TIL: Finding Duplicate Files With fdupes

Collecting audio datasets means downloading the same file twice more often than you think. fdupes finds duplicates by checksum.

fdupes -r ~/audio-datasets/

Scans the directory tree and prints groups of identical files. The -r flag makes it recursive.

fdupes -r -d ~/audio-datasets/

Interactive mode asks which duplicates to delete. You keep one, it removes the rest.

fdupes -r -S ~/audio-datasets/

Show file sizes alongside duplicate groups. The 500MB file you accidentally downloaded three times becomes obvious fast.

Does fdupes compare content or name?

Content. It computes checksums and compares bytes. Two files with different names but identical content are duplicates.

Can I install fdupes on macOS?

brew install fdupes