Learn How to Generate and Verify Files with MD5 Checksum in Linux

A checksum is a digit which serves as a sum of correct digits in data, which can be used later to detect errors in the data during storage or transmission. MD5 (Message Digest 5) sums can be used as a checksum to verify files or strings in a Linux file system.

MD5 Sums are 128-bit character strings (numerals and letters) resulting from running the MD5 algorithm against a specific file. The MD5 algorithm is a popular hash function that generates 128-bit message digest referred to as a hash value, and when you generate one for a particular file, it is precisely unchanged on any machine no matter the number of times it is generated.

It is normally very difficult to find two distinct files that results in same strings. Therefore, you can use md5sum to check digital data integrity by determining that a file or ISO you downloaded is a bit-for-bit copy of the remote file or ISO.

Suggested Read: Progress – Monitor Progress for (cp, mv, dd, tar, etc.) Commands in Linux

In Linux, the md5sum program computes and checks MD5 hash values of a file. It is a constituent of GNU Core Utilities package, therefore comes pre-installed on most, if not all Linux distributions.

Take a look at the contents of /etc/group saved as groups.cvs below.

root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:syslog,aaronkilik
tty:x:5:
disk:x:6:
lp:x:7:
mail:x:8:
news:x:9:
uucp:x:10:
man:x:12:
proxy:x:13:
kmem:x:15:
dialout:x:20:
fax:x:21:
voice:x:22:
cdrom:x:24:aaronkilik
floppy:x:25:
tape:x:26:
sudo:x:27:aaronkilik
audio:x:29:pulse
dip:x:30:aaronkilik

The md5sums command below will generate a hash value for the file as follows:

$ md5sum groups.csv

bc527343c7ffc103111f3a694b004e2f  groups.csv

When you attempt to alter the contents of the file by removing the first line, root:x:0: and then run the command for a second time, try to observe the hash value:

$ md5sum groups.csv

46798b5cfca45c46a84b7419f8b74735  groups.csv

You will notice that the hash value has now changed, indicating that the contents of the file where altered.

Now, put back the first line of the file, root:x:0: and rename it to group_file.txt and run the command below to generate its hash value again:

$ md5sum groups_list.txt

bc527343c7ffc103111f3a694b004e2f  groups_list.txt

From the output above, the hash value is still the same even when the file has been renamed, with its original content.

Important: md5 sums only verifies/works with the file content rather than the file name.

The file groups_list.txt is a duplicate of groups.csv, so, try to generate the hash value of the files at the same time as follows.

You will see that they both have equal hash values, this is because they have the exact same content.

$ md5sum groups_list.txt  groups.csv 

bc527343c7ffc103111f3a694b004e2f  groups_list.txt
bc527343c7ffc103111f3a694b004e2f  groups.csv

You can redirect the hash value(s) of a file(s) into a text file and store, share them with others. For the two files above, you can issues the command below to redirect generated hash values into a text file for later use:

$ md5sum groups_list.txt  groups.csv > myfiles.md5

To check that the files have not been modified since you created the checksum, run the next command. You should be able to view the name of each file along with “OK”.

Suggested Read: Find Top 15 Processes by Memory Usage in Linux

The -c or --check option tells md5sums command to read MD5 sums from the files and check them.

$ md5sum -c myfiles.md5

groups_list.txt: OK
groups.csv: OK

Remember that after creating the checksum, you can not rename the files or else you get a “No such file or directory” error, when you try to verify the files with new names.

For instance:

$ mv groups_list.txt new.txt
$ mv groups.csv file.txt
$ md5sum -c  myfiles.md5

md5sum: groups_list.txt: No such file or directory
groups_list.txt: FAILED open or read
md5sum: groups.csv: No such file or directory
groups.csv: FAILED open or read
md5sum: WARNING: 2 listed files could not be read

The concept also works for strings alike, in the commands below, -n means do not output the trailing newline:

$ echo -n "Tecmint How-Tos" | md5sum - 

afc7cb02baab440a6e64de1a5b0d0f1b  -

$ echo -n "Tecmint How-To" | md5sum - 

65136cb527bff5ed8615bd1959b0a248  -

In this guide, I showed you how to generate hash values for files, create a checksum for later verification of file integrity in Linux. Although security vulnerabilities in the MD5 algorithm have been detected, MD5 hashes still remains useful especially if you trust the party that creates them.

Verifying files is therefore an important aspect of file handling on your systems to avoid downloading, storing or sharing corrupted files. Last but not least, as usual reach us by means of the comment form below to seek any assistance, you can as well make some important suggestions to improve this post.