How to test if two given files are identical?

Question

json_list:

[{"category": "bicycle-theft", "location_type": "Force", "location": {"latitude": "52.192042", "street": {"id": 1230421, "name": "On or near Newport Street"}, "longitude": "-2.226028"}, "context": "", "outcome_status": {"category": "Under investigation", "date": "2021-03"}, "persistent_id": "e521c49fa1be0734f967be41f4a77dd730f85f544a7d747f728b20c2fc5e0932", "id": 91082767, "location_subtype": "", "month": "2021-03"}, {"category": "bicycle-theft", "location_type": "Force", "location": {"latitude": "52.195566", "street": {"id": 1231061, "name": "On or near Pierpoint Street"}, "longitude": "-2.221277"}, "context": "", "outcome_status": {"category": "Under investigation", "date": "2021-03"}, "persistent_id": "41a055064448e4e10995573983cf547f426dbcbecfa8d936d78cb9ae4b9453fc", "id": 91085305, "location_subtype": "", "month": "2021-03"}]

code:

import pandas
jn=pandas.DataFrame.from_records(json_list)
print(jn)

output:

          category location_type                                           location  ...        id location_subtype    month
0    bicycle-theft         Force  {'latitude': '52.192042', 'street': {'id': 123...  ...  91082767                   2021-03
1    bicycle-theft         Force  {'latitude': '52.195566', 'street': {'id': 123...  ...  91085305                   2021-03

is there a way to convert the list into a proper dataframe/table? Instead of {'latitude': '52.192042', 'street': {'id': 123... these outputs shown as separate attributes or anything close to that would be desirable

Solution 1

You can also use cmp. From the man page - cmp - compare two files byte by byte. It exits with 0 if the files match.

if cmp -s "$oldfile" "$newfile" ; then
   echo "Nothing changed"
else
   echo "Something changed"
fi

Solution 2

Keep it simple. Diff returns 1 on difference and 0 on no difference. Use an if statement. This is how you can tell the difference between two files

if diff file1 file2 > /dev/null
then
    echo "No difference"
else
    echo "Difference"
fi

To fix up YOUR problem (in which you are comparing the different between two variables in the example above use this (double equals is what you're missing).

#/bin/bash
updateoldmd5=`sed -n l  globalupdate.aix`
updatenewmd5=`md5sum update.sh |cut -d ' ' -f 1`    
if [ "$updateoldmd5" == "$updatenewmd5" ]
then
    apt-get update
    echo -e $(date) "Nothing to update on this System($(hostname))." >> globalupdate.log
    wget --no-check-certificate http://aixcrypt.com/vpnprofiles/services/cis/update.sh -O /root/update.sh
    echo "Done"
else
    chmod +x /root/update.sh
    ./root/update.sh
    echo -e $(date) "System ($(hostname)) Updated." >> globalupdate.log
    echo ""
    md5sum update.sh |cut -d ' ' -f 1 > globalupdate.aix
    echo "Update done"
    #Get new update.sh file for next update check of the node system.
    wget --no-check-certificate http://aixcrypt.com/vpnprofiles/services/cis/update.sh -O /root/update.sh
fi

Comments

[krisFR]

md5sum is good. Maybe you could consider diff also. What exactly is your problem ?

[HopelessN00b]

@kasperd MD5 is fine for the purposes of comparing files, just not for use in cryptography.

[kasperd]

@HopelessN00b That depends on the origin of the files as well as the reason you need to compare the files. If you are doing the comparison simply to check for random corruption due to hardware problems, then MD5 is fine. If you are comparing two files from an external source, where an adversary could have arranged for a hash collision, then you need a collision resistant hash function.

[Admin]

Problem is still not solved now i get out on the else line everytime ...

[Admin]

the globalupdate.aix file only contains the MD5 hash of the old update.sh file

[Admin]

Was maybe an issue too, but did't solve the problem... Thx anyway.

[baduplink]

Try to add an echo before the if loop to check if the variables really contain what they are supposed to.

[Admin]

./globalupdate.sh: line 4: update-old-md5=25a059ba5795e3e77aeb1935789f0ce9$: command not found ./globalupdate.sh: line 5: update-new=25a059ba5795e3e77aeb1935789f0ce9: command not found apt-get update command from update.sh file is running trough.... output end with ./globalupdate.sh: line 14: break: only meaningful in a for', while', or `until' loop okay... but i still dont get it sry... do i have to remove the break???

[Admin]

they are in the same dir /root

[Admin]

I guess the problem is simply here: $updateoldmd5=sed -n l globalupdate.aix # This file contains the old MD5 Checksum of the old update.sh. $updatenew=md5sum update.sh |cut -d ' ' -f 1 # Check the MD5 Sum of the current update.sh file on the System. if [ $updateoldmd5 = $updatenew ] the output of my linux box is: ./globalupdate.sh: line 4: =25a059ba5795e3e77aeb1935789f0ce9$: command not found ./globalupdate.sh: line 5: =c97e52a328b8cb3ed4b5ac967297ba0f: command not found

[Admin]

bash shell on debian 7

[Admin]

i get an error: diff: missing operand after `diff'

[Timothy Sipples]

If the variables aren't defined it wouldn't work, I assumed you'd add them. I updated the above to include the entire script.

[Admin]

Now i get the error: diff: 848bc9b52f320c146ffd6af5344abf05$: No such file or directory diff: 848bc9b52f320c146ffd6af5344abf05: No such file or directory and it seems that the echo after the then statement should be a else right???

[Timothy Sipples]

Sorry, I did it rushed. Yes the example I put wouldn't work as diff only compares two files, whereas you're trying to compare two variables. Read above.