How to find which line is missing in another file

Question

on Linux box I have one file as below A.txt

1
2
3
4

Second file as below B.txt

1
2
3
6

I want to know what is inside A.txt but not in B.txt i.e. it should print value 4

I want to do that on Linux.

Solution 1

awk 'NR==FNR{a[$0]=1;next}!a[$0]' B A

didn't test, give it a try

Solution 2

Use comm if the files are sorted as your sample input shows:

$ comm -23 A.txt B.txt
4

If the files are unsorted, see @Kent's awk solution.

Solution 3

You can also do this using grep by combining the -v (show non-matching lines), -x (match whole lines) and -f (read patterns from file) options:

$ grep -v -x -f B.txt A.txt
4

This does not depend on the order of the files - it will remove any lines from A that match a line in B.

Solution 4

(An addition to @rjmunro's answer)

The proper way to use grep for this is:

$ grep -F -v -x -f B.txt A.txt
4

Without the -F flag, grep interprets PATTERN, read from B.txt, as a basic regular expression (BRE), which is undesired here, and can cause troubles. -F flag makes grep treat PATTERN as a set of newline-separated strings. For instance:

$ cat A.txt
&
^
[
]

$ cat B.txt
[
^
]
|

$ grep -v -x -f B.txt A.txt
grep: B.txt:1: Invalid regular expression

$ grep -F -v -x -f B.txt A.txt
&

Solution 5

Using diff:

diff --changed-group-format='%<' --unchanged-group-format='' A.txt B.txt

View more solutions

Comments

[twalberg]

Not sure why this was considered "unclear" - comm is the command you're looking for. In this specific case comm -23 A.txt B.txt.

[Kent]

I am not sure if diff will work on this problem. think about in A, I have from 1-10, but in B I have 9-1 and 100-200. all are unsorted. the output should be 10.

[gipsh]

On the question the files are sorted. It wont work on unsorted files. You can use sort or any other shell command to sort the file.

[Wang]

Thanks! this is exactly what we want!

[Wang]

I think this should be marked as the actual answer!

[Vijay Varadan]

Fantastic! Works like a charm, doesn't even require lines to be sorted. Thanks.

[nonsleepr]

Explanation of why it works

[Serge Stroobandt]

This is obviously the best answer. It can be written slightly more compact as grep -Fvx -f B.txt A.txt

[Ed Morton]

That would populate a[] with the superset of values from both files and so use more memory than necessary. It's better to do awk 'NR==FNR{a[$0];next} !($0 in a)' B A instead so it only has to hold the contents of B in memory.