In python, how can I print lines that do NOT contain a certain string, rather than print lines which DO contain a certain string:

Question

I am trying to condense a very large log file, and to do so, I must eliminate every line which contains the string "StatusRequest" and "StatusResponse", while printing the other lines w/o this string. The code I have so far is as follows (to run from the command prompt):

   if (sys.argv[1])=="--help":
       print ("\n")
       print ("Argument 1: Enter name of '.py' file")
       print ("-i or --input: name of Catalina log")
       print ("-o or --output: file to output to")
       print ("\n")
   if (sys.argv[1])=="-h":
       print ("\n")
       print ("Argument 1: Enter name of '.py' file")
       print ("-i or --input: name of Catalina log")
       print ("-o or --output: file to output to")
       print ("\n")

   else:
       print 'Number of arguments:', len(sys.argv), 'arguments.'
       print 'Argument List:', str(sys.argv)

       Numarg = (len(sys.argv))
       i=1
       while i<=(Numarg-4):
           search1="StatusRequest"
           search2="StatusResponse"
           if (sys.argv[Numarg-2])=="-o":
               outputfile=sys.argv[Numarg-1]

           if (sys.argv[Numarg-2])=="--output":
               outputfile=sys.argv[Numarg-1]

           if (sys.argv[i])=="-i":
               filename=(sys.argv[i+1])

               log=(filename)
               print ("You entered the log: " + log)

               f=open(log, 'r')
               read_data = f.read()
               f.close

               f=open(log, 'r')
               readlines_data=f.readlines()
               f.close()
               i=i+1
           if (sys.argv[i])=="--input":
               filename=(sys.argv[i+1])
               log=(filename)
               print ("You entered the log: " + log)

               f=open(log, 'r')
               read_data = f.read()
               f.close

               f=open(log, 'r')
               readlines_data=f.readlines()
               f.close()
               i=i+1
           for line in readlines_data:
               if not ("StatusRequest" or "StatusResponse") in line:
                   result=line
                   print (line)
       f=open(outputfile, 'a')
       f.write(result + "\n")
       f.close()

You can just focus on the end of the script to answer my question, really...Anyways, I am not sure why this doesn't work...It is outputting every line still. And I already tried switching the place of the not so it would make more sense idiomatically, but it didn't change anything with the code. Any help is much appreciated :)

Solution 1

The problem isn't your use of not, it's that or doesn't mean what you think it does (and if you think it through, it couldn't):

if not ("StatusRequest" or "StatusResponse") in line:

You're asking whether the expression ("StatusRequest" or "StatusResponse") appears in line. But that expression is just the same thing as "StatusRequest".

Put it in English: you're not trying to say "if neither of these is in line". Python doesn't have a neither/none function, but it does have an any function, so you can do this:

if not any(value in line for value in ("StatusRequest", "StatusResponse")):

This isn't quite as nice as English; in English, you can just say "if none of the values 'StatusRequest' and 'StatusResponse' are in line", but in Python, you have to say "if none of the values coming up are in line, for values 'StatusRequest' and 'StatusResponse'".

Or, maybe more simply in this case:

if "StatusRequest" not in line and "StatusResponse" not in line:

(Also, notice that you can use not in, instead of using in and then negating the whole thing.)

Solution 2

Replace this line:

if not ("StatusRequest" or "StatusResponse") in line:

With this one:

if "StatusRequest" not in line and "StatusResponse" not in line:

Not super elegant, but it will do the trick. I'm not sure if there's a faster way to compare two strings against the same line.

Solution 3

You have to put each condition separately:

for line in readlines_data:
    if ("StatusRequest" not in line) and ("StatusResponse" not in line):
        result = line
        print(line)

Solution 4

The not can be used to negate the expression inside the parenthesis like you first had it. You just need to modify what it's negating, which is that the string is found within line:

if not ("StatusRequest" in line or "StatusResponse" in line):

View more solutions

Comments

[TheSoundDefense]

A better explanation than my answer, for certain.

[user3877194]

@abarnert Thank you for the thorough explanation. The only other question I have is that if there is a "StatusResponse" or "StatusRequest" at the very beginning of the file, there will be no value stored in the variable 'result'. I know I didn't address this initially, but any help is welcomed. Could I just initialize the result variable to [null] or is there a better way to do this?