How to read and write a text file in the same time in python?

15,717

Solution 1

You can try the following, the problem at the moment is that you are only storing and writing the final occurrence of the modified line, instead it's better to create a copy of the modified file in memory and then writing out (see below)

remove_commas = re.compile("House")
answer = {}

with open("\DEMO_houses.txt", "r") as inp:
    new_file_lines = []
    for line in inp:
        if remove_commas.match(line):
            line = line.replace(',', '')
        new_file_lines.append(line)

with open("DEMO_houses.txt", "w") as document1:
        document1.writelines(new_file_lines)

Solution 2

What is happening in your code right now is that you first read all the lines in the with open("\DEMO_houses.txt", "r") as inp: block and then in the with open("DEMO_houses.txt", "w") as document1: block your code only writes back the last read line. Since write mode "w" erases the previous file, only the last line of your original file remains after your code finishes execution.

You might be better off first reading all the lines into memory, then mdifying those lines, and then writing them back into the same file like so:

import re

remove_commas = re.compile("House")

data = []
with open("DEMO_houses.txt", "r") as inp:  #Read phase
    data = inp.readlines()  #Reads all lines into data at the same time

for index, item in enumerate(data): #Modify phase, modification happens at once in memory
    if remove_commas.match(item):
        data[index] = item.replace(',', '')

with open("DEMO_houses.txt", "w") as document1: #write back phase
        document1.writelines(data)

Provided the file can be stored in memory without issues, this is a far superior method to reading and modifying the file one line at a time as modifications in memory are much faster, and the file in secondary storage would only be modified once.

Share:
15,717
Dj_ 96
Author by

Dj_ 96

Updated on June 04, 2022

Comments

  • Dj_ 96
    Dj_ 96 almost 2 years

    I am trying to open a text file, read it and after using a regex function to find which lines to edit, modify my text file. However what happens is that after finding the lines and edit them i can not write the modified Content in the text file again.

    remove_commas = re.compile("House")
    answer = {}
    
    global line1
    
    with open("\DEMO_houses.txt", "r") as inp:
    
        for line in inp:
            if remove_commas.match(line):
    
                line1 = line.replace(',', '')
                print line1
    
    with open("DEMO_houses.txt", "w") as document1:
            document1.write(line1)
    

    What happens is that it just delets my text file, and writes only the first line modified.

    Text file is something like this:

    Street : Blue, Red
    House: Big, enough
    Garden : green, not green
    

    And in the new text file i Need something like:

    Street : Blue, Red
    House: Big enough
    Garden : green, not green
    

    If anyone could help me I would really appreciate it. Thanks

    • Dj_ 96
      Dj_ 96 over 5 years
      And if my text file is something like: Street : Blue, Red House: Big, enough Garden : green, not green House: Not, enough In the new text file it only prints : House: Big enough
  • vintprox
    vintprox over 3 years
    it open and closes the file twice
  • MuhsinFatih
    MuhsinFatih over 3 years
    you can easily run into a race condition with different processes if you use two open statements like this
  • Maghil vannan
    Maghil vannan almost 3 years
    @MuhsinFatih I doubt so since those open conditions don't get executed at same time
  • MuhsinFatih
    MuhsinFatih almost 3 years
    @Maghilvannan Imagine the following: [ a multi-threaded / multi-process server starts and wants to increase a number in a text file on each request ] [ process 1 enters the first open block ] [ process 1 reads value x and exits the block] [ process 2 enters the first open block ] [ process 2 reads value x and exits the block ] [ process 1 enters the second open block and writes x+1 and exits the block ] [ process 2 enters the second open block and writes x+1 and exits the block ]. Intended result: file contains x+2. Actual result: file contains x+1
  • Maghil vannan
    Maghil vannan almost 3 years
    @MuhsinFatih didn't think about multi threads. I thought you meant opening and closing the file in a single thread. yep if we use multiple threads race conditions will occurs