# Modifying or deleting a line from a text file the low-level way?

12,343

## Solution 1

I find this an interesting question, so I made a small console app.

I used 3 methods:

• TStringList
• Text file

All methods are timed and repeated 100 times with a text file of 10kb in size and a text file 1Mb in size. Here is the program:

program Project16;

{\$APPTYPE CONSOLE}

uses
SysUtils, Classes, StrUtils, Diagnostics, IOUtils;

procedure DeleteLine(StrList: TStringList; SearchPattern: String);

var
Index : Integer;

begin
for Index := 0 to StrList.Count-1 do
begin
if ContainsText(StrList[Index], SearchPattern) then
begin
StrList.Delete(Index);
Break;
end;
end;
end;

procedure DeleteLineWithStringList(Filename : string; SearchPattern : String);

var StrList : TStringList;

begin
StrList := TStringList.Create;
try
DeleteLine(StrList, SearchPattern);
// don't overwrite our input file so we can test
StrList.SaveToFile(TPath.ChangeExtension(Filename, '.new'));
finally
StrList.Free;
end;
end;

procedure DeleteLineWithStreamReaderAndWriter(Filename : string; SearchPattern : String);

var
Writer    : TStreamWriter;
Line      : String;
DoSearch  : Boolean;
DoWrite   : Boolean;

begin
Writer := TStreamWriter.Create(TPath.ChangeExtension(Filename, '.new'));
try
DoSearch := True;
DoWrite := True;
begin
if DoSearch then
begin
DoSearch := not ContainsText(Line, SearchPattern);
DoWrite := DoSearch;
end;
if DoWrite then
Writer.WriteLine(Line)
else
DoWrite := True;
end;
finally
Writer.Free;
end;
end;

procedure DeleteLineWithTextFile(Filename : string; SearchPattern : String);

var
InFile    : TextFile;
OutFile   : TextFile;
Line      : String;
DoSearch  : Boolean;
DoWrite   : Boolean;

begin
AssignFile(InFile, Filename);
AssignFile(OutFile, TPath.ChangeExtension(Filename, '.new'));
Reset(InFile);
Rewrite(OutFile);
try
DoSearch := True;
DoWrite := True;
while not EOF(InFile) do
begin
if DoSearch then
begin
DoSearch := not ContainsText(Line, SearchPattern);
DoWrite := DoSearch;
end;
if DoWrite then
Writeln(OutFile, Line)
else
DoWrite := True;
end;
finally
CloseFile(InFile);
CloseFile(OutFile);
end;
end;

var
Count : Integer;
Sw    : TStopWatch;

begin
Writeln(Format('Delete line with stream reader/writer - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with stream reader/writer - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;

procedure TimeDeleteLineWithStringList(Iterations : Integer);

var
Count : Integer;
Sw    : TStopWatch;

begin
Writeln(Format('Delete line with TStringlist - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStringList('c:\temp\text10kb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with TStringlist - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStringList('c:\temp\text1Mb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;

procedure TimeDeleteLineWithTextFile(Iterations : Integer);

var
Count : Integer;
Sw    : TStopWatch;

begin
Writeln(Format('Delete line with text file - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithTextFile('c:\temp\text10kb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with text file - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithTextFile('c:\temp\text1Mb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;

begin
try
TimeDeleteLineWithStringList(100);
TimeDeleteLineWithTextFile(100);
Writeln('Press ENTER to quit');
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
end.


Output:

Delete line with TStringlist - file 10kb, 100 iterations
Elapsed time : 188 milliseconds
Delete line with TStringlist - file 1Mb, 100 iterations
Elapsed time : 5137 milliseconds
Delete line with stream reader/writer - file 10kb, 100 iterations
Elapsed time : 456 milliseconds
Delete line with stream reader/writer - file 1Mb, 100 iterations
Elapsed time : 22382 milliseconds
Delete line with text file - file 10kb, 100 iterations
Elapsed time : 250 milliseconds
Delete line with text file - file 1Mb, 100 iterations
Elapsed time : 9656 milliseconds
Press ENTER to quit


As you can see is TStringList the winner here. Since you are not able to use TStringList, TextFile is not a bad choice after all...

P.S. : this code omits the part where you have to delete the inputfile and rename the outputfile to the original filename

## Solution 2

Without loading the entire file into a container like TStringList, your only option is to:

• Open the file for input
• Open a separate copy for output
• Start a loop
• Read the content line by line from the input file
• Write the content out line by line to the output file until you reach the line you want to change/delete
• Break the loop
• Read the input line from the input file
• Write the changed line (or skip writing the line you want to delete) to the output file
• Start a new loop
• Read the remainder of the input content, line by line
• Write the rest of that input to the output file, line by line
• Break the loop
• Close the files

if N = UpperCase(Name) then begin
//How to re-write this line?
Break;
end;


WriteLn the new output to the second (output) file.

if N = UpperCase(Name) then begin
//How to delete this line?
Break;
end;


Just skip the WriteLn that outputs the indicated line to the second (output) file.

Your artificial limitation of "I don't want to use TStringList" simply complicates the task for you, when you can simply:

• Load the original file into TStringList using LoadFromFile
• Locate the line you want to modify, either by index, iteration, or IndexOf()
• Modify the line by changing it directly, or deleting it from the TStringList
• Write the entire content out to the original file using TStringList.SaveToFile

The only reasons I've found to not use TStringList to perform these kinds of operations have been that the file size exceeds the capacity of a TStringList (never happened) or when dealing with a file that is text but isn't really "line" oriented (for instance, EDI files that are typically one very long single line of text, or XML files that may not contain line feeds and therefore are also one very long single line of text). Even in the case of EDI or XML, though, it's quite frequently to load them into a TStringList, make the conversion to line-based format (inserting line breaks or whatever), and do the retrieval from the stringlist.

## Solution 3

Basically, you can't do what you want to do if you treat the files as simple text files. Such files can be read (from the beginning only) or written to (either from the start, thus creating a new file) or from the end (appending to an existing file). They are not random access files.

On the other hand, you might want to consider defining a file of type string: each record in the file would be a string, and you can access this file in a random fashion. The problem then becomes in knowing which record to access for which string.

A third possibility is using INI files which are more structured and sound like a better bet for your purposes. Apart from the section header, they are a series of strings, key=value, and can be accessed on the basis of the key.

Share:
12,343
Author by

### Jerry Dodge

I'm a Delphi developer. I work for a software company which does solutions for retail management, including inventory, POS, reporting, BI, Tags, and more. It's been in Delphi since Delphi's been around. I am actively in Stack Overflow monitoring the Delphi tag, and looking for those questions I can answer and also contributing my time to keep Stack Overflow in order. I'm not an expert in anything, a jack of all trades rather. But I love to help people when I'm able to. I've known Delphi since about 2007 now, and before that, I had learned VB6. I havn't gone back to VB since I learned Delphi. I also taught myself QBasic and HTML as a kid. It hasn't been until the past 5 years that I've been diving into programming. Since then I've also become vaguely familiar with ASP.NET with C#, as well as some C# windows apps. But I'm not too fond of the whole .NET idea. .NET is good for web platforms and such, but not for win apps. My latest work has been with Delphi 10 Seattle mobile development. I'm still very raw on the subject, but see a huge potential behind it. My strengths: Understanding the bigger picture of projects Writing Custom Classes, Components, and Controls Code organization (within unit or namespace) Writing purely independent classes (as opposed to cross-referencing units or namespaces) User Friendly UI's Developer Friendly Classes Encapsulating layers of business logic My weaknesses: Lower-level coding (such as Assembly) Platform-specific design (using Firemonkey) Web Design It's always nice to know you're able to do something, even if you never use it.

Updated on June 04, 2022

• Jerry Dodge 7 months

I'm working with a Text File in Delphi, and I don't wish to use the method of loading/saving with a string list. I intend to maintain an open filestream where I read and write my data there, keeping massive amounts of data on the hard disk instead of in the memory. I have the simple concept of writing new lines to a text file and reading them, but when it comes to modifying and deleting them, I cannot find any good resources.

Each line in this file contains a name, and equals sign, and the rest is data. For example, SOMEUNIQUENAME=SomeStringValue. I intend to keep a file open for a period of time inside of a thread. This thread performs incoming requests to either get, set, or delete certain fields of data. I use WriteLn and ReadLn in a loop, evaluating EOF. Below is an example of how I read the data:

FFile = TextFile;

...

var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
Delete(S, 1, Pos('=', S));
Result:= S;
Break;
end;
end;
end;


...and then I trigger an event which informs sender of result. The requests are inside of a queue, which is sort of a message pump for these requests. The thread simply processes the next request in the queue repeatedly, similar to how typical applications work.

I have procedures ready to be able to write and delete these fields, but I don't know what I have to do to actually perform the action on the file.

procedure TFileWrapper.WriteData(const Name, Value: String);
var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
//How to re-write this line?
Break;
end;
end;
end;

procedure TFileWrapper.DeleteData(const Name: String);
var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
//How to delete this line?
Break;
end;
end;
end;


In the end, I need to avoid loading the entire file into the memory to be able to accomplish this.

• Jerry Dodge about 10 years
Yes, Ini files were my first direction, but I wanted to skip a bit and not require any header section, but I might as well implement it anyway.
• Jerry Dodge about 10 years
That does indeed sound like the only solution, I'm just pondering how SQL Server is able to maintain its data feed in the file stream, and any complex database for that matter. I'm just trying to avoid having to use up the memory many times over for massive amounts of data.
@Jerry SQL Server or any modern database engine does not use text files to store data. The data is usually stored in pages, and each page contains none, one or many records. When you update a record, if, for example, a varchar column does not fit in the same space as the old record, it is moved to the end of the page or to a different page. There are books about how this works. Read for example Understanding SQL Server Storage Structures or search google for "(sql server/oracle/firebird/db2) internal storage"
Other little thing: by definition a text file is a sequential file, because the nature of different length lines, and that's traditionally have a different treatment from a random access file. In a text file, without traversing the entire file, you have no method to predict at which file position is stored any particular line, and you can't really change any line size directly in the file. In a random access file, you can go to any record, page or byte position, because all the pages, records or bytes have the same length, so you can do some arithmetic to get that position.
• Ken White about 10 years
One thing to note, though: IIRC, TextFile is not Unicode-aware, so it won't work with anything but ASCII text.
• Ken White about 10 years
Jerry, as @jachguate said, SQL Server (or any other RDBMS, for that matter) doesn't use anything even remotely resembling text files. They use complex structures of binary (very often compressed) data that are maintained in pages. Comparing the two is like comparing a tricycle (the text file) to a Porsche (SQL Server) (Oracle would be the Ferrari <g>). In other words, you can't.
@KenWhite: That is not correct. I tested some files in UTF-8 format and it works like a charm
• Ken White about 10 years
What Delphi version? This wasn't the case in D2009-XE, because it was a common problem on the EMBT forums (which is where I got my info from - I haven't used TextFile in a decade because it's simply old Pascal methodology, and there are better ways now).
@Remy: Great. :-) The question was about using TextFile - how exactly do you do all those things (especially reading/writing in chunks, ignoring lines, and using memory mapping)? I don't see any of those things in the help file for TextFile. (Note the question didn't ask about writing anything new, but specifically about using TextFile and not a container like TStringList.)
@KenWhite: I was commenting on your list of required steps, not the actually implementation of them. Obviously, you can't some things with TextFile, you have to ditch it in favor offor more direct I/O, like FileRead() or TFileStream.
@Remy: My point is that I was referring specifically to using TextFile, which I clearly indicated in my answer; pointing out ways to optimize it via other methods wasn't quite fair. :-) I don't comment on your answers about how to do things in Indy by pointing out easier ways to accomplish them in ICS or Synapse. <g>