batch script - read line by line

Question

I have a log file which I need to read in, line by line and pipe the line to a next loop.

Firstly I grep the logfile for the "main" word (like "error") in a separate file - to keep it small. Now I need to take the seperate file and read it in line by line - each line needs to go to another loop (in these loop I grep the logs and divide it in blocks) but I stuck here.

The log looks like

xx.xx.xx.xx - - "http://www.blub.com/something/id=?searchword-yes-no" 200 - "something_else"

with a for /f loop I just get the IP instead of the complete line.

How can I pipe/write/buffer the whole line? (doesn't matter what is written per line)

Solution 1

Try this:

@echo off
for /f "tokens=*" %%a in (input.txt) do (
  echo line=%%a
)
pause

because of the tokens=* everything is captured into %a

edit: to reply to your comment, you would have to do that this way:

@echo off
for /f "tokens=*" %%a in (input.txt) do call :processline %%a

pause
goto :eof

:processline
echo line=%*

goto :eof

:eof

Because of the spaces, you can't use %1, because that would only contain the part until the first space. And because the line contains quotes, you can also not use :processline "%%a" in combination with %~1. So you need to use %* which gets %1 %2 %3 ..., so the whole line.

Solution 2

The "call" solution has some problems.

It fails with many different contents, as the parameters of a CALL are parsed twice by the parser.
These lines will produce more or less strange problems

one
two%222
three & 333
four=444
five"555"555"
six"&666
seven!777^!
the next line is empty

the end

Therefore you shouldn't use the value of %%a with a call, better move it to a variable and then call a function with only the name of the variable.

@echo off
SETLOCAL DisableDelayedExpansion
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ t.txt"`) do (
    set "myVar=%%a"
    call :processLine myVar
)
goto :eof

:processLine
SETLOCAL EnableDelayedExpansion
set "line=!%1!"
set "line=!line:*:=!"
echo(!line!
ENDLOCAL
goto :eof

Solution 3

This has worked for me in the past and it will even expand environment variables in the file if it can.

for /F "delims=" %%a in (LogName.txt) do (
     echo %%a>>MyDestination.txt
)

Solution 4

For those with spaces in the path, you are going to want something like this: n.b. It expands out to an absolute path, rather than relative, so if your running directory path has spaces in, these count too.

set SOURCE=path\with spaces\to\my.log
FOR /F "usebackq delims=" %%A IN ("%SOURCE%") DO (
    ECHO %%A
)

To explain:

(path\with spaces\to\my.log)

Will not parse, because spaces. If it becomes:

("path\with spaces\to\my.log")

It will be handled as a string rather than a file path.

"usebackq delims=" 

See docs will allow the path to be used as a path (thanks to Stephan).

View more solutions

Comments

[Ignacio Vazquez-Abrams]

possible duplicate of DOS batch files: How to read a file?

[pille]

thx - works perfect. could you explain why following statement did not work for /F %%f in (searchpattern.txt) do (call :PROCESS4 %%f ) GOTO END4 :PROCESS4 set var4=%1 goto end4 :end4 its basicly the same. i understand that without a definde "tokens" it will take the whole line put it in the var4 and i could work with it. but why it do not work please bring light in the darkness thank you

[wimh]

@user553499, I edited my answer explaining how to do this in combination with call.

[Eddie Garcia]

I wasn't asking a question, I was answering it. So I guess the answer was not clear. :)

[Stephan]

that's why the switch usebackq exists. More is a bad choice for big files, as it will wait for a keypress every 64k (if I remember correctly) in batch-mode and replaces TABs with SPACEs. Type would be a better choice. Best solution: `for /f "usebackq delims=" %%A in ("file with spaces.txt") do ...'

[Haggunenom]

Thanks Stephan! Was at the end of a long day trying to figure out what went wrong with a colleague's script, and found they'd copy-pasta'd the code from here, so wanted to float this so others don't do the same thing! 'Type' is indeed what I wanted. As a note - 'more' works on a page-by-page basis, without a page to limit it to, my undertanding is it will just keep going (just ran it over a 2MB log).

[Stephan]

"Without a page limit" is not quite right. I admit 65536 lines is a far stretched limit, but it is a limit (and a log file might easily overcome that)

[Haggunenom]

Ahh, I read that as 64k bytes (thus mentioning a file size). And yes, a log certainly would. Further - more seems to be incredibly slow with long files (presumably because this was never its intended use).