Split a file using windows batch script
14,067
This would do the trick assuming your lines are roughly the same size.
Its advantage is that it is only a 2 pass solution, One for counting the lines and the other for printing them.
@rem echo off
@rem usage: batchsplit.bat <file-to-split> <size-limit>
@rem it will generate files named <file-to-split>.part_NNN
setlocal EnableDelayedExpansion
set FILE_TO_SPLIT=%1
set SIZE_LIMIT=%2
for /f %%s in ('dir /b %FILE_TO_SPLIT%') do set SIZE=%%~Zs
for /f %%c in ('type "%FILE_TO_SPLIT%"^|find "" /v /c') do set LINE_COUNT=%%c
set /a AVG_LINE_SIZE=%SIZE%/%LINE_COUNT%
set /a LINES_PER_PART=%SIZE_LIMIT%/%AVG_LINE_SIZE%
set "cmd=findstr /R /N "^^" %FILE_TO_SPLIT%"
for /f "tokens=1,2* delims=:" %%a in ('!cmd!') do @(
set /a ccc = %%a / %LINES_PER_PART%
echo %%b >> %FILE_TO_SPLIT%.part_!ccc!
)
save it as batchsplit.bat
and run it using:
batchsplit.bat myfile.csv 100000000
Author by
Admin
Updated on June 21, 2022Comments
-
Admin almost 2 years
I have a csv file and i need to split it in to n files such that each split file should not exceed 100 mb. I need to achieve it in windows batch script. I tried the below way but its taking lot of time as my unsplit file is in GBs
@echo off setlocal enableextensions enabledelayedexpansion set count=1 set maxbytesize=100000000 set size=1 type NUL > output_1.csv FOR /F "tokens=*" %%i in (myfile.csv) do ( FOR /F "usebackq" %%A in ('!filename!_!count!.csv') do ( set size=%%~zA) if !size! LSS !maxbytesize! ( echo %%i>>!filename!_!count!.csv) else ( set /a count+=1 echo %%i>>!filename!_!count!.csv ))
please let me know if there is a better way to achieve this. I cant go to any other scripting languages as my server is windows