What happens when I truncate a file that is in use?
Note although the system call is called truncate, it actually is better interpreted as saying "Make my file report this many bytes in size". As per the system call manpage:
The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes.
If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('\0').
So, one can truncate a file and have it be larger, rather than smaller.
So my questions is, if my process has an opened file in pos 1000 and i did truncate -s0 filename, if the truncate works, what happening in the next process write?
- You have truncated. The file size at this stage is 0 bytes. The offset is 1000.
- The write at position 1001 occurs.
- The file size is 1002 bytes. Bytes 0-1000 contain '\0' (null). Bytes 1001+ contain data written.
When you write to a file from a position larger than the file itself, the data between the end of the file and the new write becomes null bytes and the file data between those two points is referred as being sparse.
Indeed, you can do the following and produce the same affect.
import os, sys
f = open('data.txt','w')
f.seek(1048576)
f.write('a')
f.flush()
f.close()
You also mentioned that opening in append mode avoids this behaviour. This is true because you are instructing the kernel in that case to "write to the actual end of the file every time". If you truncate then the end of the file does change. In append you cannot reposition your file pointer.
Here is a sample program which demonstrates what happens to the file, the offsets and the data in a file that has been truncated.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <err.h>
#define FPATH "/tmp/data.txt"
#define FSIZE 65536
int main() {
int a,b;
char buf[FSIZE];
char byte;
struct stat st;
memset(buf, 'A', FSIZE);
a = open(FPATH, O_WRONLY|O_CREAT);
b = open(FPATH, O_RDONLY);
if (a < 0 || b < 0)
errx(EXIT_FAILURE, "Could not open file");
printf("Writing %d * 'A' into file\n", FSIZE);
/* Write some bytes */
if(write(a, buf, FSIZE) != FSIZE)
errx(EXIT_FAILURE, "Couldn't write complete size out");
/* Seek to a new position in the file */
lseek(b, FSIZE/2, SEEK_SET);
printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
stat(FPATH, &st);
printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);
/* OK -- now, read the byte at the position */
if (read(b, &byte, 1) < 0)
err(EXIT_FAILURE, "Could not read file");
printf("Character at current position of handle 'b': '%c'\n", byte);
/* Truncate the file in the 'a' handle */
printf("Performing truncate...\n");
if (ftruncate(a, 0) < 0)
err(EXIT_FAILURE, "Cannot truncate file");
printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
stat(FPATH, &st);
printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);
printf("Writing one byte via handle 'a'\n");
if (write(a, buf, 1) < 0)
err(EXIT_FAILURE, "Cannot perform second write");
printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
stat(FPATH, &st);
printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);
if (read(b, &byte, 1) < 0)
err(EXIT_FAILURE, "Could not read file");
printf("Character at current position of handle 'b': '%c'\n", byte);
close(a);
close(b);
exit(0);
}
This results in the following output;
Writing 65536 * 'A' into file
Current position of handle 'a': 65536
Current position of handle 'b': 32768
Reported size on filesystem of /tmp/data.txt: 65536
Character at current position of handle 'b': 'A'
Performing truncate...
Current position of handle 'a': 65536
Current position of handle 'b': 32769
Reported size on filesystem of /tmp/data.txt: 0
Writing one byte via handle 'a'
Current position of handle 'a': 65537
Current position of handle 'b': 32769
Reported size on filesystem of /tmp/data.txt: 65537
Character at current position of handle 'b': ''
Related videos on Youtube
c4f4t0r
I work as Linux system administrator for Ops area and sometime for area project, in my free time I love to play basketball, I love to improve my "(ruby|perl|python)" programming skills reading books and online topics.
Updated on September 18, 2022Comments
-
c4f4t0r over 1 year
Around the web many people say you can truncate a file using
> filename
ortruncate -s0 filename
while file begin usedI know everytime a process write in a file, the process uses a offset for write in a file, doing a test with an script like this.
#!/usr/bin/env python import os, time with open("passwd","w") as f: #copy of passwd file in my current directory f.seek(0) for x in xrange(1,1000): f.write("hello world \n" + time.ctime() + "\n") f.flush() time.sleep(2)
every time my script make a write syscall the offset in
/proc/pid_number/fdinfo/3 pos
field is changed, but when i try to truncate the the file using the method listed above, in my file i see many characters like this^@
when i open the file usingvim
or less-u
and the file type is change fromASCII text
todata
and when i usels -l filename
the size isn't changedSo, when truncate the file the offset of the file are not report back, i'm testing this in
Centos 7
and inRedhat 5
, so i can tell changed the file size while the file is in use by processes doesn't free space and make dirty my file.So my questions is, if my process has an opened file in
pos 1000
and i didtruncate -s0 filename
, if the truncate works, what happening in the next process write?strace truncate -s0 passwd open("passwd", O_WRONLY|O_CREAT|O_NONBLOCK, 0666) = 3 ftruncate(3, 0) = 0 close(3) = 0 close(1) = 0 close(2) = 0 exit_group(0) = ? ls -l passwd -rw-rw-r--. 1 user91 users 13832 Feb 23 17:16 passwd
As you can see my file wasn't truncated
This problem doesn't happen if i open the in append mode, for example with this code.
#include <stdio.h> #include <stdlib.h> #include <fcntl.h> #include <errno.h> #include <sys/types.h> #include <unistd.h> #include <sys/stat.h> int main(){ int range = 1000; int x; x = open("passwd", O_WRONLY|O_CREAT|O_APPEND); int i = 0; for( i = 0; i <= range; range++) write(x,"hello world\n",12); sleep(2); }
-
Zoredache about 9 yearsThis seems more like a general computing question rather then anything related to system administration.
-
Matthew Ife about 9 years@Zoredache I disagree. I feel this is a question of file semantics in posix systems and is a platform question. I am answering.
-
-
c4f4t0r about 9 yearsyes, looking in the source kernel code i saw when the file is opened with O_APPEND, the pos = i_read_size(inode), anyway your answer solved my problem.