How do I recover free space on deleted files without restarting the referencing processes?

Question

I work with Discord.js User object and Mongoose Schema. But the problem doesn't seems to be part of those.

var Message = require('../app/models/message'); //Mongoose Schema
  ...
var newMessage = new Message();
  ...
//taggedUser is an object containing all the info about user. id property contains user id which is number.
const taggedUser = message.mentions.users.first(); 

newMessage.message.to       = taggedUser.id;
console.log(taggedUser.id);
console.log(newMessage.message.to);

The code above should assign user ID to Schema. Everything works, but...

442090269928849410

442090269928849400

Last 2 characters aren't the same among these variables now. How is this even possible? The = changed the actual data inside the variable?

In case it is Mongoose here is how Schema looks like:

var msgSchema = mongoose.Schema({

    message          : {
        from         : Number,
        to           : Number,
        content      : String,
        time         : Date
    }

});

---

Edit:

If I change

to           : Number,

to string:

to           : String,

It works properly. I still need the answer on why does this work incorrectly with number. Right above the problematic line I have another id which works perfectly fine:

newMessage.message.from     = msg.author.id;

I have already tried to parse taggedUser.id to integer or creating Number() object but that didn't help. So every time I turn taggedUser.id into a Number or parse it to int it changes to the slightly different number.

---

I don't know what to think. How can data change during the assignment?

If there is not enough data provided in the question please ask me and I'll add everything needed. I can't imagine what might be causing this bug.

Solution 1

find /proc/*/fd -ls 2> /dev/null | grep '(deleted)'

Find all opened file descriptors.

Grep deleted.

StdError to /dev/null

Output:

160448715    0 lrwx------   1 user      user            64 Nov 29 15:34 /proc/28680/fd/113 -> /tmp/vteT3FWPX\ (deleted)

Or you can use awk

find /proc/*/fd -ls 2> /dev/null | awk '/deleted/ {print $11}';

awk output(tested in bash Ubuntu 12.04):

/proc/28680/fd/113

Find and truncate all deleted files(tested in bash Ubuntu 12.04):

(DON'T DO THIS IF YOU DON'T KNOW WHAT YOU DO)

find /proc/*/fd -ls 2> /dev/null | awk '/deleted/ {print $11}' | xargs -p -n 1 truncate -s 0

-p prompt before execute truncate

Better way is manual truncate

Manual truncate:

: > /proc/28680/fd/113

or:

> /proc/28680/fd/113

or:

truncate -s 0 /proc/28680/fd/113

Enjoy ;)

Solution 2

Here is a simple example with less:

Let's assume we have a file, my10MBfile:

$ dd if=/dev/zero of=/tmp/my10MBfile bs=1M count=10
10+0 enregistrements lus
10+0 enregistrements écrits
10485760 octets (10 MB) copiés, 0,0454491 s, 231 MB/s

$ ls -l /tmp/my10MBfile
-rw-r--r-- 1 max max 10485760 avril 23 22:49 /tmp/my10MBfile

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9225       4662  67% /

Now I open that file with less (yes, it is a binary file... never mind)

$ less /tmp/my10MBfile &

$ lsof -p $(pidof less) | grep 10MBfile
less    29351  max    4r   REG    8,3 10485760 521464 /tmp/my10MBfile

Then I remove that file

$ rm /tmp/my10MBfile

$ lsof -p $(pidof less) | grep 10MBfile
less    29351  max    4r   REG    8,3 10485760 521464 /tmp/my10MBfile (deleted)

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9225       4662  67% /

It is still there, but deleted. Look at the 4th column of the lsof output: File Descriptor number 4 open for Reading (4r)

Let's run GDB!

$ gdb -p $(pidof less)

GNU gdb (GDB) 7.4.1-debian
....
Attaching to process 29351
....

(gdb) p close(4)
$1 = 0
(gdb) q

That's it!

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9215       4672  67% /

Our 10 MB are welcome back :)

$ ls /proc/29351/fd
0  1  2  3

$ ps 29351
29351 pts/0    S+     0:00 less /tmp/my10MBfile

The process is still running.

Solution 3

This command will show all deleted files still open on a Solaris system:

find /proc/*/fd -type f -links 0

You can truncate the ones you are sure you want with this command:

:> /proc/p/fd/x

with p being the process id and x the file descriptor returned by the first command.

Don't worry if with some programs the size reported by ls is restored to the size before truncation after a while, the actual size used on disk will be much smaller as the file is now a sparse one.

Comments

[jlliagre]

@Mike cp /dev/null is a null command as cp has nothing to copy, a simple redirection is strictly equivalent :>file or even >file

[Telion]

@Neil Lunn I am not retrieving any data from MongoDB. I am creating a new object. The "duplicate" is not related to my question at all.

[Neil Lunn]

Show the surrounding code where you get the object from and making the assignment then. Your question currently gives no indication of where newMessage even comes from. Don't blame us for calling out when it "looks like a duck". It's up to you to show a clear example.

[Telion]

@Neil Lunn noone asked me to provide more code, it was just aggressively marked as a duplicate. I updated the question, there is nothing else in the code that anyhow may affect the process.

[Neil Lunn]

So you don't actually assign newMessage.message anywhere right? And the "only" code goes straight to newMessage.message.to = <something>. Because that's what it's looking like here. Also tone down your language. Nobody is "aggressively" doing anything.

[Telion]

@Neil Lunn yes I am not assigning it anywhere except where you can see. So this is really weird. How can only 1 symbol change and why?

[maxxvw]

do you have an exemple where this method works? Cannot find a way to alter FD from there

[ujjain]

Yes, this works, but how to find the pid of the process?

[longneck]

Ok, but for how long? many processes simply exit if they can't write to their log file.

[maxxvw]

Logrotate can't do this for you?

[longneck]

logrotate sends the process a signal to close the log file and open a new one.

[79E09796]

+1, but I also required sudo to run these commands

[Telion]

Thanks for the quick response. I tried but nothing changed.

[Telion]

Oh... I never thought about that. Thanks so much! So I should use string to store these IDs? How does Discord store them? (theoretically)

[Rodrigo Soriano]

Yes, discord uses ids to recognize users: For example if you are trying to register a new user with the id = '123' Discord will save that user like this: '123': username:'abc' etc...(json format) And basically you cannot use numbers as names for your json fields, that's why you use strings instead! (And the precision is a problem too!)

[Sergey Nemchinov]

Try this way to see all deleted files: lsof +L1. You can filter the output with grep