A way for saving running state of a program

Question

I have a program that will perform a lengthy calculation which may take several days to be completed.

Is there a way to save the program's running state; so I will be able to turn off the computer and run the program from a saved state. Or if the program crashed for any reason I can restart it from a saved state (and not from the outset)?

Solution 1

Making restartable snapshots of a process is very hard, because a process can have all kinds of interactions with the filesystem. As a rule, don't expect unix systems to support this. There have been unix variants with restartable core dumps, but I don't think this is the case on modern ones (they've become too complex).

Lisp systems typically have a dump command that creates a restartable image. So you could write your program in a Lisp dialect that supports dumping.

You can run your code in a virtual machine and use the virtual machine manager (e.g. VirtualBox) to create periodic snapshots. Depending on what your program does, this may or may not hurt performance.

The best solution is probably for you to build a snapshot feature into your program. For purely computational programs, this is often only moderatly difficult. In a multithreaded computational program, snapshot points are typically global synchronization points, where all the threads communicate. Try to structure your program as a bag of tasks, and make the entry point a dispatcher which starts a task whenever a processor is free. Upon receipt of a signal, the dispatcher waits for all current tasks to finish, saves the program states, and starts dispatching tasks again.

Solution 2

I'd suggest hibernating instead of shutting down your computer, but if you really cannot avoid this there exists a snapshot/restore system called CRIU (Checkpoint/Restore in Userspace), that could suit your needs. With this framework you could quite easily »freeze« your processes (actually saving their state to disk) and later defrost them again.

I didn't extensively test this yet, but they seem to have some rather good documentation, so you might find that helpful. Also note that for using CRIU you might have to enable several kernel features which most probably aren't enabled on most stock distributions (CONFIG_CHECKPOINT_RESTORE and CONFIG_MEM_SOFT_DIRTY), so chance is good you'll need to compile your own kernel to get it running.

Note: If your program crashes for a reason freezing it beforehand is probably not the way to go, since most probably your program will run into the same conditions again (in case it's no external reason like full disk or memory).

Comments

[Admin]

To prevent process hangup when you need to turn off your computer, you could hibernate instead.