Where is the .NET JIT-compiled code cached?

Question

A .NET program is first compiled into MSIL code. When it is executed, the JIT compiler will compile it into native machine code.

I am wondering:

Where is these JIT-compiled machine code stored? Is it only stored in address space of the process? But since the second startup of the program is much faster than the first time, I think this native code must have been stored on disk somewhere even after the execution has finished. But where?

Solution 1

Memory. It can be cached, that's the job of ngen.exe. It generates a .ni.dll version of the assembly, containing machine code and stored in the GAC. Which automatically gets loaded afterward, bypassing the JIT step.

But that has little to do with why your program starts faster the 2nd time. The 1st time you have a so-called "cold start". Which is completely dominated by the time spent on finding the DLLs on the hard drive. The second time you've got a warm start, the DLLs are already available in the file system cache.

Disks are slow. An SSD is an obvious fix.

Fwiw: this is not a problem that's exclusive to managed code. Large unmanaged programs with lots of DLLs have it too. Two canonical examples, present on most dev machines are Microsoft Office and Acrobat Reader. They cheat. When installed, they put an "optimizer" in the Run registry key or the Startup folder. All that these optimizers do is load all the DLLs that the main program uses, then exit. This primes the file system cache, when the user subsequently uses the program, it will start up quickly since its warm start is fast.

Personally, I find this extraordinarily annoying. Because what they really do is slow down any other program that I may want to start after logging in. Which is rarely Office or Acrobat. I make it a point to delete these optimizers, repeatedly if necessary when a blasted update puts it back.

You can use this trick too, but use it responsibly please.

Solution 2

As others have pointed out, code is JIT'd on a per process basis in your case, and is not cached - the speed-up you are seeing on second load is OS disk caching (i.e. in-memory) of the assemblies.

However, whilst there is no caching (apart from OS disk caching) in the desktop\server version of the framework, there is caching of JIT'd machine code in another version of the framework.

Of interest is what is happening in the .Net Compact Framework (NETCF for Windows Phone 7 relase). Recent advances see sharing of some JIT'd framework code between processes where the JIT'd code is indeed cached. This has been primarily carried out for better performance (load time and memory usage) in constrained devices such as mobile phones.

So in answer to the question there is no direct framework caching of JIT'd code in the desktop\server version of the CLR, but there will be in the latest version of the compact framework i.e. NETCF.

Reference: We Believe in Sharing

http://blogs.msdn.com/b/abhinaba/archive/2010/04/28/we-believe-in-sharing.aspx

Solution 3

JIT compiled machine code is cached in memory per-method, each time that a method is executed for the first time. I don't think it is ever cached to disk.

You may find that the process is faster to load the second time because Windows cached (in memory) the files used by your process (dlls, resources etc etc) on the first run. On the second run there is no need to go to disk, where this may have been done on the first run.

You could confirm this by running NGen.exe to actually pre-compile the machine code for your architecture, and compare the performance of the first and second runs. My bet is that the second run would still be faster, due to caching in the OS.

Solution 4

In short, the IL is JIT-compiled for each invocation of the program and is maintained in code pages of the process address space. See Chapter 1 of Richter for great coverage of the .NET execution model.

Solution 5

Yes, NGEN.EXE will place a JIT compiled version of a .NET executable in the GAC, even when the MSIL version is not there. I have tried that, but to no avail. I believe, unless the original MSIL version is also in the GAC and would be loaded from there, the JIT version in the GAC will not be used. I also believe that on-the-fly JIT compiles (not NGEN) are never cached; they occupy process memory only.

I believe this from reading the MS doc and from various experiments. I would welcome either a confirmation or rebuttal of my assertions from those "who know".

View more solutions

Comments

[Ben Voigt]

Only kernel drivers can prevent code from being swapped out. JIT compiled code can be paged out just like any other page in application memory.

[Zan Lynx]

Vista and Windows 7 have this DLL caching trick built in, so you don't need to make "app accelerators" anymore. Superfetch records what applications you use most and will read those files in after boot. It's better than an accelerator app because it pays attention to current memory use and actual application use.

[Sreekumar P]

@Hans: You told "It generates a .ni.dll version of the assembly, containing machine code and stored in the GAC" . I have some doubt.. GAC is only for storing public assemblies, right ? Is it stores a Native Image of a exe that I am running ?

[Sreekumar P]

Do you know where windows is caching these files used by process (dlls, resources etc etc) ?

[user1703401]

Yes, that's what the answer says. It is hidden from view but you can see it by starting the command prompt. Type dir c:\windows\assembly to list the directories in the GAC.

[SamB]

@Sreekumar: In RAM, the same as just about every other OS these days.