Increasing no of file handles in Windows 7 64 bit

Question

I have an application that has 10000 threads running at a time. Each thread opens the same file. The problem is whenever I launch the application with 10K threads, the application terminates after creating 500 threads(file handles). I have tried the same application on Linux and is running fine after I tweaked the ulimit option. Is there any limit on the file handles a process can open in Windows? I have been googling and all I get is to change the entries in config.nt file in C\Windows\System32....

But I found out that the said file does not exist for 64 bit OS. Is there any way that I can change the limit in Windows?

My OS is WINDOWS 7 64 bit.

Solution 1

To view the total number of handles (not just file handles) your application opened at a given time: Just to make sure it's the handle limit.

Download Process Explorer from https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx Make sure to set appropiate refresh speed. Open it and go to View -> Select Columns -> press on the tab "Process Performance"and click on "Handle Count".

For Windows 7 x64 bit, a process can have 16.711.680 handles opened simultaneously. If you want to check limits for yourself then read below. Check that by using a tool from Windows Internals Book (https://technet.microsoft.com/en-us/sysinternals/bb963901.aspx). Tool's name is TestLimit and you will find it in the lower part of the page under the Book Tools header.

There are no ways to increase this limit for Windows Operating Systems as far as I know, and I looked also.

As others stated, think of a method to minimize the large number of threads. Maybe your application closes the file, but not the handle. My advice, if you really need using very large handle count, start a new process every time handle count is about 16m.

Solution 2

running out of Microsoft C runtime library file descriptors (Harry Johnston) appears to be the right diagnosis. https://docs.microsoft.com/en-us/cpp/c-runtime-library/file-handling. The default max is 512 open file descriptors. The solution therefore is a line of code early in main:

_setmaxstdio(newmax);

How to do that in Python is another question.