Close an open h5py data file
Solution 1
This is how it could be done (I could not figure out how to check for closed-ness of the file without exceptions, maybe you will find):
import gc
for obj in gc.get_objects(): # Browse through ALL objects
if isinstance(obj, h5py.File): # Just HDF5 files
try:
obj.close()
except:
pass # Was already closed
Another idea:
Dpending how you use the files, what about using the context manager and the with
keyword like this?
with h5py.File("some_path.h5") as f:
f["data1"] = some_data
When the program flow exits the with-block, the file is closed regardless of what happens, including exceptions etc.
Solution 2
pytables
(which h5py
uses) keeps track of all open files and provides an easy method to force-close all open hdf5
files.
import tables
tables.file._open_files.close_all()
That attribute _open_files
also has helpful methods to give you information and handlers for the open files.
Solution 3
I've found that hFile.bool() returns True if open, and False otherwise. This might be the simplest way to check. In other words, do this:
hFile = h5py.File(path_to_file)
if hFile.__bool__():
hFile.close()
Related videos on Youtube
Adriaan Rol
Updated on September 09, 2020Comments
-
Adriaan Rol over 3 years
In our lab we store our data in
hdf5
files trough the python packageh5py
.At the beginning of an experiment we create an
hdf5
file and store array after array of array of data in the file (among other things). When an experiment fails or is interrupted the file is not correctly closed. Because our experiments run fromiPython
the reference to the data object remains (somewhere) in memory.Is there a way to scan for all open h5py data objects and close them?
-
zerocog over 7 yearsThanks for the info on the "Another idea". I was just searching to make sure my h5py.Files were closing with the "with"
-
rhedak over 4 yearsThank you that was exactly what I needed. For some reason the previous answer didn't work for me but this one is much easier anyways :-)
-
LemmeTestThat almost 4 yearsThis seems true, but is there any description of this behavior in the official documentation? It could be risky if some unrelated conditions also affect the value of __bool__()
-
Avinash Pujala almost 4 yearsNot that I am aware of unforunately!