cx-freeze doesn't find all dependencies
20,044
Solution 1
In case anyone ever runs across this, the problem seemed to be that cxfreeze does not play nicely with eggs. Once I extracted the code out of the egg (renamed it to a zip and unzipped), and, created for example: Python27\Lib\site-packages\socketpool for the code, then ran the build, things seemed to work fine.
Seems like a hack. Time for a shower.
Solution 2
I know this thread is old but I just spent 4 days figuring this out and it might be helpful for someone else so here goes:
The description for include_files option in cx_freeze is misleading. If you want to use the include_files option for build_exe here is an example of how you do that
include_files=[
(r"C:\Python27\Scripts\mk2ifcoremd.dll", "mk2ifcoremd.dll"),
(r"C:\Python27\Scripts\mk2ifportmd.dll", "mk2ifportmd.dll"),
(r"C:\Python27\Scripts\mk2imalloc.dll", "mk2imalloc.dll"),
(r"C:\Python27\Scripts\mk2iomp5md.dll", "mk2iomp5md.dll"),
(r"C:\Python27\Scripts\mk2mmd.dll", "mk2mmd.dll"),]
You must have a list of tuples with an absolute path for the source and ONLY THE FILENAME for the destination. I ran into trouble with cx_freeze because it did/ does not copy all the required DLLs into the dist directory. So my exe tested fine on my machine but did not work on another machine that did not have Python installed. In order to debug this I had to repeatedly move my Python installation out of the way (renamed C:\Python27 to C:_Python27) and see if there were missing libraries.
Here is an example setup.py file that works for me:
# invoke using:
# python setup.py build
from cx_Freeze import setup, Executable
import sys
import glob
import os
import zlib
import shutil
# Remove the existing folders folder
shutil.rmtree("build", ignore_errors=True)
shutil.rmtree("dist", ignore_errors=True)
########################################
# Here is a list of the Executable options
########################################
#"script": #the name of the file containing the script which is to be frozen
#"initScript": #the name of the initialization script that will be executed before the actual script is executed; this script is used to set up the environment for the executable; if a name is given without an absolute path the names of files in the initscripts subdirectory of the cx_Freeze package is searched
#"base": #the name of the base executable; if a name is given without an absolute path the names of files in the bases subdirectory of the cx_Freeze package is searched
#"path": #list of paths to search for modules
#"targetDir": #the directory in which to place the target executable and any dependent files
#"targetName": #the name of the target executable; the default value is the name of the script with the extension exchanged with the extension for the base executable
#"includes": #list of names of modules to include
#"excludes": #list of names of modules to exclude
#"packages": #list of names of packages to include, including all of the package's submodules
#"replacePaths": #Modify filenames attached to code objects, which appear in tracebacks. Pass a list of 2-tuples containing paths to search for and corresponding replacement values. A search for '*' will match the directory containing the entire package, leaving just the relative path to the module.
#"compress": #boolean value indicating if the module bytecode should be compressed or not
#"copyDependentFiles": #boolean value indicating if dependent files should be copied to the target directory or not
#"appendScriptToExe": #boolean value indicating if the script module should be appended to the executable itself
#"appendScriptToLibrary":#boolean value indicating if the script module should be appended to the shared library zipfile
#"icon": #name of icon which should be included in the executable itself on Windows or placed in the target directory for other platforms
#"namespacePackages": #list of packages to be treated as namespace packages (path is extended using pkgutil)
#"shortcutName": #the name to give a shortcut for the executable when included in an MSI package
#"shortcutDir": #the directory in which to place the shortcut when being installed by an MSI package; see the MSI Shortcut table documentation for more information on what values can be placed here.
MY_TARGET_EXE = Executable(
# what to build
script = "main.py",
initScript = None,
base = 'Win32GUI',
targetDir = r"dist",
targetName = "MyProgram.exe",
compress = True,
copyDependentFiles = True,
appendScriptToExe = False,
appendScriptToLibrary = False,
icon = None
)
########################################
#Here is a list of the build_exe options
########################################
#1) append the script module to the executable
append_script_to_exe=False
#2) the name of the base executable to use which, if given as a relative path, will be joined with the bases subdirectory of the cx_Freeze installation; the default value is "Console"
base="Console"
#3) list of names of files to exclude when determining dependencies of binary files that would normally be included; note that version numbers that normally follow the shared object extension are stripped prior to performing the comparison
bin_excludes=[]
#4) list of names of files to include when determining dependencies of binary files that would normally be excluded; note that version numbers that normally follow the shared object extension are stripped prior to performing the comparison
bin_includes=[]
#5) list of paths from which to exclude files when determining dependencies of binary files
bin_path_excludes=[]
#6) list of paths from which to include files when determining dependencies of binary files
bin_path_includes=[]
#7) directory for built executables and dependent files, defaults to build/
build_exe="dist/"
#8) create a compressed zip file
compressed=False
#9) comma separated list of constant values to include in the constants module called BUILD_CONSTANTS in form <name>=<value>
constants=[]
#10) copy all dependent files
copy_dependent_files=True
#11) create a shared zip file called library.zip which will contain all modules shared by all executables which are built
create_shared_zip=True
#12) comma separated list of names of modules to exclude
excludes = ['_gtkagg', '_tkagg', 'bsddb', 'curses', 'email', 'pywin.debugger',
'pywin.debugger.dbgcon', 'pywin.dialogs', 'tcl',
'Tkconstants', 'Tkinter']
#13) include the icon in the frozen executables on the Windows platform and alongside the frozen executable on other platforms
icon=False
#13) comma separated list of names of modules to include
includes = ['sip', 'matplotlib.backends.backend_wxagg']
#15) list containing files to be copied to the target directory;
# it is expected that this list will contain strings or 2-tuples for the source and destination;
# the source can be a file or a directory (in which case the tree is copied except for .svn and CVS directories);
# the target must not be an absolute path
#
# NOTE: INCLUDE FILES MUST BE OF THIS FORM OTHERWISE freezer.py line 128 WILL TRY AND DELETE dist/. AND FAIL!!!
# Here is a list of ALL the DLLs that are included in Python27\Scripts
include_files=[
(r"C:\Python27\Scripts\mk2ifcoremd.dll", "mk2ifcoremd.dll"),
(r"C:\Python27\Scripts\mk2ifportmd.dll", "mk2ifportmd.dll"),
(r"C:\Python27\Scripts\mk2imalloc.dll", "mk2imalloc.dll"),
(r"C:\Python27\Scripts\mk2iomp5md.dll", "mk2iomp5md.dll"),
(r"C:\Python27\Scripts\mk2mmd.dll", "mk2mmd.dll"),
(r"C:\Python27\Scripts\mk2_avx.dll", "mk2_avx.dll"),
(r"C:\Python27\Scripts\mk2_blacs_ilp64.dll", "mk2_blacs_ilp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_intelmpi_ilp64.dll", "mk2_blacs_intelmpi_ilp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_intelmpi_lp64.dll", "mk2_blacs_intelmpi_lp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_lp64.dll", "mk2_blacs_lp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_mpich2_ilp64.dll", "mk2_blacs_mpich2_ilp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_mpich2_lp64.dll", "mk2_blacs_mpich2_lp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_msmpi_ilp64.dll", "mk2_blacs_msmpi_ilp64.dll"),
(r"C:\Python27\Scripts\mk2_blacs_msmpi_lp64.dll", "mk2_blacs_msmpi_lp64.dll"),
(r"C:\Python27\Scripts\mk2_cdft_core.dll", "mk2_cdft_core.dll"),
(r"C:\Python27\Scripts\mk2_core.dll", "mk2_core.dll"),
(r"C:\Python27\Scripts\mk2_def.dll", "mk2_def.dll"),
(r"C:\Python27\Scripts\mk2_intel_thread.dll", "mk2_intel_thread.dll"),
(r"C:\Python27\Scripts\mk2_mc.dll", "mk2_mc.dll"),
(r"C:\Python27\Scripts\mk2_mc3.dll", "mk2_mc3.dll"),
(r"C:\Python27\Scripts\mk2_p4n.dll", "mk2_p4n.dll"),
(r"C:\Python27\Scripts\mk2_pgi_thread.dll", "mk2_pgi_thread.dll"),
(r"C:\Python27\Scripts\mk2_rt.dll", "mk2_rt.dll"),
(r"C:\Python27\Scripts\mk2_scalapack_ilp64.dll", "mk2_scalapack_ilp64.dll"),
(r"C:\Python27\Scripts\mk2_scalapack_lp64.dll", "mk2_scalapack_lp64.dll"),
(r"C:\Python27\Scripts\mk2_sequential.dll", "mk2_sequential.dll"),
(r"C:\Python27\Scripts\mk2_vml_avx.dll", "mk2_vml_avx.dll"),
(r"C:\Python27\Scripts\mk2_vml_def.dll", "mk2_vml_def.dll"),
(r"C:\Python27\Scripts\mk2_vml_mc.dll", "mk2_vml_mc.dll"),
(r"C:\Python27\Scripts\mk2_vml_mc2.dll", "mk2_vml_mc2.dll"),
(r"C:\Python27\Scripts\mk2_vml_mc3.dll", "mk2_vml_mc3.dll"),
(r"C:\Python27\Scripts\mk2_vml_p4n.dll", "mk2_vml_p4n.dll"),
# These next DLLs appear to be copied correctly or as needed by cxfreeze...
# (r"C:\Python27\Scripts\libgcc_s_sjlj-1.dll", "libgcc_s_sjlj-1.dll"),
# (r"C:\Python27\Scripts\libgfortran-3.dll", "libgfortran-3.dll"),
# (r"C:\Python27\Scripts\libssp-0.dll", "libssp-0.dll"),
# (r"C:\Python27\Scripts\libstdc++-6.dll", "libstdc++-6.dll"),
# (r"C:\Python27\Scripts\pythoncom27.dll", "pythoncom27.dll"),
# (r"C:\Python27\Scripts\pywintypes27.dll", "pywintypes27.dll"),
]
#,("Microsoft.VC90.MFC", mfcfiles), ]
#16) include the script module in the shared zip file
include_in_shared_zip=True
#17) include the Microsoft Visual C runtime DLLs and (if necessary) the manifest file required to run the executable without needing the redistributable package installed
include_msvcr =False
#18) the name of the script to use during initialization which, if given as a relative path, will be joined with the initscripts subdirectory of the cx_Freeze installation; the default value is "Console"
init_script=""
#19) comma separated list of packages to be treated as namespace packages (path is extended using pkgutil)
namespace_packages=[]
#20) optimization level, one of 0 (disabled), 1 or 2
optimize=0
#21) comma separated list of packages to include, which includes all submodules in the package
packages = ['numpy.linalg']
#22) comma separated list of paths to search; the default value is sys.path
path = []
#23) Modify filenames attached to code objects, which appear in tracebacks. Pass a comma separated list of paths in the form <search>=<replace>. The value * in the search portion will match the directory containing the entire package, leaving just the relative path to the module.
replace_paths=[]
#24) suppress all output except warnings
silent=False
#25) list containing files to be included in the zip file directory; it is expected that this list will contain strings or 2-tuples for the source and destination
zip_includes=[]
setup(
version = "0.0",
description = "This is a program that works",
author = "Your Name Here",
name = "A text description",
options = {"build_exe": {
# "append_script_to_exe": append_script_to_exe,
# "base": base,
# "bin_excludes": bin_excludes,
# "bin_includes": bin_includes,
# "bin_path_excludes": bin_path_excludes,
# "bin_path_includes": bin_path_includes,
"build_exe": build_exe,
"compressed": compressed,
# "constants": constants,
"copy_dependent_files": copy_dependent_files,
# "create_shared_zip": create_shared_zip,
"excludes": excludes,
# "icon": icon,
"includes": includes,
"include_files": include_files,
# "include_in_shared_zip":include_in_shared_zip,
# "include_msvcr": include_msvcr,
# "init_script": init_script,
# "namespace_packages": namespace_packages,
# "optimize": optimize,
"packages": packages,
"path": path,
# "replace_paths": replace_paths,
# "silent": silent,
# "zip_includes": zip_includes,
}
},
executables = [MY_TARGET_EXE]
)
Author by
Doo Dah
Updated on August 13, 2022Comments
-
Doo Dah almost 2 years
I have a python script (2.7) with some "interesting" imports at the top. I initially wanted to use py2exe to compile this into an exe that I can more easily distribute (py2exe doesn't find all dependencies).
I have given up and am trying to use cx-freeze instead. But, I am having problem there as well. The problems seem to be libraries I have added to Python (jinja2 and restkit). I see them in my python directory ./Lib/site-packages/Jinja2-2.6-py2.7.egg/jinja2 and here ./Lib/site-packages/restkit-4.2.1-py2.7.egg/restkit.
Here are the imports in my script:
import datetime from jinja2 import Environment, PackageLoader from optparse import OptionParser from datetime import date, timedelta from restkit import Resource, BasicAuth, requestI am using a setup.py with cx-freeze. Here is the setup.py:
from cx_Freeze import setup, Executable packages = ["restkit", "jinja2" , "restkit.client" ] includes = [] includefiles = [] eggsacutibull = Executable( script = "myScript.py", initScript = None, targetName = "myScript.exe", compress = True, copyDependentFiles = True, appendScriptToExe = False, appendScriptToLibrary = False, icon = None ) setup( name = "myScript", version = "0.1", author = 'vickery', description = "MyScript description", options = {"build_exe": {"includes":includes, "include_files": includefiles, "packages": packages}}, executables = [eggsacutibull] )I run cxfreeze like this:
cxfreeze myScript.py --target-dir exeI get this in my build:
Missing modules: ? __pypy__ imported from jinja2.debug ? http_parser.http imported from restkit.client ? jinja2._debugsupport imported from jinja2.debug ? jinja2._markupsafe._speedups imported from jinja2._markupsafe ? jinja2.debugrenderer imported from jinja2.debug ? markupsafe imported from jinja2.utils ? pretty imported from jinja2.utils ? socketpool imported from restkit.connAnd, when I try to run the exe, I get this:
Traceback (most recent call last): File "c:\Python27\lib\site-packages\restkit-4.2.1-py2.7.egg\restkit\__init__.py", line 9, in <module> from restkit.conn import Connection File "c:\Python27\lib\site-packages\restkit-4.2.1-py2.7.egg\restkit\conn.py", line 14, in <module> from socketpool import Connector ImportError: No module named socketpool Traceback (most recent call last): File "c:\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py", line 27, in <module> exec code in m.__dict__ File "myScript.py", line 12, in <module> ImportError: cannot import name ResourceEdit: I am now running cxfreeze correctly like this:
python setup.py buildI also added socketpool to my setup.py:
packages = [ "restkit", "jinja2" , "restkit.client", "restkit.conn", "socketpool" ]But, when I try to build now, I get a build error:
$ python setup.py build running build running build_exe Traceback (most recent call last): File "setup.py", line 32, in <module> executables = [eggsacutibull] File "c:\python27\lib\site-packages\cx_Freeze\dist.py", line 365, in setup distutils.core.setup(**attrs) File "c:\python27\lib\distutils\core.py", line 152, in setup dist.run_commands() File "c:\python27\lib\distutils\dist.py", line 953, in run_commands self.run_command(cmd) File "c:\python27\lib\distutils\dist.py", line 972, in run_command cmd_obj.run() File "c:\python27\lib\distutils\command\build.py", line 127, in run self.run_command(cmd_name) File "c:\python27\lib\distutils\cmd.py", line 326, in run_command self.distribution.run_command(command) File "c:\python27\lib\distutils\dist.py", line 972, in run_command cmd_obj.run() File "c:\python27\lib\site-packages\cx_Freeze\dist.py", line 235, in run freezer.Freeze() File "c:\python27\lib\site-packages\cx_Freeze\freezer.py", line 570, in Freeze self.finder = self._GetModuleFinder() File "c:\python27\lib\site-packages\cx_Freeze\freezer.py", line 325, in _GetModuleFinder finder.IncludePackage(name) File "c:\python27\lib\site-packages\cx_Freeze\finder.py", line 534, in IncludePackage module = self._ImportModule(name, deferredImports) File "c:\python27\lib\site-packages\cx_Freeze\finder.py", line 274, in _ImportModule raise ImportError("No module named %r" % name) ImportError: No module named 'socketpool'What has me confused here, is, my script compiles just fine. In addition, I can import these modules from a python shell. For example:
$ python Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socketpool >>> from restkit import Resource, BasicAuth, request >>>How is python resolving modules that is differnt than cxfreeze?
Edit2: From python I can do this:
>>> import socketpool >>> print socketpool.__file__ c:\python27\lib\site-packages\socketpool-0.5.2-py2.7.egg\socketpool\__init__.pycIs that a non-standard place to look for a package? Can I used PYTHONPATH to coerce cxfreez into looking there for socketpool?
Thanks
-
SilentSteel almost 11 yearsPlease give more details! I'm new to Python and don't know what an egg is, but I think this is my problem. Can you give detailed instructions? Mine also keeps looking in C:\Python27 -
Doo Dah almost 11 yearscx-freeze could not look inside the egg to find what it needed, so, I renamed the egg to zip and unzipped. That was for windows. On linux, you could just unzip the egg directly. Of course, if you install a new version of a library, the new code will be in an egg, and, not picked up by cx-freeze.
-
Thomas K over 10 yearsThere is indeed a bug finding modules inside eggs that are zip files. I've got a pull request that should fix it.
