lxml will never finish building on ubuntu

python ubuntu lxml
16,714

Solution 1

Note that if you run top in a separate terminal you'll notice that your system is busy building the lxml package:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1645 root 20 0 322224 305740 5384 R 100.0 29.8 3:27.47 /usr/lib/gcc/arm-linux-gnueabihf/5/cc1 -quiet -I /usr/include/libxml2 -I src/lxml/includes -I /usr/include/python2.7 -imultiarch arm-linux-gnueabihf -D_REENTRANT -D ...

If you're building on a VM or SBC with limited resources this can take a while...

Solution 2

I ran into the same problem when trying to "pip install" a package that depends on lxml on a Raspberry Pi 1 B+. After several hours (and several attempts), the SSH connection dropped without finishing the wheel build. It seems like an out-of-memory situation.

I opted on installing the pre-built binary packages using apt install python-lxml or apt install python3-lxml respectively.

Solution 3

I ran this on VM with 768MB ram (which i didn't notice was the value i set) and it took several hours and didn't finish.

Changing ram to 3GB made it finish in few minutes after restarting and trying again.

Solution 4

The newest version of pip, unnecessarily spends excessive amount of time to build wheels and resolve dependency issues while building a package with requirements.txt or a setuptools/distutils setup.py

A quick fix is to revert to a version of pip prior to 21.0 (ideally in a virtual environment), where the legacy resolver is still supported, according to this discussion: sunsetting legacy resolver

As of July 28th, 2021 pip v21.2.1 still seems to have the legacy resolver support.

pip install pip==20.2.4

Command below uses an earlier version of dependency resolver to install lxml.

pip install --use-deprecated=legacy-resolver lxml

Saved me a lot of time and hassle.

Share:
16,714
Rorschach
Author by

Rorschach

Fullstack Engineer

Updated on June 06, 2022

Comments

  • Rorschach
    Rorschach about 2 years

    I am running ubuntu 14.04 LTS and python 2.7.5 on a vwmare.

    When I run:

    sudo pip install lxml

    I get:

    Collecting lxml
  Using cached lxml-3.4.4.tar.gz
Building wheels for collected packages: lxml
  Running setup.py bdist_wheel for lxml

    which runs eternally.

    I can't find any resources related to this, nor do I have any experience with similar errors.

    Note, the solution below does not work:

    sudo apt-get build-dep -y lxml;
pip install lxml

    I have also tried:

    sudo apt-get install -y build-essential python-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version.
python-dev is already the newest version.
The following packages were automatically installed and are no longer required:
  linux-headers-3.13.0-32 linux-headers-3.13.0-32-generic
  linux-headers-3.13.0-57 linux-headers-3.13.0-57-generic
  linux-image-3.13.0-32-generic linux-image-3.13.0-57-generic
  linux-image-extra-3.13.0-32-generic linux-image-extra-3.13.0-57-generic
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 75 not upgraded.

    EDIT:

    I just ran it again and got:

    sudo pip install lxml

Collecting lxml
/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Using cached lxml-3.4.4.tar.gz
Building wheels for collected packages: lxml
  Running setup.py bdist_wheel for lxml

    EDIT 2:

     pip install lxml

The directory '/home/myusername/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/myusername/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting lxml
/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading lxml-3.4.4.tar.gz (3.5MB)
    100% |████████████████████████████████| 3.5MB 7.0kB/s 
Installing collected packages: lxml
  Running setup.py install for lxml

    EDIT 3: After waiting a long time the command gave:

    pip install lxml
The directory '/home/mstaffel/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/mstaffel/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting lxml
/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading lxml-3.4.4.tar.gz (3.5MB)
    100% |████████████████████████████████| 3.5MB 7.0kB/s 
Installing collected packages: lxml
  Running setup.py install for lxml
    Complete output from command /usr/local/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-u3iJ67/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-0tAWPp-record/install-record.txt --single-version-externally-managed --compile:
    /usr/local/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.4.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-i686-2.7
    creating build/lib.linux-i686-2.7/lxml
    copying src/lxml/pyclasslookup.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/cssselect.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/__init__.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/_elementpath.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/doctestcompare.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/builder.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/ElementInclude.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/usedoctest.py -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/sax.py -> build/lib.linux-i686-2.7/lxml
    creating build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/__init__.py -> build/lib.linux-i686-2.7/lxml/includes
    creating build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/__init__.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/_diffcommand.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/soupparser.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/builder.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/usedoctest.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/html5parser.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/_setmixin.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/_html5builder.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/defs.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/clean.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/ElementSoup.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/diff.py -> build/lib.linux-i686-2.7/lxml/html
    copying src/lxml/html/formfill.py -> build/lib.linux-i686-2.7/lxml/html
    creating build/lib.linux-i686-2.7/lxml/isoschematron
    copying src/lxml/isoschematron/__init__.py -> build/lib.linux-i686-2.7/lxml/isoschematron
    copying src/lxml/lxml.etree.h -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/lxml.etree_api.h -> build/lib.linux-i686-2.7/lxml
    copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/tree.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/schematron.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xinclude.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/c14n.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xpath.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xslt.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/uri.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/relaxng.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/config.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/etree_defs.h -> build/lib.linux-i686-2.7/lxml/includes
    copying src/lxml/includes/lxml-version.h -> build/lib.linux-i686-2.7/lxml/includes
    creating build/lib.linux-i686-2.7/lxml/isoschematron/resources
    creating build/lib.linux-i686-2.7/lxml/isoschematron/resources/rng
    copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/rng
    creating build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl
    copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl
    copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl
    creating build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    running build_ext
    building 'lxml.etree' extension
    creating build/temp.linux-i686-2.7
    creating build/temp.linux-i686-2.7/src
    creating build/temp.linux-i686-2.7/src/lxml
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-u3iJ67/lxml/src/lxml/includes -I/usr/local/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.7/src/lxml/lxml.etree.o -w
    gcc -pthread -shared build/temp.linux-i686-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.7/lxml/etree.so
    building 'lxml.objectify' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-u3iJ67/lxml/src/lxml/includes -I/usr/local/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-i686-2.7/src/lxml/lxml.objectify.o -w
    gcc -pthread -shared build/temp.linux-i686-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.7/lxml/objectify.so
    running install_lib
    creating /usr/local/lib/python2.7/site-packages/lxml
    error: could not create '/usr/local/lib/python2.7/site-packages/lxml': Permission denied

    ----------------------------------------
Command "/usr/local/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-u3iJ67/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-0tAWPp-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-u3iJ67/lxml

Recents

Why Is PNG file with Drop Shadow in Flutter Web App Grainy?
How to troubleshoot crashes detected by Google Play Store for Flutter app
Cupertino DateTime picker interfering with scroll behaviour
Why does awk -F work for most letters, but not for the letter "t"?
Flutter change focus color and icon color but not works
How to print and connect to printer using flutter desktop via usb?
Critical issues have been reported with the following SDK versions: com.google.android.gms:play-services-safetynet:17.0.0
Flutter Dart - get localized country name from country code
navigatorState is null when using pushNamed Navigation onGenerateRoutes of GetMaterialPage
Android Sdk manager not found- Flutter doctor error
Flutter Laravel Push Notification without using any third party like(firebase,onesignal..etc)
How to change the color of ElevatedButton when entering text in TextField

Related

How to install lxml on Ubuntu
python parse html table using lxml
Install Microsoft ODBC Driver 13 for SQL Server in Ubuntu 16.04
lxml (or lxml.html): print tree structure
How to parse broken HTML with LXML
how to install cloud9 IDE on ubuntu server
No module named 'lxml' Windows 8.1
How to update XML file with lxml
Just returning the text of elements in xpath (python / lxml)
pyqtgraph - ImportError: No module named pyqtgraph