"OSError: [Errno 1] Operation not permitted" when installing Scrapy in OSX 10.11 (El Capitan) (System Integrity Protection)
Solution 1
I also think it's absolutely not necessary to start hacking OS X.
I was able to solve it doing a
brew install python
It seems that using the python / pip that comes with new El Capitan has some issues.
Solution 2
pip install --ignore-installed six
Would do the trick.
Source: github.com/pypa/pip/issues/3165
Solution 3
As the other answers said, it's because of the new System Integrity Protection, but I believe the other answers are overcomplicated.
If you're only gonna use that package in the current user, you should be able to install it just fine, without the need to disable the SIP, by using the --user
flag. Like this:
sudo pip install --user packagename
Solution 4
The high voted answers didn't work for me, it seems to work for El Capitan users. But for MacOS Sierra users try the following steps
brew install python
sudo pip install --user <package name>
Solution 5
Warnings
I would suggest very strongly against modifying the system Python on Mac; there are numerous issues that can occur.
Your particular error shows that the installer has issues resolving the dependencies for Scrapy without impacting the current Python installation. The system uses Python for a number of essential tasks, so it's important to keep the system installation stable and as originally installed by Apple.
I would also exhaust all other possibilities before bypassing built in security.
Package Manager Solutions:
Please look into a Python virtualization tool such as virtualenv first; this will allow you to experiment safely.
Another useful tool to use languages and software without conflicting with your Mac OS is Homebrew. Like MacPorts or Fink, Homebrew is a package manager for Mac, and is useful for safely trying lots of other languages and tools.
"Roll your own" Software Installs:
If you don't like the package manager approach, you could use the /usr/local
path or create an /opt/local
directory for installing an alternate Python installation and fix up your paths in your .bashrc
. Note that you'll have to enable root for these solutions.
How to do it anyway:
If you absolutely must disable the security check (and I sincerely hope it's for something other than messing with the system languages and resources), you can disable it temporarily and re-enable it using some of the techniques in this post on how to Disable System Integrity-Protection.
Related videos on Youtube
Luis U.
Updated on October 22, 2020Comments
-
Luis U. over 3 years
I'm trying to install Scrapy Python framework in OSX 10.11 (El Capitan) via pip. The installation script downloads the required modules and at some point returns the following error:
OSError: [Errno 1] Operation not permitted: '/tmp/pip-nIfswi-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
I've tried to deactivate the rootless feature in OSX 10.11 with the command:
sudo nvram boot-args="rootless=0";sudo reboot
but I still get the same error when the machine reboots.
Any clue or idea from my fellow StackExchangers?
If it helps, the full script output is the following:
sudo -s pip install scrapy Collecting scrapy Downloading Scrapy-1.0.2-py2-none-any.whl (290kB) 100% |████████████████████████████████| 290kB 345kB/s Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /Library/Python/2.7/site-packages (from scrapy) Requirement already satisfied (use --upgrade to upgrade): queuelib in /Library/Python/2.7/site-packages (from scrapy) Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from scrapy) Collecting w3lib>=1.8.0 (from scrapy) Downloading w3lib-1.12.0-py2.py3-none-any.whl Collecting lxml (from scrapy) Downloading lxml-3.4.4.tar.gz (3.5MB) 100% |████████████████████████████████| 3.5MB 112kB/s Collecting Twisted>=10.0.0 (from scrapy) Downloading Twisted-15.3.0.tar.bz2 (4.4MB) 100% |████████████████████████████████| 4.4MB 94kB/s Collecting six>=1.5.2 (from scrapy) Downloading six-1.9.0-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Twisted>=10.0.0->scrapy) Requirement already satisfied (use --upgrade to upgrade): setuptools in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from zope.interface>=3.6.0->Twisted>=10.0.0->scrapy) Installing collected packages: six, w3lib, lxml, Twisted, scrapy Found existing installation: six 1.4.1 DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project. Uninstalling six-1.4.1: Exception: Traceback (most recent call last): File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/basecommand.py", line 223, in main status = self.run(options, args) File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/commands/install.py", line 299, in run root=options.root_path, File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/req/req_set.py", line 640, in install requirement.uninstall(auto_confirm=True) File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/req/req_install.py", line 726, in uninstall paths_to_remove.remove(auto_confirm) File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove renames(path, new_path) File "/Library/Python/2.7/site-packages/pip-7.1.0-py2.7.egg/pip/utils/__init__.py", line 314, in renames shutil.move(old, new) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move copy2(src, real_dst) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2 copystat(src, dst) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat os.chflags(dst, st.st_flags) OSError: [Errno 1] Operation not permitted: '/tmp/pip-nIfswi-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
-
hectorpal over 8 yearsPossible duplicate of Unable to install nltk on Mac OS El Capitan
-
hectorpal over 8 yearsThe questions is being discussed here: apple.stackexchange.com/questions/209572/…
-
nvg58 over 8 yearsMaybe you it helps you without disable SIP: stackoverflow.com/a/32930419/3090526
-
Arabasta over 8 yearsI had the exact same problem when I tried installing "virtualenvwrapper" using pip but this command worked for me: "sudo pip install --ignore-installed virtualenvwrapper" So it might be apply to your case as well without having to go through a painful round-about
-
-
rholmes almost 9 yearsIt's a bit of an investment in time and requires some command line proficiency, but the result is well worth it - I use similar virtual environments for Ruby (RVM), Perl, etc. Homebrew is useful for safely trying lots of other languages and tools as well.
-
Sacrilicious over 8 yearsno, no, no. This is a util running in user space. Please see stackoverflow.com/a/33024464/743638 for a relatively simple way around this. Disabling protection from 1. unsigned kernel extensions 2. modify systems binaries 3. inspecting and mounting over system paths etc. etc. is just foolhardy. At least recommend TEMPORARILY removing SIP until after install, and then turning it back on.
-
Mario over 8 yearsI would not do this. This is asking for trouble. For a more graceful solution approach see my answer below.
-
Ergun Ozyurt over 8 yearsthe question was unanswered more than a month. i think, this was the only "findable" solution at that time. i will edit my answer with some warning to enable SIP after everyting has been installed.
-
Luke over 8 yearsThis worked well for me when installing awscli on El Capitan without messing with the python installation:
sudo pip install --ignore-installed awscli
-
harinsa over 8 yearscouldn't import scrapy when installed with this method
ImportError: cannot import name xmlrpc_client
-
Friendly King over 8 years
--ignore-installed
flag did the trick for the AWS Elastic Beanstalk Cli for me as well. Finally... -
Paul Carlton over 8 yearsthis worked well for me as well. I think this should be the correct answer.
-
Verticon over 8 yearsThis worked for me. Thank you for saving me from going into the OS tweaks snake pit!
-
Russell_A about 8 yearsWorked for me too, and got me the next Python 2.7.X. This answer should win
-
Sophia Feng about 8 yearsThis worked for me in
sudo pip install --ignore-installed scrapy
. -
Translunar about 8 yearsThis didn't work for me for Numpy/Matplotlib. I'm trying to upgrade Matplotlib, but can't seem to uninstall Numpy 1.8.0rc1 and replace it with Numpy 1.11.0.
-
olleh about 8 yearsstill needs sudo tho!
-
dmgig almost 8 yearsWorked for me installing TensorFlow
-
loretoparisi over 7 yearsAmazing, this finally solved my problem on El Capitan! My temporary solution was to use
virtualenv
that worked as well, but for a specific environment I needed packages on whole system to be installed. Now it worked like a charm:sudo pip install --ignore-installed --upgrade -r requirements.txt
-
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz over 7 yearsThis is the best answer for El Capitan Mac users
-
Ramesh Ramasubramanian over 7 years@Luke I had the same issue yesterday when I tried to install the latest version of numpy. Your suggestion worked and saved me all kinds of time. Thank you.
-
xdhmoore about 7 yearsI'm unclear on precisely how these different flags work, but this answer seems to make the most sense to me. (Also, it worked :)
-
dietbacon about 7 years@xdhmoore Thanks, I think so too haha. No need to disable SIP or to install a new python or mess with dependencies.
-
Muaaz salagar about 7 yearsYou have pointed it out correctly ! This should be accepted answer!
-
Rob almost 7 yearsFor mkvirtualenv, this did not install the package in the expected location /usr/local/bin. Instead,
sudo pip install mkvirtualenv --ignore-installed six
works as expected, putting virtualenvwrapper.sh in /usr/local/bin -
Rob almost 7 yearsYes, this works for
sudo pip install virtualenvwrapper --ignore-installed six
as well, avoiding this error:OSError: [Errno 1] Operation not permitted: '/tmp/..../System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
-
dietbacon almost 7 years@Rob It really doesn't. The --user flag only installs for current user, so it makes sense that it wouldn't install in a folder that is not user-specific. For me, this command installs my packages in "/Users/myusername/Library/Python/2.7/lib/python/site-packages" for .py files and "/Users/myusername/Library/Python/2.7/bin" for .sh files
-
Rob almost 7 years@dietbacon I think we are in agreement? The command I shared put it in a system-wide available location, yours is for installing it for a given user.
-
dietbacon almost 7 years@Rob Yeah, sort of. But /usr/local/bin shouldn't be the expected location if you're installing something only for the current user, as I wrote that was the case in the answer.
-
Timothy Swan over 6 yearsThis appeared to install in the user space for me, but then wasn't linked. What is the step to make sure that it is linked? OS X El Capitan 10.11.6
-
dietbacon over 6 years@TimothySwan That's weird. If it's in "/Users/myusername/Library/Python/2.7/lib/python/site-packages" or "/Users/myusername/Library/Python/2.7/bin", python should find it automatically, unless you're using another version of python you manually installed I think. You can see where python is loading packages from by opening the python shell, then running
import sys
andsys.path
. That should show you the folders where python is looking for packages. -
QADeveloper over 6 yearsThis worked for me.. i think this is a better solution
-
Datageek about 6 years
sudo pip install --user <package name>
worked for me, without the need to reinstall the python -
HibernatedGuy about 6 yearsworked for me. I faced the same issue while I was installing virtualenvwrapper on HighSierra.
-
nantitv almost 6 yearsThis worked for me for a rasa-nlu . sudo pip install --user rasa_nlu
-
Michael Guild over 5 yearsthat did it for me. ( Sierra 10.13.6 )
-
Mario about 5 years@codersofthedark maybe
brew install python3
could solve this as on newer systems python3 is now the default python. -
Thomas Crawford over 4 yearsWorked for me as well.