Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
10,842
It need to install rust compiler in your environment, you can use the command:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
and then you can do it successfully.
Related videos on Youtube
Author by
Jaswanth Ch
Updated on June 04, 2022Comments
-
Jaswanth Ch almost 2 years
I am trying to install the ernie and tokenizers on Windows, but for both cases it is showing the following error:
ERROR: Command errored out with exit status 1: command: 'c:\users\my pc\appdata\local\programs\python\python39\python.exe' 'c:\users\my pc\appdata\local\programs\python\python39\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_wheel 'C:\Users\MYPC~1\AppData\Local\Temp\tmpyfqm86h3' cwd: C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7 Complete output (38 lines): running bdist_wheel running build running build_py creating build creating build\lib creating build\lib\tokenizers copying tokenizers\__init__.py -> build\lib\tokenizers running build_ext running build_rust Updating crates.io index cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --target x86_64-pc-windows-msvc --release --verbose -- --crate-type cdylib warning: unused manifest key: target.x86_64-apple-darwin.rustflags Compiling proc-macro2 v1.0.30 Compiling unicode-xid v0.2.2 Compiling syn v1.0.80 Compiling autocfg v1.0.1 Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.30\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=0b72d9facdf9b740 -C extra-filename=-0b72d9facdf9b740 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\proc-macro2-0b72d9facdf9b740" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` Running `rustc --crate-name unicode_xid "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-xid-0.2.2\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=c5404e573edb1c5f -C extra-filename=-c5404e573edb1c5f --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.80\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b3126876d28730d4 -C extra-filename=-b3126876d28730d4 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\syn-b3126876d28730d4" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` Running `rustc --crate-name autocfg "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\autocfg-1.0.1\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=077a4bc4a690ef7e -C extra-filename=-077a4bc4a690ef7e --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` Compiling memchr v2.4.1 Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.4.1\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=420338dd5ea363a5 -C extra-filename=-420338dd5ea363a5 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\memchr-420338dd5ea363a5" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` error: linker `link.exe` not found | = note: The system cannot find the file specified. (os error 2) note: the msvc targets depend on the msvc linker but `link.exe` was not found note: please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option error: could not compile `syn` due to previous error Caused by: process didn't exit successfully: `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.80\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b3126876d28730d4 -C extra-filename=-b3126876d28730d4 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\syn-b3126876d28730d4" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` (exit code: 1) warning: build failed, waiting for other jobs to finish... error: build failed error: cargo failed with code: 101 ---------------------------------------- ERROR: Failed building wheel for tokenizers Failed to build tokenizers ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
-
umitu over 2 yearsYous should copy and paste all lines of the error. Also, please include the OS information.
-
wovano over 2 yearsI fixed the code formatting for you, but this is not the full error message. Please copy-paste the actual error message you get into the question, at least the part from "Building wheel for tokenizers" until the end! This part only contains the "summary" which is that the command failed, however, it does not include the relevant information about what failed. This makes it difficult to answer the question.
-
wovano over 2 yearsNow that you've included the relevant information, the cause of the error is pretty clear, right? error: linker
link.exe
not found And the solution is also mentioned in the error message: please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option -
Jaswanth Ch over 2 yearsSorry the solution was not clear, could you please elaborate. I am using the visual studio code and i installed an extension called visual c++ option in visual studio code but what is vs 2013 and how do i ensure it.
-
wovano over 2 yearsYou need Visual Studio (the compiler) version 2013 or higher to build this Python library. Visual Studio Code is an IDE. Although the name is similar it's something completely different! You can download Visual Studio here. I recently did it to install another Python library. This answer might be helpful as well.
-
-
Admin over 2 yearsAs it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
-
wovano over 2 yearsCould you explain why you think this is the problem/solution?
-
Jaswanth Ch over 2 yearsSorry my question was clear, I edited the question. I think the problem is while building the wheel for tokenizers, for other packages the wheel is working absolutely fine
-
Ali Abdi over 2 yearsI had a problem like this before & I thought that
wheel
wasn't installed on your machine; for tokenizers, have you seen this link? I think the solutions here will work.