MSYS2 pip install numpy failed - numpy

I am new to MSYS2, so I tried to install numpy using pip (mingw-w64-x86_64-python3-pip-9.0.1-3), but the the installation is failed. Here is the output:
(...)
C compiler: x86_64-pc-msys-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe
compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
x86_64-pc-msys-gcc: _configtest.c
/usr/bin/bash: x86_64-pc-msys-gcc: command not found
/usr/bin/bash: x86_64-pc-msys-gcc: command not found
failure.
removing: _configtest.c _configtest.o
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-3d852664/numpy/setup.py", line 394, in <module>
setup_package()
File "/tmp/pip-build-3d852664/numpy/setup.py", line 386, in setup_package
setup(**metadata)
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/install.py", line 62, in run
r = self.setuptools_run()
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
return distutils_install.run(self)
File "/usr/lib/python3.6/distutils/command/install.py", line 551, in run
self.run_command('build')
File "/usr/lib/python3.6/distutils/cmd.py", line 314, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/build.py", line 47, in run
old_build.run(self)
File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.6/distutils/cmd.py", line 314, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/build_src.py", line 148, in run
self.build_sources()
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
self.build_library_sources(*libname_info)
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/build_src.py", line 294, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "/tmp/pip-build-3d852664/numpy/numpy/distutils/command/build_src.py", line 377, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 675, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-3d852664/numpy/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-4nnf7xbj-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-3d852664/numpy/
I have already build tools installed: I used the tutorial on https://github.com/orlp/dev-on-windows/wiki/Installing-GCC--&-MSYS2 (I only 64-bit packages)
I looked for similar issues on the net but I found nothing relevant.
Any comment or suggestion is welcome
Thanks in advance

The proper gcc compiler has not been installed. For my installation, I used the msys2 shell (not mingw) and installed the msys/gcc package:
pacman -S msys/gcc

pacman -S mingw-w64-x86_64-python-scipy
pacman -S mingw-w64-i686-python-scipy

Related

Failure at building wheel for numpy in a virtualenv, apparently not python version related - Windows 10

I'm trying to install numpy on a Virtual Environment using pip, there is a failure when building wheel for it, however.
The problem is only present when trying to install it in the virtualenv, I can install and update it on my system just fine.
It is running on Windows 10, Python 3.10.5, pip 22.3.1.
Running from numpy source directory.
setup.py:67: DeprecationWarning:
`numpy.distutils` is deprecated since NumPy 1.23.0, as a result
of the deprecation of `distutils` itself. It will be removed for
Python >= 3.12. For older Python versions it will remain present.
It is recommended to use `setuptools < 60.0` for those Python versions.
For more details, see:
https://numpy.org/devdocs/reference/distutils_status_migration.html
import numpy.distutils.command.sdist
Processing numpy/random\_bounded_integers.pxd.in
Processing numpy/random\bit_generator.pyx
Processing numpy/random\mtrand.pyx
Processing numpy/random\_bounded_integers.pyx.in
Processing numpy/random\_common.pyx
Processing numpy/random\_generator.pyx
Processing numpy/random\_mt19937.pyx
Processing numpy/random\_pcg64.pyx
Processing numpy/random\_philox.pyx
Processing numpy/random\_sfc64.pyx
Cythonizing sources
INFO: blas_opt_info:
INFO: blas_armpl_info:
Looking for python310.dll
Traceback (most recent call last):
File "C:\stable_diffusion\diffusers_venv\lib\python3.10\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
main()
File "C:\stable_diffusion\diffusers_venv\lib\python3.10\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\stable_diffusion\diffusers_venv\lib\python3.10\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "C:\Users\Vitor\AppData\Local\Temp\pip-build-env-amr3d935\overlay\lib\python3.10\site-packages\setuptools\build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "C:\Users\Vitor\AppData\Local\Temp\pip-build-env-amr3d935\overlay\lib\python3.10\site-packages\setuptools\build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "C:\Users\Vitor\AppData\Local\Temp\pip-build-env-amr3d935\overlay\lib\python3.10\site-packages\setuptools\build_meta.py", line 267, in run_setup
super(_BuildMetaLegacyBackend,
File "C:\Users\Vitor\AppData\Local\Temp\pip-build-env-amr3d935\overlay\lib\python3.10\site-packages\setuptools\build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 479, in <module>
setup_package()
File "setup.py", line 471, in setup_package
setup(**metadata)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\core.py", line 135, in setup
config = configuration()
File "setup.py", line 118, in configuration
config.add_subpackage('numpy')
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 1050, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 1016, in get_subpackage
config = self._get_configuration_from_setup_py(
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 958, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\setup.py", line 9, in configuration
config.add_subpackage('core')
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 1050, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 1016, in get_subpackage
config = self._get_configuration_from_setup_py(
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\misc_util.py", line 958, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\core\setup.py", line 853, in configuration
blas_info = get_info('blas_opt', 0)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 585, in get_info
return cl().get_info(notfound_action)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 845, in get_info
self.calc_info()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 2077, in calc_info
if self._calc_info(blas):
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 2063, in _calc_info
return getattr(self, '_calc_info_{}'.format(name))()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 1979, in _calc_info_armpl
info = get_info('blas_armpl')
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 585, in get_info
return cl().get_info(notfound_action)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 845, in get_info
self.calc_info()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 1337, in calc_info
info = self.check_libs2(lib_dirs, armpl_libs)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 1001, in check_libs2
exts = self.library_extensions()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 960, in library_extensions
c = customized_ccompiler()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\system_info.py", line 216, in customized_ccompiler
global_compiler = _customized_ccompiler()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\__init__.py", line 62, in customized_ccompiler
c = ccompiler.new_compiler(plat=plat, compiler=compiler, verbose=verbose)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\ccompiler.py", line 780, in new_compiler
compiler = klass(None, dry_run, force)
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\mingw32ccompiler.py", line 64, in __init__
build_import_library()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\mingw32ccompiler.py", line 348, in build_import_library
return _build_import_library_amd64()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\mingw32ccompiler.py", line 397, in _build_import_library_amd64
dll_file = find_python_dll()
File "C:\Users\Vitor\AppData\Local\Temp\pip-install-1sr3obr4\numpy_fa19a5e39e9448e7b6553e0f5a275159\numpy\distutils\mingw32ccompiler.py", line 219, in find_python_dll
raise ValueError("%s not found in %s" % (dllname, lib_dirs))
ValueError: python310.dll not found in ['C:\\stable_diffusion\\diffusers_venv\\', 'C:\\stable_diffusion\\diffusers_venv\\lib', 'C:\\stable_diffusion\\diffusers_venv\\bin', 'C:\\Program Files\\Inkscape\\', 'C:\\Program Files\\Inkscape\\lib', 'C:\\Program Files\\Inkscape\\bin', 'C:\\WINDOWS\\System32']
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for numpy
Failed to build numpy
ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
As I faced a similar problem before, my instinct was to update pip, change the running python version or try an older, specific, version of numpy, but nothing seems to work and the problem persist.

in ubuntu, I try to ninja -C out/Release_arm system_webview_apk , error happened

in ubuntu, I try to ninja -C out/Release_arm system_webview_apk , error happened
ACTION //v8:generate_bytecode_builtins_list(//build/toolchain/android:android_clang_arm)
FAILED: gen/v8/builtins-generated/bytecodes-builtins-list.h
python3 ../../v8/tools/run.py ./clang_x86_v8_arm/bytecode_builtins_list_generator gen/v8/builtins-generated/bytecodes-builtins-list.h
Traceback (most recent call last):
File "/mnt/d/code/chromium/src/out/Release_arm/../../v8/tools/run.py", line 12, in
result = subprocess.call(sys.argv[1:])
File "/usr/lib/python3.9/subprocess.py", line 349, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.9/subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: './clang_x86_v8_arm/bytecode_builtins_list_generator'

Issue installing picamera (ERROR: Command errored out with exit status 1:)

Im trying to install picamera (pip install picamera) and I get the following error:
ERROR: Command errored out with exit status 1:
Can someone please help?
Python 3.9.1
pip 20.3.3
Full error message:
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python#3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"'; __file__='"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-wheel-x36s0i1b
cwd: /private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/
Complete output (61 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/picamera
copying picamera/streams.py -> build/lib/picamera
copying picamera/color.py -> build/lib/picamera
copying picamera/__init__.py -> build/lib/picamera
copying picamera/encoders.py -> build/lib/picamera
copying picamera/renderers.py -> build/lib/picamera
copying picamera/display.py -> build/lib/picamera
copying picamera/camera.py -> build/lib/picamera
copying picamera/frames.py -> build/lib/picamera
copying picamera/exc.py -> build/lib/picamera
copying picamera/bcm_host.py -> build/lib/picamera
copying picamera/mmalobj.py -> build/lib/picamera
copying picamera/array.py -> build/lib/picamera
copying picamera/mmal.py -> build/lib/picamera
running egg_info
writing picamera.egg-info/PKG-INFO
writing dependency_links to picamera.egg-info/dependency_links.txt
writing requirements to picamera.egg-info/requires.txt
writing top-level names to picamera.egg-info/top_level.txt
reading manifest file 'picamera.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'debian'
no previously-included directories found matching 'docs'
writing manifest file 'picamera.egg-info/SOURCES.txt'
installing to build/bdist.macosx-10.15-x86_64/wheel
running install
Traceback (most recent call last):
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 101, in run
with io.open('/proc/cpuinfo', 'r') as cpuinfo:
FileNotFoundError: [Errno 2] No such file or directory: '/proc/cpuinfo'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 145, in <module>
main()
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 119, in main
setup(
File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 335, in run
self.run_command('install')
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 113, in run
raise ValueError('Unable to open /proc/cpuinfo')
ValueError: Unable to open /proc/cpuinfo
----------------------------------------
ERROR: Failed building wheel for picamera
Running setup.py clean for picamera
Failed to build picamera
Installing collected packages: picamera
Running setup.py install for picamera ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python#3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"'; __file__='"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-record-qme1_imo/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/picamera
cwd: /private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/
Complete output (25 lines):
running install
Traceback (most recent call last):
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 101, in run
with io.open('/proc/cpuinfo', 'r') as cpuinfo:
FileNotFoundError: [Errno 2] No such file or directory: '/proc/cpuinfo'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 145, in <module>
main()
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 119, in main
setup(
File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python#3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py", line 113, in run
raise ValueError('Unable to open /proc/cpuinfo')
ValueError: Unable to open /proc/cpuinfo
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/opt/python#3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"'; __file__='"'"'/private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-install-rhshjkgn/picamera_1bc7cc4f85134726b19ddbe1c70f3c0a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/mk/d5qfrxc13nb6rq4d1v_ntzp40000gn/T/pip-record-qme1_imo/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/picamera Check the logs for full command output.

Protobuf errors while using Tensorflow Object Detection API locally

I got tensorflow and object detection API on my machine.
Test run shows that everything works.
~ $ cd models/research
research $ protoc object_detection/protos/*.proto --python_out=.
research $ export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
research $ python3 object_detection/builders/model_builder_test.py
...............
----------------------------------------------------------------------
Ran 15 tests in 0.144s
OK
Then I tried to retrain a model and got the protobuf error
research $ cd object_detection
object_detection $ python3 train.py --logtostderr --train_dir=training/ --pipeline_config_path=ssdlite_mobilenet_v2_coco_2018_05_09/pipeline.config
WARNING:tensorflow:From /Users/me/models/research/object_detection/trainer.py:257: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
Traceback (most recent call last):
File "/Users/me/models/research/object_detection/utils/label_map_util.py", line 135, in load_labelmap
text_format.Merge(label_map_string, label_map)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 533, in Merge
descriptor_pool=descriptor_pool)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 587, in MergeLines
return parser.MergeLines(lines, message)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 620, in MergeLines
self._ParseOrMerge(lines, message)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 635, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 735, in _MergeField
merger(tokenizer, message, field)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 823, in _MergeMessageField
self._MergeField(tokenizer, sub_message)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 722, in _MergeField
tokenizer.Consume(':')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/text_format.py", line 1087, in Consume
raise self.ParseError('Expected "%s".' % token)
google.protobuf.text_format.ParseError: 3:10 : Expected ":".
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/internal/python_message.py", line 1083, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/internal/python_message.py", line 1105, in InternalParse
(tag_bytes, new_pos) = local_ReadTag(buffer, pos)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/internal/decoder.py", line 181, in ReadTag
while six.indexbytes(buffer, pos) & 0x80:
TypeError: unsupported operand type(s) for &: 'str' and 'int'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 184, in <module>
tf.app.run()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 180, in main
graph_hook_fn=graph_rewriter_fn)
File "/Users/me/models/research/object_detection/trainer.py", line 264, in train
train_config.prefetch_queue_capacity, data_augmentation_options)
File "/Users/me/models/research/object_detection/trainer.py", line 59, in create_input_queue
tensor_dict = create_tensor_dict_fn()
File "train.py", line 121, in get_next
dataset_builder.build(config)).get_next()
File "/Users/me/models/research/object_detection/builders/dataset_builder.py", line 155, in build
label_map_proto_file=label_map_proto_file)
File "/Users/me/models/research/object_detection/data_decoders/tf_example_decoder.py", line 245, in __init__
use_display_name)
File "/Users/me/models/research/object_detection/utils/label_map_util.py", line 152, in get_label_map_dict
label_map = load_labelmap(label_map_path)
File "/Users/me/models/research/object_detection/utils/label_map_util.py", line 137, in load_labelmap
label_map.ParseFromString(label_map_string)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/message.py", line 185, in ParseFromString
self.MergeFromString(serialized)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/protobuf/internal/python_message.py", line 1089, in MergeFromString
raise message_mod.DecodeError('Truncated message.')
google.protobuf.message.DecodeError: Truncated message.
object_detection $
I tried a bunch of similar problems' solutions, but they didn't work for my case. For example this one suggest to encode pbtxt file with ASCII.
Python 2 gives an error too. Here is it's last line
google.protobuf.message.DecodeError: Unexpected end-group tag.
Context:
macOS 10.13.4
Local run on CPU
Python 3.6.4
protobuf 3.5.1
libprotoc 3.4.0
tensorflow 1.8.0
Google Cloud SDK 200.0.0
bq 2.0.33
core 2018.04.30
gsutil 4.31

xmlsec on el capitan

I'm trying to install xmlsec on El Capitan and i'm not able to do so.
I'm following instructions on this page:
https://github.com/concordusapps/python-xmlsec
when I do pip install xmlsec I get:
Downloading xmlsec-0.3.1.tar.gz
Complete output from command python setup.py egg_info:
no previously-included directories found matching 'doc/.build'
Installed /private/tmp/easy_install-kotwF5/pkgconfig-1.1.0/.eggs/nose-1.3.7-py2.7.egg
zip_safe flag not set; analyzing archive contents...
Installed /private/tmp/pip-build-my91LX/xmlsec/.eggs/pkgconfig-1.1.0-py2.7.egg
Searching for setuptools-cython
Reading https://pypi.python.org/simple/setuptools_cython/
Best match: setuptools-cython 0.2.1
Downloading https://pypi.python.org/packages/source/s/setuptools_cython/setuptools_cython-0.2.1.tar.gz#md5=c29fc5bf2894e69cbd98e3800cef4707
Processing setuptools_cython-0.2.1.tar.gz
Writing /tmp/easy_install-xnAHIU/setuptools_cython-0.2.1/setup.cfg
Running setuptools_cython-0.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xnAHIU/setuptools_cython-0.2.1/egg-dist-tmp-zk1125
zip_safe flag not set; analyzing archive contents...
Copying setuptools_cython-0.2.1-py2.7.egg to /private/tmp/pip-build-my91LX/xmlsec/.eggs
Installed /private/tmp/pip-build-my91LX/xmlsec/.eggs/setuptools_cython-0.2.1-py2.7.egg
Searching for Cython
Reading https://pypi.python.org/simple/Cython/
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/private/tmp/pip-build-my91LX/xmlsec/setup.py", line 121, in <module>
make_extension('xmlsec.template'),
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 268, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1093, in obtain
return installer(requirement)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 626, in easy_install
not self.always_copy, self.local_index
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 590, in fetch_distribution
self.find_packages(requirement)
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 427, in find_packages
self.scan_url(self.index_url + requirement.unsafe_name+'/')
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 761, in scan_url
self.process_url(url, True)
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 304, in process_url
f = self.open_url(url, "Download error on %s: %%s -- Some packages may not be found!" % url)
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 700, in open_url
return open_with_auth(url, self.opener)
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 894, in _socket_timeout
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/setuptools/package_index.py", line 1007, in open_with_auth
fp = opener(request)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
context=self._context)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open
r = h.getresponse(buffering=True)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse
response.begin()
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 409, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv
return self.read(buflen)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read
v = self._sslobj.read(len or 1024)
ssl.SSLError: ('The read operation timed out',)
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-my91LX/xmlsec
and when I download the source and do it manually I get:
Processing /Users/myuser/python-xmlsec
Requirement already satisfied (use --upgrade to upgrade): lxml>=3.0 in /usr/local/lib/python2.7/site-packages (from xmlsec==0.1.2)
Installing collected packages: xmlsec
Running setup.py install for xmlsec
Complete output from command /usr/local/opt/python/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-ad8Fqh-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-X7VjyO-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.11-x86_64-2.7
creating build/lib.macosx-10.11-x86_64-2.7/xmlsec
copying src/xmlsec/__init__.py -> build/lib.macosx-10.11-x86_64-2.7/xmlsec
copying src/xmlsec/meta.py -> build/lib.macosx-10.11-x86_64-2.7/xmlsec
running build_ext
cythoning src/xmlsec/constants.pyx to src/xmlsec/constants.c
building 'xmlsec.constants' extension
creating build/temp.macosx-10.11-x86_64-2.7
creating build/temp.macosx-10.11-x86_64-2.7/src
creating build/temp.macosx-10.11-x86_64-2.7/src/xmlsec
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/lib/python2.7/site-packages/lxml -I/usr/local/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/local/Cellar/libxml2/2.9.2/include/libxml2 -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/xmlsec/constants.c -o build/temp.macosx-10.11-x86_64-2.7/src/xmlsec/constants.o
In file included from src/xmlsec/constants.c:256:
src/xmlsec.h:1:10: fatal error: 'xmlsec/crypto.h' file not found
#include <xmlsec/crypto.h>
^
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
Command "/usr/local/opt/python/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-ad8Fqh-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-X7VjyO-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-ad8Fqh-build
Is there something i'm missing?
This finally worked:
brew install -v libxmlsec1
Obviously this is using homebrew on macOS.
Installing (or reinstalling) limbxmlsec1 didn't work for me on OSX 10.14. Here's what I had to do:
There are some SDK headers that come with xcode. Copy them to the desktop:
cp /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg ~/Desktop
Install the package by double-clicking it, or by running this command:
sudo installer -pkg ~/Desktop/macOS_SDK_headers_for_macOS_10.14.pkg -target /
Try installing again:
pip install xmlsec