PyInstaller --onefile throws Qt error with pandas, matplotlib and sklearn - matplotlib

Using PyInstaller with the --onefile flag, I can successfully build the following script into a .exe:
import sys
from PyQt4 import QtGui
class Example(QtGui.QMainWindow):
def __init__(self):
super(Example, self).__init__()
self.initUI()
def initUI(self):
self.statusBar().showMessage('Ready')
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Statusbar')
self.show()
def main():
app = QtGui.QApplication(sys.argv)
ex = Example()
sys.exit(app.exec_())
if __name__ == '__main__':
main()
I get the warnings below when building: (for readability I use "PYINSTALLERDIR" to replace the full path, which is "C:\Users\name\Downloads\pyinstaller-pyinstaller-v2.0-544-g337ae69\pyinstaller-pyinstaller-337ae69\".
PYINSTALLERDIR>pyinstaller.py --onefile --log-level=WARN MainWindowHello.py
1306 WARNING: library python%s%s required via ctypes not found
1468 INFO: Adding Microsoft.VC90.CRT to dependent assemblies of final executable
2957 WARNING: library python%s%s required via ctypes not found
But the outputted 14 MB .exe runs fine and displays a Qt window. However, when I try to add pandas, matplotlib or sklearn, I run into problems with Qt.
Adding either import matplotlib or import sklearn to line 3 of my script gives me these warnings when building:
PYINSTALLERDIR>python pyinstaller.py --onefile --log-level=WARN MainWindowHello.py
1371 WARNING: library python%s%s required via ctypes not found
1528 INFO: Adding Microsoft.VC90.CRT to dependent assemblies of final executable
3051 WARNING: library python%s%s required via ctypes not found
27108 INFO: Adding Microsoft.VC90.MFC to dependent assemblies of final executable
33329 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
When I try to run the resulting .exe (44 MB for matplotlib, 87 MB for sklearn), no Qt window is displayed and I get this error message:
WARNING: file already exists but should not: C:\Users\name\AppData\Local\Temp\_MEI75002\Include\pyconfig.h
Traceback (most recent call last):
File "<string>", line 2 in <module>
File "PYINSTALLERDIR\PyInstaller\loader\pyi_importers.py", line 409, in load_module
ImportError: could not import module 'PySide.QtCore'
With import pandas at line 3, I get the same warnings (as well as warnings about libzmq.pyd, but I have gotten them earlier with working programs). When I try to run the 119 MB .exe, the program crashes and throws the following error:
WARNING: file already exists but should not: C:\Users\name\AppData\Local\Temp\_MEI85162\include\pyconfig.h
Sub class of QObject not inheriting QObject!? Crash will happen when using Example.
I have tried with both PyInstaller 2.0 and the dev version. All three scenarios work well when using the default --onedir instead of --onefile. Can anyone help me figure out what goes awry when using --onefile?
UPDATE: I tried building with pandas in PyInstaller 2.1 and I still get the same error when using --onefile. Again, everything works when not using --onefile.

I was having the same issue with a script where I was importing PyQt4 as well as some modules that were importing PySide. PyInstaller worked fine with the --onedir option (the default), but I was getting ImportError: could not import module 'PySide.QtCore' when using the --onefile option.
After reading this I tried adding 'PySide' as an exclude in my spec file to force exclusive use of PyQt4 and the exe now runs fine. The modules you've listed should work fine with PyQt4 so it should hopefully solve your issue too.
Also, while it's not a huge problem, a solution to the file already exists warning is described here. Just add these lines in the spec file after a = Analysis... to remove the duplicate that causes the warning:
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break

Related

Import does not function as bat file; but works in spyder

I am still not the most sophisticated python user; but I cannot overcome this probably simple problem. I have a code that works perfectly with the spyder interface. I would like to make it a recurring task via creating a bat file. The bat file which in turn triggers a cmd interface does not import pandas_data reader and the code gets stuck and aborts.
import pandas_datareader.data as web
this line above creates the error below. It's a lengthy text.
File "C:\Users\myself\anaconda3\lib\site-packages\pandas_datareader\__init__.py", line 2, in <module>
from .data import ( File "C:\Users\myself\anaconda3\lib\site-packages\pandas_datareader\data.py", line 9, in <module>
from pandas.util._decorators import deprecate_kwarg File "C:\Users\myself\anaconda3\lib\site-packages\pandas\__init__.py", line 17, in <module>
"Unable to import required dependencies:\n" + "\n".join(missing_dependencies) ImportError: Unable to import required dependencies: numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\myself\anaconda3\python.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.17.0" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.

IntelliJ IDEA issue: xarray & pyparsing exception on import

This has something to do with the IntelliJ IDEA 2017.1.1 IDE. I do not get the following issue when executing my code via the command line.
===========================================================================
Python version: 3.6.1
xarray version: 0.9.6
pandas version: 0.20.3
numpy version: 1.12.1
I, for the first time, would like to use xarray.
I imported the module (no problem here) and then, without even using the module, ran my code. For example:
import xarray as xr
def something():
print("doing something...")
something()
This immediately throws an exception when I run it:
Exception ignored in: at 0x05A287B0>
Traceback (most recent call last): File "C:\Program Files
(x86)\Python36-32\lib\site-packages\pyparsing.py", line 160, in
_generatorType = type((y for y in range(1))) SystemError: error return without exception set
If I delete the import xarray as xr and rerun the code, I get no exception.
From the exception message, it looks like something called pyparsing.py
Any ideas?
pyparsing is probably installed as a dependency from some other package. I have run the pyparsing unit tests on both Python 3.6.1 and 3.6.2 (as well as most other popular Python versions back to 2.6) without any error.
I suspect that something in your environment is defining range to be something other than the normal builtin range method, and this is then causing the pyparsing code to fail.
I will fix this in pyparsing, to replace range(1) with just an empty list, which should give the same results for pyparsing, but without the susceptibility to being overwritten by a monkeypatch to range.
In the meantime, try explicitly importing pyparsing before importing xarray, or anything else for that matter. A simple import pyparsing should do.

Error with PyQtDeploy

I'm using PyQt 5.3.1 and I'm deploying with pyqtdeploy 0.4 and when i want to build a project i obtain this error message:
Generating code...
Cleaning E:\ProgramasPython3\PythonQT\QTCalculator\build.
Freezing C:\Users\Tobal\AppData\Local\Temp\bootstrap_py3.py
Freezing E:\ProgramasPython3\PythonQT\QTCalculator\qtcalculator.py
Freezing E:/ProgramasPython3/PythonQT\QTCalculator\__init__.py
Freezing E:/ProgramasPython3/PythonQT\QTCalculator\calculator_ui.py
Freezing E:/ProgramasPython3/PythonQT\QTCalculator\img_rc.py
Freezing E:/ProgramasPython3/PythonQT\QTCalculator\qtcalculator.py
Freezing C:\Python34\libs\site-packages\PyQt5\__init__.py
Unable to freeze C:\Python34\libs\site-packages\PyQt5\__init__.py.
Traceback (most recent call last):
File "C:\Users\Tobal\AppData\Local\Temp\freeze.py", line 103, in <module>
freeze_as_data(py_file, options.as_data)
File "C:\Users\Tobal\AppData\Local\Temp\freeze.py", line 36, in freeze_as_data
code = _get_marshalled_code(py_filename)
File "C:\Users\Tobal\AppData\Local\Temp\freeze.py", line 71, in _get_marshalled_code
source_file = open(py_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Python34\\libs\\site-packages\\PyQt5\\__init__.py'
I think this is a bug. someone does know how to fix it?
Thanks
Did you build PyQt5 statically? The host and target Python are two different things, and often in different directories. The target Python has PyQt module built statically, the host Python must also have PyQt5 installed (because pyqtdeploy uses Qt for its GUI) but its usually a dynamic library in the host Python.
In the pyqtdeploy GUI on "Locations" tab, make sure that "Standard library directory" is correct.

OMP warning when numpy 1.8.0 is packaged with py2exe

import numpy
When I packaged above one line script as a single executable window application using py2exe, I get following warnings upon launch.
OMP: Warning #178: Function GetModuleHandleEx failed:
OMP: System error #126: The specified module could not be found.
This warning happen only when I build as single executable (i.e., only when bundle_files=1). Here's my setup.py for this.
from distutils.core import setup
import py2exe
setup(
options = {'py2exe': {'bundle_files': 1}},
windows=['testnumpy.py'],
zipfile = None,
)
This problem started with numpy 1.8.0. When I revert back to 1.6.2, the warnings don't show up.
Usually a single executable packaged by py2exe will catch warnings and traceback and save them into a log file. But somehow these warnings are not captured and the app creates a console window to show warning. I want to suppress this additional console window to show up.
How can I fix this warning problem?
What I tried (nothing worked):
I tried this redirecting sys.stderr.
I searched github numpy source for openMP assuming the OMP stands for it as mentioned here. But, nothing useful came out.
I have copied libiomp5md.dll to the same folder as setup.py.
I tried filterwarnings:
I tried sys.excepthook.
As I wrote in the comment, installing numpy 1.8.1rc1 from sourceforge did fix the issue, although I don't really know the differences...
I had this issue with numpy 1.13.1+mkl and scipy 1.19.1. Reverting to numpy 1.8.1rc1 is not an acceptable solution.
I tracked this issue to the scipy.integrate subpackage. The warning message pops up when this package is imported. It seems that perhaps libraries that use MKL don't like being invoked from library.zip, which is where py2exe places packages when using bundle option 2.
The solution is to exclude scipy and numpy in the py2exe setup script and copy their entire package folders into the distribution directory and add that directory to the system path at the top of the main python script.

Cannot import "logging" module using Jython in embedded Jetty -> ImportError: no os specific module found

I'm trying to use Jython (embedded) in a Jetty server (all through Maven) to invoke a simple Python script.
My script works fine as long as I don't try to use any of the standard library's such as 'logging.' Whenever I try to import one of the standard library's it fails with the exception "ImportError."
The exception I get is:
File "<string>", line 1, in <module>
File "c:\home\work\sample\content\helloworld\helloworld.py", line 10, in <module>
import logging
File "c:\home\work\sample\content\Lib\logging\__init__.py", line 29, in <module>
import sys, os, types, time, string, cStringIO, traceback
File "c:\home\work\sample\content\Lib\os.py", line 119, in <module>
raise ImportError, 'no os specific module found'
ImportError: no os specific module found
at org.python.core.PyException.fillInStackTrace(PyException.java:70)
at java.lang.Throwable.<init>(Throwable.java:181)
at java.lang.Exception.<init>(Exception.java:29)
at java.lang.RuntimeException.<init>(RuntimeException.java:32)
at org.python.core.PyException.<init>(PyException.java:46)
at org.python.core.PyException.doRaise(PyException.java:200)
at org.python.core.Py.makeException(Py.java:1159)
at org.python.core.Py.makeException(Py.java:1163)
at os$py.f$0(c:\home\work\sample\content\Lib\os.py:692)
at os$py.call_function(c:\home\work\sample\content\Lib\os.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.imp.createFromCode(imp.java:325)
at org.python.core.imp.createFromPyClass(imp.java:144)
at org.python.core.imp.loadFromSource(imp.java:504)
at org.python.core.imp.find_module(imp.java:410)
at org.python.core.imp.import_next(imp.java:620)
at org.python.core.imp.import_first(imp.java:650)
at org.python.core.imp.import_name(imp.java:741)
at org.python.core.imp.importName(imp.java:791)
at org.python.core.ImportFunction.__call__(__builtin__.java:1236)
at org.python.core.PyObject.__call__(PyObject.java:367)
at org.python.core.__builtin__.__import__(__builtin__.java:1207)
at org.python.core.__builtin__.__import__(__builtin__.java:1190)
at org.python.core.imp.importOne(imp.java:802)
at logging$py.f$0(c:\home\work\sample\content\Lib\logging\__init__.py:1372)
at logging$py.call_function(c:\home\work\sample\content\Lib\logging\__init__.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.imp.createFromCode(imp.java:325)
at org.python.core.imp.createFromPyClass(imp.java:144)
at org.python.core.imp.loadFromSource(imp.java:504)
at org.python.core.imp.find_module(imp.java:410)
at org.python.core.imp.import_next(imp.java:620)
at org.python.core.imp.import_first(imp.java:650)
at org.python.core.imp.import_name(imp.java:741)
at org.python.core.imp.importName(imp.java:791)
at org.python.core.ImportFunction.__call__(__builtin__.java:1236)
at org.python.core.PyObject.__call__(PyObject.java:367)
at org.python.core.__builtin__.__import__(__builtin__.java:1207)
at org.python.core.__builtin__.__import__(__builtin__.java:1190)
at org.python.core.imp.importOne(imp.java:802)
at helloworld.helloworld$py.f$0(c:\home\work\sample\content\helloworld\helloworld.py:19)
at helloworld.helloworld$py.call_function(c:\home\work\sample\content\helloworld\helloworld.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.imp.createFromCode(imp.java:325)
at org.python.core.imp.createFromPyClass(imp.java:144)
at org.python.core.imp.loadFromSource(imp.java:504)
at org.python.core.imp.find_module(imp.java:410)
at org.python.core.PyModule.impAttr(PyModule.java:109)
at org.python.core.imp.import_next(imp.java:622)
at org.python.core.imp.import_name(imp.java:761)
at org.python.core.imp.importName(imp.java:791)
at org.python.core.ImportFunction.__call__(__builtin__.java:1236)
at org.python.core.PyObject.__call__(PyObject.java:367)
at org.python.core.__builtin__.__import__(__builtin__.java:1207)
at org.python.core.imp.importFromAs(imp.java:869)
at org.python.core.imp.importFrom(imp.java:845)
at org.python.pycode._pyx1.f$0(<string>:1)
at org.python.pycode._pyx1.call_function(<string>)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.Py.runCode(Py.java:1197)
at org.python.core.Py.exec(Py.java:1241)
at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:138)
My scripts looks like:
from java.util import Random
from java.util import Date
import sys
print(sys.path)
print(sys.builtin_module_names)
import logging
logging.basicConfig(level=logging.WARNING)
logger1 = logging.getLogger('aaa')
logger1.warning('************* This message comes from one module')
def say_hello():
return 'hello world1'
I've tried the following so far but nothing has worked:
Include the zip of the 'Lib' directory in my classpath
Hard-coding the 'Lib' path when i setup the interpreter.
If I do it directly from the interactive Jython shell the script works fine (and a logging message appears).
Thanks.
KJQ
I think for now i've found an answer to my own question...
Basically, i knew it had something to do with my paths but could not figure out how to do them.
I ended up creating a "standalone" version of the Jython jar through the installer (and it includes the /Libs directory) and using that.