after importing dill and dumping session , the notebook cannot be opened, the code cannot be retrieved - crash

i worked on some experiment and wanted to find a way to save the lists of data i collected manually. i found dill and decided to use it .
installed via pip. typed in the following code:
import dill
dill.dump_session('notebook_env.db')
after that i closed the kernel and opened the notebook and to my shocking surprise got the following message:
Unreadable Notebook: /Users/nimrodros/Untitled.ipynb UnicodeDecodeError('utf-8', b'\x80\x03cdill._dill\n_import_module\nq\x00X\x08\x00\x00\x00__main__q\x01\x85q\x02Rq\x03}q\x04(X\x08\x00\x00\x00__name__q\x05h\x01X\x07\x00\x00\x00__doc__q\x06X#\x00\x00\x00Automatically created module for IPython interactive environmentq\x07X\x0b\x00\x00\x00__package__q\x08NX\x08\x00\x00\x00__spec__q\tNX\x0b\x00\x00\x00__builtin__q\nh\x00X\x08\x00\x00\x00builtinsq\x0b\x85q\x0cRq\rX\x03\x00\x00\x00_ihq\x0e]q\x0f(X\x00\x00\x00\x00q\x10X\x1a\x00\x00\x00eye_correction_middle_fallq\x11X\r\x00\x00\x00fuc=[1,2,3,4]q\x12X\x16\x00\x00\x00eye_correction_voltageq\x13X\r\x00\x00\x00fuc=[1,2,3,4]q\x14X/\x00\x00\x00import dill\ndill.dump_session(\'untitled.ipynb\')q\x15eX\x03\x00\x00\x00_ohq\x16}q\x17X\x03\x00\x00\x00_dhq\x18]q\x19X\x10\x00\x00\x00/Users/nimrodrosq\x1aaX\x02\x00\x00\x00Inq\x1bh\x0fX\x03\x00\x00\x00Outq\x1ch\x17X\x01\x00\x00\x00_q\x1dh\x10X\x02\x00\x00\x00__q\x1eh\x10X\x03\x00\x00\x00___q\x1fh\x10X\x02\x00\x00\x00_iq X\r\x00\x00\x00fuc=[1,2,3,4]q!X\x03\x00\x00\x00_iiq"X\x16\x00\x00\x00eye_correction_voltageq#X\x04\x00\x00\x00_iiiq$X\r\x00\x00\x00fuc=[1,2,3,4]q%%X\x03\x00\x00\x00_i1q&X\x1a\x00\x00\x00eye_correction_middle_fallq\'X\x03\x00\x00\x00_i2q(h%%X\x03\x00\x00\x00fucq)]q*(K\x01K\x02K\x03K\x04eX\x03\x00\x00\x00_i3q+h#X\x03\x00\x00\x00_i4q,h!X\x03\x00\x00\x00_i5q-X/\x00\x00\x00import dill\ndill.dump_session(\'untitled.ipynb\')q.X\x04\x00\x00\x00dillq/h\x00h/\x85q0Rq1ub.', 0, 1, 'invalid start byte')
why did this happen , and most importantly , how can i retrieve the code ? that data is already lost probably but i had valuable code there .
i believe if could open the file without running and just erase the line
import dill
dill.dump_session('notebook_env.db')
then i could again open the file without any problems . so maybe there's a way of doing that?
update
i tried to use
pip uninstall dill
hoped it would run the program again and stop when not recognizing dill , but it didn't go that way . still looking for a solution
update 2
tried to run the following on terminal :
iconv -f ISO-8859-1 -t UTF-8//TRANSLIT /Users/nimrodros/Untitled.ipynb -o /Users/nimrodros/New.ipynbh-MacBook-Pro-sl-nimrod:~ nimrodroTRANSLIT /Users/nimrodros/Untitled.ipynb -o /Users/nimrodros/New.ipynb
returned the following:
cdill._dill
_import_module
q__main__qqRq}q(__name__qhX__doc__qX#Automatically created module for IPython interactive environmentqX
__package__N__spec__q NX
__builtin__q
hbuiltinsq
q
fuc=[1,2,3,4]qX/import dillon_voltageqX
dill.dump_session('untitled.ipynb')qeX_ohq}qX_dhq]qX/Users/nimrodrofuc=[1,2,3,4]q%X_i1q&Xeye_correction_middle_fallq'X_i2q(h%Xfucq)]q*(KKKKeX_i3q+h#X_i4q,h!X_i5q-X/import dill
dill.dump_session('untitled.ipynb')q.Xdillq/hh/q0Rq1ub.iconv: -o: No such file or directory
iconv: /Users/nimrodros/New.ipynb: No such file or directory
h-MacBook-Pro-sl-nimrod:~ nimrodros$
the idea was to just convert it to a file that won't cause the notebook to crash so i could extract the valuable code. any ideas on how to continue?

Related

Pandas Import Error when converting Python script to .exe with Pyinstaller

I am currently trying to convert my Python script that automates an Excel task for my boss to an executable so that I can send it to him and he can use it without having Python installed. One of the libraries I used in the script is Pandas, and when I run the .exe after successfully building it, I get "failed to execute script 'main' due to unhandled exception: No module named pandas."
This is what I used to build the .exe:
pyinstaller --onefile -w main.py
Obviously I have Pandas installed on my machine and the script works perfectly fine when I run it normally, so I don't really know why I am getting this error especially since I thought the whole point of me converting the script to an executable was to get rid of the need for other packages to be installed.
I have read other articles on here about a similar error relating to numpy, but none of the other solutions have helped me. I have tried doing --hidden-import pandas while building the executable and downgrading pandas to an older version, both to no success.
Any suggestions with help!

How can I fix the module import error when I run it on the command line after making SSH connection?

I am not working on lego mindstorm project using ev3-brick and successfully connected pc and ev3-brick using Pycharm and then transferred the code from my laptop to EV3-brick.
But whenever I try to implement the python file on SSH shell using terminal, the module import error occurs that no module named 'libs'. But since I transferred all the file and even set the path using export PYTHONPATH="${PYTHONPATH}:/home/robot/csse120/libs", it has to import all the modules. It seems that when I ran another file not using terminal, it correctly imports other modules or packages but whenever I run on the shell, it cannot find files on other packages.
I even tried inserting following code which is,
import os
os.environ['PATH'] += ':/home/robot/csse120/libs'
but it didn't work also in another file, it says it cannot import paho-mqtt thought it worked well when I didn't run it on the shell.
Can you help me solve this problem? I would appreciate your help.
enter image description here

vscode, the interactive mode, I can open ipynb but cannot import the modules which I have installed

I have created a conda environment (say, called 'ds0') and installed some packages (e.g., python, pandas etc.). I then in vscode set the interpreter to be the one which I just created. I expect I could have my code working propery in the conda environment in vscode.
but then I have a problem, when I use the interactive mode in the ipynb file, I cannot import the packages,
e.g.,
import pandas as pd
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-11-f9ebee165770> in <module>
----> 1 import pandas as pd
ModuleNotFoundError: No module named 'pandas'
other code in the ipynb is running properly (e.g., to print a message etc.).
so it looks like vscode in the interactive mode is not using the right environment - but I did set the environment as 'ds0', and also if I save the ipynb file as a python script(e.g., as 'test1.py'), I can actually run it and vscode does recognize the pandas package.
so, how can I fix the issue so that I can run the code properly in the interacrive mode (e.g., in ipynb file).
** this is macbook m1 laptop, I just have the latest anaconda and vscode installed. I also tried to create additonal conda environments and the isse is the same. I am not sure if there is a problem with the ipykernel.
Thanks you!
I now have a solution to the problem - though I do not quite know if what I found was the culprit.
What I did and what did NOT solve the problem:
I installed ipython, ipykernel in the vscode terminal and the conda terminal (may be duplicated) manually - what happened was that, if I create a new ipynb file, I can select the both the interpreter and the kernel properly, and the code runs perfect - but if I open the previous ipynb file, I still have the problem - it looks like vscode is using different kernels for two different ipynb files
What I did: I reinstalled the Mac OS Big Sur (this may be unnecessary though) and I installed a clean miniconda and vscode - however this does not solve the problem. vscode still does not use the right kernel (but it looks like it uses the right interpreter/env which I created via conda). It looks like vscode is not using the right kernel (the icon in the upper right hand side suggests that the jupyter server local is 'disconnected'), also it puts a warning msg saying 'invalid version -final'
What I did and what DID solve the problem:
Then I deleted all the existing ipynb file and re- cloned from Github. this solves the problem - but I do not know why and how it solved the problem.
In the top right corner, you can see the name of the kernel used ("Python 3.8.1 64-bit: Idle", for example). Click on that, and select the kernel you want to use. That's how I fixed the problem. The Kernel is named within the ipynb file, and if there's a mismatch between that and your system, the notebook will not run.

How to automatically reload modules in IPython?

Before I begin, I want to say that I am not a programmer; I am a geek and an engineer. Thus, I love coding and use it academically. Stackoverflow taught me more than 80% of what I know about python.
My problem is I need to manually reload the modules in my scripts by first importing importlib into my terminal and than using importlib.reload(*modulename*) to reload them. I want my IPython terminal to automatically reload the modules in my python scripts when I run them through my IPython terminal. This functionally was provided in previous version using the magic command %autoreload, which does not seem to work for me.
I have looked # the IPython documentation (link 1), tried using the %load_ext autoreload command (link 2) and the import ipy_autoreload followed by %autoreload 2 command (link 3). I found more than 4 other answers in stackoverflow telling me to do the things in either link 2 or 3; it didn't work for me. If anyone knows how to bring back autoreloading, it would make my fingers a bit happier.
Link 1: https://ipython.org/ipython-doc/3/config/extensions/autoreload.html
Link 2: https://stackoverflow.com/a/18216967/5762140
Link 3: https://stackoverflow.com/a/4765191/5762140
I am using a 64 bit installation of Windows 7. I have IPython 4.0.1 which came with my installation of Anaconda3 (3.18.9 64bit). Screenies about my error traceback from the IPython terminal when i try to use %load_ext autoreload can be provided on request.
All the links you have above use commands within ipython. You should try editing your config file. Open up your terminal and complete the following steps.
Step 1: Make sure you have the latest ipython version installed
$ ipython --version
Step 2: find out where your config file is
$ ipython profile create
Step 3: Open the config file with an editor based on the location of your config file. I use atom. For example:
$ atom ~/.ipython/profile_default/ipython_config.py
Step 4: Look for the following lines in the config file:
c.InteractiveShellApp.extensions = []
change it to:
c.InteractiveShellApp.extensions = ['autoreload']
and then uncomment that line
find:
c.InteractiveShellApp.exec_lines = []
change it to:
c.InteractiveShellApp.exec_lines = ['%autoreload 2']
and then uncomment that line
Done.

PyQt5 - pyuic5 module PyQt5.uic not found

I've upgraded my deveolpment environment from python-3.2/ Qt-4.8/ PyQt4 to python 3.4.2 / Qt5.3 / PyQt5.
Everything is Ok but the python source code generation with pyuic5.
When calling this commands I get the error
/usr/bin/python3: No module named uic
The Qt uic program is located at
~/Qt/5.3/gcc_64/bin/uic
The PyQt5 pacjakge is located at
/usr/local/lib/python3.4/site-packages/PyQt5
Even if the PyQt5 environment is operational as runtime I can't build any new GUI interface, so I can't move forward with this application upgrade.
Any idea ?
Thanks for help.
Thanks for the answer from #Akhil. The code below really helps me avoid the "module not found" problem, and successfully create a .py file from a .ui file.
exec python -m PyQt5.uic.pyuic youruifile -o yourpyfile -x
call python -m PyQt5.uic.pyuic -x filename.ui -o filename.py
this one worked for me (28th june 2017, PyQt5, python 3.6, win10, 64bit)
I faced the same issue. I had installed PyQt into a virtual environment and the issue was the the pyuic5 script was not using the python interpreter for that virtual environment.
So I opened up the pyuic5 script
exec pythonw2.7 -m PyQt5.uic.pyuic ${1+"$#"}
to
exec python -m PyQt5.uic.pyuic ${1+"$#"}'
This resolved the issue for me.
I know this is late, but I just recently encountered this issue when setting up external tools for Qt Creator. I know its not exactly what you ran into but it produces the same error. I don't like the idea of modifying files in my anaconda/bin directory so I didn't want to implement either of the other answers.
What I did find is that when you are setting up an external tool Qt Creator operates as if you are working in a new environment (i.e. your PATH is not the same as your development path). What I found is that you need to set the Environment field to have the same path as your development environment. This occurs by default when you setup Build and Run Settings for your project, but not for external tools.
Like #ekhumoro said in his comment You must pay attention to your PATH at all times.
I believe you are a Mac user and I had faced a similar issue.
It seems you'll simply have to provide the full path of the pyuic file (for me, it was under a hidden folder usr in the home directory: /usr/local/Cellar/pyqt/5.10.1_1/bin/pyuic5).
In your terminal, change directory to where the *.ui files( which you wish to convert to a *.py file) lies.
So for instance, if you have a Qt designer file saved by the name untitled.ui on your desktop, put in the following command in your terminal:
Amars-MacBook-Pro:Desktop amaradak$ /usr/local/Cellar/pyqt/5.10.1_1/bin/pyuic5 -x untitled.ui -o untitled.py
Hope this helps...
Cheers
Try this: i've been searching for 2 days and all problem solved!
First Rule in Python: Don't use .XML this is not Java instead use .PY for me i don't need uic at all
Let's begin, my pyuic5 actually there is no in PyQt5 folder instead in Scripts Folder
C:\......Python\Python37-32\Scripts\pyuic5.exe
you have to make sure that the yourFile.ui is in the same location of your pyuic5.exe
Follow the pict below, you will be understand:
How to convert ui to py for easy way
uic not found solved
it works like a Charm !
Hi everybody , see last pict, i just converted ui to py for the first time in my life !