I understand this is a little vague but not sure where else to go to or things to debug. My python script was running fine yesterday. I made minor changes today and now it only runs successfully on my Amazon LightSail (ec2) machine. Everything I read about segmentation errors is that there is not enough memory, however my local machine has 16gb of ram while the cloud machine only has 1gb. Plus I am not working with big files? The files being imported/manipulated are typically under 2mb and there are like 7-10 files.
I feel it may be something related to my terminal/zsh rather than my codes.
The below is the error code I can not seem to manage to get around.
I've done enough research to find the python faulthandler module import faulthandler; faulthandler.enable() to give the debugging below:
Fatal Python error: Segmentation fault
Current thread 0x000000010c58edc0 (most recent call first):
File "/Users/garrett/opt/anaconda3/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 1795 in <genexpr>
File "/Users/garrett/opt/anaconda3/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 1797 in <listcomp>
File "/Users/garrett/opt/anaconda3/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 1797 in count
File "GmailDownloader.py", line 215 in <module>
zsh: segmentation fault python *.py
The code seems to regularly break on line 215 while trying to compute a gorupby in pandas but it is very similar to other groupbys in the code that were successful before it.
I am on a Mac Catlina using the pre-baked zsh for my terminal handling but even when I switch to good ol' bash using chsh -s /bin/bash in my terminal and then running the code I still get a zsh segmentation error.
I have recently tried out PyCharm today and it asked for permissions to store something in a bin folder to which I just said yes. I'm not sure if that is correlated at all or not.
The full code repository: https://github.com/GarrettMarkScott/AutomotiveCRMPuller
Ongoing list of other things I have tried:
Trashing the Terminal preferences (~/Library/Preferences/com.apple.Terminal.plist)
I almost threw in the towel but tried to reinstall my pandas since it was mentioned in my bug error and what do you know it worked after running pip install --upgrade pandas
Would of been impossible without the FaultHandler! Hopefully this helps someone out there!
Related
I work on a dual boot (Windows 10 and Linux) machine. I am attempting to share the Miniconda distribution I used in Linux (Manjaro) with WSL (Pengwin). As I navigated the working directory to the binary folder of Miniconda (miniconda/bin) and issued ./conda activate. Error arose:
import-im6.q16: unable to open X server `localhost:0' # error/import.c/ImportImageCommand/346.
./conda: line 6: syntax error near unexpected token `sys.argv'
./conda: line 6: `if len(sys.argv) > 1 and sys.argv[1].startswith('shell.') and sys.path and sys.path[0] == '':'
It is strange that a command line utility should rely on a display server. Though, I installed Xming and again issued ./conda activate. This time no error arose, but the program hung on, seemingly waiting for some of my input but I am not sure what. I saw no information anywhere.
So, is it possible to share the Linux distribution of Miniconda with WSL? How can I overcome the roadblock mentioned above? Please help! Thanks!
I am currently trying to convert my Python script that automates an Excel task for my boss to an executable so that I can send it to him and he can use it without having Python installed. One of the libraries I used in the script is Pandas, and when I run the .exe after successfully building it, I get "failed to execute script 'main' due to unhandled exception: No module named pandas."
This is what I used to build the .exe:
pyinstaller --onefile -w main.py
Obviously I have Pandas installed on my machine and the script works perfectly fine when I run it normally, so I don't really know why I am getting this error especially since I thought the whole point of me converting the script to an executable was to get rid of the need for other packages to be installed.
I have read other articles on here about a similar error relating to numpy, but none of the other solutions have helped me. I have tried doing --hidden-import pandas while building the executable and downgrading pandas to an older version, both to no success.
Any suggestions with help!
I've uploaded a directory of codes in Google Colabs. I need to run python command lines in a terminal that I'm unable to open.
I tried each and every solution suggested in How can I run shell (terminal) in Google Colab? but to no avail.
Updated 20220202
Can you try to execute your python scripts using the exclamation mark ! from Colab's cell directly?
I believe I encountered identical issue back in my Colab days, the process will hung or froze over some time, especially when I am dealing with GBs of datasets. So, ultimately I just ran using the exclamation mark directly from my Colab's cell to resolve the issue.
Have you tried this with the following syntax?
!pip install google-colab-shell
from google_colab_shell import getshell
getshell()
getshell(height=400)
I understood that you have tried the solution from another post, but just in case that you missed out the !, and ignored the warning message, and that exclamation mark was actually causing the shell not spawning.
This is my first time asking a question here so I will try to be descriptive. I am relatively new to python and tensorflow, and have been learning it specifically for a project.
I am currently working to deploy a tensorflow model using tensorflow serving and flask with wsgi. I have been following this architecture: https://github.com/Vetal1977/tf_serving_flask_app
I am running tensorflow_model_server on port=9000.
I know that tensorflow_model_server is working, because when I execute the tensorflow_serving_client.py from command line, I get the expected response. I have tested this command line execution on every user account.
Similarly, I know that Flask + WSGI is working, because I can see log.info points dropping into the apache error log as it works its way through the script. If I return something before it gets to the line in question, it works just fine.
However, when the application is executed with Flask + WSGI, it hangs at this line: result = stub.Predict.future(request, 5.0) # 5 seconds (https://github.com/Vetal1977/tf_serving_flask_app/blob/master/api/gan/logic/tf_serving_client.py#L70)
I can see that it hangs as I monitor top and tail -f error.log and see the same apache process sit there until it is killed or apache is restarted.
I am really stuck on the fact that it works when executed via command line, but not when Flask + WSGI runs it. Can anyone provide suggestions or point me in the right direction? Am I headed down the right path with this? Any assistance at all would be greatly appreciated.
EDIT: I have uploaded the minimal code to a github repo here: https://github.com/raymondtri/client-test along with a minimal setup that does require flask, wsgi, tensorflow, and tensorflow-serving.
Thanks in advance,
Ray
After much research and many hours, I think that this has something to do with how mod_wsgi forks processes, and how grpc has some known issues with that. I have a feeling that things are getting orphaned as the process is forked and that is what is causing the script to hang.
I could be wrong though, that is just my current hypothesis.
Problem when calling netconvert in sumo:
I am trying to create my own scenario for simulation purposes.
I am using OpenStreetMaps for this.
python osmWebWizard.py
opens the browser and I select the area which I download.
netconvert --osm-files osm_bbox.osm.xml -o osm.net.xml
The error message I get is
Error: Cannot import network data without PROJ-Library. Please install packages proj before building sumo
Warning: Environment variable SUMO_HOME is not set, using built in type maps.
Quitting (on error).
My attempt to fix the problem is:
sudo apt-get install libproj*
But it seems like a dead end there and I am out of options.
Thank you.
EDIT
I have a gut feeling it has to do with libproj0 not being available anymore.