In windows power shell ,I can run scrapy shell 'http://www.hao123.com',
scrapy shell 'http://www.hao123.com
I can run ipython
I can run ipython but not scrapy shell 'http://www.hao123.com'
ipython then scrapy shell 'http://www.hao123.com
In ipython notebook,I can't run scrapy shell 'http://www.hao123.com'also
scrapy shell 'http://www.hao123.com'
File "<ipython-input-3-be4048c8f90b>", line 1
scrapy shell 'http://www.hao123.com'
^
SyntaxError: invalid syntax
Ipython is installed by anaconda,scrapy is installed by pip,anaconda and pip is in different file.
Please help me!
That's not a feature you can have in ipython. scrapy shell is a command, it's own application completely separate from ipython.
However, there are two things you can do:
If you have a Spider and Response objects from somewhere you can simply use scrapy.shell.inspect_repsonse
from scrapy.shell import inspect_response
# You need a scrapy spider object that has crawler instance attached to it
some_spider = Spider()
# you need response object
some_response = Response()
inspect_response(some_spider, some_response)
# new ipython shell will be embedded, like one from scrapy shell command
Otherwise you can spawn a subprocess:
import subprocess
subprocess.call('scrapy shell http://stackoverflow.com', shell=True)
# new ipython shell will be embedded, like one from scrapy shell command
if you are using Anaconda on windows 7 .
Follow: environment --> root --> Open terminal. Then you can write:
scrapy shell 'http://www.hao123.com'
Related
I developed spider by scrapy, and run it by command
scrapy crawl myspider
Now I try to run it background by command:
nohup scrapy crawl myspider &
but after I close ssh session, the scrapy stop, why?
My guess is this has nothing to do with Scrapy.
Does this suggested solution work?
https://unix.stackexchange.com/questions/658535/what-is-the-real-reason-why-nohup-a-out-dies-when-ssh-session-times-out
Before we could run a script this way
script = 'script.py'
!python '$script'
However, recently, this error occurs
python3: can't open file '$script': [Errno 2] No such file or directory
Is there an updated way to run a script with python variables in colab?
You can try this instead.
!python {script}
Somehow,
!python '$script'
Is working again.
I have a Centos 7 server with cPanel and I'm working on a Telegram bot for my business needs. The bot should be able to run a terminal command with os.system or subprocess.Popen, however both options do not work when configured through a webhook + wsgi process.
I tested both with bot.polling method and they worked as a charm, however after I switched to webhook method served by flask and wsgi, both stopped working for me. I have tried the following:
mycommand = "python3.6 GoReport.py --id 31-33 --format word"
os.chdir('dir_to_run_command_from')
os.system(mycommand)
and the following one:
mycommand = "python3.6 GoReport.py --id 31-33 --format word"
subprocess.Popen(mycommand, cwd="dir_to_run_command_from", shell=True)
Both options simply do nothing right now. I tried to print them both and received 0 as a response. I wonder if the issue is caused by permissions or something.
I expect both options to work through webhook + wsgi as good as they work through bot.polling method.
I think I got it wrong. Your script writes a report to a specific directory. You do not need a result in your application route.
I wrote a small test application called tryout. It runs in a virtual environment.
$ mkdir tryout
$ cd tryout
$ python3 -m venv tryout
$ source tryout/bin/activate
$ export FLASK_APP=tryout/app
$ export FLASK_ENV=development
$ flask run
Directory structure:
/tryout
/app/*
/bin/*
/include/*
/lib/*
/subdir/*
Application:
# /tryout/app/__init__.py
import sys, os
from flask import Flask
def create_app(env=os.getenv('FLASK_ENV', 'development')):
app = Flask(__name__)
#app.route('/run-script')
def run_script():
import subprocess
cmd = 'python script.py'
cwd = 'subdir'
ret = subprocess.check_output(cmd, cwd=cwd, shell=True)
print(ret)
return ret, 200
return app
app = create_app()
Script:
# /subdir/script.py
import os, sys
def main():
with open('report.txt', 'w+') as fp:
fp.write('Info\n')
sys.stdout.write('It works!')
if __name__ == '__main__':
main()
It works!
A new file named "report.log" is written into the "subdir"-directory.
In Browser appears "It works!".
Hope I could help you or I have no real idea of what you want to do.
If you want to run an external script from inside flask, you could use subprocess to run the script from the command line. This is the right solution.
#app.route('/run-script')
def run_script():
cmd = '<your command here!>'
result = subprocess.check_output(cmd, cwd='<your wordir>', shell=True)
return render_template('results.html', **locals())
Have fun!
#Bogdan Kozlowskyi
Is it possible to pipe on the command line? Do you need to return a result to the user?
cmd = 'first_cmd | tee report.log'
result = subprocess.check_output(cmd, cwd='<your wordir>', shell=True)
Perhaps you should look for shell commands like '>>', '>' and 'tee'.
Seems to be a user-groups permission problem (execute and write).
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I can't convert ui to py
it's giving this:
Instead of installing Python packages by hand, I would consider using conda and pip from a recent Anaconda install (https://www.anaconda.com/download/).
After installing Anaconda with python 3.6, open a privileged (Run as Administrator) cmd or git bash and run the following commands:
Installing PyQt5
PyQt5 is the default one for Python 3.6. You can check available packages by running (conda search pyqt)
conda install pyqt
Generating .py file from .ui
python -m PyQt5.uic.pyuic -x [FILENAME].ui -o [FILENAME].py
Importing generated .py on your Python code
Now, suppose that your file is called MainWindow.py, and its type is QMainWindow. This is how you import it on Python
from PyQt5 import QtWidgets
from mainwindow import Ui_MainWindow
import sys
class ApplicationWindow(QtWidgets.QMainWindow):
def __init__(self):
super(ApplicationWindow, self).__init__()
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
def main():
app = QtWidgets.QApplication(sys.argv)
application = ApplicationWindow()
application.show()
sys.exit(app.exec_())
if __name__ == "__main__":
main()
you are using the correct syntax: pyuic5 -x file.ui -o file.py
but you have to make sure that the file.ui is in the same location of your pyuic5.bat
python -m PyQt5.uic.pyuic -x [FILENAME].ui -o [FILENAME].py
This worked for me. Thanks to Danilo Gasques
Sorry for my bad english
You can find file "pyuic5.exe" (For example it is "C:\Python\venv\Scripts\pyuic.exe")
Through the command line go to the folder with the file "needToConvert.ui"
Enter the following command line: C:\Python\venv\Scripts\pyuic.exe needToConvert.ui -o needToConvert.py
What you need is python3.dll file which is missing and you have to place in your python directory
Go here (https://winpython.github.io/).
Download the version of python you have and also see which bit
version
Download the zero version and extract it somewhere temporarily
In extracted folder search for python3.dll in the search bar
Extract where your python setup is and try then it will work
Use this .bat file to automatically convert all *.ui files to python files.
All you need is:
Save the script below to ui2py.bat file
Edit the file with notepad and specify the pythonPath directory from your PC
Save changes and execute the ui2py.bat file convertor
#echo off
rem set the python path
set pythonPath=G:\Programming\WinPython-64bit-3.6.3.0Qt5\python-3.6.3.amd64
echo [START] converting .ui files...
rem convert all .ui files in current directory
for %%i in (*.ui) do (
rem Display the file name
echo %%i -- ui_%%~ni.py
rem converting
%pythonPath%\python.exe -m PyQt5.uic.pyuic -x %%i -o ui_%%~ni.py
)
echo [END] converting .ui files...
I am running a shell command through jython os.system command. My code is given below:
import os
import sys
vCmdLine = "sh C:/scripts/WAQ.sh PARAM_1 PARAM_2 >> C:/logs/WAQ.log 2>&1"
vCmdRC = os.system(vCmdLine)
My shell has several echo commands which I want to get written in C:/logs/WAQ.log file.
The code is working fine if I execute the same on unix OS. However when I run the same on windows, it does not write anything in C:/logs/WAQ.log file. I am using unix util lib on windows.
What is the issue?
I have changed a code and replace os.system with subprocess.call and the logging is now perfect. Below is my code
import subprocess
vLogFileName = 'C:/logs/WAQ.log'
vCmdLine = "sh C:/scripts/WAQ.sh PARAM_1 PARAM_2"
f = open(vLogFileName, "a")
subprocess.call(vCmdLine, stdout=f)
f.close()
Why the logging is not working with os.system on windows?