I have deployed my Selenium Script which runs across multiple web pages on a particular website and it works fine on my system but after deploying on Google Cloud and Heroku it is showing an Error after running. I found that the possible reason is memory limit was exceeded.
I have uploaded app.yaml, requirements.txt and main.py file on Google Cloud.
and Procfile, requirements.txt and main.py file on Heroku.
On both platforms it is showing error. Please help me out.
I have encountered this problem once when deploying my selenium script as well.
For me it was indeed memory-related.
There are a few ways I implemented to control memory use mainly by running a headless webdriver and by disabling extensions.
For a chrome webdriver for example you can add the following options:
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--headless")
I, on the other hand, had to monitor the navigation and interact with it via selenium-grid so I could not run it in headless mode and my solution was to simply add computational resources.
Related
I am working on an integration test, using Selenium/Java/Cucumber. When I run the test on localhost, or remotely on a VM, it passes fine. But when it runs as part of a Jenkins pipeline it hangs with the error "Timed out receiving message from renderer". I have googled this and there are several possible causes, including version conflicts. But I can't seem to figure out how to check these versions in the jenkins pipeline. Any pointers on what to look into would be helpful. I believe the tests are running headless on the pipeline, because there is no Chromedriver pluigin installed in our Jenkins. The test is pretty straightforward - it involves clicking on an element within an iframe (this is where it hangs) and checking for some output. I have even tested it running manually on the VM created by the very same pipeline and it always works without an issue.
so I have this selenium and flask application that I deployed on digital ocean ubuntu VPS. when I run the application in development mode "main#ubutu$python3 pyfile.py" everything runs perfectly without throwing any error. But when I run the application on production mode, i.e when the app is deployed I get the following error.
The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed
I have the latest google-chrome and chromedriver.
main#ubuntu:~$ google-chrome --version
Google Chrome 102.0.5005.61
also in my code, I have included all the options that are required to run on a VPS without display.
option = Options()
service = Service("/home/path/to/chromedriver")
#option.binary_location = "/usr/bin/google-chrome-stable"
option.add_argument("--headless")
option.add_argument("--no-sandbox")
option.add_argument("--disable-extensions")
option.add_argument("--disable-setuid-sandbox")
option.add_argument("--remote-debugging-port=9222")
option.add_argument("--disable-gpu")
option.add_argument("--start-maximized")
option.add_argument("--disable-dev-shm-usage")
driver = webdriver.Chrome(service=service, options=option)
many people have experienced this problem but the option --no-sandbox and --headless seemed to have solved their problems. but even after that I still have the same issue.
google-chrome path /urs/bin/google-chrome.
any help will be appreciated thanks.
I was using digital ocean ubuntu VPS, so I followed their documentation on how to deploy flask applications on their servers. if at all you are following their documentation and you run into this. here is the solution.
thanks to #Lewis Morris
the problem was the path ENVIRONMENT_PATH that I used in the /etc/systemd/system/myapp.service, initially, it was
Environment="PATH=/home/main/searchapp/venv/bin
then I changed it to:
Environment="PATH=/home/main/searchapp/venv/bin:/usr/bin:/bin"
hope it helps someone in the future so they don't have to waste their time.
Currently, we use Github Actions to run our Testcafe suites, but the problem is running Safari tests on the macos-latest image, which is 10.15 Catalina, gives the following error:
UnableToAccessScreenRecordingAPIError: The find-window process cannot access the Screen Recording API.
This is because MacOS images don't come with that permission granted by default. Apparently you can't handle this programmatically, and there's a feature called SIP that is turned on in 10.15 Catalina that stops you from getting around it. This is discussed here.
Since we're relying on Github to provide images, they've not been able to provide us with a non-SIP image as discussed here. The workarounds provided there have been of no help.
Any other ideas?
As you already mentioned, TestCafe doesn't capture screenshots and resize them because of SIP enabled on machines with MacOS. However, it is possible to use the remote mode since the platform specific functionality is disabled by default. You can execute these steps:
testcafe remote test.js --hostname ${HOSTNAME} --ports ${PORT1},${PORT2} &
pid=$!
open -a Safari http://${HOSTNAME}:${PORT1}/browser/connect
wait $pid
I know I'll get crap for this but it's worth it if someone can help.
I can't find a guide, tutorial, or instructions anywhere for installing web driver on a windows machine. I've got a site running on a homestead vagrant box and need a way of running acceptance tests locally. I had tests setup using PhpBrowser but those don't simulate JavaScript. If anyone knows of a guide to do this or a better way to run acceptance tests it would help immensely. I've got Ajax calls so PhpBrowser and resources like it won't work.
Thanks!
Just download the selenium webserver the jar file and run it in a command shell with java -jar selenium-server-standalone-2.43.1.jar. The selenium server will now listen on the default port 4444. If you run your test it should work as expected. Keep in mind that selenium webserver opens firefox and uses it for testing. It's pretty useful for watching the test cases.
If you want a headless test (no visible browser) you need to download phantomjs. Unpack it and run the phantomjs.exe with --webdriver=4444 as an argument (so phantomjs.exe --webdriver=4444).
Download and Run the selenium-server-standalone-2.43.1.jar as stated in the comment. I had to add firefox_binary: C:\Program Files (x86)\Mozilla Firefox\firefox.exe under capabilities: in my acceptance.suite.yml file. I also added it to the path variable but I'm not sure if that made a difference. Had to add the firefox_binary to make it work.
I am using phantomjs to build a web crawler. My current crawler is a Python script using selenium, which requires a running Firefox browser. While Selenium is good for debugging scripts (as I can use firebug to inspect the webpage), it cannot be deployed on linux servers without GUI. So I am trying to translate my Python script to phantomjs.
To debug phantomjs scripts, I save the page source html and render a png screenshot to disk, and open it in Firefox to inspect the source page. I am wondering if there is a better way to do this, e.g. a plugin for phantomjs, etc.
This little hack is using a simple technique: It grabs the screen as PhantomJS or CasperJS sees it with captureBase64('png') and then it is POSTing the image into the receiving server which then sends it via socket.io to the browser which displays it is as inline image.
The source code is available on github:
https://github.com/maciejjankowski/flaming-octo-puss
I'm not sure if PhantomJS has it, but CasperJS does (and the latter adds some extra functions to PhantomJS)
and use remote debugging as others suggest
As for most of the JS scripts, it's not so trivial to debug phantomjs script, because there is no IDE/compiler behind this.
First, I higly suggest you to read the Troubleshooting section.
As said by torazaburo, your best option is to use Remote Debugging : scripts can be run in a Web Inspector interface-like console : step-in, step-out, break points, local variables... many classic debugger features are available. If you're familliar with Chrome developpers tools, it's pretty the same.
Though not a solution to your Phantomjs troubles,I think Selenium with Python bindings can be used very efficiently as a headless scraper in a Linux environment.
You can use PyVirtualDisplay, a Python wrapper for Xvfb and Xephyr to fake a display.PyVirtualDisplay needs Xvfb as a dependency. On Ubuntu, first install Xvfb:
sudo apt-get install xvfb
then install PyVirtualDisplay from Pypi:
pip install pyvirtualdisplay
Sample Selenium script in Python in a headless mode with PyVirtualDisplay:
#!/usr/bin/env python
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(800, 600))
display.start()
# now Firefox will run in a virtual display.
# you will not see the browser.
browser = webdriver.Firefox()
browser.get('http://www.google.com')
print browser.title
browser.quit()
display.stop()