I am trying to set up sock5 proxy setup with firefox + python + selenium. I am having
IP and PORT of the proxy server. Even i added username and password it is giving pop up windows.
from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.proxy import *
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference('network.proxy.ssl_port', int(PROXY_PORT))
profile.set_preference('network.proxy.ssl', PROXY_IP)
profile.set_preference("network.proxy.http", PROXY_IP)
profile.set_preference("network.proxy.http_port", int(PROXY_PORT))
profile.set_preference("network.proxy.no_proxies_on", 'localhost, 127.0.0.1')
profile.set_preference("network.proxy.socks_username", PROXY_USERNAME)
profile.set_preference("network.proxy.socks_password", PROXY_PASSWORD)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile, executable_path="./geckodriver")
driver.get("https://www.whatismyip.com/")
time.sleep(10)
driver.close()
System Information:
Python 3.8.10
Ubuntu 20.04
Mozilla Firefox 99.0
That's just your proxy service, I had that same problem. I use a service called smartproxy.com. You can write your computer's IP address on the website for the authentication process. If the IP address that is running the code matches the IP address you listed on their site, there will be no pop up.
I do not know the proxy service you are using but it may have an authenticated method similar to the one I explained.
Related
I have Integrated the Selenium with JMeter to perform Performance Test, For this I have used Firefox driver Cofig and gecko driver. Script is executing successfully in local window machine But when trying to execute it on Linux server it giving the following error.
"The proxy moz-proxy://host:port is requesting a username and password. The site says: “LDAP”:"
Note:
gecko driver is compatible with Firefox version.
username and password passed while setting proxy on server, and also through JMeter command.
It looks like you're behind a corporate proxy server which requires authentication and unfortunately this is not something you can bypass using Firefox Driver Config
You will need to switch to JSR223 Sampler with Groovy language and instantiate the browser manually providing the proxy host/port/username/password/etc. Take a look at Proxy class JavaDoc
Example code:
import org.openqa.selenium.Proxy
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxOptions
Proxy proxy = new Proxy()
proxy.setHttpProxy("http://username:password#proxy-host:proxy.port")
proxy.setSslProxy("http://username:password#proxy-host:proxy.port")
FirefoxOptions options = new FirefoxOptions()
options.setCapability("proxy", proxy)
FirefoxDriver driver = new FirefoxDriver(options)
driver.get('http://example.com')
So I have figured out how to get started and open a Kameleo browser profile using Python. However, I find the Session ID and Port the chrome browser was started with. I think I have this, but my session ID is throwing an error.
I was expecting the /profiles/{guid}/start endpoint would return a JSON dictionary with the session id and port, also would be nice to have that under the profiles/{guid}/status http calls. I couldn't find it in the swaggerhub documentation.
This is the code I'm using
from kameleo.local_api_client.kameleo_local_api_client import KameleoLocalApiClient
from kameleo.local_api_client.builder_for_create_profile import BuilderForCreateProfile
client = KameleoLocalApiClient()
base_profiles = client.search_base_profiles(
device_type='desktop',
browser_product='chrome'
)
# Create a new profile with recommended settings
# for browser fingerprinting protection
create_profile_request = BuilderForCreateProfile \
.for_base_profile(base_profiles[0].id) \
.set_recommended_defaults() \
.build()
profile = client.create_profile(body=create_profile_request)
# Start the browser
client.start_profile(profile.id)
According to the documentation you don't need to get the port and the sessionID manually as you can make the connection to the browser through Kameleo.CLI.exe port.
If you keep on reading the README you will find an example where they showcase the W3C WebDriver connection.
# Connect to the running browser instance using WebDriver
options = webdriver.ChromeOptions()
options.add_experimental_option("kameleo:profileId", profile.id)
driver = webdriver.Remote(
command_executor=f'{kameleoBaseUrl}/webdriver',
options=options
)
# Use any WebDriver command to drive the browser
# and enjoy full protection from Selenium detection methods
driver.get('https://google.com')
I could also find this code in Kameleo's example repository.
I've run into the following problem while attempting to emulate a mobile browser with chrome on heroku: Selenium ERR_NAME_NOT_RESOLVED on heroku with mobile testing. Based on Josh Lewis's answer, and the python code sample at https://sites.google.com/a/chromium.org/chromedriver/mobile-emulation
from selenium import webdriver
mobile_emulation = { "deviceName": "Nexus 5" }
chrome_options = webdriver.ChromeOptions()
chrome_options.add_experimental_option("mobileEmulation", mobile_emulation)
driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities = chrome_options.to_capabilities())
It may be that I need to use a command_executor (defined at https://seleniumhq.github.io/selenium/docs/api/py/webdriver_remote/selenium.webdriver.remote.webdriver.html) as
command_executor - Either a string representing URL of the remote server or a
custom
remote_connection.RemoteConnection object. Defaults to
‘http://127.0.0.1:4444/wd/hub'
to get my code working. How would I found this with the Heroku chrome buildpack, https://github.com/heroku/heroku-buildpack-google-chrome
edit:
You can download my project for both windows and heroku use at:
https://github.com/kc1/mobiletest
(keep in mind that if you deploy to heroku you have to set FLASK_CONFIG to production. )
I'm using selenium with phantomjs under Ubuntu 16.04. I want to make use of Tor proxy. There is, of course, possibility to set a proxy via host:port pair like so:
from selenium import webdriver
service_args = [
'--proxy=127.0.0.1:9050',
'--proxy-type=socks5',
]
driver = webdriver.PhantomJS(service_args=service_args)
But I wonder if there is a possibility to make phantomjs use a unix domain socket (configured in Tor, of course) instead of host:port pair.
Setting it simply like --proxy=/run/tor/tor.socket does not make the trick - the request to check.torproject.org shows that. Maybe there is some special syntax for that option?
I want to use Selenium Webdriver and I am unable to do so because when I run my code, I get the following exception.
My code is very basic and as follows.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
Exception Message
selenium.common.exceptions.WebDriverException: Message: '<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD>\n<BODY>\n<FONT face="Helvetica">\n<big><strong></strong></big><BR>\n</FONT>\n<blockquote>\n<TABLE border=0 cellPadding=1 width="80%">\n<TR><TD>\n<FONT face="Helvetica">\n<big>Access Denied (authentication_failed)</big>\n<BR>\n<BR>\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nYour credentials could not be authenticated: "Credentials are missing.". You will not be permitted access until your credentials can be verified.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nThis is typically caused by an incorrect username and/or password, but could also be caused by network problems.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica" SIZE=2>\n<BR>\nFor assistance, contact your network support team.\n</FONT>\n</TD></TR>\n</TABLE>\n</blockquote>\n</FONT>\n</BODY></HTML>\n'
It opens firefox but after that it is unable to connect to google or any other local sites.
The exception is at driver = webdriver.Firefox()
I googled around and I followed the link on SO.
But unfortunately I still get the same error.
I cannot run as the root user. I changed my proxy settings and set No Proxy element for localhost as well as mentioned in the link.
I am using Python 2.7 and have installed selenium 2.31 version.
I also tried setting proxy.
myProxy = "*********:8080"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': myProxy,
'ftpProxy': myProxy,
'sslProxy': myProxy,
'noProxy': 'localhost,127.0.0.1,*.abc'
})
driver = webdriver.Firefox(proxy=proxy)
I also tried to set the proxy to system's proxy i.e., in the above code, 'proxyType': ProxyType.SYSTEM
But it again gives the above exception message.
is there a place where I have to set my username and password?
Any help would be appreciated!
Remove proxy settings from all the browsers on the system manually. I had IE, Firefox and Google Chrome.
When I removed the proxy settings of all the browsers and enabled the proxy only on Firefox,it worked without giving any errors. I do not know the exact reason why this works like this, may be got to do with the registry settings on windows which I am not sure about.
After doing the above said, I ran the basic code and it worked fine.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
I didn't set the proxy explicitly also. By default, it had taken the system's proxy settings. Hope this would help others facing similar issue.