I'm trying to bypass a form protected by recaptcha using CasperJS. I'm using a third party service the provide the function to bypass recaptcha, however the captcha is never rendered.
var casper = require('casper').create({
pageSettings: {
userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11"
},
verbose: true,
logLevel: "debug"
});
casper.start("https://website.com/protected/form");
casper.then(function() {
casper.wait(5000, function() {
casper.capture('/home/phantomjs/downloaders/screenshot-1.png');
});
});
casper.run();
Upon viewing the screenshot I see the recaptcha form is never rendered. I've tried adjusting the wait time to a longer period, no matter how long I wait it's never rendered.
Related
We’re using Azure B2C to authenticate the users in the Word add-in. The library in Angular (14.1.2) that we use is MSAL which works as expected on Windows.
However, when using it on Mac, the redirect inside the add-in dialog gets stuck and doesn’t go to the Azure B2C page. It writes the cookies and local storage, but never redirects.
angular 14.1.2
azure/msal-angular 2.5.1
azure/msal-browser 2.32.1
Call
this.msalService.loginRedirect().subscribe();
Msal Configuration
`export function msalInstanceFactory(appConfigService: AppConfigService): IPublicClientApplication {
const appConfig = appConfigService.config;
return new PublicClientApplication({
auth: {
clientId: appConfig.aadB2CClientId,
authority: 'https://${appConfig.aadB2CDomain}/${appConfig.aadB2CTenant}/${appConfig.aadB2CSIPolicy}',
redirectUri: appConfig.aadB2CRedirectUri,
postLogoutRedirectUri: getLocationOriginWithPath('logout'),
knownAuthorities: [appConfig.aadB2CDomain],
navigateToLoginRequestUrl: false,
},
cache: {
cacheLocation: BrowserCacheLocation.LocalStorage,
storeAuthStateInCookie: isIE,
},
system: {
loggerOptions: {
loggerCallback: msalLoggerCallback,
logLevel: LogLevel.Verbose,
piiLoggingEnabled: true,
},
},
});
}
export function msalGuardConfigFactory(appConfigService: AppConfigService): MsalGuardConfiguration {
return {
interactionType: InteractionType.Redirect,
authRequest: {
scopes: appConfigService.config.aadB2CConsentScopes,
loginHint: getLoginHint(),
},
};
}`
The funny fact is that if I set a break point just before the execution of the loginRedirect and then press continue it works. So, I tried to set a timeout before that to see if it works but it doesn't.
Also tried to disable ITP security in Safari, but didn't take any effect.
I've searched related topics. The most significant I found is this Setting user-agent in browsers with testcafe
But it doesn't provide any real aswers.
My goal is to run the test spoofing a different OS: Since I'm in Linux and the app I'm testing isn't supported for that, it shows a couple of warnings that I would want to get rid when tests are running.
We tried cypress, in which you just add the UserAgent string on a config file and that's it. But I haven't found a straightforward way of doing it on testcafe without a CLI parameter.
Is there a way to spoof an OS or userAgent in testcafe?
You can modify user-agent using the RequestHooks mechanism.
I prepared an example to demonstrate this approach:
import { RequestHook } from 'testcafe';
class UserAgentRequestHook extends RequestHook {
onRequest (e) {
e.requestOptions.headers['user-agent'] = 'Mozilla/5.0 (Android 4.4; Tablet; rv:41.0) Gecko/41.0 Firefox/41.0';
}
onResponse (e) {
}
}
const hook = new UserAgentRequestHook();
fixture `f`
.page `https://www.whatismybrowser.com/detect/what-is-my-user-agent/`;
test.requestHooks(hook)(`test`, async t => {
await t.debug();
});
Please note that TestCafe is using UserAgent internally, so incorrect UA value can lead to unpredictable results.
I am using agora-rtc-sdk 3.3.0. When i click on "share screen" and gave permission (problem is not with getDisplayMedia) safari reload page. The line which cause it, is client.publish(stream), if comment this line stream created successfully (according to console), same as client but i cant publish my "screen sharing" stream. This bug appears only in safari 13, other browsers works fine. Adding this part of code.
const beginShare = async() => {
agoraShareStream = AgoraRTC.createStream({
streamID: SHARE_ID,
audio: false,
video: false,
screen: true
});
await agoraShareStream.init(() => {
console.log('init local stream success')
});
setShareStream(agoraShareStream);
enqueueSnackbar("Started screen sharing", { variant: "info" });
}
const createShareClient = async() => {
const agoraShareClient = AgoraRTC.createClient({
mode: "rtc",
codec: "vp8"
});
await agoraShareClient.init(appId);
await agoraShareClient.join(getAgoraToken({ uid: SHARE_ID, channel }), channel, SHARE_ID);
agoraShareClient.publish(agoraShareStream);
setShareClient(agoraShareClient);
}
Unfortunately, Agora doesn't support screen sharing on Safari as noted in the documentation here: https://docs.agora.io/en/Video/screensharing_web?platform=Web#introduction
The Agora RTC Web SDK supports screen sharing in the following desktop browsers:
Chrome 58 or later
Firefox 56 or later
Edge 80 or later on Windows 10+
I'm a Japanese chess player and I would like to plot popularity of a strategy in function of time. To do that, I have a website database with a link for the first strategy called Yagura:
https://shogidb2.com/strategy/%E7%9F%A2%E5%80%89/page/1
What I would like to do is to store the years that appears at the beginning of each game (like this I can store it, then count). In this page "2017". But, it is impossible to get the text information. I also tried to find the web links to get data from the game page... But the links don't appear...
Here is my code, if you have any tip, you are welcome, I start to be crasy ^^
import requests
from bs4 import BeautifulSoup
def downloadString(url, params = {}, cookies = {}):
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
data = requests.get(url, params = params, headers = headers, cookies = cookies)
return data.text
url = "https://shogidb2.com/strategy/%E7%9F%A2%E5%80%89"
html_doc = downloadString(url, params = {}, cookies = {})
soup = BeautifulSoup(html_doc)
links = []
for link in soup.find_all("a"):
print(link.get("href"))
The problem is the website is built with ReactJS, it creates VirtualDom to populate the data. BeautifulSoup on the other hand looks for DOM elements. Since DOM is not created for the elements, it will get null values. The best solution is to use casperjs (http://casperjs.org/)
The only reason I'm suggesting something like casperjs is much simpler to use than python supported scraping modules like selenium. If you are very serious about your pythonic way, Selenium should work for you. But its hard to configure first time.
Do install phantomjs and casperjs with npm install -g phantomjs casperjs.
PS: Phantomjs is used by casperjs, its just a dependency of casperjs.
// scrape.js
var casper = require('casper').create();
var links;
function getLinks() {
// Scrape the links from top-right nav of the website
var links = document.querySelectorAll('a');
return Array.prototype.map.call(links, function (e) {
return e.getAttribute('href')
});
}
// Opens casperjs homepage
casper.start('https://shogidb2.com/strategy/%E7%9F%A2%E5%80%89');
casper.then(function () {
links = this.evaluate(getLinks);
});
casper.run(function () {
for(var i in links) {
console.log(links[i]);
}
casper.done();
});
To run the script: casperjs scrape.js
After trying this answer
How to login into a website with CasperJS? I didn't work apparently fill function is the one failing
Now facebook doesn't have emai, and pass as direct children of login_form does this affect the code ? I figured it stops at this.test.assert...
my code
var casper = require('casper').create({
pageSettings: {
loadImages: false, // The WebPage instance used by Casper will
loadPlugins: false, // use these settings
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4'
}
});
var url = 'https://www.facebook.com/';
casper.start(url, function() {
console.log("page loaded");
this.test.assertExists('form#login_form', 'form is found');
this.fill('form#login_form', {
email: 'email',
pass: 'pass'
}, true);
});
casper.then(function() {
this.evaluateOrDie(function() {
return /message sent/.test(document.body.innerText);
}, 'sending message failed');
});
casper.run(function() {
this.echo('message sent').exit();
});
In case no answers I got what I need working with PhantomJS only
Version problem, not clear what was the problem.
Change the call to phantomjs in the batchbin/casperjs.bat file from
call phantomjs "%CASPER_BIN%bootstrap.js" --casper-path="%CASPER_PATH%" --cli %ARGV%
to
call phantomjs --ignore-ssl-errors=yes "%CASPER_BIN%bootstrap.js" --casper-path="%CASPER_PATH%" --cli %ARGV%
This simply forces the ignore-ssl-errors on every call to phantomjs, which for my use case was fine. This is not a fix, just a hack.
https://github.com/n1k0/casperjs/issues/49