I am attempting to do screen scraping using phantom js.
I have copied some phantomjs code from this site: http://snippets.aktagon.com/snippets/534-How-to-scrape-web-pages-with-PhantomJS-and-jQuery
Starting with that script, I have modified into this: http://jsfiddle.net/dqfTa/ (see javascript)
My aim is to collect the prices from a website, which are the inner html of the ".price" tags, into a javascript array. Right now I am trying to console.log() them to my screen.
I am running phantomjs v1.6 and jquery v1.8 through the ubuntu 12.04 console. I am setting the user agent to "iPhone".
Here is my output:
nwo#aws-chaos-us-w-1:~/sandbox$ phantomjs usingjqueryandphantom.js
hello
success
============================================
Step "0"
============================================
It never gets past step 0. Take a look at my code, I did a console.log("h1"); but it won't output it. What am I doing wrong here?
Phantomjs requires you to hook into the console output coming from it's page context. From the API reference:
This callback is invoked when there is a JavaScript console message on
the web page. The callback may accept up to three arguments: the
string for the message, the line number, and the source identifier.
By default, console messages from the web page are not displayed.
Using this callback is a typical way to redirect it.
page.onConsoleMessage = function(msg) {
console.log("This message came from the webpage: "+ msg);
};
Related
How can i show API request in terminal of vs code?
So i'm using vs code and i would like to my API response in terminal.
for example
GET -> API URL -> Status code
for each page that i enter.
Programatically if you wanna print the response in your terminal then you need to something like
axios().then(resp => console.log(resp));
If you wanna test your api's from within VSCode then do
Check this out a VSCode extension
https://www.thunderclient.io
I am not sure if their is anyway possible for you to fetch data from API on the terminal of VScode. Unless you can create your won Vs Code plugin to handle on the same.
Check out this article, it will help you with that.
https://blog.bitsrc.io/vs-codes-rest-client-plugin-is-all-you-need-to-make-api-calls-e9e95fcfd85a
I'm trying to run simple cypress test and get error like this:
but if i change visit url to another - it opens normally:
I cant understand why this does this error come with this url:
"baseUrl":"https://apps.powerapps.com/play/f1t75er8-03c9-4cc7-996d-6743cd7eac4a"
another visible error:
code:
describe("Create request", () => {
it("Should have a link that navigates to office365 login", () => {
cy.visit("/");
cy.get('div[class="loginNeededText_uf5ohz"]').contains('Sign in to start using Power Apps');
});
});
P.S. Also can't load https://apps.powerapps.com/ page. Error:
In general
With Cypress, you should not try to visit or interact with sites or servers you do not control. The main purpose of Cypress is testing, so when you try to do anything else with it, the task will become significantly more complicated (you can read more about it in the Cypress documentation).
If you really need information from a 3rd party site, it is recommended you do it through an API — mainly with the cy.request() method (e.g., Microsoft has quite an extensive documentation on how to work with the Office 365 Management API).
Your case
When you visit the apps.powerapps.com, you are being redirected to the Microsoft login page — it has a different domain name, which breaks a same-origin policy causing the cross-origin error in the Cypress console.
Seems that the page you specified in the baseUrl returns a 404 status code, along with some ugly CORS errors in the console, which suggest an issue with the app itself. After you solved those, you would most probably face the need of authentication anyway, and it is better to do it through an API.
When I use your base URL https://apps.powerapps.com/play/f1t75er8-03c9-4cc7-996d-6743cd7eac4a I see only blank screen and 404 error in the console, I am not redirected to login.
Maybe your URL should be https://apps.powerapps.com and after you login you can navigate to "/play/f1t75er8-03c9-4cc7-996d-6743cd7eac4a".
I am working on a Laravel 8 project. I have noticed that a couple of things have changed in the Laravel 8 including authentication. I am using Jetstream and Intertia JS for authentication and admin panel. I am now having trouble writing Browser test using Dusk for login. It seems to be that the login is not working in the test even though it is actually working.
This is my test
function test userCanLogin()
{
$this->browse(function (Browser $browser) {
$admin = User::factory()->create();//I make sure that the password is Test.1234 and I can login on the browser.
$browser->visit('/login')->type('email', $admin->email)
->type('password', 'Test.1234')
->press('Login')
->assertAuthenticatedAs($admin);
});
}
When I run the test, I am getting the following error.
The currently authenticated user is not who was expected.
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
## ##
-Array &0 (
- 'id' => 1
- 'className' => 'App\Models\User'
-)
+Array &0 ()
What is wrong with my code and how can I fix it?
I'm not 100% sure but I think by default Dusk waits for a page refresh before doing asserts. Since Inertia is loading via JS you need to either explicitly pause() or waitFor() some element that would indicate that the page has been loaded. (See Dusk docs on waiting)
There's also this library that adds events to Inertia.js that you can listen for in Dusk
In our project we are using Selenium to test our Angular Web Application using a docker image and chrome with version 70.0.3538.77 (Official Build) (64-bit). Since the application uses basic authentication on our servers we created our own extension to listen to "onAuthRequired" and whenever this happens we return the username and password (Original Source).
A simple test for our application would be to navigate to the main page (basic auth login required with the popup window) and then to another subpage that is hosted inside of this page in an iframe where the basic authentication is required again.
We added some logging to the extension just to check whether the onAuthRequired gets triggered so the extension currently looks like this:
​if (!chrome.webRequest.onAuthRequired.hasListener(retrieveCredentials)) {
chrome.webRequest.onAuthRequired.addListener(
retrieveCredentials,
{urls: ["<all_urls>"]},
['blocking']
);
}
function retrieveCredentials(details) {
console.log("onAuthRequired");
return {
authCredentials: {
username: username,
password: password
}
};
}
What we found out is that with the Version 72.0.3626 this was still working. The extension would fill out both of the login prompts that occur and we would see two messages with "onAuthRequired" in the console.
However with the Version 73.0.3683 and newer, this seems to not work anymore. Only one time the onAuthRequired (retrieveCredentials-Method) is called and the second popup gets ignored by the extension. This lets the Selenium tests fail with a timeout exception since the navigation doesn't work anymore.
Does anyone know how to handle this in the newer Versions of Chrome or sees a mistake that we made in our extension?
Any help appreciated
I am running testcafe tests on an authentication page and I can see that testcafe is modifying/removing/adding the headers when sending the requests to the website and this is blocking me to do 2FA on this page
As soon as I got the issue, I tried to do the automation with Selenium just to confirm it is testcafe issue. As selenium doesn't create a proxy to insert the js scripts and automate the website I could do the automation with selenium, but I want to use testcafe as the site is developed in react.
await t.typeText(this.emailInput, config.userEmail)
.click(this.nextButton)
.typeText(this.passwordInput, config.userPassword)
.click(this.nextButton)
.click(this.otpOption)
.typeText(this.otpInput, this.token)
.click(this.signinButton)
}
When clicking on the next button I should have the 2FA form asking for the code, but I got a page saying was not possible to do the authentication (Something wrong happened) and I saw the response code for the BeginAuth endpoint was 222 without any response instead of 200.
The URL is that I am using to authenticate looks like this one:
https://login.microsoftonline.com/client uuid/oauth2/authorize?response_type=code%20id_token&response_mode=form_post&client_id=client uuid&scope=openid&x-client-Ver=4.0.0
Testcafe team found out this is a bug on testcafe-hammerhead, they have fixed and it is going to be included in the next release.
https://github.com/DevExpress/testcafe-hammerhead/issues/2052
For now I am generating the cookie in the automation and sending it in the header.