I'm running puppeteer on an express.js server. Some of the webpages I go to require puppeteer to login. I'm using
const browser = await puppeteer.launch({
userDataDir: "./user_data/",
});
to store the session cookies so that I don't have to login each time, however, whenever I restart the express.js server the cookies seem to disappear and I'm required to login again.
Why does this keep happening? And is there a way to stop it from happening?
Hmm, I can't say I know exactly what's causing your issue, but I also experienced a number of difficulties getting puppeteer to properly save all session data (building bots to login to bank accounts to check balances, etc.) To remedy this I created some helper functions that use zlib to pack up the cookies (both from the direct Chrome Dev Tools client command, and the puppeteer API) along with the entire user data directory as a base64 string.
Here's a gist I created that you can run. You will need to decide where to store the base64 string containing the session data, but once done, you can import it into any puppeteer instance using the appropriate function and replicate the exact browser state. In my case, I am using firestore but obviously any storage solution will do.
Related
A user of my app reported an issue today about authorizing the user with Google (using OAuth 2.0). So far the application was opening a new BrowserWindow (node integration disabled, session is separated from the main application). You can see the implementation here since the library is OSS. I am using this to authorize the user to access application data on Google Drive.
Today after logging in I see the following message:
This browser or app may not be secure.
Try using a different browser. If you’re already using a supported browser, you can refresh your screen and try again to sign in.
The learn more link has a section for developers. This section has 2 links. One is how to upgrade the application to PWA. Because the application is an API testing tool it won't be possible to run it in a web browser.
The second link points to a document describing how to migrate to authorization for native application. However described flow requires authorization_code grant. This means I need to include OAuth secret into my application. Electron application, however, is still web application and there's no notion of compiling sources. I would expose client secret to the public which is not secured. Potentially I could build a server application to support it but the app is OSS project. It does not have funding to run a server for authorization.
My question is now how should I implement OAuth 2 for Electron application then. I can't use PWA's and server authorization flow (code grant) is far from ideal in this case.
As Paweł explained, changing the user agent will do the trick. However, you can easily set the user agent by passing an object when loading the URL
win = new BrowserWindow({width: 800, height: 600});
win.loadURL(authUrl, {userAgent: 'Chrome'})
I have tested it and it worked like a charm
Warning: This answer relies on changing the browser's user-agent. As of Jan. 2021, Google disapproves of this and warns not to do this (see EDIT4). Use at your own risk!
The other answers didn't work for me (in Electron 9.0.5), but I eventually found this, which worked:
app.on("ready", ()=> {
session.defaultSession.webRequest.onBeforeSendHeaders((details, callback) => {
details.requestHeaders["User-Agent"] = "Chrome";
callback({ cancel: false, requestHeaders: details.requestHeaders });
});
CreateMainWindow(); // your regular code to create root browser window
});
EDIT: Two other approaches, which I haven't tested, but which may also work:
app.on("ready", ()=>{
session.defaultSession.setUserAgent("Chrome");
...
}
app.userAgentFallback = "Chrome";
EDIT2: Trying again sometime later, approach #2 did not work, but #1 still did. Haven't tried #3 yet.
EDIT3: Trying again later still, it seems that none of these workarounds are needed anymore! Google appears to accept sign-in popups from Electron apps again, without modifications to the user-agent. (odd that they'd revert this; perhaps I just did something wrong in my re-attempt)
EDIT4: While approach #1 still works atm, I recently found this blog post: https://developers.googleblog.com/2020/08/guidance-for-our-effort-to-block-less-secure-browser-and-apps.html Apparently Google is restricting usage of Google sign-in in non-standard browsers (which presumably includes Electron) starting in Jan. 2021, and warns developers not to modify their browser's user-agent (which all three of the possibilities I mention do). Use at your own risk! (they don't make clear what outcome will result, but for my own use, I'm opting to use the alternative shown below from now on)
As an alternative to using a Google sign-in popup in your app (which some might be wary of, since Electron apps could in principle insert code into the popup to read the raw password -- not that it matters that much, since Electron apps could just install keyloggers or the like anyway), you could instead open a tab in the user's regular external browser, pointed to a special page that triggers a sign-in popup there, and then just sends the credentials to your Electron app afterward.
Instructions can be seen here (approach 3): https://stackoverflow.com/a/64328193/2441655
After taking a wild guess I decided to alter the user agent string and to remove application name from it as well as Electron/ with version. After this alteration it started working again.
Example implementation:
const win = new BrowserWindow(params);
let ua = win.webContents.userAgent;
ua = ua.replace(/APPLICATION NAME HERE\/[0-9\.-]*/,'');
ua = ua.replace(/Electron\/*/,'');
win.webContents.userAgent = ua;
This assumes the application is using symver and no pre-release tags. Otherwise you would have to tweak the regexp a bit.
I'm trying to call my /auth/user endpoint to get the current user that's logged into my website. But because of the new Chrome update I need to somehow set 'sameSite' and 'secure'. Anyone know of how I can get around this? Am I doing something wrong with cookie-session?
The cookie gets sent by express just fine, but it doesn't come with sameSite and secure settings that I specify in the cookie-session settings (see image). I tried with express-session as well, but for some reason the sameSite and secure settings never propogate to cookie used for oauth.
Btw, the authentication works on localhost addresses, but when I deploy from frontend and backend with heroku, I encounter the issue where I need to set sameSite. Would setting up a proxy or something get around the sameSite issue?
I am working on the same issue. SameSite=None needs the cookie to be secure
https://www.chromestatus.com/feature/5633521622188032
Eventhough I have set "secure: true" I am still seeing the cookie not being created on Chrome, but I do see it on Edge
After 3 days trying to figure It out. I finally found a way around this issue, It's not a fix, I'm quite sure PassportJS will come with a solution for that eventually, but for now It allowed me to get the user from the authentication.
Since we are not being able to get the user from the cookie, but the information is in the server session, the way to get this information is to add to the 'server.js' a route to get the user directly from the server session:
app.get('/api/getUser', (req, res) => {
res.json(req.session.user);
});
For some reason, I suppose the lack of cookie somehow, using the req.session inside of a router is returning undefined, but If used inside 'server.js' (or your server index file) It gets the session.
If you need the req.user._id or some other sensitive information for other requests, I would recommend returning a jwtToken with this information to the frontend (in res.json), then save the token directly in localStorage and pass the token in the body of your requests, is not the ideal, but It's the safer way I could think to keep the ids safe.
I hope It can help you!
We have developped a SPA SaaS and went to a soft production launch recently.
Everything was fine until one of our customers told us they had trouble using the app.
Once they open the app, the first request to our backend triggers their proxy credential prompt. Hopefully on the login request.
They have to enter their proxy credentials to let the request go. All subsequent requests are passing properly and they can use the app.
The problem is:
When they stop using the app, close the browser and then come back the day after, the persistent login tries to connect them to our backend, but the proxy credentials prompt is not triggered and the request fails. All subsquent requests fail also.
For it work again, they have to delete all app data in chrome (so the service worker is unregistered, the localstorage and cache are cleared). The next api call will trigger their proxy credentials prompt and they will be able to work again.
So is there any way for the app to know if the proxy is set or not ? Any way of triggering the proxy prompt if not set or whatever ?
I don't exactly know how those proxies work and we have zero access to the proxy settings.
It surely is something with the credentials expiration after some time but that's all we can figure out right now. Maybe we could monitor some params in the request headers ?
We are using VueJS with axios for the requests.
My guess is when user session credentials get expired, your UI is not handling redirection to login page. When the user login for the first time you should store that the user has logged in successfully in browser localstorage. If your server returns 401 error code, you can delete the flag and redirect the user to login page. You can achieve that using meta fields in router.
Check out this link on how to use meta fields https://router.vuejs.org/guide/advanced/meta.html
On my computer, can I login to my web app, grab the cookie values, and have my selenium script use it to test the web app without having to login?
Is it also possible to modify the cookie expiration after logging in on my computer so that the cookie will won't expire when the selenium tests are running?
Is it possible to avoid having to write additional selenium logic to use the login form? If not, does it raise a cookie expired exception so that I know I need to re-login?
What about testing cookieless session webapps like Meteor.js?
Using .NET I know you can set the cookies, or at least retrieve them. Try placing your cookies there.
driver.Manage().Cookies = Your_Cookie_Object_Here;
I have a php/apache service and meteor on the same server. I am using the accounts-ui package.
Is there anyway to know in my php script, that a user is logged in, given the login token (session id?)
This is my original need: upload a profile picture for a logged in user.
Very simple right? But I have not found an answer after hours of googling.
First solution would be using html5 File apis to send data to meteor server and the server save the image. But this solution wont even work for IE9.
Second solution is what I am trying: Using a html form to upload picture to a php script (or whatever script, it can be a nodejs script if needed). This script will save the image like a traditional php script does. The thing is I cannot know if the upload request is authorized, otherwise everybody can change profile picture of anybody. I must add some information in the upload request and verify them in the php code before saving the image. I am thinking about sending a request from php script to meteor server but I need to know which parameters to send and how meteor responses it.
How can I achieve the second solution or if someone has a another solution for my origin problem that would be great.
Thank you.
Meteor uses an a protocol called DDP to communicate between the client and server. But as of now there isn't a PHP ddp client so you would have to use a REST type communication method between your meteor server and your PHP server.
If you feel you could build a PHP client for your meteor client, it would greatly help you as you could do stuff like run Meteor.call from your php scripts and have them subscribe to collections. The full DDP spec (pre1) can be found at : https://github.com/meteor/meteor/blob/master/packages/livedata/DDP.md
To do a REST method you should use Meteor Router to allow you to create server side routes. It is installed via meteorite which helps you access a list of community packages at [atmosphere.meteor.com].1
sever side js
Meteor.Router.add('/checklogin', 'post', function() {
var userId = this.params.userId;
var loginToken = this.params.loginToken;
if(userId && loginToken) {
return (!!Meteor.findOne({_id:userId,"services.resume.loginTokens.token":loginToken}));
}
});
You can then do a POST request with PHP to /checklogin with two params, one is userId which is the userId (found with Meteor.userId() or localStorage.getItem("Meteor.userId"). The other is the login token found via localStorage.getItem("Meteor.loginToken") on your Meteor client.