I am trying to log in to a site using requests as follows:
s = requests.Session()
login_data = {"userName":"username", "password":"pass", "loginPath":"/d2l/login"}
resp = requests.post("https://d2l.pima.edu/d2l/login?login=1", login_data)
although I am getting a 200 response, when I say
print(resp.content)
b"<!DOCTYPE html><html><head><meta charset='utf-8' /><script>var hash = window.location.hash;if( hash ) hash = '%23' + hash.substring( 1 );window.location.replace('/d2l/login?sessionExpired=0&target=%2fd2l%2ferror%2f404%2flog%3ftargetUrl%3dhttp%253A%252F%252Fd2l.pima.edu%253A80%252Fd2l%252Flogin%253Flogin%253D1' + hash );</script><title></title></head><body></body></html>"
notice it says session expired.
What I've tried:
logging back out and in in the actual browser, no success.
http basic auth, no success.
I'm thinking maybe I need to authenticate myself to this site using cookies?
If so how do I determine which cookies to send it?
I tried figuring this out by saying
resp.cookies
Out[4]: <RequestsCookieJar[]>
shouldn't this be giving me names of cookies? I'm not sure what to do with such output.
Main Point: HOW DO I FIGURE OUT HOW TO AUTHENTICATE MYSLEF TO THIS WEBSITE?
Help is appreciated.
I would rather not use selenium.
From loading this page https://d2l.pima.edu/d2l/login and viewing its source, you'll notice the POST target path is /d2l/lp/auth/login/login.d2l. Try using that as your POST path. Your other fields look consistent with the form's expectations.
Note: with python requests if you create a session object use it to make your requests:
resp = s.post(<blah blah>, login_data)
The session will hold any cookies set by the login server, and you can continue to use the s object to make requests in the authenticated session.
using keystone LocalFile type to handle image uploads. similar to the Cloudinary autoCleanup option, I want to be able to delete the uploaded file itself, in addition to the corresponding mongo entry when deleting entries through the admin ui.
in this case, I want to delete an "Album", and it's corresponding album cover.
Album.schema.pre('remove', function(next){
var path = this._original.album_cover.path + "/" + this._original.album_cover.filename
fs.unlink(path, function () {
console.log('deleted');
})
I get "CSRF failure" when using the fs module. I thought all CSRF protection was handled internally with Keystone.
Anyone know of a better solution to this?
Took a 10 minute break and came back and it seems to be working now. I also found this, which seems to be the explanation.
"Moreover double check your session timeout. In my dev settings the session duration is set to 3 minutes. So, if I end up editing something for more than that time, Keystone will return a CSRF error on save because the new session (generate in the meantime) invalidates the old token."
https://github.com/keystonejs/keystone/issues/1330
I am getting the following exception whenever i try to fetch defects from rally:
com.google.gson.JsonSyntaxException:
com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 12
at com.google.gson.JsonParser.parse(JsonParser.java:65)
at com.google.gson.JsonParser.parse(JsonParser.java:45)
at com.rallydev.rest.response.Response.<init>(Response.java:25)
at com.rallydev.rest.response.QueryResponse.<init>(QueryResponse.java:16)
at com.rallydev.rest.RallyRestApi.query(RallyRestApi.java:168)
at Test.main(Test.java:86)
Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 12
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1505)
at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1386)
at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:531)
at com.google.gson.stream.JsonReader.peek(JsonReader.java:414)
at com.google.gson.JsonParser.parse(JsonParser.java:60)
... 5
What intrigues me most is the code works perfectly fine on few machines and throws the above exception on few.
code snippet :
RallyRestApi restApi =
new RallyRestApi(new URI("http://rally1.rallydev.com"),apiKey);
QueryRequest queryRequest = new QueryRequest("defects");
queryRequest.setFetch(new Fetch("Project","FormattedID","Release"));
QueryFilter filter1 = new QueryFilter("FormattedID", "=", defetctID);
QueryResponse queryResponse1 = restApi.query(queryRequest);
Try a curl command to read the same defect using the same apiKey (in zsessionid header) on the same machine from which your java code fails.
curl --header "ZSESSIONID: _abc123" "https://rally1.rallydev.com/slm/webservice/v2.0/defect/123456789"
At least you will know if this is specific to java or not. Yes, it is strange that it fails on some machines and works on others, but the timing of those tests is not obvious from your post, and I wonder if this has anything to do with the underlying user credentials. (A user gets disabled for a period of time after a number of unsuccessful attempts). I am not positive that this is the issue you experience but I have seen when expired password caused the exact same error. API Keys are tied to a user, so when a user's password is expired, or when a user is inactivated (disabled) the same permissions(or the lack of them) is reflected in the key. For example, a user did not know that the password was expired because in the Rally UI they used SSO authentiation, but in the code they used either username/password or APIKey since the toolkit does not support SSO at this point. A 401 error would be more helpful, but instead a malformed JSON is generated.
I've read all the documentation I can find on migrating from Google OpenID 2 to OAuth 2/OpenIDConnect, and am currently using a nice class from phpclasses.org . This class seems to work quite well with both Google and Facebook (haven't yet tried other providers), but I'm having a problem with just one aspect of Google's migration path that is quite critical to me: obtaining the google user's old OpenID identifier in addition to the new OpenIDConnect 'sub' value for that user. I've got users registered in my database only through their old OpenID identifiers.
According to Step 3 in Google's Migration Guide it looks like all I should need to do is add a parameter "openid.realm=http://www.example.com" to the authentication request sent to https://accounts.google.com/o/oauth2/auth.
I looked up in my old code what the realm was that I used for its OpenID registration process (it was 'http://' . $_SERVER['HTTP_HOST'];), and then I made sure that the redirect urls in my application were compatible with that realm.
I added that value (url-encoded) as the value of an openid.realm parameter passed on the authentication request made within the class. But when the class exchanged the token for an access token, it got back the correct email, name, sub, etc, but there was no openid_id parameter present. BTW, my scope parameter is 'openid email profile'
Does anyone have a suggestion for what else I should try, or what I can do to determine what the problem is? Does anyone have successful experience getting the openid_id parameter value in php code? I'd really rather not go the client-side route with their "Sign-in with Google" button, and according to the docs that really shouldn't be necessary (plus there's no particular reason to believe it would solve my problem if I did it).
Just discovered it's in the id_token returned along with the access_token when you exchange the authorization_code for the access_token.
In the Migration Document, Step 3 first two paragraphs:
When you send an OpenID Connect authentication request URI to Google
as described in Step 1, you include an openid.realm parameter. The
response that is sent to your redirect_uri includes an authorization
code that your application can use to retrieve an access token and an
ID token. (You can also retrieve an ID token directly from the OpenID
Connect authentication request by adding id_token to the response_type
parameter, potentially saving a back-end call to the token endpoint.)
The response from that token request includes the usual fields
(access_token, etc.), plus an openid_id field and the standard OpenID
Connect sub field. The fields you need in this context are openid_id
and sub:
This is confusing and misleading/wrong. What token request? The authentication request returns an authorization code that you can exchange for an access_token and an id_token. The parenthetical remark about adding id_token to the response_type doesn't help much, as the various ways I tried to do that resulted in an error. But in any event, the
"usual fields (access_token, etc.), plus an openid_id field..."
is wrong. The access_token never appears in the same list at the openid_id field. The access_token appears in a list with the id_token, and the openid_id field is encoded within the id_token!
For testing purposes, you can decode an id_token using https://www.googleapis.com/oauth2/v1/tokeninfo?id_token=<string>
In this documentation I couldn't find a useful description for how to decode an id_token, only caveats about their being sensitive, and how to validate them (though validation is not needed if obtained directly from a google endpoint as is the case here). I downloaded google's php client, and extracted code from it (src/Google/Auth/OAuth2.php and src/Google/Utils.php). And from that it's easy enough to figure out how to decode the id_token string: explode on ., base64_decode element 1, and json_decode that.
Update 2015-05-21: In reply to #Arthur's "answer", which would have been more appropriate as a comment on this answer. I would have commented on that answer myself, but comments aren't allowed to be very long and don't allow image uploads, plus I thought this extra info improves my answer...
Below is a screenshot from netbeans/xdebug, showing the array elements I get when decoding the id_token I get. Interesting that the intersection of the fields listed here with the fields listed by #Arthur is the null set. So I suspect that whatever #Arthur is decoding, it is not an id_token of the kind described here. I'm not familiar enough with this stuff even to guess what it is that's being decoded in that answer.
I'm afraid I don't have the time to dig through the library I use to extract the exact code path that produces the id_token I decoded to get this array using the simple algorithm I described. But I can tell you that the library I use is this: http://www.phpclasses.org/package/7700-PHP-Authorize-and-access-APIs-using-OAuth.html
Using it just as documented does not give you the id_token you need for this for two reasons:
The pre-configured server for Google with Oauth 2 doesn't handle the openid.realm parameter. To handle that, I added the following server definition to the oauth_configuration.json file:
"Google-OpenIdConnect":
{
"oauth_version": "2.0",
"dialog_url": "https://accounts.google.com/o/oauth2/auth?response_type=code&client_id={CLIENT_ID}&redirect_uri={REDIRECT_URI}&scope={SCOPE}&state={STATE}&openid.realm={REALM}",
"offline_dialog_url": "https://accounts.google.com/o/oauth2/auth?response_type=code&client_id={CLIENT_ID}&redirect_uri={REDIRECT_URI}&scope={SCOPE}&state={STATE}&access_type=offline&approval_prompt=force",
"access_token_url": "https://accounts.google.com/o/oauth2/token"
},
Just after the call to Initialize(), you need to add
$client->store_access_token_response = true;
Without that, the actual access_token response is not accessible (at least not the way I'm using the class). With those two changes in place, my exact code to get the openid_id using this class is as follows:
protected function jwt_decode($jwt) {
$segments = explode(".", $jwt);
if (count($segments) != 3) {
throw new Exception("Wrong number of segments in token: $jwt");
}
// Parse envelope.
$envelope = json_decode($this->urlSafeB64Decode($segments[0]), true);
if (!$envelope) {
throw new Exception("Can't parse token envelope: " . $segments[0]);
}
// Parse token
$json_body = $this->urlSafeB64Decode($segments[1]);
$payload = json_decode($json_body, true);
return $payload;
}
protected function getOpenid_id() {
require_once 'Phpclasses/Http/Class.php';
require_once 'Phpclasses/OauthClient/Class.php';
require 'Phpclasses/Google/private/keys.php';
$client = new oauth_client_class;
$client->configuration_file = $phpclasses_oauth_dir . '/oauth_configuration.json';
$client->server = 'Google-OpenIdConnect';
$client->redirect_uri = 'http://' . $_SERVER['HTTP_HOST'] . strtok($_SERVER['REQUEST_URI'], '?');
$client->client_id = $GOOGLE_APPID;
$client->client_secret = $GOOGLE_APPSECRET;
$client->scope = 'openid email';
$client->realm = $this->getRequest()->getScheme() . '://' . $this->getRequest()->getHttpHost();
$me = null;
if (($success = $client->Initialize())) {
// set *after* the call to Initialize
$client->store_access_token_response = true;
if (($success = $client->Process())) {
if (strlen($client->authorization_error)) {
$client->error = $client->authorization_error;
$success = false;
}
elseif (strlen($client->access_token)) {
$success = $client->CallAPI('https://www.googleapis.com/oauth2/v1/userinfo', 'GET', array(), array('FailOnAccessError' => true), $user);
$me = (array) $user;
if (!array_key_exists('id_token', $client->access_token_response)) {
throw new Exception('No id_token in \$client->access_token_response');
}
$openid_id = $this->jwt_decode($client->access_token_response['id_token']);
$me['openid_id'] = $openid_id;
}
}
$success = $client->Finalize($success);
}
if ($client->exit)
exit;
$client->ResetAccessToken();
if ($success) {
return $me;
}
// Code to handle failure...
}
Despite sootsnoot's (own) answer I still can't find the openid_id field anywhere. When decoding the id_token there are only "issuer", "issued_to", "audience", "user_id" , "expires_in" , "issued_at", "email" and "nonce" fields.
No "openid_id" field in sight..
Any ideas?
In response to sootsnoot's response :) And I apologize for not having enough reputation to comment, otherwise would have done so.
Am using an OpenID Connect library that takes endpoints from auto-config: https://accounts.google.com/.well-known/openid-configuration
So assume the endpoints are not the problem. Indeed it seems I was checking the wrong id_token. However, even when checking the correct one I still don't see the "openid_id" field. I now see everything you have, except that I have a "nonce" field instead of the "openid_id" field:
stdClass::__set_state(array( 'iss' => 'https://accounts.google.com', 'sub' => ****, 'azp' => ****, 'email' => ****, 'nonce' => ****, 'at_hash' => ****, 'email_verified' => true, 'aud' => ****, 'iat' => ****, 'exp' => 1432300788, ))
Must be doing something wrong, but what...
Final update:
Found the issue: was passing realm parameter as openid_realm=... instead of openid.realm=...
Oh do I feel stupid... :)
I 'm a newbie to twitter api/twitterizer. I get the following exception for the code shown below. The error occurs when i request access token. Any suggestion on how to fix this??
I downloaded the latest dll(2.3.1) from the twitterizer website.
{"Value cannot be null. Parameter name: String"}
Dim OAuthTokens As New OAuthTokens
Dim accessToken As New Twitterizer.OAuthTokenResponse
accessToken = OAuthUtility.GetAccessToken(ConsumerKey, ConsumerSecretkey, "oauth_token", "oauth_verifier")
thanks
As I said on the forums, the oauth_token and oauth_verifier are given to you in previous steps of the authorization process. Calling the GetAccessToken method is the last step in the process.
Here's an over-simplification of the steps:
1. Get a request token
2. Send/Redirect the user to the authorization/authentication url
3. (The user logs in, grants you access, and Twitter directs them back to your application)
4. Exchange the request token for an access token