I am trying to switch over to the new bing search api which is now hosted on azure and as most of you know that the Appid parameter has been removed for authentication, I can't figure out an easy way to get the results of the query.
I am using http module for nodejs and the current code uses http.get to query the api.bing.net
.
Can someone show me the code to use the new one correctly?
I tried this, but the url fails to return anything
http://gavinmhackeling.com/blog/2012/05/using-the-bing-search-api-in-python/
Sorry, there was a typo in my blog post.
Make sure that you are using your default account key from https://datamarket.azure.com/account/keys.
To use the basic authentication, replace <YourDefaultAccountKey> in https://user:<YourDefaultAccountKey>#api.datamarket.azure.com/Bing/SearchWeb/Web?Query=%27leo%20fender%27&Market=%27en-US%27&$top=50&$format=JSON. You will be returned results in JSON.
You can also use https://datamarket.azure.com/dataset/explore/5BA839F1-12CE-4CCE-BF57-A49D98D29A44 to test query parameters and formatting.
You can use this module that encapsulates the requests, so you can use it like:
var Bing = require('node-bing-api')({ accKey: "your-account-key" });
Bing.web("stack overflow", function(error, res, body){
console.log(body);
},
{
top: 50,
market: 'en-US'
});
It works with the Azure version. You only have to replace your account key.
Related
I'm trying to write a react native app which will stream some tracks from Soundcloud. As a test, I've been playing with the API using python, and I'm able to make requests to resolve the url, pull the playlists/tracks, and everything else I need.
With that said, when making a request to the stream_url of any given track, I get a 401 error.
The current url in question is:
https://api.soundcloud.com/tracks/699691660/stream?client_id=PGBAyVqBYXvDBjeaz3kSsHAMnr1fndq1
I've tried it without the ?client_id..., I have tried replacing the ? with &, I've tried getting another client_id, I've tried it with allow_redirects as both true and false, but nothing seems to work. Any help would be greatly appreciated.
The streamable property of every track is True, so it shouldn't be a permissions issue.
Edit:
After doing a bit of research, I've found a semi-successful workaround. The /stream endpoint of the API is still not working, but if you change your destination endpoint to http://feeds.soundcloud.com/users/soundcloud:users:/sounds.rss, it'll give you an RSS feed that's (mostly) the same as what you'd get by using the tracks or playlists API endpoint.
The link contained therein can be streamed.
Okay, I think I have found a generalized solution that will work for most people. I wish it were easier, but it's the simplest thing I've found yet.
Use API to pull tracks from user. You can use linked_partitioning and the next_href property to gather everything because there's a maximum limit of 200 tracks per call.
Using the data pulled down in the JSON, you can use the permalink_url key to get the same thing you would type into the browser.
Make a request to the permalink_url and access the HTML. You'll need to do some parsing, but the url you'll want will be something to the effect of:
"https://api-v2.soundcloud.com/media/soundcloud:tracks:488625309/c0d9b93d-4a34-4ccf-8e16-7a87cfaa9f79/stream/progressive"
You could probably use a regex to parse this out simply.
Make a request to this url adding ?client_id=... and it'll give you YET ANOTHER url in its return json.
Using the url returned from the previous step, you can link directly to that in the browser, and it'll take you to your track content. I checked on VLC by inputting the link and it streams correctly.
Hopefully this helps some of you out with your developing.
Since I have the same problem, the answer from #Default motivated me to look for a solution. But I did not understand the workaround with the permalink_url in the steps 2 and 3. The easier solution could be:
Fetch for example user track likes using api-v2 endpoint like this:
https://api-v2.soundcloud.com/users/<user_id>/track_likes?client_id=<client_id>
In the response we can finde the needed URL like mentioned from #Default in his answer:
collection: [
{
track: {
media: {
transcodings:[
...
{
url: "https://api-v2.soundcloud.com/media/soundcloud:tracks:713339251/0ab1d60e-e417-4918-b10f-81d572b862dd/stream/progressive"
...
}
]
}
}
...
]
Make request to this URL with client_id as a query param and you get another URL with that you can stream/download the track
Note that the api-v2 is still not public and the request from your client probably will be blocked by CORS.
As mentioned by #user208685 the solution can be a bit simpler by using the SoundCloud API v2:
Obtain the track ID (e.g. using the public API at https://developers.soundcloud.com/docs)
Get JSON from https://api-v2.soundcloud.com/tracks/TRACK_ID?client_id=CLIENT_ID
From JSON parse MP3 progressive stream URL
From stream URL get MP3 file URL
Play media from MP3 file URL
Note: This link is only valid for a limited amount of time and can be regenerated by repeating steps 3. to 5.
Example in node (with node-fetch):
const clientId = 'YOUR_CLIENT_ID';
(async () => {
let response = await fetch(`https://api.soundcloud.com/resolve?url=https://soundcloud.com/d-o-lestrade/gabriel-ananda-maceo-plex-solitary-daze-original-mix&client_id=${clientId}`);
const track = await response.json();
const trackId = track.id;
response = await fetch(`https://api-v2.soundcloud.com/tracks/${trackId}?client_id=${clientId}`);
const trackV2 = await response.json();
const streamUrl = trackV2.media.transcodings.filter(
transcoding => transcoding.format.protocol === 'progressive'
)[0].url;
response = await fetch(`${streamUrl}?client_id=${clientId}`);
const stream = await response.json();
const mp3Url = stream.url;
console.log(mp3Url);
})();
For a similar solution in Python, check this GitHub issue: https://github.com/soundcloud/soundcloud-python/issues/87
https://developer.here.com/documentation/maps/dev_guide/topics/routing.html
{"error":"Unauthorized","error_description":"ApiKey invalid. ApiKey not found."}
I am using here maps api for routing and it's showing api key error key not found when i am passing key parameter in json format
// Instantiate a map and platform object:
var platform = new H.service.Platform({
'apikey': '{YOUR_APIKEY}'
});
The maps is working find when i pass the key as simple string i don't know if there is any error in documentation or at the api please update this on your site and check it once
the working code is below:
// Instantiate a map and platform object:
var platform = new H.service.Platform({
'apikey': 'YOUR_APIKEY'
});
just remove "{}" and it is working find
you should take your request and paste it on google search (the webpage search top bar). Remove any questionmark in the end, and you will see it will work, and will re-format the request. Copy the request from Google and try it on Postman now; hopefully it will work...
It worked for me.
I am trying to get JSON data from SportsRadar using an API request. My trial url is:
http://api.sportradar.us/nba/trial/v4/en/games/2018/03/03/schedule.json?api_key=4j9ge4a4rgsbq597f29p9rgb
When I copy this url into my google browser, the data I get back is as expected, but when I try to use/add the API request to my meteor project the API request does not return any data. As a test, in my client/main.js file I have added:
HTTP.call('GET',Meteor.absoluteUrl("http://api.sportradar.us/nba/trial/v4/en/games/2018/03/03/schedule.json?api_key=4j9ge4a4rgsbq597f29p9rgb"),
function(err,result){
console.log(result.data);
});
The console log result come back as null. Any guidance or thoughts will be appreciated - cfp
You need to call your callback function correctly. Try this;
HTTP.call('GET','http://api.sportradar.us/nba/trial/v4/en/games/2018/03/03/schedule.json?api_key=4j9ge4a4rgsbq597f29p9rgb'),
function(err,result){
if (result) {
console.log(result.data);
}
console.log(err);
});
Edit: The parameters of The HTTP.call() is corrected by removing Meteor.absoluteUrl()in the question upon Derrick's comment below.
You can also refer to the official documentation here.
I've read all the documentation I can find on migrating from Google OpenID 2 to OAuth 2/OpenIDConnect, and am currently using a nice class from phpclasses.org . This class seems to work quite well with both Google and Facebook (haven't yet tried other providers), but I'm having a problem with just one aspect of Google's migration path that is quite critical to me: obtaining the google user's old OpenID identifier in addition to the new OpenIDConnect 'sub' value for that user. I've got users registered in my database only through their old OpenID identifiers.
According to Step 3 in Google's Migration Guide it looks like all I should need to do is add a parameter "openid.realm=http://www.example.com" to the authentication request sent to https://accounts.google.com/o/oauth2/auth.
I looked up in my old code what the realm was that I used for its OpenID registration process (it was 'http://' . $_SERVER['HTTP_HOST'];), and then I made sure that the redirect urls in my application were compatible with that realm.
I added that value (url-encoded) as the value of an openid.realm parameter passed on the authentication request made within the class. But when the class exchanged the token for an access token, it got back the correct email, name, sub, etc, but there was no openid_id parameter present. BTW, my scope parameter is 'openid email profile'
Does anyone have a suggestion for what else I should try, or what I can do to determine what the problem is? Does anyone have successful experience getting the openid_id parameter value in php code? I'd really rather not go the client-side route with their "Sign-in with Google" button, and according to the docs that really shouldn't be necessary (plus there's no particular reason to believe it would solve my problem if I did it).
Just discovered it's in the id_token returned along with the access_token when you exchange the authorization_code for the access_token.
In the Migration Document, Step 3 first two paragraphs:
When you send an OpenID Connect authentication request URI to Google
as described in Step 1, you include an openid.realm parameter. The
response that is sent to your redirect_uri includes an authorization
code that your application can use to retrieve an access token and an
ID token. (You can also retrieve an ID token directly from the OpenID
Connect authentication request by adding id_token to the response_type
parameter, potentially saving a back-end call to the token endpoint.)
The response from that token request includes the usual fields
(access_token, etc.), plus an openid_id field and the standard OpenID
Connect sub field. The fields you need in this context are openid_id
and sub:
This is confusing and misleading/wrong. What token request? The authentication request returns an authorization code that you can exchange for an access_token and an id_token. The parenthetical remark about adding id_token to the response_type doesn't help much, as the various ways I tried to do that resulted in an error. But in any event, the
"usual fields (access_token, etc.), plus an openid_id field..."
is wrong. The access_token never appears in the same list at the openid_id field. The access_token appears in a list with the id_token, and the openid_id field is encoded within the id_token!
For testing purposes, you can decode an id_token using https://www.googleapis.com/oauth2/v1/tokeninfo?id_token=<string>
In this documentation I couldn't find a useful description for how to decode an id_token, only caveats about their being sensitive, and how to validate them (though validation is not needed if obtained directly from a google endpoint as is the case here). I downloaded google's php client, and extracted code from it (src/Google/Auth/OAuth2.php and src/Google/Utils.php). And from that it's easy enough to figure out how to decode the id_token string: explode on ., base64_decode element 1, and json_decode that.
Update 2015-05-21: In reply to #Arthur's "answer", which would have been more appropriate as a comment on this answer. I would have commented on that answer myself, but comments aren't allowed to be very long and don't allow image uploads, plus I thought this extra info improves my answer...
Below is a screenshot from netbeans/xdebug, showing the array elements I get when decoding the id_token I get. Interesting that the intersection of the fields listed here with the fields listed by #Arthur is the null set. So I suspect that whatever #Arthur is decoding, it is not an id_token of the kind described here. I'm not familiar enough with this stuff even to guess what it is that's being decoded in that answer.
I'm afraid I don't have the time to dig through the library I use to extract the exact code path that produces the id_token I decoded to get this array using the simple algorithm I described. But I can tell you that the library I use is this: http://www.phpclasses.org/package/7700-PHP-Authorize-and-access-APIs-using-OAuth.html
Using it just as documented does not give you the id_token you need for this for two reasons:
The pre-configured server for Google with Oauth 2 doesn't handle the openid.realm parameter. To handle that, I added the following server definition to the oauth_configuration.json file:
"Google-OpenIdConnect":
{
"oauth_version": "2.0",
"dialog_url": "https://accounts.google.com/o/oauth2/auth?response_type=code&client_id={CLIENT_ID}&redirect_uri={REDIRECT_URI}&scope={SCOPE}&state={STATE}&openid.realm={REALM}",
"offline_dialog_url": "https://accounts.google.com/o/oauth2/auth?response_type=code&client_id={CLIENT_ID}&redirect_uri={REDIRECT_URI}&scope={SCOPE}&state={STATE}&access_type=offline&approval_prompt=force",
"access_token_url": "https://accounts.google.com/o/oauth2/token"
},
Just after the call to Initialize(), you need to add
$client->store_access_token_response = true;
Without that, the actual access_token response is not accessible (at least not the way I'm using the class). With those two changes in place, my exact code to get the openid_id using this class is as follows:
protected function jwt_decode($jwt) {
$segments = explode(".", $jwt);
if (count($segments) != 3) {
throw new Exception("Wrong number of segments in token: $jwt");
}
// Parse envelope.
$envelope = json_decode($this->urlSafeB64Decode($segments[0]), true);
if (!$envelope) {
throw new Exception("Can't parse token envelope: " . $segments[0]);
}
// Parse token
$json_body = $this->urlSafeB64Decode($segments[1]);
$payload = json_decode($json_body, true);
return $payload;
}
protected function getOpenid_id() {
require_once 'Phpclasses/Http/Class.php';
require_once 'Phpclasses/OauthClient/Class.php';
require 'Phpclasses/Google/private/keys.php';
$client = new oauth_client_class;
$client->configuration_file = $phpclasses_oauth_dir . '/oauth_configuration.json';
$client->server = 'Google-OpenIdConnect';
$client->redirect_uri = 'http://' . $_SERVER['HTTP_HOST'] . strtok($_SERVER['REQUEST_URI'], '?');
$client->client_id = $GOOGLE_APPID;
$client->client_secret = $GOOGLE_APPSECRET;
$client->scope = 'openid email';
$client->realm = $this->getRequest()->getScheme() . '://' . $this->getRequest()->getHttpHost();
$me = null;
if (($success = $client->Initialize())) {
// set *after* the call to Initialize
$client->store_access_token_response = true;
if (($success = $client->Process())) {
if (strlen($client->authorization_error)) {
$client->error = $client->authorization_error;
$success = false;
}
elseif (strlen($client->access_token)) {
$success = $client->CallAPI('https://www.googleapis.com/oauth2/v1/userinfo', 'GET', array(), array('FailOnAccessError' => true), $user);
$me = (array) $user;
if (!array_key_exists('id_token', $client->access_token_response)) {
throw new Exception('No id_token in \$client->access_token_response');
}
$openid_id = $this->jwt_decode($client->access_token_response['id_token']);
$me['openid_id'] = $openid_id;
}
}
$success = $client->Finalize($success);
}
if ($client->exit)
exit;
$client->ResetAccessToken();
if ($success) {
return $me;
}
// Code to handle failure...
}
Despite sootsnoot's (own) answer I still can't find the openid_id field anywhere. When decoding the id_token there are only "issuer", "issued_to", "audience", "user_id" , "expires_in" , "issued_at", "email" and "nonce" fields.
No "openid_id" field in sight..
Any ideas?
In response to sootsnoot's response :) And I apologize for not having enough reputation to comment, otherwise would have done so.
Am using an OpenID Connect library that takes endpoints from auto-config: https://accounts.google.com/.well-known/openid-configuration
So assume the endpoints are not the problem. Indeed it seems I was checking the wrong id_token. However, even when checking the correct one I still don't see the "openid_id" field. I now see everything you have, except that I have a "nonce" field instead of the "openid_id" field:
stdClass::__set_state(array( 'iss' => 'https://accounts.google.com', 'sub' => ****, 'azp' => ****, 'email' => ****, 'nonce' => ****, 'at_hash' => ****, 'email_verified' => true, 'aud' => ****, 'iat' => ****, 'exp' => 1432300788, ))
Must be doing something wrong, but what...
Final update:
Found the issue: was passing realm parameter as openid_realm=... instead of openid.realm=...
Oh do I feel stupid... :)
I have a spreadsheet on my Google Drive and I want to download a CSV from another website and put it into my spreadsheet. The problem is that I have to login to the website first, so I need to use some HTTP request to do that.
I have found this site and this. If either of these sites has the answer on it, then I clearly don't understand them enough to figure it out. Could someone help me figure this out? I feel that the second site is especially close to what I need, but I don't understand what it is doing.
To clarify again, I want to login with an HTTP request and then make a call to the same website with a different URL that is the call to get the CSV file.
I have done a lot of this in the past month so I should be able to help you, we are trying to emulate the browsers behaviour here so first you need to use chrome's developer tools(or something similar) and note down the exact things the browser does like the form values posted, the url that is called and so on. The following example shows the general techinique to be used:
The first step is to login to the website and get the session cookie:
var payload =
{
"user_session[email]" : "username",
"user_session[password]" : "password",
};// The actual values of the post variables (like user_session[email]) depends on the site so u need to get it either from the html of the login page or using the developer tools I mentioned.
var options =
{
"method" : "post",
"payload" : payload,
"followRedirects" : false
};
var login = UrlFetchApp.fetch("https://www.website.com/login" , options);
var sessionDetails = login.getAllHeaders()['Set-Cookie'];
We have logged into the website (In order to confirm just log the sessionDetails and match it with the cookies set by chrome). The next step is purely dependent on the website so I will give u a general example
var downloadPayload =
{
"__EVENTTARGET" : 'ctl00$ActionsPlaceHolder$exportDownloadLink1',
};// This is just an example it may or may not be needed, if needed u need to trace the values from the developer tools.
var downloadCsv = UrlFetchApp.fetch("https://www.website.com/",
{"headers" : {"Cookie" : sessionDetails},
"method" : "post",
"payload" : downloadPayload,
});
Logger.log(downloadCsv.getContentText())
The file should now be logged, you can then parse the csv using hte GAS inbuilt function and dump the data in the spreadsheet.
A few points to note:
I have assumed that all form post values are static and can be
hardcoded, in case this is not true then let me know I will give you
a function that can extract values from the html.
Some websites require the browser to send a token value(the value will be present in the html) along with the credentials. In this case you need to extract the values and then post it.