I am trying the Amazon Product Advertising API, following their tutorial here.
Before doing any coding, I am attempting to follow the tutorial and use the Signed Requests Helper form online to prove the concept. I enter in my Access Key ID and Secret Access Key, and then enter in the following in the Unsigned URL box:
_http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService
&Version=2010-11-01
&Operation=ItemSearch
&SearchIndex=Books
&Keywords=harry+potter
I copy and paste the contents of Signed URL box into my browser's address bar (with correct access key and signature):
_http://ecs.amazonaws.com/onca/xml?AWSAccessKeyId=[MY_ACCESS_KEY]&Keywords=harry%20potter&Operation=ItemSearch%0D&SearchIndex=Books%0D&Service=AWSECommerceService%0D&Timestamp=2011-05-30T07%3A56%3A17.000Z&Version=2010-11-01%0D&Signature=[MY_SIGNATURE]
In return I get a HTTP 500 Internal Server Error.
(Note I have inserted the unscore before both example hyperlinks as stackoverflow won't allow more than 2 hyperlinks for newbies.)
I've tried IE8, and Chrome, and have also tried the .co.uk domain as I reside in the UK. I have also looked at the AWS forum with no luck. Any help would be appreciated...
I have tried generating Signed URL with the page you linked to using my Access Key ID and Secret Access Key and I do get the correct response.
One thing I noticed is that your signed URL contains an extra character (%0D - newline character?) ending some of your request parameters values. Were these characters been added when you copied the sample URL in the unsigned URL input? Please try fixing the unsigned URL until these characters do not show up in the signed URL box, then try that signed URL again.
Related
This article in the AWS Developer Blog describes how to generate pre-signed urls for S3 files that will be encrypted on the server side: https://aws.amazon.com/blogs/developer/generating-amazon-s3-pre-signed-urls-with-sse-kms-part-2/ . The part that describes how to generate a url makes sense, but then the article goes on to describe how to use the url in a put request, and it says that, in addition to the generated url, one must add to the http request a header specifying the encryption algorithm. Why is this necessary when the encryption algorithm was included in the url's generation?
// Generate a pre-signed PUT URL for use with SSE-KMS
GeneratePresignedUrlRequest genreq = new GeneratePresignedUrlRequest(
myExistingBucket, myKey, HttpMethod.PUT)
.withSSEAlgorithm(SSEAlgorithm.KMS.getAlgorithm());
...
HttpPut putreq = new HttpPut(URI.create(puturl.toExternalForm()));
putreq.addHeader(new BasicHeader(Headers.SERVER_SIDE_ENCRYPTION,
SSEAlgorithm.KMS.getAlgorithm()));
I ask in part due to curiosity but also because the code that has to execute the put request in my case is running on a different machine from the one that generates the url. I won't go into the details, but it's a real hassle to make sure that the header that one machine generates matches the url that the other machine generates.
I don't know how "clear" the justification is, but my assumption is that the encryption parameters are required to be sent as headers in order to keep them from appearing in logs that log the query string.
Why is this necessary when the encryption algorithm was included in the url's generation
This aspect is easier to answer. A signed request is a way of proving to the system that someone in possession of your access-key-secret authorized this exact, specific request, down to the last byte. Change anything about the request that was included in the signature generation, and you have invalidated the signature, because now the request differs from what was authorized.
When S3 receives your request, it looks up your secret key and does exactly what your local code does... it signs the request it received and checks whether its generated signature matches the one you supplied.
A common misconception is that signed URLs are generated by the service, but they aren't. Signed URLs are generated entirely locally. The algorithm is not computationally-feasible to reverse-engineer, and for any given request, there is exactly 1 possible valid signature.
It looks like the information about encryption doesn't get included in the presigned url. I'm guessing the only reason it's included in the GeneratePresignedUrlRequest is for generating a hash that's checked for authentication. After reading up on the question of when to use url parameters vs custom headers, I have to wonder if there is any clear justification for S3's using custom headers instead of url parameters here. As mentioned in the original question, having to include these headers makes using this API difficult. I wouldn't have this problem if url parameters were used instead. Any comments on the matter would be appreciated.
So I've just been fighting shadows in the Amazon labyrinth. Went to use the Amazon Product Advertising API Scratchpad to test out my Access Key Id and Secret Access Key, as well as entering the Associate Tag which isn't used in critically in the API query, just important if you want credit for the query later.
I kept getting the following error, no matter how many times I went and created a new set of keys or verified them in the download section of the AWS Management Console for the root user - IAM users don't work in the API seemingly.
Error! SignatureDoesNotMatch
HTTP Status 403: Forbidden
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
Rinse and repeat. Many, many times. Complete and utter frustration....
Lo and behold, I come across this oasis of sanity - Signed Request Helper - which provides a successful result to my query with the same keys as used above.
So, can anyone else confirm problems with the Scratchpad that didn't bear out in other applications, like the Signed Request Helper or their own code? At this point I'm betting that there is a bug in the Amazon Scratchpad. I guess I will go roll something to test in Python but the apparent craziness of the URL construction makes me wary. Seemingly it's soo hard even the Amazon guys got it wrong...
Yes, this reminds me of the dark days I had trying to get the signature just right.
I too had similar troubles when I started playing with the API. Ultimately, I ended up using the master credentials. This link will take you to the right spot after you login. Open the "Access Keys" tab. This is the area I made a new master access key for signing requests. When you get into the users/groups/roles/policies, I had trouble.
But I have a key there and I can use the scratchpad no problem. I have an application running that uses the key, but I just went and tried a query to confirm it's all still good.
Note that in the scratchpad the Associate Tag is irrelevant like you said. To get the error you have, it's for sure the SECRET ACCESS KEY that is incorrect. If you entered the ACCESS KEY ID field incorrectly, you would get this error:
Error! InvalidClientTokenId HTTP Status 403: Forbidden The AWS Access
Key Id you provided does not exist in our records.
For others who may come across this, let me impart my findings. Importantly, I was able to verify the keys with the Signed Request Helper but wasn't able to get the query to work in the Scratchpad. The error received clearly informs me that it is the signature that is wrong:
Error! SignatureDoesNotMatch HTTP Status 403: Forbidden The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
So one ponders, as instructed, on the Secret Key and the signing method. Since one is using the scratchpad, it must be the key. But the same key works in the signed request helper! What to do.
It turns out that the error was in one of the supplied parameters. If the scratchpad produces a signed url with a faulty parameter, it will result in that error. There is some validation going on, but you can still wind up with a bad parameter. So, if you get the above error, try a simple query with your key to prove that your secret access key is valid, then start investigating your supplied parameter values.
I would like to know if there is an API I can use to get a LinkShare merchant domain URL.
The merchant search endpoint only returns their uid and name.
According to the Linkshare help docs you can get a list of default URLs for all advertisers in your program (by joining data from two APIs), but those will still be affiliate links on Linkshare redirect domains. You would then need to write a script to visit those links and return the final destination URL, then grab the root domain of the URL.
Steps:
Follow the instructions here: http://helpcenter.linkshare.com/publisher/questions.php?questionid=1030
Write a script in your language of choice to cURL the affiliate link and grab curl_getinfo(curl_init(), CURLINFO_EFFECTIVE_URL); to see where it's redirecting to.
Parse the response with a RegExp to grab just the root domain like http[s]?://([^/]+)
This is a duplicate of this Stack Overflow question this is what worked for me:
The following URL will allow you to get a list of merchants with data that you have permission to access without the need to login. It will return an XML list of your merchants:
http://findadvertisers.linksynergy.com/merchantsearch?token=YOUR_LINKSHARE_ACCESS_TOKEN
I am trying to add an api key from my bigcommerce store to my developer account. I go in and add a user that has access to api, and this generates a key, and I click save. I then registered as developer at developer.bigcommerce.com and tried to add the key, but it gave me an invalid credentials message. What am I doing wrong?
I tried recreating this and it seems to be working fine. Did you make sure you added the store url too for the new api key.
Look at the attached screenshot.
Assuming you've inputed the user name and key correctly, you also want to be sure the store url matches the API URL given when you created the new user BUT without the /api/v2 at the end.
So if your API URL happened to be https://store-bwvr466.mybigcommerce.com/api/v2
Just input https://store-bwvr466.mybigcommerce.com for the store url
In my application I allow users to connect their Facebook accounts via oauth for the purpose of posting via our interface. We support both page accounts and regular accounts that simply manage pages.
We also inspect the result of the opengraph API call to get a valid URL to their profile, or page. The primary endpoint we use is https://graph.facebook.com/me (with oauth credentials). For some page-only accounts, the returned object has a 'link' value that, when entered into a web browser, 404s.
The bad URLs I have seen fall into two distinct cases:
The URL can be of the form 'www.facebook.com/{page_id}' which 404s on some pages, but not others.
The URL can be of the form 'www.facebook.com/profile.php?id={user_id}' which more often than not 404s.
The only URL format I have seen that works for all accounts is www.facebook.com/profile.php?id={page_id}. In the first case, we detect that the 'link' field isn't of the proper form (using profile.php?id=...), and construct a URL with the proper structure, and it works.
My next heuristic I'm considering adding is to see if the URL is of the proper form....but uses the {user_id} as the id argument to profile.php, and just construct the URL using the {page_id}. Obviously, this is getting ridiculous.
So, is there a good way to know if an account will give back a link that is invalid? Is this a bug in the API? What is the most reliable way to, given a User on the open graph API, to get a working link to their profile/page?
Using 'www.facebook.com/{page_id}' or 'www.facebook.com/profile.php?id={user_id}' will always work - they are both the same. The only reason you'll see a 404 is if the Page has been unpublished / deleted or if the user has deactivated their account.