I’ve been working on a simple function with the serverless framework that gets some data in a http POST, does some analysis and sends the results back. I got it working locally on my machine using serverless-offline but when it comes to deploying it I’m getting an error parsing the event.body.
Logging out the event.body it’s a string that looks like this:
----------------------------267253304929569989286258
Content-Disposition: form-data; name="text"
TEST
----------------------------267253304929569989286258--
so it makes sense that the parse is failing but I have no idea why this error it happening.
Any suggestions? I’ve tried a bunch of different things but am completely stumped.
Thanks in advance!
You can't JSON.parse that event.body cause it isn't JSON. It looks like whatever POSTed that data is using a multipart form POST style request rather than sending JSON. How are you invoking the HTTP POST?
I had the same issue and after a lot of debugging noticed 2 important things:
1.When the content type is application/x-www-form-urlencoded you might need to parse the data in a different way:
const qs = require('querystring');
module.exports.run = async event => {
try {
const data = qs.parse(event.body);
console.info('DATA:', data);
} catch(e) {
console.error(e.message);
}
}
2.When the Content-Type of the request is multipart/form-data the parsing will be even more complicated. I will suggest extra dependency to parse it like multiparty or any other of your choice
Thank you #Brian Winant! I am putting the answer here as a screenshot so it's clearer. In Postman, do the following:
AWS Lambda would return event.body as encoded query strings if the content-type is x-www-urlencoded. To have it return a JSON string you can then parse, send JSON data and set content-type as application/json.
Related
All welcome, I ran into a problem in which at the time of passing id and body in the method it is received, it gives 500 errors. In swagger I have an infected method, it looks like this:
swagger
Accordingly, in the body of the request, I pass all the parameters of the fields it needs (I drive them in strictly, just to check for operability):
code
In response I get this:
response
If you try to do the same in the swagger, then the method works and a 200 response comes:
swagger 200 response
What am I doing wrong?
I tried to make the code asynchronous, I tried to pass fields in different ways, but nothing happens, the answer comes 500
Try to put the id field inside the obj.
const form = {
id: 790,
...
}
You are passing too much params inside the callback (id, form).
We're automating our test with karate framework. In one of our features we need to decode a token and get a scope in the response. Everything works well, except this code in js.
function(token) {
return JSON.parse(new Buffer(token.split('.')[1],'base64').toString('ascii')).scope;
}
Error:
Caused by: <eval>:2 ReferenceError: "Buffer" is not defined
Caused by: jdk.nashorn.internal.runtime.ECMAException
In official tutorials it is said that javascript is 'native' to karate, so we don't understand why Buffer is not recognized? What we should do? Thanks for any help
I was able to successfully base64Decode a JWT token payload to JSON using the following code without the need for a Java method:
Background:
* def parseJwt =
"""
function(token) {
var base64Url = token.split('.')[1];
var base64Str = base64Url.replace(/-/g, '+').replace(/_/g, '/');
var Base64 = Java.type('java.util.Base64');
var decoded = Base64.getDecoder().decode(base64Str);
var String = Java.type('java.lang.String')
return new String(decoded)
};
"""
Scenario: JWT Token
Given path 'jwt/authenticate'
And header x-goog-authenticated-user-email = 'email'
And request {}
When method get
Then status 200
* json result = parseJwt(responseHeaders['Set-Cookie'][0])
* match result == {permissions: [1,2,3], iss: "us", exp: "#number", email: "email"}
Note: It does seem to be required to use json rather than def as Karate does better if it parses the string to json itself. Also, you may obtain the token from a header rather than a cookie as in this example if so, just change the responseHeader that you are looking for.
I'm pretty sure that Buffer is advanced / non-standard or NodeJS so it probably is not supported by the JVM JS engine (Nashorn).
Here's my recommendation. For this case, do the work using Java utilities.
For example, look at the Karate basic-auth example in the doc which uses Base64 encoding.
If it is really complex, simply create a Java static function, it will be much easier to test as a side-benefit. Hope this helps !
With respect to the post Karate: Problem with requests using bearer token | Karate is showing "org/apache/commons/codec/binary/Base64" in response instead of HTTP 401
My query was marked duplicate hence posting an answer/solution which we have found in our project.
We did some debugging in karate core about the error. It seemed a dependency was missing.
We added the dependency in the POM.xml
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.15</version>
</dependency>
Afterwards the problem was resolved and we started getting HTTP 401.
I am not sure it can be added to the karate core.
We're automating our test with karate framework. In one of our features we need to decode a token and get a scope in the response. Everything works well, except this code in js.
function(token) {
return JSON.parse(new Buffer(token.split('.')[1],'base64').toString('ascii')).scope;
}
Error:
Caused by: <eval>:2 ReferenceError: "Buffer" is not defined
Caused by: jdk.nashorn.internal.runtime.ECMAException
In official tutorials it is said that javascript is 'native' to karate, so we don't understand why Buffer is not recognized? What we should do? Thanks for any help
I was able to successfully base64Decode a JWT token payload to JSON using the following code without the need for a Java method:
Background:
* def parseJwt =
"""
function(token) {
var base64Url = token.split('.')[1];
var base64Str = base64Url.replace(/-/g, '+').replace(/_/g, '/');
var Base64 = Java.type('java.util.Base64');
var decoded = Base64.getDecoder().decode(base64Str);
var String = Java.type('java.lang.String')
return new String(decoded)
};
"""
Scenario: JWT Token
Given path 'jwt/authenticate'
And header x-goog-authenticated-user-email = 'email'
And request {}
When method get
Then status 200
* json result = parseJwt(responseHeaders['Set-Cookie'][0])
* match result == {permissions: [1,2,3], iss: "us", exp: "#number", email: "email"}
Note: It does seem to be required to use json rather than def as Karate does better if it parses the string to json itself. Also, you may obtain the token from a header rather than a cookie as in this example if so, just change the responseHeader that you are looking for.
I'm pretty sure that Buffer is advanced / non-standard or NodeJS so it probably is not supported by the JVM JS engine (Nashorn).
Here's my recommendation. For this case, do the work using Java utilities.
For example, look at the Karate basic-auth example in the doc which uses Base64 encoding.
If it is really complex, simply create a Java static function, it will be much easier to test as a side-benefit. Hope this helps !
With respect to the post Karate: Problem with requests using bearer token | Karate is showing "org/apache/commons/codec/binary/Base64" in response instead of HTTP 401
My query was marked duplicate hence posting an answer/solution which we have found in our project.
We did some debugging in karate core about the error. It seemed a dependency was missing.
We added the dependency in the POM.xml
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.15</version>
</dependency>
Afterwards the problem was resolved and we started getting HTTP 401.
I am not sure it can be added to the karate core.
I'm trying to write a react native app which will stream some tracks from Soundcloud. As a test, I've been playing with the API using python, and I'm able to make requests to resolve the url, pull the playlists/tracks, and everything else I need.
With that said, when making a request to the stream_url of any given track, I get a 401 error.
The current url in question is:
https://api.soundcloud.com/tracks/699691660/stream?client_id=PGBAyVqBYXvDBjeaz3kSsHAMnr1fndq1
I've tried it without the ?client_id..., I have tried replacing the ? with &, I've tried getting another client_id, I've tried it with allow_redirects as both true and false, but nothing seems to work. Any help would be greatly appreciated.
The streamable property of every track is True, so it shouldn't be a permissions issue.
Edit:
After doing a bit of research, I've found a semi-successful workaround. The /stream endpoint of the API is still not working, but if you change your destination endpoint to http://feeds.soundcloud.com/users/soundcloud:users:/sounds.rss, it'll give you an RSS feed that's (mostly) the same as what you'd get by using the tracks or playlists API endpoint.
The link contained therein can be streamed.
Okay, I think I have found a generalized solution that will work for most people. I wish it were easier, but it's the simplest thing I've found yet.
Use API to pull tracks from user. You can use linked_partitioning and the next_href property to gather everything because there's a maximum limit of 200 tracks per call.
Using the data pulled down in the JSON, you can use the permalink_url key to get the same thing you would type into the browser.
Make a request to the permalink_url and access the HTML. You'll need to do some parsing, but the url you'll want will be something to the effect of:
"https://api-v2.soundcloud.com/media/soundcloud:tracks:488625309/c0d9b93d-4a34-4ccf-8e16-7a87cfaa9f79/stream/progressive"
You could probably use a regex to parse this out simply.
Make a request to this url adding ?client_id=... and it'll give you YET ANOTHER url in its return json.
Using the url returned from the previous step, you can link directly to that in the browser, and it'll take you to your track content. I checked on VLC by inputting the link and it streams correctly.
Hopefully this helps some of you out with your developing.
Since I have the same problem, the answer from #Default motivated me to look for a solution. But I did not understand the workaround with the permalink_url in the steps 2 and 3. The easier solution could be:
Fetch for example user track likes using api-v2 endpoint like this:
https://api-v2.soundcloud.com/users/<user_id>/track_likes?client_id=<client_id>
In the response we can finde the needed URL like mentioned from #Default in his answer:
collection: [
{
track: {
media: {
transcodings:[
...
{
url: "https://api-v2.soundcloud.com/media/soundcloud:tracks:713339251/0ab1d60e-e417-4918-b10f-81d572b862dd/stream/progressive"
...
}
]
}
}
...
]
Make request to this URL with client_id as a query param and you get another URL with that you can stream/download the track
Note that the api-v2 is still not public and the request from your client probably will be blocked by CORS.
As mentioned by #user208685 the solution can be a bit simpler by using the SoundCloud API v2:
Obtain the track ID (e.g. using the public API at https://developers.soundcloud.com/docs)
Get JSON from https://api-v2.soundcloud.com/tracks/TRACK_ID?client_id=CLIENT_ID
From JSON parse MP3 progressive stream URL
From stream URL get MP3 file URL
Play media from MP3 file URL
Note: This link is only valid for a limited amount of time and can be regenerated by repeating steps 3. to 5.
Example in node (with node-fetch):
const clientId = 'YOUR_CLIENT_ID';
(async () => {
let response = await fetch(`https://api.soundcloud.com/resolve?url=https://soundcloud.com/d-o-lestrade/gabriel-ananda-maceo-plex-solitary-daze-original-mix&client_id=${clientId}`);
const track = await response.json();
const trackId = track.id;
response = await fetch(`https://api-v2.soundcloud.com/tracks/${trackId}?client_id=${clientId}`);
const trackV2 = await response.json();
const streamUrl = trackV2.media.transcodings.filter(
transcoding => transcoding.format.protocol === 'progressive'
)[0].url;
response = await fetch(`${streamUrl}?client_id=${clientId}`);
const stream = await response.json();
const mp3Url = stream.url;
console.log(mp3Url);
})();
For a similar solution in Python, check this GitHub issue: https://github.com/soundcloud/soundcloud-python/issues/87
I have a resource that looks something like this:
/users/{id}/summary?format={format}
When format is "xml" or "json" I respond with a user summary object that gets automagically encoded by WCF - fine so far. But when format equals "pdf", I want my response to consist of a trivial HTTP response body and a PDF file attachment.
How is this done? Hacking on WebOperationContext.Current.OutgoingResponse doesn't seem to work, and wouldn't be the right thing even if it did. Including the bits of the file in a CDATA section or something in the response isn't safe. Should I create a subclass of Message, then provide a custom IDispatchMessageFormatter that responds with it? I went a short distance down that path but ultimately found the documentation opaque.
What's the right thing?
It turns out that what I need is WCF "raw" mode, as described here. Broadly speaking, I want to do this:
[OperationContract, WebGet(UriTemplate = "/users/{id}/summary?format={format}"]
public Stream GetUserSummary(string id, string format)
{
if(format == "pdf")
{
WebOperationContext.Current.OutgoingResponse.ContentType = "application/pdf";
return new MemoryStream(CreatePdfSummaryFileForUser(id));
}
else
{
// XML or JSON serialization. I can't figure out a way to not do this explicitly, but my use case involved custom formatters anyway so I don't care.
}
}
In theory you could do it with a multi-part content MIME type (see http://www.faqs.org/rfcs/rfc2387.html). However, it would be much easier to return a URL in the XML/JSON response and the client can do a GET on that link to return the file.