I have a (NodeJS) Lambda function that handles Twilio API calls. I'd like to add a simple, static web page to the repo and deploy it in the same function using Express.
I'm aware that S3 or Cloudfront are options for this task, but it is appealing to me to have a single repo and deploy mechanic for this small side project.
API Gateway is setup as a proxy and is correctly serving up HTML, JS and CSS assets. However, the images are not getting served up correctly.
Here's what I know for certain.
When running the Node/Express app locally, everything works correctly.
When running in AWS, everything except the images works correctly.
The "Binary Media Types" settings in API Gateway have been both 'image/jpeg' and '/'. I've re-deployed after each experiment. Neither setting works.
When inspecting the HTTP request in the browser, the content-type for the image is 'image/jpeg'. This implies that API Gateway is handling the Lambda pass-through correctly.
When I inspect the CloudWatch logs for API Gateway, there are two curious things,
The entry for 'Endpoint response headers' shows a content-type of 'application/json'. I think this is the Lambda response but I can't reproduce this in isolation with the Lambda code by itself.
The content-length is different between the 'Endpoint response headers' log entry and the 'Method response headers' log entry immediately after that. This implies some sort of transformation occurring.
(4387c38d-23a2-4bac-bade-224573109ac6) Endpoint response headers: {Date=Fri, 04 Nov 2022 13:49:24 GMT, Content-Type=application/json, Content-Length=279668, Connection=keep-alive, x-amzn-RequestId=b5a1e560-31b8-4530-9677-026125b09c74, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-63651863-4abde29e529f267550fe4627;sampled=0}
body transformation occurs
(4387c38d-23a2-4bac-bade-224573109ac6) Method response headers: {x-powered-by=Express, accept-ranges=bytes, cache-control=public, max-age=0, last-modified=Tue, 01 Nov 2022 14:19:17 GMT, etag=W/"18871-184338f8e88", content-type=image/jpeg, content-length=100465, date=Fri, 04 Nov 2022 13:49:24 GMT, connection=close, X-Amzn-Trace-Id=Root=1-63651863-4abde29e529f267550fe4627;Sampled=0}
In this debug example, I'm only fetching the image. There should never be a json response here.
Related
This is essentially a duplicate of Can not add account for custom Sonos service, but there's no accepted answer and I am not able to add a comment to ask if they ever resolved their issue.
I've inherited a project and am trying to add the development service. I've configured it via /customsd.htm, set the header policy to Session ID, have both secure and insecure endpoints working.
When I go to add the channel, I see the request for strings.xml. However, I never see any requests come in for getSessionId. Meanwhile the SONOS reports "Account Not Found. ***** server did not recognize your login information." I am able to make the request with SoapUI, and I get a valid response.
If it's worth mentioning, I am in SONOS' beta program and am on version 6.2, build 31926010 (Mac desktop app).
UPDATE:
While I'm not sure there's anything useful here, looking at logs at [deviceIP]:1400/support/aggregate, I see the following. Note that the redacted URL and IP do resolve. IP is for a loadbalancer, URL is behind it.
Feb 28 11:07:42 Sonos[84168] <![CDATA[<]]>Error<![CDATA[>]]>: (SCLib) dns(1): [redacted URL] -<![CDATA[>]]> [redacted IP]
Feb 28 11:07:42 Sonos[84168] <![CDATA[<]]>Error<![CDATA[>]]>: (SCLib) control_client(1): getSessionId failed, res = 1000, tvStart = 1456679262 s 250163 us, m_tvConnectDone = 1456679282 s 250162 us, m_tvDone = 1456679302 s 250162 us, tvNow = 1456679262 s 509982 us
Feb 28 11:07:42 Sonos[84168] <![CDATA[<]]>Error<![CDATA[>]]>: (SCLib) soap(1): - param username = [redacted username]
UPDATE #2:
I inspected the packets via Wireshark, and behavior of the production service and development one seem the same except that for the production service, the controller / my computer kicks off a POST request to the Sonos before the server hangs up. That process, outlined in red in the attached image, does not occur for the customsd service.
I also experimented with using the production service endpoints in the customsd configuration, but that request failed in the same manner. FWIW, all ssl_validation tests pass just fine, as do various content tests.
For want of a trailing slash...
I finally sorted this, and it was incredibly minor. In the customsd configuration, my endpoint urls did not have a trailing slash. Adding them made it all work.
I finally realized this when I decrypted the calls the controller made to the server for both the production service and my customsd service registered to the production endpoints. The call made by the customsd service was getting back the load balancer's 400 error. The only difference between the calls were the headers, specifically:
Production service: POST / HTTP/1.1
Dev service : POST HTTP/1.1
LB properly said the call was invalid. Adding the trailing slash made it all work.
For what it's worth, I'm pretty sure the URLs without trailing slashes worked both via curl and, I believe, via SoapUI. The Sonos controller requires them, however.
There is no authentication on server side so authentication should not be issue.
URL format: PUT
https://localhost/api/v1/protections?integrationKey=111&userKey=1111&group=111&category=foo
Payload:
{"action":"BLOCK"}
This is working fine in Postman.
In SOAP UI , I am giving input as under:
EndPoint: https://localhost
Resource: /api/v1/protections
Parameters:?integrationKey=111&userKey=1111&group=111&category=foo
in Media type, I am selecting "application/json"
and entering {"action": "BLOCK"} but getting "Wed Jan 20 16:25:27 PST 2016:DEBUG:Receiving response: HTTP/1.1 403 Forbidden
"
Is there any suggestion to get the output in SOAP UI.
Depending on the server where the rest is exposed service generates an HTTP 403, you should verify that server is and thus find the fastest response.
Also try making a GET request from the browser to see if you can answer correctly because problem lock your machine to the server.
As is https, it may be that you lack some certificate set SOAPUI. possibly Postman you use already has configured. Try to check this setting.
In my case, I missed the Header "User-Agent" and "accept". I put in Soap UI and Works.
In Postman, this headers it put automatically.
I am calling json api built using Symfony2 using LAMP stack.
URL for api is like
http://ab.ab.com/new/358342/17/12.948468/77.718571
Response I get back correctly:
{"Result":{"statusCode":1,"statusMsg":"Created Successfully"}}
However http headers contain the information I am logging in server. Example is
-wf-1-1-1-3:185|[{"Type":"INFO","File":"","Line":"","Label":"app"},"Data From Publisher Device Id:358342045834581 Route Id: 17 Lat:12.948468 Lng77.718571 Timestamp:2014-09-18 13:23:20 Data:tstsst|S-1"]|
Above logging info is coming back in Http header of response.
How should I disable the server logging info coming back.
Finally I figured it out.
I was using firephp handler and firephp adds the logging information back in response through http headers.
firephp is useful tool for debugging json apis, however should be disabled in production to avoid revealing sensitive information.
I switched to fingers_crossed handler.
Using WSO2 API Manager 1.3.1. Trying to use the API Manager to proxy to a REST service. I have set up the service in API Mgr and can successfully post and get responses, typically json, though some are text.
However, when I try to GET a resource that returns binary content (a zip "file", content-type:application/octet-stream), the API Manager does not seem to respond and I can see an error in the console window (i'm running wso2server.bat in console):
[2013-07-03 11:52:05,048] WARN - SourceHandler Connection time out
while writing the response: 173.21.1.22:1268->173.21.1.22:8280
I have an HTTPModule on my internal service and it seems to be responding with the appropriate content (I can see the GET and response data logged). I can also call to the internal service directly and get a response, so that end of things seems OK. But going through the API Manager seems to fail.
I found information on enabling other content-types:
WSO2 API Manager - Publishing API with non-XML response
http://wso2.com/library/articles/binary-relay-efficient-way-pass-both-xml-non-xml-content-through-apache-synapse
Using that information I tried to enable the application/octet-stream for messageFormatter and messageBuilder using the binary relay and it did not help (or seem to make a difference). I have even disabled all other content-types and use the binary relay for all content-types and it does not help.
Currently, I'm running with just the following in both axis2.xml and axis2_client.xml (in their appropriate sections):
<messageBuilder contentType=".*" class="org.wso2.carbon.relay.BinaryRelayBuilder"/
<messageFormatter contentType=".*" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
I still get my json and text responses, but WSO2 times out getting the zip content. I saw the JIRA referenced in axis2.xml about enabling the ".*" relay, but as the other requests seem to work, I'm not sure it's an issue for me. I did try adding
'format="rest"' to the API definition, but it seemed to break all operations even the ones that worked prior so I've pulled it back out.
Any ideas on what is happening or how to dig in and debug this will help. Thanks!
After working with this for much too long, it turns out that my WSO2 configuration was correct, using the Message Relay and BinaryRelayBuilder, etc. While my REST service could reply immediately, I was setting a HTTP header that I assume WSO2 does not like, because when i removed it WSO2 would reply at an expected rate (instantly).
I was setting the header:
Transfer-Encoding: binary
When I removed that header from my service reply, then WSO2 operated as expected. I don't know if that's a "bug" in WSO2 or if I was implementing incorrectly, but I do have what seems like a "workaround" by omitting that header from my service response.
I'm using Azure Blob Storage to store media files and providing access to these files using Shared Access Signatures; everything is working well in this regard.
However, I have a client application that needs to "resume" access to these files and does so using an HTTP RANGE header. When it makes a request like this, it is unhappy with the result it gets back from Azure.
I'm not sure how to view the details on the Azure side to see if the request failed, or if it just returned something the client didn't expect, and I have no debugging visibility into the client.
Here's what the incoming range header looks like:
RANGE: bytes=4258672-
From the Azure documentation I've read it appears to support RANGE headers, however I'm wondering if there is a conflict using RANGE and Shared Access Signatures together?
Update:
It appears that Azure may be returning an incorrect status code for RANGE requests, which is causing my client apps to reject the response. The documentation states that Azure will respond with an HTTP status code of 206 when responding to a RANGE request, however when I issue a RANGE request like this:
curl -I -H "User-Agent: Bonos" -r 500- "https://murfie.blob.core.windows.net/168464/1.mp3?st=2013-07-03T16%3A34%3A32.4832235Z&se=2013-07-03T17%3A34%3A32.4613735Z&sr=b&sp=r&sig=mJgQGW%2Fr3v8HN2%2BVV3Uady7J68nFqeHyzQb37HAhfuE%3D"
Azure returns the following:
HTTP/1.1 200 OK
Content-Length: 19988911
Content-Type: application/octet-stream Charset=UTF-8
Last-Modified: Fri, 07 Jun 2013 16:44:50 GMT
ETag: 0x8D031B57670B986
Server: Blob Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 77312761-65a9-42ef-90cd-ff718a80b231
Date: Wed, 03 Jul 2013 16:41:01 GMT
We got this straightened out.
As #BrentDaCodeMonkey mentioned, Azure returns the expected 206 response if you're using API version 2011-01-18 or better, but in our case we don't originate the request so we can't specify this using the request header.
However, some Microsoft friends tipped us of to the fact that you can set the API version globally for a storage account, but you need to use the REST API to do so (it's not something you can do in the management UI). This post explains how:
http://msdn.microsoft.com/en-us/library/windowsazure/hh452235.aspx
After setting the DefaultServiceVersion to 2011-01-18, we're now getting back the expected 206 status for RANGE requests.
For those who are struggling with the Azure Service API and the tricky Authorization, I recommend the this very simple C# snippet that does exactly the same in a very simpler way (at least for me).
var credentials = new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials("storagename", "storagekey");
var account = new Microsoft.WindowsAzure.Storage.CloudStorageAccount(credentials, true);
var client = account.CreateCloudBlobClient();
var properties = client.GetServiceProperties();
properties.DefaultServiceVersion = "2013-08-15";
client.SetServiceProperties(properties);
You'll need to add a nuget package WindowsAzure.Storage v9.3.3 (obsolete, but still works)
I reached out to some members of the product team and was given the following...
The 200 vs 206 is due to the presents of the "-I" flag in the curl command. This results in a HEAD request instead of a GET which is essentially as "get blob properties" call instead of a "get blob" which will cause the range header to be ignored. Also be sure to specify the version headers as "x-ms-version:2011-08-18" or later since the "startByte-" range format was only supported on that version of later.
For more information on range headers, see: http://msdn.microsoft.com/en-us/library/windowsazure/ee691967.aspx
Yes, it works. I've used SAS to stream video to mobile phones, which use Range headers.
Its easy to verify with a bit of code too.