411 Length required error with App Proxies - shopify

I set up an app proxy for my app deployed on a normal shared hosted server (not Heroku or anything like that). It works like a charm (as do my other apps) until I set the content type to application/liquid.
As soon as I do that I get a 411 Length Required error by nginx which is generated by my server (my guess). I tried to resolve it by setting content length to 0. It worked for a while but then it stopped. I tried other values and it works depending on its mood. Funnily, sometimes the output is truncated at content length, and at times I get the whole output (a simple page refresh can give different outputs). Also, sometimes it doesn't work AT ALL and shopify throws a "we're having tech. difficulties" error.
To summarize, content length is not reliable at all.
Now I am not sure exactly what causes a 411 error and what can I do about it. And why is it thrown only when content type is liquid. Moreover, content-length doesn't result in a consistent output (no output/predictable output/truncated output/shopify error).
Anyone knows what's up?

Perhaps your responses are using chunked transfer encoding. I don't think nginx supports this by default, so would return a 411 error in this case because chunked encoding doesn't use a Content-Length header.
If you do want to use chunked responses, there is the http://wiki.nginx.org/HttpChunkinModule module that should add support for this. Otherwise, disable the chunked encoding in your app, and make sure the Content-Length header is consistent with the length of the body of the response.

Related

Not able to use string over 260 characters as a segment of URL in .NET Core

I'm making a request that works great and acts as supposed to. The actual authorization is provided using headers and working as expected too. This is the URL of it.
https://localhost:44385/api/security/check
By coincidence, I happened to replace the verbatim string check with the actual token, so the URL changed to
https://localhost:44385/api/security/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ...
All in all, the token happens to be 475 characters long. Then, when executing that call, I get the error message as follows.
Error: connect ECONNREFUSED 127.0.0.1:44300
I don't understand the issue and the status code 400 tells me only that the request is bad. Is it purely due ot the length of the URL? It seems like a bit too short (although there is a limitation for that but we're talking about a few thousands characters)...
The signature of the receiving method in the controller looks like this. It resides in the controller with path Security.
[HttpHead("{check}"), Authorize]
public IActionResult IsAuthorized(string check) { ... }
I also tried GET instead of HEAD with the same result. It's difficult to learn more about the error based on 400 Bad request only. It's a bit like something went wrong somewhere kind of error.
After some experimenting, I can confirm that it's not the length of the URL as such but rather the length of the segment between slashes. The first request works, the other does too but the third doesn't. The xxx part is precisely 260 characters and **yyy* part is precisely 261.
https://localhost:44385/api/test/xxx
https://localhost:44385/api/testtest/xxx
https://localhost:44385/api/test/yyy
What is this about?! It's like string in a method in my WebAPI can't be longer than 260 characters. Not 256, which at least would make some kind of sense...
Googling gave a veeery wide range of vastly spread hits and gave me nothing that I could relate to. Postman provides pretty much the same, limited information. The browser's network tab give even less.
A bit confused how to get to know more, how to diagnose it further and/or what to google for. Since it's a non-problem for the production environment, I can't bother my colleagues - the question is purely academic.
The limit you're hitting is UrlSegmentMaxLength (260).
This is all the way down in Http.Sys and only configurable in the
registry.
https://support.microsoft.com/en-us/help/820129/http-sys-registry-settings-for-windows
Workaround: break it up into multiple path segments, or move it to the
query or body. Or use Kestrel without IIS.
Resource: https://github.com/aspnet/AspNetCore/issues/2823#issuecomment-360921436
Here's a related post:
Setting UrlSegmentMaxLength from commadline

xbuf_frurl does not work properly without server header of content length?

I try to get some info from other sites with xbuf_frurl.
I got some site OK but some Not OK.
By Now, I still can not make sure what is going wrong.
But some sites are missing the content length header.
Who can tell whether xbuf_frurl() relies on the (potentially missing) content length header, esp. when growing the buffer?
xbuf_frurl() is indeed reading a body IF an HTTP content-length header is present. It will not try to decode chunked responses.
To deal with those servers using chunked replies, use the G-WAN curl.c example provided with the distribution. With libcurl you have even the opportunity to use SSL/TLS.
If that's not resolving your problem, the only way to troubleshoot this kind of issues is to give a non-working example, with both the full request that you have sent and the full reply received from the server.
That's why the xbuf_xcat("%v") format has been added: to give hexdumps, even with binary replies.
Edit your question and add this information to let people help you with a well-defined problem.

WebAPI hangs indefinitely when receiving a POST with incorrect Content-Length in header

I have a project set up using ASP.NET WebAPI on top of Azure, and am having a problem whenever I try to make an HTTP Post where the content-length is too long in the header.
Normally I would've just ignored this problem, because you should be correctly setting the content-length on POST, but it turns out that when this happens, it causes the session to hang indefinitely, and then the Azure emulator crashes.
I have a custom JSON Formatter which extends MediaTypeFormatter, and I set a breakpoint on the first line of my implementation of OnReadFromStreamAsync(). However, the breakpoint is never hit because the hangup happens before ever hitting the JSON Deserializer.
I really have no idea where this hanging is occurring from because I receive no exception, just an indefinite hang and occasional Azure emulator crash.
Thank you in advance for any help or insight you might provide!
This sounds like a bug. The good thing is that you can get updated developer bits form codeplex.
There is a chance what your experiencing is related to one of these:
WebAPI: Stream uploading under webhost is not working
DevDiv 388456 -- WebHost should not use Transfer-Encoding chunked when
content length is known.
Zero ContentLength without content type header in body is throwing
If the updated bits don't fix your problem I suggest you try the standard media formatters to rule in/out your formatter. Failing that, then submit an issue.

What does "Predicate Mismatch for View"

I am writing a iOS client for a an existing product that uses a legacy SOAP webservice. I got the proper URL to send my SOAP/XML messages too and even have some samples. However, none of them seem to work...
I always get a 404 error with the following error text "Predicate mismatch for View"
I am using an ASIFormDataRequest for the actual request and apending the data (SOAP XML in this case) via [someFormRequest appenData:myData].
I am flat out of ideas here and am wondering what, if anything I am doing wrong. Or should I ping one of the back end guys? Could this error be a result of something on the server side?
This is an error message spit out by the pyramid web framework when attempting to access a URL without supplying all of the required parameters. You definitely want to double check that the URL you are using has all of the required params (headers, query string options, request body, etc) and if you're convinced that what you are sending is correct then but your backend guys because it's definitely a miscommunication or a bug between the two of you.

Suggestions for best way of implementing HTTP upload resume

I'm working on a project which will allow large files (GB+) to be uploaded via HTTP PUT and I need to implement a method for resuming the upload. Once a file is uploaded and finalized it is complete and cannot be modified. So far, I have 2 options in mind but neither of which fit perfectly:
Option 1
Client sends an initial HEAD request on the file which will return either 404 if it does not exist or the file details including current size along with an HTTP X-Header along the lines of X-Can-Resume or something like that to specify whether the file can be resumed and a Range header specifying which bytes it has. This seems OK but I'm not keen on the X-Header as it removes from the HTTP standard.
Option 2
Client sends a PUT request with a Content-Length header of 0 bytes and no body, the server can then send back a 308 Resume Incomplete (as proposed here http://code.google.com/p/gears/wiki/ResumableHttpRequestsProposal) or a 202 Accepted header to indicate whether to resume or start from the beginning. This also seems acceptable apart from the use of a non-standard header.
Any other suggestions on the best way to implement this?
Thanks,
J
In either solutions there is no existing client and server implementations, so I'm guessing you'll code both. I think you should just find a right balance between the simplest and what is described in the gears proposal (by the way, you probably know Gears is dead), and be prepared to change when a standard emerges.
If I were to implement this feature, I'd make it possible for the client to upload in chunks and I would add a message digest on the whole content and the chunks.