How to add custom header ( for example header 'accept-encoding: gzip' ) for downloadPage in Twisted ?
If you look at the API documentation for downloadPage, you'll notice that it accepts *args and **kwargs. It refers you to HTTPDownloader for documentation regarding those parameters.
If you look at the API documentation for HTTPDownloader, you'll see that it accepts a headers argument. This argument isn't documented, but it's what you're looking for. Its value should be a dict mapping header name to header value. So,
downloadPage(..., headers={'accept-encoding': 'gzip'})
should do what you want. Also, note the newer client API provided by Twisted, twisted.web.client.Agent. Since Twisted 11.1, this supports content encodings at a higher level, and has specific support for gzip.
Related
As described in the documentation, using the version_labels field, you can specify a label to a model version in order to handle canary deployments.
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/serving_config.md#assigning-string-labels-to-model-versions-to-simplify-canary-and-rollback
For example, you can have model 43 labeled as stable and model 44 labeled as canary.
That feature sounds really neat, but I did not find in the doc how to adapt my POST request to specify the label I want to use.
Until now, I was using something of the sort:
curl -d '{"instances": <<my input data>>}' -X POST http://localhost:8501/v1/models/<<my model name>>:predict
Any idea ?
Update:
Based on comments on this GitHub Issue, #misterpeddy states that, as of August 14th 2019:
Re: not being able to access the version using labels via HTTP - this is something that's not possible today (AFAIR) - only through the grpc interface can you declare labels :(
To the best of my knowledge, this feature is yet to be implemented.
Original Answer:
It looks like the current implementation of the HTTP API Handler expects the version to be numeric.
You can see the regular expression that attempts to parse the URL here.
prediction_api_regex_(
R"((?i)/v1/models/([^/:]+)(?:/versions/(\d+))?:(classify|regress|predict))")
The \d defines an expectation for a numeric version indicator rather than a text label.
I've opened a corresponding TensorFlow Serving issue here.
The REST API for TensorFlow Serving is defined here: https://www.tensorflow.org/tfx/serving/api_rest#url_4
For the predict method it would be:
http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]:predict
where ${MODEL_VERSION} would be stable or canary
While using Google Cloud HTTPS Load Balancer we hit the following bug. Couldn't find any information on it.
We have a custom http header in our request:
X-<Company name>-abcde. If we are working directly against the server all is good, but once we are working through the load balancer, than our custom header is missing. We didn't find any reference in the documentation that there is a need to white list our headers or something like that.
Why my custom header is not being transferred to my backend server while working through Google Cloud Load Balancer? And how to make it work?
Thanks
Data
After a lot of testing, these are the results I've come up with:
The Google Cloud HTTPS Load Balancer does transfer custom HTTP headers to the backend service.
However, it changes them to lower-case.
So, in your case, X-Custom-Header is transformed to x-custom-header.
Solution
Simply change your code to read the lower-case version of your custom HTTP header. This is a simple fix, but one which may not be supported in the long-term by Google (there's not a word on this in Google's documentation so it's subject to change with no notice).
Petition Google to change this idiosyncratic behaviour or at the very least mention it clearly in their documentation.
A little extra
As far as I know, the RFC 2047 which specified the X- prefix for custom HTTP headers and propagated the pseudo-standard of a capital letter for each word has been deprecated and replaced by RFC 6648 which recommends against the X- prefix in general and mentions nothing regarding the rest of the words in the custom HTTP header key name. If I were Google, I would change this behaviour to pass custom HTTP headers as is and let developers deal with the strings as they've set them.
The RFC (RFC 7230) for HTTP/1.1 Message Syntax and Routing says that header fields have a case-insensitive field name. If you're relying on case to match the header that doesn't align with the RFC.
Way back in the day I looked through either the Tomcat of Jetty source and they worked with everything as a .toLower().
Go has a CanonicalMIMEHeaderKey where it'll format the headers in a common way to be sure everything is on the same page.
Python still harkens back to the RFC822 (hg.python.org/cpython/file/2.7/Lib/rfc822.py#l211) days, but it forces a .lower() on headers to standardize.
Basically though what the GCP HTTP(S) Load Balancer is doing is acceptable as far as the RFC is concerned.
This is most likely an application bug.
As other answers have stated, HTTP header names are case insensitive. Ime, every time headers appear to be case sensitive, it is because there is a request wrapper somewhere in the application call stack.
Request wrappers like this are common (usually necessary) in Java servlet filters. It's a common, newbie mistake to use case-sensitive matching (e.g. a regular Java HashMap<String, T>()) for the header names in the wrapper.
That's where I would start looking for your bug.
A reasonable way to create a Java Map<String, T> that is both case insensitive and that doesn't modify the keys is to use new TreeMap<String, T>( String.CASE_INSENSITIVE_ORDER ).
Hey guys I am trying to call Docebo's REST APIs and I am finding it hard to understand the method for it. Basically, Calling an API requires you to place an X-Authorization Parameter in the request header. The Docebo documentation on implementing this is just a paragraph that is very confusing to read. A similar question has been asked and answered here :
Docebo - constructing authorisation header
I read the code but couldn't quite grasp the explanation as there was little and the code very difficult for me to understand. I have two questions-
1)What is the X-Authorization parameter?
2)How does one compute the X-Authorization parameter to add to the request header to make calls to the Docebo API?
Detailed explanation of how the code works would be great! Thanks in Advance!
Hey I finally figured it out.
What is the X-Authorization parameter?
It is the parameter that has to be added to the header of the request. This header is used to authenticate the call to an API and the server first checks for this parameter to know whether the call is coming from a trusted source. This kind of requests that have a custom X- headers are called pre-flighted requests that require the sender to first send an HTTP OPTIONS request. The server responds with a list of allowed actions that can be performed. Only if the origin(of the sender) is allowed to have the specific header/Have access to the server resources, the request is actually executed.
How does one compute the X-Authorization parameter to add to the request header to make calls to the Docebo API?
It is as follows :
First take a look at the API documentation of Docebo for the specific API you want to call. It will have a list of parameters that the call requires. Then, you need to have the API keys from docebo handy since both of them are used in generating this X-Authorization parameter.Then proceed as follows :
1)Suppose you have n parameters that the call needs.Do the following :
SHA1 encoding of the following string between the brackets - (param-1,param-2,param-3.....param-n,secretKey).Don't forget the commas! Take the SHA1 hash generated in this step and proceed to step 2
2)A UTF-8 base64 encoding of the following string between the brackets-
(PublicKey:hash from step 1).Again, don't forget the colon! and you will obtain an alphanumeric string.
3) The X-Authorization parameter is - Docebo code(notice the space between Docebo and the code).
4)Add the parameter named X-Authorization to the request header before sending it and you will receive the response.
Hope this helps..
This is related to:
How should I implement a COUNT verb in my RESTful web service? , Paging in a Rest Collection
and Using the HTTP Range Header with a range specifier other than bytes?
Actually I think the -1 rated anwser here is correct https://stackoverflow.com/a/1434701/1237617
Generally anwsers say that you can use custom units citing the sec 3.12
range-unit = bytes-unit | other-range-unit
bytes-unit = "bytes"
other-range-unit = token
However when you read the HTTP spec please notice the production rules are thus:
Content-Range = "Content-Range" ":" content-range-spec
content-range-spec = byte-content-range-spec
byte-content-range-spec = bytes-unit SP
byte-range-resp-spec "/"
( instance-length | "*" )
The header spec only references bytes-unit from sec 3.12, not range-units, so I think that actually it's against the spec to use custom units here.
Am I missing something or is the popular anwser wrong?
EDIT: Since this probbably isn't clear, the gist of my question is:
rfc2616 sec14.16 only references bytes-unit. It never mentions range-unit, so range-unit production is not relevant for Content-Range, and thus only byte-units can be used.
I think this adresses my concerns best, although I needed some time to understand it (plus I wanted to make sure, that there is something wrong with the wording).
This reflects the fact that, apparently, the first set of grammar rules has been specifically made for parsing and the second one for producing HTTP requests
thanks to elgaton
The spec, as being revised, allows custom range units. See HTTPbis Part 5, Section 2.
If you read the HTTP/1.1 RFC, section 3.12, you will see that:
The only range unit defined by HTTP/1.1 is "bytes". HTTP/1.1 implementations MAY ignore ranges specified using other units.
So, the other-range-unit token has been introduced only to make servers more "liberal" when accepting. This reflects the fact that, apparently, the first set of grammar rules has been specifically made for parsing and the second one for producing HTTP requests, so that servers could accept even invalid requests (they will be simply ignored) and clients would use only the universally-accepted bytes unit.
Therefore, I personally recommend to:
use only the bytes unit when acting as a client, and
accept other units (discarding the Content-Range header if they are invalid) when acting as a server.
This is a purely personal opinion, but I think it is fairly consistent with how other HTTP extensions (custom methods or headers) are used. Here is how I read it: Yes I can use custom range units and no, I shouldn't submit a bug report when it gets ignored when passing through firewalls, web proxies, and other intermediaries. I conform to the HTTP spec when I'm sending it and they conform to HTTP when they ignore it. WebDAV uses HTTP extensions correctly, IMO, but rarely works over the Internet for exactly this reason. As I said, a personal opinion only.
Apparently it's OK to use custom units, because:
This reflects the fact that, apparently, the first set of grammar
rules has been specifically made for parsing and the second one for
producing HTTP requests
What are the valid values for http header pragma . I know no-cache is one but i wnat to enable caching so what should i set it. I did some googleing and all that i got was most clients ignore this but no info on other values it accepts.
Surprisingly there is only one parameter defined by default, which is no-cache and no new Pragma directives will be defined in HTTP as per RFC.
ref: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.32
Moreover, you will need to use the Cache-Control header for managing the caching behaviors rather than the Pragma directive which seems to be still included only to support the legacy HTTP/1.0.
ref: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9
Bonus: http://www.mnot.net/cache_docs/
You're probably looking for Cache-Control, this is supported in HTTP/1.1 and defines more states than Pragma.
Some more information, that might help some people that are less interested in caching, and more interested in http headers in general. i.e the literal interpretation of the original question, "what are the valid values for the http header pragma"?
The reference in the accepted answer (https://stackoverflow.com/a/7376516/3246928) is the RFC http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.32. It defines the snytax as:
Pragma = "Pragma" ":" 1#pragma-directive
pragma-directive = "no-cache" | extension-pragma
extension-pragma = token [ "=" ( token | quoted-string ) ]
This implies that any 'token=value' pair is acceptable (with the value being optional). The spec goes on to say
No new Pragma directives will be defined in HTTP.
and I would guess this is also meant to cover the "extension-pragma" part, but I wish they had been more unambiguous here.
This header does not seem to be specifically created for caching; the description in the RFC says:
The Pragma general-header field is used to include implementation-
specific directives that might apply to any recipient along the
request/response chain
So, in theory, you could add things here, and they could work. However, despite much searching, I have not found any reference to any other values ever being used here. It is effectively a dead and embarrassing part of http/1.
It seems like the normal thing to do is:
Only use pragma with the no-cache flag. This is the only value anyone should ever use. (And of course you should also use the cache-control header for your caching to behave as expected).
If you want to put some special information into a http header - i.e. If you want to "include implementation-specific directives that might apply to any recipient along the request/response chain", then create a custom http header. Google and Amazon, for example, do this:
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html and
https://cloud.google.com/storage/docs/reference-headers
Note the naming convention on the http header. The "x-" prefix is deprecated
by https://www.rfc-editor.org/rfc/rfc6648, but everyone seems to use it
anyway.