How to handle multiple arguments with spaces in a passed sh var? - printf

I have a posix sh script that runs another command. The second command can have zero, one or multiple options passed. These options are passed as an env var, e.g. $OPTIONS. However, some of the options can have spaces in them.
E.g. If my executed command is curl, then my option is CURL_OPTIONS.
#!/bin/sh
# do some processing
curl $CURL_OPTIONS https://some.example.com
As long as none of the CURL_OPTIONS contains whitespace, I am fine, e.g.
CURL_OPTIONS="--user me:pass --user-agent foo"
However, if the options contain space, the sh expands it out and treats each like its own var:
CURL_OPTIONS="--header 'Authorization: token abcdefg'"
I can run curl --header 'Authorization: token abcdefg' https://some.url.com, since the sh will interpret the single quotes ' and pass Authorization: token abcdefg as a single parameter to curl, and all is fine.
However, because I am working with a var, the sh expands it out upon seeing $CURL_OPTIONS and does not interpret the single-quoted Authorization: token abcdefg as a single parameter. The result of the above is:
curl: (6) Could not resolve host: token
curl: (6) Could not resolve host: abcdefg'
(Note that it even treats the single quote as part of abcdefg'.)
Yes, I have seen this question and its answers , none of which seems to work in this case.
UPDATE:
Oh, now this is interesting https://stackoverflow.com/a/31485948/736714
Why didn't I think of xargs?

Actually, this works.
CURL_OPTIONS=(--header "Authorization: token abcdefg")
curl "${CURL_OPTIONS[#]}" www.example.com
How to pass quoted arguments from variable to bash script
UPDATE:
I'm guessing this next snippet should be POSIX-compliant (it's a bit of an escape-hell, though)
CURL_OPTIONS="--header \\\"Authorization:\ token\ abcdefg\\\""
eval set -- $CURL_OPTIONS
curl "$#" www.example.com

Related

How to update gitlab merge request with curl? Getting 401 all the time

Need to update labels in merge request during gitlab ci. Tried different variants:
curl -X PUT https://gitlab.com/api/v4/projects/{project_id}/merge_requests/134?access_token={token}&labels=merged
Tried passing token as header and as data.
Tried passing "personal_token", "access_token", "token" and etc.
Getting {"message":"401 Unauthorized"} all the time.
I think that you should use private_token as it said on documentation:
curl https://gitlab.example.com/api/v4/projects?private_token=<your_access_token>
or in the header, as you said:
curl --header "PRIVATE-TOKEN: xxx" "https://gitlab.com/api/v4/projects"
Have a look on the official page:
https://docs.gitlab.com/ee/api/README.html#personal-access-tokens
Example:
Getting information from merge request with iid 1(the actual labels are testlabel and testlabel3):
#curl -X GET https://gitlab.com/api/v4/projects/11209705/merge_requests/1?private_token={XXXXXX}
{"id":28761206,"iid":1,"project_id":1120970.......,"labels":["testlabel","testlabel3"],...}
Sending put with curl with "--data" and the parameter to update "labels" to "test":
# curl -X PUT https://gitlab.com/api/v4/projects/11209705/merge_requests/1?private_token={XXXXX} --data "labels=test"
Getting information from merge request with iid 1(the actual labels is "test"):
#curl -X GET https://gitlab.com/api/v4/projects/11209705/merge_requests/1?private_token={XXXXXX}
{"id":28761206,"iid":1,"project_id":1120970.......,"labels":["test"],...}

how to set the audio file sample rate for watson text to speech in node-red

watson text to speech accepts extra parameters such as the sampling (default is 44khz)
see https://www.ibm.com/watson/developercloud/text-to-speech/api/v1/#synthesize_audio for options such as this one
(...)
audio/l16;rate=rate
(You can optionally specify endianness=big-endian
or endianness=little-endian; the default is little endian.)
(...)
but I can't see a way to set these options in node red
In the Node-red, doesn't have the option, because you need to add in your cURL.
As you can see, you can use Accept or accept query parameter header to specify the audio format.
For example:
curl -X POST -u "{username}":"{password}"
--header "Content-Type: application/json"
--header "Accept: audio/l16;endianness=big-endian"
--data "{\"text\":\"Hello world\"}"
--output hello_world.wav
"https://stream.watsonplatform.net/text-to-speech/api/v1/synthesize?voice=en-US_AllisonVoice"
See the Official Text to Speech v1 API Explorer.
Please raise this as an improvement request on the github repo for the nodes, we can continue the discussion there. No promises, but I am open to accepting a pull request, if done right.

how to send correct curl command to webserver

So I got the data that is being sent to a specific server. Now I want to do the same using curl from my local machine to play around with specific repsonses from the server and learn more about curl as well.
Here is part of my data
POST /auth HTTP/1.1
platform: android
X-Auth-Token: <censored>
Content-Type: application/json; charset=utf-8
Host: api.blabla.com
Accept-Encoding: gzip
And the data that is being sent:
{"blabla_token": "sdsadsad", "blahblah_id": "23213", "locale": "us"}
Now when I try cURL in my dos shell, I try
curl --insecure -X POST https://api.blabla.com/auth --data '{"blabla_token": "sdsadsad", "blahblah_id": "23213", "locale": "us"}'
The response I get from cURL is this:
{"code":401,"error":"blablaTokenRequired"}
Even though I specified the token. So there are two possible scenarios because the token is correct:
It has something to do with the SSL thing? (I use --insecure because I get an SSL error otherwise)
Something about my command is not correct but I can't figure out what.
Can someone kindly help me out? I am trying everything I can without success
I am not sure if I understand your application specific right, but probably one thing you need to take into account:
man curl says:
-d, --data <data>
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when
a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the
server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. --data-raw is almost the same but does not have a special interpreta‐
tion of the # character. To post data purely binary, you should instead use the --data-binary option. To URL-
encode the value of a form field you may use --data-urlencode.
As I can't see in your example the necessity of sending data as HTML form input, probably your application expects just a "raw" POST body and then you have to try this:
curl --insecure -X POST https://api.blabla.com/auth --data--binary '{"blabla_token": "sdsadsad", "blahblah_id": "23213", "locale": "us"}'
PS and for sure this is error is not about using --insecure which just asks curl to neglect ssl verification
you forgot the headers and enabling compressed encoding (gzip), however, i believe you can't force curl to only support gzip encoding using the curl command line alone, you will have to use libcurl, this will make the request say "Accept-Encoding: gzip,deflate" on most systems, using --compressed .. if that's not acceptable to you, rewrite it using libcurl (where you can force it to say only "gzip", if you wish, via CURLOPT_ENCODING )
curl -X POST https://api.blabla.com/auth --data '{"blabla_token": "sdsadsad", "blahblah_id": "23213", "locale": "us"}' --header 'platform: android' --header 'X-Auth-Token: <censored>' --header 'Content-Type: application/json; charset=utf-8' --header 'Host: api.blabla.com' --compressed
another gotcha: on some systems, there will be a default useragent header (like debian 6), while on some systems, curl comes without a default useragent (like debian 8).. you might want to use --user-agent '' too

Can I set multiple headers with Siege?

I want to use siege to target a number of URLs on my app, each with different headers. I can set headers for one request
siege -u http://localhost/xyz -d1 -r1000 -c25 --header="Token: f2840fc1"
(this appears to be undocumented)
I can specify a list of URLs in the URL file, with custom headers for each URL. But I can't see a way in the docs.
I suggest using two concurrent calls to siege. Write a URL file that uses Header-A, and another for Header-B.
For my API testing, I've got a get_urls.txt file and post_json_urls.txt file, which I call on two instances of Siege at once. This way one gets called with Content-Type: text/json and the other doesn't. Short of rewriting the Siege url parser, this is the only way I know to do it.
For example:
siege -f get_urls.txt & siege -H 'Content-Type: text/json' -f post_json_urls.txt
As far as I can see from the man page and reading around I think you are right. The only way to specify headers is on the command line using the -H --header options not in the URL file.
You can try this example siege --concurrent=5 --reps=100 --header='sdk:3.0, config:3.0,zid:0' 'https://google.com/api/REGME POST uid=a8qn&aid=43ZK0'

cURL - Invalid character is found in given range 'start' - 'stop'

I'm making a restful api call, but I'm getting an error.
Warning: Invalid character is found in given range. A specified range MUST
Warning: have only digits in 'start'-'stop'. The server's response to this
Warning: request is uncertain.
HTTP/1.1 200 OK
Date: Thu, 22 Aug 2013 17:43:19 GMT
Server: Apache
Content-Length: 208
Allow: GET, HEAD, POST
Vary: Authorization
Content-Type: application/json
Has anyone seen this WARNING error before about an invalid character? How do I fix it?
I was calling:
curl https://my/url/for/api --include -H "Content-Type:application/json" -request POST '{ \"type\": \"code\", \"objects\" : [ \"123456\" ] } ' -u user:pass
More info:
extra response at the end of return
curl: (6) Could not resolve host: POST; nodename nor servname provided, or not known
curl: (3) [globbing] nested braces not supported at pos 33
I had a similar situation - my curl request was essentially working, and returning the information I needed, but with warnings, using the following code:
curl -request http://...
It turned out to be the -request parameter, which I'd incorrectly typed. It either needs to be something like:
curl --request GET http://...
or simply not there at all (because GET is the default):
curl http://...
A corrected command line probably looks something like:
curl https://my/url/for/api \
--include \
-H "Content-Type:application/json" \
-d '{ \"type\": \"code\", \"objects\" : [ \"123456\" ] } ' \
-u user:pass
(I split it to multiple lines for readability.) The changes I did:
1 - --request needs two dashes but since you set POST, it was superfluous and I removed it
2 - use -d (or --data) to specify what to POST. --request only takes a single method string
that you can replace POST with in the HTTP request but I don't think you want that.
cURL gets confused about the square brackets in your command. It interprets them as a range of characters it should process. Either way your question would probably fit better on SuperUser than on Stackoverflow.