Request in Postman with same cookies and headers like in browser returns 401, but in browser all works fine - authentication

I trying to scraping data from this url:
https://rgis.mosreg.ru/v3/swagger/map/layer?SERVICE=GeoJSON&layer=34&bbox=37.51027598519073,55.58991,37.84716401480926,55.89414999999997&zoom=11
In web browser, if I visit main page https://rgis.mosreg.ru first (to get cookies), and next - go to this url - all works fine.
But when I trying to perform this request in Postman - its fault with 401 "Unautorized" error.
In Postman I use all same headers and cookies, like in web-browser, but it does not help.
All cookies and headers are synced with browser using Postman INTERCEPTOR
What I missing out?
Chrome screen with headers
Postman screen. Header "mojo" looks line auth header

The server seems to only accept HTTP/2 and reject HTTP/1.1 call. If you have curl compiled with http2 support, you can test this directly:
curl --http2 'https://rgis.mosreg.ru/v3/swagger/map/layer?SERVICE=GeoJSON&layer=34&bbox=37.51027598519073,55.58991,37.84716401480926,55.89414999999997&zoom=11'
Output
< HTTP/2.0 200
< server:nginx/1.19.5 (MOGT Edition # rgis-pub-app-01)
otherwise it returns 401
At the moment, you can't run this request in Postman, because Postman doesn't have http2 support yet
You can also test it with python using the httpx package (pip install httpx[http2]):
import httpx
import asyncio
url = 'https://rgis.mosreg.ru/v3/swagger/map/layer?SERVICE=GeoJSON&layer=34&bbox=37.51027598519073,55.58991,37.84716401480926,55.89414999999997&zoom=11'
r = httpx.get(url)
print(r.http_version)
print(r.status_code)
client = httpx.AsyncClient(http2=True)
async def get():
response = await client.get(url)
print(response.http_version)
print(response.status_code)
asyncio.run(get())
Output
HTTP/1.1
401
HTTP/2
200

Related

How to query SFTP server with Karate via API?

The app that I am trying to test has an SFTP server that can be queried via API. Swagger shows the following sample cURL request to get domain files information, and I am having a hard time making this call with Karate:
curl -X GET -H 'Accept: application/json' 'https://{host}:{port}/api/{clientId}/'
I do the following where baseUrl is defined as https://sftp.mydomain.com:22 where 22 is the port number that I can successfully use to connect to the SFTP server via Cyberduck:
Feature:
Background:
* url baseUrl
* def moduleBase = '/api/12345/'
Scenario:
* path moduleBase
When method get
Then status 200
The error that I get is this:
ERROR com.intuit.karate - src/test/java/mytest.feature:9
When method get
http call failed after 815 milliseconds for url: https://sftp.mydomain.com:22/api/12345/
What am I doing wrong?
Clearly it may not be HTTP so I don't think Swagger and all is legit.
Maybe you can just delegate to the OS. Refer: https://stackoverflow.com/a/64352676/143475

Karate returns HTTP 415 for soap request

I'm trying to hit a SOAP service using karate and it always returns 415, whereas the same service is successful in SOAP UI.
Response in Karate:
20:05:37 20:05:37.236 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 37.15
20:05:37 1 < 415
20:05:37 1 < Content-Length: 0
Given url soapServiceURL
And header Content-Type = 'application/xml'
And request requestPayLoad
When method post
Then status 200
Kindly advise if I'm missing something here
Please read the docs and use soap action if needed: https://github.com/intuit/karate#soap-action
Karate can make any HTTP request, but you need to figure out the right headers etc. One hint is export to cURL and then you will be able to do it easily.

Too many authorization bearers in MFP 8.0

I've followed the steps provided in https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/8.0/authentication-and-security/protecting-external-resources/ to protect an external resource and https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/8.0/application-development/resource-request/javascript/ to call via Cordova.
I'm making 2 request to the same REST method, which is protected with the scope "aovLogin".
It seems that every call is generating a new bearer token, which needs 4 extra calls to MFP.
Also, the first time a method is called, it makes several extra calls (it always goes http 401, then 403 then 200, making extra calls to MFP in the middle). If i have a very granular API, it's making a lot of extra calls.
I've seen that the server API has a cache for the bearers and the scope is configured for being valid for 10 minutes.
Why is the client sending so many authorization requests?
POST /com.costaisa.app.api/api/mfprest/delegation/detail/private HTTP/1.1
HTTP/1.1 401 Unauthorized
----------
POST /mfp/api/preauth/v1/preauthorize HTTP/1.1
{"scope":"","client_id":"3deccec7-3f18-4ee2-8464-de90a7c64685"}
HTTP/1.1 400 Bad Request
{"errorCode":"INVALID_CLIENT_ID","errorMsg":"Invalid client ID."}
------
POST /mfp/api/registration/v1/self HTTP/1.1
{"signedRegistrationData":{"header":"XXXXX","payload":"XXXXX","signature":"XXXXX"}}
HTTP/1.1 201 Created
-----
POST /mfp/api/preauth/v1/preauthorize HTTP/1.1
{"scope":"","client_id":"84c45e4a-b75d-4125-ab9a-98f390d5bd3a"}
HTTP/1.1 200 OK
{"successes":{"clockSynchronization":{"serverTimeStamp":1480322130967}}}
--------
GET /mfp/api/az/v1/authorization?response_type=code&scope=&client_id=84c45e4a-b75d-4125-ab9a-98f390d5bd3a&redirect_uri=http://mfpredirecturi&isAjaxRequest=true&x=0.1757133661526875 HTTP/1.1
HTTP/1.1 302 Found
------
POST /mfp/api/az/v1/token HTTP/1.1
XXXXX
HTTP/1.1 200 OK
{"access_token":"XXXXX","token_type":"Bearer","expires_in":3599,"scope":""}
---
POST /com.costaisa.app.api/api/mfprest/delegation/detail/private HTTP/1.1
Authorization: Bearer XXXXX
{"idDelegation":"0801"}
HTTP/1.1 403 Forbidden
---
POST /mfp/api/preauth/v1/preauthorize HTTP/1.1
{"scope":"aovLogin","client_id":"84c45e4a-b75d-4125-ab9a-98f390d5bd3a"}
HTTP/1.1 401 Unauthorized
{"successes":{"clockSynchronization":{"serverTimeStamp":1480322131320}},"challenges":{"aovLogin":{"remainingAttempts":5,"errorMsg":null}}}
---
POST /mfp/api/preauth/v1/preauthorize HTTP/1.1
{"challengeResponse":{"aovLogin":{"username":"XXXXX","tokenSEA":"XXXXX"}},"scope":"aovLogin","client_id":"84c45e4a-b75d-4125-ab9a-98f390d5bd3a"}
HTTP/1.1 200 OK
{"successes":{"aovLogin":{"user":{"id":"XXXXX","displayName":"XXXXX","authenticatedAt":1480322139874,"authenticatedBy":"aovLogin","attributes":{"tokenSEA":"XXXXX"}}},"clockSynchronization":{"serverTimeStamp":1480322139874}}}
--------
GET /mfp/api/az/v1/authorization?response_type=code&scope=aovLogin&client_id=84c45e4a-b75d-4125-ab9a-98f390d5bd3a&redirect_uri=http://mfpredirecturi&isAjaxRequest=true&x=0.5223292209780417 HTTP/1.1
HTTP/1.1 302 Found
---
POST /mfp/api/az/v1/token HTTP/1.1
XXXXX
HTTP/1.1 200 OK
{"access_token":"XXXXX","token_type":"Bearer","expires_in":599,"scope":"aovLogin"}
---
POST /com.costaisa.app.api/api/mfprest/delegation/detail/private HTTP/1.1
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImp3ayI6eyJrdHkiOiJSU0EiLCJlIjoiQVFBQiIsImtpZCI6Ijg0YzQ1ZTRhLWI3NWQtNDEyNS1hYjlhLTk4ZjM5MGQ1YmQzYSIsIm4iOiJBTTBEZDd4QWR2NkgteWdMN3I4cUNMZEUtM0kya2s0NXpnWnREZF9xczhmdm5ZZmRpcVRTVjRfMnQ2T0dHOENWNUNlNDFQTXBJd21MNDEwWDlJWm52aHhvWWlGY01TYU9lSXFvZS1ySkEwdVp1dzJySGhYWjNXVkNlS2V6UlZjQ09Zc1FOLW1RSzBtZno1XzNvLWV2MFVZd1hrU093QkJsMUVocUl3VkR3T2llZzJKTUdsMEVYc1BaZmtOWkktSFU0b01paS1Uck5MelJXa01tTHZtMDloTDV6b3NVTkExNXZlQ0twaDJXcG1TbTJTNjFuRGhIN2dMRW95bURuVEVqUFk1QW9oMmluSS0zNlJHWVZNVVViTzQ2Q3JOVVl1SW9iT2lYbEx6QklodUlDcGZWZHhUX3g3c3RLWDVDOUJmTVRCNEdrT0hQNWNVdjdOejFkRGhJUHU4PSJ9fQ.eyJpc3MiOiJjb20uaWJtLm1mcCIsInN1YiI6Ijg0YzQ1ZTRhLWI3NWQtNDEyNS1hYjlhLTk4ZjM5MGQ1YmQzYSIsImF1ZCI6ImNvbS5pYm0ubWZwIiwiZXhwIjoxNDgwMzIyNzM5ODc0LCJzY29wZSI6ImFvdkxvZ2luIn0.jGJAhZaV6NFHZKj-LKBmJ6Gqb7ZrZX20xDKEPkNtORZ1tanLo8MSklY2HogK-wKs7APIuWESLSsskrwR9p0EnrmHgUYZf3BPY9HDUSBojUN9-vd_I9kavcg34Hes1KTvYG4Wi-9XbZQ2T1-SbHhn-mqsToeLIGBGkzsugwQG9tIKG3Qr0BixDIfuhxux4Gdo30HCyn9SB5ZaY5wdxaD2_kJjnJih_SsAuuXRNAXEO_PgExnZ6Mr1qyqyOfwc3k9jmgRpuEQigYYRYOP-Tvs_i59IVYOdpsQ70gi-Ky09orx5Jy3hVJv-J45Dx7FHdR3ZPTn7pYW7IRmRo4CZ2COoCg
HTTP/1.1 200 OK
.....
--- CALL AGAIN, new bearer is generated
POST /mfp/api/az/v1/introspection HTTP/1.1
POST /mfp/api/preauth/v1/preauthorize HTTP/1.1
GET /mfp/api/az/v1/authorization?XXX HTTP/1.1
POST /mfp/api/az/v1/token HTTP/1.1
POST /com.costaisa.app.api/api/mfprest/delegation/detail/private HTTP/1.1
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImp3ayI6eyJrdHkiOiJSU0EiLCJlIjoiQVFBQiIsImtpZCI6IjM1NDcyYWNhLWVlNmItNGNhZi04OGQ2LWQxY2ExNjQ0NzM4NyIsIm4iOiJBTTBEZDd4QWR2NkgteWdMN3I4cUNMZEUtM0kya2s0NXpnWnREZF9xczhmdm5ZZmRpcVRTVjRfMnQ2T0dHOENWNUNlNDFQTXBJd21MNDEwWDlJWm52aHhvWWlGY01TYU9lSXFvZS1ySkEwdVp1dzJySGhYWjNXVkNlS2V6UlZjQ09Zc1FOLW1RSzBtZno1XzNvLWV2MFVZd1hrU093QkJsMUVocUl3VkR3T2llZzJKTUdsMEVYc1BaZmtOWkktSFU0b01paS1Uck5MelJXa01tTHZtMDloTDV6b3NVTkExNXZlQ0twaDJXcG1TbTJTNjFuRGhIN2dMRW95bURuVEVqUFk1QW9oMmluSS0zNlJHWVZNVVViTzQ2Q3JOVVl1SW9iT2lYbEx6QklodUlDcGZWZHhUX3g3c3RLWDVDOUJmTVRCNEdrT0hQNWNVdjdOejFkRGhJUHU4PSJ9fQ.eyJpc3MiOiJjb20uaWJtLm1mcCIsInN1YiI6IjM1NDcyYWNhLWVlNmItNGNhZi04OGQ2LWQxY2ExNjQ0NzM4NyIsImF1ZCI6ImNvbS5pYm0ubWZwIiwiZXhwIjoxNDgwMzM5OTU0NjE2LCJzY29wZSI6ImFvdkxvZ2luIn0.JSm3nrW6BD5i66GossHYM4-6GqQfC-ZSH5P-X4M9mws2jBNvCkFKgv_XbRAb3km-0NMZz3FHsrY_0h0dx7fpJYiR9CIjaY-PFw75zdKbyEpzbhAX7OjZtYOtZblKEYLkT8mH-0mLc6VE_YBPFd2q55HMmECCLirAAdWwzMGgEzL02OKTd1GVuJyjqjlxeOJypFglaHezuByd6eGVMFJvnfDX3h_o6k8sWcv-g7UFa8jtcMNZpbzFOYG9Q2nGQ-oYIt17QyF4CVKPMN4anMwRRQ_2cjuvg-1ZuU450hxBX3u09wBxJ21mQklgg72t7fdLKgT7EIPmQlPP3wrX9qzy7A
HTTP/1.1 200 OK
Update:
The HTTP 401 and 403 calls to the external resource and serveral calls to MFP can be avoided if the scope is send in WLResourceRequest
It generates a new token calling an external resource using an absolute URL but also calling a standard protected adapter using a relative URL
Sample calling a protected adapter:
var resourceRequest = new WLResourceRequest(
"/adapters/AOS42_AOV_API/resource/protectedResource",
WLResourceRequest.GET,
{'scope' : 'aovLogin'} // it avoids 401 and 403 responses
);
resourceRequest.send().then(
function (response) {
alert("response ok protectedResource " + response.responseText);
},
function (response) {
alert("response ko protectedResource " + response.responseText);
}
);
Sample calling an external resource:
var resourceRequest = new WLResourceRequest(
"https://someurl.com/someApp/protectedResource",
WLResourceRequest.GET,
{'scope' : 'aovLogin'} // it avoids 401 and 403 responses
);
Update 2:
We've made a change: Instead of calling to a protected external resource, receiving HTTP 401 and then sending the challenge, now we call to WLAuthorizationManager.login before.
In Android, it continues calling MFP 3 times before each call, but now the server returns the same Bearer Token.
The same Cordova Application calling the same Rest API protected by MFP and using the same security adapter in MFP works perfectly fine in iOS.
Once the bearer is obtained, we see only calls to the external API.
This bug has been resolved in a just-released iFix for MobileFirst Foundation 8.0. The build number is 8.0.0.0-IF20170125-0919. Please login to IBM Fix Central to download the iFix.
The associated APAR is:
PI74988 MULTIPLE AUTHORIZATION CALLS ARE MADE FOR EACH REST CALL IN ANDROID APPLICATION
Since you're using Cordova, I believe updating the cordova-plugin-mfp plug-in to #8.0.2017012210 should suffice.

CORS request not valid - missing headers?

I am trying to upload an image to a finagle (netty) server. For the OPTIONS request I return the following:
curl -X OPTIONS http://localhost:8686/images -i
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST
Browser (FireBug): http://s15.postimg.org/vtdzyfshn/Screen_Shot_2014_09_02_at_9_49_05_PM.png
The following POST request fails with
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote
resource at http://localhost:8686/images. This can be fixed by moving the resource
to the same domain or enabling CORS.
FireBug 1 (Console): http://s30.postimg.org/9utq4ridt/Screen_Shot_2014_09_02_at_9_53_41_PM.png
FireBug 2 (Net Tab): http://s16.postimg.org/jyblxfcv9/Screen_Shot_2014_09_02_at_9_54_37_PM.png
FireBug 3 (Net Tab - POST): http://s14.postimg.org/e8czua2wh/Screen_Shot_2014_09_02_at_9_54_47_PM.png
Any idea what I am missing?
(I am using this upload script: http://www.extremecss.com/creating-asynchronous-file-upload-system-using-html5-file-api/
You have to return the CORS headers (Access-Control-Allow-Origin: *) within the response header to your POST as well, instead of just within the OPTIONS response.

Datastore API always returning "dsid: Missing Value" error

I'm trying to follow the datastore API tutorial and this simple request (sent via Fiddler):
POST https://api.dropbox.com/1/datastores/get_or_create_datastore HTTP/1.1
User-Agent: Fiddler
Host: api.dropbox.com
Content-Length: 12
Authorization: Bearer [snipped]
dsid=default
always results in this error response:
HTTP/1.1 400 Bad Request
{"error": {"dsid": "Missing value"}}
The access token was created from the developer app console, and my test app has full dropbox permissions. Running the list_datastores API call succeeds and reports that I do have one datastore with a dsid of default.
I think you'll need a header of Content-Type: application/x-www-form-url-encoded, since you're sending form-encoded parameters.