Getting Truncated JSON response when calling UPS API using google app script - api

I am using google app script to Call UPS api and generate shipping label. However the API response is truncated and i am unable to decode base64 encoded image which is part of the JSON response object as it is truncated.
I am also not getting any truncation error messages or responses from the UPS servers, neither is google apps script throwing an error
Have contacted UPS support with the JSON request and it seems to works fine at their end.
// Here is the code for API call.
function getLabel() {
var userName = "myUPS_username";
var password = "*********";
var accessKey = "my_access_key";
var transId = "Trans123";
var transactionSrc = "upstest";
var url = "https://wwwcie.ups.com/ship/v1807/shipments";
var header = {
'AccessLicenseNumber' : accessKey,
'password' : password,
'transId' : transId,
'transactionsrc' : transactionSrc,
'username' : userName
};
// parameters for url fetch
var params = {
'method': 'GET',
'contentType': 'application/json',
'headers': header,
'payload' : JSON.stringify(payload)
};
// call the UPS Shipment API
var response = UrlFetchApp.fetch(url, params);
}
Not including the JSON payload here

Answer:
UrlFetchApp() has functionality and response limitations, including POST and response sizes.
More Information:
As per the Apps Script Documentation, URL Fetch has Limitations which are implemented in the methods themselves. The limitations are as follows:
URL Fetch response size: 50MB/call
URL Fetch headers: 100/call
URL Fetch header size: 8kB/call
URL Fetch POST size: 50MB/call
URL Fetch URL length: 2kB/call
Unfortunately, there is no way to get around this.
Feature Request:
There is a Feature Request on Google's Issue Tracker requesting the increase of the UrlFetch response size limit already. This Feature Request can be found here, which you can give a star (☆) in the top left to let Google know more people wish for this request. There is already a response from them saying 'We'll consider raising the quota if there is enough interest from the developer community.', so letting them know this is a wanted feature is a good way to go.
References:
Google's Issue Tracker
Increase the UrlFetch Total Bytes quota Feature Request
Quotas for Google Services
Current Quotas
Current Limitations

Related

Sync Token in Postman

A fresher to postman, currently working on API project where I need to delivery to the API and Token the client to integrate with them system, good is I successfully configure the Authorization as OAuth Type as Password Credentials and receiving perfect response as 200.
The issue/confusion is Token is getting expire every hour, I need to Get new Access Token every time.
So, the question is, is it anyway I can overcome this issue?
that no need to get new/refresh token.
can provide the one fix token to client.
You can do it like here. You can get the token in the pre-request field of the collection or request.
https://stackoverflow.com/a/73911458/10126763
EDIT
We can adapt it like this:
Take this and paste it in the "pre-request" field of the collection or the request you will use directly. Create an "environment" value named "accessToken". When each request is run, this method will run first and send the token value to the value in the environment.
// Set refresh and access tokens
const loginRequest = {
url: "exampleurl.com/etc/etc", //YOUR URL
method: 'GET',
header: {
'content-type': 'application/json',
'Accept': "*/*"
} //Since you will be using GET, I deleted the body. If you are sending value you can get the body field from the other example in the link.
};
pm.sendRequest(loginRequest, function (err, res) {
pm.environment.set("accessToken", res.json().accessToken); //The token returned in the response and the environment value to which the value will be sent
});

403 access denied to the website with proper login/pass through google script

var url = "https://web-site_name/page/?format=json&var_data-organization_dates&xlsexport=true";
var payload =
{
"login" : "login",
"password" : "pass",
};
var options =
{
"method" : "post",
"payload" : payload,
"followRedirects" : false
};
var login = UrlFetchApp.fetch("https://web-site_name/page/" , options);
var sessionDetails = login.getAllHeaders()['Set-Cookie'];
Logger.log(login.getAllHeaders());
here is the part of the code I try to use, to automate export of the data from web-site, i do have proper login and password and able to download file in json (opened in xsl) manually, I've got the address to the downloaded file in network in developer tools, but i have a problem on the first stage - when trying to authorize to the web-site - access denied. I've tried the code, given in answers on stackoverflow, but it still doesn't work.
How to make an url fetch request correctly, depends on the website you want to access and the authentication they uses
In the simplest case, your website requires HTTP basic authentification, in this case the correct syntax would be
var authHeader = 'Basic ' + Utilities.base64Encode(login + ':' + pass);
var options = {
headers: {Authorization: authHeader}
}
If your website uses a different authentication form, you might need to provide an access token.
In any case: the authentication credentials go into headers, not into payload!
payload is the data that you want to post = upload to the website.
If you want export data from the website - that is download data - you do not need a payload and the correct method would be get, not post. Btw., if the method is get, you do not need to specify it.
Please see here for more information and samples.

Webhook call failed. Error: Failed to parse webhook JSON response: Expect message object but got: [Chinese letters]

I'm building my own WebhookClient for dialog flow. My code is the following (using Azure Functions, similar to Firebase Functions):
module.exports = async function(context, req) {
const agent = new WebhookClient({ request: context.req, response: context.res });
function welcome(agent) {
agent.add(`Welcome to my agent!!`);
}
let intentMap = new Map();
intentMap.set("Look up person", welcome);
agent.handleRequest(intentMap);
}
I tested the query and the response payload looks like this:
{
"fulfillmentText": "Welcome to my agent!!",
"outputContexts": []
}
And the headers in the response look like this:
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Tue, 11 Dec 2018 18:16:06 GMT
But when I test my bot in dialog flow, it returns the following:
Webhook call failed. Error: Failed to parse webhook JSON response:
Expect message object but got:
"笀ഀ਀  ∀昀甀氀昀椀氀氀洀攀渀琀吀攀砀琀∀㨀 ∀圀攀氀挀漀洀攀 琀漀 洀礀 愀最攀渀琀℀℀∀Ⰰഀ਀  ∀漀甀琀瀀甀琀䌀漀渀琀攀砀琀猀∀㨀 嬀崀ഀ਀紀".
There's Chinese symbols!? Here's a video of me testing it out in DialogFlow: https://imgur.com/yzcj0Kw
I know this should be a comment (as it isn't really an answer), but it's fairly verbose and I didn't want it to get lost in the noise.
I have the same problem using WebAPI on a local machine (using ngrok to tunnel back to Kestrel). A friend of mine has working code (he's hosting in AWS rather than Azure), so I started examining the differences between our responses. I've notice the following:
This occurs with Azure Functions and WebAPI (so it's not that)
The JSON payloads are identical (so it's not that)
Working payload isn't chunked
Working payload doesn't have a content type
As an experiment, I added this code to Startup.cs, in the Configure method:
app.Use(async (context, next) =>
{
var original = context.Response.Body;
var memory = new MemoryStream();
context.Response.Body = memory;
await next();
memory.Seek(0, SeekOrigin.Begin);
if (!context.Response.Headers.ContentLength.HasValue)
{
context.Response.Headers.ContentLength = memory.Length;
context.Response.ContentType = null;
}
await memory.CopyToAsync(original);
});
This code disables response chunking, which is now causing a new and slightly more interesting error for me in the google console:
*Webhook call failed. Error: Failed to parse webhook JSON response: com.google.gson.stream.MalformedJsonException: Unterminated object at line 1 column 94 path $.\u0000\\"\u0000f\u0000u\u0000l\u0000f\u0000i\u0000l\u0000l\u0000m\u0000e\u0000n\u0000t\u0000M\u0000e\u0000s\u0000s\u0000a\u0000g\u0000e\u0000s\u0000\\"\u0000.\
I thought this could be encoding at first, so I stashed my JSON as a string and used the various Encoding classes to convert between them, to no avail.
I fired up Postman and called my endpoint (using the same payload as Google) and I can see the whole response payload correctly - it's almost as if Google's end is terminating the stream part-way through reading...
Hopefully, this additional information will help us figure out what's going on!
Update
After some more digging and various server/lambda configs, I spotted this post here: https://github.com/googleapis/google-cloud-dotnet/issues/2258
It turns out that json.net IS the culprit! I guess it's something to do with the formatters on the way out of the pipeline. In order to prove this, I added this hard-coded response to my POST controller and it worked! :)
return new ContentResult()
{
Content = "{\"fulfillmentText\": null,\"fulfillmentMessages\": [],\"source\": null,\"payload\": {\"google\": {\"expectUserResponse\": false,\"userStorage\": null,\"richResponse\": {\"items\": [{\"simpleResponse\": {\"textToSpeech\": \"Why hello there\",\"ssml\": null,\"displayText\": \"Why hello there\"}}],\"suggestions\": null,\"linkOutSuggestion\": null}}}}",
ContentType = "application/json",
StatusCode = 200
};
Despite the HTTP header saying the charset is utf-8, that is definitely using the utf-16le character set, and then the receiving side is treating them as utf-16be. Given you're running on Azure, it sounds like there is some configuration you need to make in Azure Functions to represent the output as UTF-8 instead of using UTF-16 strings.

How to get thumbnails for shared OneDrive files to unauthorized users? This useful OneDrive feature provided in the old API is broken now

My app uses OneDrive API feature to let unauthorized users get thumbnails for shared OneDrive files using old API request:
https:// apis.live.net/v5.0/skydrive/get_item_preview?type=normal&url=[shared_link_to_OneDrive_file]
This feature is broken now (any such links return XMLHttpRequest connection error 0x2eff).
And my Windows Store app can not longer provide this feature.
Anyone can try to check it, link to shared OneDrive file:
https://onedrive.live.com/redir?resid=AABF0E8064900F8D!27202&authkey=!AJTeSCuaHMc45eY&v=3&ithint=photo%2cjpg
links to a preview image for shared OneDrive file (according to old OneDrive API
"Displaying a preview of a OneDrive item" - https:// msdn.microsoft.com/en-us/library/jj680723.aspx):
https://apis.live.net/v5.0/skydrive/get_item_preview?type=normal&url=https%3A%2F%2Fonedrive.live.com%2Fredir%3Fresid%3DAABF0E8064900F8D!27202%26authkey%3D!AJTeSCuaHMc45eY%26v%3D3%26ithint%3Dphoto%252cjpg
generates error: SCRIPT7002: XMLHttpRequest: Network error 0x2eff
Сurrent OneDrive API thumbnail feature:
GET /drive/items/{item-id}/thumbnails/{thumb-id}/{size}
is just for authorized users and can not provide access to thumbnails for shared OneDrive files to unauthorized users
How can a Windows Store app let unauthorized users get thumbnails for shared OneDrive files (videos etc.) using the current OneDrive API?
Any ideas?
You need to make a call to the following API:
GET /drive/items/{item-id}/thumbnails/{thumb-id}/{size}/content
This call needs to use authorization and returns a redirect to a cache-safe thumbnail location. You can then use this new url to serve thumbnails to unauthenticated users.
e.g.
Request:
GET https://api.onedrive.com/v1.0/drive/items/D094522DE0B10F6D!152/thumbnails/0/small/content
Authorization: bearer <access token>
Response:
HTTP/1.1 302 Found
Location: https://qg3u2w.bn1302.livefilestore.com/y3m1LKnRaQvGEEhv_GU3mVsewg_-aizIXDaVczGwGFIqtNcVSCihLo7s2mNdUrKicuBnB2sGlSwMQTzQw7v34cHLkchKHL_5YC3IMx1SMcpndtdb9bmQ6y2iG4id0HHgCUlgctvYsDrE24XALwXv2KWRUwCCvDJC4hlkqYgnwGBUSQ
You can now use the link in the Location header to access the thumbnail without signing in. This url will change only if the contents of the file change.
You can read more in the documentation here.
I just figured it out. It is based on the information in this article from Microsoft...
https://learn.microsoft.com/en-ca/onedrive/developer/rest-api/api/driveitem_list_thumbnails?view=odsp-graph-online
... look at the section, "Getting thumbnails while listing DriveItems." It shows you the relevant JSON return structure from a call such as:
GET /me/drive/items/{item-id}/children?$expand=thumbnails
Basically, the JSON return structure gives you string URL's for each of the thumbnail formats. You then create URLSession's to upload these URL's (once you've converted them from String to URL)
Here is an excerpt of code using Swift (Apple):
////////////////////////////////////////////////////////////////////////////////
//
// Download a thumbnail with a URL and label the URLSession with an ID.
//
func downloadThumbnail(url: URL, id: String) {
// Create a URLSession. This is an object that controls the operation or flow
// control with respect to asynchronous operations. It sets the callback delegate
// when the operation is complete.
let urlSession: URLSession = {
//let config = URLSessionConfiguration.default
let config = URLSessionConfiguration.background(withIdentifier: id)
config.isDiscretionary = true
config.sessionSendsLaunchEvents = true
//config.identifier = "file download"
return URLSession(configuration: config, delegate: self as URLSessionDelegate, delegateQueue: OperationQueue.main)
}()
// Create the URLRequest. This is needed so that "Authorization" can be made, as well
// as the actual HTTP command. The url, on initialization, is the command... along
// with the "GET" setting of the httpMethod property.
var request = URLRequest(url: url)
// Set the Authorization header for the request. We use Bearer tokens, so we specify Bearer + the token we got from the result
request.setValue("Bearer \(self.accessToken)", forHTTPHeaderField: "Authorization")
request.httpMethod = "GET"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
// This initiates the asynchronous data task
let backgroundTask = urlSession.downloadTask(with: request)
//backgroundTask.earliestBeginDate = Date().addingTimeInterval(60 * 60)
backgroundTask.countOfBytesClientExpectsToSend = 60
backgroundTask.countOfBytesClientExpectsToReceive = 15 * 1024
backgroundTask.resume()
}
... of course you need to have the correct "accessToken," (shown above) but you also have to have written the generic callback function for URLSession, which is:
func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask,
didFinishDownloadingTo location: URL) {
Swift.print("DEBUG: urlSession callback reached")
// This was the identifier that you setup URLSession with
let id = session.configuration.identifier
// "location" is the temporary URL that the thumbnail was downloaded to
let temp = location
// You can convert this URL into any kind of image object. Just Google it!
}
Cheers, Andreas

Google Apps Script: Salesforce API Call

Just finished breakfast and already hit a snag. I'm trying to call the salesforce REST api from my google sheets. I've written a working script locally in python, but converting it into JS, something went wrong:
function authenticateSF(){
var url = 'https://login.salesforce.com/services/oauth2/token';
var options = {
grant_type:'password',
client_id:'XXXXXXXXXXX',
client_secret:'111111111111',
username:'ITSME#smee.com',
password:'smee'
};
var results = UrlFetchApp.fetch(url, options);
}
Here is the error response:
Request failed for https://login.salesforce.com/services/oauth2/token
returned code 400. Truncated server response:
{"error_description":"grant type not
supported","error":"unsupported_grant_type"} (use muteHttpExceptions
option to examine full response) (line 12, file "Code")
Mind you, these exact parameters work fine in my local python script (putting the key values inside quotations).
Here are the relevant docs:
Google Script: Connecting to external API's
Salesforce: REST API guide
Thank you all!
Google's UrlFetchApp object automatically defaults to a GET request. To authenticate, you have to explicitly set in the options the method "post":
function authenticateSF(){
var url = 'https://login.salesforce.com/services/oauth2/token';
var payload = {
'grant_type':'password',
'client_id':'XXXXXXXXXXX',
'client_secret':'111111111111',
'username':'ITSME#smee.com',
'password':'smee'
};
var options = {
'method':'post',
'payload':payload
};
var results = UrlFetchApp.fetch(url, options);
}