RestSharp RestResponse is truncating content to 64 kb - httprequest

Hi I am using the RestSharp to create the request to my web API. Unfortunately the response.content does not contain full response, which I am able to see when I perform request through browser or fiddler. The content is being truncated to 64 kb. I am attaching my code below.
Could you please advice what could solve this issue?
var request = new RestRequest("Products?productId={productId}&applicationId={applicationId}", Method.GET);
request.RequestFormat = DataFormat.Json;
request.AddParameter("productId", id, ParameterType.UrlSegment);
request.AddParameter("applicationId", Settings.ApplicationId, ParameterType.UrlSegment);
request.AddHeader("X-AppKey", token.AppKey);
request.AddHeader("X-Token", token.Token);
request.AddHeader("X-IsWebApi", "true");
RestResponse response = (RestResponse) client.Execute(request);
if (response.StatusCode == HttpStatusCode.Found)
{
// The following line failes because response.Content is truncated.
ShowProductModel showProductModel =
new JavaScriptSerializer().Deserialize<ShowProductModel>(response.Content);
// Do other things.
return ShowProductApi(showProductModel, q, d, sort, breadcrumb);
}

This is happening because RestSharp uses the HttpWebRequest class from the .NET Framework. This class has a static attribute called DefaultMaximumErrorResponseLength. This attribute determines the max length of an error response, and the default value for this attribute is 64Kb.
You can change the value of that atribbute before instatiating the RestRequest class.
Here's some code:
HttpWebRequest.DefaultMaximumErrorResponseLength = 1048576;
var request = new RestRequest("resource" + "/", Method.POST)
{
RequestFormat = DataFormat.Json,
JsonSerializer = new JsonSerializer()
};
That way your error response can be longer without problemns.

It looks like HttpStatusCode.Found may be causing the issue. That equates to Http Status Code 302 which is a form of redirect. I'm not entirely sure if that's necessarily the right thing to do in this case. If you have "found" the data you are looking for you should return a success level status code, e.g. 200 (Ok). Wikipedia has a list of HTTP Status Codes with summaries about what they mean and links off to lots of other resources.
I've created a little demonstrator solution (You can find it on GitHub) to show the difference. There is a WebApi server application that returns a list of values (Hex codes) and a Console client application that consumes the resources on the WebApi application.
Here is the ValuesFound resource which returns HTTP Status Code 302/Found:
public class ValuesFoundController : ApiController
{
public HttpResponseMessage Get(int count)
{
var result = Request.CreateResponse(HttpStatusCode.Found, Values.GetValues(count));
return result;
}
}
And the same again but returning the correct 200/OK response:
public class ValuesOkController : ApiController
{
public HttpResponseMessage Get(int count)
{
var result = Request.CreateResponse(HttpStatusCode.OK, Values.GetValues(count));
return result;
}
}
On the client side the important part of the code is this:
private static void ProcessRequest(int count, string resource)
{
var client = new RestClient("http://localhost:61038/api/");
var request = new RestRequest(resource+"?count={count}", Method.GET);
request.RequestFormat = DataFormat.Json;
request.AddParameter("count", count, ParameterType.UrlSegment);
RestResponse response = (RestResponse) client.Execute(request);
Console.WriteLine("Status was : {0}", response.StatusCode);
Console.WriteLine("Status code was : {0}", (int) response.StatusCode);
Console.WriteLine("Response.ContentLength is : {0}", response.ContentLength);
Console.WriteLine("Response.Content.Length is: {0}", response.Content.Length);
Console.WriteLine();
}
The count is the number of hex codes to return, and resource is the name of the resource (either ValuesOk or ValuesFound) which map to the controllers above.
The console application asks the user for a number and then shows the length of response for each HTTP Status Code. For low values, say 200, both versions return the same amount of content, but once the response content exceeds 64kb then the "Found" version gets truncated and the "Ok" version does not.
Trying the console application with a value of about 9999 demonstrates this:
How many things do you want returned?
9999
Waiting on the server...
Status was : OK
Status code was : 200
Response.ContentLength is : 109990
Response.Content.Length is: 109990
Status was : Redirect
Status code was : 302
Response.ContentLength is : 109990
Response.Content.Length is: 65536
So, why does RestSharp do this? I've no idea why it truncates content in one instance and not in the other. However, it could be assumed that in a situation where the server has asked the client to redirect to another resource location that content exceeding 64kb is unlikely to be valid.
For example, if you use Fiddler to look at what websites do, the responses in the 300 range (Redirection) such as 302/Found do have a small content payload that simply contain a little HTML so that the user can click the link to manually redirect if the browser did not automatically redirect for them. The real redirect is in the Http "Location" header.

Related

How do I get the message from an API using Flurl?

I've created an API in .NET Core 2 using C#. It returns an ActionResult with a status code and string message. In another application, I call the API using Flurl. I can get the status code number, but I can't find a way to get the message. How do I get the message or what do I need to change in the API to put the message someway Flurl can get it?
Here's the code for the API. The "message" in this example is "Sorry!".
[HttpPost("{orderID}/SendEmail")]
[Produces("application/json", Type = typeof(string))]
public ActionResult Post(int orderID)
{
return StatusCode(500, "Sorry!");
}
Here's the code in another app calling the API. I can get the status code number (500) using (int)getRespParams.StatusCode and the status code text (InternalError) using getRespParams.StatusCode, but how do I get the "Sorry!" message?
var getRespParams = await $"http://localhost:1234/api/Orders/{orderID}/SendEmail".PostUrlEncodedAsync();
int statusCodeNumber = (int)getRespParams.StatusCode;
PostUrlEncodedAsync returns an HttpResponseMessage object. To get the body as a string, just do this:
var message = await getRespParams.Content.ReadAsStringAsync();
One thing to note is that Flurl throws an exception on non-2XX responses by default. (This is configurable). Often you only care about the status code if the call is unsuccessful, so a typical pattern is to use a try/catch block:
try {
var obj = await url
.PostAsync(...)
.ReceiveJson<MyResponseType>();
}
catch (FlurlHttpException ex) {
var status = ex.Call.HttpStatus;
var message = await ex.GetResponseStringAsync();
}
One advantage here is you can use Flurl's ReceiveJson to get the response body directly in successful cases, and get the error body (which is a different shape) separately in the catch block. That way you're not dealing with deserializing a "raw" HttpResponseMessage at all.

Google Sheets API v4 receives HTTP 401 responses for public feeds

I'm having no luck getting a response from v4 of the Google Sheets API when running against a public (i.e. "Published To The Web" AND shared with "Anyone On The Web") spreadsheet.
The relevant documentation states:
"If the request doesn't require authorization (such as a request for public data), then the application must provide either the API key or an OAuth 2.0 token, or both—whatever option is most convenient for you."
And to provide the API key, the documentation states:
"After you have an API key, your application can append the query parameter key=yourAPIKey to all request URLs."
So, I should be able to get a response listing the sheets in a public spreadsheet at the following URL:
https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}?key={myAPIkey}
(with, obviously, the id and key supplied in the path and query string respectively)
However, when I do this, I get an HTTP 401 response:
{
error: {
code: 401,
message: "The request does not have valid authentication credentials.",
status: "UNAUTHENTICATED"
}
}
Can anyone else get this to work against a public workbook? If not, can anyone monitoring this thread from the Google side either comment or provide a working sample?
I managed to get this working. Even I was frustrated at first. And, this is not a bug. Here's how I did it:
First, enable these in your GDC to get rid of authentication errors.
-Google Apps Script Execution API
-Google Sheets API
Note: Make sure the Google account you used in GDC must be the same account you're using in Spreadsheet project else you might get a "The API Key and the authentication credential are from different projects" error message.
Go to https://developers.google.com/oauthplayground where you will acquire authorization tokens.
On Step 1, choose Google Sheets API v4 and choose https://www.googleapis.com/auth/spreadsheets scope so you have bot read and write permissions.
Click the Authorize APIs button. Allow the authentication and you'll proceed to Step 2.
On Step 2, click Exchange authorization code for tokens button. After that, proceed to Step 3.
On Step 3, time to paste your URL request. Since default server method is GET proceed and click Send the request button.
Note: Make sure your URL requests are the ones indicated in the Spreadsheetv4 docs.
Here's my sample URL request:
https://sheets.googleapis.com/v4/spreadsheets/SPREADSHEET_ID?includeGridData=false
I got a HTTP/1.1 200 OK and it displayed my requested data. This goes for all Spreadsheetv4 server-side processes.
Hope this helps.
We recently fixed this and it should now be working. Sorry for the troubles, please try again.
The document must be shared to "Anyone with the link" or "Public on the web". (Note: the publishing settings from "File -> Publish to the web" are irrelevant, unlike in the v3 API.)
This is not a solution of the problem but I think this is a good way to achieve the goal. On site http://embedded-lab.com/blog/post-data-google-sheets-using-esp8266/ I found how to update spreadsheet using Google Apps Script. This is an example with GET method. I will try to show you POST method with JSON format.
How to POST:
Create Google Spreadsheet, in the tab Tools > Script Editor paste following script. Modify the script by entering the appropriate spreadsheet ID and Sheet tab name (Line 27 and 28 in the script).
function doPost(e)
{
var success = false;
if (e != null)
{
var JSON_RawContent = e.postData.contents;
var PersonalData = JSON.parse(JSON_RawContent);
success = SaveData(
PersonalData.Name,
PersonalData.Age,
PersonalData.Phone
);
}
// Return plain text Output
return ContentService.createTextOutput("Data saved: " + success);
}
function SaveData(Name, Age, Phone)
{
try
{
var dateTime = new Date();
// Paste the URL of the Google Sheets starting from https thru /edit
// For e.g.: https://docs.google.com/---YOUR SPREADSHEET ID---/edit
var MyPersonalMatrix = SpreadsheetApp.openByUrl("https://docs.google.com/spreadsheets/d/---YOUR SPREADSHEET ID---/edit");
var MyBasicPersonalData = MyPersonalMatrix.getSheetByName("BasicPersonalData");
// Get last edited row
var row = MyBasicPersonalData.getLastRow() + 1;
MyBasicPersonalData.getRange("A" + row).setValue(Name);
MyBasicPersonalData.getRange("B" + row).setValue(Age);
MyBasicPersonalData.getRange("C" + row).setValue(Phone);
return true;
}
catch(error)
{
return false;
}
}
Now save the script and go to tab Publish > Deploy as Web App.
Execute the app as: Me xyz#gmail.com,
Who has access to the app: Anyone, even anonymous
Then to test you can use Postman app.
Or using UWP:
private async void Button_Click(object sender, RoutedEventArgs e)
{
using (HttpClient httpClient = new HttpClient())
{
httpClient.BaseAddress = new Uri(#"https://script.google.com/");
httpClient.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("utf-8"));
string endpoint = #"/macros/s/---YOUR SCRIPT ID---/exec";
try
{
PersonalData personalData = new PersonalData();
personalData.Name = "Jarek";
personalData.Age = "34";
personalData.Phone = "111 222 333";
HttpContent httpContent = new StringContent(JsonConvert.SerializeObject(personalData), Encoding.UTF8, "application/json");
HttpResponseMessage httpResponseMessage = await httpClient.PostAsync(endpoint, httpContent);
if (httpResponseMessage.IsSuccessStatusCode)
{
string jsonResponse = await httpResponseMessage.Content.ReadAsStringAsync();
//do something with json response here
}
}
catch (Exception ex)
{
}
}
}
public class PersonalData
{
public string Name;
public string Age;
public string Phone;
}
To above code NuGet Newtonsoft.Json is required.
Result:
If your feed is public and you are using api key, make sure you are throwing a http GET request.In case of POST request, you will receive this error.
I faced same.
Getting data using
Method: spreadsheets.getByDataFilter has POST request

Windows Phone Silverlight request does not update

I'm quite new to the Windows Phone dev and I have to do an application to communicate with a Restful API. Everything works fine to get the informations back from the API but my problem occurs when I try to update the content. For example, I have a profile and I try to update the user's information (change the city let's say). On the server side I can see that my update worked properly but when I go back to my profile in my WP app nothing has changed, the city is still the same as the old one. This is my code :
public MainPage()
{
InitializeComponent();
this.ApplicationBar = this.Resources["HomeBar"] as ApplicationBar;
Requester requester = new Requester();
requester.initGetRequest("/me/", GetResponseCallback, true);
}
private void GetResponseCallback(IAsyncResult asynchronousResult)
{
try
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
string read = streamRead.ReadToEnd();
GlobalData.GetInstance().user = JsonConvert.DeserializeObject<MeClass>(read);
Dispatcher.BeginInvoke(() =>
{
MessageBox.Show(read);
});
//Create the profile and stuff
streamResponse.Close();
streamRead.Close();
response.Close();
}
catch (WebException webException)
{
HttpStatusCode status = ((HttpWebResponse)webException.Response).StatusCode;
Dispatcher.BeginInvoke(() =>
{
MessageBox.Show(status.ToString());
});
}
}
I figured out that the string 'read' is always equal to the old one, even after the update so this is why the content is not updated but how can the response be exactly the same as before, even if the update worked fine on the server side (if I check in Postman after my update, I can see that my city is the new one). If I restart my app I can see the update.
I can also show you my initGetRequest() :
public void initGetRequest(String endPoint, Action<IAsyncResult> callback, Boolean header)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + endPoint);
if (header == true)
request.Headers["Authorization"] = GlobalData.GetInstance().Header;
request.BeginGetResponse(new AsyncCallback(callback), request);
}
Thank you for your help !
I finally found why my request was still the same even after the update. The HttpWebRequest uses a cache by default. I only added a small bit of code before calling my request :
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + endPoint);
if (header == true)
request.Headers["Authorization"] = GlobalData.GetInstance().Header;
request.Headers[HttpRequestHeader.IfModifiedSince] = DateTime.UtcNow.ToString();
request.BeginGetResponse(new AsyncCallback(callback), request);
I had no idea about that cache so I hope this answer will help someone having the same issue !

WCF API and API Key authorisation

Written or started to write a WEB API rest service in WCF. It's all going relatively well. However, I've come across a small problem. I've implemented this;
http://blogs.msdn.com/b/rjacobs/archive/2010/06/14/how-to-do-api-key-verification-for-rest-services-in-net-4.aspx
For key validation. (I'm not sure if this is the correct approach for WCF WEB API, since it looks more like the rest service implementation).
Anyway, it seems to work. However, when the api key is not provided the exception is not been displayed in the browser. I.e. if I provide the key, it returns correctly, if I don't it just shows a blank page.
private static void CreateErrorReply(OperationContext operationContext, string key)
{
// The error message is padded so that IE shows the response by default
using (var sr = new StringReader("<?xml version=\"1.0\" encoding=\"utf-8\"?>" + APIErrorHTML))
{
XElement response = XElement.Load(sr);
using (Message reply = Message.CreateMessage(MessageVersion.None, null, response))
{
HttpResponseMessageProperty responseProp = new HttpResponseMessageProperty() { StatusCode = HttpStatusCode.Unauthorized, StatusDescription = String.Format("'{0}' is an invalid API key", key) };
responseProp.Headers[HttpResponseHeader.ContentType] = "text/html";
reply.Properties[HttpResponseMessageProperty.Name] = responseProp;
operationContext.RequestContext.Reply(reply);
// set the request context to null to terminate processing of this request
operationContext.RequestContext = null;
}
}
}
Instead of this showing an error, the result is a blank response. Can anyone help?

How to communicate WCF exceptions to WebClient

I have a WCF web service which throws exceptions when invalid data is submitted. The data is submitted via an HTTP Post using the WebClient object.
Here is the code for the web service:
[WebInvoke(UriTemplate = "update", Method = "POST")]
public JsonValue Update(HttpRequestMessage message)
{
var context = new Entities();
dynamic response = new JsonObject();
// in order to retrieve the submitted data easily, reference the data as a dynamic object
dynamic data = message.Content.ReadAs(typeof(JsonObject), new[] { new FormUrlEncodedMediaTypeFormatter() });
// retrieve the submitted data
int requestId = data.requestId;
int statusId = data.statusId;
string user = data.user;
string encryptedToken = data.token;
string notes = data.notes;
// retrieve the request with a matching Id
var request = context.Requests.Find(requestId);
// make sure the request exists
if (request == null)
throw new FaultException("The supplied requestId does not exist.");
// make sure the submitted encrypted token is valid
var token = DecryptToken(encryptedToken);
if (token == null)
throw new FaultException("Invalid security token.");
// TODO: Validate other token properties (e.g. email)?
if (!request.User.UserName.Equals(token.UserName))
throw new FaultException("Invalid security token.");
// additional logic removed ...
}
And here is the code that submits data to the web service:
// use the WebClient object to submit data to the WCF web service
using (var client = new WebClient())
{
client.Encoding = Encoding.UTF8;
// the data will be submitted in the format of a form submission
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
var data = new NameValueCollection();
// prepare the data to be submitted
data.Add("requestId", requestId.ToString());
data.Add("statusId", this.StatusId);
data.Add("token", token.ToString());
data.Add("user", this.User);
data.Add("notes", this.Notes);
// submit the data to the web service
var response = client.UploadValues(this.Address, data);
}
I keep getting an exception with message: "The remote server returned an error: (500) Internal Server Error" at client.UploadValues(this.Address, data);.
Is there a way I can make sure that more detailed information is returned to the WebClient?
Also, how can I make sure that these exceptions (in the WCF service) are logged to the EventLog? (Basically I just need to know what happened).
Take a look at HttpResponseException (namespace Microsoft.ApplicationServer.Http.Dispatcher) - they're the way where you can control the response for error cases. You can specify the status code, and you have control over the HttpResponseMessage, in which you can control the message body.
On the client side, when you call WebClient.UploadValues, wrap that call and catch a WebException. If the service returns a response with a non-successful status code (e.g., 500, 400), the Response property of the WebException will have the body, in which you can read in your client.
Another option is to use HttpClient instead of the WebClient, in which case you can simply look at the HttpResponseMessage directly.