RestRequest.AddFileBytes posting corrupted files - restsharp

I'm trying to POST image files to a service using RestRequest.AddFileBytes, however when I do this, the file seems to be corrupted. When I download it back, the filesize is ever so slightly off, and depending on the file type, it may appear to have the wrong extension or be uninterpretable (presumably b/c the header is messed up)
When I attempt to do the same thing with Postman it file is OK, so it would seem to be something with the way I'm posting the file. Please advise!
private static bool AddPageBytes(string user, string password, string documentId, string fileName, string contentType, byte[] bytes)
{
System.Console.WriteLine($"AddPageBytes documentId:{documentId} fileName:{fileName} contentType:{contentType} bytes:{bytes.Length}");
var client = new RestClient(BASE_URL);
var request = new RestRequest("v1/document/{id}/page", Method.POST);
request.AddUrlSegment("id", documentId);
request.AddHeader("X-IntegrationServer-Username", user);
request.AddHeader("X-IntegrationServer-Password", password);
request.AddHeader("Content-Type", "application/octet-stream");
request.AddHeader("X-IntegrationServer-FileSize", bytes.Length.ToString());
request.AddHeader("X-IntegrationServer-Resource-Name", fileName);
request.AddFileBytes(fileName, bytes, fileName, contentType);
var response = client.Execute(request);
return response.IsSuccessful;
}

Woudln't you know it, like clockwork I banged my head for the better part of 2 days then posted this question and almost immediately figured out how to make it work. I wouldn't stay that I know why one is right and one is only mostly right (which is obviously not good enough) but
I just had to change
request.AddFileBytes(fileName, bytes, fileName, contentType);
to
request.AddParameter(contentType, bytes, ParameterType.RequestBody);

Related

.NET Core pdf downloader "No output formatter was found for content types 'application/pdf'..."

I'm creating a .NET Core 3.1 web api method to download a pdf for a given filename. This method is shared across teams where their client code is generated using NSwag.
I recently changed produces attribute to Produces("Application/pdf") from json, this change is required so other teams can generate valid client code. However since this change, I haven't been able to download any files from this method. Requests to download documents return with a 406 error (in Postman) and the following error is logged to the server event viewer.
No output formatter was found for content types 'application/pdf, application/pdf' to write the response.
Reverting the produced content-type to 'application/json' does allow documents to be downloaded, but as mentioned, this value is required to be pdf.
Any suggestions would be greatly appreciated.
Method:
[HttpGet("{*filePath}")]
[ProducesResponseType(typeof(FileStreamResult), StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
[ProducesResponseType(StatusCodes.Status400BadRequest)]
[ProducesResponseType(StatusCodes.Status401Unauthorized)]
[Produces("Application/pdf")]
public async Task<ActionResult> GetDocument(string fileName) {
RolesRequiredHttpContextExtensions.ValidateAppRole(HttpContext, _RequiredScopes);
var memoryStream = new MemoryStream();
var memoryStream = new MemoryStream();
using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true)) {
stream.CopyTo(memoryStream);
}
memoryStream.Seek(offset: 0, SeekOrigin.Begin);
return new FileStreamResult(memoryStream, "Application/pdf");
}
I just came across the same error and after some investigation I found out that the cause of the exception was indeed in the model binding error. You already wrote about it in your answer, but on closer inspection it became obvious that the reason was not related to binding itself, rather to the response body.
Since you specified [Produces("application/pdf")] the framework assumes this content type is the only possible for this action, but when an exception is thrown, you get application/json containing error description instead.
So to make this work for both "happy path" and exceptions, you could specify multiple response types:
[Produces("application/pdf", "application/json")]
public async Task<ActionResult> GetDocument(string fileName)
{
...
}
I'am using
public asnyc Task<IActionResult> BuildPDF()
{
Stream pdfStream = _pdfService.GetData();
byte[] memoryContent = pdfStream.ToArray();
return File(memoryContent, "application/pdf");
}
and it works. Could you please try?
The issue was caused by renaming the method parameter and not updating [HttpGet("{*filePath}")] to [HttpGet("{*fileName}")]
I had the same error, it is very confusing in some cases.
I got this error after adding new parameter of type int[] to my method forgetting [FromQuery] attribute for it.
After adding [FromQuery] attribute error gone.

Winrt StreamWriter & StorageFile does not completely Overwrite File

Quick search here yielded nothing. So, I have started using some rather roundabout ways to use StreamWriter in my WinRT Application. Reading works well, writing works differently. What' I'm seeing is that when I select my file to write, if I choose a new file then no problem. The file is created as I expect. If I choose to overwrite a file, then the file is overwritten to a point, but the point where the stream stops writing, if the original file was large, then the old contents exist past where my new stream writes.
The code is as such:
public async void WriteFile(StorageFile selectedFileToSave)
{
// At this point, selectedFileToSave is from the Save File picker so can be a enw or existing file
StreamWriter writeStream;
Encoding enc = new UTF8Encoding();
Stream dotNetStream;
dotNetStream = await selectedFileToSave.OpenStreamForWriteAsync();
StreamWriter writeStream = new StreamWriter(dotNetStream, enc);
// Do writing here
// Close
writeStream.Write(Environment.NewLine);
await writeStream.FlushAsync();
await dotNetStream.FlushAsync();
}
Can anyone offer clues on what I could be missing? There are lots of functions missing in WinRT, so not really following ways to get around this
Alternatively you can set length of the stream to 0 with SetLength method before using StreamWriter:
var stream = await file.OpenStreamForWriteAsync();
stream.SetLength(0);
using (var writer = new StreamWriter(stream))
{
writer.Write(text);
}
Why not just use the helper methods in FileIO class? You could call:
FileIO.WriteTextAsync(selectedFileToSave, newTextContents);
If you really need a StreamWriter, first truncate the file by calling
FileIO.WriteBytesAsync(selectedFileToSave, new byte[0]);
And then continue with your existing code.

Crawl Wikipedia using ASP.NET HttpWebRequest

I am new to Web Crawling, and I am using HttpWebRequest to crawl data from sites.
As of now I was successfully able to crawl and get data from my wordpress site. This data was a simple user profile data. (like name, email, AIM id etc...)
Now as an exercise I want to crawl wikipedia, where I will search using the value entered into textbox at my end and then crawl wikipedia with the search value and get the appropriate title(s) from the search.
Now I have the following doubts/difficulties.
Firstly, is this even possible ? I have heard that wiki has robot.txt setup to block this. Though I have heard this only from a friend and hence not sure.
I am using the same procedure I used earlier, but I am not getting the required results.
Thanks !
Update :
After some explanation and help from #svick, I tried the below code, but still not able to get any value (see last line of code, there I am expecting an html markup of the search result page)
string searchUrl = "http://en.wikipedia.org/w/index.php?search=Wikipedia&title=Special%3ASearch";
var postData = new StringBuilder();
postData.Append("search=" + model.Query);
postData.Append("&");
postData.Append("title" + "Special:Search");
byte[] data2 = Crawler.GetEncodedData(postData.ToString());
var webRequest = (HttpWebRequest)WebRequest.Create(searchUrl);
webRequest.Method = "POST";
webRequest.UserAgent = "Crawling HW (http://yassershaikh.com/contact-me/)";
webRequest.AllowAutoRedirect = false;
ServicePointManager.Expect100Continue = false;
Stream requestStream = webRequest.GetRequestStream();
requestStream.Write(data2, 0, data2.Length);
requestStream.Close();
var responseCsv = (HttpWebResponse)webRequest.GetResponse();
Stream response = responseCsv.GetResponseStream();
// Todo Parsing
var streamReader = new StreamReader(response);
string val = streamReader.ReadToEnd();
// val is empty !! <-- this is my problem !
and here is my GetEncodedData method defination.
public static byte[] GetEncodedData(string postData)
{
var encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
return data;
}
Pls help me on this.
You probably don't need to use HttpWebRequest. Using WebClient (or HttpClient if you're on .Net 4.5) will be much easier for you.
robots.txt doesn't actually block anything. If something doesn't support it (and .Net doesn't support it), it can access anything.
Wikipedia does block requests that don't have their User-Agent header set. And you should use an informative User-Agent string with your contact information.
A better way to access Wikipedia is to use its API, rather than scraping. This way, you will get an answer that's specifically meant to be read by a custom applications, formatted as XML or JSON. There are also dumps containing all information from Wikipedia available for download.
EDIT: The problem with your newly posted code is that your query returns a 302 Moved Temporarily response to the searched article, if it exists. Either remove the line that forbids AllowAutoRedirect, or add &fulltext=Search to your query, which will mean you won't get redirected.

vb- networkstream write only returns first result (need all)

My code looks very similar to this post Read bytes from NetworkStream (Hangs), which I copied below. ( I realize this is c# - I need a vb solution)
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
TcpClient client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close(); client.Close();
My issue lies in this:
I have a form that takes user input on which tif file(s) to find (which ends up being the getBytes(message) ). When one does this, it always returns one result - the first tif file that fits the criteria, However, I know in certain instances I should recieve more than one match.
Then I send the results to a picturebox and should be able to scroll through results (which this part works fine)
I've tried a few ways to get multiple results, but maybe I'm missing the obvious? My best guess is using the asyncronous beginread/write....
I've attempted using a for loop but I end up getting a bunch of the same tif files as a result...
Could anyone help me (even with a generic direction)? I'm not a pro. Thanks in advance for the help.
I ended up finding out what the issue was. the data i was looking for wasn't standardized as so the results were the same data over and over. I changed what items I was looking for and tada it worked. anyway, thanks for the help

Why am I getting System.FormatException: String was not recognized as a valid Boolean on a fraction of our customers machines?

Our c#.net software connects to an online app to deal with accounts and a shop. It does this using HttpWebRequest and HttpWebResponse.
An example of this interaction, and one area where the exception in the title has come from is:
var request = HttpWebRequest.Create(onlineApp + string.Format("isvalid.ashx?username={0}&password={1}", HttpUtility.UrlEncode(username), HttpUtility.UrlEncode(password))) as HttpWebRequest;
request.Method = "GET";
using (var response = request.GetResponse() as HttpWebResponse)
using (var ms = new MemoryStream())
{
var responseStream = response.GetResponseStream();
byte[] buffer = new byte[4096];
int read;
do
{
read = responseStream.Read(buffer, 0, buffer.Length);
ms.Write(buffer, 0, read);
} while (read > 0);
ms.Position = 0;
return Convert.ToBoolean(Encoding.ASCII.GetString(ms.ToArray()));
}
The online app will respond either 'true' or 'false'. In all our testing it gets one of these values, but for a couple of customers (out of hundreds) we get this exception System.FormatException: String was not recognized as a valid Boolean Which sounds like the response is being garbled by something. If we ask them to go to the online app in their web browser, they see the correct response. The clients are usually on school networks which can be fairly restrictive and often under proxy servers, but most cope fine once they've put the proxy details in or added a firewall exception. Is there something that could be messing up the response from the server, or is something wrong with our code?
Indeed, it's possible that the return result is somehow different.
Is there any particular reason you are doing the reasonably elaborate method of reading the repsonse there? Why not:
string data;
using(HttpWebResponse response = request.GetResponse() as HttpWebResponse){
StreamReader str = new StreamReader(response.GetResponseStream());
data = str.ReadToEnd();
str.Close();
}
string cleanResult = data.Trim().ToLower();
// log this
return Convert.ToBoolean(cleanResult);
First thing to note is I would definitely use something like:
bool myBool = false;
Boolean.TryParse(Encoding.ASCII.GetString(ms.ToArray()), myBool);
return myBool;
It's not some localisation issue is it? It's expecting the Swahili version of 'true', and getting confused. Are all the sites in one country, with the same language, etc?
I'd add logging, as suggested by others, and see what results you're seeing.
I'd also lean towards changing the code as silky suggested, though with a few further changes from me (code 'smell' issues, IMO); Use using around the stream reader, as well as the response.
Also, I don't think the use of as is appropriate in this instance. If the Response can't be cast to HttpWebResponse (which, admittedly is unlikely, but still) you'll get a NullRef exception on the response.GetResponseStream() bit which is both a vague error, and you've lost the original line number. Using (HttpWebResponse)request.GetResponse() will give you a more correct error, and the correct line number of the actual error.