The ADLS .NET SDK has some good examples for reading and creating text files. This uses StreamReader and this shouldn't be used with binary files. I tried using BinaryReader but have been unsuccessful.
https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-net-sdk
//Read file contents
using (var readStream = new StreamReader(client.GetReadStream(fileName)))
{
string line;
while ((line = readStream.ReadLine()) != null)
{
Console.WriteLine(line);
}
}
Can the .NET SDK create/read binary? If so, are there any examples of doing this?
Can the .NET SDK create/read binary? If so, are there any examples of doing this?
Short answer is yes, please refer to the following demo code.
Create binary File
AdlsClient adlsClient = AdlsClient.CreateClient($"{datalakeAccount}.azuredatalakestore.net", clientCreds);
using (var stream = adlsClient.CreateFile("file name", IfExists.Overwrite))
{
byte[] textByteArray = File.ReadAllBytes(#"local file path");
stream.Write(textByteArray, 0, textByteArray.Length);
}
read binary file and write to local file
using (var filesream = adlsClient.GetReadStream("1.png"))
{
MemoryStream memorystream = new MemoryStream();
filesream.CopyTo(memorystream);
memorystream.Position = 0;
File.WriteAllBytes(#"filename", memorystream.ToArray());
}
Related
I have a filename pointing to a text file, including its path, as a string. Now I'd like to load this .csv file into memory stream. How should I do that?
For example, I have this:
Dim filename as string="C:\Users\Desktop\abc.csv"
Dim stream As New MemoryStream(File.ReadAllBytes(filename))
You don't need to load a file into a MemoryStream.
You can simply call File.OpenRead to get a FileStream containing the file.
If you really want the file to be in a MemoryStream, you can call CopyTo to copy the FileStream to a MemoryStream.
I had an XML file being read from disk, using the old XmlReader API. How to read the XML file into memory, and then work with it in memory, instead of reading the disk repeatedly? Based on VB answer from Centro (upvoted) but with a Using block, and in C#.
The key line:
MemoryStream myXMLDocument = new MemoryStream(File.ReadAllBytes(#"c:\temp\myDemoXMLDocument.xml"));
Re the OP's question, if you wanted to load a CSV file into a MemoryStream:
MemoryStream myCSVDataInMemory = new MemoryStream(File.ReadAllBytes(#"C:\Users\Desktop\abc.csv"));
Following is a code snippet showing code to reads through XML document now that it's in a MemoryStream. Basically the same code as when it was coming from a FileStream that pointed to a file on disk. Yes, the XMLTextReader API is old and clunky, but it's what I had to work with in this app.
string myXMLFileName = #"c:\temp\myDemoXMLDocument.xml";
using (MemoryStream myXMLDocument = new MemoryStream(File.ReadAllBytes(myXMLFileName)))
{
myXMLTextReader = new XmlTextReader(myXMLDocument);
myXMLTextReader.WhitespaceHandling = WhitespaceHandling.None;
myXmlTextReader.Read(); // read the XML declaration node, advance to <Batch> tag
while (!myXmlTextReader.EOF)
{
if (myXmlTextReader.Name == "xml" && !myXmlTextReader.IsStartElement()) break;
// advance to <Batch> tag
while (myXmlTextReader.Name == "Batch" && myXmlTextReader.IsStartElement())
{
string BatchIdentifier = myXmlTextReader.GetAttribute("BatchIdentifier");
myXmlTextReader.Read(); // advance to next tag
while (!myXmlTextReader.EOF)
{
if (myXmlTextReader.Name == "Transaction" && myXmlTextReader.IsStartElement())
{
// Start a new set of items
string transactionID = myXmlTextReader.GetAttribute("ID");
myXmlTextReader.Read(); // Read next element, possibly another Transaction tag
}
}
//All Batch tags are completed.Move to next tag
myXmlTextReader.Read();
}
// Close the XML memory stream.
myXmlTextReader.Close();
myXmlDocument.Close();
}
}
You can copy it to a file stream like so:
string fullPath = Path.Combine(filePath, fileName);
FileStream fileStream = new FileStream(fullPath, FileMode.Open);
Image image = Image.FromStream(fileStream);
MemoryStream memoryStream = new MemoryStream();
image.Save(memoryStream, ImageFormat.Jpeg);
//Close File Stream
fileStream.Close();
I have a .NET Core API that I'd like to extend to save uploaded images asynchronously.
Using ImageSharp I should be able to check uploads and resize if predefined size limits are exceeded. However I can't get a simple async save working.
A simple (non-async) save to file works without problem:
My Controller extracts IFormFile from the upload and calls the following method without any problem
public static void Save(IFormFile image, string imagesFolder)
{
var fileName = Path.Combine(imagesFolder, image.FileName);
using (var stream = image.OpenReadStream())
using (var imgIS = Image.Load(stream, out IImageFormat format))
{
imgIS.Save(fileName);
}
}
ImageSharp is currently lacking async methods so a workaround is necessary.
The updated code below saves the uploaded file but the format is incorrect - when viewing the file I get the message "It appears we don't support this file format".
The format is extracted from the ImageSharp Load method. and used when saving to MemoryStream.
MemoryStream CopyToAsync method is used to save to FileStream to make the upload asynchronous.
public static async void Save(IFormFile image, string imagesFolder)
{
var fileName = Path.Combine(imagesFolder, image.FileName);
using (var stream = image.OpenReadStream())
using (var imgIS = Image.Load(stream, out IImageFormat format))
using (var memoryStream = new MemoryStream())
using (var fileStream = new FileStream(fileName, FileMode.OpenOrCreate))
{
imgIS.Save(memoryStream, format);
await memoryStream.CopyToAsync(fileStream).ConfigureAwait(false);
fileStream.Flush();
memoryStream.Close();
fileStream.Close();
}
}
I can't work out whether the issue is with ImageSharp Save to MemoryStream, or the MemoryStream.CopyToAsync.
I'm currently getting 404 on SixLabors docs - hopefully not an indication that the project has folded.
How can I make the upload async and save to file in the correct format?
CopyToAsync copies a stream starting at its current position. You must change the current position of memoryStream back to start before copying:
// ...
memoryStream.Seek(0, SeekOrigin.Begin);
await memoryStream.CopyToAsync(fileStream).ConfigureAwait(false);
// ...
I have a mvc 4.5 application where I show a grid. The first column of the grid is a document name. The document name is an hyper link to the actual document that is hosted on our site and is available via a url. The documents can be pdf or doc or ppt. I can access these documents only via url and I do not have access to the actual physical document on our server.
I am providing users an option to select one or many of these documents from the grid and then they can download them. What I am trying to achieve is read each of the selected documents via the url and write it to a zip file and make the zip file downloadable. So users will be downloading one file instead of multiple files.
I have tried to stream the documents via url in memory and then add it to the zip file using ZipArchive Library from Microsoft. This is not working for me.
I was able to add documents that was on disk to zip file using Zip Archive and it works great. But I do not have access to the physical document as I can access the documents only through URL. My next option is to download each of these documents into a temp location on server and then add it to zip file using Zip Archive.But I am trying to avoid downloading files into a temp location
Please suggest how I can achieve reading documents via url in memory and adding each of these document to zip file and make zip file downloadable.
Any help will be appreciated.
Thank you Cbroe for commenting. I figured the answer. The problem was I was reading the pdf from the url and convert it to a memory stream and then was trying to add the memory stream to ZipArchive which was not working but instead I extracted the byte array out of the memory stream and then added it to the zip archive and it worked.
Here is the code snippet that might be useful for some one. My first contribution to Stack OverFlow.
public FileResult DownloadZip()
{
MemoryStream memoryStream = new MemoryStream();
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("Pdf123.pdf");
var convertedStream = ConvertTobyte("http://www.example.com/Pdf123.pdf");
using (var entryStream = demoFile.Open())
{
entryStream.Write(convertedStream, 0, convertedStream.Length);
}
demoFile = archive.CreateEntry("Pdf456.pdf");
convertedStream = ConvertTobyte("http://www.example.com/Pdf456.pdf");
using (var entryStream = demoFile.Open())
{
entryStream.Write(convertedStream, 0, convertedStream.Length);
}
}
//This option is to write the zip to your local disk
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
//This option is to donload the zip via browser
memoryStream.Seek(0, SeekOrigin.Begin);
return new FileStreamResult(memoryStream, "application/zip")
{
FileDownloadName = "Archive.zip"
};
}
private static byte[] ConvertTobyte(string fileUrl)
{
byte[] imageData = null;
using (var wc = new System.Net.WebClient())
imageData = wc.DownloadData(fileUrl);
return imageData;
}
I'm a bit new to WinRT developing platform, and it's already driving me crazy (I'm a long-time .Net developer, and all those removed APIs are quite annoying)
I'm experiencing a problem while zipping all files present in the Windows.Storage.ApplicationData.Current.TemporaryFolder
Here is my current code (VB.Net, based on MSDN code, and "file" is the zip file I'll put all my files into) :
Using zipMemoryStream As New MemoryStream()
Using zipArchive As New Compression.ZipArchive(zipMemoryStream, Compression.ZipArchiveMode.Create)
For Each fileToCompress As Windows.Storage.StorageFile In (Await Windows.Storage.ApplicationData.Current.TemporaryFolder.GetFilesAsync())
Dim buffer As Byte() = WindowsRuntimeBufferExtensions.ToArray(Await Windows.Storage.FileIO.ReadBufferAsync(fileToCompress))
Dim entry As ZipArchiveEntry = zipArchive.CreateEntry(fileToCompress.Name)
Using entryStream As Stream = entry.Open()
Await entryStream.WriteAsync(buffer, 0, buffer.Length)
End Using
Next
End Using
Using zipStream As Windows.Storage.Streams.IRandomAccessStream = Await file.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite)
Using outstream As Stream = zipStream.AsStreamForWrite()
Dim buffer As Byte() = zipMemoryStream.ToArray()
outstream.Write(buffer, 0, buffer.Length)
outstream.Flush()
End Using
End Using
End Using
It builds well, but when I launch the code, I have the exception :
UnauthorizedAccessException : Access denied. (Exception de HRESULT : 0x80070005 (E_ACCESSDENIED))
On line : WindowsRuntimeBufferExtensions.ToArray(blahblah...
I'm wondering what is wrong. Any idea ?
Thanks in advance !
I rewrote your method in C# to try it out:
var file = await ApplicationData.Current.LocalFolder.CreateFileAsync("test.zip");
using (var zipMemoryStream = new MemoryStream())
{
using (var zipArchive = new System.IO.Compression.ZipArchive(zipMemoryStream, System.IO.Compression.ZipArchiveMode.Create))
{
foreach (var fileToCompress in (await ApplicationData.Current.TemporaryFolder.GetFilesAsync()))
{
var buffer = WindowsRuntimeBufferExtensions.ToArray(await FileIO.ReadBufferAsync(fileToCompress));
var entry = zipArchive.CreateEntry(fileToCompress.Name);
using (var entryStream = entry.Open())
{
await entryStream.WriteAsync(buffer, 0, buffer.Length);
}
}
}
using ( var zipStream = await file.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite))
{
using (var outstream = zipStream.AsStreamForWrite())
{
var buffer = zipMemoryStream.ToArray();
outstream.Write(buffer, 0, buffer.Length);
outstream.Flush();
}
}
}
It works flawlessly - it creates the zip file in local folder as expected. Since you get the exception in ToArray call, the reason could be that the file you're trying to open is already locked from somewhere else. If you are creating these files yourself or even only accessing them, make sure you're closing the streams.
To test this method you could manually create a folder inside temp folder, put a couple of files in it and then run the method on that folder (the files are in C:\Users\<Username>\AppData\Local\Packages\<PackageName>\TempState) just to exclude any other reason for error.
I've been trying to read a pre-built file with Car Maintenance tips, there's one in each line of my "Tips.txt" file. I've tried to follow around 4 or 5 different approaches but It's not working, it compiles but I get an exception. Here's what I've got:
using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
{
using (StreamReader sr = new StreamReader(store.OpenFile("Tips.txt", FileMode.Open, FileAccess.Read)))
{
string line;
while ((line = sr.ReadLine()) != null)
{
(App.Current as App).MyTips.Insert(new DoubleNode(line));
}
}
}
I'm getting this "Operation not permitted on IsolatedStorageFileStream", from the info inside the 2nd using statement. I tried with the build action of my "Tips.txt" set to resource, and content, yet I get the same result.
Thanks in advance.
Since you've added it to your project directory, you can't read it using Isolated Storage methods. There are various ways you can load the file. One way would be to set the text file's build type to Resource, then read it in as a stream:
//Replace 'MyProject' with the name of your XAP/Project
Stream txtStream = Application.GetResourceStream(new Uri("/MyProject;component/myTextFile.txt",
UriKind.Relative)).Stream;
using(StreamReader sr = new StreamReader(txtStream))
{
//your code
}