I had a question when using android-async-http. After reading the source code, I knew about how to add a File or InputStream as a parameter into RequestParam. Then the RequestParam would be transferred into an AsyncHttpClient which would use RequestParam to get/put/post....Just like this:
String url = ...;
File file = ...;
ResponseHandlerInterface respHandler = ...;
AsyncHttpClient client = new AsyncHttpClient();
RequestParams params = new RequestParams();
params.add("upload_file", file);
client.get(url, params, respHandler);
As we all know, Files of any types are bits essentially. So when delivered over the internet, files could be transferred into a byte stream. But I didn't find any codes about this conversion. So, I wonder how android-async-http completes this, or did I miss something when reading source codes?
I thought I found the way android-async-http handled with files/inputstreams. Uploading a file depends on the call of put(?)/post(?), but not get(?). By searching the overwrited methods of put(?)/post(?), you will find paramsToEntity(RequestParams, ResponseHandlerInterface) which will return a HttpEntity. And then, HttpPost/HttpPut will setEntity(HttpEntity). Because get(?)s don't support uploading files, then you can't find operations about uploading files in get(?)s.
Related
I'm trying to get a large CSV file from S3 but the download fails with “java.net.SocketException: Connection reset”, which is probably due to the InputStream simply being open for too long (the download often takes more than an hour since I am doing multiple time-consuming processes on the streamed content). This is how I currently parse the file:
InputStream inputStream = new GZIPInputStream(s3Client.getObject("bucket", "key").getObjectContent());
Reader decoder = new InputStreamReader(inputStream, Charset.defaultCharset());
BufferedReader isr = new BufferedReader(decoder);
CSVParser csvParser = new CSVParser(isr, CSVFormat.DEFAULT);
CSVRecord nextRecord = csvParser.iterator().next();
...
I know I have to split the download into multiple short getObject-calls with a defined offset for the GetObjectRequest, but I'm wondering how to define this offset in case of a CSV, since I need complete lines.
Do I have to ditch the parser library and parse each line into an Object myself so I can keep a count of the read bytes and use it as an offset for the next batch? That doesn't seem very robust to me. Is there any best practice way to achieve "batch downloading" of CSV records?
I decided on simply using the dedicated getObject(GetObjectRequest getObjectRequest, File destinationFile) method to copy the entire CSV to a temporary file on disk. This closes the HTTP connection as soon as possible and allows me to get the InputStream from the local file with no problems. It doesn't resolve the question of the best way to download in batches, but it's a nice and simple workaround.
I am new to S3 and need to use it for image storage. I found a half dozen versions of an s2wrapper for cf but it appears that the only one set of for v4 is one modified by Leigh
https://gist.github.com/Leigh-/26993ed79c956c9309a9dfe40f1fce29
Dropped in the com directory and created a "test" page that contains the following code:
s3 = createObject('component','com.S3Wrapper').init(application.s3.AccessKeyId,application.s3.SecretAccessKey);
but got the following error :
So I changed the line 37 from
variables.Sv4Util = createObject('component', 'Sv4').init(arguments.S3AccessKey, arguments.S3SecretAccessKey);
to
variables.Sv4Util = createObject('component', 'Sv4Util').init(arguments.S3AccessKey, arguments.S3SecretAccessKey);
Now I am getting:
I feel like going through Leigh code and start changing things is a bad idea since I have lurked here for year an know Leigh's code is solid.
Does any know if there are any examples on how to use this anywhere? If not what I am doing wrong. If it makes a difference I am using Lucee 5 and not Adobe's CF engine.
UPDATE :
I followed Leigh's directions and the error is now gone. I am addedsome more code to my test page which now looks like this :
<cfscript>
s3 = createObject('component','com.S3v4').init(application.s3.AccessKeyId,application.s3.SecretAccessKey);
bucket = "imgbkt.domain.com";
obj = "fake.ping";
region = "s3-us-west-1"
test = s3.getObject(bucket,obj,region);
writeDump(test);
test2 = s3.getObjectLink(bucket,obj,region);
writeDump(test2);
writeDump(s3);
</cfscript>
Regardless of what I put in for bucket, obj or region I get :
JIC I did go to AWS and get new keys:
Leigh if you are still around or anyone how has used one of the s3Wrappers any suggestions or guidance?
UPDATE #2:
Even after Alex's help I am not able to get this to work. The Link I receive from getObjectLink is not valid and getObject never does download an object. I thought I would try the putObject method
test3 = s3.putObject(bucketName=bucket,regionName=region,keyName="favicon.ico");
writeDump(test3);
to see if there is any additional information, I received this :
I did find this article https://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html but it is pretty old and since S3 specifically suggests using dots in bucketnames I don't that it is relevant any longer. There is obviously something I am doing wrong but I have spent hours trying to resolve this and I can't seem to figure out what it might be.
I will give you a rundown of what the code does:
getObjectLink returns a HTTP URL for the file fake.ping that is found looking in the bucket imgbkt.domain.com of region s3-us-west-1. This link is temporary and expires after 60 seconds by default.
getObject invokes getObjectLink and immediately requests the URL using HTTP GET. The response is then saved to the directory of the S3v4.cfc with the filename fake.ping by default. Finally the function returns the full path of the downloaded file: E:\wwwDevRoot\taa\fake.ping
To save the file in a different location, you would invoke:
downloadPath = 'E:\';
test = s3.getObject(bucket,obj,region,downloadPath);
writeDump(test);
The HTTP request is synchronous, meaning the file will be downloaded completely when the functions returns the filepath.
If you want to access the actual content of the file, you can do this:
test = s3.getObject(bucket,obj,region);
contentAsString = fileRead(test); // returns the file content as string
// or
contentAsBinary = fileReadBinary(test); // returns the content as binary (byte array)
writeDump(contentAsString);
writeDump(contentAsBinary);
(You might want to stream the content if the file is large since fileRead/fileReadBinary reads the whole file into buffer. Use fileOpen to stream the content.
Does that help you?
I have a WCF written in vb.net that I want to return a zip file from. Now I can get that working, but I can't figure out how to set the suggested name the file should be saved as (it defaults to the Operation Name with no extension). When I save the file and then rename it with a .zip extension it is the zip file I expect.
So I'd like to be able to set the file extension, I guess I'm not that bothered by the file name.
Here is my example code:
<WebGet(UriTemplate:="Export/", ResponseFormat:=WebMessageFormat.Xml, BodyStyle:=WebMessageBodyStyle.Bare)>
Public Function ModelVersionExport() As System.IO.Stream
WebOperationContext.Current.OutgoingResponse.ContentType = "application/octet-stream"
'then I call my code that creates the io.stream of the zip file (that works fine)
End Function
I thought maybe I could set something on the OutGoingResponse but nothing looks right.
This fixes the issue:
WebOperationContext.Current.OutgoingResponse.ContentType = "application/zip"
Should have been fairly obvious but all examples I'd seen used the WebOperationContext.Current.OutgoingResponse.ContentType = "application/octet-stream" setting
So, using vb.net, I retrieve from my server the byte data for a file that the user wishes to download. I always know what the filename and extension is, but what I don't know is how to start downloading the byte data and in the proper file format. How do I got about doing this?
EDIT: Just to clarify, I already retrieve the data in byte format in code, I just need to download it as the proper file type which is also known. I'm keeping the URL to the file hidden at all times so it's never exposed.
If you want to download the file directly to the hard drive, the easiest solution is to use WebClient.DownloadFile. The MSDN page contains a nice example.
If you want to put the file into a byte array instead of a file on disk, use WebClient.DownloadData instead:
Dim myWebClient As New WebClient()
Dim myByteArray = myWebClient.DownloadData("http://...")
Again, a larger example can be found on the MSDN page.
If you want your program to stay responsive while downloading, check out the asynchronous versions of those methods.
EDIT: I'm still having a hard time understanding your situation, but it you already have a byte array and just want to write it to the disk, you can use File.WriteAllBytes:
File.WriteAllBytes("C:\my\path\myfile.bin", myByteArray)
Okay, I figured it out. Using BinaryWrite with the other Response functions like AddHeader and ContentType I got it to work. GetMimeType is a function I made. Code below:
Response.Clear()
Response.AddHeader("Content-Disposition", "attachment; filename=" + FileName)
Response.ContentType = GetMimeType(FileName)
Response.BinaryWrite(data)
Response.End()
Response.Flush()
Thanks to those who tried to help!
I'm trying to write/create a JSON file from a AIR app, I'm trying not so show a 'Save as' dialogue box.
Here's the code I'm using:
var fileDetails:Object = CreativeMakerJSX.getFileDetails();
var fileName:String = String(fileDetails.data.filename);
var path:String = String(fileDetails.data.path);
var f:File = File.userDirectory.resolvePath( path );
var stream:FileStream = new FileStream();
stream.open(f, FileMode.WRITE );
stream.writeUTFBytes( jsonToExport );
stream.close();
The problem I'm having is that I get a 'Error 3013. File or directory in use'. The directory/path is gathered from a Creative Suite Extension I'm building, this path is the same as the FLA being developed in CS that the Extension is being used with.
So I'm not sure if the problem is that there are already files in the directory I'm writing the JSON file to?
Do I need to add a timer in order to close the stream after a slight delay, giving some time to writing the file?
Can you set up some trace() commands? I would need to know what the values of the String variables are, and the f.url.
Can you read from the file that you are trying to write to, or does nothing work?
Where is CreativeMakerJSX.getFileDetails() coming from? Is it giving you data about a file that is in use?
And from Googling around, this seems like it may be a bug. Try setting up a listener for when you are finished, if you have had the file open previously.
I re-wrote how the file was written, no longer running into this issue.