Regarding file upload in play framework 2.2.3.
According to update on this question,
adding the following to application.conf should enable upload of files upto 10MB.
parsers.MultipartFormData.maxLength = 10240K
this does not work for me. I get "413, Request entity too large" error code for any file greater than 1MB.
I tried setting another field
parsers.text.maxLength=10M
Still upload fails with 413.
I upload files using XHR with a FormData that a can contain multiple files.
Upload Controller Code :
public Result uploadAttendeeFiles(){
try{
MultipartFormData body = request().body().asMultipartFormData();
List<FilePart> uploadedFiles = body.getFiles();
Map<String, String> returnMessages = new HashMap<String, String>();
String fileName ="", fileExtension="", fieldId = "";
int fileCounter = 0;
if (!CommonUtils.isEmpty(uploadedFiles)) {
for (FilePart filePar : uploadedFiles) {
try {
fileExtension = Files.getFileExtension(filePar.getFilename());
fieldId = body.asFormUrlEncoded().get("fieldId")[fileCounter];
fileName = body.asFormUrlEncoded().get("fileName")[fileCounter++];
InputStream in = new FileInputStream(filePar.getFile());
Object objectId = uploadService.loadFile(in, fileName, fileExtension);
if(objectId != null)
returnMessages.put(fieldId, objectId.toString());
else
returnMessages.put(fieldId, "failed-Failed to save file!");
} catch (IOException e) {
returnMessages.put(fileName+"."+fileExtension, "failed-Error while uploading file!");
e.printStackTrace();
}
}
}
else{
return ok("{\"errormessage\":\"No files selected!\"}");
}
return ok(Json.toJson(returnMessages));
} catch(Exception e) {
e.printStackTrace();
return ok("{\"errormessage\":\"Error while uploading files!\"}");
}
}
Related
I'm trying to save pdf in wildfly, I'm using RestEasy MultipartFormDataInput provided with wildfly 20.0.1,
but it doesn't work.
This is what I have:
public static Response uploadPdfFile(MultipartFormDataInput multipartFormDataInput) {
// local variables
MultivaluedMap<String, String> multivaluedMap = null;
String fileName = null;
InputStream inputStream = null;
String uploadFilePath = null;
try {
Map<String, List<InputPart>> map = multipartFormDataInput.getFormDataMap();
List<InputPart> lstInputPart = map.get("poc");
for(InputPart inputPart : lstInputPart){
// get filename to be uploaded
multivaluedMap = inputPart.getHeaders();
fileName = getFileName(multivaluedMap);
if(null != fileName && !"".equalsIgnoreCase(fileName)){
try {
// write & upload file to UPLOAD_FILE_SERVER
//here I have the error: Unable to find a MessageBodyReader for media type:
//application/pdf
inputStream = inputPart.getBody(InputStream.class,InputStream.class);
uploadFilePath = writeToFileServer(inputStream, fileName);
}catch (Exception e) {
e.printStackTrace();
}
// close the stream
inputStream.close();
}
}
}
catch(IOException ioe){
ioe.printStackTrace();
}
finally{
// release resources, if any
}
return Response.ok("File uploaded successfully at " + uploadFilePath).build();
}
I'm using postman for test, http POST method, in the body I send: form-data - file and selected the file.pdf.
When I sent the request, I have the next RunTimeException when I try:
inputStream = inputPart.getBody(InputStream.class,null);
I get:
java.lang.RuntimeException: RESTEASY007545: Unable to find a MessageBodyReader for media type: application/pdf and class type org.jboss.resteasy.util.Base64$InputStream
At the moment I am saving the file receiving it in Base64, but I think that with MultipartFormDataInput it is the correct way.
This is what I have when debug:
Thanks for your support.
I solved this changing the InputStream from "org.jboss.resteasy.util.Base64.InputStream"
to "java.io.InputStream"
I use the code stated here to upload files through a webapi http://bartwullems.blogspot.pe/2013/03/web-api-file-upload-set-filename.html. I also made the following api to list all the files I have :
[HttpPost]
[Route("sharepoint/imageBrowser/listFiles")]
[SharePointContextFilter]
public async Task<HttpResponseMessage> Read()
{
string pathImages = HttpContext.Current.Server.MapPath("~/Content/images");
DirectoryInfo d = new DirectoryInfo(pathImages);//Assuming Test is your Folder
FileInfo[] Files = d.GetFiles(); //Getting Text files
List<object> lst = new List<object>();
foreach (FileInfo f in Files)
{
lst.Add(new
{
name = f.Name,
type = "f",
size = f.Length
});
}
return Request.CreateResponse(HttpStatusCode.OK, lst);
}
When calling this api, all the files uploaded are listed. But when I go to azure I dont see any of them (Content.png is a file I manually uploaded to azure)
Why are the files listed if they dont appear on azure.
According to your description, I suggest you could firstly use azure kudu console to locate the right folder in the azure web portal to see the image file.
Open kudu console:
In the kudu click the debug console and locate the site\wwwroot\yourfilefolder
If you find your file is still doesn't upload successfully, I guess there maybe something wrong with your upload codes. I suggest you could try below codes.
Notice: You need add image folder in the wwwort folder.
{
public class UploadingController : ApiController
{
public async Task<HttpResponseMessage> PostFile()
{
// Check if the request contains multipart/form-data.
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = Environment.GetEnvironmentVariable("HOME").ToString() + "\\site\\wwwroot\\images";
//string root = HttpContext.Current.Server.MapPath("~/images");
var provider = new FilenameMultipartFormDataStreamProvider(root);
try
{
StringBuilder sb = new StringBuilder(); // Holds the response body
// Read the form data and return an async task.
await Request.Content.ReadAsMultipartAsync(provider);
// This illustrates how to get the form data.
foreach (var key in provider.FormData.AllKeys)
{
foreach (var val in provider.FormData.GetValues(key))
{
sb.Append(string.Format("{0}: {1}\n", key, val));
}
}
// This illustrates how to get the file names for uploaded files.
foreach (var file in provider.FileData)
{
FileInfo fileInfo = new FileInfo(file.LocalFileName);
sb.Append(string.Format("Uploaded file: {0} ({1} bytes)\n", fileInfo.Name, fileInfo.Length));
}
return new HttpResponseMessage()
{
Content = new StringContent(sb.ToString())
};
}
catch (System.Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e);
}
}
}
public class FilenameMultipartFormDataStreamProvider : MultipartFormDataStreamProvider
{
public FilenameMultipartFormDataStreamProvider(string path) : base(path)
{
}
public override string GetLocalFileName(System.Net.Http.Headers.HttpContentHeaders headers)
{
var name = !string.IsNullOrWhiteSpace(headers.ContentDisposition.FileName) ? headers.ContentDisposition.FileName : Guid.NewGuid().ToString();
return name.Replace("\"", string.Empty);
}
}
}
Result:
I am trying to download data Bigquery data through Google cloud storage. Am able to send data from BigQuery to GCS but when downloading data from GCS to load the files are corrupted randomly.
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);
out = fs.create(pathDir, true);
getObject.executeMediaAndDownloadTo(out);
boolean match= ismd5HashValid(o.getMd5Hash(), pathDir);
and to check md5 checksum
private boolean ismd5HashValid(String md5hash, String path) {
org.apache.hadoop.fs.Path pathDir = new org.apache.hadoop.fs.Path(path);
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
InputStream is = null;
try {
FileSystem fs = FileSystem.get(conf);
MessageDigest md = MessageDigest.getInstance("MD5");
is = fs.open(pathDir);
byte[] bytes = new byte[1024];
int numBytes;
while ((numBytes = is.read(bytes)) != -1) {
md.update(bytes, 0, numBytes);
}
byte[] digest = md.digest();
String result = new String(Base64.encodeBase64(digest));
Log.info("Source file md5hash {} Downloaded file md5hash {}", md5hash, result);
if (md5hash.equals(result)) {
Log.info("md5hash check is valid");
return true;
}
} catch (IOException e) {
// TODO Auto-generated catch block
Log.warn(e.getMessage(), e);
} catch (NoSuchAlgorithmException e) {
// TODO Auto-generated catch block
Log.warn(e.getMessage(), e);
} finally {
IOUtils.closeQuietly(is);
}
return false;
}
I am trying to set up file upload example using JAX RS. I could set up the project and successfully upload file in a server location. But i get the following error when file size is more than 10KB (weird!!)
com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:9090/DOAFileUploader/rest/file/upload returned a response status of 400
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:607)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:507)
at com.sony.doa.rest.client.DOAClient.upload(DOAClient.java:75)
at com.sony.doa.rest.client.DOAMain.main(DOAMain.java:34)
I am new to JAX RS and i'm not sure what exactly the issue is. Do i need to set some parameters client side or server side (like size, timeout etc)?
This is the client side code calling webservice:
public void upload() {
File file = new File(inputFilePath);
FormDataMultiPart part = new FormDataMultiPart();
part.bodyPart(new FileDataBodyPart("file", file, MediaType.APPLICATION_OCTET_STREAM_TYPE));
WebResource resource = Client.create().resource(url);
String response = resource.type(MediaType.MULTIPART_FORM_DATA_TYPE).post(String.class, part);
System.out.println(response);
}
This is the server side code:
#Path("/file")
public class UploadFileService {
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(
#FormDataParam("file") InputStream uploadedInputStream,
#FormDataParam("file") FormDataContentDisposition fileDetail) {
String uploadedFileLocation = "e://uploaded/"
+ fileDetail.getFileName();
writeToFile(uploadedInputStream, uploadedFileLocation);
String output = "File uploaded to : " + uploadedFileLocation;
return Response.status(200).entity(output).build();
}
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(
uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[16000];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
} } }
Please let me know what settings i have to change for file sizes greater than 10KB?
Thanks!
I use org.apache.commons.fileupload.servlet.ServletFileUpload in a Jersey context, and it works fine., and yes, it set the max file size, sorry I missed this before.
here is a snipet of code I use (this is a multipart form, so there are other fields along with the file)
private LibraryUpload parseLibraryUpload(HttpServletRequest request) {
LibraryUpload libraryUpload;
File libraryZip = null;
String name = null;
String version = null;
ServletFileUpload upload = new ServletFileUpload();
upload.setFileSizeMax(MAX_FILE_SIZE);
FileItemIterator iter;
try {
iter = upload.getItemIterator(request);
while (iter.hasNext()) {
....
if (item.isFormField()) {
....
}else{
BufferedInputStream buffer = new BufferedInputStream(stream);
buffer.mark(MAX_FILE_SIZE);
libraryZip = File.createTempFile("fromUpload", null);
IOUtils.copy(buffer, new FileOutputStream(libraryZip));
...
}
I have encountered the same problem with Jersey. I have activated jersey trace but nothing help me.
I have changed the library by an apache Library and I see than the problem with linked to a repository for temporary files for tomcat. The repository was not exist. For files under 10k, the repository was not used.
So, after the repository creation, I used jersey library and all works fine.
I am developing an testing automation tool in linux system. I dont have write permissions for tomcat directory which is located on server. I need to develop an application where we can select an excel file so that the excel content is automatically stored in already existing table.
For this pupose i have written an form to select an file which is posted to a servlet CommonsFileUploadServlet where i am storing the uploaded file and then calling ReadExcelFile class which reads the file path and create a vector for data in file which is used to sstore data in database.
My problem is that i am not able to store the uploaded file in directory. Is it necessary to have permission rights for tomcat to do this. Can i store the file on my system and pass the path to ReadExcelFile.class
Please guide me
My code is as follows:
Form in jsp
CommonsFileUploadServlet class code:
public void init(ServletConfig config) throws ServletException {
super.init(config);
}
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
PrintWriter out = response.getWriter();
response.setContentType("text/plain");
out.println("<h1>Servlet File Upload Example using Commons File Upload</h1>");
DiskFileItemFactory fileItemFactory = new DiskFileItemFactory ();
fileItemFactory.setSizeThreshold(1*1024*1024);
fileItemFactory.setRepository(new File("/home/example/Documents/Project/WEB-INF/tmp"));
ServletFileUpload uploadHandler = new ServletFileUpload(fileItemFactory);
try {
List items = uploadHandler.parseRequest(request);
Iterator itr = items.iterator();
while(itr.hasNext()) {
FileItem item = (FileItem) itr.next();
if(item.isFormField()) {
out.println("File Name = "+item.getFieldName()+", Value = "+item.getString());
} else {
out.println("Field Name = "+item.getFieldName()+
", File Name = "+item.getName()+
", Content type = "+item.getContentType()+
", File Size = "+item.getSize());
File file = new File("/",item.getName());
String realPath = getServletContext().getRealPath("/")+"/"+item.getName();
item.write(file);
ReadExcelFile ref= new ReadExcelFile();
String res=ref.insertReq(realPath,"1");
}
out.close();
}
}catch(FileUploadException ex) {
log("Error encountered while parsing the request",ex);
} catch(Exception ex) {
log("Error encountered while uploading file",ex);
}
}
}
ReadExcelFile code:
public static String insertReq(String fileName,String sno) {
//Read an Excel File and Store in a Vector
Vector dataHolder=readExcelFile(fileName,sno);
//store the data to database
storeCellDataToDatabase(dataHolder);
}
public static Vector readExcelFile(String fileName,String Sno)
{
/** --Define a Vector
--Holds Vectors Of Cells
*/
Vector cellVectorHolder = new Vector();
try{
/** Creating Input Stream**/
//InputStream myInput= ReadExcelFile.class.getResourceAsStream( fileName );
FileInputStream myInput = new FileInputStream(fileName);
/** Create a POIFSFileSystem object**/
POIFSFileSystem myFileSystem = new POIFSFileSystem(myInput);
/** Create a workbook using the File System**/
HSSFWorkbook myWorkBook = new HSSFWorkbook(myFileSystem);
int s=Integer.valueOf(Sno);
/** Get the first sheet from workbook**/
HSSFSheet mySheet = myWorkBook.getSheetAt(s);
/** We now need something to iterate through the cells.**/
Iterator rowIter = mySheet.rowIterator();
while(rowIter.hasNext())
{
HSSFRow myRow = (HSSFRow) rowIter.next();
Iterator cellIter = myRow.cellIterator();
Vector cellStoreVector=new Vector();
short minColIndex = myRow.getFirstCellNum();
short maxColIndex = myRow.getLastCellNum();
for(short colIndex = minColIndex; colIndex < maxColIndex; colIndex++)
{
HSSFCell myCell = myRow.getCell(colIndex);
if(myCell == null)
{
cellStoreVector.addElement(myCell);
}
else
{
cellStoreVector.addElement(myCell);
}
}
cellVectorHolder.addElement(cellStoreVector);
}
}catch (Exception e){e.printStackTrace(); }
return cellVectorHolder;
}
private static void storeCellDataToDatabase(Vector dataHolder)
{
Connection conn;
Statement stmt;
String query;
try
{
// get connection and declare statement
int z;
for (int i=1;i<dataHolder.size(); i++)
{
z=0;
Vector cellStoreVector=(Vector)dataHolder.elementAt(i);
String []stringCellValue=new String[10];
for (int j=0; j < cellStoreVector.size();j++,z++)
{
HSSFCell myCell = (HSSFCell)cellStoreVector.elementAt(j);
if(myCell==null)
stringCellValue[z]=" ";
else
stringCellValue[z] = myCell.toString();
}
try
{
//inserting into database
}
catch(Exception error)
{
String e="Error"+error;
System.out.println(e);
}
}
stmt.close();
conn.close();
System.out.println("success");
}
catch(Exception error)
{
String e="Error"+error;
System.out.println(e);
}
}
POI will happily open from an old InputStream, it needn't be a File one.
I'd suggest you look at the Commons FileUpload Streaming API and consider just passing the excel part straight to POI without touching the disk