So here's my problem. My admins need to upload files that can be anywhere from 2-500 MB's each. I've set my php.ini settings appropriately and all is well with this requirement. But now I've been asked to allow guests to upload files from the front-end. Obviously, I do not want to give them the ability to upload 500 MB files.
I've searched around and have been unable to find a decent answer for allowing large file uploads in the admin while limiting front-end guests to smaller file sizes.
So how do you allow your admin's to continue uploading extremely large files while restricting front-end users to a smaller file sizes?
Here's my solution:
public function saveAction()
{
$post = $this->getRequest()->getPost();
$helper = Mage::helper('my_module');
if ( $post ) {
try {
if ($_FILES['size'] >= 2000000) { // Limit is set to 2 MB
$errors[] = $helper->__('You have exceeded the max file size.');
$error = true;
}
if ($error) {
throw new Exception();
}
// Perform save operations here.
} catch (Exception $e) {
foreach($errors as $error) {
Mage::getSingleton('core/session')->addError($error);
}
$this->_redirect('*/*/*');
return;
}
}
}
This checks to see if the file exceeds the limit. If it does, it throws an exception.
Anyway, I'm looking for better/alternative solutions to this same problem. Post them if you've got them!
I can give you the basic idea on how you can achieve this. In your saveAction() of admin add this Mage::app()->getWebsite()->getId() This will return the current website ID. The admin website ID will be always 0. So add an IF condition in the saveAction() to check whether the current website ID is 0 or other.
If the return value is not equal to 0 then you can limit the size to 50MB or what ever u want. If it is equal to 0 then you can allow the limit upto 500MB.
Hope this helps.
Related
I want to upload files to a minio file container.
Smaller files work as expected with this code:
private Mono<Boolean> saveFileToMinio(FilePart filePart) {
log.info("About to save database to minio container...");
Mono<Boolean> result = Mono.from(
filePart.content().flatMap(dataBuffer -> {
var bytes = dataBuffer.asByteBuffer().array();
dataBuffer.read(bytes);
DataBufferUtils.release(dataBuffer);
return Flux.just(bytes);
})
.flatMap(databaseFileService::write)
.then(Mono.just(true))
.onErrorMap(throwable -> {
log.error(throwable.getMessage(), throwable);
return throwable;
}));
log.info("Successfully saved database to minio container...");
return result;
}
I need to provide a byte[] for my minio service to be uploaded.
Smaller files work as expected (will be stored to the container). But larger files (12 MB in my test) don´t work.
I get this exception:
java.lang.IndexOutOfBoundsException: readPosition 0 and length 1024 should be smaller than writePosition 808
I´ve tried a suggestion DataBufferUtils.join from another SO post. This is kind of odd but i think the following code does the job:
private Mono<Boolean> saveFileToMinio(FilePart filePart) {
var result = DataBufferUtils.join(filePart.content()).map(dataBuffer -> {
var bytes = dataBuffer.asByteBuffer().array();
dataBuffer.read(bytes);
DataBufferUtils.release(dataBuffer);
return bytes;
}).map(databaseFileService::write).then(Mono.just(true))
.onErrorMap(throwable -> {
log.error(throwable.getMessage(), throwable);
return throwable;
});
log.info("Successfully saved database to minio container...");
return result;
}
Especially this line seems to do the trick:
DataBufferUtils.join
I don´t know why but it seems to work.
EDIT:
Looking up the above static join sets a parameter maxByteCount to -1. Maybe the other (not working function for larger files) sets some limits. But i don´t know.
We have a scanner that is going to scan documents into a specific directory.
I need ideas on where to start on an application/service that I can schedule to run every hour or so and upload the documents to a Cloud DB.
Every single example I can find involves the uploader that requires a user to browse/select the file; I don't want that.
The documents will be stored as binary if that matters.
Maybe this will help you get started. Build a console app that does something along these lines
void Main()
{
while(true)
{
var files = Directory.GetFiles("c:\\your\\path\\here\\","*.pdf");
foreach (var file in files)
{
UploadToCloudDB(file);
//possibly delete the file now
}
Thread.Sleep(30000); //pause for 30 seconds between scans
}
}
static void UploadToCloudDB(string path)
{
var bytes = File.ReadAllBytes(path);
//upload those bytes to the cloud DB ....
}
My error is Maximum request length exceeded.
I want file upload must be smaller than 2MB. Please help me to fix code bellow, thanks
my controller:
public ActionResult Index()
{
var path = "~/Images/upload/";
if (Request.Files["UpFile"] != null && Request.Files["UpFile"].ContentLength < 2048)
{
var upload = Request.Files["UpFile"];
upload.SaveAs(Server.MapPath(path + upload.FileName));
}
else
{
ModelState.AddModelError("", "The size of file too big");
}
return View();
}
Try to manage your maximum request length for reducing errors to minimum:
Maximum request length exceeded
I think it's a good practice to use try..catch when working with uploading files even if you have global exception handler.
I'm looking into developing apps for a project and I started researching permissions. I know the technical definition of GET_TASKS- Allows an application to get information about the currently or recently running tasks: a thumbnail representation of the tasks, what activities are running in it, etc. What I don't know is just what a "thumbnail" representation is- is it an actual picture (i.e screenshot of what is going on in another app), or is it just some information about the app? Also, what does the definition mean by "what activities are running in it"? Does that mean that someone can develop an app that can practically tell exactly what someone is doing, almost like a spy app (i.e if someone were checking their bank information on their browser, could the app see that?). Thanks for all the help, I'm a total noob here just trying to get used to the developer tools.
It is used within the "context" of a Context... so, gives your Activities. E.g.
Context context = this.hostActivity.getApplicationContext();
ActivityManager am = (ActivityManager)context.getSystemService("activity");
List taskInfo = null;
try {
taskInfo = am.getRunningTasks(1);
if ((taskInfo != null) && (!taskInfo.isEmpty())) {
ComponentName topActivity = ((ActivityManager.RunningTaskInfo)taskInfo.get(0)).topActivity;
if (!topActivity.getPackageName().equals(context.getPackageName())) {
this.logger.debug("The application was displaced by new one.");
needPause = true;
} else {
this.logger.debug("The activity was displaced by new one in the same application.");
}
}
} catch (SecurityException e) {
needPause = true;
this.logger.warn("The application has no GET_TASKS permission.");
}
I created an upload script in node.js using express/formidable. It basically works, but I am wondering where and when to check the uploaded file e. g. for the maximum file size or if the file´s mimetype is actually allowed.
My program looks like this:
app.post('/', function(req, res, next) {
req.form.on('progress', function(bytesReceived, bytesExpected) {
// ... do stuff
});
req.form.complete(function(err, fields, files) {
console.log('\nuploaded %s to %s', files.image.filename, files.image.path);
// ... do stuff
});
});
It seems to me that the only viable place for checking the mimetype/file size is the complete event where I can reliably use the filesystem functions to get the size of the uploaded file in /tmp/ – but that seems like a not so good idea because:
the possibly malicious/too large file is already uploaded on my server
the user experience is poor – you watch the upload progress just to be told that it didnt work afterwards
Whats the best practice for implementing this? I found quite a few examples for file uploads in node.js but none seemed to do the security checks I would need.
With help from some guys at the node IRC and the node mailing list, here is what I do:
I am using formidable to handle the file upload. Using the progress event I can check the maximum filesize like this:
form.on('progress', function(bytesReceived, bytesExpected) {
if (bytesReceived > MAX_UPLOAD_SIZE) {
console.log('### ERROR: FILE TOO LARGE');
}
});
Reliably checking the mimetype is much more difficult. The basic Idea is to use the progress event, then if enough of the file is uploaded use a file --mime-type call and check the output of that external command. Simplified it looks like this:
// contains the path of the uploaded file,
// is grabbed in the fileBegin event below
var tmpPath;
form.on('progress', function validateMimetype(bytesReceived, bytesExpected) {
var percent = (bytesReceived / bytesExpected * 100) | 0;
// pretty basic check if enough bytes of the file are written to disk,
// might be too naive if the file is small!
if (tmpPath && percent > 25) {
var child = exec('file --mime-type ' + tmpPath, function (err, stdout, stderr) {
var mimetype = stdout.substring(stdout.lastIndexOf(':') + 2, stdout.lastIndexOf('\n'));
console.log('### file CALL OUTPUT', err, stdout, stderr);
if (err || stderr) {
console.log('### ERROR: MIMETYPE COULD NOT BE DETECTED');
} else if (!ALLOWED_MIME_TYPES[mimetype]) {
console.log('### ERROR: INVALID MIMETYPE', mimetype);
} else {
console.log('### MIMETYPE VALIDATION COMPLETE');
}
});
form.removeListener('progress', validateMimetype);
}
});
form.on('fileBegin', function grabTmpPath(_, fileInfo) {
if (fileInfo.path) {
tmpPath = fileInfo.path;
form.removeListener('fileBegin', grabTmpPath);
}
});
The new version of Connect (2.x.) has this already baked into the bodyParser using the limit middleware: https://github.com/senchalabs/connect/blob/master/lib/middleware/multipart.js#L44-61
I think it's much better this way as you just kill the request when it exceeds the maximum limit instead of just stopping the formidable parser (and letting the request "go on").
More about the limit middleware: http://www.senchalabs.org/connect/limit.html