I am thinking to have subfolders for each user on my website to be able to upload their files(images, videos etc). My approach is to have a folder structure like this
uploads
user1
file1
file2
user2
file1
file2
Is this a good approach and does it have any impact when the number of users increases?
Related
Both users are uploading unique images but their camera roll has the same image name.
Should I use a uuid as a file name instead?
You can't prevent overwrites in S3 through the API.
For avoidance, there are multiple strategies that all come down to namespacing or UUIDs. What you want to use depends on what you plan to do with the data.
Approach 1
s3://<bucket>/<userId>/<filename>
This way, you avoid users overwriting each other's files, but a user could still overwrite their own files. You could, with relative ease, list the uploads a specific user has made (but it could get expensive).
Approach 2
s3://<bucket>/<userId>/<uuid>.jpg
You still avoid users being able to overwrite each other's data and make it exceedingly unlikely that a user overwrites their own images - but you lose the information about the original file name.
Approach 3
s3://<bucket>/<userId>/<uuid>/<filename>
This key schema retains the benefits of the first two approaches and also allows you to retain the original filename, but it will be more annoying if you want to look at the data in the console because there will be more "directory" levels.
Approach 4
s3://<bucket>/<uuid>.jpg
This way, you don't namespace anything and just rely on UUIDs to avoid overwriting data. You lose the information about the original file name and which user the object belongs to unless you have a secondary data structure (e.g. an index of you data in DynamoDB).
All of these options (and more) are completely valid, personally, I'd pick something that at least namespaces my data by the user id because that makes it easier to delete specific users if necessary and also allows me to write IAM policies to allow or deny access to specific users.
I set CSV file in jmeter via CSV data Set Config this file contains 6 usernames and passwords.
What should be number of Threads in Thread Group page.
Also what should i do if i wanted to check with 100 users?
Should i increase number of users in my CSV file or should i use number of thread?
Answer really depends on what your test does and which load you want to inject.
But to make a realistic test, your CSV file should have as many logins as you have Virtual Users (threads in JMeters).
And you should ensure that 2 threads never use the same user (if your application does not accept it).
I need to store some data so it can be accessed by the application from any user account.
The amount of data is not big, just several strings.
The storage does not have to be secure, although it would be beneficial. A least the users without admin permission should not be able to spoil it.
I would not mind if the application would require to provide admin password to store the data or remove it. However, when application is launched from another user account without admin rights, it should be able to read this data not asking for any passwords.
Should I use keychain or /Library/Application Support/ folder or maybe something else?
You can share file between different users by using ACL.
Working with Access Control Lists
Share any files between users on the same Mac
Make a folder read-write for all users without using ACLs
I hope this will help.
We are in the process of building a system which allows users to upload multiple images and videos to our servers.
The team I'm working with have decided to save all the assets belonging to a user in a folder named using the user's unique identifier. This folder in turn will be a sub-folder of our main assets folder on the file server.
The file structure they have proposed is as follows:
[asset_root]/userid1/assets1
[asset_root]/userid1/assets2
[asset_root]/userid2/assets1
[asset_root]/userid2/assets2
etc.
We are expecting to have thousands or possibly a million+ users in the life time of this system.
I always thought that it wasn't a good idea to have many sub-folders in a single location and suggested a year/month/day approach as follows:
[asset_root]/2010/11/04/userid1/assets1
[asset_root]/2010/11/04/userid1/assets2
[asset_root]/2010/11/04/userid2/assets1
[asset_root]/2010/11/04/userid2/assets2
etc.
Does anyone know which of the above approaches would be better suited for this many assets? Is there a better method to organize images/videos on a server?
The system in question will be an Windows IIS 7.5 with a SAN.
Many thanks in advance.
In general you are correct, in that many file systems impose a limit on the number of files and folders which may be in one folder. If you hit that limit with the number of users you have, your in trouble.
In general, I would simply use a uuid for each image, with some dimension of partitioning. e.g. A hash of ABCDEFGH would end up as [asset_root]/ABC/DEFGH. Using a hash gives you a greater degree of assurance about the number of files which will end up in each folder and prevents you from having to worry about, for example, not knowing which month an image you need was stored in.
I'm presuming your file system is NTFS? IF so, you've got a limit of 4,294,967,295 files on the disk - the limit of files in a folder is the same. If you have on the order of millions of users you should be fine, though you might want to consider having only one folder per user instead of several as your example indicates.
For the past 5 years, my typical solution for storing uploaded files (images, videos, documents, etc) was to throw everything into an "upload" folder and give it a unique name.
I'm looking to refine my methods for storing uploaded content and I'm just wondering what other methods are used / preferred.
I've considered storing each item in their own folder (folder name is the Id in the db) so I can preserve the uploaded file name. I've also considered uploading all media to a locked folder, then using a file handler, which you pass the Id of the file you want to download in the querystring, it would then read the file and send the bytes to the user. This is handy for checking access, and restricting bandwidth for users.
I think the file handler method is a good way to handle files, as long as you know to how make good use of resources on your platform of choice. It is possible to do stupid things like read a 1GB file into memory if you don't know what you are doing.
In terms of storing the files on disk it is a question of how many, what are the access patterns, and what OS/platform you are using. For some people it can even be advantageous to store files in a database.
Creating a separate directory per upload seems like overkill unless you are doing some type of versioning. My personal preference is to rename files that are uploaded and store the original name. When a user downloads I attach the original name again.
Consider a virtual file system such as SolFS. Here's how it can solve your task:
If you have returning visitors, you can have a separate container for each visitors (and name it by visitor login, for example). One of the benefits of this approach is that you can encrypt the container using visitor's password.
If you have many probably one-time visitors, you can have one or several containers with files grouped by date of upload.
Virtual file system lets you keep original filenames either as actual filesnames, or as a metadata for the files being stored.
Next, you can compress the data being stored in the container.