My media storage is Openstack object storage (swift) in the cloud (OVH).
Regarding the user-rights on the uploaded media:
Images [A] are viewable by all users, but only deletable by
user-owner/ uploader.
Images [B] are very private. CRUD by user-owner/ uploader and
viewable by some other users.
I looked around for solutions and came across pre-signed (temporary) urls., see also this article.
I was wondering whether this provides an acceptable security level. An alternative I could think of is authenticating all users via openstack's authentication module, Keystone. But maybe that's just completely stupid and/ or overkill. I started to look in that direction as it might be similar to AWS S3 use of IAM policies.
My questions:
Is the pre-signed url solution the way to go? And if not why not?
How would processing images (creating thumbnails) look like? You
grab it from the storage, process and store it back and delete local
versions, I suppose?
Related
I'm evaluating potential content management systems I want to use for a project. Many of the users will need to upload static files and include links to the in their posts.
In the Admin UI I can only see the ability to upload an image in a post. Does anyone know if it is possible to upload files to Keystone through the Admin UI?
You could use their Amazon S3 storage adapter. Depending on which version of Keystone you're using (3 or 4), you'll have to do some different things. Either way, you need to make some credentials for Amazon S3's service and configure Keystone to work with it. From there, you can use Types.S3File to allow a certain part of your MongoDB model to be a reference to an S3 object. See this page for more info on the S3File type in Keystone.
We are developing a mobile app using Parse as our BAAS solution but using Amazon S3 for storage of our media files. All of our users upload media files into their own individual folders inside of our app's bucket. As the user uploads media files we update their records in Parse so it knows where to download the files. That's the easy part.
I've spent quite a bit of time researching the different policies for S3 buckets and I am trying to get a grip on the proper way to ensure the security of the content uploaded. If you do all of your work with DynamoDB or SimpleDB then it's easy because you're essentially adjusting your ACLs with the IAM accounts and whatnot. If you use Amazon Cognito it's also easy because authentication happens through Google, Facebook or Amazon accounts. In my case I am using Parse to authenticate users which cannot speak to Amazon directly.
My goal is that only the currently logged in Parse user with ID #1234567 can access their own 1234567 folder and files (as well as any other user given permission by this person for collaboration). Here is a post similar to what I'm trying to accomplish: amazon S3 bucket policy - restricting access by referer BUT not restricting if urls are generated via query string authentication
...but how do I accomplish this with the current user's ID number?
Even better question is whether that post mentioned above is best practice or should I instead be looking at creating an EC2 server to handle access to these files? Should I be looking at CloudFront to serve private content? Or is there another method that works better for what I am trying to accomplish? I am going in circles and my head is spinning.
Thanks to whoever can help straighten me out.
Well since Parse is being shut down I am migrating to another service. This question is no longer relevant.
We have been working on a web service (http://www.genomespace.org/) that allows computational biology tools that implement our REST API to, among other things, read and write files stored on "The Cloud".
Our default file storage is a common Amazon S3 bucket, but we now allow users to mount their own private S3 bucket as well as files on Dropbox.
We are now trying to enable similar functionality for Google Drive and have run into some problems unique to Google Drive that we have not encountered with S3 or Dropbox.
Only way to allow clients that are not Google-authenticated to read files unobtrusively is to make the files "Public". Our preference would be that once the user has authorized access to our application via OAuth2, the user files could remain "Private" in Google Drive.
However, even though the user has already authorized our web service to offline access to their "Private" files, we have not found a way to generate a URL that a client authorized by our system can use to GET the file directly without the client being logged into Google as well.
The closest we have come to this functionality has been to change the file permissions to "Anyone with Link", except that for files greater than 20MB Google insists on returning an intermediate web page warning that the file has not been scanned for viruses. In addition to having to mess with file permissions, this would break our existing clients. Only when the file is "Public" and we utilize URLs of the form https://googledrive.com/host/PARENT_FOLDER_ID/FILENAME can non-Google clients read the files without interference.
Have not found any way for clients that are not Google-authenticated to upload a file to Google Drive. Our API allows our authorized clients to PUT files directly to the backing file storage using URLs provided by our server. However, even if a folder is marked "Public", the client requires Google authentication credentials to save to Google Drive. We could deal with both of these issues with intermediate hops through our system (e.g., our web server would first download the file from Google Drive and then allow the client to GET it) however this would be woefully inefficient and, hopefully, unnecessary. These problems have been discussed multiple times before on stackoverflow (e.g. here and here and have read the responses very carefully, but have not seen any recent discussion.
The Google folks direct their API users to post on stackoverflow for support, so I am hoping for a fresh look from insiders.
The general answer is: dont make the drive requests through the user's browser. Insead do everything from your servers. You are the one having the (refresh) tokens for users, so you should make all requests like a proxy between the user and Drive. Same for downloading, you download it and return to the user. As long as you use each drive's token there shouldnt be rate limit/quota issues.
I have an amazon s3 account. As i am expecting a large amount of traffic to my site i want to put the videos to be placed in my s3 account and stream it up to my website. How can i do that?
Thanks
Prady
The simplest way is to upload it using the AWS Management Console for S3, use that to set its permissions to be publicly accessible, and then just access the usual S3 URL for it:
http://bucket-name.s3.amazonaws.com/key-name
Depending on exactly how much traffic you're getting, you can look into using Amazon's CloudFront distribution network. That will speed things up for your users, especially if they span the globe.
I'd like to host my photo galleries on Amazon S3. Some would be public, some password-protected. What is the best way to achieve this? As far as I know S3 doesn't supports .htaccess files.
Thanks,
Istvan
It is "possible" to manage user permissions in S3 natively, but it's not really practical or scalable.
A much more practical solution (in my opinion) is to manage authentication and permissions in some other web application and then have that application provide temporary (expiring) links to the files in S3.