So I have been reading about signed URL's and some of its benefits. Especially the part about hot linking. Our app doesn't allow users to embed media (photo, video, audio) from our site. So signed URL's looks like the right direction. Mostly to prevent hotlinking.
So now that I know my requirements. I have a few questions.
Does this mean I have to add a policy to my bucket, denying read-write access to any of the files or folders in the bucket?
Do I have to create signed URL's for each page visit? So let's say 100 users visit the same page where the song can be played. Does this mean I have to create 100 signed URL's?
Creating S3 signed URL's are free?
Touching on point #2. Is it normal practice for Amazon S3 to create several signed URL's? I mean what happens if 1,000 users end up coming to the same song page..
Your thoughts?
REFERENCE:
For anyone interested on how I was able to generate signed url's. Based on https://github.com/appoxy/aws gem and the docs at http://rubydoc.info/gems/aws/2.4.5/frames :
s3 = Aws::S3.new(APP_CONFIG['amazon_access_key_id'], APP_CONFIG['amazon_secret_access_key'])
bucket_gen = Aws::S3Generator::Bucket.create(s3, 'bucket_name')
signed_url = bucket_gen.get(URI.unescape(URI.parse(URI.escape('http://bucket_name.s3.amazonaws.com/uploads/foobar.mp3')).path[1..-1]), 1.hour)
By default, your bucket will be set to private. When you upload files to S3, you can set the ACL (permissions) - in your case, you'll want to make sure the files are private.
The simplest solution is to create new signed urls for each visitor. You could do something like generate new urls everyday, store them somewhere, and then use those but that adds complexity for little benefit. The one place where you might need this though is too enable client side caching. Everytime you create a new url, the browser sees it as a different file and will download a fresh copy. If this isn't the behaviour you want, you need to generate urls that expire far in the future and reuse those - but that will reduce the effectiveness of preventing hotlinking.
Yes, generating urls are free. They are generated on the client and don't touch S3. I suppose there is a time/processing cost, but I have created pages with hundreds of urls that are generated on each visit and have not noticed any performance issues.
Related
Sanity team said "Asset files are not private, so even images uploaded to a private dataset can be viewed by unauthenticated users."
(https://www.sanity.io/docs/keeping-your-data-safe)
The material on my site is only viewable by those who pay a monthly fee.
The material is mainly a pdf file.
Uploading a pdf file with type="file" creates an asset url.
Anyone can view it by typing the path into a web browser.
In the case of Vimeo, videos are played only when accessing from a specific domain.
Similarly, is it possible to make sanity assets available only within a specific domain?
I am in a similar situation and have done some resarch. It seems like private assets can't exist on Sanity. I believe the main reason is that every asset under 10MB in size will be cached on a CDN which does not have the authentication logic required to protect the assets.
IMO they should provide a toggle to make certain assets private, with the disclaimer that it will not be cached on a CDN.
Read more about Sanity CDN: https://www.sanity.io/docs/asset-cdn
EDIT:
You can however use a custom asset source, but it doesn't really sovle the problem: https://www.sanity.io/docs/custom-asset-sources
I am trying to make sure I did not miss anything in the AWS CloudFront documentation or anywhere else ...
I have a (not public) S3 bucket configured as origin in a CloudFront web distribution (i.e. I don't think it matters but I am using signed urls).
Let's say a have a file in a S3 path like
/someRandomString/someCustomerName/someProductName/somevideo.mp4
So, perhaps the url generated by CloudFront would be something like:
https://my.domain.com/someRandomString/someCustomerName/someProductName/somevideo.mp4?Expires=1512062975&Signature=unqsignature&Key-Pair-Id=keyid
Is there a way to obfuscate the path to actual file on the generated URL. All 3 parts before the filename can change, so I prefer not to use "Origin Path" on Origin Settings to hide the begging of the path. With that approach, I would have to create a lot of origins mapped to the same bucket but different paths. If that's the only way, then the limit of 25 origins per distribution would be a problem.
Ideally, I would like to get something like
https://my.domain.com/someRandomObfuscatedPath/somevideo.mp4?Expires=1512062975&Signature=unqsignature&Key-Pair-Id=keyid
Note: I am also using my own domain/CNAME.
Thanks
Cris
One way could be to use a lambda function that receives the S3 file's path, copies it into an obfuscated directory (maybe it has a simple mapping from source to origin) and then returns the signed URL of the copied file. This will ensure that only the obfuscated path is visible externally.
Of course, this will (potentially) double the data storage so you need some way to clean up the obfuscated folders. That could be done on a time-based manner, so if each signed URL is expected to expire after 24 hours, you could create folders based on date, and each of the obfuscated directories could be deleted every other day.
Alternatively, you could use a service like tinyurl.com or something similar to create a mapping. It would be much easier, save on storage, etc. The only downside would be that it would not reflect your domain name.
If you have the ability to modify the routing of your domain then this is a non-issue, but I presume that's not an option.
Obfuscation is not a form of security.
If you wish to control which objects users can access, you should use Pre-Signed URLs or Cookies. This way, you can grant access to private objects via S3 or CloudFront and not worry about people obtaining access to other objects.
See: Serving Private Content through CloudFront
I am using AMWS s3 in a ruby on rails project to store images for my models. Everything is working fine. I was just wondering if it okay/normal that if someone right clicks an image, it shows the following url:
https://mybucketname.s3.amazonaws.com/uploads/photo/picture/100/batman.jpg
Is this a hacking risk, letting people see your bucket name? I guess I was expecting to see a bunch of randomized letters or something. /Noob
Yes, it's normal.
It's not a security risk unless your bucket permissions allow unauthenticated actions like uploading and deleting objects by anonymous users (obviously, having the bucket name would be necessary if a malicious user wanted to overwrite your files) or your bucket name itself provides some kind of information you don't want revealed.
If it makes you feel better, you can always associate a CloudFront distribution with your bucket -- a CloudFront distribution has a default hostname like d1a2b3c4dexample.cloudfront.net, which you can use in your links, or you can associate a vanity hostname with the CloudFront distribution, like assets.example.com, neither of which will reveal the bucket name.
But your bucket name, itself, is not considered sensitive information. It is common practice to use links to objects in buckets, which necessarily include the bucket name.
I am using aws s3 for storing files and returning links to those files. It's very convenient but I only know how to generate a pre-signed URL which only lasts for a set amount of time.
Is there anyway to generate a default URL which lasts permanently? I need the user to be able to access the photo from within their app. They take a photo, it's uploaded. They view it. Pretty standard.
You have to use Handlers (DOI, PURL, PURLZ, persistant URIs)
All are free of charge, except for DOIs.
You create that handler, then you add it to your project.
I'm building an application (Rails 3.2.8) where user can upload music tracks and associated clip. While clips can be publicly accessible, track can't be accessible without purchasing. I'm using carrierwave for uploading the both types of files. However, I do not want to expose the actual track path to the users.
What techniques such services use to protect hotlinking and/or unauthorized access to the files?
Currently, the carrierwave path is like:
def store_dir
"tracks/#{model.user_id}/"
end
However, this is very vulnerable. Anyone can easily guess the url.
For authorised downloading, i can consider:
1. Static download link (this link is valid all time for that user. however, no guests or other users can use that URL)
2. General temporary links for each download!
Please enlighten me with the ways I can consider (i will study them) so that i can secure the files from downloading without purchases.
Seems like you want both, public for the clip and private for the track.
I am trying to implement this as well with the approach below (not tested)
def fog_public
return #job.job_kind == 'public'
end
s3 allows you to store private file, they will be available only for a given period and with an access token.
With carrierwave, you simply have to set the fog_publig to false as described here https://github.com/jnicklas/carrierwave#using-amazon-s3