I've tried Riak CS and Walrus, and read a few other's documentation pages but can't tell whether they would support this or not.
What I have is an application that uses S3 policies to allow the client to upload and download directly from their browser. I'm looking for a way to replace S3 (for some customers, who would prefer their data not in Amazon's cloud), without having to maintain two different branches of code everywhere I currently talk to S3.
Example of what I do now:
https://aws.amazon.com/articles/1434
Help would be greatly appreciated, I'm stumped!
What you want is called "POST OBJECT". Just check document of S3 compatible implements to see support status.
For example, Walrus claims that it supports POST OBJECT. https://github.com/eucalyptus/eucalyptus/wiki/Walrus-S3-API
On the other hand, Riak CS does not support POST OBJECT http://docs.basho.com/riakcs/latest/references/apis/storage/s3/
Ps.: Many S3 implements claims that they are compatible with AWS S3. In fact, it is not ture. If you want fully compatibility, try google cloudian.
Related
Implementing a requirement to store images in AWS bucket instead of NetSuite. Since the bucket is private, I have to upload and generate the URL in backend/suitelet.
I tried to include AWS SDK into Suitelet by defining, but that doesn't work.
I want to get to know whether can we use/include SDKs inside Suitelet?
How can I implement a solution for this without using any third party solutions?
How are permissions for the links managed? Can you make them publicly viewable? Remember unless the links you generate are timestamped anyone with the link can get to the image.
In terms of uploading the images check out https://github.com/DeepChannel/netsuite-savedsearch-s3
If you need to keep have each image have a magic link you could use a Heroku app or an AWS lambda. The app would check a hash based on link parameters and proxy the image if the hash is valid. If your images are supposed to be private to a customer this would be the way to go.
If you are using the images generally on a website then just make the bucket publicly readable and use the API to upload.
I've seen the recently Google Drive pricing changes and they are amazing.
1Tb in Google Drive = $9.99
1Tb in Amazon S3 = $85 ($43 if you have more than 5000TB with them)
This changes everything !
We have a SaaS website in which we keep customer's files. Does anyone know if Google Drive can be used to keep this kind of files/service or it's just for personal use?
Does it have a robust API for uploading, downloading, and create public URL's to access files as S3 have ?
Edit: I saw the SDK here (https://developers.google.com/drive/v2/reference/). The main concern is if this service can be used for keeping customer's files, I mean, a SaaS website offering a service and keeping files there.
This doesn't really change anything.
“Google Drive storage is for users and Google Cloud Storage is for developers.”
— https://support.google.com/a/answer/2490100?hl=en
The analogous service with comparable functionality to S3 is Google Cloud Storage, which is remarkably similar to S3 in pricing.
https://developers.google.com/storage/pricing
Does anyone know if Google Drive can be used to keep this kind of files/service or it's just for personal use?
Yes you can. That's exactly why the Drive SDK exists. You can either store files under the user's own account, or under an "app" account called a Service Account.
Does it have a robust API for uploading, downloading, and create public URL's to access files as S3 have ?
"Robust" is a bit subjective, but there is certainly an API.
There are a number of techniques you can use to access the stored files. Look at https://developers.google.com/drive/v2/reference/files to see the various URLs which are provided.
Por true public access, you will probably need to have the files under a public directory. See https://support.google.com/drive/answer/2881970?hl=en
NB. If you are in the TB space, be very aware that Drive has a bunch of quotas, some of which are unpublished. Make sure you test any proof of concept at full scale.
Sorry to spoil your party, before you get too excited, look at this issue. It is in Google's own product, and has been active since November 2013 (i.e.4 months). Now imagine re-syncing a few hundred GB of files once a while. Or better, ask your customers to do it with their files after you recommended Drive to them.
With Docs API deprecated and some nice new functionality in Drive API, I'm working on updating some of my file migration scripts. However, I've found there doesn't seem to be a Drive API equivalent to the writersCanInvite parameter in Docs API v3. Being able to set this attribute on a file would be extremely important during migration scenarios as without it, the migrated file would be left open to having editors share it out further.
Not having writersCanInvite avaialble in addition to not being able to see email addresses (or some Unique ID) for the ACLs is preventing me from fully porting over my migration script to Drive API v2.
Thanks Guys,
Jay
It looks like we now have a writersCanShare attribute that controls this:
https://developers.google.com/drive/v2/reference/files#resource
at the moment, it's not marked as writeable but I had no problems making it False for a new document so I'm guessing that's just a documenation issue.
I'm trying to figure out how to store a database consisting of metadata in Amazon SimpleDB, with the actual content the metadata refers to (videos) in S3. As I understand it, I should place a pointer in SimpleDB that refers to the videos in S3. What is this pointer, exactly? Is it the URL of the video located in S3?
Also, are there any code samples that would pertain to this?
Thanks!
You're right, just enter the url on simpleDB and you're done.
What you're trying to do is pointed as an use case: http://aws.amazon.com/en/simpledb/usecases_metadata_indexing/
Taking a look at the code library, you can filter by S3 or SimpleDB and you'll find examples like SimpleDB PHP Sample Program Set and Travel Log - Sample Java Web Application.
Regards.
The documentation for Google Documents List API, seems to say that you can create a local document and upload it. Is there no way to actually create and edit a document on Google Docs through an API?
While the docs call it "uploading", everything boils down to sending an appropriately formatted HTTP POST request, so of course it can actually be a new creation rather than an actual "upload" of an otherwise existing file. (Creation through POST requests is similar to what's normally described as a REST API, though in real REST you'd typically use a PUT request instead of course).
You just need to create a blob of data representing your document in any of the formats listed here -- depending on your programming language, simplest may be text/csv for a spreadsheet and application/rtf for a text-document -- then put in in an appropriately formatted POST data. For example, to make a spreadsheet in the simplest way (no metadata), you could POST something like:
POST /feeds/default/private/full HTTP/1.1
Host: docs.google.com
GData-Version: 3.0
Authorization: <your authorization header here>
Content-Length: 81047
Content-Type: text/csv
Slug: Example Spreadsheet
ColumnA, ColumnB
23, 45
Each specific programming language for which a dedicated API is supplied may offer help with this not-so-hard task; for example, in Python, per the docs, the API recommends using ETags to avoid overwriting changes when multiple clients are simultaneously "uploading" (i.e., creating or updating docs). But preparing the POST directly is always possible, since the almost-REST API is documented as the protocol underlying all language-specific APIs.
Alex's answer, while undoubtedly correct, begs the question: "how do I do that via the Google Docs API?"
Here's a way (in Python, 'cause I'm that kind of guy):
import gdata.docs.service
import StringIO
client = gdata.docs.service.DocsService()
client.ClientLogin(username, password,
source='Spreadsheet Creation Example')
content = 'COL_A, COL_B, COL_C, COL_D\ndata1, data2, data3, data4'
ms = gdata.MediaSource(file_handle=StringIO.StringIO(content),
content_type='text/csv',
content_length=len(content))
entry = client.Upload(ms, 'Test Spreadsheet')
This is a small mashup of techniques that I found in http://code.google.com/p/gdata-python-client/source/browse/tests/gdata_tests/docs/service_test.py , which I in turn found via this post from the Google Group for the GData Docs API.
The key insights (for me anyway) were:
realizing that the MediaSource constructor's formal parameter "file_handle" will take any file-like object, and
discovering (as the OP's followup to the Google Group post mentions) that the unit tests are a great source of examples
(I wasn't able to find the Python-specific developer's guide referenced by Alex's doc link -- possibly it's been lost or buried in Google's move of documentation assets from code.google.com to developers.google.com. Alex's link now redirects to the more generic document that shows mostly .NET and Java examples, but only a little Python.)
As of Feb 4, 2019, Google Docs now has a REST API.
See documentation:
https://developers.google.com/docs/api/
(Sep 2019) There are 3 ways to create a document in Google Docs programmatically:
Google Docs REST API (low-level; Python, JS/Node.js, Java, C#/.NET, PHP, Ruby, Go, etc.)
Google Apps Script (high-level; JavaScript-only)
Google Drive API (low-level like Docs API above; both alternatives above can create or edit documents, but this one is create- or delete-only plus editing sharing/permissions)
The Docs API was officially launched in Feb 2019. I produced a high-level video overview of what a mail merge application using the API would look like. (It's not a full-fledged G Suite Dev Show episode but does link to a working sample.) Check out the various guides on using the API, including Quickstart examples in a variety of programming languages.
OTOH, Apps Script is a simpler, higher-level alternative. It's a custom server-side JavaScript runtime supporting apps that are hosted+executed in Google's cloud. Use objects to talk to various Google APIs (G Suite & beyond) without knowledge of HTTP, REST, nor OAuth2. You can also access external databases with its JDBC Service or call other apps via its URL Fetch Service.
With Apps Script, you can create standalone applications, document-bound applications (only works for a single document), or Google Docs Add-ons to extend the functionality of Google Docs. Here are the Google Docs Apps Script overview page as well as the Apps Script reference documentation for Google Docs (Document Service). I've also produced a variety of Apps Script videos if that's your preferred learning vehicle. If you're new to Apps Script, see my answer to a similar SO question for more learning resources.
Typically the Docs, Sheets, Slides, etc., APIs are used to perform document-oriented functionality while the Drive API is used primarily for file-based operations. However "create" is a special case where you can use either. See my answer to another SO question which shows the difference b/w creating a new Google Sheet via the Sheets API vs. the Drive API. (Both samples in Python.) Read this if you're interested in managing sharing or updating permissions of Google Docs.