We have multiple clients using our service.
Each client may create multiple projects.
Each client may upload multiple files to any of his projects.
Each file may have custom meta data associated.
Each client may "share" any of the projects to another client.
Each client may comment any of his or shared projects/files.
My question is about file storing in a cloud. What will best solution? I thought about Amazon S3 but maybe there are better alternatives?
You can explore Box.com solution. They are an advanced file management solution in the cloud and support fine-grained permission management as you explained above. Dropbox for Teams is also another option - The permission model is not as extensive as Box, but the sync client is very stable here. In one of my recent projects, I used box.com mainly due to their fine-grained permission controls
You can also build this on S3 (Dropbox and I guess Box too is behind the scenes built on S3). To achieve all the functionality as you mentioned, it is quite some programming work !
Related
I am exploring the possibilities of exposing an EMC Documentum folder, and the files/folders within, as a network file share.
The reason is so we can enable another application to read and write files to what it thinks is a standard UNC path, but really the repository is in Documentum.
That Documentum product doesn’t seem to offer this, however does expose an API.
A few thoughts here were a bespoke ‘driver’ for SAMBA, possibly something using WebDAV, but really I haven’t investigated these much yet, so both may be unviable.
Basically, how can I wrap an API up to look like a network drive?
I’ll keep self exploring this but hopefully someone can provide some leads here too..?
Update: using FUSE for Linux.
Documentum "folder" as you see it is not something like Windows folder. It is a database record of object with its related properties. Nothing else.
Documentum "documents" are somehow more related to Windows documents but still are only database record of objects with related properties and specific content stored somewhere in storage. Storage can be something like:
file share on Windows / Linux OS
specialized storage soluton like
Centra
specialized storage cloud solution
So you have misunderstanding of what you call Documentum folder. Your requirement can still be achieved in some way, thats for sure.
For example you could make integration between windows folder to Documentum via Spring Intergration framework (SI) from Windows folder side and at Documentum side implement listeners to hook SI and implement BOF (Business Object Framework) services to process events from SI. This is just one of the options.
Technically it is possible to create an interface to Documentum repository using any standard (SMB, CIFS, WebDav, IMAP, .... ) which can represent a document.
The fun task / hard part is mapping Documentum functionality to your chosen standard.
For example: back in 2013 I wrote a basic proof of concept Webdav interface to Documentum repository. I used the Miltion WebDav java library (http://milton.io).
With a WebDav interface, the Documentum Repository was exposed to a Windows computer as a drive using Add Network Location.
We identified that we can use FUSE on Linux.
I'm going to deploy a web application with multiple Pyramid application servers and nginx as a load balancer.
This application will have a feature for uploading files which should be available for downloading afterwards.
Total size of uploaded files may be very big so I'd like to deploy a separate file webserver to serve these static files. (this is one reason why I don't like rsync solution proposed here).
What is the best solution to handle file upload and syncronization in this case? I was thinking about NFS or something like that, but I'm not sure it is a good way to solve the problem. I suppose there must be some best-practices here or even a tool or library for these purposes.
UPDATE:
I don't want use cloud services like Dropbox, it would be nicer to find some syncronization solution inside the network segment.
UPDATE2:
I finished with setting up NFS, for now it works perfectly.
not really a python or pyramid related question. But, you should investigate distributed file systems and CDN's both of which are for this kind of thing. gridfs is easy enough to get going with. But there are plenty of other options. Both Amazon and Google have similar services.
I am developing an application which needs to fetch some data from an XML file for the automatic update process and for some other functions. This approach requires the files to be located in a direct link, so it can be hard-programmed to use that specific URL.
I heard that you can use a lot of free-to-use file-sharing services such as Google Drive, Box and Dropbox. Can you tell me if it's true or not? And are there any other services beside those I mentioned?
I don't need a web hosting that supports PHP and other frameworks, I just want to store files and make my application access it when required.
Yes, both Dropbox and Google Drive provide web hosting of your public folders, but there is a 10GB bandwidth limit with Dropbox.
You can use any free web hosting like 110mb or 5gbfree too.
You can try github, bitbucket or mega.co.nz
The web based AWS console seems so limited in what it can do. For example, to create a private stream distribution, you have to create CloudFront Origin Access Identity, create private content distribution, and modify ALC on the private objects, all through XML calls (WTF?). I really expect something so common to be integrated into their Console.
Is there a client smart enough that allows me to do simple tasks in simple ways?
Some options are:
ylastic - A web based tool that does automate many multi-step operations.
Cloudberry Explorer - A Windows only client application
Bucket Explorer - A cross platform client application
I'm not sure if these perform the tasks you need by they are worth a look.
You can configure CloudFront Private Content with cloudberry freeware. http://blog.cloudberrylab.com/2010/03/how-to-configure-private-content-for.html
More generally, you should probably look for cloud management softwares. They are an additional layer on top of Amazon Web Services and leverage AWS API to offer automations tools like automated backups, auto-scaling, failover by default...
Ylastic was mentionned by Geoff, but you can try Scalr (disclaimer: I work there), RightScale or enStratus.
I have felt this way too...but it seems the aws command line tools is the only options that I know..
Basic answer: No! It is even hard to find a GUI to manage more than one aws product
And no one will put in effort to develop this as aws keep changing the API interface.
For me:
For stable aws service like S3, I use Cyberduck
And the rest of them, I write my own program to do it, it is more customized for my own need and not easy to make mistake (and help me to familiar with the api)
I made a small backup application that simply creates an archive out specified files and folders. Now I need an online service to backup that online. Which service can i use that can be integrated into my app ?
Options I considered:
dropbox is ideal, but they have all but abandoned the desktop.
skydrive has no api.
I couldn't find any free reliable backup service that uses ftp .
anything else ? it should provide 1-2 gb of free space and be reasonably reliable.
Thanks
My app is in C#, but can be ported to any other language as well..
In your case, Amaxon's S3 seems more fitting but that's not free.
Depending on your target audience, you can create a local archive and have that picked up by your regular backup solution. You might try Wuala,or SpiderOak. Expand Wuala by adding your own space. Spideroak is free up to 2GB (more if you invite friends), and also provides a good alternative to Dropbox (if you want to see how to migrate from dropbox to spideroak see my blogpost about that).
Try box.net, now known as box.com or simply Box
reference: http://developers.box.com/docs