ruby on rails AWS-S3 list files in a bucket - ruby-on-rails-3

I have a select box the I wish to fill with filenames from a clients S3 bucket.
In my controller I set the variable as this:
#files = AWS::S3::Bucket.find("clientsbucket").objects
which when called in the view as options_for_select(#files) gives a list of objects but in the format of <AWS::S3::Object:0x4f9e5b8>, <AWS::S3::Object:0x4f9e5a0> etc
For the life of me I cant figure out how to list the filename instead of this object info?
Any help muchly appreciated

Well, access the key property of the each object in the view!
key property is the entire path to the file in the bucket.
objects.each do |object|
= object.key
Even though the AWS SDK Documentation isn't as informative try and dig around..
Use the as_tree method on the objects so you can get the specific data you want.
http://docs.amazonwebservices.com/AWSRubySDK/latest/AWS/S3/Tree.html
Good luck!!

Related

Is there a way to list the directories in a using PySpark in a notebook?

I'm trying to see every file is a certain directory, but since each file in the directory is very large, I can't use sc.wholeTextfile or sc.textfile. I wanted to just get the filenames from them, and then pull the file if needed in a different cell. I can access the files just fine using Cyberduck and it shows the names on there.
Ex: I have the link for one set of data at "name:///mainfolder/date/sectionsofdate/indiviual_files.gz", and it works, But I want to see the names of the files in "/mainfolder/date" and in "/mainfolder/date/sectionsofdate" without having to load them all in via sc.textFile or sc.Wholetextfile. Both those functions work, so I know my keys are correct, but it takes too long for them to be loaded.
Considering that the list of files can be retrieve by one single node, you can just list the files in the directory. Look at this response.
wholeTextFiles returns a tuple (path, content) but I don't know if the file content is lazy to get only the first part of the tuple.

Which is the best practice either to save image name or full URL in database

Which is the better approach for storing image name in database? I have two choices first one is to store just image name e.g. apple.png and second choice is to store full image URL e.g. abc.com/src/apple.png.
Any help will be appreciated. Thanks.
Best practice is not save full path to image like abc.com/src/apple.png but saving specific domain path to image. Ex:
Users image : /user/{id}/avatar/img.png
Product image: /product/{id}/1.png
In this case you avoid sticking images to defined server, server path, url etc. For example, you will decide to move all your images to another server, in this case you don't need to change all records in DB.
The 2 answers already covered it pretty well. It is indeed best practice to save the directory path instead of saving the entire URL path. Some of the reasons were already covered, such as making it easy to move your folders to another server without having to make any changes whatsoever in your file logic.
What you could do, is also have everything in one directory, refer to that, and then just save the image name. However, I would not recommend that. The other structure simply makes it way easier to navigate and look through. Good file structure is something you'll thank yourself for later in case you ever have to go through things manually for one reason or another.
With that said, I'd like to add this trick into the mix:
$_SERVER['DOCUMENT_ROOT']. This always makes you start from the root directory as opposed to having to do tedious things, such as ../../ etc. It looks like a mess.
So in the end as an image path, you'd have something like:
<img src="<?php echo $_SERVER['DOCUMENT_ROOT'].'/'.$row['filePath']; ?>" >
$row['filePath'] being your stored filepath from the database.
Depending on how your file path is saved, you can lose the / in the image source link.
first of all you need to upload all images in public folder of your project , so no need to save domain name
If you are storing all images in one directory , then there is no problem storing only imagename in database
you can easily access images like <img src="/foldername/imagename.jpg" />
but if in your project there are multiple directory like
profile :to save user avatar image ,
background : to save background images,
then it is better to save image with path in database like "/profile/avatar.jpg"
so you can access image like <img src="imagepathhere" />
Another common way is to create image table with cols
id
type (enum or int)
name (file name)
Define in your app (better in model) types like
USER_AVATAR = 1;
PRODUCT_IMG = 2;
Define path map foreach image type like:
$paths = [
USER_AVATAR => '/var/www/project/web/images/users',
...
];
and use id's from this image table in another tables. It is called polymorphic association. It is most flexible way to store images.

how to handle multiple uploads?

I'm writing a program that can upload files to multiple FTP servers.
There is a table, at the top row there are the sites, and at the far left column there are the files. through this table I define what should be uploaded to where.
the program is already working, but what i want to do now is to upload the files parallely on each site. so when i hit start each column will go through the rows on its own and upload the files to that site if the content of the specific cell says so. sites can be any number between 1 and 50. and all uploads should be in parallel. (one file at the time for each site)
what i am asking is what is the best way to handle such thing? i know i have to set up multiple uploaders, but what is confusing me is how to keep track what each site is doing. the only thing i can come up with is an array of arrays. where each position is for a site, and the array at that position defines what file is beeing uploaded and all the informations it needs for that. would that be a good solution?
thanks!
You can put your data into array then use for loop
use this code
$web = ['www.firstSite.com','www.secondSite.com']
$user = ['firstUser','secondUser']
$pass = ['firstPass','secondPass']
for($i=0;$i<sizeof($web);$i++)
{
$conn_id = ftp_connect($web[$i]);
$login_result = ftp_login($conn_id,$user[$i],$pass[$i]);
if (ftp_put($conn_id, $server_file, $local_file, FTP_BINARY))
{echo "Success";}
else {echo "Failed";}
}
You can make a custom class and use a List(Of SiteFiles) for a collection of them. You iterate the sites data and create a new SiteFiles object for each site and add the files names to the Files property that need to be uploaded to that site. Then when your done making this List(Of SiteFiles) then you can iterate each file in SiteFiles.Files for each SiteFiles object and use threading/async methods if needed and upload the files. This gives you a neat and tidy way to organize what your doing.
Public Class SiteFiles
Public Property Site As String
Public Property Files As New List(Of String)
End Class

Plone 4 - Get url of a file in a plone.app.blob.field.FileField

I have a custom content type with 3 FileFields (plone.app.blob.field.FileField) and I want to get their url's, so i can put them on my custom view and people will be able to download these files.
However, when using Clouseau to test and debug, I call :
context.getFirst_file().absolute_url()
Where getFirst_file() is the accessor to the first file (field called 'first_file').
The url returned is 'http://foo/.../eat.00001', where 'eat.00001' is the object of my custom type that contains the file fields...
The interesting thing is, if I call:
context.getFirst_file().getContentType()
It returns 'application/pdf', which is correct since it's a pdf file.
I'm pretty lost here, any help is appreciated. Thanks in advance!
File fields do not support a absolute_url method; instead, through acquisition you inherit the method from the object itself, hence the results you see. Moreover, calling getFirst_field() will return the actual downloadable contents of the field, not the field itself which could provide such information.
Instead, you should use the at_download script appended to the object URL, followed by the field id:
First File
You can also re-use the Archetypes widget for the field, by passing the field name to the widget method:
<metal:field use-macro="python:context.widget('first_field', mode='view')">
First File
</metal:field>
This will display the file size, icon (if available), the filename and the file mime type.
In both these examples, I assumed the name of the field is 'first_field'.

Getting the data from backend to display it on Front end in X-CART

I am working on a X-Cart Module. I need the backend configuration values to the front end. In admin/configuration file I have created an array which contains the value which I need to use it in adv_search.php which is in the root folder. I don't know how to pass that array to adv_search.php.
Thanks in advance.
It is not quite clear how you store the configuration values. It may be easier to store them in the table xcart_config and access them directly as $config['Option-category']['option_name'] in php files and {$config.Option-category.option_name} in the Smarty-templates.
Or just define an array in module's config.php file and this array will be available everywhere in php-scripts. Look at the modules/UPS_OnLine_Tools/config.php file for example.