Import CSV to existing sqlite database table - ruby-on-rails-3

I've searched all over this forum for solution to the above problem, but everything I tried didn't work. Basically, i have a model Library, with corresponding libraries table in my sqlite3 database. I have a csv file named libraries.csv which contains all the data I want to import into the database.
I tried the method on the second answer on this page but it's still not working. I made sure to create my rake file 'import_libraries.rake in the lib/tasks folder and I also saved the libraries.csv file in that folder but i keep getting this error message:
rake aborted!
Don't know how to build task 'import_libraries' (See
full trace by running task with --trace)
This is the current code I'm using:
require 'csv'
desc "Imports a CSV file into an ActiveRecord table"
task :import, [:filename] => :environment do
CSV.foreach('libraries.csv', :headers => true) do |row|
Library.create!(row.to_hash)
end
end
But when I run bundle exec rake import_libraries, I get the error message above.
Is there anything I am doing wrong? i would appreciate your help guys. Thanks
EDIT
I renamed the rake file from import_libraries.rake to just import.rake On running bundle exec rake import, the error message i now get is:
rake aborted! invalid byte sequence in UTF-8
C:/Users/username/rails_app_name/lib/tasks/import.rake:4:in `block in
' Tasks: TOP => import (See full trace by running task
with --trace)

Based on the error you're getting and the task you have defined, you should be calling:
bundle exec rake import #=> you're currently calling import_libraries which indeed doesn't exist
With rake you call a task based on the name you give to the tasks, not on the name of the file (remember you can have many task inside each of those rake files).

I finally solved the problem by using this code:
namespace :csv do
desc "Import CSV Data"
task :import => :environment do
require 'csv'
csv_file_path = 'lib/tasks/libraries.csv'
CSV.foreach(csv_file_path, headers: true) do |row|
row = Library.create!({
:column_name1 => row[0],
:column_name2 => row[1],
:column_name3 => row[2],
.
.
:last_column => row[6]
})
puts "Success!"
end
end
end

Related

Passing a yaml array containing package data to puppet through Hiera

puppet version: 4.9.4
hiera version: 3.3.1
What i am trying to do is have httpd reload when a new version of package-x/y is installed, and it doesn't seem like the array from Hiera is being passed correctly.
for my httpd.pp file, I have:
class service::common::httpd (
$service_state = undef, # undef = unmanaged
$service_run_at_boot = undef,
$packages = undef
) {
service { 'httpd':
ensure => $service_state,
enable => $service_run_at_boot,
subscribe => $packages,
restart => "/usr/sbin/apachectl graceful"
}
}
and in the yaml file for hiera, I have:
service::common::httpd::packages: [Package['package-x'],Package['package-y']]
running puppet with this gives the error
Error: Evaluation Error: Error while evaluating a Function Call, Lookup of key 'allow_virtual_packages' failed: Unable to parse (/root/repos/puppet-config/data/nodes/<location of yaml file>): did not find expected ',' or ']' while parsing a flow sequence
also saying that its missing a comma between flow collection entries. I've tried many different combinations of spaces and commas as well..
I have also tried including the packages inside the class with an include statement.
What am i doing wrong?
The yamllint utility is quite useful for analysing Puppet Hiera YAML files. When I tried it on your file I got:
▶ yamllint spec/fixtures/hiera/data/common.yaml
spec/fixtures/hiera/data/common.yaml
2:25 error syntax error: expected ',' or ']', but got '['
2:39 error too few spaces after comma (commas)
The syntax error there reveals that the file is simply invalid YAML.
But how to fix it?
Confusingly, a line in a Puppet manifest like:
subscribe => [Package['package-x'], Package['package-y']]
When compiled into a JSON Puppet catalog becomes:
"subscribe": ["Package[package-x]", "Package[package-y]"]
And you can place the same JSON string in the YAML file to make valid YAML like this:
service::common::httpd::packages: ["Package[package-x]", "Package[package-y]"]
You can also use single quotes in the YAML i.e.
service::common::httpd::packages: ['Package[package-x]', 'Package[package-y]']
More info on how to compile the Puppet catalog in my blog post here.
quotes.
hiera doesn't know what Package is. just quote it, since its a string.
service::common::httpd::packages: [ "Package['package-x']",
"Package['package-y']" ]
works perfectly.
or you can just change [ "Package['package-x']", "Package['package-y']" ] to [ 'package-x', 'package-y' ]
works flawlessly. see below.
host01.yaml
beats::packetbeat::packages: [ acl, htop ]
packetbeat.pp
class beats::packetbeat (
$packages = undef
) {
package {
"packetbeat":
ensure => "$version",
subscribe => Package[$packages],
;
}
}
Notice: /Stage[main]/Beats::Packetbeat/Package[acl]/ensure:
current_value 'absent', should be '2.2.51-14.el7' (noop)
Notice: /Stage[main]/Beats::Packetbeat/Package[packetbeat]: Would have
triggered 'refresh' from 1 event
Notice: /Stage[main]/Beats::Packetbeat/Service[packetbeat]: Would have
triggered 'refresh' from 2 events
Notice: Class[Beats::Packetbeat]: Would have triggered 'refresh' from
3 events

Is there an option to list the targets (maybe with a description) in fake?

In Ruby (RAKE) you can document your tasks in the following way
# rakefile
desc "cleans ./temp"
task :clean do
p :cleaning
end
desc "compile the source"
task :compile => :clean do
p :compiling
end
$ rake -T # Display the tasks with descriptions, then exit.
rake clean # cleans ./temp
rake compile # compile the source
Is this possible with fake ?
The same is implemented in FAKE, as I found out when reading the source
// build.fsx
Description "remove temp/"
Target "Clean" (fun _ ->
CleanDirs [buildDir; deployDir]
)
// ....so on
Dependency graph is shown with .../fake.exe --listTargets or -lt
Available targets:
- Clean - remove temp/
Depends on: []
- Build
Depends on: ["Clean"]
- Deploy
Depends on: ["Test"]
- Test
Depends on: ["Build"]

CSV::can't convert String into Integer

I would like to import csv file into my database. I am using Ruby 1.8.7 and Rails 3.2.13 and the gem 'csv_importer'.
I am going to fetch productname and release_date from csv file
In my controller
def import
csv_text = File.read('test.csv')
csv = CSV.parse(csv_text, :headers => true)
csv.each do |row|
puts row #getting entire values from csv
puts row['productname'] #getting error
end
If I print row/row[0], I am getting entire values from csv file as
productname,release_date
xxx,yyy
in my log.
If I print row['productname'], I am getting error as can't convert String into Integer.
How can I rectify this error?
It looks like you are actually expecting the FasterCSV API, which does support a signature like parse(csv_text, :headers => true).
In Ruby version 1.9 the CSV library in stdlib was replaced with the FasterCSV library. Your code looks like it might work straight up in Ruby 1.9+.
If you want to use the FasterCSV style API without upgrading to a newer Ruby, you can grab the gem and use it instead of CSV:
require 'fastercsv'
csv_text = File.read('test.csv')
csv = FasterCSV.parse(csv_text, :headers => true)
csv.each do |row|
puts row['productname']
end
From http://ruby-doc.org/stdlib-1.8.7/libdoc/csv/rdoc/CSV.html#method-c-parse:
CSV.parse(str_or_readable, fs = nil, rs = nil, &block)
Parse lines from given string or stream. Return rows as an Array of Arrays.
... so row in your case is an Array. Array[] takes an Integer as argument, not aString`, which is why you're getting the error.
In other words; row['anything'] cannot work, but row[0] and row[1] will give you the values from column 1 and 2 of the row.
Now, in your case, you are actually calling CSV.parse like so:
CSV.parse(csv_text, :headers => true)
Looking at the docs, we see that the second argument to CSV.parse is the field separator. You're passing :headers => true as a field separator. That tells CSV to split each row whenever it encounters the string "headerstrue" - it doesn't, so it doesn't split each row.
If you remove the second argument to CSV.parse you should be closer to what you expect.

Ruby CSV read strings and numbers from MySQL exported csv

I exported tables and queries from SQL.
The ruby (1.9+) way to read csv appears to be:
require 'csv'
CSV.foreach("exported_mysql_table.csv", {:headers=>true}) do |row|
puts row
end
Which works great if your data is like this:
"name","email","potato"
"Bob","bob#bob.bob","omnomnom"
"Charlie","char#char.com","andcheese"
"Doug","diggyd#diglet.com","usemeltattack"
Works fine (The first line is a header, the attributes). However, if the data is like this:
"id","name","email","potato"
1,"Bob","bob#bob.bob","omnomnom"
2,"Charlie","char#char.com","andcheese"
4,"Doug","diggyd#diglet.com","usemeltattack"
Then we get the error:
.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/csv.rb:1894:in `block (2 levels) in shift': Missing or stray quote in line 2 (CSV::MalformedCSVError)
I think this is because the id is stored as a number, not a string, and thus has no quotes, and the csv parser expects ALL the entries to have quotes. Ideally I'd like to read "Bob" as a string and 1 as a number (and stuff it into a Hash of hashes)
(Have tried 'FasterCSV', that gem became 'csv' since ruby 1.9)
EDIT:
Was pointed out that the example worked fine (derp), was looking in the wrong place, it was an error with multi-line fields, question moved to Ruby CSV read multiline fields
Using the input you provided, I am unable to reproduce this.
1.9.3p194 :001 > require 'csv'
=> true
1.9.3p194 :002 > CSV.foreach("test.txt", {:headers => true}) { |row| puts row }
1,Bob,bob#bob.bob,omnomnom
2,Charlie,char#char.com,andcheese
4,Doug,diggyd#diglet.com,usemeltattack
=> nil
The only difference I see between our environments is that you are using rbenv, and I am using RVM. I also verified this on another machine I have with ruby 1.9.3-p194. Does the input you provided exactly match what is in your csv?

Capistrano run in sequential mode

I was wondering is there is a way to specify to the run command of Capistrano a way to operate in sequential mode rather than in parallel (typically via an environment variable)
If possible, I would like to avoid a conditional statement to switch between a run call and a find_servers_for_task(current_task).each do |hostname| ... end loop.
Any advice?
Ok, I found the correct syntax finally. You can limit the scope of a run command via the :hosts specifier (thanks Andriy Yanko). Example:
before "mytask", 'set_roles'
########### mytask #####################
desc "Example of a sequential run of capistrano"
task :mytask, :roles => :dynhosts do
find_servers_for_task(current_task).each do |hostname|
info "=> run mytask on #{hostname}"
run "whoami", :hosts => [ "#{hostname}" ]
end
end