I have downloaded a CSV file and am trying to use it for a SQL project (I am using Jupyter notebooks). Do I even need the CSV file or is there a way to use it without downloading it? I'm very new to all of this!
This is the link to the data that I downloaded:
https://github.com/new-york-civil-liberties-union/NYPD-Misconduct-Complaint-Database
What's your goal? Are you trying to learn SQL, or do you just want to work with the data?
If all you want is to load that csv into a table in a SQLite database, it would be easiest to do using the sqlite command line shell.
I'm on Windows, so forgive me if you aren't...
Open the Command Prompt
Navigate to the folder in which you want the new sqlite db file (e.g. cd C:\Users\User\Data)
sqlite3 NewDBName.db (e.g. sqlite3 MyNewDb.db)
.mode csv
.import path/to/downloaded/csv.csv NewTableName (e.g. .import C:\Users\User\Downloads\CCRB_database_raw.csv CCRB
That should be it. You can check it worked by running .schema- you should see the structure of your new tables.
Now you can try out some sql statements:
SELECT * FROM CCRB LIMIT 10;
Here are some more detailed instructions.
Related
I am under strict corporate environment and don't have access to Postgres' psql. Therefore I can't do what's shown e.g. in the SO Convert SQLITE SQL dump file to POSTGRESQL. However, I can generate the sqlite dump file .sql. The resulting dump.sql file is 1.3gb big.
What would be the best way to import this data into Postgres? I also have DBeaver and can connect to both databases simultaneously but unfortunately can't do INSERT from SELECT.
I think the term for that is 'absurd', not 'strict'.
DBeaver has an 'execute script' feature. But who knows, maybe it will be blocked.
EnterpriseDB offers binary downloads. If you unzip those to a local drive you might be able to execute psql from the bin subdirectory.
If you can install psycopg2 or pg8000 for python, you should be able to connect to the database and then loop over the dump file sending each line to the database with cur.execute(line) . It might take some fiddling if the dump file has any multi-line commands, but the example you linked to doesn't show any of those.
I want to load chinook database in SQLite but I have no idea how can I do that.
First, I used this command but it just made a file without anything in it:
sqlite3 chinook.db
I also downloaded this archive that contains chinook database but I couldn't find chinook.db file to open it with this command:
.open chinook.db
If you have a copy of the chinook database then just use the restore database feature.
cd /home/sqlite
mv sample.db sample.db.old
sqlite3 sample.db < sample.bak
For more information go to http://www.ibiblio.org/elemental/howto/sqlite-backup.html
We should script the database before using it. Here is the link that discussed the whole procedure.
I am currently working on a project and I want to know how to save an sqllite database in rails as a csv file. I want it when you click the button, the current database on the system download. Can anybody help me? Thanks!
Your problem isn't really specific to Rails. Instead, you're mostly dealing with an administrative issue. You should write a script to export your database as csv, something like this:
#!/bin/bash
./bin/sqlite3 ./my_app/db/my_database.db <<!
.headers on
.mode csv
.output my_output_file.csv
select * from my_table;
!
This script exports a single table. If you have additional tables, you'll want to add them to your script.
The only Rails related issue is the matter of calling that script. Save the script within your application structure; I'd suggest my_app/assets or some similar location.
Now you can run that script using system(command) where command is the absolute path for your script, within a set of double-quotes.
This is probably a simple question, but I could use some help. I am trying to build a small database for an application that will only be run on my computer so I want to create a local database.
To do this I am trying to use sqlite. I can use the command prompt to make what seems to be a database by using the sqlite3 databaseName; functionality, but I do not know where it is being stored.
I need to be able to find the database to access it through the application I am experimenting with. I already know all of the basic sql and such for creating the database tables and data, but I cannot figure out how to simply make the database connection.
is there a way to specify where the database .db file will be stored, and why can I not find the file it seems to be making?
Using sqlite3 shell? Some help using sqlite3 -help:
Usage: sqlite3 [OPTIONS] FILENAME [SQL]
If FILENAME is not supplied, shell uses an temporary database.
If you start shell without supplying a filename, you may save temporary database at any time using:
sqlite> .backup MAIN "folder\your_file.extension"
Or you can ATTACH an existing database an use SQL methods:
sqlite> ATTACH DATABASE "path\stored.db" AS other;
sqlite> INSERT OR REPLACE INTO other.table1 SELECT * FROM this_table1;
sqlite> DETACH other;
For doing such things , you can use Sqlite Manager , you'll get it as a Firefox addon. It's excellent in creating / Managing Sqlite database.
https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager/
Thanks everyone for answering, but it turns out my issue was much simpler than I thought.
I was trying to name the database after already starting the shell.
I was supposed to create the database from command line by doing sqlite3 name.db
But I was trying to use that command within the sqlite shell so nothing was being created.
i have created a table in Hive "sample" and loaded a csv file "sample.txt" into it.
now i need that data from "sample" into my local /opt/zxy/sample.txt.
How can i do that?
Hortonworks' Sandbox lets you do it through its HCatalog menu. Otherwise, the syntax is
INSERT OVERWRITE LOCAL DIRECTORY '/tmp/c' SELECT a.* FROM b
as per Hive language manual
Since your intention is just to copy the entire file from HDFS to your local FS, I would not suggest you to do it through a Hive query, because of the following reasons :
It'll start a Mapreduce job which will take more time than a normal copy.
It'll create file(s) with different names(000000_0, 000001_0 and so on), which will require you to rename the file manually afterwards.
You might face problem in opening these files as they are without any extension. Your OS would be unable to choose an application to open these files on its own. In such a case you either have to rename the file or manually select an application to open it.
To avoid these problems you could use HDFS get command :
bin/hadoop fs -get /user/hive/warehouse/sample/sample.txt /opt/zxy/sample.txt
Simple n easy. But if you need to copy some selected data, then you have to use a Hive query.
HTH
I usually run my query directly through Hive on the command line for this kind of thing, and pipe it into the local file like so:
hive -e 'select * from sample' > /opt/zxy/sample.txt
Hope that helps.
Readers who are accessing Hive from Windows OS can check out this script on Github.
It's a Python+paramiko script that extracts Hive data to local Windows OS file-system.