I have oracle table that holds more than 30 million records, I need to pull all that data into file and store it.. Can anyone suggest me what will be easiest way to do that and what kind of file do I have to use to store all that data. If there is a way I can put data into multiple files that will work too.. You can tell me manual or programatical method. Thank you
Ask Tom has an answer http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::p11_question_id:88212348059
Also, you could do it in SQL*Plus by doing a select with spool on (from a shell script)
sqlplus /nolog <<EOF
conn /as sysdba
set pagesize 0 heading off feedback off verify off echo off trimspool on
spool test.log
select 'dsmc inc "'||file_name||'">>hot_WISDOM_$BCKNAME.log' from dba_data_files
where rownum<5;
spool off;
exit
EOF
More info on the spool stuff at http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:25323432223677
That page is where I found that snippet above.
You will want to replace the connection details with your username / pw.
Starting with Oracle 10g you can create a new table with ORGANIZATION EXTERNAL and INSERT records from the original table into it. See this example from the Oracle 10gR2 documentation.
One of the easiest formats to store data in is comma seperated value (.csv). You can define your delimiter to be any character (comma is a default) as long as you know the delimiter when you are parsing it. (For instance, the ~ character would be a good decision). Usually this format can easily be opened in a spreadsheet program like Excel. It can also be easily parsed to be re-inserted into a database of your choosing.
To export from oracle, you can do something like:
spool backup.csv;
select column1||','||column2||','|| ... from table;
spool off;
After exporting, here is a resource for working with csv files in perl:
http://perlmeme.org/tutorials/parsing_csv.html
Apache commons has a great library for java:
http://commons.apache.org/sandbox/csv/apidocs/org/apache/commons/csv/CSVParser.html
Related
I have a very large set of data that is formed similar to this:
I want to produce report output that drops the table headers (field1, field2, field3) and instead use data row1 as headers in output. I am able to do this nicely in SQLPlus and SPOOL to CSV, but I want to do everything in PLSQL so that I can log to concurrent request output.
I cannot simply load the data into a new table using row1 as header because I cannot predict what the new header names will be (for the purpose of extraction). The steps of dropping table headers and extracting to file system need to happen together.
Looking for some suggestions. I am OK with continuing the use of SQLPlus for the extraction if there is a way to use fnd_file.put_line or some other method to write to concurrent request log. As far as I can tell fnd_file.put_line commands do not work in SQLPlus.
Before you all go off on me, I dont want to hear about how its bad design to have dynamic table headers. I have my reasons in this particular case.
I solved this within the SQLPlus program. Before starting to SPOOL the output I issue the PROMPT command with text that I want to be written to the request output.
The below will place 'this is some log text' in the request output
prompt this is some log text
Using SQL tool, I want to create a text file that has a VARCHAR records.
Using SQL Developer I have successfully created the output text file by calling a script. But based on my research, the TRIMSPOOl command is currently not supported by SQL Developer, making my output to have trailing spaces.
Here is the code in the script:
spool on
set echo off
set verify off
set feedback off
set heading off
set trimspool on
set termout off
spool C:\SQLFiles\Output.txt;
select cust_name || cust_addr as Cust_Details
from customer_db;
spool off;
I now have the SQL Plus 11g, and I'm trying to run the script i created in SQL Developer. I'm getting an SP2-0606 error, saying:
SP2-0606: Cannot create SPOOL file "on.LST:
Based on the research I did, it is because of the Spool command, and I don't have the right to access the default folder??..
Can you help please on what setup I should change do to make the desired result in SQL Plus.
You have to put the file name in double quotes - I guess SQLPlus uses C-style escape sequences, giving the backslash \ a special meaning. Also, you should remove the semicolon ';':
So if you replace
spool C:\SQLFiles\Output.txt;
with
spool "C:\SQLFiles\Output.txt"
it should work as expected (if C:\SQLFiles exists and is writable for your account).
UPDATE
As #LalitKumarB has pointed out, spool works perfectly fine without the double quotes. The real problem is elsewhere:
spool on
This tells SQLPlus to create an output file on in the current directory. So you'll get this error message if you don't have access to the directory you're starting SQLPlus from.
You are getting this error because of permission issues. Either open your command prompt window in administrator mode or save your file to a location where admin permission is not required.
From the docs,
SP2-0606 Cannot create file file_name
Cause: The STORE command was unable to create the specified file. There may be insufficient disk space, too many open files, or
read-only protection on the output directory.
Action: Check that there is sufficient disk space and that the protection on the directory allows file creation.
My test case shows there is nothing wrong with the spool command. I don't think there is any need of quotes at all.
For example,
Client: 12c
OS : Windows 7
SQL> spool D:\text.txt
SQL> select ename||to_char(empno) from emp;
ENAME||TO_CHAR(EMPNO)
--------------------------------------------------
SMITH7369
ALLEN7499
WARD7521
JONES7566
MARTIN7654
BLAKE7698
CLARK7782
SCOTT7788
KING7839
TURNER7844
ADAMS7876
ENAME||TO_CHAR(EMPNO)
--------------------------------------------------
JAMES7900
FORD7902
MILLER7934
14 rows selected.
SQL> spool off;
And the spool file looks like:
So it works without any double-quotation marks. I guess you need to really look at the directory you are using to spool.
this error is just because of , not having "WRITE" privileges on directory where you are creating the spool file.
Solution
give or ask to give the permission to you who is owner of that directory, if its not owned by you using chmod. for ex- full permission to all chmod 777 <file/directory name>
When you get this error check for spaces. For example, below you see both single quotes (') but the first one has a space after it and the second one doesn't.
SP2-0606: Cannot create SPOOL file ' E:\SQL CLASS\New folder\sql class prt'
SQL> spool 'E:\SQL CLASS\New folder\sql class prt'
This question already has answers here:
Procedure to export table to multiple csv files
(2 answers)
Closed 9 years ago.
please help me to create a procedure to export data to specified path from oracle database as .csv file take study names from another table.
QUERY : select * from enrollment where study_name = 'FTY67' ;
I have another table(studies) in same database with all studynames.
is there any way to create procedure that will take study names from studies table and repeat this procedure to create .csv files for all studies?
read some articles in internet but not found anything related to this.
please help.
You should look into the spool and set command. Since the database is usually running on a remote server, it can not write files to your local computer. In order to achieve this yu must write an sql where you disable certain characteristics in the terminal, and then spool the result into a file that you can access.
Something like this might get you started:
set term off colsep ";" pause off
spool myfile
select * from x;
spool off
exit
For an overview of the options you can use with SET refer to the oracle documentation here: http://docs.oracle.com/cd/B19306_01/server.102/b14357/ch12040.htm
With the proper set commands you are able to create the CSV file.
The above set commands are just a few you might need, but you will probably need addtional parameters to make your CSV usable.
Best to write this in a file.sql and run it using sqlplus:
sqlplus user#db #file
I have a SQL code which spools out data into a .csv file from Oracle databases.
set echo off
set feedback off
set linesize 1000
set pagesize 0
set sqlprompt ''
set trimspool on
set verify off
spool test.csv
/*Code Part*/
/
spool off
The problem is that, if at all any error occurs (e.g: resource busy issue) while executing code part, those error messages are getting copied into .csv file along with the spooled data. Is there any way to avoid it?
It would be much more helpful if some one suggests me a way to redirect these error messages to a .txt file (I dont know if it is possible or not).
Thanks!
You may want to look into using the UTL_FILE package to create the file you want rather than counting on SQL*Plus to redirect your output. While this would perhaps require some rewriting on your part you'd end up with better control of what is being written.
Another option would be to filter the output file to eliminate . For example, if the lines you don't want to see all start with "ORA-" you might be able to use something like the grep command:
grep -v ^"ORA-" test.csv
Or you might have to use something like sed or awk if your filtering requirements are more complicated.
Share and enjoy.
Is there a way to display the output of a sqlplus command without having to first issue the spool off command?
I am spooling the results of a sqlplus session to a file while at the same time tailing the file. The reason for this is that for table with very long rows the format is easier to look at from a file. The problem is to see the output i have to issue the spool off command everytime i run a command in sqlplus.
Can i configure sqlplus so that after i have issued the spool command all the output is viewable straight away on the file.
(Formating the way the rows are displayed on the screen is not an option. )
THanks
SPOOL is really intended for creating a file of SQL*Plus output, for whatever purpose: logging, input to another process, etc. There is no facility for inflight viewing of its output.
There are a number of ways of solving this particular problem, but the easiest is surely to use an IDE which includes a data browser, thus obviating the need to tail a file. There are a number on the market, including Quest's TOAD and Allround Automation's PL/SQL Developer, but if you don't want to spring for a license fee then you should have a look at Oracle's own (free) SQL Developer.
If you are spooling the results of multiple statements you could turn spooling off and then turn it back on between each statement. When you turn spooling back on add the append keyword so that it will continue in the same file rather than overwriting it.
If you want to see the results of one query in your spool file you could break the query up into multiple queries that returned specific ranges of the data. This would be slower, but you could cycle spooling to get faster feedback.
If your problem is that you can't open the output file (as the spool process has a lock on it) then try copying the file output to another file and opening that file instead.
Since it sounds like your real problem is formatting of output in SQLPlus -- can you make your SQLPlus window wider and SET LINESIZE so the output looks better in SQLPlus to start with? Then you might not need to spool at all.
I tried to add a comment but for some reason it doesnt save it so ill try the "Answer your question" option :)
I do use SQLDeveloper but there are situations where i have to use sqlplus where SQLDeveloper is not available then i am stuck with plain old sqlplus.
There are other situations where i would use sqlplus over sqldeveloper purely for the fact that it would take me 1/2 minute to find out what i am looking for in sqlplus rather than several minutes with SQLDeveloper as it would take ages to load.
I have checked the time it takes before the output is flushed out and it looks like it does flush it out after a certain number of rows. Isnt there a way to reduce the buffer so that they are flushed out quicker?
There is no problem opening the file the problem is even with the file opened i cant see the output unless i issue the "spool off" command or the output has several hundred rows. I am using a free program called baretail (http://www.baretail.com) to tail the spool file on windows.
Thanks