I have a very large set of data that is formed similar to this:
I want to produce report output that drops the table headers (field1, field2, field3) and instead use data row1 as headers in output. I am able to do this nicely in SQLPlus and SPOOL to CSV, but I want to do everything in PLSQL so that I can log to concurrent request output.
I cannot simply load the data into a new table using row1 as header because I cannot predict what the new header names will be (for the purpose of extraction). The steps of dropping table headers and extracting to file system need to happen together.
Looking for some suggestions. I am OK with continuing the use of SQLPlus for the extraction if there is a way to use fnd_file.put_line or some other method to write to concurrent request log. As far as I can tell fnd_file.put_line commands do not work in SQLPlus.
Before you all go off on me, I dont want to hear about how its bad design to have dynamic table headers. I have my reasons in this particular case.
I solved this within the SQLPlus program. Before starting to SPOOL the output I issue the PROMPT command with text that I want to be written to the request output.
The below will place 'this is some log text' in the request output
prompt this is some log text
Related
I've been testing multiple theories but having issues. I've created an SP and a BAT command to export a file via BCP and send it to a third party. Normally I would BCP it within the SP, but due to server:folder connectivity, I'm trying to perform the %OUTFILE% within the BAT (If I'm over complicating it, let me know.)
I can't post entire code, so I'll psuedo replace it.
CREATE PROCEDURE
{{{populates a temp table}}}
SELECT {requirements} FROM #table;
SELECT {requirements2} FROM #table;
SELECT {requirements3} FROM #table;
END
Now this works in live form, just fine.
The BAT file I sent the client is
SET hourVAR
SET OUTFILE="{FileDirectory}"
bcp "exec SPICreated" queryout %OUTFILE% params
Normally I would do this either within a multiple step job (I can't do a job for them, though) or I would make the BAT file include the entire BCP "SELECT FROM" but the select is ~30 columns long, and due to the 3rd party vendor I'm trying to put all the 'bulk' in the SP.
Can anyone provide insight on how I may better do this? If I assign a variable to the "SELECT" portion, can I call it from the BAT file? SQL Server is not my forte.
(Trying not to create 3 duplicated SPs, and trying to avoid a ~100 line BAT file.)
--- For those wondering, running the SP with all 3 "SELECTs" caused it to become broken in compilation and somehow becomes.
Additional Info: This is all from the same table, but I need 3 different data sets in 3 different documents.
Data Resembles:
1|2|3|4|5
A|B|C|D|E
Z|X|Y|V|C
AA|BB|3|D|5
I need Document One to be
1|2|3
A|B|C
Z|X|Y
AA|BB|D
I need document Two to be
1|5
A|E
Z|C
AA|3
I need document Three to be
1|3
A|D
Z|V
AA|D
EDIT: Added data examples to assist query. Queries already work to get the data within a view, but not for BCP
I am writing the shell script to move the data from the SQL plus to Sybase. But before that I need to take the output to the csv file which I have done.
I want to remove the "Connected." word form the csv file so that I can have the data columns only so that I can read the data from the file and insert it in the sybase.
I tried searching but unable to find a proper command. I have used the following but none worked for me.
set heading off
set echo off
set feedback off
I found the solution:
I used /dev/null for silencing all the notifications and the spool command to output the result. It works great.
I have a SQL code which spools out data into a .csv file from Oracle databases.
set echo off
set feedback off
set linesize 1000
set pagesize 0
set sqlprompt ''
set trimspool on
set verify off
spool test.csv
/*Code Part*/
/
spool off
The problem is that, if at all any error occurs (e.g: resource busy issue) while executing code part, those error messages are getting copied into .csv file along with the spooled data. Is there any way to avoid it?
It would be much more helpful if some one suggests me a way to redirect these error messages to a .txt file (I dont know if it is possible or not).
Thanks!
You may want to look into using the UTL_FILE package to create the file you want rather than counting on SQL*Plus to redirect your output. While this would perhaps require some rewriting on your part you'd end up with better control of what is being written.
Another option would be to filter the output file to eliminate . For example, if the lines you don't want to see all start with "ORA-" you might be able to use something like the grep command:
grep -v ^"ORA-" test.csv
Or you might have to use something like sed or awk if your filtering requirements are more complicated.
Share and enjoy.
I have oracle table that holds more than 30 million records, I need to pull all that data into file and store it.. Can anyone suggest me what will be easiest way to do that and what kind of file do I have to use to store all that data. If there is a way I can put data into multiple files that will work too.. You can tell me manual or programatical method. Thank you
Ask Tom has an answer http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::p11_question_id:88212348059
Also, you could do it in SQL*Plus by doing a select with spool on (from a shell script)
sqlplus /nolog <<EOF
conn /as sysdba
set pagesize 0 heading off feedback off verify off echo off trimspool on
spool test.log
select 'dsmc inc "'||file_name||'">>hot_WISDOM_$BCKNAME.log' from dba_data_files
where rownum<5;
spool off;
exit
EOF
More info on the spool stuff at http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:25323432223677
That page is where I found that snippet above.
You will want to replace the connection details with your username / pw.
Starting with Oracle 10g you can create a new table with ORGANIZATION EXTERNAL and INSERT records from the original table into it. See this example from the Oracle 10gR2 documentation.
One of the easiest formats to store data in is comma seperated value (.csv). You can define your delimiter to be any character (comma is a default) as long as you know the delimiter when you are parsing it. (For instance, the ~ character would be a good decision). Usually this format can easily be opened in a spreadsheet program like Excel. It can also be easily parsed to be re-inserted into a database of your choosing.
To export from oracle, you can do something like:
spool backup.csv;
select column1||','||column2||','|| ... from table;
spool off;
After exporting, here is a resource for working with csv files in perl:
http://perlmeme.org/tutorials/parsing_csv.html
Apache commons has a great library for java:
http://commons.apache.org/sandbox/csv/apidocs/org/apache/commons/csv/CSVParser.html
Is there a way to display the output of a sqlplus command without having to first issue the spool off command?
I am spooling the results of a sqlplus session to a file while at the same time tailing the file. The reason for this is that for table with very long rows the format is easier to look at from a file. The problem is to see the output i have to issue the spool off command everytime i run a command in sqlplus.
Can i configure sqlplus so that after i have issued the spool command all the output is viewable straight away on the file.
(Formating the way the rows are displayed on the screen is not an option. )
THanks
SPOOL is really intended for creating a file of SQL*Plus output, for whatever purpose: logging, input to another process, etc. There is no facility for inflight viewing of its output.
There are a number of ways of solving this particular problem, but the easiest is surely to use an IDE which includes a data browser, thus obviating the need to tail a file. There are a number on the market, including Quest's TOAD and Allround Automation's PL/SQL Developer, but if you don't want to spring for a license fee then you should have a look at Oracle's own (free) SQL Developer.
If you are spooling the results of multiple statements you could turn spooling off and then turn it back on between each statement. When you turn spooling back on add the append keyword so that it will continue in the same file rather than overwriting it.
If you want to see the results of one query in your spool file you could break the query up into multiple queries that returned specific ranges of the data. This would be slower, but you could cycle spooling to get faster feedback.
If your problem is that you can't open the output file (as the spool process has a lock on it) then try copying the file output to another file and opening that file instead.
Since it sounds like your real problem is formatting of output in SQLPlus -- can you make your SQLPlus window wider and SET LINESIZE so the output looks better in SQLPlus to start with? Then you might not need to spool at all.
I tried to add a comment but for some reason it doesnt save it so ill try the "Answer your question" option :)
I do use SQLDeveloper but there are situations where i have to use sqlplus where SQLDeveloper is not available then i am stuck with plain old sqlplus.
There are other situations where i would use sqlplus over sqldeveloper purely for the fact that it would take me 1/2 minute to find out what i am looking for in sqlplus rather than several minutes with SQLDeveloper as it would take ages to load.
I have checked the time it takes before the output is flushed out and it looks like it does flush it out after a certain number of rows. Isnt there a way to reduce the buffer so that they are flushed out quicker?
There is no problem opening the file the problem is even with the file opened i cant see the output unless i issue the "spool off" command or the output has several hundred rows. I am using a free program called baretail (http://www.baretail.com) to tail the spool file on windows.
Thanks