I have been given a requirement to create a pl/sql procedure which will accept select statement as an input parameter. All data must be fetched from the query and printed in DBMS_OUTPUT.
I've researched native dynamic SQL and DBMS_SQL but was unable to figure out how to fetch and process data from a table whose structure is unknown.
Since the table name will be provided during run time, i just want to know how to store the data fetched from the query because i cant define variables or collections since the structure of table is unknown
First off, the requirement seems incredibly dubious. You should never depend on data that is written to the DBMS_OUTPUT buffer-- it is entirely up to the client to enable a buffer, to ensure that the buffer is large enough, and to display the data from the buffer to the user. By default, none of that will happen. And writing a procedure to manipulate a table whose structure is completely unknown would be incredibly unusual.
If you are really determined, however, you would likely want to take Tom Kyte's SQL Unloader which uses DBMS_SQL to write data from an arbitrary query to a flat file and modify it to write it to DBMS_OUTPUT instead.
There is an open-source utility package for generating Excel readable files within PL/SQL.
https://code.google.com/p/plsql-utils/
However, I would recommend you look into using a more general purpose language for your tool if at all possible. PL/SQL can be incredibly useful for database logic, but for interacting with the outside world, I expect you will achieve a more maintainable solution using something like Python or Java.
Although, as always YMMV :-)
Related
I am developing one automation tool for data migration from one table to other table, here I am looking for one function or SP for which I will pass source column and destination column as input parameter. I want output parameter to return true when source column data is compatible to copy to destination column. If not then it should return false.
For example, if a source column is varchar and a destination column is integer, the script should check all the data in a source column in good enough to move to an integer column or not and return the output flag. I want a script to work like this for all types of data types. Any suggestions will be helpful.
If you're on SQL Server 2012 you have TRY_CAST(), TRY_CONVERT() and TRY_PARSE() at your disposal (see this post by Biz Nigatu of blog.dbandbi for comparison).
That said, you still need to check for truncation errors, e.g. by converting to target datatype and back, then comparing the original value with the one after conversions.
I've seen similar tools in the past, might be a good idea to see if one isn't already available online for free. Even a purchase might be less expensive than the time you put into developing and troubleshooting your own tool.
I have an SSIS solution for this that I put together using EzAPI. I've got it posted to GitHub, so feel free to look:
https://github.com/thevinnie/SyncDatabases
Now, the part of that which is relevant to you would be where I use the information schema to make sure that the source and destination are schema matches. If not, C# script task will generate the statement to create, alter or drop the necessary column(s).
The EzAPI part is cool with SSIS because it will allow you to programatically generate an SSIS package. For the project requirements, I needed to be able to load data every time and not let schema changes in the source break the process.
Comments and recommendations are welcome. Hopefully it'll help, but I think you'll be looking at the INFORMATION_SCHEMA.COLUMNS either way.
I have some hard coded database values in my SQL and I need to convert to variables , I have declared them in places but I need Production2 to be changed to #Source_Database_Name variable below but I dont know how to place it in with the Information Schema just after it without getting a syntax error
IF EXISTS(SELECT * FROM Production2.INFORMATION_SCHEMA.COLUMNS
I guess that the only way you can do this is dynamic sql generation (unfortunately). And there's actually quite a few reasons (from database engine's perspective) for not allowing a user to parametrise queries in a way you want. The one that sits on top of my head is that it will make impossible to validate syntax of your query (no way to know that you're referring to what actually exists).
In case you're talking about "being able to execute the same set of SQL against different database(s)" and you're actually executing this sql from code (.NET / anything), you can achieve the same result by specifying target database in connection string (i.e. by changing the level where you set database -- not in the [sql] script, but rather at some external point).
I'm getting a little confused about using parameters with SQL queries, and seeing some things that I can't immediately explain, so I'm just after some background info at this point.
First, is there a standard format for parameter names in queries, or is this database/middleware dependent ? I've seen both this:-
DELETE * FROM #tablename
and...
DELETE * FROM :tablename
Second - where (typically) does the parameter replacement happen? Are parameters replaced/expanded before the query is sent to the database, or does the database receive params and query separately, and perform the expansion itself?
Just as background, I'm using the DevArt UniDAC toolkit from a C++Builder app to connect via ODBC to an Excel spreadsheet. I know this is almost pessimal in a few ways... (I'm trying to understand why a particular command works only when it doesn't use parameters)
With such data access libraries, like UniDAC or FireDAC, you can use macros. They allow you to use special markers (called macro) in the places of a SQL command, where parameter are disallowed. I dont know UniDAC API, but will provide a sample for FireDAC:
ADQuery1.SQL.Text := 'DELETE * FROM &tablename';
ADQuery1.MacroByName('tablename').AsRaw := 'MyTab';
ADQuery1.ExecSQL;
Second - where (typically) does the parameter replacement happen?
It doesn't. That's the whole point. Data elements in your query stay data items. Code elements stay code elements. The two never intersect, and thus there is never an opportunity for malicious data to be treated as code.
connect via ODBC to an Excel spreadsheet... I'm trying to understand why a particular command works only when it doesn't use parameters
Excel isn't really a database engine, but if it were, you still can't use a parameter for the name a table.
SQL parameters are sent to the database. The database performs the expansion itself. That allows the database to set up a query plan that will work for different values of the parameters.
Microsoft always uses #parname for parameters. Oracle uses :parname. Other databases are different.
No database I know of allows you to specify the table name as a parameter. You have to expand that client side, like:
command.CommandText = string.Format("DELETE FROM {0}", tableName);
P.S. A * is not allowed after a DELETE. After all, you can only delete whole rows, not a set of columns.
I've got the SQL stored procedure from hell that I've created and all input parameters are parameterised for security but it's not running as quick as I'd like so I wanted to make it dynamic and so a bit more efficient.
I know I can keep my input parameters to my stored procedure, then within it create a dynamic SQL statement into which I can then pass the input parameters of the stored procedure, but are there any security implications I need to be aware of when doing this? I'm guessing not as it just another set of parameters and they should be treated the same as the parameters passed to the current stored procedure.
Obviously, producing code like this "WHERE OrderNo = ' + #orderno is asking for trouble - I will be doing 'WHERE OrderNo = #orderno' in the dynamic SQL, but is there anything else I need to be aware of?
Thx MH
PS - before anyone suggests it, I can't create the SQL dynamically at the client side using LINQ or similar - it all (for various reasons) has to be contained and controlled at the database level
There is a form of SQL injection that many people don't think about when doing dynamic SQL in stored procedures: SQL Truncation attacks.
With a SQL truncation attack, the attacker injects a long peace of text making the used text variable overflow and lose part of the query.
This article gives more information about this.
Where your parameters are always Data Items, both when being passed to the StoredProc and when used in yor DynamicSQL, everything will stay safe.
Should any of your StoredProc's parameters end up being table or field names, and so forming part of the structure of the DynamicSQL itself, you introduce a new risk : That the parameter can be used to inject rogue SQL Code.
To prevent against such an injection attack you should always validate any such parameters.
One example of how to do this would be to use the input parameter as a token, rather than substitute it directly into the DynamicSQL...
SET #SQL = #SLQ + CASE targetTable WHEN '1' THEN 'table1'
WHEN 'tx' THEN 'tableX'
END
Some people suggest you only need to validate on the client application. But that means that if someone becomes able to execute you SP's directly, the SP has become a point of attack. I always prefer to validate both on the client AND in the server.
EDIT Performance
Note that using DynamicSQL isn't always a guarnatee of performance increases. If you use parameterised queries, the execution plans can indeed be stored. But if the queries do vary greatly, you may still find a significant overhead in compiling the SQL.
There is also the fact that dependancy tracking is lost. It's not possible to see what tables the SP is dependant on, because the code is hidden away as strings.
I have very rarely found that DynamicSQL is needed. Often a complex query can be reformed as several optimised queries. Or the data can be re-structured to meet the new demands. Or even a rethink of both the data and the algorithm using the data. One might even be able to suggest that a dependancy on DynamicSQL is an indicator of another underlying problem.
Perhaps it's not in the scope of your question, but it would be interesting to see the actual puzzle you're facing; to see if anyone has any alternative approaches for you.
I am rather new to SAS and I have run into a problem that I think probably has a better solution than what I've found so far.
I need to update a Oracle db table that has around 1 million rows with data from a SAS data set that has about 10,000 records.
I used an update statement within proc sql, but it takes hours to update the Oracle table. Right now, I am loading the data from the SAS data set into a temporary table in the Oracle db and doing a proc sql pass through execute statement to update the main table from the temporary table. This takes only a couple of minutes at most.
However, this is rather cumbersome to program and and I need to update the Oracle table from multiple functions within my SAS code.
Is there an analog to JDBC batch update in SAS (I uses to do Java programming before getting involved in SAS)? Something that is faster than using an update statement in proc sql, but easier to code than temp table + update using pass through?
Are you using SAS/Access to connect your SAS sessions to Oracle?
In my situation, I use SAS/Connect JDBC.
SAS/Connect is a very simple but effective strategy for interfacing the SAS substrate system to JEE. Essentially sas/connect is yet another telnet implementation by sas to execute sas -dmr.
I draw the sas data out using sas/connect jdbc into my jsp and then push the data into oracle or sql server using java programming techniques we are all familiar with.
Read my ancient paper on using sas/connect to connect sas to JEE:
http://www.nesug.org/proceedings/nesug04/ap/ap02.pdf.
BTW do not try to contact me with the contacts listed on the paper - they are ancient.
In response to your further statement:
I thought you wanted a way to use JDBC to insert the data into Oracle?
My paper shows you how to embed a whole block of SAS macro or SQL or any text in a JSP and then submit that block of text to be run through SAS/Connect.
String datasetname = request.getParameter("datasetname");
String where = request.getParameter("where");
<t:text id="macHello">
%macro hello(datasetname);
data &datasetname;
/* code to create your data */
run;
%mend;
%hello(<%=datasetname%>);
</t:text>
sasConnect.submit(macHello);
<t:text id="SQLgetRecs">
SELECT *
FROM <%=datasetname%>
WHERE <%=where%>
</t:text>
ResultSet mydata =
sasConnJDBC.executeQuery(SQLgetRecs);
Then do whatever you need to do with Java,
either by interweaving insertion in Oracle per iteration of Resultset
or iterate resultset to produce a text block of SQL insert VALUES
which you then submit to Oracle JDBC.
It would just be a single JSP, provided you know how to work a JSP and willing to understand how the text-block tag library I wrote works. You see, I use this technique to allow a JSP run SAS macros that have been running in production batch mode for ages, without any change to the macros. Not only so, the tag lib allows me to embed java and jsp variable resolution into the macros or sas/sql blocks.
I wrote this block-text tag lib because I used to do such operations in Perl (prior to 2003), where Perl (and other scripting languages) allows you to assign a variable to a continuous block of text within the code of the script.
Instructions on using the tag lib:
http://h2g2java.blessedgeek.com/2009/07/jsp-text-custom-tag.html
http://h2g2java.blessedgeek.com/2009/07/referencing-text-jsp-custom-tag-defined.html