I am using Microsoft SQL server 2005. I need to sync data between SQL server and an Oracle db. First thing I need is to find out if the count of data on Oracle side with certain filters(here I use ID as a simple example).
SELECT COUNT(*) FROM oracleServer..owner.table1 WHERE id = #id;
The problem I have is that the table on the lined server or Oracle is very big with 4M rows of data. The above query took about 2minutes to get data back. This code is just a simplied piece. Actually my SP has some other queries to update, insert data from the lined server to my SQL server. The SP took hours or 10+ hours to run with large Oracle db. Therefore T-SQL with lined server is not good for me.
Recently I found OPENQUERY and EXEC (...) AT linedServer. OPENQUERY() is very fast. It took about 0 time to get the same result. However, it does not support variable query or expressions. The query has to be a literal constant string.
EXEC() is in the same way to pass-through query to Oracle. It is fast as well. For example:
EXEC ('SELECT COUNT(*) FROM owner.table1 WHERE id = ' + CAST(#id AS VARCHAR))
AT oracleServer
The problem I have is how to pass the result COUNT(*) back. I tried to google examples in web and msdn. All I can find are SQL or ExpressSQL linedServer examples like:
EXEC ('SELECT ? = COUNT(*) FROM ...', #myCount OUTPUT) AT expressSQL
This query does not work for Oracle. It seems in Oracle, you can set value as output in this way:
SELECT COUNT(*) INTO myCount ...
I tried this:
EXEC ('SELECT COUNT(*) INTO ? FROM ...', #myCount OUTPUT) AT oracleServer
EXEC ('SELECT COUNT(*) INTO : FROM ...', #myCount OUTPUT) AT oracleServer
EXEC ('SELECT : = COUNT(*) FROM ...', #myCount OUTPUT) AT oracleServer
None of those working. I got error message saying query not executable on Oracle server.
I could write a .Net SQL Server project to do the job. Before that, I just wonder if there is anyway way to pass value out as oupput parameter so that I put the better performance T-SQL codes in my SP?
Just a quick update on this. I think I got the solution. I found it in a discussion on a similar issue at Dev NewsGroup. Based on the information, I tried this:
DECLARE #myCount int;
DECLARE #sql nvarchar(max);
set #sql =
N'BEGIN
select count(*) into :myCount from DATAPARC.CTC_MANUAL_DATA;
END;'
EXEC (#sql, #myCount OUTPUT) AT oracleServer;
PRINT #myCount; -- 3393065
Wa! I got the result back in 3 seconds comparing T-SQL query directly on Orable DB (+2minutes). The important thing is to use "BEGIN" and "END;" to wrap the query as anonymous block and don't miss ";" after END
You need anonymous block for output parameters. If you only have input or no parameters, you don't need the block and the query works fine.
Enjoy it! By the way, this is a quick update. If you don't see me again, I would not have any trouble on this issue.
With Linked Services the biggest issue is performance (IMHO)
[linkedserver]...[dbo.RemoteTable] vs OPENQUERY(linkedserver, 'Select * from dbo.RemoteTable') always use the second one
Now to answer the question. OPENQUERY and EXEC() AT is much quicker.
EXEC(Select * from dbo.RemoteTable) AT linkedserver will show the results, but there is no way to re-use.
My simple solution:
SELECT * INTO LocalTable FROM OPENQUERY(linkedserver, 'Select * from dbo.RemoteTable')
OR
INSERT INTO LocalTable SELECT * FROM OPENQUERY(linkedserver, 'Select * from dbo.RemoteTable')
much^10 faster than
SELECT * INTO LocalTable FROM [linkedserver]...[dbo.RemoteTable]
Related
One of the product teams I manage has an abundance of queries that use dynamic sql queries injecting the table in using concatenation. While it has sanitisation, I am trying to completely remove dynamic sql.
Is there a way to parameterise the table name?
I am trying to think of how I can query a table something like:
SELECT * FROM (SELECT DISTINCT Table_Name FROM INFORMATION_SCHEMA.Tables WHERE Table_Name = :queryParam)
Is this possible?
There is no way to "properly" prevent SQL injection completely from within SQL, the calling application layer should do this prior to executing any SQL statement.
Solve the problem by using an ORM or building the code to protect yourself from SQL injection when you generate the SQL in the application code.
This feels like a classic XY problem, try to take a step back and consider that you need to protect the access to the SQL server itself rather than sanitise everything from within "after" your SQL server has been accessed.
I am trying to completely remove dynamic sql.
You can't do it without Dynamic SQL.
Is this possible?
No, it's not possible, you cannot parameterize identifiers in SQL queries.
Why?
From the Books online page for Variables, Variables can be used only in expressions, not in place of object names or keywords. To construct dynamic SQL statements, use EXECUTE.
This is the only way to do it:
DECLARE #Column SysName = N'Table_Name',
#Param NVARCHAR(128) = N'ParamValue';
DECLARE #SQL NVARCHAR(MAX)=N'SELECT *
FROM(
SELECT '+ QUOTENAME(#Column) +
'FROM INFORMATION_SCHEMA.Tables
WHERE ' + QUOTENAME(#Column) + ' = #Param
) T';
EXECUTE sp_executesql #SQl,
N'#Param NVARCHAR(128)',
#Param;
EDIT: The things I've tried below came directly from the alleged duplicate. The solutions actually do work fine with a user defined sp (and probably most system sp's), but for whatever reason it doesn't work with this one.
I can run exec sp_showpendingchanges on the distribution publication database without any issues. However I want to capture the results in a table
I've tried:
SELECT * INTO #tmpTable
FROM OPENROWSET('SQLNCLI', 'Server=SERVER; Trusted_Connection=yes;',
'EXEC sp_showpendingchanges')
and:
SELECT * INTO #tmpTable
FROM OPENQUERY(SERVER, 'exec sp_showpendingchanges')
Both of these statements return an error that says: Invalid object name 'sysmergepublications'.
I tried to specify the initial catalog in the connection string and even tried adding a USE statement in the last parameter of each statement (i.e. I used an embedded EXEC statement with double-single quotes and all that). But I still end up with the same error.
So how can I get the results from exec sp_showpendingchanges into a temporary table, and preferably without having to define the table myself? If all else fails I will make a program in C#, but really hoping there's a simpler way to just do this with just SQL.
Here is a working example
You create a table
DECLARE #result_table TABLE
(
destination_server SYSNAME ,
pub_name SYSNAME ,
destination_db_name SYSNAME ,
is_dest_subscriber BIT ,
article_name SYSNAME ,
pending_deletes INT ,
pending_ins_and_upd INT
)
execute the script
INSERT INTO #result_table
EXEC sp_showpendingchanges
view the results
SELECT * FROM #result_table
I read your question but definetly cannot understand what the problem to create temp table. Anyway, if you can execute SP but get an error when you do it through linkedserver or openrowset - than problem is in permissions.
Check permissions on sysmergepublications table. If user, which you use for linked server or openrowset, has no grant on do select this table you need to add this permission to user.
I hope it will help you.
I believe what I am attempting to achieve may only be done through the use of Dynamic SQL. However, I have tried a couple of things without success.
I have a table in database DB1 (lets say DB1.dbo.table1, in a MS SQL server) that contains the names of other databases in the server (DB2,DB3, etc). Now, all the dbs listed in that table contain a particular table (lets call it desiredTable) which I want to query. So what I'm looking for is a way of creating a stored procedure/script/whatever that queries DB1.dbotable1 for the other DBs and then run a statement on each of the dbs retrieved, something like:
#DBNAME = select dbName from DB1.dbo.table1
select value1 from #DBNAME.dbo.desiredTable
Is that possible? I'm planning on running the sp/script in various systems DB1.dbo.table1 being a constant.
You need to build a query dinamically and then execute it. Something like this:
DECLARE #MyDynamicQuery VARCHAR(MAX)
DECLARE #MyDynamicDBName VARCHAR(20)
SELECT #MyDynamicDBName = dbName
FROM DB1.dbo.table1
SET #MyDynamicQuery = 'SELECT value1 FROM ' + #MyDynamicDBName + '.dbo.desiredTable'
EXEC(#MyDynamicQuery)
You can use the undocumented stored procedure, sp_MSForEachDB. The usual warnings about using an undocumented stored procedure apply though. Here's an example of how you might use it in your case:
EXEC sp_MSForEachDB 'SELECT value1 FROM ?.dbo.desiredTable'
Notice the use of ? in place of the DB name.
I'm not sure how you would limit it to only DBs in your own table. If I come up with something, then I'll post it here.
In SQL Server I could copy sql code out of an application and paste it into SSMS, declare & assign vars that exist in the sql and run. yay great debugging scenario.
E.g. (please note I am rusty and syntax may be incorrect):
declare #x as varchar(10)
set #x = 'abc'
select * from sometable where somefield = #x
I want to do something similar with Postgres in pgAdmin (or another postgres tool, any recommendations?) where I can just drop my SQL (params & all) into something that will run against Postgres DB.
I realise you can create pgscript, but it doesn't appear to be very good, for example, if I do the equivalent of above, it doesn't put the single quotes around the value in #x, nor does it let me by doubling them up and you don't get a table out after - only text...
Currently I have a piece of SQL someone has written that has 3 unique variables in it which are used around 6 times each...
So the question is how do other people debug SQL efficiently, preferably in a similar fashion to my SQL Server days.
You can achieve this using the PREPARE, EXECUTE, DEALLOCATE commands for handling statements, which is really what we are talking about here.
For example:
PREPARE test AS SELECT * FROM users WHERE first_name = $1;
EXECUTE test ('paul');
DEALLOCATE test;
Perhaps not as graphical as some may like, but certainly workable.
I would give a shot at writing a SQL function that wraps your query. It can be something as simple as
CREATE OR REPLACE FUNCTION my_function(integer, integer)
RETURNS integer
AS
$$
SELECT $1 + $2;
$$
LANGUAGE SQL;
SELECT my_function(1, 2);
I would do this instead of a PREPARE since it will be simpler to update it. Depending on how complex the function is, you might want to also look at some of the other PL's in Postgres.
SQL procs are notoriously hard to debug. My lame but practical solution has been to write log messages to a log table, like this (please excuse syntax issues):
create table log_message (
log_timestamp timestamp not null default current_timestamp,
message varchar(1000)
);
then add lines to your stored proc like:
insert into log_message (message) values ("The value of x is " || #x);
Then after a run:
select * from log_message order by 1;
It's not pretty, but works in every DB.
I have a SQL Insert query inside a stored proc, for inserting rows into a linked server table.
Since the stored proc is getting called within a parent transaction, this Insert statement tries to use a DTC for inserting rows into the linked server.
I would like to avoid DTC from getting involved.
Is there any way I can do that (like a hint) for the Insert SQL statement to ignore transactional scope?
My suggestion is that you store whatever you want to insert into a staging table, and once the procedure is over run the cross server insert. To my knowledge there is no way of ignoring the transaction you are in once you are within the SProc execution.
In contrast, if you use .NET 2.0's System.Transaction namespace, you can tell specific statements not to participate in any parent scope transaction. This would require you to write some of your logic in code rather than stored procedures, but would work.
Here's a relevant link.
Good luck,
Alan.
Try using openquery to call the linked server query/sp instead of direct calling
That worked for me
so instead of
insert into ...
select * from mylinkedserver.pubs.dbo.authors
e.g.
DECLARE #TSQL varchar(8000), #VAR char(2)
SELECT #VAR = 'CA'
SELECT #TSQL = 'SELECT * FROM OPENQUERY(MyLinkedServer,''SELECT * FROM pubs.dbo.authors WHERE state = ''''' + #VAR + ''''''')'
INSERT INTO .....
EXEC (#TSQL)