I have an old SQL script that is currently run by loading it into SQL Server Management studio and running it. I'd like to clean this up by turning it into a series of functions that are stored in the database itself.
The basic sequence of steps that the current code does is like this:
(Miles of SQL logic)
Create a temporary table
BULK INSERT from a CSV file into the temporary table
Massage the data
Merge the data into the "real" table
DROP the temporary table
(Miles of SQL logic)
I'd like to wrap steps 1-5 in a function, but I'm stuck at how to perform a BULK INSERT when you can't BULK INSERT into a table variable, and you're also not allowed to create temporary tables from within a function.
So what's the right way to fix this issue?
Thanks!
As already mentionned in the comment, the solution that differs the less to yours is doing that in a stored procedure rather than in a functoin, which is intended to modify the content of a table.
On a short term perspective, this should be clearly the easiest to implement for you but on a long term learnin SSIS could be a good investment.
Related
I would like to know if there was a trigger which i can use to drop a temporary sql table when a report is closing in ssrs?
Background
I deserialise data in a stored procedure and after deserialising i want to reuse the data, so the plan is to put the deserialised data in to a temporary table and create multiple stored procedures with different result sets (To be used with multiple datasets in ssrs report) and finally drop the temporary table with deserialised data.
Currently i am thinking may be i should create a sql agent job which will drop the table everyday at a certain time (say 1am in the morning). However I have a sneaky feeling that there might be a better way to do this?
Any help is greatly appreciate. Thanks.
My assumptions:
You can't de-serialise the data when you run the report because it
takes too long?
You are happy for the data to be out of date for your report - because you are thinking of creating a table at 01:00 and potentially running the report at e.g. 17:00
You use the phrase temporary table but that's not what you mean. A temporary table in SQL server is one which is available for the lifetime of the connection and is then dropped.
If my assumptions are correct your plan is fine. You wouldn't drop the table though - Have a look at TRUNCATE TABLE.
You may want to use SSIS, however it's not worth it for really simple procedures.
I need to run a query in SQL Server, where I have a particular number of values stored individually on separate lines in a text file, and I need to run a query in SQL server to check if a value in a column of the table matches any one of the value stored in the txt file.
How should I go about doing this ?
I am aware of how to formulate various types of queries in SQL Server, just not sure how to run a query that is dependent on a file for its query parameters.
EDIT :
Issue 1 : I am not doing this via a program since the query that I need to run traverses over 7 million datapoints which results in the program timing out before it can complete, hence the only alternative I have left is to run the query in SQL Server itself without worrying about the timeout.
Issue 2 : I do not have admin rights to the database that I am accessing which is why there is no way I could create a table, dump the file into it, then perform a query by joining those tables.
Thanks.
One option would be to use BULK INSERT and a temp table. Once in the temp table, you can parse the values. This is likely not the exact answer you need, but based on your experience, I'm sure you could tweak as needed.
Thanks...
SET NOCOUNT ON;
USE Your_DB;
GO
CREATE TABLE dbo.t (
i int,
n varchar(10),
d decimal(18,4),
dt datetime
);
GO
BULK INSERT dbo.t
FROM 'D:\import\data.txt'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\n');
There are lots of approaches.
Mine would be to import the file to a table, do the comparison with a regular SQL query, and then delete the file-data table if you don't need it anymore.
Bulk import the data from text file into a temporary table.
Execute the query to do the comparison between your actual physical table & temporary table.
I have a application written by other team in our company that insert data in one table.
Let's say they write data into table Log1 with fields:
Id (auto-generated primary key);
KeyId;
Value1;
Value2;
Value3.
For now I need to have another additional record in another table (Log2) from them that has only part of their data:
Id (it will be my own auto-generated Id);
KeyId;
Value1.
I see 2 ways to do that:
Create trigger that on adding records into Log1 will automatically create record in Log2 with required data;
Implement SP that will accept all required data for Log1 table and will create records in both tables, then ask those applications authors use SP instead of direct INSERT query.
What do you think is the best way in this case and why?
Thank you very much for your help.
P.S. I'm using MS SQL 2005
Go with option 1.
It means that the tables will be synchronised properly even if the "correct" stored procedure interface isn't used and it will be easier and more efficient to insert multiple rows (How would you do this with a stored procedure in SQL Server 2005? - Call it multiple times? Convert all the data to XML format first?)
If you use a trigger, be aware that as it seems both Log1 and Log2 use identity columns, that you can't use SELECT ##IDENTITY to return the PK of Log1 - you will need to use SCOPE_IDENTITY().
On the other hand, if you use a SPROC, what you can do is revoke INSERT privileges to your table from (just about) everyone, and instead grant EXEC on your SPROC. This way access to your table should be fairly well guarded.
The only way to really guarantee your data integrity is with a trigger. There is always a chance that someone will execute an operation (bulk operation, sql insert statement, etc.) that will bypass your SP.
Go with option 2.
Triggers should be avoided whenever possible.
One not so obvious reason: Have you ever used SQL Server replication facilities? Triggers won't be very straightforward to replicate. (ie it is not as easy as a couple of clicks, like it is for tables for instance). But I'm going off topic ... bottom line, triggers are evil... avoid when you can.
EDIT
More reasons: Triggers are not easy to see like other objects in the DBMS. On the application side, they are invisible, and if not well documented, they tend to be forgotten. If there are changes to the schema ... oh well, it's just easier to maintain stuff with stored procedures.
I read about temporary tables, global temporary tables and table variables. I understood it but could not imagine a condition when I have to use this. Please elaborate on when I should use the temporary table.
Most common scenario for using temporary tables is from within a stored procedure.
If there is logic inside a stored procedure which involves manipulation of data that cannot be done within a single query, then in such cases, the output of one query / intermediate results can be stored in a temporary table which then participates in further manipulation via joins etc to achieve the final result.
One common scenario in using temporary tables is to store the results of a SELECT INTO statement
The table variable is relatively new (introduced in SQL Server 2005 - as far as i can remember ) can be used instead of the temp table in most cases. Some differences between the two are discussed here
In a lot of cases, especially in OLTP applications, usage of temporary tables within your procedures means that you MAY possibly have business processing logic in your database and might be a consideration for you to re-look your design - especially in case of n tier systems having a separate business layer in their application.
The main difference between the three is a matter of lifetime and scope.
By a global table, I am assuming you mean a standard, run of the mill, table. Tables are used for storing persistent data. They are accessible to all logged in users. Any changes you make are visible to other users and vice versa.
A temporary table exist solely for storing data within a session. The best time to use temporary tables are when you need to store information within SQL server for use over a number of SQL transactions. Like a normal table, you'll create it, interact with it (insert/update/delete) and when you are done, you'll drop it. There are two differences between a table and a temporary table.
The temporary table is only visible to you. Even if someone else creates a temporary table with the same name, no one else will be able to see or affect your temporary table.
The temporary table exists for as long as you are logged in, unless you explicitly drop it. If you log out or are disconnected SQL Server will automatically clean it up for you. This also means the data is not persistent. If you create a temporary table in one session and log out, it will not be there when you log back in.
A table variable works like any variable within SQL Server. This is used for storing data for use in a single transaction. This is a relatively new feature of TSQL and is generally used for passing data between procedures - like passing an array. There are three differences between a table and a table variable.
Like a temporary table, it is only visible to you.
Because it is a variable, it can be passed around between stored procedures.
The temporary table only exists within the current transaction. Once SQL Server finishes a transaction (with the GO or END TRANSACTION statements) or it goes out of scope, it will be deallocated.
I personally avoid using temporary tables and table variables, for a few reasons. First, the syntax for them is Microsoft specific. If your program is going to interact with more than one RDBMS, don't use them. Also, temporary tables and table variables have a tendency to increase the complexity of some SQL queries. If your code can be accomplished using a simpler method, I'd recommend going with simple.
I have a VB.Net application that populates an Oracle 11g global temporary table. I can see data in the temporary table by loading it into a grid and everything looks correct.
However, when I call an Oracle stored procedure from VB.Net that would manipulate the data in this temporary table, the stored procedure reports that my temporary table is empty.
I understood that data in an Oracle global temporary table should be visible to all sessions so I'm not sure why this is happening and what the solution would be.
Please advise.
Regards,
M. R.
I think you are wrong, my understanding is that the data is session specific so it won't be visible to other sessions.
Another similar thing that can sometimes be confusing is that unless the table has been created with ON COMMIT PRESERVE ROWS, then it'll clear it self out when you commit.