I've successfully created many connections in an Excel file to a Database so the issue is directly related only to this particular scenario. Server is 2012, Excel is 2013.
I have the following SP:
IF OBJECT_ID('usp_LockUnlockCustomer', 'P') IS NOT NULL
DROP PROCEDURE usp_LockUnlockCustomer
GO
CREATE PROCEDURE usp_LockUnlockCustomer
#Lock AS CHAR(10), #Id AS nvarchar(50), #LockedBy AS nvarchar(50)=null
AS BEGIN
SET NOCOUNT ON;
IF #Lock = 'Lock'
INSERT INTO IO_Call_DB..IdLock (Id, LockedBy, LockedDtTm)
VALUES(#Id,#LockedBy,GETDATE())
;
IF #Lock = 'Unlock'
DELETE FROM IO_Call_DB..IdLock
WHERE Id = #Id
;
END;
--EXEC usp_LockUnlockCustomer 'Lock','123456789', 'Test User'
The above SP is called via some VBA as follows:
With ActiveWorkbook.Connections("usp_LockUnlockCustomer").OLEDBConnection
.CommandText = "EXECUTE dbo.usp_LockUnlockCustomer '" & bLock & "','" & Id & "','" & LockedBy & "'"
End With
I have tested the string and the string is formatted correct & contains all required data.
The connection between Excel and SQL is created via "Data > From Other Sources > From SQL Server", it's a fairly standard process which has worked for all other SP's and general queries.
I think, because this connection is not returning data to Excel (I only set up the connection rather than specifying that Excel should return data to a cell) that this may be the issue.
Has anyone experienced this issue before?
EDIT1: I have resolved the issue but it's not a particularly great outcome. Some help would be appreciated.
To resolve the issue, you have to include a "select * from" process at the end of the stored procedure and also tell Excel to output the data to a range within the workbook. This allows the .Refresh portion of the VBA to do whatever it does & submit the SP to SQL.
Essentially, you're being forced to create a data table - but I don't want any data, I just want to submit a command.
So, how do you submit a command and not have Excel require that you 1) explicitly state where the data should be put 2) include a SELECT statement within the stored procedure when I don't require any data to be returned.
My fix was to "select top 0" from the table, at least that way the data table being output to Excel won't grow.
In my experience if you generate the database connection in VBA, (there are multiple previous questions about that), rather than rely on an existing workbook connection, your stored procedure will execute regardless of what it returns.
The problem I have is that by merely creating the connection without specifying a cell to return data to, nothing happens.
In addition, if I specify a cell to return data to, nothing happens unless I use my 'fix' which is to create an empty table at the end of the SP.
Related
I would appreciate some ideas, discussion and feasibility of same, regarding bulk updates on a large table in SQL Server 2008+.
Currently I have a table with 10,000 rows and 160 columns. This table is updated very frequently with 1 to 100+ column changes per row depending on the process. Using the 'standard newbie' table update using a DataAdapter is very slow and unsuitable.
The quest is to find a faster way. I have tried fine-tuning the DataAdapter.Update with batch size, regardless the more heavy updates take 10-15 seconds. In the meanwhile SqlBulkCopy imports the whole table in (ball park) 1-3 seconds. When the update procedure takes place 30-50 times in a process the 10s-15s add up!
Being internet self thought, I have gaps in my experience, however there are 2 possibilities that I can think of that may be better at accomplishing the task of the update.
Dump the table content from the database and repopulate the table using SqlBulkcopy.
Using a stored procedure with a table passed to it with a merge SQL statement.
The main issue is data safety, although this is a local single user application there needs to be a way to handle errors roll back. From my understanding the dump and replace would be simpler, but perhaps more prone to data loss? The stored procedure would be far more extensive to set up as the update statement would have to have all the update columns typed individually and maintained for changes. Unless there is one 'Update *' statement :).
In trying to keep this short I want to keep this at a concept level only , but will appreciate any different ideas or links and advice.
EDIT further info:
The table has only one index, the ID column. Its a simple process of storing incoming (and changing) data to a simple datatable. and the update can be anywhere between 1 row to 1000 rows. The program stores the information to the database very often, and can be some or nearly all the columns. Building a stored procedure for each update would be impossible as I don't know which data will be updated, you can say that all of the columns will be updated (except the ID column and a few 'hard' data columns) it depends on what the update input is. So there is no fine tuning the update to specific columns unless I list nearly all of them each time. In which case one stored procedure would do it.
I think the issue is the number of 'calls' to the database are made using the current data adapter method.
EDIT:
3 WHat about a staging table where I bulk copy the data to and then have a store procedure do the update. Wouldn't that cut down the SQL trafic? I think that is the problem with the dataadapter update.
Edit: Posted an atempt of concept 1 in an answer to this thread.
Thank you
Dropping the table and reloading the entire thing with a bulk copy is not the correct way.
I suggest creating a stored procedure for each process that updates the table. The procedure should take as input only the columns that need to be updated for that specific process and then run a standard SQL update command to update those columns on the specified row. If possible, try to have indexes on the column(s) that you use to find the record(s) that need to be updated.
Alternately, depending on which version of the .Net framework you're using, you might try using Entity Framework if you don't want to maintain a whole list of stored procedures.
I have coded the following mockup to dump all rows from the table, bulkcopy the table in memory into an sql staging table and then move the data back into the original table. So by updating the data in that table.
Time taken 1.1 to 1.3 seconds
certainly a very attractive time compared to the 10-15s it takes to update. I have placed the truncete code for the staging table on top so that there is always one copy of the information in the database. Although the original table will not have the updated info untill the process completed.
What are the pitfals associate with this aproach ? What can I do about them? I must state that the table is unlikely to ever get beyond 10000 rows so the process will work.
Try
ESTP = "Start Bulk DBselection Update"
Dim oMainQueryT = "Truncate Table DBSelectionsSTAGE"
Using con As New SqlClient.SqlConnection(RacingConStr)
Using cmd As New SqlClient.SqlCommand(oMainQueryT, con)
con.Open()
cmd.ExecuteNonQuery()
con.Close()
End Using
End Using
ESTP = "Step 1 Bulk DBselection Update"
Using bulkCopy As SqlBulkCopy = New SqlBulkCopy(RacingConStr)
bulkCopy.DestinationTableName = "DBSelectionsSTAGE"
bulkCopy.WriteToServer(DBSelectionsDS.Tables("DBSelectionsDetails"))
bulkCopy.Close()
End Using
ESTP = "Step 2 Bulk DBselection Update"
oMainQueryT = "Truncate Table DBSelections"
Using con As New SqlClient.SqlConnection(RacingConStr)
Using cmd As New SqlClient.SqlCommand(oMainQueryT, con)
con.Open()
cmd.ExecuteNonQuery()
con.Close()
End Using
End Using
ESTP = "Step 3 Bulk DBselection Update"
oMainQueryT = "Insert INTO DBSelections Select * FROM DBSelectionsSTAGE"
Using con As New SqlClient.SqlConnection(RacingConStr)
Using cmd As New SqlClient.SqlCommand(oMainQueryT, con)
con.Open()
cmd.ExecuteNonQuery()
con.Close()
End Using
End Using
Data_Base.TextBox25.Text = "Deleting data - DONE "
Data_Base.TextBox25.Refresh()
Catch ex As Exception
ErrMess = "ERROR - occured at " & ESTP & " " & ex.ToString
Call WriteError()
Call ViewError()
End Try
I have a form button that runs a delete query and an append query. I would like to be able to show the user how many rows will be deleted and how many will be appended and store them on the form. The queries already exist outside the VBA for the button. So I am just calling the queries with DoCmd.OpenQuery. I turned off the warnings so the popup doesn't happen that says "Do you really want to delete this many rows?" But I would still like to find out how many rows will be deleted or appended and store that number as a variable somewhere so the user can see it. How can I make that happen?
I tried a DCount function but because these are action queries it won't work. I would also like to avoid putting the SQL for the queries in my VBA. I like that they are separate from the VBA and can be edited on their own.
Assuming ACE / Jet, you can use an instance of currentdb to run the queries and then look at the RecordsAffected property.
Dim db As Database
Set db = CurrentDb
db.Execute "Delete From Table1 Where Id = 1"
Debug.Print db.RecordsAffected
Or with a saved query
Dim db As Database
Set db = CurrentDb
db.Execute "ADelete"
Debug.Print db.RecordsAffected
I have a stored procedure with four parameters, I'm trying to connect it to an Excel 2010 workbook but I'm having difficulty passing multiple parameters.
I followed the steps outlined in Running a SQL Stored Procedure in Excel and was able to connect my stored procedure to Excel and pass one parameter.
But after I got that to work I had to modify my stored procedure and add three more parameters and now I can't figure out the syntax for the command text in my Excel connection to pass multiple parameters.
My stored procedure name is [VIDIR].[_cornie_Report_Shipped_Serial_Numbers] and I used the following VBA code in my Excel workbook to successfully refresh the connection with only one parameter.
Sub RefreshQuery()
With ActiveWorkbook.Connections("VIDIR3 Arborg_Test_App country").OLEDBConnection
.CommandText = "[VIDIR].[_cornie_Report_Shipped_Serial_Numbers] '%" & Range("C3").Value & "%'"
End With
ActiveWorkbook.Connections("VIDIR3 Arborg_Test_App country").Refresh
End Sub
The four parameter names in the modified stored procedure are:
#BillName varchar(60)
,#ShipName varchar(60)
,#Item varchar(40)
,#FamilyCode varchar(40)
Use the logic described in https://superuser.com/questions/294779/pass-a-cell-contents-as-a-parameter-to-an-excel-query-in-excel-2007 (second part of the first answer, involving CreateParameter and the Parameters collection. See also http://www.ozgrid.com/forum/showthread.php?t=170413 for another example.
I'm trying to define a Data Driven Subscription for a report in SSRS 2005.
In Step 3 of the set up you're asked for:
" a command or query that returns a list of recipients and optionally returns fields used to vary delivery settings and report parameter values for each recipient"
This I have written and it returns the data without a hitch. I press next and it rolls onto the next screen in the set up which has all the variables to set for the DDS and in each case it has an option to "Select Value From Database"
I select this radio button and press the drop down. No fields are available to me.
Now the only way I could vary the number of parameters returned by the SP was to have the SP write the SQL to an nvarchar variable and then at the end execute the variable as sql. I have tested this in the Management Studio and it returns the expected fields. I even named them after the fields in SSRS but the thing won't put the field names into the dropdowns.
I've even taken the query body out of the Stored Proc, verified it in SSRS and then tried that. It doesn't work either.
Can anyone shed any light into what I'm doing wrong?
You may need to start your stored proc with something like this:
CREATE PROCEDURE [GetRecipients]
AS
SET NOCOUNT ON
If 1=0
BEGIN
Select CAST(NULL as nvarchar(50)) as RecipientEmail,
CAST(NULL as integer) as Param1,
CAST(NULL as nvarchar(10)) as Param2,
CAST(NULL as DATETIME) as Param3
END
... insert your code here ...
End;
This was needed for a procedure I used as a data source in SSIS that used temp tables. The Select at the top with the format of the final output is never run due to the If contstruct, but it allows SSIS (and possibly SSRS) to see and derive the metadata for the output. I believe that this is due to SSIS and SSRS looking for the first select in your code to try and derive the metadata.
I've been asked to implement some code that will update a row in a MS SQL Server database and then use a stored proc to insert the update in a history table. We can't add a stored proc to do this since we don't control the database. I know in stored procs you can do the update and then call execute on another stored proc. Can I set it up to do this in code using one SQL command?
Either run them both in the same statement (separate the separate commands by a semi-colon) or a use a transaction so you can rollback the first statement if the 2nd fails.
You don't really need a stored proc for this. The question really boils down to whether or not you have control over all the inserts. If in fact you have access to all the inserts, you can simply wrap an insert into datatable, and a insert into historytable in a single transasction. This will ensure that both are completed for 'success' to occur. However, when accessing to tables in sequence within a transaction you need to make sure you don't lock historytable then datatable, or else you could have a deadlock situation.
However, if you do not have control over the inserts, you can add a trigger to certain db systems that will give you access to the data that are modified, inserted or deleted. It may or may not give you all the data you need, like who did the insert, update or delete, but it will tell you what changed.
You can also create sql triggers.
Insufficient information -- what SQL server? Why have a history table?
Triggers will do this sort of thing. MySQL's binlog might be more useful for you.
You say you don't control the database. Do you control the code that accesses it? Add logging there and keep it out of the SQL server entirely.
Thanks all for your reply, below is a synopsis of what I ended up doing. Now to test to see if the trans actually roll back in event of a fail.
sSQL = "BEGIN TRANSACTION;" & _
" Update table set col1 = #col1, col2 = #col2" & _
" where col3 = #col3 and " & _
" EXECUTE addcontacthistoryentry #parm1, #parm2, #parm3, #parm4, #parm5, #parm6; " & _
"COMMIT TRANSACTION;"
Depending on your library, you can usually just put both queries in one Command String, separated by a semi-colon.