How do I use a cursor to scroll a resultset from an ODBC datasource in PowerShell ?
Problem:
The SQL statement takes too long to execute (due to returning millions of rows). I only want to return the top 5 rows but this database does not support the top, limit or rownum select statement clauses).
Current code (that does not use cursors):
$datasourceName = "big_database"
$sqlQuery = "SELECT * as 'millions_of_rows' FROM big_table"
$odbcConnection = new-object Data.Odbc.OdbcConnection
$odbcConnection.ConnectionString = "dsn=$datasourceName"
$odbcConnection.open()
$resultSet = (new-object Data.Odbc.OdbcCommand($sqlQuery, $odbcConnection)).ExecuteReader()
$table = new-object "System.Data.DataTable"
$table.Load($resultSet)
#/* Display results */
$table
The like statement in my SQL query in powershell isn't producing the intended results. My 'like' statement is getting treated like an '=' sign.
for example
I have a table with the following values:
FieldNames
1. George Bush;George Foreman;Mary jones;
2. George Foreman;
3. George Foreman; Michael Smith;
4. Mary Jones;George Foreman
Variable:
$Expression = "George Foreman"
Three options I tried:
1.
Select FieldNames from table where FieldNames like '%$Expression%'
2.
Select FieldNames from table where FieldNames like '%$($Expression)%'
3.
Select FieldNames from table where FieldNames like '%George Foreman%'
I receive the same results for all three options (only row 2 returns). I would expect all the rows to return.
I can run this same query in SSMS and all rows return.
Has anyone ran across this problem or have any suggestions on how I can modify my select query to get all the rows using 'like'?
Formatted Powershell expression:
#Variables:
$Expression = "George Foreman"
$UserName = "UserName"
$password = "Password"
$Database = "Database"
$Server = "Server"
$Query="Select FieldNames from table where FieldNames like '%$Expression%' "
$ConnectionString = "Server=$Server ;User ID=$UserName; password=$password;
Integrated Security=True; database=$Database;"
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connection
$connection.Open()
$command = $connection.CreateCommand()
$command.CommandText = $Query
$result = $command.ExecuteReader()
$connection.Close()
I have a requirement where i need to see data for every week using a sql query.
I used query like SELECT * from table Between '27-08-2012' and '30-08-2012'.
Now my requirement is that i need a batch file that can given me this script based on user selection. Like user can give start date and end date and sql query should generate automatically.
A much less error prone way to get a date range is to use a graphical date picker
than manual user input which has to be checked for validity.
(month/day names will match your locale, not my German one)
This PowerShell script:
# Function Pick-Date
[void][System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
Function Pick-Date {
$Cal = new-object System.Windows.Forms.MonthCalendar
$Cal.ShowWeekNumbers = $true
$Cal.MaxSelectionCount = 10 # change this value for the max date distance
$Cal.Dock = 'Fill'
$Form = new-object Windows.Forms.Form
$Form.text = "Drag the mouse to select a date range then hit [enter]"
$Form.Size = new-object Drawing.Size #(656,620)
$btnSelect = new-object System.Windows.Forms.Button
$btnSelect.Size = "1,1"
$btnSelect.add_Click({ $Form.close() })
$Form.Controls.Add($btnSelect )
$Form.AcceptButton = $btnSelect
$Form.Controls.Add($Cal)
$Form.Add_Shown({$Form.Activate()})
[void]$Form.showdialog()
return ("SELECT * from table Between '"+
(Get-Date($Cal.SelectionStart) -format 'dd-MM-yyyy')+
"' and '"+
(Get-Date($Cal.SelectionEnd) -format 'dd-MM-yyyy')+
"'")
}
Pick-Date
Will have this output, you can save to a file.sql
PS> .\Pick-Date.ps1
SELECT * from table Between '27-08-2012' and '31-08-2012'
EDIT
This batch wrapper for the powerShell script will store the query in the variable SqlQuery
#Echo off&SetLocal EnableExtensions EnableDelayedExpansion
::Wrap Pick-Date.ps1 in same folder as batch
For /F "delims=" %%%A in (
'Powershell -NoP -NonI -NoLogo -File Pick-Date.ps1 '
) Do Set "SqlQuery=%%A"
Set SqlQuery
I am new to SQL, and what I needed to do was to combine 2 .mdf databases into one. I did that using SQL Server 2008 Manager - Tasks > Import/Export tables.The tables and views were copied successfully, but there are no Stored procedures in the new database. Is there any way to do that?
Right click on database
Tasks
Generate Scripts
Select the objects you wish to script
Script to File
Run generated scripts against target database
This code copies all stored procedures in the Master database to the target database, you can copy just the procedures you like by filtering the query on procedure name.
#sql is defined as nvarchar(max), #Name is the target database
DECLARE c CURSOR FOR
SELECT Definition
FROM [ResiDazeMaster].[sys].[procedures] p
INNER JOIN [ResiDazeMaster].sys.sql_modules m ON p.object_id = m.object_id
OPEN c
FETCH NEXT FROM c INTO #sql
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = REPLACE(#sql,'''','''''')
SET #sql = 'USE [' + #Name + ']; EXEC(''' + #sql + ''')'
EXEC(#sql)
FETCH NEXT FROM c INTO #sql
END
CLOSE c
DEALLOCATE c
Late one but gives more details that might be useful…
Here is a list of things you can do with advantages and disadvantages
Generate scripts using SSMS
Pros: extremely easy to use and supported by default
Cons: scripts might not be in the correct execution order and you might get errors if stored procedure already exists on secondary database. Make sure you review the script before executing.
Third party tools
Pros: tools such as ApexSQL Diff (this is what I use but there are many others like tools from Red Gate or Dev Art) will compare two databases in one click and generate script that you can execute immediately
Cons: these are not free (most vendors have a fully functional trial though)
System Views
Pros: You can easily see which stored procedures exist on secondary server and only generate those you don’t have.
Cons: Requires a bit more SQL knowledge
Here is how to get a list of all procedures in some database that don’t exist in another database
select *
from DB1.sys.procedures P
where P.name not in
(select name from DB2.sys.procedures P2)
I originally found this post looking for a solution to copying stored procedures from my remote production database to my local development database. After success using the suggested approach in this thread, I realized I grew increasingly lazy (or resourceful, whichever you prefer) and wanted this to be automated. I came across this link, which proved to be very helpful (thank you vincpa), and I extended upon it, resulting in the following file (schema_backup.ps1):
$server = "servername"
$database = "databaseName"
$output_path = "D:\prod_schema_backup"
$login = "username"
$password = "password"
$schema = "dbo"
$table_path = "$output_path\table\"
$storedProcs_path = "$output_path\stp\"
$views_path = "$output_path\view\"
$udfs_path = "$output_path\udf\"
$textCatalog_path = "$output_path\fulltextcat\"
$udtts_path = "$output_path\udtt\"
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.ConnectionInfo") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SmoExtended") | out-null
$srvConn = new-object Microsoft.SqlServer.Management.Common.ServerConnection
$srvConn.ServerInstance = $server
$srvConn.LoginSecure = $false
$srvConn.Login = $login
$srvConn.Password = $password
$srv = New-Object Microsoft.SqlServer.Management.SMO.Server($srvConn)
$db = New-Object ("Microsoft.SqlServer.Management.SMO.Database")
$tbl = New-Object ("Microsoft.SqlServer.Management.SMO.Table")
$scripter = New-Object Microsoft.SqlServer.Management.SMO.Scripter($srvConn)
# Get the database and table objects
$db = $srv.Databases[$database]
$tbl = $db.tables | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$storedProcs = $db.StoredProcedures | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$views = $db.Views | Where-object { $_.schema -eq $schema }
$udfs = $db.UserDefinedFunctions | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$catlog = $db.FullTextCatalogs
$udtts = $db.UserDefinedTableTypes | Where-object { $_.schema -eq $schema }
# Set scripter options to ensure only data is scripted
$scripter.Options.ScriptSchema = $true;
$scripter.Options.ScriptData = $false;
#Exclude GOs after every line
$scripter.Options.NoCommandTerminator = $false;
$scripter.Options.ToFileOnly = $true
$scripter.Options.AllowSystemObjects = $false
$scripter.Options.Permissions = $true
$scripter.Options.DriAllConstraints = $true
$scripter.Options.SchemaQualify = $true
$scripter.Options.AnsiFile = $true
$scripter.Options.SchemaQualifyForeignKeysReferences = $true
$scripter.Options.Indexes = $true
$scripter.Options.DriIndexes = $true
$scripter.Options.DriClustered = $true
$scripter.Options.DriNonClustered = $true
$scripter.Options.NonClusteredIndexes = $true
$scripter.Options.ClusteredIndexes = $true
$scripter.Options.FullTextIndexes = $true
$scripter.Options.EnforceScriptingOptions = $true
function CopyObjectsToFiles($objects, $outDir) {
#clear out before
Remove-Item $outDir* -Force -Recurse
if (-not (Test-Path $outDir)) {
[System.IO.Directory]::CreateDirectory($outDir)
}
foreach ($o in $objects) {
if ($o -ne $null) {
$schemaPrefix = ""
if ($o.Schema -ne $null -and $o.Schema -ne "") {
$schemaPrefix = $o.Schema + "."
}
#removed the next line so I can use the filename to drop the stored proc
#on the destination and recreate it
#$scripter.Options.FileName = $outDir + $schemaPrefix + $o.Name + ".sql"
$scripter.Options.FileName = $outDir + $schemaPrefix + $o.Name
Write-Host "Writing " $scripter.Options.FileName
$scripter.EnumScript($o)
}
}
}
# Output the scripts
CopyObjectsToFiles $tbl $table_path
CopyObjectsToFiles $storedProcs $storedProcs_path
CopyObjectsToFiles $views $views_path
CopyObjectsToFiles $catlog $textCatalog_path
CopyObjectsToFiles $udtts $udtts_path
CopyObjectsToFiles $udfs $udfs_path
Write-Host "Finished at" (Get-Date)
$srv.ConnectionContext.Disconnect()
I have a .bat file that calls this, and is called from Task Scheduler. After the call to the Powershell file, I have:
for /f %f in ('dir /b d:\prod_schema_backup\stp\') do sqlcmd /S localhost /d dest_db /Q "DROP PROCEDURE %f"
That line will go thru the directory and drop the procedures it is going to recreate. If this wasn't a development environment, I would not like programmatically dropping procedures this way. I then rename all the stored procedure files to have .sql:
powershell Dir d:\prod_schema_backup\stp\ | Rename-Item -NewName { $_.name + ".sql" }
And then run:
for /f %f in ('dir /b d:\prod_schema_backup\stp\') do sqlcmd /S localhost /d dest_db /E /i "%f".sql
And that iterates through all the .sql files and recreates the stored procedures. I hope that any part of this will prove to be helpful to someone.
use
select * from sys.procedures
to show all your procedures;
sp_helptext #objname = 'Procedure_name'
to get the code
and your creativity to build something to loop through them all and generate the export code :)
You can use SSMS's "Generate Scripts..." function to script out whatever you need to transfer. Right-click on the source database in SSMS, choose "Generate Scripts...", and follow the wizard along. Then run your resultant script that will now contain the stored procedure create statements.
You can generate scriptof the stored proc's as depicted in other answers. Once the script have been generated, you can use sqlcmd to execute them against target DB like
sqlcmd -S <server name> -U <user name> -d <DB name> -i <script file> -o <output log file>
Another option is to transfer stored procedures using SQL Server Integration Services (SSIS). There is a task called Transfer SQL Server Objects Task. You can use the task to transfer the following items:
Tables
Views
Stored Procedures
User-Defined Functions
Defaults
User-Defined Data Types
Partition Functions
Partition Schemes
Schemas
Assemblies
User-Defined Aggregates
User-Defined Types
XML Schema Collection
It's a graphical tutorial for Transfer SQL Server Objects Task.
In Mgmt Studio, right-click on your original database then Tasks then Generate Scripts... - follow the wizard.
SELECT definition + char(13) + 'GO' FROM MyDatabase.sys.sql_modules s INNER JOIN MyDatabase.sys.procedures p ON [s].[object_id] = [p].[object_id] WHERE p.name LIKE 'Something%'" queryout "c:\SP_scripts.sql -S MyInstance -T -t -w
get the sp and execute it
-- This program copies (CREATE OR ALTER) a single PROCEDURE from one database to another
declare #SourceDatabase nvarchar(50);
declare #SourceSchemaName nvarchar(50)
declare #TargetDatabase nvarchar(50);
declare #ProceduresName nvarchar(50);
declare #sql nvarchar(max)
set #SourceDatabase = N'Northwind' -- The name of the source database
set #SourceSchemaName = N'dbo' -- The name of Procedure SCHEME
set #ProceduresName = N'CustOrderHist' -- The name of Procedure
set #TargetDatabase = N'AdventureWorks' -- The name of the Target database
-- -------- - - -
-- If the PROCEDURE SCHEME does not exist, create it
set #sql = ' use [' +#TargetDatabase +'] ' +
' IF NOT EXISTS (SELECT * FROM sys.schemas WHERE name = ''' + #SourceSchemaName+''') '+
' BEGIN ' +
' EXEC('' CREATE SCHEMA '+ #SourceSchemaName +''') ' +
' END'
exec (#sql);
set #sql = ''
--
set #sql = #sql + ' use [' + #TargetDatabase +'] ;' +
' declare #sql2 nvarchar(max) ;' +
' SELECT #sql2 = coalesce(#sql2,'';'' ) + [definition] + '' ; '' ' +
' FROM ['+#sourceDatabase+'].[sys].[procedures] p ' +
' INNER JOIN ['+#sourceDatabase+'].sys.sql_modules m ON p.object_id = m.object_id '+
' where SCHEMA_NAME([schema_id]) = ''' +#SourceSchemaName +''' and [name] = N''' + #ProceduresName + ''' ; ' +
' set #sql2 = replace(#sql2,''CREATE PROCEDURE'',''CREATE OR ALTER PROCEDURE'')' +
' exec (#sql2)'
exec (#sql)
I am using a function that collects data from a SQL server:
function Invoke-SQLCommand {
param(
[string] $dataSource = "myserver",
[string] $dbName = "mydatabase",
[string] $sqlCommand = $(throw "Please specify a query.")
)
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = `
"Server=$dataSource;Database=$dbName;Integrated Security=True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $sqlCommand
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
}
It works great but returns only one table. I am passing several Select statements, so the dataset contains multiple tables.
I replaced
$DataSet.Tables[0]
with
for ($i=0;$i -lt $DataSet.tables.count;$i++){
$Dataset.Tables[$i]
}
but the console only shows the content of the first table and blank lines for each records of what should be the second table. The only way to see the result is to change the code to
$Dataset.Tables[$i] | out-string
but I do not want strings, I want to have table objects to work with.
When I assign what is returned by the Invoke-SQLCommand to a variable, I can see that I have an array of datarow objects but only from the first table. What happened to the second table?
Any help would be greatly appreciated.
Thanks
I tried your function (that returns $DataSet.Tables) and it worked pretty well for me. This command returned rows from both tables:
$t = Invoke-sqlcommand '.\sql2005' 'AdventureWorksDW' `
"SELECT * FROM DimOrganization; SELECT * FROM DimSalesReason"
$t[0] #returns rows from first table
$t[1] #returns rows from second table
Anyway, what I would recommend:
First I would discard output from Fill:
$SqlAdapter.Fill($DataSet) > $null
It is returned as well, but that's not probably desired.
As in your case Invoke-SqlCommand doesn't work, I would try to return 1 dim array like this:
function Invoke-SQLCommand {
...
,$DataSet.Tables
}
Consider that PowerShell treats DataTable specially and when trying to format it, it unravels Rows collection (credits to x0n). That's why just executing $t from my example displays all the rows returned from the command.
Thank you for your answer.
Well, I can't explain why it works for you and not for me.
If I run the exact same command as you (except for the data source, mysqlserver\sqlexpress in my case), $t[0] only returns the first row of the first table and $t[1] the second row.
What seems to be happening in my case is that the rows from all tables are merged so I end up with one big set of datarows, not the individual tables expected.
I ended replacing:
for ($i=0;$i -lt $DataSet.tables.count;$i++){
$Dataset.Tables[$i]
}
with just
$Dataset
I can then reference the individual tables from my script by using $t.Tables[0] and $t.Tables[1].
Thanks again