I have a massive SQL query that returns 4 tables. I'm able to return the first table but not the other tables here is the code I'm using below. I'm using a powershell script that I found online. I was about to get the second table by changing $DataSetTable = $DataSet.Tables[0] to $DataSetTable = $DataSet.Tables[1] the second table was return in the excel sheet but not the first one.
How can I all the tables?
$SQLServer = 'SQLCL01'
$Database = 'home'
$SqlQuery = #' Query goes here'#
## - Connect to SQL Server using non-SMO class 'System.Data':
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
## - Extract and build the SQL data object '$DataSetTable':
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSetTable = $DataSet.Tables[0]
## --------- Working with Excel ------- ##
## - Create an Excel Application instance:
$xlsObj = New-Object -ComObject Excel.Application
## - Create new Workbook and Sheet (Visible = 1 / 0 not visible)
$xlsObj.Visible = 1
$xlsWb = $xlsobj.Workbooks.Add()
$xlsSh = $xlsWb.Worksheets.item(1)
$xlsSh = $xlsWb.Worksheets.Item("Sheet1")
$xlsSh.Name = "Back date dump"
$xlsSh = $xlsWb.ActiveSheet
## - Build the Excel column heading:
[Array] $getColumnNames = $DataSetTable.Columns | Select ColumnName
## - Build column header:
[int] $RowHeader = 1
foreach ($ColH in $getColumnNames) {
$xlsSh.Cells.item(1, $RowHeader).font.bold = $true
$xlsSh.Cells.item(1, $RowHeader) = $ColH.ColumnName
$RowHeader++
}
## - Adding the data start in row 2 column 1:
[int] $rowData = 2
[int] $colData = 1
foreach ($rec in $DataSetTable.Rows) {
foreach ($Coln in $getColumnNames) {
## - Next line convert cell to be text only:
$xlsSh.Cells.NumberFormat = "#"
## - Populating columns:
$xlsSh.Cells.Item($rowData, $colData) = $rec.$($Coln.ColumnName).ToString()
$ColData++
}
$rowData++; $ColData = 1
}
## - Adjusting columns in the Excel sheet:
$xlsRng = $xlsSH.usedRange
$xlsRng.EntireColumn.AutoFit()
Loop over all tables in the collection:
$xlsObj = New-Object -ComObject Excel.Application
foreach ($DataSetTable in $DataSet.Tables) {
...
}
Related
I need to copy all data in MyDatabase on SERVER1 to SERVER2.
SERVER1 and SERVER2 are in different domains.
SERVER1 can be reached from SERVER2 using SQL Authentication.
SERVER1 and SERVER2 are not linked.
I only have SELECT rights on Mydatabase on SERVER1.
Only tooling to be used is PowerShell
My idea was to write a powershell script that get the names of all tables in MyDatabase on SERVER1, create a SELECT * FROM table statement and execute those one by one.. Insert the returned records in a new table in MyDatabase on SERVER2.
I used that trick to write the data from a database to CSV files. However, now I want to write the data not to a file but to a table.
See the code below
$server_src = "SERVER1" #$args[0]
$database_src = "MyDatabase" #$args[1]
$auth_src = "UID=MyUser;PWD=Mypassword" #$args[2]
$server_dst = "SERVER2" #$args[3]
$database_dst = "MyDatabase" #$args[4]
$auth_dst = "UID=MyUser;PWD=Mypassword" #$args[5]
# set source
$connection_string_src = "Server = $server_src; Database = $database_src; $auth_src;"
$connection_src = New-Object System.Data.SqlClient.SqlConnection $connection_string_src
# set destination
$connection_string_dst = "Server = $server_dst; Database = $database_dst; $auth_dst;"
$connection_dst = New-Object System.Data.SqlClient.SqlConnection $connection_string_dst
# sql to execute on source - get all tables in database $database_src
$sql_src = "select distinct
'$database_src' as dbname
,s.name as schemaname
,t.name as tablename
, 'SELECT * FROM ' + quotename('$database_src')+'.'+
quotename(s.name)+'.'+
quotename(t.name)
AS selectquery
,UPPER(s.name+'.'+t.name) as [dst_name]
from $database_src.sys.schemas s
inner join $database_src.sys.tables t on s.schema_id = t.schema_id"
# actually open the connection and execute $sql_src
$command_src = New-Object System.Data.SqlClient.SqlCommand $sql_src,$connection_src
$connection_src.Open()
$adapter_src = New-Object System.Data.SqlClient.SqlDataAdapter $command_src
$dataset_src = New-Object System.Data.DataSet
[void] $adapter_src.Fill($dataset_src)
$connection_src.Close()
#loop through source query's and insert data in $records
[array] $rows=($dataset_src.Tables | Select-Object -Expand Rows)
$i=0
foreach ($row in $rows)
{
$query = $rows[$i].selectquery
$dst_name = $rows[$i].dst_name
$command_src = New-Object System.Data.SqlClient.SqlCommand $query,$connection_src
$connection_src.Open()
$adapter_src = New-Object System.Data.SqlClient.SqlDataAdapter $command_src
$wanteddata_src = New-Object System.Data.DataSet
[void] $adapter_src.Fill($wanteddata_src)
$connection_src.Close()
# I think I only need to alter the 2 lines below to write the data to a table called $dst_name in the database $database_dst on the server $server_dstinstead of a file. Any existing table on the destination should be destroyed.
$outputfile = "C:\ExportedData\"+$def_filename+ ".csv"
$records= ($wanteddata_src.Tables | Select-Object -Expand Rows) | export-csv -Path $outputfile -Delimiter "|" -NoTypeInformation
$i++
}
I have a script which pulls data from a SQL database and gives the o/p in excel but the o/p comes under single column. I want to have it in different colums. This is my script:
$SQLServer = ""
$SQLDBName = ""
$SqlQuery = ""
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $SQLDBName; Integrated Security = True; User ID = $uid; Password = $pwd;"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSet.Tables[0] | out-file "C:\Users\sql.csv"
I also tried this script:
$SQLServer = ""
$SQLDBName = ""
$SqlQuery = "select request_number,actioneer,audited_date from form_submission where actioneer ='XXXX' AND audited_date between '2020-05-19 00:00:00:000' and '2020-05-19 19:16:00:000';"| | Select #{n='request_number';e={request_number}}, #{n='actioneer';e={actioneer}} ,#{n ='audited_date';e={audited_date}}
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database =
$SQLDBName; Integrated Security = True; User ID = $uid; Password = $pwd;"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSet.Tables[0] | out-file "C:\Users\sql.csv"
But this leads to the following error:
Exception calling "Fill" with "1" argument(s): "Must declare the scalar variable "#"."
At C:\Users\sraghur\Documents\sql powershell.ps1:12 char:1
+ $SqlAdapter.Fill($DataSet)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : SqlException
Is it really necessary to use .net methods instead of "Invoke-Sqlcmd"?
I usually use something like
Invoke-Sqlcmd -ServerInstance $strSQLServer -Database $strDatabase -QueryTimeout 180 -ConnectionTimeout 180 -query "select sqlcolumn1, sqlcolumn2, sqlcolumn3 from xxx" `
| Select-Object -Property sqlcolumn1, sqlcolumn2, sqlcolumn3 `
| Export-Csv "C:\Temp\SQLOutput.csv" -Encoding UTF8 -Delimiter ";" -NoTypeInformation
(You might have to install powershell module "SqlServer" beforehand)
On the other hand: Depending on the delimiter char used, the method of opening the file in Excel might be the problem: Restart Excel and choose "File > Open > ..." instead of double-clicking.
In case both doesn't work, you could also select - as LordPupazz recommended - the "single column" you were talking about and use "data > text to columns" (+ choose the correct delimiter).
Try this
Import-Module -Name SqlServer -Force
#Download ImportExcel Module if you don't have it already: https://www.powershellgallery.com/packages/ImportExcel/7.1.0
Import-Module -Name "C:\Program Files\WindowsPowerShell\Modules\ImportExcel\ImportExcel.psm1" -force
##########################################################################################################################################################################################
#Create Excel File in Temp
##########################################################################################################################################################################################
$Folder = "C:\Temp";
$ReportName = "ReportName";
#$FileNameCSV = $Folder + "\" + $ReportName + ".csv"
$FileNameExcel = $Folder + "\" + $ReportName +"_"+ $(get-date -f MM-dd-yyyy) + ".xlsx"
#$TestPath1 = Test-Path -Path $FileNameCSV;
$TestPath2 = Test-Path -Path $FileNameExcel;
#If($TestPath1 -eq $true){Remove-Item -Path $FileNameCSV -Force -ErrorAction Ignore};
If($TestPath2 -eq $true){Remove-Item -Path $FileNameExcel -Force -ErrorAction Ignore};
##########################################################################################################################################################################################
#Connect to SQL and load report table to PS variable
##########################################################################################################################################################################################
#Variables for your source SQL Server information
$SqlServer = "sqlserver.domain,port"
$SQLDB = "main"
$SQLQuery = "SELECT * FROM dbo.FOO"
#Wrap everything into a PS variable
$SQLProc = #{
ServerInstance = ""
Database = ""
Query = ""
}
#Set PS variable values
$SQLProc.ServerInstance = $SqlServer
$SQLProc.Database = $SQLDB
$SQLProc.Query = $SQLQuery
#Run SQL Query
$Results = Invoke-Sqlcmd #SQLProc
##########################################################################################################################################################################################
##Load data to Excel file on the specified Excel tab
##########################################################################################################################################################################################
#From the results dataset, select what columns you want in the Excel File
$Export = $Results | Select Column1,Column2,Column3
$Export | Export-Excel -Path $FileNameExcel -WorkSheetname "SheetName" -TableName "SQLData" -TableStyle Medium15 -AutoSize -BoldTopRow
Since you are using PowerShell already, why not to try dbatools.io and ImportExcel.
dbatools will help you to get data from the SQL with an easy way. Here is an example (for more check my series about dbatools:
$data = Invoke-DbaQuery -SqlInstance "localhost,1433" -Query "SELECT name,database_id from sys.databases"
The other will help you to get CSV exported to the XLSX.
$data | Export-Excel -Path HowToExcel.xlsx -Show
See how to use ImportExcel on my blog too
I'm running an enormous PowerShell script, which loops through every table and view in my database, and creates *.CSV text files for each table/view, containing the entire dataset.
(SELECT * FROM <table/view name>)
Any columns that are DATE or DATETIME are produced in the format MM/DD/YYYY
I need them to come out in the format YYYY/MM/DD
There are over 1,200 tables and views, so I can't manually specify the format for each affected column.
Can I change the default output format at the server level?
... or at the database level?
... or at the session level?
Here's my PowerShell code:
$server = "myserver"
$database = "mydb"
$tablequery = "SELECT s.name AS schema_name, t.name AS table_name FROM sys.tables t LEFT JOIN sys.schemas s ON s.schema_id = t.schema_id UNION ALL SELECT s.name, v.name FROM sys.views v LEFT JOIN sys.schemas s ON s.schema_id = v.schema_id"
#Delcare Connection Variables
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $tablequery
$command.Connection = $connection
#Load up the Tables in a dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$SqlAdapter.SelectCommand.CommandTimeout=600
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
# Loop through all tables and export a CSV of the Table Data
foreach ($Row in $DataSet.Tables[0].Rows)
{
$queryData = "SELECT * FROM [$($Row[0])].[$($Row[1])]"
#Specify the output location of your dump file
$extractFile = "C:\mydb\$($Row[0])_$($Row[1]).csv"
$command.CommandText = $queryData
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation
}
What if you add this line:
[System.Threading.Thread]::CurrentThread.CurrentCulture=[System.Globalization.CultureInfo]"fr-CA"
in this way:
...
...
# Loop through all tables and export a CSV of the Table Data
[System.Threading.Thread]::CurrentThread.CurrentCulture=[System.Globalization.CultureInfo]"fr-CA"
foreach ($Row in $DataSet.Tables[0].Rows)
{
$queryData = "SELECT * FROM [$($Row[0])].[$($Row[1])]"
#Specify the output location of your dump file
$extractFile = "C:\mydb\$($Row[0])_$($Row[1]).csv"
$command.CommandText = $queryData
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation
}
I'm trying to create a parameterized insert statement in my PS script and I keep getting an error about converting nvarchar:
Exception calling "ExecuteNonQuery" with "0" argument(s): "Conversion failed when converting the nvarchar value '(select e.endpointid from endpoints e inner join meters m on m.meterid=e.meterid where m.meterno ='06700' )' to data type int.".
Here's the relevant part of my script:
$linesplit = $line -split ","
$prefix = "$($linesplit[0]),$($linesplit[1]),$($linesplit[2]),$($linesplit[3])"
$numberOfMeters = $linesplit.length - 4
for($i = 1; $i -le $numberOfMeters; $i++){
$meternumber = $linesplit[3 + $i]
$meternumber = "'"+$meternumber+"'"
$notecontent = "'"+$prefix+"'"
$endpointid = "(select e.endpointid from endpoints e inner join meters m on m.meterid=e.meterid where m.meterno ="+$meternumber+" )"
write-host $endpointid -fore Cyan
$userid = "55"
$created = "(select GETDATE())"
$notetype = "0"
$note = $notecontent
write-host $note -fore Cyan
$sqlinsert = "
insert into endpointnotes (endpointId, userId, created, noteType, note)
VALUES (#endpointid,#userid,#created,#notetype,#note)
"
# Connect to SQL database and run insert statement
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "server=$Server;database=$CSDatabase;uid=;pwd="
$SqlConnection.open()
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.Parameters.Add("#endpointid", $endpointid)
$SqlCmd.Parameters.Add("#userid", $userid)
$SqlCmd.Parameters.Add("#created", $created)
$SqlCmd.Parameters.Add("#notetype", $notetype)
$SqlCmd.Parameters.Add("#note", $note)
$SqlCmd.CommandText = $sqlinsert
$SqlCmd.Connection = $SqlConnection
$SqlCmd.ExecuteNonQuery()
$SqlConnection.close()
}
Is it even possible to use a SELECT statement in one of my parameters?
Any help would be greatly appreciated!
Thanks
Hi I am trying to update a column though doesnt seem to work. It extracts the data and does the date conversion in the string but the update section of my code below does not want to insert the changes Powershell makes.
## connect to db and do stuff
$Connection = new-object system.data.SqlClient.SQLConnection("Data Source= (local);Integrated Security=SSPI;Initial Catalog=TESTDB");
$sqlqry = "select row_id, DestFileName from FL_Input Where DestFileName like '%[0-9] ' + 'Jan%' + ' [0-9]%'"
$Command = new-object system.data.sqlclient.sqlcommand
$Command.CommandText = $sqlqry
$Command.Connection = $Connection
$Connection.Open()
$UpdateCmd = new-object System.Data.SqlClient.SqlCommand
$UpdateCmd.Connection = $Connection
## for each filename, update timestamp - this replaces your $file in $files loop.
$Result = $Command.ExecuteReader()
while ($Result.Read()) {
$destfile = $Result['DestFileName']
$srcfile = $destfile
$row_id = $Result['row_id']
## do all your regex stuff here
$destfile -match '\d{2}\s\w+\s\d{2,4}' | Out-Null <# Test #>
$destfile -match '\d{2}\-\w+\-\d{4}' | Out-Null <# Test #>
$destFile -replace "$(($matches).values)" , "$(get-date "$(($matches).Values)" -Format yyyyMMdd)"
write-host "{0}->{1}" -f $destfile, $srcfile
# when you're finished doing your regex, push the result result back to the db:
$updateqry = "update FL_Input set DestFileName='{0}' WHERE Row_ID = {1} " -f $destfile, $row_id
$UpdateCmd.CommandText = $updateqry
$UpdateCmd.ExecuteNonQuery()
}
## all done
$Connection.Close
Aren't you missing single quotes around the row_id e.g.:
$updateqry = "update FL_Input set DestFileName='$destfile' WHERE Row_ID = '$row_id'"