PowerShell Script that Queries SQL Table to CSV File using Loop - sql

I have a basic SQL table of Employees. Using a powershell script I want to export all the employees who have made over 1000 sales to a .csv file and the rest into a different .csv file. I want to accomplish this task by using a loop. I am new to powershell and want to learn the foundations. Can anyone help?
SQL Table (not real employees)
This is what I have so far:
$connection.Open()
[System.Data.SqlClient.SqlDataReader]$result = $cmd.ExecuteReader()
$highDestFile = "C:\high-sales.csv"
$lowDestFile = "C:\low-sales.csv"
while($result.Read()) {
$ename = $result.GetValue(3);
$job = $result.GetValue(4);
$sales = $result.GetValue(7);
$tableArray = New-Object System.Collections.ArrayList
$tableArray.Add($ename)
$tableArray.Add($job)
$tableArray.Add($sales)
if($sales -ge 1000) {
Out-File -FilePath $highDestFile -InputObject $tableArray -Encoding ASCII -Append
} else {
Out-File -FilePath $lowDestFile -InputObject $tableArray -Encoding ASCII -Append
}
}
$connection.Close()

I'm not real familiar with the method you're using to get your results, but I think I have something similar that might be easier to work with for you. It will get all results into PS, and you can filter things from there, rather than getting one result at a time. You obviously know how to make your own SqlConnection and SqlCommand, I'm just including them for future readers to reference.
# Define SQL query
$sqlQuery = #"
SELECT *
FROM MyTable
"#
# Create a SqlConnection to connect to the SQL DB
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection
$sqlConnection.ConnectionString = "Server = $SqlServer; Database =$SqlCatalog; User Id = $User; Password = $Password"
# Create a SqlCommand object to define the query
$sqlCmd = New-Object System.Data.SqlClient.SqlCommand
$sqlCmd.CommandText = $sqlQuery
$sqlCmd.Connection = $sqlConnection
# Create a SqlAdapter that actually does the work and manages everything
$sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$sqlAdapter.SelectCommand = $sqlCmd
# Create an empty DataSet for the query to fill with its results
$dataSet = New-Object System.Data.DataSet
# Execute the query and fill the DataSet (then disconnect)
$sqlAdapter.Fill($dataSet)
$sqlConnection.Close()
# Convert DataSet table to array for ease of use
[Array]$Results = $dataSet.Tables[0]
Beyond that you could just use a pair of Where statements to filter your results, and output to files.
$Results | ?{[int]$_.Sales -ge 1000} | Set-Content $highDestFile
$Results | ?{[int]$_.Sales -lt 1000} | Set-Content $lowDestFile

Related

Backup database with multiple tables powershell

My objective is to backup a database to a spreadsheet. The database uses multiple tables. I'd like to save each table into it's own sheet in the spreadsheet. That way everything is in a single file. I also want to avoid using excel.
$server = "***\***"
$database = "***"
$username = "***"
$password = '***'
$tablequery = "SELECT schemas.name as schemaName, tables.name as tableName from sys.tables inner join sys.schemas ON tables.schema_id = schemas.schema_id"
#Delcare Connection Variables
$connectionTemplate = "Data Source={0};Integrated Security=false;Initial Catalog={1};User ID={2};Password={3}"
$connectionString = [string]::Format($connectionTemplate, $server, $database, $username, $password)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $tablequery
$command.Connection = $connection
#Load up the Tables in a dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
Import-Module ImportExcel
foreach ($Row in $DataSet.Tables[0].Rows) {
$queryData = "SELECT * FROM [$($Row[0])].[$($Row[1])]"
$command.CommandText = $queryData
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$Columns = $DataSet.Tables[0].Columns.ColumnName
$DataSet.Tables[0] | Select-Object -Property $Columns | Export-Excel -workSheetName $Row[1] -path C:\Users\bsben\Desktop\Demo.xlsx
}
Here's my current code. It does what I need. The problem is it has to write to the file for every table. The end result is for my current database it writes to the file 94 times. It ends up taking about 10 seconds.
The way I see it working is adapting what's going to be saved into an object.
This Idea is not an optimal solution and not recommended. Imagine if you have 1000+ tables in your DB and imaging you have 50+ DB inside your DB instance; then 1000+ sheets would be required in one excel file. Do you really think that it is possible to read after that in that level ?
Although there is a direct way to export the tables of DB to EXCEL. Please refer the link below :
Export-sql-server-tables-to-Excel/
Storing everything in one single Flat file is going to make life more complicated when you cannot do anything with the file in future.
Take backups based on the backup tools or script the mdf,ldf,ndf in proper paths.
As per the comment, here is the link for sql-database-backups-using-powershell-module-dbatools

Powershell import csv to SQL queries, export multiple csvs

I'm struggling to get my head around this one, I've only just begun looking at scripting in SQL, and my powershell is very limited. The requirments are basically this:
Utilisng Powershell, import a csv file which contains one column that needs to feed into multiple SQL queries via a loop, exporting a seperate csv file for each different query.
example import of csv:
Project (heading)
1000
1001
1002
Powershell:
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
import-csv $importProjectsCSV | ForEach-Object {
$query = "
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = $_.Project;
Select ProjectClient, Name
FROM dbo.CLIENTS
Where Project = $_.Project;
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $query
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$dataset.Table[0] | Export-csv "E:\" + $_.ProjectName + ".csv"
$dataset.Table[1] | Export-csv "E:\" + $_.ProjectName + ".csv"
The problem is that the variable isn't coming into the SQL query.
Is there a better way to handle this type of example?
Appreciate any pointers
Paul.
I would do something like this:
I must admit i havent been able to test it, and personally i usually use c# to query sql servers. So i might have gone a bit wrong somewhere.
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
$Projects = Import-Csv -Path $importProjectsCSV | % {$_.Project}
$DS_Projects = New-Object System.Data.DataSet
$DS_Clients = New-Object System.Data.DataSet
$query_pro = "Select ProjectLeader, ProjectTitle, Project FROM dbo.PROJECTS";
$query_Clients = "Select ProjectClient, Name, Project FROM dbo.CLIENTS";
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = $connection.CreateCommand();
$command.CommandText = $query_pro;
$sqlAdap = New-Object System.Data.SqlClient.SqlDataAdapter($command)
$sqlAdap.Fill($DS_Projects)
$command2 = $connection.CreateCommand();
$command2.CommandText = $query_Clients;
$sqlAdap2 = New-Object System.Data.SqlClient.SqlDataAdapter($command2)
$sqlAdap2.Fill($DS_Clients)
$connection.Close();
foreach($project in $Projects)
{
$DS_Projects.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
$DS_Clients.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
}
If I get every right you could do this with something like this:
$Projects = Import-Csv -Path 'C:\Temp\Projects.csv'
ForEach ($Project in $Projects.Projects) {
$Query= #"
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = '%{0}';
"# -f $Project
# just dummy action
$results = Invoke-Sqlcmd -ServerInstance "foo" -Database "bar" -Query $Query
# In Case results is a dataset do something like
$results | Export-Csv -Path ("E:\{0}.csv" -f $Project )
}

PowerShell Function multiple SQL queries output as CSV

I'm having problems getting PowerShell to run multiple SQL queries and export the results as CSV.
I'm trying to accomplish this using a Function but the problem occurs in the Process block when I expect two queries to run and output two CSV files.
I tried creating one function to run the query and a second function to create the CSV files but that didn't even run the SQL queries. I'm doing this without SQL being installed where this powershell script is executed from. -thanks!
Function Run-Query {
param([string[]]$queries,[string[]]$sheetnames)
Begin{
$SQLServer = 'ServerName'
$Database = 'DataBase'
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
}#End Begin
Process{
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $queries
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "C:\Scripts\$sheetnames.csv"
}#End Process
End{
$SqlConnection.Close()
}
}#End function run-query.
$queries = #()
$queries += #'
Select * from something
'#
$queries += #'
Select * from something2
'#
$sheetnames = #()
$sheetnames += 'Cert'
$sheetnames += 'Prod'
Run-Query -queries $queries
I'm not sure if SQL processes multiple queries separately so while you might be passing two different queries the SQL server might be interpreting them as one query (Not 100% sure this is happening, just a guess really)
You've put your queries in an array so we can easily loop through the array, run each query by itself and put the results into a CSV.
Here's how i'd modify your code to start with:
Function Run-Query
{
param([string[]]$queries,[string[]]$sheetnames)
Begin
{
$SQLServer = 'ServerName'
$Database = 'DataBase'
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
}#End Begin
Process
{
# Loop through each query
For($i = 0; $i -lt $queries.count; $i++)
{
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
# Use the current index ($i) to get the query
$SqlCmd.CommandText = $queries[$i]
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
# Use the current index ($i) to get the sheetname for the CSV
$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "C:\Scripts\$($sheetnames[$i]).csv"
}
}#End Process
End
{
$SqlConnection.Close()
}
}#End function run-query.
$queries = #()
$queries += #'
Select * from something
'#
$queries += #'
Select * from something2
'#
$sheetnames = #()
$sheetnames += 'Cert'
$sheetnames += 'Prod'
Run-Query -queries $queries -sheetnames $sheetnames

Powershell array correlation to SQL table dataset from powershell

First off, I'm new to stack. I have referenced stack many times in the past, but recently I have been stuck on this issue for quite sometime. So here goes.
My goal:
I am attempting to correlate an array output from VMware that matches a custom value on each VM machine. ( an asset ID ) to a value ( ID Key ) on a microsoft SQL 2000 server.
As such, since this server is pre 2005 I am unable to use the invoke-sqlcmd powershell command. I have to utilize the full SQL connection string and command structure to return a value out of this database. This sql statement and script works fine on its own. Meaning that the sql portion of this script, functioning on its own will pull results out of the database with a manual tag number put in place of my variable "$etag". I'm fairly new to powershell, and sql use from powershell.
So here is my script with names of the protected taken out.
#========================================================================
# Created on: 12/4/2013 2:01 PM
# Created by: Shaun Belcher
# Filename:
#========================================================================
function get-inventory
{
Add-PSSnapin VMware.VimAutomation.Core
$date=get-date
$vcenterserver = #("srv-1","srv-2","srv-3")
Connect-VIServer -server $vcenterserver
$toAddr="user#domain.com"
$fromAddr="user#domain.com"
$smtpsrv="mail.domain.com"
#Variables
$mdesks=#()
$sqlServer = "serverdb"
$sqlDBNAME = "instance"
$sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection
$DataSet = New-Object System.Data.DataSet
$sqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.connection = $sqlConnection
$sqlAdapter.SelectCommand = $sqlCmd
#db Connection
$sqlConnection.ConnectionString = "Server = $sqlServer; Database = $sqlDBname; Integrated Security=True;"
$SqlCmd.connection = $SqlConnection
$SqlCmd.commandtext = $sqlQuery
$sqlAdapter.SelectCommand = $sqlCmd
$sqlQuery += "SELECT INVHARDW_PropTag as proptag, invhardw_clientID as ClientID, invhardw_notes as Notes FROM INV_Hardware where invhardw_proptag = '$etag';"
$SqlCmd.commandtext = $sqlQuery
$sqlAdapter.SelectCommand = $sqlCmd
$sqlAdapter.Fill($DataSet)
$DataSet.Tables[0]
$sqlConnection.Close()
$mdesks = #($DataSet.Tables[0] | select propTag, ClientID, Notes)
$virtuals= #(Get-VM | select Name,vmhost,memoryMB,#{N="Datastore";E={[string]::Join(',',(Get-Datastore -Id $_.DatastoreIdList | Select -ExpandProperty Name))}})
$etags = #(Get-vm | Get-Annotation |select value,#{N="mDeskNote";E={[string]::Join(',',($mdesk | Where-Object {$mdesks.propTag = $_;}))}},#{N="mDeskClientID";E={[string]::Join(',',($mdesk | Where-Object {$mdesks.propTag = $_;}))}})
if($virtuals -ne $null){
$body = #("
<center><table border=1 width=50 % cellspacing=0 cellpadding=8 bgcolor=Black cols=3>
<tr bgcolor=White><td>Virtual Machine</td><td>Host Machine</td><td>Memory Allocated</td><td>DatastoreList</td><td>Asset Tag</td><td>App Note</td><td>App Client ID</td></tr>")
$i = 0
do {
#if($i % 2){$body += "<tr bgcolor=#D2CFCF><td>$($virtuals[$i].Name)</td></tr>";$i++}
#else {$body += "<tr bgcolor=#EFEFEF><td>$($virtuals[$i].Name)</td></tr>";$i++}
if($i % 2){$body += "<tr bgcolor=#D2CFCF><td>$($virtuals[$i].Name)</td><td>$($virtuals[$i].VMHost)</td><td>$($virtuals[$i].MemorymB)</td><td>$($virtuals[$i].datastore)</td><td>$($etags[$i].value)</td><td>$mdesks[$i].notes</td><td>$mdesks[$i].ClientID</td></tr>";$i++}
else {$body += "<tr bgcolor=#EFEFEF><td>$($virtuals[$i].Name)</td><td>$($virtuals[$i].VMHost)</td><td>$($virtuals[$i].memorymb)</td><td>$($virtuals[$i].datastore)</td><td>$($etags[$i].value)</td><td>$mdesks[$i].notes</td><td>$mdesks[$i].ClientID</td></tr>";$i++}
}
while ($virtuals[$i] -ne $null)
$body += "</table></center>"
# Send email.
if($attachmentPref){
$virtuals | Export-CSV "Inventory $($date.month)-$($date.day)-$($date.year).csv"
Send-MailMessage -To "$toAddr" -From "$fromAddr" -Subject "$vcenterserver Inventory = $countvms" -Body "$body" -Attachments "Inventory $($date.month)-$($date.day)-$($date.year).csv" -SmtpServer "$smtpsrv" -BodyAsHtml
Remove-Item "Inventory $($date.month)-$($date.day)-$($date.year).csv"
}
Else{
Send-MailMessage -To "$toAddr" -From "$fromAddr" -Subject "Inventory $vcenterserver = $countvms" -Body "$body" -SmtpServer "$smtpsrv" -BodyAsHtml
}
}
Disconnect-VIServer -Server $vcenterserver -Confirm:$false exit
get-inventory
This returns the information and sends it in an email with columns and rows of the information. Again, these are two working scripts that just do not return the result that is sought after.

Two SQL queries and pass each as a dataset to pass as parameters

Im very new to Powershell and I've scraped together the following code for a script that will check for an 'old' folder and create it if not found. It will then move the compressed weblogs from current location to the 'old' folder. I am wanting this written to pull the server names and website names from a sql query so it can be setup to run nightly and does not need to be updated with new or deleted servers. I have the following written so far but since I'm new I cannot figure out the last bit of syntax.
clear
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = "select hsa.servername from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlQuery1 = "select hah.vchhost from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
What I would like to do is have each $SQLQuery run and store each one in it's own dataset table so I can pass each one as a parameter for the following code. What I need to figure out is:
1)How do I write the above to run both SQL queries and have each one be either it's own dataset or table so I can them as parameters; $Servername and $HostedGroup?
2)How do I set this up to recurse the below code on each of the servers from the $Servername parameter?
$Servername = Dataset1
$Hostedgroup = Dataset2
$OldFolder = "\\$Servername\C$\Ren\Weblogs\$Hostedgroup\old"
$FolderExists = Test-Path $OldFolder
if($FolderExists -eq $False)
{
new-item \\$Servername\C$\Ren\Weblogs\$Hostedgroup\old -type directory
}
then
{
if(Test-Path \\$Servername\C$\Ren\Weblogs\old\W3SVC2)
{
get-childitem -path '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\W3SVC2' -recurse -include *.zip | move-item -destination '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\old'
$SqlAdapter.Fill($DataSet) returns an integer; you should prevent your script/function from returning it by assigning it to $null or using Out-Null. (e.g. $null = $SqlAdapter.Fill($DataSet))
For the answer to your question, you can return both fields from a single query, then iterate over the results. The pattern would look like this:
function Get-HostedServerApp {
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = #"
select hsa.vchappservername, hah.vchhost
from SERVER.dbo.hostedserversapp hsa
inner join SERVER.dbo.hostedapphosts hah on hsa.vchappservername = hah.vchappservername
where hsa.tirecordstatus = 1
order by hsa.vchappservername desc
"#
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$null = $SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$Dataset.Tables[0]
}
# Place your logic in this function
function SomeFunction {
param(
$Servername,
$Hostedgroup
)
"\\$Servername\C`$\Renaissance\Weblogs\$Hostedgroup\old"
}
$data = Get-HostedServerApp
$data| foreach{SomeFunction -ServerName $_.vchappservername -HostedGroup $_.vchhost}