Powershell import csv to SQL queries, export multiple csvs - sql

I'm struggling to get my head around this one, I've only just begun looking at scripting in SQL, and my powershell is very limited. The requirments are basically this:
Utilisng Powershell, import a csv file which contains one column that needs to feed into multiple SQL queries via a loop, exporting a seperate csv file for each different query.
example import of csv:
Project (heading)
1000
1001
1002
Powershell:
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
import-csv $importProjectsCSV | ForEach-Object {
$query = "
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = $_.Project;
Select ProjectClient, Name
FROM dbo.CLIENTS
Where Project = $_.Project;
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $query
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$dataset.Table[0] | Export-csv "E:\" + $_.ProjectName + ".csv"
$dataset.Table[1] | Export-csv "E:\" + $_.ProjectName + ".csv"
The problem is that the variable isn't coming into the SQL query.
Is there a better way to handle this type of example?
Appreciate any pointers
Paul.

I would do something like this:
I must admit i havent been able to test it, and personally i usually use c# to query sql servers. So i might have gone a bit wrong somewhere.
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
$Projects = Import-Csv -Path $importProjectsCSV | % {$_.Project}
$DS_Projects = New-Object System.Data.DataSet
$DS_Clients = New-Object System.Data.DataSet
$query_pro = "Select ProjectLeader, ProjectTitle, Project FROM dbo.PROJECTS";
$query_Clients = "Select ProjectClient, Name, Project FROM dbo.CLIENTS";
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = $connection.CreateCommand();
$command.CommandText = $query_pro;
$sqlAdap = New-Object System.Data.SqlClient.SqlDataAdapter($command)
$sqlAdap.Fill($DS_Projects)
$command2 = $connection.CreateCommand();
$command2.CommandText = $query_Clients;
$sqlAdap2 = New-Object System.Data.SqlClient.SqlDataAdapter($command2)
$sqlAdap2.Fill($DS_Clients)
$connection.Close();
foreach($project in $Projects)
{
$DS_Projects.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
$DS_Clients.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
}

If I get every right you could do this with something like this:
$Projects = Import-Csv -Path 'C:\Temp\Projects.csv'
ForEach ($Project in $Projects.Projects) {
$Query= #"
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = '%{0}';
"# -f $Project
# just dummy action
$results = Invoke-Sqlcmd -ServerInstance "foo" -Database "bar" -Query $Query
# In Case results is a dataset do something like
$results | Export-Csv -Path ("E:\{0}.csv" -f $Project )
}

Related

Backup database with multiple tables powershell

My objective is to backup a database to a spreadsheet. The database uses multiple tables. I'd like to save each table into it's own sheet in the spreadsheet. That way everything is in a single file. I also want to avoid using excel.
$server = "***\***"
$database = "***"
$username = "***"
$password = '***'
$tablequery = "SELECT schemas.name as schemaName, tables.name as tableName from sys.tables inner join sys.schemas ON tables.schema_id = schemas.schema_id"
#Delcare Connection Variables
$connectionTemplate = "Data Source={0};Integrated Security=false;Initial Catalog={1};User ID={2};Password={3}"
$connectionString = [string]::Format($connectionTemplate, $server, $database, $username, $password)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $tablequery
$command.Connection = $connection
#Load up the Tables in a dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
Import-Module ImportExcel
foreach ($Row in $DataSet.Tables[0].Rows) {
$queryData = "SELECT * FROM [$($Row[0])].[$($Row[1])]"
$command.CommandText = $queryData
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$Columns = $DataSet.Tables[0].Columns.ColumnName
$DataSet.Tables[0] | Select-Object -Property $Columns | Export-Excel -workSheetName $Row[1] -path C:\Users\bsben\Desktop\Demo.xlsx
}
Here's my current code. It does what I need. The problem is it has to write to the file for every table. The end result is for my current database it writes to the file 94 times. It ends up taking about 10 seconds.
The way I see it working is adapting what's going to be saved into an object.
This Idea is not an optimal solution and not recommended. Imagine if you have 1000+ tables in your DB and imaging you have 50+ DB inside your DB instance; then 1000+ sheets would be required in one excel file. Do you really think that it is possible to read after that in that level ?
Although there is a direct way to export the tables of DB to EXCEL. Please refer the link below :
Export-sql-server-tables-to-Excel/
Storing everything in one single Flat file is going to make life more complicated when you cannot do anything with the file in future.
Take backups based on the backup tools or script the mdf,ldf,ndf in proper paths.
As per the comment, here is the link for sql-database-backups-using-powershell-module-dbatools

Powershell Rename file based on SQL query

We take a lot of photograph's for product imagery . I have found a piece of software that will read a Barcode embeded in the image and rename the Image to the Barcode within the image , what i now want to do is rename the File again with the Productid using powershell with an SQL query within ,
declare #$CliRef nvarchar(max)
set #$CliRef = '5015807416980'
Select convert(nvarchar(max), productid) as ID FROM products WHERE products.Barcode = #$CliRef
I Found a Powershell script online but I must be doing something wrong as its renaming the file blank rather than a productid , I'm really new to Powershell so a little stuck so any help appreciated
$Database = "training"
$Server = "ART-WM01"
##Not sure i need this bit
function Select-Info($CliRef)
{
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$Server;Initial Catalog=$Database;Integrated Security = True"
$SqlConnection.Open()
$conn.open()
$query = “Select convert(varchar,producdid )as ID FROM products WHERE barcode = '$CliRef'”
$cmd = New-Object System.Data.SqlClient.SqlCommand($sql,$conn)
$cmd.connection = $SqlConnection
$cmd.commandtext = $query
$result = $cmd.executenonquery()
$SqlConnection.Close()
return $query
}
## Return ID from Database
function Return-Info($CliRef)
{
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$Server;Initial Catalog=$Database;Integrated Security = True"
$SqlConnection.Open()
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.commandtext = “Select convert(varchar, productid) as ID FROM products WHERE barcode = '$CliRef'”
$cmd.connection = $SqlConnection
$result = $cmd.ExecuteScalar()
$SqlConnection.Close()
return $result
}
## Collect the image names
$FiNms = Get-ChildItem C:\JJC\Test_Dump -Name
## Loop through each Image name
foreach ($FiNm in $FiNms)
{
## Variable for current File path
$file = “C:\JJC\Test_Dump\” + $FiNm
## Variable for new File path
$newFile = “C:\JJC\Test_Dump\renamed” + $FiNm
$ID = Return-Info $FiNm
$ID = $ID + “.tiff”
##Copy-Item $file -Destination $newFile
##Copy-Item $file -Destination = “C:\JJC\\Test_Dump\renamed\”
Rename-Item $file $ID -force
}

How to use powershell to obtain the data from excel and select from SQL

I have an excel file that contain a list of staff information,
now I want to use the column Staff_ID in xlsx to select those Staff_ID from sql table and get the result by export_CSV, so I try to write a powershell that could be run in schedule task.
How should I get the information from excel and select those data from sql table?
$SQLServer = "ServerName"
$SQLDBName = "DBName"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database =
$SQLDBName; User ID= YourUserID; Password= YourPassword"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = 'StoredProcName'
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$gSqlAuthMode = True
SELECT * FROM [DBName].[dbo].[DETAILS] WHERE [STATUS] not like 'Inactive-%' ORDER BY NUMBER ASC
$List = Import-XLSX -Path "D:\Stafflist.xlsx"
foreach ($item in $List){
If ((?? -eq $item.StaffCuid.toUpper().trim())) {
$Report = [PSCustomObject] #{
}
}
}
$Report | Export-Csv D:\report.csv -Delimiter "," -NoTypeInformation -append
}
You could use Import-Excel cmdlet from the famous ImportExcel module and Invoke-SQLCmd (part of SQL management tools)
$Excel = Import-Excel -Path D:\Stafflist.xlsx
Foreach($Row in $Excel) {
$StaffID = $Row.StaffCuid.toUpper().trim()
$Query = "create your query here with $StaffId"
Invoke-SQLCmd -Query $Query -ServerInstance "$ServerName\$DBName"
}

Convert SQL varbinary content into .pkg file using PowerShell

I use SQL Server and I have a SQL table called [dbo].[TemplatePackageContent] which has only two fields:
[TemplatePackageContentId] [uniqueidentifier] NOT NULL,
[Content] [varbinary](max) NULL
I'd like to create PowerShell script which reads whole content from this table and for each row, it will generate a file, in a given directory with format {TemplatePackageContentId}.pkg based on the Content field.
So far I've managed how to read the whole content of the table:
param(
[string] $dataSource = "(localdb)\mssqlLocalDb",
[string] $database = "Hda_tenancy1",
[string] $sqlCommand = $("SELECT * FROM TemplatePackageContent")
)
$connectionString = "Data Source=$dataSource; " +
"Integrated Security=SSPI; " +
"Initial Catalog=$database"
$connection = new-object system.data.SqlClient.SQLConnection($connectionString)
$command = new-object system.data.sqlclient.sqlcommand($sqlCommand,$connection)
$connection.Open()
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataSet) | Out-Null
$connection.Close()
$dataSet.Tables
Now I would like to make a mentioned conversion of received result, presumably looping thru each row.
I found this article which solves a similar problem in c# and I was thinking about using some of the logic from there and try to convert it into a PowerShell script.
What is the most optimal way to convert all the "Content" fields into files with .pkg format and what library should I use or what approach?
Any ideas?
Cheers
I ended up with this solution:
param(
[string] $dataSource = "(localdb)\mssqlLocalDb",
[string] $database = "Hda_tenancy1",
[string] $templatePath = "C:\dev\hubadvance\Seeding\Templates\"
)
$sqlCommand = $("SELECT * FROM TemplatePackageContent");
$connection = new-object System.Data.SqlClient.SQLConnection("Data Source=$dataSource;Integrated Security=SSPI;Initial Catalog=$database");
$cmd = new-object System.Data.SqlClient.SqlCommand($sqlCommand, $connection);
$connection.Open();
$reader = $cmd.ExecuteReader();
$results = #();
while ($reader.Read())
{
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i);
}
$results += new-object psobject -property $row;
}
$connection.Close();
foreach ($row in $results)
{
Write-Host $row.TemplatePackageContentId;
$path = $templatePath + $row.TemplatePackageContentId + ".pkg";
[System.IO.File]::AppendAllText($path, $row.Content);
}

Two SQL queries and pass each as a dataset to pass as parameters

Im very new to Powershell and I've scraped together the following code for a script that will check for an 'old' folder and create it if not found. It will then move the compressed weblogs from current location to the 'old' folder. I am wanting this written to pull the server names and website names from a sql query so it can be setup to run nightly and does not need to be updated with new or deleted servers. I have the following written so far but since I'm new I cannot figure out the last bit of syntax.
clear
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = "select hsa.servername from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlQuery1 = "select hah.vchhost from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
What I would like to do is have each $SQLQuery run and store each one in it's own dataset table so I can pass each one as a parameter for the following code. What I need to figure out is:
1)How do I write the above to run both SQL queries and have each one be either it's own dataset or table so I can them as parameters; $Servername and $HostedGroup?
2)How do I set this up to recurse the below code on each of the servers from the $Servername parameter?
$Servername = Dataset1
$Hostedgroup = Dataset2
$OldFolder = "\\$Servername\C$\Ren\Weblogs\$Hostedgroup\old"
$FolderExists = Test-Path $OldFolder
if($FolderExists -eq $False)
{
new-item \\$Servername\C$\Ren\Weblogs\$Hostedgroup\old -type directory
}
then
{
if(Test-Path \\$Servername\C$\Ren\Weblogs\old\W3SVC2)
{
get-childitem -path '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\W3SVC2' -recurse -include *.zip | move-item -destination '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\old'
$SqlAdapter.Fill($DataSet) returns an integer; you should prevent your script/function from returning it by assigning it to $null or using Out-Null. (e.g. $null = $SqlAdapter.Fill($DataSet))
For the answer to your question, you can return both fields from a single query, then iterate over the results. The pattern would look like this:
function Get-HostedServerApp {
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = #"
select hsa.vchappservername, hah.vchhost
from SERVER.dbo.hostedserversapp hsa
inner join SERVER.dbo.hostedapphosts hah on hsa.vchappservername = hah.vchappservername
where hsa.tirecordstatus = 1
order by hsa.vchappservername desc
"#
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$null = $SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$Dataset.Tables[0]
}
# Place your logic in this function
function SomeFunction {
param(
$Servername,
$Hostedgroup
)
"\\$Servername\C`$\Renaissance\Weblogs\$Hostedgroup\old"
}
$data = Get-HostedServerApp
$data| foreach{SomeFunction -ServerName $_.vchappservername -HostedGroup $_.vchhost}