Backup database with multiple tables powershell - sql

My objective is to backup a database to a spreadsheet. The database uses multiple tables. I'd like to save each table into it's own sheet in the spreadsheet. That way everything is in a single file. I also want to avoid using excel.
$server = "***\***"
$database = "***"
$username = "***"
$password = '***'
$tablequery = "SELECT schemas.name as schemaName, tables.name as tableName from sys.tables inner join sys.schemas ON tables.schema_id = schemas.schema_id"
#Delcare Connection Variables
$connectionTemplate = "Data Source={0};Integrated Security=false;Initial Catalog={1};User ID={2};Password={3}"
$connectionString = [string]::Format($connectionTemplate, $server, $database, $username, $password)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $tablequery
$command.Connection = $connection
#Load up the Tables in a dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
Import-Module ImportExcel
foreach ($Row in $DataSet.Tables[0].Rows) {
$queryData = "SELECT * FROM [$($Row[0])].[$($Row[1])]"
$command.CommandText = $queryData
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$Columns = $DataSet.Tables[0].Columns.ColumnName
$DataSet.Tables[0] | Select-Object -Property $Columns | Export-Excel -workSheetName $Row[1] -path C:\Users\bsben\Desktop\Demo.xlsx
}
Here's my current code. It does what I need. The problem is it has to write to the file for every table. The end result is for my current database it writes to the file 94 times. It ends up taking about 10 seconds.
The way I see it working is adapting what's going to be saved into an object.

This Idea is not an optimal solution and not recommended. Imagine if you have 1000+ tables in your DB and imaging you have 50+ DB inside your DB instance; then 1000+ sheets would be required in one excel file. Do you really think that it is possible to read after that in that level ?
Although there is a direct way to export the tables of DB to EXCEL. Please refer the link below :
Export-sql-server-tables-to-Excel/
Storing everything in one single Flat file is going to make life more complicated when you cannot do anything with the file in future.
Take backups based on the backup tools or script the mdf,ldf,ndf in proper paths.
As per the comment, here is the link for sql-database-backups-using-powershell-module-dbatools

Related

Cyclic csv-Import to SQL file by file

I try to write a short script that cyclic imports csv-files to a SQL data table that are being dropped in an import folder ($sourceSQL). Each csv-file consists of one line of information (here three columns). To make sure that the file was successfully written to the data table, I check if the unique Id can be found in the table.
So far it works with the first file. However, the second file gets only moved to the destination folder without being written to the data table. I can't find the problem. Is it because of the $data variable?
$StartButton.Add_Click({
$script:ActiveLoop = $true
while ($script:ActiveLoop){
If (Test-Path $sourceSQL){
$data = $null
Do{
$data = import-csv $impcsv -Header A,B,C
foreach($i in $data)
{
$Id = $i.A
$State = $i.B
$Sequence = $i.C
$query = INSERT INTO $SQLTable (Id, State, Sequence)
VALUES ('$Id','$State','$Sequence')"
$impcsv = invoke-sqlcmd -Database $SQLDatabase -Query $query -serverinstance $SQLInstance -Username $SQLUsername -Password $SQLPassword}
$SqlQueryId = "SELECT TOP 1 Id from $SQLTable ORDER BY Id DESC"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLInstance; Database = $SQLDatabase; User ID = $SQLUsername; Password = $SQLPassword"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQueryId
$SqlCmd.Connection = $SqlConnection
$SqlConnection.Open()
$IdCheck= [string]$SqlCmd.ExecuteScalar()
$SqlConnection.Close()
} Until ($Id -eq $IdCheck)
Move-Item $sourceSQL -Destination $destinationSQL
}
[System.Windows.Forms.Application]::DoEvents()
}
})
$objForm.Controls.Add($StartButton)

PowerShell Script that Queries SQL Table to CSV File using Loop

I have a basic SQL table of Employees. Using a powershell script I want to export all the employees who have made over 1000 sales to a .csv file and the rest into a different .csv file. I want to accomplish this task by using a loop. I am new to powershell and want to learn the foundations. Can anyone help?
SQL Table (not real employees)
This is what I have so far:
$connection.Open()
[System.Data.SqlClient.SqlDataReader]$result = $cmd.ExecuteReader()
$highDestFile = "C:\high-sales.csv"
$lowDestFile = "C:\low-sales.csv"
while($result.Read()) {
$ename = $result.GetValue(3);
$job = $result.GetValue(4);
$sales = $result.GetValue(7);
$tableArray = New-Object System.Collections.ArrayList
$tableArray.Add($ename)
$tableArray.Add($job)
$tableArray.Add($sales)
if($sales -ge 1000) {
Out-File -FilePath $highDestFile -InputObject $tableArray -Encoding ASCII -Append
} else {
Out-File -FilePath $lowDestFile -InputObject $tableArray -Encoding ASCII -Append
}
}
$connection.Close()
I'm not real familiar with the method you're using to get your results, but I think I have something similar that might be easier to work with for you. It will get all results into PS, and you can filter things from there, rather than getting one result at a time. You obviously know how to make your own SqlConnection and SqlCommand, I'm just including them for future readers to reference.
# Define SQL query
$sqlQuery = #"
SELECT *
FROM MyTable
"#
# Create a SqlConnection to connect to the SQL DB
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection
$sqlConnection.ConnectionString = "Server = $SqlServer; Database =$SqlCatalog; User Id = $User; Password = $Password"
# Create a SqlCommand object to define the query
$sqlCmd = New-Object System.Data.SqlClient.SqlCommand
$sqlCmd.CommandText = $sqlQuery
$sqlCmd.Connection = $sqlConnection
# Create a SqlAdapter that actually does the work and manages everything
$sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$sqlAdapter.SelectCommand = $sqlCmd
# Create an empty DataSet for the query to fill with its results
$dataSet = New-Object System.Data.DataSet
# Execute the query and fill the DataSet (then disconnect)
$sqlAdapter.Fill($dataSet)
$sqlConnection.Close()
# Convert DataSet table to array for ease of use
[Array]$Results = $dataSet.Tables[0]
Beyond that you could just use a pair of Where statements to filter your results, and output to files.
$Results | ?{[int]$_.Sales -ge 1000} | Set-Content $highDestFile
$Results | ?{[int]$_.Sales -lt 1000} | Set-Content $lowDestFile

Powershell import csv to SQL queries, export multiple csvs

I'm struggling to get my head around this one, I've only just begun looking at scripting in SQL, and my powershell is very limited. The requirments are basically this:
Utilisng Powershell, import a csv file which contains one column that needs to feed into multiple SQL queries via a loop, exporting a seperate csv file for each different query.
example import of csv:
Project (heading)
1000
1001
1002
Powershell:
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
import-csv $importProjectsCSV | ForEach-Object {
$query = "
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = $_.Project;
Select ProjectClient, Name
FROM dbo.CLIENTS
Where Project = $_.Project;
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $query
$command.Connection = $connection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$dataset.Table[0] | Export-csv "E:\" + $_.ProjectName + ".csv"
$dataset.Table[1] | Export-csv "E:\" + $_.ProjectName + ".csv"
The problem is that the variable isn't coming into the SQL query.
Is there a better way to handle this type of example?
Appreciate any pointers
Paul.
I would do something like this:
I must admit i havent been able to test it, and personally i usually use c# to query sql servers. So i might have gone a bit wrong somewhere.
$importProjectsCSV = e:\Projects.csv
$server = servername
$database = database
$Projects = Import-Csv -Path $importProjectsCSV | % {$_.Project}
$DS_Projects = New-Object System.Data.DataSet
$DS_Clients = New-Object System.Data.DataSet
$query_pro = "Select ProjectLeader, ProjectTitle, Project FROM dbo.PROJECTS";
$query_Clients = "Select ProjectClient, Name, Project FROM dbo.CLIENTS";
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = $connection.CreateCommand();
$command.CommandText = $query_pro;
$sqlAdap = New-Object System.Data.SqlClient.SqlDataAdapter($command)
$sqlAdap.Fill($DS_Projects)
$command2 = $connection.CreateCommand();
$command2.CommandText = $query_Clients;
$sqlAdap2 = New-Object System.Data.SqlClient.SqlDataAdapter($command2)
$sqlAdap2.Fill($DS_Clients)
$connection.Close();
foreach($project in $Projects)
{
$DS_Projects.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
$DS_Clients.Tables[0].Select("Project=$project") | Export-Csv "E:\$project.csv"
}
If I get every right you could do this with something like this:
$Projects = Import-Csv -Path 'C:\Temp\Projects.csv'
ForEach ($Project in $Projects.Projects) {
$Query= #"
Select ProjectLeader, ProjectTitle
FROM dbo.PROJECTS
Where Project = '%{0}';
"# -f $Project
# just dummy action
$results = Invoke-Sqlcmd -ServerInstance "foo" -Database "bar" -Query $Query
# In Case results is a dataset do something like
$results | Export-Csv -Path ("E:\{0}.csv" -f $Project )
}

PowerShell Function multiple SQL queries output as CSV

I'm having problems getting PowerShell to run multiple SQL queries and export the results as CSV.
I'm trying to accomplish this using a Function but the problem occurs in the Process block when I expect two queries to run and output two CSV files.
I tried creating one function to run the query and a second function to create the CSV files but that didn't even run the SQL queries. I'm doing this without SQL being installed where this powershell script is executed from. -thanks!
Function Run-Query {
param([string[]]$queries,[string[]]$sheetnames)
Begin{
$SQLServer = 'ServerName'
$Database = 'DataBase'
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
}#End Begin
Process{
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $queries
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "C:\Scripts\$sheetnames.csv"
}#End Process
End{
$SqlConnection.Close()
}
}#End function run-query.
$queries = #()
$queries += #'
Select * from something
'#
$queries += #'
Select * from something2
'#
$sheetnames = #()
$sheetnames += 'Cert'
$sheetnames += 'Prod'
Run-Query -queries $queries
I'm not sure if SQL processes multiple queries separately so while you might be passing two different queries the SQL server might be interpreting them as one query (Not 100% sure this is happening, just a guess really)
You've put your queries in an array so we can easily loop through the array, run each query by itself and put the results into a CSV.
Here's how i'd modify your code to start with:
Function Run-Query
{
param([string[]]$queries,[string[]]$sheetnames)
Begin
{
$SQLServer = 'ServerName'
$Database = 'DataBase'
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
}#End Begin
Process
{
# Loop through each query
For($i = 0; $i -lt $queries.count; $i++)
{
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
# Use the current index ($i) to get the query
$SqlCmd.CommandText = $queries[$i]
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
# Use the current index ($i) to get the sheetname for the CSV
$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "C:\Scripts\$($sheetnames[$i]).csv"
}
}#End Process
End
{
$SqlConnection.Close()
}
}#End function run-query.
$queries = #()
$queries += #'
Select * from something
'#
$queries += #'
Select * from something2
'#
$sheetnames = #()
$sheetnames += 'Cert'
$sheetnames += 'Prod'
Run-Query -queries $queries -sheetnames $sheetnames

Two SQL queries and pass each as a dataset to pass as parameters

Im very new to Powershell and I've scraped together the following code for a script that will check for an 'old' folder and create it if not found. It will then move the compressed weblogs from current location to the 'old' folder. I am wanting this written to pull the server names and website names from a sql query so it can be setup to run nightly and does not need to be updated with new or deleted servers. I have the following written so far but since I'm new I cannot figure out the last bit of syntax.
clear
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = "select hsa.servername from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlQuery1 = "select hah.vchhost from SERVER.dbo.serversapp hsa
inner join SERVER.dbo.apphosts hah on hsa.vchservername = hah.vchservername
where hsa.tirecordstatus = 1
order by hsa.vchservername desc"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
What I would like to do is have each $SQLQuery run and store each one in it's own dataset table so I can pass each one as a parameter for the following code. What I need to figure out is:
1)How do I write the above to run both SQL queries and have each one be either it's own dataset or table so I can them as parameters; $Servername and $HostedGroup?
2)How do I set this up to recurse the below code on each of the servers from the $Servername parameter?
$Servername = Dataset1
$Hostedgroup = Dataset2
$OldFolder = "\\$Servername\C$\Ren\Weblogs\$Hostedgroup\old"
$FolderExists = Test-Path $OldFolder
if($FolderExists -eq $False)
{
new-item \\$Servername\C$\Ren\Weblogs\$Hostedgroup\old -type directory
}
then
{
if(Test-Path \\$Servername\C$\Ren\Weblogs\old\W3SVC2)
{
get-childitem -path '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\W3SVC2' -recurse -include *.zip | move-item -destination '\\$Servername\C$\Ren\WebLogs\$Hostedgroup\old'
$SqlAdapter.Fill($DataSet) returns an integer; you should prevent your script/function from returning it by assigning it to $null or using Out-Null. (e.g. $null = $SqlAdapter.Fill($DataSet))
For the answer to your question, you can return both fields from a single query, then iterate over the results. The pattern would look like this:
function Get-HostedServerApp {
$SqlServer = "SERVER"
$SqlCatalog = "DATABASE"
$SqlQuery = #"
select hsa.vchappservername, hah.vchhost
from SERVER.dbo.hostedserversapp hsa
inner join SERVER.dbo.hostedapphosts hah on hsa.vchappservername = hah.vchappservername
where hsa.tirecordstatus = 1
order by hsa.vchappservername desc
"#
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlCatalog; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$null = $SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$Dataset.Tables[0]
}
# Place your logic in this function
function SomeFunction {
param(
$Servername,
$Hostedgroup
)
"\\$Servername\C`$\Renaissance\Weblogs\$Hostedgroup\old"
}
$data = Get-HostedServerApp
$data| foreach{SomeFunction -ServerName $_.vchappservername -HostedGroup $_.vchhost}