I am trying to convert an excel file containing multiple sheets to csv files.
$currentDir = $PSScriptRoot
$csvPATH = Join-Path -Path $currentDir -ChildPath CSV_Files
New-Item -ItemType Directory -Force -Path $csvPATH | out-null
function Convert-ExcelSheetsToCsv {
param(
[Parameter(Mandatory, ValueFromPipelineByPropertyName)]
[ValidateNotNullOrEmpty()]
[Alias('FullName')]
[string]$Path,
[Parameter(Mandatory = $false, ValueFromPipeline, ValueFromPipelineByPropertyName)]
[bool]$AppendFileName
)
Begin {
$excel = New-Object -ComObject Excel.Application -Property #{
Visible = $false
DisplayAlerts = $false
}
}
Process {
$root = Split-Path -Path $Path
$filename = [System.IO.Path]::GetFileNameWithoutExtension($Path)
$workbook = $excel.Workbooks.Open($Path)
foreach ($worksheet in $workbook.Worksheets) {
if ($AppendFileName) {
$name = Join-Path -Path $csvPATH <# $root #> -ChildPath "${filename}_$($worksheet.Name).csv"
}
else {
$name = Join-Path -Path $csvPATH <# $root #> -ChildPath "$($worksheet.Name).csv"
}
try {
$worksheet.SaveAs($name, 6) #6 to ignore formatting and covert to pure text, otherwise, file could end up containing rubbish
} catch {
Write-Error -Message "Failed to save csv! Path: '$name'. $PSItem"
}
}
}
End {
$excel.Quit()
$null = [System.Runtime.InteropServices.Marshal]::ReleaseComObject($excel)
}
}
Get-ChildItem -Path $currentDir -Filter *.xlsx 0 | Convert-ExcelSheetsToCsv
This is giving me the following error:
Get-ChildItem : A positional parameter cannot be found that accepts
argument '0'.
or if i put the 0 (for false) after like this: Get-ChildItem -Path $currentDir -Filter *.xlsx | Convert-ExcelSheetsToCsv 0
i get this error:
Convert-ExcelSheetsToCsv : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the
input and its properties do not match any of the parameters that take pipeline input.
basically, i am trying to have an option where if $AppendFileName is false, then the generated csv files will only be named by the sheet name, which is this else statement
$name = Join-Path -Path $csvPATH <# $root #> -ChildPath "$($worksheet.Name).csv"
Position needs to be specified.
[Parameter(Mandatory, ValueFromPipelineByPropertyName,Position=1)] [Parameter(Mandatory=$false,ValueFromPipeline,ValueFromPipelineByPropertyName,Position=0)]
and then:
Get-ChildItem -Path $currentDir -Filter *.xlsx | Convert-ExcelSheetsToCsv 0
Related
I have a Powershell script to read .sql files from a specific folder and run them against a database depending on the name of the filename.
The filenames are always the same: myDatabase.script.SomeRandomCharacters.csv
There can be many files which is why the script has a foreach loop.
[CmdletBinding()]
param (
[parameter(Mandatory = $true)][ValidateSet('dev')][String]$serverName,
[parameter(Mandatory = $true)][String]$databaseName,
)
$dir = Split-Path $MyInvocation.MyCommand.Path
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name -like "$databaseName*" }
foreach ($s in $scripts) {
$script = $s.FullName
Invoke-Sqlcmd -ServerInstance $serverName -Database $databaseName -InputFile $script
}
The issue here is that if I would have 2 databases "myDatabase" and "myDatabase2", running the script with the former input would run the latter as well since the Where-Object filtering uses an asterisk.
I can't figure out how to modify the script so that I get the absolute value of whatever is before the first fullstop in the filename. What I would also what to do is to validate the value between the first and second fullstops, in the example filename it is script.
Any help is appreciated!
Use the database names to construct a regex pattern that will match either:
param(
[Parameter(Mandatory = $true)][ValidateSet('dev')][String]$ServerName,
[Parameter(Mandatory = $true)][String[]]$DatabaseNames,
)
# Construct alternation pattern like `db1|db2|db3`
$dbNamesEscaped = #($DatabaseNames |ForEach-Object {
[regex]::Escape($_)
}) -join '|'
# Construct pattern with `^` (start-of-string anchor)
$dbNamePattern = '^{0}' -f $dbNamesEscaped
# Fetch scripts associated with either of the database names
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" -and $_.Name -match $dbNamePattern }
# ...
You can use the StartsWith function to fix your filter:
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name.StartsWith("$($databaseName).") }
I was trying to script out all the SQL Server Agent jobs for category 'Data Warehouse' into a single file
I was able to do it using PowerShell, where every single job creates a single file.
But I need one file for all the SQL Server Agent jobs under category ID = 100 (or Category : = 'Data Warehouse')
Code I'm currently using:
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.Smo') | Out-Null
$serverInstance = "APAAUHC7DB01VD"
$server = New-Object ('Microsoft.SqlServer.Management.Smo.Server') $serverInstance
$jobs = $server.JobServer.Jobs
#$jobs = $server.JobServer.Jobs | where-object {$_.category -eq "100"}
if ($jobs -ne $null)
{
$serverInstance = $serverInstance.Replace("\", "-")
ForEach ( $job in $jobs )
{
$FileName = "C:\SQLBackup\SQLJobs\" + $serverInstance + "_" + $job.Name + ".sql"
$job.Script() | Out-File -filepath $FileName
}
}
Give $FileName a single file name for the whole set. Then you can leave out the whole foreach block:
$FileName = "C:\SQLBackup\SQLJobs\whatever.sql"
$jobs | %{ $_.Script() } | Out-File -filepath $FileName
As a newbie in powershell, im trying to read thru a folder which has multiple sql files and iterate them through poweshell scripts read the data from oracle and export to CSV.
If my sqlfile has a single line statement no issues with the code, its working fine, If my sql file has multiple line statement - as always it has,
the powershell errors out saying
"Get-DataTable : Cannot process argument transformation on parameter 'sql' Cannot convert value to type System.String."
could you please help me how to resolve this issue? Below my code snapshot.
function Get-DataTable{
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[Oracle.DataAccess.Client.OracleConnection]$conn,
[Parameter(Mandatory=$true)]
[string]$sql
)
$cmd = New-Object Oracle.DataAccess.Client.OracleCommand($sql,$conn)
$da = New-Object Oracle.DataAccess.Client.OracleDataAdapter($cmd)
$dt = New-Object System.Data.DataTable
[void]$da.Fill($dt)
return ,$dt
}
foreach ($file in Get-ChildItem -path $ScriptsDirectory -Filter *.sql | sort-object -desc )
{
$SQLquery = get-content "$ScriptsDirectory\$file"
echo $SQLquery
$fileName = $file.name.split(".")[0]
$dt = Get-DataTable $conn $SQLquery
Write-Host "Retrieved records:" $dt.Rows.Count -ForegroundColor Green
$dt | Export-Csv -NoTypeInformation -LiteralPath $WorkingDirectory\$fileName.csv
Write-Host "Output Written to :" $WorkingDirectory\$fileName.csv -ForegroundColor Green }
Get-Content returns an array of lines. If you're using PowerShell v3 or higher you can use the -Raw parameter to read the file as one big string:
$SQLquery = get-content "$ScriptsDirectory\$file" -Raw
Alternatively you could re-join the array with line endings:
$SQLquery = $SQLquery -join "`r`n"
Or you can read the file all at once with .net classes:
$SQLquery = [System.IO.File]::ReadAllText("$ScriptsDirectory\$file")
I want to read a file that contains a line separated list of *.sql file names (all located at the same directory) to execute the following:
$reader = [System.IO.File]::OpenText("_Scripts.txt")
try {
for(;;) {
$line = $reader.ReadLine()
if ($line -eq $null) { break }
#output
$out = $line.split(".")[0] + ".txt" ;
# -serverinstance u should change the value of it according to use
invoke-sqlcmd -inputfile $line -serverinstance "." | format-table | out-file -filePath $out
$line
}
}
finally {
$reader.Close()
}
I'm trying to execute this script file by using a batch file containing the command:
powershell.exe -ExecutionPolicy Bypass -Command "_scripts.ps1"
but I get the error shown below:
Can anyone help me fix my ps1 script please?
This works for me:
$lines = Get-Content C:\Temp\TEST\_Scripts.txt
ForEach ($line in $lines)
{
$out = $line.split(".")[0] + ".txt" ;
Invoke-Sqlcmd -InputFile $line -ServerInstance "localhost" -Database "master" | Format-Table | Out-File -FilePath $out
}
I have a very simple question. My purpose here to retrieve login names from a txt file into a variable into SQL and query the SQL table while predicating against that same variable.
So for example:
the txt file would have:
forde
blain
martin
Alex
so the idea to feed each name to a variable and output the designated computer name.
Declare #loginName varchar (25)
--open the file
--while the end of the file has not reached, read each line and place the name into #loginName variable
select *
from computerinfo
where loginname = #loginname
I don't necessarily need to bulk import to a temp table at this point.
Thanks.
i had to do this some weeks ago and the simple way i found was Powershell.
I had no SSIS else it's the best of course.
# You may want to adjust these
function Invoke-Sqlcmd2
{
[CmdletBinding()]
param(
[Parameter(Position=0, Mandatory=$true)] [string]$ServerInstance,
[Parameter(Position=1, Mandatory=$false)] [string]$Database,
[Parameter(Position=2, Mandatory=$false)] [string]$Query,
[Parameter(Position=3, Mandatory=$false)] [string]$Username,
[Parameter(Position=4, Mandatory=$false)] [string]$Password,
[Parameter(Position=5, Mandatory=$false)] [Int32]$QueryTimeout=600,
[Parameter(Position=6, Mandatory=$false)] [Int32]$ConnectionTimeout=15,
[Parameter(Position=7, Mandatory=$false)] [ValidateScript({test-path $_})] [string]$InputFile,
[Parameter(Position=8, Mandatory=$false)] [ValidateSet("DataSet", "DataTable", "DataRow")] [string]$As="DataRow"
)
if ($InputFile)
{
$filePath = $(resolve-path $InputFile).path
$Query = [System.IO.File]::ReadAllText("$filePath")
}
$conn=new-object System.Data.SqlClient.SQLConnection
if ($Username)
{ $ConnectionString = "Server={0};Database={1};User ID={2};Password={3};Trusted_Connection=False;Connect Timeout={4}" -f $ServerInstance,$Database,$Username,$Password,$ConnectionTimeout }
else
{ $ConnectionString = "Server={0};Database={1};Integrated Security=True;Connect Timeout={2}" -f $ServerInstance,$Database,$ConnectionTimeout }
$conn.ConnectionString=$ConnectionString
#Following EventHandler is used for PRINT and RAISERROR T-SQL statements. Executed when -Verbose parameter specified by caller
if ($PSBoundParameters.Verbose)
{
$conn.FireInfoMessageEventOnUserErrors=$true
$handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] {Write-Verbose "$($_)"}
$conn.add_InfoMessage($handler)
}
$conn.Open()
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$cmd.CommandTimeout=$QueryTimeout
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
[void]$da.fill($ds)
$conn.Close()
switch ($As)
{
'DataSet' { Write-Output ($ds) }
'DataTable' { Write-Output ($ds.Tables) }
'DataRow' { Write-Output ($ds.Tables[0]) }
}
} #Invoke-Sqlcmd2
$files = #(get-childitem "filelocationformultiplefile" -include *.txt -exclude *.bak -recurse | where-object {($_.LastWriteTime -le (Get-Date).AddDays(-0))-and ($_.psIsContainer -eq $false)})
if ($files -ne $NULL)
{
for ($idx = 0; $idx -lt $files.Length; $idx++)
{
$file = $files[$idx]
$Query = #"
Bulk INSERT db.dbo.tbl from '$file' with (FirstRow = 1, FieldTerminator ='";', RowTerminator = '\n')
"#
Invoke-sqlcmd2 -ServerInstance "servername" -Database "db" -Query $Query
}
}
And i'm not crazy, except the 10 ending lines everything else is coming from a microsoft official blog.
You don't need everything, powershell is present on every computer, it requires to save this in a file with extension ps1 and configure the 4 variables $fileout2 -> $tablename