PowerShell script failing when run from SQL Agent - sql

I have created a PowerShell script that does the following:
Copies a template Excel file to another directory
Imports data into this template file using the ImportExcel module created by Doug Finke ( https://www.powershellgallery.com/packages/ImportExcel/7.1.0 )
Runs a macro that is contained in the template file
Renames the file as an .xlsx file type
When I run this script from the server, it executes perfectly and does what is expected. But when I tried to set up a SQL Agent job to run this same PS script, it is failing and I cannot find the reason it would do so.
Here is my PS script:
# Directory Information
$RootDirectory = "D:\MSSQL\SSIS\BIN Commission Report\"
$ArchiveFolder = "Archive\"
$SourceFolder = "Source\"
$TemplateFolder = "Template\"
# File Information
$TemplateFile = "Commission_Invoices_Template.xlsm" #Template file with macro
$MacroFile = "Commission_Invoices.xlsm" #Macro-enabled file with vendor data
$VendorFile = "Commission_Invoices.xlsx" #File provided by the vendor
$DataImportFile = "Commission_Invoices_Import.xlsx" #File used to import data into SQL
$AppliedMacroFile = "Commission_Invoices_Import.xlsm"
#Full paths for each file
$TemplatePath = $RootDirectory, $TemplateFolder, $TemplateFile -Join ""
$MacroPath = $RootDirectory, $SourceFolder, $MacroFile -Join ""
$VendorPath = $RootDirectory, $SourceFolder, $VendorFile -Join ""
$DataImportPath = $RootDirectory, $SourceFolder, $DataImportFile -Join ""
$AppliedMacroPath = $RootDirectory, $SourceFolder, $AppliedMacroFile -Join ""
If ( Test-Path $MacroPath ) { Remove-Item $MacroPath }
If ( Test-Path $AppliedMacroPath ) { Remove-Item $AppliedMacroPath }
#Copy template file to use for date insertion and macro
Copy-Item $TemplatePath -Destination $MacroPath
# Copy data from vendor spreadsheet into macro-enabled workbook
Import-Excel -Path $VendorPath -WorksheetName Sheet1 | Export-Excel $MacroPath -WorksheetName Commission_Invoice
# Run the macro in Excel
$MacroName = "CommissionPrep"
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = $false
$WB = $Excel.Workbooks.Add($MacroPath)
$Excel.Run($MacroName)
#Close Excel
$WB.Close($false)
$Excel.Quit()
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($WB)
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel)
Remove-Variable -Name Excel
# Copy data to xlsx file from xlsm for data import into SQL
Import-Excel -Path $AppliedMacroPath -WorksheetName Commission_Invoice | Export-Excel $DataImportPath -WorksheetName Commission_Invoice -AutoSize
If ( Test-Path $MacroPath ) { Remove-Item $MacroPath }
If ( Test-Path $VendorPath ) { Remove-Item $VendorPath }
If ( Test-Path $AppliedMacroPath ) { Remove-Item $AppliedMacroPath }
This is what my SQL Agent job looks like:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -File "D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1"
And here is the error message when run as a SQL Agent job:
Message
Executed as user: CompanyX\CompanyXSQLADMIN.
Microsoft Excel cannot access the file 'D:\MSSQL\SSIS\BIN Commission Report\Source\Commission_Invoices.xlsm'.
There are several possible reasons:
The file name or path does not exist.
The file is being used by another program.
The workbook you are trying to save has the same name as a currently open workbook.
At D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1:35 char:1 + $WB = $Excel.Workbooks.Add($MacroPath) + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: (:) [], COMException + FullyQualifiedErrorId : System.Runtime.InteropServices.COMExceptionException calling "Run" with "1" argument(s): "Cannot run the macro 'CommissionPrep'. The macro may not be available in this workbook or all macros may be disabled."
At D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1:36 char:1 + $Excel.Run($MacroName) + ~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : COMException
You cannot call a method on a null-valued expression.
At D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1:39 char:1 + $WB.Close($false) + ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullException calling "ReleaseComObject" with "1" argument(s): "Object reference not set to an instance of an object."
At D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1:45 char:1 + [System.Runtime.Interopservices.Marshal]::ReleaseComObject($WB) + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : NullReferenceException 0 'D:\MSSQL\SSIS\BIN Commission Report\Source\Commission_Invoices_Import.xlsm' file not found
At C:\Program Files\WindowsPowerShell\Modules\ImportExcel\7.0.1\Public\Import-Excel.ps1:105 char:17 + throw "'$($Path)' file not found" + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: ('D:\MSSQL\SSIS\... file not found:String) [], RuntimeException + FullyQualifiedErrorId : 'D:\MSSQL\SSIS\BIN Commission Report\Source\Commission_Invoices_Import.xlsm' file not found. Process Exit Code 1. The step failed.
I am unclear as to how this script executes perfectly when run from the PowerShell ISE shell when run locally on the server but fails it is run via a SQL Agent job. Any suggestions on how to fix this?

have you tried to set the executionpolicy for your SQL Agent job:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -executionpolicy bypass -File "D:\MSSQL\SSIS\BIN Commission Report\Scripts\CommissionInvoices_PS.ps1"

Related

Output which Excel files are password protected

I am trying to modify the below script to be able to search for multiple Excel files that contain macros. The issue I'm having is being able to access password protected files. It is prompting to enter a password for each password protected file. I would like to skip it and just process the unprotected ones. Is there a way or script that I could use to output which files are password protected and just process the unprotected ones?
Here is the script I'm currently using:
Get-ChildItem -path "server path" -recurse | where {$_.extension -match "^\.xls(m|)$"} | ForEach-
Object -Begin {
$thisThread = [System.Threading.Thread]::CurrentThread
$originalCulture = $thisThread.CurrentCulture
$thisThread.CurrentCulture = New-Object System.Globalization.CultureInfo('en-US')
$excel = new-object -comobject excel.application
Try{
$excel.Workbooks.open($path, 0, 0, 5, $password)}
Catch{
Write-Host 'Book is password protected.'}
} -Process {
$workbook = $excel.workbooks.Open($_.FullName)
$_.FullName + " : " + $workbook.HasVBProject
$workbook.Close($false)
} -end {
$excel.Quit()
$thisThread.CurrentCulture = $originalCulture
}| out-file c:\results.csv
Here is the errors:
Book is password protected.
Unable to get the Open property of the Workbooks class
At line:11 char:5
+ $workbook = $excel.workbooks.Open($_.FullName)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
The object invoked has disconnected from its clients. (Exception from HRESULT: 0x80010108 (RPC_E_DISCONNECTED))
At line:13 char:4
+ $workbook.Close($false)
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
Unable to get the Open property of the Workbooks class
At line:11 char:5
+ $workbook = $excel.workbooks.Open($_.FullName)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
The object invoked has disconnected from its clients. (Exception from HRESULT: 0x80010108 (RPC_E_DISCONNECTED))
At line:13 char:4
+ $workbook.Close($false)
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException

SQL Server error when updating/creating records in database from PowerShell

Here is the function I use to send queries to SQL
function Invoke-SQL
{
param (
[string]$server,
[string]$database,
[string]$Query
)
$connectionString = "Data Source=$server; " +
"Integrated Security=SSPI; " +
"Initial Catalog=$database"
$connection = new-object
system.data.SqlClient.SQLConnection($connectionString)
$command = new-object system.data.sqlclient.sqlcommand($Query, $connection)
$connection.Open()
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataSet) | Out-Null
$connection.Close()
$dataSet.Tables
}
When I run a query like this:
Invoke-SQL -server 'ServerName' -database 'DBname' -Query "SELECT * FROM [DBname].[dbo].[TableName] WHERE UserID = '$userid' AND ComputerName = '$computername'"
I get the following error:
ERROR: Exception calling "Fill" with "1" argument(s): "Conversion failed when converting from a character string to uniqueidentifier."
PSDesk_Client.ps1 (358, 2): ERROR: At Line: 358 char: 2
ERROR: + $adapter.Fill($dataSet) | Out-Null
ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: + CategoryInfo : NotSpecified: (:) [],
MethodInvocationException
ERROR: + FullyQualifiedErrorId : SqlException
ERROR:
ERROR: Exception calling "Fill" with "1" argument(s): "Conversion failed when converting from a character string to uniqueidentifier."
PSDesk_Client.ps1 (358, 2): ERROR: At Line: 358 char: 2
ERROR: + $adapter.Fill($dataSet) | Out-Null
ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: + CategoryInfo : NotSpecified: (:) [],
MethodInvocationException
ERROR: + FullyQualifiedErrorId : SqlException
ERROR:
It seems it is having this error twice because:
First, it runs a query to get information from the database
Second, it runs a query to create/update information as needed on the database
Whether I have it return the data or just send a command it is unhappy with $adapter.Fill($dataSet) | Out-Null. Why are my two uniqueidentifiers causing this error and what can I do to fix this?
Upon trying to enter in a record manually, it throws an error:
Error Message: Guid should contain 32 digits with 4 dashes.
Is uniqueidentifier the wrong data type then? It needs to be a string but certainy shouldn't be required to be in that format or any format

PowerShell to insert code in Excel VBA

I'm creating a script for my Work, where i need to insert some VBA code in Excel. I've all worked out, except how to insert the code in the VBProject. I need 2 codes inserted. I need a code inserted in Sheet1(aka tmpArk1) and in ThisWorkbook. How do i do this?
My code so far:
Function Get-FileName($initialDirectory)
{
[System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms") | Out-Null
$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
$OpenFileDialog.initialDirectory = $initialDirectory
$OpenFileDialog.filter = "xlsx (*.xlsx)| *.xlsx"
$OpenFileDialog.ShowDialog() | Out-Null
$OpenFileDialog.filename
}
$inputplace = Get-FileName "P:\" ".xlsx"
$inputname = [System.IO.Path]::GetFileNameWithoutExtension($inputplace)
$inputpath = [System.IO.Path]::GetDirectoryName($inputplace)
########################################
########### Starting Excel #############
########################################
$Excel = New-Object -ComObject Excel.Application
$ExcelVersion = $Excel.Version
New-ItemProperty -Path "HKCU:\Software\Microsoft\Office\$ExcelVersion\Excel\Security" -Name AccessVBOM -Value 1 -Force | Out-Null
New-ItemProperty -Path "HKCU:\Software\Microsoft\Office\$ExcelVersion\Excel\Security" -Name VBAWarnings -Value 1 -Force | Out-Null
$Excel.Visible = $true
$Excel.DisplayAlerts = $false
########################################
####### Working with workbook ##########
########################################
#Opens workbook to be transfered from, ReadOnly
$workbook1 = $Excel.Workbooks.Open($inputplace,$null, $true)
#Create workbook to work with
$workbook2 = $Excel.Workbooks.add()
########################################
####### The code to be inserted ########
########################################
#Code Alpha
#$codeAlfa = #"
# Some code here
#"#
#Code beta
#$codeBeta = #"
# Some code here
#"#
########################################
####### Working with worksheets ########
########################################
#Add 1 worksheets to workbook
$workbook2.Worksheets.Add()
#Transfer Sheet from opened workbook to new workbook
$SheetToCopy = $workbook1.sheets.item(1)
$TransferTarget = $workbook2.sheets.item(1)
$SheetToCopy.copy($TransferTarget)
#Change name of all 4 worksheets
$workbook2.Worksheets.Item(1).name = "tmpArk1"
$workbook2.Worksheets.Item(2).name = "output"
$workbook2.Worksheets.Item(3).name = "net"
$workbook2.Worksheets.Item(4).name = "data"
$workbook2.Worksheets.Item(5).name = "bilag"
#Close Workbook1
$Workbook1.Close()
#inserting the codes
$ExcelModuleAlfa = $workbook2.VBProject.VBComponents.Add(1)
$ExcelModuleAlfa.CodeModule.AddFromString($codeAlfa)
########################################
######### Clean up enviroment ##########
########################################
#Save workbook to hard drive
Add-Type -AssemblyName Microsoft.Office.Interop.Excel
$workbook2.SaveAs("P:\" + $inputname + "_auto",[Microsoft.Office.Interop.Excel.XlFileFormat]::xlExcel8)
#Close workbook and quit Excel
#$Excel.Workbooks.Close()
#$Excel.Quit()
#Stop Excel process
#Get-Process excel | Stop-Process -Force
#Release com object
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel)
When I try to run the script, I get this as my output error.
You cannot call a method on a null-valued expression.
At C:\Users\G017116\Desktop\PowerShell InWork.ps1:995 char:57
+ $ExcelModuleAlfa = $workbook2.VBProject.VBComponents.Add <<<< (1)
+ CategoryInfo : InvalidOperation: (Add:String) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression.
At C:\Users\G017116\Desktop\PowerShell InWork.ps1:996 char:42
+ $ExcelModuleAlfa.CodeModule.AddFromString <<<< ($codeAlfa)
+ CategoryInfo : InvalidOperation: (AddFromString:String) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Try running PowerShell as Administrator.
I was able to replicate the issue when the "Trust access to the VBA project object model" was disabled. Yes, the two New-ItemProperty ... HKCU ... settings should set those options, but access to the registry may need Administrator access in order for it to be correctly set.
Try manually setting/Verifying that the Trust Center Permissions are:
Select "Enable all macros (not recommended; potentially dangerous code can run"
Check "Trust access to the VBA project object model"
Try running the minimal set of commands here:
$Excel = New-Object -ComObject Excel.Application
$ExcelVersion = $Excel.Version
New-ItemProperty -Path "HKCU:\Software\Microsoft\Office\$ExcelVersion\Excel\Security" -Name AccessVBOM -Value 1 -Force | Out-Null
New-ItemProperty -Path "HKCU:\Software\Microsoft\Office\$ExcelVersion\Excel\Security" -Name VBAWarnings -Value 1 -Force | Out-Null
$Excel.Visible = $true
$workbook2 = $Excel.Workbooks.add()
$workbook2.VBProject.VBComponents
The last statement $workbook2.VBProject.VBComponents should return something like:
Saved : True
Name : ThisWorkbook
Designer :
CodeModule : System.__ComObject
Type : 100
VBE : System.__ComObject
Collection : System.__ComObject
HasOpenDesigner : False
Properties : System.__ComObject
DesignerID :
Saved : True
Name : Sheet1
Designer :
CodeModule : System.__ComObject
Type : 100
VBE : System.__ComObject
Collection : System.__ComObject
HasOpenDesigner : False
Properties : System.__ComObject
DesignerID :
If it does not, or errors out as null, then the Trust Center permissions didn't get set. (you should be able to use the opened Excel document to go and verify the permissions)

How do I get the last file from a hot folder using PowerShell? [duplicate]

This question already has answers here:
Powershell: Count items in a folder with PowerShell
(8 answers)
Closed 8 years ago.
We have a client that uploads one .pkg file each hour to a specified folder on our FTP. I want to create a SQL Server Agent Job to grab that file, import the data into a table in a SQL Server DB, then move and rename the file.
I am successful in doing this (using the code below) except when there is only 1 file left. When there is only one file left it will not import... and then it moves the actual folder with the file in it (renaming the folder as it does this). I have also provided the errors below.
Script:
Function AutoImportCommaFlatFilesTopOne($location, $server, $database)
{
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = "Data Source=" + $server + ";Database=" + $database + ";integrated security=true"
$files = Get-ChildItem $location
$fileName = $files[0]
$full = $location + $fileName
$table = "rawUSPS"
$insertData = New-Object System.Data.SqlClient.SqlCommand
$insertData.CommandText = "EXECUTE stp_CommaBulkInsert #1,#2"
$insertData.Parameters.Add("#1", $full)
$insertData.Parameters.Add("#2", $table)
$insertData.Connection = $connection
$connection.Open()
$insertData.ExecuteNonQuery()
$connection.Close()
move-item $full (("C:\Test\archiveFolder\{0:yyyyMMdd_HHmmss}" + "_" + $fileName.BaseName + ".log") -f (get-date))
}
AutoImportCommaFlatFilesTopOne -location "C:\Test\testFolder\" -server "LAN-DP-03" -database "FlatFileInsertTestingDB"
The errors:
PS C:\Users\rcurry> C:\Scripts\140627.ps1
Unable to index into an object of type System.IO.FileInfo.
At C:\Scripts\140627.ps1:8 char:24
+ $fileName = $files[ <<<< 0]
+ CategoryInfo : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : CannotIndex
Exception calling "ExecuteNonQuery" with "0" argument(s): "Cannot bulk load because the file "C:\Test\testFolder\" could not be opened. Operating system error
code 3(The system cannot find the path specified.)."
At C:\Scripts\140627.ps1:21 char:32
+ $insertData.ExecuteNonQuery <<<< ()
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException
I figured out the problem. Per hydropowerdeveloper at stackoverflow.com/questions/14714284/… : > Well, it turns out that this is a quirk caused precisely because there was only one file in the directory. Some searching revealed that in this case, PowerShell returns a scalar object instead of an array. This object doesn’t have a count property, so there isn’t anything to retrieve. So I simply needed to force an array using '#'.

Powershell using file? "being used by another process"

I have this powershell script running. The first time it runs it runs flawlessly, the second time it runs i get the error that the .csv cannont be access "because it is being used by another process. Any idea which part of the script is "holding onto" the file and how i can make it let it go at the end?
clear
set-executionpolicy remotesigned
# change this to the directory that the script is sitting in
cd d:\directory
#############################################
# Saves usernames/accountNumbers into array #
# and creates sql file for writing to #
#############################################
# This is the location of the text file containing accounts
$accountNumbers = (Get-Content input.txt) | Sort-Object
$accountID=0
$numAccounts = $accountNumbers.Count
$outString =$null
# the name of the sql file containing the query
$file = New-Item -ItemType file -name sql.sql -Force
###################################
# Load SqlServerProviderSnapin100 #
###################################
if (!(Get-PSSnapin | ?{$_.name -eq 'SqlServerProviderSnapin110'}))
{
if(Get-PSSnapin -registered | ?{$_.name -eq 'SqlServerProviderSnapin110'})
{
add-pssnapin SqlServerProviderSnapin100
Write-host SQL Server Provider Snapin Loaded
}
else
{
}
}
else
{
Write-host SQL Server Provider Snapin was already loaded
}
#################################
# Load SqlServerCmdletSnapin100 #
#################################
if (!(Get-PSSnapin | ?{$_.name -eq 'SqlServerCmdletSnapin100'}))
{
if(Get-PSSnapin -registered | ?{$_.name -eq 'SqlServerCmdletSnapin100'})
{
add-pssnapin SqlServerCmdletSnapin100
Write-host SQL Server Cmdlet Snapin Loaded
}
else
{
}
}
else
{
Write-host SQL Server CMDlet Snapin was already loaded
}
####################
# Create SQL query #
####################
# This part of the query is COMPLETELY static. What is put in here will not change. It will usually end with either % or '
$outString = "SELECT stuff FROM table LIKE '%"
# Statement ends at '. loop adds in "xxx' or like 'xxx"
IF ($numAccounts -gt 0)
{
For ($i =1; $i -le ($AccountNumbers.Count - 1); $i++)
{
$outString = $outstring + $AccountNumbers[$accountID]
$outString = $outString + "' OR ca.accountnumber LIKE '"
$accountID++
}
$outString = $outString + $AccountNumbers[$AccountNumbers.Count - 1]
}
else
{
$outString = $outString + $AccountNumbers
}
# This is the end of the query. This is also COMPLETELY static. usually starts with either % or '
$outString = $outString + "%'more sql stuff"
add-content $file $outString
Write-host Sql query dynamically written and saved to file
###########################
# Create CSV to email out #
###########################
#Make sure to point it to the correct input file (sql query made above) and correct output csv.
Invoke-Sqlcmd -ServerInstance instance -Database database -Username username -Password password -InputFile sql.sql | Export-Csv -Path output.csv
####################################
# Email the CSV to selected people #
####################################
$emailFrom = "to"
$emailTo = "from"
$subject = "test"
$body = "test"
$smtpServer = "server"
# Point this to the correct csv created above
$filename = "output.csv"
$att = new-object Net.mail.attachment($filename)
$msg = new-object net.mail.mailmessage
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$msg.from = $emailFrom
$msg.to.add($emailto)
$msg.subject = $subject
$msg.body = $body
$msg.attachments.add($att)
$smtp.Send($msg)
Can you try to add at th end :
$att.Dispose()
$msg.Dispose()
$smtp.Dispose()
You could also try and use a tool like procmon and see what does the script do whenever it acquires a lock on the file and doesn't release it. Also, since (supposedly) the problem is with the .csv file, you could load it as byte array instead of passing it's path as an attachment. This way the file should be read once and not locked.