PowerShell 7. ForEach-Object -Parallel Does Not Autheticate Against Azure PowerShell - azure-powershell

We wrote a script that supposed to execute Azure PowerShell commands in parallel. The problem is when we increase -ThrottleLimit higher than one, some of the commands are not being performed properly. The script is:
# Writing IPs for whitelisting into file.
Add-Content -Path IPs.txt -Value ((Get-AzWebApp -ResourceGroupName "ResourceGroup1" -Name "WebApp1").OutboundIpAddresses).Split(",")
Add-Content -Path IPs.txt -Value ((Get-AzWebApp -ResourceGroupName "ResourceGroup1" -Name "WebApp1").PossibleOutboundIpAddresses).Split(",")
# Writing new file with inique IPs.
Get-Content IPs.txt | Sort-Object -Unique | Set-Content UniqueIPs.txt
# Referencing the file.
$IPsForWhitelisting = Get-Content UniqueIPs.txt
# Assigning priotiry number to each IP
$Count = 100
$List = foreach ($IP in $IPsForWhitelisting) {
$IP|Select #{l='IP';e={$_}},#{l='Count';e={$Count}}
$Count++
}
# Whitelisting all the IPs from the list.
$List | ForEach-Object -Parallel {
$IP = $_.IP
$Priority = $_.Count
$azureApplicationId ="***"
$azureTenantId= "***"
$azureApplicationSecret = "***"
$azureSecurePassword = ConvertTo-SecureString $azureApplicationSecret -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($azureApplicationId , $azureSecurePassword)
Connect-AzAccount -Credential $credential -TenantId $azureTenantId -ServicePrincipal | Out-null
echo "IP-$Priority"
echo "$IP/24"
echo $Priority
Add-AzWebAppAccessRestrictionRule -ResourceGroupName "ResourceGroup1" -WebAppName "WebApp1" -Name "IP-$Priority" -Priority $Priority -Action Allow -IpAddress "$IP/24"
} -ThrottleLimit 1
If ThrottleLimit is set to 1 - 8 rules are being created, if ThrottleLimit is set to 2 - 7 rules are being created, 3 - 4 rules, 10 - 1 rule, hence some rules are being skipped.
What is the reason for such behavior?

In short - the -Parallel parameter does not (yet perhaps) magically import all dependent variables that fall in the scope of the For-EachObject block. In reality PWSH spans separate processes and only the array that is looped over will be implicitly passed, all other variables need explicit designations.
One should use the $using: directive (prefix) to denote which variables are to be imported (made visible) in the parallel code block.
Example:
$avar = [Int]10
$bvar = [Int]20
$list = #('here', 'it', 'eees')
$list | ForEach-Object -Parallel {
Write-Output "(a, b) is here ($($using:avar), $($using:bvar))"
Write-Output "(a, b) missing ($($avar), $($bvar))"
Write-Output "Current element is $_"
}```
*thus - the described behavior is likely due to the fact that config. variables are not imported (at all) and thus the operations silently fail.*

Related

Extract values into variables from filename in Powershell

I have a Powershell script to read .sql files from a specific folder and run them against a database depending on the name of the filename.
The filenames are always the same: myDatabase.script.SomeRandomCharacters.csv
There can be many files which is why the script has a foreach loop.
[CmdletBinding()]
param (
[parameter(Mandatory = $true)][ValidateSet('dev')][String]$serverName,
[parameter(Mandatory = $true)][String]$databaseName,
)
$dir = Split-Path $MyInvocation.MyCommand.Path
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name -like "$databaseName*" }
foreach ($s in $scripts) {
$script = $s.FullName
Invoke-Sqlcmd -ServerInstance $serverName -Database $databaseName -InputFile $script
}
The issue here is that if I would have 2 databases "myDatabase" and "myDatabase2", running the script with the former input would run the latter as well since the Where-Object filtering uses an asterisk.
I can't figure out how to modify the script so that I get the absolute value of whatever is before the first fullstop in the filename. What I would also what to do is to validate the value between the first and second fullstops, in the example filename it is script.
Any help is appreciated!
Use the database names to construct a regex pattern that will match either:
param(
[Parameter(Mandatory = $true)][ValidateSet('dev')][String]$ServerName,
[Parameter(Mandatory = $true)][String[]]$DatabaseNames,
)
# Construct alternation pattern like `db1|db2|db3`
$dbNamesEscaped = #($DatabaseNames |ForEach-Object {
[regex]::Escape($_)
}) -join '|'
# Construct pattern with `^` (start-of-string anchor)
$dbNamePattern = '^{0}' -f $dbNamesEscaped
# Fetch scripts associated with either of the database names
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" -and $_.Name -match $dbNamePattern }
# ...
You can use the StartsWith function to fix your filter:
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name.StartsWith("$($databaseName).") }

Invoke-AzVMRunCommand as a job

I am trying to use Invoke-AzVMRunCommand as a job. when I executed below script the job is created and executed successfully but I am failing to write the output like which job result belongs to which vm.
Invoke-AzVMRunCommand is used to invoke a command on a particular VM. You should have this information beforehand.
Here is some information on -AsJob parameter
https://learn.microsoft.com/en-us/powershell/module/az.compute/invoke-azvmruncommand?view=azps-2.6.0#parameters
As suggested by AmanGarg-MSFT, you should have that information before hand. You can use a hashtable $Jobs to store the server name and Invoke-AzVMRunCommand output and later iterate through using the $Jobs.GetEnumerator().
$Jobs = #{}
$Servers = "Server01","Server02"
[System.String]$ScriptBlock = {Get-Process}
$FileName = "RunScript.ps1"
Out-File -FilePath $FileName -InputObject $ScriptBlock -NoNewline
$Servers | ForEach-Object {
$vm = Get-AzVM -Name $_
$Jobs.Add($_,(Invoke-AzVMRunCommand -ResourceGroupName $vm.ResourceGroupName -Name $_ -CommandId 'RunPowerShellScript' -ScriptPath $FileName -AsJob))
}

tst10 telnet scripting continuously

I am using this website (http://npr.me.uk/scripting.html) to connect to telnet and run command. It returns me some information. I need to get this info every 4 seconds. How do I do that? Now it runs but reconnects everytime, so I have to wait while it opens a connection and it takes much more than 4s. Bat file:
echo off
cls
if exist r1.txt del r1.txt
if exist r2.txt del r2.txt
tst10.exe /r:stats.txt /o:r1.txt /m
for /f "skip=30 tokens=*" %%A in (r1.txt) do echo %%A >> r2.txt
del r1.txt
start r2.txt
And stats file:
192.168.xxx.xxx
WAIT "login:"
SEND "myuser\m"
WAIT "Password:"
SEND "mypass\m"
WAIT ">"
SEND "mycommand\m"
WAIT ">"
Use Powershell to program using a csv file with the connections, I am using it for re-programming mfd's
I have a file mfd.txt and a script that reads it in.
I have a telnet script template to change the settings on the mfd and the powershell script creates custom scripts for each mfd and sets dns and hostname parameters. When run, a logfile is piped into a directory for checking later
Script is as follows:
#Process for updating devices quickly using telnet
#Check file exists
c:
cd 'C:\Resources\Telnet'
cls
$fileisthere = $false
$fileisthere = test-path 'C:\Resources\Telnet\mfds.csv'
if ($fileisthere -ne $true)
{
""
Write-Host ("There is no MFD import list C:\Resources\telnet\mfds.csv") | out-file -filepath $logfile -force
""
exit
}
Write-Host ("MFD import List is present")
# for each device in devices:
$mfds = import-csv 'C:\Resources\Telnet\mfds.csv'
foreach ($mfd in $mfds)
{
# ping device and check for response
$mfdname = $mfd.name
$mfdip = $mfd.ipaddress
$mfddns1 = $mfd.dns1
$mfddns2 = $mfd.dns2
$mfdhostname = $mfd.serial
""
Write-Host ("Updating device $($mfdname) on IP address $($Mfdip) ")
""
("Updating device $($mfdname) on IP address $($Mfdip) ") | out-file -filepath $logfile -Append -force
if(!(Test-Connection -Cn $mfdip -BufferSize 16 -Count 1 -ea 0 -quiet))
{
Write-Host ""
Write-Host ("MFD $($mfdname) is offline or not at this address")
Write-Host ""
"" | out-file $logfile -Append -force
("MFD $($mfdname) is offline or not at this address") | out-file $logfile -Append -force
"" | out-file $logfile -Append -force
}
else
{
#find replace script
# Device is present and add to script header
$tststring = "$($mfdip) 23"
$tstfile = "$($mfdname)-$($mfdip).txt"
$tstlogfile = "$($mfdname)-$($mfdip).log"
$tststring | out-file $tstfile -force
type dns.txt >> $tstfile
$location1 = "C:\Resources\telnet\$($tstfile)"
$change1 = get-content $location1
$change1 | ForEach-Object { $_ -replace "dns 1 server", "dns 1 server $($mfddns1)"} | Set-Content $location
$location2 = "C:\Resources\telnet\$($tstfile)"
$change2 = get-content $location2
$change2 | ForEach-Object { $_ -replace "dns 2 server", "dns 2 server $($mfddns2)"} | Set-Content $location
$location3 = "C:\Resources\telnet\$($tstfile)"
$change3 = get-content $location3
$change3 | ForEach-Object { $_ -replace "hostname ether name", "hostname ether name $($mfdhostname)"} | Set-Content $location
$location4 = "C:\Resources\telnet\$($tstfile)"
$change4 = get-content $location4
$change4 | ForEach-Object { $_ -replace "devicename name", "devicename name $($mfdhostname)"} | Set-Content $location
# Create variables for update
Write-Host ("Updating $($Mfdname) on IP Address $($mfdIP) ")
$parameter1 = "/r:$($tstfile)"
$parameter2 = "/o:$($tstlogfile)"
#& cmd tst10 $parameter1 $paremeter2
write-host ("$($tstfile) $($tstlogfile)")
new-item $tstfolder -Type directory
move-item $tstfile $tstfolder
move-item $tstlogfile $tstfolder -ErrorAction SilentlyContinue
}
}

Powershell using file? "being used by another process"

I have this powershell script running. The first time it runs it runs flawlessly, the second time it runs i get the error that the .csv cannont be access "because it is being used by another process. Any idea which part of the script is "holding onto" the file and how i can make it let it go at the end?
clear
set-executionpolicy remotesigned
# change this to the directory that the script is sitting in
cd d:\directory
#############################################
# Saves usernames/accountNumbers into array #
# and creates sql file for writing to #
#############################################
# This is the location of the text file containing accounts
$accountNumbers = (Get-Content input.txt) | Sort-Object
$accountID=0
$numAccounts = $accountNumbers.Count
$outString =$null
# the name of the sql file containing the query
$file = New-Item -ItemType file -name sql.sql -Force
###################################
# Load SqlServerProviderSnapin100 #
###################################
if (!(Get-PSSnapin | ?{$_.name -eq 'SqlServerProviderSnapin110'}))
{
if(Get-PSSnapin -registered | ?{$_.name -eq 'SqlServerProviderSnapin110'})
{
add-pssnapin SqlServerProviderSnapin100
Write-host SQL Server Provider Snapin Loaded
}
else
{
}
}
else
{
Write-host SQL Server Provider Snapin was already loaded
}
#################################
# Load SqlServerCmdletSnapin100 #
#################################
if (!(Get-PSSnapin | ?{$_.name -eq 'SqlServerCmdletSnapin100'}))
{
if(Get-PSSnapin -registered | ?{$_.name -eq 'SqlServerCmdletSnapin100'})
{
add-pssnapin SqlServerCmdletSnapin100
Write-host SQL Server Cmdlet Snapin Loaded
}
else
{
}
}
else
{
Write-host SQL Server CMDlet Snapin was already loaded
}
####################
# Create SQL query #
####################
# This part of the query is COMPLETELY static. What is put in here will not change. It will usually end with either % or '
$outString = "SELECT stuff FROM table LIKE '%"
# Statement ends at '. loop adds in "xxx' or like 'xxx"
IF ($numAccounts -gt 0)
{
For ($i =1; $i -le ($AccountNumbers.Count - 1); $i++)
{
$outString = $outstring + $AccountNumbers[$accountID]
$outString = $outString + "' OR ca.accountnumber LIKE '"
$accountID++
}
$outString = $outString + $AccountNumbers[$AccountNumbers.Count - 1]
}
else
{
$outString = $outString + $AccountNumbers
}
# This is the end of the query. This is also COMPLETELY static. usually starts with either % or '
$outString = $outString + "%'more sql stuff"
add-content $file $outString
Write-host Sql query dynamically written and saved to file
###########################
# Create CSV to email out #
###########################
#Make sure to point it to the correct input file (sql query made above) and correct output csv.
Invoke-Sqlcmd -ServerInstance instance -Database database -Username username -Password password -InputFile sql.sql | Export-Csv -Path output.csv
####################################
# Email the CSV to selected people #
####################################
$emailFrom = "to"
$emailTo = "from"
$subject = "test"
$body = "test"
$smtpServer = "server"
# Point this to the correct csv created above
$filename = "output.csv"
$att = new-object Net.mail.attachment($filename)
$msg = new-object net.mail.mailmessage
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$msg.from = $emailFrom
$msg.to.add($emailto)
$msg.subject = $subject
$msg.body = $body
$msg.attachments.add($att)
$smtp.Send($msg)
Can you try to add at th end :
$att.Dispose()
$msg.Dispose()
$smtp.Dispose()
You could also try and use a tool like procmon and see what does the script do whenever it acquires a lock on the file and doesn't release it. Also, since (supposedly) the problem is with the .csv file, you could load it as byte array instead of passing it's path as an attachment. This way the file should be read once and not locked.

How do I check for the SQL Server Version using Powershell?

What's the easiest way to check for the SQL Server Edition and Version using powershell?
Just an option using the registry, I have found it can be quicker on some of my systems:
$inst = (get-itemproperty 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server').InstalledInstances
foreach ($i in $inst)
{
$p = (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server\Instance Names\SQL').$i
(Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server\$p\Setup").Edition
(Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server\$p\Setup").Version
}
Invoke-Sqlcmd -Query "SELECT ##VERSION;" -QueryTimeout 3
http://msdn.microsoft.com/en-us/library/cc281847.aspx
[reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | out-null
$srv = New-Object "Microsoft.SqlServer.Management.Smo.Server" "."
$srv.Version
$srv.EngineEdition
Obviously, replace "." with the name of your instance. If you want to see all the methods available, go here.
Hacked up advice from this thread (and some others), this went in my psprofile:
Function Get-SQLSvrVer {
<#
.SYNOPSIS
Checks remote registry for SQL Server Edition and Version.
.DESCRIPTION
Checks remote registry for SQL Server Edition and Version.
.PARAMETER ComputerName
The remote computer your boss is asking about.
.EXAMPLE
PS C:\> Get-SQLSvrVer -ComputerName mymssqlsvr
.EXAMPLE
PS C:\> $list = cat .\sqlsvrs.txt
PS C:\> $list | % { Get-SQLSvrVer $_ | select ServerName,Edition }
.INPUTS
System.String,System.Int32
.OUTPUTS
System.Management.Automation.PSCustomObject
.NOTES
Only sissies need notes...
.LINK
about_functions_advanced
#>
[CmdletBinding()]
param(
# a computer name
[Parameter(Position=0, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$ComputerName
)
# Test to see if the remote is up
if (Test-Connection -ComputerName $ComputerName -Count 1 -Quiet) {
# create an empty psobject (hashtable)
$SqlVer = New-Object PSObject
# add the remote server name to the psobj
$SqlVer | Add-Member -MemberType NoteProperty -Name ServerName -Value $ComputerName
# set key path for reg data
$key = "SOFTWARE\Microsoft\Microsoft SQL Server\Instance Names\SQL"
# i have no idea what this does, honestly, i stole it...
$type = [Microsoft.Win32.RegistryHive]::LocalMachine
# set up a .net call, uses the .net thingy above as a reference, could have just put
# 'LocalMachine' here instead of the $type var (but this looks fancier :D )
$regKey = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($type, $ComputerName)
# make the call
$SqlKey = $regKey.OpenSubKey($key)
# parse each value in the reg_multi InstalledInstances
Foreach($instance in $SqlKey.GetValueNames()){
$instName = $SqlKey.GetValue("$instance") # read the instance name
$instKey = $regKey.OpenSubkey("SOFTWARE\Microsoft\Microsoft SQL Server\$instName\Setup") # sub in instance name
# add stuff to the psobj
$SqlVer | Add-Member -MemberType NoteProperty -Name Edition -Value $instKey.GetValue("Edition") -Force # read Ed value
$SqlVer | Add-Member -MemberType NoteProperty -Name Version -Value $instKey.GetValue("Version") -Force # read Ver value
# return an object, useful for many things
$SqlVer
}
} else { Write-Host "Server $ComputerName unavailable..." } # if the connection test fails
}
To add to Brendan's code.. this fails if your machine is 64-bit, so you need to test appropriately.
Function Get-SQLSvrVer {
<#
.SYNOPSIS
Checks remote registry for SQL Server Edition and Version.
.DESCRIPTION
Checks remote registry for SQL Server Edition and Version.
.PARAMETER ComputerName
The remote computer your boss is asking about.
.EXAMPLE
PS C:\> Get-SQLSvrVer -ComputerName mymssqlsvr
.EXAMPLE
PS C:\> $list = cat .\sqlsvrs.txt
PS C:\> $list | % { Get-SQLSvrVer $_ | select ServerName,Edition }
.INPUTS
System.String,System.Int32
.OUTPUTS
System.Management.Automation.PSCustomObject
.NOTES
Only sissies need notes...
.LINK
about_functions_advanced
#>
[CmdletBinding()]
param(
# a computer name
[Parameter(Position=0, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$ComputerName
)
# Test to see if the remote is up
if (Test-Connection -ComputerName $ComputerName -Count 1 -Quiet) {
$SqlVer = New-Object PSObject
$SqlVer | Add-Member -MemberType NoteProperty -Name ServerName -Value $ComputerName
$base = "SOFTWARE\"
$key = "$($base)\Microsoft\Microsoft SQL Server\Instance Names\SQL"
$type = [Microsoft.Win32.RegistryHive]::LocalMachine
$regKey = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($type, $ComputerName)
$SqlKey = $regKey.OpenSubKey($key)
try {
$SQLKey.GetValueNames()
} catch { # if this failed, it's wrong node
$base = "SOFTWARE\WOW6432Node\"
$key = "$($base)\Microsoft\Microsoft SQL Server\Instance Names\SQL"
$regKey = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($type, $ComputerName)
$SqlKey = $regKey.OpenSubKey($key)
}
# parse each value in the reg_multi InstalledInstances
Foreach($instance in $SqlKey.GetValueNames()){
$instName = $SqlKey.GetValue("$instance") # read the instance name
$instKey = $regKey.OpenSubkey("$($base)\Microsoft\Microsoft SQL Server\$instName\Setup") # sub in instance name
# add stuff to the psobj
$SqlVer | Add-Member -MemberType NoteProperty -Name Edition -Value $instKey.GetValue("Edition") -Force # read Ed value
$SqlVer | Add-Member -MemberType NoteProperty -Name Version -Value $instKey.GetValue("Version") -Force # read Ver value
# return an object, useful for many things
$SqlVer
}
} else { Write-Host "Server $ComputerName unavailable..." } # if the connection test fails
}
Try this
Invoke-SqlCmd -query "select ##version" -ServerInstance "localhost"
Check all available method to Get the build number of the latest Cumulative Update / Service Pack that has been installed in SQL Server
Here is a version I cobbled together from some sources here and there*.
This version does not hit the registry, does not hit SQL, and doesn't even require that the instance be running. It does require that you know the instance name. If you don't know the instance name, you should be able to trivially work it out from this code.
To get this to work, replace "YourInstanceNameHere" with the name of your instance. Don't touch the $ if you do it won't work.
$ErrorActionPreference = "Stop"
$instanceName = "MSSQL`$YourInstanceNameHere"
$sqlService = Get-Service -Name $instanceName
$WMISQLservices = Get-WmiObject -Class Win32_Product -Filter "Name LIKE 'SQL Server % Database Engine Services'" | Select-Object -Property Name,Vendor,Version,Caption | Get-Unique
foreach ($sqlService in $WMISQLservices)
{
$SQLVersion = $sqlService.Version
$SQLVersionNow = $SQLVersion.Split("{.}")
$SQLvNow = $SQLVersionNow[0]
$thisInstance = Get-WmiObject -Namespace "root\Microsoft\SqlServer\ComputerManagement$SQLvNow" -Class SqlServiceAdvancedProperty | Where-Object {$_.ServiceName -like "*$instanceName*"} | Where-Object {$_.PropertyName -like "VERSION"}
}
$sqlServerInstanceVersion = $thisInstance.PropertyStrValue
if ($sqlServerInstanceVersion)
{
$majorVersion = $thisInstance.PropertyStrValue.Split(".")[0]
$versionFormatted = "MSSQL$($majorVersion)"
}
else
{
throw "ERROR: An error occured while attempting to find the SQL Server version for instance '$($instanceName)'."
}
$versionFormatted
*I also received help from and help from this this friend of mine https://stackoverflow.com/users/1518277/mqutub and I didn't want it to go uncredited.
All you need is to connect to SQL Server and run this query:
select ##version
This, of course, will work for any client tool.
Additionally, this is also available:
SELECT SERVERPROPERTY('productversion'),
SERVERPROPERTY ('productlevel'),
SERVERPROPERTY ('edition')
More ways to determine the SQL Server version here: http://support.microsoft.com/kb/321185
Just an expansion of Ben Thul's answer, It loops through a list of all my DB Servers and prints out the current version of the database engine:
[reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | out-null
$computers = #(‘XXXX-OMG-DB-01’,’XXXX-PRO-DB-01’,’XXXX-PRO-DB-02’,
’XXXX-QAT-DB-01', 'XXXX-TST-DB-01’,'YYYY-PRO-DB-01',
'YYYY-PRO-DB-02','YYYY-QAT-DB-01','YYYY-QAT-DB-02',
'YYYY-TST-DB-01','ZZZZ-DEV-DB-01','ZZZZ-DEV-DB-02')
$computers | % {
$srv = New-Object "Microsoft.SqlServer.Management.Smo.Server" $_
if ($null -eq $srv.ComputerNamePhysicalNetBIOS) {
$s = $_.tostring() + ' is unavailable'
$s.tostring()
} else {
$srv.ComputerNamePhysicalNetBIOS + ' ' +
$srv.VersionString + ' ' +
$srv.DatabaseEngineEdition
}
}
Well, here's the old school way, that's easy:
sqlcmd -Q "select ##version;"
And here's how I use it from Serverspec:
require 'windows_spec_helper'
describe 'MS SQL Server Express' do
describe service('MSSQLSERVER') do
it { should be_enabled }
it { should be_running }
end
describe port(1433) do
it { should be_listening }
end
describe command('sqlcmd -Q "select ##version;"') do
its(:stdout) { should match /Microsoft SQL Server 2008 R2 (SP2) - 10.50.4000.0 (X64)/ }
end
end