I just learned the benefit of using Write-Verbose & Write-Debug over my own Write-Log function, which you can find below:
Function Write-Log
{
param($logType, $logString, $logFile, [switch]$newLine)
$time = get-date -Format "HH:mm:ss"
$date = get-date -Format "yyyy-MM-dd"
$line = "[${date}][${time}][$logType] ${logString}"
if ($logFile)
{
$retryDelay = 0.5;
$maxRetries = 10;
$retries = 0;
while($retries -lt $maxRetries)
{
try
{
$line | out-file -Encoding utf8 -Append $logFile
break;
}
catch
{
++$retries;
Sleep $retryDelay;
}
}
}
if ($logType -eq 'INFO')
{
write-host -ForegroundColor Green $line
}
elseif ($logType -eq 'WARN')
{
write-host -ForegroundColor Yellow $line
}
elseif ($logType -eq 'ERROR')
{
write-host -ForegroundColor Red $line
}
if ($newLine -eq $true)
{
write-host
}
}
This helps to me keep my scripts output as little cluttered as possible and includes a timestamp which is handy when it comes to debugging.
Question
Is there a way to overload Write-Verbose so it behaves in the following way?
PS > Write-Verbose -Message 'I am a verbose message!'
[2016-02-25][07:44:36] VERBOSE: I am a verbose message!
Edit
I have found the following, which unfortunately isn't honoring the $VerbosePreference variable:
$VerbosePreference = "SilentlyContinue"
Function Private:Write-Verbose ($Message)
{
$time = get-date -Format "HH:mm:ss"
$date = get-date -Format "yyyy-MM-dd"
$line = "[${date}][${time}] "
Write-Host $line -NoNewline
&{Write-Verbose -Message $Message}
}
Write-Verbose -Message "Test"
The above will just output the date and timestamp, without the message.
Write-Verbose resides within Microsoft.PowerShell.Utility , so this is not possible afaik; without manipulating and change built-in behavior in Powershell (which should be avoided).
You could either create your own "Write-Verbose" function in your script/session scope; which would output the desired result (using cmdletbinding()); or live with an output message such as "VERBOSE: [2016-02-25][07:44:36] Your log message" (rely on the default behavior of write-verbose).
I'd recommend the latter unless you have some funky output requirements for your host.
If you go on and create your own Write-Verbose function, you should use the [cmdletbinding()] before your params; as this enables default parameters/switches to be passed to your functions(such as -verbose / -information, -debug etc).
For more information about cmdletbinding and parameter-binding see:
https://blogs.technet.microsoft.com/heyscriptingguy/2012/07/07/weekend-scripter-cmdletbinding-attribute-simplifies-powershell-functions/
https://posh2scripting.wordpress.com/2013/06/05/what-is-cmdletbinding/
Last thing; it's not recommended of using Write-host directly in your scripts as this messes with default stream redirects (etc). I would highly suggest on using Write-Verbose, Write-Debug, Write-information, Write-Output cmdlets if you're printing information to streams.
More information for not using Write-host can be found on:
http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/
http://powershell.com/cs/blogs/donjones/archive/2012/04/06/2012-scripting-games-commentary-stop-using-write-host.aspx
Hope this answers your question.
Agree with #CmdrTchort on the answer to this question.
This answer's provided to give an implementation of a custom implementation of Write-Verbose which could be used instead (i.e. by making calls to Write-CustomVerbose instead of Write-Verbose. Obviously this won't affect any existing code or code in referenced libraries which still use Write-Verbose.
function Write-CustomVerbose {
[CmdletBinding(DefaultParameterSetName='UseTimestamp')]
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[AllowEmptyString()]
[string]$Message
,
[Parameter(Mandatory = $false, ParameterSetName='UseTimestamp')]
[string]$TimestampFormat = ((Get-Culture).DateTimeFormat.UniversalSortableDateTimePattern)
,
[Parameter(Mandatory = $false, ParameterSetName='UseTimestamp')]
[switch]$UseLocalTime #defaults to UTC
,
[Parameter(Mandatory = $false, ParameterSetName='DoNotUseTimestamp')]
[switch]$ExcludeTimestamp #defaults to include the timestamp (as that's why we're using this function over the standard write-verbose
)
begin {
[string]$FormattedMessage = '{0}'
if(-not $ExcludeTimestamp.IsPresent) {
$FormattedMessage = "{1:$TimestampFormat}: $FormattedMessage"
}
}
process {
#get the time here rather than in begin as we want it to be accurate per message from pipeline
[DateTime]$Now = Get-Date
if(-not $UseLocalTime.IsPresent){$Now = $Now.ToUniversalTime()}
#output the results
write-verbose ($FormattedMessage -f $Message, $Now)
}
}
Example Usage: 1..1000 | Write-CustomVerbose -Verbose -UseLocalTime -TimestampFormat 'HH:mm'
Update
Here's a slightly more advanced version allowing you to hijack all streams at once:
function Write-Custom {
[CmdletBinding(DefaultParameterSetName='UseTimestamp')]
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[PSCustomObject]$InputObject
,
[Parameter(Mandatory = $false, ParameterSetName='UseTimestamp')]
[string]$TimestampFormat = ((Get-Culture).DateTimeFormat.UniversalSortableDateTimePattern)
,
[Parameter(Mandatory = $false, ParameterSetName='UseTimestamp')]
[switch]$UseLocalTime #defaults to UTC
,
[Parameter(Mandatory = $false, ParameterSetName='DoNotUseTimestamp')]
[switch]$ExcludeTimestamp #defaults to include the timestamp (as that's why we're using this function over the standard write-verbose
)
begin {
[string]$FormattedMessage = '{0}'
if(-not $ExcludeTimestamp.IsPresent) {
$FormattedMessage = "{1:$TimestampFormat}: $FormattedMessage"
}
}
process {
#get the time here rather than in begin as we want it to be accurate per message from pipeline
[DateTime]$Now = Get-Date
if(-not $UseLocalTime.IsPresent){$Now = $Now.ToUniversalTime()}
#determine output back to original stream
[bool]$outputStream = $true
if($InputObject.WriteErrorStream) {$outputStream=$false;write-error ($FormattedMessage -f $InputObject, $Now)}
if($InputObject.WriteWarningStream){$outputStream=$false;write-warning ($FormattedMessage -f $InputObject, $Now)}
if($InputObject.WriteVerboseStream){$outputStream=$false;write-verbose ($FormattedMessage -f $InputObject, $Now) -Verbose}
if($InputObject.WriteDebugStream) {$outputStream=$false;write-debug ($FormattedMessage -f $InputObject, $Now) -Debug}
if($outputStream){$InputObject}
}
}
#demo
1..20 | %{
if($_ % 2 -eq 0) {Write-Output $_}
if($_ -eq 11) {Write-Error $_ -ErrorAction Continue 2>&1} #bit of a hack required to get error output to flow further along the pipeline.
if($_ -eq 13) {Write-Warning $_}
if($_ -eq 15) {Write-Verbose $_ -Verbose}
if($_ -eq 17) {Write-Debug $_ -Debug}
} *>&1 | Write-Custom
Related
I have a Powershell script to read .sql files from a specific folder and run them against a database depending on the name of the filename.
The filenames are always the same: myDatabase.script.SomeRandomCharacters.csv
There can be many files which is why the script has a foreach loop.
[CmdletBinding()]
param (
[parameter(Mandatory = $true)][ValidateSet('dev')][String]$serverName,
[parameter(Mandatory = $true)][String]$databaseName,
)
$dir = Split-Path $MyInvocation.MyCommand.Path
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name -like "$databaseName*" }
foreach ($s in $scripts) {
$script = $s.FullName
Invoke-Sqlcmd -ServerInstance $serverName -Database $databaseName -InputFile $script
}
The issue here is that if I would have 2 databases "myDatabase" and "myDatabase2", running the script with the former input would run the latter as well since the Where-Object filtering uses an asterisk.
I can't figure out how to modify the script so that I get the absolute value of whatever is before the first fullstop in the filename. What I would also what to do is to validate the value between the first and second fullstops, in the example filename it is script.
Any help is appreciated!
Use the database names to construct a regex pattern that will match either:
param(
[Parameter(Mandatory = $true)][ValidateSet('dev')][String]$ServerName,
[Parameter(Mandatory = $true)][String[]]$DatabaseNames,
)
# Construct alternation pattern like `db1|db2|db3`
$dbNamesEscaped = #($DatabaseNames |ForEach-Object {
[regex]::Escape($_)
}) -join '|'
# Construct pattern with `^` (start-of-string anchor)
$dbNamePattern = '^{0}' -f $dbNamesEscaped
# Fetch scripts associated with either of the database names
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" -and $_.Name -match $dbNamePattern }
# ...
You can use the StartsWith function to fix your filter:
$scripts = Get-ChildItem $dir | Where-Object { $_.Extension -eq ".sql" } | Where-Object { $_.Name.StartsWith("$($databaseName).") }
Can anyone help me to get the script to remove the backup file older than 45 days from storage Blob container.
Actually i want to run the script in Managed instance ( PAAS) through SQL server agent job.
Please help me on this.
According this post.
I think you can modifier the PS script. Change $FromDate = ((Get-Date).AddDays(-6)).Date to $FromDate = ((Get-Date).AddDays(-45)).Date . The script is as follows:
#PowerShell Script to delete System Databases .bak files from Azure Block Blob
#-eq = equals
#-ne = not equals
#-lt = less than
#-gt = greater than
#-le = less than or equals
#-ge = greater than or equals
#Set All Variables
$YesterdayDate = ((Get-Date).AddDays(-1)).Date #Get Yesterday date
$FromDate = ((Get-Date).AddDays(-45)).Date #Get 6 Days back date from Today
$BlobType = "Pageblob"
$bacs = Get-ChildItem $location -Filter *.bak
$container="dwhdatabasesbackup"
$StorageAccountName="analytics"
$StorageAccountKey="xxxxxx"
$context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$filelist = Get-AzureStorageBlob -Container $container -Context $context
#Foreach Loop With a Condition $_.LastModified.Date -eq $YesterdayDate to make Sure there was a File Uploaded Yesterday
foreach ($filein$filelist | Where-Object {$_.LastModified.Date -eq $YesterdayDate -and $_.BlobType -eq $BlobType -and ($_.Name -like "*.bak")})
{
$Yesterdayfile = $file.Name
if ($Yesterdayfile -ne $null)
{
$FileFullLength = $Yesterdayfile.Length
$FileNameWithoutDatePart = $Yesterdayfile.SubString(0, $FileFullLength-30)
Write-Output ("File Name Without Date Part: " +$FileNameWithoutDatePart)
#Foreach Loop With a Condition $_.LastModified.Date -lt $FromDate to Remove Files those are 5 Days Old
foreach ($filein$filelist | Where-Object {$_.LastModified.Date -lt $FromDate -and $_.BlobType -eq $BlobType -and ($_.Name -like "$FileNameWithoutDatePart*.bak")})
{
$removefile = $file.Name
$RemoveFileFullLength = $removefile.Length
$ModifiedDate = $file.LastModified.Date
if (($removefile -ne $null) -and ($RemoveFileFullLength -eq $FileFullLength))
{
Write-Output ("Remove File Name: ("+$removefile +") as Modified Date: ("+ $ModifiedDate +") of File is Older Than Date: ("+ $FromDate + ")")
}
}
}
}
Using SQL2017 version 14.0.1.439. I need to change dataSource path of Connections in Tabular database with Powershell.
Here is my code:
$ServerName="localhost\tabular"
$loadInfo =
[Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices")
$server = New-Object Microsoft.AnalysisServices.Server
$server.connect($ServerName)
if ($server.name -eq $null) {
Write-Output ("Server '{0}' not found" -f $ServerName)
break
}
foreach ($d in $server.Databases )
{
Write-Output ( "Database: {0}; Status: {1}; Size: {2}MB; Data Sources: {3} " -f $d.Name, $d.State, ($d.EstimatedSize/1024/1024).ToString("#,##0"), $d.DataSources.Count )
}
My problem, that $d.DataSources.Count is always 0.
I am looking for some way to edit it with PS.
You should be accessing the data sources through Model.Datasources
I can confirm that the following code returns the number of data sources for my Tabular models.
$ServerName="localhost\tabular"
$loadInfo =
[Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices")
$server = New-Object Microsoft.AnalysisServices.Server
$server.connect($ServerName)
if ($server.name -eq $null) {
Write-Output ("Server '{0}' not found" -f $ServerName)
break
}
foreach ($d in $server.Databases )
{
Write-output ( "Database: {0}; Status: {1}; Size: {2}MB; Data Sources: {3} " -f $d.Name, $d.State, ($d.EstimatedSize/1024/1024).ToString("#,##0"), $d.Model.DataSources.Count )
}
In the next week we want to relocate our database from one server to another one.
On http://msdn.microsoft.com/en-us/library/ms187858%28v=sql.110%29.aspx
I read about detaching the database from the old location and attach it to the new location.
The problem is, that I don't have access to the file system of the server, I don't even know where exactly the server is physically located^^
Is there a way to relocate a database from one Server to another without the need to access the file system of the old Server?
You could use the Import/Export tool in SQL Server to copy the data directly which will create a new database in the destination location. The good thing about this is the new DB will work as you might expect since it is created from scratch on the target server, but that also means that you might have old, deprecated syntax in your stored procs or functions or whatever which won't work unless you lower the compatibility level (although that shouldn't be hard). also be aware of any possible collation conflicts (your old server might have SQL_Latin1_General_CP1_CI_AS and the new one might be Latin1_General_CI_AS which can cause equality operations to fail amongst other things).
In addition, if you have a big database then it'll take a long time, but I can't think of any other method off the of of my head which doesn't require some level of access to the file system as you'd still need to get to the file system to take a copy of a backup, or if using a UNC path for the backup the source server would need to be able to write to that location and you'd need to be able to access it afterwards. If anyone else can think of one I'd be interested because it would be a useful bit of knowledge to have tucked away.
Edit:
Should also have mentioned the use of Powershell and SMO - it's not really any different to using the Import/Export wizard but it does allow you to fine tune things. The following is a PS script I have been using to create a copy of a DB (schema only) on a different server to the original but with certain facets missing (NCIs, FKs Indeitites etc) as the copy was destined to be read-only. You could easily extend it to copy the data as well.
param (
[string]$sourceServerName = $(throw "Source server name is required."),
[string]$destServerName = $(throw "Destination server is required."),
[string]$sourceDBName = $(throw "Source database name is required."),
[string]$destDBName = $(throw "Destination database name is required"),
[string]$schema = "dbo"
)
# Add an error trap so that at the end of the script we can see if we recorded any non-fatal errors and if so then throw
# an error and return 1 so that the SQL job recognises there's been an error.
trap
{
write-output $_
exit 1
}
# Append year to destination DB name if it isn't already on the end.
$year = (Get-Date).AddYears(-6).Year
if (-Not $destDBName.EndsWith($year)) {
$destDBName+=$year
}
# Load assemblies.
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.ConnectionInfo") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SmoExtended") | out-null
# Set up source connection.
$sourceSrvConn = new-object Microsoft.SqlServer.Management.Common.ServerConnection
$sourceSrvConn.ServerInstance = $sourceServerName
$sourceSrvConn.LoginSecure = $false
$sourceSrvConn.Login = "MyLogin"
$sourceSrvConn.Password = "xxx"
# Set up destination connection.
$destSrvConn = new-object Microsoft.SqlServer.Management.Common.ServerConnection
$destSrvConn.ServerInstance = $destServerName
$destSrvConn.LoginSecure = $false
$destSrvConn.Login = "MyLogin"
$destSrvConn.Password = "xxx"
$sourceSrv = New-Object Microsoft.SqlServer.Management.SMO.Server($sourceSrvConn)
$sourceDb = New-Object ("Microsoft.SqlServer.Management.SMO.Database")
$destSrv = New-Object Microsoft.SqlServer.Management.SMO.Server($destSrvConn)
$destDb = New-Object ("Microsoft.SqlServer.Management.SMO.Database")
$tbl = New-Object ("Microsoft.SqlServer.Management.SMO.Table")
$scripter = New-Object Microsoft.SqlServer.Management.SMO.Scripter($sourceSrvConn)
# Get the database objects
$sourceDb = $sourceSrv.Databases[$sourceDbName]
$destDb = $destSrv.Databases[$destDbName]
# Test to see databases exist. Not as easy to test for servers - if you got those wrong then this will fail and throw an error
# so it's down to the user to check their values carefully.
if ($sourceDb -eq $null) {throw "Database '" + $sourceDbName + "' does not exist on server '" + $sourceServerName + "'"}
if ($destDb -eq $null) {throw "Database '" + $destDbName + "' does not exist on server '" + $destServerName + "'"}
# Get source objects.
$tbl = $sourceDb.tables | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$storedProcs = $sourceDb.StoredProcedures | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$views = $sourceDb.Views | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$udfs = $sourceDb.UserDefinedFunctions | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$catalogs = $sourceDb.FullTextCatalogs
$udtts = $sourceDb.UserDefinedTableTypes | Where-object { $_.schema -eq $schema -and -not $_.IsSystemObject }
$assemblies = $sourceDb.Assemblies | Where-object { -not $_.IsSystemObject }
# Set scripter options to ensure only schema is scripted
$scripter.Options.ScriptSchema = $true;
$scripter.Options.ScriptData = $false;
#Exclude GOs after every line
$scripter.Options.NoCommandTerminator = $false;
$scripter.Options.ToFileOnly = $false
$scripter.Options.AllowSystemObjects = $false
$scripter.Options.Permissions = $true
$scripter.Options.DriForeignKeys = $false
$scripter.Options.SchemaQualify = $true
$scripter.Options.AnsiFile = $true
$scripter.Options.Indexes = $false
$scripter.Options.DriIndexes = $false
$scripter.Options.DriClustered = $true
$scripter.Options.DriNonClustered = $false
$scripter.Options.NonClusteredIndexes = $false
$scripter.Options.ClusteredIndexes = $true
$scripter.Options.FullTextIndexes = $true
$scripter.Options.NoIdentities = $true
$scripter.Options.DriPrimaryKey = $true
$scripter.Options.EnforceScriptingOptions = $true
$pattern = "(\b" + $sourceDBName + "\b)"
$errors = 0
function CopyObjectsToDestination($objects) {
foreach ($o in $objects) {
if ($o -ne $null) {
try {
$script = $scripter.Script($o)
$script = $script -replace $pattern, $destDBName
$destDb.ExecuteNonQuery($script)
} catch {
#Make sure any errors are logged by the SQL job.
$ex = $_.Exception
$message = $o.Name + " " + (Get-Date)
$message += "`r`n"
#$message += $ex.message
$ex = $ex.InnerException
while ($ex.InnerException) {
$message += "`n$ex.InnerException.message"
$ex = $ex.InnerException
}
#Write-Error $o.Name
Write-Error $message # Write to caller. SQL Agent will display this (or at least some of it) in the job step history.
# Need to use Set-Variable or changes to the variable will only be in scope within the function and we want to persist this.
if ($errors -eq 0) {
Set-Variable -Name errors -Scope 1 -Value 1
}
}
}
}
}
# Output the scripts
CopyObjectsToDestination $assemblies
CopyObjectsToDestination $tbl
CopyObjectsToDestination $udfs
CopyObjectsToDestination $views
CopyObjectsToDestination $storedProcs
CopyObjectsToDestination $catalogs
CopyObjectsToDestination $udtts
# Disconnect from databases cleanly.
$sourceSrv.ConnectionContext.Disconnect()
$destSrv.ConnectionContext.Disconnect()
# Did we encounter any non-fatal errors along the way (SQL errors and suchlike)? If yes then throw an exception which tells the
# user to check the log files.
if ($errors -eq 1) {
throw "Errors encountered - see log file for details"
}
I have a very simple question. My purpose here to retrieve login names from a txt file into a variable into SQL and query the SQL table while predicating against that same variable.
So for example:
the txt file would have:
forde
blain
martin
Alex
so the idea to feed each name to a variable and output the designated computer name.
Declare #loginName varchar (25)
--open the file
--while the end of the file has not reached, read each line and place the name into #loginName variable
select *
from computerinfo
where loginname = #loginname
I don't necessarily need to bulk import to a temp table at this point.
Thanks.
i had to do this some weeks ago and the simple way i found was Powershell.
I had no SSIS else it's the best of course.
# You may want to adjust these
function Invoke-Sqlcmd2
{
[CmdletBinding()]
param(
[Parameter(Position=0, Mandatory=$true)] [string]$ServerInstance,
[Parameter(Position=1, Mandatory=$false)] [string]$Database,
[Parameter(Position=2, Mandatory=$false)] [string]$Query,
[Parameter(Position=3, Mandatory=$false)] [string]$Username,
[Parameter(Position=4, Mandatory=$false)] [string]$Password,
[Parameter(Position=5, Mandatory=$false)] [Int32]$QueryTimeout=600,
[Parameter(Position=6, Mandatory=$false)] [Int32]$ConnectionTimeout=15,
[Parameter(Position=7, Mandatory=$false)] [ValidateScript({test-path $_})] [string]$InputFile,
[Parameter(Position=8, Mandatory=$false)] [ValidateSet("DataSet", "DataTable", "DataRow")] [string]$As="DataRow"
)
if ($InputFile)
{
$filePath = $(resolve-path $InputFile).path
$Query = [System.IO.File]::ReadAllText("$filePath")
}
$conn=new-object System.Data.SqlClient.SQLConnection
if ($Username)
{ $ConnectionString = "Server={0};Database={1};User ID={2};Password={3};Trusted_Connection=False;Connect Timeout={4}" -f $ServerInstance,$Database,$Username,$Password,$ConnectionTimeout }
else
{ $ConnectionString = "Server={0};Database={1};Integrated Security=True;Connect Timeout={2}" -f $ServerInstance,$Database,$ConnectionTimeout }
$conn.ConnectionString=$ConnectionString
#Following EventHandler is used for PRINT and RAISERROR T-SQL statements. Executed when -Verbose parameter specified by caller
if ($PSBoundParameters.Verbose)
{
$conn.FireInfoMessageEventOnUserErrors=$true
$handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] {Write-Verbose "$($_)"}
$conn.add_InfoMessage($handler)
}
$conn.Open()
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$cmd.CommandTimeout=$QueryTimeout
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
[void]$da.fill($ds)
$conn.Close()
switch ($As)
{
'DataSet' { Write-Output ($ds) }
'DataTable' { Write-Output ($ds.Tables) }
'DataRow' { Write-Output ($ds.Tables[0]) }
}
} #Invoke-Sqlcmd2
$files = #(get-childitem "filelocationformultiplefile" -include *.txt -exclude *.bak -recurse | where-object {($_.LastWriteTime -le (Get-Date).AddDays(-0))-and ($_.psIsContainer -eq $false)})
if ($files -ne $NULL)
{
for ($idx = 0; $idx -lt $files.Length; $idx++)
{
$file = $files[$idx]
$Query = #"
Bulk INSERT db.dbo.tbl from '$file' with (FirstRow = 1, FieldTerminator ='";', RowTerminator = '\n')
"#
Invoke-sqlcmd2 -ServerInstance "servername" -Database "db" -Query $Query
}
}
And i'm not crazy, except the 10 ending lines everything else is coming from a microsoft official blog.
You don't need everything, powershell is present on every computer, it requires to save this in a file with extension ps1 and configure the 4 variables $fileout2 -> $tablename