How to schedule automatic backups in teamcity? - backup

We are using Teamcity 6.5.6 professional version, which gives me the option to run a backup but I do not see any option to schedule it to a particular time.
I am not sure if this version of teamcity even supports scheduled backups. If it is not possible through teamcity GUI, I wonder if there is any other option?
Could someone please help?
Thanks.

I wrote Powershell script for TeamCity auto backups, which you can schedule and run backup.
Here's the code:
function Execute-HTTPPostCommand() {
param(
[string] $url,
[string] $username,
[string] $password
)
$authInfo = $username + ":" + $password
$authInfo = [System.Convert]::ToBase64String([System.Text.Encoding]::Default.GetBytes($authInfo))
$webRequest = [System.Net.WebRequest]::Create($url)
$webRequest.ContentType = "text/html"
$PostStr = [System.Text.Encoding]::Default.GetBytes("")
$webrequest.ContentLength = $PostStr.Length
$webRequest.Headers["Authorization"] = "Basic " + $authInfo
$webRequest.PreAuthenticate = $true
$webRequest.Method = "POST"
$requestStream = $webRequest.GetRequestStream()
$requestStream.Write($PostStr, 0, $PostStr.length)
$requestStream.Close()
[System.Net.WebResponse] $resp = $webRequest.GetResponse();
$rs = $resp.GetResponseStream();
[System.IO.StreamReader] $sr = New-Object System.IO.StreamReader -argumentList $rs;
[string] $results = $sr.ReadToEnd();
return $results;
}
function Execute-TeamCityBackup() {
param(
[string] $server,
[string] $addTimestamp,
[string] $includeConfigs,
[string] $includeDatabase,
[string] $includeBuildLogs,
[string] $includePersonalChanges,
[string] $fileName
)
$TeamCityURL = [System.String]::Format("{0}/httpAuth/app/rest/server/backup?addTimestamp={1}&includeConfigs={2}&includeDatabase={3}&includeBuildLogs={4}&includePersonalChanges={5}&fileName={6}",
$server,
$addTimestamp,
$includeConfigs,
$includeDatabase,
$includeBuildLogs,
$includePersonalChanges,
$fileName);
Execute-HTTPPostCommand $TeamCityURL "USER" "PASSWORD"
}
$server = "http://YOUR_SERVER"
$addTimestamp = $true
$includeConfigs = $true
$includeDatabase = $true
$includeBuildLogs = $true
$includePersonalChanges = $true
$fileName = "TeamCity_Backup_"
Execute-TeamCityBackup $server $addTimestamp $includeConfigs $includeDatabase $includeBuildLogs $includePersonalChanges $fileName

You could use the REST API to run the backup. We actually use TeamCity to run a scheduled build at midnight each day. That build makes a call to the rest api to do the backup.

If you don't want to write programs to perform your task, simply run this command:
wget --user=*** --password=*** "http://localhost:8085/httpAuth/app/rest/server/backup?includeConfigs=true&includeDatabase=true&includeBuildLogs=true&fileName=ScheduledBackup-" --post-data=
where stars should be replaced with your TeamCity login information.
On Windows, you can get WGET as a part of Cygwin package

For those who wants to trigger builds from Mac OS (use "command line" runner on TeamCity):
curl --basic --user user:password -X POST "http://team.city.server:8111/httpAuth/app/rest/server/backup?includeConfigs=true&includeDatabase=true&includeBuildLogs=true&fileName=ScheduledBackup"

We run maintainDB.cmd and schedule it with the Windows Task Scheduler, it's a one line command and requires no extra software. maintainDB is fully documented in the TeamCity documentation.

You can also use the environment variable to have the team city server address resolve at build time:
curl --basic --user user:pasword -X POST "%teamcity.serverUrl%/httpAuth/app/rest/server/backup?includeConfigs=true&includeDatabase=true&includeBuildLogs=true&fileName=ScheduledBackup"

Starting from #Ivan Leonenko script I added some lines of code to wait that the backup ends before exit.
function Execute-HTTPCommand() {
param(
[string] $method,
[string] $url,
[string] $username,
[string] $password
)
$authInfo = $username + ":" + $password
$authInfo = [System.Convert]::ToBase64String([System.Text.Encoding]::Default.GetBytes($authInfo))
$webRequest = [System.Net.WebRequest]::Create($url)
$webRequest.ContentType = "text/html"
$PostStr = [System.Text.Encoding]::Default.GetBytes("")
$webrequest.ContentLength = $PostStr.Length
$webRequest.Headers["Authorization"] = "Basic " + $authInfo
$webRequest.PreAuthenticate = $true
$webRequest.Method = $method
if ($method -ne "GET")
{
$requestStream = $webRequest.GetRequestStream()
$requestStream.Write($PostStr, 0, $PostStr.length)
$requestStream.Close()
}
[System.Net.WebResponse] $resp = $webRequest.GetResponse();
$rs = $resp.GetResponseStream();
[System.IO.StreamReader] $sr = New-Object System.IO.StreamReader -argumentList $rs;
[string] $results = $sr.ReadToEnd();
return $results;
}
function Execute-TeamCityBackup() {
param(
[string] $server,
[string] $addTimestamp,
[string] $includeConfigs,
[string] $includeDatabase,
[string] $includeBuildLogs,
[string] $includePersonalChanges,
[string] $fileName,
[string] $username,
[string] $password
)
$TeamCityURL = [System.String]::Format("{0}/httpAuth/app/rest/server/backup?addTimestamp={1}&includeConfigs={2}&includeDatabase={3}&includeBuildLogs={4}&includePersonalChanges={5}&fileName={6}",
$server,
$addTimestamp,
$includeConfigs,
$includeDatabase,
$includeBuildLogs,
$includePersonalChanges,
$fileName);
Write-Host "Starting TeamCity backup"
Execute-HTTPCommand "POST" $TeamCityURL $username $password
}
function Wait-TeamCityBackup() {
param(
[string] $server,
[string] $username,
[string] $password
)
$GetBackupStatusUrl = [System.String]::Format("{0}/httpAuth/app/rest/server/backup",
$server);
do {
Start-Sleep -Seconds 1
$backupStatus = Execute-HTTPCommand "GET" $GetBackupStatusUrl $username $password
Write-Host $backupStatus
} while ($backupStatus -eq 'Running')
}
$server = "http://YOUR_SERVER"
$addTimestamp = $true
$includeConfigs = $true
$includeDatabase = $true
$includeBuildLogs = $true
$includePersonalChanges = $true
$fileName = "TeamCity_Backup_"
$username = "USERNAME" # Must be a TeamCity Admin
$password = "PASSWORD"
Execute-TeamCityBackup $server $addTimestamp $includeConfigs $includeDatabase $includeBuildLogs $includePersonalChanges $fileName $username $password
Wait-TeamCityBackup $server $username $password

Related

Create SQL user fails from powershell command with incorrect syntax near password field

I am trying to run below command from powershell script on AWS EC2 instance through SSM:
$Params = #{
Comment = 'Create a SQL user'
DocumentName = 'AWS-RunPowerShellScript'
Targets = #(
#{
Key = 'InstanceIds'
Values = #(
"InstanceID"
)
})
Parameters = #{
commands = #(
'$sqlConn = New-Object System.Data.SqlClient.SqlConnection'
'$userID = "testuser"'
'$password = "testpassword"'
'$sqlConn.ConnectionString = (Get-SSMParameterValue -Name "/ops/DbConnectionString" -WithDecryption $true).Parameters.Value'
'$sqlConn.Open()'
'$sqlQuery = #(
"CREATE LOGIN [$userID] WITH PASSWORD='$password' MUST_CHANGE, CHECK_EXPIRATION = ON, CHECK_POLICY = ON",
"CREATE USER [$userID] FOR LOGIN [$userID]",
"ALTER ROLE [db_datareader] ADD MEMBER [$userID]",
"ALTER ROLE [db_datawriter] ADD MEMBER [$userID]"
)'
'$Command = New-Object System.Data.SQLClient.SQLCommand'
'$Command.Connection = $sqlConn'
'foreach($query in $sqlQuery) {
Write-Host "Command is : $query"
$Command.CommandText = $query
$Reader = $Command.ExecuteReader()
$Reader.close()
}'
'$sqlConn.close()'
)
}
}
$JsonStr = $Params | ConvertTo-Json -Depth 4
$FileName = 'SSMCommandTempFile-' + $(get-date -f MM-dd-yyyy_HH_mm_ss) + '.json'
$FilePattern = 'file://' + $FileName
$JsonStr | Out-File -FilePath $FileName -Encoding ASCII
aws ssm send-command --document-name "AWS-RunPowerShellScript" --cli-input-json $FilePattern
Remove-Item $FileName
But getting below error:
... H PASSWORD='$password' MUST_CHANGE, CHECK_EXPIRATION = ON, CHECK_POLI ...
I tried double quotes around password as well as no quotes but nothing seems to be working.

Convert SQL varbinary content into .pkg file using PowerShell

I use SQL Server and I have a SQL table called [dbo].[TemplatePackageContent] which has only two fields:
[TemplatePackageContentId] [uniqueidentifier] NOT NULL,
[Content] [varbinary](max) NULL
I'd like to create PowerShell script which reads whole content from this table and for each row, it will generate a file, in a given directory with format {TemplatePackageContentId}.pkg based on the Content field.
So far I've managed how to read the whole content of the table:
param(
[string] $dataSource = "(localdb)\mssqlLocalDb",
[string] $database = "Hda_tenancy1",
[string] $sqlCommand = $("SELECT * FROM TemplatePackageContent")
)
$connectionString = "Data Source=$dataSource; " +
"Integrated Security=SSPI; " +
"Initial Catalog=$database"
$connection = new-object system.data.SqlClient.SQLConnection($connectionString)
$command = new-object system.data.sqlclient.sqlcommand($sqlCommand,$connection)
$connection.Open()
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataSet) | Out-Null
$connection.Close()
$dataSet.Tables
Now I would like to make a mentioned conversion of received result, presumably looping thru each row.
I found this article which solves a similar problem in c# and I was thinking about using some of the logic from there and try to convert it into a PowerShell script.
What is the most optimal way to convert all the "Content" fields into files with .pkg format and what library should I use or what approach?
Any ideas?
Cheers
I ended up with this solution:
param(
[string] $dataSource = "(localdb)\mssqlLocalDb",
[string] $database = "Hda_tenancy1",
[string] $templatePath = "C:\dev\hubadvance\Seeding\Templates\"
)
$sqlCommand = $("SELECT * FROM TemplatePackageContent");
$connection = new-object System.Data.SqlClient.SQLConnection("Data Source=$dataSource;Integrated Security=SSPI;Initial Catalog=$database");
$cmd = new-object System.Data.SqlClient.SqlCommand($sqlCommand, $connection);
$connection.Open();
$reader = $cmd.ExecuteReader();
$results = #();
while ($reader.Read())
{
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i);
}
$results += new-object psobject -property $row;
}
$connection.Close();
foreach ($row in $results)
{
Write-Host $row.TemplatePackageContentId;
$path = $templatePath + $row.TemplatePackageContentId + ".pkg";
[System.IO.File]::AppendAllText($path, $row.Content);
}

Get the SQL Versions of all servers with get-wmiobject

I would like to get all the installed version values of SQL on over 200 different Servers.
The plan is, to have all the Server Names in the ServerListSQLVersions.txt
and to get all the SQL Versions into the CSV.
$Username = ''
$Password = ''
$pass = ConvertTo-SecureString -AsPlainText $Password -Force
$SecureString = $pass
# Users you password securly
$MySecureCreds = New-Object -TypeName
System.Management.Automation.PSCredential -ArgumentList $Username,$SecureString
$Array = #()
##Create a Folder called SQLVersions##
$scriptPath = "C:\Transfer to SV000229\SQL Script"
$server = Get-Content "$scriptPath\ServerListSQLVersions.txt"
$wmiobj = Get-WmiObject -class Win32_product | where Name -like '*SQL*' | Select-Object name,version
function getWMIObject($server, $wmiobj, $MySecureCreds) {
$result = Get-WmiObject $wmiobj -ComputerName $server -Credential $MySecureCreds
#Write-Host "Result: "$result
$Array+= $Result
}
$Array = Export-Csv $scriptpath\output.csv -NoTypeInformation
My output in the CSV is:
Length
0
I used a
foreach($computer in $computers){
instead of the function and gave the information manually.
Also the output was not abled to Export, because i used an = instead of an |
Works now.

Access ASANA via windows powershell using APIKEY

I have created this following two codes in powershell to access ASANA.
But both of them dont work. I always get this error
"The remote server returned an error: (401) Unauthorized."
Any help appreciated. If you have a working C# code that can connect to any secure server with only APIKey, please post it. I could convert it to powershell code.
Code 1
Function Get-WebResponseString
{
param (
[Parameter(Mandatory=$true)]
[String]$Url,
[Parameter(Mandatory=$true)]
[String]$Method,
[Parameter(Mandatory=$false)]
[System.Net.NetworkCredential]$Credential
)
$Request = [System.Net.WebRequest]::Create($Url)
$Request.Method = $Method
$Request.ContentType = "application/x-www-form-urlencoded";
if ($Credential -ne $null)
{
$Request.Credentials = $credential
write-host "****" -foregroundcolor blue
}
$Response = $Request.GetResponse()
$StreamReader = New-Object System.IO.StreamReader $Response.GetResponseStream()
$StreamReader.ReadToEnd()
}
$Url = "https://app.asana.com/api/1.0/tasks"
$Username = "MMuthusamy#xxxxxx.xom"
$apikey="xxxxxxxxx"
$credential = New-Object System.Net.NetworkCredential #($Username, $apikey)
Get-WebResponseString -Url $Url -Credential $credential -Method "GET"
Code 2
$sha = New-Object System.Security.Cryptography.SHA1CryptoServiceProvider
$apikey="xxxxxxx"
#Add colon
$authinfo=$apikey+":";
$string1 = $authinfo
Write-Host $string1 -ForeGroundColor Green
#Encoding format
$enc = [system.Text.Encoding]::UTF8
#get bytes
$data1 = $enc.GetBytes($string1)
#Encode
$result1 = $sha.ComputeHash($data1)
#convert to 64 bit
$mykey=[System.Convert]::ToBase64String($result1)
Write-Host $mykey -ForeGroundColor Green
$url = "https://app.asana.com/api/1.0/tasks"
$url="https://app.asana.com/api/1.0/users"
$request = [System.Net.WebRequest]::Create($url)
$authorization = "Authorization: Basic " + $myKey
Write-Host $authorization -ForeGroundColor Green
$request.Headers.Add($authorization)
#$request.Headers.Add("Authorization: BASIC $mykey")
$response = $request.GetResponse()
Write-Host $Response -ForeGroundColor Green
(I work at Asana)
The 401 header is a clue that the problem lies somewhere in your authorization header.
The HTTP Basic Auth spec does not call for a SHA1 hash of the username:password. It's just straight base64 encoding of that string. Try passing $authinfo to your call to ToBase64String instead of hashed data.

Multithreading Help w/Powershell

So I have a script that will go through and ping all the servers from a list that is stored in SQL Server. The script works fine but it does it all sequentially (lame).
Can someone help me out as to how I would change this to use multithreading instead of a foreach loop?
$Server = "ServerName"
$Database = "DatabaseName"
$con = "server=$Server;database=$Database;Integrated Security=sspi"
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"
$da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con)
$dt = new-object System.Data.DataTable
$da.fill($dt) | out-null
foreach ($srv in $dt)
{
$ping = new-object System.Net.NetworkInformation.Ping
$Reply = $ping.send($srv.ServerName)
$ServerName = $srv.ServerName
$ServerName
$Reply.status
if ($Reply.status –eq “Success”)
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'"
}
else
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'"
}
$Reply = ""
invoke-sqlcmd -serverinstance $Server -database $Database -query $sql
}
(Edited as per Chad Miller's Suggestion + Throttling Requirement + Wait-Job fix + STA fix)
Support.ps1
powershell -File "Main.ps1" -Sta
Main.ps1
$Server = "ServerName"
$Database = "DatabaseName"
$con = "server=$Server;database=$Database;Integrated Security=sspi"
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"
$da = New-Object System.Data.SqlClient.SqlDataAdapter -ArgumentList $cmd, $con
$dt = New-Object System.Data.DataTable
$da.Fill($dt) | Out-Null
$ThrottleLimit = 10
$activeJobs = New-Object 'System.Collections.Generic.List[Int32]'
$JobStateChanged = {
param (
[System.Object]$Sender,
[System.Management.Automation.JobStateEventArgs]$EventArgs
)
switch ($EventArgs.JobStateInfo.State)
{
Blocked { return }
Completed { $activeJobs.Remove($Sender.Id); break }
Failed { $activeJobs.Remove($Sender.Id); break }
NotStarted { return }
Running { return }
Stopped { $activeJobs.Remove($Sender.Id); break }
}
Unregister-Event -SourceIdentifier ("{0}.StateChanged" -f $Sender.Name)
}
foreach ($srv in $dt)
{
while ($true)
{
if ($activeJobs.Count -lt $ThrottleLimit)
{
$job = Start-Job -InitializationScript {
Add-PSSnapin -Name SqlServerCmdletSnapin100
} -ScriptBlock {
param (
[String]$Server,
[String]$Database,
[String]$ServerName
)
if (Test-Connection -ComputerName $ServerName -Quiet)
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'"
}
else
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'"
}
Invoke-SqlCmd -ServerInstance $Server -Database $Database -Query $sql
} -ArgumentList $Server, $Database, $srv.ServerName
$activeJobs.Add($job.Id)
Register-ObjectEvent -InputObject $job -EventName StateChanged -SourceIdentifier ("{0}.StateChanged" -f $job.Name) -Action $JobStateChanged
break
}
}
}
Get-Job | Where-Object { $_.State -eq "Running" } | Wait-Job
Get-Job | Remove-Job
If have PowerShell 2.0 you could make use of background jobs. You'll need to break up your server list into "groups". Given a source table with serverName and groupName:
CREATE TABLE [dbo].[vwServerListActive](
[serverName] [varchar](50) NULL,
[groupName] [char](1) NULL
)
A slight modification to your script (save as forum.ps1):
param($groupName)
$Server = "$env:computername\sql2k8"
$Database = "dbautility"
$con = "server=$Server;database=$Database;Integrated Security=sspi"
$cmd = "SELECT ServerName FROM dbo.vwServerListActive WHERE groupName ='$groupName'"
$da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con)
$dt = new-object System.Data.DataTable
$da.fill($dt) | out-null
foreach ($srv in $dt)
{
$ping = new-object System.Net.NetworkInformation.Ping
$Reply = $ping.send($srv.ServerName)
new-object PSObject -Property #{ServerName=$($srv.ServerName); Reply=$($Reply.status)}
}
You can then call the script for different groups:
#groupName A
start-job -FilePath .\forum.ps1 -Name "Test" -ArgumentList "A"
#groupName B
start-job -FilePath .\forum.ps1 -Name "Test" -ArgumentList "B"
Get-Job -name "test" | wait-job | out-null
Get-Job -name "test" | receive-job
#get-job -name "test" |remove-job
If you're using PowerShell V1 or sqlps you could use System.Diagnostics.ProcessStartInfo to start separate powershell.exe processes and pass the group name.
param($groupName)
$StartInfo = new-object System.Diagnostics.ProcessStartInfo
$StartInfo.FileName = "$pshome\powershell.exe"
$StartInfo.Arguments = " -NoProfile -Command C:\scripts\forum.ps1 $groupName"
$StartInfo.WorkingDirectory = "C:\scripts"
$StartInfo.LoadUserProfile = $true
$StartInfo.UseShellExecute = $true
[System.Diagnostics.Process]::Start($StartInfo) > $null
Here's a page with a script which might be useful for you. I haven't used it myself yet, so I can't comment on it beyond that.
Powershell doesn't really do multithreading at all. I've managed to crowbar it into place by faking it with a fire-and-forget script kicked off with "start [powershell path] scriptname.ps1". It'll fire off multiple insances, but you can't get data back from them without doing an end-run by way of a database or other message passing mechanism. Tracking when the child processes terminate is tricky as well.
cmd /c "start /min /low C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -command .\evtlogparse.ps1 "
In your case as you're setting a SQL string as part of the foreach loop, you can try to put the DB update code into a second script that you fire off. You'll potentially have many different processes attempting to update the same database table, so the potential for timing issues is pretty large.
Also, since you're kicking off umpty new powershell instances you're going to eat a lot of memory to make it work. The foreach loop may just be faster than kicking off a bunch of processes.
So, it can be done, but it isn't anything even resembling pretty.
First, I suggest to only create once the variable $ping outside of the 'foreach..'.
Maybe a simpler solution... now that you're using SQL 2008, why not using the SMO method 'enumAvailableSqlServers: "...SMOApplication]::EnumAvailableSqlServers($false)". This will give you a list of all available server on the network. Here's the Microsoft MSDN link so you can read about it:
http://msdn.microsoft.com/en-us/library/ms210350.aspx
so close.... this is what I've got
add-pssnapin SqlServerCmdletSnapin100
$Server = "ServerName"
$Database = "DatabaseName"
$con = "server=$Server;database=$Database;Integrated Security=sspi"
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"
$da = New-Object System.Data.SqlClient.SqlDataAdapter -ArgumentList $cmd, $con
$dt = New-Object System.Data.DataTable
$da.Fill($dt) | Out-Null
foreach ($srv in $dt)
{
Start-Job -ScriptBlock {
param (
[String]$Server,
[String]$Database,
[String]$ServerName
)
if (Test-Connection -ComputerName $ServerName -quiet)
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'"
}
else
{
$sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'"
}
Invoke-SqlCmd -ServerInstance $Server -Database $Database -Query $sql
} -ArgumentList $Server, $Database, $srv.ServerName
}
and it looks to be starting multiple jobs... but my table never gets updated. If I remove the "Start-Job" stuff and arguement list and use $srv.ServerName then it works sequentially as it did before. Any ideas? (Thanks so much BTW for all the responses)
Here's a script from Jim Truher for background jobs in PowerShell v1.0:
http://jtruher.spaces.live.com/blog/cns!7143DA6E51A2628D!130.entry
PowerShell v2.0 has background jobs built-in:
http://technet.microsoft.com/en-us/library/dd347692.aspx
-Oisin