Authenticate Azure Blob Storage Account in cloud using a runbook - azure-powershell

In AZURE ANALYSIS SERVICES Tabular model (with compatibility level 1400) I imported a Blob storage account as a Data Source. It's Authentication Kind is Key kind of authentication. The key is a Static Key.
But while refreshing the Tabular using a Runbook in Automation Account (Cloud PowerShell) is there a way to pass the key/credentials so that it could authenticate?
Otherwise the PowerShell fails with below message
The given credential is missing a required property. Data source kind: AzureBlobs. Authentication kind: Key. Property name: Key. The exception was raised by the IDbConnection interface.
Here is the Source definition copied from Model.bim file:
{
"createOrReplace": {
"object": {
"database": "azureanalysisservicesdatabase",
"dataSource": "OMSLogs"
},
"dataSource": {
"type": "structured",
"name": "OMSLogs",
"connectionDetails": {
"protocol": "azure-blobs",
"address": {
"account": "storage",
"domain": "blob.core.windows.net"
},
"authentication": null,
"query": null
},
"credential": {
"AuthenticationKind": "Key",
"kind": "AzureBlobs",
"path": "https://storage.blob.core.windows.net/",
"PrivacySetting": "Organizational"
}
}
}
}
this is the code I ran in PowerShell to process the Database:
Invoke-ProcessASDatabase -databasename $DatabaseName -server $AnalysisServerName -RefreshType "Full" -Credential $SPCredential

Okay I also hit a similar issue and found the solution, add "Key" to the "credential" object:
"credential": {
"AuthenticationKind": "Key",
"kind": "AzureBlobs",
"path": "https://storage.blob.core.windows.net/",
"PrivacySetting": "Organizational",
"Key": "<StorageAccountKey>"
}
This isn't well documented by Microsoft, but this worked for me
Update with PowerShell sample:
Get-ChildItem -Filter "drop" -Recurse -Path $sourcePath -Directory |
Get-ChildItem -recurse -filter *.asdatabase -file | ForEach-Object {
$filename = $_.fullname
$generatedFile = $buildPath + $_.BaseName + ".xmla"
"Processing $filename"
& $deploymentWizard $filename /o:$generatedFile
# Have to add Blob Key now, as Deployment Wizard doesn't like
# adding the Key (bug maybe? Or DeloyWizard isn't up to date)
$file = Get-Content $generatedFile -Raw | ConvertFrom-Json
$file.createOrReplace.database.model.dataSources | ForEach-Object {
# Add Blob Key to credential object
if ($_.name.StartsWith("AzureBlobs/")) {
$_.credential | Add-Member -Name "Key" -Value $storageKey -MemberType NoteProperty -Force
}
}
$file = $file | ConvertTo-Json -Depth 32
$file | Set-Content -Path $generatedFile -Encoding utf8
}

Related

How to create contact point in grafana using API?

I am trying to create a contact point in grafana for pagerduty using grafana API.
Tried with the help of these URLS: AlertProvisioning HTTP_API
API call reference
YAML reference of data changed to JSON and tried this way, the YAML reference
But getting error as
{"message":"invalid object specification: type should not be an empty string","traceID":"00000000000000000000000000000000"}
My API code below, replaced with dummy integration key for security.
curl -X POST --insecure -H "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -H "Content-Type: application/json" -d '{
"contactPoints": [
{
"orgId": 1,
"name": "test1",
"receivers": [
{
"uid": "test1",
"type": "pagerduty",
"settings": {
"integrationKey": "XXXXXXXXXXXXXXXX",
"severity": "critical",
"class": "ping failure",
"component": "Grafana",
"group": "app-stack",
"summary": "{{ `{{ template \"default.message\" . }}` }}"
}
}
]
}
]
},
"overwrite": false
}' http://XXXXXXXXXXXXXXXX.us-east-2.elb.amazonaws.com/api/v1/provisioning/contact-points
I would recommend to enable Grafana swagger UI. You will see POST /api/v1/provisioning/contact-points model there:
Example:
{
"disableResolveMessage": false,
"name": "webhook_1",
"settings": {},
"type": "webhook",
"uid": "my_external_reference"
}

kubernetes api access forbidden

I'm trying to get cluster info like pods through curl -k https://172.26.2.101:6443/api/v1/pods but i'm getting bellow forbidden error, however, I checked the admin rights and made sure it's in the "system:masters" group.
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "pods is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"kind": "pods"
},
"code": 403
Any idea ?!
Solved by capturing certs from the .kube/config file
client-key-data:
echo -n "LS0...Cg==" | base64 -d > admin.key
client-certificate-data:
echo -n "LS0...C==" | base64 -d > admin.crt
certificate-authority-data:
echo -n "LS0...g==" | base64 -d > ca.crt
Then, use
curl https://172.26.2.101:6443 \
--key admin.key \
--cert admin.crt
--cacert ca.crt

bash / grep: getting multiple matching elements from a json

In my BitBucket+Bamboo setup, I'm trying to get a list of email addresses of people having access to a particular repository. This is the output from the BitBucket API:
{
"size": 3,
"limit": 25,
"isLastPage": true,
"values": [
{
"user": {
"name": "name1",
"emailAddress": "name1.lastname1#domain.com",
"id": 1,
"displayName": "Name1 Lastname1",
"active": true,
"slug": "name1",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name1"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name2",
"emailAddress": "name2.lastname2#domain.com",
"id": 2,
"displayName": "Name2 Lastname2",
"active": true,
"slug": "name2",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name2"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name3",
"emailAddress": "name3.lastname3#domain.com",
"id": 3,
"displayName": "Name3 Lastname3",
"active": true,
"slug": "name3",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name3"
}
]
}
},
"permission": "REPO_WRITE"
}
],
"start": 0
}
is there an easy way to, say, put all 3 email addresses into an array or a coma-separated variable within a bash script? I tried using grep and splitting the API output somehow (e.g. by 'permission'), but no luck so far. Let me note that I may be forced to use standard tools like grep, sed or awk, meaning I may not be able to use tools like jq (to process json in bash) since I cannot really temper with available build agents.
Any help would be much appreciated!
Consider using JQ (or another JSON query tool). It will handle any valid Json, even one that is not pretty-printed or formatted in a specific way. Ca be compined with readarray to build the array in bash.
readarray -t emails <<< "$(jq -r '.values[].user.emailAddress' < file)"
Will produce an array 'emails'
declare -p emails
declare -a emails=([0]=$'name1.lastname1#domain.com' [1]=$'name2.lastname2#domain.com' [2]=$'name3.lastname3#domain.com')
Note 2020-07-22: Added '-t' to strip trailing new lines from result array
Assuming your input is always that regular, this will work using any awk in any shell on every UNIX box:
$ awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Save the output in a variable or a file as you see fit, e.g.:
$ var=$(awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file)
$ echo "$var"
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Take a look on the python:
You can access directly to your api like this:
import urllib.request
import json
with urllib.request.urlopen('http://your/api') as url:
data = json.loads(url.read().decode())
or as an example with the local file with the same data as you provided:
import json
with open('./response.json') as f:
data = json.load(f)
result = {}
for x in data['values']:
node = x['user']
result[node['emailAddress']] = x['permission']
result is {'name1.lastname1#domain.com': 'REPO_WRITE', 'name2.lastname2#domain.com': 'REPO_WRITE', 'name3.lastname3#domain.com': 'REPO_WRITE'}
$ grep -oP '(?<="emailAddress": ).*' file |
tr -d '",' |
paste -sd,
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
or
$ grep '"emailAddress":' file |
cut -d: -f2 |
tr -d '", ' |
paste -sd,

How to check whether any window is open in i3

How could I find out whether any window is opened in i3 or not? Like, check if any workspace contains any window.
You can check if any 'visible' window is open in i3 by using xdotool:
You can install xdotool with sudo pacman -S xdotool
WINDOWS=$(xdotool search --all --onlyvisible --desktop $(xprop -notype -root _NET_CURRENT_DESKTOP | cut -c 24-) "" 2>/dev/null)
NUM=$(echo "$WINDOWS" | wc -l)
if [ $NUM -eq 0 ]; then
echo "No windows open."
fi
maybe try i3-save-tree. You must install perl-anyevent-i3 and perl-json-xs first.
https://i3wm.org/docs/layout-saving.html
Example:
$ i3-save-tree --workspace 10
// vim:ts=4:sw=4:et
{
"border": "pixel",
"current_border_width": 1,
"floating": "auto_off",
"geometry": {
"height": 720,
"width": 1366,
"x": 0,
"y": 0
},
"name": "Waterfox Start Page - Waterfox",
"percent": 1,
"swallows": [
{
// "class": "^Waterfox$",
// "instance": "^Navigator$",
// "title": "^Waterfox\\ Start\\ Page\\ \\-\\ Waterfox$",
// "transient_for": "^$",
// "window_role": "^browser$"
}
],
"type": "con"
}

run powershell script from sql server agent (type: powershell)

I am trying to run simple powershell script prom a job(sql server agent).
script:
[string]$SrcFolder = "G:\MSSQL\Test";
[string]$TrgFolder = "\\xx.xx.xx.xxx\d$\sql\logshipping" ;
if (-not(Get-Module -Name SQLPS))
{
if (Get-Module -ListAvailable -Name SQLPS) {
Push-Location
Import-Module -Name SQLPS -DisableNameChecking
Pop-Location
};
};
if ($SrcFolder -eq $null -or $TrgFolder -eq $null )
{
Write-Host "The source Folder = $SrcFolder ,OR target folder = $TrgFolder is not valid/Null";
};
$prafix = "[A-Za-z]+_[0-9]+_[0-9].trn" ;
Set-Location -Path C:\ ;
# Copy to Destination
foreach ($file in gci -Path $SrcFolder | Where-Object{ $_.Mode -eq '-a---' -and $_.Extension -eq '.trn' -and $_.Name -match $prafix})
{
write-host "Starting Copy File: $($file.FullName) ." ;
Copy-Item -Path $file.FullName -Destination $TrgFolder -Force -ErrorAction Stop ;
if (Test-Path -LiteralPath "$TrgFolder\$($file.Name)")
{
write-host "End Copy File: $($file.FullName) ." ;
Move-Item -Path $file.FullName -Destination "$SrcFolder\Moved" -Force ;
}
else
{
Write-Host "The Copy File: $TrgFolder\$($file.BaseName) . Failed "
};
}
The script doing: Copy .bak files to remote server. then check if the bak file exists in the remote server, and if it's exists his move the bak file to a local folder .
The job is failing message:
Date 9/1/2016 6:29:31 PM Log Job History (LS_Manual-Copy)
Step ID 3 Server SQL2012ENT-IR-3 Job Name LS_Manual-Copy Step
Name Delete Old Logs Duration 00:00:00 Sql Severity 0 Sql Message
ID 0 Operator Emailed Operator Net sent Operator Paged Retries
Attempted 0
Message Unable to start execution of step 3 (reason: line(23): Syntax
error). The step failed.
the sql server agent is a member in the Administrator group .
please help .
10X