how to use the puppetdb API to combine facts - api

I'm using puppet version 5.3.6.
I'm able to query the puppetdb and get lots of useful information like this:
$ curl -s -X GET http://localhost:8080/pdb/query/v4/facts --data-urlencode 'query=["extract", [["function","count"],"value"],["=","name","operatingsystem"],["group_by", "value"]]' | python -mjson.tool
[
{
"count": 339,
"value": "OracleLinux"
},
{
"count": 73,
"value": "RedHat"
}
]
AND:
$ curl -s -X GET http://localhost:8080/pdb/query/v4/facts --data-urlencode 'query=["extract", [["function","count"],"value"],["=","name","operatingsystemmajrelease"],["group_by", "value"]]' | python -mjson.tool
[
{
"count": 38,
"value": "5"
},
{
"count": 217,
"value": "6"
},
{
"count": 157,
"value": "7"
}
]
How can I combine the two together and get each Oracle/Red Hat release & major release grouped together in an easy to see view. I've tried a few different ways to do it but I'm not able to find any examples or docs that can explain to me how to do it.
Other useful combinations would be all Red Hat servers in a particular DC running operatingsystemmajrelease 6 (or show all of them?). This would involve combining three facts.
This would be very ussful.
Thanks for your help!
Regards

Related

Trying to POST data to ElasticSearch server 8.6, but getting error "no handler found for uri"

I'm trying to send data to an ElasticSearch server using CURL. There is an index called 'datastream2' which has a lot of fields sorta like this:
"datastream2": {
"mappings": {
"properties": {
"UA": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 512
}
}
},
"accLang": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}...
I'd like to use CURL to send data to this index.
I've been using CURL for the attempted POST like this:
curl -v -X POST http://66-228-66-111.ip.linodeusercontent.com:9200/datastream2/newdocname -H "Content-type: application/json" --user elastic:u34XXXc2qYNGnVS4XXXA -d '{"UA":"Mozilla","acclang":"eng"}'
but it's failing with the message:
{"error":"no handler found for uri [/datastream2/newdocname] and method [POST]"}%
I will admit that I'm not sure what to put after the indexname of '/datastream2/' , but I've tried various different values. Some documentation says to list the type (which I'm not sure where to find) and some docs say that this is no longer necessary on ElasticSearch 8+ .
Any ideas how I can get this data posted into ElasticSearch?
You just need to replace newdocname by _doc and it will work
curl -v -X POST http://66-228-66-111.ip.linodeusercontent.com:9200/datastream2/_doc

Withings API Body Sample

I'm trying to get data back from this Withings endpoint: https://developer.withings.com/api-reference/#operation/measure-getmeas
But every combination of things I've tried simply returns:
status body error
503 Invalid Params
This is the most recent body that isn't working: action=getmeas&meastype=meastype&meastypes=11&category=1&startdate=1641168000&enddate=1641254399
For reference: https://developer.withings.com/api-reference/#operation/measure-getmeas
Based on what you posted, the problem is your parameter meastype=meastype. If you remove this then it should run fine.
Assuming you have followed the procedure to get an access token your call from PowerShell would look like this:
Invoke-RestMethod -Method 'Post' -Headers #{ "Authorization" = "Bearer XXXXXXXXXXXXXXXXXX" } -Body "action=getmeas&meastypes=11&category=1&startdate=1641168000&enddate=1641254399" -Uri 'https://wbsapi.withings.net/measure'
This will return a JSON structure as per the docs you link to in the question e.g.
{
"status": 0,
"body": {
"updatetime": "string",
"timezone": "string",
"measuregrps": [
{
"grpid": 12,
"attrib": 1,
"date": 1594245600,
"created": 1594246600,
"category": 1594257200,
"deviceid": "892359876fd8805ac45bab078c4828692f0276b1",
"measures": [
{
"value": 65750,
"type": 1,
"unit": -3,
"algo": 3425,
"fm": 1,
"fw": 1000
}
],
"comment": "A measurement comment"
}
],
"more": 0,
"offset": 0
}
}
If your "measuregrps" is empty (like mine is below) then it means there is no data available for the time period you selected so either your device doesn't record that parameter or the data has not been synchronised to your Withings account.
What I get when I run it (my device doesn't record HR):
status body
------ ----
0 #{updatetime=1641470158; timezone=Europe/London; measuregrps=System.Object[]}
Another option is to use Windows Subsystem for Linux to run curl commands. You essentially get the same thing:
curl --header "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXX" --data "action=getmeas&meastype=11&category=1&startdate=1609925332&enddate=1641461360" 'https://wbsapi.withings.net/measure'
gives
{
"status":0,
"body":{
"updatetime":1641470640,
"timezone":"Europe\/London",
"measuregrps":[]
}
}

bash / grep: getting multiple matching elements from a json

In my BitBucket+Bamboo setup, I'm trying to get a list of email addresses of people having access to a particular repository. This is the output from the BitBucket API:
{
"size": 3,
"limit": 25,
"isLastPage": true,
"values": [
{
"user": {
"name": "name1",
"emailAddress": "name1.lastname1#domain.com",
"id": 1,
"displayName": "Name1 Lastname1",
"active": true,
"slug": "name1",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name1"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name2",
"emailAddress": "name2.lastname2#domain.com",
"id": 2,
"displayName": "Name2 Lastname2",
"active": true,
"slug": "name2",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name2"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name3",
"emailAddress": "name3.lastname3#domain.com",
"id": 3,
"displayName": "Name3 Lastname3",
"active": true,
"slug": "name3",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name3"
}
]
}
},
"permission": "REPO_WRITE"
}
],
"start": 0
}
is there an easy way to, say, put all 3 email addresses into an array or a coma-separated variable within a bash script? I tried using grep and splitting the API output somehow (e.g. by 'permission'), but no luck so far. Let me note that I may be forced to use standard tools like grep, sed or awk, meaning I may not be able to use tools like jq (to process json in bash) since I cannot really temper with available build agents.
Any help would be much appreciated!
Consider using JQ (or another JSON query tool). It will handle any valid Json, even one that is not pretty-printed or formatted in a specific way. Ca be compined with readarray to build the array in bash.
readarray -t emails <<< "$(jq -r '.values[].user.emailAddress' < file)"
Will produce an array 'emails'
declare -p emails
declare -a emails=([0]=$'name1.lastname1#domain.com' [1]=$'name2.lastname2#domain.com' [2]=$'name3.lastname3#domain.com')
Note 2020-07-22: Added '-t' to strip trailing new lines from result array
Assuming your input is always that regular, this will work using any awk in any shell on every UNIX box:
$ awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Save the output in a variable or a file as you see fit, e.g.:
$ var=$(awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file)
$ echo "$var"
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Take a look on the python:
You can access directly to your api like this:
import urllib.request
import json
with urllib.request.urlopen('http://your/api') as url:
data = json.loads(url.read().decode())
or as an example with the local file with the same data as you provided:
import json
with open('./response.json') as f:
data = json.load(f)
result = {}
for x in data['values']:
node = x['user']
result[node['emailAddress']] = x['permission']
result is {'name1.lastname1#domain.com': 'REPO_WRITE', 'name2.lastname2#domain.com': 'REPO_WRITE', 'name3.lastname3#domain.com': 'REPO_WRITE'}
$ grep -oP '(?<="emailAddress": ).*' file |
tr -d '",' |
paste -sd,
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
or
$ grep '"emailAddress":' file |
cut -d: -f2 |
tr -d '", ' |
paste -sd,

How to add test steps in a test case work item using Rest API - TFS2018 / Python

I am using Microsoft's TFS 2018 and I have started writing some Selenium test cases using Python 3.7 in Visual Studio 2018.
I have managed to use the REST API of TFS to return my TFS projects and create new test cases.
What I couldn't find is how to use this API to pass a list with all the test steps of this test case. I am not sure how and if you can add them in the body of the request as a string or array.
At the moment I am trying to make this work on Postman first and then I am going to try in python as well.
This is the request:
curl -X POST \
'https://TFSLINK:443/DefaultCollection/TFS/_apis/wit/workitems/$Test%20Case?api-version=4.1' \
-H 'Authorization: Basic MYKEY' \
-H 'Content-Type: application/json-patch+json' \
-H 'cache-control: no-cache' \
-d '[
{
"op": "add",
"path": "/fields/System.Title",
"from": null,
"value": "Sample task 2"
}
]'
Is there a way to achieve adding steps ? The API didn't mention anything about this.
In the response I get after creating a test case I get a section called 'fields' which should have included the steps but I can't see them in my response.
{
"id": 731,
"rev": 1,
"fields": {
"System.AreaPath": "TFS",
"System.TeamProject": "TFS",
"System.IterationPath": "TFS",
"System.WorkItemType": "Test Case",
"System.State": "Design",
"System.Reason": "New",
"System.AssignedTo": "Marialena <TFS\\marialena>",
"System.CreatedDate": "2019-01-09T08:00:50.51Z",
"System.CreatedBy": "Marialena <TFS\\marialena>",
"System.ChangedDate": "2019-01-09T08:00:50.51Z",
"System.ChangedBy": "Marialena <TFS\\marialena>",
"System.Title": "Sample task 2",
"Microsoft.VSTS.Common.StateChangeDate": "2019-01-09T08:00:50.51Z",
"Microsoft.VSTS.Common.ActivatedDate": "2019-01-09T08:00:50.51Z",
"Microsoft.VSTS.Common.ActivatedBy": "Marialena <TFS\\marialena>",
"Microsoft.VSTS.Common.Priority": 2,
"Microsoft.VSTS.TCM.AutomationStatus": "Not Automated"
},
"_links": {
"self": {
"href": "https://TFSLINK/DefaultCollection/_apis/wit/workItems/731"
},
"workItemUpdates": {
"href": "https://TFSLINK/DefaultCollection/_apis/wit/workItems/731/updates"
},
"workItemRevisions": {
"href": "https://TFSLINK/DefaultCollection/_apis/wit/workItems/731/revisions"
},
"workItemHistory": {
"href": "https://TFSLINK/DefaultCollection/_apis/wit/workItems/731/history"
},
"html": {
"href": "https://TFSLINK/web/wi.aspx?pcguid=07b658c4-97e5-416f-b32d-3dd48d7f56cc&id=731"
},
"workItemType": {
"href": "https://TFSLINK/DefaultCollection/18ca0a74-cf78-45bf-b163-d8dd4345b418/_apis/wit/workItemTypes/Test%20Case"
},
"fields": {
"href": "https://TFSLINK/DefaultCollection/_apis/wit/fields"
}
},
"url": "https://TFSLINK/DefaultCollection/_apis/wit/workItems/731"
}
I have tried creating this PATCH request to update the steps but it didn't work
curl -X PATCH \
'https://TFSLINK:443/DefaultCollection/TFS/_apis/wit/workItems/730?api-version=4.1' \
-H 'Authorization: Basic MYKEY' \
-H 'Content-Type: application/json-patch+json'
-d '[
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.Steps",
"from": null,
"value": "Test"
},
{
"op": "add",
"path": "/fields/Steps",
"from": null,
"value": "Test"
}
]'
And maybe this is a another topic but if the above is achievable, can you also pass the results after you run the test and update the test plan perhaps ? If this is unrelated please help me only with the test steps and ignore this question.
Many thanks.
This is the way to add test steps in Test Case with Rest API:
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.Steps",
"value": "<steps id=\"0\" last=\"1\"><step id=\"2\" type=\"ValidateStep\"><parameterizedString isformatted=\"true\">Input step 1</parameterizedString><parameterizedString isformatted=\"true\">Expectation step 1</parameterizedString><description/></step></steps>"
}
For a few steps (3 on this example):
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.Steps",
"value": "<steps id=\"0\" last=\"4\"><step id=\"2\" type=\"ValidateStep\"><parameterizedString isformatted=\"true\"><P>step 1 \"Action\"</P></parameterizedString><parameterizedString isformatted=\"true\"><P>step 1 \"Expected\"<BR/></P></parameterizedString><description/></step><step id=\"3\" type=\"ValidateStep\"><parameterizedString isformatted=\"true\"><P>step 2 \"Action\"<BR/></P></parameterizedString><parameterizedString isformatted=\"true\"><P>step 2 \"Expected\"<BR/></P></parameterizedString><description/></step><step id=\"4\" type=\"ValidateStep\"><parameterizedString isformatted=\"true\"><P>step 3 \"Action\"<BR/></P></parameterizedString><parameterizedString isformatted=\"true\"><P>step 3 \"Expected\"<BR/></P></parameterizedString><description/></step></steps>"
}

elasticsearch not_analyzed doesn't work

Why elastic search not_analyzed doesn't work I am trying to solve
The mapping is visible here: http://i.stack.imgur.com/dGj7A.png
curl -XGET 'http://localhost:9200/gb/_analyze?field=tag?pretty' -d 'Black-cats'
curl -XGET 'http://localhost:9200/gb/_analyze?field=tweet?pretty' -d 'Black-cats'
the results are the same
{
"tokens": [{
"token": "black",
"start_offset": 0,
"end_offset": 5,
"type": "<ALPHANUM>",
"position": 1
}, {
"token": "cats",
"start_offset": 6,
"end_offset": 10,
"type": "<ALPHANUM>",
"position": 2
}]
}
You have your URL wrong, i.e. you have two ? characters, the one before pretty should be an ampersand &, try this one
curl -XGET 'http://localhost:9200/gb/_analyze?field=tag&pretty' -d 'Black-cats'
^
|
this should be a &, not a ?