bash / grep: getting multiple matching elements from a json - awk

Question

In my BitBucket+Bamboo setup, I'm trying to get a list of email addresses of people having access to a particular repository. This is the output from the BitBucket API:
{
"size": 3,
"limit": 25,
"isLastPage": true,
"values": [
{
"user": {
"name": "name1",
"emailAddress": "name1.lastname1#domain.com",
"id": 1,
"displayName": "Name1 Lastname1",
"active": true,
"slug": "name1",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name1"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name2",
"emailAddress": "name2.lastname2#domain.com",
"id": 2,
"displayName": "Name2 Lastname2",
"active": true,
"slug": "name2",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name2"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name3",
"emailAddress": "name3.lastname3#domain.com",
"id": 3,
"displayName": "Name3 Lastname3",
"active": true,
"slug": "name3",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name3"
}
]
}
},
"permission": "REPO_WRITE"
}
],
"start": 0
}
is there an easy way to, say, put all 3 email addresses into an array or a coma-separated variable within a bash script? I tried using grep and splitting the API output somehow (e.g. by 'permission'), but no luck so far. Let me note that I may be forced to use standard tools like grep, sed or awk, meaning I may not be able to use tools like jq (to process json in bash) since I cannot really temper with available build agents.
Any help would be much appreciated!

Answer 1

Consider using JQ (or another JSON query tool). It will handle any valid Json, even one that is not pretty-printed or formatted in a specific way. Ca be compined with readarray to build the array in bash.
readarray -t emails <<< "$(jq -r '.values[].user.emailAddress' < file)"
Will produce an array 'emails'
declare -p emails
declare -a emails=([0]=$'name1.lastname1#domain.com' [1]=$'name2.lastname2#domain.com' [2]=$'name3.lastname3#domain.com')
Note 2020-07-22: Added '-t' to strip trailing new lines from result array

Answer 2

Assuming your input is always that regular, this will work using any awk in any shell on every UNIX box:
$ awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Save the output in a variable or a file as you see fit, e.g.:
$ var=$(awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file)
$ echo "$var"
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com

Answer 3

Take a look on the python:
You can access directly to your api like this:
import urllib.request
import json
with urllib.request.urlopen('http://your/api') as url:
data = json.loads(url.read().decode())
or as an example with the local file with the same data as you provided:
import json
with open('./response.json') as f:
data = json.load(f)
result = {}
for x in data['values']:
node = x['user']
result[node['emailAddress']] = x['permission']
result is {'name1.lastname1#domain.com': 'REPO_WRITE', 'name2.lastname2#domain.com': 'REPO_WRITE', 'name3.lastname3#domain.com': 'REPO_WRITE'}

Answer 4

$ grep -oP '(?<="emailAddress": ).*' file |
tr -d '",' |
paste -sd,
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
or
$ grep '"emailAddress":' file |
cut -d: -f2 |
tr -d '", ' |
paste -sd,