Why rebol fails with stackoverflow api? - rebol

If I type in a browser (see https://stackapps.com/questions/2/getting-started-with-the-api) :
http://api.stackoverflow.com/1.0/stats
it returns
{
"statistics": [
{
"total_questions": 800830,
"total_unanswered": 131356,
"total_accepted": 500653,
"total_answers": 2158752,
"total_comments": 3125048,
"total_votes": 7601765,
"total_badges": 798091,
"total_users": 289282,
"questions_per_minute": 1.50,
"answers_per_minute": 3.12,
"badges_per_minute": 1.20,
"views_per_day": 455215.44,
"api_version": {
"version": "1.0",
"revision": "2010.7.17.1"
},
"site": {
"name": "Stack Overflow",
"logo_url": "http://sstatic.net/stackoverflow/img/logo.png",
"api_endpoint": "http://api.stackoverflow.com",
"site_url": "http://stackoverflow.com",
"description": "Q&A for professional and enthusiast programmers",
"icon_url": "http://sstatic.net/stackoverflow/apple-touch-icon.png",
"state": "normal",
"styling": {
"link_color": "#0077CC",
"tag_foreground_color": "#3E6D8E",
"tag_background_color": "#E0EAF1"
}
}
}
]
}
If I type this in rebol console:
read http://api.stackoverflow.com/1.0/stats
It returns some weird binary chars.

probe load to-string gunzip to-string read/binary http://api.stackoverflow.com/1.0/stats
connecting to: api.stackoverflow.com
{
"statistics": [
{
"total_questions": 801559,
"total_unanswered": 131473,
"total_accepted": 501129,
"total_answers": 2160171,
"total_comments": 3127759,
"total_votes": 7607247,
"total_badges": 798608,
"total_users": 289555,
"questions_per_minute": 0.93,
"answers_per_minute": 1.83,
"badges_per_minute": 0.73,
"views_per_day": 455579.60,
"api_version": {
"version": "1.0",
"revision": "2010.7.17.2"
},
"site": {
"name": "Stack Overflow",
"logo_url": "http://sstatic.net/stackoverflow/img/logo.png",
"api_endpoint": "http://api.stackoverflow.com",
"site_url": "http://stackoverflow.com",
"description": "Q&A for professional and enthusiast programmers",
"icon_url": "http://sstatic.net/stackoverflow/apple-touch-icon.png",
"state": "normal",
"styling": {
"link_color": "#0077CC",
"tag_foreground_color": "#3E6D8E",
"tag_background_color": "#E0EAF1"
}
}
}
]
}

REBOL is ignoring the Content-Encoding: gzip response header, which stackoverflow seems adamant to use, regardless of what you put in your Accept-Encoding: header. On Unix, wget and curl have the same problem, but I can do this to see the intended content:
curl http://api.stackoverflow.com/1.0/stats | zcat
Does REBOL have a way to uncompress gzip content?

Based on http://www.mail-archive.com/rebol-bounce#rebol.com/msg03531.html
>> do http://www.rebol.org/download-a-script.r?script-name=gunzip.r
connecting to: www.rebol.org
Script: "gunzip" (30-Dec-2004)
>> print to-string gunzip read http://api.stackoverflow.com/1.0/stats
connecting to: api.stackoverflow.com
{
"statistics": [
{
"total_questions": 801316,
"total_unanswered": 131450,
"total_accept���������������������accept������E531450,
"tocomment312672�vote7605283badge7984187946531450,
tal_unans_per_minutet.0531450,
....
almost works :)
so the core code is all there, just not exposed properly... it's a pity indeed...
but stackoverflow is not nice either not being complaint to the http specs and ignoring the accept-encoding header...

Related

Trying to POST data to ElasticSearch server 8.6, but getting error "no handler found for uri"

I'm trying to send data to an ElasticSearch server using CURL. There is an index called 'datastream2' which has a lot of fields sorta like this:
"datastream2": {
"mappings": {
"properties": {
"UA": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 512
}
}
},
"accLang": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}...
I'd like to use CURL to send data to this index.
I've been using CURL for the attempted POST like this:
curl -v -X POST http://66-228-66-111.ip.linodeusercontent.com:9200/datastream2/newdocname -H "Content-type: application/json" --user elastic:u34XXXc2qYNGnVS4XXXA -d '{"UA":"Mozilla","acclang":"eng"}'
but it's failing with the message:
{"error":"no handler found for uri [/datastream2/newdocname] and method [POST]"}%
I will admit that I'm not sure what to put after the indexname of '/datastream2/' , but I've tried various different values. Some documentation says to list the type (which I'm not sure where to find) and some docs say that this is no longer necessary on ElasticSearch 8+ .
Any ideas how I can get this data posted into ElasticSearch?
You just need to replace newdocname by _doc and it will work
curl -v -X POST http://66-228-66-111.ip.linodeusercontent.com:9200/datastream2/_doc

How can i custom config CHANGELOG.md using standard-version npm package?

I'm using the command standard-version each time I want to publish new version, but the yielded changes in the CHANGELOG.md look like this:
### [10.1.9](https://github.com/my-project-name/compare/v10.1.8...v10.1.9) (2021-03-29)
### [10.1.8](https://github.com/my-project-name/compare/v10.1.7...v10.1.8) (2021-03-29)
### [10.1.7](https://github.com/my-project-name/compare/v10.1.6...v10.1.7) (2021-03-29)
first the links do not work - the github url is not correct and i want to configure it to the right url, and second, I'd like to configure the link that's shown in the changeslog file (there are some types)
I tried to use this documentation but didn't find anything that can help me
https://github.com/conventional-changelog/conventional-changelog
so how do I configure the way standard-version works on the CHANGELOG.md ? can someone provide example?
yes.
according to doc:
You can configure standard-version either by:
Placing a standard-version stanza in your package.json (assuming your project is JavaScript).
Creating a .versionrc, .versionrc.json or .versionrc.js.
If you are using a .versionrc.js your default export must be a configuration object, or a function returning a configuration object.
Any of the command line parameters accepted by standard-version can instead be provided via configuration.
Please refer to the conventional-changelog-config-spec for details on available configuration options.
example:
.versionrc
{
"types": [
{
"type": "feat",
"section": "Features"
},
{
"type": "fix",
"section": "Bug Fixes"
},
{
"type": "chore",
"hidden": true
},
{
"type": "docs",
"hidden": true
},
{
"type": "style",
"hidden": true
},
{
"type": "refactor",
"section": "Refactor"
},
{
"type": "perf",
"section": "Performance"
},
{
"type": "test",
"hidden": true
}
]
}

bash / grep: getting multiple matching elements from a json

In my BitBucket+Bamboo setup, I'm trying to get a list of email addresses of people having access to a particular repository. This is the output from the BitBucket API:
{
"size": 3,
"limit": 25,
"isLastPage": true,
"values": [
{
"user": {
"name": "name1",
"emailAddress": "name1.lastname1#domain.com",
"id": 1,
"displayName": "Name1 Lastname1",
"active": true,
"slug": "name1",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name1"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name2",
"emailAddress": "name2.lastname2#domain.com",
"id": 2,
"displayName": "Name2 Lastname2",
"active": true,
"slug": "name2",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name2"
}
]
}
},
"permission": "REPO_WRITE"
},
{
"user": {
"name": "name3",
"emailAddress": "name3.lastname3#domain.com",
"id": 3,
"displayName": "Name3 Lastname3",
"active": true,
"slug": "name3",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://bitbucket.com/stash/users/name3"
}
]
}
},
"permission": "REPO_WRITE"
}
],
"start": 0
}
is there an easy way to, say, put all 3 email addresses into an array or a coma-separated variable within a bash script? I tried using grep and splitting the API output somehow (e.g. by 'permission'), but no luck so far. Let me note that I may be forced to use standard tools like grep, sed or awk, meaning I may not be able to use tools like jq (to process json in bash) since I cannot really temper with available build agents.
Any help would be much appreciated!
Consider using JQ (or another JSON query tool). It will handle any valid Json, even one that is not pretty-printed or formatted in a specific way. Ca be compined with readarray to build the array in bash.
readarray -t emails <<< "$(jq -r '.values[].user.emailAddress' < file)"
Will produce an array 'emails'
declare -p emails
declare -a emails=([0]=$'name1.lastname1#domain.com' [1]=$'name2.lastname2#domain.com' [2]=$'name3.lastname3#domain.com')
Note 2020-07-22: Added '-t' to strip trailing new lines from result array
Assuming your input is always that regular, this will work using any awk in any shell on every UNIX box:
$ awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Save the output in a variable or a file as you see fit, e.g.:
$ var=$(awk -F'"' '$2=="emailAddress"{addrs=addrs sep $4; sep=","} END{print addrs}' file)
$ echo "$var"
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
Take a look on the python:
You can access directly to your api like this:
import urllib.request
import json
with urllib.request.urlopen('http://your/api') as url:
data = json.loads(url.read().decode())
or as an example with the local file with the same data as you provided:
import json
with open('./response.json') as f:
data = json.load(f)
result = {}
for x in data['values']:
node = x['user']
result[node['emailAddress']] = x['permission']
result is {'name1.lastname1#domain.com': 'REPO_WRITE', 'name2.lastname2#domain.com': 'REPO_WRITE', 'name3.lastname3#domain.com': 'REPO_WRITE'}
$ grep -oP '(?<="emailAddress": ).*' file |
tr -d '",' |
paste -sd,
name1.lastname1#domain.com,name2.lastname2#domain.com,name3.lastname3#domain.com
or
$ grep '"emailAddress":' file |
cut -d: -f2 |
tr -d '", ' |
paste -sd,

is there a wikipedia api call that can retrieve restriction status of the article?

Otherwise I must do querySelector on the page content to find if there is a some kind of padlock and by try and error check what (id or class) is unique to that icon.
Other source to find is this info is to go on information page by adding $action=info to the url params. But then another problem comes in that the protection status is written in that's particular wiki language.
Using the API is the right way to do it, but you need to use action=query. The padlocks icons are inconsistent across wikis, and most wikis probably don't even have them.
If you use the right parameters for your API query, you should be getting the results you're looking for.
Example for the English Wikipedia:
https://en.wikipedia.org/w/api.php?action=query&prop=info&format=json&inprop=protection&titles=Elton%20John gives you this result:
{
"batchcomplete": "",
"query": {
"pages": {
"5052197": {
"pageid": 5052197,
"ns": 0,
"title": "Elton John",
"contentmodel": "wikitext",
"pagelanguage": "en",
"touched": "2015-10-02T03:49:24Z",
"lastrevid": 683730854,
"length": 115931,
"protection": [
{
"type": "edit",
"level": "autoconfirmed",
"expiry": "infinity"
},
{
"type": "move",
"level": "sysop",
"expiry": "infinity"
}
],
"restrictiontypes": [
"edit",
"move"
]
}
}
}
}
Here the protection array tells you that only sysops can move the page, and only autoconfirmed users can edit it.
If you make a similar query on another wiki, say the French Wikipedia: https://fr.wikipedia.org/w/api.php?action=query&prop=info&format=json&inprop=protection&titles=Malia%20Obama , you get this in response (trimmed):
"protection": [
{
"type": "edit",
"level": "sysop",
"expiry": "infinity"
},
{
"type": "move",
"level": "sysop",
"expiry": "infinity"
}
],
"restrictiontypes": [
"edit",
"move"
]
In this case, sysops are the only one who can move and edit the page.

Error loading file stored in Google Cloud Storage to Big Query

I have been trying to create a job to load a compressed json file from Google Cloud Storage to a Google BigQuery table. I have read/write access in both Google Cloud Storage and Google BigQuery. Also, the uploaded file belongs in the same project as the BigQuery one.
The problem happens when I access to the resource behind this url https://www.googleapis.com/upload/bigquery/v2/projects/NUMERIC_ID/jobs by means of a POST request. The content of the request to the abovementioned resource can be found as follows:
{
"kind" : "bigquery#job",
"projectId" : NUMERIC_ID,
"configuration": {
"load": {
"sourceUris": ["gs://bucket_name/document.json.gz"],
"schema": {
"fields": [
{
"name": "id",
"type": "INTEGER"
},
{
"name": "date",
"type": "TIMESTAMP"
},
{
"name": "user_agent",
"type": "STRING"
},
{
"name": "queried_key",
"type": "STRING"
},
{
"name": "user_country",
"type": "STRING"
},
{
"name": "duration",
"type": "INTEGER"
},
{
"name": "target",
"type": "STRING"
}
]
},
"destinationTable": {
"datasetId": "DATASET_NAME",
"projectId": NUMERIC_ID,
"tableId": "TABLE_ID"
}
}
}
}
However, the error doesn't make any sense and can also be found below:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "invalid",
"message": "Job configuration must contain exactly one job-specific configuration object (e.g., query, load, extract, spreadsheetExtract), but there were 0: "
}
],
"code": 400,
"message": "Job configuration must contain exactly one job-specific configuration object (e.g., query, load, extract, spreadsheetExtract), but there were 0: "
}
}
I know the problem doesn't lie either in the project id or in the access token placed in the authentication header, because I have successfully created an empty table before. Also I specify the content-type header to be application/json which I don't think is the issue here, because the body content should be json encoded.
Thanks in advance
Your HTTP request is malformed -- BigQuery doesn't recognize this as a load job at all.
You need to look into the POST request, and check the body you send.
You need to ensure that all the above (which seams correct) is the body of the POST call. The above Json should be on a single line, and if you manually creating the multipart message, make sure there is an extra newline between the headers and body of each MIME type.
If you are using some sort of library, make sure the body is not expected in some other form, like resource, content, or body. I've seen libraries that use these differently.
Try out the BigQuery API explorer: https://developers.google.com/bigquery/docs/reference/v2/jobs/insert and ensure your request body matches the one made by the API.