I am trying to reset my Neptune instance following the documentation provided,
https://aws.amazon.com/blogs/database/resetting-your-graph-data-in-amazon-neptune-in-seconds/
https://docs.aws.amazon.com/neptune/latest/userguide/manage-console-fast-reset.html
When I try these approach to awscurl, I am getting BadRequestException error
{"requestId":"21cd6d80-some-more-code-25566575a4ba","code":"BadRequestException","detailedMessage":"Bad route: /system"}
This was my awscurl
awscurl -X POST --access_key ACCESS_KEY --secret_key SECRET_KEY --service neptune-db "https://neptunedbinstance-somecode.somemorecode.us-east-2.neptune.amazonaws.com:8182/system" -H 'Content-Type: application/json' --region us-east-2 -d '{ "action" : "initiateDatabaseReset" }'
Adding an answer in case others find this question.
The fast reset feature requires that the Neptune engine version needs to be 1.0.4.0 or later. There is a related blog post that describes the feature in detail.
Related
I am having issues getting these steps to work, documented in Securing Applications and Services Guide. This thread in stackoverflow showed an option which did not work as well. Can some one point me to the steps needed to fix this to allow naked impersonation?
Realm/client/policy/user details in the attached picture.
enter image description here
Query:
curl -k -X POST
–header ‘Content-Type: application/x-www-form-urlencoded’
–data-urlencode “client_id=starting-client”
–data-urlencode “client_secret=.....”
–data-urlencode “grant_type=urn:ietf:params:oauth:grant-type:token-exchange”
–data-urlencode “requested_subject=target-user”
https://localhost:8543/auth/realms/target_realm/protocol/openid-connect/token
Response:
{“error”:“access_denied”,“error_description”:“Client not allowed to exchange”}
Server logs:
22:40:08,396 WARN [org.keycloak.events] (default task-149) type=TOKEN_EXCHANGE_ERROR, realmId=target_realm, clientId=starting-client, userId=null, ipAddress=172.17.0.1, error=not_allowed, reason=‘client not allowed to impersonate’, auth_method=token_exchange, grant_type=urn:ietf:params:oauth:grant-type:token-exchange, requested_subject=target-user, client_auth_method=client-secret
This is an issue with the documentation in Securing Applications and Services Guide. Instead of add client policy for scope "user-impersonated", policy must added for scope "impersonate". Once that change was done it started to work.
Please use the below request format, and it should work. Tested and verified.
curl --location --request POST 'https://api.kong.stg.vrpintegration.dev/requestJWT/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode 'client_id=CLIENT_ID' \
--data-urlencode 'client_secret=bddf3a83-f4ec-45f6-afa0-cf9c94cb5518' \
--data-urlencode 'subject_token=xxxxxx' \
--data-urlencode 'audience=CLIENT_ID' \
--data-urlencode 'requested_token_type=urn:ietf:params:oauth:token-type:refresh_token'
Make sure you checked the user exist, because if it doesn't exist it will return "Client cannot impersonate".
Also, there is some contraction on the doc.
Reading it from
redhat keycloak it is using impersonate instead of user-impersonate in user permission while for keycloak official doc it added user-impersonate, I used impersonate and added my starting-client. It works for me.
GraphDB Free/9.4.1, RDF4J/3.3.1
I'm working on using the /rest/data/import/server/{repo-id} endpoint to initiate the importing of an RDF/XML file.
Steps:
put SysML.owl in the ${graphdb.workbench.importDirectory} directory.
chmod a+r SysML.owl
create repository test1 (in Workbench - using all defaults except RepositoryID := "test1")
curl http://127.0.0.1:7200/rest/data/import/server/test1 => as expected:
[{"name":"SysML.owl","status":"NONE"..."timestamp":1606848520821,...]
curl -XPOST --header 'Content-Type: application/json' --header 'Accept: application/json' -d ' { "fileNames":[ "SysML.owl" ] }' http://127.0.0.1:7200/rest/data/import/server/test1 => SC==202
after 60 seconds, curl http://127.0.0.1:7200/rest/data/import/server/test1 =>
[{"name":"SysML.owl","status":"DONE","message":"Imported successfully in 7s.","context":null,"replaceGraphs":[],"baseURI":
"file:/home/steve/graphdb-import/SysML.owl", "forceSerial":false,"type":"file","format":null,"data":null,"timestamp":
1606848637716, [...other json content deleted]
Repository test1 now has the 263,119 (824 inferred) statements from SysML.owl loaded
BUT if I then
delete the repository using the Workbench page at http://localhost:7200/repository, wait 180 seconds
curl http://127.0.0.1:7200/rest/data/import/server/test => same as in step 5 above, despite repository having been deleted.
curl -X GET --header 'Accept: application/json' 'http://localhost:7200/rest/repositories' => test1 not shown.
create the repository again, using the Workbench - same settings as previously. wait 60 seconds. Initial 70 statements present.
curl http://127.0.0.1:7200/rest/data/import/server/test1 =>
The same output as from the earlier usage - when I was using the prior repository instance. "status":"DONE", same timestamp - which is prior to the time at which I deleted, recreated the test1 repository.
The main-2020-12-01.log shows the INFO messages pertaining to the repository test1, plugin registrations, etc. Nothing indicating why the prior repository instance's import status is lingering.
And this is of concern because I was expecting to use some polling of the status to determine when the data is loaded so my processing can proceed. Some good news - I can issue the import server file request again and after waiting 60 seconds, the 263,119 statements are present. But the timestamp on the import is the earlier repo instance's timestamp. It was not reset via the latest import request.
I'm probably missing some cleanup step(s), am hoping someone knows which.
Thanks,
-Steve
The status is simply for your reference and doesn't represent the actual presence of data in the repository. You could achieve a similar thing simply by clearing all data in the repository without recreating it.
If you really need to rely on those status records you can clear the status for a given file once you polled it and determined it's done (or prior to starting an import) with this curl:
curl -X DELETE http://127.0.0.1:7200/rest/data/import/server/test1/status \
-H 'content-type: application/json' -d '["SysML.owl"]'
Note that this is an undocumented API and it may change without notice.
I am trying to enable kv secret engine at secret path in my vault setup..
I can easily do it using CLI
vault secrets enable -path=secret kv
But I have to make it work using Vault's HTTP APIs.
I have gone through documentation but could not find any endpoint for the above command.
Thanks in advance
This is covered under the System Backend/sys/mounts API reference page.
Issue a POST request to /v1/sys/mounts/<mountpoint> with a payload containing the type (kv) and various configuration options. For KV, you probably want to specify version: 2 (or type kv-v2) unless you want to stick to V1.
See the above link for details on the possible parameters.
Here is the example from the docs:
payload.json:
{
"type": "aws",
"config": {
"force_no_cache": true
}
}
Request:
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data #payload.json \
http://127.0.0.1:8200/v1/sys/mounts/my-mount
I can load the MonDO ontology into GraphDB Free 9 with the /rest/data/import/upload/{repositoryID}/url method with this body:
{
"context": "http://purl.obolibrary.org/obo/mondo.owl",
"data": "https://github.com/monarch-initiative/mondo/releases/download/current/mondo.owl",
"format": "RDF/XML"
}
I can also load this via the Workbench, but not programatically:
http://data.bioontology.org/ontologies/ICD9CM/submissions/17/download?apikey=8b5b7825-538d-40e0-9e9e-5ab9274a9aeb
I set the format line to "Turtle" and I'm getting 202 responses, but the workbench doesn't show any import. It seems like some of the time I see error messages on the workbench's import page, but I don't understand what corrective action to take.
For example, if I intentionally import ICD9 via the workbench, with the wrong format (RDF/XML), then I see
RDF Parse Error: Content is not allowed in prolog. [line 2, column 1]
curl 'http://localhost:7200/rest/data/import/upload/w2/url' -H 'Content-Type: application/json;charset=UTF-8' -H 'Accept: application/json, text/plain, */*' --data-binary '{"type":"url","name":"http://data.bioontology.org/ontologies/ICD9CM/submissions/17/download?apikey=8b5b7825-538d-40e0-9e9e-5ab9274a9aeb","format":"text/turtle","data":"http://data.bioontology.org/ontologies/ICD9CM/submissions/17/download?apikey=8b5b7825-538d-40e0-9e9e-5ab9274a9aeb","status":"NONE","message":"","context":"","replaceGraphs":[],"baseURI":null,"forceSerial":false,"timestamp":1534939094325,"parserSettings":{"preserveBNodeIds":false,"failOnUnknownDataTypes":false,"verifyDataTypeValues":false,"normalizeDataTypeValues":false,"failOnUnknownLanguageTags":false,"verifyLanguageTags":true,"normalizeLanguageTags":false,"verifyURISyntax":true,"verifyRelativeURIs":true,"stopOnError":true}}
GraphDB handles api key but you should provide file format in the way up or "format":"text/turtle". Hope this helps.
Thanks to Sava from Ontotext, I was able to construct this minimal curl command that successfully loads the ICD9CM Turtle file from the NCBO BioPortal.
curl -d \
'{"type":"url","format":"text/turtle","data":"http://data.bioontology.org/ontologies/ICD9CM/submissions/17/download?apikey=8b5b7825-538d-40e0-9e9e-5ab9274a9aeb","context":"http://data.bioontology.org/ontologies/ICD9CM/"}' \
-H 'Content-Type: application/json;charset=UTF-8' \
-X POST http://localhost:7200/rest/data/import/upload/disease_diagnosis_dev/url
I left out many of the keys, including the timestamp and all of the parserSettings.
I used the - d ... -X POST curl style instead of the --data-binary style
I don't claim to know all of the consequences of those decisions.
Here's my approach in R
library(httr)
post.endpoint <- "http://localhost:7200//rest/data/import/upload/disease_diagnosis_dev/url"
update.body <- '{
"type":"url",
"format":"text/turtle",
"context": "http://purl.bioontology.org/ontology/ICD9CM/",
"data": "http://data.bioontology.org/ontologies/ICD9CM/submissions/17/download?apikey=9cf735c3-a44a-404f-8b2f-c49d48b2b8b2"
}'
post.result <- POST(post.endpoint,
body = update.body,
content_type("application/json"))
curl 'http://localhost:7200/rest/data/import/upload/abc/url' -H 'Content-Type: application/json;charset=UTF-8' -H 'Accept: application/json, text/plain, */*' --data-binary '{"type":"url","name":"http://www.w3.org/TR/owl-guide/wine.rdf","format":"","data":"http://www.w3.org/TR/owl-guide/wine.rdf","status":"NONE","message":"","context":"","replaceGraphs":[],"baseURI":null,"forceSerial":false,"timestamp":1534939094325,"parserSettings":{"preserveBNodeIds":false,"failOnUnknownDataTypes":false,"verifyDataTypeValues":false,"normalizeDataTypeValues":false,"failOnUnknownLanguageTags":false,"verifyLanguageTags":true,"normalizeLanguageTags":false,"verifyURISyntax":true,"verifyRelativeURIs":true,"stopOnError":true}}'
where,
abc - is the repository id
http://www.w3.org/TR/owl-guide/wine.rdf - is the URL to import
1534939094325 - the current timestamp in seconds since epoch (in bash, equivalent to date +%s)
If you want to perform regular updates I advise you to put each file in its own graph (using the "context":"<file's url>") and then replace it with "replaceGraphs":"<file's url>". The database will create a delta and update only the changed statements.
I'm working on an app using Pushbullet's API, but I'm running into odd errors when running through the sample code at https://docs.pushbullet.com/v2/pushes/.
I'm executing the following cURL command (in Windows):
curl -k -u <MY_API_KEY>: -X POST https://api.pushbullet.com/v2/pushes --header 'Content-Type: application/json' --data-binary '{"type": "note", "title": "Note Title", "body": "Note Body"}'
...but it keeps generating the following error:
{"error": {"type":"invalid_request","message":"The param 'type' has an invalid value.","param":"type","cat":"\u003e:3"}}
It also produces this error:
The other commands for the other endpoints in the documentation work fine...it's just this one.
Got any suggestions? Thanks for the help! :)
It looks like windows doesn't support those kinds of quotes on the command line. Here's an example that works:
curl https://api.pushbullet.com/v2/pushes -X POST -u <access token>: --header "Content-Type: application/json" --data-binary "{\"type\": \"note\", \"title\":\"Note Title\", \"body\": \"Note Body\"}"
I think I'm going to try to replace the curl examples with something that has less confusing behavior.
I figured it out - I don't really know why, but the cURL command wasn't working through the DOS prompt, and also wasn't working using the Postman REST client for Chrome, but I got it working in the DHC extension for Chrome. The trick was setting the Authorization header to "Basic", which resolves the Pushbullet access token to some other form, and makes a successful the HTTP request.
Hope this helps someone down the road if they run into this on Windows!