I am new to elasticsearch and trying to find a way to create a document from url(http-API). I have tried below given options but none of them worked.
http://Myserver:9200/dilbert/user/3 -d '{ "name" : "Praveena" }
http://Myserver:9200/dilbert/user/3{ "name" : "Praveena" }
http://Myserver:9200/dilbert/user/3?pretty _create name=Praveena
I expect this to add a record. Here dilbert is Index name. user is type & 3 is id. This index only contains a single element called name.
First option should work, but you must explicitly set request type to PUT.
It seems to me that you use curl to insert data. If you send data to server with -d option, curl issues the POST request by default. But you specify id of document in URL, so Elasticsearch waits for PUT request (see official documentation). You can use POST requests only in case of automatic ID generation.
So your request may look as follows:
curl -X PUT http://Myserver:9200/dilbert/user/3 -d '{ "name" : "Praveena" }'
Related
I try to post Intent and entity via wit.ai HTTP API.
My JSON format:
{"entities"=>[{"id"=>"intent", "lookups"=>["trait"], "values"=> [{"value"=>"ask_info", "expressions"=>["How old are you ?"]}]}, {"id"=>"age", "values"=>[{"value"=>"old", "expressions"=>["How old are you ?"]}]}]}
Input sentence is "How old are you ?"
Intent is ask_info
Entity is age for value 'old'
I call post entitiles API twice for Intent and Entity
$ curl -XPOST 'https://api.wit.ai/entities?v=20160526' \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"id"=>"intent", "lookups"=>["trait"], "values"=> [{"value"=>"ask_info", "expressions"=>["How old are you ?"]}]}'
$ curl -XPOST 'https://api.wit.ai/entities?v=20160526' \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"id"=>"age", "values"=>[{"value"=>"old", "expressions"=>["How old are you ?"]}]}'
In wit.ai page, entity age not mapping to expression "How old are you ?".
Just display in Synonyms
Display Image
Downloaded Dataset only show intent no entity
{
"text" : "How old are you ?",
"entities" : [
{
"entity" : "intent",
"value" : "\"ask_info\""
}
]
}
wit.ai GUI work very nice
{
"text" : "How old are you ?",
"entities" : [
{
"entity" : "intent",
"value" : "\"ask_info\""
},
{
"entity" : "age",
"value" : "\"old\"",
"start" : 2,
"end" : 3
}
]
}
Do you have any method could solve this problem?
I've bypassed this bug by manually crafting an archive which I later used for deploy.
There are multiple issues with this approach:
Apps cannot be updated from an archive, the importer only supports creating new apps as far as I know. This means the history will be lost between deploys and the server token will need to be changed.
The importer seems to be a little flaky. Tampering with the archive is possible, but zipping back is complicated: all files should be in an directory, no directory entries should be present, and the file order also seems to be important.
The upside is multiple features that are not available in the API are exposed, such as setting the 'free-text' lookups option.
I've downloaded an sample application and proceeded from there.
I'm planning to migrate a couple hundred bugs tracked in another (home-rolled) system into GitHub's issue system. Most of these bugs were closed in the past. I can use github's API to create an issue, e.g.
curl -u $GITHUB_TOKEN:x-oauth-basic https://api.github.com/repos/my_organization/my_repo/issues -d '{
"title": "test",
"body": "the body"
}'
... however, this will leave me with a bunch of open issues. How to close those? I've tried just closing at the time of creation, e.g.:
curl -u $GITHUB_TOKEN:x-oauth-basic https://api.github.com/repos/my_organization/my_repo/issues -d '{
"title": "test",
"body": "the body",
"state": "closed"
}'
... but the result is to create an open issue (i.e. the "state" is ignored).
It looks to me like I should be able to "edit" an issue to close it (https://developer.github.com/v3/issues/#edit-an-issue) ... but I'm unable to figure out what the corresponding curl command is supposed to look like. Any guidance?
Extra credit: I'd really like to be able to assign a "closed" date, to agree with the actual closed date captured in our current system. It's not clear that this is possible.
Thanks!
migrating a bunch of issues to github with the command line? are you crazy?
anyway, using php and hhb_curl from https://github.com/divinity76/hhb_.inc.php/blob/master/hhb_.inc.php ,
this worked for me, unfortunately couldn't set the "closed_at" date (it was ignored by the api), but i could emulate it using labels, then it looked like
, the code should give you something to work on when porting it to command line:
<?php
declare(strict_types = 1);
require_once ('hhb_.inc.php');
$hc=new hhb_curl();
define('BASE_URL','https://api.github.com');
$hc->_setComfortableOptions();
$data=array(
'state'=>'closed',
'closed_at'=> '2011-04-22T13:33:48Z',// << unfortunately, ignored
'labels'=>array(
'closed at 2011-04-22T13:33:48Z' // << we can fake it using labels...
)
);
$data=json_encode($data);
$hc->setopt_array(array(
CURLOPT_CUSTOMREQUEST=>'PATCH',
// /repos/:owner/:repo/issues/:number
// https://github.com/divinity76/GitHubCrashTest/issues/1
CURLOPT_URL=>BASE_URL.'/repos/divinity76/GitHubCrashTest/issues/1',
CURLOPT_USERAGENT=>'test',
CURLOPT_HTTPHEADER=>array(
'Accept: application/vnd.github.v3+json',
'Content-Type: application/json',
'Authorization: token <removed>'
),
CURLOPT_POSTFIELDS=>$data,
));
$hc->exec();
hhb_var_dump($hc->getStdErr(),$hc->getResponseBody());
(i modified the "Authorization: token" line before posting it on stackoverflow ofc)
As suggested by hanshenrik, the correct altered curl command is:
curl -u $GITHUB_TOKEN:x-oauth-basic https://api.github.com/repos/my_organization/my_repo/issues/5 -d '{
"state": "closed"
}'
I'd failed to understand the documentation referenced in his answer:
/repos/:owner/:repo/issues/:number
translates to
https://api.github.com/repos/my_organization/my_repo/issues/5
(I now understand that fields starting with ":" are variables)
For the record, I'm planning to script the calls to curl. :)
I am trying to replicate the curl method mentioned in splunk rest api doc into R to perform search using R. Sorry, I won't be able provides details on the parameters to replicate. Hence attaching the link for reference.
curl -u admin:changeme -k https://localhost:8089/services/search/jobs -d search="search *"
This returns me a sid from curl. However when I try to replicate the same in R using httr it returns list of all search details. I have tried using both POST & GET in httr just in case. Below is sample code. Ideally below one should return me a sid. however it returns list of existing search details. Not sure what I am missing. I am new to Rcurl,httr. I tried curlperform as well, there also same. Seems, something is missing out. What exactly -d does in curl, is this the thing I am missing to replicate ?
response <- GET(splunk_server,path=search_job_export_endpoint,
config(ssl_verifyhost=FALSE, ssl_verifypeer=0),
authenticate(username, password),
query=list(search=urlencode(search_terms)),
verbose())
result <- read.table(text=content(response, as="text"), sep=",", header=TRUE,
stringsAsFactors=FALSE)
In the repository home page , i can see comments posted in recent activity at the bottom, bit it only shows 10 commnets.
i want to all the comments posted since beginning.
Is there any way
Comments of pull requests, issues and commits can be retrieved using bitbucket’s REST API.
However it seems that there is no way to list all of them at one place, so the only way to get them would be to query the API for each PR, issue or commit of the repository.
Note that this takes a long time, since bitbucket has seemingly set a limit to the number of accesses via API to repository data: I got Rate limit for this resource has been exceeded errors after retrieving around a thousand results, then I could retrieve about only one entry per second elapsed from the time of the last rate limit error.
Finding the API URL to the repository
The first step is to find the URL to the repo. For private repositories, it is necessary to get authenticated by providing username and password (using curl’s -u switch). The URL is of the form:
https://api.bitbucket.org/2.0/repositories/{repoOwnerName}/{repoName}
Running git remote -v from the local git repository should provide the missing values. Check the forged URL (below referred to as $url) by verifying that repository information is correctly retrieved as JSON data from it: curl -u username $url.
Fetching comments of commits
Comments of a commit can be accessed at $url/commit/{commitHash}/comments.
The resulting JSON data can be processed by a script. Beware that the results are paginated.
Below I simply extract the number of comments per commit. It is indicated by the value of the member size of the retrieved JSON object; I also request a partial response by adding the GET parameter fields=size.
My script getNComments.sh:
#!/bin/sh
pw=$1
id=$2
json=$(curl -s -u username:"$pw" \
https://api.bitbucket.org/2.0/repositories/{repoOwnerName}/{repoName}/commit/$id/comments'?fields=size')
printf '%s' "$json" | grep -q '"type": "error"' \
&& printf "ERROR $id\n" && exit 0
nComments=$(printf '%s' "$json" | grep -o '"size": [0-9]*' | cut -d' ' -f2)
: ${nComments:=EMPTY}
checkNumeric=$(printf '%s' "$nComments" | tr -dc 0-9)
[ "$nComments" != "$checkNumeric" ] \
&& printf >&2 "!ERROR! $id:\n%s\n" "$json" && exit 1
printf "$nComments $id\n"
To use it, taking into account the possibility for the error mentioned above:
A) Prepare input data. From the local repository, generate the list of commits as wanted (run git fetch -a prior to update the local git repo if needed); check out git help rev-list for how it can be customised.
git rev-list --all | sort > sorted-all.id
cp sorted-all.id remaining.id
B) Run the script. Note that the password is passed here as a parameter – so first assign it to a variable safely using stty -echo; IFS= read -r passwd; stty echo, in one line; also see security considerations below. The processing is parallelised onto 15 processes here, using the option -P.
< remaining.id xargs -P 15 -L 1 ./getNComments.sh "$passwd" > commits.temp
C) When the rate limit is reached, that is when getNComments.sh prints !ERROR!, then kill the above command (Ctrl-C), and execute these below to update the input and output files. Wait a while for the request limit to increase, then re-execute the above one command and repeat until all the data is processed (that is when wc -l remaining.id returns 0).
cat commits.temp >> commits.result
cut -d' ' -f2 commits.result | sort | comm -13 - sorted-all.id > remaining.id
D) Finally, you can get the commits which received comments with:
grep '^[1-9]' commits.result
Fetching comments of pull requests and issues
The procedure is the same as for fetching commits’ comments, but for the following two adjustments:
Edit the script to replace in the URL commit by pullrequests or by issues, as appropriate;
Let $n be the number of issues/PRs to search. The git rev-list command above becomes: seq 1 $n > sorted-all.id
The total number of PRs in the repository can be obtained with:
curl -su username $url/pullrequests'?state=&fields=size'
and, if the issue tracker is set up, the number of issues with:
curl -su username $url/issues'?fields=size'
Hopefully, the repository has few enough PRs and issues so that all data can be fetched in one go.
Viewing comments
They can be viewed normally via the web interface on their commit/PR/issue page at:
https://bitbucket.org/{repoOwnerName}/{repoName}/commits/{commitHash}
https://bitbucket.org/{repoOwnerName}/{repoName}/pull-requests/{prId}
https://bitbucket.org/{repoOwnerName}/{repoName}/issues/{issueId}
For example, to open all PRs with comments in firefox:
awk '/^[1-9]/{print "https://bitbucket.org/{repoOwnerName}/{repoName}/pull-requests/"$2}' PRs.result | xargs firefox
Security considerations
Arguments passed on the command line are visible to all users of the system, via ps ax (or /proc/$PID/cmdline). Therefore the bitbucket password will be exposed, which could be a concern if the system is shared by multiple users.
There are three commands getting the password from the command line: xargs, the script, and curl.
It appears that curl tries to hide the password by overwriting its memory, but it is not guaranteed to work, and even if it does, it leaves it visible for a (very short) time after the process starts. On my system, the parameters to curl are not hidden.
A better option could be to pass the sensitive information through environment variables. They should be visible only to the current user and root via ps axe (or /proc/$PID/environ); although it seems that there are systems that let all users access this information (do a ls -l /proc/*/environ to check the environment files’ permissions).
In the script simply replace the lines pw=$1 id=$2 with id=$1, then pass pw="$passwd" before xargs in the command line invocation. It will make the environment variable pw visible to xargs and all of its descendent processes, that is the script and its children (curl, grep, cut, etc), which may or may not read the variable. curl does not read the password from the environment, but if its password hiding trick mentioned above works then it might be good enough.
There are ways to avoid passing the password to curl via the command line, notably via standard input using the option -K -. In the script, replace curl -s -u username:"$pw" with printf -- '-s\n-u "%s"\n' "$authinfo" | curl -K - and define the variable authinfo to contain the data in the format username:password. Note that this method needs printf to be a shell built-in to be safe (check with type printf), otherwise the password will show up in its process arguments. If it is not a built-in, try with print or echo instead.
A simple alternative to an environment variable that will not appear in ps output in any case is via a file. Create a file with read/write permissions restricted to the current user (chmod 600), and edit it so that it contains username:password as its first line. In the script, replace pw=$1 with IFS= read -r authinfo < "$1", and edit it to use curl’s -K option as in the paragraph above. In the command line invocation replace $passwd with the filename.
The file approach has the drawback that the password will be written to disk (note that files in /proc are not on the disk). If this too is undesirable, it is possible to pass a named pipe instead of a regular file:
mkfifo pipe
chmod 600 pipe
# make sure printf is a builtin, or use an equivalent instead
(while :; do printf -- '%s\n' "username:$passwd"; done) > pipe&
pid=$!
exec 3<pipe
Then invoke the script passing pipe instead of the file. Finally, to clean up do:
kill $pid
exec 3<&-
This will ensure the authentication info is passed directly from the shell to the script (through the kernel), is not written to disk and is not exposed to other users via ps.
You can go to Commits and see the top line for each commit, you will need to click on each one to see further information.
If I find a way to see all without drilling into each commit, I will update this answer.
Is there some way I can programmatically change the users account image on OSX?I know I can retrieve it, but can it be changed like apple does on the account page in the settings app?
You can use the Address Book framework. You need to use the me method to get the record for the current user, and then the setImageData: method to set the user's image:
#import <AddressBook/AddressBook.h>
#implementation YourClass
- (void)setUserImage:(NSImage*)anImage
{
ABPerson* me = [[ABAddressBook addressBook] me];
[me setImageData:[anImage TIFFRepresentation]];
}
#end
There's more detail here in the docs.
You can path out to a file and merge it with the current record using the /usr/bin/dsimport command, which could be run from a NSTask. Here is an example of how to do so with BASH as root, this could be done with passed credentials as well
export OsVersion=`sw_vers -productVersion | awk -F"." '{print $2;exit}'`
declare -x UserPicture="/path/to/$UserName.jpg"
# Add the LDAP picture to the user record if dsimport is avaiable 10.6+
if [ -f "$UserPicture" ] ; then
# On 10.6 and higher this works
if [ "$OsVersion" -ge "6" ] ; then
declare -x Mappings='0x0A 0x5C 0x3A 0x2C'
declare -x Attributes='dsRecTypeStandard:Users 2 dsAttrTypeStandard:RecordName externalbinary:dsAttrTypeStandard:JPEGPhoto'
declare -x PictureImport="/Library/Caches/$UserName.picture.dsimport"
printf "%s %s \n%s:%s" "$Mappings" "$Attributes" "$UserName" "$UserPicture" >"$PictureImport"
# Check to see if the username is correct and import picture
if id "$UserName" &>/dev/null ; then
# No credentials passed as we are running as root
dsimport -g "$PictureImport" /Local/Default M &&
echo "Successfully imported users picture."
fi
fi
fi
Gist of this code
I am almost certain that you can do this through the OpenDirectory interface. See this guide:
https://developer.apple.com/library/mac/#documentation/Networking/Conceptual/Open_Directory/Introduction/Introduction.html#//apple_ref/doc/uid/TP40000917
Basically you have to open a Open Directory Node (say the /Search node), and then find the ODRecord for your user, then use:
setValue:forAttribute:error
on the ODRecord to set the JPEGPhoto attribute.
If you query using dscl from the command line for this attribute you will see the attribute's value:
dscl /Search read /Users/luser JPEGPhoto
I believe the dscl tool uses the Open Directory framework (or the older/harder to use/deprecated directory service framework) to read and write attributes to user records. You can read and write any other attribute using this tool and the associated framework. I don't see any reason the JPEGPhoto would be any different.
The /Search node MIGHT be read only (since it is kind of a meta node). Not really sure. You might have to explicitly open the appropriate node (for example the /Local/Default node) before you can write to the record:
dscl /Local/Default read /Users/luser JPEGPhoto