Fetch All Pull-Request Comments Via Bitbucket REST API - api

This is how retrieve a particular pull-request's comments according to bitbucket's documentation:
While I do have the pull-request ID and format a correct URL I still get a 400 response error. I am able to make a POST request to comment but I cannot make a GET. After further reading I noticed the six parameters listed for this endpoint do not say 'optional'. It looks like these need to be supplied in order to retrieve all the comments.
But what exactly are these parameters? I don't find their descriptions to be helpful in the slightest. Any and all help would be greatly appreciated!

fromHash and toHash are only required if diffType is'nt set to EFFECTIVE. state also seems optional to me (didn't give me an error when not including it), and anchorState specifies which kind of comments to fetch - you'd probably want ALL there. As far as I understand it, path contains the path of the file to read comments from. (ex: src/a.py and src/b.py were changed -> specify which of them to fetch comments for)
However, that's probably not what you want. I'm assuming you want to fetch all comments.
You can do that via /rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/pull-requests/{pullRequestId}/activities which also includes other activities like reviews, so you'll have to do some filtering.
I won't paste example data from the documentation or the bitbucket instance I tested this once since the json response is quite long. As I've said, there is an example response on the linked page. I also think you'll figure out how to get to the data you want once downloaded since this is a Q&A forum and not a "program this for me" page :b
As a small quickstart: you can use curl like this
curl -u <your_username>:<your_password> https://<bitbucket-url>/rest/api/1.0/projects/<project-key>/repos/<repo-name>/pull-requests/<pr-id>/activities
which will print the response json.
Python version of that curl snippet using the requests module:
import requests
url = "<your-url>" # see above on how to assemble your url
r = requests.get(
url,
params={}, # you'll need this later
auth=requests.auth.HTTPBasicAuth("your-username", "your-password")
)
Note that the result is paginated according to the api documentation, so you'll have to do some extra work to build a full list: Either set an obnoxiously high limit (dirty) or keep making requests until you've fetched everything. I stronly recommend the latter.
You can control which data you get using the start and limit parameters which you can either append to the url directly (e.g. https://bla/asdasdasd/activity?start=25) or - more cleanly - add to the params dict like so:
requests.get(
url,
params={
"start": 25,
"limit": 123
}
)
Putting it all together:
def get_all_pr_activity(url):
start = 0
values = []
while True:
r = requests.get(url, params={
"limit": 10, # adjust this limit to you liking - 10 is probably too low
"start": start
}, auth=requests.auth.HTTPBasicAuth("your-username", "your-password"))
values.extend(r.json()["values"])
if r.json()["isLastPage"]:
return values
start = r.json()["nextPageStart"]
print([x["id"] for x in get_all_pr_activity("my-bitbucket-url")])
will print a list of activity ids, e.g. [77190, 77188, 77123, 77136] and so on. Of course, you should probably not hardcode your username and password there - it's just meant as an example, not production-ready code.
Finally, to filter by action inside the function, you can replace the return values with something like
return [activity for activity in values if activity["action"] == "COMMENTED"]

Related

Mockoon | Configure mockoon with a seed to always return the same random data between requests

Although I am using faker to generate random data I would like the responses of a given endpoint to be the same on every request.
I noticed the general settings have an input text for setting a seed but does nothing regarding what I am aiming at.
Faker.js seeding is not always generating the same information.
Instead, it will always generate the same sequence. I'm not sure it's documented on Faker.js' website, but I added this to Mockoon's documentation.
If you want the answer to always be the same you can either:
use the inline body with "static" data (without templating), something like:
{
"username": "john"
}
use the data buckets feature as they are only generated once when the server starts.

Why I only getting id's back in response? IGDB api

I already read the documentation and I think I am making the simplest request in the correct way, but it always returns only the IDs, instead of all the fields of the games
Documentation example: Documentation Example
The request header is fine. I know this because I can get the expected request if fields = * as querystring
this is my request:
You have to provide the fields you want inside the body.
Like this:
fields age_ratings,aggregated_rating,aggregated_rating_count,alternative_names,artworks,bundles,category,checksum,collection,cover,created_at,dlcs,expansions,external_games,first_release_date,follows,franchise,franchises,game_engines,game_modes,genres,hypes,involved_companies,keywords,multiplayer_modes,name,parent_game,platforms,player_perspectives,rating,rating_count,release_dates,screenshots,similar_games,slug,standalone_expansions,status,storyline,summary,tags,themes,total_rating,total_rating_count,updated_at,url,version_parent,version_title,videos,websites;"

How to pass some information between parse_item calls?

Ok, imagine a website with some list. The items of this list have one piece of information needed. The second piece is located at some other url, which is unique from item to item.
Currently our crawler opens a list page, scrapes each item, and for each item it opens that 2nd URL and gets the 2nd piece of the info from there. We use requests lib which is excellent in almost all cases but now it seems to be slow and ineffective. It looks that the whole Twisted is being blocked until one 'requests' request ends.
pseudo-code:
def parse_item():
for item in item_list:
content2 = requests.get(item['url'])
We can't just let Scrapy parse these 2nd urls because we need to 'connect' the first and the second url somehow. Something like Redis would work, but hey, is there any better (simpler, faster) way to do that in Scrapy? I can't believe the things must be so complicated.
You can do this my passing variable in meta
For example:
req = Request(url=http://somedonain.com/path, callback=myfunc)
req.meta['var1'] = 'some value'
yeld(req)
And in ur myfunc, you read passed variable as:
myval = response.request.meta['var1']

REST API: How to search for other attribute

I use node.js as REST API.
There are following actions available:
/contacts, GET, finds all contacts
/contacts, POST, creats new contact
/contacts/:id, GET, shows or gets specifiy contact by it's id
/contacts/:id, PUT, updates a specific contact
/contacts/:id, DELETE, removes a specific contact
What would now be a logic Route for searching, quering after a user?
Should I put this to the 3. route or should I create an extra route?
I'm sure you will get a lot of different opinions on this question. Personally I would see "searching" as filtering on "all contacts" giving:
GET /contacts?filter=your_filter_statement
You probably already have filtering-parameters on GET /contacts to allow pagination that works well with the filter-statement.
EDIT:
Use this for parsing your querystring:
var url = require('url');
and in your handler ('request' being your nodejs http-request object):
var parsedUrl = url.parse(request.url, true);
var filterStatement = parsedUrl.query.filter;
Interesting question. This is a discussion that I have had several times.
I don't think there is a clear answer, or maybe there is and I just don't know it or don't agree with it. I would say that you should add a new route: /contacts/_search performing an action on the contacts list, in this case a search. Clear and defined what you do then.
GET /contacts finds all contacts. You want a subset of all contacts. What delimiter in a URI represents subsets? It's not "?"; that's non-hierarchical. The "/" character is used to delimit hierarchical path segments. So for a subset of contacts, try a URI like /contacts/like/dave/ or /contacts/by_name/susan/.
This concept of subsetting data by path segments is for more than collections--it applies more broadly. Your whole site is a set, and each top-level path segment defines a subset of it: http://yoursite.example/contacts is a subset of http://yoursite.example/. It also applies more narrowly: /contacts/:id is a subset of /contacts, and /contacts/:id/firstname is a subset of /contacts/:id.

Preventing YQL from URL encoding a key

I am wondering if it is possible to prevent YQL from URL encoding a key for a datatable?
Example:
The current guardian API works with IDs like this:
item_id = "environment/2010/oct/29/biodiversity-talks-ministers-nagoya-strategy"
The problem with these IDs is that they contain slashes (/) and these characters should not be URL encoded in the API call but instead stay as they are.
So If I now have this query
SELECT * FROM guardian.content.item WHERE item_id='environment/2010/oct/29/biodiversity-talks-ministers-nagoya-strategy'
while using the following url defintion in my datatable
<url>http://content.guardianapis.com/{item_id}</url>
then this results in this API call
http://content.guardianapis.com/environment%2F2010%2Foct%2F29%2Fbiodiversity-talks-ministers-nagoya-strategy?format=xml&order-by=newest&show-fields=all
Instead the guardian API expects the call to look like this:
http://content.guardianapis.com/environment/2010/oct/29/biodiversity-talks-ministers-nagoya-strategy?format=xml&order-by=newest&show-fields=all
So the problem is really just that the / characters gets encoded as %2F which I don't want to happen in this case.
Any ideas on how this can be achieved?
You can also check the full datatable I am using:
http://github.com/spier/yql-tables/blob/master/guardian/guardian.content.item.xml
The URI-template expansions in YQL (e.g. {item_id}) only follow the version 3 spec. With version 4 it would be possible to simply (only slightly) change the expansion to do what you want, but alas not currently with YQL.
So, a solution. You could bring a very, very basic <execute> block into play: one which adds the item_id value to the path as needed.
<execute><![CDATA[
response.object = request.path(item_id).get().response;
]]></execute>
Finally, see the diff against your table (with a few other, minor tweaks to allow the above to work).