I have created a shell script to create and accept a merge request.
The first action (creating) is working fine, but the second action (accepting) fails.
This is my code:
curl -X PUT -H "PRIVATE-TOKEN: abc123" -d id=86 -d merge_request_id=323 https://gitlab/api/v3/projects/86/merge_requests/323/merge
API feedback:
{"message":"404 Not found"}
GitLab API documentation on merge requests:
https://gitlab.com/help/api/merge_requests.md#accept-mr
If you are using GitLab API v4
In this API version you should use iid, in the MR API calls, so the same id which you see in the web UI MR URL.
Source: https://docs.gitlab.com/ee/api/v3_to_v4.html :
API uses merge request IIDs (internal ID, as in the web UI) rather
than IDs. This affects the merge requests, award emoji, todos, and
time tracking APIs.
If you are using GitLab API v3
Use the globally unique internal id of the MR, NOT its local id.
So if you have a web UI URL of the MR you want to accept, like this:
https://gitlab.domain.com/group/project/merge_requests/11/commits
^^--- ..then *DON'T* use this id!
Instead if you create the MR with the API (or get a list of them in a project) and see its JSON:
{
"id": 16393, # <--- ...then USE THIS id...
"iid": 11,
"project_id": 1162,
"title": "MR title...",
(...)
}
...in your MR accept request:
curl -X PUT https://gitlab.domain.com/api/v3/projects/1162/merge_requests/16393/merge
# the globally unique internal id of the MR ---^^^^^
(In my opinion this is kind of misleading, because as of March 2017 the GitLab API docs call the global one iid (internal id)...)
Shortly after my previous post I have found out that the URL is incorrect.
The structure is not kept the same across the API.
The request that works is:
https://gitlab/api/v3/projects/86/merge_request/323/merge (notice REQUEST, not REQUESTS)
Related
What I want
I have a python backend application, using a service account, running in docker.
I have a cloud build trigger that is connected to a bitbucket repository. This trigger uses a webhook. For revision I use tags.
I want to trigger this webhook with my backend application. I want to provide a specific tag (using a placeholder variable).
I want the backend to give me a list of all available tags (like I get on the console.google.com frontend, see screenshot)
What I tried
I tried this API endpoint using a Bearer token (which works fine), but it doesn't provide me with a tag list: Source Repo API
curl https://sourcerepo.googleapis.com/v1/projects/<project>/repos/<repo>' --header "Authorization: Bearer $(gcloud auth print-access-token)" --header 'Accept: application/json'
Because it is possible to retrieve all tags in the cloud console, I used the developer tools to find the endpoint that provides me with all available tags:
https://console.cloud.google.com/m/source/repos/get?project=<project>&repo=<repo>
My issue here is that it takes cookies to authenticate, if I use the Bearer token it does not work.
Is it possible to authenticate my service account automatically against console.google.com to use this endpoint? Or is there another way to get a list of tags?
From what you have explained I understand that your concerns are:
1. If there is a way to get the list of tags from your repository that you are able to see in the GCP console using the endpoint that you have found.
The information that the console displays regarding tags do not come from any REST or gRPC API (the APIs provided by Google), but rather it comes directly from the git API. The console frontend runs a command similar to git tag in order to get the tags from your repository. The tags are not stored within the GCP system, the console only queries the git repo for the tags.
2. Can I authenticate with a service account on the console?
No. The APIs used by the web frontends (i.e. APIs starting with https://console.cloud.google.com) will only allow cookie authentication, which only user accounts can obtain. There is usually a way to translate a frontend API (https://console.cloud.google.com) to a GCP API (https://*.googleapis.com), where you can use regular authentication to retrieve the information. However,in this case, the tag information is not in a GCP API (but rather inside the git repo), so there is no translation available.
3. If there is another way to list the possible tags present in the repository?
I tried to reproduce your situation to find a way to be able to get the list of the tags present in one repository, in this case a Bitbucket repository, and I found that you will be able to get this data using the $ git tag command. In this documentation you will be able to find all the commands related to Repository tags.
Knowing this, after linking the Bitbucket Repository to my code, I was able to get the list of tags after using the $ git tag command.
I have created a job of JDBC to BigQuery using the web interface and it worked just fine.
Now I want to create the same job from the REST API of GCP so I took the rest equivalent of the request from the site and tried to send it from Postman.
I'm sending POST request for the following URL:
https://dataflow.googleapis.com/v1b3/projects/test-data-308414/templates:launch?gcsPath=gs://dataflow-templates/latest/Jdbc_to_BigQuery
which I got from the example in the GCP documentation.
I also pass the JSON that the GCP gave me in the body.
And the API key as get parameter in the next format "?key=[API_KEY]"
I'm getting 401 response from the server with the following message:
Request is missing required authentication credential. Expected OAuth
2 access token, login cookie or other valid authentication credential.
See
https://developers.google.com/identity/sign-in/web/devconsole-project.
With a status of:
UNAUTHENTICATED
I looked up at the link and found a tutorial on how to create google authentication on the front end
witch is not helpful to me.
I'm pretty sure that I'm passing the API key in the wrong format and that the reason it failed to authenticate.
But I couldn't find any documentation that says how to do it correctly.
PS> I have also tried passing it at the headers as I saw in one place
in the next format
Authorization : [API_KEY]
but it failed with the same message
Few days back I was trying to integrate GCP into MechCloud and struggling to figure out how to invoke a microservice ( which is acting as a proxy to GCP) with credentials for different projects which will be passed to this microservice on the fly. I was surprised that in spite of spending good amount of time I could not figure out how to achieve it because GCP documentation is focused on working with one project credentials at a time using application default credentials. Another frustrating thing is that API explorer shows both OAuth 2.0 and API Key by default for all the APIs when the fact is that API Key is hardly supported for any API. Finally I found the solution for this problem here.
Here are the steps to invoke a GCP rest api -
Create a service account for your project and download the json file associated with it.
Note down values of client_email, private_key_id and private_key attribues from service account json file.
Define following environment variables using above values -
GCP_SERVICE_ACCOUNT_CLIENT_EMAIL=<client_email>
GCP_SERVICE_ACCOUNT_PRIVATE_KEY_ID=<private_key_id>
GCP_SERVICE_ACCOUNT_PRIVATE_KEY=<private_key>
Execute following python code to generate jwt_token -
import time, jwt, os
iat = time.time()
exp = iat + 3600
client_email = os.getenv('GCP_SERVICE_ACCOUNT_CLIENT_EMAIL')
private_key_id = os.getenv('GCP_SERVICE_ACCOUNT_PRIVATE_KEY_ID')
private_key = os.getenv('GCP_SERVICE_ACCOUNT_PRIVATE_KEY')
payload = {
'iss': client_email,
'sub': client_email,
'aud': 'https://compute.googleapis.com/',
'iat': iat,
'exp': exp
}
private_key1 = private_key.replace('\\n', '\n')
# print(private_key1)
additional_headers = {'kid': private_key_id}
signed_jwt = jwt.encode(
payload,
private_key1,
headers=additional_headers,
algorithm='RS256'
)
print(signed_jwt)
Use generated jwt token from previous step and use it as a bearer token to invoke any GCP rest api. E.g.
curl -X GET --header 'Authorization: Bearer <jwt_token>' 'https://compute.googleapis.com/compute/v1/projects/{project}/global/networks'
The best practice to authenticate a request is to use your application credentials. Just make sure you installed the google cloud SDK.
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d #request.json \
https://dataflow.googleapis.com/v1b3/projects/PROJECT_ID/templates:launch?gcsPath=gs://dataflow-templates/latest/Jdbc_to_BigQuery
My server is a closed-network and i want to make a whitelist only for bigquery client and gsutil.
I have tried adding following url
https://cloud.google.com/bigquery
https://console.cloud.google.com/
But it turns out bigquery needs a little more url for authenticating purposes and many more. For example like the urls below
https://accounts.google.com/o/oauth2/auth
https://www.googleapis.com/oauth2/v1/certs
Does anyone know all the url that is used for bigquery-client and gsutil. Not only for authenticating but also for creating jobs and fetching data from storage
Per the official documentation you can see that the URL https://www.googleapis.com/auth/bigquery is the one used for BigQuery to authenticate:
#This is a Python example
appflow = flow.InstalledAppFlow.from_client_secrets_file(
"client_secrets.json", scopes=["https://www.googleapis.com/auth/bigquery"]
)
But, then you need to whitelist the URL for the API authorization of your project, for example:
curl -H "Authorization: Bearer $ACCESS_TOKEN" \
"https://www.googleapis.com/bigquery/v2/projects/$GOOGLE_CLOUD_PROJECT/datasets"
I recommend you to whitelist all of GCS/Google APIs or if you want to be more detailed deploy a firewall and see the traffic you are getting from GCS/BigQuery
UPDATED QUESTION to clarify more.
I have two servers : wso2greg containing custom artefacts and wso2am (api manager).
The gouvernance register is intended to be used internally (intranet) and some of it's feature are intended to be exposed externally and acceded via it's rest API.
In order to manage wso2greg API accesss, we used an wso2am (api manager) server, where we did define an API that act as a proxy for the wso2greg API.
In our experiments, we found the strange behaviour described below :
Successfully direct calling wso2greg artefact rest api (the exact url is
https://localhost:9443/resource/1.0.0/artifact/_system/governance/myartifact/art1 and i'm following the documentation of this REST API from here https://docs.wso2.com/display/Governance520/Resources+with+REST+API ) using postman. The API reacts as described in the documentation.
Error 500, reported on wso2greg server, when calling the same REST API, but via API manager (using publisher application).
So as a summary :
Directly calling wso2greg rest API woks fine and a got results back
Adding this working API, in wso2am and than calling it gives always error 500.
As extra information:
Tested with other wso2greg REST API , such as "rating" or "comments" and I was able to call it successfully either directly or throw Api manager.
Tried to get information related to default wso2greg artifacts such as restservice, but i got exactly the same pb when (error 500) interagting with API manager.
Working with the last version of both products : wso2greg version 5.2.0 and wso2am version 1.10.0
Please advice, or clarify how could i debug this internal error in wso2greg.
Regards,
I hope your not using G-Reg + APIM feature installed version. I presume your using separate G-reg 5.2.0 and APIM x.x.x. Since your using the second option you can't use G-Reg REST API with APIM, therefore you have to use APIM REST API instead.
Get an API
Description: Get details of a specific API.
URI: http://localhost:9763/publisher/site/blocks/listing/ajax/item-list.jag
URI Parameters: action=getAPI&name=xxx&version=xxx&provider=xxx
HTTP Methods: POST
Example:
curl -X POST -b cookies http://localhost:9763/publisher/site/blocks/listing/ajax/item-list.jag -d "action=getAPI&name=PhoneVerification&version=1.0.0&provider=admin"
Please follow this link to see all the api usage.
Thanks #thusharaK for providing help.
The solution was to add Message Mediation Policies which forces json header on output.
Now i got my correct response from API manager (in fact, i got a binary response in test console and a json response in postman) .
I'm trying to fork a repo using the GitHub V3 API via REST, however, I am having issues making a POST request as per the docs (https://developer.github.com/v3/repos/forks/#create-a-fork).
Basically, what I have so far:
A logged-in user with an OAuth Token
A POST request setup to the API (URL: https://api.github.com/repos/carmichaelalonso/infiniteflight/forks/) - I am testing this with hurl.it to begin with.
Headers in the request: one has the name Authorization with the value 'token ...', the other specifies Content-Type with the value application/json
A body with the following JSON: {"organization" : "shortlisthome"} (shortlisthome is the account I am trying to fork the repo to.
I am not intending to fork this to an organization, instead of a standard user account, which is where I am getting confused. When I run the request, I do not get any authentication errors or 404 errors (I previously had but I had been entering incorrect values by mistake, causing such errors).
When I run this request though, I get the following result (a 422 unprocessable request):
{
"message": "Validation Failed",
"documentation_url": "---url-to-docs---",
"errors": [
{
"resource": "Fork",
"code": "invalid",
"field": "organization"
}
]
}
I am unsure whether or not I am able to fork this to a standard user, or if it is an error with my request. Please let me know if I can provide any more info (first post here so a bit unfamiliar with the convention).
Thanks!
In order for shortlisthome to fork the repository you need to authenticate as them. The repository you're trying to fork is public, so all you need to do is obtain a OAuth token for shortlisthome and then make a similar request to the one you're making now. The only difference will be that you do not need to provide the JSON body of {"organization": "shortlisthome"}.
For what it is worth, the optional JSON body is intended for you to use when you are a member of an organization with proper permissions and you want to fork the repository to that organization. You cannot fork a repository to someone else's account unless you are authenticated as them.
I've so far avoided OAuth2, and do not know hurl. Yet, perhaps this can help.
This post shows how it can be done using cURL's -u username flag:
curl -u 'myusername' https://api.github.com/repos/carmichaelalonso/infiniteflight/forks/ -d ''
The -d (or alternatively --data) flag turns it into a POST request.
Without that flag cURL defaults to a GET request, which is not what you want.
Since, the data is part of the URI for this request, send an empty string for the data that must follow the -d param, as such: -d ''.
Of course, using -u will require you to also supply a password..
Here is what GitHub shows using OAuth2 with their API:
OAuth2 Token (sent in a header):
curl -H "Authorization: token OAUTH-TOKEN" https://api.github.com
OAuth2 Token (sent as a parameter):
curl https://api.github.com/?access_token=OAUTH-TOKEN
Read more about OAuth2. Note that OAuth2 tokens can be acquired programmatically, for applications that are not websites.
OAuth2 Key/Secret
curl 'https://api.github.com/users/whatever?client_id=xxxx&client_secret=yyyy'
I suspect that adding -d '' (or some equivalent in hurl),
plus one of the formats above for sending OAuth2 info might get you most of the way there.
I used this command to fork on github enterprise
curl -vX POST https://git.redacted.com/api/v3/repos/<org-to-fork-from>/<repo-to-fork>/forks?access_token=<api-token> -d #gh-fork.json --header "Content-Type: application/json"
gh-fork.json is just
{
"organization": "org-to-fork-to",
"description": "",
"homepage": "https://git.redacted.com",
"private": false
}