Using latestSuccessfulBuild number in Jenkins groovy job configuration - api

I have a jenkins job and I want to use it's lastSuccessfulBuild number in my Build Flow groovy script.
I can get the last successful build number from Jenkins api at:
http://{JENKINS_DOMAIN}/job/{JOB_NAME}/lastSuccessfulBuild/buildNumber
I tried using groovy's RESTClient in my Build Flow groovy script but when importing the groovyx.net.http.RESTClient library I get syntax error.
Does any one know away of getting around this error or getting the api result in some other way?

maybe this will help you:
import hudson.model.Build;
println(build.getProject().getLastSuccessfulBuild())
for example we have simple build flow groovy script building only one item "JobA". If we want check and print its last successful build we can write such script:
import hudson.model.Build;
def buildA = build("jobA")
println(buildA.getProject().getLastSuccessfulBuild())

Possibly a little overkill, but you can use HttpClient, as all you need is a get request on the url.
Here's one I knocked up from some code I had lying around
Tested it on our own Jenkins instance which has basic auth over ssl.
import org.apache.http.HttpResponse
import org.apache.http.HttpVersion
import org.apache.http.client.HttpClient
import org.apache.http.client.methods.HttpGet
import org.apache.http.client.params.ClientPNames
import org.apache.http.conn.ClientConnectionManager
import org.apache.http.conn.scheme.PlainSocketFactory
import org.apache.http.conn.scheme.Scheme
import org.apache.http.conn.scheme.SchemeRegistry
import org.apache.http.conn.ssl.SSLSocketFactory
import org.apache.http.impl.client.DefaultHttpClient
import org.apache.http.impl.conn.PoolingClientConnectionManager
import org.apache.http.params.BasicHttpParams
import org.apache.http.params.HttpConnectionParams
import org.apache.http.params.HttpParams
import org.apache.http.params.HttpProtocolParams
class LastSuccessfulBuild {
def static main(args) {
println new LastSuccessfulBuild().connect("your.jenkins.com", "443", "/path/to/job/YourJob/lastSuccessfulBuild/buildNumber", "your.user:your-password")
}
def connect(host, port, path, auth) {
def url = new URL("https", host, Integer.parseInt(port), path)
HttpClient client = createClient()
HttpGet get = new HttpGet(url.toURI())
get.setHeader("Authorization", "Basic ${auth.getBytes().encodeBase64().toString()}")
HttpResponse response = client.execute(get)
def status = response.statusLine.statusCode
if (status != 200) {
throw new IOException("Failed to get page, status: $response.statusLine")
}
return response.entity.content.text
}
def createClient() {
HttpParams params = new BasicHttpParams()
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1)
HttpProtocolParams.setContentCharset(params, "UTF-8")
params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS, true)
SchemeRegistry registry = new SchemeRegistry()
registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80))
registry.register(new Scheme("https",SSLSocketFactory.getSocketFactory(),443))
ClientConnectionManager ccm = new PoolingClientConnectionManager(registry)
HttpConnectionParams.setConnectionTimeout(params, 8000)
HttpConnectionParams.setSoTimeout(params, 5400000)
HttpClient client = new DefaultHttpClient(ccm, params)
return client
}
}

Related

Cannot call an API with Form params using JAX-RS Invocation Builder, returns a 400 status code

Trying an example to hit a rest API , but seeing a 400 status code. Is this the correct way to call an API using form params?
import javax.json.JsonObject;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.core.Form;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.client.Invocation;
import javax.ws.rs.HttpMethod;
import javax.ws.rs.core.Response;
...
public JsonObject getUserProfile() throws Exception {
Form userform = new Form();
userform.param("grant_type", "password")
.param("client_id", "cexxx")
.param("username", "theuser")
.param("password", "password")
.param("scope", "user.read openid profile offline_access");
Client client = ClientBuilder.newClient();
String serverUrl = "https://login.microsoftonline.com/547xx/oauth2/v2.0/token";
WebTarget target = client.target(serverUrl);
final Invocation.Builder invocationBuilder = target.request();
invocationBuilder.header("Content-Type", "application/x-www-form-urlencoded");
final Response response = invocationBuilder.method(
HttpMethod.POST,
Entity.entity(userform, MediaType.APPLICATION_FORM_URLENCODED),
Response.class);
System.out.println("r.getStatus()=" + response.getStatus());
...
}
The same works on Postman:
Thanks Giorgi for the hint.
Actually, the code we have above to programmatically make an API call works. The issue is that we are seeing error from server side.
Using this, we are able to see the error message from server:
System.out.println(response.readEntity(String.class));
{"error":"invalid_grant","error_description":"AADSTS50034: The user account sx does not exist in the 547..b32 directory. To sign into this application, the account must be added to the directory.\r\nTrace ID: 4616...3c00\r\nCorrelation ID: 617...a01\r\nTimestamp: 2020-09-29 22:25:41Z","error_codes":[50034],"timestamp":"2020-09-29 22:25:41Z","trace_id":"461...c00","correlation_id":"617..a01","error_uri":"https://login.microsoftonline.com/error?code=50034"}

Having trouble with authentication when doing rest api to retrieve Bigquery job information

I am trying to do http request to get information for a job that was submitted by another script. This script has the job id and the project. I read on Oauth20 and saw that using Application Default Credentials is the recommended way.
I have exported the default auth:
export GOOGLE_APPLICATION_CREDENTIALS= /test/proj-service-key-file.json
Here is my code:
from google.oauth2.service_account import Credentials
from google.cloud import bigquery
import requests
from oauth2client.client import GoogleCredentials
from google.auth.transport.requests import AuthorizedSession
import google.auth
credentials = GoogleCredentials.get_application_default()
session = google.auth.transport.requests.AuthorizedSession(credentials)
job_url="https://www.googleapis.com/bigquery/v2/projects/" + self.project + "/jobs/" + job_id + "?maxResults=1"
job_query_results = session.request('GET',job_url)
I am getting the following error:
self.credentials.before_request(
AttributeError: '_JWTAccessCredentials' object has no attribute 'before_request'
Any suggestions is appreciated.
Try deleting the space in your export statement:
export GOOGLE_APPLICATION_CREDENTIALS=/test/proj-service-key-file.json
I got it to work. I had to set up the scopes explicity and that did the trick:
scopes = ['https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.full_control',
'https://www.googleapis.com/auth/bigquery']
credentials = credentials.create_scoped(scopes)
http = httplib2.Http()
if credentials.access_token_expired:
credentials.refresh(http)
http = credentials.authorize(http)
response, content = http.request(job_url)

How to create a jenkins credentials via API?

Does anybody know how to create a new jenkins (2.8) credentials (f.e for a git access) via API or POST request in Jenkins? I have tried to use this code (from another stackoverflow topic), but it does nothing:
import json
import requests
def main():
data = {
'credentials': {
'scope': "GLOBAL",
'username': "jenkins",
'privateKeySource': {
'privateKey': "-----BEGIN RSA PRIVATE KEY-----\nX\n-----END RSA PRIVATE KEY-----",
'stapler-class': "com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource"
},
'stapler-class': "com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey"
}
}
payload = {
'json': json.dumps(data),
'Submit': "OK",
}
r = requests.post("http://%s:%d/credential-store/domain/_/createCredentials" % (localhost, 8080), data=payload)
if r.status_code != requests.codes.ok:
print r.text
I did it this way:
java -jar /tmp/jenkins-cli.jar -s http://localhost:8080/ \
groovy /tmp/credentials.groovy id username password
credentials.groovy
import jenkins.model.*
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*
domain = Domain.global()
store = Jenkins.instance.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()
usernameAndPassword = new UsernamePasswordCredentialsImpl(
CredentialsScope.GLOBAL,
args[0],
"",
args[1],
args[2]
)
store.addCredentials(domain, usernameAndPassword)
I ran into the same issue and after a bit of digging/testing it seems you need to change this
/credential-store/domain/_/createCredentials
to this
/credentials/store/system/domain/_/createCredentials
It's doesn't work: /credential-store/domain/_/api/json
You have to use this url: /credentials/store/system/domain/_/api/json

ScrapyDeprecationWaring: Command's default `crawler` is deprecated and will be removed. Use `create_crawler` method to instantiate crawlers

Scrapy version 0.19
I am using the code at this page ( Run multiple scrapy spiders at once using scrapyd ). When I run scrapy allcrawl, I got
ScrapyDeprecationWaring: Command's default `crawler` is deprecated and will be removed. Use `create_crawler` method to instantiate crawlers
Here is the code:
from scrapy.command import ScrapyCommand
import urllib
import urllib2
from scrapy import log
class AllCrawlCommand(ScrapyCommand):
requires_project = True
default_settings = {'LOG_ENABLED': False}
def short_desc(self):
return "Schedule a run for all available spiders"
def run(self, args, opts):
url = 'http://localhost:6800/schedule.json'
for s in self.crawler.spiders.list(): #this line raise the warning
values = {'project' : 'YOUR_PROJECT_NAME', 'spider' : s}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
log.msg(response)
How do I fix the DeprecationWarning ?
Thanks
Use:
crawler = self.crawler_process.create_crawler()

OAuth2Decorator: Using developer's token to run API calls for user

For the "normal" oauth2 dance, I get to specify the user and get a corresponding token.
This allows me to make API calls masquerading as that user, i.e. on his behalf.
It can also allow the user to make calls masquerading as me.
A use case is bigquery where I don't have to grant table access to the user and I can specify my own preferred level of control.
Using the simplified OAuth2Decorator, I don't seem to have this option.
Am I right to say that?
Or is there a work-around?
In general, what is the best practice? To use the proper oauth (comprising of Flow, Credentials and Storage)? Or to use OAuth2Decorator.
Thank you very much.
You can certainly use an OAuth2Decorator
Here is an example:
main.py
import bqclient
import httplib2
import os
from django.utils import simplejson as json
from google.appengine.api import memcache
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from oauth2client.appengine import oauth2decorator_from_clientsecrets
PROJECT_ID = "xxxxxxxxxxx"
DATASET = "your_dataset"
QUERY = "select columns from dataset.table"
CLIENT_SECRETS = os.path.join(os.path.dirname(__file__),'client_secrets.json')
http = httplib2.Http(memcache)
decorator = oauth2decorator_from_clientsecrets(CLIENT_SECRETS,
'https://www.googleapis.com/auth/bigquery')
bq = bqclient.BigQueryClient(http, decorator)
class MainHandler(webapp.RequestHandler):
#decorator.oauth_required
def get(self):
data = {'data': json.dumps(bq.Query(QUERY, PROJECT_ID))}
template = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(render(template, data))
application = webapp.WSGIApplication([('/', MainHandler),], debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
bqclient.py that gets imported in your main.py which handles BigQuery actions
from apiclient.discovery import build
class BigQueryClient(object):
def __init__(self, http, decorator):
"""Creates the BigQuery client connection"""
self.service = build('bigquery', 'v2', http=http)
self.decorator = decorator
def Query(self, query, project, timeout_ms=10):
query_config = {
'query': query,
'timeoutMs': timeout_ms
}
decorated = self.decorator.http()
queryReply = (self.service.jobs()
.query(projectId=project, body=query_config)
.execute(decorated))
jobReference=queryReply['jobReference']
while(not queryReply['jobComplete']):
queryReply = self.service.jobs().getQueryResults(
projectId=jobReference['projectId'],
jobId=jobReference['jobId'],
timeoutMs=timeout_ms).execute(decorated)
return queryReply
where all your authentication details are kept in a json file client_secrets.json
{
"web": {
"client_id": "xxxxxxxxxxxxxxx",
"client_secret": "xxxxxxxxxxxxxxx",
"redirect_uris": ["http://localhost:8080/oauth2callback"],
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token"
}
}
finally, don't forget to add these lines to your app.yaml:
- url: /oauth2callback
script: oauth2client/appengine.py
Hope that helps.
I am not sure I completely understand the use case, but if you are creating an application for others to use without their having to authorize access based on their own credentials, I would recommend using App Engine service accounts.
An example of this type of auth flow is described in the App Engine service accounts + Prediction API article.
Also, see this part and this part of the App Engine Datastore to BigQuery codelab, which also uses this authorization method.
The code might look something like this:
import httplib2
# Available in the google-api-python-client lib
from apiclient.discovery import build
from oauth2client.appengine import AppAssertionCredentials
# BigQuery Scope
SCOPE = 'https://www.googleapis.com/auth/bigquery'
# Instantiate and authorize a BigQuery API client
credentials = AppAssertionCredentials(scope=SCOPE)
http = credentials.authorize(httplib2.Http())
bigquery_service = build("bigquery", "v2", http=http)
# Make some calls to the API
jobs = bigquery_service.jobs()
result = jobs.insert(projectId='some_project_id',body='etc, etc')