I want to use the Jenkins Remote API, and I am looking for safe solution. I came across Prevent Cross Site Request Forgery exploits and I want to use it, but I read somewhere that you have to make a crumb request.
How do I get a crumb request in order to get the API working?
I found this https://github.com/entagen/jenkins-build-per-branch/pull/20, but still I don't know how to fix it.
My Jenkins version is 1.50.x.
Authenticated remote API request responds with 403 when using POST request
I haven't found this in the documentation either. This code is tested against an older Jenkins (1.466), but should still work.
To issue the crumb use the crumbIssuer
// left out: you need to authenticate with user & password -> sample below
HttpGet httpGet = new HttpGet(jenkinsUrl + "crumbIssuer/api/json");
String crumbResponse = toString(httpclient, httpGet);
CrumbJson crumbJson = new Gson().fromJson(crumbResponse, CrumbJson.class);
This will get you a response like this
{"crumb":"fb171d526b9cc9e25afe80b356e12cb7","crumbRequestField":".crumb"}
This contains two pieces of information you need
the field name with which you need to pass the crumb
the crumb itself
If you now want to fetch something from Jenkins, add the crumb as header. In the sample below I fetch the latest build results.
HttpPost httpost = new HttpPost(jenkinsUrl + "rssLatest");
httpost.addHeader(crumbJson.crumbRequestField, crumbJson.crumb);
Here is the sample code as a whole. I am using gson 2.2.4 to parse the response and Apache's httpclient 4.2.3 for the rest.
import org.apache.http.auth.*;
import org.apache.http.client.*;
import org.apache.http.client.methods.*;
import org.apache.http.impl.client.*;
import com.google.gson.Gson;
public class JenkinsMonitor {
public static void main(String[] args) throws Exception {
String protocol = "http";
String host = "your-jenkins-host.com";
int port = 8080;
String usernName = "username";
String password = "passwort";
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(host, port),
new UsernamePasswordCredentials(usernName, password));
String jenkinsUrl = protocol + "://" + host + ":" + port + "/jenkins/";
try {
// get the crumb from Jenkins
// do this only once per HTTP session
// keep the crumb for every coming request
System.out.println("... issue crumb");
HttpGet httpGet = new HttpGet(jenkinsUrl + "crumbIssuer/api/json");
String crumbResponse= toString(httpclient, httpGet);
CrumbJson crumbJson = new Gson()
.fromJson(crumbResponse, CrumbJson.class);
// add the issued crumb to each request header
// the header field name is also contained in the json response
System.out.println("... issue rss of latest builds");
HttpPost httpost = new HttpPost(jenkinsUrl + "rssLatest");
httpost.addHeader(crumbJson.crumbRequestField, crumbJson.crumb);
toString(httpclient, httpost);
} finally {
httpclient.getConnectionManager().shutdown();
}
}
// helper construct to deserialize crumb json into
public static class CrumbJson {
public String crumb;
public String crumbRequestField;
}
private static String toString(DefaultHttpClient client,
HttpRequestBase request) throws Exception {
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = client.execute(request, responseHandler);
System.out.println(responseBody + "\n");
return responseBody;
}
}
Or you can use Python and requests instead
req = requests.get('http://JENKINS_URL/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)', auth=(username, password))
print(req.text)
will give you the name and the crumb:
Jenkins-Crumb:e2e41f670dc128f378b2a010b4fcb493
This Python function gets the crumb, and additionally uses the crumb to post to a Jenkins endpoint. This is tested with Jenkins 2.46.3 with CSRF protection turned on:
import urllib.parse
import requests
def build_jenkins_job(url, username, password):
"""Post to the specified Jenkins URL.
`username` is a valid user, and `password` is the user's password or
(preferably) hex API token.
"""
# Build the Jenkins crumb issuer URL
parsed_url = urllib.parse.urlparse(url)
crumb_issuer_url = urllib.parse.urlunparse((parsed_url.scheme,
parsed_url.netloc,
'crumbIssuer/api/json',
'', '', ''))
# Use the same session for all requests
session = requests.session()
# GET the Jenkins crumb
auth = requests.auth.HTTPBasicAuth(username, password)
r = session.get(crumb_issuer_url, auth=auth)
json = r.json()
crumb = {json['crumbRequestField']: json['crumb']}
# POST to the specified URL
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
headers.update(crumb)
r = session.post(url, headers=headers, auth=auth)
username = 'jenkins'
password = '3905697dd052ad99661d9e9f01d4c045'
url = 'http://jenkins.example.com/job/sample/build'
build_jenkins_job(url, username, password)
Meanwhile you can generate an API token in order to prevent having to include your password in the source code provided by the solutions above:
https://wiki.jenkins.io/display/JENKINS/Authenticating+scripted+clients
Refer - https://support.cloudbees.com/hc/en-us/articles/219257077-CSRF-Protection-Explained
If you authenticate with a username and a user API token then a crumb is not needed from Jenkins 2.96 weekly/2.107 LTS. For more information please refer to CSRF crumb no longer required when authenticating using API token or JENKINS-22474.
User cheffe's answer helped 90%. Thanks for giving us the right direction.
The missing 10% revolved around HTTP username and password authentication.
Since the Codenameone Java API I was using did not have the Authentication Class,
new UsernamePasswordCredentials(usernName, password));
I used:
String apiKey = "yourJenkinsUsername:yourJenkinsPassword";
httpConnection.addRequestHeader("Authorization", "Basic " + Base64.encode(apiKey.getBytes()));
User cheffe's Java snippet worked great for me on Jenkins v2.89.3 (Eclipse.org) and another Jenkins instance I use, at v2.60.3 (once enabled1).
I've added this to a Maven mojo2 I use for pushing locally-edited config.xml changes back to the server.
1 CSRF Protection
2 Hudson job sync plugin
In any of these answers I didn't find an option to use Jenkins API token.
I really tried all of these options but if you're enabling CSRF protection, you should access Jenkins APIs with Jenkins API token instead of normal password.
This token can be generated by each individual user in the user config page.
The token can be used as follows-
JenkinsApi::Client.new(server_url: jenkins_url, username: jenkins_user, password: jenkins_token)
P.S. - This initialization is for a Ruby Jenkins API client
Related
Can't believe I'm stuck with a LOGIN :( hate when this happens.
Can somebody enlight me how to connect TF.EXE by using PAT password or in the best case an OAuth token?
I might add that I already have a Pat token and an OAuth token, not a problem while trying to get those, but every time I try this example:
TF.exe workspaces /collection:xxxx.visualstudio.com/xxxx /loginType:OAuth /login:.,MyPatTokenOrMyOauthToken /noprompt
I get the following response:
TF30063: You are not authorized to access xxxx.visualstudio.com\xxxx.
So, I Know command it's ok, because if I don't specify a login, a modal window prompts for credentials, and I tested already with that approach and works fine.
For the end, I might change everything to change tf.exe for the TFS api, but I'm unable to find same methods in the api (see reference: https://learn.microsoft.com/es-es/rest/api/vsts/?view=vsts )
If API has same methods than TF.exe, that will be useful, but so far I don't see same methods in the API.
Hope somebody has the solution for my problem.
Thanks in advance.
From my test, PAT token doesn't work in the following command, you have to get a OAuth token:
tf workspaces /collection:https://xxxx.visualstudio.com /loginType:OAuth /login:.,[OAuth token]
For the api that authenticate with Visual Studio Team Services (VSTS), you could refer to the examples in this link:
Here is an example getting a list of projects for your account:
REST API
using System.Net.Http;
using System.Net.Http.Headers;
...
//encode your personal access token
string credentials = Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "", personalAccessToken)));
ListofProjectsResponse.Projects viewModel = null;
//use the httpclient
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://{accountname}.visualstudio.com"); //url of our account
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials);
//connect to the REST endpoint
HttpResponseMessage response = client.GetAsync("_apis/projects?stateFilter=All&api-version=1.0").Result;
//check to see if we have a succesfull respond
if (response.IsSuccessStatusCode)
{
//set the viewmodel from the content in the response
viewModel = response.Content.ReadAsAsync<ListofProjectsResponse.Projects>().Result;
//var value = response.Content.ReadAsStringAsync().Result;
}
}
.Net Client Libraries
using Microsoft.TeamFoundation.Core.WebApi;
using Microsoft.VisualStudio.Services.Common;
...
//create uri and VssBasicCredential variables
Uri uri = new Uri(url);
VssBasicCredential credentials = new VssBasicCredential("", personalAccessToken);
using (ProjectHttpClient projectHttpClient = new ProjectHttpClient(uri, credentials))
{
IEnumerable<TeamProjectReference> projects = projectHttpClient.GetProjects().Result;
}
Add a screenshot:
Update:
I've tested with a new account, and the result is as below. If I remove /loginType and /login parameters, a window will pop up to ask me logon.
The screenshot without /loginType and /login parameters:
The screenshot with /loginType and /login parameters:
Trying to build a service that will grab info on a JIRA ticket based on an ID passed to it.
I'm calling the API to take the ID passed to the service, tack it onto the URL for the API and get the JSON object.
Problem is, it appears one must be logged on or registered on JIRA in order to use the API.
So if I use the code below to make my request, I get a 404 error, as I do on any browser which I've not used to log onto Jira
public string Get(string id)
{
string html = string.Empty;
string url = #"https://company.atlassian.net/rest/api/latest/issue/" + id;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
html = reader.ReadToEnd();
}
return html;
}
I can add credentials to the request like so
request.Credentials = new NetworkCredential("vinnie#company.com","mypassword");
but I've no idea exactly what needs sending. I've tried the email address with which I'm set up in Jira but that doesn't work.
I have a suspicion that Jira adds a cookie to my browser which it uses to validate after the initial config - is that so? If so, what can I add/include on my web request to get it to run?
Am I just wildly off on the right way to access it? Or are there changes that can be made to the Jira side to allow requests?
You have to encode your credentials in Base64 format first and then these credentials can be put into your request as shown below:
string mergedCredentials = string.Format("{0}:{1}", m_Username, m_Password);
byte[] byteCredentials = UTF8Encoding.UTF8.GetBytes(mergedCredentials);
string base64Credentials = Convert.ToBase64String(byteCredentials);
request.Headers.Add("Authorization", "Basic " + base64Credentials);
Hope you're able to solve your problem by this approach!
I'm using the onelogin REST api to log a user in: https://developers.onelogin.com/api-docs/1/samples/login-user-via-api.
I have followed all the steps successfully to generate a session token with no issues.
The documentation then says to post the session token to this url: https://admin.us.onelogin.com/session_via_api_token
However, when do the post to that URL with the session token it simply re-directs me to the onelogin Sign On Page.
Here is the c# code for the post. I have a valid session token in variable: session_token:
string url = "https://admin.us.onelogin.com/session_via_api_token";
StringBuilder postData = new StringBuilder();
postData.Append("session_token=" + HttpUtility.UrlEncode(session_token) + "&");
postData.Append("auth_token=" + HttpUtility.UrlEncode(""));
//ETC for all Form Elements
// Now to Send Data.
StreamWriter writer = null;
request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postData.ToString().Length;
try
{
writer = new StreamWriter(request.GetRequestStream());
writer.Write(postData.ToString());
}
finally
{
if (writer != null)
writer.Close();
}
This appears to be server side code so this will never be able to successfully get a session with the end-user's browser.
In order for this flow to work properly, you need to redirect the end-user's browser to the https://admin.us.onelogin.com/session_via_api_token URL with just the auth_token value as a POST parameter.
All the above code will do is allow your back end server to get a session cookie, which doesn't help your end-user establish a session at all.
More details can be found here: https://developers.onelogin.com/api-docs/1/samples/login-user-via-api
I have a scenario and any suggestions in implementing that will be of great help. I have a servlet created on publish that will have POST requests coming from a lot of other third party applications. This servlet just stores the incoming posted data in JCR. I have successfully created this servlet but now the requirement is to make this servlet secured so that only applications hitting this servlet with particular username and password should be entertained.
What can I do to achieve this?
The way I would go for it:
Ask those 3rd party applications to send you the username and password so you can validate them on your servlet, then decide if you will allow or reject the request.
from the servlet calling (the 3rd party application)
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// ...
request.setAttribute("username", "a_valid_user");
request.setAttribute("password", "a_valid_password");
request.getRequestDispatcher("yourApp/YourServlet").forward(req, resp);
}
On your servlet:
String username = request.getParameter("username");
String password = request.getParameter("password");
if("a_valid_user".equals(username) && "a_valid_password".equals(password) {
// validate and go further
} else {
// do not process the request
}
The above example is valid just in case you can validate them on your side.
If this sample doesn't answer to your question, please provide more information about those 3rd party applications and the way you want to validate them.
You might consider using Google Client Library. I used it for authentication of users in an AEM publish instance. After the third party server is authenticated, you could use a separate AEM service account to handle POST processing.
Here' a SO post I made about integrating those libraries into AEM.
Google Client API in OSGI
With this you should be able set up authentication of the third party service account... as discussed here
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
I haven't actually done server to server auth in AEM, but it should be possible. But in a separate project (non AEM) I've used the Google Client Library for authenticating Service Accounts.
I recommend to use a two step process:
Step 1: Authentication and generate a token, you can use 3rd party service also to generate token.
Step 2: Call your servlet with this token, the servlet will validate token first and then use post data.
Thanks everyone for your replies. In the end I implemented the below code for authentication in cq :
final String authorization = request.getHeader("Authorization");
if (authorization != null && authorization.startsWith("Basic")) {
StringTokenizer st = new StringTokenizer(authorization);
if (st.hasMoreTokens()) {
String basic = st.nextToken();
if (basic.equalsIgnoreCase("Basic")) {
String decodedStr = Base64.decode(st.nextToken());
LOGGER.info("Credentials: " + decodedStr);
int p = decodedStr.indexOf(":");
if (p != -1) {
String login = decodedStr.substring(0, p).trim();
String password = decodedStr.substring(p + 1).trim();
Credentials credentials = new SimpleCredentials(login, password.toCharArray());
adminSession = repository.login(credentials);
if (null != adminSession) {
// means authenticated and do your stuff here
}
}
}
}
}
Also in the webservice code which is calling the servlet of publish, below is the code on how I am supplying the credentials in auth headers :
String authStr = usrname+":"+password;
// encode data on your side using BASE64
byte[] bytesEncoded = Base64.encodeBase64(authStr.getBytes());
String authEncoded = new String(bytesEncoded);
connection.setRequestProperty("Authorization", "Basic "+authEncoded);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write("jsondata={sample:jsoncontent}");
writer.close();
When using groovy's http-builder with basic authentication the default behavior is to send an unauthenticated request first and resend the request with credentials after receiving a 401 in the first place.
Apache's Httpclient offers preemptive authentication to send the credentials directly on the first request.
How can I use preemptive auth in Groovy's http-builder? Any code examples are appreciated.
You can also solve it groovy style with
http = new RESTClient('http://awesomeUrl/')
http.headers['Authorization'] = 'Basic '+"myUsername:myPassword".getBytes('iso-8859-1').encodeBase64()
Based on a JIRA issue you can do something like that :
def http = new RESTClient('http://awesomeUrl/')
http.client.addRequestInterceptor(new HttpRequestInterceptor() {
void process(HttpRequest httpRequest, HttpContext httpContext) {
httpRequest.addHeader('Authorization', 'Basic ' + 'myUsername:myPassword'.bytes.encodeBase64().toString())
}
})
def response = http.get(path: "aResource")
println response.data.text
Used following with Jenkins.
def accessToken = "ACCESS_TOKEN".bytes.encodeBase64().toString()
def req = new URL("https://raw.githubusercontent.com/xxxx/something/hosts").openConnection();
req.setRequestProperty("Authorization", "Basic " + accessToken)
def content = req.getInputStream().getText()