Jenkins - Email notifications - notifications

I'm trying to customize email using Groovy with the email-ext plugin. As I add new features to these emails, I may introduce errors in the scripts and so receive bad mails containing the StackTrace. So, I'd like to be able to send notifications on finished jobs as my jobs may take many hours (more than 4 currently).
Is there a way to ask jenkins to send notifications on finished jobs (using Groovy or any other scripting language)?

find a solution :
import java.text.SimpleDateFormat;
import java.util.GregorianCalendar;
import groovy.text.Template
import groovy.text.SimpleTemplateEngine
import javax.mail.*
import javax.mail.internet.*
//-------------- Job params --------------
projectName="YourProjectName";
buildNum = 10;
templateName="groovy-html-cobertura.template";
recipients="someone#somewhere.com";
sender="jenkins#somewhere.com";
smtpHost="mysmtphost";
//------------End Job params -------------
for (hudson.model.AbstractProject p : hudson.model.Hudson.instance.projects) {
if(p.name.equals(projectName)){
for (hudson.model.AbstractBuild b : p.getBuilds() ) {
if(b.getNumber() == buildNum){
println b;
b.reload();
def binding = [ "build" : b,
"project" : b.getProject(),
"rooturl" : hudson.model.Hudson.getInstance().getRootUrl(),
"it" : new hudson.plugins.emailext.plugins.content.ScriptContentBuildWrapper(b),
"spc" : " " ]
def engine = new SimpleTemplateEngine()
java.io.File file = new java.io.File(hudson.model.Hudson.getInstance ().getRootPath().getBaseName()+"/email-templates/"+templateName);
mailBody = engine.createTemplate(file.getText()).make(binding).toString();
port = 25
props = new Properties()
props.put('mail.smtp.host', smtpHost)
props.put('mail.smtp.port', port.toString())
session = Session.getDefaultInstance(props, null)
// Construct the message
msg = new MimeMessage(session)
msg.from = new InternetAddress(sender)
msg.sentDate = new Date()
msg.subject = 'Template Test'
msg.setRecipient(Message.RecipientType.TO, new InternetAddress(recipients))
msg.setHeader('Organization', 'i-BP')
msg.setContent(mailBody,
'text/html')
// Send the message
Transport.send(msg)
}
}
}
}

Related

How can I pass a value from a TeamCity failure Condition to an e-mail notification?

I want to show the status of the build in an e-mail with just a simple FAIL or PASS text appear in the body. There does not seem to be any kind of predefined "buildStatus" variable that I can access or setup in TeamCity. I guess I need to access the "failureConditions" function at bottom but not sure how, tried lots of things but nothing worked, this is my script:
package _Self.buildTypes
import jetbrains.buildServer.configs.kotlin.v2019_2.failureConditions.BuildFailureOnText
import jetbrains.buildServer.configs.kotlin.v2019_2.failureConditions.failOnText
import jetbrains.buildServer.configs.kotlin.v2019_2.triggers.schedule
allowExternalStatus = true
params {
param("MinorVersion", "0")
param("RevisionVersion", "0")
}
powerShell {
name = "Email"
scriptMode = script {
content = """
function Send-ToEmail([string]${'$'}email, [string]${'$'}attachmentpath){
${'$'}message = new-object Net.Mail.MailMessage;
${'$'}message.From = "teamcity#blog.com";
${'$'}message.To.Add(${'$'}email);
${'$'}message.Subject = "%env.TEAMCITY_PROJECT_NAME% | %VersionNumber% ";
${'$'}message.Body = "The build: PASS or FAIL text here";
${'$'}smtp = new-object Net.Mail.SmtpClient("blog.local", "25");
${'$'}smtp.EnableSSL = ${'$'}true;
${'$'}smtp.send(${'$'}message);
write-host "Mail Sent" ;
}
Send-ToEmail -email "me#blog.com" -attachmentpath ${'$'}path;
""".trimIndent()
}
}
}
failureConditions {
failOnText {
conditionType = BuildFailureOnText.ConditionType.CONTAINS
pattern = "FAIL"
reverse = false
}
}
To solve this in T/C:
Create new custom env. var. e.g. MyBuildStatus
Create new build step e.g. Set build status (only executes if build successful)
Create a new Parameter name='env.MyBuildStatus.Status' value='SUCCESS'
Add variable to email subject: $message.Subject = "%env.MyBuildStatus.Status%
Why T/C has no build in "buildstatus" env. variable is interesting.

How to list all assets (groupId, name, size) in a Nexus3 repository?

I know it is very simple to do in OrientDB :
select * from asset where bucket.repository_name = 'my-repo-release';
but I need to get this list remotely, not in local orientdb console, so I need a groovy script and I can't find it anywhere.
Here is the example script :
https://github.com/sonatype-nexus-community/nexus-scripting-examples
it can be found in the "nexus-script-example" project :
import org.sonatype.nexus.repository.storage.Asset
import org.sonatype.nexus.repository.storage.Query
import org.sonatype.nexus.repository.storage.StorageFacet
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
def assetListFile = new File('/tmp/assetListFile.txt')
def request = new JsonSlurper().parseText("{\"repoName\":\"maven-releases\",\"startDate\":\"2016-01-01\"}")
assert request.repoName: 'repoName parameter is required'
assert request.startDate: 'startDate parameter is required, format: yyyy-mm-dd'
log.info("Gathering Asset list for repository: ${request.repoName} as of startDate: ${request.startDate}")
def repo = repository.repositoryManager.get(request.repoName)
StorageFacet storageFacet = repo.facet(StorageFacet)
def tx = storageFacet.txSupplier().get()
try {
tx.begin()
Iterable<Asset> assets = tx.findAssets(Query.builder().where('last_updated > ').param(request.startDate).build(), [repo])
assets.each {Asset asset ->
assetListFile << asset.name() + '\n'
}
}
finally {
tx.close()
}
All properties of an asset (object org.sonatype.nexus.repository.storage.Asset) are accessible, including asset.size().

Google Cloud Pubsub Data lost

I'm experiencing a problem with GCP pubsub where a small percentage of data was lost when publishing thousands of messages in couple seconds.
I'm logging both message_id from pubsub and a session_id unique to each message on both the publishing end as well as the receiving end, and the result I'm seeing is that some message on the receiving end has same session_id, but different message_id. Also, some messages were missing.
For example, in one test I send 5,000 messages to pubsub, and exactly 5,000 messages were received, with 8 messages lost. The log lost messages look like this:
MISSING sessionId:sessionId: 731 (missing in log from pull request, but present in log from Flask API)
messageId FOUND: messageId:108562396466545
API: 200 **** sessionId: 731, messageId:108562396466545 ******(Log from Flask API)
Pubsub: sessionId: 730, messageId:108562396466545(Log from pull request)
And the duplicates looks like:
======= Duplicates FOUND on sessionId: 730=======
sessionId: 730, messageId:108562396466545
sessionId: 730, messageId:108561339282318
(both are logs from pull request)
All missing data and duplicates look like this.
From the above example, it is clear that some messages has taken the message_id of another message, and has been sent twice with two different message_ids.
I wonder if anyone would help me figure out what is going on? Thanks in advance.
Code
I have an API sending message to pubsub, which looks like this:
from flask import Flask, request, jsonify, render_template
from flask_cors import CORS, cross_origin
import simplejson as json
from google.cloud import pubsub
from functools import wraps
import re
import json
app = Flask(__name__)
ps = pubsub.Client()
...
#app.route('/publish', methods=['POST'])
#cross_origin()
#json_validator
def publish_test_topic():
pubsub_topic = 'test_topic'
data = request.data
topic = ps.topic(pubsub_topic)
event = json.loads(data)
messageId = topic.publish(data)
return '200 **** sessionId: ' + str(event["sessionId"]) + ", messageId:" + messageId + " ******"
And this is the code I used to read from pubsub:
from google.cloud import pubsub
import re
import json
ps = pubsub.Client()
topic = ps.topic('test-xiu')
sub = topic.subscription('TEST-xiu')
max_messages = 1
stop = False
messages = []
class Message(object):
"""docstring for Message."""
def __init__(self, sessionId, messageId):
super(Message, self).__init__()
self.seesionId = sessionId
self.messageId = messageId
def pull_all():
while stop == False:
m = sub.pull(max_messages = max_messages, return_immediately = False)
for data in m:
ack_id = data[0]
message = data[1]
messageId = message.message_id
data = message.data
event = json.loads(data)
sessionId = str(event["sessionId"])
messages.append(Message(sessionId = sessionId, messageId = messageId))
print '200 **** sessionId: ' + sessionId + ", messageId:" + messageId + " ******"
sub.acknowledge(ack_ids = [ack_id])
pull_all()
For generating session_id, sending request & logging response from API:
// generate trackable sessionId
var sessionId = 0
var increment_session_id = function () {
sessionId++;
return sessionId;
}
var generate_data = function () {
var data = {};
// data.sessionId = faker.random.uuid();
data.sessionId = increment_session_id();
data.user = get_rand(userList);
data.device = get_rand(deviceList);
data.visitTime = new Date;
data.location = get_rand(locationList);
data.content = get_rand(contentList);
return data;
}
var sendData = function (url, payload) {
var request = $.ajax({
url: url,
contentType: 'application/json',
method: 'POST',
data: JSON.stringify(payload),
error: function (xhr, status, errorThrown) {
console.log(xhr, status, errorThrown);
$('.result').prepend("<pre id='json'>" + JSON.stringify(xhr, null, 2) + "</pre>")
$('.result').prepend("<div>errorThrown: " + errorThrown + "</div>")
$('.result').prepend("<div>======FAIL=======</div><div>status: " + status + "</div>")
}
}).done(function (xhr) {
console.log(xhr);
$('.result').prepend("<div>======SUCCESS=======</div><pre id='json'>" + JSON.stringify(payload, null, 2) + "</pre>")
})
}
$(submit_button).click(function () {
var request_num = get_request_num();
var request_url = get_url();
for (var i = 0; i < request_num; i++) {
var data = generate_data();
var loadData = changeVerb(data, 'load');
sendData(request_url, loadData);
}
})
UPDATE
I made a change on the API, and the issue seems to go away. The changes I made was instead of using one pubsub.Client() for all request, I initialized a client for every single request coming in. The new API looks like:
from flask import Flask, request, jsonify, render_template
from flask_cors import CORS, cross_origin
import simplejson as json
from google.cloud import pubsub
from functools import wraps
import re
import json
app = Flask(__name__)
...
#app.route('/publish', methods=['POST'])
#cross_origin()
#json_validator
def publish_test_topic():
ps = pubsub.Client()
pubsub_topic = 'test_topic'
data = request.data
topic = ps.topic(pubsub_topic)
event = json.loads(data)
messageId = topic.publish(data)
return '200 **** sessionId: ' + str(event["sessionId"]) + ", messageId:" + messageId + " ******"
Talked with some guy from Google, and it seems to be an issue with the Python Client:
The consensus on our side is that there is a thread-safety problem in the current python client. The client library is being rewritten almost from scratch as we speak, so I don't want to pursue any fixes in the current version. We expect the new version to become available by end of June.
Running the current code with thread_safe: false in app.yaml or better yet just instantiating the client in every call should is the work around -- the solution you found.
For detailed solution, please see the Update in the question
Google Cloud Pub/Sub message IDs are unique. It should not be possible for "some messages [to] taken the message_id of another message." The fact that message ID 108562396466545 was seemingly received means that Pub/Sub did deliver the message to the subscriber and was not lost.
I recommend you check how your session_ids are generated to ensure that they are indeed unique and that there is exactly one per message. Searching for the sessionId in your JSON via a regular expression search seems a little strange. You would be better off parsing this JSON into an actual object and accessing fields that way.
In general, duplicate messages in Cloud Pub/Sub are always possible; the system guarantees at-least-once delivery. Those messages can be delivered with the same message ID if the duplication happens on the subscribe side (e.g., the ack is not processed in time) or with a different message ID (e.g., if the publish of the message is retried after an error like a deadline exceeded).
You shouldn't need to create a new client for every publish operation. I'm betting that the reason that that "fixed the problem" is because it mitigated a race that exists in the publisher client side. I'm also not convinced that the log line you've shown on the publisher side:
API: 200 **** sessionId: 731, messageId:108562396466545 ******
corresponds to a successful publish of sessionId 731 by publish_test_topic(). Under what conditions is that log line printed? The code that has been presented so far does not show this.

Using spark-kernel comm API

I started using spark-kernel recently.
As given in tutorial and sample code, I was able to set up client and use it for executing code snippets on spark-kernel and retrieving back results as given in this example code.
Now, I need to use comm API provided with spark-kernel. I tried this tutorial, but I am not able to make it work. In fact, I have no understanding of how to make that work.
I tried the following code, but when I run this code, I get this error "Received invalid target for Comm Open: my_target" on the kernel.
package examples
import scala.runtime.ScalaRunTime._
import scala.collection.mutable.ListBuffer
import com.ibm.spark.kernel.protocol.v5.MIMEType
import com.ibm.spark.kernel.protocol.v5.client.boot.ClientBootstrap
import com.ibm.spark.kernel.protocol.v5.client.boot.layers.{StandardHandlerInitialization, StandardSystemInitialization}
import com.ibm.spark.kernel.protocol.v5.content._
import com.typesafe.config.{Config, ConfigFactory}
import Array._
object commclient extends App{
val profileJSON: String = """
{
"stdin_port" : 48691,
"control_port" : 44808,
"hb_port" : 49691,
"shell_port" : 40544,
"iopub_port" : 43462,
"ip" : "127.0.0.1",
"transport" : "tcp",
"signature_scheme" : "hmac-sha256",
"key" : ""
}
""".stripMargin
val config: Config = ConfigFactory.parseString(profileJSON)
val client = (new ClientBootstrap(config)
with StandardSystemInitialization
with StandardHandlerInitialization).createClient()
def printResult(result: ExecuteResult) = {
println(s"${result.data.get(MIMEType.PlainText).get}")
}
def printStreamContent(content:StreamContent) = {
println(s"${content.text}")
}
def printError(reply:ExecuteReplyError) = {
println(s"Error was: ${reply.ename.get}")
}
client.comm.register("my_target").addMsgHandler {
(commWriter, commId, data) =>
println(data)
commWriter.close()
}
// Initiate the Comm connection
client.comm.open("my_target")
}
Can someone tell me how shall I run this piece of code:
// Register the callback to respond to being opened from the client
kernel.comm.register("my target").addOpenHandler {
(commWriter, commId, targetName, data) =>
commWriter.writeMsg(Map("response" -> "Hello World!"))
}
I would really appreciate if someone can point me to complete working example on usage of comm API.
Any help will be appreciated. Thanks
You can use your client to run this server (kernel) side registration once in one program. Then your other programs can communicate to kernel using this channel.
Here is a way I ran my registration in the first program I mentioned above:
client.execute(
"""
// Register the callback to respond to being opened from the client
kernel.comm.register("my target").
addOpenHandler {
(commWriter, commId, targetName, data) =>
commWriter.writeMsg(org.apache.toree.kernel.protocol.v5.MsgData("response" -> "Toree Hello World!"))
}.
addMsgHandler {
(commWriter, _, data) =>
if (!data.toString.contains("closing")) {
commWriter.writeMsg(data)
} else {
commWriter.writeMsg(org.apache.toree.kernel.protocol.v5.MsgData("closing" -> "done"))
}
}
""".stripMargin
)

Error occurred creating story: "Cannot parse object reference from", Rally

I am trying to create user stories in rally by java. Am failing to get response and it throwing " Cannot parse object reference from ".
Brief on my rally : Let assume I have Project name "Characters" and its has subs as A, B, C, D. I have to create my user stories in C. Is my following code correct in this prospect? Can anyone please help?
package samplerally;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import com.google.gson.JsonObject;
import com.rallydev.rest.RallyRestApi;
import com.rallydev.rest.request.CreateRequest;
import com.rallydev.rest.request.GetRequest;
import com.rallydev.rest.response.CreateResponse;
import com.rallydev.rest.util.Ref;
public class CreateStory {
public static void main(String[] args) throws URISyntaxException,IOException {
String host = "https://rally1.rallydev.com";
String username = "name#comp.com";
String password = "password";
String wsapiVersion = "v2.0";
String applicationName = "C";
RallyRestApi restApi = new RallyRestApi(new URI(host),username,password);
restApi.setWsapiVersion(wsapiVersion);
restApi.setApplicationName(applicationName);
try {
for (int i = 0; i < 3; i++) {
// Add a story
System.out.println("Creating a story...");
JsonObject newStory = new JsonObject();
newStory.addProperty("Name", "my story");
newStory.addProperty("Project", "Characters");
CreateRequest createRequest = new CreateRequest("hierarchicalrequirement", newStory);
CreateResponse createResponse = restApi.create(createRequest);
System.out.println("Response ::: "+createResponse.wasSuccessful());
if (createResponse.wasSuccessful()) {
System.out.println(String.format("Created %s",createResponse.getObject().get("_ref").getAsString()));
// Read story
String ref = Ref.getRelativeRef(createResponse.getObject().get("_ref").getAsString());
System.out.println(String.format("\nReading Story %s...",ref));
GetRequest getRequest = new GetRequest(ref);
} else {
String[] createErrors;
createErrors = createResponse.getErrors();
System.out.println("Error occurred creating story: ");
for (int j = 0; j < createErrors.length; j++) {
System.out.println(createErrors[j]);
}
}
}
} finally {
// Release all resources
restApi.close();
}
}
}
And I am getting error as :
Creating a story...
Response ::: false
Error occurred creating story:
Could not read: Cannot parse object reference from "BSS HWSE Team Columbia"
Creating a story...
Response ::: false
Error occurred creating story:
Could not read: Cannot parse object reference from "BSS HWSE Team Columbia"
Creating a story...
Response ::: false
Error occurred creating story:
Could not read: Cannot parse object reference from "BSS HWSE Team Columbia"
Picked up JAVA_TOOL_OPTIONS: -agentlib:jvmhook
Picked up _JAVA_OPTIONS: -Xrunjvmhook -Xbootclasspath/a:"C:\Program Files\HP\Unified Functional Testing\bin\java_shared\classes";"C:\Program Files\HP\Unified Functional Testing\bin\java_shared\classes\jasmine.jar"
Please help. New to rally. Thanks in advance
When setting reference fields such as Project, Iteration, Release, WorkProduduct, Parent, etc that point to full objects use the reference,and not the Name.
Find out the unique ObjectID of the Project "C", e.g. 123456.
Replace:
newStory.addProperty("Project", "Characters");
with
newStory.addProperty("Project", "/project/123456");
To find ObjectID of a project you may either query by its name in WS API, or look at the URL of one of Rally pages in the address bar, while in that project, e.g. URL of a Backlog page: https://rally1.rallydev.com/#/123456/backlog
If there is a d or u appended to ObjectID string, e.g. 123456d, do not include it in string. In the URL it indicates if you are currently scoping up or down.