migrating CQ 5.6.1(jdk 1.7) to AEM 6.4(jdk 1.8), Unable to deserialze byte array while reading session from Cassandra using Serializer<SimpleSession> - apache

i am migrating java bundles from CQ 5.6.1 to AEM 6.4, bundle is working in jdk 1.7,
code is using apache shiro to serealize and deserialize session, here is the code to save and read session stored in Cassandra
it works good with AEM 5.6.1 and 6.1 (works in jdk 1.7) when i migrate it to AEM 6.4 (working in jdk 1.8) code is
giving exception in "method doReadSession" when it is trying to deserialize session in bite array
it throw an exception
In method doReadSession where it return "serializer.deserialize(bytes)"
exception thrown is
Caused by: org.apache.shiro.io.SerializationException: Unable to deserialze argument byte array.
at org.apache.shiro.io.DefaultSerializer.deserialize(DefaultSerializer.java:82)
at com.xyz.web.platform.common.security.shiro.cassandra.CassandraSessionDAO.doReadSession(CassandraSessionDAO.java:252)
at org.apache.shiro.session.mgt.eis.AbstractSessionDAO.readSession(AbstractSessionDAO.java:168)
at org.apache.shiro.session.mgt.DefaultSessionManager.retrieveSessionFromDataSource(DefaultSessionManager.java:236)
import com.xyz.web.platform.common.security.shiro.session.idgenerator.UUIDSessionIdGenerator;
import com.xyz.web.platform.common.tracelogging.TMTraceLogger;
import java.io.Serializable;
import java.nio.ByteBuffer;
import java.util.Collection;
import java.util.Collections;
import org.apache.shiro.ShiroException;
import org.apache.shiro.io.DefaultSerializer;
import org.apache.shiro.io.Serializer;
import org.apache.shiro.session.Session;
import org.apache.shiro.session.UnknownSessionException;
import org.apache.shiro.session.mgt.SimpleSession;
import org.apache.shiro.session.mgt.eis.AbstractSessionDAO;
import org.apache.shiro.util.Destroyable;
import org.apache.shiro.util.Initializable;
import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.Row;
public class CassandraSessionDAO extends AbstractSessionDAO implements Initializable, Destroyable {
private static final TMTraceLogger CASSANDRA_LOGGER = TMTraceLogger
.getLogger("cassandraLogger" + "." + CassandraSessionDAO.class.getName());
private String DEFAULT_CONSISTENCY_LEVEL = ConsistencyLevel.LOCAL_QUORUM.toString();
private static final String DEFAULT_CACHING = "ROWS_ONLY";
private static final String DEFAULT_DATA_CENTER = "DC1";
private String keyspaceName;
private String tableName;
private Cluster cluster;
private Serializer<SimpleSession> serializer;
private String readConsistencyLevel;
private String writeConsistencyLevel;
private String defaultTTL;
private String gc_grace_seconds;
private String caching = DEFAULT_CACHING;
private String dataCenter = DEFAULT_DATA_CENTER;
private PreparedStatement deletePreparedStatement;
private PreparedStatement savePreparedStatement;
private PreparedStatement readPreparedStatement;
private com.datastax.driver.core.Session cassandraSession;
public CassandraSessionDAO() {
setSessionIdGenerator(new UUIDSessionIdGenerator());
this.serializer = new DefaultSerializer<SimpleSession>();
}
private SimpleSession assertSimpleSession(Session session) {
if (!(session instanceof SimpleSession)) {
throw new IllegalArgumentException(CassandraSessionDAO.class.getName() + " implementations only support "
+ SimpleSession.class.getName() + " instances.");
}
return (SimpleSession) session;
}
public Cluster getCluster() {
return cluster;
}
public void setCluster(Cluster cluster) {
this.cluster = cluster;
}
public String getKeyspaceName() {
return keyspaceName;
}
public void setKeyspaceName(String keyspaceName) {
this.keyspaceName = keyspaceName;
}
public String getTableName() {
return tableName;
}
public void setTableName(String tableName) {
this.tableName = tableName;
}
public String getReadConsistencyLevel() {
return readConsistencyLevel;
}
public void setReadConsistencyLevel(String readConsistencyLevel) {
this.readConsistencyLevel = readConsistencyLevel;
}
public String getWriteConsistencyLevel() {
return writeConsistencyLevel;
}
public void setWriteConsistencyLevel(String writeConsistencyLevel) {
this.writeConsistencyLevel = writeConsistencyLevel;
}
public String getDefaultTTL() {
return defaultTTL;
}
public void setDefaultTTL(String defaultTTL) {
this.defaultTTL = defaultTTL;
}
public String getGc_grace_seconds() {
return gc_grace_seconds;
}
public void setGc_grace_seconds(String gc_grace_seconds) {
this.gc_grace_seconds = gc_grace_seconds;
}
public String getCaching() {
return caching;
}
public void setCaching(String caching) {
this.caching = caching;
}
public String getDataCenter() {
return dataCenter;
}
public void setDataCenter(String dataCenter) {
this.dataCenter = dataCenter;
}
public void init() throws ShiroException {
com.datastax.driver.core.Session systemSession = cluster.connect();
boolean create = false;
try {
if (!isKeyspacePresent(systemSession)) {
create = true;
createKeyspace(systemSession);
if (!isKeyspacePresent(systemSession)) {
throw new IllegalStateException("Unable to create keyspace " + keyspaceName);
}
}
} finally {
systemSession.shutdown();
}
cassandraSession = cluster.connect(keyspaceName);
if (create) {
createTable();
}
prepareReadStatement();
prepareSaveStatement();
prepareDeleteStatement();
}
public void destroy() throws Exception {
if (cassandraSession != null) {
cassandraSession.shutdown();
cluster.shutdown();
}
}
protected boolean isKeyspacePresent(com.datastax.driver.core.Session systemSession) {
PreparedStatement ps = systemSession.prepare("select * from system.schema_keyspaces");
BoundStatement bs = new BoundStatement(ps);
ResultSet results = systemSession.execute(bs);
// ResultSet results = systemSession.execute("select * from
// system.schema_keyspaces");
for (Row row : results) {
if (row.getString("keyspace_name").equalsIgnoreCase(keyspaceName)) {
return true;
}
}
return false;
}
protected void createKeyspace(com.datastax.driver.core.Session systemSession) {
String query = "create keyspace " + this.keyspaceName
+ " with replication = {'class' : 'NetworkTopologyStrategy', '" + dataCenter + "': 2};";
systemSession.execute(query);
}
protected void createTable() {
long defaultTTLLong = Long.parseLong(defaultTTL);
long gc_grace_secondsLong = Long.parseLong(gc_grace_seconds);
String query = "CREATE TABLE " + tableName + " ( " + " id varchar PRIMARY KEY, " + " start_ts timestamp, "
+ " stop_ts timestamp, " + " last_access_ts timestamp, " + " timeout bigint, "
+ " expired boolean, " + " host varchar, " + " serialized_value blob " + ") " + "WITH "
+ " gc_grace_seconds = " + gc_grace_secondsLong + " AND default_time_to_live = " + defaultTTLLong
+ " AND caching = '" + caching + "' AND " + " compaction = {'class':'LeveledCompactionStrategy'};";
cassandraSession.execute(query);
}
#Override
protected Serializable doCreate(Session session) {
SimpleSession ss = assertSimpleSession(session);
Serializable timeUuid = generateSessionId(session);
assignSessionId(ss, timeUuid);
save(ss);
return timeUuid;
}
#Override
protected Session doReadSession(Serializable sessionId) {
long startTime = System.currentTimeMillis();
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("In the doReadSession()..");
CASSANDRA_LOGGER.logDebug("The start time is " + startTime + " ms");
}
// put a not-null & not-empty check
if (sessionId != null && !sessionId.equals("")) {
String id = sessionId.toString();
PreparedStatement ps = prepareReadStatement();
/*
* String query = "SELECT * from " + tableName + " where id = ?";
* PreparedStatement ps = cassandraSession.prepare(query);
*/
BoundStatement bs = new BoundStatement(ps);
bs.bind(id);
if (readConsistencyLevel == null) {
readConsistencyLevel = DEFAULT_CONSISTENCY_LEVEL;
}
bs.setConsistencyLevel(ConsistencyLevel.valueOf(readConsistencyLevel));
ResultSet results = cassandraSession.execute(bs);
for (Row row : results) {
String rowId = row.getString("id");
if (id.equals(rowId)) {
ByteBuffer buffer = row.getBytes("serialized_value");
if (buffer != null) {
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
return serializer.deserialize(bytes);
}
}
}
}
long endTime = System.currentTimeMillis();
long timeTaken = endTime - startTime;
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("The total time taken is " + timeTaken + " ms");
}
return null;
}
private PreparedStatement prepareReadStatement() {
if (this.readPreparedStatement == null) {
String query = "SELECT * from " + tableName + " where id = ?";
this.readPreparedStatement = cassandraSession.prepare(query);
}
return this.readPreparedStatement;
}
// In CQL, insert and update are effectively the same, so we can use a single
// query for both:
protected void save(SimpleSession ss) {
// long timeoutInSeconds = ss.getTimeout() / 1000;
/*
* String query = "UPDATE " + tableName + " SET " + "start_ts = ?, " +
* "stop_ts = ?, " + "last_access_ts = ?, " + "timeout = ?, " + "expired = ?, "
* + "host = ?, " + "serialized_value = ? " + "WHERE " + "id = ?";
* PreparedStatement ps = cassandraSession.prepare(query);
*/
long startTime = System.currentTimeMillis();
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("In the save()..");
CASSANDRA_LOGGER.logDebug("The start time is " + startTime + " ms");
}
PreparedStatement ps = prepareSaveStatement();
BoundStatement bs = new BoundStatement(ps);
byte[] serialized = serializer.serialize(ss);
ByteBuffer bytes = ByteBuffer.wrap(serialized);
bs.bind(ss.getStartTimestamp(), ss.getStopTimestamp() != null ? ss.getStartTimestamp() : null,
ss.getLastAccessTime(), ss.getTimeout(), ss.isExpired(), ss.getHost(), bytes, ss.getId().toString());
if (writeConsistencyLevel == null) {
writeConsistencyLevel = DEFAULT_CONSISTENCY_LEVEL;
}
bs.setConsistencyLevel(ConsistencyLevel.valueOf(writeConsistencyLevel));
cassandraSession.execute(bs);
long endTime = System.currentTimeMillis();
long timeTaken = endTime - startTime;
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("The total time taken is " + timeTaken + " ms");
}
}
private PreparedStatement prepareSaveStatement() {
if (this.savePreparedStatement == null) {
String query = "UPDATE " + tableName + " SET " + "start_ts = ?, " + "stop_ts = ?, " + "last_access_ts = ?, "
+ "timeout = ?, " + "expired = ?, " + "host = ?, " + "serialized_value = ? " + "WHERE " + "id = ?";
this.savePreparedStatement = cassandraSession.prepare(query);
}
return this.savePreparedStatement;
}
public void update(Session session) throws UnknownSessionException {
SimpleSession ss = assertSimpleSession(session);
save(ss);
}
public void delete(Session session) {
/*
* String query = "DELETE from " + tableName + " where id = ?";
* PreparedStatement ps = cassandraSession.prepare(query);
*/
long startTime = System.currentTimeMillis();
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("In the delete()..");
CASSANDRA_LOGGER.logDebug("The start time is " + startTime + " ms");
}
PreparedStatement ps = prepareDeleteStatement();
BoundStatement bs = new BoundStatement(ps);
bs.bind(session.getId().toString());
cassandraSession.execute(bs);
long endTime = System.currentTimeMillis();
long timeTaken = endTime - startTime;
if (CASSANDRA_LOGGER.isDebugEnabled()) {
CASSANDRA_LOGGER.logDebug("The total time taken is " + timeTaken + " ms");
}
}
private PreparedStatement prepareDeleteStatement() {
if (this.deletePreparedStatement == null) {
String query = "DELETE from " + tableName + " where id = ?";
this.deletePreparedStatement = cassandraSession.prepare(query);
}
return this.deletePreparedStatement;
}
public Collection<Session> getActiveSessions() {
return Collections.emptyList();
}
}

Need to add your class or pacakge in the configuration in whitelist section
https://helpx.adobe.com/experience-manager/6-4/sites/administering/using/mitigating-serialization-issues.html

Related

Visual Basic / SQL Server: identity_insert is set to off error

I just recently started toying with all of this programming stuff, so I'm not quite sure on how to fix a minor problem I seem to have with the communication between the server and the client.
when I start up the server, players are able to connect and log out when they are done playing without any problems, but the server throws an error and doesn't save the item in the table items.
It's almost as if the server/client can't communicate properly with the dbo.items table :/
Any help would be appreciated! Thank you.
And sorry for the bad English
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.Data;
namespace Goose
{
/**
* Item, holds the actual item data
*
* Each item in game is separate
* Holds the original template and modified/added stats
*
*/
public class Item : IItem
{
public int ItemID { get; set; }
public int TemplateID { get; set; }
public ItemTemplate Template { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public int GraphicEquipped { get; set; }
public int GraphicTile { get; set; }
public int GraphicR { get; set; }
public int GraphicG { get; set; }
public int GraphicB { get; set; }
public int GraphicA { get; set; }
int weapondamage = 0;
public int WeaponDamage
{
get
{
return this.weapondamage + (int)Math.Ceiling(this.Template.WeaponDamage * this.StatMultiplier);
}
set
{
this.weapondamage = value;
}
}
/**
* Body pose/state 1 for normal, 3 for staff, 4 for sword
*/
public int BodyState { get; set; }
public AttributeSet BaseStats { get; set; }
public AttributeSet TotalStats { get; set; }
public decimal StatMultiplier { get; set; }
/**
* Dirty, has data changed since loading
*
*/
public bool Dirty { get; set; }
/**
* Delete, item is no longer in game so delete it
*/
public bool Delete { get; set; }
public long Value { get; set; }
bool bound = false;
public bool IsBound
{
get { return this.bound; }
set { this.bound = value; }
}
/**
* These properties are read only and just pass along from the templates properties
*
*/
public int WeaponDelay { get { return this.Template.WeaponDelay; } }
public int StackSize { get { return this.Template.StackSize; } }
public bool IsLore { get { return this.Template.IsLore; } }
public bool IsBindOnPickup { get { return this.Template.IsBindOnPickup; } }
public bool IsBindOnEquip { get { return this.Template.IsBindOnEquip; } }
public bool IsEvent { get { return this.Template.IsEvent; } }
public ItemTemplate.ItemSlots Slot { get { return this.Template.Slot; } }
public ItemTemplate.ItemTypes Type { get { return this.Template.Type; } }
public ItemTemplate.UseTypes UseType { get { return this.Template.UseType; } }
public int MinLevel { get { return this.Template.MinLevel; } }
public int MaxLevel { get { return this.Template.MaxLevel; } }
public long MinExperience { get { return this.Template.MinExperience; } }
public long MaxExperience { get { return this.Template.MaxExperience; } }
/**
* This is a bitmask
* Therefore only limited to about 64 classes, which should be enough.
* If the bit is set then that class id CAN'T use the item.
*
*/
public long ClassRestrictions { get { return this.Template.ClassRestrictions; } }
public SpellEffect SpellEffect { get { return this.Template.SpellEffect; } }
public decimal SpellEffectChance { get { return this.Template.SpellEffectChance; } }
public int LearnSpellID { get { return this.Template.LearnSpellID; } }
public bool Unsaved { get; set; }
public Item()
{
this.Unsaved = true;
this.ItemID = 0;
this.TotalStats = new AttributeSet();
this.BaseStats = new AttributeSet();
this.StatMultiplier = 1;
this.Dirty = true;
this.Delete = false;
}
/**
* LoadFromTemplate, loads item from a template
*
* This is when we want an item the same as the template.
*
*/
public void LoadFromTemplate(ItemTemplate template)
{
this.Template = template;
this.TemplateID = this.Template.ID;
this.TotalStats += this.Template.BaseStats;
this.Name = this.Template.Name;
this.Description = this.Template.Description;
this.GraphicEquipped = this.Template.GraphicEquipped;
this.GraphicTile = this.Template.GraphicTile;
this.GraphicR = this.Template.GraphicR;
this.GraphicG = this.Template.GraphicG;
this.GraphicB = this.Template.GraphicB;
this.GraphicA = this.Template.GraphicA;
this.Value = this.Template.Value;
this.BodyState = this.Template.BodyState;
}
/**
* LoadTemplate, adds template to item
*
* This is when we want to just add the templates stats to our item
* ie when loading the items database, eg for surname/titled items
*
* Note: Doesn't load the value from the template as we want to keep the value as 0
* if the item is custom for example
*
*/
public void LoadTemplate(ItemTemplate template)
{
this.TotalStats += template.BaseStats;
this.TotalStats *= this.StatMultiplier;
this.TotalStats += this.BaseStats;
}
/**
* AddItem, adds item to database
*
*/
public void AddItem(GameWorld world)
{
SqlParameter nameParam = new SqlParameter("#itemName", SqlDbType.VarChar, 64);
nameParam.Value = this.Name;
SqlParameter descriptionParam = new SqlParameter("#itemDescription", SqlDbType.VarChar, 64);
descriptionParam.Value = this.Description;
string query = "INSERT INTO items (item_id, item_template_id, item_name, item_description, " +
"player_hp, player_mp, player_sp, stat_ac, stat_str, stat_sta, stat_dex, stat_int, " +
"res_fire, res_water, res_spirit, res_air, res_earth, weapon_damage, item_value, " +
"graphic_tile, graphic_equip, graphic_r, graphic_g, graphic_b, graphic_a, stat_multiplier, " +
"bound, body_state) VALUES (" +
this.ItemID + "," +
this.TemplateID + ", " +
"#itemName, " +
"#itemDescription, " +
this.BaseStats.HP + ", " +
this.BaseStats.MP + ", " +
this.BaseStats.SP + ", " +
this.BaseStats.AC + ", " +
this.BaseStats.Strength + ", " +
this.BaseStats.Stamina + ", " +
this.BaseStats.Dexterity + ", " +
this.BaseStats.Intelligence + ", " +
this.BaseStats.FireResist + ", " +
this.BaseStats.WaterResist + ", " +
this.BaseStats.SpiritResist + ", " +
this.BaseStats.AirResist + ", " +
this.BaseStats.EarthResist + ", " +
this.weapondamage + ", " +
this.Value + ", " +
this.GraphicTile + ", " +
this.GraphicEquipped + ", " +
this.GraphicR + ", " +
this.GraphicG + ", " +
this.GraphicB + ", " +
this.GraphicA + ", " +
this.StatMultiplier + ", " +
(this.bound ? "'1'" : "'0'") + ", " +
this.BodyState + ")";
this.Dirty = false;
this.Unsaved = false;
SqlCommand command = new SqlCommand(query, world.SqlConnection);
command.Parameters.Add(nameParam);
command.Parameters.Add(descriptionParam);
command.BeginExecuteNonQuery(new AsyncCallback(GameWorld.DefaultEndExecuteNonQueryAsyncCallback), command);
}
/**
* SaveItem, updates item info in database
*
*/
public void SaveItem(GameWorld world)
{
SqlParameter nameParam = new SqlParameter("#itemName", SqlDbType.VarChar, 64);
nameParam.Value = this.Name;
SqlParameter descriptionParam = new SqlParameter("#itemDescription", SqlDbType.VarChar, 64);
descriptionParam.Value = this.Description;
string query = "UPDATE items SET " +
"item_template_id=" + this.TemplateID + ", " +
"item_name=" + "#itemName, " +
"item_description=" + "#itemDescription, " +
"player_hp=" + this.BaseStats.HP + ", " +
"player_mp=" + this.BaseStats.MP + ", " +
"player_sp=" + this.BaseStats.SP + ", " +
"stat_ac=" + this.BaseStats.AC + ", " +
"stat_str=" + this.BaseStats.Strength + ", " +
"stat_sta=" + this.BaseStats.Stamina + ", " +
"stat_dex=" + this.BaseStats.Dexterity + ", " +
"stat_int=" + this.BaseStats.Intelligence + ", " +
"res_fire=" + this.BaseStats.FireResist + ", " +
"res_water=" + this.BaseStats.WaterResist + ", " +
"res_spirit=" + this.BaseStats.SpiritResist + ", " +
"res_air=" + this.BaseStats.AirResist + ", " +
"res_earth=" + this.BaseStats.EarthResist + ", " +
"weapon_damage=" + this.weapondamage + ", " +
"item_value=" + this.Value + ", " +
"graphic_tile=" + this.GraphicTile + ", " +
"graphic_equip=" + this.GraphicEquipped + ", " +
"graphic_r=" + this.GraphicR + ", " +
"graphic_g=" + this.GraphicG + ", " +
"graphic_b=" + this.GraphicB + ", " +
"graphic_a=" + this.GraphicA + ", " +
"stat_multiplier=" + this.StatMultiplier + ", " +
"bound=" + (this.bound ? "'1'" : "'0'") + ", " +
"body_state=" + this.BodyState +
" WHERE item_id=" + this.ItemID;
SqlCommand command = new SqlCommand(query, world.SqlConnection);
command.Parameters.Add(nameParam);
command.Parameters.Add(descriptionParam);
command.BeginExecuteNonQuery(new AsyncCallback(GameWorld.DefaultEndExecuteNonQueryAsyncCallback), command);
this.Dirty = false;
}
/**
* DeleteItem, deletes item from database
*
*/
public void DeleteItem(GameWorld world)
{
SqlCommand command = new SqlCommand(
"DELETE FROM items WHERE item_id=" + this.ItemID,
world.SqlConnection);
command.BeginExecuteNonQuery(new AsyncCallback(GameWorld.DefaultEndExecuteNonQueryAsyncCallback), command);
}
}
}
public void AddItem(GameWorld world)
{
SqlParameter nameParam = new SqlParameter("#itemName", SqlDbType.VarChar, 64);
nameParam.Value = this.Name;
SqlParameter descriptionParam = new SqlParameter("#itemDescription", SqlDbType.VarChar, 64);
descriptionParam.Value = this.Description;
string query = "INSERT INTO items (item_template_id, item_name, item_description, " +
"player_hp, player_mp, player_sp, stat_ac, stat_str, stat_sta, stat_dex, stat_int, " +
"res_fire, res_water, res_spirit, res_air, res_earth, weapon_damage, item_value, " +
"graphic_tile, graphic_equip, graphic_r, graphic_g, graphic_b, graphic_a, stat_multiplier, " +
"bound, body_state) VALUES (" +
this.TemplateID + ", " +
"#itemName, " +
"#itemDescription, " +
this.BaseStats.HP + ", " +
this.BaseStats.MP + ", " +
this.BaseStats.SP + ", " +
this.BaseStats.AC + ", " +
this.BaseStats.Strength + ", " +
this.BaseStats.Stamina + ", " +
this.BaseStats.Dexterity + ", " +
this.BaseStats.Intelligence + ", " +
this.BaseStats.FireResist + ", " +
this.BaseStats.WaterResist + ", " +
this.BaseStats.SpiritResist + ", " +
this.BaseStats.AirResist + ", " +
this.BaseStats.EarthResist + ", " +
this.weapondamage + ", " +
this.Value + ", " +
this.GraphicTile + ", " +
this.GraphicEquipped + ", " +
this.GraphicR + ", " +
this.GraphicG + ", " +
this.GraphicB + ", " +
this.GraphicA + ", " +
this.StatMultiplier + ", " +
(this.bound ? "'1'" : "'0'") + ", " +
this.BodyState + ")";
this.Dirty = false;
this.Unsaved = false;
SqlCommand command = new SqlCommand(query, world.SqlConnection);
command.Parameters.Add(nameParam);
command.Parameters.Add(descriptionParam);
command.BeginExecuteNonQuery(new AsyncCallback(GameWorld.DefaultEndExecuteNonQueryAsyncCallback), command);
}
dont insert column item_id because it is identity and it will be inserted automatically

Failed to parse resource-request java.net.URISyntaxException: Expected scheme name at index 0: ://

I try to make yarn application master and runs application on hadoop 2. These are sources
=== Client.java
public class MyClient {
Configuration conf = new YarnConfiguration();
public void run(String[] args) throws Exception {
final int n = 1;
final Path jarPath = new Path("./");
YarnConfiguration conf = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start();
YarnClientApplication app = yarnClient.createApplication();
ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);
amContainer.setCommands(
Collections.singletonList(
"$JAVA_HOME/bin/java" + " -Xmx256M" + " com.aaa.yarn.MyApplicationMaster" + " " + jarPath +
" " + String.valueOf(n) +
" 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
" 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
)
);
LocalResource appMasterJar = Records.newRecord(LocalResource.class);
setupAppMasterJar(jarPath, appMasterJar);
amContainer.setLocalResources(Collections.singletonMap("YarnHelloWorld.jar", appMasterJar));
Map<String, String> appMasterEnv = new HashMap<String, String>();
setupAppMasterEnv(appMasterEnv);
amContainer.setEnvironment(appMasterEnv);
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(256);
capability.setVirtualCores(1);
ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext();
appContext.setApplicationName("Yarn_Hello_World"); // application name
appContext.setAMContainerSpec(amContainer);
appContext.setResource(capability);
appContext.setQueue("default"); // queue
ApplicationId appId = appContext.getApplicationId();
System.out.println("Submitting application " + appId);
yarnClient.submitApplication(appContext);
ApplicationReport appReport = yarnClient.getApplicationReport(appId);
YarnApplicationState appState = appReport.getYarnApplicationState();
while (appState != YarnApplicationState.FINISHED && appState != YarnApplicationState.KILLED &&
appState != YarnApplicationState.FAILED) {
Thread.sleep(100);
appReport = yarnClient.getApplicationReport(appId);
appState = appReport.getYarnApplicationState();
}
System.out.println("Application " + appId + " finished with" + " state " + appState + " at " + appReport.getFinishTime());
}
private void setupAppMasterJar(Path jarPath, LocalResource appMasterJar) throws IOException {
FileStatus jarStat = FileSystem.get(conf).getFileStatus(jarPath);
appMasterJar.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
appMasterJar.setSize(jarStat.getLen());
appMasterJar.setTimestamp(jarStat.getModificationTime());
appMasterJar.setType(LocalResourceType.FILE);
appMasterJar.setVisibility(LocalResourceVisibility.PUBLIC);
}
private void setupAppMasterEnv(Map<String, String> appMasterEnv) {
StringBuilder classPathEnv = new StringBuilder(Environment.CLASSPATH.$$())
.append(ApplicationConstants.CLASS_PATH_SEPARATOR).append("./*");
for (String c : conf.getStrings(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
YarnConfiguration.DEFAULT_YARN_CROSS_PLATFORM_APPLICATION_CLASSPATH)) {
classPathEnv.append(ApplicationConstants.CLASS_PATH_SEPARATOR);
classPathEnv.append(c.trim());
}
appMasterEnv.put("CLASSPATH", classPathEnv.toString());
}
public static void main(String[] args) throws Exception{
// TODO Auto-generated method stub
MyClient c = new MyClient();
c.run(args);
}
}
=== My Application Master
public class MyApplicationMaster {
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
final int n = 1;
Configuration conf = new YarnConfiguration();
AMRMClient<ContainerRequest> rmClient = AMRMClient.createAMRMClient();
rmClient.init(conf);
rmClient.start();
NMClient nmClient = NMClient.createNMClient();
nmClient.init(conf);
nmClient.start();
System.out.println("registerApplicationMaster 0");
rmClient.registerApplicationMaster("", 0, "");
System.out.println("registerApplicationMaster 1");
Priority priority = Records.newRecord(Priority.class);
priority.setPriority(0);
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(128);
capability.setVirtualCores(1);
for (int i = 0; i < n; ++i) {
ContainerRequest containerAsk = new ContainerRequest(capability, null, null, priority);
System.out.println("Making res-req " + i);
rmClient.addContainerRequest(containerAsk);
}
int responseId = 0;
int completedContainers = 0;
while (completedContainers < n) {
AllocateResponse response = rmClient.allocate(responseId++);
for (Container container : response.getAllocatedContainers()) {
ContainerLaunchContext ctx = Records.newRecord(ContainerLaunchContext.class);
ctx.setCommands(
Collections.singletonList(
"$JAVA_HOME/bin/java" + " -Xmx256M" + " com.aaa.yarn.YarnHelloWorld" +
" 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
" 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
));
System.out.println("Launching container " + container.getId());
nmClient.startContainer(container, ctx);
}
for (ContainerStatus status : response.getCompletedContainersStatuses()) {
++completedContainers;
System.out.println("Completed container " + status.getContainerId());
}
Thread.sleep(100);
}
rmClient.unregisterApplicationMaster(
FinalApplicationStatus.SUCCEEDED, "", "");
}
}
But deployment is failed. In nodemanager log the following exception is thrown
WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Failed to parse resource-request
java.net.URISyntaxException: Expected scheme name at index 0: ://
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.failExpecting(URI.java:2854)
at java.net.URI$Parser.parse(URI.java:3046)
at java.net.URI.<init>(URI.java:746)
at org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:80)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:632)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:590)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:992)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:76)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:1065)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:1058)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
at java.lang.Thread.run(Thread.java:745)
I think there is some problem on this line
amContainer.setCommands(
Collections.singletonList(
"Environment.JAVA_HOME.$$()" + "/bin/java" + " -Xmx256M" + " com.aaa.yarn.MyApplicationMaster" + " " + jarPath +
" " + String.valueOf(n) +
" 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
" 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
)
);
But I have no idea.

To Get the VM Created Date

I am new to the VMWare Sdk Programming,i have a requirement to get the Virtual Machine (VM) Deployed date.
I have written the below code to get the other required details.
package com.vmware.vim25.mo.samples;
import java.net.URL;
import com.vmware.vim25.*;
import com.vmware.vim25.mo.*;
public class HelloVM {
public static void main(String[] args) throws Exception
{
long start = System.currentTimeMillis();
int i;
ServiceInstance si = new ServiceInstance(new URL("https://bgl-clvs-vc.bgl.com/sdk"), "sbibi", "sibi_123", true);
long end = System.currentTimeMillis();
System.out.println("time taken:" + (end-start));
Folder rootFolder = si.getRootFolder();
String name = rootFolder.getName();
System.out.println("root:" + name);
ManagedEntity[] mes = new InventoryNavigator(rootFolder).searchManagedEntities("VirtualMachine");
System.out.println("No oF vm:" + mes.length);
if(mes==null || mes.length ==0)
{
return;
}
for(i=0;i<mes.length; i++){
VirtualMachine vm = (VirtualMachine) mes[i];
VirtualMachineConfigInfo vminfo = vm.getConfig();
VirtualMachineCapability vmc = vm.getCapability();
vm.getResourcePool();
System.out.println("VM Name " + vm.getName());
System.out.println("GuestOS: " + vminfo.getGuestFullName());
System.out.println("Multiple snapshot supported: " + vmc.isMultipleSnapshotsSupported());
System.out.println("Summary: " + vminfo.getDatastoreUrl());
}
si.getServerConnection().logout();
}
}
Can anyone help me how I can get the VM created date?
I have found the Vm Creation Date using the below codes.
EventFilterSpecByUsername uFilter =
new EventFilterSpecByUsername();
uFilter.setSystemUser(false);
uFilter.setUserList(new String[] {"administrator"});
Event[] events = evtMgr.queryEvents(efs);
// print each of the events
for(int i=0; events!=null && i<events.length; i++)
{
System.out.println("\nEvent #" + i);
printEvent(events[i]);
}
/**
* Only print an event as Event type.
*/
static void printEvent(Event evt)
{
String typeName = evt.getClass().getName();
int lastDot = typeName.lastIndexOf('.');
if(lastDot != -1)
{
typeName = typeName.substring(lastDot+1);
}
System.out.println("Time:" + evt.getCreatedTime().getTime());
}
Hope this code might help others.
private DateTime GetVMCreatedDate(VirtualMachine vm)
{
var date = DateTime. Now;
var userName = new EventFilterSpecByUsername ();
userName . SystemUser = false;
var filter = new EventFilterSpec ();
filter . UserName = userName;
filter . EventTypeId = ( new String [] { "VmCreatedEvent" , "VmBeingDeployedEvent" ,"VmRegisteredEvent" , "VmClonedEvent" });
var collector = vm .GetEntityOnlyEventsCollectorView(filter);
foreach (Event e in collector . ReadNextEvents(1 ))
{
Console .WriteLine(e . GetType(). ToString() + " :" + e. CreatedTime);
date = e. CreatedTime;
}
Console .WriteLine( "---------------------------------------------------" );
return date;
}

How to store fingerprint template to sql server

I am having trouble in storing fpt templates in SQL Server.
Here is my code:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.IO;
//imports
using DHELTASSys.AuditTrail;
using DHELTASSys.Modules;
namespace Enrollment
{
delegate void Function(); // a simple delegate for marshalling calls from event handlers to the GUI thread
public partial class CreateAccount : Form
{
HRModuleBL obj = new HRModuleBL();
DHELTASSysAuditTrail audit = new DHELTASSysAuditTrail();
public CreateAccount(int emp_id)
{
InitializeComponent();
audit.Emp_id = emp_id;
}
protected void EmptyFields()
{
}
private void CloseButton_Click(object sender, EventArgs e)
{
Close();
}
private void OnTemplate(DPFP.Template template)
{
this.Invoke(new Function(delegate()
{
Template = template;
VerifyButton.Enabled = SaveButton.Enabled = (Template != null);
if (Template != null)
MessageBox.Show("The fingerprint template is ready for verification and saving", "Fingerprint Enrollment");
else
MessageBox.Show("The fingerprint template is not valid. Repeat fingerprint enrollment.", "Fingerprint Enrollment");
}));
}
private DPFP.Template Template;
private void EnrollButton_Click(object sender, EventArgs e)
{
EnrollmentForm Enroller = new EnrollmentForm();
Enroller.OnTemplate += this.OnTemplate;
Enroller.ShowDialog();
}
private void SaveButton_Click(object sender, EventArgs e)
{
obj.Last_name = txtLastname.Text;
obj.First_name = txtFirstName.Text;
obj.Middle_name = txtMiddleName.Text;
obj.Position_name = cmbPosition.Text;
obj.Company_name = cmbCompany.Text;
obj.Password = txtTempPassword.Text;
obj.Department_name = cmbDepartment.Text;
if (obj.Last_name == string.Empty) //Validation for empty texts
{
MessageBox.Show("Last name can't be empty!");
} else if (obj.First_name == string.Empty)
{
MessageBox.Show("First name can't be empty!");
}
else if (obj.Middle_name == string.Empty)
{
MessageBox.Show("Middle name can't be empty!");
}
else if (obj.Position_name == string.Empty)
{
MessageBox.Show("Position name can't be empty!");
}
else if (obj.Department_name == string.Empty)
{
MessageBox.Show("Deparment can't be empty!");
}
else if (obj.Company_name == string.Empty)
{
MessageBox.Show("Company name can't be empty!");
}
else if (obj.Password == string.Empty)
{
MessageBox.Show("Password can't be empty!");
}
else if (txtConfirmTempPassword.Text == string.Empty)
{
MessageBox.Show("Please verify your input password!");
}
else
{
if (txtTempPassword.Text != txtConfirmTempPassword.Text)
{
MessageBox.Show("Password does not match", "Password Mismatch",
MessageBoxButtons.OK);
}
else
{
MemoryStream fingerprintData = new MemoryStream();
Template.Serialize(fingerprintData);
fingerprintData.Position = 0;
BinaryReader br = new BinaryReader(fingerprintData);
byte[] bytes = br.ReadBytes((Int32)fingerprintData.Length);
obj.Biometric_code = bytes;
obj.AddAccountSetTempPassword();
audit.AddAuditTrail("Created account for " + obj.First_name + " " + obj.Last_name + ".");
MessageBox.Show("Account Created for " + txtLastname.Text + "," + txtFirstName.Text);
txtConfirmTempPassword.Text = "";
txtFirstName.Text = "";
txtLastname.Text = "";
txtMiddleName.Text = "";
txtTempPassword.Text = "";
cmbCompany.Text = "";
cmbDepartment.Text = "";
cmbPosition.Text = "";
}
}
}
private void VerifyButton_Click(object sender, EventArgs e)
{
VerificationForm Verifier = new VerificationForm();
Verifier.Verify(Template);
}
}
}
Wherein I have initiated the object obj for the class used to insert data to SQL Server.
Using the AddAccountSetTempPassword() method, I store the data into the database.
Problem is, when I look into the database, the image field for all fingerprint data is written as this 0x53797374656D2E427974655B5D.
UPDATE #1
Here is the class that I used to insert data into the database.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
//Imports
using System.Data;
using DHELTASSys.DataAccess;
using DHELTASSys.AuditTrail;
namespace DHELTASSys.Modules
{
class HRModuleBL
{
DHELTASSysAuditTrail audit = new DHELTASSysAuditTrail();
private int emp_id;
public int Emp_id
{
get { return emp_id; }
set { emp_id = value; }
}
private string password;
public string Password
{
get { return password; }
set { password = value; }
}
private string last_name;
public string Last_name
{
get { return last_name; }
set { last_name = value; }
}
private string first_name;
public string First_name
{
get { return first_name; }
set { first_name = value; }
}
private string middle_name;
public string Middle_name
{
get { return middle_name; }
set { middle_name = value; }
}
private string position_name;
public string Position_name
{
get { return position_name; }
set { position_name = value; }
}
private string company_name;
public string Company_name
{
get { return company_name; }
set { company_name = value; }
}
private string email;
public string Email
{
get { return email; }
set { email = value; }
}
private string gender;
public string Gender
{
get { return gender; }
set { gender = value; }
}
private string address;
public string Address
{
get { return address; }
set { address = value; }
}
private string primary_Number;
public string Primary_Number
{
get { return primary_Number; }
set { primary_Number = value; }
}
private string alternative_Number;
public string Alternative_Number
{
get { return alternative_Number; }
set { alternative_Number = value; }
}
private string city;
public string City
{
get { return city; }
set { city = value; }
}
private DateTime birthdate;
public DateTime Birthdate
{
get { return birthdate; }
set { birthdate = value; }
}
private int sss_Number;
public int Sss_Number
{
get { return sss_Number; }
set { sss_Number = value; }
}
private int philHealth_number;
public int Philhealth_number
{
get { return philHealth_number; }
set { philHealth_number = value; }
}
private byte[] biometric_code;
public byte[] Biometric_code
{
get { return biometric_code; }
set { biometric_code = value; }
}
private string employee_status;
public string Employee_status
{
get { return employee_status; }
set { employee_status = value; }
}
private int company_id;
public int Company_id
{
get { return company_id; }
set { company_id = value; }
}
private string department_name;
public string Department_name
{
get { return department_name; }
set { department_name = value; }
}
//Methods
//Create account and set temporary password for employee
public void AddAccountSetTempPassword()
{
if (Position_name == "Supervisor") //If employee is supervisor
{
string cmd = "EXECUTE AddAccountSetTempPassword '" + Password + "',"
+ "'" + Last_name + "',"
+ "'" + First_name + "',"
+ "'" + Middle_name + "',"
+ "'" + Position_name + "',"
+ "'" + Company_name + "',"
+ "'" + Department_name + "',"
+ "'" + Biometric_code + "'";
DHELTASSysDataAccess.Modify(cmd);
string cmdTwo = "Execute AddSupervisor";
DHELTASSysDataAccess.Modify(cmdTwo);
}
else //If employee isn't a supervisor
{
string cmd = "EXECUTE AddAccountSetTempPassword '" + Password + "',"
+ "'" + Last_name + "',"
+ "'" + First_name + "',"
+ "'" + Middle_name + "',"
+ "'" + Position_name + "',"
+ "'" + Company_name + "',"
+ "'" + Department_name + "',"
+ "'" + Biometric_code + "'";
DHELTASSysDataAccess.Modify(cmd);
}
}
//Employee will enter permanent password for account
public void AddPermanentPasswordForAccount()
{
string cmd = "EXECUTE AddPermanentPasswordForAccount"
+ "'" + Password + "',"
+ "'" + Emp_id + "',";
DHELTASSysDataAccess.Modify(cmd);
//DHELTASSysAuditTrail.AddAuditTrail( "Changed password.");
}
//Employee adds his/her account details
public void AddAccountDetails()
{
string cmd = "EXECUTE AddAccountDetails"
+ "'" + Email + "',"
+ "'" + Gender + "',"
+ "'" + Address + "',"
+ "'" + Primary_Number + "',"
+ "'" + Alternative_Number + "',"
+ "'" + City + "',"
+ "'" + Birthdate + "',"
+ "'" + Sss_Number + "',"
+ "'" + Philhealth_number + "',"
+ "'" + Emp_id + "'";
DHELTASSysDataAccess.Modify(cmd);
//DHELTASSysAuditTrail.AddAuditTrail("Account details updated.");
}
//Displays all employees' information
public DataTable ViewEmployeeInformation()
{
string cmd = "EXECUTE ViewEmployeeInformation"
+ "'" + Company_id + "'";
DataTable dtEmployees = DHELTASSysDataAccess.Select(cmd);
return dtEmployees;
}
//Verifies account login in the forms application
public DataTable AccountEnrollmentLogin()
{
string cmd = "EXECUTE AccountEnrollmentLogin"
+ "'" + Emp_id + "',"
+ "'" + Password + "'";
DataTable dt = DHELTASSysDataAccess.Select(cmd);
return dt;
}
//Check if HR Manager
public DataTable CheckIfHRManager()
{
string cmd = "EXECUTE CheckIfHRManager"
+ "'" + Emp_id + "'";
DataTable dt = DHELTASSysDataAccess.Select(cmd);
return dt;
}
}
}
And the table looks like this:
table : employee
emp_id->int->PK
password->nvarchar(MAX)
last_name->varchar(50)
first_name->varchar(50)
middle_name->varchar(50)
position_id->int->FK
department_id->int->FK
company_id->int->FK
email_address->varchar(50)
gender->varchar(10)
address->varchar(100)
primary_contact_number->varchar(20)
alternative_contact_number->varchar(20)
city->varchar(50)
birthdate->date
sssNumber->int
philhealth_number->int
employee_status->bit
biometrics_image->image
I wish to convert the template into bytes to save it into the database.
Thanks in advance.

Customizing summary section of TestNG emailable report

TestNG generates an emailable report. I have seen that this report can be customized by using Listeners. But could not get what i wanted. My requirement is to include extra details in the summary section of this report. I want to be able to add may be a new table or extra columns to show the environment details of the test execution.
Trying to attach a screenshot but apparently missing something and it does not come up.
That is what I have on my framework. I'll try to explain it (sorry my English)
Copy ReporterListenerAdapter.java and rename as MyReporterListenerAdapter.java, put it on your java project (/listener folder for example)
public class MyReporterListenerAdapter implements IReporter {
public void generateReport(List<XmlSuite> xml, List<ISuite> suites, String outdir) {}
}
Next, copy ReporterListener.java and rename as MyReporterListener.java
Too much code to paste here, but on createWriter function change the report name. For example: "emailable-MyFramework-report".
MyReporterListener.java
public class MyReporterListener extends MyReporterListenerAdapter {
private static final Logger L = Logger.getLogger(MyReporterListener.class);
// ~ Instance fields ------------------------------------------------------
private PrintWriter m_out;
private int m_row;
private Integer m_testIndex;
private int m_methodIndex;
private Scanner scanner;
// ~ Methods --------------------------------------------------------------
/** Creates summary of the run */
#Override
public void generateReport(List<XmlSuite> xml, List<ISuite> suites,
String outdir) {
try {
m_out = createWriter(outdir);
} catch (IOException e) {
L.error("output file", e);
return;
}
startHtml(m_out);
generateSuiteSummaryReport(suites);
generateMethodSummaryReport(suites);
generateMethodDetailReport(suites);
endHtml(m_out);
m_out.flush();
m_out.close();
}
protected PrintWriter createWriter(String outdir) throws IOException {
java.util.Date now = new Date();
new File(outdir).mkdirs();
return new PrintWriter(new BufferedWriter(new FileWriter(new File(
outdir, "emailable-FON-report"
+ DateFunctions.dateToDayAndTimeForFileName(now)
+ ".html"))));
}
/**
* Creates a table showing the highlights of each test method with links to
* the method details
*/
protected void generateMethodSummaryReport(List<ISuite> suites) {
m_methodIndex = 0;
startResultSummaryTable("methodOverview");
int testIndex = 1;
for (ISuite suite : suites) {
if (suites.size() > 1) {
titleRow(suite.getName(), 5);
}
Map<String, ISuiteResult> r = suite.getResults();
for (ISuiteResult r2 : r.values()) {
ITestContext testContext = r2.getTestContext();
String testName = testContext.getName();
m_testIndex = testIndex;
resultSummary(suite, testContext.getFailedConfigurations(),
testName, "failed", " (configuration methods)");
resultSummary(suite, testContext.getFailedTests(), testName,
"failed", "");
resultSummary(suite, testContext.getSkippedConfigurations(),
testName, "skipped", " (configuration methods)");
resultSummary(suite, testContext.getSkippedTests(), testName,
"skipped", "");
resultSummary(suite, testContext.getPassedTests(), testName,
"passed", "");
testIndex++;
}
}
m_out.println("</table>");
}
/** Creates a section showing known results for each method */
protected void generateMethodDetailReport(List<ISuite> suites) {
m_methodIndex = 0;
for (ISuite suite : suites) {
Map<String, ISuiteResult> r = suite.getResults();
for (ISuiteResult r2 : r.values()) {
ITestContext testContext = r2.getTestContext();
if (r.values().size() > 0) {
m_out.println("<h1>" + testContext.getName() + "</h1>");
}
resultDetail(testContext.getFailedConfigurations());
resultDetail(testContext.getFailedTests());
resultDetail(testContext.getSkippedConfigurations());
resultDetail(testContext.getSkippedTests());
resultDetail(testContext.getPassedTests());
}
}
}
/**
* #param tests
*/
private void resultSummary(ISuite suite, IResultMap tests, String testname,
String style, String details) {
if (tests.getAllResults().size() > 0) {
StringBuffer buff = new StringBuffer();
String lastClassName = "";
int mq = 0;
int cq = 0;
for (ITestNGMethod method : getMethodSet(tests, suite)) {
m_row += 1;
m_methodIndex += 1;
ITestClass testClass = method.getTestClass();
String className = testClass.getName();
if (mq == 0) {
String id = (m_testIndex == null ? null : "t"
+ Integer.toString(m_testIndex));
titleRow(testname + " — " + style + details, 5, id);
m_testIndex = null;
}
if (!className.equalsIgnoreCase(lastClassName)) {
if (mq > 0) {
cq += 1;
m_out.print("<tr class=\"" + style
+ (cq % 2 == 0 ? "even" : "odd") + "\">"
+ "<td");
if (mq > 1) {
m_out.print(" rowspan=\"" + mq + "\"");
}
m_out.println(">" + lastClassName + "</td>" + buff);
}
mq = 0;
buff.setLength(0);
lastClassName = className;
}
Set<ITestResult> resultSet = tests.getResults(method);
long end = Long.MIN_VALUE;
long start = Long.MAX_VALUE;
for (ITestResult testResult : tests.getResults(method)) {
if (testResult.getEndMillis() > end) {
end = testResult.getEndMillis();
}
if (testResult.getStartMillis() < start) {
start = testResult.getStartMillis();
}
}
mq += 1;
if (mq > 1) {
buff.append("<tr class=\"" + style
+ (cq % 2 == 0 ? "odd" : "even") + "\">");
}
String description = method.getDescription();
String testInstanceName = resultSet
.toArray(new ITestResult[] {})[0].getTestName();
buff.append("<td><a href=\"#m"
+ m_methodIndex
+ "\">"
+ qualifiedName(method)
+ " "
+ (description != null && description.length() > 0 ? "(\""
+ description + "\")"
: "")
+ "</a>"
+ (null == testInstanceName ? "" : "<br>("
+ testInstanceName + ")") + "</td>"
+ "<td class=\"numi\">" + resultSet.size() + "</td>"
+ "<td>" + start + "</td>" + "<td class=\"numi\">"
+ (end - start) + "</td>" + "</tr>");
}
if (mq > 0) {
cq += 1;
m_out.print("<tr class=\"" + style
+ (cq % 2 == 0 ? "even" : "odd") + "\">" + "<td");
if (mq > 1) {
m_out.print(" rowspan=\"" + mq + "\"");
}
m_out.println(">" + lastClassName + "</td>" + buff);
}
}
}
/** Starts and defines columns result summary table */
private void startResultSummaryTable(String style) {
tableStart(style, "summary");
m_out.println("<tr><th>Class</th>"
+ "<th>Method</th><th># of<br/>Scenarios</th><th>Start</th><th>Time<br/>(ms)</th></tr>");
m_row = 0;
}
private String qualifiedName(ITestNGMethod method) {
StringBuilder addon = new StringBuilder();
String[] groups = method.getGroups();
int length = groups.length;
if (length > 0 && !"basic".equalsIgnoreCase(groups[0])) {
addon.append("(");
for (int i = 0; i < length; i++) {
if (i > 0) {
addon.append(", ");
}
addon.append(groups[i]);
}
addon.append(")");
}
return "<b>" + method.getMethodName() + "</b> " + addon;
}
private void resultDetail(IResultMap tests) {
for (ITestResult result : tests.getAllResults()) {
ITestNGMethod method = result.getMethod();
m_methodIndex++;
String cname = method.getTestClass().getName();
m_out.println("<h2 id=\"m" + m_methodIndex + "\">" + cname + ":"
+ method.getMethodName() + "</h2>");
Set<ITestResult> resultSet = tests.getResults(method);
generateForResult(result, method, resultSet.size());
m_out.println("<p class=\"totop\">back to summary</p>");
}
}
/**
* Write the first line of the stack trace
*
* #param tests
*/
private void getShortException(IResultMap tests) {
for (ITestResult result : tests.getAllResults()) {
m_methodIndex++;
Throwable exception = result.getThrowable();
List<String> msgs = Reporter.getOutput(result);
boolean hasReporterOutput = msgs.size() > 0;
boolean hasThrowable = exception != null;
if (hasThrowable) {
boolean wantsMinimalOutput = result.getStatus() == ITestResult.SUCCESS;
if (hasReporterOutput) {
m_out.print("<h3>"
+ (wantsMinimalOutput ? "Expected Exception"
: "Failure") + "</h3>");
}
// Getting first line of the stack trace
String str = Utils.stackTrace(exception, true)[0];
scanner = new Scanner(str);
String firstLine = scanner.nextLine();
m_out.println(firstLine);
}
}
}
/**
* Write all parameters
*
* #param tests
*/
private void getParameters(IResultMap tests) {
for (ITestResult result : tests.getAllResults()) {
m_methodIndex++;
Object[] parameters = result.getParameters();
boolean hasParameters = parameters != null && parameters.length > 0;
if (hasParameters) {
for (Object p : parameters) {
m_out.println(Utils.escapeHtml(Utils.toString(p)) + " | ");
}
}
}
}
private void generateForResult(ITestResult ans, ITestNGMethod method,
int resultSetSize) {
Object[] parameters = ans.getParameters();
boolean hasParameters = parameters != null && parameters.length > 0;
if (hasParameters) {
tableStart("result", null);
m_out.print("<tr class=\"param\">");
for (int x = 1; x <= parameters.length; x++) {
m_out.print("<th>Param." + x + "</th>");
}
m_out.println("</tr>");
m_out.print("<tr class=\"param stripe\">");
for (Object p : parameters) {
m_out.println("<td>" + Utils.escapeHtml(Utils.toString(p))
+ "</td>");
}
m_out.println("</tr>");
}
List<String> msgs = Reporter.getOutput(ans);
boolean hasReporterOutput = msgs.size() > 0;
Throwable exception = ans.getThrowable();
boolean hasThrowable = exception != null;
if (hasReporterOutput || hasThrowable) {
if (hasParameters) {
m_out.print("<tr><td");
if (parameters.length > 1) {
m_out.print(" colspan=\"" + parameters.length + "\"");
}
m_out.println(">");
} else {
m_out.println("<div>");
}
if (hasReporterOutput) {
if (hasThrowable) {
m_out.println("<h3>Test Messages</h3>");
}
for (String line : msgs) {
m_out.println(line + "<br/>");
}
}
if (hasThrowable) {
boolean wantsMinimalOutput = ans.getStatus() == ITestResult.SUCCESS;
if (hasReporterOutput) {
m_out.println("<h3>"
+ (wantsMinimalOutput ? "Expected Exception"
: "Failure") + "</h3>");
}
generateExceptionReport(exception, method);
}
if (hasParameters) {
m_out.println("</td></tr>");
} else {
m_out.println("</div>");
}
}
if (hasParameters) {
m_out.println("</table>");
}
}
protected void generateExceptionReport(Throwable exception,
ITestNGMethod method) {
m_out.print("<div class=\"stacktrace\">");
m_out.print(Utils.stackTrace(exception, true)[0]);
m_out.println("</div>");
}
/**
* Since the methods will be sorted chronologically, we want to return the
* ITestNGMethod from the invoked methods.
*/
private Collection<ITestNGMethod> getMethodSet(IResultMap tests,
ISuite suite) {
List<IInvokedMethod> r = Lists.newArrayList();
List<IInvokedMethod> invokedMethods = suite.getAllInvokedMethods();
for (IInvokedMethod im : invokedMethods) {
if (tests.getAllMethods().contains(im.getTestMethod())) {
r.add(im);
}
}
Arrays.sort(r.toArray(new IInvokedMethod[r.size()]), new TestSorter());
List<ITestNGMethod> result = Lists.newArrayList();
// Add all the invoked methods
for (IInvokedMethod m : r) {
result.add(m.getTestMethod());
}
// Add all the methods that weren't invoked (e.g. skipped) that we
// haven't added yet
for (ITestNGMethod m : tests.getAllMethods()) {
if (!result.contains(m)) {
result.add(m);
}
}
return result;
}
#SuppressWarnings("unused")
public void generateSuiteSummaryReport(List<ISuite> suites) {
tableStart("testOverview", null);
m_out.print("<tr>");
tableColumnStart("Test");
tableColumnStart("Methods<br/>Passed");
tableColumnStart("Scenarios<br/>Passed");
tableColumnStart("# skipped");
tableColumnStart("# failed");
tableColumnStart("Error messages");
tableColumnStart("Parameters");
tableColumnStart("Start<br/>Time");
tableColumnStart("End<br/>Time");
tableColumnStart("Total<br/>Time");
tableColumnStart("Included<br/>Groups");
tableColumnStart("Excluded<br/>Groups");
m_out.println("</tr>");
NumberFormat formatter = new DecimalFormat("#,##0.0");
int qty_tests = 0;
int qty_pass_m = 0;
int qty_pass_s = 0;
int qty_skip = 0;
int qty_fail = 0;
long time_start = Long.MAX_VALUE;
long time_end = Long.MIN_VALUE;
m_testIndex = 1;
for (ISuite suite : suites) {
if (suites.size() > 1) {
titleRow(suite.getName(), 8);
}
Map<String, ISuiteResult> tests = suite.getResults();
for (ISuiteResult r : tests.values()) {
qty_tests += 1;
ITestContext overview = r.getTestContext();
startSummaryRow(overview.getName());
int q = getMethodSet(overview.getPassedTests(), suite).size();
qty_pass_m += q;
summaryCell(q, Integer.MAX_VALUE);
q = overview.getPassedTests().size();
qty_pass_s += q;
summaryCell(q, Integer.MAX_VALUE);
q = getMethodSet(overview.getSkippedTests(), suite).size();
qty_skip += q;
summaryCell(q, 0);
q = getMethodSet(overview.getFailedTests(), suite).size();
qty_fail += q;
summaryCell(q, 0);
// NEW
// Insert error found
m_out.print("<td class=\"numi" + (true ? "" : "_attn") + "\">");
getShortException(overview.getFailedTests());
getShortException(overview.getSkippedTests());
m_out.println("</td>");
// NEW
// Add parameters for each test case (failed or passed)
m_out.print("<td class=\"numi" + (true ? "" : "_attn") + "\">");
// Write OS and Browser
// m_out.println(suite.getParameter("os").substring(0, 3) +
// " | "
// + suite.getParameter("browser").substring(0, 3) + " | ");
getParameters(overview.getFailedTests());
getParameters(overview.getPassedTests());
getParameters(overview.getSkippedTests());
m_out.println("</td>");
// NEW
summaryCell(
DateFunctions.dateToDayAndTime(overview.getStartDate()),
true);
m_out.println("</td>");
summaryCell(
DateFunctions.dateToDayAndTime(overview.getEndDate()),
true);
m_out.println("</td>");
time_start = Math.min(overview.getStartDate().getTime(),
time_start);
time_end = Math.max(overview.getEndDate().getTime(), time_end);
summaryCell(
formatter.format((overview.getEndDate().getTime() - overview
.getStartDate().getTime()) / 1000.)
+ " seconds", true);
summaryCell(overview.getIncludedGroups());
summaryCell(overview.getExcludedGroups());
m_out.println("</tr>");
m_testIndex++;
}
}
if (qty_tests > 1) {
m_out.println("<tr class=\"total\"><td>Total</td>");
summaryCell(qty_pass_m, Integer.MAX_VALUE);
summaryCell(qty_pass_s, Integer.MAX_VALUE);
summaryCell(qty_skip, 0);
summaryCell(qty_fail, 0);
summaryCell(" ", true);
summaryCell(" ", true);
summaryCell(" ", true);
summaryCell(" ", true);
summaryCell(
formatter.format(((time_end - time_start) / 1000.) / 60.)
+ " minutes", true);
m_out.println("<td colspan=\"3\"> </td></tr>");
}
m_out.println("</table>");
}
private void summaryCell(String[] val) {
StringBuffer b = new StringBuffer();
for (String v : val) {
b.append(v + " ");
}
summaryCell(b.toString(), true);
}
private void summaryCell(String v, boolean isgood) {
m_out.print("<td class=\"numi" + (isgood ? "" : "_attn") + "\">" + v
+ "</td>");
}
private void startSummaryRow(String label) {
m_row += 1;
m_out.print("<tr"
+ (m_row % 2 == 0 ? " class=\"stripe\"" : "")
+ "><td style=\"text-align:left;padding-right:2em\"><a href=\"#t"
+ m_testIndex + "\">" + label + "</a>" + "</td>");
}
private void summaryCell(int v, int maxexpected) {
summaryCell(String.valueOf(v), v <= maxexpected);
}
private void tableStart(String cssclass, String id) {
m_out.println("<table cellspacing=\"0\" cellpadding=\"0\""
+ (cssclass != null ? " class=\"" + cssclass + "\""
: " style=\"padding-bottom:2em\"")
+ (id != null ? " id=\"" + id + "\"" : "") + ">");
m_row = 0;
}
private void tableColumnStart(String label) {
m_out.print("<th>" + label + "</th>");
}
private void titleRow(String label, int cq) {
titleRow(label, cq, null);
}
private void titleRow(String label, int cq, String id) {
m_out.print("<tr");
if (id != null) {
m_out.print(" id=\"" + id + "\"");
}
m_out.println("><th colspan=\"" + cq + "\">" + label + "</th></tr>");
m_row = 0;
}
/** Starts HTML stream */
protected void startHtml(PrintWriter out) {
out.println("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\" \"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">");
out.println("<html xmlns=\"http://www.w3.org/1999/xhtml\">");
out.println("<head>");
out.println("<title>Hector Flores - TestNG Report</title>");
out.println("<style type=\"text/css\">");
out.println("table {margin-bottom:10px;border-collapse:collapse;empty-cells:show}");
out.println("td,th {border:1px solid #009;padding:.25em .5em}");
out.println(".result th {vertical-align:bottom}");
out.println(".param th {padding-left:1em;padding-right:1em}");
out.println(".param td {padding-left:.5em;padding-right:2em}");
out.println(".stripe td,.stripe th {background-color: #E6EBF9}");
out.println(".numi,.numi_attn {text-align:right}");
out.println(".total td {font-weight:bold}");
out.println(".passedodd td {background-color: #0A0}");
out.println(".passedeven td {background-color: #3F3}");
out.println(".skippedodd td {background-color: #CCC}");
out.println(".skippedodd td {background-color: #DDD}");
out.println(".failedodd td,.numi_attn {background-color: #F33}");
out.println(".failedeven td,.stripe .numi_attn {background-color: #D00}");
out.println(".stacktrace {white-space:pre;font-family:monospace}");
out.println(".totop {font-size:85%;text-align:center;border-bottom:2px solid #000}");
out.println("</style>");
out.println("</head>");
out.println("<body>");
}
/** Finishes HTML stream */
protected void endHtml(PrintWriter out) {
out.println("<center> Report customized by Hector Flores [hectorfb#gmail.com] </center>");
out.println("</body></html>");
}
// ~ Inner Classes --------------------------------------------------------
/** Arranges methods by classname and method name */
private class TestSorter implements Comparator<IInvokedMethod> {
// ~ Methods
// -------------------------------------------------------------
/** Arranges methods by classname and method name */
#Override
public int compare(IInvokedMethod o1, IInvokedMethod o2) {
// System.out.println("Comparing " + o1.getMethodName() + " " +
// o1.getDate()
// + " and " + o2.getMethodName() + " " + o2.getDate());
return (int) (o1.getDate() - o2.getDate());
// int r = ((T) o1).getTestClass().getName().compareTo(((T)
// o2).getTestClass().getName());
// if (r == 0) {
// r = ((T) o1).getMethodName().compareTo(((T) o2).getMethodName());
// }
// return r;
}
}
}
With those steps you already have your listener ready to listen.
How to call it?
If you use testng.xml add the following lines:
<listeners>
<listener class-name='[your_class_path].MyReporterListener'/>
</listeners>
If you run your tests from a java class, add the following lines:
private final static MyReporterListener frl = new MyReporterListener();
TestNG testng = new TestNG();
testng.addListener(frl);
With those steps, when you execute your tests you'll have two emailable reports, customized and original.
Now it's time to pimp your report.
In my case I had to add error messages, parameters and times (start and end), because it's very useful if you want to paste on an excel file.
My customized report:
You have to mainly modify generateSuiteSummaryReport(List suites) function.
Play with that and ask me if you have any problem.
Make a you CSS and embed it in EmailableReporter.class. This class file can be found in TestNG folder> org.testNg.reporters> EmailableReporter.class.
There you can edit the Style of the TestNG report HTML file which starts with
protected void startHtml(PrintWriter out)
Better you can use extent report html reporting library jar file. However i am using extentreport1.4.jar jar file. So you will get summary details at right corner of your report
Using the above code, I am facing the below null pointer issue, HTML report works fine when fewer tests are included in testng.xml, but when I include more tests, the HTML report does not get generated and the below exception is thrown
[TestNG] Reporter report.MyListener#60e07aed failed
java.lang.NullPointerException: Cannot invoke "String.length()" because "s" is null
at java.base/java.io.StringReader.<init>(StringReader.java:51)
at java.base/java.util.Scanner.<init>(Scanner.java:766)
at report.MyListener.getShortException(MyListener.java:294)
at report.MyListener.generateSuiteSummaryReport(MyListener.java:473)
at report.MyListener.generateReport(MyListener.java:72)
at org.testng.TestNG.generateReports(TestNG.java:1093)
at org.testng.TestNG.run(TestNG.java:1036)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)
To fix the above added a null check to avoid Null Pointer at line 295
if(str!=null) {
scanner = new Scanner(str);
String firstLine = scanner.nextLine()+"<br>";
m_out.println(firstLine);
}