Appending data to existing CSV using Jackson CSV - jackson

How Can I append the data to existing CSV File using Jackson CSV (https://github.com/FasterXML/jackson-dataformats-text/tree/master/csv)
Here is my code
public synchronized Boolean AppendToFile(Path fullFilePath, InputDataFormat obj) throws IOException {
if (mapper == null || schema == null)
init();
File outputFile = new File(fullFilePath.toUri());
if (!outputFile.exists()) {
outputFile.createNewFile();
}
ObjectWriter writer = mapper.writer(schema);
mapper.writer(schema).writeValue(outputFile, obj);
return true;
}
It writes to the file but does not retain the existing data

Resolved using OutputStream and passing append flag as true
public Boolean AppendToFile(Path fullFilePath, Account obj) {
logger.info(" Acquired Write lock on file " + fullFilePath.getFileName());
mapper = new CsvMapper();
mapper.configure(JsonGenerator.Feature.IGNORE_UNKNOWN, true);
schema = mapper.schemaFor(Account.class).withColumnSeparator('|');
File outputFile = new File(fullFilePath.toUri());
if (!outputFile.exists()) {
outputFile.createNewFile();
}
ObjectWriter writer = mapper.writer(schema);
OutputStream outstream = new FileOutputStream(outputFile , true);
writer.writeValue(outstream,obj);
return true;
}

Related

autodesk design automation

FATAL ERROR: Unhandled Access Violation Reading 0x0008 Exception at 1d8257a5h
Failed missing output
I finally made it work with HostApplicationServices.getRemoteFile in local AutoCAD, then migrated it to Design Automation. It is also working now. The below is the command of .NET plugin.
To have a simple test, I hard-coded the URL in the plugin. you could replace the URL with the workflow at your side (either by an json file, or input argument of Design Automation)
My demo ReadDWG the entities from the remote URL file, then wblock the entities to current drawing (HostDWG), finally save current drawing.
Hope it helps to address the problem at your side.
.NET command
namespace PackageNetPlugin
{
class DumpDwgHostApp: HostApplicationServices
{
public override string FindFile(string fileName,
Database database,
FindFileHint hint)
{
throw new NotImplementedException();
}
public override string GetRemoteFile(Uri url,
bool ignoreCache)
{
//return base.GetRemoteFile(url, ignoreCache);
Database db =
Autodesk.AutoCAD.ApplicationServices.Application.
DocumentManager.MdiActiveDocument.Database;
string localPath = string.Empty;
if (ignoreCache)
{
localPath =
Autodesk.AutoCAD.ApplicationServices.Application.
GetSystemVariable("STARTINFOLDER") as string;
string filename =
System.IO.Path.GetFileName(url.LocalPath);
localPath += filename;
using (var client = new WebClient())
{
client.DownloadFile(url, localPath);
}
}
return localPath;
}
public override bool IsUrl(string filePath)
{
Uri uriResult;
bool result = Uri.TryCreate(filePath,
UriKind.Absolute, out uriResult)
&& (uriResult.Scheme == Uri.UriSchemeHttp ||
uriResult.Scheme == Uri.UriSchemeHttps);
return result;
}
}
public class Class1
{
[CommandMethod("MyPluginCommand")]
public void MyPluginCommand()
{
try {
string drawingPath =
#"https://s3-us-west-2.amazonaws.com/xiaodong-test-da/remoteurl.dwg";
DumpDwgHostApp oDDA = new DumpDwgHostApp();
string localFileStr = "";
if (oDDA.IsUrl(drawingPath)){
localFileStr = oDDA.GetRemoteFile(
new Uri(drawingPath), true);
}
if(!string.IsNullOrEmpty(localFileStr))
{
//source drawing from drawingPath
Database source_db = new Database(false, true);
source_db.ReadDwgFile(localFileStr,
FileOpenMode.OpenTryForReadShare, false, null);
ObjectIdCollection sourceIds =
new ObjectIdCollection();
using (Transaction tr =
source_db.TransactionManager.StartTransaction())
{
BlockTableRecord btr =
(BlockTableRecord)tr.GetObject(
SymbolUtilityServices.GetBlockModelSpaceId(source_db),
OpenMode.ForRead);
foreach (ObjectId id in btr)
{
sourceIds.Add(id);
}
tr.Commit();
}
//current drawing (main drawing working with workitem)
Document current_doc =
Autodesk.AutoCAD.ApplicationServices.Application.
DocumentManager.MdiActiveDocument;
Database current_db = current_doc.Database;
Editor ed = current_doc.Editor;
//copy the objects in source db to current db
using (Transaction tr =
current_doc.TransactionManager.StartTransaction())
{
IdMapping mapping = new IdMapping();
source_db.WblockCloneObjects(sourceIds,
SymbolUtilityServices.GetBlockModelSpaceId(current_db),
mapping, DuplicateRecordCloning.Replace, false);
tr.Commit();
}
}
}
catch(Autodesk.AutoCAD.Runtime.Exception ex)
{
Autodesk.AutoCAD.ApplicationServices.Application.
DocumentManager.MdiActiveDocument.Editor.WriteMessage(ex.ToString());
}
}
}
}

How to encrypt payload file streamingly via WSO2 ESB

I have to implement a scenario by using WSO2 ESB, as encrypting the binary payload streamingly while response to the client side (I assume the content-type in the case is Application/Octet-Stream), below is some details by my thought:
An Endpoint like "http://myhost/backend/" which provides business functionality;
A proxy which pass messages through the endpoint;
I attempt to write an OutSequence to check the Content-type: if the Content-Type matches Application/Octet-Stream, invoke my customized class mediator to encrypt the fileStream Streamingly and response.
I have no idea on how to write the class mediator to make it implemented? How could I get/read the file stream from the message as well as how to put the outputStream back to the response while I could only see mc.getEnvelope().getBody() in mediation method? Below is my current mediator which doesn't work.
public boolean mediate(MessageContext mc) {
org.apache.axis2.context.MessageContext amc = ((Axis2MessageContext) mc).getAxis2MessageContext();
try {
String contentID = amc.getAttachmentMap().getAllContentIDs()[0];
DataHandler dh = amc.getAttachment(contentID);
dh.getDataSource().getName();
InputStream is = null;
try {
is = dh.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String line = null;
while ((line = br.readLine()) != null) {
System.out.println("client read:" + line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
return true;
}
Many thanks if anybody with experience would kindly help.
Pasted my current solution for anyone else who confronts similar issue.
In the mediator, I read the file content from response stream via OMText.InputStream and use net.lingala.zip4j package to write a zip file(in memory) with the raw file encrypted; Finally I write the zip file content as ByteArray back to the OMElement of the soap message.
public boolean mediate(MessageContext mc) {
System.out.println("========================Mediator log start================================");
org.apache.axis2.context.MessageContext amc = ((Axis2MessageContext) mc).getAxis2MessageContext();
try {
#SuppressWarnings("unchecked")
Map<String, String> responseHeaders = (Map<String, String>) amc.getProperty("TRANSPORT_HEADERS");
String rawFileName = "";
String[] contentDisps = responseHeaders.get("Content-Disposition").split(";");
for (String item : contentDisps) {
System.out.println("item::" + item);
if (item.trim().startsWith(CONTENT_DISPOSITION_FILENAME)) {
rawFileName = item.substring(item.indexOf("\"") + 1, item.length() - 1);
break;
}
}
responseHeaders.put(
"Content-Disposition",
responseHeaders.get("Content-Disposition").replace(rawFileName,
rawFileName.substring(0, rawFileName.lastIndexOf(".")) + ".myzip"));
OMElement binaryPayload =
amc.getEnvelope().getBody()
.getFirstChildWithName(new QName("http://ws.apache.org/commons/ns/payload", "binary"));
OMText binaryNode = (OMText) binaryPayload.getFirstOMChild();
DataHandler dataHandler = (DataHandler) binaryNode.getDataHandler();
InputStream is = dataHandler.getInputStream();
ByteArrayOutputStream responseOutputStream = new ByteArrayOutputStream();
ZipOutputStream zipOutputStream = getZipOutputStreamInstance(responseOutputStream, rawFileName);
// write to zipOutputStream
byte data[] = new byte[BUFFER_SIZE];
int count;
while ((count = is.read(data, 0, BUFFER_SIZE)) != -1) {
zipOutputStream.write(data, 0, count);
zipOutputStream.flush();
}
zipOutputStream.closeEntry();
zipOutputStream.finish();
InputStream in = new ByteArrayInputStream(responseOutputStream.toByteArray());
DataHandler zipDataHandler = new DataHandler(new StreamingOnRequestDataSource(in));
OMFactory factory = OMAbstractFactory.getOMFactory();
OMText zipData = factory.createOMText(zipDataHandler, true);
zipData.setBinary(true);
binaryPayload.getFirstOMChild().detach();
binaryPayload.addChild(zipData);
amc.setProperty("TRANSPORT_HEADERS", responseHeaders);
System.out.println("========================Mediator end==================================");
} catch (Exception ex) {
System.out.println("exception found here:");
ex.printStackTrace();
}
return true;
}

inserting image into mongo from java result in a strange error

I have the following code for saving image in mongodb:
public static void insertImage() throws Exception {
String newFileName = "mkyong-java-image";
File imageFile = new File("c:\\JavaWebHosting.png");
GridFS gfsPhoto = new GridFS(db, "photo");
GridFSInputFile gfsFile = gfsPhoto.createFile(imageFile);
gfsFile.setFilename(newFileName);
gfsFile.save();
}
And I got this from this link:
link for code
But when I use that I get the following error and I do not know how to fix it ... Can anyone help?
Exception in thread "main" java.lang.NullPointerException
at com.mongodb.gridfs.GridFS.<init>(GridFS.java:97)
for more explanation the error is at exactly this line:
GridFS gfsPhoto = new GridFS(db, "photo");
Update:
Here is the code for creating db connection
public static DB getDBConnection() {
// If it's not connected to the database, make connection
if (db == null) {
initialize();
makeConnections();
}
return db;
}
private static void makeConnections() {
MongoCredential credential = MongoCredential.createMongoCRCredential(dbUser, dbName, dbPass.toCharArray());
MongoClient mongoClient;
try {
mongoClient = new MongoClient(new ServerAddress(dbHost, Integer.parseInt(dbPort)), Arrays.asList(credential));
db = mongoClient.getDB(dbName);
} catch (UnknownHostException e) {
e.printStackTrace();
}
}
Update:
String newFileName = "mkyong-java-image";
File imageFile = new File("D:/1.jpg");
db = MongoDB.getDBConnection();
collection = db.getCollection("test");
// create a "photo" namespace
GridFS gfsPhoto = new GridFS(db, "photo");
// get image file from local drive
GridFSInputFile gfsFile = gfsPhoto.createFile(imageFile);
// set a new filename for identify purpose
gfsFile.setFilename(newFileName);
// save the image file into mongoDB
gfsFile.save();
// print the result
DBCursor cursor = gfsPhoto.getFileList();
while (cursor.hasNext()) {
System.out.println(cursor.next());
}
// get image file by it's filename
GridFSDBFile imageForOutput = gfsPhoto.findOne(newFileName);
// save it into a new image file
imageForOutput.writeTo("D:\\JavaWebHostingNew.jpg");
// remove the image file from mongoDB
// gfsPhoto.remove(gfsPhoto.findOne(newFileName));
System.out.println("Done");

How to extract text from PDFs using a PIG UDF and Apache Tika?

I'm attempting to write a PIG eval function (UDF) to extract text from pdf files using Apache Tika. However, my function only writes 0 or 1 bytes to output whenever I try to run the function. How could I fix my code?
public class ExtractTextFromPDFs extends EvalFunc<String> {
#Override
public String exec(Tuple input) throws IOException {
String pdfText;
if (input == null || input.size() == 0 || input.get(0) == null) {
return "N/A";
}
DataByteArray dba = (DataByteArray)input.get(0);
InputStream is = new ByteArrayInputStream(dba.get());
ContentHandler contenthandler = new BodyContentHandler();
Metadata metadata = new Metadata();
Parser pdfparser = new AutoDetectParser();
try {
pdfparser.parse(is, contenthandler, metadata, new ParseContext());
} catch (SAXException | TikaException e) {
e.printStackTrace();
}
pdfText = contenthandler.toString();
//close the input stream
if(is != null){
is.close();
}
return pdfText;
}
}
I run the code using 'c = foreach b generate ExtractTextFromPDFs(content);' where b is a pdf and content is a bytearray.

excel file upload using apache file upload

I am developing an testing automation tool in linux system. I dont have write permissions for tomcat directory which is located on server. I need to develop an application where we can select an excel file so that the excel content is automatically stored in already existing table.
For this pupose i have written an form to select an file which is posted to a servlet CommonsFileUploadServlet where i am storing the uploaded file and then calling ReadExcelFile class which reads the file path and create a vector for data in file which is used to sstore data in database.
My problem is that i am not able to store the uploaded file in directory. Is it necessary to have permission rights for tomcat to do this. Can i store the file on my system and pass the path to ReadExcelFile.class
Please guide me
My code is as follows:
Form in jsp
CommonsFileUploadServlet class code:
public void init(ServletConfig config) throws ServletException {
super.init(config);
}
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
PrintWriter out = response.getWriter();
response.setContentType("text/plain");
out.println("<h1>Servlet File Upload Example using Commons File Upload</h1>");
DiskFileItemFactory fileItemFactory = new DiskFileItemFactory ();
fileItemFactory.setSizeThreshold(1*1024*1024);
fileItemFactory.setRepository(new File("/home/example/Documents/Project/WEB-INF/tmp"));
ServletFileUpload uploadHandler = new ServletFileUpload(fileItemFactory);
try {
List items = uploadHandler.parseRequest(request);
Iterator itr = items.iterator();
while(itr.hasNext()) {
FileItem item = (FileItem) itr.next();
if(item.isFormField()) {
out.println("File Name = "+item.getFieldName()+", Value = "+item.getString());
} else {
out.println("Field Name = "+item.getFieldName()+
", File Name = "+item.getName()+
", Content type = "+item.getContentType()+
", File Size = "+item.getSize());
File file = new File("/",item.getName());
String realPath = getServletContext().getRealPath("/")+"/"+item.getName();
item.write(file);
ReadExcelFile ref= new ReadExcelFile();
String res=ref.insertReq(realPath,"1");
}
out.close();
}
}catch(FileUploadException ex) {
log("Error encountered while parsing the request",ex);
} catch(Exception ex) {
log("Error encountered while uploading file",ex);
}
}
}
ReadExcelFile code:
public static String insertReq(String fileName,String sno) {
//Read an Excel File and Store in a Vector
Vector dataHolder=readExcelFile(fileName,sno);
//store the data to database
storeCellDataToDatabase(dataHolder);
}
public static Vector readExcelFile(String fileName,String Sno)
{
/** --Define a Vector
--Holds Vectors Of Cells
*/
Vector cellVectorHolder = new Vector();
try{
/** Creating Input Stream**/
//InputStream myInput= ReadExcelFile.class.getResourceAsStream( fileName );
FileInputStream myInput = new FileInputStream(fileName);
/** Create a POIFSFileSystem object**/
POIFSFileSystem myFileSystem = new POIFSFileSystem(myInput);
/** Create a workbook using the File System**/
HSSFWorkbook myWorkBook = new HSSFWorkbook(myFileSystem);
int s=Integer.valueOf(Sno);
/** Get the first sheet from workbook**/
HSSFSheet mySheet = myWorkBook.getSheetAt(s);
/** We now need something to iterate through the cells.**/
Iterator rowIter = mySheet.rowIterator();
while(rowIter.hasNext())
{
HSSFRow myRow = (HSSFRow) rowIter.next();
Iterator cellIter = myRow.cellIterator();
Vector cellStoreVector=new Vector();
short minColIndex = myRow.getFirstCellNum();
short maxColIndex = myRow.getLastCellNum();
for(short colIndex = minColIndex; colIndex < maxColIndex; colIndex++)
{
HSSFCell myCell = myRow.getCell(colIndex);
if(myCell == null)
{
cellStoreVector.addElement(myCell);
}
else
{
cellStoreVector.addElement(myCell);
}
}
cellVectorHolder.addElement(cellStoreVector);
}
}catch (Exception e){e.printStackTrace(); }
return cellVectorHolder;
}
private static void storeCellDataToDatabase(Vector dataHolder)
{
Connection conn;
Statement stmt;
String query;
try
{
// get connection and declare statement
int z;
for (int i=1;i<dataHolder.size(); i++)
{
z=0;
Vector cellStoreVector=(Vector)dataHolder.elementAt(i);
String []stringCellValue=new String[10];
for (int j=0; j < cellStoreVector.size();j++,z++)
{
HSSFCell myCell = (HSSFCell)cellStoreVector.elementAt(j);
if(myCell==null)
stringCellValue[z]=" ";
else
stringCellValue[z] = myCell.toString();
}
try
{
//inserting into database
}
catch(Exception error)
{
String e="Error"+error;
System.out.println(e);
}
}
stmt.close();
conn.close();
System.out.println("success");
}
catch(Exception error)
{
String e="Error"+error;
System.out.println(e);
}
}
POI will happily open from an old InputStream, it needn't be a File one.
I'd suggest you look at the Commons FileUpload Streaming API and consider just passing the excel part straight to POI without touching the disk