I have a spark dataset that needs to be sent as an attachment to an email.
I found the below link but it explains things in scala. Can someone help me with java code for sending the dataset as an attachment to an email?
sending-spark-dataframe-via-mail
Dataset<Row> finaldataset-> This is my dataset
Thanks in advance,
private void ConstructDataForEmail(Dataset<Row> datasetForEmail) {
try {
// Saving Dataset into CSV file into a single partition
datasetForEmail.repartition(1).write().
format("com.databricks.spark.csv").mode(SaveMode.Overwrite).option("header", true).save("/tmp/output");
//Getting Path of saved CSV file, as file is saved with name like part-0000-asgditda-yewd-yea-......csv
Optional<File> file = Arrays.stream(FileUtil.listFiles(new File("/tmp/output")).filter(fl -> fl.getName().endsWith(".csv")).findFirst();
String filePath = null;
if (file.isPresent())
filePath = file.get().getPath();
String subject = "Subject of the Email";
String emailBody = "Body for the Email";
String fileName = "Meaningful name to attached File";
//Method to send the email
sendMail(subject, emailBody, filePath, fileName);
} catch (Exception exp) {
LOG.info(" Error while preparing the data to send", exp);
}
}
Related
Please suggest me how to achieved this:
The other table having set of data which I need to send:
Now by using SSIS we have to send the set of data to that email. In production I'm having 135 emails and have send them a separate data which contain their set of area.
#pp#gmial.cpm will get only data where subject is maths.
I have no idea to create variable of how to pass variable.
Can any one help me with this?
I use this method a lot for sending emails:
public static void SendEmail(List<string> to, string subject, string body, string filepathName = null)
{
using (MailMessage mail = new MailMessage())
{
string from = ConfigurationManager.AppSettings["EmailFrom"];
SmtpClient SmtpServer = new SmtpClient("your email server");
mail.From = new MailAddress(from);
foreach(string t in to)
mail.To.Add(new MailAddress(t));
mail.Subject = subject;
mail.ReplyToList.Add("your reply to email");
mail.IsBodyHtml = true;
mail.Body = body;
mail.Bcc.Add("enter your email if you want to be copied");
if (filepathName != null)
{
Attachment file;
file = new Attachment(filepathName);
mail.Attachments.Add(file);
}
SmtpServer.Port = 2525; //this is unique to you
SmtpServer.Credentials = new NetworkCredential(from, ConfigurationManager.AppSettings["EmailPassword"]);
SmtpServer.EnableSsl = true;
SmtpServer.Send(mail);
}
}
This won't completely solve your problem but will give you option of sending HTML emails straight from SSIS Script task.
You won't be able to use configuration manager, you will need to "hard code" that.
I am using org.apache.commons.net.ftp.FTPClient for retrieving files from a ftp server. It is crucial that I preserve the last modified timestamp on the file when its saved on my machine. Do anyone have a suggestion for how to solve this?
This is how I solved it:
public boolean retrieveFile(String path, String filename, long lastModified) throws IOException {
File localFile = new File(path + "/" + filename);
OutputStream outputStream = new FileOutputStream(localFile);
boolean success = client.retrieveFile(filename, outputStream);
outputStream.close();
localFile.setLastModified(lastModified);
return success;
}
I wish the Apache-team would implement this feature.
This is how you can use it:
List<FTPFile> ftpFiles = Arrays.asList(client.listFiles());
for(FTPFile file : ftpFiles) {
retrieveFile("/tmp", file.getName(), file.getTimestamp().getTime());
}
You can modify the timestamp after downloading the file.
The timestamp can be retrieved through the LIST command, or the (non standard) MDTM command.
You can see here how to do modify the time stamp: that: http://www.mkyong.com/java/how-to-change-the-file-last-modified-date-in-java/
When download list of files, like all files returned by by FTPClient.mlistDir or FTPClient.listFiles, use the timestamp returned with the listing to update timestemp of local downloaded files:
String remotePath = "/remote/path";
String localPath = "C:\\local\\path";
FTPFile[] remoteFiles = ftpClient.mlistDir(remotePath);
for (FTPFile remoteFile : remoteFiles) {
File localFile = new File(localPath + "\\" + remoteFile.getName());
OutputStream outputStream = new BufferedOutputStream(new FileOutputStream(localFile));
if (ftpClient.retrieveFile(remotePath + "/" + remoteFile.getName(), outputStream))
{
System.out.println("File " + remoteFile.getName() + " downloaded successfully.");
}
outputStream.close();
localFile.setLastModified(remoteFile.getTimestamp().getTimeInMillis());
}
When downloading a single specific file only, use FTPClient.mdtmFile to retrieve the remote file timestamp and update timestamp of the downloaded local file accordingly:
File localFile = new File("C:\\local\\path\\file.zip");
FTPFile remoteFile = ftpClient.mdtmFile("/remote/path/file.zip");
if (remoteFile != null)
{
OutputStream outputStream = new BufferedOutputStream(new FileOutputStream(localFile));
if (ftpClient.retrieveFile(remoteFile.getName(), outputStream))
{
System.out.println("File downloaded successfully.");
}
outputStream.close();
localFile.setLastModified(remoteFile.getTimestamp().getTimeInMillis());
}
We are using the MultipartFormDataStreamProviderto save file upload by clients. I have a hard requirement that file size must be greater than 1KB. The easiest thing to do would of course be the save the file to disk and then look at the file unfortunately i can't do it like this. After i save the file to disk i don't have the ability to access it so i need to look at the file before its saved to disk. I've been looking at the properties of the stream provider to try to figure out what the size of the file is but unfortunately i've been unsuccessful.
The test file i'm using is 1025 bytes.
MultipartFormDataStreamProvider.BufferSize is 4096
Headers.ContentDisposition.Size is null
ContentLength is null
Is there a way to determine file size before it's saved to the file system?
Thanks to Guanxi i was able to formulate a solution. I used his code in the link as the basis i just added a little more async/await goodness :). I wanted to add the solution just in case it helps anyone else:
private async Task SaveMultipartStreamToDisk(Guid guid, string fullPath)
{
var user = HttpContext.Current.User.Identity.Name;
var multipartMemoryStreamProvider = await Request.Content.ReadAsMultipartAsync();
foreach (var content in multipartMemoryStreamProvider.Contents)
{
using (content)
{
if (content.Headers.ContentDisposition.FileName != null)
{
var existingFileName = content.Headers.ContentDisposition.FileName.Replace("\"", string.Empty);
Log.Information("Original File name was {OriginalFileName}: {guid} {user}", existingFileName, guid,user);
using (var st = await content.ReadAsStreamAsync())
{
var ext = Path.GetExtension(existingFileName.Replace("\"", string.Empty));
List<string> validExtensions = new List<string>() { ".pdf", ".jpg", ".jpeg", ".png" };
//1024 = 1KB
if (st.Length > 1024 && validExtensions.Contains(ext, StringComparer.OrdinalIgnoreCase))
{
var newFileName = guid + ext;
using (var fs = new FileStream(Path.Combine(fullPath, newFileName), FileMode.Create))
{
await st.CopyToAsync(fs);
Log.Information("Completed writing {file}: {guid} {user}", Path.Combine(fullPath, newFileName), guid, HttpContext.Current.User.Identity.Name);
}
}
else
{
if (st.Length < 1025)
{
Log.Warning("File of length {FileLength} bytes was attempted to be uploaded: {guid} {user}",st.Length,guid,user);
}
else
{
Log.Warning("A file of type {FileType} was attempted to be uploaded: {guid} {user}", ext, guid,user);
}
var responseMessage = new HttpResponseMessage(HttpStatusCode.BadRequest)
{
Content =
st.Length < 1025
? new StringContent(
$"file of length {st.Length} does not meet our minumim file size requirements")
: new StringContent($"a file extension of {ext} is not an acceptable type")
};
throw new HttpResponseException(responseMessage);
}
}
}
}
}
You can also read the request contents without using MultipartFormDataStreamProvider. In that case all of the request contents (including files) would be in memory. I have given an example of how to do that at this link.
In this case you can read header for file size or read stream and check the file size. If it satisfy your criteria then only write it to desire location.
I am working on Adobe Echo sign,I have downloaded the sample code from their website, I am using this sample code for sendingdocument, it has some code missing in sendDocument method so I have changed it. It's giving SoapHeader Exception,with nothing in InnerException,
{"apiActionId=XHZI4WF4BV693YS"}
below is my code of sending document
public static void sendDocument(string apiKey, string fileName, string recipient)
{
ES = new EchoSignDocumentService16();
FileStream file = File.OpenRead(fileName);
secure.echosign.com.FileInfo[] fileInfos = new secure.echosign.com.FileInfo[1];
fileInfos[0] = new secure.echosign.com.FileInfo(fileName, null, file);
SenderInfo senderInfo = null;
string[] recipients = new string[1];
recipients[0] = recipient;
DocumentCreationInfo documentInfo = new DocumentCreationInfo(
recipients,
"Test from SOAP: " + fileName,
"This is neat.",
fileInfos,
SignatureType.ESIGN,
SignatureFlow.SENDER_SIGNATURE_NOT_REQUIRED
);
DocumentKey[] documentKeys;
senderInfo = new SenderInfo(recipient, "password", "APIKEY");
documentKeys = ES.sendDocument(apiKey, senderInfo, documentInfo);
Console.WriteLine("Document key is: " + documentKeys[0].documentKey);
}
its giving exception on this line
documentKeys = ES.sendDocument(apiKey, senderInfo, documentInfo);
Can anyone suggest some sample code of Adobe Echo Sign?
On the account page of your login there is an API log you can check. If you check the log entry for your request you may find more information there.
I can't see anything immediately wrong with your code however the EchoSign API guide says that the 'tos' field is deprecated and that the recipients field should be used instead. Helpfully this means you can't use the paramaterised constructor. Try creating your document creation info as such (this is C# but if you need Java it should be straightforward to figure out):
RecipientInfo[] recipientInfo = new RecipientInfo[1];
recipientInfo[0] = new RecipientInfo
{
email = "recipient",
role = RecipientRole.SIGNER,
roleSpecified = true
};
DocumentCreationInfo documentCreationInfo = new DocumentCreationInfo
{
recipients = recipientInfo,
name = "Test from SOAP: " + fileName,
message = "This is neat.",
fileInfos = fileInfos,
signatureType = SignatureType.ESIGN,
signatureFlow = SignatureFlow.SENDER_SIGNATURE_NOT_REQUIRED
};
Note that when using the recipientInfo array it seems that the roleSpecified field must be set to true. This little field tripped me up for ages and I was receiving errors similar to yours.
I am trying to upload a file via a form and then save in in SQL as a blob.
I already have my form working fine, my database is fully able to take the blob and I have a controller that take the file, saves it in a local directory:
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult FileUpload(int id, HttpPostedFileBase uploadFile)
{
//allowed types
string typesNonFormatted = "text/plain,application/msword,application/pdf,image/jpeg,image/png,image/gif";
string[] types = typesNonFormatted.Split(',');
//
//Starting security check
//checking file size
if (uploadFile.ContentLength == 0 && uploadFile.ContentLength > 10000000)
ViewData["StatusMsg"] = "Could not upload: File too big (max size 10mb) or error while transfering the file.";
//checking file type
else if(types.Contains(uploadFile.ContentType) == false)
ViewData["StatusMsg"] = "Could not upload: Illigal file type!<br/> Allowed types: images, Ms Word documents, PDF, plain text files.";
//Passed all security checks
else
{
string filePath = Path.Combine(HttpContext.Server.MapPath("../Uploads"),
Path.GetFileName(uploadFile.FileName)); //generating path
uploadFile.SaveAs(filePath); //saving file to final destination
ViewData["StatusMsg"] = "Uploaded: " + uploadFile.FileName + " (" + Convert.ToDecimal(uploadFile.ContentLength) / 1000 + " kb)";
//saving file to database
//
//MISSING
}
return View("FileUpload", null);
}
Now all I am missing is putting the file in the database. I could not find anything on the subject... I found some way to do it in a regular website but nothing in MVC2.
Any kind of help would be welcome!
Thank you.
This could help: http://byatool.com/mvc/asp-net-mvc-upload-image-to-database-and-show-image-dynamically-using-a-view/
Since you have HttpPostedFileBase in your controllers method, all you need to do is:
int length = uploadFile.ContentLength;
byte[] tempImage = new byte[length];
myDBObject.ContentType = uploadFile.ContentType;
uploadFile.InputStream.Read(tempImage, 0, length);
myDBObject.ActualImage = tempImage ;
HttpPostedFileBase has a InputStream property
Hope this helps.
Alright thanks to kheit, I finaly got it working. Here's the final solution, it might help someone out there.
This script method takes all the file from a directory and upload them to the database:
//upload all file from a directory to the database as blob
public void UploadFilesToDB(long UniqueId)
{
//directory path
string fileUnformatedPath = "../Uploads/" + UniqueId; //setting final path with unique id
//getting all files in directory ( if any)
string[] FileList = System.IO.Directory.GetFiles(HttpContext.Server.MapPath(fileUnformatedPath));
//for each file in direcotry
foreach (var file in FileList)
{
//extracting file from directory
System.IO.FileStream CurFile = System.IO.File.Open(file, System.IO.FileMode.Open);
long fileLenght = CurFile.Length;
//converting file to a byte array (byte[])
byte[] tempFile = new byte[fileLenght];
CurFile.Read(tempFile, 0, Convert.ToInt32(fileLenght));
//creating new attachment
IW_Attachment CurAttachment = new IW_Attachment();
CurAttachment.attachment_blob = tempFile; //setting actual file
string[] filedirlist = CurFile.Name.Split('\\');//setting file name
CurAttachment.attachment_name = filedirlist.ElementAt(filedirlist.Count() - 1);//setting file name
//uploadind attachment to database
SubmissionRepository.CreateAttachment(CurAttachment);
//deleting current file fromd directory
CurFile.Flush();
System.IO.File.Delete(file);
CurFile.Close();
}
//deleting directory , it should be empty by now
System.IO.Directory.Delete(HttpContext.Server.MapPath(fileUnformatedPath));
}
(By the way IW_Attachment is the name of one of my database table)