I am using Amazon SES to send transactional application emails. I want an audit log of every email sent by the system.
As an example, I might want to see a log of every email we sent to john.smith#example.com.
I followed the instructions for using Cloudwatch to log SES events. However, this only actually logs metrics, not data logs. So all I see in Cloudwatch is a graph of how many emails were sent at different times under the metrics tab. I can't search a log anywhere to find individual SES events.
I also looked into using Cloudtrail to log SES events, but Cloudtrail only logs management events. It does not log data events like emails being sent.
I have setup SNS notifications on all SES events (such as send or bounce). This is really useful, but does not achieve my aim of having a long term audit log.
As far as I can tell Amazon do not support the kind of logging I want to record. Maybe I could write events to our application database as we produce them, but it seems a shame to have to introduce my own custom logging system.
Does anyone know a way to have AWS store my SES data events?
Yes, Amazon doesn't have any easy to monitor you ask, an easier solution would be to add a Configuration set header and a unique message tag whenever any email is being sent to john.smith#example.com, In configuration set, you can enable Cloudwatch or preferred SNS Destination and create delivery dashboard using link below:
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/bouncecomplaintdashboard.html
This isn't the logs but It creates a nice excel file of email details which can be used for Audit purpose.
I opened a ticket with AWS about this, here is their response.
I understand that you wanted to know whether SES provides logs about events of an email that was accepted by SES for delivery in a searchable format. Please correct me if I’m wrong.
SES provides logs as notification[1] for each event(Delivery, Bounce, Complaint) to SNS and it doesn’t provide event logs in a consolidated manner which would be helpful for indexing according to email addresses and searching purpose. However, in SNS you can configure a variety of subscriber like(Email, SQS, Lambda, HTTP endpoint) where the logs in JSON format would be delivered. In the destination of SNS like Lambda and HTTP Endpoint or email, you can parse the JSON file and store in a suitable format which would be helpful for indexing and searching purpose.
If SQS is being used, each JSON log will be stored individually in the SQS queue for some time. You can periodically query the SQS endpoint and retrieve the logs and consolidate it to a single file and use it for analysis.
Apart from that, unfortunately SES doesn’t provide any other format of logs for an email it accepted for delivery.
Do let me know if you require any further assistance, I’ll be happy to help.
References:
[1]. https://docs.aws.amazon.com/ses/latest/DeveloperGuide/monitor-sending-using-notifications.html
Related
I provide SES Services to many of my clients in different regions of the world. So their apps send out emails through my AWS SES account. It has become a problem to trace out how to measure their usage and stats, IAM wise or some other ways and then be able to bill them for what they have consumed. There must be a way to break stats into some kind of Sub Accounts or something
Can someone help regarding this ?
You have two options for such fine-grained monitoring, according to the official docs:
Feedback notifications
This option allows the most detailed reporting for your billing needs. However, it also requires more work on your part, because you need to build handlers (i.e. lambda functions) which will log sending/bounces/complaints in some persistence (i.e. DynamoDB table). More details here.
Event publishing
This option uses CloudWatch for persistence and you have the (probably) familiar CloudWatch UI for reporting. You will have to enforce (somehow) your clients to use a AWS specific SMTP header or SendEmail / SendRawEmail API calls. Less work on your part but you need to find a way to make your clients help you. More details here.
I have a web app hosted on aws under free tier limit. What I want to achieve is that whenever I received an email, i want the system to download its attachment(will be images only), upload that image to s3 and save the image id in database with email's sender email address. I don't want to use zapier api etc, I want to code it my self. How can I achieve this?
This really depends upon how your email is hosted.
You could use Receive Email with Amazon Simple Email Service.
The flow could then either be:
SES -> S3 -> Trigger Event -> AWS Lambda function, or
SES -> SNS -> AWS Lambda function
You would then need to write a Lambda function to do the processing you described.
If, on the other hand, your email is being hosted elsewhere, then you will need a mechanism to trigger some code when an email is received (somehow on your email system) or a scheduled Lambda function to poll the email system to see whether new mail is available.
Is it possible to get MailChimp or Mandrill to notify a webhook URL whenever a message is either created for a list, or scheduled to be sent, along with the list and message IDs?
I have a client that wants to intercept messages from his campaign, add special data from his server, then send the resulting template through his connected Mandrill account. I'm trying to figure out how to implement the first step in this process.
Although I know this is old someone may stumble into this thread, have you checked out the webhook information?
http://help.mandrill.com/entries/58303976-Message-Event-Webhook-format
It has an easy way to implement it inside the account. You just setup a URL to intercept and parse the incoming data. I recommend first saving the data then using a scheduled task to parse the job separately so you don't lose data (although mandrill will try 100x).
If the unique id is not enough for you with your events, and you are concerned about specific campaigns you can tag the emails upon send and they will have the tagged information with the incoming event.
I'm using Amazon SES/SNS API of Amazon for sending email and getting notification.
Is Amazon provide any scheduler Api for scheduling the email for future? As i want to schedule my email by using Amazon API.
No, SES does not provide this functionality. You would need to do that yourself using a task scheduler such as the built-in windows or perhaps quartz.net if you need something more sophisticated
While they may not offer it directly in SES or SNS, they do offer similar functionality with SQS or SWF.
I made the service, huhumails.com
With HuhuMails, you can schedule your transactional emails with ease. It also makes sure emails are sent within your rate-limits. The service accepts date and timezone parameters to schedule your email at a specific time in the future.
I couldn't find anything on the market that's according to what I'm looking for and easy to use, so I created the service. I'm currently using it on my other production websites. Hopefully, it will be useful to others too.
While the SES dashboard shows aggregated statistics about the bounce rate of emails sent through the service, I do not see a way of retrieving the individual addresses that bounced. Is this possible? Our situation is that the 'from' address we had set in certain emails was incorrect and resolved to a non existant mailbox on our (verfied sender) domain, so anything SES would have forwarded to the from address is likely gone.
Use the Amazon SNS (simple notification service), and then you can add your email address - or Amazon SQS service for holding a log of all bounces/complaints.
The answer is no, they are gone. Lesson: make sure you from address is valid (good practice obviously) and goes to a mailbox that resolves (and/or set up and process a SQS queue for them to go to)
I had the same problem. The SES report didn't show enough details for the accruing bounce error. I modified the sesreport.zip, where the deliveries, from-emails/source-emails, and the subject column are added and are included in the report.
You can find my modification here:
https://github.com/Morning-Train/AWS-SES-Report
I hope my answer helps you with your problem.