In an effort to keep us from being labeled spammers by major ISPs (in addition to SPF records, privacy policies, CANSPAM compliance and the like) - I wanted to limit the amount of mail we send out an hour.
Is this possible in W2K3 SMTP server? I was looking at outbound connection properties in the SMTP virtual server config screens...It's just not that clear if tinkering with those settings are going to do what I want.
In a nutshell, I'd love mail being sent by this server to queue up and send for example, 5,000 messages every 10 minutes or so.
Is this possible?
So, I found out what I wanted - in case others look for it.
I'm using the config settings for ASPNET to drop the mail into a different pickup directory on the server (rather than the default Queue directory). Then I can simply use a scheduled task to check the pickup directory every 30 minutes or so and copy N number of messages to the SMTP Queue directory...Where they will be picked up and mailed. The configuration section/settings is:
<system.net>
<mailSettings>
<smtp deliveryMethod="SpecifiedPickupDirectory">
<specifiedPickupDirectory pickupDirectoryLocation="C:\YOUR_CUSTOM_PICKUP_DIR" />
</smtp>
</mailSettings>
</system.net>
What are you suign to send the mail, As far as i know i dont think you can set the sending server, But if you have some code which puts the mail together you could just put the send function on a timer.
5000 email every 10 mins is still going to get you marked as SPAM though after a few days. as thats close to 1 million emails a day.
Related
So I'm trying to understand what practical problems Queues solve. By reading all the information from Google, I get the high-level.
Push message to Queue for processing at a later time
So I'm looking at an architecture from Company A and they have different use cases for Job Queueing like for example
chat messages
file conversion
searching
Heavy sql queries
Why process it at a later time?
Here's my best guess...
Let's say I have an application that can process 10 "things" at a time.
My application then maxes out it's processing capacity.
an 11th request came in so app puts it in the Queue for later processing
Assuming this is a valid Use Case, wouldn't adding more servers to process more "things" make sense? Is it because it's more costly to add more servers than employ a Queue and sacrifice response time a little bit?
Given my Use Case examples, what other problems would Queues solve for them?
Have you ever lined up at a bank when it is busy? You would have waited in a queue.
"But," you could say, "wouldn't adding more staff to process more customers make sense? Is it because it's more costly to add more staff than employ a Queue and sacrifice response time a little bit?"
That would be correct. It can be quite costly to staff a bank based on the peak number of customers who would arrive each day. It is cheaper to staff below this level and have some customers wait in a queue.
Also, the number of customers each day are not 100% predictable. A queue allows excess demand to wait without breaking the system.
Queues enable decoupling.
For example, imagine an online store where customers purchase an item. They select the item, provide a credit card number and click 'Purchase'. If the credit card is declined, the online store can immediately prompt them to re-enter the number. This interaction has to take place immediately while the customer is still online.
However, there is no need to have the customer wait while an invoice is generated, a record is added to the accounting system and inventory is pulled off the shelf. This can be decoupled from the ordering process. A good way to do it is to push the order into a queue, which can be handled by the next system.
If that 'next system' happens to be offline at the moment, there is no reason to cancel the whole sale. The transaction can be processed when the 'next system' comes back online. This is much better than failing the whole process just because one component (which is not required immediately) has a failure.
Bottom line: Queues are excellent. They enable better handling of failures. They makes things more resilient (just wait a few minutes and try again!). They should be used at all times when the process is compatible with a queuing architecture.
Let's do scenarios
Scenario 1 without queue:
you request an endpoint /blabla/do-eveything/
this request do
download an image from very slow FTP
e.g 1.5 sec (can error, retry ? add +X sec)
attach the image to an email
send an email (3 sec)
e.g 1 sec (can error, retry ? add +X sec)
confirmation received > store confirmation to a third company tracking stuff
e.g 1.5 (can error, retry ? add +X sec)
when tracking confirm, update your data from another third company for big data purpose
e.g 2 sec (can error, retry ? add +X sec)
... you get the idead
return the response e.g 11 sec later (this is to slow) or more or timeout when everything failed
End user said internet was faster 20 years ago, maybe I need to change my internet connection or change my 16 threads
Scenario 2 queue everything you can:
you request an endpoint /blabla/do-eveything/
this request do
Queue job "DO_EVERYTHING"
e.g 0.02 sec
Return the response less then 0.250 sec
End user said that is website/app is too fast, I can keep my 56K internet connection
on queue/event system one failed job can be retry later without affeting the end user
you can pause job, add a unlimited number a task/step after the original message
better fault tolerance
Working with queue will allow you a better micro/nano service architecture, better testing because, you can test a single job, intead of a full controller that do everything...
Ye, is maybe more work, more thinking, but a the end no need to think about the work when holidays
I need help to make Visual Basic not stop while sending emails.
I am using Smtp_Server.Send(e_mail) everytime it's sends an email the program stops for 5 seconds and then it's stop work for maybe 3 sec and then it sends the mail.
I want to be able to still use the program while it's sends the mail Help plz!
You can use some logging or careful observation in the debugger to confirm, but the conversation between your client and the SMTP server is probably taking 5 seconds (which is quite long, but plausible).
The solution is to use multiple threads to send email.
If you are sending batches of emails, try starting with 5-10 threads (with too many, the SMTP server on the other end might start rejecting some connection attempts).
If you are trying to send a single email at a time, but it is blocking your main application, you can use a single separate thread to perform the send.
Here is my application I'm attempting to put together using NServiceBus:
I have a 1000 files that need to be processed by a service. So far I'm thinking I'd have one endpoint, the client, find all of those files and send them out on the bus to be processed
My other endpoint, the server that does the processing, would listen for these client messages, when one comes in process the file, and return the results.
Client takes the results, marks the file as processed, and waits for the next 999 files to be processed. Client doesn't care the order of the messages that come back, just as long as they all get processed at some point. (In reality the client is going to do something more with the data after it is processed that can't be done by the server, so I can't just fire and forget the request for processing.)
Since processing a single message can take over an hour I would scale out the application to have multiple servers all attempting to eat through the 1000 files that need to be processed.
Conceptually, its like building a personal SETI at home service to run on all of my servers.
The issues I'm having is, how do I stop midway through processing the 1000 files?
I want to keep all of my servers working as much as they can on my data, so when the client starts does it publish a 1000 commands for the 1000 files to process and then sit back and wait? And if it does this, and decides to stop, how can it clear the bus of all of those commands to process files?
If my client only pushes one or two messages on the bus at a time I could easily stop sending messages if I decide to stop on the client, but then I have two other problems
The servers could be underutilized and I'd end up with idle servers.
How do I stop the servers that are loaded up and processing data? Send them a second command of a different message format?
Thoughts, ideas? Am I approaching this problem using the right tool/right methodology?
One of things you might want to think about is how you are going to correlate the message processing. I would use a saga for this and have the client generate some kind of batch id which is attached to all the files to be processed. This allows your client to be able to send a CancelProcessing message to the saga, the handler for which could then stop the processing / sending of messages to the file processing endpoints and perform any clean-up operations such as completing the saga and removing data from the database.
So you would have client endpoint, saga endpoint and one or more file processing endpoints (which would sit behind a distributor). Your client would be responsible for initiating / sending the files to the saga. The saga manages the file correlation and processing activities, while your processing endpoints focus doing the work.
Remember that the processing endpoints don't necessarily have to be physical endpoints. You can have many of these on one server if you wanted to and use monitoring tools to determine whether or not you need to add or remove nodes.
I've got a puzzling qmail situation (at least to me, but I am not a qmail expert, this is a work-related issue that I'm looking at by default...)
There are about a dozen emails sent to a particular address that have been in the qmail queue on our mailserver for 2 days. In the logs I see each delivery attempt ends with
delivery NNNN: deferral: Connected_to_XXX.XXX.X.XX_but_connection_died._Possible_duplicate!_(#4.4.2)/
All this time, literally thousands of other emails to the exact same address are being sent successfully. There's nothing I can see about the dozen failed messages that is different, although that is the only logical explanation I can think of.
The destination host (XXX.XXX.X.XX) is a machine on our own internal network.
Any suggestions on what I could check? Any way to get additional details on what happened to cause 'connection died'? E.g., perhaps the email server at the destination rejected the messages due to some particular content?
Any suggestions will be much appreciated.
it is possible that the mail is looping.
If the mail header contains "Delivered To:" line then qmail-local usually gives a deferral.
Also check the destination server for any logs, if multiple copies of same mail are been hit there for any random reasons.
I have a bunch of message on an outgoing MSMQ that have been addressed incorrectly due to a typo. They are stuck on the outgoing queue since the box they have been directed at does not exist.
The name of the queue is of the form
DIRECT=OS:foo\private$\MyQueue and is in state Waiting to connect
and I want to send to
DIRECT=OS:bar\private$\MyQueue
Will an entry in the hosts file suffice, or if not is there any other way that I can go this? I don't want to have to create a server called and a process that recieves the messages and forwards them on to the correct server if at all possible
I tried the idea I suggested above of adding the incorrect entry into the Hosts file with the correct IP address. This caused the messages to get dispatched to the remote computer, but I think WCF dropped them as they didn't have the correct endpoint adress.
Not recommended if you don't want to lose the messages! Fortunately for me these were not critical, so it was worth a try.
Oh well
Modan,
I experienced something similar years ago and the only way I could find was to setup the box and Q (in a VM) and forward them to correct queue. Maybe you can delete and resend the messages with the fix? Is that possible? Unfortunately don't think there's a good way around this.
-Bryan