I am looking for a load balancer that will direct load based on the user.
For example, I have a REST API that supports 3 different document types and 10 servers, each document type can not take up more than 5 servers, or another option where each document type is given 3 servers each. I don't want to lose the requests, but instead, queue them if possible.
I am pretty sure a load balancer like this already exists, but I can not find the name of it/an implementation. Or even better is there an AWS ELB that is already capable of handling this?
You can put a Web Application Firewall in front of an Application Load Balancer and then use a rule like this to block traffic over the specified 5 minute rule.
{
"Priority": 0,
"Action": {
"Count": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": { "Fn::Sub": "${Site}-Overall-Rate-Limit" }
},
"Name": { "Fn::Sub": "${Site}-Overall-Rate-Limit" },
"Statement": {
"RateBasedStatement": {
"Limit": { "Ref": "OverallRateLimit" },
"AggregateKeyType": "IP"
}
}
}
Note that WAF is not free. You pay for the ACL and for the rule evaluation.
Another option here is to use a CDN like cloudfront to offload the content delivery to your users. It's pretty much impossible to DOS a CDN like cloudfront. Whether this method is appropriate for you depends on whether your content is static and shared or dynamic and unique to clients.
Having read your edits, I'll offer you another path. If your goal is to route different traffic to different servers, you can do that with ALB Listener Rules. https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-update-rules.html
You would have to give the users a header to route them to the correct Target Group on the backend.
You could use sticky sessions to keep them on a specific server beyond that, but that can have its own implications.
Related
In my application I maintain couple of Shapes. And these shapes are calculated with respect to some reference point. And this reference point is considered as (0, 0). For example someone wants to move this reference point by (x, y), then distance of all existing Shapes need to be re-calculated with new reference point.
I have an REST API, to change this reference point, so internally all the Shapes distance is re-calculated with that (x, y) movement.
What HTTP method should be used for such operation POST, PUT or PATCH? As I don't find any one of these correct in the context, if I go by the definition of these HTTP method.
What HTTP method should be used for such operation POST, PUT or PATCH? As I don't find any one of these correct in the context, if I go by the definition of these HTTP method.
It depends.
We use PUT/PATCH when the client edits a local copy of a resource, and sends the edited version of the document back to the server (aka "remote authoring"). Think "Save File".
We use POST when... well, pretty much for everything else.
POST serves many useful purposes in HTTP, including the general purpose of “this action isn’t worth standardizing.” -- Fielding, 2009
Think HTML forms here - information is dispatched to the origin server in a representation that is completely unrelated to the representation of the resource itself.
For example, in a resource model where we have a resource for the reference point itself
GET /example
200 OK
Content-Type: appplication/json
{ "referencePoint": { "x": 0, "y": 0 } }
The using remote authoring semantics is pretty reasonable
PUT /example
Content-Type: appplication/json
{ "referencePoint": { "x": 0, "y": 999 } }
If the representation of the resource were very big (much bigger than the HTTP headers), and the changes were small, then we might use a patch document to communicate the information, rather than a complete document.
PATCH /example
Content-Type: application:json-patch+json
[ { "op": "replace", "path": "referencePoint/y", "value": 999 } ]
In both cases, the basic idea is the same - we are requesting that the origin server change its copy of the resource to match the edited version on the client.
It is okay to use POST for everything else.
Let's say I'd like to implement two functions:
Register for a course
Pay the course fee
I understood that I might have two RESTful API endpoints like this
register the course for the student:
send POST request to /myapp/api/students/{id}/courses
with request body like
{
"course_id": 26,
"is_discount": true,
"reg_date": "2020-04-23T18:25:43.511Z"
}
create payment record of the student for the course:
send POST request to /myapp/api/payment-records
with request body like
{
"student_id": 204,
"course_id": 26,
"amount": 500
}
My question is, how this can be done in one action (or within one transaction) from client side by just calling to one RESTful endpoint without separating them into two like the above? Because if one fails to make successful payment, due to network failure of card system, for example, then the course registered by the student should be rollbacked accordingly.
Or, should I do it like:
send POST request to /myapp/api/course-registration
with request body like this?
{
course: {
"course_id": 26,
"is_discount": true,
"reg_date": "2020-04-23T18:25:43.511Z"
},
payment: {
"record_id": 1,
"student_id": 204,
"course_id": 26,
"amount": 500
}
}
My question is, how this can be done in one action (or within one transaction) from client side by just calling to one RESTful endpoint without separating them into two like the above?
How would you do it on the web?
You'd have a web page with a form to collect all of the information you need from the student -- some fields might be pre-filled in, others might be "hidden" so that they aren't part of the presentation. When the form is submitted, the browser would collect all of that information into a single application/x-www-form-urlencoded document, and would include that document in a POST request that targets whatever URI was specified by the form.
Note that you might also have two smaller forms - perhaps part of the same web page, perhaps somewhere else, to do the registration and payment separately.
Two things to note about the target URI. The browser does not care what the spelling of the target-uri is; it's just going to copy that information into the HTTP request. But the browser does care if the target URI is the same as something that is available in its local cache; see the specification of invalidation in RFC 7234.
So a heuristic that you might use for choosing the target is to think about which cached document must be refreshed if the POST is successful, and use the identifier for that document as the target-uri.
I am asked to write a small HTTP based service that will accept requests that look like this:
[
{ name: 'James Smith', address: '10 Lake Drive' }
{ name: 'Jones', phone_number: '999-123-4567' }
{ name: 'Mr. Lucas', address: 'Detroit, MI' }
]
(but with a larger number of requests) and to attempt to determine whether an account exists for each one, returning data that looks like this:
[
{ name: 'James Smith', address: '10 Lake Drive', account='123ABC' }
{ name: 'Jones', phone_number: '999-123-4567', account='Not Found' }
{ name: 'Mr. Lucas', address: 'Detroit, MI', account='654CBA' }
]
The client of this service (at least, the first client I know about) will use this data synchronously -- it cannot continue until it receives the account numbers back from the service. (The actual processing done to match up the specifications request items to the accounts is not important.)
I'm asked to provide a RESTful API to this service. The only way I can see to encapsulate this in a RESTful API is to create the concept of a "LookupRequestDocument" which contains one or more lookup requests. The client would POST this to a URI at \LookupRequests\, receive a server-generated URL for the entire request, and then use GET to poll that URL until the response is ready.
This feels uncomfortable to me for the following reasons:
I've created a new concept (the LookupRequestDocument "resource") solely to allow me to make the API RESTful -- it has no other existence in the problem statement.
I've introduced asynchronicity into what is otherwise a synchronous problem.
It seems more natural to me to POST the data to a URI like \DoMatches and get the completed results in the returned document. But this does not seem to meet my client's requirement that the API be RESTful.
Question: Is my encapsulation of the problem into a RESTful API the best way of "being RESTful" for this problem? Is my \DoMatches solution actually RESTful even though it does not involve resources as I understand them?
So it seems to that your LookUpRequests process is not RESTful. Its not a stateless transaction, it requires the information in the post to be stored and processed on the server then returned in a different request.
I would do the \DoMatches but are you really "POSTING" anything your "GETTING" information right ? so why wouldn't that be a GET request with the response being the answer/answers ?
Neither one is RESTful. This is RPC, pure and simple.
The RESTful approach would be treating each account as a resource, identified by an URI, and checking each one of them on a single request.
If you don't want to have multiple requests like that, you can have a higher level interface as a workaround, where you submit a list of URIs (mimetype text/uri-list) via POST and get a 207 Multi-Status with the equivalent responses.
I'm currently creating an application (let's say, notes app, for instance - webapplication + mobile app). I wanted to use RESTful API here, so I read a lot about this topic and I found out there's a lot of ambiguity over there.
So let's start at the beginning. And the beginning in REST is that we have to first request the / (root), then it returns list of paths we can further retrieve, etc, etc. Isn't this the the first part where REST is completely wasteful? Instead of rigid paths, we have to obtain them each time we want to do something. Nah.
The second problem I encountered was bulk operations. How to implement them in REST? Let's say, user didn't have access to the internet for a while, made a few changes and they all have to be made on server as well. So, let's say user modified 50 notes, added 30 and removed 20. We have to make 100 separate requests now. A way to make bulk operations would be very helpful - I saw this stackoverflow topic: Patterns for handling batch operations in REST web services? but I didn't found anything interesting here actually. Everything is okay as long as you want to do one type of operation on one type of resources.
Last, but not least - retrieving whole collection of items. When writing an example app I mentioned - notes app - you probably want to retrieve all collection of items (notes, tags, available notes colors, etc...) at once. With REST, you have to first retrieve list of item links, then fetch the items one by one. 100 notes = over 100 requests.
Since I'm currently learning all this REST stuff, I may be completely wrong at what I said here. Anyway, the more I read about it, the more gruesome it looks like for me. So my question finally is: where am I wrong and/or how to solve problems I mentioned?
It's all about resources. Resources that are obtained through a uniform interface (usually via URI and HTTP methods).
You do not have to navigate through root every time. A good interface keeps their URIs alive forever (if they go stale, they should return HTTP Moved or similar). Root offering pathways to navigate is a part of HATEOAS, one of Roy Fieldings defined architectural elements of REST.
Bulk operations are a thing the architectural style is not strong on. Basically nothing is stopping you to POST a payload containing multiple items to a specific resource. Again, it's all up to what resource you are using/offering and ultimately, how your server implementation handles requests. Your case of 100 requests: I would probably stick with 100 requests. It is clean to write and the overhead is not that huge.
Retrieving a collection: It's about resources what the API decides to offer. GET bookCollection/ vs GET book/1 , GET/book/2 ... or even GET book/all. Or maybe GET book/?includeDetails=true to return all books with same detail as GETting them one-by-one.
I think that this link could give you interesting hints to design a RESTful service: https://templth.wordpress.com/2014/12/15/designing-a-web-api/.
That said, here are my answers to your questions:
There is no need to implement a resource for the root path. With this, I think that you refered to HATEOS. In addition, no link within the payload is also required. Otherwise you can use available formats like Swagger or RAML to document your RESTful service. This show to your end users what is available.
Regarding bulk operations, you are free to use methods POST or PATCH to implement this on the list resource. I think that these two answers could be helpful to you:
REST API - Bulk Create or Update in single request - REST API - Bulk Create or Update in single request
How to Update a REST Resource Collection - How to Update a REST Resource Collection
In fact, you are free to regarding the content you want for your methods GET. This means the root element managed by the resources (list resource and element resource) can contain its hints and also the ones of dependencies. So you can have several levels in the returned content. For example, you can have something like this for an element Book that references a list of Author:
GET /books
{
"title": "the title",
(...)
"authors": [
{
"firstName": "first name",
"lastName": last name"
}, {
(...)
},
(...)
]
}
You can notice that you can leverage query parameters to ask the RESTful service to get back the expected level. For example, if you want only book hints or book hints with corresponding authors:
GET /books
{
"title": "the title",
(...)
}
GET /books?include=authors
{
"title": "the title",
(...)
"authors": [
{
"firstName": "first name",
"lastName": last name"
}, {
(...)
},
(...)
]
}
You can notice that you can distinguish two concepts here:
The inner data (complex types, inner objects): data that are specific to the element and are embedded in the element itself
The referenced data: data that reference and correspond other elements. In this case, you can have a link or the data itself embedded in the root element.
The OData specification addresses such issue with its feature "navigation links" and its query parameter expand. See the following links for more details:
http://www.odata.org/getting-started/basic-tutorial/ - Section "System Query Option $expand"
https://msdn.microsoft.com/en-us/library/ff478141.aspx - Section "Query and navigation"
Hope it helps you,
Thierry
We use AWS's DynamoDB Session Provider in our app to store session data.
I recently moved to an environment where I can have NewRelic monitoring my app and it started throwing alerts regarding Dynamo DB access. However, NewRelic is the only monitoring tool that is getting it. I cannot see anything related to this problem in my application logging (log4net) or the Windows event viewer.
I searched a lot and even went through the source code of the provider but came out empty.
I'm getting (400) Bad Request from what is seems to be all the calls made during a period of 1 or 2 minutes at a time happening 3 or 4 times per hour.
The stacktrace I could get is not promising:
at System.Net.HttpWebRequest.GetResponse()
at System.Net.HttpWebRequest.GetResponse()
at Amazon.Runtime.AmazonWebServiceClient.getResponseCallback(IAsyncResult result)
And the offending URL is:
dynamodb.us-east-1.amazonaws.com/Stream/GetResponse
From the time-graphs below we can see that all requests are fine during most of time (graph 1), but when the problem occurs the number of successful requests made to DynamoDB goes to 0 (graph 1). And, at the same time, there is a spike in the number of errors thrown (graph 2).
UPDATE: During a low usage period in the weekend I ran Fiddler on the production server too see what the error from AWS looks like. I'm getting "The conditional request failed" which seems to happen because the value was updated while requesting and old value and therefore the value is not consistent to what was expected. Below is a full request/response as a sample.
Request:
POST https://dynamodb.us-east-1.amazonaws.com/ HTTP/1.1
X-Amz-Target: DynamoDB_20120810.UpdateItem
Content-Type: application/x-amz-json-1.0
User-Agent: aws-sdk-dotnet-35/2.0.15.0 .NET Runtime/4.0 .NET Framework/4.0 OS/6.2.9200.0 SessionStateProvider TableSync
Host: dynamodb.us-east-1.amazonaws.com
X-Amz-Date: 20140510T153947Z
X-Amz-Content-SHA256: e7a4886acac6ccf16f0da9be962d3a68bd50e381c202277033d0d2bb3208aa8a
Authorization: AWS4-HMAC-SHA256 Credential=redacted/20140510/us-east-1/dynamodb/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-target, Signature=redacted
Accept: application/json
X-NewRelic-ID: redacted
X-NewRelic-Transaction: redacted
Content-Length: 399
{
"TableName": "ASP.NET_SessionState",
"Key": {
"SessionId": {
"S": "redacted"
}
},
"AttributeUpdates": {
"LockId": {
"Value": {
"S": "42a9ed29-7a92-4455-8733-2f56c7d974b3"
},
"Action": "PUT"
},
"Locked": {
"Value": {
"N": "1"
},
"Action": "PUT"
},
"LockDate": {
"Value": {
"S": "2014-05-10T15:39:47.324Z"
},
"Action": "PUT"
}
},
"Expected": {
"Locked": {
"Value": {
"N": "0"
},
"Exists": true
}
},
"ReturnValues": "ALL_NEW"
}
Response:
HTTP/1.1 400 Bad Request
x-amzn-RequestId: redacted
x-amz-crc32: redacted
Content-Type: application/x-amz-json-1.0
Content-Length: 120
Date: Sat, 10 May 2014 15:33:17 GMT
{
"__type": "com.amazonaws.dynamodb.v20120810#ConditionalCheckFailedException",
"message": "The conditional request failed"
}
Graph 1
Graph 2
Any help is appreciated. Thanks!
The conditional lock failure can occur if your application makes multiple requests at the same time which access the session state. This can be common with Ajax calls. The article The Downsides of ASP.NET Session State provides a good explanation about how ASP.NET serializes access to a particular session state with some work arounds:
The first issue we'll look at is one that a lot developers don't know about; by default the ASP.NET pipeline will not process requests belonging to the same session concurrently. It serialises them, i.e. it queues them in the order that they were received so that they are processed serially rather than in parallel. [...]
These errors should not be bubbling up to application level. The AWS SDK for .NET throws exceptions for conditional update failures which the session provider is interpreting that as failure to get the lock. That is passed back to the ASP.NET framework which queues the request till it can get the lock:
[...] This means that if a request is in progress and another request from the same session arrives, it will be queued to only begin executing when the first request has finished. Why does ASP.NET do this? For concurrency control, so that multiple requests (i.e. multiple threads) do not read and write to session state in an inconsistent way.
Update
Norm Johanson's answer surfaces the root cause of the issue at hand, I'm keeping my respectively adjusted answer for the parts that still apply and the pointers to related issues.
Initial Answer
I haven't faced the exact issue you describe, but it rings a bell regarding similar patterns encountered in the context of investigating the AWS API's Eventual Consistency, see e.g. my answer to Deterministically creating and tagging EC2 instances for more on this. Things have considerably improved since then:
I've just updated my answer regarding the extended documentation about Troubleshooting API Request Errors being available meanwhile.
E.g. the AWS SDK for Java added more elaborate exponential backoff handling along the lines of what's proposed in Error Retries and Exponential Backoff in AWS, which also seems to be available in the AWS SDK for .NET to some extent.
Now, what I suspect is something like this:
New Relic is instrumenting the .NET byte code, which allows them to e.g. log all exceptions, regardless of whether they are handled or not.
Your client is e.g. getting throttled for request limit violations, which is causing a retryable 400 - ThrottlingException as per the API Error Codes, i.e. it triggers an exception that is handled and kicking off the exponential retry in turn, ultimately succeeding the request eventually, and leaving no trace for other tools accordingly.
Update: the exceptions at hand turn out to be the non retryable 400 - ConditionalCheckFailedException, thus this suspicion doesn't apply here.
In case, the question obviously is what might be causing this - even though the issue description doesn't match yours, the discussion in Performance issue in 2.0.12.0 hints on an ongoing threading issue in the 2.0.x releases of the .NET SDK, which might surface differently depending on the usage pattern at hand?
Update: Norm Johanson's answer surfaces the root cause of the issue at hand.