I have a Web Api 2 service that will be deployed across 4 production servers. When a request doesn't pass validation a custom response object is generated and returned to the client.
A rudimentary example
if (!ModelState.IsValid)
{
var responseObject = responseGenerator.GetResponseForInvalidModelState(ModelState);
return Ok(responseObject);
}
Currently the responseGenerator is aware of what environment it is in and generates the response accordingly. For example, in development it'll return a lot detail but in production it'll only return a simple failure status.
How can I implement a "switch" that turns details on without requiring a round trip to the database each time?
Due to the nature of our environment using a config file isn't realistic. I've considered using a flag in the database and then caching it at the application layer but environmental constraints make refreshing the cache on all 4 servers very painful.
I ended up going with the parameter suggestion and then implementing a token system on the back end. If a Debug token is present in the request the service validates it against the database. If it's a valid and active token it returns the additional detail.
This allows us to control things from our end while keeping things simple for the vendors and only adds that extra round trip to the database during debugging.
Related
I am trying to work around a issue with a 3rd party filter. My current plan is to put a filter in front of that filter to "fix" the query string so it does not error out.
I made an ActionFilterAttribute and added it into the filter list. It is running fine. I am adding my logic in the OnActionExecuting method.
The first item of context.HttpContext.Request.Query has a Key that is a json structure. I need to change that Key to be {}.
Problem is that both context.HttpContext.Request.Query and context.HttpContext.Request.QueryString are read-only.
How can I alter the context.HttpContext.Request.Query or the context.HttpContext.Request.QueryString?
EDIT - The Underlying Problem:
BreezeJS did a minimal level upgrade to support .NET Core. In this upgrade, part of the code expects that every call that has any parameters to return an IQueryable (QueryFns.cs Line 32). From reading the code it seems like this is an error (the calling function (the actual filter) seems to just expect null to be returned not an Exception.)
Either way, this makes moving to .NET Core very hard.
I considered my other options and if this fails, I will continue to pursue them:
Submit a pull request to fix the issue: The project has not accepted any pull requests in over a year and a half. So it seems unlikely my request will be taken.
Fork my own branch: I would rather not have to create and maintain a separate version with my own build and publishing pipeline.
Find a way to make the Breeze filter ignore the call when the result is not an IQueryable: I am currently looking into this one. (This question.)
Find a way to send my call from the client differently so that breeze ignores calls that do not return IQueryable: The return type of the call is owned by the service. And this is an issue with the service. I would rather not have to have tight coupling between the service and the client such that the client is crafting workarounds for service filter issues.
How optimize an API call in symfony?
I call with Guzzle bundle, but the time in some situations is very long.
In client application call a function from the server.
In server application extract the objects from the database and send back to the client.
In client creat the new object with properties from server respons.
One of the ways to improve your API calls is to use caching. In Symfony there are many different ways to achieve this. I can show you one of them (PhpFileCache example):
In services.yml create cache service:
your_app.cache_provider:
class: Doctrine\Common\Cache\PhpFileCache
arguments: ["%kernel.cache_dir%/path_to/your_cache_dir", ".your.cached_file_name.php"]
(Remember, you need Doctrine extension in your app to work)
Then pass your caching service your_app.cache_provider to any service where you need caching:
Again in your services.yml:
some_service_of_yours:
class: AppBundle\Services\YourService
arguments: ['#your_app.cache_provider']
Finally, in your service (where you want to perform API caching):
use Doctrine\Common\Cache\CacheProvider;
class YourService
{
private $cache;
public function __construct(CacheProvider $cache)
{
$this->cache = $cache;
}
public function makeApiRequest()
{
$key = 'some_unique_identifier_of_your_cache_record';
if(!$data = $this->cache->fetch($key))
{
$data = $provider->makeActualApiCallHere('http://some_url');
$this->cache->save($key, serialize($data), 10800); //10800 here is amount of seconds to store your data in cache before its invalidated, change it to your needs
}
return $data; // now you can use the data
}
}
This is quite GENERIC example, you should change it to your exact needs, but idea is simple. You can cache data and avoid unnecessary API calls to speed things up. Be careful though, because cache has drawback of presenting stale(obsolete) data. Some things can (and should) be cached, but some things don't.
If you control the server
You should put a cache reverse proxy like Varnish on top of your PHP server. The PHP app must send HTTP cache headers to tell to the proxy how many time it must cache the request. Alternatively, you can use a library like FOSHttpCache to setup a cache invalidation strategy (the PHP server will purge the cache from the proxy when an update of the data occurs - it's a more advanced and complex scenario).
The PHP server will not even be called if the requested resource is in the reverse proxy cache.
You should also use a profiler like Blackfire.io or xhprof to find why some parts of your PHP code (or your SQL queries) take so many time to be executed, then optimize.
If you control the client
You can use this HTTP cache middleware for Guzzle to cache every API result according to HTTP headers sent by the API.
There is a ton of documentation on academic theory and best practices on how to manage versioning for RESTful Web Services, however I have not seen much discussion on how multiple REST APIs interact with data.
I'd like to see various architectural strategies or documentation on how to handle hosting multiple versions of your app that rely on the same data pool.
For instance, suppose you make a database level destructive change to a database table that causes you to have to increment your major API version to v2.
Now at any given time, users could be interacting with the v1 web service and the v2 web service at the same time and creating data that is visible and editable by both services. How should this be handled?
Most of changes introduced to API affect the content of the response, till changes introduced are incremental this is not a very big problem (note: you should never expose the exact DB model directly to the clients).
When you make a destructive/significant change to DB model and new API version of API is introduced, there are two options:
Turn the previous version off, filter out all queries to reply with 301 and new location.
If 1. is impossible to need to maintain both previous and current version of the API. Since this might time and money consuming it should be done only for some time and finally previous version should be turned off.
What with DB model? When two versions of API are active at the same time I'd try to keep the DB model as consistent as possible - having in mind that running two versions at the same time is just temporary. But as I wrote earlier, DB model should never be exposed directly to the clients - this may help you to avoid a lot of problems.
I have given this a little thought...
One solution may be this:
Just because the v1 API should not change, it doesn't mean the underlying implementation cannot change. You can modify the v1 implementation code to set a default value, omit the saving of a field, return an unchecked exception, or do some kind of computational logic that helps the v1 API to be compatible with the shared datasource. Then, implement a better, cleaner, more idealistic implementation in v2.
when you are going to change any thing in your API structure that can change the response, you most increase you'r API Version.
for example you have this request and response:
request post: a, b, c, d
res: {a,b,c+d}
and your are going to add 'e' in your response fetched from database.
if you don't have any change based on 'e' in current client versions, you can add it on your current API version.
but if you'r new changes are going to change last responses, for example:
res: {a+e, b, c+d}
you most increase API number to prevent crashing.
changing in the request input's are the same.
I'm creating a web API based on the new ASP.NET Web API. I'm trying to understand the best way to handle people submitting multiple data-sets at the same time. If they have 100,000 requests it would be nice to let them submit 1,000 at a time.
Let's say I have a create new Contact method in my Contacts Controller:
public string Put(Contact _contact)
{
//add new _contact to repository
repository.Add(_contact);
//return success
}
What's the proper way to allow users to "Batch" submit new contacts? I'm thinking:
public string BatchPut(IEnumerable<Contact> _contacts)
{
foreach (var contact in _contacts)
{
respository.Add(contact);
}
}
Is this a good practice? Will this parse a GET request with a JSON array of Contacts (assuming they are correctly formatted)?
Lastly, any tips on how best to respond to Batch requests? What if 4 out of 300 fail?
Thanks a million!
When you PUT a collection, you are either inserting the whole collection or replacing an existing collection as if it was a single resource. It is very similar to GET, DELETE or POST a collection. It is an atomic operation. Using is as a substitute for individual calls to PUT a contact may not be very RESTfull (but that is really open for debate).
You may want to look at HTTP pipelining and send multiple PutContact requests of the same socket. With each request you can return standard HTTP status for that single request.
I implemented batch updates in the past with SOAP and we encountered a number of unforeseen issues when the system was under load. I suspect you will run into the same issues if you don't pay attention.
For example, the database may timeout in the middle of the batch update and the all hell broke loose in terms of failures, reliability, transactions etc. And the poor client had to figure out what was actually updated and try again.
When there was too many records to update, the HTTP request would time out because we took too long. That opened another can of worms.
Another concern was how much data would we accept during the update? Was 10MB of contacts enough? Perhaps 1MB? Larger buffers has numerous implications in terms of memory usage and security.
Hence my suggestion to look at HTTP pipelining.
Update
My suggestion would to handle batch creation of contacts as an async process. Just assume that a "job" is the same as a "batch create" process. So the service may look as follows:
public class JobService
{
// Post
public void Create(CreateJobRequest job)
{
// 1. Create job in the database with status "pending"
// 2. Save job details to disk (or S3)
// 3. Submit the job to MSMQ (or SQS)
// 4. For 20 seconds, poll the database to see if the job completed
// 5. If the job completed, return 201 with a URI to "Get" method below
// 6. If not, return 202 (aka the request was accepted for processing, but has not completed)
}
// Get
public Job Get(string id)
{
// 1. Fetch the job from the database
// 2. Return the job if it exists or 404
}
}
The background process that consumes stuff from the queue can update the database or alternatively perform a PUT to the service to update the status of Job to running and completed.
You'll need another service to navigate through the data that was just processed, address errors and so forth.
You background process may be need to be tolerant of validation errors. If not, or if your service does validation (assuming you are not doing database calls etc for which response times cannot be guaranteed), you can return a structure like CreateJobResponse that contains enough information for your client to fix the issue and resubmit the request. If you have to do some validation that is time consuming, do it in the background process, mark the job as failed and update the job with the information that will allow a client to fix the errors and resubmit the request. This assumes that the client can do something with the fact that the job failed.
If the Create method breaks the job request into many smaller "jobs" you'll have to deal with the fact that it may not be atomic and pose numerous challenges to monitor whether jobs completed successfully.
A PUT operation is supposed to replace a resource. Normally you do this against a single resource but when doing it against a collection that would mean you replace the original collection with the set of data passed. Not sure if you are meaning to do that but I am assuming you are just updating a subset of the collection in which case a PATCH method would be more appropriate.
Lastly, any tips on how best to respond to Batch requests? What if 4 out of 300 fail?
That is really up to you. There is only a single response so you can send a 200 OK or a 400 Bad Request and put the details in the body.
I have a Dispatch MessageInspector which is deserializing a SAML Token contained in the SOAP message header.
To do the deserialization I am using a variation of the following code:
List<SecurityToken> tokens = new List<SecurityToken>();
tokens.Add(new X509SecurityToken(CertificateUtility.GetCertificate()));
SecurityTokenResolver outOfBandTokenResolver = SecurityTokenResolver.CreateDefaultSecurityTokenResolver(new ReadOnlyCollection<SecurityToken>(tokens), true);
SecurityToken token = WSSecurityTokenSerializer.DefaultInstance.ReadToken(xr, outOfBandTokenResolver);
The problem I am seeing is that the performance of the ReadToken call varies depending on the account that is running the windows service (in which the WCF service is hosted).
If the service is running as a windows domain account the elapsed time for the ReadToken call is virtually zero. When running as a local machine account the call takes between 200 and 1000 milliseconds.
Can anyone shed any light on what is going on here and why the account running this bit of code makes a difference as to its performance?
Thanks,
Martin
When the service is running under a local account that there is considerably more activity taking place, examples of this are :
Accessing and using C:\WINDOWS\system32\certcli.dll
Accessing and using C:\WINDOWS\system32\atl.dll
Attempting to access registry keys e.g.
HKLM\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration
None of this extra activity appears to occur when running under a domain account.
A quick search on the internet for "certcli.dll domain user" brings up microsoft knowledge base article 948080 which sounds similar.
Unsure how to resolve this as ultimately a .Net method is being called (WSSecurityTokenSerializer.ReadToken) where you have little to no control over the internals.
This appears to also describe the same problem :
http://groups.google.com/group/microsoft.public.biztalk.general/browse_thread/thread/402a159810661bf6?pli=1