My Active Resource connects to some stupid external service that takes a while to respond for whatever reason. This is a little too nagging. I would like to stub Active Resource during development to speed up my development time.
Is this a good thing to do? I think it is. If you think otherwise, please explain.
And is there a mechanism to stub it out based on a switch in environment configuration file, possibly any gem/plugin that you have used for this purpose?
What and how do you do all these in your experience?
I recommend using FakeWeb. I used this on a project recently and it allowed me to register a number of external urls with a predefined response. In your test setup you could do:
FakeWeb.register_uri(:get, %r|users.xml|, :body => File.read("spec/factories/xml/users.xml"))
Now whenever active resource requests anyhost.com/users.xml (in test environment), you'll instead immediately get the contents of the file your referred to. I like this approach because when you're testing a model, you don't really want to be testing the external service too. I'd leave that level of testing to an integration test.
This won't affect development or production environments, so you can use your stupid external service as usual.
Related
How is the worklightserverhost attribute on the app-builder task used? This is important as when deploying an tested application into a production environment, you normally wouldn't do a new build (as this could introduce regression problems). However, the fact that this is a mandatory property and contains in this scenario the test server URL and context - does it force you then to do a new build for the production environment?
Yes, a re-build for each environment does seem to be the usual approach. While we might prefer a "build once, promote through the stages" pattern, I think by careful use of tagging in your source repository you can get pretty good defence against regression.
Alternatively, I think with care you could set up your network so that the app is built once directing to, say,
myco.mobile.hostv21
you could then have that resolve to the different stages as appropriate.
I am designing a AWS deployment solution for a new dynamic website project. I have acquired an EC2 instance for testing the environment. Need some help on how do I do a load testing on an Ec2 instance to determine how many HTTP requests it can safely handle... P.S. I am new to the AWS platform.
Thanks...
RedLine offers an EC2 Load Testing solution that will automate the distribution of load tests on your own EC2 instances.
Late to the party but could help someone in the future:
A possible tool for load tests, stress tests, whatever you may call them, is Apache JMeter, but there are plenty of alternatives.
A simple starting setup, further explained in this excellent tutorial on DigitalOcean, can exist of a Thread Group containing an HTTP Request Sampler and a View Results in Table Listener. The Thread Group can be used to configure the amount of "clients" you want to simulate. The Request Sampler will be used to configure the server's properties (hostname, path, etc). The Table View Listener outputs a handy CSV file that can be used to calculate means, compare different types of EC2 instances,...
JMeter is a beautiful program with a GUI that can be run on your local workstation, producing an XML file that can be executed on another EC2 instance, for instance. You can even do simple manual edits to the XML file on your server afterward, if necessary.
Take a look at Amazon's testing policy to make sure you're not doing anything illegal.
A couple of quick points;
Set the environment up exactly like it's supposed to run. If there's a database involved, you'll want to involve that in the testing too. Synthetic <?php echo "ok"; CPU based benchmarks won't help you much since normally very little of the time spent replying to HTTP requests is actual CPU time.
A recommendation is to use a service for the benchmarking. Setting load testing up is not without its complexities, and unless you consider benchmarking your core business, you're probably better off using something like Neustar to load and measure your site (there are many services, they're not necessarily what fits you best, just pulled one out of memory)
Of course you can set a load test up yourself, but getting that done right is not anything that can be described in a few sentences. There are very well paid people that only do that for a living :)
There is good experience in using curl-loader aka Davilka tool, also on Amazon EC2 env
http://curl-loader.sourceforge.net
In my opinion, web server is responsible to deliver content to client. If it is static content like pictures and static html document, web server just deliver them as bitstream directly. If it is some dynamic content that is generated during processing client's request, the web server will not generate the conetnt itself but call some external proram to genearte the content.
AFAIK, this kind of dynamice content generation technologies include the following:
CGI
ISAPI
...
And from here, I noticed that:
...In IIS 7, modules replace ISAPI
filters...
Is there any others? Could anyone help me complete the above list and elabrate on or show some links to their evolution? I think it would be very helpful to understand application such as IIS, TomCat, and Apache.
I once wrote a small CGI program, and though it serves as a content generator, it is still nothing but a normal standalone program. I call it normal because the CGI program has a main() entry point. But with the recenetly technology like ASP.NET, I am not writing complete program, but only some class library. Why does such radical change happens?
Many thanks.
well, the biggest missing piece in your question is that you can have the webserver generating the content dynamically as well. This is common with most platforms outside of PHP and Perl. You often set that website behind apache or nginx used as a proxy, but it doesn't "call an external progam" in any reasonable sense, it forwards the http request to the proxied server. This is mostly done so you can have multiple sites on the same server, and also so you can have apache/nginx protect you against incorrect requests.
But sure, we can, for the sake of the question, say that "proxying" is a way to call an external program. :-)
Another way to "call the external program" is Pythons WSGI, where you do call a permanently running server. So again you don't start an external program, it's more like calling the module in ASP (although it's a separate program, not a module, but you don't start it with every request, you use an API).
The change from calling external programs as in CGI to calling modules like in ASP.NET, process with WGI or proxying to another webserver happened because with CGI you have to start a new prpogram for each request. The PERL/PHP interpreter needs to be laoded into memory, and all modules they use as well. This quickly becomes very heavy and process/memory intensive.
Therefore, to be able to use bigger systems that are permanently running, other techniques have been developed. Most of them are platform/language dependent, and the only one that is platform independent is really to make a complete webserver and then use apache/nginx as a proxy in front (in which case the apache/nginx strictly isn't necessary any more).
I hope this cleared things up a bit.
fastcgi and wsgi are two more interfaces content generators can use to talk to a webserver -- the reason more recent interfaces aren't complete programs is that forking and executing things that expect to be executables is costly.
OTOH, writing your little generator in such a way that it doesn't leak anything between invocations is harder than having the liberty to just exit at the end (and rely on environment variables and command line arguments like a normal executable).
This is all for performance reasons, but then you have more complicated content generators and process management in the webservers.
Has any put much thought into this? Personally, I think managing endpoints in configuration files are a pain. Are there any pros/cons to doing one over the other?
Only points in favour of configuration files from me.
Managing endpoints in configuration files mean that you don't have to update your application if (or perhaps I should say when) the endpoints change.
You can also have several instances of the application running with different endpoints.
I tend to like the config approach myself too, other than the config file can get pretty big.
The one thing I have noticed with WCF configuration is that there is a lot of stuff that you can do from code that you can't do in XML config without adding your own custom extensions. In other words, doing config in code will allow more flexibility, of course you could also just code your own extensions and use those from configuration.
However, do note that there is what I would consider a 'bug' in Visual Studio that if you start making your own extensions and including them in XML, then VS won't like your config file any more and will tag them as errors, and then if you try to add a new service through the wizards, it will fail to add the endpoint to the configuration.
This is sort of a followup to my own answer:
After months of having everything in xml configuration, I'm changing everything to construct the endpoints and bindings in code. I found a really good case for having it in code;
When you want to have a deployable / sharable .dll that contains WCF clients.
So for example if you have a CommonClients.dll that contains all your WCF interfaces and contracts to communicate with some remote server, then you don't want to also say "here is 100 lines of xml that you also have to drop into your app.config for every client to make it work". Having it all constructed in code works out much better in this case.
There is also a "feature" of .NET 3.5 where if you have some wcf extensions, you have to specify the fully qualified assembly name. This means that if your assembly containing the extensions changes the version nnumber, you have to go change the assembly name in the config file too. It is supposedly fixed in .NET 4 to use a short assembly name and not require the full name.
Offhand, an endpoint in a config file doesn't need to be recompiled when it's changed. This also means that you just need to update your config file when moving an application from Development to UAT to Production.
If your just coding something for your own use at home, then there's no real difference. However in a business environment, having the enpoint defined in your config file saves all sorts of headaches.
When using an app.config, your application does not need to be recompiled to adjust to a change. Also it can be resused in multiple situations with the exact same code. Finally, hardcoding your endpoints (or anything subject to change) is poor coding practice. Don't fear the configuration file, it's declarative programming. You say, "I want to use this endpoint." and it does the work for you.
I generally do programmatic configuration, as I don't want to expose my applications internal structure the the user. The only thing I keep configurable is service address, but even this I keep in userSettings section, not system.ServiceModel.
I prefer and recommend the configuration file approach. It offeres a lot of flexibility by allowing to make change to your server without the need to recompile the applcation.
If you need security, you can encrypt the config file.
The biggest worry with plain config files could be that it can be accidentally (or on purpose) modified by the end user causing your app to crash. To overcome this you could make some tests in code to check the configuration is ok in the config file and if not, initialize it programatically to some defaults. I presented how you could do that in another answer to this question.
It's just a question of how much flexibility you need.
Usually I prefer the config file approach.
Check out the .NET StockTrader app. It uses a repository to store config data and has a separate app to manage the configuration. The setup and structure is pretty advanced and there's a fair bit of head scratching for anyone like me that only has the basics of WCF configuration so far, but I would say it's worth a look.
I am designing an application that is going to consist of 3-4 services that run as separate processes and are linked by a suitable IPC. The system is going to have a web interface and I want to use whatever webserver is there.
The web interface should be accessed under some URL that allows to have other URLs on the same webserver doing totally different things. I'm planning to use the path below that URL to specify what the web interface should do. It has facilities for use by other applications over the net and for humans to interact with in a browser.
Off the cuff, I'd work as follows:
make the webserver fire up a CGI process for every request it receives (like SetHandler in Apache)
let the CGI connect to the IPC
let it get whatever it needs from the backend services
let the CGI return HTML / XML and whatever HTTP Status based on the services' answers
Now, what I really want is to avoid the first two steps, or if I can't, avoid the second one, because I'm afraid that I'm wasting performance on unneccesary overhead (the requests coming from other applications might be frequent).
PHP, for example, can open persistent connections to a MySQL database that survive the script's runtime and don't need to be recreated next time, though I don't know how they actually do it. Also, as I understand it, the Apache modules are loaded once when the server starts, so that might remove the first step but would tie me to Apache.
So, what are good ways to hook a handler for specific URLs into different webservers? I don't want to handle the HTTP, otherwise I might just use a proxy setup to a second server, but it just seems to be so reinventing-the-wheel. If you think, CGI is fine and have examples where it handles large numbers of request of a similar structure, please let me know.
OK, I overlooked this previously. Explaining my question here brought me onto it:
Instead of creating a new process for every request, FastCGI can use a single persistent process which handles many requests over its lifetime. -- Wikipedia: FastCGI
Even under moderate loads, CGI is a pretty unscalable beast. FastCGI is an option, but you'll probably also find a mod_XXXX package where XXXX is the name of your language. There's a mod for ruby, perl, and python for instance and probably a fair few others.