I'm new to cloud computing and, for the life of me, I can't figure out how "much" 1/8th of a core is in practical terms.
I know what kind of CPUs Amazon EC2 are using for m1.small, but let's say (for education purposes) that it is a single-core 1GHz CPU.
How is 1/8th of core calculated? Does it mean my application will run at 128MB RAM and 1/1GHz of CPU? Or will my application be able to run only a certain number of operations/CPU cycles before I'll be charged for an addition app-cell?
What I need is a practical explanation of the phrase. Perhaps, on an a simple vert.x HTTP server, where each successful connection calculates 2 + 3? Vert.x uses less than 128MB of RAM.
Afaik, you don't have a limit on the number of cycles: if you application requires many CPU cycles it will probably run slower since it would only use 1/8 of core.
Regarding the memory, if you are just using 1 app cell but your app requires more than 128MB, then it will probably result in an OUT OF MEMORY exception.
slicing of the server to 8th isn't as mathematic as you expect. Sharing server resource with multiple tenant allows to better use CPU globaly, compared to a classic server, so even you path inly 1/8 of the server you actually get more resources, but only when you application actually use them.
I have installed MongoDB 2.4.4 on Amazon EC2 with ubuntu 64 bit OS and 1.6 GB RAM.
On this server, only MongoDB running nothing else.
But sometime CPU usage reach to 99% and load average: 500.01, 400.73,
620.77
I have also installed MMS on server to monitor what's going on server.
Here is MMS detail
As per MMS details, indexing working perfectly for each queries.
Suspect details as below
1) HIGH non-mapped virtual memory
2) HIGH page faults
Can anyone help me to understand what exactly causing high CPU usage ?
EDIT:
After comments of #Dylan Tong, i have reduced active connetions but
still there is high non-mapped virtual memory
Here's a summary of a few things to look into:
1. Observed a large number of connections and cursors (13k):
- fix: make sure your connection pool is appropriate. For reporting, and your current request rate, you only need a few connections at most. Also, I'm guessing you have a m1small instance, which means you only have 1 core.
2. Review queries and indexes:
- run your queries with explain(), to observe how the queries are executed. The right model normally results in queries only pulling very few documents and utilization of an index.
3. Memory (compact and readahead setting):
- make the best use of memory. 1.6GB is low. Check how much free memory you have, and compare it to what is reported as resident. A couple of common causes of low resident memory is due to fragmentation. If there are alot of documents moving, changing size and such, you should run the compact command to defragment your data files. Also, a bad readahead can lead to poor use of memory as well. Check your readahead setting (http://manpages.ubuntu.com/manpages/lucid/man2/readahead.2.html). Try a few values starting with low values (http://docs.mongodb.org/manual/administration/production-notes/). The production notes recommend 32 (for standard 512byte blocks). Sometimes higher values are optimal if your documents are larger. The hope is that resident memory should be close to your available memory and your page faults should start to lower.
If you're using resources to the fullest after this, and you're still capped out on CPU then it means you need to up your resources.
How much CPU/RAM would I need to host 5 Ruby on Rails 3 applications?
I am talking about applications that will not get more than 300 hits per day each.
That's only a few hits per minute, even after allowing for peak hours and bursts.
It's hard for me to imagine a reasonably new machine that would have any problems with that.
But to answer your question, it depends a bit on which web server you choose but about 300 MB / Rails server is a starting point for planning a big application rollout. Since you won't be needing lots of simultaneous transactions, a couple of threads should do and therefore a totally random 2GB machine should be more than enough.
I wouldn't really bother deploying a server without at least 8 or 16 GB, though, even if not immediately needed. Given the other costs involved, even a small budget allocation for memory should result in way more than your scenario needs.
Given that:
SSD’s are now [high end] mainstream
Two+ cores are not hard to come across
24+ Inch monitors are plentiful
Dual Video Outputs are the norm.
64-Bit OS’s complement very cheap memory
Can I ask two questions to hardware enthused developers [not the gamers!]
What high-end hardware item could you not develop without - [what is your hardware crutch]?
What should a baseline [no frills] dev machine look like and what basic specs should it have to ensure that any dev can still be productive?
Note: It might be worth mentioning what platform and dev-env your base line is for?
The most important hardware update (and most underrated) is the monitor.
If you're coding 8+ hours a day don't hesitate on costs and get a nice high end 24" at least, or even a pair of them.
Absolute must have is a good monitor which is easy on the eyes, afterall, you stare at it all day. I go with the 24" Samsung (forget model). I used to go with two monitors but prefer the one wide screen now. You need to be able to get docs and code on the same screen.
Secondly is a good chair and desk (sorry not very technical).
Followed lastly by plenty of RAM (2Gb minimum). Once you get over any thrashing due to paging you are fine. Anything with a dual core had enough processing power.
This is entirely dependent upon what you are developing for. Take your target system requirements, and double them and use that as your minimum specs for the dev machines. That may seem odd, but it is about the point I've found that I've needed at least of when developing various projects.
As others have mentioned the importance of getting good monitors, keyboard, and chairs is underrated. If you are going to spend a lot of time at this PC, those are very important.
RAM is cheap, and you'll likely never have enough. If you are running 32bit Windows, max it out at 4GB of RAM. If you are using another OS that supports more than 4GB of ram (Linux, or 64bit Windows for example), start at 8GB minimum, and if you are working on multimedia projects be ready to upgrade from there.
Best bang for the buck on CPUs seems to be Quad cores right now, so I would say that at least a quad core (2.4Ghz or so) should be the minimum. You may not see much difference going up beyond there, until you get until dual quad core, which is a large price jump.
Find a reliable hard drive or two. Reliability and speed are going to be more important than size. Personally I currently go for a pair of 640GB western digital drives in all machines I build.
24 inch or larger monitor
Baseline dev machine would be a 15 inch MacBook Pro with 4GB of RAM. (For web development)
A pair of the fastest hard drives avaílable. I never recognized how much difference separate and fast System and Data drives can make.
(And please, none of those slow SSDs that you usually get nowadays in <$2000 Laptops - if you really want to hop on the SSD train, get a proper one, otherwise you could as well use a 32 GB SDHC Card)
There's been a study on the optimum size of computer monitors by the Utah University
Wall street journal article. Not surprising is that bigger monitors will boost the speed of work. Surprising is that there seems to be an optimum size of 26". There's no explanation why though.
I am not a developer, but do sit at the computer all day.
For me the must have is a desk that is a good height or easily adjusted, I prefer dual monitors, a 26" and a second wide screen that can turn sideways to view documents full lenght without the need for a lot of scrolling, a computer with dual core(prefer 4) and at east 4gb of ram(I tend to do a lot of vm work), and as stated above, a good chair that has lumbar support and will allow me to lean back when I am reading or pondering a situation. The last one is specific for me since I have glasses and tend to hear high frequencies, I prefer to have incandescent lighting with a slightly warm spectrum. I can hear a fluorescent ballast above someone playing loud speakers. I also find I get less glare and I can focus my eyes for longer periods of time with incandescent.
Ram, lots and lots of ram. Ram compensates for many performance bottlenecks.
But do make sure you keep an eye on the memory usage of whatever you're building. When you're building a 60 MB footprint app on a system with 2 gigs of developer tools loaded at run-time, it's easy to lose that footprint in the noise, even when it doubles.
Don't bother shelling out for a high-end cpu. The cpu is the most overpowered component in modern systems. A standard cheap dual-core should be more than enough. Compiles tend to be disk-bound, not cpu bound, so that money is better invested in a faster drive.
Dell Outlet sells 30" LCD monitors for about $800.00.
That is a good place to start.
Besides that, invest time into tweaking your OS to your needs and automate as much as possible.
It's like I keep telling people, "I'll upgrade to the latest Mac when it somehow manages to help me run more Terminal windows and Text Editors." Until then, you're better off saving the money for a new machine and investing it into a decent monitor and keyboard.
It depends on the project.
For large imaging application like medical imaging applications, You may require: large monitors(we have to view the images properly and in detail), powerful graphics, lots of RAM and a good processor(imaging applications usually need lots of power).
I'm going to echo most people on the large monitors part, and you can always make good use of a pair.
Second to that is a good keyboard. What that mean varies depending on which school of keyboard design you subscribe to. I'm with the ergonomic camp.
Following that is 2Gb+ of RAM, and a recent desktop CPU (anything released in the past 2-3 years really).
As has been previously said, large monitors are essential. These days is not that expensive to have 2 hooked up to a machine. At work I'm lucky enough to have 3 hooked up to one PC and it make a huge amount of difference to how I work.
A decent keyboard and mouse are essential. For the last 10 or so years I've always taken my own mouse and keyboard to work as you typically end up with whatever comes from the PC manufacturer. I use a Microsoft ergonomic keyboard and it's very hard to find these in the workplace, or to get your employer to stump up for one, but I've never worked anywhere where the employer has an issue with taking your own in.
High-end hardware I cannot do without:
Kinesis countoured ergonomic keyboard ($300)
Fast twin SATA drives, striped for speed ($150)
Affordable luxuries I could do without:
Dell 30" widescreen monitor ($900)
Twin Velociraptor hard drives ($600)
We have decided to go with a virtualization solution for a few of our development servers. I have an idea of what the hardware specs would be like if we bought separate physical servers, but I have no idea how to consolidate that information into the specification for a generalized virtual server.
I know intuitively that the specs are not additive - I shouldn't just add up all the RAM requirements from each machine to get the RAM required for the virtual server. I can't really treat them as parallel systems either because no matter how good the virtualization software is, it can't abstract away two servers trying to peg the CPU at the same time.
So my question is - is there a standard method to estimating the hardware requirements for a virtualized system given hardware requirement estimations for the underlying virtual machines? Is there a +C constant for VMWare/MS Virtual Server overhead (and if so, what is C?)?
P.S. I promise to move this over to serverfault once it goes into beta (Promise kept)
Yes add 25% additional resources to manage the VM. So if I need 4 servers that are equal to single core 2 ghz machines with 2 gigs of ram I will need 10 ghz processing power plus 10 gigs of ram. This will allow all systems to redline and still be ok.
In the real world this will never happen though, all your servers will not always be running all the time. You can get a feel for usage by profiling your current servers and determine their exact requirements and then adding an additional 25% in resources.
Check out this software for profiling utilization http://confluence.atlassian.com/display/JIRA/Profiling+Memory+and+CPU+usage+with+YourKit
The requirements are in fact additive. You should add up the memory requirements for each VM, and the disk requirements, and have at least one processor core per VM. Then add on whatever you need for the host system.
VMs can share a CPU, to some extent, if you have really low performance requirements, but they cannot share disk space or memory.
Answers above are far too high, second (1 core per VM) is closer. You can either 1) plan ahead and probably over-purchase 2) add just-in-time. Do you have some reason that you must know well ahead (yearly budget? your chosen host platform doesn't cluster hosts, so you can't add later?)
Unless you have an incredible simple usage profile, it will be hard to predict before and you'll over purchase. The answer above (+25%) would be several times more than you need for an modern server virtualization software (VMware, Zen, etc) that manages resources smartly. It's accurate only for desktop products like VPC. I chose to rough it out on a napkin and profile my first environment (set of machines) on the host. I'm happy.
Examples of things that will confound your estimation
Disk space, Some systems (Lab
Manager) use only the difference in
space from the base template. 10
deployed machines with 10 GB drives
using about 10 GB (template) + 200MB.
Disk space: You'll then find you
don't like the deltas in specific
scenarios.
CPU / Memory: This is dev
shop - so you'll have erratic load.
Smart hosts don't reserve memory and CPU.
CPU / Memory: But then you'll
want to do perf testing, and want to
reserve CPU cycles (not all hosts can
do that)
We all virtualize for different reasons. Many of the guests
in our environment don't have much work. We want them there to see how something behaves with a cluster of 3 servers of type X. Or, we have a bundle of weird client desktops waiting around, being used one at time by a tester. They rarely consume many host resources.
So, if you are using something like that doesn't do delta disks, disk space might be somewhat calculable. If lab manager (delta disk), disk space is really hard to predict.
Memory and processor usage: You'll have to profile or over-purchase heavily. I have many more guest CPUs than host CPUS, and don't have perf problems - but that's because of the choppy usage in our QA environments.