Which is the maximum amount of memory one can achieve in .NET managed code? Does it depend on the actual architecture (32/64 bits)?
There are no hard, exact figure for .NET code.
If you run on 32 bit Windows; your process can address up to 2 GB, 3 GB if the /3GB switch is used on Windows Server 2003.
If you run a 64 bit process on a 64 bit box your process can address up to 8 TB of address space, if that much RAM is present.
This is not the whole story however, since the CLR takes some overhead for each process. At the same time, .NET will try to allocate new memory in chunks; and if the address space is fragmented, that might mean that you cannot allocate more memory, even though some are available.
In C# 2.0 and 3.0 there is also a 2G limit on the size of a single object in managed code.
The amount of memory your .NET process can address depends both on whether it is running on a 32/64 bit machine and whether or not it it running as a CPU agnostic or CPU specific process.
By default a .NET process is CPU agnostic so it will run with the process type that is natural to the version of Windows. In 64 bit it will be a 64 bit process, and in 32 bit it will be a 32 bit process. You can force a .NET process though to target a particular CPU and say make it run as a 32 bit process on a 64 bit machine.
If you exclude the large address aware setting, the following are the various breakdowns
32 bit process can address 2GB
64 bit process can address 8TB
Here is a link to the full breakdown of addressable space based on the various options Windows provides.
http://msdn.microsoft.com/en-us/library/aa366778.aspx
For 64 bit Windows the virtual memory size is 16 TB divided equally between user and kernel mode, so user processes can address 8 TB (8192 GB). That is less than the entire 16 EB space addressable by 64 bits, but it is still a whole lot more than what we're used to with 32 bits.
I have recently been doing extensive profiling around memory limits in .NET on a 32bit process. We all get bombarded by the idea that we can allocate up to 2.4GB (2^31) in a .NET application but unfortuneately this is not true :(. The application process has that much space to use and the operating system does a great job managing it for us, however, .NET itself seems to have its own overhead which accounts for aproximately 600-800MB for typical real world applications that push the memory limit. This means that as soon as you allocate an array of integers that takes about 1.4GB, you should expect to see an OutOfMemoryException().
Obviously in 64bit, this limit occurs way later (let's chat in 5 years :)), but the general size of everything in memory also grows (I am finding it's ~1.7 to ~2 times) because of the increased word size.
What I know for sure is that the Virtual Memory idea from the operating system definitely does NOT give you virtually endless allocation space within one process. It is only there so that the full 2.4GB is addressable to all the (many) applications running at one time.
I hope this insight helps somewhat.
I originally answered something related here (I am still a newby so am not sure how I am supposed to do these links):
Is there a memory limit for a single .NET process
The .NET runtime can allocate all the free memory available for user-mode programs in its host. Mind that it doesn't mean that all of that memory will be dedicated to your program, as some (relatively small) portions will be dedicated to internal CLR data structures.
In 32 bit systems, assuming a 4GB or more setup (even if PAE is enabled), you should be able to get at the very most roughly 2GB allocated to your application. On 64 bit systems you should be able to get 1TB. For more information concerning windows memory limits, please review this page.
Every figure mentioned there has to be divided by 2, as windows reserves the higher half of the address space for usage by code running in kernel mode (ring 0).
Also, please mind that whenever for a 32 bit system the limit exceeds 4GB, use of PAE is implied, and thus you still can't really exceed the 2GB limit unless the OS supports 4gt, in which case you can reach up to 3GB.
Yes, in a 32 bits environment you are limited to a 4GB address-space but Windows claims about half. On a 64 bits architecture it is, well, a lot bigger. I believe it's 4G * 4G
And on the Compact Framework it usually is in the order of a few hundred MB
I think other answers being quite naive, in real world after 2GB of memory consumption your application will behave really badly. In my experience GUIs generally go massively clunky, unsusable after lots of memory consumptions.
This was my experience, obviously actual cause of this can be objects grows too big so all operations on those objects takes too much time.
The following blog post has detailed findings on x86 and x64 max memory. It also has a small tool (source available) which allows easy easting of the different memory options:
http://www.guylangston.net/blog/Article/MaxMemory.
I am wondering what are the possible pros and cons of switching from 32 bit to 64 bit VM. Most of answers I encountered suggest that with VM you don't need to worry about switch and you get extra addressable memory, but I am afraid that there is more to the problem.
Here is what I came up with:
Pros:
substantially more addressable memory
Cons:
whatever uses pointers under the hood will double up the memory usage - I know VMs do optimisations in order to avoid this problem, but I guess it's safe to assume that same logic under 64bit VM will potentially take more memory
all deterministic outputs relying on memory hashes are likely to change
Anything more?
I am happy to take concrete example like JVM.
I am planning to configure Redis in Master/Slave configuration.
I have got three machines (8GB RAM, 8 cores), planing to to use one master and two slaves.
What would be the recommended hardware configuration for these machines?
Redis is not CPU intensive, so you should get at least 2 cores per server (one for redis, one for backups, maybe one more to do basic stuff on the server?), more is not really relevant. Redis is single-threaded.
Get as much RAM as you can as it defines the size of your store. Also making a dump consumes RAM so your true space size is less than you can think. Monitor your RAM usage to prevent surprises.
For RAM type, if it fails, redis fails and sometimes silently (consistency broken). If you need to be careful with your data always use ECC RAM, it is expensive but maybe less expensive than broken data in RAM accessed through redis causing unknown effects. Redis has no known checks against hardware errors from RAM, even if it is quite rare (less likely to happen than a broken hard drive) it does happen.
I've a strange situation: A server, containing 64GB of memory, runs a SQL server process (64 bit) which consumes 32 GB of memory. There is about 17 GB memory available.
MS Dynamics Nav is running on top of SQL
Besides the 64bit SQL process, there is another SQL process and a NAS, both running 32 bits.
Every now and then, an error message is logged in the eventviewer, saying
There is not enough memory to execute this function.
If you work in a single-user installation, you can try reducing the
value of the 'cache' program property. You can find information about
how to optimize the operating system in the documentation for yo
Now I'm wondering what the problem is, since there is still 17 GB memory available. Is it possible that a 32-bit process cannot allocate memory in the last segment (60 to 64 GB)?
32 bit processes are limited to about 4 GB of memory usage. The x64 architecture should allow a 32bit process to run in any of the available memory space, but your 32bit process will still be limited by it's maximum addressible space (~4GB).
I am not sure what is meant by 16-bit or 32-bit applications. Is that a 16-bit application is an application which would not require more than 2^16 bytes of memory space? Does this 16-bit refers to the max size of the application?
It means the application has been compiled for a processor that has 16 bits of memory addressing or 32 bit of memory addressing. Same goes for 64 bit applications.
The number refers to the maximum amount of memory that the application can address.
See wikipedia - 16-bit, 32-bit, 64-bit (and more).
A 32-bit application is software that runs in a 32-bit flat address space.
Answers to common questions
Will a 64 bit CPU run a standard (32-bit) program on a 64-bit version of an OS?
Yes it will. 64 bit systems are backward compatible with the 32 bit counterparts.
Will a 64-bit OS run a standard application on a 64 bit processor?
Again, it will. This is because of backward compatibility.
Can I run W2K and WXP on an 64 bit CPU, and use old software?
Yes, a 32 bit OS (W2K and WXP) will run on a 64 bit processor. Also, you should be able to run "old software" on a 64 bit OS.
The number(32 or 16 of the assembler directive of the addressmode (example "[use16]" and "[use32]")) does not refers to the maximum amount of memory that the application can address!
Because with the 80386+ it is also possible to use operandsize- and adresssize prefixes in combination with the 16 bit PM for to address up to 4 GB of ram.
(The maximum amount of memory that our application can be use is refering to the segment entries of the segmentsize inside of a GDT/LDT selector, or by the default size for a segment of 64 kb.)
The only one differnce between the 32 bit - and the 16 bit addressmode is the meaning and the usage of those operandsize- and addresssize prefixes.
[use16]
So if we want to use in the 16 bit addressmode 32 bit operands/addresses, then we have to add those prefixes to our opcode. Without those prefixes we can only use 16 bit.
[use32]
In the 32 bit addressmode we found a diametrical opposite situation, so if we want to use 32 bit operands/addresses, then we have to leave out those prefixes from our opcode and only if we want to use 16 operand/addresses, then we have to add those prefixes to our opcode.
If we use these size-directives above(or similar notation) carefully, then our assembler will do this job.
Operand size prefix in 16-bit mode
Dirk