I've recently assembled a new AMD Desktop, to replace an older Dell Latitude E7540 laptop.
The AMD Desktop:
Ryzen 3 3100 # 3.8GHz (4C/8T), 32GB DDR4 3600 CL17 RAM, Corsair P600
Gen4 SSD
The DELL Laptop:
Dell Latitude E7540: Intel I7-5600U # 2.6GHz (2C/4T), 16 MB RAM DDR3 1600, Samsung mSATA PM851
On the new AMD Desktop, when executing a docker build command, two situations occur:
The performance is dreadful, even building a simple image, it takes a long time for the command to start. After starting, it takes a long long time to complete (when it completes)
The build window crashes almost 50% of the time.
The benchmarks indicate that the new AMD Desktop is 3.5x faster at single core, and 6x faster at multicore.
As such, I was expecting a much better performance with the new AMD Desktop.
Unfortunately, that's not the case, and for the same Dockerfile (which generates a very big image):
The Dell starts faster
The Dell completes faster (10m vs 30m)
On the Dell, the build window never crashes.
The only difference between both systems is that one is an Intel platform, the new one an Ryzen 3 AMD.
Environment Details are the same on both machines:
Windows Version: Windows 10 Ent. 19049
Docker Desktop Version: Docker 3.0.0
What can explain this abysmal performance on Docker-Desktop on the new AMD system?
After a few troubling days, i can confirm that the problem is not AMD related.
The culprit is the Antivirus, that when ON, its scanning the files used by Docker, which cause all the problems i've described.
Docker documentation states how to disable the antivirus to scan Docker related files:
https://docs.docker.com/engine/security/antivirus/
When antivirus software scans files used by Docker, these files may be locked in a way that causes Docker commands to hang.
One way to reduce these problems is to add the Docker data directory (/var/lib/docker on Linux, %ProgramData%\docker on Windows Server, or $HOME/Library/Containers/com.docker.docker/ on Mac) to the antivirus’s exclusion list. However, this comes with the trade-off that viruses or malware in Docker images, writable layers of containers, or volumes are not detected. If you do choose to exclude Docker’s data directory from background virus scanning, you may want to schedule a recurring task that stops Docker, scans the data directory, and restarts Docker.
I have a Lenovo Z51-70 laptop (Windows10). it had 8GB RAM by default and SSHD. When I used to compile large projects 20K c++ files Task Manager always showed 90-100% CPU utilization. A week back I upgraded SSHD to SSD and 8GB RAM to 16GB RAM for gaining speed in the compilation. But build time hasn't improved (it is almost same) but Task Manager always shows roughly 50% CPU utilization. Why it is not able to utilize anyway near 90-100%? and why same build on SSHD & 8GB RAM always used to consume roughly 90-100% CPU utilization? It is not specific to a particular build system, i have tried MSBUILD, NINJA. All build system show same CPU utilization. I have tried to compile different projects for excluding any reason which may be the project-specific.
Any thoughts?
I am going to set up a standalone Hortonworks cluster.
What is the needed system configuration that could process 1TB of data.
Requirement such as:
RAM space
Hard disk space
quad-/hex-/octo-core CPUs running how much GHZ
Cent os which version etc.,
System Configuration depends on the usecase.
Disk - Assuming 1 replication, 1 TB + 25% processing space
If you are using Hive or MapReduce I would start with 16 Gig, 4 or 8 core. CentOS 7.0
A production tomcat stopped responding after 8 hours of adding more cpu and ram during runtime. There is no traceable log in catalina.out.
java process was still running in the background. There is no load on the system during that time period. No sign of Out of memory.
apache-tomcat-7.0.55
tomcat-native-1.1.31
java version "1.8.0_66"
Red Hat Enterprise Linux Server release 6.6 (Santiago)
Could it be because of the cpu and ram upgrade? Thanks.
I saw a post about it for java 6. I am not sure it is related to my issue.
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6840239
We are facing a problem that java.exe process occupies complete cpu utilization (100 %),
OS: Windows Server SP2 2007 Edition,
Processor: Intel(R) Xeon(R) CPU X7542 # 2.67 Ghz 2.67 Ghz
Memory (RAM): 8.00 GB
System type: 32-bit Operating System
Java Heap Memory: 1 GB Allocated
There are many reasons for CPU usage going up to 100%.
Java is known for its thread and queuing process. So make it to process one by one.
Have a doubt whether your java.exe hits any db that is does it calls any Database to process.
If u could tell in detail I could help with more accurate solution