I know INFO CPU call getrusage to get cpu resource usage.
but the number accumulated since the launch of the Redis instance,see this.
so, I think even if this number is large （eg, used_cpu_user:203.56）,but it does not reflect the current cpu load.
I wonder what information should I get from it? or how can i judge cpu load from the command?
Thank you very much.
Related
I am trying to test cpu consumption of an agent / daemon process written in Java. To avoid getting skewed by garbage collection, I keep trying longer periods for each profiling run. In the beginning I tried 15 minutes, then later arrived at 2 hours. Yet I just found out that, even with 2 hour runs, I can get very inconsistent results. - One run of 2 hours gave me cpu of 6%, another of 2 hours gave me cpu of 12%.
Any suggestions to get consistent results?
Are you controlling for CPU frequency? If there isn't much work to do, the OS (or CPU itself) might reduce the clock frequency to save power. With an aggressive power-management strategy, the CPU will always run at max when it's running at all, so looking CPU% can be meaningful.
On Linux on a Skylake or later CPU, you might set the EPP for each core to performance, to get it to run at max speed whenever it's running at all.
sudo sh -c 'for i in /sys/devices/system/cpu/cpufreq/policy[0-9]*/energy_performance_preference;do echo performance > "$i";done'
Otherwise maybe measure in core clock cycles (like Linux perf stat java ...) instead of CPU %, or at least look at average clock speed while it was running. (Lower clock speed relative to DRAM can skew things, since a cache miss stall for fewer cycles.)
I am struggling to understand how yarn containers are limited to allocated resources, especially the CPU.
I am running Spark or Flink jobs in the YARN cluster. Each executor or task manager requests a yarn container that has 1 CPU. Basically, the number of containers is equal to the number of CPUs available in the host.
I understand that YARN monitors the memory usage, and if the container exceeds the limit, it sends a kill signal. I am wondering about how CPU scheduling really works.
My JVM job in the YARN container(1CPU) can try to create multiple CPU-bound work threads. Will JVM be limited to 1 CPU core to execute those threads, or will it steal resources from other containers? Can technically a YARN container affect other containers' CPU performance?
Let's say I have 10 CPU in the host and I created a single container. Will that containers CPU performance be 10% of the host CPU performance?
By Default, yarn only allocates resources by RAM. so by default it hopes everyone plays nicely and you can get affected by CPU hungry jobs. You can change this:
From Apache:
yarn.scheduler.capacity.resource-calculator The ResourceCalculator
implementation to be used to compare Resources in the scheduler. The
default i.e.
org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator only
uses Memory while DominantResourceCalculator uses Dominant-resource to
compare multi-dimensional resources such as Memory, CPU etc. A Java
ResourceCalculator class name is expected.
In general it's enough to estimate by Memory. Most people actually estimate they're requirements for memory and threads very poorly. It's usually best to ignore [threads] unless you encounter issues. If it maintains to be an issue then maybe consider looking at DominantResourceCalculator. If/when you turn on resourceDominantCalculator be ready for a lot of people to feel the impact. You may have grossly over allocated threads and when we start counting threads, they will suddenly have to account for what they've asked for. (Or at least this was my experience.) This could grossly appear to shrink capacity of your cluster as space is reserved where it wasn't before.
TLDF: Don't touch this unless you have a good reason. (Wait until it's a problem, don't optimize until there is a bottleneck ). Users can make innocent mistakes in their resource estimation and it can be painful to grow their ability to correctly estimate what they need.
Firstly, I'm working on Visual Basic
With Ref. to this pre-posted thread What is the correct Performance Counter to get CPU and Memory Usage of a Process?
I wanted to get the cpu and memory used by a particular process in a label or listbox, let's say i want to get the processor % time of notepad.exe
Actually i am making a MYSQL Oriented tool that connects to MYSQL and i want to record how much CPU Consumption is actually caused to make the whole process happen. So i need to add the CPU and Memory consumptions of few particular processes concerned that i can see in task Manager.
So i want to supply the names of all the processes in code and get the output.
I need to figure this out using performance counter for one process and rest i can make out.
Thanks a lot for any help in advance.
You migh want to consider using this:
Dim cpu as New System.Diagnostics.PerformanceCounter()
With cpu
.CategoryName = "Processor"
.CounterName = "% Processor Time"
.InstanceName = "MyProcess"
End With
cpuLabel.Text = cpu.NextValue()
Note that this can quickly become quite heavy on the processor itself if doen for multiple processes. For an alternative take a look at How To Get Hardware Information. Its in C# but should be relatively easy to convert to VisualBasic.NET either manually or using this tool.
Hope this helps.
I've set up SOLR indices on my computer, and everything works fine.
I'm not experienced with SOLR but after profiling the "start.jar" process for a while, I've noticed that the RAM consumption jumps around a lot, anywhere from 150MB to 400MBish. And this is just for 10K documents!
So as a response, I wrote a script that just waits for SOLR to go past my RAM consumption limit (on shared hosting), and when it does, it kills start.jar and restarts it.
Does this have any adverse effect? And if so, what better solutions are there, besides get more RAM or use cloud based SOLR (which also costs money)? Sorry if this sounds stupid but I just need a working solution.
Thank you.
You need to provide some Index stats:
How big is your index (no. of docs & size in MB/GB)?
How many fields indexed/stored?
How much memory is allocated to JVM?
Do you optimize/commit, very often/realtime?
What is the SOLR Query time you see in logs?
Best
The BOINC client (does distributed processing jobs like SETI#home does) is able to turn processing on or off based on whether other processes are using a certain percentage of CPU time. That is, if the user starts to do some work and their processes start using 60% CPU, BOINC can pause to avoid interfering with the user's work.
I would like to do the same thing (monitor CPU usage by other processes). The difficulty as I see it is not monitoring CPU usage, but rather making sure that the information isn't skewed by my own usage. For example, if my process is using a ton of CPU time it may prevent another process from using enough to trigger the pause.
Can someone point me in the right direction? Even a suggestion for what to search for would be useful. I'm not really sure what this feature would be called.
You can use NSTask to set the 'nice' value of the process when your process starts.
Also [[NSThread mainThread] setThreadPriority:0.0]
where priority value is between 0.0 and 1.0 is a Cocoa API which may save you frakking about with sudo