In the old (single-threaded) days we instructed our testing team to always report the CPU time and not the real-time of an application. That way, if they said that in version 1 an action took 5 CPU seconds, and in version 2 it took 10 CPU seconds, that we had a problem.
Now, with more and more multi-threading, this doesn't seem to make sense anymore. It could be that the version 1 of an application takes 5 CPU seconds, and version 2 10 CPU seconds, but that version 2 is still faster if version 1 is single-threaded, and version 2 uses 4 threads (each consuming 2.5 CPU seconds).
On the other hand, using real-time to compare performance isn't reliable either since it can be influenced by lots of other elements (other applications running, network congestion, very busy database server, fragmented disk, ...).
What is in your opinion the best way to 'numerate' performance?
Hopefully it's not intuition since that is not an objective 'value' and probably leads to conflicts between the development team and the testing team.
Performance needs to be defined before it is measured.
Is it:
memory consumption?
task completion times?
disk space allocation?
Once defined, you can decide on metrics.
Related
I am using optaplanner 8.17.FINAL with Java 17.0.2 inside a kubernetes cluster, my server has 32 cores + hyper threading. My app scales to 14 pods and I use moveThreadCount = 4 . On a single run, everything works fine, but on a parallel run, the speed of the optaplanner drops. With 7 launches, the drop is insignificant, 5-10%. But with 14 launches, the speed drop is about 50%. Of course, you can say that there are not enough physical cores, but I'm not sure that hyperthreading works like that. In resource monitoring, I see that 60 logical cores are involved with 14 launches, but why then do the speed drop twice?
I'm tried to inscrease heap size and change garbage collector (G1GC, SerialGC, ParallelGC), but it has little effect
I am not an expert on hyperthreading by any means but perhaps OptaPlanner, by
fully utilizing the entire core(s), cannot benefit from HT so much. If so, you just don't have enough CPU cores to run so many solvers in parallel, which leads to context switching and performance drop, as a result.
You can prove that by adding more cores. If it helps, it means there is no artificial bottleneck for this amount of tasks.
I successfully implemented a solver that fits my needs. However, I need to run the solver on 1500+ different "problems" at 0:00 precisely, everyday. Because my web-app is in ruby, I built a quarkus "micro-service" that takes the data, calculate a solution and return it to my main app.
In my application.properties, I set:
quarkus.optaplanner.solver.termination.spent-limit=5s
which means each request take ~5s to solve. But sending 1500 requests at once will saturate the CPU on my machine.
Is there a way to tell OptaPlanner to stop when the solution is good enough ? ( for example if the score is stable ... ). That way I can maybe reduce the time from 5s to 1-2s depending on the problem?
What are your recommandations for my specific scenario?
The SolverManager will automatically queue solver jobs if too many come in, based on its parallelSolverCount configuration:
quarkus.optaplanner.solver-manager.parallel-solver-count=3
In this case, it will run 3 solvers in parallel. So if 7 datasets come in, it will solve 3 of them and the other 4 later, as the earlier solvers terminate. However if you use moveThreadCount=2, then each solver uses at least 2 cpu cores, so you're using at least 6 CPU cores.
By default parallelSolverCount is currently set to half your CPU cores (it currently ignores moveThreadCount). In containers, it's important to use JDK 11+: the CPU count of the container is often different than from the bare metal machine.
You can indeed tell the OptaPlanner Solvers to stop when the solution is good enough, for example when a certain score is attained or the score hasn't improved in an amount of time, or combinations thereof. See these OptaPlanner docs. Quarkus exposes some of these already (the rest currently still need a solverConfig.xml file), some Quarkus examples:
quarkus.optaplanner.solver.termination.spent-limit=5s
quarkus.optaplanner.solver.termination.unimproved-spent-limit=2s
quarkus.optaplanner.solver.termination.best-score-limit=0hard/-1000soft
I have an application, using the HSQL database, that at very rare ocassions needs to copy a lot of data (5 million rows+) into the database. On an i7 this takes about 3 hours which is perfectly fine.
The issue I have is that on a weaker CPU like an i3 it is not possible to do this copy task at all since the CPU usage is at 100% at all cores and as a consequence the whole application freezes.
I'm looking for a solution to "throttle" the data entering process. It's totally okay if the copy process takes much longer as long as it completes and doesn't freeze the application.
I have been looking through the official documentation here: http://hsqldb.org/doc/guide/guide.html but couldn't find what I was looking for.
What would be the best approach to get this task working also on weaker CPUs?
The problem is described and answered here: https://groups.google.com/forum/#!topic/redis-db/egyA1xvhGfo
Unfortunately I do not fully understand the answer.
My concerns are if redis takes up 100% CPU every 5 minutes and if my server only has a single CPU (i.e. staging) would that mean it will freeze my httpd process every 5 minutes?
Would this not be of a concern if my server has multiple CPUs?
Depending on the type of persistence you select, this will happen. The reason is because the standard persist method ( fork and copy-on-write aka cow ) happens after x number of object changes ( or however you have it configured ) and will eat up a fair amount of I/O persisting the database to disk. You'll want at least a spare core on your server for the persistence to happen but it's not so much actual CPU that's being utilized as it is the wait for the I/O. Faster I/O will mitigate the impact of the db saves.
How much CPU/RAM would I need to host 5 Ruby on Rails 3 applications?
I am talking about applications that will not get more than 300 hits per day each.
That's only a few hits per minute, even after allowing for peak hours and bursts.
It's hard for me to imagine a reasonably new machine that would have any problems with that.
But to answer your question, it depends a bit on which web server you choose but about 300 MB / Rails server is a starting point for planning a big application rollout. Since you won't be needing lots of simultaneous transactions, a couple of threads should do and therefore a totally random 2GB machine should be more than enough.
I wouldn't really bother deploying a server without at least 8 or 16 GB, though, even if not immediately needed. Given the other costs involved, even a small budget allocation for memory should result in way more than your scenario needs.