How to build historgram of methods by time spent inside with Mono? - mono

I have tried the following:
mono --profile=log myprog.exe
to collect profiler data. Then to interpret those I invoke:
> mprof-report output.mlpd
Mono log profiler data
Profiler version: 2.0
Data version: 14
Arguments: log
Architecture: x86-64
Operating system: linux
Mean timer overhead: 51 nanoseconds
Program startup: Fri Jul 20 00:11:12 2018
Program ID: 19840
Server listening on: 59374
JIT summary
Compiled methods: 8349
Generated code size: 2621631
JIT helpers: 0
JIT helpers code size: 0
GC summary
GC resizes: 0
Max heap size: 0
Object moves: 0
Metadata summary
Loaded images: 16
Loaded assemblies: 16
Exception summary
Throws: 0
Thread summary
Thread: 0x7fb49c50a700, name: ""
Thread: 0x7fb49d27b700, name: "Threadpool worker"
Thread: 0x7fb49d07a700, name: "Threadpool worker"
Thread: 0x7fb49ce79700, name: "Threadpool worker"
Thread: 0x7fb49cc78700, name: "Threadpool worker"
Thread: 0x7fb49d6b9700, name: ""
Thread: 0x7fb4bbff1700, name: "Finalizer"
Thread: 0x7fb4bfe3f740, name: "Main"
Domain summary
Domain: (nil), friendly name: "myprog.exe"
Domain: 0x1d037f0, friendly name: "(null)"
Context summary
Context: (nil), domain: (nil)
However, there's no information concerning which methods were called often and took long to complete, which was the only one thing I expected from profiling.
How do I use Mono profiling to gather and output information about method calls' total run time? Like hprof with cpu=times will generate.

The Mono docs are "slightly" wrong as the methods calls are not tracked by default. This option creates huge profile log output and massively slows down "total" execution time and when combined with other options like alloc, effect the execution time of the methods and thus any timings that are being collected.
Personally I would recommend using calls profiling by itself adjusting the calldepthto a level that matters to your profiling. i.e. do you need to profile into the framework calls or not? Also a smaller call depth also greatly decreases the size of the log produced.
Example:
mono --profile=log:calls,calldepth=10 Console_Ling.exe
Produces:
Method call summary
Total(ms) Self(ms) Calls Method name
53358 0 1 (wrapper runtime-invoke) <Module>:runtime_invoke_void_object (object,intptr,intptr,intptr)
53358 2 1 Console_Ling.MainClass:Main (string[])
53340 2 1 Console_Ling.MainClass:Stuff ()
53337 0 3 System.Linq.Enumerable:ToList<int> (System.Collections.Generic.IEnumerable`1<int>)
53194 13347 1 System.Linq.Enumerable/WhereListIterator`1<int>:ToList ()
33110 13181 20000000 Console_Ling.MainClass/<>c__DisplayClass0_0:<Stuff>b__0 (int)
19928 13243 20000000 System.Collections.Generic.List`1<int>:Contains (int)
6685 6685 20000000 System.Collections.Generic.GenericEqualityComparer`1<int>:Equals (int,int)
~~~~
Re: http://www.mono-project.com/docs/debug+profile/profile/profiler/#profiler-option-documentation

Related

JVM Runtime.availableProcessors() returns 2 when it should be 4

I'm running openjdk11 on alpine linux in a container in an AWS EKS cluster.
The application determines the size of a threadpool based on the number of CPUs as returned by Runtime.getRuntime().availableProcessors()
This call is returning 2 processors even though the container shows that 4 CPUs are available:
# cat /proc/cpuinfo | grep processor
processor : 0
processor : 1
processor : 2
processor : 3
Any idea why and how to solve the problem?
Update
Doing some more digging (prompted by some great questions from #gohm'c in the comments), I found a way to add some trace log prints to the JVM with -Xlog:os+container=trace
[0.001s][trace][os,container] CPU Shares is: 1536
[0.001s][trace][os,container] CPU Share count based on shares: 2
Now, I defined in resources.requests.cpu: "1500m".
I don't know why the slight discrepancy but I changed the value of the CPU request, and indeed the CPU Shares in the log trace changes accordingly.
I understand how the resources.limits.cpu value could affect the CPUs that the JVM sees. But why is the resources.requests.cpu value doing that! This seems like a bug to me? Any thoughts?

Local Search phase needs to start from an initialized solution - But how?

I am writing a modified version of the Task Assignment example with my own domain model.
In my model each Task can have a NextTask and a PreviousTask and an Assignee. All 3 are configured as PlanningVariables:
...
/** PreviousTask is a calculated task that the Resource will complete before this one. */
#PlanningVariable(valueRangeProviderRefs = { "tasksRange" }, graphType = PlanningVariableGraphType.CHAINED)
public Task PreviousTask;
#InverseRelationShadowVariable(sourceVariableName = "PreviousTask")
public Task NextTask;
#AnchorShadowVariable(sourceVariableName = "PreviousTask")
public Resource Assignee;
I have been stuck on the Local Search Phase step for some time now as it appears to require an initialized state of my planning variables (Task.PreviousTask in this case).
Error log:
2020-07-16 15:00:15.341 INFO 4616 --- [pool-1-thread-1] o.o.core.impl.solver.DefaultSolver : Solving started: time spent (65), best score (-27init/[0]hard/[0/0/0/0]soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
2020-07-16 15:00:15.356 INFO 4616 --- [pool-1-thread-1] .c.i.c.DefaultConstructionHeuristicPhase : Construction Heuristic phase (0) ended: time spent (81), best score (-27init/[0]hard/[0/0/0/0]soft), score calculation speed (90/sec), step total (0).
2020-07-16 15:00:15.376 ERROR 4616 --- [pool-1-thread-1] o.o.c.impl.solver.DefaultSolverManager : Solving failed for problemId (5e433f57-8c75-4756-8a9c-4c4ca4a83d6d).
java.lang.IllegalStateException: Local Search phase (1) needs to start from an initialized solution, but the planning variable (Task.PreviousTask) is uninitialized for the entity (com.redhat.optaplannersbs.domain.Task#697ea710).
Maybe there is no Construction Heuristic configured before this phase to initialize the solution.
I've been pouring over the documentation and trying to figure out what I've missed or broken between it and the source example but I cannot work it out. I have no Construction Heuristic configured (Same as the example), and I could not see where in the example does it ever set the previousTaskOrEmployee variable before solving.
I could supply a random PreviousTask in the initial solution model, but surely at least 1 of the tasks would have no previous task?
May I ask if you have <localSearch/> in your solverConfig.xml? If so, the <constructionHeuristic/> has to be there as well.
If none of these phases (CH nor LS) is configured, both Construction Heuristic and Local Search is added with default parameters. But once the appears in the solverConfig.xml, it's considered an override of the defaults, and a user is supposed to take care of initializing the solution (most often by providing the Construction Heuristic configuration).

Vulkan physical device

I'm learning Vulkan by API spec (http://vulkan-spec-chunked.ahcox.com/ch02s09.html), and I'm little confused about how physical devices are in Vulkan. I do have only one intel physical video card device, but vkEnumeratePhysicalDevices returns count of 2. The devices are identical, but the queue flags seems differ, and the queue flags are undocumented (actually they are, but only to flag 8, in second queue I do have the flag values 16 and 32).
typedef enum VkQueueFlagBits {
VK_QUEUE_GRAPHICS_BIT = 0x00000001,
VK_QUEUE_COMPUTE_BIT = 0x00000002,
VK_QUEUE_TRANSFER_BIT = 0x00000004,
VK_QUEUE_SPARSE_BINDING_BIT = 0x00000008,
} VkQueueFlagBits;
here is the output of my vulkan code:
GPU count: 2 ( physical devices )
Physical Device 0:
Device API version: 1.0.42 - 4194346
Device Vendor Id: 0x8086
Device Id: 1916
Device Driver version: 0.0.1 - 1
Device type: 1
Device Name: Intel(R) HD Graphics 520 (Skylake GT2)
Device Pipeline UID: f557cfd4
Queue Properties:
Flags: 7
Count: 1
ts Valid Bits: 24
Physical Device 1:
Device API version: 1.0.42 - 4194346
Device Vendor Id: 0x8086
Device Id: 1916
Device Driver version: 0.0.1 - 1
Device type: 1
Device Name: Intel(R) HD Graphics 520 (Skylake GT2)
Device Pipeline UID: f557cfd4
Queue Properties:
Flags: 49
Count: 0
ts Valid Bits: 1
Someone can help me understand why there is 2 physical devices for the same real device and the missing flags ?
The count=0 of the second device is curious. More seriously, its flags and tsVB values are corrupted (49 is not an valid value for flags and 1 not valid for tsVB).
This pretty much boils down to there being one extraneous *.json file on your system.
These *.json files store informations about ICDs present on the machine. They are stored in standard location(s).
vkEnumeratePhysicalDevices+vkGetPhysicalDeviceProperties are relatively dumb commands doing nothing much else than reading said *.json file(s) and returning its contents. I think though that calling something "real" like vkCreateDevice would not work on the badly installed driver.
What exactly happened to creare this problem is up to your curiosity to explore. For starters I believe on Linux distros there is a command to map a file to its originating package. It will probably be something about bad cleanup of previous driver, or possibly bug in the installation script of the new one. At least one person had this problem before.
Based on what I explained here, I believe this is relatively benign bug. The first device should work just fine. And you can just ignore the second one. Or simply delete its *.json manifest to prevent it from showing up in vkEnumeratePD.

How to Configure the Web Connector from metrics.log Values

I am reviewing the ColdFusion Web Connector settings in workers.properties to hopefully address a sporadic response time issue.
I've been advised to inspect the output from the metrics.log file (CF Admin > Debugging & Logging > Debug Output Settings > Enable Metric Logging) and use this to inform the adjustments to the settings max_reuse_connections, connection_pool_size and connection_pool_timeout.
My question is: How do I interpret the metrics.log output to inform the choice of setting values? Is there any documentation that can guide me?
Examples from over a 120 hour period:
95% of entries -
"Information","scheduler-2","06/16/14","08:09:04",,"Max threads: 150 Current thread count: 4 Current thread busy: 0 Max processing time: 83425 Request count: 9072 Error count: 72 Bytes received: 1649 Bytes sent: 22768583 Free memory: 124252584 Total memory: 1055326208 Active Sessions: 1396"
Occurred once -
"Information","scheduler-2","06/13/14","14:20:22",,"Max threads: 150 Current thread count: 10 Current thread busy: 5 Max processing time: 2338 Request count: 21 Error count: 4 Bytes received: 155 Bytes sent: 139798 Free memory: 114920208 Total memory: 1053097984 Active Sessions: 6899"
Environment:
3 x Windows 2008 R2 (hardware load balanced)
ColdFusion 10 (update 12)
Apache 2.2.21
Richard, I realize your question here is from 2014, and perhaps you have since resolved it, but I suspect your problem was that the port set in the CF admin (below the "metrics log" checkbox) was set to 8500, which is your internal web server (used by the CF admin only, typically, if at all). That's why the numbers are not changing. (And for those who don't enable the internal web server at installation of CF, or later, most values in the metrics log are null).
I address this problem in a blog post I happened to do just last week: http://www.carehart.org/blog/client/index.cfm/2016/3/2/cf_metrics_log_part1
Hope any of this helps.

What are the numbers in the square brackets in NSLog() output?

What is the stuff between the [] in the log message below? I get this in my iPhone app, and I have no idea where the message is coming from. My first guess would be a line number, but which file would it be in?
2010-10-19 08:56:12.006 Encore[376:6907]
The first number is the process ID, the second is the logging thread's Mach port. A desktop example:
2010-10-19 17:37:13.189 nc_init[28617:a0f] nc <CFNotificationCenter 0x10010d170 [0x7fff70d96f20]> - default <CFNotificationCenter 0x10010d2a0 [0x7fff70d96f20]>
(gdb) i thread
Thread 1 has current state "WAITING"
Mach port #0xa0f (gdb port #0x4203)
frame 0: main () at nc_init.m:10
pthread ID: 0x7fff70ebfc20
system-wide unique thread id: 0x167b49
dispatch queue name: "com.apple.main-thread"
dispatch queue flags: 0x0
total user time: 13232000
total system time: 16099000
scaled cpu usage percentage: 0
scheduling policy in effect: 0x1
run state: 0x3 (WAITING)
flags: 0x0
number of seconds that thread has slept: 0
current priority: 31
max priority: 63
suspend count: 0.
(gdb) p/x (int)mach_thread_self()
$1 = 0xa0f
Notice how 0xa0f is reported as the thread's Mach port.
first number is the process ID, unsure about second, this line will precede every line thats printed to console from your application.
Possibly a NSLog(#""); is causing this.
Is your application running or has it crashed by this stage?
The first number is the process ID, as the others have said. The second number is the thread ID, at least I'm pretty sure that's what it is...
It's a process ID in fact. You can see that in the GDB console with a line somewhere that should read "[Switching to process 376]".