GenerateResource / CoreResGen suddenly taking almost 30 times as long - vb.net

Starting a few weeks ago, compiling a project (VB.NET, .NET 2.0, VS 2010) has taken several times as long as before. In Task Manager, I noticed ResXtoResources.exe taking lots of CPU for a while. I've finally been able to get some data on this using MSBuild's 'Diagnostic' output setting, and comparing that output to what I see in a branch from a few months back. Most striking are the final lines, which give timings. Before:
Target Performance Summary:
[..]
1395 ms CoreResGen 1 calls
1930 ms CompileLicxFiles 1 calls
2135 ms GenerateApplicationManifest 1 calls
2844 ms CoreCompile 1 calls
Task Performance Summary:
[..]
1391 ms GenerateResource 1 calls
1929 ms LC 1 calls
2134 ms GenerateApplicationManifest 1 calls
2843 ms Vbc 1 calls
Build succeeded.
Time Elapsed 00:00:09.50
========== Rebuild All: 5 succeeded, 0 failed, 0 skipped ==========
After:
Target Performance Summary:
1348 ms CompileLicxFiles 1 calls
1747 ms GenerateApplicationManifest 1 calls
2595 ms CoreCompile 1 calls
39575 ms CoreResGen 1 calls
Task Performance Summary:
1347 ms LC 1 calls
1745 ms GenerateApplicationManifest 1 calls
2593 ms Vbc 1 calls
39570 ms GenerateResource 1 calls
Build succeeded.
Time Elapsed 00:00:47.34
========== Rebuild All: 5 succeeded, 0 failed, 0 skipped ==========
Both projects were compiled on the same system with the same settings. We've made numerous changes, to be sure, but nothing to the order of magnitude that would justify such a change in timings (and only for this one task!). I assume resource generation is getting stuck on something — a circular reference, a missing one, etc. I have been unable, however, to find anything useful on how to trace such a problem down to what I assume is just a single resource file.
Short of looking through thousands of checkins or temporarily removing some forms (and thus, their resource files) from the project, is there anything else I can do to figure out the issue? I can't seem to find individual per-resource file timings.
Findings so far:
I've created a new, empty project with all the same .resx files in place.
The issue is not reproducible in .NET 4.0: compiling the exact same test project takes less than a second.
The issue is reproducible in .NET 2.0 as soon as I also add one of the forms from the original project; apparently, it will otherwise not compile the resources "properly".
Removing individual .resx files will reduce the timings 'proportionally'; that is: I have unfortunately not found a single file that is the culprit.

Looks like this blog entry gives the answer.
In a nut, search your .resx files for assembly references that don't actually exist (such as System.Windows.Forms, Version 4.0.0.0), and replace them with ones that do (Version 2.0.0.0). I used grepWin to accomplish this.
My CoreResGen / GenerateResource timings are now roughly what they used to be. CruiseControl.NET says build time is down from 92 seconds to 40. :)

I found the reason here... the resources contained a png file save in special Adobe Fireworks format (PNG). I exported the file to png (without layer information) and now the compile takes 6 seconds.

Related

Karate DSL surefire - reports duplicated time

In my work, we implemented a lot le features that call another feature because de reuse scenarios for many scenarios.
But, when see the html reporte, this one show 5 minutes execution when, in console said 2.5 minutes.
We found in sunfire reports that the time of the feature son, the step that call a web service delay 30 ms, but also the step that call this feature son has 30 ms. So is 60 ms.
feature parent
call (feature Son.feature) 30ms
this is the son
given url 0 ms
where status 200 30 ms
feature report
Column duration 60 ms
Excuse me por my bad english. Thanks for any help
2 things.
If you use the parallel runner, you will see different time (actual / elapsed)
When you call features, just focus on the time reported by the parent
Can you refer this video, so you can troubleshoot better: https://twitter.com/KarateDSL/status/1049321708241317888

How to optimize golang program that spends most time in runtime.osyield and runtime.usleep

I've been working on optimizing code that analyzes social graph data (with lots of help from https://blog.golang.org/profiling-go-programs) and I've successfully reworked a lot of slow code.
All data is loaded into memory from db first, and the data analysis from there appears CPU bound (max memory consumption < 10MB, CPU1 # 100%)
But now most of my program's time seems to be in runtime.osyield and runtime.usleep. What's the way to prevent that?
I've set GOMAXPROCS=1 and the code does not spawn any goroutines (other than what the golang libraries may call).
This is my top10 output from pprof
(pprof) top10
62550ms of 72360ms total (86.44%)
Dropped 208 nodes (cum <= 361.80ms)
Showing top 10 nodes out of 77 (cum >= 1040ms)
flat flat% sum% cum cum%
20760ms 28.69% 28.69% 20850ms 28.81% runtime.osyield
14070ms 19.44% 48.13% 14080ms 19.46% runtime.usleep
11740ms 16.22% 64.36% 23100ms 31.92% _/C_/code/sc_proto/cloudgraph.(*Graph).LeafProb
6170ms 8.53% 72.89% 6170ms 8.53% runtime.memmove
4740ms 6.55% 79.44% 10660ms 14.73% runtime.typedslicecopy
2040ms 2.82% 82.26% 2040ms 2.82% _/C_/code/sc_proto.mAvg
890ms 1.23% 83.49% 1590ms 2.20% runtime.scanobject
770ms 1.06% 84.55% 1420ms 1.96% runtime.mallocgc
760ms 1.05% 85.60% 760ms 1.05% runtime.heapBitsForObject
610ms 0.84% 86.44% 1040ms 1.44% _/C_/code/sc_proto/cloudgraph.(*Node).DeepestChildren
(pprof)
The _ /C_/code/sc_proto/* functions are my code.
And the output from web:
(better, SVG version of graph here: https://goo.gl/Tyc6X4)
Found the answer myself, so I'm posting this here for anyone else who is having a similar problem. And special thanks to #JimB for sending me down the right path.
As can be seen from the graph, the paths which lead to osyield and usleep are garbage collection routines. This program was using a linked list which generated a lot of pointers, which created a lot of work for the gc, which occasionally blocked execution of my code while it cleaned up my mess.
Ultimately the solution to this problem came from https://software.intel.com/en-us/blogs/2014/05/10/debugging-performance-issues-in-go-programs (which was an awesome resource btw). I followed the instructions about the memory profiler there; and the recommendation to replace collections of pointers with slices cleared up my garbage collection issues, and my code is much faster now!

I am getting 'Local Search phase started with an uninitialized Solution' when I run on a larger dataset

I am developing a solver using Optaplanner 6.1.0, similar to the Vehicle Routing Problem. When I run my solver on 700 installers and 200 bookings, it will successfully solve the planning problem. But, when I used against a larger dataset (700 installers and 1220 bookings), I get
Caused by: java.lang.IllegalStateException: Local Search phase started with an uninitialized Solution. First initialize the Solution. For example, run a Construction Heuristic phase first.
but right before the exception,
16:10:40,378 INFO [DefaultConstructionHeuristicPhase] [http-listener-1(4)] Construction Heuristic phase (0) ended: step total (194), time spent (30693), best score (-1hard/-688803soft).
I am using <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
in my config.
Am I using it wrong?
Maybe the value range for a planning variable is empty. Especially with value range provider from entity, this is more likely. Feel free to file a jira that the error message should improve in such a case.
Diagnostic todo: Comment out the local solver phase, run the solver (so it only does the construction heuristic) and then iterate through the planning entities and print out the value for each planning value. Check if there are any nulls in there.
The fact that you have 194 steps, instead 200 steps in your CH indicates this. (If those other 6 planning entities are immovable, this won't trigger this exception (more info), so that's not the problem.)

How do I determine what a Crystal Reports function does?

I have a custom crystal report which retrieves invoices from a database. There is a formula in the report that has the following code:
V6AttachmentsGetAttachment ({Command.AttachmentID},{?ReportAttachmentChannel} )
From my understanding, the formula has a function called 'V6AttachmentsGetAttachment' which takes two parameters (the first is a report field and the second is a report parameter).
This calculates a dynamic hyperlink.
How can I determine how exactly this is calculated? I am trying to figure out if I can replicate this calculation in SQL.
Is V6AttachmentsGetAttachment something that is stored in the SQL database? I can not find any references to it in Crystal Reports.
This is a custom function created by Viewpoint Construction Software (V6). It is used to connect with Construction Imaging, a document management system, to return images of things like invoices.
I believe the custom function is available if you have the Viewpoint client installed.
It could either be a custom function or one contained in a user-function library (UFL).
Custom Function
Edit a formula (any formula will do)
Assuming that it is a custom function, it will be listed below the Report Custom Functions node:
UFL
If this function is contained in a user-function library (UFL), you should see it listed in the Function tree:
u252000.dll contains a single function DateTimeto2000().
If this is the case, then you will need to locate the source code for the UFL.
CRUFLE and CRUFLV6 are viewpoint dlls. They are registered as global access controls using regasm. Crystal reports can then access the functionality within the dll.
https://support.viewpoint.com/s/knowledgedetail?c__urlname=Issue-109356-Error-when-running-crystal-reports-via-remote-access-not-VRL-UFL-u212com-dll-or-u2lcom-dll-that-implements-this-function-is-missing
CRUFLE and CRUFLV6
file version 22.2.0.716
I use crystal reports 2016, 32 bit
I downloaded the 32 bit crufle.dll and cruflv6
I copied the crufle.dll and cruflv6 to C:\Program Files (x86)\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\win32_x86
%SystemRoot%\Microsoft.NET\Framework\v4.0.30319\regasm.exe crufle.dll
%SystemRoot%\Microsoft.NET\Framework\v4.0.30319\regasm.exe cruflv6.dll
regedit and search for the crufle and cruflv6 ensuring there were no other versions register. The u2lcom.dll error message was misleading because the dll is loaded from C:\Program Files (x86)\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\win32_x86. you can see u2lcom.dll is loaded in the crystal report about-> more info which shows the loaded modules.
dumpbin will show that u2lcom.dll is loaded and availabe
dumpbin /EXPORTs u2lcom.dll
Section contains the following exports for U2LCOM.dll
00000000 characteristics
5A54B943 time date stamp Tue Jan 9 05:44:51 2018
0.00 version
1 ordinal base
15 number of functions
15 number of names
ordinal hint RVA name
3 0 00005670 InitPerThread
4 1 00005680 IsThreadSafe
5 2 000056A0 UFCallCOMFunction
6 3 000056C0 UFEndJob
7 4 000056F0 UFErrorRecovery
8 5 00005720 UFGetFunctionDefStrings
9 6 00005730 UFGetFunctionExamples
10 7 00005740 UFGetFunctionTemplates
11 8 00005750 UFGetVersion
12 9 00005760 UFInitialize
1 A 00005610 UFSetPreferredViewingLocale
2 B 00005630 UFSetProductLocale
13 C 000057E0 UFStartJob
14 D 00005810 UFTerminate
15 E 00005860 UninitPerThread
Summary
1000 .data
1000 .gfids
4000 .rdata
1000 .reloc
1000 .rsrc
8000 .text

Apache ODE - BPEL compensation handler - weird behavior (or may be I'm wrong somewhere

I've got this example of BPEL from this location https://svn.wso2.org/repos/wso2/carbon/platform/trunk/products/bps/modules/samples/product/src/main/resources/bpel/2.0/SampleCompensationHandlers/FlightReservationProcess/
The example sets a given variable when executes a given scope.
The last scope throws an error, so the fault triggers the relevant handler for that scope, which rethrows the fault. That way the fault handler for the process is triggered, where the compensation is made for every successfully completed scope.
I've created a BPEL project in Eclipse and I've put the example in there, then I've started some tests. But I've found a very strange behavior:
I've got correct results just few times:
CarReservationActivity: 1 CarReservationCompensated: 1 HotelReservationActivity: 1 HotelReservationCompensated: 1 FlightReservatoinActivity: 1
In all other cases I've got incorrect results:
A)
CarReservationActivity: 1 CarReservationCompensated: 0 HotelReservationActivity: 1 HotelReservationCompensated: 1 FlightReservatoinActivity: 1
B)
CarReservationActivity: 1 CarReservationCompensated: 1 HotelReservationActivity: 1 HotelReservationCompensated: 0 FlightReservatoinActivity: 1
And when the result is incorrect then the case A) dominates.
I can not find out where is the problem. Everything looks fine.
Can someone help me to solve the issue ?
Used software :
- Windows 7 Enterprise, SP1, 32bit
- Apache Tomcat v.6.0.18
- Apache ODE v.1.3.5
- Eclipse Indigo v.3.7.2 SR2
- BPEL designer v.1.0.1
- Java 7 (v.1.7.0_07)