Need help debugging why server crashing. IIS Server on Win 2008, CPU load normal, Memory usage log, nothing seems wrong.
Randomly w3p process crashes. In event log I see first:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: System.Runtime.Serialization.ni.dll, version: 4.0.30319.18044, time stamp: 0x51271348
Exception code: 0xc00000fd
Fault offset: 0x0000000000091ac6
Faulting process id: 0x1928
Faulting application start time: 0x01ceaf1919acdfd4
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Runteb92aa12#\9a8ecadcc9e05f435158bedc1f1a3e74\System.Runtime.Serialization.ni.dll
Report Id: 5beef11c-1b0c-11e3-8eec-0026b9347fe0
And then I see:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: clr.dll, version: 4.0.30319.18052, time stamp: 0x5173c114
Exception code: 0xc00000fd
Fault offset: 0x00000000000055a7
Faulting process id: 0x1760
Faulting application start time: 0x01ceaf191e746321
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: 60deebcd-1b0c-11e3-8eec-0026b9347fe0
And then:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: clr.dll, version: 4.0.30319.18052, time stamp: 0x5173c114
Exception code: 0xc00000fd
Fault offset: 0x0000000000073dc4
Faulting process id: 0x2208
Faulting application start time: 0x01ceaf193c19060b
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: 7e58b5f2-1b0c-11e3-8eec-0026b9347fe0
After that application pool goes into stopped state.
All webservice exceptions handled, I have no idea where to look. System can be stable for a long time (days) and then just randomly crash.
Is there any way to get to root cause of such problem?
Is there any way to monitor service and restart in case of such crash inside IIS?
EDIT:
Got more details. Debug diagnostics tool was able to read memory dump. Seems like WCF call causing it. Looks like when I create instance of WCF service and it starts to deserialize message it goes into infinite loop. Any ideas on how to get BETTER details on what's going on? In this stack I don't see any of my code. I cut middle portion out, it's just repeating calls in serialization DLL
Process Image c:\Windows\System32\inetsrv\w3wp.exe
System Up-Time 21 day(s) 11:44:13
Process Up-Time 1 day(s) 19:43:33
Thread 49 - System ID 1104
.NET Call Stack
Full Call Stack
Function Arg 1 Arg 2 Arg 3 Arg 4 Source
clr!GetRuntimeAssemblyHelper+1d 000007fe`f9cc471c 00000000`024dd240 00000000`ffba5cb0 00000000`00000000
clr!RuntimeTypeHandle::GetAssembly+8c 00000000`ffba5cb0 00000003`00053d30 000007fe`f87bd608 00000000`ffd338a8
mscorlib_ni+4ec579 00000004`0313e578 00000004`015e1ef0 00000001`fff8f2c0 000007fe`f1ee8f0e
mscorlib_ni+4f0a39 00000004`0313e578 000007fe`f1ed0af3 00000001`ffbbb300 00000004`0313e5a0
0x000007fe`9ad39c8a 00000000`ffd32520 000007fe`f9c6c4c5 00000000`0000008b 00000000`00000000
0x000007fe`9b6e2976 00000000`ffd32520 00000000`000054e2 00000000`00000000 000007fe`f1ec2e00
System_Runtime_Serialization_ni+b2ce1 00000004`015e1ef0 00000003`00023dc0 00000000`0e656548 00000000`0e6564e8
System_Runtime_Serialization_ni+b29ed 00000002`030caad8 00000001`00b65cc8 00000001`ffc11890 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000000`00000066 00000000`00000000 00000004`015e1f98 00000003`00023dc0
0x000007fe`9bda513d 00000001`ffc25548 00000000`000054e1 00000000`00000000 000007fe`f1ec2e00
System_Runtime_Serialization_ni+b2ce1 00000004`015e1ef0 00000003`00023dc0 00000004`0313e578 00000003`00023dc0
System_Runtime_Serialization_ni+b29ed 00000002`030cbe08 000007fe`f203bb28 00000003`ffe868e8 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000003`ffe86520 000007fe`9ab63858 00000000`00000147 00000004`ffb5afb0
0x000007fe`9bda4ef9 00000003`ffe86520 00000004`0313e5a0 00000000`00000000 00000003`00083558
System_Runtime_Serialization_ni+b08c1 00000002`030cbcf0 000007fe`f213c7cd 00000002`030c7c20 00000002`030c7c40
System_Runtime_Serialization_ni+b29ed 00000004`003e33f0 00000002`030c7c40 00000003`00083558 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000001`ffc1eaa0 00000004`015e2018 00000004`015e2264 00000004`015e2080
0x000007fe`9bda4687 00000001`ffc1eaa0 00000000`000054dc 00000000`00000000 000007fe`f1ec2e00
System_Runtime_Serialization_ni+b2ce1 00000004`015e1ef0 00000003`00023dc0 00000004`0313e578 00000003`00023dc0
System_Runtime_Serialization_ni+b29ed 00000002`018a8af0 000007fe`f203bb28 00000003`ffe85768 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f
.....
0x000007fe`9b8e780b 00000001`ffcb8690 00000000`00000016 00000000`00000000 000007fe`f1ec2e00
System_Runtime_Serialization_ni+b2ce1 00000004`015e1ef0 00000003`00023dc0 00000003`028539a0 000007fe`9a97d658
System_Runtime_Serialization_ni+b29ed 00000001`ffc22190 00000003`028539c0 00000003`000837c0 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000001`ffc1eaa0 000007fe`f213d7a0 00000003`00052e00 00000004`015e2018
0x000007fe`9bda44cf 00000001`ffc1eaa0 000007fe`f1edf4fc 00000000`0e6cc8e0 00000000`00000000
System_Runtime_Serialization_ni+b2ce1 00000001`ffc1eaf0 000007fe`f9cf5c2f 00000000`ffba5b60 00000000`00000000
System_Runtime_Serialization_ni+20c45b 00000004`015e2018 00000004`015e2018 00000000`00000011 000007fe`f1ee5e19
System_Runtime_Serialization_ni+20c766 00000004`015e1ef0 00000003`00023dc0 00000001`ffbbb3f0 00000000`00000007
System_Runtime_Serialization_ni+30a177 00000004`0313e578 00000004`0313e5a0 00000001`fff8ac40 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000001`ffbbaec8 000007fe`9ab625a8 80000000`00000000 000007fe`f860d2a2
0x000007fe`9b6e1823 00000001`fff8a948 000007fe`f2035929 00000000`0e6ccbd0 00000000`00000000
System_Runtime_Serialization_ni+b08c1 00000001`fff8a990 00000000`00000000 00000001`ffbe0180 00000001`ffbbae00
System_Runtime_Serialization_ni+20c45b 00000004`015e2018 000007fe`f821e740 00000000`00000010 000007fe`f1ee5e19
System_Runtime_Serialization_ni+20c766 00000004`015e1ef0 00000003`00023dc0 00000001`ffdadf58 000007fe`f86157d7
System_Runtime_Serialization_ni+30a177 00000001`fff7d2f0 000007fe`f1ee8efa 00000001`ffdb1d68 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000001`ffdafc80 00000004`0313e5e8 00000000`00000000 000007fe`f1edf848
0x000007fe`9b6e0d8c 00000001`ffdafc80 000007fe`f1edf4fc 00000000`0e6cced0 00000000`00000000
System_Runtime_Serialization_ni+b2ce1 00000001`ffdafcd0 000007fe`f9cf5c2f 00000003`ffc3f3a8 00000004`0313e500
System_Runtime_Serialization_ni+20c45b 00000004`015e2018 000007fe`f821e740 00000000`00000005 000007fe`f1ee5e19
System_Runtime_Serialization_ni+20c766 00000004`015e1ef0 00000003`00023dc0 02261db5`f87be0c0 000007fe`f9c6797c
System_Runtime_Serialization_ni+30a177 000007fe`9ac3c088 000007fe`fa074137 00000001`ffdac0f8 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000004`0313cee0 00000001`ffdac0f8 00000004`0313e578 00000000`0e6cd220
0x000007fe`9a85a630 00000001`ffdab308 000007fe`f2035900 00000000`0e6cd1c0 000007fe`f1edf484
System_Runtime_Serialization_ni+b2ce1 00000001`ffdab358 000007fe`f1ee7149 00000004`015e1f20 00000000`00000004
System_Runtime_Serialization_ni+308197 00000004`015e1ef0 00000003`00023dc0 00000004`0313e578 00000003`00023dc0
System_Runtime_Serialization_ni+b29ed 00000004`0313cf38 000007fe`f203bb28 00000001`ffda1c40 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000004`015e1f98 000007fe`9b01bda0 00000001`ffbbb3f0 00000000`00000003
0x000007fe`9a85a430 00000001`ffda1938 00000000`00000003 00000000`00000000 000007fe`f1ec2e00
System_Runtime_Serialization_ni+b08c1 00000004`015e1ef0 00000003`00023dc0 00000004`015e1ef0 000007fe`f20879f9
System_Runtime_Serialization_ni+b29ed 000007fe`9ac3bf38 000007fe`fa074137 00000002`fffe6060 000007fe`f8e31860
System_Runtime_Serialization_ni+20c91f 00000003`ffc41ef8 00000004`0313cef8 00000004`0313e5a0 00000004`015e1ef0
0x000007fe`9a8597e7 00000002`fffe4a28 000007fe`f2137f69 000007fe`9a811e88 000007fe`f1ed0a23
System_Runtime_Serialization_ni+b2ce1 000007fe`9a811e88 000007fe`f1ed082d 00000001`ffd15c14 00000003`00023c74
System_Runtime_Serialization_ni+a0c3c 00000000`00000000 00000003`00023cb0 00000004`015e1ef0 00000000`0e6cd760
System_Runtime_Serialization_ni+1fbd92 00000001`ffd8f520 000007fe`f76e5277 00000004`0313cef8 000007fe`f0e81779
System_Runtime_Serialization_ni+1fc626 00000003`027f98f8 00000000`00000000 00000000`00000000 00000000`00000000
System_Runtime_Serialization_ni+a0715 00000001`ffd8f508 00000004`015e2018 00000004`0313cef8 00000004`015e2080
System_Runtime_Serialization_ni+a03f5 00000000`00000066 000007fe`f1ee66fc 00000000`00000065 00000000`00000000
System_ServiceModel_ni+5eaf06 00000004`015e1ef0 00000002`fffe2158 00000002`fffe4908 000007fe`f1ec1577
System_ServiceModel_ni+5eade9 00000001`00000064 00000003`ffb31420 00000001`ffd06138 00000001`ffd06138
System_ServiceModel_ni+5eac97 00000000`00000000 00000004`0000000e 00000004`015e1ef0 00000004`015e1ef0
System_ServiceModel_ni+5eab05 00000004`015e1e88 00000003`0000bd68 00000000`00000000 00000000`00000000
System_ServiceModel_ni+650ed8 00000000`00000000 00000000`00000000 00000001`ffbcf838 00000000`00000000
System_ServiceModel_ni+650b93 00000000`0e6cdd20 00000004`015e1de8 00000001`ffec0ef0 00000004`0313d388
System_ServiceModel_ni+635c07 00000000`0e6cde30 00000001`ffbf92b0 00000004`0313d308 000007fe`7fffffff
0x000007fe`9a6a601c 00000001`ffbcb2f0 000007fe`ef84b555 00000003`04f95618 00000004`0313d488
System_ServiceModel_ni+5f316c 00000000`0e6ce060 00000000`00000001 00000000`0e6ce060 000007fe`f4ff80f3
System_ServiceModel_ni+5f2c8e 00000000`00000000 00000000`00000000 00000000`00000000 00000003`04f956c0
System_ServiceModel_ni+62380a 00000000`0e6ce000 00000003`054831d0 01ceb097`e03b5200 00000000`00000000
System_ServiceModel_ni+6232d1 00000000`00000000 00000004`0313d308 00000000`00000000 00000000`00000000
System_ServiceModel_ni+618d4b 00000003`04f95618 00000004`0313d308 00000000`23c34600 00000000`00000000
System_ServiceModel_ni+618bf3 00000003`0000c190 00000000`0e6ce820 00000003`0000c190 00000001`ffbd7038
System_ServiceModel_ni+6189ad 00000003`0000c190 00000000`0e6ce820 00000000`00000000 000007fe`f9d5d900
System_ServiceModel_ni+617a2a 00000003`0000c190 00000000`0e6ce820 000007fe`9a7857b8 00000000`0e6ce820
System_ServiceModel_ni+6124c6 00000003`0000c190 00000000`0e6ce820 00006a2a`00000000 00000000`0f48e8a0
System_ServiceModel_ni+611b2b 00000003`000299c8 000007fe`f5f1d74b 00000001`ffbf9d28 00000000`00000000
System_ServiceModel_ni+61053e 00000000`0e6ce820 000007fe`f1ed0d00 00000000`00000000 000007fe`f8621f10
System_ServiceModel_ni+60fa57 00000000`ffb46740 00000003`04f95618 00000000`00000000 00000000`00000000
System_ServiceModel_ni+60de03 00000003`ffc0fb60 00000001`ffbf92b0 80000000`00000000 00000003`000296d0
System_ServiceModel_ni+60d92b 00000000`00000000 000007fe`f9e33824 00000000`0122f070 00000000`00000000
System_ServiceModel_Internals_ni+4c352 00000000`00000000 00000000`00000000 00000003`054834a0 00000000`0e6cec18
System_ServiceModel_Internals_ni+47d2a 00000004`04299c58 000007fe`f76ef200 00000000`00000000 00000000`0e6ceca0
System_ServiceModel_Internals_ni+4c5f1 00000000`0e6cec20 00000003`04f95890 00000003`04f96060 000007fe`f56a4c65
System_ServiceModel_Internals_ni+4c1cc 00000000`ffb465a0 00000000`0e6cedf0 00000003`05450001 000007fe`f5efdda6
System_ServiceModel_Internals_ni+4bd9e 00000001`ffbf9620 00000000`00000000 00000000`00000000 00000000`00000000
System_ServiceModel_ni+622eb2 00000003`04f95890 00000003`05451a10 00000003`04f957b0 00000003`04f95890
System_ServiceModel_ni+62281f 00000001`ffbf9620 00000003`04f95790 00000003`04f957b0 00000003`04f95780
System_ServiceModel_ni+62268a 00000003`04f95890 000007fe`f4f9b001 00000000`00000000 00000000`00000000
System_ServiceModel_Internals_ni+e93f8 00000003`04f95618 000007fe`f56d0e96 00000003`04f95550 000007fe`f9cc5699
System_ServiceModel_ni+621582 00000001`ffbf9620 00000003`04f95618 00000000`00000001 00000003`04f956d8
System_ServiceModel_ni+620c38 00000003`04f956d8 00000003`ffc0f940 00000001`01d742a8 000007fe`ef858cb0
System_ServiceModel_ni+620a2a 00000000`00000000 00000000`00000000 00000003`ffc0fb60 00000003`04f949b8
System_ServiceModel_Activation_ni+71d48 00000002`00040004 00000001`02b1f848 00000003`04f953a8 000007fe`9a7aad00
System_ServiceModel_Activation_ni+6ac44 00000001`01d742a8 00000001`ffb3a450 00000000`0e6cf608 00000000`00000000
System_ServiceModel_Activation_ni+6ae35 00000001`0173afd8 000007fe`f8617fd7 00000000`0e6cf280 00000000`00000000
System_ServiceModel_Activation_ni+6aec3 00000003`ffc0daf0 00000000`00000000 00000003`04f94528 000007fe`ef84c79c
System_ServiceModel_Activation_ni+2c7a2 00000001`ffb3a498 00000001`ffb384a8 00000000`d2d6d2d7 000007fe`f4f9e809
System_ServiceModel_Activation_ni+6afe3 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
System_ServiceModel_Internals_ni+4e651 00000000`0e6cf4c0 000007fe`fa0bb84b 00000000`0f48e8a0 00000000`00000002
System_ServiceModel_Internals_ni+4e5d0 00000000`0e6cf550 00000000`0e6cf470 00000000`0e6cf5e0 00000001`ffb384c8
mscorlib_ni+534fa6 00000000`00000000 00000000`00000000 00000001`ffb3a498 00000000`11dc0470
clr!CallDescrWorkerInternal+83 00000000`10b0ec30 00000000`11f43e20 00000000`001d0000 00000000`00000468
clr!CallDescrWorkerWithHandler+4a 00000000`00000000 00000000`0e6cf5e0 00000000`0e6cf608 000007fe`f854af18
clr!DispatchCallSimple+85 00000000`0e6cfca0 00000001`ffb3a498 00000000`00000000 00000000`00000000
clr!BindIoCompletionCallBack_Worker+b8 00000001`ffb3a450 00000000`0e6cfc10 00000000`0e6cfcc0 00000000`0e6cfc10
clr!DebuggerU2MCatchHandlerFrame::DebuggerU2MCatchHandlerFrame+a0 00000000`0e6cfc10 00000000`00000000 00000000`120a12f8 000007fe`ecad6585
clr!ManagedPerAppDomainTPCount::DispatchWorkItem+38e 00000000`ffffffff 000007fe`f9c63108 00000000`0f48e8a0 000007fe`f9c63108
clr!ManagedPerAppDomainTPCount::DispatchWorkItem+2bd 00000000`0e6cf9b8 00000000`0f48e8a0 00000000`0e6cfc10 00000000`0f48e8a0
clr!ReturnToPreviousAppDomainHolder::Init+39 00000000`00000000 00000000`023e0340 00000000`0e6cfcc0 00000000`0e6cfcc0
clr!Thread::DoADCallBack+234 00000000`00000002 00000000`00000002 00000000`0e6cfc10 00000000`0e6cfc10
clr!DebuggerU2MCatchHandlerFrame::DebuggerU2MCatchHandlerFrame+d6 00000000`0e6cfc10 00000000`00000000 00000000`00000030 00000000`120a0000
clr!ManagedPerAppDomainTPCount::DispatchWorkItem+38e 00000000`80004005 00000000`0e6cfc40 000007fe`fa324dd8 00000000`0f48e8a0
clr!ManagedPerAppDomainTPCount::DispatchWorkItem+2bd 80000000`00000000 00000000`0f48e8a0 00000000`0f48e8a0 00000000`00000000
clr!ManagedPerAppDomainTPCount::DispatchWorkItem+23b 00000000`00000002 00000000`0e6cfcf0 00000000`00008000 000007fe`fd5216ad
clr!BindIoCompletionCallbackStubEx+cd 00000001`00040004 00000000`00000001 00000000`0f48e8a0 00000001`00000001
clr!BindIoCompletionCallbackStub+9 00000002`00040004 00000002`00040004 00000000`00000001 00000000`00000000
clr!ThreadpoolMgr::CompletionPortThreadStart+361 00000000`00000000 00000000`00000000 00000000`00000001 00000000`00000000
clr!Thread::intermediateThreadProc+7d 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
kernel32!BaseThreadInitThunk+d 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
ntdll!RtlUserThreadStart+1d 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
Exception Information
CLR!GETRUNTIMEASSEMBLYHELPER+1DIn WER45CE.tmp.mdmp the assembly instruction at clr!GetRuntimeAssemblyHelper+1d in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll from Microsoft Corporation has caused a stack overflow exception (0xC00000FD) when trying to write to memory location 0x0e655ff8 on thread 49
The details in the event log does point out to a serialization problem, however I don't recall serialization problems causing the w3wp to crash.
First of all, turn on WCF tracing to get the WCF logs and see the exact serialization issue (if that is the case).
http://msdn.microsoft.com/en-us/library/ms733025.aspx
Second, regarding fail-over, if an application pool crashes it will automatically restart itself. However, if after a restart the pool crashes again, several times during a given period (5 crashed during 5 minutes is the default) the application pool will be stopped to prevent more crashes - this should have an indication in the event log.
As for the serialization issue - if you can't figure out the issue from the trace, can you possibly publish the xml content here? or upload it somewhere and provide a link? If you are using an HTTP binding you can simply open Fiddler and copy the request content. If you are using TCP or other bindings, you can use the WCF Message Logging to log the message (don't forget to set LogEntireMessage to true).
http://msdn.microsoft.com/en-us/library/ms730064.aspx
There is a hotfix out to potentially fix the stack overflow:
http://support.microsoft.com/kb/2640103
Is there any way to monitor service and restart in case of such crash inside IIS?
Have you tried turning off Rapid Fail Protection in IIS? Rapid failure protection will stop your w3p service if concurrent failures happen within the specified interval. see http://www.iisunderground.com/disable-rapid-fail-protection/
Causes:
A worker process does not start within the startup time limit.
A worker process does not shut down within the shutdown time limit.
A worker process shuts itself down because of a fatal error and sends the WWW service an error code.
A worker process fails to respond to a ping message.
The WWW service detects that a worker process is sending non-standard communications (the worker process may have been taken over).
Have you tried setting up diagnostics in config file?
<system.diagnostics>
<trace autoflush="true" />
<sources>
<source name="System.ServiceModel"
switchValue ="Critical, Error"
propagateActivity="true">
<listeners>
<add name="sdt"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "c:\logs\servicelog.e2e" />
</listeners>
</source>
</sources>
</system.diagnostics>
You need to add permission to service to write to C:\logs or whichever folder you select. This has helped me many times to get low level stack information on which method has exception occured.
You have the 64bit version of the framework on the server, was the app developed on a 32 bit system then deployed?
If yes, re build app on 64 bit system then re deploy.
Also any 3rd party dll's you have refererenced in your app, you may need to aquire thier 64bit equivalents
I was seeing a similar issue. In my case I had added aspNetCompatibilityEnabled=”true” to my web.config and I was running in asp.net 2.0 compatibility mode. This KB article describes the problem in more detail
https://support.microsoft.com/en-us/kb/977104
Related
Faulting application name: httpd.exe, version: 2.4.48.0, time stamp: 0x60a37f00
Faulting module name: opencv_videoio_ffmpeg460_64.dll, version: 2022.5.0.0, time stamp: 0x00000000
Exception code: 0x40000015
Fault offset: 0x0000000000f7fd52
Faulting process id: 0x102d0
Faulting application start time: 0x01d913a882251b82
Faulting application path: C:\AI_People_Counter\Software_Files\Apache24\bin\httpd.exe
Faulting module path: C:\Program Files\Python38\Lib\site-packages\cv2\opencv_videoio_ffmpeg460_64.dll
Report Id: 63bbc67f-ae93-4b02-83d9-f9541a09f63b
Faulting package full name:
Faulting package-relative application ID:
How can I resolve this issue?
When I launch a detox test with an emulator(one that I specify in detox config file) already running, detox finds the emulator, attaches it to it, and runs tests on it.
When I don't have an emulator running, it helps load the emulator itself. But then when detox tries to install the app on the emulator next, it gets stuck and then eventually fails.
Trace logs(trimmed out artifacts config from the log for brevity).
Goto the log entry related to ALLOCATE_DEVICE after which it should've installed the device.
$ detox test -c android.emu.release --loglevel trace
00:24:21.224 detox[17068] INFO: [test.js] DETOX_CONFIGURATION="android.emu.release" DETOX_LOGLEVEL="trace" DETOX_REPORT_SPECS=true DETOX_START_TIMESTAMP=1663526361217 DETOX_USE_CUSTOM_LOGGER=true jest --config D:\Myapp\app\mobile\e2e\config.json --testNamePattern "^((?!:ios:).)*$" e2e
00:24:26.614 detox[19864] TRACE: [DETOX_CREATE] created a Detox instance with config:
{
appsConfig: {
default: {
type: 'android.apk',
binaryPath: 'android/app/build/outputs/apk/release/app-release.apk',
testBinaryPath: 'android/app/build/outputs/apk/androidTest/release/app-release-androidTest.apk',
build: 'cd android && gradlew clean && gradlew assembleRelease assembleAndroidTest -DtestBuildType=release && cd ..'
}
},
behaviorConfig: {
init: {
reinstallApp: true,
exposeGlobals: true
},
cleanup: {
shutdownDevice: false
},
launchApp: 'auto'
},
cliConfig: {
configuration: 'android.emu.release',
loglevel: 'trace',
useCustomLogger: true
},
configurationName: 'android.emu.release',
deviceConfig: {
type: 'android.emulator',
gpuMode: 'swiftshader_indirect',
device: {
avdName: '31_aosp'
}
},
runnerConfig: {
testRunner: 'jest',
runnerConfig: 'D:\\Myappp\\app\\mobile\\e2e\\config.json',
specs: 'e2e',
skipLegacyWorkersInjection: true
},
sessionConfig: {
autoStart: true,
sessionId: '8a6a88d9-1e5b-5df3-6cda-4c90f05b1e3b',
debugSynchronization: 10000
}
}
00:24:26.625 detox[19864] DEBUG: [WSS_CREATE] Detox server listening on localhost:51367...
00:24:26.643 detox[19864] DEBUG: [WSS_CONNECTION, #51368] registered a new connection.
00:24:26.645 detox[19864] TRACE: [WS_OPEN] opened web socket to: ws://localhost:51367
00:24:26.647 detox[19864] TRACE: [WS_SEND] {"type":"login","params":{"sessionId":"8a6a88d9-1e5b-5df3-6cda-4c90f05b1e3b","role":"tester"},"messageId":0}
00:24:26.649 detox[19864] TRACE: [WSS_GET_FROM, #51368] {"type":"login","params":{"sessionId":"8a6a88d9-1e5b-5df3-6cda-4c90f05b1e3b","role":"tester"},"messageId":0}
00:24:26.650 detox[19864] TRACE: [SESSION_CREATED] created session 8a6a88d9-1e5b-5df3-6cda-4c90f05b1e3b
00:24:26.651 detox[19864] TRACE: [WSS_SEND_TO, #tester] {"type":"loginSuccess","params":{"testerConnected":true,"appConnected":false},"messageId":0}
00:24:26.652 detox[19864] TRACE: [SESSION_JOINED] tester joined session 8a6a88d9-1e5b-5df3-6cda-4c90f05b1e3b
00:24:26.653 detox[19864] TRACE: [WS_MESSAGE] {"type":"loginSuccess","params":{"testerConnected":true,"appConnected":false},"messageId":0}
00:24:26.754 detox[19864] DEBUG: [EXEC_CMD, #0] "C:\Users\PRADEEP\AppData\Local\Android\Sdk\emulator\emulator.EXE" -list-avds --verbose
00:24:26.826 detox[19864] TRACE: [EXEC_SUCCESS, #0] 31_aosp
31_api
31_playstore
00:24:26.827 detox[19864] DEBUG: [EXEC_CMD, #1] "C:\Users\PRADEEP\AppData\Local\Android\Sdk\emulator\emulator.EXE" -version
00:24:27.413 detox[19864] TRACE: [EXEC_SUCCESS, #1] INFO | Duplicate loglines will be removed, if you wish to see each indiviudal line launch with the -log-nofilter flag.
Android emulator version 31.3.10.0 (build_id 8807927) (CL:N/A)
Copyright (C) 2006-2017 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).
This software is licensed under the terms of the GNU General Public
License version 2, as published by the Free Software Foundation, and
may be copied, distributed, and modified under those terms.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
INFO | Android emulator version 31.3.10.0 (build_id 8807927) (CL:N/A)
00:24:27.416 detox[19864] DEBUG: [EMU_BIN_VERSION_DETECT] Detected emulator binary version { major: 31, minor: 3, patch: 10, toString: [Function: toString] }
00:24:27.416 detox[19864] DEBUG: [ALLOCATE_DEVICE] Trying to allocate a device based on "31_aosp"
00:24:27.419 detox[19864] DEBUG: [EXEC_CMD, #2] "C:\Users\PRADEEP\AppData\Local\Android\Sdk\platform-tools\adb.EXE" devices
00:24:27.520 detox[19864] DEBUG: [EXEC_SUCCESS, #2] List of devices attached
00:24:27.523 detox[19864] DEBUG: [ALLOCATE_DEVICE] Settled on emulator-12586
00:24:27.524 detox[19864] DEBUG: [SPAWN_CMD] C:\Users\PRADEEP\AppData\Local\Android\Sdk\emulator\emulator.EXE -verbose -no-audio -no-boot-anim -gpu swiftshader_indirect -port 12586 #31_aosp
00:26:26.610 detox[19864] ERROR: [DETOX_INIT_ERROR]
DetoxRuntimeError: Aborted detox.init() execution, and now running detox.cleanup()
HINT: Most likely, your test runner is tearing down the suite due to the timeout error
at DetoxRuntimeErrorComposer.abortedDetoxInit (D:\Myapp\app\mobile\node_modules\detox\src\errors\DetoxRuntimeErrorComposer.js:13:12)
at Detox.[_assertNoPendingInit] (D:\Myapp\app\mobile\node_modules\detox\src\Detox.js:238:48)
at Detox.beforeEach (D:\Myapp\app\mobile\node_modules\detox\src\Detox.js:122:37)
at DetoxExportWrapper.<computed> (D:\Myapp\app\mobile\node_modules\detox\src\DetoxExportWrapper.js:94:32)
at DetoxAdapterImpl.beforeEach (D:\Myapp\app\mobile\node_modules\detox\runners\jest\DetoxAdapterImpl.js:28:22)
at DetoxAdapterJasmine.beforeEach (D:\Myapp\app\mobile\node_modules\detox\runners\jest\DetoxAdapterJasmine.js:20:5)
at Object.<anonymous> (D:\Myapp\app\mobile\e2e\init.ts:21:3)
00:26:26.617 detox[19864] TRACE: [ARTIFACTS_LIFECYCLE] artifactsManager.onBeforeCleanup()
00:26:26.622 detox[19864] DEBUG: [WSS_CLOSE] Detox server has been closed gracefully
I use selenium + opensuse + codeception + php7.3 + firefox91 + geckodriver
My acceptance.suite.yml
class_name: AcceptanceTester
modules:
enabled:
- WebDriver:
url: 'http://suz97.rp.ru/'
browser: firefox # 'chrome' or 'firefox'
- Yii2:
part: [ orm, fixtures ]
entryScript: index.php
My php test is
<?php
use yii\helpers\Url as Url;
class LoginCest
{
public function ensureThatLoginWorks(\AcceptanceTester $I)
{
$I->amOnPage(Url::toRoute('/'));
$I->see('login');
and selenium answer is
17:17:30.682 INFO [LocalDistributor.newSession] - Session request received by the distributor:
[Capabilities {browserName: firefox}]
1638454650685 geckodriver INFO Listening on 127.0.0.1:49233
1638454650695 mozrunner::runner INFO Running command: "/usr/bin/firefox" "--marionette" "-no-remote" "-profile" "/tmp/rust_mozprofileLUqga9"
1638454650965 Marionette INFO Marionette enabled
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at /tmp/rust_mozprofileLUqga9/search.json.mozlz4", (void 0)))
console.error: Region.jsm: "Error fetching region" (new TypeError("NetworkError when attempting to fetch resource.", ""))
console.error: Region.jsm: "Failed to fetch region" (new Error("NO_RESULT", "resource://gre/modules/Region.jsm", 419))
1638454653723 Marionette INFO Listening on port 38273
17:17:33.835 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
17:17:33.837 INFO [LocalDistributor.newSession] - Session created by the distributor. Id: 750142cf-f983-4263-93c1-6068155dddc5, Caps: Capabilities {acceptInsecureCerts: false, browserName: firefox, browserVersion: 91.3.0, moz:accessibilityChecks: false, moz:buildID: 20211028170545, moz:geckodriverVersion: 0.30.0, moz:headless: false, moz:processID: 23483, moz:profile: /tmp/rust_mozprofileLUqga9, moz:shutdownTimeout: 60000, moz:useNonSpecCompliantPointerOrigin: false, moz:webdriverClick: true, pageLoadStrategy: normal, platformName: linux, platformVersion: 5.3.18-lp152.41-default, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
1638454654371 Marionette INFO Stopped listening on port 38273
JavaScript error: resource://activity-stream/lib/ActivityStreamPrefs.jsm, line 27: NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIPrefBranch.removeObserver]
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "Sqlite.jsm: wait until all connections are closed" is finished, it is too late to register completion condition "protections.sqlite#0: waiting for shutdown"
JavaScript error: resource://activity-stream/lib/PlacesFeed.jsm, line 173: NS_ERROR_UNEXPECTED: Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsINavBookmarksService.addObserver]
JavaScript error: resource://gre/modules/ExtensionSettingsStore.jsm, line 122: Error: The ExtensionSettingsStore was accessed before the initialize promise resolved.
console.log: "RemoteSettingsWorker error: Error: Can't import when we've started shutting down."
console.error: "Could not load engine google#search.mozilla.org: [Exception... \"AddonManager is not initialized\" nsresult: \"0xc1f30001 (NS_ERROR_NOT_INITIALIZED)\" location: \"JS frame :: resource://gre/modules/AddonManager.jsm :: installBuiltinAddon :: line 2536\" data: no]"
console.warn: SearchService: "_init: abandoning init due to shutting down"
JavaScript error: , line 0: uncaught exception: 2147500036
JavaScript error: resource://gre/modules/JSONFile.jsm, line 196: Error: Data is not ready.
JavaScript error: , line 0: AbortError: A request was aborted, for example through a call to IDBTransaction.abort.
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
what could be the problem ?
Error snapshot:
This error message...
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at /tmp/rust_mozprofileLUqga9/search.json.mozlz4", (void 0)))
...implies that Selenium driven GeckoDriver was unable to initiate firefox Browsing Context.
Possibly you have stepped into the classic case of rust_mozprofile folders are cluttering TEMP.
As #andreastt mentions:
It’s a known bug that geckodriver sometimes leaves the throwaway
profile behind, see
#299.
Setting the TMPDIR environment variable is the only way to configure
where the throwaway profiles are created. You don’t have to set it
system-wide: just make sure it is part of the geckodriver process’
environment before it is started:
% TMPDIR=/some/location ./geckodriver
#andreastt further added:
I just discovered that TMPDIR only applies to Unix systems. On Windows
the following
variables are
read, in order:
TMP
TEMP
USERPROFILE
Quick Solution
A quick solution would be to delete the rust_mozprofile sub-directories from the temp directories.
tl; dr
Retrieve the crash data
at the time of running titanium application i am getting error, i have tried all the solutions suggested by google, but problem still remains, please help if anyone can understand the error log i pasted below. Thanks.
Appcelerator Command-Line Interface, version 4.1.2
Copyright (c) 2014-2015, Appcelerator, Inc. All Rights Reserved.
TRACE | __command__ search paths:
[
"C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package",
"C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package\\node_modules",
"C:\\Users\\acer\\Desktop\\node_modules",
"C:\\Users\\acer\\node_modules",
"C:\\Users\\node_modules",
"C:\\node_modules",
"C:\\Users\\acer\\.appcelerator\\.npm\\lib\\node_modules"
]
DEBUG | [PLUGIN-LOAD] 0ms C:\Users\acer\.appcelerator\install\4.1.2\package\appc.js
DEBUG | [PLUGIN-LOAD] 623ms C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\appc-cli-titanium\appc.js
DEBUG | [PLUGIN-LOAD] 2ms C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\arrow\appc.js
log level set to "trace"
executing command "run"
set environment to {"registry":"https://software.appcelerator.com","security":"https://security.appcelerator.com","baseurl":"https://platform.appcelerator.com"}
checking credentials for existing session
Attempting to load session info from config file
check if session is invalidated
session expiry 1439531789135 false
+---------------------------------------------------------------------------------------+
+ This is a Developer trial account. You may use this software for evaluation purposes. +
+ Once you are ready to go to production, upgrade at https://billing.appcelerator.com +
+---------------------------------------------------------------------------------------+
Arrow Cloud config file: C:\Users\acer\.acs
found Arrow Cloud login { mid: 'dfa5b79881cc650462ff93ae5df5f65d5760e1cd',
publishPort: 443,
publishHost: 'https://admin.cloudapp-enterprise.appcelerator.com',
username: 'mymail',
cookie: [ 'connect.sid=s%3Ab9eP%2F3NylwuSkAqEtFC3La1k.RDNuz84uHznd9QjZUQ9Rm6UxHfan0GZcGKug2GtaQjg; Path=/; Expires=Fri, 14 Aug 2015 05:56:35 GMT; HttpOnly' ],
defaultEP:
{ publishHost: 'https://admin.cloudapp-enterprise.appcelerator.com',
publishPort: 443 } } , checking nodeACSEndpoint= https://admin.cloudapp-enterprise.appcelerator.com
Arrow Cloud cookie expiry [ 1439531795000 ]
session already loaded in opts.session
getCredentials() session:
{
"ipaddress": "192.168.1.13",
"username": "mymail",
"password": "<OMITTED>",
"session": "<OMITTED>",
"nonce": "<OMITTED>",
"environment": {
"name": "production",
"isProduction": true,
"acsBaseUrl": "https://api.cloud.appcelerator.com",
"acsAuthBaseUrl": "https://secure-identity.cloud.appcelerator.com",
"nodeACSEndpoint": "https://admin.cloudapp-enterprise.appcelerator.com"
},
"token": "<OMITTED>",
"fingerprint": "dfa5b79881cc650462ff93ae5df5f65d5760e1cd",
"fingerprint_description": "Windows Machine ID: bcd68e35-cc53-4567-ab68-66246b73cc67",
"org_id": 100057656,
"expiry": 1439531789135
}
loading plugins for command "run"
run search paths:
[
"C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package",
"C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package\\node_modules",
"C:\\Users\\acer\\Desktop\\node_modules",
"C:\\Users\\acer\\node_modules",
"C:\\Users\\node_modules",
"C:\\node_modules",
"C:\\Users\\acer\\.appcelerator\\.npm\\lib\\node_modules"
]
[PLUGIN-LOAD] 0ms C:\Users\acer\.appcelerator\install\4.1.2\package\appc.js
[PLUGIN-LOAD] 251ms C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\appc-cli-titanium\appc.js
run plugin: C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\appc-cli-titanium
[PLUGIN-LOAD] 0ms C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\arrow\appc.js
run plugin: C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\arrow
plugin "arrow" failed its "when" function check, skipping...
loading plugin "titanium" for command "run" CLI options via function
loading plugin "titanium" for command "run" CLI options via array
Duplicate option "colors" for command "run", removing...
executing command "run" with the following plugins:
["titanium"]
TRACE | Attempting to load session info from config file
TRACE | check if session is invalidated
TRACE | session expiry 1439531789135 false
TRACE | session already loaded in opts.session
DEBUG | Titanium Downloads Last Checked: 1439447168501
TRACE | "C:\Program Files\nodejs\node.exe" "C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\appc-cli-titanium\node_modules\titanium\bin\titanium" config -o json-object
TRACE | "C:\Program Files\nodejs\node.exe" "C:\Users\acer\.appcelerator\install\4.1.2\package\node_modules\appc-cli-titanium\node_modules\titanium\bin\titanium" sdk -o json
TRACE | checking for titanium, result:
{ activeSDK: '4.1.0.GA',
defaultInstallLocation: 'C:\\ProgramData\\Titanium',
installLocations:
[ 'C:\\ProgramData\\Titanium',
'C:\\Users\\acer\\AppData\\Roaming\\Titanium',
'C:\\ProgramData\\Application Data\\Titanium' ],
installed: { '4.1.0.GA': 'C:\\ProgramData\\Titanium\\mobilesdk\\win32\\4.1.0.GA' } }
TRACE | C:\Program Files\nodejs\node.exe [ 'C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package\\node_modules\\appc-cli-titanium\\node_modules\\titanium\\bin\\titanium',
'build',
'--platform',
'android',
'--log-level',
'trace',
'--sdk',
'4.1.0.GA',
'--project-dir',
'D:\\Titanium WS\\ex',
'--target',
'emulator',
'--android-sdk',
'C:\\android-sdk-win',
'--device-id',
'Appp',
'--skip-js-minify',
'--no-colors',
'--no-progress-bars',
'--no-prompt',
'--prompt-type',
'socket-bundle',
'--prompt-port',
'62727',
'--plugin-paths',
'C:\\Users\\acer\\.appcelerator\\install\\4.1.2\\package\\node_modules',
'--config-file',
'C:\\Users\\acer\\AppData\\Local\\Temp\\build-1439448105417.json',
'--no-banner' ]
[WARN] :
TRACE | titanium exited with exit code 1
[ERROR] Application Installer abnormal process termination. Process exit value was 1
After 24 hours i get my project installing on my device. i am unable to get the exact problem but i want to share my tried steps to solve the problem.
I installed all sdk's one by one.
I switched my current work space and created new one. (old work space still gives error)
Now my project is installing fine, every time.
Thanks.
I met a problem while creating
"New Domain" iso/nfs storage,it prints "Error while executing action
New NFS Storage Domain: Storage domain remote path not mounted"
and the error code is 477.
I followed the "http://wiki.ovirt.org/wiki/Troubleshooting_NFS_Storage_Issues" and find that the vdsm user can't now use mount.
"mount: only root can do that"
the version I use:
oVirt Engine Version: 3.1.0-2.fc17
oVirt Node Hypervisor 2.5.4-0.1.fc17
the error log:
2012-11-08 09:15:00,004 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-77) Checking autorecoverable storage domains done
2012-11-08 09:17:28,920 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (ajp--0.0.0.0-8009-2) calling GetConfigurationValueQuery (StorageDomainNameSizeLimit) with null version,
using default general for version
2012-11-08 09:17:29,333 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (ajp--0.0.0.0-8009-3) [7720b88f] START, ValidateStorageServerConnectionVDSCommand(vdsId = 12bcf124-29a4-11e2-bcba-00505680002a, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6556c55d-42a4-4dcc-832c-4d8987ebe6bd, connection: 200.200.101.219:/usr/lwq/iso };]), log id: 52777a80
2012-11-08 09:17:29,388 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (ajp--0.0.0.0-8009-3) [7720b88f] FINISH, ValidateStorageServerConnectionVDSCommand, return: {6556c55d-42a4-4dcc-832c-4d8987ebe6bd=0}, log id: 52777a80
2012-11-08 09:17:29,392 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp--0.0.0.0-8009-3) [7720b88f] Running command: AddStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-11-08 09:17:29,404 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp--0.0.0.0-8009-3) [7720b88f] START, ConnectStorageServerVDSCommand(vdsId = 12bcf124-29a4-11e2-bcba-00505680002a, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6556c55d-42a4-4dcc-832c-4d8987ebe6bd, connection: 200.200.101.219:/usr/lwq/iso };]), log id: 36cb94f
2012-11-08 09:17:29,656 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp--0.0.0.0-8009-3) [7720b88f] FINISH, ConnectStorageServerVDSCommand, return: {6556c55d-42a4-4dcc-832c-4d8987ebe6bd=477}, log id: 36cb94f
2012-11-08 09:17:29,658 ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper] (ajp--0.0.0.0-8009-3) [7720b88f] The connection with details 200.200.101.219:/usr/lwq/iso failed because of
error code 477 and error message is: 477
2012-11-08 09:17:29,717 INFO [org.ovirt.engine.core.bll.storage.AddNFSStorageDomainCommand] (ajp--0.0.0.0-8009-11) [1661aa36] Running command: AddNFSStorageDomainCommand internal: false. En
tities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-11-08 09:17:29,740 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--0.0.0.0-8009-11) [1661aa36] START, CreateStorageDomainVDSCommand(vdsId = 12bcf12
4-29a4-11e2-bcba-00505680002a, storageDomain=org.ovirt.engine.core.common.businessentities.storage_domain_static#4a900545, args=200.200.101.219:/usr/lwq/iso), log id: 50b803a0
2012-11-08 09:17:35,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp--0.0.0.0-8009-11) [1661aa36] Failed in CreateStorageDomainVDS method
2012-11-08 09:17:35,234 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp--0.0.0.0-8009-11) [1661aa36] Error code StorageDomainFSNotMounted and error message VDSGeneri
cException: VDSErrorException: Failed to CreateStorageDomainVDS, error = Storage domain remote path not mounted: ('/rhev/data-center/mnt/200.200.101.219:_usr_lwq_iso',)
2012-11-08 09:17:35,260 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp--0.0.0.0-8009-11) [1661aa36] Command org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageD
omainVDSCommand return value
As far as I know VDSM does sudo to run privileged commands, but still that mount error is strange. Can you send share the vdsm log?
Also, you may get more attention on the engine-users or vdsm mailing lists.