Subversion checkout fails with ACCESS_VIOLATION in svn_wc_add_repos_file4() - apache

Question

We reproducibly experience errors when attempting to checkout or update working copies.
Environment
Our environment is as follows:-
svn 1.8.9 (r1591380) client and server running on the same server (also happens with client on another server, but less often)
Server runs Windows Server 2008 (64 bit)
Apache httpd server
We are running the svn checkout from QuickBuild.
Client Error
This error is reported from the checkout:-
svn checkout http://qvsvn101/PayWay/PayWay/Branches/2014.R1/ D:\quickbuild_workspace\PayWay\Application\PointRelease\Release --non-interactive --username SrvAcc --password ****** -r 11523
Command return code: -1073741819
Command error output: This application has halted due to an unexpected error.
A crash report and minidump file were saved to disk, you can find them here:
C:\Users\SrvAcc\AppData\Local\Temp\svn-crash-log20140527164109.log
C:\Users\SrvAcc\AppData\Local\Temp\svn-crash-log20140527164109.dmp
Please send the log file to users_at_subversion.apache.org to help us analyze
and solve this problem.
NOTE: The crash report and minidump files can contain some sensitive information
(filenames, partial file content, usernames and passwords etc.)
Apache httpd error log
At the same time, the Apache error.log contains this:
[Tue May 27 16:41:12 2014] [error] [client 192.168.40.47] Provider encountered an error while streaming a REPORT response. [500, #0]
[Tue May 27 16:41:12 2014] [error] [client 192.168.40.47] A failure occurred while driving the update report editor [500, #106]
Subversion Crash Log File
Subversion writes out a log file as follows:
Process info:
Cmd line: svn checkout http://qvsvn101/PayWay/PayWay/Branches/2014.R1/ D:\quickbuild_workspace\PayWay\Application\PointRelease\Release --non-interactive --username SrvAcc --password ****** -r 11523
Working Dir: D:\quickbuild_workspace\PayWay\Application\PointRelease\Release
Version: 1.8.9 (r1591380), compiled May 8 2014, 04:25:41
Platform: Windows OS version 6.1 build 7601 Service Pack 1
Exception: ACCESS_VIOLATION
Registers:
eax=7259a7c0 ebx=00000000 ecx=01e5e138 edx=72746e65 esi=01e5e138 edi=61006469
eip=7259a7cf esp=003cf44c ebp=003cf458 efl=00010202
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b
Stacktrace:
#1 0x7259a7cf in svn_wc_add_repos_file4()
#2 0x732396b7 in svn_ra_svn_init()
#3 0x7323470b in svn_ra_svn_init()
#4 0x73234abe in svn_ra_svn_init()
#5 0x731f3110 in (unknown function)
Loaded modules:
0x00020000 D:\SubversionServer\svn.exe (1.8.9.18516, 241664 bytes)
0x77a20000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247, 1572864 bytes)
0x75f00000 C:\Windows\SysWOW64\kernel32.dll (6.1.7601.18229, 1114112 bytes)
0x75e00000 C:\Windows\SysWOW64\KERNELBASE.dll (6.1.7601.18229, 290816 bytes)
0x73600000 D:\SubversionServer\libapr-1.dll (1.5.1.0, 163840 bytes)
0x764a0000 C:\Windows\SysWOW64\ws2_32.dll (6.1.7601.17514, 217088 bytes)
0x75800000 C:\Windows\SysWOW64\msvcrt.dll (7.0.7601.17744, 704512 bytes)
0x75350000 C:\Windows\SysWOW64\rpcrt4.dll (6.1.7601.18205, 983040 bytes)
0x75100000 C:\Windows\SysWOW64\sspicli.dll (6.1.7601.18270, 393216 bytes)
0x750f0000 C:\Windows\SysWOW64\CRYPTBASE.dll (6.1.7600.16385, 49152 bytes)
0x75ee0000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385, 102400 bytes)
0x75950000 C:\Windows\SysWOW64\nsi.dll (6.1.7600.16385, 24576 bytes)
0x74510000 C:\Windows\System32\mswsock.dll (6.1.7601.18254, 245760 bytes)
0x76010000 C:\Windows\SysWOW64\user32.dll (6.1.7601.17514, 1048576 bytes)
0x75960000 C:\Windows\SysWOW64\gdi32.dll (6.1.7601.18275, 589824 bytes)
0x779f0000 C:\Windows\SysWOW64\lpk.dll (6.1.7601.18177, 40960 bytes)
0x758b0000 C:\Windows\SysWOW64\usp10.dll (1.626.7601.18009, 643072 bytes)
0x759f0000 C:\Windows\SysWOW64\advapi32.dll (6.1.7601.18247, 655360 bytes)
0x76510000 C:\Windows\SysWOW64\shell32.dll (6.1.7601.18222, 12886016 bytes)
0x75e70000 C:\Windows\SysWOW64\shlwapi.dll (6.1.7601.17514, 356352 bytes)
0x73260000 D:\SubversionServer\MSVCR100.DLL (10.0.30319.1, 778240 bytes)
0x73580000 D:\SubversionServer\libsvn_client-1.dll (1.8.9.18516, 319488 bytes)
0x74010000 D:\SubversionServer\libsvn_delta-1.dll (1.8.9.18516, 122880 bytes)
0x733e0000 D:\SubversionServer\libaprutil-1.dll (1.5.3.0, 204800 bytes)
0x73ee0000 D:\SubversionServer\libapriconv-1.dll (1.2.1.0, 36864 bytes)
0x72a80000 D:\SubversionServer\libsvn_subr-1.dll (1.8.9.18516, 1077248 bytes)
0x73670000 C:\Windows\System32\shfolder.dll (6.1.7600.16385, 20480 bytes)
0x755c0000 C:\Windows\SysWOW64\ole32.dll (6.1.7601.17514, 1425408 bytes)
0x75230000 C:\Windows\SysWOW64\crypt32.dll (6.1.7601.18277, 1179648 bytes)
0x75ed0000 C:\Windows\SysWOW64\msasn1.dll (6.1.7601.17514, 49152 bytes)
0x74a30000 C:\Windows\System32\version.dll (6.1.7600.16385, 36864 bytes)
0x73cc0000 D:\SubversionServer\libsvn_diff-1.dll (1.8.9.18516, 86016 bytes)
0x731f0000 D:\SubversionServer\libsvn_ra-1.dll (1.8.9.18516, 454656 bytes)
0x738f0000 D:\SubversionServer\libsasl.dll (2.1.23.0, 81920 bytes)
0x73f10000 D:\SubversionServer\libsvn_fs-1.dll (1.8.9.18516, 225280 bytes)
0x73f60000 D:\SubversionServer\libsvn_repos-1.dll (1.8.9.18516, 180224 bytes)
0x735d0000 C:\Windows\System32\secur32.dll (6.1.7601.18270, 32768 bytes)
0x731a0000 D:\SubversionServer\ssleay32.dll (1.0.1.7, 286720 bytes)
0x72940000 D:\SubversionServer\libeay32.dll (1.0.1.7, 1306624 bytes)
0x72550000 D:\SubversionServer\libsvn_wc-1.dll (1.8.9.18516, 544768 bytes)
0x76340000 C:\Windows\System32\imm32.dll (6.1.7601.17514, 393216 bytes)
0x75160000 C:\Windows\SysWOW64\msctf.dll (6.1.7600.16385, 835584 bytes)
0x74960000 C:\Windows\System32\profapi.dll (6.1.7600.16385, 45056 bytes)
0x744f0000 C:\Windows\System32\nlaapi.dll (6.1.7601.17761, 65536 bytes)
0x744e0000 C:\Windows\System32\NapiNSP.dll (6.1.7600.16385, 65536 bytes)
0x742c0000 C:\Windows\System32\dnsapi.dll (6.1.7601.17570, 278528 bytes)
0x744d0000 C:\Windows\System32\winrnr.dll (6.1.7600.16385, 32768 bytes)
0x742a0000 C:\Windows\System32\IPHLPAPI.DLL (6.1.7601.17514, 114688 bytes)
0x74290000 C:\Windows\System32\winnsi.dll (6.1.7600.16385, 28672 bytes)
0x74250000 C:\Windows\System32\FWPUCLNT.DLL (6.1.7601.18283, 229376 bytes)
0x74240000 C:\Windows\System32\rasadhlp.dll (6.1.7600.16385, 24576 bytes)
0x74500000 C:\Windows\System32\WSHTCPIP.DLL (6.1.7600.16385, 20480 bytes)
0x74ae0000 C:\Windows\System32\dbghelp.dll (6.1.7601.17514, 962560 bytes)
0x73170000 C:\Windows\System32\powrprof.dll (6.1.7600.16385, 151552 bytes)
0x76110000 C:\Windows\SysWOW64\setupapi.dll (6.1.7601.17514, 1691648 bytes)
0x75470000 C:\Windows\SysWOW64\cfgmgr32.dll (6.1.7601.17621, 159744 bytes)
0x763a0000 C:\Windows\SysWOW64\oleaut32.dll (6.1.7601.17676, 585728 bytes)
0x75e50000 C:\Windows\SysWOW64\devobj.dll (6.1.7601.17621, 73728 bytes)

Answer 1

Here is the related thread in dev# Subversion mailing list.
It looks like your server interrupts the connection and Subversion 1.8 client built with serf 1.3.5 library crashes on this failure. That's why you see the error with older Subversion client but the client built with serf 1.3.5 crashes.
Serf 1.3.5 fails to process the error and thus crash on the client occurs. There is a great chance that the crash is caused by the bug in Serf library (on client side) which is fixed in the version 1.3.6:
Revert r2319 from serf 1.3.5: this change was making serf call
handle_response multiple times in case of an error response, leading
to unexpected behavior.
I suggest trying Subversion command-line client which is built against Serf 1.3.6. Subversion 1.8.x binaries built with serf 1.3.6 are going to be available soon.

Answer 2

I posted the same dump file to the users#subversion.apache.org mailing list but have had no reply in 2 weeks. To get around this issue, I switched QuickBuild to use jsvn.bat from the pure Java distribution of SVNKit and the issue seems to be resolved. Jsvn has the same command-line interface as the Apache svn binary, so it's a simple drop-in replacement.
I initially had an authentication issue because we use NTLM to authenticate with our active directory. The error was "svn: E170001: Authentication required for repsoitory" The solution was to add the following to svnkit\bin\jsvn.bat after the existing EXTRA_JVM_ARGUMENTS environment variable:
rem Adding this to resolve authentication issue as per http://subversion.1072662.n5.nabble.com/SVN-authentication-problem-td1560.html
set EXTRA_JVM_ARGUMENTS=%EXTRA_JVM_ARGUMENTS% "-Dsvnkit.http.methods=Basic,Digest,Negotiate,NTLM"

Answer 3

The only thing worked for me is to install the previous version of Tortoise svn(TortoiseSVN-1.8.6.25419-x64-svn-1.8.8) which runs with the pervious version of svn client 1.8.8 and then use the old version of the svn.exe. That worked against the newer version of the server (1.8.9) too.
(I had the same issue. I upgraded yesterday my collabnet subversion to the latest (svn version 1.8.9-3871.129) and both the command prompt svn checkout or the latest tortoise svn (1.8.7) fails. I have the same ACCESS_VIOLATION error in the dump log. And my computer is Windows 7 64 bit.)