Azure Devop TestThe active test run was aborted. Reason: Test host process crashed : Unhandled Exception - testing

Question

We have one PR that fails due to our tests sets. They only fail with this PR, with the following error:
The active test run was aborted. Reason: Test host process crashed : Unhandled Exception: System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain.
It always crashes with this error while other branches work fine. It's always the same exception but not everytime at the same position. We tried to enable diagnostic, but it gives absolutely no additional informations:
##[debug]ProxyRunRequest: HandleLogMessage Entered
##[error]RunMessage : The active test run was aborted. Reason: Test host process crashed : Unhandled Exception: System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain.
##[debug]Processed: ##vso[task.logissue type=error;]RunMessage : The active test run was aborted. Reason: Test host process crashed : Unhandled Exception: System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain.
We use Nunit, and the testing part of our YML is the following:
- task: VSTest#2
timeoutInMinutes: 180
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: '*.Test*.dll'
searchFolder: '$(System.DefaultWorkingDirectory)/$(buildConfiguration)'
codeCoverageEnabled: false
platform: 'Any CPU'
uiTests: false
configuration: '$(buildConfiguration)'
runInParallel: true
resultsFolder: 'testResults'
runSettingsFile: '$(Build.SourcesDirectory)\azure-tests.runsettings'
rerunFailedTests: true
The PR is like 200 files and we can't find what could cause the issue, especially since we spent a lot of time testing as its a critical feature for us.
Any idea what we can do to have a better idea of what the problem is? Or what could causes this exception?