iOS tests fails in Release but works in Debug - detox

Question

I just did a small set of tests for my react-native app and they work well when testing in iOS - Debug mode. But when I try to run it in iOS - Release it just fails, saying can't find the components.
I know more information is needed but I have no idea how to actually move forward.
Notes:
- All testID are in <View />
- I tried to run with simulator open and the app actually opens
- Did trace and it has no much useful data (but the hierarchy shows no accessibilityID or testID)
Follows the hierarchy:
Cannot find UI Element.
Exception with Assertion: {
"Assertion Criteria": "assertWithMatcher:matcherForSufficientlyVisible(>=0.750000)",
"Element Matcher": "((!(kindOfClass('RCTScrollView')) && (respondsToSelector(accessibilityIdentifier) && accessibilityID('WelcomeScreen_Contents'))) || (((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches(kindOfClass('RCTScrollView'))) && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && parentThatMatches((respondsToSelector(accessibilityIdentifier) && accessibilityID('WelcomeScreen_Contents'))))))",
"Recovery Suggestion": "Check if the element exists in the UI hierarchy printed below. If it exists, adjust the matcher so that it accurately matches element."
}
Error Trace: [
{
"Description": "Interaction cannot continue because the desired element was not found.",
"Error Domain": "com.google.earlgrey.ElementInteractionErrorDomain",
"Error Code": "0",
"File Name": "GREYElementInteraction.m",
"Function Name": "-[GREYElementInteraction matchedElementsWithTimeout:error:]",
"Line": "124"
}
]
Hierarchy: <UIWindow:0x7fb2a5622f30; AX=N; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
|--<RCTRootView:0x7fb2a5620e70; AX=N; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| |--<RCTRootContentView:0x7fb2a57c5200; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | |--<RCTView:0x7fb2a558c7a0; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | |--<RCTView:0x7fb2a55baaf0; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | |--<RCTView:0x7fb2a55a6f50; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | |--<RCTView:0x7fb2a5517b40; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | |--<RCTView:0x7fb2a5791380; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | |--<RCTView:0x7fb2a5791070; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | |--<RCTView:0x7fb2a5790d60; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | | |--<RCTView:0x7fb2a5790a50; AX=N; AX.frame={{0, 0}, {414, 88}}; AX.activationPoint={207, 44}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 88}}; opaque; alpha=1>
| | | | | | | | | | |--<RCTView:0x7fb2a5790740; AX=N; AX.frame={{-414, 0}, {414, 88}}; AX.activationPoint={-207, 44}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{-414, 0}, {414, 88}}; opaque; alpha=1>
| | | | | | | | | | | |--<RCTView:0x7fb2a579d570; AX=N; AX.frame={{-414, 0}, {414, 88}}; AX.activationPoint={-207, 44}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 88}}; opaque; alpha=1>
| | | | | | | | | | | | |--<RCTView:0x7fb2a579d260; AX=N; AX.frame={{-414, 44}, {414, 43.5}}; AX.activationPoint={-207, 65.75}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 44}, {414, 43.5}}; opaque; alpha=1>
| | | | | | | | | |--<RCTView:0x7fb2a578a1b0; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | | | |--<RCTView:0x7fb2a5593d10; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | | | | |--<RCTView:0x7fb2a55a5c90; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | | | | | |--<RCTView:0x7fb2a55d0c60; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 0}, {414, 896}}; AX.activationPoint={207, 448}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 896}}; opaque; alpha=1>
| | | | | | | | | | | | | |--<RCTView:0x7fb2a55d92f0; AX=N; AX.label='Welcome! Let's go! Sign in'; AX.frame={{0, 44}, {414, 818}}; AX.activationPoint={207, 453}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 44}, {414, 818}}; opaque; alpha=1>
| | | | | | | | | | | | | | |--<RCTView:0x7fb2a55d0950; AX=N; AX.label='Sign in'; AX.frame={{0, 816}, {414, 36}}; AX.activationPoint={207, 834}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 772}, {414, 36}}; opaque; alpha=1>
| | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cd670; AX=Y; AX.id='Touchable'; AX.label='Sign in'; AX.frame={{0, 816}, {414, 36}}; AX.activationPoint={207, 834}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 36}}; opaque; alpha=1>
| | | | | | | | | | | | | | | | |--<RCTTextView:0x7fb2a55b3040; AX=Y; AX.label='Sign in'; AX.frame={{0, 816}, {414, 36.5}}; AX.activationPoint={207, 834.25}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{0, 0}, {414, 36.5}}; alpha=1>
| | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cd360; AX=N; AX.label='Welcome! Let's go!'; AX.frame={{0, 44}, {414, 772}}; AX.activationPoint={207, 430}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {414, 772}}; opaque; alpha=1>
| | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cd050; AX=N; AX.frame={{207, 500}, {0, 316}}; AX.activationPoint={207, 658}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{207, 456}, {0, 316}}; opaque; alpha=1>
| | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55ccd40; AX=N; AX.label='Let's go!'; AX.frame={{147, 460}, {120, 40}}; AX.activationPoint={207, 480}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{147, 416}, {120, 40}}; opaque; alpha=1>
| | | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cca30; AX=Y; AX.id='Touchable'; AX.label='Let's go!'; AX.frame={{147, 460}, {120, 40}}; AX.activationPoint={207, 480}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 0}, {120, 40}}; opaque; alpha=1>
| | | | | | | | | | | | | | | | | |--<RCTTextView:0x7fb2a55cc800; AX=Y; AX.label='Let's go!'; AX.frame={{164, 471}, {86, 18.5}}; AX.activationPoint={207, 480.25}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{17, 11}, {86, 18.5}}; alpha=1>
| | | | | | | | | | | | | | | |--<RCTTextView:0x7fb2a55cc210; AX=Y; AX.label='Answer two quick questions to get started.'; AX.frame={{68, 379.5}, {278, 50.5}}; AX.activationPoint={207, 404.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{68, 335.5}, {278, 50.5}}; alpha=1>
| | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cbd20; AX=N; AX.label='Welcome to '; AX.frame={{97.5, 44}, {219, 336}}; AX.activationPoint={207, 212}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{97.5, 0}, {219, 336}}; opaque; alpha=1>
| | | | | | | | | | | | | | | | |--<RCTView:0x7fb2a55cba10; AX=N; AX.label='Welcome to '; AX.frame={{97.5, 327}, {219, 33}}; AX.activationPoint={207, 343.5}; AX.traits='UIAccessibilityTraitNone'; AX.focused='N'; frame={{0, 283}, {219, 33}}; opaque; alpha=1>
| | | | | | | | | | | | | | | | | |--<RCTImageView:0x7fb2a55cb730; AX=N; AX.id='/Users/.../Library/Developer/CoreSimulator/Devices/4FF22CAE-8C08-4FA8-8CB7-DEC31EB77737/data/Containers/Data/Application/608DC539-4BB6-4C2D-934E-98C4863ED4D1/Library/Application Support/CodePush/76158f1b9b351e2881c905cc31e6f30ea33a2e52829f92e0b092dcc45be9cfde/CodePush/assets/app/assets/images/logo#2x.png'; AX.frame={{258.5, 333}, {58, 21}}; AX.activationPoint={287.5, 343.5}; AX.traits='UIAccessibilityTraitImage'; AX.focused='N'; frame={{161, 6}, {58, 21}}; alpha=1>
| | | | | | | | | | | | | | | | | |--<RCTTextView:0x7fb2a55cb500; AX=Y; AX.label='Welcome to '; AX.frame={{97.5, 327}, {162, 33.5}}; AX.activationPoint={178.5, 343.75}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{0, 0}, {162, 33.5}}; alpha=1>
9 |
10 | it('WelcomeScreen: Loaded', async () => {
> 11 | await expect(element(by.id('WelcomeScreen_Contents'))).toBeVisible();
| ^
12 | });
I also tried some other values:
<View
style={styles.contents}
testID="WelcomeScreen_Contents1"
accessibilityHint="WelcomeScreen_Contents2"
accessibilityLabel="WelcomeScreen_Contents3"
>
And the following tests:
it('WelcomeScreen: Loaded 1', async () => {
await expect(element(by.id('WelcomeScreen_Contents1'))).toBeVisible();
});
it('WelcomeScreen: Loaded 2', async () => {
await expect(element(by.id('WelcomeScreen_Contents2'))).toBeVisible();
});
it('WelcomeScreen: Loaded 3', async () => {
await expect(element(by.id('WelcomeScreen_Contents3'))).toBeVisible();
});
Detox config at package.json
"detox": {
"configurations": {
"iphonexr": {
"binaryPath": "ios/build/xxx/Build/Products/Release-iphonesimulator/xxx.app",
"build": "RCT_NO_LAUNCH_PACKAGER=1 xcodebuild -quiet -workspace ios/xxx.xcworkspace -UseNewBuildSystem=NO -configuration Release -scheme xxx -derivedDataPath ios/build/xxx -sdk iphonesimulator",
"type": "ios.simulator",
"name": "iPhone Xʀ"
},
"iphonese": {
"binaryPath": "ios/build/xxx/Build/Products/Release-iphonesimulator/xxx.app",
"build": "RCT_NO_LAUNCH_PACKAGER=1 xcodebuild -quiet -workspace ios/xxx.xcworkspace -UseNewBuildSystem=NO -configuration Release -scheme xxx -derivedDataPath ios/build/xxx -sdk iphonesimulator",
"type": "ios.simulator",
"name": "iPhone SE"
}
},
"test-runner": "jest"
}
}

Answer 1

After hours of debugging I found out the problems were:
- CodePush overriding my changes
- RN's View Flattening ( https://twitter.com/shergin/status/1058393187079704576 )
I added a small check so I can disable CodePush for Test builds and it works properly now.