1. The Summary of the Problem
I would like a Travis CI setup that would let me run flutter driver tests inside an Android and an iOS environments. For this to happen, I expect I somehow have to install Flutter, Android and iOS in different environments.
2. What I Have So Far
Most of the posts I've been able to find on this topic are terribly outdated or feature incredibly complicated setups. Some of those that keep on appearing in my searches are:
Test Flutter apps on Travis, by Yegor Jbanov. This one covers unit and widget testing (flutter test), but not integration tests.
It's from early 2017 and Travis CI has maybe simplified its API, because I've managed to make it work with only this:
language: dart
dart:
- stable
dart_task:
- dartfmt
install:
- git clone https://github.com/flutter/flutter.git -b stable
script:
- ./flutter/bin/flutter doctor
- ./flutter/bin/flutter test
One resource that I've found very useful is the .travis.yml in the Flutter samples repo. The setup there seems very complicated to me though.
The closest I could get to what I wanted is similar to Maurice McCabe's Flutter unit, widget and integration testing with IOS and Android emulators on Travis-CI.
Again, this seems overcomplicated and outdated.
3. The Sketch of What I Have in Mind
The script and install steps in the example I mentioned previously could be replaced by jobs with stages. In this way, each stage would represent one sort of step. Unit and Widget stages in one, integration tests on Android and iOS in two others, which is similar to what Maurice McCabe and Flutter samples show. For example:
jobs:
include:
- stage: Flutter Test
language: dart
os: linux
install: git clone $FLUTTER_GITHUB -b stable
before_script:
- ./flutter/bin/flutter doctor
script:
- ./flutter/bin/flutter test
- stage: Integration Test on Android
os: linux
dist: trusty
language: android
android: # the things here are what probably needs to be fixed
components:
- build-tools-28.0.3
- android-28
install: git clone $FLUTTER_GITHUB -b stable
before_script:
- ./flutter/bin/flutter doctor
script:
- ./flutter/bin/flutter drive --target=test_driver/app.dart
If I could create a stage for the dartfmt task that would also be nice in terms of organization.
1. The Overview
Kudos to #MirceaMatei and Maurice McCabe for helping out in this.
I haven't yet been able to make Android Integration Tests work (the last stage of the code below), but at least iOS is working.
Android is much harder to get right because of different versions and licenses, something Apple does much better.
The code you will find below is part of a custom mono repo setup I'm working on.
The folder structure consists of an app and a packages folders.
I haven't yet segmented code coverage by those two folders either, so this setup is currently overwriting code coverage from one folder to another.
Using a top level install section makes Travis repeat that setup for every stage, so that avoids having to do that repeatedly for every stage.
I'm sharing my current — albeit incomplete — setup below, but continuous improvement will happen through this Github Gist, which is a much better way of interacting with the community to improve code than StackOverflow — it's not its purpose.
All in all, I think Travis CI is really dropping the ball when it comes to Flutter — and maybe Android in general. An easier alternative that has been hyped by many practitioners I've come into contact recently is Codemagic, which also offers code signing and automated deployment to the iOS and Android app stores.
2. Useful Resources
I didn't really like Travis documentation for the integration tests. People creating pure Android apps filed similar issues.
Anyway, here are some useful resources I've found during my searches:
Issue Comment on "Error: Target id is not valid. Use 'android list targets' to get the target ids."
“Invalid --abi armeabi-v7a for the selected target” with Google APIs
Is there a way to start android emulator in Travis CI build?
Travis-CI Android 28 licenses have not been accepted
Flutter Samples Travis Setup
#MirceaMatei's Travis Setup Recommendation
Travis Building Android Projects Documentation
Maurice McCabe's Unit and Integration Tests with Travis Article on Medium
3. The Code
Please, do not post code improvement suggestions here, but in the the Github Gist. I'll be updating the yaml code below myself every time a useful iteration arrives.
language: dart
env:
global:
- DARTSDK=./flutter/bin/cache/dart-sdk/bin
- DARTFMT=$DARTSDK/dartfmt
- FLUTTER=./flutter/bin/flutter
- FLUTTER_UP=../flutter/bin/flutter
- FLUTTER_GITHUB=https://github.com/flutter/flutter.git
- CODECOV=https://codecov.io/bash
- PACKAGES=packages
- APP=app
- ANDROID_API=28
install:
- git clone $FLUTTER_GITHUB -b stable --depth 1
- $FLUTTER doctor
- $FLUTTER pub get
jobs:
include:
- stage: Formatting
script:
- $DARTFMT -n $PACKAGES --set-exit-if-changed
- stage: Packages Flutter Test
script:
- $FLUTTER test --coverage $PACKAGES
after_success:
- bash <(curl -s $CODECOV)
- stage: App Flutter Test
script:
- cd $APP
- $FLUTTER_UP test --coverage
after_success:
- bash <(curl -s $CODECOV)
- stage: iOS Integration Tests
os: osx
osx_image: xcode11
before_script:
- open /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app
- export HOMEBREW_NO_AUTO_UPDATE=1
- brew install libimobiledevice
- brew install ideviceinstaller
- brew install ios-deploy
- brew install cocoapods || echo 'ignore exit(1)'
- brew link --overwrite cocoapods
script:
- cd $APP
- $FLUTTER_UP driver --target=test_driver/app.dart77
- stage: Android Integration Tests
language: android
dist: trusty
android:
components:
- tools
- platform-tools
- build-tools-25.0.3
- android-24
- android-22
- extra-google-google_play_services
- extra-google-m2repository
- extra-android-m2repository
- sys-img-armeabi-v7a-android-22
licenses:
- 'android-sdk-preview-license-52d11cd2'
- 'android-sdk-license-.+'
- 'google-gdk-license-.+'
before_install:
- yes | sdkmanager "platforms;android-28"
- echo no | android create avd --force -n test -t android-22 --abi armeabi-v7a -c 32M
- emulator -avd test -no-audio -no-window &
- android-wait-for-emulator
- adb devices
- adb shell input keyevent 82 &
script:
- $FLUTTER --version
- $FLUTTER doctor
- cd $APP
- $FLUTTER_UP devices
- $FLUTTER_UP driver --target=test_driver/app.dart
You can take a look at this project
Sqlite Scaffolding Generator # Dart Framework ORM M8
I think it has all the components you are looking for. It's a Dart package that has, in the solution, an example integration project for Flutter.
The CI pipeline is built for Travis CI (based mostly on Maurice McCabe's article). It's possible that the travis.yml file has all the sections you need.
Related
One magical day I found a reference to an 'epilogue' key to be used in the Serverless.yml file. It's the best. We use it to cleanup after testing that occurs inside our CI/CD pipeline.
- name: Test Integration
dependencies:
- Deploy Dev
task:
jobs:
- name: Test endpoints
commands:
- cache restore
- checkout
- sem-version python 3.8
- cd integration_tests
- pip install -r requirements.txt
- // our various testing scripts...
epilogue:
always: // This runs, no matter what. There are other options!!
commands:
- python3 99_cleanup.py
secrets:
- name: secret_things_go_here
Today, I don't want epilogue: always: , but rather epilogue: when it doesn't fail: . I cannot find one shred of documentation about this option. Nothing to even explain how I got here in the first place.
Oh, internet: How do I run something only when my tests have passed?
WOO!
I was barking up the wrong tree. The solution is within SemaphoreCI, not Serverless.
https://docs.semaphoreci.com/reference/pipeline-yaml-reference/#the-epilogue-property
Options include: on_pass and on_fail.
Whew.
I want to run Kotlin scripts in CI without relying on a Gradle project, so I can easily do operations that would be hard to program using shell/bash/batch, and so that I can use libraries if needed.
Having the Kotlin script run only on Ubuntu/Linux is fine, though ideally, there's a way to make it run on Windows and macOS targets as well for platform specific projects.
UPDATE: Kotlin is now pre-installed on GitHub Actions runners, no need to install it beforehand anymore.
First, ensure that you have a proper Kotlin script, ending in .kts, or better, .main.kts as that latter one will be recognized better by the IDE (e.g. IntelliJ IDEA, Android Studio), especially when it comes to autocompletion and type analysis.
Second, ensure that its first line is the shebang pointing to the right place:
#!/usr/bin/env kotlin
That will be helpful to test the script locally before running in CI, as the IDE will show a run button in the gutter, next to the shebang.
If you add the execute permission to the file (chmod +x YouScript.main.kts on Linux/macOS), you'll also be able to run it just like any other script, without having to type kotlinc -script before, and that will apply on GitHub Actions as well.
Finally, here's an example manual GitHub Action (aka. workflow file) that will take an input and pass it to your Kotlin script (usable in the args property/parameter) after it installed Kotlin:
name: Run Kotlin script
on:
workflow_dispatch:
inputs:
awesome-input:
description: 'Awesome parameter'
default: 'You'
required: true
jobs:
awesome-action:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Run Kotlin script
run: kotlinc -script ./YourScript.main.kts ${{ github.event.inputs.awesome-input }}
Note that if the script has the execute (x) permission, as I told previously, you can remove the kotlinc -script part and it will still run.
Bonus: it is possible to have Kotlin code directly in the workflow file (though I'd not recommend doing it), by using kotlin as a shell.
See this YouTrack comment to see how: https://youtrack.jetbrains.com/issue/KT-43534#focus=Comments-27-4640716.0-0
Kotlin runner is now pre-installed on GitHub Actions environments (GitHub issue, YouTube video).
Refer to the GitHub Actions runner images1 to see all the installed software.
So, you can easily run your .main.kts scripts like this:
name: Example
on:
push:
branches:
- main
jobs:
example-action:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Run the script
run: kotlin /path/in/repo/to/my-script.main.kts
And here is an example my-script.main.kts file:
#file:JvmName("MyScript")
#file:CompilerOptions("-jvm-target", "11")
#file:Repository("https://repo.maven.apache.org/maven2")
// #file:DependsOn("com.example:library:1.2.3")
import java.io.File
val input = File("README.md") // Assuming you ran checkout before
val output = File("result.txt")
val readmeFirstLine = input.readLines().first()
output.writeText(readmeFirstLine)
There is also a GitHub action called setup-kotlin that lets you install your desired version of Kotlin and also provides some more features. Check out this issue.
...
- uses: actions/checkout#v3
- uses: fwilhe2/setup-kotlin#main
with:
version: 1.7.0
- name: Run the script
run: kotlin /path/in/repo/to/my-script.main.kts
It was previously called virtual environments
I'm using GitLab's shared CI runners for training. Since this morning I can't build
my project because the pipeline's status is "pending".
Is it because the number of shared runners is maxed out ? Too busy running other people's
code ?
Is there a setting I need to check ? I know I can pay for a dedicated runner but I'm not
in commercial setting at this point therefore I'm sticking with shared ones.
Thank you for assistance.
In such situations I would highly recommend to make the best use of tags. You can use tags to select a specific runner from the list of all runners that are available for the project.
In this example, the job is run by a runner that
has both ruby and postgres tags defined.
job:
tags:
- ruby
- postgres
If you have your own runners setup then you can use tags to run different jobs on different platforms. For
example, if you have an OS X runner with tag osx and a Windows runner with tag
windows, you can run a job on each platform:
windows job:
stage:
- build
tags:
- windows
script:
- echo Hello, %USERNAME%!
osx job:
stage:
- build
tags:
- osx
script:
- echo "Hello, $USER!"
If this is still a problem then consider using your own private runner.
I've got a simple CI pipeline with the .gitlab-ci.yml file as below:
include:
template: SAST.gitlab-ci.yml
variables:
SAST_GOSEC_LEVEL: 2
test:
script:
- apt-get update -qy
- apt-get install -y python-dev python-pip
- pip install -r requirements.txt
All actions in the script section works well, but the templated included was not picked up the build.
Thanks,
Paul
This feature is available to only gold/ultimate accounts. Mine is a free one.
GitLab 15.6 (November 2022) seems to refers to this feature, but for all GitLab tiers.
More accurate SAST rules for Python
The GitLab Vulnerability Research team has updated the rules used for Python SAST scanning to catch more relevant problems with fewer false-positive results.
The updated rules are included in the latest release of the Semgrep-based SAST analyzer.
If you use the default GitLab-managed CI/CD template for SAST on GitLab 15.0 or higher, your pipelines automatically update the Semgrep-based SAST analyzer to use the new rules.
On your first scan after the update, existing findings we’ve identified as false positives will be marked “No longer detected”, and new findings may be created.
See Documentation and Issue.
Is it possible to download the Android APK file from Fabric Beta? We have multiple releases uploaded.
Mike from Fabric here. We currently don't provide a way to download the .APK, they are only provided via the Beta by Crashlytics apps.
Late answer but someone may need this. You can download it in a hacky way from devices that apps install by Beta or any way:
Connect the device to your computer and run the following command, ensure that you have configured the adb correctly:
adb shell pm list packages | grep xyz # get the package name of the app
adb shell pm path app.xyz.stg # get the path of the app
adb pull /data/app/app.xyz.stg/base.apk . # pull the app to PWD
the name of the app is base.apk, change it to xyz. This can be used for the same device.
Mesut's answer is correct. Just to make it more clear.
adb shell pm path ${package_name}
adb pull /data/app/${package_name_2}/base.apk
In the second command, the value ${package_name_2}/base.apk is from the first command. Sometimes it's not exactly the package name.
In my case, it's ${package_name}-1/base.apk
If you just want to download a specific build, say "1.0(143)" then you can choose that build in the beta app and download it.
If your need is to upload multiple apks from same build (say an apk for each deployment environment such as prevalidation, validation, production) then you need to setup your gradle to define productFlavors for each deployment environment like this:
android {
...
flavorDimensions "deploymentEnvironment"
productFlavors {
prevalidation {
dimension "deploymentEnvironment"
}
validation {
dimension "deploymentEnvironment"
}
production {
dimension "deploymentEnvironment"
}
}
...
}
Then you publish multiple APKs from the same build (one for each target deployment environment) to the same Fabric project using following gradle tasks as illustrative examples. Actual tasks depend on the variants defined for your project:
./gradlew -s assemblePrevalidationRelease assembleValidationRelease
./gradlew -s crashlyticsUploadDistributionPrevalidationRelease crashlyticsUploadDistributionValidationRelease
The Fabric console beta page does show both apks and you can choose to download and install one or the other. The only missing part is that both variants are listed as exactly the same (since they have the same versionName and versionCode). This could easily be solved if Fabric console shows the actual apk name in addition to the version / build info. I would love for the awesome Fabric team to address this small feature request sometime soon.
Until then a workaround I use is to identify the build based on order in Fabric beta console (risky but works) and put the target deployment info in the release notes for each apk in Fabric for a given build.