I'm working on a Flutter app using Android Studio as my IDE. I'm attempting to write tests and check the code coverage but I can't work out how to view the data in the IDE or any other application.
By running flutter test --coverage, a coverage report seems to be generated into a file /coverage/lcov.info. That file looks something like this:
SF:lib\data\Customer.g.dart
DA:9,2
DA:10,2
DA:11,2
DA:12,2
DA:13,2
DA:20,0
DA:21,0
DA:22,0
DA:23,0
DA:24,0
...
Looking at the file it seems to have a list of my project files with line by line coverage data. Is there a way to view this information in Android Studio?
You can also install lcov and convert the lcov.info file to HTML pages and then see the result in the browser with sorting option.
1. Installation
1.1. Installing in Ubuntu
sudo apt-get update -qq -y
sudo apt-get install lcov -y
1.2. Installing in Mac
brew install lcov
2. Run tests, generate coverage files and convert to HTML
flutter test --coverage
genhtml coverage/lcov.info -o coverage/html
3. Open coverage report in browser
open coverage/html/index.html
Note This way you can add it to circleci artifacts and coveralls as well.
This is what you want to run to see tests coverage in your browser on macOS
flutter test --coverage
genhtml coverage/lcov.info -o coverage/html
open coverage/html/index.html
You can view the code coverage generated by flutter with the Atom editor.
You just need to install the Dart and lcov-info packages.
Then you load your project folder and press Ctrl+Alt+c, coverage will be displayed with a summary of the whole projects coverage and also with specific line highlighting.
There doesn't appear to be any plugin for Android studio which does this as of yet.
Update 9/18/2021:
See new answer for how it's done within the editor
Update 5/9/2020:
Turns out you can just run flutter test --coverage, then in the same terminal session run bash <(curl -s https://codecov.io/bash) -t token token should be the repository token you get from CodeCov. That command should automatically find and upload the coverage data and will be visible on your CodeCov dashboard. So you don't need Bitrise.
Original:
I've been using Bitrise for continuous integration on my flutter project and there is an easy way to send your reports to CodeCov then visualize it there. This requires you to gain some knowledge on how to set up and use Bitrise but a lot of its automatic so don't worry, also if you're a small team you should be fine with the free tier. Here are the key points for getting CodeCov to work.
Make sure you add the --coverage variable to the Flutter Test workflow.
Add the token from CodeCov as a secret key, you will need to sign up for CodeCov and link your repository to receive a token.
Add the CodeCov workflow and select the CODECOV_TOKEN key.
After that, you should be able to fire off a build and if successful you should see your dashboard update at CodeCov.
The Flutter Enhancement Suite does exactly that. It is an Android Studio/IntelliJ plugin which generates coverage reports.
It shows the coverage per file and also highlights covered lines (red/green bars next to the line numbers):
install the plugin from the plugin options (Preferences > Plugins > Marketplace tab > Search for Flutter Enhancement Suite).
Create a new Run Configuration for testing with coverage
(Run > Edit Configurations > click the plus button to add a new configuration > Choose Flutter Test in the dropdown)
Name your configuration (e.g. "All tests"), set the scope and the file or directory containing your tests.
Run your tests with coverage from the top menu.
I just developed a simple dart package (test_cov_console), so you can run it directly from Android Studio terminal. The tool would read the lcov.info that was generated by flutter test --coverage. Find this link for source code.
You can install the lib globally, so it would not change your current project:
flutter pub global activate test_cov_console
And run it:
flutter pub global run test_cov_console
Here is the sample of output:
flutter pub run test_cov_console
---------------------------------------------|---------|---------|---------|-------------------|
File |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/ | | | | |
print_cov.dart | 100.00 | 100.00 | 88.37 |...,149,205,206,207|
print_cov_constants.dart | 0.00 | 0.00 | 0.00 | no unit testing|
lib/ | | | | |
test_cov_console.dart | 0.00 | 0.00 | 0.00 | no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
All files with unit testing | 100.00 | 100.00 | 88.37 | |
---------------------------------------------|---------|---------|---------|-------------------|
The output can be saved to CSV file:
flutter pub run test_cov_console -c --output=coverage/test_coverage.csv
Sample CSV output file:
File,% Branch,% Funcs,% Lines,Uncovered Line #s
lib/,,,,
test_cov_console.dart,0.00,0.00,0.00,no unit testing
lib/src/,,,,
parser.dart,100.00,100.00,97.22,"97"
parser_constants.dart,100.00,100.00,100.00,""
print_cov.dart,100.00,100.00,82.91,"29,49,51,52,171,174,177,180,183,184,185,186,187,188,279,324,325,387,388,389,390,391,392,393,394,395,398"
print_cov_constants.dart,0.00,0.00,0.00,no unit testing
All files with unit testing,100.00,100.00,86.07,""
With the release of Flutter 2.5, you can now view test coverage within IntelliJ and Android Studio.
See this post
In addition, the latest IJ/AS plugin for Flutter allows you to see the
coverage information for both unit test and integration test runs. You
can access this from the toolbar button next to the “Debug” button:
Android Studio and IntelliJ:
Coverage reporting is now available on Android Studio
You can use SonarQube with an additional plugin for Flutter as per this link SonarQube plugin for Flutter / Dart.
I have tried it with the free version of SonarQube on docker, and if you have configured it correctly, you just need to run the following commands on Android Studio terminal:
# Download dependencies
flutter pub get
# Run tests with User feedback (in case some test are failing)
flutter test
# Run tests without user feedback regeneration tests.output and coverage/lcov.info
flutter test --machine --coverage > tests.output
# Run the analysis and publish to the SonarQube server
sonar-scanner
Here is the sample of the report, and you can drill deep into line codes.
So, the actual answer is no, you cannot view a coverage report within Android Studio (or in IntelliJ IDEA) at this time.
Unlike JavaScript/TypeScript and Java and probably Python, the IntelliJ IDE (and by extension, Android Studio) do not have integrated IDE support for showing test coverage of Flutter code in the editor. This is a shame because the ability to see your untested code branches and lines highlighted in the source code of your editor is a beautiful thing. Not sure why a plugin for this does not exist yet, since it is well-supported for other languages, and a standard lcov.info file is generated.
There is a bundled code coverage tool window in IntelliJ that is supposed to allow you to browse the lcov.info file in a tree/table drill-down format, but it doesn't seem to work with the coverage report generated by flutter (flutter test --coverage). I thought it might be the relative paths in the lcov.info and my multi-module app structure, but I tried to manually edit the file paths in lcov.info, but I had no luck getting the stats to show.
FOR WINDOWS
Required:
Chocolatey
Perl
LCOV
1. INSTALL CHOCOLATEY
Open PowerShell (with Admin)
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
2. INSTALL PERL
choco install strawberryperl
Add path to the environmental variable
3. INSTALL LCOV
choco install lcov
LCOV OPERATION
go to the android studio terminal & run this flutter test --coverage
now next, open your project root dir in the power shell in my case (eg :C:\Users\CIPL\Documents\Project\Flutter\myProject)
& run this cmd perl C:\ProgramData\chocolatey\lib\lcov\tools\bin\genhtml coverage/lcov.info -o coverage/html
that's it open the html folder and click the HTML file to view the visual in the browser.
NOTE: when I tried to do 3'rd point, I faced this error "ERROR: cannot create directory 'coverage/html'"
so manually created the html folder and tried 3rd point again.
Found this Windows solution in this https://blog.tech-andgar.me/flutter-test-coverage
Is it possible to download the Android APK file from Fabric Beta? We have multiple releases uploaded.
Mike from Fabric here. We currently don't provide a way to download the .APK, they are only provided via the Beta by Crashlytics apps.
Late answer but someone may need this. You can download it in a hacky way from devices that apps install by Beta or any way:
Connect the device to your computer and run the following command, ensure that you have configured the adb correctly:
adb shell pm list packages | grep xyz # get the package name of the app
adb shell pm path app.xyz.stg # get the path of the app
adb pull /data/app/app.xyz.stg/base.apk . # pull the app to PWD
the name of the app is base.apk, change it to xyz. This can be used for the same device.
Mesut's answer is correct. Just to make it more clear.
adb shell pm path ${package_name}
adb pull /data/app/${package_name_2}/base.apk
In the second command, the value ${package_name_2}/base.apk is from the first command. Sometimes it's not exactly the package name.
In my case, it's ${package_name}-1/base.apk
If you just want to download a specific build, say "1.0(143)" then you can choose that build in the beta app and download it.
If your need is to upload multiple apks from same build (say an apk for each deployment environment such as prevalidation, validation, production) then you need to setup your gradle to define productFlavors for each deployment environment like this:
android {
...
flavorDimensions "deploymentEnvironment"
productFlavors {
prevalidation {
dimension "deploymentEnvironment"
}
validation {
dimension "deploymentEnvironment"
}
production {
dimension "deploymentEnvironment"
}
}
...
}
Then you publish multiple APKs from the same build (one for each target deployment environment) to the same Fabric project using following gradle tasks as illustrative examples. Actual tasks depend on the variants defined for your project:
./gradlew -s assemblePrevalidationRelease assembleValidationRelease
./gradlew -s crashlyticsUploadDistributionPrevalidationRelease crashlyticsUploadDistributionValidationRelease
The Fabric console beta page does show both apks and you can choose to download and install one or the other. The only missing part is that both variants are listed as exactly the same (since they have the same versionName and versionCode). This could easily be solved if Fabric console shows the actual apk name in addition to the version / build info. I would love for the awesome Fabric team to address this small feature request sometime soon.
Until then a workaround I use is to identify the build based on order in Fabric beta console (risky but works) and put the target deployment info in the release notes for each apk in Fabric for a given build.
Since upgrading to Xcode 8, when I do a build with fastlane, I get the following error message:
There does not seem to be a CURRENT_PROJECT_VERSION key set for this project
If I go to Xcode > Build Settings and go down to Versioning, there is a Current Project Version key, as shown below:
The help text says to enter an integer or floating point number, but when I click on the field, there is no opportunity to enter a number in either the Debug or Release field. This is different from the screen shot shown in this apple tech Q&A so there appears to have been a change in Xcode since the Q&A was released.
Don't. Modify the values in your app's info.plist file instead.
This means not using agvtool (as I learned).
Why? Over the years, Apple has come up with several manners of changing version and build numbers. Many of them are now outdated and poor practice. Changing CURRENT_PROJECT_VERSION modifies values within your project's project.pbxproj file and if you are running a distributed team, this will cause merge conflicts if the other half of the team tries to update and while they were asleep, you updated this internal value. If you are using pods, you'll get several more merge conflicts per pod that you add to the project.
So, CURRENT_PROJECT_VERSION?
Don't use it.
Within the info.plist file are these keys.
CFBundleVersion
CFBundleShortVersionString
Use CFBundleVersion for your app's build number.
Use CFBundleShortVersionString for your app's version number.
Use Plistbuddy to do it.
<key>CFBundleShortVersionString</key>
<string>3.0.7</string>
<key>CFBundleVersion</key>
<string>934</string>
</dict>
</plist>
Try the script below.
#!/bin/sh
# To make executable, use: chmod u+x Build-Versioning-Scripts/Increment_Build_Number.sh
# to locate your target's info.plist use
# ${PRODUCT_SETTINGS_PATH}
echo "----"
echo "Info.plist for target: ${PRODUCT_SETTINGS_PATH}"
buildNum=$(/usr/libexec/Plistbuddy -c "Print CFBundleVersion" "${PRODUCT_SETTINGS_PATH}")
echo "Current build #: $buildNum"
if [ -z "$buildNum" ]; then
echo "No build number found in $PRODUCT_SETTINGS_PATH"
exit 2
fi
buildNum=$(expr $buildNum + 1)
echo "Build # incremented to: $buildNum"
/usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNum" "$PRODUCT_SETTINGS_PATH"
echo "----"
exit 0
By adding this script to your build or archive process on your build machine, this will automatically update the app's build number.
If you wish to increment your app's version number, change CFBundleShortVersionString (Bundle versions string, short) in the info.plist manually.
Currently when you get that fastlane error, terminal logs redirects you to
Automating Version and Build Numbers Using agvtool
to understand what you need to do.
Summary
Enable agvtool.
Build Settings > Current Project Version > $(CURRENT_PROJECT_VERSION)
Build Settings > Versioning System > Apple Generic
Set up your version and build numbers.
Target > Info > Bundle versions string, short (CFBundleShortVersionString) > "your init
version"
Target > Info > Bundle version (CFBundleVersion) > "your
init value"
That helps me a lot.
you may try change project format ?
I suggest you can change this xcode format version.
I have an application which uses FFMPEG and FFPROBE to perform some tasks on a video the user can open with my application.
For non-sandboxed applications everything works fine, but when my app is running in a sandbox FFPROBE doesn't seem to get started.
The Console says the following:
08.06.15 12:27:55,803 secinitd[281]: ffprobe[4049]: registration request failed: (0x11, 0x0) Container object initialization failed.
failed to get bundleid for app "/Users/Alex/.../ffprobe"
The path to ffprobe mentioned in this messages points to the MacOS directory within the app bundle (a Build Phase copies these two binaries into the executable directory).
I've searched a lot and found some hints regarding entitlements. Of course my sandboxed app has its entitlements and when building my application it gets signed (with --deep signing flag). It even passes the technical App Store check for entitlements.
Now I'm stuck and wonder why my application is not able to launch FFPROBE (and FFMPEG).
Does anybody have a clue?
It seems like I've found a solution. I don't know whether all of these steps are necessary, but here is what I've tried and what seems (!) to work:
I added an entitlements file which contains true for the keys com.apple.security.inheritand com.apple.security.app-sandbox
I added a plist file for each used binary and filled the keys CFBundleName and CFBundleIdentifier with suitable values (I don't know whether this step is necessary)
I added a Run Script build phase which executes codesign -f -s "your certificate" --entitlements ./ffmpeg.entitlements ./Build/Intermediates/ArchiveIntermediates/App\ Store/InstallationBuildProductsLocation/Applications/<my app>/Contents/MacOS/ffmpeg
for each binary used.
These steps result in suitable entitlements when trying to submit the app to the Mac App Store and it results in a correct usage of the embedded binaries (at least on my development Mac and the Mac of some colleagues).
I'm trying to get my application, which does not appear in the Dock, to have an option to launch at login. This is tricky, and involves creating a second, helper application which you add as a startup item. This helper app is only responsible for launching the main app and then exiting.
I've followed the instructions here and here and it works like a charm - the problem is, of course, code signing. I have two targets; the helper app target is copied to the Contents/Library/LoginItems subdirectory of the main bundle at compile time. Each bundle has its own bundle identifier and own deployment provisioning profile, but when I validate my archive for the app store, I get the following error:
Invalid provisioning profile. The provisioning profile included in the bundle BUNDLE NAME [BUNDLE NAME.app] is invalid. For more information, visit the Mac OS Developer Portal.
If I remove the helper bundle from my main target, there's no problem. It looks like the presence of another provisioning profile is setting off the error.
How can I include two signed bundles and pass the validation?
I was finally able to resolve this problem by using codesign on a coworker's computer (there must have been something wrong with my Keychain) and deleting the embedded.provisionprofile file from the helper app by adding the following run script:
if [ -f "${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app/Contents/embedded.provisionprofile" ];
then
rm "${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app/Contents/embedded.provisionprofile"
echo "Removed embedded provisioning profile."
else
echo "No profile found"
fi
You should use the same Mac App Store Production Certificate to sign both the helper app and the main application. I haven't tried this in Xcode — we have a helper app that's a bundle resource, but our code signing is a command line script. We didn't have any problems with the app store system.
I'm not sure why you're ending up with a provisioning profile in the built product, and I don't think this is required for app store submission. You can try using codesign manually:
codesign -f -s "3rd Party Mac Developer Application: My Company" \
-i "com.mycompany.loginitem" \
--entitlements path/to/loginitem.entitlements" \
path/to/appname.app/Contents/Library/LoginItems/loginitem.app
codesign -f -s "3rd Party Mac Developer Application: My Company" \
-i "com.mycompany.appname" \
--entitlements path/to/app.entitlements" \
path/to/appname.app
I had the same problem. Instead of removing embedded.provisionprofile from the helper app I've just disabled provisioning (Provision profile: None) leaving code signing identify and entitlements in place. Submitted my app for review without any issues.