Hi, anyone out there testing flutter apps yet?
Looking into speculative ways to do so, but the entire framework is just a lot of black magic right now and the landscape and platform support keeps changing thing mature. Anyone also looking at all?
2 Likes
Hi @conrad.braam ,
I did a basic tutorial long time ago and put up some sample repo base (not sure it helps/addresses your needs): Testing Flutter (iOS, Android, web, desktop) Applications - Xray Cloud Documentation - Xray
However, I’m not an expert on flutter; so others may want to jump in and perhaps better insights on this.
Regards,
Sergio
1 Like
According to testing level you are use ,for example component test you can use "flutter driver "
for system /integration test you can use “appium flutter driver”
Flutter driver :
appium flutter driver :
note: lesson learned from my trial if you gona do POC do the full cycle specially CI/CD as you will face challenges with every phase
2 Likes
Hi Conrad
Two recent years of testing a Flutter app.
The app was very unreliable and the team was very chaotic when I arrived, so focus on making sure developers had Android and iOS devices (stop relying on emulators) and then test together. Then focus on unit tests for business logic and widget tests for visual layer. There was no way we could get started with higher level automation as the app was extremely wobbly and crashy.
We went like this:
Stage 0: Just get devs to build the app locally and make sure it compiles on BOTH iOS and Android. The most common wheeze was to have the ability to run one platform. Add build pipeline in CodeMagic or similar on pull request and push. Review pull requests together, the basics of being alive in software development.
Stage 1: Add unit and/or widget tests for each pull request. Testing on devices by developers, I would then test the more risky work. Standard linting for Flutter/Dart.
Stage 2: Golden tests for more detailed safety around visual changes: flutter/docs/contributing/testing/Writing-a-golden-file-test-for-package-flutter.md at master · flutter/flutter · GitHub
Stage 3: Flutter integration tests used sparingly (just the stuff that will get you fired) - Introduction | Flutter running in the pipeline. Integration tests spin up the entire app, its just what they call them. Flutter is migrating away from flutter driver too.
Stage 4: Feature flags with Growthbook or similar, has a Flutter SDK. Mobile development is still treacherous, protect yourself accordingly.
General advice - many Flutter packages on pub.dev are flakey and have dubious test coverage. Check very carefully. Its still early days for Flutter after all.
1 Like
This is interesting, and I wish I could say thank you more.
We started learning by adding dart/flutter to a recently added new mobile app almost 2 years back, but that has not progressed much, instead we opted to write a desktop app first. And then come back and re-do the mobile app later, so a lot is in flight and we are seeing the kinds of pain you describe, but even difficult more on desktop. The framework does not feel ready for prime time. That CI/CD build pain does appear to hit everyone, so some defensive strategy to our testing is going to be what I take away from this advice, thank you @ash_winter .
Yes we also saw the migration away from flutter-driver, and we are trying to be building out a wrapper for their integration-driver instead. Yet more CI/CD build output variants security security pain. Cheers.
1 Like
Had to check yesterday for a profile update, first app I tested was early 2020 and about 10 flutter apps to date.
I’ve found them less problematic than react native for example, navigation, touches etc. Mostly same codebase across Android and IOS. So the basic functional testing pretty much same as any mobile app.
Traffic interception was a pain, originally I needed to hard code my IP address into the code when that app was built, these days I usually have a local build if I need to view traffic.
Local mobile builds. Initially painful, was a bit surprised that mobile devs often feel the same pain. Lots of dependencies in flutter so I suggest pairing with a dev when you start up and then update the readme addressing general mobile issues. The dev tools are much improved and useful. There is an accessibility plugin but I have not used it much in testing to be fair.
I contributed to a series of articles on automation earlier in the year, two in the series are flutter specific - Rebelapps
Unit, widget, golden, integration layers. Patrol gave access to the native views and has an awesome finder feature so fairly clean code. Note automating mobile app at UI layer is in my view more problematic than web by far so it took us some time to get tests stable so we keep it light at that layer. More info in the articles.
Oh, flutter apps also have a sort of glass pane so if on web many tools and inspectors will not work as you’d expect.