Parallelize your subset runs with the Launchable CLI
Step-by-step process to harness split-subsetting in Launchable
Key Takeaways
You don't need to choose between subsetting or parallelizing your tests.
Combine this new feature with Predictive Test Selection to supercharge your testing velocity.
Should you subset your tests, or should you parallelize them? Now you can do both with launchable split-subset, which makes it easy to divide a subset into equal chunks to be run in parallel.
Test Subsetting Example
For example, let's say you currently run ~80 minutes of tests split coarsely into four bins and run in parallel across four workers:
Worker 1: 20 minutes of tests
Worker 2: 15 minutes of tests
Worker 3: 20 minutes of tests
Worker 4: 25 minutes of tests
By combining launchable subset --split and launchable split-subset, you can generate a subset of the full 80 minutes of tests and then call Launchable once in each worker to get the bin of tests for that runner.
To continue the example above, if your subset target is to run 25% of the total duration (or 20 minutes of tests), then this lets you split those 20 minutes of tests across your existing four runners (5 minutes each).
The high level flow to subset in Launchable
First, request a subset of tests to run from Launchable by running
launchable subsetwith the--splitoption. Instead of outputting a list of tests, the CLI will output a subset ID that you can save to pass into each worker.Start up your parallel test workers, e.g. four runners from the example above, passing in the subset ID value (either as a file or a variable, etc.).
In each worker, get the tests that worker should run with
launchable split-subset, e.g.launchable split-subset --subset [SUBSET_ID] --bin 1/2 bazel > tests.txt.This will output the tests for that that worker tests to a file that you can use to run the tests for that worker.
After each run finishes in each worker, record test results using launchable record tests with the --subset-id option set to the ID you saved earlier.
This new feature lets you take advantage of predictive test selection and parallelization at the same time. To use launchable split-subset, upgrade your Launchable CLI to version 1.24.0, and check out the documentation to learn more.
