Test Suite Insights Speeding Test Failure Analysis for Development
Development Managers Can Now Fix Flaky Testing
Key Takeaways
Introducing the latest addition to the Launchable platform, Flaky Tests Insights (beta).
Access to these insights will help development teams strengthen CICD pipeline observability.
To get started, sign up for a free trial, start sending data to Launchable, and your insights should be available soon after.
SAN JOSE, CA, Sept 15, 2021 — Launchable, the intelligence platform layer for all software testing, announced their latest addition to the platform, Flaky Tests Insights (beta). Access to these insights will help development teams strengthen CICD pipeline observability by quickly identifying and pinpointing the top flaky tests in a test suite to prioritize fixing the appropriate tests.
Flaky tests are a pervasive problem for development teams. Unreliable tests are an issue for development velocity, where development teams waste time diagnosing test failures to separate the real from false failures.
Today there are no tools that flag flaky tests for development teams. The knowledge that some tests are flaky builds by observing false failures in multiple runs. The bigger the test suite, the harder it becomes to keep track of all the flakes. Nobody has a comprehensive picture of flakes across test suites.
Data-driven insights on flakes can help teams prioritize their efforts on fixing the costliest issues first.
“Unreliable of flaky tests are a systemic problem in eroding development velocity, said Alastair Wilkes, Director of Product Management at Launchable. “It has been amongst the top 3 problems that development teams have consistently asked us to solve. The Launchable Intelligence Platform can easily identify and prioritize these tests helping teams to concentrate fixing the most impactful problems.”
Launchable now analyzes test runs to identify flaky tests in the suite so customers can fix them. To do this, Launchable assigns each test a flakiness score from 0-1, with higher scores indicating higher likelihood of flakiness, giving developers a clearer picture of which flaky tests to tackle first. The dashboard can be used as an information radiator for the development team to go fix issues that are impacting them the most.
“At any point in time, only a small number of tests are flaky but over a period of time decay sets into the tests,” said D.T., Engineering Director. “The problem is that people continue to live with inefficiencies. A tool that flags tests as flakes would go a long way in helping us prioritize and remove the inefficiencies.”
About Predictive Test Selection
Launchable’s Predictive Test Selection helps developers determine what tests to run. In order to take advantage of the machine learning capabilities, Launchable optionally enables teams to first test for flakiness with Flaky Test Insights, ensuring the test suites being run aren’t flaky, and in turn, fix issues that are creating significant noise for developers.
The synergy by pairing Flaky Tests Insights with the Predictive Test Selection further amplifies the feedback loop effectiveness. Tests get stronger, test selection gets better, feedback is analyzed quicker, and developers are happier with the ability to easily triage problems.
To get started, sign up for a free trial, start sending data to Launchable, and your insights should be available soon after.
About Launchable
Launchable is an intelligence platform layer for all software testing that shortens wait times for testing, speeds up and optimizes CI pipeline efficiency, leading to shipping higher-quality software faster. Through machine learning in Launchable's SaaS, developers and Dev teams can test what matters and find bugs faster, reduce risk, increase commit frequency, and decrease time-to-production, all leading to Continuous Quality. Test what matters at https://www.launchableinc.com.
