Test Impact Analysis - My story so far
It's been three months now that I have started my journey in a new working environment to implement a flavor of the Test Impact Analysis.
Being a fan of ThoughtWorks and following Martin Fowler's blog I was impressed about reading an article that spoke about the rise of TIA(Test Impact Analysis)
The definition of TIA from martin fowler "Test Impact Analysis (TIA) is a modern way of speeding up the test automation phase of a build. It works by analyzing the call-graph of the source code to work out which tests should be run after a change to production code. Microsoft has done some extensive work on this approach, but it's also possible for development teams to implement something useful quite cheaply."
Problems to solve:
- Let's run all tests every time a change is pushed
- Tests that run late in the integration cycle - Implicitly Shifting right
- Number of tests that run in the pipeline
- The number of tests in the regression suite
- What shape is the testing pyramid?
- Where are the requirements covered for each slice of the pyramid?
- Everything that you test pre-integrate should immediately be tested post-integrate in the Continuous Integration (CI) infrastructure
- Deployment pipeline to run slower tests later.
- Tagging tests - how do we understand and track the domain knowledge? (Bird's eye view)
- Tests with their notional 'size' designations (execution time)
- Automated subsetting of tests to run per commit based on this baked-in intelligence.
- TIA support in IDEs - Microsoft also has a powerful Live Unit Testing feature in Visual Studio that, if enabled, automatically runs impacted unit tests even as you edit the code.
- TIA is all about incremental validation by automatic test selection.