IT Engineering

Continuous Testing

"The traditional testing methods and processes we have in Nationwide lack flexibility. Our response to this in the Test CoE is Continuous Testing. Continuous Testing is designed to provide fast and continuous feedback to the level of business risk carried in that build or release. The aim is ultimately still the validation of both the functional and non-functional characteristics of the solution
under test, which is achieved through unit, system, integration testing and NFT. However, there’s an architectural shift from driving tests at the UI to testing the Application Programming Interface (API) level. The API is considered the most stable interface in a solution and typically subject to the least change, which it lends itself very well to automated testing as scripts are easier to maintain, so can be shifted left into Agile development sprints.

"The key difference with delivering Speed Layer compared to traditional projects is that we adopted an engineering-led approach. Automation typically wins the plaudits, but without a sound foundation of test intelligence, the tests you apply over the top will be brittle and meaningless. A fool with a tool, is still a fool! Led by Simon Snow, my team undertook a significant data archaeology exercise into the world of mainframe, so that we knew individual data attributes, edge cases and transition states from source to presentation layer. Once complete, we created our functional tests which are automated and orchestrated in our pipeline.

"Another shift in mindset is recognising non-functionals as constraints on our stories, not an activity to push to the right of a plan. By componentising NFR’s, we were able to prove our solution’s horizontal scalability in less than five months from the first line of code being written.

"We also had good fun dusting off an old testing technique from 1971 called mutation testing. It’s designed to evaluate the effectiveness of our tests. We implemented this during the Unit testing phase, to improve tests and reduce the volume defects found in system test. A true shift left.”

Look at the stats

They speak volumes. This was the first time that Nationwide has ever tested across different data centres. The Speed Layer team tested everything, no handoffs at all and they could then be 100% confident with the tests.

Greenfield technology

Grant Valentine, the Delivery Lead for Speed Layer told us “With Speed Layer being a greenfield technology, we had a real chance to redefine how we deliver a heavy lifting project, embrace new ways of working, and introduce engineering-led techniques such as test driven development and automation. It was great to have someone like Martin on the team who was so enthusiastic about taking Nationwide testing to the next level in terms of our test engineering capability. A really impressive and pleasing outcome.”

COO Patrick Eltridge said “The conception of the need for Speed Layer exemplifies our innovative strategy and architecture thinking, and the pursuit of an implementation that delivers resilience, improved member service, and the potential for game-changing analytical insight in the moment shows our ambition and determination to deliver for our members and our other stakeholders. And the way it has been delivered, the engineering-led approach is game changing. Well done to Steve, Grant and the team on this critical first step and I look forward to future developments in this area.”

Massive congratulations to everyone involved and we look forward to seeing what Speed Layer brings next.