Cleverly designed tests for products and systems are becoming increasingly important nowadays. The driving force behind this is the growing number of interfaces resulting from the increasing variety of features in combination with a high level of flexibility. Helbling has greatly refined its test methodology and tool expertise in customer projects in recent years. The focus here is on purposeful tests that measure the quality and resilience of systems.
What do Diesel Gate, a grounded aircraft fleet and a character-replacing scanner have in common? Appropriate holistic testing of the products and systems involved would probably have prevented the immense damage to the manufacturers' image.
In a world in which systems are becoming ever more complex - from individual devices to networked machines - the requirements for reliability and resilience are increasing alongside the risk of errors. An array of features in combination with high product flexibility lead to a growing number of interfaces. In such a complex system, how do you ensure that it functions as desired, remains robust and meets high customer expectations?
Testing plays an important role alongside methodology, development processes and reviews. Depending on the characteristics of a system, there are different things to consider; test abstractions must be specifically defined. In order to implement these tests efficiently and reliably, the ability to write high-quality code and hardware design are paramount, while broad experience in other areas is also relevant: Interface design, software architecture, tools for unit, integration and user interface testing, as well as build, deployment and test automation, not to mention dedicated test system hardware. Helbling combines these competencies all under one roof. In addition to product development, Helbling offers the development of test systems and test automation. This interdisciplinary expertise is the result of many years of experience.
Purposeful testing ensures cost-efficient quality
Imagine a system is developed and critical errors occur after launch in the field. Such errors can be expensive, reputation damaging, impossible to repair or even life-threatening. In the worst case, the product has to be withdrawn from the market. This is why tests - at different levels of granularity and at different points in time - are essential. By simulating or replicating the relevant use cases and real operating conditions, flaws and faults can be identified and rectified at an early stage. This includes not only the straightforward scenario, but also mistakes in the form of human error or technical faults, for example.
Achieving 100% test coverage would require immense resources. It is therefore important to test in a purposeful and efficient manner. Helbling's refined methods of risk assessment for the probability of error occurrence and their impact allow us to identify the appropriate measures and avoid unnecessary ones. Measures are not only test cases, but also, for example, changes to the design or processes. This systematic approach allows us to narrow our focus and concentrate on the essential tests.
The test pyramid - a structured approach
A well-structured test strategy comprises several levels - from unit tests to component and interface tests through to full system tests. This structure enables early and well-localized error detection, effective test maintainability and clear assignment of project roles for tests.
The test pyramid in Fig. 1 represents the comprehensive test structure taking into account the abstraction levels from hardware to the cloud.
Prerequisite: Testability and reliable test execution
A well-structured system and software architecture with clear interfaces and consideration of the testability of all components right from the design phase enable effective testing. A well-defined branching strategy and appropriate merge and release management can ensure that tests are optimally integrated into the development process and can be automated at all stages.
Automation, which is essential for reliable, stable, and cost-effective test execution, is typically implemented on a build server or a DevOps platform. These generate the software artifacts via pipelines, test them on various target platforms and, if successful, make them available as a new release. Infrastructure as Code (IaC) tools are essential for independent test execution and reliable deployment, especially in the cloud or when using containers. IaC describes the required infrastructure in the form of code, for example for servers, databases or network settings. This allows entire development and test environments to be set up and managed quickly and consistently. As a result, non-functional or destructive tests such as stress tests, penetration tests or recovery tests can be carried out in a repeatable, scalable, time and cost-efficient manner.
Further, the targeted use of various specialized tools can ensure that the product requirements are taken into account in the tests and that the tests are carried out in a traceable manner. In addition, the test results can be made accessible to a wider audience in a straightforward, easily understandable way.
Component and interface tests: Focus on integration
These tests involve the integration of one, two or several components, which are tested individually as a black box or in combination via the interfaces. Components that are not yet on the market or have not been tested, are simulated. Clearly defined interfaces and the simulation of untested components can reduce complexity and identify errors more quickly.
Three specific examples are presented in a fact box below. They refer to the test pyramid above and show the range of projects implemented by Helbling as well as the advantages of the structured approach in practice.
Summary: Specific methodology for efficient quality assurance
Purposeful and structured tests are the key to ensuring the quality and resilience of complex systems. On the basis of a tried-and-tested methodology, interdisciplinary teams and the latest tools, Helbling provides clients with comprehensive support in ensuring the resilience and stability of products. Both the smallest component and the entire system are tested.
Authors: Daniela Biberstein, Walter Pirrotta, André Wangler
Main Image: Helbling / Midjourney
