Integration testing

 

Integration testing tests interfaces between components, interactions to different parts of a system such as an operating system, file system and hardware or interfaces between systems. Objectives of integration testing include:

• Reducing risk
• Verifying whether the functional and non-functional behaviors of the interfaces are as designed and specified
• Building confidence in the quality of the interfaces
• Finding defects (which may be in the interfaces themselves or within the components or systems)
• Preventing defects from escaping to higher test levels

In some cases automated integration regression tests provide confidence that changes have not broken existing interfaces, components, or systems.

There are two different levels of integration testing, which may be carried out on test objects of varying size as follows:

• Component integration testing focuses on the interactions and interfaces between integrated components. Component integration testing is performed after component testing, and is generally automated. In iterative and incremental development, component integration tests are usually part of the continuous integration process. 
• System integration testing focuses on the interactions and interfaces between systems, packages, and microservices. System integration testing can also cover interactions with, and interfaces provided by, external organizations (e.g., web services). In this case, the developing organization does not control the external interfaces, which can create various challenges for testing (e.g., ensuring that test-blocking defects in the external organization’s code are resolved, arranging for test environments, etc.). System integration testing may be done after system testing or in parallel with ongoing system test activities (in both sequential development and iterative and incremental development).
  

Test basis

• Software and system design
• Sequence diagrams
• Interface and communication protocol specifications
• Use cases
• Architecture at component or system level
• Workflows
• External interface definitions

Test objects

• Subsystems
• Databases
• Infrastructure
• Interfaces
• APIs
• Microservices

Typical defects and failures

Typical defects and failures for component integration testing include:

• Incorrect data, missing data, or incorrect data encoding
• Incorrect sequencing or timing of interface calls
• Interface mismatch
• Failures in communication between components
• Unhandled or improperly handled communication failures between components
• Incorrect assumptions about the meaning, units, or boundaries of the data being passed between components

Typical defects and failures for system integration testing include:

• Inconsistent message structures between systems
• Incorrect data, missing data, or incorrect data encoding
• Interface mismatch
• Failures in communication between systems
• Unhandled or improperly handled communication failures between systems 
• Incorrect assumptions about the meaning, units, or boundaries of the data being passed between systems
• Failure to comply with mandatory security regulations
  

Specific approaches and responsibilities

Component integration tests and system integration tests should concentrate on the integration itself. For example, if integrating module A with module B, tests should focus on the communication between the modules, not the functionality of the individual modules, as that should have been covered during component testing. If integrating system X with system Y, tests should focus on the communication between the systems, not the functionality of the individual systems, as that should have been covered during system testing. Functional, non-functional, and structural test types are applicable.

Component integration testing is often the responsibility of developers. System integration testing is generally the responsibility of testers. Ideally, testers performing system integration testing should understand the system architecture, and should have influenced integration planning.

Big-bang testing is one extreme is that all components or systems are integrated simultaneously, after which everything is tested as a whole. Big-bang testing has the advantage that everything is finished before integration testing starts. There is no need to simulate (as yet unfinished) parts. The major disadvantage is that in general it is time-consuming and difficult to trace the cause of failures with this late integration.  

Incremental testing is another extreme is that all programs are integrated one by one, and a test is carried out after each step. The incremental approach has the advantage that the defects are found early in a smaller assembly when it is relatively easy to detect the cause. A disadvantage is that it can be time-consuming since stubs and drivers have to be developed and used in the test. Within incremental integration testing a range of possibilities exist, partly depending on the system architecture:

• Top-down: testing takes place from top to bottom, following the control flow or architectural structure (e.g. starting from the GUI or main menu). Components or systems are substituted by stubs.
• Bottom-up: testing takes place from the bottom of the control flow upwards. Components or systems are substituted by drivers.
• Functional incremental: integration and testing takes place on the basis of the functions or functionality, as documented in the functional specification.

The preferred integration sequence and the number of integration steps required depend on the location in the architecture of the high-risk interfaces. The best choice is to start integration with those interfaces that are expected to cause most problems. Doing so prevents major defects at the end of the integration test stage. In order to reduce the risk of late defect discovery, integration should normally be incremental rather than 'big-bang'. If integration tests are planned before components or systems are built, they can be developed in the order required for most efficient testing.

The greater the scope of integration, the more difficult it becomes to isolate defects to a specific component or system, which may lead to increased risk and additional time for troubleshooting. This is one reason that continuous integration, where software is integrated on a component-by-component basis (i.e., functional integration), has become common practice. Such continuous integration often includes automated regression testing, ideally at multiple test levels.