How does a Test Execution Technique differ from a Test Design Technique?

A Test Design Technique (like Equivalence Partitioning or Boundary Value Analysis) is about what to test. It helps you create a minimal yet effective set of test cases from a large set of possibilities. In contrast, a Test Execution Technique is about how and in what order to run those designed test cases. Design is about creating the tests; execution is about running them strategically.

In a CI/CD pipeline, what kind of Test Execution Technique is most common?

In a CI/CD pipeline, the most common techniques are automated, parallel, and risk-based. Parallel execution is vital to run large test suites quickly and not slow down the pipeline. The execution is almost always automated. Often, a risk-based or impact-analysis approach is used to select a subset of regression tests to run on every commit, saving the full regression suite for less frequent runs (e.g., nightly builds).

Test Execution Technique

Q: Why is choosing an appropriate Test Execution Technique important?

Choosing the right Test Execution Technique is crucial for efficiency and effectiveness. It helps in finding critical defects early (e.g., with risk-based testing), saves time and resources (e.g., with parallel execution or targeted regression), and provides the most relevant feedback to developers at the right time. A poor technique can lead to wasted effort, delayed feedback, and critical bugs slipping into production.

Intermediate

A Test Execution Technique is a method or strategy for running tests. It defines the order in which test cases are executed to efficiently find defects, manage resources, and provide timely feedback, based on factors like risk, requirements, or available time.

First used·1980s

Definitions·1

Synonyms·3

Category·Software Testing

Also known as

Test Run StrategyTest Execution MethodTest Sequencing Approach

Definitions

What it means.

01
General Software Testing Context
A Test Execution Technique is a systematic method used to determine the order and manner in which test cases are executed. It is a crucial component of a test strategy that focuses on the how and when of running tests, rather than just the what to test.

The primary goal is to maximize defect detection efficiency, manage resources effectively, and provide timely feedback to the development team. The choice of technique depends on factors like project goals, identified risks, available time, and the nature of the application under test.

Key Concepts

Test Prioritization: Ordering tests based on importance, risk, or business impact. High-priority tests are run first to find critical defects early.

Test Scheduling: Planning when specific test suites will be run, often aligned with build cycles or release schedules.

Automation vs. Manual: Deciding which tests are executed by automated scripts and which require human intervention.

Common Techniques

Risk-Based Execution: Prioritizing and executing test cases that cover the highest-risk features or areas of the application first.

Requirements-Based Execution: Executing tests in an order that follows the flow of business requirements or user stories.

Exploratory Testing: A simultaneous learning, test design, and execution technique where the tester explores the application to find defects. This is less structured but highly effective for finding non-obvious bugs.

Regression Test Execution: Running a subset of existing tests after a code change to ensure that the changes haven't introduced new defects. This can involve full regression, partial regression, or using test impact analysis to select only relevant tests.

Parallel Execution: Running multiple tests simultaneously on different machines or threads to significantly reduce the total execution time, which is crucial for CI/CD pipelines.

Origin

Where it comes from.

Etymology

The term is a composite. 'Test' originates from the Old French 'test' (an earthen pot), used for assaying precious metals. 'Execution' comes from the Latin 'exsequi', meaning 'to follow out' or 'carry out'. 'Technique' derives from the Greek 'tekhnikos', meaning 'of art, skillful'. Combined, it signifies the skillful method of carrying out tests.

Historical context

In the early days of software (1950s-60s), testing was ad-hoc and unstructured, often performed by developers simply running the program to see if it crashed. The concept of a formal Test Execution Technique was non-existent.

The 1970s brought structured programming, which led to more organized testing, but execution remained largely manual and sequential. It was in the 1980s, with the rise of formal methodologies like the V-Model, that distinct test levels (unit, integration, system) necessitated more sophisticated strategies for test execution. This is when concepts like risk-based testing began to influence the order of test runs.

The 1990s and 2000s were transformed by test automation tools. This introduced new execution techniques like data-driven and keyword-driven testing, shifting the focus to creating reusable and maintainable test suites.

From the 2010s to the present, Agile and DevOps have championed continuous integration and continuous testing (CI/CT). Modern Test Execution Techniques now heavily feature parallel execution, execution in containerized environments, and intelligent test selection algorithms (like test impact analysis) to accelerate feedback loops.

Usage

In context.

For our critical release, we adopted a risk-based Test Execution Technique to ensure the most fragile parts of the system were validated first.
The QA lead decided that a Test Run Strategy involving parallel execution was necessary to meet the tight deadlines of the CI/CD pipeline.
Instead of following a rigid script, the team used an exploratory Test Execution Method to uncover usability issues that automated checks would miss.

FAQ

Common questions.

Choosing the right Test Execution Technique is crucial for efficiency and effectiveness. It helps in finding critical defects early (e.g., with risk-based testing), saves time and resources (e.g., with parallel execution or targeted regression), and provides the most relevant feedback to developers at the right time. A poor technique can lead to wasted effort, delayed feedback, and critical bugs slipping into production.

Taxonomy

What it means.

General Software Testing Context

Where it comes from.

In context.

Common questions.

Filed under.