ISTQB CTFL Syllabus Uncovered: Your Ultimate Guide Vol4

5.1. Test Planning

5.1.1. Purpose and Content of a Test Plan

A test plan is like a roadmap for testing. It tells us what we want to test, how we’ll do it, and who’s involved. It’s important because it helps us do testing the right way and prepares us for challenges.

The plan includes:

• What we’re testing
• Assumptions and limits
• People involved
• How we communicate
• Possible problems
• How we’ll test
• Time and money

You can learn more in the ISO/IEC/IEEE 29119–3 standard.

5.1.2. Tester’s Contribution to Iteration and Release Planning

In iterative software development, there are two types of planning:

* Release Planning: Prepares for the product release. It refines tasks and involves testers in writing testable user stories, estimating test effort, and planning testing for the entire release.

* Iteration Planning: Focuses on a single iteration. Testers analyze risks, break down tasks, estimate test effort for those tasks, and refine aspects of what needs to be tested.

5.1.3. Entry Criteria and Exit Criteria

Entry criteria are the conditions that must be met before starting a task. If they aren’t met, the task may become harder, costlier, and riskier. Exit criteria are the conditions that must be fulfilled to finish a task. These criteria vary based on the objectives.

For instance, entry criteria include having resources, test materials, and an acceptable initial quality level. Exit criteria involve measuring thoroughness and confirming completion. In Agile, they’re known as “Definition of Done” for completion and “Definition of Ready” for starting work. If time or budget is exhausted, it can be a valid reason to end testing, with stakeholder approval.

5.1.4. Estimation Techniques

Test effort estimation means predicting how much work is needed for testing in a project. It’s based on assumptions and may have errors. For accurate estimates, break big tasks into smaller ones.

Four estimation methods:

• Ratios: Use past project data to find test efforts based on development efforts.
• Extrapolation: Collect early data, then estimate the rest using a model, good for iterative projects.
• Wideband Delphi: Experts make estimates, discuss, and adjust until they agree (like Planning Poker in Agile).
• Three-point estimation: Experts provide optimistic, likely, and pessimistic estimates; the final estimate is the weighted average.

These methods help in planning and are used in the field of software testing.

5.1.5. Test Case Prioritization

Test cases are organized into schedules for execution. Prioritizing test cases depends on factors like risk, coverage, and requirements. Common strategies are:

• Risk-based: Execute test cases based on identified risks. Start with those addressing the most important risks.
• Coverage-based: Prioritize based on coverage (e.g., statement coverage). Begin with test cases achieving the highest coverage.
• Requirements-based: Order based on stakeholder-defined requirement priorities. Test cases linked to the most crucial requirements go first.

In an ideal scenario, tests are run by priority, but dependencies and resource availability can influence the order. For instance, if a high-priority test depends on a low-priority one, the low-priority test goes first. Resource availability must also be considered.

5.1.6. Test Pyramid

The test pyramid is a methodology that assists teams in planning their tests. It demonstrates that different tests serve different goals and have varying sizes.

Small, short tests at the bottom examine individual sections. Many of these are required for adequate coverage. At the top are larger, slower tests that go across the entire system. You just need a handful of them.

The tiers and nomenclature of the pyramid might vary, such as unit tests, integration tests, and end-to-end tests. It serves as a manual for test automation and effort allocation.

5.1.7. Testing Quadrants

The testing quadrants help manage testing in Agile. The group test types and activities are based on their purpose.

There are four quadrants:

• Technology-facing, supports the team (e.g., component tests, automation).
• Business-facing, supports the team (e.g., functional tests, user story tests, can be manual or automated).
• Business-facing, critiques the product (e.g., exploratory testing, user acceptance testing, often manual).
• Technology-facing, critiques the product (e.g., smoke tests, non-functional tests, often automated).

This model helps everyone understand which tests are needed for different aspects of the software.

5.2. Risk Management

Risk management helps organizations achieve their goals and improve product quality by dealing with uncertainties. It involves identifying and assessing risks and then taking actions to control and monitor them.

In testing, risk-based testing means choosing, prioritizing, and managing test activities based on risk analysis and control. It helps focus testing efforts where they matter most.

5.2.1. Risk Definition and Risk Attributes

Risk is a possible problem with two parts: how likely it is to happen and how bad it would be if it does. The higher the risk, the more you need to deal with it.

5.2.2. Project Risks and Product Risks

In software testing, we worry about two types of risks: project risks and product risks.

Project risks involve issues with managing the project, like delays, skill shortages, or technical problems. When these happen, they can affect the project’s schedule, budget, and goals.

Product risks are about problems in the software itself, such as missing features, errors, or security issues. When these occur, they can lead to user dissatisfaction, financial losses, and even legal trouble.

5.2.3. Product Risk Analysis

Product risk analysis in testing aims to understand and reduce the risks in a software product. It involves identifying and assessing risks early in the development process.

Risk analysis includes finding and categorizing risks, evaluating their likelihood and impact, and coming up with ways to deal with them. This analysis helps decide the scope of testing, what types of tests to use, and how much effort is needed. It also helps prioritize testing to find critical issues early and consider other ways to reduce risk.

5.2.4. Product Risk Control

Product risk control means taking actions in response to identified risks in software. It involves two parts: risk mitigation and risk monitoring.

Risk mitigation means doing things to reduce the risk, like selecting skilled testers, using different testing techniques, and performing dynamic testing.

Risk monitoring keeps an eye on whether the actions are working and helps identify new risks. Response options can include testing to reduce risks, accepting them, transferring them, or having a backup plan.

5.3.Test Monitoring, Test Control and Test Completion

Test monitoring is about collecting data to track testing progress and see if goals are met.

Test control uses this data to make decisions and take action for more effective testing, like changing priorities or adjusting schedules.

Test completion gathers data and lessons learned when a testing phase is finished, a project is completed, or software is released.

5.3.1. Metrics Used in Testing

Test metrics are data used to track progress, quality, and efficiency in testing. They help control and complete testing.

Common test metrics include data about project and test progress, product quality, defects, risks, coverage, and costs. These metrics help evaluate how well testing is going and if it’s on schedule and budget.

5.3.2. Purpose, Content, and Audience for Test Reports

Test reporting is about sharing information during and after testing. It helps control and evaluate testing.

Test progress reports are regular updates that include the test period, progress status, issues, test metrics, and plans for the next period. They help make adjustments as needed.

Test completion reports summarize a specific testing stage and include a test summary, evaluation, deviations, test metrics, unresolved issues, and lessons learned. These reports are more formal and are prepared when a testing phase is finished.

The content and formality of reports depend on the audience and purpose.

5.3.3. Communicating the Status of Testing

How you share test information depends on your needs. Options include talking to team members and using dashboards, emails, online documents, or formal reports. Choose the best method based on your situation, like using formal reports for distributed teams. Tailor communication for different stakeholders.

5.4. Configuration Management

Configuration management in testing is about keeping track of test-related documents and tools. It records changes, versions, and relationships, creating baselines for approved items. This helps trace and reproduce results. CM ensures unique IDs, version control, and clear references in test documentation.

In automated DevOps pipelines, configuration management is typically included in continuous integration, delivery, and deployment processes.

5.5. Defect Management

Defect management is crucial for finding and resolving issues in testing. Defects might not always be real problems; they can be false alarms or change requests. The process includes steps like reporting, analysis, classification, resolution, and closure. Defect reports provide details about the issue, helping to fix it and track product quality. They include a unique ID, summary, date, author, context, description, severity, and more. Automated tools can help manage defect data. Templates and examples are available in standards like ISO/IEC/IEEE 29119–3.

6.1. Tool Support for Testing

Test tools aid various testing tasks, such as managing the software development lifecycle, reviewing code, generating test cases, automating testing, and ensuring non-functional aspects. They also help with DevOps, communication, scalability, and deployment standardization. Even tools as simple as spreadsheets can assist in testing.

6.2. Benefits and Risks of Test Automation

Test automation can save time, reduce errors, provide objective measures, offer better test management, speed up defect detection, and free testers for more effective testing. But, there are risks like unrealistic expectations, inaccurate estimations, overreliance on tools, vendor dependency, choosing the wrong tool, and compliance issues. Automated testing isn’t always the best choice, and it’s essential to consider these factors for successful implementation.