Testing strategy / Phases and readiness
Test the product, the connections and the business outcome.
For a package-led change, SCP would agree a route from component checks through product testing, System Integration Testing (SIT) and User Acceptance Testing (UAT), followed by cutover and go-live readiness. Define each phase’s scope, owner, environment, entry and exit criteria, evidence and approval authority in the test strategy.
Agree the terminology: Product Acceptance Testing (PAT) is used here for the client/provider’s agreed package-level acceptance phase, such as a configured or extended ERP product. Its name and boundary vary between organisations. System Testing (ST) tests the complete application; it is distinct from unit testing. Avoid duplicating ST and PAT where one agreed phase can cover the required objectives.
Component / unit and component integration testing
Developers check individual components and their interactions before broader application testing.
SCP delivery focus: confirm provider responsibilities and retain relevant results for changed calculations, extensions and internal interfaces.
System Testing (ST)
Test the complete application’s behaviour, including relevant functional and non-functional requirements.
SCP delivery focus: link application-level scenarios to the functional specification and agreed criteria; record the tested build, configuration and defects.
Product Acceptance Testing (PAT)
For the agreed package scope, demonstrate that configuration and modifications meet the specified product requirements before wider integration testing.
SCP delivery focus: agree who accepts the package and what evidence is needed. For an ERP change, check configured rules and extensions, including failure and boundary cases, within the package test boundary. Use controlled substitutes for external systems where necessary.
System Integration Testing (SIT)
Test interactions between systems and external services.
SCP delivery focus: exercise ERP, WMS, transport and other relevant handovers using realistic end-to-end scenarios. Include mappings, timing, missing or duplicate messages, failure recovery and reconciliation. Resolve or explicitly disposition issues before business acceptance testing.
User Acceptance Testing (UAT)
Business users assess whether the solution meets agreed business needs and acceptance criteria.
SCP delivery focus: business representatives execute realistic work and exceptions, with exploratory challenge as well as planned scenarios. Record results, defects and unmet needs; route changes through change control. UAT is more than trying to break the solution: it establishes whether people can perform the required work.
Operational Acceptance Testing (OAT)
Confirm that the service can be operated and supported in the agreed production conditions, with operations and support owners involved.
SCP delivery focus: demonstrate monitoring and alerts, incident escalation, access administration, backup and restore, recovery or failover, scheduled jobs and support runbooks as relevant. Record operational acceptance against agreed service criteria.
Cutover readiness, go live and early-life support
Use an authorised readiness decision to move from an accepted solution to production operation.
SCP delivery focus: rehearse migration and reconciliation, confirm access, training, support, monitoring, rollback or recovery arrangements, owners and communications. Record the go/no-go decision, conditions and outstanding risks; verify production operation after release.
OAT and performance checks can run alongside delivery
Plan OAT alongside SIT and UAT where dependencies and a suitable environment allow; it is not simply a final step after UAT. Combine business acceptance, operational acceptance and the remaining readiness evidence at the go/no-go decision.
Performance and other non-functional testing: response times, throughput, concurrent-user load, peak demand, endurance and recovery behaviour need measurable targets and representative workloads. Performance testing is a test type that can span system testing, SIT and operational readiness; it is not a synonym for OAT. Agree specialist ownership and environment requirements early.
Illustrative plan: component checks → ST / agreed PAT → SIT → UAT → cutover and go live. OAT and relevant non-functional tests run in parallel when ready, with their exit criteria feeding the same release decision. The exact overlap and gates are agreed in the delivery plan.
Testing terminology in plain English
- Unit / component testing
- Checks an individual piece of the application.
- ST — System Testing
- Checks the complete application or product.
- PAT — Product Acceptance Testing
- The agreed package-level acceptance scope; define this local term with the client and provider.
- SIT — System Integration Testing
- Checks the handovers and interactions between systems.
- UAT — User Acceptance Testing
- Business users assess fitness for agreed business needs.
- OAT — Operational Acceptance Testing
- Operations and support teams assess readiness to run and support the service.
- Non-functional testing
- Checks quality characteristics such as performance, reliability, security and accessibility.
- Regression testing
- Checks that a change has not adversely affected other behaviour.
- Cutover
- The controlled transition of systems, data and people into production operation.
Entry and exit criteria: agree when to start and when to move on
Projects and programmes use decision gates between phases and sub-phases. Entry criteria define what must be ready before work starts; exit criteria define the evidence and conditions required to complete that phase. These apply across discovery, design, build, testing and deployment, as well as between testing phases such as PAT, SIT and UAT.
Agree the criteria, measures, evidence owners and approval authority in advance. Completing one phase does not automatically make the next phase ready: check its entry criteria separately. Phases may overlap where the agreed scope, dependencies and risks allow.
Example: entry to UAT
- The agreed SIT scope has met its exit criteria and its results are available.
- The correct build, configuration, interfaces, access and representative test data are ready in a stable environment.
- Business scenarios and acceptance criteria are agreed; users, support and time are available.
- Blocking defects are resolved and retested, and remaining defects and workarounds are understood.
Example: exit from UAT
- At least 95% of the agreed in-scope tests have been executed and at least 95% of those executed have passed.
- All mandatory and business-critical tests have been executed and passed, including relevant exception paths.
- No open Priority 1 / critical blocking defects remain, using the project's agreed severity and priority definitions.
- Remaining failures, blocked or unexecuted tests, defects and residual risks have a recorded disposition, owner and action date.
- The authorised business owner reviews the evidence and records the acceptance decision and any conditions.
Illustrative thresholds, not a universal rule: 95% is an example to agree for each project. Execution coverage and pass rate are different measures and should both be reported against a controlled test scope. A headline percentage cannot override a failed critical process, a blocking defect or a mandatory control. Priority and severity should be defined separately, including which defects block progression.
Record the gate decision: proceed, proceed with explicitly authorised conditions, or do not proceed. Any permitted exception needs a recorded rationale, risk acceptance, accountable owner and follow-up date. UAT exit approval does not itself authorise deployment: cutover, operational readiness and the next phase's entry criteria still need approval.
Make the handover between phases explicit
For an SCP engagement, a readiness record would identify the version and scope tested, requirements and acceptance criteria covered, results, open defects, residual risks, accountable owner and decision. Agree severity thresholds and blocking conditions before execution. Passing PAT does not prove cross-system integration; passing SIT does not constitute business acceptance; UAT approval does not by itself authorise go live.
The sequence is a planning view, not a requirement to wait until late in delivery to involve users. Prepare criteria and test cases early. Agile and hybrid teams can repeat the relevant levels for each increment, with risk-based regression testing after changes. Performance, security, accessibility, data migration and operational readiness checks should be planned at suitable levels, rather than left entirely to UAT.
Test-level terminology: ISTQB Foundation Level syllabus v4.0.1, section 2.2 ↗. The PAT boundary, package-led sequence, delivery examples and readiness controls above describe SCP’s proposed application, to be agreed for each engagement.