When Should a Supply Chain Pilot Scale, Change or Stop?

A pilot needs a decision, not just a demonstration

A planning tool produces a convincing dashboard. A new replenishment rule reduces stock on a selected product range. A workflow handles the cases shown in a demonstration. These are useful observations, but they do not yet establish that the change is ready for wider use.

Before extending a pilot, the business needs to understand what worked, under which conditions, what it cost and what remains uncertain. SCP would build that decision into the pilot from the start, with an owner, a review date and explicit options to scale, change or stop.

ASQ describes Plan–Do–Check–Act as a repeating improvement cycle: plan a change, test it on a small scale, study the results and act on the learning. Its guidance supports iteration when a change does not work and wider implementation when the evidence supports it. ASQ: the PDCA cycle.

Define acceptance before collecting results

Start with the operating problem and a credible baseline. Specify which orders, sites, users and exceptions are included. Record the business requirements and agree measurable acceptance criteria before testing begins. A requirement such as ‘improve visibility’ needs to become an observable behaviour that users can assess.

Illustrative example, not a client result: a delivery risk management pilot might require that held inventory is excluded from available stock, missing supplier confirmations remain visible and recommendations cannot change customer commitments without the agreed authority. Define the expected outcome for each case, including the awkward cases, before reviewing the solution.

Keep the links clear: business requirement → acceptance criteria → test cases → results and UAT evidence → acceptance decision. Criteria describe what must be true; evidence records what actually happened. A pass rate alone cannot explain whether an untested business-critical exception remains.

Separate a defect from a requirement that needs to change

If the solution does not meet its agreed specification, investigate a defect. If it behaves as specified but the business needs something different, raise a change request. Record the reason, proposed requirement, impact on cost and timing, dependencies and the person authorised to decide.

Updating the requirement should also update the affected design, acceptance criteria and tests. Retest the changed behaviour and relevant existing functions. This prevents a pilot from drifting into an undocumented solution that nobody has formally accepted.

Check readiness beyond the happy path

The testing approach depends on the solution. A configured package may need product acceptance testing before integration testing. UAT then asks business users to assess representative end-to-end work against agreed criteria. Operational acceptance activities can run alongside this work, covering matters such as performance, recovery, support and monitoring.

Check that users can manage exceptions, that support ownership is understood and that cutover and rollback arrangements are workable. Passing functional tests does not by itself establish operating readiness or a financial benefit. Compare outcomes with the baseline and consider other changes in demand, staffing or product mix that could explain the difference.

Make the three options explicit

• Scale: agreed criteria are met, material risks have acceptable controls and the operating owner can support the next stage. Expand in a defined step, with continued measurement.
• Change: the idea remains credible, but a specific gap needs attention. Agree the change request, revised scope, cost and next review point before extending the pilot.
• Stop: the evidence does not justify further investment, the problem has changed or a simpler alternative offers a better route. Retain the learning and close the pilot deliberately.

SCP recommends recording the decision, evidence, unresolved issues, accountable owner and follow-up date in a short review record. Where evidence is inconclusive, describe the uncertainty and the smallest useful next test instead of treating the absence of a failure as proof of success.

Connect the pilot to continuous improvement

Our six-stage approach connects understanding and diagnosis with design, testing and sustained improvement. A pilot review may send the team back to clarify a requirement or investigate a cause. It can also establish a controlled next step towards wider delivery.

The useful outcome is a decision the business can explain and act on. For a Control Tower or delivery risk management pilot, that means connecting reliable inputs and tested decisions with named operating owners and benefits that can be measured after implementation.