How production planning, governance and team adoption can help pharma organisations turn an AI pilot into a useful operational service. Many pilots show promise and then stall; the reasons are usually organisational rather than technical, and most can be anticipated.
Plan for production during the pilot
An AI pilot can demonstrate a useful capability without establishing whether it will work in everyday operations. Data, users, support and review requirements may be very different outside the trial environment. Planning for those conditions early makes the pilot more informative.
Define the workflow the pilot is testing, the people involved and the decision it will support. Use representative data and include users with different levels of experience. Agree what evidence is needed before a wider release, including acceptable performance, escalation and support.
A pilot designed only to show that something is possible will usually succeed, because it is run by enthusiasts on carefully prepared material. A pilot designed to inform a production decision asks harder questions. Will the tool cope with the messy, incomplete inputs that arrive in real work? Can it be connected to the systems where users already work, or will they have to copy information between applications? Who will support it at the end of the trial, and how will that support be funded?
Consider an illustrative pilot in which a commercial operations team tests a tool that drafts summaries of internal reports. During the pilot, a small group uses it on a curated set of documents and finds it helpful. When the tool is opened to the wider team, users apply it to document types it was never tested on, the summaries become less reliable and confidence drops. Testing on a broader and more representative set of material during the pilot, and defining clearly which document types are in scope, would have exposed the issue earlier.
Exit criteria worth agreeing upfront
Agree before the pilot starts what result would justify expansion, what would justify further testing and what would justify stopping. Typical criteria cover output quality on representative cases, user willingness to keep using the tool, the effort needed to review outputs and the practicality of supporting the tool at scale. Writing these down prevents a pilot from drifting on indefinitely because nobody is sure whether it has succeeded.
Build governance into the delivery plan
Review requirements influence how an AI system should be designed. Access, information handling, source traceability, evaluation and human decisions need attention before implementation is complete.
Bring the relevant business, IT, security, legal and compliance teams into discovery. Document responsibilities, limitations and the conditions under which the system can be used. The controls should match the intended use; a drafting assistant and a tool supporting a clinical decision require different consideration.
When governance is added after a pilot, it often requires changes to the design: logging that was not built in, access controls that need restructuring or source references that the tool was not set up to provide. Each change costs time and can undermine confidence. Involving these teams early lets their requirements shape the design from the start. Depending on the use, the relevant considerations in pharma may include GxP validation expectations, data protection under GDPR and the risk-based obligations introduced by the EU AI Act.
A useful governance artefact is a short use statement for each tool, describing what it is for, what it must not be used for, who reviews its outputs and what users should do if something looks wrong. This gives users clarity and gives auditors a clear starting point.
Prepare people and processes
Introducing AI can change the work around it. Teams may need to review outputs, record decisions, manage exceptions or use information differently. Training on the tool is only one part of that change.
Explain the purpose, involve users in testing and give them a route to raise concerns. Agree who owns adoption and how feedback will influence the next release. Business leaders, delivery teams and people responsible for training all have a role.
Adoption tends to falter in predictable ways. Users who were not involved in the pilot may not understand why the tool exists. Managers may expect immediate productivity gains while users are still learning how far to trust the output. Process documentation may not be updated, so people are unsure whether the AI-assisted route is officially sanctioned. Addressing these points directly, through clear communication, practical training on real cases and updated procedures, matters as much as the technology.
Training is most effective when it covers what to do when the tool gets things wrong, not only how to use it when it works. Users who know the common failure patterns and the escalation route are better placed to use the tool well.
Measure use after launch
A release is a point for review, rather than proof that the investment has succeeded. Compare the agreed measures with the baseline and examine quality, adoption, support effort and unintended effects.
Plan a structured review a few weeks after release and at regular intervals afterwards. Look at who is using the tool and who is not, and ask why. Examine whether review effort has fallen as expected or whether checking has become a hidden cost. Watch for workarounds, which often indicate that the tool does not fit the workflow as well as intended. Our workflow application for a pharma company’s European operations is an example of a workflow designed with the people who run the process day to day.
Where to start
Start with a defined scope, review the evidence and expand only when there is a clear case. Glyz can support readiness, delivery planning, governance and team capability around that process, through our AI strategy and governance work, development and automation and capability and change programmes.
