Why Trust Is the Real Problem with AI — And What to Do About It

A few months ago I was asked to contribute a chapter to a new book called The AI Opportunity. The editors wanted a practitioner's perspective — not another "AI is coming" think piece, but something grounded in how AI actually lands, or fails to land, in environments where physical things get made. I wrote Chapter 14: Designing AI-Enabled Operations That Humans Can Trust.

The book is now published, and I want to share the core ideas here in a practical format that is a bit more direct. Because honestly, the message is something I have been trying to get across for years in various ways, and the book format gave me the chance to put it all in one place.

Here's the short version: The reason most AI deployments in manufacturing fail isn't the technology. It's the fundamental approach and architecture, and that is what I define as composability.

Let me unpack why.

The Trust Problem Is Not What You Think

When manufacturing leaders talk about "trusting AI," the conversation usually gravitates toward ethics, model accuracy, or data quality. Those things matter — but they are not where trust actually breaks down on the shop floor.

In operations, trust is earned through repeated, consistent performance in the flow of everyday work. It's not about whether a model is statistically impressive in a benchmark. It's about whether a system behaves predictably when conditions change, supports human judgment under pressure, and fails safely when uncertainty rises.

Manufacturing makes accountability impossible to avoid. When AI influences how a deviation gets handled, whether a product ships, or how a piece of equipment gets configured — responsibility doesn't transfer to the algorithm. Humans are still on the hook. So when an AI system obscures its reasoning, works from incomplete data, or overrides operator expertise, trust evaporates fast. And in regulated industries like pharmaceuticals or medical devices, where explainability isn't optional, an untrustworthy system isn't just frustrating — it's ungovernable.

The trust problem in AI is not primarily an ethical or philosophical issue. It is a design problem.

That is the chapter's central argument, and I think it's the most important thing for executives and practitioners to internalize before they make another AI investment.

Why Most AI Deployments Disappoint — And What To Do About It

We have all seen the statistics — AI pilots that don't scale, initiatives that live forever in proof-of-concept purgatory, strategy decks full of AI ambition and shop floors that look the same as they always did. At this point we have all heard this ad nausoum - on a daily basis! But the pattern is real and worth understanding, because the failure mode is consistent and fixable.

AI gets layered onto existing operational systems — systems that were engineered for stability and repeatability, not adaptation. These systems encode yesterday's processes — and not just yesterday's, but the assumptions of the 1980s electronic era that gave rise to the monoliths that still runs manufacturing today. Layering AI onto these foundations doesn't fix the mismatch; it deepens it. More digital on top of the wrong structure is still the wrong structure, its literally lipstick on a pig.

There's also a velocity problem. AI capabilities now evolve on a monthly cadence. Operational systems and governance models move in multi-year cycles. The gap between what AI can do and what organizations can absorb keeps widening, creating what I call "velocity paralysis" — organizations that recognize the opportunity but can't act fast enough to capture it.

And then there's the context problem, which is the one I've been writing about for years in this blog. More than half of operational knowledge exists as "dark data" — observations, judgment calls, visual cues, and institutional experience that never enter digital systems. When AI is built on this incomplete foundation, its recommendations feel disconnected from what operators see and know. It reinforces skepticism rather than trust. And skepticism, as I've written before, kills more innovation than failure ever does.

AI now makes the digital divide even more clear, while at the same time providing a unique opportunity to cross it faster.  

Composability and Agentic AI: The Design Answer

The answer to the failures described above is composability — and I want to be precise about what I mean, because the word gets misused. Composability is not a software architecture concept. It is the method for transforming operations in the digital era, defined by five reinforcing pillars: bottom-up adoption, agility (what I call Augmented Lean), democratization, human-centricity, and compliance. Together, these pillars create operational systems that can absorb change continuously without losing control — which is exactly what AI deployment requires. I've written about the 5 Pillars and digital maturity at length elsewhere, so I won't retread all of it here. The point for this chapter is that composability is the precondition — without it, AI accelerates fragility rather than productivity.

The book chapter also introduces the framework for deploying agentic AI within this composable foundation. The core idea: a human actor — an engineer, a citizen developer, an operations lead — remains at the center, defining intent rather than writing code, and retaining accountability for outcomes. Two classes of AI agents operate around them: Staff or Companion Agents that augment human judgment with insight, pattern recognition, and recommendations without executing changes independently; and Builder Agents that create operational artifacts — workflows, connectors, data models — within explicit governance constraints. The result is an agentic system that can evolve rapidly without destabilizing operations, because scope is bounded and outputs are transparent. I published a full treatment of this Composable Agentic Framework last year — if this is new to you, that's the place to start.

What Executives and Practitioners Actually Need to Know

I'll close with the practical takeaways, because that's why this chapter exists.

For executives: The risk calculus has flipped. For decades, the conservative position in manufacturing was "wait and see." That position is no longer conservative — it's dangerous. Organizations that learn to apply AI responsibly within their operations will become structurally more productive, more adaptive, and more competitive over time. The gap between those who do this well and those who don't will compound. As I put it in the chapter: what begins as prudent caution can quietly turn into strategic disadvantage. Don't be the last dinosaur.

For practitioners: The question is not whether to implement AI — it's whether to implement it in a way that actually works in your environment. That means resisting the pressure to layer AI onto rigid monolithic architectures. It means designing for human agency from the start, not as an afterthought. It means treating data integrity as a precondition, not a downstream problem. And it means choosing composable, modular approaches that allow your operations to absorb change continuously — because the pace of AI evolution is not going to slow down to match your ERP upgrade cycle.

For both: Trust is not a soft concept you address in the governance policy document. It's a hard design requirement that shapes every architectural decision. Build systems that humans can understand, intervene in, and be held accountable for — and AI becomes a genuine multiplier. Build systems that obscure logic, override expertise, and create black boxes on the shop floor — and you'll spend the next three years explaining why the pilot succeeded but the rollout stalled.

The future of AI will not be decided in models or labs alone. It will be decided where work happens, one operation at a time, by organizations that have the courage to engage, the discipline to design for trust, and the humility to let humans and machines learn together.

More Is Coming

If these ideas resonate — trust as a design problem, composability as the operational method, agentic AI governed by human accountability — there is a lot more on the way. I am currently working on writing the Augmented Lean Field Guide, the practitioner companion to the original Augmented Lean book. It goes significantly deeper than any single chapter or blog post can: full frameworks, worked design patterns, and the full arc from first pilot to continuous transformation. It will be out by the end of 2026, and I'll be sharing more from it here as we get closer.

Democratization Is Not Anarchy. Learn to Deal with IT!

In nearly every conversation I have with manufacturing executives about their digital transformation programs, the topic of democratization surfaces the same way. As a problem. "How do we make sure people aren't building things outside of our control?", "How do we enforce compliance with IT governance?" "How do we prevent shadow IT from getting into production?" "What's our policy on AI-generated code?" The conversation turns to containment before it ever reaches potential. And in my experience, that is exactly the wrong place to start.

I've written about democratization as one of the 5 Pillars of Composability for years. It generates more internal friction than any other pillar — not because it is the most dangerous, but because it is the most misunderstood. Organizations treat it like a problem to be controlled. What they should be treating it like is the engine that has been quietly driving their best productivity gains for the past three decades — whether they recognized it or not.

The evidence has been right in front of us the entire time. We just keep refusing to see it.

The Proof Has Been Hiding in Plain Sight

Look at Microsoft Excel, there is no piece of enterprise technology that has driven more distributed manufacturing productivity gains than a spreadsheet application that anyone can use. Long before no-code platforms existed, Excel was what people reached for when the digital solution they needed simply didn't exist — or when the monolithic system in place couldn't provide it. From simple hour-by-hour production trackers to complex inventory management models, engineers and operators built what they needed with what they had. The gap was always there, because ERP, MES, WMS, QMS, EAM — every monolithic system of record was designed for structured, standardized workflows. The contextual, the ad-hoc, the highly specific operational problem always fell through the cracks. Excel caught them. It democratized data manipulation — it placed analytical and tracking capability in the hands of every engineer, quality manager, and production planner on the floor. IT organizations hated it. Shadow IT. Ungoverned. Risky. And yet — it worked. The productivity it unlocked was enormous, precisely because the people building the tools were the people who understood the problems.

The same story played out with Human-Machine Interfaces on the plant floor. When SCADA and DCS vendors began providing configuration environments that operators and process engineers could use directly, the pace of process improvement accelerated. The automation engineers closest to the process could suddenly instrument, visualize, and adjust without waiting months for a programmer to interpret their requirements, write a specification, queue the work, and deploy a change. Democratization of configuration capability drove productivity. Again.

The pattern is consistent: 

When you give operationally knowledgeable people the tools to solve their own problems, the rate at which problems get solved increases dramatically. This is not a hypothesis — it is a thirty-year track record.

This is precisely the type of democratization that no-code and low-code platforms have extended into frontline operations over the past decade. The shift from "IT builds, operations uses" to "operations builds what they need" compresses the lag between identifying a problem and solving it from weeks to hours. As I've described through the lens of digital maturity, organizations that make this shift don't just get more efficient — they develop a fundamentally different operational capability. They become adaptive

AI Is Closing the Last Gap — and That Changes Everything

No-code lowered the barrier to building digital solutions, but it still required learning a new environment, a new paradigm, a new way of thinking about logic and workflow. AI is closing that gap entirely. You can now describe what you need in plain language and have a working solution generated in front of you. The barrier to content creation for is approaching zero.

Think about what that means for the examples we just discussed. The engineer who used to spend days building a complex spreadsheet model to track WIP and yield can now describe the logic conversationally and have the model built for them. The process engineer who needed a SCADA supplier's configuration team to update an HMI display can now specify the change in plain language and iterate in real time. AI isn't just another wave of democratization, it super charges it. It skips no-code as the primary mechanism by which non-programmers create digital solutions.

And here is the implication that most organizations are missing: as the barrier to creation approaches zero, the value of the platform it runs on increases dramatically. Anyone can generate a solution. Not everyone is generating solutions that are version-controlled, validated, connected to the right data sources, maintainable by someone other than the person who built them, and operating within a compliant environment. The purpose-built platform — composable, human-centric, compliance-ready — becomes more critical, not less, as AI democratizes creation. Ungoverned AI generation without a platform foundation is not democratization. It is the digital equivalent of everyone writing their own procedures on sticky notes.

As I described in A Composable Agentic Framework for Frontline Operations, the emergence of builder agents — AI that helps domain experts design and iterate digital solutions in real time — is the realization of this shift. The question it raises for every manufacturing organization is not "how do we control this?" It is: do we have the platform foundation to make what our teams are about to build actually work?

And the organizational response? The same one we've seen before. Fear. The containment reflex. "What's our policy on AI-generated code in production?" "How do we prevent people from deploying things that haven't been validated?" "Who is accountable when something built with AI goes wrong?"

I am not saying those are wrong questions. I am saying they are being asked before the far more important one: what becomes possible when your process engineers, quality specialists, and production leads can build and iterate the tools they need in hours rather than weeks? That is the question that unlocks value. Governance comes second — as the framework that makes value creation sustainable — not as a replacement for asking whether value is even being pursued.

In both previous waves of democratization, the organizations that responded with blanket restriction fell behind the ones that built governance structures capable of channeling the new capability. As I've been observing as the industry traverses the digital divide, the companies pulling ahead are not the most cautious — they are the ones with a sound strategy and culture that enables effective governance.

Why Democratization Gets Managed Instead of Harnessed

The answer is structural, and it runs deep. Most manufacturing organizations still operate with a mental model inherited from the era of monolithic systems — where digital capability was scarce, expensive, and necessarily centralized. In that model, IT was the gatekeeper because it had to be. Building anything digital required specialized skills, expensive licenses, and careful change management. The architecture was fragile. A mistake in one place could propagate across the whole system. In that context, tight central control was not a choice — it was a necessity.

The decline of monolithic architectures didn't just change the technology. It changed the risk profile. Composable platforms are designed for distributed development — with version control, role-based permissions, validated templates, and isolated workspaces that contain failure to a single application or station. But organizational culture moves slower than technology. The gatekeeping mindset persists long after the scarcity that justified it has disappeared.

So we still see organizations applying the change governance frameworks designed for monolithic systems deployments to no-code app development. We see IT review boards that were built to manage quarterly release cycles now being used to evaluate whether a process engineer can add a field to a workstation app. The tools changed. The governance didn't. And the result is that organizations spend more energy suppressing the creative capacity of their most operationally knowledgeable people than they do enabling it.

There is also a subtler dynamic that I've observed consistently. Organizational skepticism tends to attach itself to democratization specifically because its outputs are distributed and visible — not because they are more dangerous than centralized systems. A poorly architected process buried inside a monolithic MES can affect the entire operation and take months to unwind. A poorly designed app built by a process engineer affects a single workstation and can be corrected in an afternoon. The distributed failure mode is actually less catastrophic. But it's more visible, and visibility triggers the control reflex — even when the underlying risk doesn't warrant it.

Democratization Is Not Anarchy. It Requires Democratic Governance.

Here is the point I want to make directly: democratization is not the absence of rules. It is the distribution of capability within a system of rules.

We have a model for this. It is called democracy. Functioning democratic systems are not anarchies — they are the most sophisticated governance structures humans have built. They have constitutions, laws, institutions, independent accountability mechanisms, and ethical norms. They distribute power not because they have abandoned governance, but because they have built governance structures capable of handling distributed power. The result — when it functions — is a more resilient, adaptive, and innovative system than any centralized alternative has ever achieved.

The comparison to manufacturing governance is direct. What I have consistently called controlled democratization means giving people the capability to solve their own problems within a governance framework designed to channel that creativity productively — not police it into submission. Policing is a dictatorial method. It is also what traditional top-down, monolithic systems use. And it is precisely why those systems generate compliance without generating adaptability.

The most well-governed democracies do not work because they have the most police. They work because they have strong institutions, a culture that understands and values the rules, and mechanisms for accountability when those rules are violated. Manufacturing organizations that want to harness democratization need to build the equivalent: platforms that enforce governance by design rather than by gatekeeping, centers of excellence that guide rather than approve, and a culture that treats accountability and empowerment as the same thing — not opposites. The power of that combination, when you've seen it operating at scale, is not incremental. It is categorical.

The Question Is Not Whether. It's How.

Your teams will use AI to build things. The only variable is whether they do it inside your governance framework or around it. Blanket restrictions don't prevent use — they push it underground, where it operates without documentation, without platform guardrails, and without accountability. That is the actual risk. And it is a risk created entirely by the policing reflex.

Every previous wave of democratization taught the same lesson. The organizations that tried to stop it accumulated missed improvements, unsolved problems, and eventually lost the people who were motivated enough to try. The organizations that built governance structures to channel it gained ground that compounded over time.

Continuous transformation is only achievable if democratization is operating at full potential. You cannot get there by treating your most operationally knowledgeable people as risks to be managed. The difference between democratization and anarchy has never been the absence of rules — it has always been the quality of governance.

Build governance worthy of the capability your teams are ready to use. Don't be the last to start.

Observing The Industry Traversing the Digital Divide — Its Finaly here!

Earlier this week at Nvidia’s GTC conference, I had a moment of reflection that, for me, brought a lot of threads together. The energy around AI was undeniable—but more importantly, it wasn’t just hype or futuristic vision. It was grounded in real capability, real deployment patterns, and a clear signal of where the industry is heading.

I shared some of my immediate thoughts in a LinkedIn post during the event, but stepping back, what stood out most was this: the conversation has fundamentally shifted. AI is no longer being discussed as an isolated capability or an experimental technology. It is being positioned as a core building block of how systems will be designed, how operations will run, and how value will be created.

For someone like me—who has been writing for years about composability, democratization, and the need for a new operational architecture—this felt less like a surprise and more like a confirmation. The pieces I’ve been describing are starting to come together in a very visible way.

And it reinforced something I’ve been saying for a long time: manufacturing is on the verge of a fundamental shift. Not another incremental improvement cycle, not another wave of disconnected digital initiatives, but a real transformation in how operations are run, improved, and scaled.

For years, that message felt like a warning. A call to prepare. Today, it feels more like an observation.

Because what I saw at GTC—and what I’ve continued to see in conversations across the industry—is that companies have reached the divide and looking at crossed it. The conversations have changed. The posture of leadership has changed. And most importantly, the level of commitment has changed.

I referred to this earlier in my 2025 trends webinar as a watershed moment, and what we are seeing now is exactly that playing out in real time. I would strongly encourage you to watch that discussion, as it frames much of what is now unfolding across the industry:

What’s important is not just that change is happening—but how it is happening!

Vibe Coding and the Realization of Democratization

One of the clearest signals of this shift is how solutions are now being created. I’ve spent a lot of time over the years writing about democratization in manufacturing—the idea that the people closest to the work should be empowered to improve it, and that technology should enable that rather than constrain it. What is emerging now with AI, and what some are starting to call “vibe coding,” is the most complete realization of that idea that I’ve seen in my career.

What makes this different from previous waves of low-code or no-code is not just accessibility, but the collapse of effort between intent and execution. The ability to describe a problem, iterate on a solution, and see something functional emerge in minutes fundamentally changes the dynamic of how operations evolve. It brings solution creation directly into the operational context, where engineers, operators, and subject matter experts can shape systems in a much more immediate and iterative way. We are now seeing a world where:

• A process engineer can describe a problem and generate a working application
• An operator can help shape a workflow in real time
• A team can iterate on solutions at a pace that was previously unimaginable

This is not incremental improvement. This is a step change in how value is created and something I have consistently pointed to in my writing on composability and frontline operations platforms.

The shift from centrally developed, rigid applications to adaptable, user-driven solutions that reflect the reality of the shop floor.

But what is becoming clear now is that AI is not just enabling this shift—it is accelerating it to a point where it is unavoidable and, I feel, it's removing the mindset barrier. The discussions about technical capabilities, or features and functions are quickly fading away, including the odd ask about monolithic systems and OOTB configurations. They are shifting to be about how quickly solutions it can be built and how effectively it can be applied. That changes expectations at every level of the organization, particularly at the executive level, where the potential for rapid productivity gains becomes much more tangible.

At the same time, this level of democratization introduces a new kind of responsibility. When the ability to create is broadly distributed, the risk of creating the wrong thing—or creating the right thing in the wrong way—also increases. This is where the narrative needs to mature beyond excitement about capability and into a deeper understanding of what it takes to operate in this new model.

Why Platforms Are Now Critical to Operational Integrity

As AI transforms the ability to create solutions, it is tempting to assume that bringing those solutions into operations will follow the same path. This is where manufacturing fundamentally pushes back. The same forces that make “vibe coding” so powerful—the speed, the accessibility, the freedom to create—also introduce a level of variability that operations simply cannot absorb without consequence. In a production environment, the introduction of new technology, solutions, logic, automation, or decision-making is not an isolated act. It becomes part of a tightly coupled system where even small inconsistencies can propagate quickly.

In these environments, the consequences of error are immediate and often irreversible. A mistake cannot be rolled back with a software update, and failures in safety, quality, or compliance can have serious and lasting impact. This reality fundamentally reshapes what trust means for AI. Trust is not about believing that a model is intelligent or statistically accurate, but about whether a system behaves predictably under changing conditions, supports human judgment, and fails safely when uncertainty arises. In operations, trust is earned through repeated, consistent performance in the flow of everyday work.

While AI can generate applications, workflows, and even autonomous behaviors with remarkable speed, manufacturing requires that every one of those elements operates within clearly understood and controlled boundaries. One misstep—whether it’s an incorrect parameter, an unexpected interaction, or an opaque decision—can create cascading effects. Quality can be compromised, performance can degrade, and most critically, safety can be put at risk. In my experience, nothing halts adoption faster in a manufacturing organization than a single visible failure that undermines confidence in the system.

You cannot afford uncontrolled experimentation in a live production environment. This is why I’ve consistently emphasized the importance of a platform-based approach—not as a technology preference, but as an operational necessity. A true operational platform provides:

• Governance over what is created and deployed
• Context so that solutions are aligned with real processes
• Control to ensure consistency, traceability, and compliance
• Resilience so that failures are contained and managed
• Connectivity so that decision and action are based on a holistic understanding
• Content that is industry specific and ready to increase quality and resilience

Accountability in this environment is unavoidable. When AI influences how equipment is configured, how deviations are handled, or whether a product is released, responsibility does not shift to the algorithm. Humans remain accountable for outcomes, which makes human-in-the-loop not just a design preference, but a requirement. If an AI system makes a mistake, and they certainly do, trust erodes quickly—and once that trust is lost, it is very difficult to regain. This is even more pronounced in regulated industries, where expectations around data integrity, traceability, and explainability are explicit, and systems must be understandable not only to technologists, but to operators, engineers, quality professionals, and regulators.

This is precisely why a platform approach is not optional—it is foundational. A manufacturing-focused platform creates the controlled, governed environment where AI can actually operate within the strict realities of production. It is what ensures that solutions are not only created quickly, but behave predictably, meet quality standards, respect safety constraints, and remain compliant over time. Without that structure, the same capabilities that make AI so powerful will introduce unacceptable risk. In manufacturing, you cannot compromise on errors, defects, or safety—and you don’t get multiple chances to get it right. A purpose-built platform is what makes it possible to harness the benefits of AI and “vibe coding” without violating the core requirements of the operation. With a platform, you enable what I often describe as controlled democratization—the ability to innovate broadly, but within a structure that protects the integrity of the operation. Without it, scale is not just difficult—it’s dangerous.

Why Domain Expertise Still Defines Success

The final and perhaps most critical element in all of this is the role of domain expertise—something that is increasingly being underestimated in the current enthusiasm around AI. There is a flawed narrative that AI can compensate for gaps in knowledge or experience, that it can generate solutions independent of deep understanding. But as I have explored in other posts, particularly when experimenting with AI as a creative partner, the technology is only as effective as the context and intent that guide it. In manufacturing, this distinction is not subtle—it is fundamental.

With the incredible democratization AI brings to creating solutions accelerates, this constraint does not disappear—it shifts. It becomes even more important and critical to define the right problem and to judge whether a solution will actually work within the realities of the operation. Manufacturing processes are complex, tightly interconnected, constraints by physical realities, driven by well defined methods, and governed regulatory requirements. Understanding how cause and effect play out in that environment is not something that can be inferred generically; it is built through experience, engineering discipline, and operational knowledge. AI can amplify that expertise, but it cannot replace it—and without it, the risk of creating solutions that fail in practice increases significantly.

In the hands of those with deep expertise, AI accelerates learning, experimentation, and scale. This becomes even more critical as we move toward more autonomous systems, where agents are expected to act within operations. Their effectiveness depends not just on data, but on the depth of understanding embedded in how they are designed—grounded in the experience of those who know how the system behaves, especially when things don’t go as planned.

The Take-Away

What we are seeing right now is the convergence of three defining forces: 

1. The democratization of solution creation through AI.
2. The need for structured platforms to govern and control that creation.
3. The enduring importance of domain expertise to ensure it all works in the reality of manufacturing operations. 

This convergence is not theoretical—it is actively reshaping how companies think about, design, and run their operations.

Crossing the digital divide was never just about connecting systems or digitizing processes. It was about enabling a fundamentally different way of operating—one where the creation, deployment, and continuous improvement of solutions are embedded directly into the fabric of the operation. What we are now beginning to see is what that actually looks like in practice, and it is both powerful and unforgiving.

As with any significant shift in manufacturing, success will not come from simply adopting the latest technology. It will come from understanding how to integrate these capabilities into the operational reality—balancing speed with control, innovation with discipline, and democratization with accountability. The companies that get this right will not just move faster—they will operate differently, and ultimately, outperform.

Video Illustration: The AI Knowledge Revolution

An Alternative Visual

This is my alternative visual narrative that explores how AI and specifically Agentic AI are fundamentally disrupting traditional manufacturing hierarchies. The video illustrates the "compression" (or collapsing) of the classic Data-Information-Knowledge-Wisdom (DIKW) pyramid, showing how AI now acts as an intelligent intermediary that instantly transforms unstructured "human language"—like deviation comments and work instructions—into actionable operational wisdom

Key themes include:

• Collapsing Complexity: Moving past the rigid, million-dollar data models of the 1990s to a system that understands context like a human.
• Knowledge Flow: Driving multi-site transformation through "Outbound" digital playbooks and "Inbound" frontline innovations.
• Augmented Lean: Democratizing expertise across the entire network so every site becomes both a consumer and a producer of wisdom.

I also talked about it in an interview for Tulip's "Human in the Loop" series.

Behind this is a body of work and a lot of written content that I will publish in the future. As I have written before I am experimenting with different formats to convey the message about composability. 

Stay tuned more content will be coming out in the future!