Governance v. Culture:
Who Will Win?
This week’s AI conversation has exposed a tension that extends far beyond model safety. Several major technology leaders have publicly acknowledged the need for stronger safeguards, more rigorous testing, and greater caution around advanced systems, yet those same organizations continue to operate in an environment defined by intense competitive pressure. The contradiction is not surprising. Governance may tell an organization when it should slow down, but culture determines whether anyone actually can or will.
Formal governance does not operate independently from the incentives surrounding it. An organization can establish policies, review gates, testing requirements, approval structures, and risk controls. Everything that people discuss as "human-in-the-loop." The reality is that those mechanisms will always compete with other priorities such as speed, market position, revenue, productivity, efficiency, and customer expectations. When leadership communicates that all of these objectives matter, the real organizational culture becomes visible at the point where they conflict and someone has to decide which priority carries the most weight.
The pressure does not have to be explicit. Few leaders will directly instruct employees to ignore a governance process or bypass a safeguard, but culture can communicate the same expectation through deadlines, performance metrics, executive attention, promotion decisions, investor pressure, and the way an organization responds when someone introduces friction into a high-priority initiative. If raising a concern consistently makes that person the one always slowing progress, governance may remain intact on paper while the organization gradually teaches people not to use it.
This is where governance becomes an operating issue rather than a policy issue. The meaningful question whether the person responsible for invoking it has the authority, organizational support, and protection necessary to do so when the decision conflicts with a business objective. Safety versus production, quality versus deadlines, compliance versus revenue, accuracy versus throughput, and security versus convenience are not new tensions. Artificial intelligence is simply making the consequences of those tensions more visible and potentially more significant.
Effective governance must account for culture as part of the governance architecture itself. A framework that assumes people will consistently choose the governed path regardless of organizational pressure is a fantasy because governance is ultimately tested at the decision point, not in the policy document. When competing objectives collide, the organization reveals what it truly values, and that is where governance either becomes real or becomes performative.
AI isn’t the problem. Alignment is.
This Week’s Insight:
When Culture, Capability, and Human Judgment Collide
The tension between governance and culture becomes consequential when the systems themselves are treated as more objective, capable, or trustworthy than the people expected to oversee them. Once AI begins to carry the appearance of authority, organizational pressure can make it harder for individuals to slow a process, question an output, or challenge the assumptions embedded within it.
In The Myth of the Unbiased Machine, I examine the assumption that replacing or supplementing human judgment with artificial intelligence necessarily produces a more objective or unbiased decision. AI can reduce some forms of inconsistency, but consistency is not the same as neutrality or objectivity. Models, retrieval systems, workflows, performance measures, and optimization criteria all reflect the biases inherent in the data as well as cultural influence about what information matters, which outcomes are desirable, and what level of error is considered acceptable. Once those choices become embedded within a system, they may become harder to see because they are presented through scores, rankings, recommendations, or polished analyses that appear technically authoritative, objective, and unbiased..
Culture becomes important because employees rarely evaluate AI outputs in a vacuum. They are working inside organizations that reward particular outcomes, prioritize certain metrics, and communicate expectations about speed, productivity, efficiency, and performance. An AI-supported process can reinforce historical assumptions contained within data, and current priorities within the organization deploying it. If the culture discourages employees from questioning a system that appears to be working efficiently, the assumptions embedded within that system gain additional authority simply because challenging them introduces friction.
That concern extends directly into AI Literacy Without Reasoning Literacy Is Dangerous. Organizations are teaching employees how to use AI, write better prompts, automate tasks, and incorporate generative systems into everyday work, but tool proficiency does not automatically create judgment. A person may become exceptionally good at obtaining sophisticated answers while becoming less capable of determining whether those answers deserve reliance. Human-in-the-loop governance means very little if the human lacks the domain knowledge, critical reasoning, or organizational confidence necessary to question, override, or reject the system when something does not look right.
These ideas point back to the initial organizational problem raised earlier. Governance depends on more than policies, technical controls, or the presence of a human reviewer. It depends on whether people recognize when judgment is being delegated, understand what assumptions are influencing the decision, and feel empowered to introduce friction when the evidence warrants it. An organization can have highly capable AI, technically sophisticated employees, and extensive governance documentation, and still fail at the decision point if its culture rewards compliance with the system more strongly than the thoughtful challenge of it.
This Week’s Practical Takeaways
- Test governance where pressure is highest. Do not evaluate a governance framework under normal operating conditions. Examine what happens when deadlines tighten, revenue is at risk, competitors move faster, or leadership wants an immediate result.
- Identify who has authority to create friction. Every AI-enabled process should make clear who can pause, challenge, escalate, override, or reject an output or decision. Authority that exists theoretically or only on paper is not meaningful governance.
- Review the incentives surrounding human oversight. If employees are rewarded primarily for speed, volume, efficiency, or delivery, asking them to slow a process for additional review may conflict with the behaviors the organization actually rewards.
- Separate consistency from objectivity. A system that produces the same kind of answer repeatedly may be consistent without being neutral, fair, or correct. Review the assumptions, data, objectives, thresholds, and organizational priorities embedded within the process.
- Develop reasoning alongside AI proficiency. Training employees to prompt, automate, and use AI tools should be paired with the ability to question assumptions, evaluate evidence, identify missing context, and recognize when an answer should not influence a decision.
- Watch what happens when someone challenges the system. Organizational culture becomes visible when an employee questions an AI-supported recommendation or asks to slow a process. The response to that challenge may reveal more about the strength of governance than the written policy ever will.
A Moment of Reflection
Take a moment this week to consider one simple question:
If governance told my organization to slow down,
would our culture really support it?
If the answer depends on the deadline, the executive involved, the competitive pressure, or the cost of delaying a decision, that is worth examining. Governance is not proven when conditions are easy. It is proven when doing the right thing introduces friction, and the organization still gives people the authority and support to act.
Closing Thoughts
AI governance is tested by the policies an organization writes when those same policies become inconvenient by the organization's culture and the choices employees have to make. The real measure is whether people are still expected, empowered, and supported to question assumptions, challenge outputs, and slow a process when pressure is pushing in the opposite direction. Culture will always reveal which priorities carry the most weight, which is why governance must be designed for how organizations want to operate, as well as for how they really behave when the stakes are highest.
Find this useful? Share it with someone who would appreciate it. |