When Reckoning Isn’t Enough

---

While we are striving to stay current, we are not experts on artificial intelligence (AI), and the field is moving at light speed. Those who are experts may see much in Brian Cantwell Smith’s writing that we have missed or perhaps have been mooted by subsequent developments. That said, we think Smith’s primary value is in laying a foundation for discovering and discussing a borderline — one that is increasingly elusive — between a machine that does not care and humanity, which does.

Judgment, as used by Smith, refers to a quality of mind (someone has “good” or “poor” judgment, or perhaps “wisdom” or “understanding”); a faculty of mind (what might be referred to as “prudence” — the capacity to weigh advantages and disadvantages and resolve thought into action); and an act of the mind¹.

In this later context, Bertrand Russell said, “In every act of judgment, there is a mind which judges and there are terms concerning which it judges.” Using these three subcategories, the notion of judgment and the divide between a machine (AI) and the human mind can begin to be clarified and understood.

The Great Divide – What Is Registered and What Is Reality

The comedian George Carlin did a bit about a tough guy who had a tattoo of barbed wire encircling his bicep. Carlin made hay by pointing out that the tattoo was only an image of barbed wire, rather than barbed wire itself. He mused on the fact that an image was hardly as intimidating as the real thing. Carlin’s comedy bit hinged on the wide gap between reality and a representation of reality. In the world of AI, Smith refers to this representation as registration, i.e., how computer systems must necessarily function — through the medium of electronic storage of data. Registration is based on counting.

In The Promise of Artificial Intelligence: Reckoning and Judgment, Smith is pointing out the same basic concept. What he calls “registration” is the data AI uses. It cannot actually “capture” real barbed wire around an arm, but it captures a sentence describing barbed wire around an arm, or a digital image of barbed wire.

Smith argues that real intelligence goes beyond the first-level concern that “understanding” has a primary root in how the world is registered in data systems. Second, that the capacity to develop and define the registration systems and concepts for AI’s use is itself an act of intelligence. And third, that anything approaching judgment requires something altogether more demanding: a commitment to that which is registered — a commitment to its truth, its existence outside of the registration schema, and perhaps a commitment to its ultimate “un-registerability” (our word, not Smith’s). In our example, AI cares about neither a tattoo nor real barbed wire, yet we care about the man and what the barbed wire says about him, whether is it merely an image or if it is real.

Topic: Can a Reckoning System Exercise Judgment?

“Reckoning” is the term that Smith uses “for the types of calculative prowess at which computer and AI systems already excel.”² Smith is concerned that our reckoning systems “lack the ethical commitment, deep contextual awareness, and ontological sensitivity of judgement.”³ In other words, AI systems are good at counting (reckoning) but not at judgment.

Smith offers multiple important insights as to the constitution of judgment, ranging from the intuitive and thoughtful, to the philosophical and esoteric.

For example, he points out that judgment is constituted with the properties of intention and semantics, and as such, is not about describing physical effect or occurrence, nor is a description of facts about what is in the machine. Consequently, “any question about the presence of judgment is a question about the system ‘under interpretation,’ not about its mechanical form.”⁴

Secondly, judgment exists at a meta level of how a system works, how things are tied together, how conflicts are resolved, how it responds, how it sorts. Smith argues that as such, “judgment is unlikely to be an isolable or categorical property of an architecture.”⁵

Third, and perhaps most importantly, “judgment involves a system’s giving priority to the world, over its internal state and representation — and over the patterns exemplified by any inputs, machinations and ‘interior’ processes.”⁶

What does Smith mean, “giving priority to the world”? In so many words, he is making a plea for understanding the nature of existence, of being itself. While this question occupies many philosophers and religious thinkers, it takes on an urgency now at the dawn of this new, powerful technology as we seek to maintain sovereignty over the technology.

Smith writes:

… the third level of commitment must be to that which we register — to that which we take to be a table, in the case of a table; to that which we register as a person, in the case of a person; and so on. Not (first stage) to the representation “table” or “person,” and not (second stage) to the table as registered as a table or to the person as registered as a person (since that would not leave us in a position to be able to say “that isn’t a person after all” — or even “what you are taking to be a table isn’t even a coherent object”). Rather only if we are committed (third stage) to that which we take to be a table, person and so on, will we be in a position to hold the registration to account. (Emphasis added.)⁷

It is this accountability to “that which is” that becomes the line in the sand. Do we have accountability? Once this prime directive comes into focus, all the other battlefields are clarified. Who uses AI, who develops AI, is it the province of “private enterprise” or the regulators, or the Chinese, or rogue actors? Whose data feeds the database? Is surveillance of every electronic remnant from every private person’s every action merely a data point available to those who can scrape it, store it, and sell it?

And while these questions are front and center facing us as a society, managers at any level, at whatever stage their organization has traveled down the path of AI, can look to the concept of accountability in how they adopt, adapt, limit, promote or exploit the promise of AI. Accountability to all stakeholders, to regulators, and to the truth of “that which” the world is.

Smith sets forth the standard that would constitute a system of full-scale judgment:⁸

1. Systems existentially committed to the world they register, represent and think about.
2. Systems that will go to bat for the truth, reject what is false, balk at what is impossible — and know the difference.
3. Systems not only in and of the world, but for which there is a world — a world that worlds, in the sense of constituting that to which all is ultimately accountable.
4. Systems that know what the world that hosts them, the entities they reason about, and all of humanity and community as well, must be treated with deference, humility and compassion. (Emphasis added)
   Smith offers the opinion, published in 2019, that he does not see the near-term possibility of constructing systems capable of full-scale judgment. He opines that it is humans who exercise judgment, and that to be fully human, a person must exercise judgment.

Smith’s discussion of AI includes both first-wave AI (what Smith calls “good old-fashioned AI,” or GOFAI) and second-wave machine learning, or ML AI.

But the principles and concepts of Reckoning and Judgment are still relevant and important to understanding AI, even though we have advanced to large language models, reasoning and learning models, and agentic AI. Recent predictions are that we will have artificial general intelligence (AGI) by 2028 or 2030. With AI “brains” harnessed to physical “bodies” — vehicles, robots, weapons, etc. — we will be faced with entities that can not only learn, predict and self-improve, but also to respond to goals rather than prompts and to take physical action. Experts in the philosophy of moral boundaries of waging a “just” war are already urging a universal ban on lethally autonomous weapons (LAWs).⁹

Perhaps your attention just skipped right over the phrase “respond to goals rather than prompts” in the previous passage. We think of AI as a computing “brain,” but when this brain is harnessed to vehicles, robots, drones and weapons, it can be autonomous in taking independent physical action.

A “machine” that can take action and at the same time can set in motion the achievement of a goal is a very different sort of entity from one that merely offers an answer to a prompt such as “make a list of ways I can sell more widgets than my competitors.” Imagine a machine that can implement actions when given the goal “make my competitors fail.” Such a capacity raises real ethical challenges.

Smith then raises the question: “What could lead a system to judgment?”¹⁰ His answer compares it to child-rearing practices, which are interactive and “progressive, building on prior experiences, and which use norms or standards.”

Conclusion: What Are We, as Managers, to Learn from Smith’s Book?

He explores the nature of the divide — representation vs. reality — in spite of human limitations, human error, human bias, etc., perceiving reality and the notion of the “thing being the thing that it is.”¹¹

To read Smith’s book is to take a dive into the heavy philosophy of knowing, of ontology (the formal representation of knowledge), epistemology (the theory of knowledge itself), of truth, of where one finds the divide between understanding the world and merely processing data representing the world. We all know the typical conversation about AI: We muse on how it affects our jobs, how it saves time, how it is scary, how it is overblown. These conversations tend to be at the level of what AI does and how it either does things well, or it affects efficiency, or it weakens human learning, to name a few topics.

And although AI has advanced considerably since the book was published, Smith lays out a foundation of principles and concepts that endure as technology advances. As we develop machines that have the capability to exercise “judgment,” it is critical for managers and decision-makers to understand how slippery the notion of a machine without empathy and authenticity is offering ideas and taking autonomous action at a foundational level.

---

Categories: Business