An Article Written by a Human: Are you an AI Architect or an AI plumber?

This article is based on my talk at a LegalQuants training webinar delivered in the context of LegalQuants’ first Residency Programme (see here for more). It was dictated through Wispr Flow and the format is intentionally informal.

Whether you are a board member, a CEO, a Chief Financial Officer, a Chief Legal Officer, or any other C-suite member, or just any N-X in any serious organisation today, you are under pressure to deliver with AI.

Whether it is ahead of the next board meeting, ExCo meeting, or simply your performance evaluation, you’ve probably been asked to deliver something, no matter what.

It goes without saying that you have to do something about it.

But are you being strategic about this?

By strategic, I mean, first of all, the long-term impact on your career, your image, and your reputation within the environment in which you evolve. I also mean the long-term impact and opportunity of AI on your business.

In my experience, there are three categories of people.

The first layer I would call the AI Plumbers. The Plumbers are those who deploy AI, just for the purpose of deploying AI, either because they enjoy building with AI or because they have to deliver something quickly to "look good" at some upcoming meeting.

The second layer of people are the exact opposite; they are the AI Sceptics... or the AI Indifferent. They believe that AI is a gimmick, a hype, and will usually post on social media stuff like a screenshot where their AI hallucinated (usually because of a badly designed prompt). Or, just people who don't care enough, or are too busy to pay attention to AI.

And there is a third category of people who take a strategic view, take a step back, understand how AI works, understand technicalities, architectures, weaknesses, and strengths of AI-powered systems. They use what they gather from that inquiry to consider whether their operating systems can be revamped, on the basis of what we know AI can do today. I call them the AI Architects.

I will not judge which category is the best. It is totally okay if strategy, holistic planning, and resilience in the long term are not of interest to you. It is totally okay if you just chose to outsource everything AI, or if you just want to build what you’re being told to build, or build what you personally designed in response to a discreet pain point that you have identified. It is also okay if you just want to create one product that solves a niche problem and try to "get rich or die trying" by selling it. That’s a personal choice.

It is also totally okay if you think that AI is a hype and a gimmick that will just pass or somehow become noise in the background for the decades to come. To be honest, I don’t have strong evidence that you are wrong. You might be right. I don’t intend to get sucked into the rabbit hole of that debate here.

If, however, your choice is to be or position yourself as an AI Architect, here are a few thoughts that come to mind. These thoughts are not the thoughts of an AI expert. Anybody who positions themselves as such should, in my humble opinion, be avoided like the plague. What follows is just a couple of thoughts from someone who has been deeply experimenting with the technology, with no prior knowledge, and who’s been through a lot of mistakes, a lot of trial and error.

I will address below what I think the AI Architect mindset is, and why it is still important for AI Architects to build. At the end of this article (if you ever actually get to the end of this article...), I have linked a Claude Code command that should ideally help leaders through that thought process.

Part One: The mindset of the AI Architect

This may not be popular in the short term with rushed AI transformation mandates, but an AI Architect doesn’t rush to deploy AI just to look good at his or her next exec meeting, board meeting, or personal performance evaluation. The AI Architect takes a step back and looks at how AI will strategically fit in his or her environment.

I have identified three gates in that thought process, to date:

The AI Architect decides whether the operating model of his or her environment earns its keep (a.) and how it could be completely revamped from first principles knowing what AI can do today (b). Then, the AI Architect decides whether processes that survive that first principles challenge are good candidates for AI automation, and whether such automation will provide a measurable ROI, whether to build or buy, and, finally, if the choice to build is made, whether that is a permanent build or a build that will be discarded after the task is complete (c).

Let me unfold those one by one:

a. Assessing your operating model irrespective of AI

A lot of the operating models, workflows, governance processes, generally policies, procedures, and ways of doing things in practice are inherited, old, and this is an opportunity to revisit them.

Before automating existing workflows, it is important to take a step back and first question the process, the governance, the practices themselves:

- Why does this approval sit with this function?

- Why does this function need to review? Is it adding any value? What exactly is this function reviewing?

- Why does this need to be signed?

- Why does this need a memo?

- Why does this need a PowerPoint deck?

- Is that because of legacy inherited governance?

- Is it because this process was drafted by someone who did not know the business?

- Is it purely political, because, for example, some function needs to feel involved and in control?

- Or is there a real risk being mitigated, and is the response to that risk proportional to the underlying likelihood and consequences of its realisation?

And so on.

Too often, I think these questions could result in restructuring the process in such a manner that AI is not even required, and the matter could then probably be handled by templatization, non-AI automation, or simply good management of human resources.

To take a quite telling example, early in my career as a lawyer, I came across a management committee that a lot of executives were challenging, and I was asked to automate it. That was before the AI era. My first action was to inquire when this committee was created, why, and by whom. I realised that the main reason this committee was created was to limit the powers of one of the executives whose integrity was being questioned (while figuring out a long exit procedure). Eventually, that executive exited the organisation. The committee, however, remained, but wasn’t mitigating any risk anymore. That analysis alone resulted in the deletion of that committee. There was nothing left to automate.

Another example is a workflow where multiple functions have to review. I have come across governance where multiple functions, such as tax, legal, health and safety, and others, were required to endorse, but when I asked what exactly they are reviewing, the answer was nebulous. It appeared that the process was diluting accountability for no reason. The process was simplified to retain only two reviewing functions (those who were really adding value both in terms of substance and controls). Automation was still required, but it became much simpler.

But I have also come across examples where the process exists for political reasons. This can be the case when the process is there so that a specific function, individual, or organisation can feel that they are in control (and that is not inherently a bad thing). In that case, revamping the process is a political decision and often more difficult to achieve. It is good to know that anyway though: if the political opportunity to revamp the process arises, the issue can be raised in due course.

Once you are set on what the right governance is, there are two more challenges your governance has to go through, this time with your AI lenses on.

b. Assessing your operating model on the basis of what AI can do today

First, knowing what AI can do today, is there a way that the system could be further simplified?

Second, is your infrastructure and data ready to allow the deployment of AI for that process?

On the first point, the idea is that there might be ways to replace a human review or any other type of human intervention by AI, therefore no longer requiring the process to include a specific function. For example, I have come across workflows where the only task of the reviewer and endorser is to check whether a transaction falls within a quarterly or annual threshold. Clearly, in most circumstances, in most scenarios, that sort of human intervention can be replaced by a deterministic automation (with or without AI).

On the second point, the idea is to check whether the automation or AI system you are considering will have access to reliable data and infrastructure. For example, if you were to replace an endorsement or review by an automation or an AI workflow, as set out in the example above, you first would need to make sure that you have a single source of truth, an authoritative source for the annual or quarterly threshold that the automated system can rely on. Otherwise, the system is unreliable and thus useless. Generally speaking, my understanding is that organizations that have not cleaned up and indexed their data and that have not set up the right infrastructure will find it challenging to extract any value from the technology. If that is the case in your organization, you can probably stop reading here, forget about AI, and go and get your house in order.

c. The Final AI Investment Decision

Finally, if your workflow has passed all the tests above (i.e., it doesn’t have to be revamped, and your data and infrastructure allow deployment), don’t rush. This is exactly the moment where you have to slow down. Before you actually deploy AI, I suggest that you ask yourself at least four questions:

- Volume. The question here is, “Does this happen often enough to matter?” If you do something twice a year, the time you spend building it and maintaining it might never be repaid. Be ruthless here, because we all overestimate how often the interesting problems recur. This is not black or white. A task that happens only once a year might still be a good candidate for AI. A task that happens only once could be a good candidate for AI too. But you have to think this through.

- Pattern. The question here is whether the task is patterned enough that a machine can do it and varied enough that a fixed template cannot. This is really the sweet spot, and it is narrower than we often think. If every instance is identical, you may not need AI at all. You may need a form. If every instance is genuinely novel, the AI may guess (or "hallucinate"), and you will not catch that guess. 

- Stakes. What happens if the robot is wrong, and who carries the accountability? If a workflow today has been externalized and is handled by an external vendor such as a law firm or a consultancy, optically and politically, that external vendor is carrying the responsibility of failure. If a workflow is automated in such a manner that it doesn’t require being externalised any more, then the in-house team is assuming full accountability. In-house teams may not want that accountability. To give you an example, at LegalQuants we developed The_Closer. The_Closer is a full-fledged autonomous AI agent which replaces a junior associate managing conditions precedent closure. It handles everything from setting up the calendar, building the satisfaction criteria for each condition precedent and getting them vetted by external counsel, all the way to evaluating documents against satisfaction criteria, asking external counsel to confirm legal opinions and closure of conditions precedent once all documents are secured. It handles chasing people to do their job, escalating to their boss, preparing presentations for the deal lead as well as audit reports and many other useful actions that any deal lead would appreciate. The tool could probably save tens of thousands of dollars of junior associate billable hours doing mechanical work and charged to the client. At first view, the ROI is great for the client. However, if one looks at this politically and strategically, from an in-house perspective, if the robot closes a condition precedent that should have remained open, or if it accidentally deletes all the case files, or does any other mistake, the in-house team no longer has a vendor to blame. Although the economic ROI would be high, the automation wouldn’t bring any value to the in-house team whose only reason for having recourse to this external counsel or consultant is to transfer visible responsibility (a sort of "CYA-insurance", so to speak). Showing that such tools can be built though may be a useful vector to challenge your consultants into building something similar to reduce the fees they are charging you.

- Verification. The question here is whether you can check the answer faster than you could have produced it yourself. This question is the one that quietly decides everything. If verifying the output takes as long as doing the work, the AI has saved you nothing. It has moved your labour from production to inspection and added a layer of false confidence on top. The tasks worth automating are the ones where you can confirm a good answer at a glance, even though writing it yourself could have taken an hour. A lot of examples come to mind, but I’m pretty sure you can imagine those yourself.

This is, in my view, what the first draft of the AI Architect’s mindset should look like.

But for an AI Architect to have that mindset, the AI Architect needs to deeply understand how the underlying systems work. I wouldn't exclude that, soon enough, dropping buzzwords during a meeting and talking about "AI agents" won’t cut it. Some people in your team (especially the younger generations) may start noticing and may soon realize that you have no idea what you are talking about, which may in turn affect your credibility and maybe even affect your leadership position itself. If this occurs, the natural selection on this may become quite merciless.

This is why, in my view, the AI Architect also has to be a builder.

Part Two: Why an AI Architect should still be a builder

Your edge as a senior leader in an organisation is not the tool. The tools are disposable, replaceable. Your edge is the judgement that has been informed by building things with AI.

The point of building with AI, for you, is not to become a developer. You will not out-engineer the engineers, you won’t learn coding in a few months, and you should probably not try. 

The point of building with AI is that it changes your judgement. Once you have actually tried to make a machine do a piece of your work, you understand in your hands where it is strong, where it is dangerous, and where it is quietly wrong in a way that only a lawyer or a doctor or CFO, an engineer, etc. would catch.

You can basically become the person in the room who can look at an AI workflow and say with authority, “Here is where this fails, and here is who is liable when it does.” You can understand when a vendor who wrote in their RFP that their solution is “powered by AI” is just using a marketing smoke-screen to charge you outrageous margins.

This judgement is worth much more than any tool you will ship in your career (unless you build a tool that you market successfully, of course).

You could of course get a general understanding on AI by watching YouTube videos or subscribing to an AI Substack, but there is so much misinformation out there, so many self-proclaimed AI experts, and that knowledge is so accessible that it will likely soon only be a floor for any employee in any organization.

In my experience, you will learn much more by building things yourself, on your own stack, using all available technologies, take the time to see them in action and how they work under the hood (familiarize yourself with Skills, API calls, MCPs,  memory, tool use, workspace management, databases, context, models, local models, Claude Code, just as an appetizer).

I would also recommend joining a community of people who are better than you at this and who will challenge you ruthlessly at every step (if you are a lawyer, LegalQuants is the place to be).

You don’t need to know how to code for any of that. You just need to understand some principles of coding which you can learn by using a good AI as your sparring partner and coach during your exploration. Most importantly, you need to understand when you have to pause, say “I don’t know”, and check with a competent consultant (which you will hopefully have identified in advance).

"But where do I start?"

I encourage you to go through Anthropic's online courses. They’re free on their website (here). Most "AI Experts" sell some sort of repackaged version of these courses, and charge you for it. Although these courses mostly cover Claude, the knowledge you will get there will already give you a very deep understanding of how AI systems work. If your organization has become a partner with Anthropic or any other serious organization that certifies architects, study hard and pass that certification. It will give you the formal recognition that you need in some environments that value diplomas and certificates. Do it even if you don't need a piece of paper recognizing your competencies, as this type of exam will help you with benchmarking yourself against the current established knowledge in this area.

Last but not least: I would recommend that you learn how to vulgarise and explain Al in simple terms for peers who are less advanced and teach them, as you will need allies and sparring partners who will build their own systems that integrate with yours for maximum efficiency. I have learned a lot by making the mistake of explaining AI in too much detail, and losing the audience. It is crucial in my view to learn how to keep things simple to be of value to your peers, and help them with getting up to speed.

Part Three: The /before-ai command

If you read up to this point... and if what I am writing resonates with you, you will like what follows.

I have compressed the thoughts of this article into a Claude Code command for you. You will find that command here: https://github.com/SaifAlYounan/before-ai

Just open Claude Code in your terminal (type "claude" in your terminal), then point Claude Code at the GitHub repository, instructing Claude to install it (typing "install this command: https://github.com/SaifAlYounan/before-ai", for example, should do the trick).

Then, exit Claude Code (Control+C x 2), and launch Claude Code again (type "claude" in the terminal).

Then, run the command (by writing "/before-ai" in the terminal).

The command will guide you with questions... the old Socratic method. You just sit back, relax and follow the flow.

If you are a Senior Leader and do not yet have Claude Code (or some other AI agent), do not know how to use Claude Code in a terminal and have no idea what a Claude Code command is, it is not too late. I encourage you to go and take a look, now.

Open your Claude Max subscription and… start asking about Claude Code, how it works, how to install it and how to use it. Then, just make do. Good luck.