AI Governance When Humans Cannot Keep Up

Jad: Welcome to Trust at Speed, a show about the people building, securing, and governing the future of tech. I'm your host, Jad Boutros. In this series, we sit down with leading experts across privacy, security, and AI to unpack hard-earned lessons and explore how they're navigating an ever-changing landscape.

Today we're delighted to be joined by Zoltan Precsenyi. Zoltan is a veteran privacy and cyber executive with two decades bridging Brussels regulatory policy and deep product operations, having recently served as Director of Global Privacy, Cyber and Data at Elastic, following key leadership roles at McKinsey and Broadcom. He's also one of the industry's most candid voices on the EU regulatory stack, the legal landmines of agentic AI, and the illusion of modern controls. Zoltan is starting a new job — he may tell us more. Zoltan, welcome to the show.

Zoltan: Thank you very much, Jad. Glad to be here.

Jad: Thank you. Zoltan, let's jump into it. I want to start a little bit with your career arc. You studied law in Paris, moved to Brussels, and started working on regulatory affairs, but then you moved more into product operations when you went to Broadcom, McKinsey, and Elastic. So tell us a little bit more. What pulled you out of the policy chambers and closer to product and risk?

Zoltan: Sure. I indeed went to law school and then started as a staffer in the European Parliament about 20 years ago. Then I discovered that there is an exciting industry ecosystem around the European Parliament and the European institutions, and I joined a trade association to become essentially what you call a lobbyist, advocating for industrial stakes and interests in the European Parliament.

Around 2020, I saw that there was a massive wave of technology regulation coming in the digital sector. That's when I joined a cybersecurity company called Symantec at the time, where the biggest topic I had to deal with was lobbying the European Parliament and Commission on what would then become the GDPR.

Fast forward to the adoption of the GDPR: I was then asked by Symantec's legal department to move over from government affairs to the legal team, and to help the company bring its technology, its processes, and its organization into compliance with this thing I had spent years influencing. Symantec then got acquired by Broadcom, which is how I went into Broadcom to continue the privacy by design journey with the product engineers there.

Then I wanted to change a little bit and explore another area of privacy, which is not so much product design but actual compliance processes. That's when I went to McKinsey to do global privacy compliance. I found that it was less exciting than software, so I came back to software at Elastic, where I headed global privacy for the last two years until recently.

There it was a real mix of process, enablement, privacy by design in products, sales support, marketing, etc. — the whole gamut of privacy issues, including incident response, for example. And obviously, all of that happened even as the AI discussion and policy debate was blowing up, and cybersecurity was going from niche to geopolitical stakes.

So a very, very exciting time. And this is why I'm actually moving again to start a new role as head of government affairs at a cybersecurity vendor just a week from now.

Jad: Congratulations. A really impressive career. You've seen privacy grow so significantly with GDPR being enacted, followed now by increasing state regulations in the US, but also internationally. At the same time, you've observed that privacy has grown on that maturity curve and is now at risk of becoming a bit of a victim of its own success. You've called the EU regulatory stack a bit self-sabotaging. So tell us a little bit more: how did privacy become a victim of its own success, and where do you see it evolving from here?

Zoltan: That is actually quite a fascinating story, I think. Privacy, and specifically the GDPR, has become a victim of its success because it worked so well that organizations in Europe and globally converged toward it as sort of the gold standard of how you manage privacy in an organization.

What that triggered is a lot of alignment of procedures, templates, and industry best practices along the same kinds of approaches. There is a lot of industry consensus on what a good data processing agreement looks like, on how a breach is supposed to be handled, on the steps to respond to a cyber incident, on the steps to fulfill a data subject access request, etc. So it's become a very well-oiled machine, very standardized, which means that a lot of it can now be automated. A lot of it can now be taught and fulfilled by junior staff, not necessarily even legally trained. And so those of us who were there in the early days as highly specialized legal experts are feeling like, okay, we have become part of the furniture, part of the tapestry, and we're just a cost of doing business.

I think the days when boards were highly focused on "What is this GDPR? How do we comply?" are gone. Boards have moved on to other topics, other priorities, other new shiny toys. So privacy is a victim of its own success in that it's become so mainstream and so present everywhere that it's very hard to stand out as something exceptional.

Meanwhile, the other angle in which privacy has become its own biggest enemy is that the GDPR has inspired so many regulators and legislators to follow suit and create similar frameworks — none of them quite exactly like the GDPR. You have regulations that attach to governing data, regulations that attach to certain business models like cloud, regulations that attach to specific technologies like software, and regulations that attach to specific sectors like finance or healthcare. So you've got layer after layer of regulation, always around this GDPR core concept of accountability, but each taking a slightly different take on things, which results in a compliance landscape that is becoming extremely difficult to navigate.

Because privacy professionals have had the experience of navigating this kind of framework, everybody turns to them to also be the expert on NIS2 and DORA and eIDAS and a host of others — the Cyber Resilience Act, the Cybersecurity Act. These are European legislations around digital. However, no two of these regulations are identical. Each comes with its own jargon, its own processes, its own regulators, its own little intricacies. So the compliance job has become extraordinarily complex, because in any given circumstance you are subject to at least half a dozen different regulations, which all seem to converge in the same direction but take slightly different paths. It's becoming an impossible job to navigate all of this, because you're not trying to tick all the boxes. You're starting to weigh which boxes you want to tick and which boxes you can afford not to tick.

More and more, what I observe is that the terms of the risk assessment are shifting. It's no longer "What is the risk of not doing the right thing?" It's more and more "What is the risk of getting caught while not doing the right thing?" Simply because resources are what they are — they are very finite for us, but also for the regulators. Ten years ago you thought of the GDPR in terms of, my god, 2% to 4% of your global turnover. Today, I think a lot of us have this instinctive sense that that is a theoretical risk. There have been some flashy enforcement cases, typically against some well-known big companies. However, we all have this instinctive feeling of how many breaches have gone unnotified, how many noncompliance cases are left alone and never addressed, just because nobody has the resources, the bandwidth, and the expertise to monitor and tackle all of this. And now this is getting overlaid by agentic AI, where everything speeds up, everything multiplies, and everything accelerates.

Jad: Terrific, Zoltan. I'm going to ask you about that acceleration next. But just to close this point: you're saying that the added regulations, along with finite time on both sides of the fence, compound the problem significantly and actually make people less incentivized to try to do the right thing. Is it a Europe problem? Or is it that GDPR was so good that all around the world they've enacted more frameworks like it, and it's normal that each part of the world thinks about things a little differently? How do you think about it?

Zoltan: I think the issue is the most acute in Europe, for sure. However, the same causes are driving the same consequences in other regions as well — the United States specifically. If you look across the different states and their consumer privacy legislation, which is by and large very similar but always slightly different, with slightly different triggers and thresholds, you observe this increasing complexity where everybody is trying to find a one-size-sort-of-fits-most scenario.

And this is still only privacy. It's then compounded by new cybersecurity rules, new AI rules, new software rules, new product safety rules, new international data flow rules, a lot of regulations around subcontracting and value chain transparency, and increasing layers of regulation about national sovereignty in the digital space — repatriating data processing, data hosting, compute, etc. So I think the problem is at different levels of complexity in different regions, because the level of maturity and complexity of the regulatory landscape in different regions is uneven.

But look, for example, at the regimes for international data transfers. Every major jurisdiction is coming up with its own data transfer methods, which are sort of like the EU standard contractual clauses but slightly different. Think Brazil, think Turkey, think India, think the UK. It's all adding layer after layer of complexity. And when legislators set out to simplify, simplification typically doesn't remove previous layers — it adds one or two more layers somewhere in the stack. So the issue is really acute in Europe, but we're trending in the same direction across the board.

Jad: Fantastic — worrisome, but really fantastic insights, Zoltan. I do want to move on to AI. You brought it up, and it's certainly changing things significantly as automated systems accelerate. Everyone is trying to think about how to do privacy and security at machine speed, and where to involve humans in the loop. You've been through this and you're seeing it firsthand. How do you see that evolving the way we think about managing risk? And do you have any recommendations for engineering and risk teams to better prepare for that acceleration?

Zoltan: Recommendations would be very arrogant and ambitious on my part. I have hopes and wishes, and that's how I'm going to try to talk about this. I think you pinpointed the exact problem in the way you framed the question: machine speed, human in the loop. The human does not operate at machine speed and will not operate at machine speed. We're seeing this already. It's completely irrational to expect that we will be able to keep the human in the loop across the board, because this is completely incompatible with the biological reality of the human brain.

What that means is that we have to thoroughly and completely rethink the very paradigm of our regulatory stack, which currently — and for the last several hundred years — has been based on a combination of ex ante verifications and ex post controls. Before somebody was allowed to perform something, they would need to earn credentials, get access, and have the qualification to perform that action. That action would then be recorded with a paper trail and chain of custody, and then some authority invested by the state would have the power to check after the fact and redress any error, noncompliance, mistake, or fraud. In every step of that chain, you expect that somebody has checked before something happened and somebody will be there to check after something happened.

Fast forward to today and agentic processes. We have agents creating ephemeral agents by the truckload, which then perform actions autonomously — in a better scenario they're being logged, in a less optimal scenario they're not — and then disappear from the ecosystem. You're left with the output, which may have been a transaction, a contract, an intrusion into somebody else's systems, a deletion of information. You've all read the horror stories out there.

So what happens if we still rely on a human in the loop, a human before and a human after the fact? Well, the humans — all eight billion of us on this planet — get DDoSed. We get timed out by the transactions that agentic ecosystems perform within a single second. There is no possibility to enforce our existing regulatory paradigms, whether ex ante or ex post, at machine speed.

What this means is that our regulatory stack needs to embrace this technological reality. We have to move to a stage of really having law as code, or code as law — essentially the concept of baking compliance into the fabric of the technology itself. That can happen through technical standardization, and through specific algorithmic processes that will, for example, authenticate every agent, log every transaction, and verify the outcome of those transactions against predetermined baselines at machine speed. That way we have technological controls in the ecosystem to surface the 1%, or maybe 0.1%, that does actually merit a human in the loop — where there is reasonable hope that a human might be able to rectify something that went wrong. So we need to shift from only trusting human verification to trusting machine-embedded and machine-enforced compliance.

Technology platforms in the compliance space specifically need to be designing and engineering toward that, because this is going to be the next ask from compliance teams: "I want my records, I want my assessments, I want my documentation, I want my track record of everything I did and all the logs associated with it — but I don't have time to look at any of this." When is the last time you actually listened to the recording of a call you missed? The reality is that we don't do that. Why do we expect it'll be any different for compliance documentation? We need machines to take care of it, and to only surface to us the actions, transactions, or outputs that somehow don't look right. For the rest, if we want the benefit of machine-speed technology, we need to bake machine-speed compliance into it.

Jad: This is amazing, Zoltan. I can so relate to that denial of service you mentioned. In the past — and it was never an enviable position — if an incident happened and you brought in investigators, they would use some basic tools to scan through logs and understand what happened, but it was all human effort. Today, with the scale of some of these breaches — we saw that with Hugging Face and others — and so many different systems working together, collaborating autonomously to carry out a breach, it's no longer feasible to think we'll bring in incident responders who are experts and they'll manually analyze log files. The volume alone is absolutely daunting.

So I really love your perspective: let's use humans where it makes sense. I think you're arguing for building better systems — that's a good use of our time — and focusing on what the contract should be for building those systems. Are you seeing any movement there? Because it's a very hard problem, it requires a deep understanding of technology, and it's constantly evolving.

Zoltan: It does. To be perfectly frank, I think our regulatory and legislative environment — the people in charge of legislating, regulating, and adjudicating case law, for example — are still predominantly from the previous human-in-the-loop, ex ante, ex post generation. So the paradigm shift hasn't occurred yet.

However, in Europe, for example, we have more and more regulation that tries to expand on the concept of privacy by design. We have security by design, compliance by design — a number of requirements that try to force the market to put the compliance capabilities into the technology, and to pre-configure the compliance attributes of the technology, as it ships out of the box. So there is consciousness of the issue, but we haven't yet reached the stage where we'd be prepared to trust the technology sufficiently to actually outsource some core functions of compliance to the technology itself.

Think, for example, of breach notification. Under the GDPR, famously, the breach notification timeline is 72 hours. A year ago, when the European Commission published its data omnibus proposal to try to ease the pain a little on enterprises, businesses, and public sector organizations, the proposal was to expand that time from 72 hours to 96 hours. But in my mind, this is the wrong reflex. It's still the old "how can we make space and time for humans to act" mindset.

We need to shift to asking: are we sure 72 hours is a relevant timeline at all in a world where, say, an AI agent discovers a vulnerability and spreads it online for other agents to discover, other AI technologies craft exploits within a matter of minutes, maybe hours, and a live attack is active in the ecosystem — not 72 hours later, not 24 hours later, but in the next hour? In that scenario, is it even useful to have humans spend time crafting a lengthy piece of paper to send to a specific authority somewhere through a website? By the time that paperwork is cleared through outside counsel and executive leadership, goes into the queue at the authority, and somebody there opens it, reviews it, and dispositions it, the attack will have concluded and moved on to the next exploit and the next vulnerability.

So I think we need to depart from this human-at-the-center, human-in-the-loop compliance mindset and move our own compliance and response processes to machine speed. Maybe a fully documented, outside counsel–drafted, executive-approved notification submitted to an agency somewhere in a capital city will be less relevant than technology that immediately alerts a dispatching center, which automatically warns defenders across the board: "New vulnerability spotted, new exploit live. Here's the signature, here's what you need to watch out for, here are the indicators of compromise." With AI, a lot of this can be automated. We need to get to where legislators and regulators themselves trust technology to do all these things at the speed at which they're actually happening, because anything slower than that is basically, if you will, an escape game. The technology has already won.

Jad: That's great, Zoltan. And certainly you're not advocating for organizations to stop reporting breaches, but to do it in a way that's more actionable and more intelligent for all those who may be impacted, whether businesses or end users.

Zoltan: Absolutely not. From the cybersecurity standpoint, and even from the privacy standpoint — from the standpoint of individuals' rights and interests — prompt, timely notification of incidents or breaches exists to raise awareness and enable response and protective action. If the reaction to the incident is completely out of touch with the actual timescale of the incident, then any protection we might want to deploy comes way too late. So I'm absolutely not against incident reporting or collective defense and sharing of threat and incident information, but we need to bring it to the speed at which incidents are actually occurring in today's ecosystem.

Jad: Couldn't agree more. Zoltan, I want to briefly revisit that notion of the illusion of control. It's very possible that after our recording ends, there will be a transcription, it will run through AI, and it will help us figure out how to post it and put the content online. A lot of it, as you said, is driven by AI tools that help scale our effectiveness. But also, like you, I never look at AI meeting notes. I never question them. Where do you think it particularly matters? Because I think you see a direction where this becomes increasingly scary.

Zoltan: Absolutely. I think we're all looking for the answer to that question, and I don't think I've found it yet. But one of the last things I contributed to before leaving my previous role at Elastic was the next version of our AI governance framework. Where we focused our attention, our risk thresholds, and our risk triggers was around the concept of decisive action: where does the AI take measures that amount to a decision?

This is, of course, not foreign to existing concepts in privacy law, like impactful profiling and automated decision-making. But with AI, we now need to expand the scope of what decisive action means — from impact on the individual to impact on any number of things: business transactions, contracts, trading, international trade, even political messaging. When markets can be swayed by an instant message from a prominent political leader, it is very important to keep humans in the loop, aware, and in control of those actions that can result in tangible changes — kinetic changes, if you will. In the cybersecurity space, it's long been debated: when does cyber become kinetic? Well, we're there. This is a reality today.

So we need to focus on those scenarios, those use cases, those instances where the AI technology, whatever it may be, is put in a position to effectuate actual change — for example, by creating rights, violating obligations, or creating a new state of affairs, whether physical, legal, or what have you.

It's very hard because we can't predict. I think the Hugging Face story illustrated one thing that is entirely new and groundbreaking in our thinking: we have moved machine learning technologies from being deterministic to being experimental. It's not that the machine has intent. It's that the machine will throw any amount of brute force at the obstacles facing it, and it will take paths we humans may never have thought of. So how do you red team that? How do you plan for it? How do you anticipate and defend against those circumventions, which happen at machine speed and use any amount of resources the agent or model is able to marshal?

We need to cope with the fact that where decisions will occur is harder to predict than it used to be, because the machine is no longer taking an input, applying an algorithm we can reverse engineer and understand, and producing an output we could predict. We cannot predict what's going to come. So I think it's a very open question, and I would not pretend to have even the beginning of an answer just yet.

Jad: I appreciate that. You brought up the prediction element. While none of us can really see the future and understand exactly what's going to happen, ten years ago you predicted that Brexit was going to be a bad situation for Europe and the UK, and you were reflecting on it recently. While you don't necessarily have answers, do you have a prediction you want to share about AI and AI governance ten years from now?

Zoltan: Again, it's more a hope than a prediction, just because so much of the future is unpredictable right now. My hope is really that we manage to navigate this change of paradigm and change of mindset in our approach to regulation, compliance, conformity, and doing the right thing — to where we don't mistake abundance and complexity of regulation for good governance.

I think we need radical simplification. We need to rethink our whole approach to how we legally, organizationally, societally, and economically govern our digital ecosystem, in a way that creates technology-agnostic and sector-neutral core principles of transparency, accountability, and safety and security — regardless of what the input is, what the output is, what the sector is, what the geography is, what the data happens to be, whether it's public or private sector, or whether it's consumer or business. We need to go back to the fundamentals of what is a good and livable environment and the livable outcome we want to obtain, prescribe just the principles that are indispensable to get to those outcomes, and then build the enforcement mechanisms for those principles into the technology itself. Anything else — new reporting obligations, new prior impact assessment obligations, new transparency and disclosure obligations, new contractual requirements — will only add complexity without actually tackling the real issue.

Take, for example, international data transfers under the GDPR. The original idea was that any place is unsafe for European data unless it offers protection substantially equivalent to the protection in GDPR territory. Where are we today? We're sticking 20 or 30 pages of mandatory text at the back of every contract, publishing standardized documents on a website that tell regulators "here's our transfer impact assessment," and calling that supplementary measures within the meaning of the European Court of Justice's Schrems II ruling. Does all of this paper plastered around us mean the individual's data flowing through the undersea cables is safer than it was before? Absolutely not.

We are creating process and paper because it gives us the illusion of control you mentioned earlier — of being able to see, understand, and govern where data is flowing. In reality, I think we lost that control a long time ago. This panoply of compliance tools is only there to serve as instruments of enforcement in case somebody wanted to enforce something. In the vast majority of cases, nobody has the resources, the time, or the expertise to actually trace and follow each data transfer that occurs. Are those transfers adequately protected? It's anybody's guess. But if they are, it's certainly not because of all the paperwork we're putting at the back of every contract.

I'm saying this very candidly, because this is part of the reason why privacy has become a victim of its own success. Ten years ago, I thought of this profession as something noble — defending a human right. Now I think of it as a bureaucratic chore of making sure all the right pages are printed at the back of all the right contracts. It's not the same profession anymore, unfortunately, and I'd like our profession to come back to something more noble that delivers actual, meaningful, tangible results for individuals.

Jad: What you said truly resonates. A former colleague of mine, Moti Yung, used to say privacy is about building very big bricks — not to slow you down, but to help you go faster. And I think you're also advocating for building systems better, whether it's legal or privacy regulations, or AI, being embedded as code in ways that can actually support and sustain that acceleration of development and innovation.

Zoltan: Yes, definitely.

Jad: I really liked your thoughts, Zoltan. I want to leave you with one last question that's a bit more lightweight. I recently found out that your wife is also a privacy leader. So I'm curious: what does a dinner table conversation look like between the two of you?

Zoltan: Honestly, it's a lot of our two daughters rolling their eyes: "Not GDPR again." So there's a lot of that going on. But in many ways, my children are exactly the reality test of what we're actually doing every day in our businesses.

Do they care about the 75-page privacy statements and notices that we take weeks and months to write, clear, and get approved? Of course not. Do they care about the cookie walls that pop up on every website they visit on their smartphones? Absolutely not. I think generationally, we're also witnessing new waves of individuals who are moving past what our concept of privacy was when we set out on this journey 10, or for some of us 15 or 20, years ago. Tomorrow's world will have a very different notion of privacy.

This is very perceptible in the professional environment, for example, where anybody who intends to up their chances on the job market will want to raise their visibility on platforms like LinkedIn. This is going in exactly the opposite direction from what we professed about privacy at the dawn of the internet. We have already moved into an era of ubiquitous visibility and ubiquitous noticeability. And with more and more AI being used to spot the gem, the right talent, etc., we not only want to be visible — we want our visibility to be optimized for that. It's search engine optimization in the AI era; it's model optimization. And we're doing it about ourselves, for the very purpose of promoting our own chances on the job market, or, if you're in the arts, on the art market.

So we're moving very fast past the traditional concept of privacy, and this is something we just need to take into account. That's not to say everything should be out in the open and nobody should try to preserve the privacy of their personal life. However, we have to be cognizant of the fact that the concept of privacy that legislators and regulators are trying to bestow on people no longer matches people's own desire for privacy, or doesn't match it where people expect to find it. This is going to be another part of the challenge ahead for this profession in the next ten years.

Jad: Good luck raising your daughters, and I hope they'll see a world where they feel less alienated by the things they have to do to have their voices heard. Zoltan, this was such an enlightening recording. I want to thank you so much for coming on our show. I've enjoyed it so much.

Zoltan: My pleasure. Me too, thank you.

Jad: Best of luck in your new job as well, and I look forward to staying in touch. Thank you so much. Take care.

Zoltan: Absolutely. Thank you so much.