AcityprocurementleadIspokewithonarecentprojectputitbluntly:shedidn'tneedanother vendor telling her what the AI could automate. She needed to know what happens to a resident's case file when the system gets it wrong.

That'sthequestionthatdecidesthesedeployments,andit'srarelytheonevendorscome prepared to answer.

I work on go-to-market for a frontier AI agent platform built for regulated, exception-heavy backofficework,thekindwhereeverytransactioncarriescomplianceexposureandamistake doesn't surface until weeks later, usually as a complaint, an appeal, or a number that doesn't reconcile. The sales pitch is almost always about capability: automate the workflow, navigate the system of record, eliminate manual work. None of that is false. It's also not what gets a public sector buyer to sign.

Public sector buyers, and the operations leaders who serve them, aren't evaluating performance. They're evaluating accountability. Can the system recognize what it doesn't know. Is there a complete, reviewable record. Who is responsible when the model is wrong, andhowfastdoesaresident,apatient,orasmallbusinessfindout.Thesearen'tcompliance checkboxes. They're the conditions under which a public institution can use AI without breaking the public's trust in it.

That distinction matters more in this sector than almost any other. A private company that deploys a flawed automation absorbs the cost internally, maybe a write-off, maybe an awkwardclientcall.Acitythatdeploysoneabsorbsitasaresidentwhogotdeniedapermit,a benefit, or a service, with no clear way to find out why or appeal it. I've sat through a demo where the vendor's answer to “what happens on an edge case” was a shrug and a promise to “tune the model.” That's an answer for a company optimizing a marketing funnel. It is not an answer for a system that decides whether someone's housing application gets reviewed this month or next.

The stakes here aren't just operational. They're about who gets to participate fairly in the systemsacityruns,andthat'sadifferentburdenthantheonemostautomationvendorswere built to carry.

I'll say the part that doesn't make for a clean vendor pitch: most automation providers aren't ready for this conversation, because their answer to “what happens when you're wrong” is thinnerthantheirdemosuggests.Aplatformthatcan'tdescribe,specifically,howitsurfaces andescalatesthecasesitcan'thandlehasn'tsolvedthepublicsectorproblem.It'ssolvedthe enterprise efficiency problem and is hoping nobody asks the harder question.

This is also why the conversation about agentic AI in cities can't stay contained to technology teams.Privacyofficers,equityandaccessadvocates,andfrontlinestaffaretheoneswhosee where automated decisions land on real people. A frontline caseworker knows, in a way a vendor's roadmap never will, which residents are most likely to fall through the cracks of an automated process: the ones without reliable internet access, the ones who don't speak the dominant language fluently, the ones who never get a callback because the system marked their case resolved. The deployments that hold up over time are the ones where those voices were in the room before the contract was signed, not brought in afterward to manage a

If you're evaluating a vendor for this kind of work, here's a concrete test: ask them to walk through their last real exception, the case itself, not a hypothetical, and tell you who got notified,howlongittook,andwhatchangedafterward.Iftheycan'tproduceone,that'sthe

I don't think there's a tidy resolution to this, and I'd be cautious of anyone who offers one. Public trust in AI gets built case by case, appeal by appeal, not by a vendor's aggregate accuracy number. The providers worth taking seriously are the ones willing to walk through theirfailurecasesindetail,unprompted,becausethat'stheconversationacityneedstohave before anyone touches a resident's file.