Barclays built a system to spot AI agents that might go rogue
Skip to main content
Research setup ‘reads the mind’ of LLM-based agents and flags if they’re about to do something risky
Quants at Barclays have built a prototype monitoring system to read the minds of AI agents and catch them before they do anything too risky.
Ariye Shater and Hariom Tatsat borrowed from research into so-called mechanistic interpretability to address what Shater calls “tool risk” in agentic systems – the risk that AI agents call software tools in ways that are wasteful, risky or damaging, such as
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe
You are currently unable to print this content. Please contact info@risk.net to find out more.
You are currently unable to copy this content. Please contact info@risk.net to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net
Op risk data: EFG takes a bath for two decades of money laundering
Also: Deepfake voice clone cons Intesa division; key couple of cryptocurrency hacks. Data by ORX News
Managing risk to accelerate agentic AI value creation
Banks can accelerate agentic AI adoption by aligning each use case’s risks, controls and capabilities, enabling safe, accountable and value-driven deployment
One legal entity. Multiple IDs. Three business risks.
How fragmented entity IDs undermine risk, compliance and revenue decisions
With no clear definition or way to gauge ROI, claims of AI adoption should be taken with a pinch of salt
BIS’s Hernández de Cos on AI, non-bank risk and tokenisation
General manager talks about how the BIS is navigating a complex global environment
For enterprise risk, influence runs deeper than any veto
Risk Benchmarking study finds formal veto rights concentrated among the largest lenders, but used only as a last resort
Intraday pricing: fixed income’s next frontier
With firms looking to price fixed income securities on an intraday basis, specialist pricing providers such as LSEG help address cost, complexity and time-to-market challenges
Tech and data fragmentation irks enterprise risk managers
Risk Benchmarking: Often at the mercy of decisions made by other parts of the bank, ERM heads gripe at disparate systems, poor UX and reporting gaps
Most read articles loading…
Back to Top
You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.
