AI Governance

India’s central bank is drawing a strict line around artificial intelligence in banking. The Reserve Bank of India has proposed rules that would force banks to keep a kill switch ready, allowing AI systems to be shut down immediately if they produce harmful or incorrect output.The draft also makes human oversight non-negotiable. Any AI-driven decision inside a bank would still need a person above it, with the power to override, suspend, or disable the system whenever necessary.Why the RBI wants tighter controlThe proposal comes as AI use expands across banks and other financial institutions. In a sector built on sensitive data and critical decision-making, even a small model failure can affect operations, compliance, reputation, and overall system stability.RBI said the draft guidance was issued for banks and other regulated entities on Wednesday. If adopted, the rules would require firms to be able to override model decisions and switch AI off instantly, including through a kill switch mechanism.Human review stays at the centerThe draft applies to every AI-related decision, not only to generative systems or advanced frontier models. RBI wants the same human control across the board, from simple spreadsheet-based calculators to complex AI tools used in business processes.That broad scope is designed to close a common blind spot. Regulators are concerned that institutions may focus on the newest models while ignoring older systems that still shape important outcomes.RBI also warned about automation bias, where staff may trust AI output too easily and stop applying their own judgment. In the bank’s view, human presence is not symbolic; it is a core control meant to prevent decisions from being taken at face value.Accountability remains with the bankThe central bank is also making it clear that responsibility cannot be outsourced. Banks will remain fully accountable for the results of any model they use, whether it is built internally, bought from a vendor, or created through a hybrid approach.Before deployment, RBI expects banks to conduct proper due diligence on the model. That warning matters because third-party tools are often seen as shifting risk toward vendors, while the proposal keeps the liability with the regulated entity.According to RBI, the growing use of artificial intelligence and machine learning in business and decision-making processes can create financial, operational, compliance, and reputational risks if governance and controls are weak. Poor management can lead to inaccurate outputs, flawed decisions, financial losses, operational disruption, compliance failures, and wider harm to institutions, customers, and the financial system.Risk levels will be mapped and reviewedThe draft also introduces a risk-based framework for model oversight. Banks would need to classify each model by risk level and apply monitoring, validation, and controls that match that level of exposure.If a model’s risk rises above a bank’s risk appetite, the draft calls for rapid action. That can include stronger controls, restricted use, remediation, suspension, removal of the model, and reporting to the board’s Risk Management Committee.Risk cannot be left unchanged for long periods. RBI wants risk levels reviewed at least once a year, and high-risk models would need approval from the Risk Management Committee of the Board before they are put into use.AI governance moves to board levelOne of the most significant shifts in the proposal is the move to board-level oversight. Every regulated entity would need a Model Risk Management Framework approved by the board and covering all models, regardless of who built them.That framework must include internally developed models, vendor-provided models, and hybrid systems. The message is clear: RBI sees model risk as a governance issue, not just a technical one handled by the IT team.The draft also highlights supply-chain risk caused by overdependence on a small number of AI model providers. Banks are expected to manage that concentration risk actively and avoid creating fresh vulnerabilities through AI adoption.Customer-facing systems get extra rulesFor systems that interact directly with customers, banks would have to inform users when they are dealing with AI. Customers must also be given the option to switch to a human at any time.Generative AI systems that face customers or external users would face additional cybersecurity controls. The concern comes amid rising attention to AI security risks, including renewed scrutiny after Anthropic’s Claude Mythos model drew attention in financial circles.RBI is accepting feedback on the draft until 24 July. If approved, the rules could become one of the toughest frameworks yet for ensuring that AI in banking can be shut down instantly, kept under human control, and governed directly from the boardroom. Source: www.indiatoday.in

Banks Face a Hard AI Reset, Regulators Demand a Kill Switch and Human Oversight