Responsible transformation: Agentic AI for the public sector

The integration of artificial intelligence (AI), particularly agentic AI, is rapidly transforming both the private and public sectors. This evolution presents unique challenges for government agencies, law enforcement, and other mission-critical organizations. While agentic AI offers significant benefits such as modernized IT workflows, expedited analysis, enhanced citizen services, and improved operational efficiency, the complexities of regulatory compliance, data security, and governance often hinder its adoption.

The Promise of Agentic AI in the Public Sector

Agentic AI is designed to operate autonomously, making decisions without the need for continuous human input. This capability is particularly valuable in the public sector, where timely and informed decision-making is crucial. By enhancing visibility and transparency, agentic AI can contribute to consistent governance, which is essential for maintaining public trust.

A recent webinar titled "Putting the responsible back into RAG and agentic AI" explored the potential of deploying AI agents safely in the public sector. The consensus is that responsible deployment is feasible, provided that governance and accountability are integral to their operation. Data security remains the most significant barrier to AI adoption in this sector, given the sensitive nature of public data and the potential national security implications of a breach. Despite these concerns, there is a growing anticipation of the benefits that AI can unlock for public organizations.

When implemented responsibly, agentic AI can streamline traditional IT workflows and enhance internal processes, leading to greater efficiency. For instance, AI agents can significantly improve citizen services, strengthen transparency, and help bridge the trust gap between public institutions and the citizens they serve. Additionally, these technologies can enhance the daily experiences of civil servants, allowing them to focus on more impactful work rather than administrative tasks.

Case Study: The Dutch Defense Organization

A notable example of effective agentic AI implementation can be found in the DATA department of the Materiel and IT Command (COMMIT) in the Netherlands. This organization developed an in-house, air-gapped large language model (LLM) that operates entirely on a closed network. By isolating the system from the internet, they ensured that sensitive information remained secure, demonstrating that safe deployment of agentic AI is achievable with the right precautions.

The key takeaway from this case is that successful agentic AI deployment in the public sector necessitates rigorous research, robust security architecture, clearly defined success metrics, and a compelling business case to guide implementation. For many organizations, the complexity and resource intensity of AI adoption may seem daunting, especially without clear expectations for outcomes. Sustainable adoption should be driven by purpose rather than external pressures or trends.

Navigating Challenges and Building Resilience

Integrating AI-driven search capabilities into public-facing websites, for example, can significantly enhance accessibility. This allows citizens to find information in straightforward language, reducing the need for them to navigate complex institutional structures. Such improvements not only enhance user experience but also alleviate pressure on frontline staff by decreasing call volumes and repetitive inquiries.

However, the public sector cannot ignore the risks associated with unrestricted access to AI tools. Without proper oversight, organizations expose themselves to privacy violations, compliance failures, and potential data leaks. The absence of a structured approach can lead to the emergence of shadow AI agents, where employees resort to unsanctioned tools, further increasing the risk of data exposure without visibility or governance.

To mitigate these risks, organizations must make strategic decisions about their position on the AI access spectrum, taking into account their specific risk profiles and tolerance levels. Additionally, adopting open standards can enhance long-term resilience and prevent vendor lock-in, ensuring that organizations are not overly dependent on external providers.

Building an internal hub of AI knowledge and expertise is essential for organizations to critically evaluate, implement, and optimize AI tools responsibly. Investing in user education and fostering internal expertise can help reduce reliance on external vendors and create long-term resilience.

Implementing AI agents in the public sector involves addressing key challenges, including data security, controlled access to information, and the need for relevance and accuracy. This is where retrieval augmented generation (RAG) becomes relevant. RAG helps ground AI responses in an organization's verified knowledge base, allowing models to generate answers based solely on approved internal data.

However, deploying RAG effectively requires access to data, which is often fragmented within public sector organizations. A data mesh approach can help connect these disparate datasets, creating a unified knowledge layer for AI applications. This strategy can lead to secure, intelligent search capabilities and advanced use cases without compromising data sovereignty.

In conclusion, while agentic AI holds great promise for the public sector, its successful implementation requires a thoughtful and strategic approach. Organizations must build a robust tooling ecosystem, ensuring that AI agents can effectively integrate with existing systems. Gradual integration and regular assessments against success criteria are crucial to maintaining control over AI deployment and maximizing its benefits.