Avoid expensive AI agents with these five design imperatives

In a recent address, Jeff Clarke, the vice-chairman and chief operating officer of Dell Technologies, highlighted the critical need for organizations to rethink their data infrastructure and enterprise software stacks to avoid the pitfalls of deploying artificial intelligence (AI) agents. He warned that without fundamental changes, businesses risk introducing "silent and expensive" AI agents that could undermine their operational efficiency and financial health.

The Challenge of Data Silos

Clarke's analysis begins with the widespread issue of data silos within enterprises. He pointed out that while AI technology thrives on data, many organizations are ill-equipped to leverage their data effectively for agentic applications. "In most enterprises, the data is scattered across dozens of systems," he explained, noting that a staggering 80 to 90 percent of this data is unstructured and disconnected. This fragmentation poses a significant barrier to scaling AI agents, which require a coherent and accessible data environment to function optimally.

To address this challenge, Clarke proposed a paradigm shift in how organizations manage their data. Rather than transferring large datasets to external cloud-based AI models, he suggested that companies should bring AI models closer to their existing data repositories. "To power agents at scale, you need a real-time connected knowledge line," he stated, emphasizing the need for a structural and architectural decision to "move AI to the data" instead of the other way around. This approach not only enhances efficiency but also aligns with the growing demands of AI technologies.

Evolving AI Workloads

Clarke further elaborated on the evolving nature of AI workloads, particularly in the context of inference. Unlike traditional AI systems, which primarily focus on model training, agentic AI systems engage in complex tasks that require multi-step reasoning and the integration of various models. He noted that the computational demands for these tasks have surged dramatically, with workloads that can be 10 to 100 times, or even 1,000 times, more intensive than those seen just a few months ago.

As AI agents transition from mere information retrieval to executing business processes—such as updating customer records or processing transactions—Clarke stressed the importance of security. He advocated for a redesign of enterprise systems to ensure that every action taken by an AI agent is logged, tracked, and can be reversed if necessary. "When an agent acts on your behalf, you need to know what it did, why it did it, and how to undo it if it got it wrong," he explained. This level of accountability is crucial for maintaining trust in AI systems that operate autonomously.

Integration and Tokenomics

Another critical point Clarke raised was the necessity for a unified software stack through application programming interface (API) orchestration. He warned that without this integration, AI agents could become isolated and costly failures. "The agent becomes the coordinator," he noted, highlighting the need for agents to plan, execute, and manage tasks across the entire software stack. This requires a robust API code architecture and workflow orchestration that supports multi-step execution.

Clarke's insights culminated in a call for IT leaders to optimize their tokenomics—essentially, the management of computational resources and costs associated with AI tasks. He cautioned against the tendency to default to the most advanced AI models for every application, arguing that efficiency lies in matching tasks with the most appropriate and cost-effective models, regardless of whether they are hosted on-premises, at the edge, or in the cloud. "The question isn’t whether consumption grows—it absolutely will. The question is, are you running the right tokens on the right infrastructure?" he asked, underscoring the importance of strategic resource allocation.

In summary, Clarke's five imperatives for structuring data infrastructure and enterprise software stacks present a roadmap for organizations looking to implement AI agents effectively. By addressing data silos, evolving AI workloads, ensuring integration, and optimizing resource management, businesses can harness the potential of AI while mitigating risks associated with its deployment.

Why it matters

The implications of Clarke's insights are profound for both creators and technologists. As AI continues to permeate various sectors, understanding the foundational requirements for successful implementation is crucial. For creators, this means recognizing the importance of data architecture and workflow design in developing AI applications. For technologists, it emphasizes the need for robust security measures and integration capabilities to support the autonomous functions of AI agents. Ultimately, adopting these imperatives can lead to more efficient, secure, and cost-effective AI solutions, fostering innovation and growth in an increasingly competitive landscape.