IAB Tech Lab proposes new AI content framework

New Delhi: The IAB Tech Lab, a global organisation that develops technical standards for digital advertising, has proposed a new framework to respond to the challenges posed by generative artificial intelligence (AI), particularly large language models (LLMs) and AI search agents.

According to the organisation, the increasing reliance on AI-generated content and summaries is reducing traffic to publisher sites and raising concerns around content attribution and brand visibility. Publishers argue that their content is being used without adequate credit or compensation, while brands worry about misrepresentation in AI-powered search results and chat interfaces.

To address these issues, the IAB Tech Lab has introduced the LLM Content Ingest API Initiative, which targets to establish a technical specification that supports fair compensation for publishers and offers brands better control over how their content is accessed and displayed by AI systems.

“It is clear that AI agents powered by large language models are shifting how users engage with content,” said Anthony Katsur, Chief Executive Officer, IAB Tech Lab. “While this is a promising new way for people to access information, we have also seen data showing publisher traffic decreases at 15% or higher, and revenue is down. Meanwhile, AI platforms are growing on the back of open web content, impacting publisher revenues and misrepresenting advertisers. This initiative is about giving publishers and brands a path forward that is fair, enforceable, and grounded in technical standards.”

The proposed framework is intended to support publishers in responding to growing AI scraping and to establish a value exchange between content owners and AI developers. It will also allow brands to better manage how their information is accessed, interpreted, and presented by AI systems.

The initiative invites publishers, brands, and AI developers to contribute to shaping the standards for future content usage. A workshop is expected to be held soon to collectively explore practical tools for managing AI content access. Topics of focus include managing bot access, LLM-compatible content discovery, and monetisation strategies.

“The LLM Content Ingest API is one part of the framework to address the challenges publishers and brands face from the growing use of AI-based tools by consumers,” said Shailley Singh, Executive Vice President, Product and Chief Operating Officer, IAB Tech Lab. “The proposed technical framework is designed to foster better collaboration between LLMs/AI agents and content owners and to lay the foundations for fair value exchange for content, two of the biggest challenges that have emerged since the proliferation of generative AI and endanger the web economy as it works today.”