Hugging Face Replicates OpenAI's Deep Research in 24 Hours with Open Source Agent

Introduction

On February 5, 2025, Hugging Face released "Open Deep Research," an open-source AI research agent designed to replicate the capabilities of OpenAI's proprietary "Deep Research" feature. This initiative, completed within a 24-hour hackathon, aims to democratize access to advanced AI agent technology, allowing developers to study and build upon it.

The Challenge: Replicating Deep Research

OpenAI's "Deep Research" feature allows AI models to autonomously browse the web and generate research reports. Hugging Face's "Open Deep Research" project was launched as a direct response to OpenAI's announcement, with the goal of matching its performance and making the underlying agentic framework publicly available. The team emphasized that while powerful open-source LLMs exist, the specific agent framework used by OpenAI was not disclosed, prompting their rapid development effort.

How AI Agents Work

Similar to OpenAI's Deep Research and Google's Gemini-based "Deep Research," Hugging Face's solution utilizes an "agent" framework. This framework enhances existing AI models by enabling them to perform multi-step tasks, such as information gathering and report generation, in an autonomous manner. The agent acts as an intermediary, orchestrating the AI model's capabilities to complete complex research objectives.

Performance and Benchmarking

Even after just one day of development, Hugging Face's Open Deep Research demonstrated competitive performance on the General AI Assistants (GAIA) benchmark. This benchmark evaluates an AI model's ability to synthesize information from multiple sources. Hugging Face's agent achieved 55.15 percent accuracy, while OpenAI's Deep Research scored 67.36 percent with a single-pass response, and 72.57 percent with a consensus mechanism.

The GAIA Benchmark Challenge

The GAIA benchmark includes complex, multi-step questions that require AI agents to access and integrate information from various disparate sources. An example question cited by Hugging Face involves identifying fruits from a specific painting that were part of a historical ship's menu, requiring the AI to cross-reference art, historical data, and culinary information.

Core AI Model Selection

An AI agent requires a foundational AI model. Open Deep Research can leverage various models, including OpenAI's GPT-4o and simulated reasoning models like o1 and o3-mini, through APIs. It is also adaptable to open-weight AI models. Aymeric Roucher, the project lead at Hugging Face, explained that while they used a closed-weight model (o1) for its effectiveness, the development process and code are fully transparent, allowing for easy integration of other models. Hugging Face is also working on the "open-R1 initiative" to potentially replace o1 with a superior open-source model.

The Importance of the Agentic Layer

Open Deep Research highlights that the agentic layer is crucial for enhancing AI model capabilities. Benchmarks show a significant improvement when an agentic framework is applied. For instance, GPT-4o alone scores an average of 29 percent on the GAIA benchmark, whereas OpenAI's Deep Research, with its agentic framework, achieves 67 percent.

Smolagents and Code Agents

Hugging Face utilized its open-source "smolagents" library, which employs "code agents" instead of JSON-based agents. Code agents write actions in programming code, reportedly making them 30 percent more efficient and better at handling complex action sequences.

The Speed of Open Source AI

The rapid development of Open Deep Research is a testament to the agility of open-source AI projects. The team benefited from external contributors and leveraged existing tools, such as Microsoft Research's "Magnetic-One" agent project, to accelerate development.

Future Developments

While Open Deep Research is still optimizing its performance and user experience compared to commercial offerings, Hugging Face plans to add support for more file formats and vision-based web browsing. They are also working on replicating OpenAI's "Operator" agent, which can control computer interfaces within a web browser.

Community Impact

Hugging Face has made the project's code publicly available on GitHub, fostering community involvement. The project has already attracted numerous contributors, demonstrating the power of open collaboration in advancing AI technology.

Conclusion

Hugging Face's Open Deep Research project showcases the rapid progress and collaborative spirit within the open-source AI community. By providing free access to advanced agent technology, they empower developers to innovate and push the boundaries of artificial intelligence.

Image Credits

Credit: 3alexd via Getty Images

Comments

43 comments