Keynote: Research in the Era of AI by Peter Lee

Keynote: Research in the Era of AI

Peter Lee, President of Microsoft Research, delivered a keynote address at the Microsoft Research Forum, discussing the profound impact of Artificial Intelligence (AI) on the landscape of scientific research. He began by drawing a historical parallel to scientific disruptions, using the example of biology in the 1700s where the prevailing theory of cell crystallization was eventually overturned by the discovery of cell division, fundamentally changing biological research and requiring new infrastructure like microscopes.

Lee posited that computer science, particularly in the realm of AI, is currently undergoing a similar disruptive phase. The advent of powerful machine learning systems and architectures like neural transformers has necessitated new research infrastructure, including large datasets, substantial GPU compute power, and advanced training pipelines. This shift is impacting virtually all areas of computer science research.

The GPT-4 Experience and the "Sparks" Paper

Microsoft's collaboration with OpenAI on GPT-4 provided a unique opportunity to study the model's capabilities. Lee detailed how Microsoft Research conducted secret investigations into GPT-4 towards the end of 2022, culminating in the publication of the paper "Sparks of Artificial General Intelligence: Early experiments with GPT-4" in March 2023. This paper explored the model's emergent cognitive capabilities, which defied simple explanations and sparked significant discussion and research within the community. The research aimed to understand GPT-4's abilities in planning, causal reasoning, common-sense reasoning, and decision-making in complex scenarios.

Responsible AI and Societal Impact

A significant portion of the keynote was dedicated to the concept of "Responsible AI" and its broader implications for "AI and Society." Lee emphasized that Microsoft has invested heavily in understanding the potential harms and risks associated with AI technologies, as well as the opportunities they present. This has led to the integration of responsible AI principles and societal impact thinking across all engineering, finance, security, and legal departments within the company. The focus is on FATE: Fairness, Accountability, Transparency, and Ethics.

Open Source and Model Development

Microsoft Research is also actively contributing to the open-source AI community. Lee highlighted the development of the Phi series of models (Phi-1, Phi-1.5, Phi-2), which focus on synthesizing training data to enhance specific reasoning and problem-solving strategies rather than just world knowledge. These models aim to foster greater understanding of transparency, alignment issues, and AI safety in the pretraining phase.

Furthermore, Microsoft is working on platform and model orchestration, exemplified by AutoGen, an open-source framework that enables multiple AI agents to collaborate on solving problems. Research on models like Orca and Orca 2 focuses on improving reasoning capabilities, while LLaVA-Med demonstrates the potential of specialized models in domains like medicine.

Specialization vs. Generalization

The interplay between specialization and generalization in AI models was another key theme. Lee discussed the question of whether AI models need extensive specialization to be effective in specific domains, using healthcare as an example. Research using prompt engineering techniques, such as "promptbase" and "Medprompt," suggests that generalist models like GPT-4, when properly prompted, can outperform specialized models. The paper "Who's Harry Potter?" was cited to illustrate how even limited specialization can lead to a model forgetting previously learned information, highlighting the complexities of specialization.

AI's Pervasive Influence and Future Outlook

Lee concluded by emphasizing that AI has permeated nearly all research areas at Microsoft, from security and privacy to program analysis and healthcare. The creation of the AI4Science lab was highlighted, showcasing rapid advancements in synthesizing new materials for batteries and discovering drug treatments for diseases like tuberculosis and coronaviruses. He expressed optimism about the future, stressing the importance of collaboration, openness, and a clear-eyed approach to both the immense potential benefits and the inherent risks of AI. The goal is to ensure that AI development benefits humanity while mitigating downstream harms.

Related Research and Events

The keynote was part of the Microsoft Research Forum, Episode 1. Related publications mentioned include "Sparks of Artificial General Intelligence," "Orca," "Phi-1.5," "Textbooks Are All You Need," and "Who's Harry Potter? Approximate Unlearning in LLMs." The event also featured other talks on topics like AI Frontiers, Generative AI in Structural Biology, Augmenting Human Cognition, and Evaluating Foundation Models.