Haiyan Zhang on Gaming AI: Collaboration, Creativity, and the Future of Play

Collaborators: Gaming AI with Haiyan Zhang

This article features an interview with Haiyan Zhang, General Manager of Gaming AI at Xbox, discussing the intersection of artificial intelligence and the gaming industry. The conversation, hosted by Dr. Gretchen Huizinga, explores Zhang's career journey, her new role, and the transformative potential of AI in gaming.

Haiyan Zhang's Role and Vision

Zhang's role as General Manager of Gaming AI at Xbox is relatively new, established to consolidate and strategize AI efforts across Xbox. She highlights Xbox's decade-long history of integrating AI, citing technologies like TrueSkill and TrueMatch as early successes. The current focus is on leveraging advancements in foundation models and large language models (LLMs) to enhance game development and player experiences. Zhang emphasizes the need for a unified AI strategy within Xbox, fostering collaboration between research and product teams.

The Evolution of AI in Gaming

Zhang reflects on the evolution of AI in gaming, noting the shift from AI being perceived as an adversary (as seen in early AI benchmarks like chess or Go) to a collaborator. She attributes this shift to the increasing capabilities of AI, particularly with LLMs and generative AI, which are moving from pure technological advancement to practical tools and "copilots." This transition is facilitated by the integration of human-computer interaction (HCI) and design principles, making AI more assistive in daily life and play.

Personal Journey and Interdisciplinary Approach

Zhang shares her personal career path, which began in software engineering and expanded into user experience and design. This diverse background informs her current approach, emphasizing the importance of understanding the "why" and "what" behind technology development. She advocates for a multidisciplinary approach, believing that diverse teams, reflecting the diversity of players, are crucial for inclusive and innovative product development. The democratization of technology through natural language prompts is seen as a key driver for multidisciplinary collaboration, allowing individuals from various fields to contribute to AI development.

AI as a Creative Tool

Zhang discusses the potential of AI to augment human creativity in gaming. She draws parallels to historical advancements in technology and art, suggesting that generative AI can empower individuals who may not have traditional artistic skills to express their ideas. This could lead to new forms of storytelling and artistic expression, pushing the boundaries of what is considered art. She highlights examples like AI-generated dialogue for non-player characters (NPCs) and the use of reinforcement learning and imitation learning to create more human-like AI agents for testing and enhancing gameplay.

Addressing the Uncanny Valley and Future of Gaming

The conversation touches upon the "uncanny valley" phenomenon in character animation, where near-perfect but not quite human-like characters can evoke discomfort. Zhang explains that overcoming this requires a blend of artistry and technology, involving collaboration with animators, neuroscientists, and user researchers. She envisions a future where AI helps create more lifelike characters and immersive experiences, but also acknowledges that the definition of immersion will evolve. The potential for AI to assist in creating personalized and adaptive gaming experiences, catering to players with different abilities or disabilities, is also explored.

Gaming as a Medium for Exploration

Zhang suggests that video games can serve as a safe space for exploring complex societal and ethical questions, similar to how science fiction has done. She believes that by interacting with lifelike AI agents in games, players can explore potential future scenarios and their implications, fostering a deeper understanding of AI's impact.

Inclusivity and Accessibility in Gaming

Xbox's commitment to inclusivity and accessibility is a key theme. Zhang highlights initiatives like the Forza racing game's feature for visually impaired players, which uses 3D sound to guide gameplay, and speech-to-text functionality in Xbox Party Chat to improve communication for players with disabilities. These efforts aim to make gaming more accessible and enjoyable for a wider audience, reflecting the democratization of gaming across various platforms.

The Future of Gaming and AI Collaboration

Looking ahead, Zhang expresses excitement about AI's potential to enhance visual fidelity, character behavior, and player accessibility. She emphasizes the importance of turning AI models into practical tools for game developers, thereby accelerating game creation and improving quality. The ultimate goal is to use AI to help people tell more human stories, foster empathy, and build a more understanding society. The interview concludes with a reflection on the journey of technology from concept to reality, underscoring the collaborative spirit that drives innovation in the gaming industry.

Key Research Areas and Projects Mentioned:

Game Intelligence: A research group focused on AI in gaming.
Project Paidia: A collaboration between Microsoft Research and Ninja Theory focused on AI agents for games.
TrueSkill™ Ranking System: An AI-powered system for matchmaking and skill assessment.
TrueMatch Matchmaking System: A system for matching players in online games.
Grounded Conversational Characters: Research on creating natural language interactions for game characters using LLMs.

Follow Microsoft Research:

X (Twitter)
Facebook
LinkedIn
YouTube
Instagram
RSS Feed