RoleAgent: Building, Interacting, and Benchmarking High-quality Role-Playing Agents from Scripts

Abstract

Believable proxies of human behavior can em- power interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyp- ing tools. Recently, generative agents have been proposed to simulate believable human behavior by using Large Language Models. However, the existing method heavily relies on human-annotated agent profiles (e.g., name, age, personality, relationships with others, and so on) for the initialization of each agent, which cannot be scaled up easily. In this paper, we propose a scalable RoleAgent framework to generate high-quality role-playing agents from raw scripts, which includes building and in- teracting stages. Specifically, in the building stage, we first use a hierarchical memory sys- tem to extract and summarize the structure and high-level information of each agent for the raw script. Then, in the interacting stage, we further propose a novel innovative mechanism with four steps to achieve a high-quality in- teraction between agents. Finally, we intro- duce a systematic and comprehensive evalua- tion benchmark called RoleAgentBench to eval- uate the effectiveness of our RoleAgent, which includes 54 roles from 5 English and 5 Chinese scripts. Extensive experimental results on our RoleAgentBench demonstrate the effectiveness of our RoleAgent.