ChatCRS: Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

This project is inspired by the idea of applying LLMs for zero-shot conversational recommendation. Advanced LLMs (e.g., ChatGPT) are limited in domain-specific CRS tasks for 1. generating grounded responses with recommendation- oriented knowledge, or 2. proactively leading the conversations through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute sig- nificantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into several sub-tasks through the implementation of 1. a knowledge retrieval agent using a tool-augmented approach to reason over external knowledge bases and 2. a goal-planning agent for dialogue goal prediction. Incorporating those external inputs, LLMs proactively plan interactions and generate outputs with rich information. Experiments on two multi-goal CRS datasets reveal that ChatCRS sets new state-of-the-art benchmarks, improving language quality of informativeness by 17% and proactivity by 27%, with tenfold recommendation accuracy enhancement