As automated self-checkout cashiers and self-order kiosks become more common in our daily lives, they offer a convenient option for purchasing low involvement products, such as groceries. However, when it comes to high involvement products, such as smart phones, consumers tend to prefer speaking with human employees who can provide advice and guidance. Social Robots such as Furhat (Video) offer interesting possibilities for providing shopping assistance in the context of high involvement products.  Furthermore, with advancements in Generative AI (e.g., ChatGPT), robot shopping assistants are able to generate more human-like and personalized responses. However, recognizing human emotions, which play an important role in purchase decisions, still remains a challenge. This leads to the question of how to design robot shopping assistants powered by large language models that can advise customers when purchasing high involvement products and take into account customers’ emotions.

Goal of the thesis

The overall goal is to extend an existing prototype application for the social robot "Furhat" with emotion processing capabilities. Ultimately, the robot should be able to recognize and interpret human emotions using the camera and/or microphone integrated in the robot. Third party libraries will be used for emotion processing building on already implemented approaches. The social robot and required technologies are provided by the h-lab. The foundational prototype of the application is ready to be developed further. Your primary task will be to develop and evaluate an emotion processing feature that can enable Furhat to adjust its responses based on the user's affective states, thus enhancing its social interaction capabilities. You are also encouraged to bring in your own innovative ideas to this project to further improve the system's effectiveness.

Skills required

• Good programming skills (e.g., Java, Kotlin, Python)
• General interest in generative AI, large language models, and/or human-computer interaction
• Good time management and organizational skills
• English skills

Contact

If you are interested and want to apply for this topic (practical seminar, bachelor thesis, master thesis), please contact Leon Hanschmann (leon.hanschmann∂kit.edu) or Dr. Ulrich Gnewuch (ulrich gnewuch ∂does-not-exist.kit edu) with a short motivation statement, your CV, and a current transcript of records. Feel free to reach out beforehand if you have any questions.