Large Language Models (LLM) enable impressive automation in text-based information processing as well as in answering questions in service support. For example, the answering of standardized questions based on documents (e.g., technical documentation) can be automated in the form of search or chat interfaces. However, this requires solving a number of conceptual challenges. This concerns, for example, the technical architecture; such as the requirements for a data/document source, the criteria for selecting a language model, or the desired depth of the technical process connection or dialogs of a support tool. Furthermore, risks and constraints such as data protection requirements or the risk of hallucinations of the language model have to be considered and excluded. Furthermore, in practical contexts the organizational embedding as well as the reaction of employees and customers play a role. Against this background, this thesis is dedicated to the evaluation of the possible use of Artificial Intelligence (AI) in the form of LLM for automation in service support.

Goal of the thesis

The goal of the thesis is to develop concepts and methods that evaluate and support the implementation of an LLM for service automation. These will be developed based on a real-world use case provided by an industry partner (see below). The thesis will evaluate whether and how the use of LLM/KI can improve service delivery in different implementation scenarios. In doing so, economic, legal as well as technical-procedural aspects and the effects on the customer and employee side are to be considered. On the theoretical side, these are to be translated into a generally applicable concept such as a process model or a catalog of criteria for evaluating the use of LLM. A first prototyping and a test of this is also possible.

Work packages and Skills required

Interested students should have a basic technical understanding or interest in NLP or LLM technology and chatbot or information retrieval applications. Furthermore, proactivity and the ability to work in a team to conduct interview studies are desired as well. Experience or skills in requirements elicitation and engineering as well as the design of software artifacts are also helpful.

Industry partner/case

TechniData IT-Service GmbH (IT-S) is a medium-sized IT systems house with around 200 employees and several locations in BaWü (including Karlsruhe). Its activities focus on the design, project planning and operation of IT infrastructures for customers (workplace, network, data center, cloud). For this purpose, IT-S also operates its own ServiceDesk, through which customer inquiries can be accepted and processed through to an underlying 2nd and 3rd level support. The main input channels are telephone and email.