User input is essential for the successful development of question-and-answer (QA) chatbots. Therefore, interactive development systems are emerging that allow developers to involve test-users in the QA chatbot development process. Although the feasibility and effectiveness of this approach have been demonstrated, there is a lack of knowledge on how to design interactive QA chatbot development systems to increase test-user engagement in this time-consuming and tedious task. To address this research gap, we propose an interactive system design based on the interactivity effects model. We instantiate the proposed design and show in two online experiments that it significantly increases subjective and objective engagement of test-users. We contribute with theoretical implications for the development of engaging interactive QA chatbot development systems based on the interactivity effects model and with practical design implications for an implementation in commercial chatbot development systems.