Introduction
This is the repository for our paper ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?. We provide our raw log files and the evaluation code used to compute the experimental results that we present in the paper.
How to run
DO.prepare will clone the MultiWOZ 2.1 dataset and the TripPy code. It will prepare the train/dev/test split of the dataset and unpack the raw log files.
DO.eval will convert the raw log files into TripPy-style predictions, run the TripPy-style evaluation and print some statistics. Said evaluation is done across domains. Additionally, this script also runs the evaluation per domain. Detailed results, errors and performance metrics are found in the respective log files.
Citation
This work is published as ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?
If you use our logs or code for your own work, please cite our work as follows:
@inproceedings{heck-etal-2023-chatgpt,
title = "{C}hat{GPT} for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?",
author = "Heck, Michael and Lubis, Nurul and Ruppik, Benjamin and Vukovic, Renato and Feng, Shutong and
Geishauser, Christian and Lin, Hsien-chin and van Niekerk, Carel and Gasic, Milica",
booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
pages = "936--950",
}