From 590b9c9ef12e5424b21cfaa67e4a7b57b63bc4fa Mon Sep 17 00:00:00 2001
From: Hsien-Chin Lin <linh@hhu.de>
Date: Wed, 19 Oct 2022 14:48:01 +0200
Subject: [PATCH] update readme, add evaluation example

---
 README.md | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 1dd5071..27b2789 100755
--- a/README.md
+++ b/README.md
@@ -3,15 +3,29 @@
 **GenTUS** is a user simulator for task-oriented dialogues, which consists of an encoder-decoder structure and is able to optimise both the user policy and natural language generation jointly. GenTUS generates both semantic actions and natural language utterances, preserving interpretability and enhancing language variation. In addition, by representing the inputs and outputs as word sequences and by using a large pre-trained language model we can achieve generalisability in feature representation.
 
 ## Training
-Building dataset
+* Building dataset
 ```
 python3 convlab2/policy/genTUS/build_data.py --add-history
 ```
-Train the model
+The data will be under `convlab2/policy/genTUS/data` as default.
+
+* Train the model
 ```
 python3 convlab2/policy/genTUS/train_model.py --batch-size 8
 ```
+The result `YOUR_MODEL_DIR` may be named as `genTUS-YY-MM-DD-hh-mm` as default.
+
+* Evaluate the semantic level performance
+The test data `TEST_DATA` will be `convlab2/policy/genTUS/data/data_test_v1.json` as default.
 
+```
+python3 convlab2/policy/genTUS/evaluate.py --do-semantic --dataset multiwoz21 --model-checkpoint YOUR_MODEL_DIR --evaluate-file TEST_DATA --only-action
+```
+
+* Evaluate the natural language performance
+```
+python3 convlab2/nlg/corpus_evaluation.py --model-checkpoint YOUR_MODEL_DIR --input-fname TEST_DATA --do-generation --model-name gentus
+```
 ## Citing
 
 If you use GenTUS in your research, please cite:
-- 
GitLab