Skip to content
Snippets Groups Projects
Commit 3393a064 authored by zqwerty's avatar zqwerty
Browse files

update requirements.txt & setup.py, move tests/ to agent_examples/, move deploy to convlab/deploy

parent 8e8eba91
Branches
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
Showing
with 0 additions and 26 deletions
tests/results/SVMNLU-RuleDST-RulePolicy-TemplateNLG/res.txt deleted 100755 → 0
+ 0
− 8
View file @ 8e8eba91
complete number of dialogs/tot: 0.842
success number of dialogs/tot: 0.704
average precision: 0.7909328570932932
average recall: 0.8878809606223936
average f1: 0.8148240530602635
average book rate: 0.7613247863247864
average turn (succ): 14.758522727272727
average turn (all): 17.736
tests/results/Sequicity/Frequency_of_domain.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/Sequicity/Frequency_of_domain.png

212 KiB

tests/results/Sequicity/Performance_for_each_domain.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/Sequicity/Performance_for_each_domain.png

182 KiB

tests/results/Sequicity/Proportions_of_the_dialogue_loop.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/Sequicity/Proportions_of_the_dialogue_loop.png

222 KiB

tests/results/Sequicity/report_multiwoz.html deleted 100755 → 0
+ 0
− 1
View file @ 8e8eba91
<html><body><h1>Test Report</h1><h3>Model Name: Sequicity</h3><h3>Dataset: multiwoz</h3><h3>Time: 2020-04-26 00:56:51</h3><h3>Overall Results</h3><p> Success Rate: 10.5 %</p><p> (Precision, Recall, F1) : (0.414, 0.308, 0.313) </p><p> Average Dialog Turn (Succ): 12.876 </p><p> Average Dialog Turn (All): 38.320 </p><table><tr><td><img src="Frequency_of_domain.png" width=600 border=0/></td><td><img src="Performance_for_each_domain.png" width=600 border=0/></td></tr></table><h3>Metric</h3><table width="900" border="1"><tr><th>&nbsp;</th><th>Total Num</th><th>Succ Rate</th><th>Precision</th><th>Recall</th><th>F1</th><th>Dialog Loop Failed Rate</th><th>Dialog Turn (Succ)</th><th>Dialog Turn (All)</th></tr><tr><td align="center">hotel</td><td align="center">244</td><td align="center">0.164</td><td align="center">0.272</td><td align="center">0.328</td><td align="center">0.275</td><td align="center">0.803</td><td align="center">10.550</td><td align="center">31.336</td></tr><tr><td align="center">restaurant</td><td align="center">332</td><td align="center">0.265</td><td align="center">0.494</td><td align="center">0.464</td><td align="center">0.456</td><td align="center">0.720</td><td align="center">6.409</td><td align="center">29.229</td></tr><tr><td align="center">train</td><td align="center">251</td><td align="center">0.191</td><td align="center">0.345</td><td align="center">0.272</td><td align="center">0.295</td><td align="center">0.789</td><td align="center">15.500</td><td align="center">32.430</td></tr><tr><td align="center">attraction</td><td align="center">234</td><td align="center">0.162</td><td align="center">0.635</td><td align="center">0.430</td><td align="center">0.488</td><td align="center">0.833</td><td align="center">5.842</td><td align="center">33.103</td></tr><tr><td align="center">police</td><td align="center">23</td><td align="center">0.087</td><td align="center">0.087</td><td align="center">0.087</td><td align="center">0.087</td><td align="center">0.913</td><td align="center">16.000</td><td align="center">37.913</td></tr><tr><td align="center">hospital</td><td align="center">30</td><td align="center">0.867</td><td align="center">0.867</td><td align="center">0.867</td><td align="center">0.867</td><td align="center">0.133</td><td align="center">7.692</td><td align="center">12.000</td></tr><tr><td align="center">taxi</td><td align="center">44</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">1.000</td><td align="center">0.000</td><td align="center">40.000</td></tr></table><h3>Domain hotel</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 16.4 %</p><p> (Precision, Recall, F1) : (0.272, 0.328, 0.275) </p><p> Average Dialog Turn (Succ): 10.550 </p><p> Average Dialog Turn (All): 31.336 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Hotel-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 75</li><li>Inform-Hotel-Stay-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Request-Hotel-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>Inform-Hotel-Stay-2&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>Inform-Hotel-Stay-3&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Request-Hotel-Stars-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Inform-Hotel-Stay-5&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Request-Hotel-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Request-Hotel-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-hotel-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 34</li><li>inform-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>inform-hotel-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>inform-hotel-internet&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>inform-hotel-parking&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>recommend-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-hotel-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 92</li><li>request-hotel-internet&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-hotel-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-hotel-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-hotel-parking&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>request-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>request-hotel-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>request-hotel-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-hotel-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 86</li><li>inform-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 43</li><li>inform-hotel-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>inform-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></div><h3>Domain restaurant</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 26.5 %</p><p> (Precision, Recall, F1) : (0.494, 0.464, 0.456) </p><p> Average Dialog Turn (Succ): 6.409 </p><p> Average Dialog Turn (All): 29.229 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Restaurant-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 129</li><li>Request-Restaurant-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 38</li><li>Request-Restaurant-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Inform-Restaurant-Day-sunday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>Request-Restaurant-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>Inform-Restaurant-Time-16:45&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Restaurant-Day-saturday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Restaurant-Time-18:30&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Restaurant-People-1&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Restaurant-Day-wednesday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-restaurant-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 33</li><li>inform-restaurant-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>inform-restaurant-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>recommend-restaurant-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-restaurant-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 166</li><li>request-restaurant-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-restaurant-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>request-restaurant-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>request-restaurant-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-restaurant-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 54</li><li>inform-restaurant-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>inform-restaurant-food&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li></ul></div><h3>Domain train</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 19.1 %</p><p> (Precision, Recall, F1) : (0.345, 0.272, 0.295) </p><p> Average Dialog Turn (Succ): 15.500 </p><p> Average Dialog Turn (All): 32.430 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Train-Time-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 62</li><li>Request-Train-Id-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 52</li><li>Request-Train-Ticket-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 42</li><li>Request-Train-Leave-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>Request-Train-Arrive-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>Inform-Train-People-6&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Train-People-5&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Train-People-3&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Train-People-1&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Train-People-8&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-train-trainID&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>offerbooked-train-price&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>inform-train-day&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>inform-train-arriveBy&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>inform-train-leaveAt&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-train-trainID&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 100</li><li>request-train-price&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 57</li><li>request-train-duration&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 46</li><li>request-train-arriveBy&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>request-train-leaveAt&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain attraction</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 16.2 %</p><p> (Precision, Recall, F1) : (0.635, 0.430, 0.488) </p><p> Average Dialog Turn (Succ): 5.842 </p><p> Average Dialog Turn (All): 33.103 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Attraction-Fee-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 109</li><li>Request-Attraction-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 71</li><li>Request-Attraction-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Request-Attraction-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Request-Attraction-Area-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-attraction-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 132</li><li>request-attraction-entrance fee&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 109</li><li>request-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>request-attraction-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>request-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>request-attraction-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>inform-attraction-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>inform-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></div><h3>Domain police</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 8.7 %</p><p> (Precision, Recall, F1) : (0.087, 0.087, 0.087) </p><p> Average Dialog Turn (Succ): 16.000 </p><p> Average Dialog Turn (All): 37.913 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Police-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>Request-Police-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-police-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li><li>request-police-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain hospital</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 86.7 %</p><p> (Precision, Recall, F1) : (0.867, 0.867, 0.867) </p><p> Average Dialog Turn (Succ): 7.692 </p><p> Average Dialog Turn (All): 12.000 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Hospital-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-hospital-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain taxi</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 0.0 %</p><p> (Precision, Recall, F1) : (0.000, 0.000, 0.000) </p><p> Average Dialog Turn (Succ): 0.000 </p><p> Average Dialog Turn (All): 40.000 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Taxi-Car-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 44</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-taxi-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 44</li><li>request-taxi-car type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 44</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div></body></html>
\ No newline at end of file
tests/results/Sequicity/res.txt deleted 100755 → 0
+ 0
− 8
View file @ 8e8eba91
complete number of dialogs/tot: 0.131
success number of dialogs/tot: 0.105
average precision: 0.4144451861741581
average recall: 0.30776920667886404
average f1: 0.31279910524458815
average book rate: 0.05128205128205128
average turn (succ): 12.876190476190477
average turn (all): 38.32
tests/results/TRADE-RulePolicy-TemplateNLG/Frequency_of_domain.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/TRADE-RulePolicy-TemplateNLG/Frequency_of_domain.png

211 KiB

tests/results/TRADE-RulePolicy-TemplateNLG/Performance_for_each_domain.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/TRADE-RulePolicy-TemplateNLG/Performance_for_each_domain.png

183 KiB

tests/results/TRADE-RulePolicy-TemplateNLG/Proportions_of_the_dialogue_loop.png deleted 100755 → 0
+ 0
− 0
View file @ 8e8eba91
tests/results/TRADE-RulePolicy-TemplateNLG/Proportions_of_the_dialogue_loop.png

237 KiB

tests/results/TRADE-RulePolicy-TemplateNLG/report_multiwoz.html deleted 100755 → 0
+ 0
− 1
View file @ 8e8eba91
<html><body><h1>Test Report</h1><h3>Model Name: TRADE-RulePolicy-TemplateNLG</h3><h3>Dataset: multiwoz</h3><h3>Time: 2020-04-29 21:10:04</h3><h3>Overall Results</h3><p> Success Rate: 25.3 %</p><p> (Precision, Recall, F1) : (0.493, 0.481, 0.444) </p><p> Average Dialog Turn (Succ): 12.735 </p><p> Average Dialog Turn (All): 24.684 </p><table><tr><td><img src="Frequency_of_domain.png" width=600 border=0/></td><td><img src="Performance_for_each_domain.png" width=600 border=0/></td></tr></table><h3>Metric</h3><table width="900" border="1"><tr><th>&nbsp;</th><th>Total Num</th><th>Succ Rate</th><th>Precision</th><th>Recall</th><th>F1</th><th>Dialog Loop Failed Rate</th><th>Dialog Turn (Succ)</th><th>Dialog Turn (All)</th></tr><tr><td align="center">hotel</td><td align="center">330</td><td align="center">0.285</td><td align="center">0.167</td><td align="center">0.286</td><td align="center">0.188</td><td align="center">0.473</td><td align="center">9.617</td><td align="center">16.448</td></tr><tr><td align="center">attraction</td><td align="center">291</td><td align="center">0.299</td><td align="center">0.614</td><td align="center">0.541</td><td align="center">0.547</td><td align="center">0.533</td><td align="center">6.368</td><td align="center">20.192</td></tr><tr><td align="center">restaurant</td><td align="center">361</td><td align="center">0.335</td><td align="center">0.479</td><td align="center">0.490</td><td align="center">0.465</td><td align="center">0.427</td><td align="center">4.744</td><td align="center">14.964</td></tr><tr><td align="center">train</td><td align="center">273</td><td align="center">0.897</td><td align="center">0.795</td><td align="center">0.865</td><td align="center">0.815</td><td align="center">0.084</td><td align="center">9.902</td><td align="center">11.773</td></tr><tr><td align="center">police</td><td align="center">23</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">1.000</td><td align="center">0.000</td><td align="center">40.000</td></tr><tr><td align="center">hospital</td><td align="center">30</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">0.000</td><td align="center">1.000</td><td align="center">0.000</td><td align="center">40.000</td></tr><tr><td align="center">taxi</td><td align="center">46</td><td align="center">0.935</td><td align="center">1.000</td><td align="center">0.967</td><td align="center">0.978</td><td align="center">0.065</td><td align="center">5.209</td><td align="center">7.478</td></tr></table><h3>Dialogue Loop</h3><img src="Proportions_of_the_dialogue_loop.png" width=800 /><h3>Domain hotel</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 28.5 %</p><p> (Precision, Recall, F1) : (0.167, 0.286, 0.188) </p><p> Average Dialog Turn (Succ): 9.617 </p><p> Average Dialog Turn (All): 16.448 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><ul><li>Inform-Hotel-Parking-yes<ul><li>Occur Num: 886</li><li>NLU Output</li><ul><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 139</li><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 72</li><li>Inform-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 51</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 47</li><li>Inform-Hotel-Type-hotel&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 37</li></ul></ul><li>Inform-Hotel-Internet-yes<ul><li>Occur Num: 732</li><li>NLU Output</li><ul><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 120</li><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 82</li><li>Inform-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 44</li><li>Inform-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 37</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 36</li></ul></ul><li>NoOffer-Hotel-Internet-yes<ul><li>Occur Num: 462</li><li>NLU Output</li><ul><li>NoOffer-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 39</li><li>NoOffer-Hotel-Type-hotels&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 36</li><li>NoOffer-Hotel-Type-hotel&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 30</li><li>NoOffer-Hotel-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 28</li><li>NoOffer-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 28</li></ul></ul><li>Inform-Hotel-Stars-4<ul><li>Occur Num: 394</li><li>NLU Output</li><ul><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 53</li><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 48</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 32</li><li>Inform-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>Inform-Hotel-Area-north&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li></ul></ul><li>Inform-Hotel-Type-guesthouse<ul><li>Occur Num: 365</li><li>NLU Output</li><ul><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 47</li><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 36</li><li>Inform-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 30</li><li>Inform-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 29</li><li>Inform-Hotel-Area-north&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li></ul></ul><li>NoOffer-Hotel-Type-hotel<ul><li>Occur Num: 341</li><li>NLU Output</li><ul><li>NoOffer-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>NoOffer-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li><li>NoOffer-Hotel-Type-hotels&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 22</li><li>NoOffer-Hotel-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 22</li><li>NoOffer-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li></ul></ul><li>Inform-Hotel-Price-moderate<ul><li>Occur Num: 302</li><li>NLU Output</li><ul><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 39</li><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 39</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 28</li><li>Inform-Hotel-Area-north&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 25</li><li>Inform-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li></ul></ul><li>NoOffer-Hotel-Parking-yes<ul><li>Occur Num: 295</li><li>NLU Output</li><ul><li>NoOffer-Hotel-Type-hotel&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 28</li><li>NoOffer-Hotel-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>NoOffer-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>reqmore-general-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>Request-Hotel-Area-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li></ul></ul><li>NoOffer-Hotel-Price-moderate<ul><li>Occur Num: 273</li><li>NLU Output</li><ul><li>NoOffer-Hotel-Type-hotels&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>NoOffer-Hotel-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>NoOffer-Hotel-Type-hotel&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>NoOffer-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>Request-Hotel-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li></ul></ul><li>Inform-Hotel-Type-hotel<ul><li>Occur Num: 251</li><li>NLU Output</li><ul><li>Inform-Hotel-Parking-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 43</li><li>Inform-Hotel-Internet-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 35</li><li>Inform-Hotel-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Inform-Hotel-Stars-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>Inform-Hotel-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li></ul></ul></ul><b> Dialog Loop</b><ul><li>Request-Hotel-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 50</li><li>Request-Hotel-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 39</li><li>Request-Hotel-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>Inform-Hotel-Stay-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>Request-Hotel-Internet-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Inform-Hotel-Parking-yes&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Request-Hotel-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Inform-Hotel-Internet-yes&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Hotel-Stay-5&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Hotel-Stay-2&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-hotel-parking&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 152</li><li>inform-hotel-internet&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 126</li><li>inform-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 25</li><li>inform-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li><li>recommend-hotel-parking&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>recommend-hotel-internet&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>recommend-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>inform-hotel-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>inform-hotel-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-hotel-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 82</li><li>request-hotel-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 68</li><li>request-hotel-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 35</li><li>request-hotel-internet&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 25</li><li>request-hotel-parking&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>request-hotel-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>request-hotel-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>request-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>request-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-hotel-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 70</li><li>inform-hotel-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 65</li><li>inform-hotel-stars&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 60</li><li>inform-hotel-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul></div><h3>Domain attraction</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 29.9 %</p><p> (Precision, Recall, F1) : (0.614, 0.541, 0.547) </p><p> Average Dialog Turn (Succ): 6.368 </p><p> Average Dialog Turn (All): 20.192 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><ul><li>Inform-Attraction-Area-centre<ul><li>Occur Num: 292</li><li>NLU Output</li><ul><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li><li>Inform-Attraction-Type-museum&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li><li>Inform-Attraction-Addr-Regent Street City Centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li><li>Inform-Attraction-Area-within&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>Inform-Attraction-Type-park&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li></ul></ul><li>Inform-Attraction-Type-museum<ul><li>Occur Num: 156</li><li>NLU Output</li><ul><li>Inform-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>Inform-Attraction-Addr-Regent Street City Centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>Inform-Attraction-Fee-free&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>Recommend-Attraction-Type-museum&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li></ul></ul><li>Request-Attraction-Type-?<ul><li>Occur Num: 138</li><li>NLU Output</li><ul><li>NoOffer-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>NoOffer-Attraction-Type-entertainment&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>Request-Attraction-Name-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Attraction-Type-college&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Request-Attraction-Area-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li></ul></ul><li>NoOffer-Attraction-Area-centre<ul><li>Occur Num: 130</li><li>NLU Output</li><ul><li>NoOffer-Attraction-Type-entertainment&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>NoOffer-Attraction-Area-that area of town&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Attraction-Type-college&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></ul><li>Inform-Attraction-Fee-?<ul><li>Occur Num: 117</li><li>NLU Output</li><ul><li>Inform-Attraction-Addr-Regent Street City Centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>Inform-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>Request-Taxi-Leave-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Inform-Attraction-Area-north&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul><li>Inform-Attraction-Fee-free<ul><li>Occur Num: 102</li><li>NLU Output</li><ul><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Inform-Attraction-Addr-Regent Street City Centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>Inform-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>Inform-Attraction-Type-park&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Request-Train-Arrive-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul><li>Inform-Attraction-Choice-44<ul><li>Occur Num: 94</li><li>NLU Output</li><ul><li>reqmore-general-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Recommend-Attraction-Name-primavera&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Recommend-Attraction-Name-scudamores punting co&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Recommend-Attraction-Post-cb21qy&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li></ul></ul><li>NoOffer-Attraction-Type-college<ul><li>Occur Num: 93</li><li>NLU Output</li><ul><li>NoOffer-Attraction-Area-west&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>NoOffer-Attraction-Type-entertainment&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>NoOffer-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></ul><li>Request-Attraction-Name-?<ul><li>Occur Num: 84</li><li>NLU Output</li><ul><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Attraction-Type-college&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Inform-Attraction-Addr-Regent Street City Centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>NoOffer-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>NoOffer-Attraction-Type-entertainment&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul><li>NoOffer-Attraction-Area-west<ul><li>Occur Num: 72</li><li>NLU Output</li><ul><li>NoOffer-Attraction-Type-college&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Attraction-Type-entertainment&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Inform-Attraction-Post-cb21ab&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>NoOffer-Attraction-Type-museum&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul></ul><b> Dialog Loop</b><ul><li>Request-Attraction-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 78</li><li>Request-Attraction-Fee-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 42</li><li>Request-Attraction-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li><li>Request-Attraction-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>Inform-Attraction-Type-college&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Attraction-Type-nightclub&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Request-Attraction-Type-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Bad Inform Dialog Act</b><ul><li>inform-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 33</li><li>recommend-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>inform-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>recommend-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 140</li><li>request-attraction-entrance fee&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 59</li><li>request-attraction-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 54</li><li>request-attraction-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 28</li><li>request-attraction-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>request-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-attraction-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 47</li><li>inform-attraction-type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 43</li><li>inform-attraction-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li></ul></div><h3>Domain restaurant</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 33.5 %</p><p> (Precision, Recall, F1) : (0.479, 0.490, 0.465) </p><p> Average Dialog Turn (Succ): 4.744 </p><p> Average Dialog Turn (All): 14.964 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><ul><li>Inform-Restaurant-Area-centre<ul><li>Occur Num: 269</li><li>NLU Output</li><ul><li>Inform-Restaurant-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>Inform-Attraction-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Request-Taxi-Arrive-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Inform-Restaurant-Price-moderate&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>Inform-Restaurant-Price-cheap&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li></ul></ul><li>Inform-Restaurant-Price-expensive<ul><li>Occur Num: 187</li><li>NLU Output</li><ul><li>Inform-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>Request-Taxi-Arrive-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Inform-Restaurant-Food-indian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Request-Taxi-Leave-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Recommend-Restaurant-Name-bangkok city&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></ul><li>Request-Restaurant-Food-?<ul><li>Occur Num: 173</li><li>NLU Output</li><ul><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>Request-Restaurant-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>NoOffer-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>NoOffer-Restaurant-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>Request-Restaurant-Area-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li></ul></ul><li>NoOffer-Restaurant-Price-expensive<ul><li>Occur Num: 163</li><li>NLU Output</li><ul><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>NoOffer-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>Request-Restaurant-Food-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Request-Restaurant-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>NoOffer-Restaurant-Food-indian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li></ul></ul><li>NoOffer-Restaurant-Area-centre<ul><li>Occur Num: 151</li><li>NLU Output</li><ul><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>Request-Restaurant-Food-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>NoOffer-Restaurant-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>NoOffer-Restaurant-Price-cheap&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>NoOffer-Restaurant-Food-indian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul></ul><li>Request-Restaurant-Price-?<ul><li>Occur Num: 119</li><li>NLU Output</li><ul><li>Request-Restaurant-Food-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>NoOffer-Restaurant-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>NoOffer-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Request-Restaurant-Area-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li></ul></ul><li>Inform-Restaurant-Food-indian<ul><li>Occur Num: 118</li><li>NLU Output</li><ul><li>Inform-Restaurant-Price-expensive&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>Inform-Restaurant-Choice-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Recommend-Restaurant-Food-indian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Inform-Restaurant-Area-east&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Inform-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul><li>NoOffer-Restaurant-Price-moderate<ul><li>Occur Num: 99</li><li>NLU Output</li><ul><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>Request-Restaurant-Food-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>NoOffer-Restaurant-Food-indian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Restaurant-Area-east&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></ul><li>NoOffer-Restaurant-Price-cheap<ul><li>Occur Num: 95</li><li>NLU Output</li><ul><li>NoOffer-Restaurant-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 15</li><li>NoOffer-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>Request-Restaurant-Food-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>Request-Restaurant-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>NoOffer-Restaurant-Area-east&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li></ul></ul><li>Inform-Restaurant-Price-cheap<ul><li>Occur Num: 94</li><li>NLU Output</li><ul><li>Inform-Restaurant-Area-centre&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>Inform-Restaurant-Food-italian&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li><li>Recommend-Restaurant-Price-cheap&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li><li>Request-Taxi-Arrive-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Recommend-Restaurant-Name-la raza&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li></ul></ul></ul><b> Dialog Loop</b><ul><li>Request-Restaurant-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 108</li><li>Request-Restaurant-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li><li>Request-Restaurant-Post-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 4</li><li>Inform-Restaurant-Day-saturday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Restaurant-People-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Restaurant-Time-14:15&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Restaurant-Day-sunday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Request-Restaurant-Price-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Restaurant-Time-16:45&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Restaurant-Time-13:30&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-restaurant-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 127</li><li>request-restaurant-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 41</li><li>request-restaurant-postcode&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>request-restaurant-pricerange&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li></ul><b> Inform But Not Request Dialog Act</b><ul><li>inform-restaurant-food&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 56</li><li>inform-restaurant-area&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 50</li><li>inform-restaurant-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li></ul></div><h3>Domain train</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 89.7 %</p><p> (Precision, Recall, F1) : (0.795, 0.865, 0.815) </p><p> Average Dialog Turn (Succ): 9.902 </p><p> Average Dialog Turn (All): 11.773 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><ul><li>OfferBook-Train-Depart-cambridge<ul><li>Occur Num: 712</li><li>NLU Output</li><ul><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 43</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 41</li><li>OfferBook-Train-Day-tuesday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 24</li><li>OfferBook-Train-Day-thursday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 22</li></ul></ul><li>OfferBook-Train-Dest-cambridge<ul><li>Occur Num: 641</li><li>NLU Output</li><ul><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 50</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 33</li><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 27</li><li>OfferBook-Train-Day-saturday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 26</li><li>OfferBook-Train-Day-tuesday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li></ul></ul><li>Inform-Train-Arrive-05:51<ul><li>Occur Num: 475</li><li>NLU Output</li><ul><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 67</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 37</li><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 31</li><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li></ul></ul><li>OfferBook-Train-Day-tuesday<ul><li>Occur Num: 256</li><li>NLU Output</li><ul><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 21</li><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li></ul></ul><li>OfferBook-Train-Day-saturday<ul><li>Occur Num: 254</li><li>NLU Output</li><ul><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 26</li><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 18</li><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 12</li><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul></ul><li>Inform-Train-Depart-cambridge<ul><li>Occur Num: 253</li><li>NLU Output</li><ul><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 32</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>OfferBook-Train-Day-tuesday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>OfferBook-Train-Day-thursday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li></ul></ul><li>Inform-Train-Leave-05:00<ul><li>Occur Num: 242</li><li>NLU Output</li><ul><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 37</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>OfferBook-Train-Day-saturday&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul></ul><li>OfferBook-Train-Day-friday<ul><li>Occur Num: 237</li><li>NLU Output</li><ul><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 19</li><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 11</li><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 7</li></ul></ul><li>OfferBook-Train-Day-thursday<ul><li>Occur Num: 233</li><li>NLU Output</li><ul><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 23</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>OfferBook-Train-none-none&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 16</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>OfferBook-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 5</li></ul></ul><li>Inform-Train-Time-51 minutes<ul><li>Occur Num: 194</li><li>NLU Output</li><ul><li>OfferBook-Train-Depart-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>OfferBook-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>Inform-Train-Time-one way&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 10</li><li>OfferBook-Train-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>OfferBook-Train-People-51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li></ul></ul></ul><b> Dialog Loop</b><ul><li>Request-Train-Time-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 9</li><li>Inform-Train-People-6&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Train-People-4&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Train-People-3&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Train-People-5&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-People-7&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-Depart-london liverpool street&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-Dest-cambridge&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-Dest-birmingham new street&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Request-Train-Leave-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Bad Inform Dialog Act</b><ul><li>offerbook-train-arriveBy&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 30</li><li>inform-train-duration&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 14</li><li>offerbook-train-leaveAt&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 8</li><li>offerbook-train-price&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>inform-train-price&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>offerbook-train-trainID&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Request But Not Inform Dialog Act</b><ul><li>request-train-duration&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 13</li><li>request-train-price&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>request-train-leaveAt&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>request-train-arriveBy&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>request-train-trainID&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain police</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 0.0 %</p><p> (Precision, Recall, F1) : (0.000, 0.000, 0.000) </p><p> Average Dialog Turn (Succ): 0.000 </p><p> Average Dialog Turn (All): 40.000 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Police-Addr-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li><li>Request-Police-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 6</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-police-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 20</li><li>request-police-address&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 17</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain hospital</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 0.0 %</p><p> (Precision, Recall, F1) : (0.000, 0.000, 0.000) </p><p> Average Dialog Turn (Succ): 0.000 </p><p> Average Dialog Turn (All): 40.000 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><p>Nothing</p><b> Dialog Loop</b><ul><li>Request-Hospital-Phone-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 30</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-hospital-phone&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 30</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div><h3>Domain taxi</h3><div style="margin-left:50"><h3>Overall Results</h3><p> Success Rate: 93.5 %</p><p> (Precision, Recall, F1) : (1.000, 0.967, 0.978) </p><p> Average Dialog Turn (Succ): 5.209 </p><p> Average Dialog Turn (All): 7.478 </p><b> System NLU Failed Dialog Act:</b><p>Nothing</p><b> User NLU Failed Dialog Act:</b><ul><li>Inform-Taxi-Car-skoda<ul><li>Occur Num: 27</li><li>NLU Output</li><ul><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Train-Time-51 minutes&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Taxi-Phone-62698234072&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Car-toyota&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-toyota<ul><li>Occur Num: 17</li><li>NLU Output</li><ul><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Taxi-Phone-62698234072&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-83153889561&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-69714552379&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-audi<ul><li>Occur Num: 16</li><li>NLU Output</li><ul><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Taxi-Phone-73297457818&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-02411795305&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>OfferBooked-Train-Time-51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-17927153736&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-ford<ul><li>Occur Num: 14</li><li>NLU Output</li><ul><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Taxi-Phone-94549609159&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-56629498460&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-44947488614&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-Time-51 minutes&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-honda<ul><li>Occur Num: 13</li><li>NLU Output</li><ul><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Taxi-Phone-52547806713&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-42223260114&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-84754151715&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-lexus<ul><li>Occur Num: 13</li><li>NLU Output</li><ul><li>Inform-Taxi-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 2</li><li>Inform-Taxi-Phone-83144288521&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-69714552379&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-19626794559&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-bmw<ul><li>Occur Num: 13</li><li>NLU Output</li><ul><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Taxi-Phone-79054617689&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-73609361703&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-49920327578&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-tesla<ul><li>Occur Num: 13</li><li>NLU Output</li><ul><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li><li>Inform-Taxi-Phone-00253869902&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-97488259243&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-36951803842&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-volkswagen<ul><li>Occur Num: 7</li><li>NLU Output</li><ul><li>Inform-Taxi-Phone-15214714504&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Arrive-05:51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-53827293134&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Hotel-Type-guesthouse&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-29737767268&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul><li>Inform-Taxi-Car-volvo<ul><li>Occur Num: 6</li><li>NLU Output</li><ul><li>Inform-Taxi-Phone-26438028145&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>OfferBooked-Train-Time-51&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-96368625434&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Train-Leave-05:00&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li><li>Inform-Taxi-Phone-10921657762&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 1</li></ul></ul></ul><b> Dialog Loop</b><ul><li>Request-Taxi-Car-?&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li></ul><b> Bad Inform Dialog Act</b><p>Nothing</p><b> Request But Not Inform Dialog Act</b><ul><li>request-taxi-car type&nbsp;&nbsp;&nbsp;&nbsp;Occur Num: 3</li></ul><b> Inform But Not Request Dialog Act</b><p>Nothing</p></div></body></html>
\ No newline at end of file
tests/results/TRADE-RulePolicy-TemplateNLG/res.txt deleted 100755 → 0
+ 0
− 8
View file @ 8e8eba91
complete number of dialogs/tot: 0.299
success number of dialogs/tot: 0.253
average precision: 0.49259671331322424
average recall: 0.48095032059206516
average f1: 0.4437872685208314
average book rate: 0.3691452991452991
average turn (succ): 12.735177865612648
average turn (all): 24.684
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment