diff --git a/.github/PULL_REQUEST_TEMPLATE/add_datasets.md b/.github/ISSUE_TEMPLATE/add_dataset.md similarity index 89% rename from .github/PULL_REQUEST_TEMPLATE/add_datasets.md rename to .github/ISSUE_TEMPLATE/add_dataset.md index dfb3ea10fe1c3604af9d8e9c2e83fb95c01bf146..71265549a13442d9ca30b98c6a58903a19219d8e 100644 --- a/.github/PULL_REQUEST_TEMPLATE/add_datasets.md +++ b/.github/ISSUE_TEMPLATE/add_dataset.md @@ -1,3 +1,12 @@ +--- +name: Add dataset +about: Add a new dataset in unified format +title: "[Dataset] " +labels: dataset, new +assignees: '' + +--- + - **Name:** *name of the dataset* - **Description:** *short description of the dataset (or link to social media or blog post)* - **Paper:** *link to the dataset paper if available* diff --git a/.github/PULL_REQUEST_TEMPLATE/add_model.md b/.github/ISSUE_TEMPLATE/add_model.md similarity index 86% rename from .github/PULL_REQUEST_TEMPLATE/add_model.md rename to .github/ISSUE_TEMPLATE/add_model.md index 387b02441605ff07c4e9c27e4b7e2f1989d1361e..cdec9801bcbdd6ebea53f93c52f4016f1dae024b 100644 --- a/.github/PULL_REQUEST_TEMPLATE/add_model.md +++ b/.github/ISSUE_TEMPLATE/add_model.md @@ -1,3 +1,12 @@ +--- +name: Add model +about: Add a new model in unified format +title: "[Model] " +labels: model, new +assignees: '' + +--- + - **Name:** *name of the model* - **Description:** *short description of the model (or link to social media or blog post)* - **Paper:** *link to the model paper if available* diff --git a/.github/PULL_REQUEST_TEMPLATE/custom.md b/.github/PULL_REQUEST_TEMPLATE.md old mode 100755 new mode 100644 similarity index 100% rename from .github/PULL_REQUEST_TEMPLATE/custom.md rename to .github/PULL_REQUEST_TEMPLATE.md diff --git a/.github/PULL_REQUEST_TEMPLATE/fix_bug.md b/.github/PULL_REQUEST_TEMPLATE/fix_bug.md deleted file mode 100755 index 719d39e99e66d74511ddddeb83204d7f8d9fee8b..0000000000000000000000000000000000000000 --- a/.github/PULL_REQUEST_TEMPLATE/fix_bug.md +++ /dev/null @@ -1,23 +0,0 @@ -**Describe the bug** -A clear and concise description of what the bug is. - -**To Reproduce** -Steps to reproduce the behavior: -1. Go to '...' -2. Click on '....' -3. Scroll down to '....' -4. See error - -**Expected behavior** -A clear and concise description of what you expected to happen. - -**Actual behavior** -A clear and concise description of the actual results. - -**Additional context** -Add any other context about the problem here. - -**How to fix** -Changes made to fix the bug. - -**Reference Issues:** #XX (XX is the issue number you work on) diff --git a/README.md b/README.md index 27c77dfad5ab641476b70d9e8383e9d15b01403b..b4b22e5f79b8bcd3611a081c3b3bd221f3ac2189 100755 --- a/README.md +++ b/README.md @@ -65,6 +65,7 @@ docker exec -it CONTAINER_ID bash ## Tutorials +- [Getting Started](https://github.com/ConvLab/ConvLab-3/blob/master/tutorials/Getting_Started.ipynb) (Have a try on [Colab](https://colab.research.google.com/github/ConvLab/ConvLab-3/blob/master/tutorials/Getting_Started.ipynb)!) - [Introduction to Unified Data Format](https://github.com/ConvLab/ConvLab-3/tree/master/data/unified_datasets) - [Utility functions for unified datasets](https://github.com/ConvLab/ConvLab-3/blob/master/convlab/util/unified_datasets_util.py) - [RL Toolkit](https://github.com/ConvLab/ConvLab-3/tree/master/convlab/policy) diff --git a/tutorials/Getting_Started.ipynb b/tutorials/Getting_Started.ipynb index a4b28edc9fbbc06e7ac354b84d9948fb04f0d469..5d32d4e1f810ef30a30cd74f77d20c6b6ba3fb4c 100644 --- a/tutorials/Getting_Started.ipynb +++ b/tutorials/Getting_Started.ipynb @@ -1,31 +1,16 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "colab": { - "name": "Tutorial", - "provenance": [], - "collapsed_sections": [], - "toc_visible": true - }, - "kernelspec": { - "name": "python3", - "display_name": "Python 3", - "language": "python" - } - }, "cells": [ { "cell_type": "markdown", "metadata": { - "id": "x3qZ8fi1FAWS", - "colab_type": "text" + "colab_type": "text", + "id": "x3qZ8fi1FAWS" }, "source": [ "# Getting Started\n", "\n", "In this tutorial, you will know how to\n", - "- use the models in **ConvLab-2** to build a dialog agent.\n", + "- use the models in **ConvLab-3** to build a dialog agent.\n", "- build a simulator to chat with the agent and evaluate the performance.\n", "- try different module combinations.\n", "- use analysis tool to diagnose your system.\n", @@ -36,27 +21,27 @@ { "cell_type": "markdown", "metadata": { - "id": "NvXG3UyBFFS3", - "colab_type": "text" + "colab_type": "text", + "id": "NvXG3UyBFFS3" }, "source": [ "## Environment setup\n", - "Run the command below to install ConvLab-2. Then restart the notebook and skip this commend." + "Run the command below to install ConvLab-3. Then restart the notebook and skip this commend." ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "qPqTs6EP0wxO", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "qPqTs6EP0wxO" }, + "outputs": [], "source": [ - "# first install ConvLab-2 and restart the notebook\n", - "! git clone https://github.com/thu-coai/ConvLab-2.git && cd ConvLab-2 && pip install -e ." - ], - "execution_count": null, - "outputs": [] + "# first install ConvLab-3 and restart the notebook\n", + "! git clone --depth 1 https://github.com/ConvLab/ConvLab-3.git && cd ConvLab-3 && pip install -e ." + ] }, { "cell_type": "code", @@ -68,18 +53,11 @@ "!python -m spacy download en_core_web_sm" ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - }, { "cell_type": "markdown", "metadata": { - "id": "1TPEtcEpFiCU", - "colab_type": "text" + "colab_type": "text", + "id": "1TPEtcEpFiCU" }, "source": [ "## build an agent\n", @@ -91,11 +69,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "OqzFlmCBCXEV", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "OqzFlmCBCXEV" }, + "outputs": [], "source": [ "# common import: convlab.$module.$model.$dataset\n", "from convlab.nlu.jointBERT.multiwoz import BERTNLU\n", @@ -109,15 +89,13 @@ "import random\n", "import numpy as np\n", "import torch" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "N-18Q6YKGEzY", - "colab_type": "text" + "colab_type": "text", + "id": "N-18Q6YKGEzY" }, "source": [ "Then, create the models and build an agent:" @@ -125,11 +103,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "-GPQgUD6GjHv", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "-GPQgUD6GjHv" }, + "outputs": [], "source": [ "# go to README.md of each model for more information\n", "# BERT nlu\n", @@ -142,15 +122,13 @@ "sys_nlg = TemplateNLG(is_user=False)\n", "# assemble\n", "sys_agent = PipelineAgent(sys_nlu, sys_dst, sys_policy, sys_nlg, name='sys')" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "FvqIiO22H0Od", - "colab_type": "text" + "colab_type": "text", + "id": "FvqIiO22H0Od" }, "source": [ "That's all! Let's chat with the agent using its response function:" @@ -158,100 +136,100 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "9LYnDLysH1nX", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "9LYnDLysH1nX" }, + "outputs": [], "source": [ "sys_agent.response(\"I want to find a moderate hotel\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "dAvi-IzsH474", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "dAvi-IzsH474" }, + "outputs": [], "source": [ "sys_agent.response(\"Which type of hotel is it ?\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "f0A6Pt4tH60c", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "f0A6Pt4tH60c" }, + "outputs": [], "source": [ "sys_agent.response(\"OK , where is its address ?\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "8VH6yO6PH8Ua", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "8VH6yO6PH8Ua" }, + "outputs": [], "source": [ "sys_agent.response(\"Thank you !\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "wto-PxgNH9gp", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "wto-PxgNH9gp" }, + "outputs": [], "source": [ "sys_agent.response(\"Try to find me a Chinese restaurant in south area .\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "MJ9C1ZhbH_JY", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "MJ9C1ZhbH_JY" }, + "outputs": [], "source": [ "sys_agent.response(\"Which kind of food it provides ?\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "DpaQNA_HIAmT", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "DpaQNA_HIAmT" }, + "outputs": [], "source": [ "sys_agent.response(\"Book a table for 5 , this Sunday .\")" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "n6uuuRonIHvW", - "colab_type": "text" + "colab_type": "text", + "id": "n6uuuRonIHvW" }, "source": [ "## Build a simulator to chat with the agent and evaluate\n", @@ -263,11 +241,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "pAMAJZSF7D5w", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "pAMAJZSF7D5w" }, + "outputs": [], "source": [ "# MILU\n", "user_nlu = MILU()\n", @@ -279,15 +259,13 @@ "user_nlg = TemplateNLG(is_user=True)\n", "# assemble\n", "user_agent = PipelineAgent(user_nlu, user_dst, user_policy, user_nlg, name='user')" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "Rghl-V2AJhRY", - "colab_type": "text" + "colab_type": "text", + "id": "Rghl-V2AJhRY" }, "source": [ "\n", @@ -298,23 +276,23 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "saUoLUUGJqDa", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "saUoLUUGJqDa" }, + "outputs": [], "source": [ "evaluator = MultiWozEvaluator()\n", "sess = BiSession(sys_agent=sys_agent, user_agent=user_agent, kb_query=None, evaluator=evaluator)" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "kevGJZhFJzTU", - "colab_type": "text" + "colab_type": "text", + "id": "kevGJZhFJzTU" }, "source": [ "Let's make this two agents chat! The key is `next_turn` method of `BiSession` class." @@ -322,11 +300,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "FIV_qkE49LzE", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "FIV_qkE49LzE" }, + "outputs": [], "source": [ "def set_seed(r_seed):\n", " random.seed(r_seed)\n", @@ -354,15 +334,13 @@ "print('final goal:')\n", "pprint(sess.evaluator.goal)\n", "print('='*100)" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "CKOQs1l8LpTR", - "colab_type": "text" + "colab_type": "text", + "id": "CKOQs1l8LpTR" }, "source": [ "## Try different module combinations\n", @@ -383,11 +361,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "G-9G0VYUNYOI", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "G-9G0VYUNYOI" }, + "outputs": [], "source": [ "# available NLU models\n", "from convlab.nlu.svm.multiwoz import SVMNLU\n", @@ -413,15 +393,13 @@ "# available E2E models\n", "from convlab.e2e.sequicity.multiwoz import Sequicity\n", "from convlab.e2e.damd.multiwoz import Damd" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "6TS2_Tp1Nzvq", - "colab_type": "text" + "colab_type": "text", + "id": "6TS2_Tp1Nzvq" }, "source": [ "NLU+RuleDST or Word-DST:" @@ -429,11 +407,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "DZMk9wAlONrz", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "DZMk9wAlONrz" }, + "outputs": [], "source": [ "# NLU+RuleDST:\n", "sys_nlu = BERTNLU()\n", @@ -445,15 +425,13 @@ "# sys_nlu = None\n", "# sys_dst = SUMBT()\n", "# sys_dst = TRADE()" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "gUUYsDMJPJRl", - "colab_type": "text" + "colab_type": "text", + "id": "gUUYsDMJPJRl" }, "source": [ "Policy+NLG or Word-Policy:" @@ -461,11 +439,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "PTJ95x9UPHt4", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "PTJ95x9UPHt4" }, + "outputs": [], "source": [ "# Policy+NLG:\n", "sys_policy = RulePolicy()\n", @@ -481,15 +461,13 @@ "# sys_policy = HDSA()\n", "# sys_policy = MDRGWordPolicy()\n", "# sys_nlg = None" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "s9lGIv0oPupn", - "colab_type": "text" + "colab_type": "text", + "id": "s9lGIv0oPupn" }, "source": [ "Assemble the Pipeline system agent:" @@ -497,22 +475,22 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "IvLx7HUkPyZ5", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "IvLx7HUkPyZ5" }, + "outputs": [], "source": [ "sys_agent = PipelineAgent(sys_nlu, sys_dst, sys_policy, sys_nlg, 'sys')" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "hR4A8WbZP2lc", - "colab_type": "text" + "colab_type": "text", + "id": "hR4A8WbZP2lc" }, "source": [ "Or Directly use an end-to-end model:" @@ -520,23 +498,23 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "8VdUmcxoP6ej", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "8VdUmcxoP6ej" }, + "outputs": [], "source": [ "# sys_agent = Sequicity()\n", "# sys_agent = Damd()" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "_v-eoBtnP9J9", - "colab_type": "text" + "colab_type": "text", + "id": "_v-eoBtnP9J9" }, "source": [ "Config an user agent similarly:" @@ -544,11 +522,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "UkHpjvR5QezN", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "UkHpjvR5QezN" }, + "outputs": [], "source": [ "user_nlu = BERTNLU()\n", "# user_nlu = MILU()\n", @@ -559,15 +539,13 @@ "user_nlg = TemplateNLG(is_user=True)\n", "# user_nlg = SCLSTM(is_user=True)\n", "user_agent = PipelineAgent(user_nlu, user_dst, user_policy, user_nlg, name='user')" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "VJTBxEDhSAqc", - "colab_type": "text" + "colab_type": "text", + "id": "VJTBxEDhSAqc" }, "source": [ "## Use analysis tool to diagnose the system\n", @@ -578,11 +556,13 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "Hu2q3lQiSMDy", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "Hu2q3lQiSMDy" }, + "outputs": [], "source": [ "from convlab.util.analysis_tool.analyzer import Analyzer\n", "\n", @@ -591,15 +571,13 @@ "\n", "set_seed(20200131)\n", "analyzer.comprehensive_analyze(sys_agent=sys_agent, model_name='sys_agent', total_dialog=100)" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", "metadata": { - "id": "AB-mDm0plQWd", - "colab_type": "text" + "colab_type": "text", + "id": "AB-mDm0plQWd" }, "source": [ "To compare several models:" @@ -607,17 +585,41 @@ }, { "cell_type": "code", + "execution_count": null, "metadata": { - "id": "GKe_DNJUlWzh", + "colab": {}, "colab_type": "code", - "colab": {} + "id": "GKe_DNJUlWzh" }, + "outputs": [], "source": [ "set_seed(20200131)\n", "analyzer.compare_models(agent_list=[sys_agent1, sys_agent2], model_name=['sys_agent1', 'sys_agent2'], total_dialog=100)" - ], - "execution_count": null, - "outputs": [] + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "Tutorial", + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3.6.9 64-bit", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.6.9" + }, + "vscode": { + "interpreter": { + "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" + } } - ] -} \ No newline at end of file + }, + "nbformat": 4, + "nbformat_minor": 0 +}