diff --git a/Docs/Acknowledgements.rst b/Docs/Acknowledgements.rst new file mode 100644 index 0000000000000000000000000000000000000000..b6e37a50852d196255e9794b39ab70ffbfdd4cb0 --- /dev/null +++ b/Docs/Acknowledgements.rst @@ -0,0 +1,18 @@ +**************** +Acknowledgements +**************** + +The following current and former group members have contributed to PyDial (in alphabetical order): + +- Paweł Budzianowski +- Iñigo Casanueva +- Milica Gašić +- Dongho Kim +- Nikola Mrkšić +- Lina Rojas-Barahona +- Stefan Ultes +- Pei-Hao Su +- David Vandyke +- Gellert Weisz +- Tsung-Hsien Wen +- Steve Young diff --git a/Docs/Agent.rst b/Docs/Agent.rst new file mode 100644 index 0000000000000000000000000000000000000000..fc6be9953381df811dac0717359d9c845c6bd62e --- /dev/null +++ b/Docs/Agent.rst @@ -0,0 +1,13 @@ +Agent - the Complete Dialogue System +************************************ + + +.. |.| raw:: html + + <br /> + + +.. automodule:: Agent + :members: + + diff --git a/Docs/BeliefTracking.rst b/Docs/BeliefTracking.rst new file mode 100644 index 0000000000000000000000000000000000000000..c2d139a9092cd471ee5671963da2058b61e9e596 --- /dev/null +++ b/Docs/BeliefTracking.rst @@ -0,0 +1,23 @@ +Belief Tracking +*********************** + +.. |.| raw:: html + + <br /> + +.. automodule:: belieftracking.BeliefTrackingManager + :members: + :private-members: + +.. automodule:: belieftracking.BeliefTracker + :members: + :private-members: + +.. automodule:: belieftracking.baseline +.. autoclass:: belieftracking.baseline.BaselineTracker +.. autoclass:: belieftracking.baseline.FocusTracker + +.. automodule:: belieftracking.BeliefTrackingUtils + :members: + + diff --git a/Docs/CEDM.rst b/Docs/CEDM.rst new file mode 100644 index 0000000000000000000000000000000000000000..1ad6cb554f5487440e01cd1e80a6dcf9486a2e4e --- /dev/null +++ b/Docs/CEDM.rst @@ -0,0 +1,23 @@ +Conversational Entity Dialogue Model +************************************ + +The Conversational Entity Dialogue Model (CEDM) aims at resolving restrictions of conventional single- or multi-domain dialogue models in their capabilities of modelling complex dialogue structures, e.g., relations. The CEDM is a novel dialogue model that is centred around entities and is able to model relations as well as multiple entities of the same type `(Ultes et al., 2018) +<http://www.pydial.org/cedm>`_. A prototype of the CEDM has been implemented and integrated into the `PyDial <http://www.pydial.org/>`_ toolkit. + +Please note that the CEDM prototype implementation has been designed in a way to exploit structures and implementations of PyDial where possible. Thus, some restrictions needed to be posed on the implementation which need to be resolved in future versions. + + +Usage Instructions +------------------ + +The CEDM is disabled by default. To activate and use it, download the latest version of PyDial. From Pydial root, run the activation shell script + +.. code-block:: shell + + python cedm/activation/activate.py + +which will copy all necessary files to their respective locations. You will now find a SimulateCEDM.py in the root folder of PyDial which you can use to run experiments with the CEDM. To test it, run + +.. code-block:: shell + + python SimulateCEDM.py -c config/cedm_example.cfg diff --git a/Docs/Evaluation.rst b/Docs/Evaluation.rst new file mode 100644 index 0000000000000000000000000000000000000000..c51edde621c352e4fc04cbc5d26593519a23c5a2 --- /dev/null +++ b/Docs/Evaluation.rst @@ -0,0 +1,18 @@ +Evaluation +*************** + +.. |.| raw:: html + + <br /> + + +.. automodule:: evaluation.EvaluationManager + :members: + :private-members: + +.. automodule:: evaluation.SuccessEvaluator +.. autoclass:: evaluation.SuccessEvaluator.ObjectiveSuccessEvaluator +.. autoclass:: evaluation.SuccessEvaluator.SubjectiveSuccessEvaluator + + + diff --git a/Docs/NLGeneration.rst b/Docs/NLGeneration.rst new file mode 100644 index 0000000000000000000000000000000000000000..049aa00dfb771e8b78e44821d0ab7d618cf92521 --- /dev/null +++ b/Docs/NLGeneration.rst @@ -0,0 +1,23 @@ +Natural Language Generation +*************************** + +.. |.| raw:: html + + <br /> + +.. automodule:: semo.SemOManager + :members: + :private-members: + +.. automodule:: semo.RuleSemOMethods +.. autoclass:: semo.RuleSemOMethods.PassthroughSemO +.. autoclass:: semo.RuleSemOMethods.BasicSemO +.. autoclass:: semo.RuleSemOMethods.BasicTemplateRule +.. autoclass:: semo.RuleSemOMethods.BasicTemplateFunction +.. autoclass:: semo.RuleSemOMethods.BasicTemplateFunctionRule +.. autoclass:: semo.RuleSemOMethods.BasicTemplateGenerator + +.. automodule:: semo.RNNSemOMethods +.. autoclass:: semo.RNNSemOMethods.RNNSemO + + diff --git a/Docs/Ontology.rst b/Docs/Ontology.rst new file mode 100644 index 0000000000000000000000000000000000000000..3c74ec78a55e6e3f81d75583cddfcfe2545c56ee --- /dev/null +++ b/Docs/Ontology.rst @@ -0,0 +1,20 @@ +Ontology +********** + +.. |.| raw:: html + + <br /> + + +.. automodule:: ontology.DataBase + :members: + +.. automodule:: ontology.FlatOntologyManager + :members: + +.. automodule:: ontology.Ontology + :members: + +.. automodule:: ontology.OntologyUtils + :members: + diff --git a/Docs/Policy.rst b/Docs/Policy.rst new file mode 100644 index 0000000000000000000000000000000000000000..e9154d32a81b18d1d3d4b052bf44580964229e7a --- /dev/null +++ b/Docs/Policy.rst @@ -0,0 +1,77 @@ +Policy +********************* + +.. |.| raw:: html + + <br /> + +.. automodule:: policy.Policy + :members: + :private-members: + +.. automodule:: policy.PolicyManager + :members: + :private-members: + +.. automodule:: policy.PolicyCommittee + :members: + :private-members: + +.. automodule:: policy.HDCPolicy +.. autoclass:: policy.HDCPolicy.HDCPolicy + +.. automodule:: policy.GPPolicy +.. autoclass:: policy.GPPolicy.GPPolicy +.. autoclass:: policy.GPPolicy.Kernel +.. autoclass:: policy.GPPolicy.GPAction +.. autoclass:: policy.GPPolicy.GPState +.. autoclass:: policy.GPPolicy.TerminalGPAction +.. autoclass:: policy.GPPolicy.TerminalGPState + +.. automodule:: policy.GPLib +.. autoclass:: policy.GPLib.GPSARSA +.. autoclass:: policy.GPLib.GPSARSAPrior +.. autoclass:: policy.GPLib.LearnerInterface + +.. automodule:: policy.HDCTopicManager +.. autoclass:: policy.HDCTopicManager.HDCTopicManagerPolicy + +.. automodule:: policy.WikipediaTools +.. autoclass:: policy.WikipediaTools.WikipediaDM + +.. automodule:: policy.SummaryAction +.. autoclass:: policy.SummaryAction.SummaryAction + +.. automodule:: policy.SummaryUtils + +.. automodule:: policy.PolicyUtils + :members: + +.. automodule:: policy.BCM_Tools + +DeepRL Policies +********************* + +.. automodule:: policy.A2CPolicy + +.. automodule:: policy.ACERPolicy + +.. automodule:: policy.BDQNPolicy + +.. automodule:: policy.DQNPolicy + +.. automodule:: policy.ENACPolicy + +.. automodule:: policy.TRACERPolicy + +FeudalRL Policies +********************* + +Traditional Reinforcement Learning algorithms fail to scale to large domains due to the curse of dimensionality. A novel Dialogue Management architecture based on Feudal RL decomposes the decision into two steps; a first step where a master policy selects a subset of primitive actions, and a second step where a primitive action is chosen from the selected subset. The structural information included in the domain ontology is used to abstract the dialogue state space, taking the decisions at each step using different +parts of the abstracted state. This, combined with an information sharing mechanism between slots, increases the scalability to large +domains. + +For more information, please look at the paper `Feudal Reinforcement Learning for Dialogue Management in Large Domains <https://arxiv.org/pdf/1803.03232.pdf>`_. + + + \ No newline at end of file diff --git a/Docs/SemanticBeliefTracking.rst b/Docs/SemanticBeliefTracking.rst new file mode 100644 index 0000000000000000000000000000000000000000..f329a9ce15fed303e75f13b2790c279824bd5455 --- /dev/null +++ b/Docs/SemanticBeliefTracking.rst @@ -0,0 +1,13 @@ +Semantic Belief Tracking +************************ + +.. |.| raw:: html + + <br /> + +.. automodule:: semanticbelieftracking.SemanticBeliefTrackingManager + :members: + :private-members: + +.. automodule:: semanticbelieftracking.ModularSemanticBeliefTracker +.. autoclass:: semanticbelieftracking.ModularSemanticBeliefTracker.ModularSemanticBeliefTracker diff --git a/Docs/SemanticParsing.rst b/Docs/SemanticParsing.rst new file mode 100644 index 0000000000000000000000000000000000000000..165bf41475f900fef35b09178ea0fe86139957c6 --- /dev/null +++ b/Docs/SemanticParsing.rst @@ -0,0 +1,18 @@ +Semantic Parsing +********************* + +.. |.| raw:: html + + <br /> + + +.. automodule:: semi.SemI + :members: + :private-members: + +.. automodule:: semi.RuleSemIMethods +.. autoclass:: semi.RuleSemIMethods.PassthroughSemI +.. autoclass:: semi.RuleSemIMethods.RegexSemI + +.. automodule:: semi.RegexSemI +.. autoclass:: semi.RegexSemI.RegexSemI diff --git a/Docs/Simulation.rst b/Docs/Simulation.rst new file mode 100644 index 0000000000000000000000000000000000000000..d6e92611ec5e6cb086ee306b305704786bb6c4cf --- /dev/null +++ b/Docs/Simulation.rst @@ -0,0 +1,29 @@ +Simulation +*************** + +.. |.| raw:: html + + <br /> + + +.. automodule:: usersimulator.SimulatedUsersManager + :members: + +.. automodule:: usersimulator.ConfidenceScorer + :members: + +.. automodule:: usersimulator.ConfusionModel + :members: + +.. automodule:: usersimulator.ErrorModel + :members: + +.. automodule:: usersimulator.NBestGenerator + :members: + +.. automodule:: usersimulator.UserModel + :members: + +.. automodule:: usersimulator.UMHdcSim + :members: + diff --git a/Docs/Systems.rst b/Docs/Systems.rst new file mode 100644 index 0000000000000000000000000000000000000000..f021417c65037cda8cb561dd3f6992446a726564 --- /dev/null +++ b/Docs/Systems.rst @@ -0,0 +1,17 @@ +Dialogue Systems +********************* + +.. |.| raw:: html + + <br /> + +.. automodule:: DialogueServer + :members: + +.. automodule:: Simulate + :members: + +.. automodule:: Texthub + :members: + + diff --git a/Docs/Tests.rst b/Docs/Tests.rst new file mode 100644 index 0000000000000000000000000000000000000000..bac4d64a96d44f2c8cbeaee72d3bb2e9a26706d1 --- /dev/null +++ b/Docs/Tests.rst @@ -0,0 +1,23 @@ +Functional Tests +*************************** + +.. |.| raw:: html + + <br /> + +.. include:: ../tests/README.rst + + +List of Current Tests +====================== + + +.. automodule:: tests.test_DialogueServer + :members: + +.. automodule:: tests.test_Simulate + :members: + +.. automodule:: tests.test_Tasks + :members: + \ No newline at end of file diff --git a/Docs/TopicTracking.rst b/Docs/TopicTracking.rst new file mode 100644 index 0000000000000000000000000000000000000000..b83adbb3e4bd91cf8664a61b981849cfec4872fb --- /dev/null +++ b/Docs/TopicTracking.rst @@ -0,0 +1,17 @@ +Topic Tracking +*************** + +.. |.| raw:: html + + <br /> + + +.. automodule:: topictracking.TopicTracking + :members: + :private-members: + +.. automodule:: topictracking.RuleTopicTrackers + :members: + + + diff --git a/Docs/Utilities.rst b/Docs/Utilities.rst new file mode 100644 index 0000000000000000000000000000000000000000..284337dc2e83aa8dc95880c474a6fc34f384088c --- /dev/null +++ b/Docs/Utilities.rst @@ -0,0 +1,31 @@ +Utility Modules +*************** + +.. |.| raw:: html + + <br /> + + +.. automodule:: utils.ContextLogger + :members: + +.. automodule:: utils.dact + :members: + + +.. automodule:: utils.DiaAct + :members: + +.. automodule:: utils.DialogueState + :members: + +.. automodule:: utils.dummyDialogueServerClient + :members: + +.. automodule:: utils.Scanner + :members: + +.. automodule:: utils.Settings + :members: + + diff --git a/Docs/cued_logo.png b/Docs/cued_logo.png new file mode 100644 index 0000000000000000000000000000000000000000..437e4e84dc18a6797a2c8e500f442073b923aa2c Binary files /dev/null and b/Docs/cued_logo.png differ diff --git a/Docs/index.rst b/Docs/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..b95cf34b485a4e86a5e864123ad4b75019a56387 --- /dev/null +++ b/Docs/index.rst @@ -0,0 +1,75 @@ +.. PyDial documentation master file, created by + sphinx-quickstart on Mon Aug 10 14:07:56 2015. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +PyDial: the CUED Python Statistical Dialog System +************************************************* + + + +Tutorials +==================== + +To get familiar with usage of PyDial, you can use the following tutorials: + +- `Introduction to PyDial <tutorials/Introduction%20to%20PyDial.html>`_ +- `Policy module <tutorials/Policy%20module.html>`_ +- `Spoken Language Understanding <tutorials/Spoken%20Language%20Understanding.html>`_ +- `Natural Language Generation <tutorials/Natural%20Language%20Generation%20Module.html>`_ +- `How to add your own module <tutorials/How%20to%20add%20your%20own%20module.html>`_ +- `How to add a new domain <tutorials/How%20to%20add%20a%20new%20domain.html>`_ +- `Deep Reinforcement Learning Policies <tutorials/Deep%20RL%20Policies.html>`_ + +System Documentation +==================== +.. toctree:: + :maxdepth: 2 + + Agent + SemanticBeliefTracking + SemanticParsing + BeliefTracking + Policy + NLGeneration + Evaluation + TopicTracking + +-------------------- + +.. toctree:: + :maxdepth: 1 + + CEDM + +-------------------- + +.. toctree:: + :maxdepth: 2 + + Systems + Simulation + +-------------------- + +.. toctree:: + :maxdepth: 2 + + Ontology + Tests + Utilities + +-------------------- + +.. toctree:: + :maxdepth: 1 + + Acknowledgements + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` + diff --git a/Docs/updateLocalDocs.sh b/Docs/updateLocalDocs.sh new file mode 100644 index 0000000000000000000000000000000000000000..2f376d179846359f89f13307b793a26fcc92bc52 --- /dev/null +++ b/Docs/updateLocalDocs.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +cp -r ../_build /tools/WorldWideWeb/old/research/dialogue/LocalDocs/cued-python-doc/