Machine Understanding for interactive StorytElling (EU ICT FP7 FET)


The MUSE project introduces a new way of exploring and understanding textual information by “bringing text to life” through 3D interactive storytelling. Taking as input natural language text like children’s stories or medical patient education materials, MUSE will process the natural language, translate it into formal knowledge that represents the actions, actors, plots and surrounding world by means of advanced machine learning techniques, and then render these as virtual 3D worlds in which the user can explore through interaction, re-enactment and guided game play.

MUSE’s long-term goal is to enable the innovative text-to-virtual-world translation system to be used in many applications and a variety of domains. Current information communicated to citizens is often very complex making written text difficult to understand. Comparable to the invention of symbolic writing systems several millennia ago, MUSE contributes to a novel symbolic system of communicating natural language utterances.

LIIR is involved in the coordination of the project and in the advanced natural language processing research.


LIIR coordinates the MUSE project. KU Leuven co-promotors are Prof. Steven Bethard (partially affiliated with the University of Boulder, USA) and Prof. Johan De Tavernier. The other partners are: Jožef Stefan Institute (Prof. Nada Lavrac), Teesside University (Prof. Marc Cavazza), Leiden University (Prof. Paul van den Broek) and Haute Autorité de Santé (Dr. Gersende Georg).

Results (Demo)

In the frame of the EU projects MUSE and TERENCE we have built an integrated system for semantic role labeling, noun phrase coreference resolution, event classification and temporal relation extraction (see Story Annotation Service).

Period From 2012-09-01 to 2015-08-31.
Financed by EU FP7-296703 (FET-open call).
Supervised by Marie-Francine Moens
Staff Parisa Kordjamshidi
Steven Bethard
Quynh Do Thi Ngoc
Oleksandr Kolomiyets
Oswaldo Ludwig
Contact Marie-Francine Moens

More information can be found on the project website


  1. KOLOMIYETS, Oleksandr & MOENS, Marie-Francine MotionML: Motion Markup Language: Shallow Approach for Annotating Motions in Text. In Proceedings of Corpus Linguistics. 2013
  2. KOLOMIYETS, Oleksandr, KORDJAMSHIDI, Parisa, BETHARD, Steven & MOENS, Maie-Francine SemEval-2013 Task 3: Spatial Role Labeling. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013) (pp. 255-262). ACL. 2013
  3. KOLOMIYETS, Oleksandr & MOENS, Moens Marie-Francine KUL: A Data-Driven Approach to Temporal Parsing of Documents. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013) (pp. 83-87). ACL. 2013
  4. KOLOMIYETS, O. & MOENS, M.-F. Towards Animated Visualization of Actors and Actions in a Learning Environment. In Proceedings of the 3rd International Workshop on Evidence Based and User centred Technology Enhanced Learning (EbuTEL 2013) (Advances in Intelligent and Soft-Computing) (in press). Springer. 2014
  5. DE MULDER, Wim, DO THI, Ngoc Quynh, VAN DEN BROEK, Paul & MOENS, Marie-Francine Machine Understanding for Interactive Storytelling. In Proceedings of the 8th International Conference on Knowledge, Information and Creativity Support Systems (KICSS'2013) (pp. 73-80). 2013
  6. KORDJAMSHIDI, Parisa & MOENS, Marie-Francine Designing Constructive Machine Learning Models based on Generalized Linear Learning Techniques. In Proceedings of the NIPS Workshop on Constructive Machine Learning. 2013
  7. KORDJAMSHIDI, Parisa & MOENS, Marie-Francine Structured Learning for Mapping Natural Language to Spatial Calculi Models. Abstract published in Proceedings of the NIPS Workshop: Knowledge Extraction from Text (KET). 2013

Back to all projects