The MUSE project introduces a new way of exploring and understanding textual information by "bringing text to lift" through 3D interactive storytelling. Taking as input natural language text like children's stories or medical patient education materials, MUSE processes natural language, translate it into formal knowledge that represents the actions, actors, plots and surrounding world by means of advanced machine learning techniques, and then render these as virtual 3D worlds in which the user can explore through interaction, re-enactment and guided game play.
MUSE's long-term goal is to enable the innovative text-to-virtual-world translation system to be used in many applications and a variety of domains. Current information communicated to citizens is often very complex making written text difficult to understand. Comparable to the invention of symbolic writing systems several millennia ago, MUSE contributes to a novel symbolic system of communicating natural language utterances.
LIIR is involved in the coordination of the project and in the advanced natural language processing research.
In the frame of the EU projects MUSE and TERENCE we have built an integrated system for semantic role labeling, noun phrase coreference resolution, event classification and temporal relation extraction (see Story Annotation Service). We have developed a novel, scalable framework for jointly learning a model for knowledge extraction from natural language text the validity of which has been proven for extraction of relations, their composing entities and attributes. We have built a novel model for mapping text and its semantic annotations to the knowledge representation used to construct the virtual world. In the framework of the MUSE project we have organized the SemEval 2015 task of SpaceEval and we are involved in the organization of the Workshop on Replicability and Reusability in Natural Language Processing: From Data to Software Sharing at the 2015 International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina on July 27, 2015. The MUSE technologies will be demonstrated at ICT 2015 Innovate, Connect, Transform in Lisbon, Portugal on October 20-22 2015.