June 2015 - June 2018

Project reference: 653590
Funded under: H2020-EU.3.7. - Secure societies - protecting freedom and security of Europe and its citizens

Automated Serious Game Scenario Generator for Mixed Reality Training


Click here to see FSEG YouTube channel playlist featuring presentation and demonstration from project AUGGMED.


Terrorism is a major threat to human life and critical infrastructure in Europe. The intruders continuously devise new strategies and execute highly sophisticated plans that are increasingly unfamiliar to the security forces and therefore more difficult to predict, prepare and defend against. In principle, in order to prepare for terrorist acts or critical incidents, security forces are trained using traditional methods such as live scenario training though full-scale field exercises. This type of training can be very costly, time-consuming and dangerous to those participating. In addition, it can only offer a limited number of scenarios that individually require major resources to plan and carryout. Thus, limiting the amount and variety of situations for which security forces are currently trained and making it difficult to maintain uniform levels of skill, judgement and preparedness across teams and individuals.
Virtual Training Environments (VTEs) and Serious Games (SGs) provide the opportunity to prepare security forces for events that cannot be, or are very difficult to train for using live scenario exercises (e.g. an attack on fully populated transport terminal). However, the nature of these systems allows for the simulation of a wide range of situations and settings in a quicker and more cost-efficient manner. These software tools can be tailored to offer trainers and trainees a broad selection of scenarios, environments, characters, props and possible outcomes. VTEs have been widely used in military, surgery and law-enforcement training, where the latter has focussed on training interview and interrogation techniques and range shooting.

Project Objectives AUGGMED (AUtomated serious Game scenario Generator for MixED reality training) is part of the European Commission’s Horizon 2020 programme. The aim of AUGGMED is to develop a serious game platform to enable single and team-based training of end-users (e.g. police, counter-terrorism, medics, firefighters) with different level of expertise from different organisations responding to terrorist and organised crime threats. The platform will automatically generate non-linear scenarios tailored to suit the needs of individual trainees with learning outcomes that will improve the acquisition of emotional management, analytical thinking, problem solving and decision making skills. The game scenarios will include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through Virtual Reality (VR) and Mixed Reality (MR) environments with multimodal interfaces. This will result in highly realistic training scenarios allowing advanced interactivity while encouraging security staff and first responders to engage and actively participate in the training process. In addition, the AUGGMED platform will include tools for trainers enabling them to set learning objectives, define scenarios, monitor training sessions, modify scenarios and provide feedback in real-time, as well as evaluate trainee performance and set training curricula for individual personnel in the post-training session phase. Finally, the platform will be offered in affordable and cost-effective Modes including Basic Mode (low VR fidelity and interactivity through mobile devices), Intermediate Mode (immersive multimodal VR) and Full Mode (immersive multimodal MR On-Site).
The partners for the project are: BMT Group Ltd, FSEG University of Greenwich, Piraeus Port Authority SA, Ferrocarrils De La Generalitat De Catalunya, University of Birmingham, GEOMOBILE GMBH, Sistema D'Emergencies Mediques, Police and Crime Commissioner for West Yorkshire, Sheffield Hallam University, ISRA-TEAM 98 Ltd, SERCO Belgium SA, Integration Power, Ministry Of Citizens Protection and Universidad Politécnica de Madrid

The AUGGMED concept
The AUGGMED concept

Work Package 1:
Lead partner: WYP
Training Requirements and AUGGMED Platform Specification

The objective of this work package is to define the security training requirements for several transport terminals and determine the training platform specifications. The overall system specification, design of the architecture indicating how the three main elements of the system (serious game scenario, VR environment and training tools) will communicate and interact with each other will be specified.

FSEG involvement in this work package consists of defining the requirements for the modelled geometry, agent and crowd simulations. New behavioural features will need to be developed for the EXODUS crowd simulation engine e.g. ability for agents representing authority figures to interact with other agents, providing them with instructions to modify their behaviour.  This will allow simulated authority figures to issue voice commands or make hand gestures to other agents instructing them to move in a particular direction, to indicate to stop or to run or to crawl, etc.  The capabilities of the EXODUS software will be expanded further to include the capability for an external person (i.e. live player) to take control of one of the agents within the simulation controlling their movement through the simulated environment and controlling the nature of their interaction with the environment and with other simulated agents or other agents guided by other real players.

Work Package 2:
Lead partner: GEO 
Automated Game Scenario Engine for Training

The objective of this work package is to design and develop an engine that will automatically generate game scenarios on-demand to suit the individual training needs of the first responders and security personnel. The engine will adapt the generated scenarios to the different levels of VR and MR environments while maximising the training objectives and learning outcomes.

FSEG will be involved in the development of appropriate game scenarios and gameplay logic. Several game scenarios will be evaluated and the most relevant scenario and game play will be realised. During this task FSEG will focus on modelling and developing the environments of the game. Algorithms will be developed to automatically generate a scenario based on selected training criteria and user profiles. Game scenarios will involve specification of: the geometry, the crowd population, the perpetrators (number and location), the security forces (number and location) and the hazards (e.g. smoke, heat, toxic gases, etc). The appropriate scenario factors will be represented within the EXODUS simulation environment. This information will be communicated to EXODUS via the communications layer and interpreted via a scenario layer within EXODUS.

Work Package 3:
Lead partner: UOB
VR and MR Environments

The objective of this work package is to design and develop the VR and MR environments that will be used to deliver the serious game to the end-users. The physical and behavioural modelling and technologies of the VR environments which will support the gaming platform will be developed in this work package. In addition, the framework for interfaces and devices allowing effective interaction between the end-user and the VR/MR environments will be developed here.

FSEG involvement in this task will include the modelling of three pilot infrastructures including appropriate agent behaviours and threats. Support for multimodal interfaces and devices delivering the interactive virtual environment to the end-users will also be developed. The agents present within the models will be simulating using the EXODUS environment. Agents will include counter-terrorist forces, first responders as well as general public and intruders. Important aspects of agent’s physical (e.g., resting, walking, running), emotional (e.g., friendly, assertive, aggressive, scared), and physiological (e.g., healthy, injured, dead) status will be simulated through modelling of interactive behaviour as well as through direct control from a user in real time during the running of the simulation (e.g., a security agent instructed other agents to perform a certain action such stop, go, change direction or change stance, i.e. crawl.) The simulations within EXODUS will be enhanced to include simulations of threats like fire spreading within a building, or explosives detonated by the intruders. In addition, sophisticated simulations of the telecommunication capabilities of the security units and first responders, the public and the intruders will be included.

Circulation scenario modelled in buildingEXODUS and represented in vrEXODUS. This will help provide the initial conditions for the scenario involving the terrorist attack to the airport.

Smoke hazard from SMARTFIRE fire simulations: Smoke at head height spreading in airport terminal, RED showing 10m visibility or less.

Heat hazard from SMARTFIRE fire simulations: Temperature at head height spreading in the airport terminal. Exposure to air Temp. above 121°C causes pain.

Heat hazard from SMARTFIRE fire simulations: Temperature iso-surfaces showing the evolving spread of the heat in the airport terminal.

Work Package 4:
Lead partner: SHU
Trainer Tools

The objective of this work package is to design and develop the tools that will enable trainers to improve the training outcome for all personnel involved in training programmes. The objective of this work package is to design and develop the trainer tools that will allow trainers to access the functionality of game scenario generation.

Work Package 5:
Lead partner: BMT
AUGGMED platform integration

The objective of this work package is to integrate the game engine developed in WP2, the VR/MR technologies developed in WP3 and the training tools developed in WP4. Device management and security issues will also be addressed in this work packages.

FSEG will be involved in implementing a communication layer that controls all access between EXODUS and the other AUGGMED components. This will be a conduit for EXODUS-AUGGMED communication (e.g. agent actions, simulated results, communication with the VR shell, instructions passed on to the client interface) and AUGGMED-EXODUS communication (user operations, SMARTFIRE environmental data, and communication with server).

Work Package 6:
Lead partner: WYP

Pilots and Evaluation

The objective of this work package is to evaluate the effectiveness of the AUGGMED platform as a training tool. The activities in this work package will focus on the pilot plans including logistics as well as evaluation protocols as defined in WP1. In addition, the analysis of the results with conclusions and recommendations for further improvements will take place in this WP.

FSEG will contribute in the formulation of detailed plans for the pilot studies including timelines, recruitment of volunteers, trainees and trainers (and a small number of public if necessary).
Prior to testing, the prototypes will be installed. Depending on the scenario and/or Mode that is evaluated, the work may include software and/or hardware installation. Once the installation has been completed a rigorous test will be carried out to ensure reliable functionality of all prototypes during evaluation.

Work Package 7:
Lead partner: N/A
Dissemination, Exploitation and Communication
Work Package 8:
Lead partner: BMT
Project Management
Project Partners
BMT Ltd Logo
University of Greenwich logo
University of Greenwich
Piraeus Port Authority SA logo
Piraeus Port Authority SA
Ferrocarrils De La Generalitat De Catalunya logo
Ferrocarrils De La Generalitat De Catalunya
University of Birmingham logo
University of Birmingham
Sistema D'Emergencies Mediques logo
Sistema D'Emergencies Mediques
Police and Crime Commissioner for West Yorkshire logo
Police and Crime Commissioner for West Yorkshire
ISRA-TEAM 98 Ltd logo
SERCO Belgium SA logo
SERCO Belgium SA
Sheffield Hallam University logo
Sheffield Hallam University
Integration Power logo
Integration Power
Ministry Of Citizens Protection logo
Ministry Of Citizens Protection
Universidad Politécnica de Madrid logo
Universidad Politécnica de Madrid
Further Information Prof. Ed Galea
Fire Safety Engineering Group
University of Greenwich
Greenwich Maritime Campus
Old Royal Naval College
Queen Mary Building
Greenwich SE10 9LS
Tel: +44 (020) 8331 8730

The AUGGMED project is funded by the European Union’s HORIZON 2020 Framework Programme for Research and Innovation