Project reference: 653590
Automated Serious Game Scenario
Generator for Mixed Reality Training
Terrorism is a major threat to human life and critical
infrastructure in Europe. The intruders continuously devise new
strategies and execute highly sophisticated plans that are
increasingly unfamiliar to the security forces and therefore more
difficult to predict, prepare and defend against. In principle, in
order to prepare for terrorist acts or critical incidents, security
forces are trained using traditional methods such as live scenario
training though full-scale field exercises. This type of training can
be very costly, time-consuming and dangerous to those participating.
In addition, it can only offer a limited number of scenarios that
individually require major resources to plan and carryout. Thus,
limiting the amount and variety of situations for which security
forces are currently trained and making it difficult to maintain
uniform levels of skill, judgement and preparedness across teams and
|AUGGMED (AUtomated serious Game scenario
Generator for MixED reality training) is part of the European
Commission’s Horizon 2020 programme. The aim of AUGGMED is to develop
a serious game platform to enable single and team-based training of
end-users (e.g. police, counter-terrorism, medics, firefighters) with
different level of expertise from different organisations responding
to terrorist and organised crime threats. The platform will
automatically generate non-linear scenarios tailored to suit the needs
of individual trainees with learning outcomes that will improve the
acquisition of emotional management, analytical thinking, problem
solving and decision making skills. The game scenarios will include
advanced simulations of operational environments, agents,
telecommunications and threats, and will be delivered through Virtual
Reality (VR) and Mixed Reality (MR) environments with multimodal
interfaces. This will result in highly realistic training scenarios
allowing advanced interactivity while encouraging security staff and
first responders to engage and actively participate in the training
process. In addition, the AUGGMED platform will include tools for
trainers enabling them to set learning objectives, define scenarios,
monitor training sessions, modify scenarios and provide feedback in
real-time, as well as evaluate trainee performance and set training
curricula for individual personnel in the post-training session phase.
Finally, the platform will be offered in affordable and cost-effective
Modes including Basic Mode (low VR fidelity and interactivity through
mobile devices), Intermediate Mode (immersive multimodal VR) and Full
Mode (immersive multimodal MR On-Site).
The partners for the project are: BMT Group Ltd, FSEG University of Greenwich, Piraeus Port Authority SA, Ferrocarrils De La Generalitat De Catalunya, University of Birmingham, GEOMOBILE GMBH, Sistema D'Emergencies Mediques, Police and Crime Commissioner for West Yorkshire, Sheffield Hallam University, ISRA-TEAM 98 Ltd, SERCO Belgium SA, Integration Power, Ministry Of Citizens Protection and Universidad Politécnica de Madrid
|Work Package 1:
Lead partner: WYP
|Training Requirements and AUGGMED
The objective of this work package is to define the security training requirements for several transport terminals and determine the training platform specifications. The overall system specification, design of the architecture indicating how the three main elements of the system (serious game scenario, VR environment and training tools) will communicate and interact with each other will be specified.
FSEG involvement in this work package consists of defining the requirements for the modelled geometry, agent and crowd simulations. New behavioural features will need to be developed for the EXODUS crowd simulation engine e.g. ability for agents representing authority figures to interact with other agents, providing them with instructions to modify their behaviour. This will allow simulated authority figures to issue voice commands or make hand gestures to other agents instructing them to move in a particular direction, to indicate to stop or to run or to crawl, etc. The capabilities of the EXODUS software will be expanded further to include the capability for an external person (i.e. live player) to take control of one of the agents within the simulation controlling their movement through the simulated environment and controlling the nature of their interaction with the environment and with other simulated agents or other agents guided by other real players.
|Work Package 2:
Lead partner: GEO
|Automated Game Scenario Engine for
The objective of this work package is to design and develop an engine that will automatically generate game scenarios on-demand to suit the individual training needs of the first responders and security personnel. The engine will adapt the generated scenarios to the different levels of VR and MR environments while maximising the training objectives and learning outcomes.
FSEG will be involved in the development of appropriate game scenarios and gameplay logic. Several game scenarios will be evaluated and the most relevant scenario and game play will be realised. During this task FSEG will focus on modelling and developing the environments of the game. Algorithms will be developed to automatically generate a scenario based on selected training criteria and user profiles. Game scenarios will involve specification of: the geometry, the crowd population, the perpetrators (number and location), the security forces (number and location) and the hazards (e.g. smoke, heat, toxic gases, etc). The appropriate scenario factors will be represented within the EXODUS simulation environment. This information will be communicated to EXODUS via the communications layer and interpreted via a scenario layer within EXODUS.
|Work Package 3:
Lead partner: UOB
|VR and MR Environments
The objective of this work package is to design and develop the VR and MR environments that will be used to deliver the serious game to the end-users. The physical and behavioural modelling and technologies of the VR environments which will support the gaming platform will be developed in this work package. In addition, the framework for interfaces and devices allowing effective interaction between the end-user and the VR/MR environments will be developed here.
FSEG involvement in this task will include the modelling of three pilot infrastructures including appropriate agent behaviours and threats. Support for multimodal interfaces and devices delivering the interactive virtual environment to the end-users will also be developed. The agents present within the models will be simulating using the EXODUS environment. Agents will include counter-terrorist forces, first responders as well as general public and intruders. Important aspects of agent’s physical (e.g., resting, walking, running), emotional (e.g., friendly, assertive, aggressive, scared), and physiological (e.g., healthy, injured, dead) status will be simulated through modelling of interactive behaviour as well as through direct control from a user in real time during the running of the simulation (e.g., a security agent instructed other agents to perform a certain action such stop, go, change direction or change stance, i.e. crawl.) The simulations within EXODUS will be enhanced to include simulations of threats like fire spreading within a building, or explosives detonated by the intruders. In addition, sophisticated simulations of the telecommunication capabilities of the security units and first responders, the public and the intruders will be included.
|Work Package 4:
Lead partner: SHU
The objective of this work package is to design and develop the tools that will enable trainers to improve the training outcome for all personnel involved in training programmes. The objective of this work package is to design and develop the trainer tools that will allow trainers to access the functionality of game scenario generation.
|Work Package 5:
Lead partner: BMT
|AUGGMED platform integration
The objective of this work package is to integrate the game engine developed in WP2, the VR/MR technologies developed in WP3 and the training tools developed in WP4. Device management and security issues will also be addressed in this work packages.
FSEG will be involved in implementing a communication layer that controls all access between EXODUS and the other AUGGMED components. This will be a conduit for EXODUS-AUGGMED communication (e.g. agent actions, simulated results, communication with the VR shell, instructions passed on to the client interface) and AUGGMED-EXODUS communication (user operations, SMARTFIRE environmental data, and communication with server).
|Work Package 6:
Lead partner: WYP
Pilots and Evaluation
The objective of this work package is to evaluate the effectiveness of the AUGGMED platform as a training tool. The activities in this work package will focus on the pilot plans including logistics as well as evaluation protocols as defined in WP1. In addition, the analysis of the results with conclusions and recommendations for further improvements will take place in this WP.
FSEG will contribute in the formulation of detailed plans for the
pilot studies including timelines, recruitment of volunteers,
trainees and trainers (and a small number of public if necessary).
|Work Package 7:
Lead partner: N/A
|Dissemination, Exploitation and Communication
|Work Package 8:
Lead partner: BMT
|Prof. Ed Galea
Fire Safety Engineering Group
University of Greenwich
Greenwich Maritime Campus
Old Royal Naval College
Queen Mary Building
Greenwich SE10 9LS
Tel: +44 (020) 8331 8730
The AUGGMED project is funded by the European Union’s HORIZON 2020 Framework Programme for Research and Innovation