AUGGMED
June 2015 - June 2018

Project reference: 653590
Funded under: H2020-EU.3.7. - Secure societies - protecting freedom and security of Europe and its citizens

Automated Serious Game Scenario Generator for Mixed Reality Training

Click here to see FSEG YouTube channel playlist featuring presentation and demonstration from project AUGGMED.

Terrorism is a major threat to human life and critical infrastructure in Europe. The intruders continuously devise new strategies and execute highly sophisticated plans that are increasingly unfamiliar to the security forces and therefore more difficult to predict, prepare and defend against. In principle, in order to prepare for terrorist acts or critical incidents, security forces are trained using traditional methods such as live scenario training though full-scale field exercises. This type of training can be very costly, time-consuming and dangerous to those participating. In addition, it can only offer a limited number of scenarios that individually require major resources to plan and carryout. Thus, limiting the amount and variety of situations for which security forces are currently trained and making it difficult to maintain uniform levels of skill, judgement and preparedness across teams and individuals.
 
Virtual Training Environments (VTEs) and Serious Games (SGs) provide the opportunity to prepare security forces for events that cannot be, or are very difficult to train for using live scenario exercises (e.g. an attack on fully populated transport terminal). However, the nature of these systems allows for the simulation of a wide range of situations and settings in a quicker and more cost-efficient manner. These software tools can be tailored to offer trainers and trainees a broad selection of scenarios, environments, characters, props and possible outcomes. VTEs have been widely used in military, surgery and law-enforcement training, where the latter has focussed on training interview and interrogation techniques and range shooting.

The AUGGMED concept

The objective of this work package is to define the security training requirements for several transport terminals and determine the training platform specifications. The overall system specification, design of the architecture indicating how the three main elements of the system (serious game scenario, VR environment and training tools) will communicate and interact with each other will be specified.

FSEG involvement in this work package consists of defining the requirements for the modelled geometry, agent and crowd simulations. New behavioural features will need to be developed for the EXODUS crowd simulation engine e.g. ability for agents representing authority figures to interact with other agents, providing them with instructions to modify their behaviour.  This will allow simulated authority figures to issue voice commands or make hand gestures to other agents instructing them to move in a particular direction, to indicate to stop or to run or to crawl, etc.  The capabilities of the EXODUS software will be expanded further to include the capability for an external person (i.e. live player) to take control of one of the agents within the simulation controlling their movement through the simulated environment and controlling the nature of their interaction with the environment and with other simulated agents or other agents guided by other real players.

The objective of this work package is to design and develop an engine that will automatically generate game scenarios on-demand to suit the individual training needs of the first responders and security personnel. The engine will adapt the generated scenarios to the different levels of VR and MR environments while maximising the training objectives and learning outcomes.

FSEG will be involved in the development of appropriate game scenarios and gameplay logic. Several game scenarios will be evaluated and the most relevant scenario and game play will be realised. During this task FSEG will focus on modelling and developing the environments of the game. Algorithms will be developed to automatically generate a scenario based on selected training criteria and user profiles. Game scenarios will involve specification of: the geometry, the crowd population, the perpetrators (number and location), the security forces (number and location) and the hazards (e.g. smoke, heat, toxic gases, etc). The appropriate scenario factors will be represented within the EXODUS simulation environment. This information will be communicated to EXODUS via the communications layer and interpreted via a scenario layer within EXODUS.

The objective of this work package is to design and develop the VR and MR environments that will be used to deliver the serious game to the end-users. The physical and behavioural modelling and technologies of the VR environments which will support the gaming platform will be developed in this work package. In addition, the framework for interfaces and devices allowing effective interaction between the end-user and the VR/MR environments will be developed here.

FSEG involvement in this task will include the modelling of three pilot infrastructures including appropriate agent behaviours and threats. Support for multimodal interfaces and devices delivering the interactive virtual environment to the end-users will also be developed. The agents present within the models will be simulating using the EXODUS environment. Agents will include counter-terrorist forces, first responders as well as general public and intruders. Important aspects of agent’s physical (e.g., resting, walking, running), emotional (e.g., friendly, assertive, aggressive, scared), and physiological (e.g., healthy, injured, dead) status will be simulated through modelling of interactive behaviour as well as through direct control from a user in real time during the running of the simulation (e.g., a security agent instructed other agents to perform a certain action such stop, go, change direction or change stance, i.e. crawl.) The simulations within EXODUS will be enhanced to include simulations of threats like fire spreading within a building, or explosives detonated by the intruders. In addition, sophisticated simulations of the telecommunication capabilities of the security units and first responders, the public and the intruders will be included.

The objective of this work package is to design and develop the tools that will enable trainers to improve the training outcome for all personnel involved in training programmes. The objective of this work package is to design and develop the trainer tools that will allow trainers to access the functionality of game scenario generation.

The objective of this work package is to integrate the game engine developed in WP2, the VR/MR technologies developed in WP3 and the training tools developed in WP4. Device management and security issues will also be addressed in this work packages.

FSEG will be involved in implementing a communication layer that controls all access between EXODUS and the other AUGGMED components. This will be a conduit for EXODUS-AUGGMED communication (e.g. agent actions, simulated results, communication with the VR shell, instructions passed on to the client interface) and AUGGMED-EXODUS communication (user operations, SMARTFIRE environmental data, and communication with server).

Pilots and Evaluation

The objective of this work package is to evaluate the effectiveness of the AUGGMED platform as a training tool. The activities in this work package will focus on the pilot plans including logistics as well as evaluation protocols as defined in WP1. In addition, the analysis of the results with conclusions and recommendations for further improvements will take place in this WP.

FSEG will contribute in the formulation of detailed plans for the pilot studies including timelines, recruitment of volunteers, trainees and trainers (and a small number of public if necessary).
Prior to testing, the prototypes will be installed. Depending on the scenario and/or Mode that is evaluated, the work may include software and/or hardware installation. Once the installation has been completed a rigorous test will be carried out to ensure reliable functionality of all prototypes during evaluation.