Homeland Security Exercise
and Evaluation
Program (HSEEP)
JANUARY 2020
Homeland Security Exercise and Evaluation Program
i
Table of Contents
INTRODUCTION........................................................................................................................ V
PURPOSE ................................................................................................................................... V
ROLE OF EXERCISES ................................................................................................................. V
APPLICABILITY AND SCOPE ..................................................................................................... VI
SUPERSESSION ......................................................................................................................... VI
DOCTRINE ORGANIZATION ...................................................................................................... VI
REVISION PROCESS ................................................................................................................. VII
TOOLS .................................................................................................................................... VII
TRAINING ............................................................................................................................... VII
1. HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP)
FUNDAMENTALS ............................................................................................................ 1-1
OVERVIEW............................................................................................................................. 1-1
FUNDAMENTAL PRINCIPLES .................................................................................................. 1-1
THE HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP) CYCLE ......... 1-2
Program Management ................................................................................................. 1-2
Exercise Design and Development .............................................................................. 1-2
Exercise Conduct ......................................................................................................... 1-2
Exercise Evaluation ..................................................................................................... 1-2
Improvement Planning ................................................................................................ 1-2
2. PROGRAM MANAGEMENT .......................................................................................... 2-1
OVERVIEW............................................................................................................................. 2-1
INTEGRATED PREPAREDNESS CYCLE ..................................................................................... 2-1
LINKAGE OF INTEGRATED PREPAREDNESS CYCLE AND HOMELAND SECURITY EXERCISE AND
EVALUATION PROGRAM (HSEEP) ........................................................................................ 2-2
ENGAGE SENIOR LEADERS .................................................................................................... 2-2
PREPAREDNESS PRIORITIES ................................................................................................... 2-2
INTEGRATED PREPAREDNESS PLANNING WORKSHOP (IPPW) ............................................... 2-3
Purpose ........................................................................................................................ 2-3
Participation ................................................................................................................ 2-3
Conduct of the Integrated Preparedness Planning Workshop (IPPW) ....................... 2-4
INTEGRATED PREPAREDNESS PLAN (IPP) .............................................................................. 2-5
Progressive Approach ................................................................................................. 2-5
Discussion-Based Exercises ........................................................................................ 2-6
Operations-Based Exercises ........................................................................................ 2-9
PROGRAM REPORTING ......................................................................................................... 2-11
MANAGE EXERCISE PROGRAM RESOURCES ........................................................................ 2-12
Exercise Budget Management ................................................................................... 2-12
Staffing ....................................................................................................................... 2-12
Other Resources ........................................................................................................ 2-12
3. EXERCISE DESIGN AND DEVELOPMENT ............................................................... 3-1
Homeland Security Exercise and Evaluation Program
ii
OVERVIEW............................................................................................................................. 3-1
EXERCISE FOUNDATION ........................................................................................................ 3-1
EXERCISE PLANNING ACTIVITIES .......................................................................................... 3-2
Exercise Planning Meetings ........................................................................................ 3-2
EXERCISE PLANNING TEAM CONSIDERATIONS ...................................................................... 3-7
Exercise Planning Team Positions .............................................................................. 3-7
EXERCISE DESIGN ............................................................................................................... 3-10
Exercise Purpose ....................................................................................................... 3-10
Exercise Scope ........................................................................................................... 3-10
Exercise Objectives ................................................................................................... 3-11
Evaluation Parameters .............................................................................................. 3-12
Scenario ..................................................................................................................... 3-12
Exercise Documentation ............................................................................................ 3-13
Planning for Media and Public Affairs ..................................................................... 3-16
EXERCISE DEVELOPMENT .................................................................................................... 3-16
Discussion-Based Exercise Logistics Planning ........................................................ 3-17
Discussion-Based Exercise Facilitation Planning .................................................... 3-18
Operations-Based Exercise Logistics Planning ........................................................ 3-19
Operations-Based Exercise Control Planning .......................................................... 3-21
Planning for Exercise Evaluation ............................................................................. 3-25
Preparing for Exercise Conduct ................................................................................ 3-25
4. EXERCISE CONDUCT .................................................................................................... 4-1
OVERVIEW............................................................................................................................. 4-1
DISCUSSION-BASED EXERCISES ............................................................................................ 4-1
Exercise/Play Preparation .......................................................................................... 4-1
Exercise Play/Conduct ................................................................................................ 4-1
OPERATIONS-BASED EXERCISES ........................................................................................... 4-3
Play Preparation ......................................................................................................... 4-3
Exercise Play/Conduct ................................................................................................ 4-4
BRIEFINGS ............................................................................................................................. 4-6
WRAP-UP ACTIVITIES ............................................................................................................ 4-7
Player Hotwash ........................................................................................................... 4-7
Debrief ......................................................................................................................... 4-7
5. EXERCISE EVALUATION ............................................................................................. 5-1
OVERVIEW............................................................................................................................. 5-1
EVALUATION PLANNING ....................................................................................................... 5-1
Evaluation Team .......................................................................................................... 5-2
Evaluation Documentation .......................................................................................... 5-3
Exercise Evaluation Guide (EEG) Development ........................................................ 5-4
OBSERVATION AND DATA COLLECTION ................................................................................ 5-5
Observation ................................................................................................................. 5-5
Data Collection ........................................................................................................... 5-6
DATA ANALYSIS .................................................................................................................... 5-7
Data Synthesis ............................................................................................................. 5-7
Event Reconstruction ................................................................................................... 5-7
Homeland Security Exercise and Evaluation Program
iii
Trend Analysis ............................................................................................................. 5-8
Root Cause Analysis .................................................................................................... 5-8
AFTER-ACTION REPORT (AAR)/IMPROVEMENT PLAN (IP) ................................................... 5-8
AFTER-ACTION MEETING (AAM) ....................................................................................... 5-10
IMPROVEMENT PLAN (IP) .................................................................................................... 5-10
6. IMPROVEMENT PLANNING ........................................................................................ 6-1
OVERVIEW............................................................................................................................. 6-1
SMART CORRECTIVE ACTIONS ............................................................................................ 6-1
CORRECTIVE ACTION TRACKING AND IMPLEMENTATION ..................................................... 6-2
IMPROVEMENT PLANNING TO SUPPORT CONTINUOUS IMPROVEMENT ................................... 6-3
ACRONYMS ................................................................................................................ Acronym-1
GLOSSARY OF TERMS ............................................................................................. Glossary-1
List of Tables
TABLE 2.1: FACTORS FOR CONSIDERATION IN DEVELOPING EXERCISE PROGRAM PRIORITIES ..... 2-4
TABLE 2.2: CONDUCT OF THE INTEGRATED PREPAREDNESS PLANNING WORKSHOP (IPPW) ....... 2-4
TABLE 2.3: DISCUSSION-BASED EXERCISE TYPES: SEMINAR ........................................................ 2-6
TABLE 2.4: DISCUSSION-BASED EXERCISE TYPES: WORKSHOP .................................................... 2-7
TABLE 2.5: DISCUSSION-BASED EXERCISE TYPES: TABLETOP EXERCISE ..................................... 2-7
TABLE 2.6: DISCUSSION-BASED EXERCISE TYPES: GAME ............................................................ 2-8
TABLE 2.7: OPERATIONS-BASED EXERCISE TYPES: DRILL ........................................................... 2-9
TABLE 2.8: OPERATIONS-BASED EXERCISE TYPES: FUNCTIONAL EXERCISE .............................. 2-10
TABLE 2.9: OPERATIONS-BASED EXERCISE TYPES: FULL-SCALE EXERCISE ............................... 2-11
TABLE 3.1: PLANNING MEETING TYPES: CONCEPT AND OBJECTIVES (C&O) MEETING ............... 3-2
TABLE 3.2: PLANNING MEETING TYPES: INITIAL PLANNING MEETING (IPM) .............................. 3-3
TABLE 3.3: PLANNING MEETING TYPES: MIDTERM PLANNING MEETING (MPM) ........................ 3-4
TABLE 3.4: PLANNING MEETING TYPES: MASTER SCENARIO EVENTS LIST (MSEL) MEETING .... 3-5
TABLE 3.5: PLANNING MEETING TYPES: FINAL PLANNING MEETING (FPM) ............................... 3-6
TABLE 3.6: FUNCTIONS OF AN EXERCISE PLANNING TEAM........................................................... 3-8
TABLE 3.7: ELEMENTS THAT DEFINE SCOPE ............................................................................... 3-10
TABLE 3.8: DISCUSSION-BASED EXERCISE DOCUMENTATION .................................................... 3-13
TABLE 3.9: OPERATIONS-BASED EXERCISE DOCUMENTATION ................................................... 3-14
TABLE 3.10: OTHER EXERCISE DOCUMENTATION ...................................................................... 3-15
TABLE 3.11: POSITIONS OF AN EXERCISE FACILITATION STRUCTURE ......................................... 3-19
TABLE 3.12: FACILITIES AND EXERCISE AREAS, DESCRIPTIONS, AND TYPES.............................. 3-20
TABLE 3.13: POSITIONS OF AN OPERATIONS-BASED EXERCISE CONTROL STRUCTURE .............. 3-22
TABLE 3.14: MSEL EVENT TYPES .............................................................................................. 3-23
TABLE 4.1: POSITIONS OF A DISCUSSION-BASED EXERCISE CONTROL STRUCTURE ...................... 4-2
TABLE 4.2: POSITIONS OF AN OPERATIONS-BASED EXERCISE CONTROL STRUCTURE .................. 4-5
TABLE 4.3: TYPES OF EXERCISE BRIEFS ........................................................................................ 4-6
TABLE 5.1: POSITIONS OF AN EXERCISE EVALUATION TEAM ........................................................ 5-3
TABLE 5.2: EVALUATION DOCUMENTATION ................................................................................. 5-3
TABLE 5.3: DATA COLLECTION METHODS .................................................................................... 5-6
TABLE 5.4: CATEGORIES OF AN AAR/IP OBSERVATION ............................................................... 5-9
Homeland Security Exercise and Evaluation Program
iv
TABLE 5.5: AFTER-ACTION MEETING ......................................................................................... 5-10
List of Figures
FIGURE 1.1: THE HSEEP CYCLE................................................................................................... 1-2
FIGURE 2.1: THE INTEGRATED PREPAREDNESS CYCLE ................................................................. 2-1
FIGURE 2.2: THE RELATIONSHIP OF THE INTEGRATED PREPAREDNESS CYCLE AND THE HSEEP
CYCLE ................................................................................................................................... 2-2
FIGURE 3.1: INDIVIDUAL EXERCISE DESIGN AND DEVELOPMENT PROCESS .................................. 3-1
FIGURE 3.2: EXAMPLE OF A FUNCTIONAL ORGANIZATIONAL STRUCTURE FOR AN EXERCISE
PLANNING TEAM ................................................................................................................... 3-8
FIGURE 3.3: PRIORITIES, OBJECTIVES, AND CAPABILITIES .......................................................... 3-11
FIGURE 3.4: SMART GUIDELINES FOR EXERCISE OBJECTIVES................................................... 3-12
FIGURE 3.5: EXAMPLE OF A DISCUSSION-BASED EXERCISE FACILITATION STRUCTURE ............. 3-19
FIGURE 3.6: EXAMPLE OF AN OPERATIONS-BASED EXERCISE CONTROL STRUCTURE ................ 3-22
FIGURE 4.1: EXAMPLE OF A DISCUSSION-BASED CONTROL STRUCTURE ...................................... 4-2
FIGURE 4.2: EXAMPLE OF AN OPERATIONS-BASED EXERCISE CONTROL STRUCTURE .................. 4-4
FIGURE 5.1: EXITING THE HSEEP CYCLE AND ENTERING THE INTEGRATED PREPAREDNESS CYCLE
.............................................................................................................................................. 5-1
FIGURE 5.2: EXAMPLE OF AN EXERCISE EVALUATION TEAM ........................................................ 5-2
FIGURE 5.3: DEVELOPMENT OF AN EXERCISE EVALUATION GUIDE (EEG) ................................... 5-4
FIGURE 6.1: EXITING THE HSEEP CYCLE AND ENTERING THE INTEGRATED PREPAREDNESS CYCLE
.............................................................................................................................................. 6-1
FIGURE 6.2: SMART GUIDELINES FOR CORRECTIVE ACTIONS ..................................................... 6-2
Homeland Security Exercise and Evaluation Program
v
Introduction
Purpose
The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of fundamental
principles for exercise programs, as well as a common approach to program management, design
and development, conduct, evaluation, and improvement planning. Exercises are an important
component of preparedness, by providing the whole community
1
with the opportunity to shape
planning, assess and validate capabilities,
2
and address areas for improvement. An exercise is an
event or activity, delivered through discussion or action, to develop, assess, or validate plans,
policies, procedures, and capabilities that jurisdictions/organizations can use to achieve planned
objectives.
Through HSEEP, exercise program managers can develop, execute, and evaluate exercises that
address the priorities established by a jurisdiction’s/organization’s senior leaders. The National
Preparedness Goal,
3
strategy documents, Threat and Hazard Identification and Risk Assessment
(THIRA) processes, capability assessments, and results from previous exercises and real-world
incidents impact the priorities. These priorities guide the overall direction of an exercise program
where individual exercises anchor to a common set of priorities or objectives, which increase in
complexity over time. These priorities guide the design and development of individual exercises
and allow planners to identify and align objectives to the capabilities being evaluated. Exercise
evaluation assesses the ability to meet the objectives and capabilities by documenting strengths,
areas for improvement, capability performance, and corrective actions in an After-Action Report
(AAR)/Improvement Plan (IP).
Through Improvement Planning, jurisdictions/organizations take the corrective actions identified
during exercises to improve plans, build and sustain capabilities, and maintain readiness.
Stakeholders are reminded of the importance of implementing corrective actions.
In this way, the use of HSEEP—in line with the National Preparedness Goal, the National
Preparedness System (NPS),
4
and stakeholder preparedness priorities—supports efforts across the
whole community that improve the nation’s capacity to build, sustain, and deliver capabilities to
better respond to a real-world incident.
Role of Exercises
Exercises play a vital role in preparedness. A well-designed exercise provides a low-risk
environment to familiarize personnel with roles and responsibilities; foster meaningful interaction
and communication across jurisdictions/organizations; assess and validate plans, policies,
procedures, and capabilities; and identify strengths and areas for improvement. Exercises bring
together and strengthen the whole community to prevent, protect against, mitigate, respond to, and
1
The whole community means involving people (individuals and families, including those with access and functional needs, businesses, faith-
based and community organizations, nonprofit groups, schools and academia, media outlets, and all levels of government including state, local,
tribal, territorial, and federal partners) in the development of national preparedness documents and ensuring their roles and responsibilities are
reflected in the content of the materials.
2
Capabilities are the means to accomplish a mission, function, or objective based on the performance of related tasks, under specified conditions,
to target levels of performance. For additional information on the core capability development sheets see
https://www.fema.gov/core-capability-
development-sheets.
3
Refer to the National Preparedness Goal: https://www.fema.gov/national-preparedness-goal.
4
Refer to the National Preparedness System: https://www.fema.gov/national-preparedness-system.
Homeland Security Exercise and Evaluation Program
vi
recover from all hazards. Overall, exercises help the whole community address the priorities
established by a jurisdiction’s/organization’s leaders; and evaluate progress towards meeting
preparedness goals.
Applicability and Scope
The HSEEP doctrine is flexible, scalable, adaptable, and for use by stakeholders across the whole
community and all mission areas. Using HSEEP supports the NPS by providing a consistent
approach to exercises and measuring progress toward building, sustaining, and delivering
capabilities. To achieve whole community preparedness goals, agencies, jurisdictions/
organizations, and stakeholders (for example, public health, transportation, education, etc.) that do
not use the core capabilities as outlined in the National Preparedness Goal should use their relevant
capabilities and elements to better prepare for real-world incidents.
The foundation of the HSEEP doctrine incorporates lessons learned and best practices from the
exercise community and current policies and plans that support training, technology systems, tools,
and technical assistance. Exercise practitioners are encouraged to apply and adapt the HSEEP
doctrine to meet their specific needs.
Supersession
The 2020 iteration of the HSEEP doctrine supersedes the 2013 version.
Doctrine Organization
The organization of the HSEEP doctrine follows:
• Chapter 1, Homeland Security Exercise and Evaluation Program (HSEEP)
Fundamentals, describes the basic principles and methodology of HSEEP.
• Chapter 2, Program Management, provides guidance for overseeing and integrating a
variety of exercises over time, conducting an Integrated Preparedness Planning Workshop
(IPPW), and developing a multi-year Integrated Preparedness Plan (IPP).
• Chapter 3, Exercise Design and Development, describes the methodology for developing
exercise objectives, conducting planning meetings, developing exercise documentation,
and planning for exercise logistics, control, and evaluation.
• Chapter 4, Exercise Conduct, provides guidance on setup, exercise play, and wrap-up
activities.
• Chapter 5, Exercise Evaluation, provides the approach to exercise evaluation planning
and conduct through data collection, analysis, and development of an After-Action Report
(AAR).
• Chapter 6, Improvement Planning, addresses corrective actions identified in the exercise
Improvement Plan and the process of tracking corrective actions to resolution.
Homeland Security Exercise and Evaluation Program
vii
Revision Process
The Department of Homeland Security’s Federal Emergency Management Agency’s (FEMA’s)
National Exercise Division reviews the HSEEP doctrine and methodology every six years, or as
needed.
Tools
The Preparedness Toolkit (PrepToolkit) is a web-based application that allows the whole
community access to a wide variety of resources to manage preparedness activities. The system is
designed to support implementation of the NPS by providing exercise planners, program
managers, resource typing and mutual aid coordinators, threat and hazard planners, and other key
stakeholders access to technologies that align to the six NPS elements. PrepToolkit supplies a
technology platform that supports implementation of HSEEP and aids exercise planners in
program management, design and development, conduct, evaluation, and improvement planning.
Refer to https://preptoolkit.fema.gov/hseep-resources for more information.
Training
Training provides the whole community with the knowledge, skills, and abilities needed to perform
key tasks required by specific capabilities. Jurisdictions/organizations should make training
decisions based on information derived from the assessments, strategies, and plans developed in
previous steps of the Integrated Preparedness Cycle. The National Training Program provides an
organized approach to training for emergency managers and emergency response providers across
the nation that supports the National Preparedness Guidelines.
5
The National Preparedness Online
Course Catalog provides searchable, integrated information on courses provided or managed by
FEMA’s Center for Domestic Preparedness (CDP), the Emergency Management Institute (EMI),
the National Fire Academy, and the National Training and Education Division (NTED).
6
5
For more information on the National Preparedness Guidelines refer to: https://www.fema.gov/media-library/assets/documents/16886.
6
For more information on training refer to:
https://www.fema.gov/training, https://www.firstrespondertraining.gov/frts, and https://www.usfa.fema.gov/training/nfa/
.
Homeland Security Exercise and Evaluation Program
1-1
1. H omeland S ecurity E xercise and E valuation
P rogram (HSEEP) Fundamentals
Overview
The HSEEP doctrine consists of fundamental principles that frame a common approach to
exercises. The intent is to enhance consistency in exercise conduct and evaluation while ensuring
exercises remain a flexible, accessible way to improve our preparedness across the nation.
Fundamental Principles
Applying the following principles to the management of an exercise program and executing
individual exercises is important to an effective evaluation of capabilities:
• Senior Leader Guidance: The early and frequent engagement of senior leaders is the key
to the success of any exercise program. Senior leaders provide the overarching guidance
and direction for the exercise and evaluation program as well as specific intent for
individual exercises.
• Informed by Risk: Identifying and assessing risks and associated impacts helps
jurisdictions/organizations identify and evaluate priorities, objectives, and capabilities
through exercises.
• Capability-Based, Objective-Driven: Jurisdictions/Organizations can use exercises to
evaluate current capability levels/targets and identify gaps. Exercises focus on assessing
performance against capability-based objectives.
• Progressive Exercise Planning Approach: A progressive approach includes the use of
various exercises aligned to a common set of program priorities and objectives with an
increasing level of complexity over time. Progressive exercise planning does not always
imply a linear progression of exercise types.
• Whole Community Integration: The use of HSEEP encourages exercise planners, where
appropriate, to engage the whole community throughout program management, design and
development, conduct, evaluation, and improvement planning.
• Common Methodology: HSEEP includes a common methodology for exercises across all
mission areas. The methodology enables jurisdictions/organizations a shared
understanding of program management, design and development, conduct, evaluation, and
improvement planning and fosters exercise-related interoperability and collaboration.
Homeland Security Exercise and Evaluation Program
1-2
The Homeland Security Exercise and Evaluation Program
(HSEEP) Cycle
HSEEP uses a common approach for planning, conducting,
and evaluating individual exercises. The following chapters
contain more detailed descriptions of each phase, as
depicted in Figure 1.1.
Program Management
Program management involves a collaborative approach
that integrates resources, jurisdictions/organizations, and
individuals to identify and achieve program priorities. An
effective exercise program maximizes efficiency,
resources, time, and funding by ensuring a coordinated and
integrated approach to building, sustaining, and delivering
capabilities. Through the management of an exercise
program, senior leaders provide oversight for specific
preparedness activities sustained over time.
Exercise Design and Development
During exercise design and development, the exercise planning team members use the intent and
guidance of senior leaders and the program priorities to shape the individual or series of exercises.
Involving the risk and hazard assessments, plans, policies, and procedures, grant or cooperative
agreements, and relevant After-Action Reports (AARs) and Improvement Plans (IPs), exercise
planners ensure that exercises assess and validate a jurisdiction’s/organization’s capabilities.
Exercise Conduct
Exercise conduct involves activities such as preparing for exercise play, managing exercise play,
and conducting immediate exercise wrap-up activities. For a discussion-based exercise, conduct
entails presentation, facilitation, and discussion. For an operations-based exercise, conduct,
encompasses all operations occurring between the designated Start of the Exercise (StartEx) and
End of the Exercise (EndEx).
Exercise Evaluation
An effective evaluation assesses performance against exercise objectives and identifies and
documents strengths and areas for improvement relative to capabilities. Evaluation is important
and considered in all phases of the exercise planning cycle, beginning when the exercise planning
team meets to establish objectives and initiate exercise design through improvement planning.
Improvement Planning
Improvement Planning activities can help shape a jurisdiction’s/organization’s preparedness
priorities and support continuous improvement. Actions identified during Improvement Planning
help to strengthen elements of a jurisdiction’s/organization’s capability to plan, organize/equip,
train, and exercise.
Figure 1.1: The HSEEP Cycle
Homeland Security Exercise and Evaluation Program
2-1
2. Program Management
Overview
Program management is the process of overseeing and integrating a variety of exercises over time.
An effective exercise program helps jurisdictions/organizations maximize efficiency, resources,
time, and funding by ensuring that exercises are part of a coordinated, integrated planning approach
to building, sustaining, and delivering capabilities. This integrated planning approach begins when
senior leaders, working with whole community stakeholders, identify and develop a set of multi-
year preparedness priorities based on relevant threats, hazards, and risks.
Integrated Preparedness Cycle
The Integrated Preparedness Cycle of planning, organizing/
equipping, training, exercising, and evaluating/improving is a
continuous process that ensures the regular examination of
ever-changing threats, hazards, and risks, as shown in Figure
2.1.
The Cycle involves the assessment of threats, hazards, and
risks; new and updated plans; and improvements implemented
from previously identified shortfalls or gaps. The
preparedness priorities are developed to ensure that the needed
preparedness elements are incorporated. This cycle provides a
continual and reliable approach to support decision making,
resource allocation, and measure progress toward building,
sustaining, and delivering capabilities based on a
jurisdiction’s/organization’s threats, hazards, and risks.
Through the Integrated Preparedness Cycle, the program manager understands the full breadth of
preparedness activities that impact their jurisdiction/organization and allows for a more deliberate
approach to multi-year preparedness activity planning. Ultimately, this integrated approach
becomes a means to achieve whole community preparedness in a consistent and measurable way.
Figure 2.1: The Integrated
Preparedness Cycle
Homeland Security Exercise and Evaluation Program
2-2
Linkage of Integrated Preparedness Cycle and Homeland S ecurity
Exercise and Evaluation Program (HSEEP)
The exercise phase of the Integrated Preparedness Cycle is linked
to the program management process and starts the design and
development of individual exercises within HSEEP. Multi-year
preparedness priorities guide the exercise program to produce
quality individual exercises. These individual exercises are used
to build, sustain, and deliver capabilities.
Effective program management is comprised of the following
components:
• Engaging senior leaders;
• Establishing multi-year preparedness priorities;
• Conducting an Integrated Preparedness Planning
Workshop (IPPW);
• Developing a multi-year Integrated Preparedness Plan
(IPP);
• Maintaining program reporting of exercise outcomes; and
• Managing exercise program resources.
Through effective program management, each exercise becomes
a supporting component of a larger exercise program and
integrated preparedness with overarching priorities.
Figure 2.2 shows the relationship and linkage of the Integrated
Preparedness Cycle and the HSEEP Cycle. Exercise practitioners are encouraged to apply and
adapt HSEEP program management guidance to meet their specific needs.
Engage Senior Leaders
Involving senior leaders early and often in the exercise process is critical in providing both the
strategic direction for the exercise program and specific guidance for individual exercises. In
developing individual exercises, the exercise planning team should continue to engage the
appropriate senior leaders throughout the exercise planning cycle to achieve the leaders’ intent for
the exercise.
Preparedness Priorities
Preparedness priorities should be established by senior leaders and informed by risk and capability
assessments, findings, and corrective actions from previous events and external requirements like
regulations and grant guidance. Preparedness priorities should be comprehensive to meet whole
community needs and drive preparedness activities throughout the Integrated Preparedness Cycle.
For exercises, these priorities guide the development of exercise objectives, related training,
staffing, resourcing, and other planning activities to ensure individual exercises assess and validate
preparedness in an integrated and coordinated way.
Figure 2.2: The Relationship of
the Integrated Preparedness
Cycle and the HSEEP Cycle
Homeland Security Exercise and Evaluation Program
2-3
Integrated Preparedness Planning Workshop (IPPW)
The Integrated Preparedness Planning Workshop (IPPW) is a meeting that establishes the strategy
and structure for an exercise program, in addition to broader preparedness efforts, while setting
the foundation for the planning, conduct, and evaluation of individual exercises. This meeting
occurs on a periodic basis depending on the needs of the program and any grant or cooperative
agreement requirements.
Purpose
The purpose of the Integrated Preparedness Planning Workshop (IPPW) is to use the guidance
provided by senior leaders to identify and set preparedness priorities and develop a multi-year
schedule of preparedness activities.
The process confirms:
• Coordination of whole community initiatives;
• Prevention of duplication of efforts;
• Assurance of the efficient use of resources and funding; and
• Avoidance of overextending key agencies and personnel.
During the Integrated Preparedness Planning Workshop (IPPW), participation from the whole
community ensures preparedness activities are included in the program’s priorities.
Participation
When identifying stakeholders, program managers should consider individuals from
jurisdictions/organizations throughout the whole community, including but, not limited to:
• Senior leaders responsible for providing direction and guidance for preparedness priorities,
planning activities, and those responsible for providing resources to support preparedness
efforts;
• Relevant planning, training, exercise, and grant program managers who would be
responsible for carrying out the activities identified during the workshop;
• Individuals with knowledge of the community’s/organization’s risks and capabilities;
• Representatives from relevant disciplines that would be part of the exercises or any real-
world incidents, including appropriate local, regional, or federal department and agency
representatives;
• Individuals with administrative responsibility relevant to exercise conduct;
• Advocates for individuals with disabilities and access and functional needs
7
; and
• Community representatives to include businesses, the healthcare sector, volunteer
organizations, nongovernmental organizations, nonprofits, faith-based, and social support
organizations.
7
“Access and functional needs” includes children, seniors, racially and ethnically diverse communities, and people with limited English proficiency.
Homeland Security Exercise and Evaluation Program
2-4
Conduct of the Integrated Preparedness Planning Workshop (IPPW)
During conduct of the Integrated Preparedness Planning Workshop (IPPW), participants review
and consider various factors to develop the preparedness priorities and the multi-year schedule.
Guidance from senior leaders confirms whole community involvement in this process, to include
the factors listed in Table 2.1.
Table 2.1: Factors for Consideration in Developing Exercise Program Priorities
Focus Factors to Consider
Threats and Hazards
▪ Jurisdictional threats and hazards
▪ National threats and hazards
▪ Hazard vulnerability analysis
▪ Risk assessments
Areas for
Improvement and
Capabilities
▪ Real-world incident corrective actions
▪ Exercise corrective actions
▪ Identified and/or perceived areas for improvement
▪ Validating training efforts
▪ Newly acquired capabilities and resources
External Sources and
Requirements
▪ Industry reports
▪ State or national preparedness reports
▪ Homeland security strategies
▪ Grants or funding-specific requirements
Accreditation
Standards and
Regulations
▪ Accreditation requirements
▪ Local, state, and federal regulations
Utilizing the above factors, the workshop facilitator leads group discussions to review program
priorities and develop a multi-year schedule. Table 2.2 shows the conduct of the Integrated
Preparedness Planning Workshop (IPPW).
Table 2.2: Conduct of the Integrated Preparedness Planning Workshop (IPPW)
Element Aspects and Activities
Focus
A forum to identify priorities and the strategies employed to achieve those priorities,
whether that’s coordinating and integrating current activities in an organized manner
or identifying areas where additional activities are needed.
Homeland Security Exercise and Evaluation Program
2-5
Element Aspects and Activities
Discussion
Points
▪ Incorporate senior leader and stakeholder guidance
▪ Review jurisdiction/organization capabilities
▪ Collect and assemble data from the Integrated Preparedness C
ycle (plan,
organize/equip, train, exercise, evaluate/improve)
▪ Review new and revised plans, policies, and procedures
▪ Review corrective actions from previous exercises (AAR/IP)
▪ Consider using multiple exercise types to accomplish multi-year goals
▪ Identify funding
▪ Discuss scheduling conflicts
▪ Review program reports
Tools
▪ Read-ahead material
▪ Meeting agenda documents
▪ Presentations materials
▪ Integrated Preparedness Planning Workshop (IPPW) guide
▪ Integrated Preparedness Plan (IPP) template
Outcomes
▪ Prioritization and schedule of future preparedness activities
▪ Development of the Integrated Preparedness Plan (IPP)
Follow-up
▪ Distribution of finalized Integrated Preparedness Plan (IPP)
▪ Periodic review of current Integrated Preparedness Plan (IPP)
At the end of the Integrated Preparedness Planning Workshop (IPPW), program managers will
have a clear understanding of specific multi-year preparedness priorities and any available
information on previously planned preparedness activities that align to those priorities. Once the
preparedness priorities are outlined, stakeholders develop the multi-year Integrated Preparedness
Plan (IPP).
Integrated Preparedness Plan (IPP)
The Integrated Preparedness Plan (IPP) is a document for combining efforts across the elements
of the Integrated Preparedness Cycle to make sure that a jurisdiction/organization has the
capabilities to handle threats and hazards. Through effective program management, each planning,
organizing/equipping, training, exercising, and evaluating/improving activity becomes a
supporting component of the larger preparedness priorities. These preparedness priorities help
exercise planners design and develop a multi-year exercise program of individual exercises that
target resources to where they are most useful. Reference the National Preparedness Goal for
information regarding how resources are critical for those responsible in delivering capabilities
toward national preparedness. This progressive approach ensures jurisdictions/organizations select
the appropriate exercise type as described in the following sections.
Progressive Approach
A progressive, multi-year exercise program enables jurisdictions/organizations to participate in a
series of increasingly complex exercises, with each exercise building upon the previous one.
Homeland Security Exercise and Evaluation Program
2-6
Regardless of exercise type, each exercise within the series is linked to a set of common program
priorities and designed to test associated capabilities. By identifying other preparedness activities
or requirements as part of the integrated preparedness planning process, jurisdictions/organizations
address known shortfalls prior to exercising capabilities.
Discussion-Based Exercises
Discussion-based exercises include seminars, workshops, tabletop exercises (TTXs), and games.
These types of exercises familiarize players with or develop new plans, policies, procedures, and
agreements. Discussion-based exercises focus on strategic, policy-oriented issues, and facilitators
or presenters lead the discussion, keeping participants moving towards meeting the exercise
objectives. The following tables (Table 2.3,
Table 2.4,Table 2.5, and Table 2.6) provide the important information for each type of discussion-
based exercise:
Table 2.3: Discussion-Based Exercise Types: Seminar
Seminar:
A discussion-based exercise that orients participants to or provides an overview of authorities, strategies,
plans, policies, procedures, protocols, resources, concepts, and ideas.
Element Considerations and Activities
Purpose
▪ Provide a common framework of understanding
▪ Provide a good starting point for developing or making major changes to existing
plans, policies, or procedures
Structure
▪ Usually in the form of multiple presentations, subject-matter expert (SME)
panels, or case study discussions
▪ Lecture-based
▪ Led by a seminar facilitator/presenter
▪ Limited feedback or interaction from participants
Participant
Goals
▪ Gain awareness of or assess interagency capabilities or inter-jurisdictional
operations
▪ Set objectives for future capabilities
Conduct
Characteristics
▪ Minimal time constraints
▪ Effective for small or large groups
Outcomes
▪ A report that captures the discussion, issues raised, and (if appropriate) action
items that will address these issues
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Homeland Security Exercise and Evaluation Program
2-7
Table 2.4: Discussion-Based Exercise Types: Workshop
Workshop:
A discussion-based exercise often employed to develop policy, plans, or procedures.
Element Considerations and Activities
Purpose
▪ Increased participant interaction with a focus on achieving or building a product
▪ Should have clearly defined objectives, products, or goals and focus on a specific
issue
Structure
▪ A group of individuals in a venue conducive to discussion
▪ Lectures, presentations, panel or case-
study discussions, or decision support
tools
▪ Facilitated working breakout sessions
▪ Led by a workshop facilitator/presenter
Participant
Goals
▪ Develop a product as a group
▪ Obtain consensus
▪ Collect or share information
Conduct
Characteristics
▪ Effective for small and large groups
▪ Broad attendance by relevant stakeholders
▪ Conducted based on clear objectives/goals
▪ More participant discussion than lecture-based
▪ Frequently uses breakout sessions to explore parts of an issue with similar groups
Outcomes
▪ Emergency Operations Plans
▪ Mutual Aid Agreements
▪ Standard Operating Procedures
▪ Continuity of operations plans
▪ Workshop Summary Report
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Table 2.5: Discussion-Based Exercise Types: Tabletop Exercise
Tabletop Exercise (TTX):
A discussion-based exercise in response to a scenario, intended to generate a dialogue of various issues
to facilitate a conceptual understanding, identify strengths and areas for improvement, and/or achieve
changes in perceptions about plans, policies, or procedures.
Element Considerations and Activities
Purpose
▪ Generate discussion of various issues regarding an exercise scenario
▪
Facilitate conceptual understanding, identify strengths and areas for
improvement, and/or achieve changes in perceptions
Homeland Security Exercise and Evaluation Program
2-8
Element Considerations and Activities
Structure
▪ Scenario is presented to describe an event at a simulated time
▪ Players apply their knowledge and skills to a list of problems presented by the
facilitator
▪ Problems are discussed as a group, and resolution m
ay be reached and
documented for later analysis
▪ Plenary or breakout session(s)
▪ Discussion led by a facilitator(s)
▪ Presentation
Participant
Goals
▪ Enhance general awareness
▪ Enhance roles and responsibility understanding
▪ Validate plans and procedures
▪ Discuss concepts and/or assess types of systems in a defined incident
Conduct
Characteristics
▪ Requires an experienced facilitator
▪ In-depth discussion
▪ Problem-solving environment
▪ All participants should be encouraged to contribute to the
discussion and be
reminded that they are making decisions in a no-fault environment
Outcomes
▪ Recommended revisions to current plans, policies, and procedures
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Table 2.6: Discussion-Based Exercise Types: Game
Game:
A discussion-
based exercise that is a structured form of play designed for individuals or teams in a
competitive or noncompetitive environment. It is an event players take part in and are guided by clear
rules, data, and procedures for its execution. Games are designed to depict an actual or hypothetical
situation to ensure that the participants make decisions and take actions that would be plausible. Games
can be used to reinforce training, stimulate team building, or enhance operational and tactical capabilities.
Element Considerations and Activities
Purpose
▪
Simulation of operations that explore the consequences of player decisions and
actions
▪ Identification of critical decision-making points is a major factor in the success
of evaluating a game
Structure
▪ Usually in an environment that can involve two or more teams, using rules, data,
and procedures designed to depict an actual or hypothetical situation
▪ Decision-making may be either slow and deliberate or rapid and more stressful,
depending on the exercise design and objectives
▪ The open, decision-based format of a game can incorporate “what if” questions
that expand exercise benefits
▪ Depending on the game’s design, the conseque
nces of player actions can be
either pre-scripted or decided dynamically
Homeland Security Exercise and Evaluation Program
2-9
Element Considerations and Activities
Participant
Goals
▪ Explore decision-making processes and consequences
▪ Conduct “what-if” analyses of existing plans
▪ Evaluate existing and potential strategies
Conduct
Characteristics
▪ No actual resources used
▪ Often involves two or more teams
▪
May include models and simulations of increasing complexity as the game
progresses
▪ May or may not include pre-scripted activities
Outcomes
▪ Validate plans, policies, and procedures or evaluating resource requirements
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Operations-Based Exercises
Operations-based exercises include drills, functional exercises (FE), and full-scale exercises
(FSE). These exercises validate plans, policies, procedures, and agreements; clarify roles and
responsibilities; and identify resource gaps. Operations-based exercises include a real-time
response such as initiating communications or mobilizing personnel and resources. The following
tables (Table 2.7, Table 2.8, and Table 2.9) provide the important information for each type of
operations-based exercise:
Table 2.7: Operations-Based Exercise Types: Drill
Drill:
An operations-based exercise often employed to validate a single operation or function.
Element Considerations and Activities
Purpose
▪ Coordinated, supervised activity to validate a specific function or capability in a
single agency/organization,
often employed to validate a single operation or
function
▪ Provide training on new equipment, validate procedures, or practice and maintain
current skills
Structure
▪ Can be standalone or used as a series of drills
▪ Clearly defined plans, procedures, and protocols need to be in place
Participant
Goals
▪ Evaluate new procedures, policies, and/or equipment
▪ Practice and maintain skills
▪ Prepare for future exercises
Conduct
Characteristics
▪ Immediate feedback
▪ Realistic environment
▪ Narrow focus
▪ Performance in isolation
▪ Results are measured against established standards
Homeland Security Exercise and Evaluation Program
2-10
Element Considerations and Activities
Outcomes
▪ Determine if plans can be executed as designed
▪ Assess whether more training is required
▪ Reinforce best practices
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Table 2.8: Operations-Based Exercise Types: Functional Exercise
Functional Exercise (FE):
An operations-based exercise is designed to test and evaluate
capabilities and functions while in a
realistic, real-time environment; however, movement of resources is usually simulated.
Element Considerations and Activities
Purpose
▪ Validate and evaluate capabilities, multiple functions and/or sub-functions, or
interdependent groups of activities
▪
Exercise plans, policies, procedures, and staffs involved in management,
direction command, and control functions
▪ Apply established plans, policies, and procedures under crisis conditions
Structure
▪ Events are projected through a realistic exercise scenario with event updates that
drive activity typically at the management level
▪ Controllers typically use a Master Scenario Events List (MSEL) to ensure
participant activity remains within predefined boundaries
▪ Evaluators observe behaviors and compare
them against established plans,
policies, procedures, and standard practices (if applicable)
Participant
Goals
▪ Validate and evaluate capabilities
▪ Focused on plans, policies, and procedures
Conduct
Characteristics
▪ Conducted in a realistic environment
▪ Usually simulated deployment of resources and personnel
▪ Use of Simulation Cell and Master Scenario Events List (MSEL)
▪ Simulators can inject scenario elements
▪ Include controller and evaluators
Outcomes
▪ Management evaluation of Emergency
Operations Center (EOCs), command
post, headquarters, and staffs
▪ Performance analysis
▪ Cooperative relationships are strengthened
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Homeland Security Exercise and Evaluation Program
2-11
Table 2.9: Operations-Based Exercise Types: Full-Scale Exercise
Full-Scale Exercise (FSE):
An operations-based exercise that is typically the most complex and resource-intensive of the exercise
types and often involves multiple agencies, jurisdictions/organizations, and real-
time movement of
resources.
Element Considerations and Activities
Purpose
▪ Often includes many players operating under cooperative systems such as the
Incident Command System (ICS) Unified Command
▪
Focus on implementing and analyzing the plans, policies, and procedures that
may have been developed in discussion-based exercises and honed during
previous, smaller exercises
Structure
▪ Events are projected through an exercise scenario with event updates that drive
activity at the operational level
▪ Involves multiple agencies, organizations, and jurisdictions
▪ Use of a MSEL drives player actions
▪ SimCell controllers inject scenario elements
▪ The level of support needed may be greater than that needed for other types of
exercises
▪ Conducted in a realistic environment to mirror a real incident by presenting
complex problems
Participant
Goals
▪ Demonstrate roles and responsibilities as addressed in plans and procedures
▪ Coordinate between multiple agencies, organizations and jurisdictions
Conduct
Characteristics
▪ Rapid problem solving; critical thinking
▪ Mobilization of personnel and resources
▪ Exercise site is usually large with many activities occurring simultaneously
▪ Site logistics require close monitoring
▪ Safety issues, particularly regarding the use of props and special effects, must be
monitored
▪ Demonstrate roles and responsibilities as addressed in plans and procedures
Outcomes
▪ Validate plans, policies, and procedures
▪ Evaluate resource requirements
▪ An After-Action Report (AAR)/Improvement Plan (IP)
Program Reporting
Program reporting is intended to provide senior leaders with an analysis of issues, trends, and key
outcomes from all exercises conducted as part of the exercise program, along with a continuous
update on corrective actions. Reporting is a communications tool to inform senior leaders and
guides the development of future preparedness activities. Program reporting, similar to the Rolling
Summary Report, is not a collection of After-Action Reports, but it does feed into the Integrated
Preparedness Plan (IPP) and is part of a continuous improvement process.
Homeland Security Exercise and Evaluation Program
2-12
Information in the report should:
• Inform senior leaders on the exercise program;
• Provide data to support preparedness assessments and reporting requirements; and
• Enable exercise planners to modify objectives and the exercise schedule to reflect
knowledge gathered from the exercises.
Manage Exercise Program Resources
An effective exercise program utilizes the full range of available resources. Exercise program
managers should plan an exercise budget, appropriate program staffing, and other needed
resources.
Exercise Budget Management
Effective budget management is essential to the success of an exercise program and for exercise
managers to maintain awareness of available resources and expected expenditures. In developing
and maintaining an exercise program budget, exercise program managers work with stakeholders
to identify financial resources and define monitoring and reporting requirements for individual
exercises.
Staffing
Exercise program managers should identify the administrative and operational staff needed to
oversee the exercise program and to carry out the necessary exercises to improve capabilities
across the whole community. The Integrated Preparedness Plan (IPP) is one tool for determining
staffing needs, in addition to grant funds or other programmatic considerations. Exercise program
managers should identify gaps between staffing availability and staffing needs and consider
alternative means of obtaining needed staff.
Other Resources
Exercise program managers may consider other resources that can support exercises, such as:
• Information technology (for example, modeling and simulation capabilities, etc.);
• Subject Matter Experts (SMEs);
• Exercise tools and resources (for example, document templates, etc.);
• Materials from previous exercises;
• Training courses;
• Mutual aid agreements, Memoranda of Understanding, and Memoranda of Agreement;
• Technical assistance; and
• Equipment and props (for example, smoke machines, mannequins, moulage kits, etc.).
Homeland Security Exercise and Evaluation Program
3-1
3. Exercise Design and Development
Overview
In the design and development phase, exercise planners use the intent and guidance of senior
leaders and the program priorities to plan individual exercises. Exercise planning teams apply the
guidance to shape the major concepts and planning considerations for a single exercise or series of
exercises.
The steps of exercise design and development include:
• Setting the exercise foundation by reviewing senior leader guidance, the Integrated
Preparedness Plan (IPP), and other factors to develop exercise-specific objectives and align
capabilities within a scenario;
• Defining exercise planning activities;
• Developing the Exercise Planning Team;
• Designing the exercise to include the purpose, scope, objectives, evaluation parameters,
scenario, documentation, and planning for media and public affairs; and
• Developing the exercise to include planning for discussion-based and operations-based
logistics, facilitation, control, exercise evaluation, and exercise conduct.
Exercise planners apply and adapt the HSEEP doctrine to exercise design and development to meet
specific needs.
Exercise Foundation
The exercise foundation is a set of components that drive the
exercise design and development process. The program manager
considers the preparedness priorities from the Integrated
Preparedness Cycle as outlined in the Integrated Preparedness
Plan (IPP) to design and develop individual, or a series of,
exercises. The transition is shown in Figure 3.1.
The exercise planning team reviews and considers the following
items before the design phase:
• The senior leader intent and guidance;
• The Integrated Preparedness Plan (IPP);
• Any relevant After-Action Reports/Improvement Plans
(AARs/IPs) from real-world incidents and exercises;
• Program reports from the exercise program managers;
• The Threat and Hazard Identification and Risk Assessment (THIRA) or other threat,
hazard, and risk assessments;
• The Stakeholder Preparedness Review, or other capability assessments;
Figure 3.1: Individual Exercise
Design and Development Process
Homeland Security Exercise and Evaluation Program
3-2
• The organization/jurisdiction plans, policies, and procedures; and
• Any grant or cooperative agreement requirements.
Exercise Planning Activities
This section describes the types of planning activities most useful in exercise design and
development. The primary planning activities include establishing an exercise timeline and
assembling the exercise planning team. The exercise planning team is typically formed following
the Concept and Objectives (C&O) Meeting. At that time, the exercise planning team members
decide the type and number of additional planning meetings necessary.
Exercise Planning Meetings
Meetings serve as a structured event or forum for completing the major milestones of exercise
design and development. The range of planning meetings are described in the following tables
(Table 3.1, Table 3.2, Table 3.3, Table 3.4, and Table 3.5), though not all meetings may be used
for every exercise. While the planning meetings are critical for accomplishing tasks as a planning
team, much of the development work will be accomplished between formal meetings.
Table 3.1: Planning Meeting Types: Concept and Objectives (C&O) Meeting
Concept & Objective (C&O) Meeting:
A meeting that is the formal beginning of the exercise planning process. It is held to identify the scope
and exercise objectives. For less complex exercises and for organizations with limited resources, the
C&O Meeting can be conducted in conjunction with the Initial Planning Meeting (IPM).
Element Considerations and Activities
Meeting Focus
The formal beginning of the planning process. Program priorities and objectives
are determined during the C&O Meeting.
Discussion Points
▪ Exercise scope
▪ Proposed objectives and aligned capabilities linked to threats and risks
▪ Location, date, and duration
▪ Participants and anticipated extent of play
▪ Exercise Planning Team makeup
▪ Assumptions and artificialities
▪ Control and evaluation concepts
▪ Security organizations and structure
▪ Available resources
▪ Logistics
▪ Planning timeline and milestones
▪ Local issues, concerns, and sensitivities
▪ Engagement with senior leaders for guidance and intent
Exercise Tools
▪ Meeting agenda and briefing, which includes the background and rationale for
conducting the exercise
Homeland Security Exercise and Evaluation Program
3-3
Element Considerations and Activities
Exercise
Outcomes
▪ Exercise concept
▪ Exercise timeline (group consensus)
▪ Extent of participant play
▪ Identification of planning team members
▪ Planning timeline, milestones, and meeting dates
Follow-up
Compile and distribute the C&O Meeting minutes and include the next meeting
time, date, and location.
Table 3.2: Planning Meeting Types: Initial Planning Meeting (IPM)
Initial Planning Meeting (IPM):
A meeting that is held to refine the scope and exercise objectives. For less complex exercises and for
organizations with limited resources, the C&O Meeting can be conducted in conjunction with the IPM.
Element Considerations and Activities
Meeting Focus
Formal beginning of the development phase. The IPM also focuses on refining the
scope and the objectives for the exercise.
Discussion Points
▪ Clearly defined objectives and aligned capabilities linked to threats and risks
▪ Evaluation requirements, including Exercise Evaluation Guide (EEG)
capability targets and critical tasks
▪ Relevant plans, policies, and procedures to be evaluated
▪ Exercise scenario
▪ Modeling and simulation
▪ Participants extent of play
▪ Optimum duration of the exercise
▪ Exercise planners’ roles and responsibilities
▪ Decision to record exercise proceedings (audio or video)
▪ Local issues, concerns, or sensitivities
▪ Discussion points typically covered during a C&O Meeting, if one is not held
▪ Consensus regarding the date, time, and location for next meeting
▪ Re-
engagement with senior leaders to ensure alignment with guidance and
intent
Exercise Tools
▪ Read-ahead document
▪ Agenda material
▪
Briefing for presenting an overview of the exercise and meeting discussion
points
▪ Threat and hazard identification assessments
▪ Proposed room layout (discussion-based)
▪ Map of proposed venue sites (operations-
based) with description of local
environment
▪ Copy of proposed timeline and milestones for design and development
▪ Copies of presentation briefing for meeting
Homeland Security Exercise and Evaluation Program
3-4
Element Considerations and Activities
Exercise
Outcomes
▪ Any outcomes from C&O Meeting; if a C&O Meeting was not conducted
▪ Clearly defined objectives and aligned capabilities
▪ Initial capability targets and critical tasks (reviewed and confirmed prior to the
next meeting)
▪ Scenario variables (threat, scope, venue, conditions)
▪ List of participating organizations and extent of play
▪ Develop exercise draft documentation (Situation Manual [SitMan], Exercise
Plan [ExPlan], etc.)
▪ Identification and availability of all source documents (plans, policies,
procedures) needed for exercise documentation
▪ Refined exercise planning timeline
▪ Identification of available subject-matter experts (SMEs) (scenario vetting)
▪ Preferred communication methods among planning team
▪ Clearly identified and assigned responsibility for logistical issues
▪ List of tasks to be accomplished by next planning meeting, to include date and
responsible planning team member
▪ Agreed-upon date, time, and location of next planning meeting
Follow-up
▪ Compile and distribute the IPM, meeting minutes, including the next meeting
time, date, and location
▪ Between meetings the planning team collaborates on assignments and prepares
draft exercise documentation
▪ Distribute draft documentation prior to the next scheduled meeting
Table 3.3: Planning Meeting Types: Midterm Planning Meeting (MPM)
Midterm Planning Meeting (MPM):
A meeting that serves as a forum to develop exercise scenario details and timeline.
Element Considerations and Activities
Meeting Focus
The MPM is a continuation of the development phase. This meeting is primarily
focused on scenario development and serves as a check-in for exercise product
development.
Discussion
Points
▪ Comments on draft exercise documentation
▪ Construction of the scenario timeline
▪ Development of the Master Scenario Events List (MSEL), if a MSEL meeting
is not scheduled
▪ Identification of exercise venue artificialities and/or limitations
▪ Agreement on final logistical items
▪ Assignment of additional responsibilities
▪ Re-engage with senior leaders to ensure alignment with guidance and intent
Homeland Security Exercise and Evaluation Program
3-5
Element Considerations and Activities
Exercise
Tools
▪ Agenda material
▪ Briefing for presenting an overview of the exercise and meeting discussion
points
▪ IPM minutes
▪ Draft scenario timeline
▪ Draft documentation (Facilitator Guide, Controller/Evaluator [C/E] Handbook,
etc.)
▪
Other selected documentation needed to illustrate exercise concepts and
provide planning guidance
Exercise
Outcomes
▪ Reviewed exercise documentation (SitMan, ExPlan, etc.)
▪ Draft Facilitator Guide or C/E Handbook, including EEGs
▪ Well-developed scenario to include injects (if no MSEL Meeting is scheduled)
▪ Agreement on the exercise site
▪ Identified logistics planning requirements
▪ Finalization of date, time, and location on the MSEL Meeting and Final
Planning Meeting (FPM)
Follow-up
▪ Compile and distribute the MPM meeting minutes, including the next meeting
date, time, and location
▪ Between meetings the planning team collaborates on assignments and prepares
draft exercise documentation
▪ Distribute draft documentation prior to the next scheduled meeting
Table 3.4: Planning Meeting Types: Master Scenario Events List (MSEL) Meeting
Master Scenario Events List (MSEL) Meeting:
A meeting for operations-based exercises that serves as a forum to build the MSEL in detail.
Element Considerations and Activities
Meeting Focus
The MSEL meeting focuses on reviewing the timeline and developing the MSEL.
Exercise planners from participating departments, agencies, jurisdictions, and
organizations are included to identify activities that must occur during the exercise.
Discussion
Points
▪ Tasks, conditions, and standards required to meet objectives
▪ Key events and critical tasks
▪ Event originator, target player, expected player actions, and timeframe
▪ Contingency injects to prompt player action (if needed)
Exercise
Tools
▪ Agenda material
▪ Previous meeting minutes
▪ Draft exercise documentation
▪ Applicable plans, policies, and procedures
▪ Agreed-upon template used to create the MSEL
Homeland Security Exercise and Evaluation Program
3-6
Element Considerations and Activities
Exercise
Outcomes
▪ Key event injects and delivery timeline identified
▪ Assignment of responsibility for conducting remaining events
▪ Revisions of draft scenario-based documentation
▪ Timeline for completion
Follow-up
▪ Compile and distribute the MSEL meeting minutes, including the next meeting
date, time, and location
▪ Between meetings the planning team collaborates on assignments and prepares
draft exercise documentation
▪ Distribute draft documentation prior to next scheduled meeting
Table 3.5: Planning Meeting Types: Final Planning Meeting (FPM)
Final Planning Meeting (FPM):
A meeting that serves as the formal end of the exercise planning process. It is held to finalize exercise
documentation and logistics.
Element Considerations and Activities
Meeting Focus
Should be conducted for all exercises to ensure that all elements of the exercise are
ready for conduct. Prior to the FPM, the planning team receives final drafts of all
exercise materials. No major changes to the exercise’s design, scope, or supporting
documentation should take place at or following the FPM.
Discussion Points
▪ Conduct a comprehensive final review
▪ Approve all remaining draft documents (for example, SitMan, MSEL, C/E
Handbook) and presentation materials
▪ Resolve any open planning issues and identify last-minute concerns
▪ Review all exercise logistical activities (for example, schedule, registration,
attire, special needs)
▪ Re-engage with senior leaders to ensure alignment with guidance and intent
Exercise Tools
▪ Agenda materials
▪ Briefing for presenting an overview of the exercise and meeting discussion
points
▪ Previous meeting minutes from the IPM, the MPM, and the MSEL, if needed
▪ All draft exercise documents and documentation
▪ Previously finalized documents
Exercise
Outcomes
▪ Final approval of exercise documents and material for production
▪ Identified issues resolved
▪ Attendees understand and approve exercise processes and procedures
▪ Task assignments and logistical elements, including facilities, equipment, and
schedules are confirmed
Homeland Security Exercise and Evaluation Program
3-7
Element Considerations and Activities
Follow-up
▪ Finalize all publications
▪ Prepare all supporting materials
▪ Rehearse presentations and briefings
▪ Prepare to conduct the exercise
▪ Disseminate documentation and any additional instructions to all appropriate
personnel prior to the exercise
Exercise Planning Team Considerations
The exercise planning team manages and is responsible for exercise design, development, conduct,
and evaluation. The membership of an exercise planning team fits the type and scope of an exercise
and is a manageable size yet represents the full range of participating jurisdictions/organizations
and stakeholders. For multi-jurisdictional/organizational exercises, planning team members
include representatives from each functional area or relevant discipline.
Typically, planning team members are not exercise players; they are trusted agents. With limited
resources, exercise planning team members may act as both planners and players but must be
careful not to divulge sensitive exercise information. Planning team members also help with
producing and distributing pre-exercise materials and conducting exercise planning meetings,
briefing, and training sessions.
To design and develop exercises most effectively, exercise planning teams:
• Identify and understand the desired objectives and associated capabilities for the exercise;
• Adhere to a clear organizational structure with roles and responsibilities;
• Use proven project management practices, processes, and tools, such as project plans and
timelines, status reports, and other communications;
• Incorporate evaluation planning and SMEs during design and development;
• Use SMEs to develop a realistic and challenging scenario; and
• Identify the type of security clearance levels needed (if required).
Community representative and individual with disabilities and access and functional needs should
be invited to participate throughout the planning process. In doing so, exercise planners incorporate
a broad set of perspectives and promote early understanding of roles, responsibilities, and planning
assumptions.
Exercise Planning Team Positions
Regardless of the scale and complexity of an exercise, the exercise planning team should adhere
to a coherent organizational structure that delineates roles and responsibilities. An Exercise
Director provides strategic oversight and direction to the exercise planning team leader, who
manages the exercise planning team. In developing a structure, the Exercise Director considers
adequate span of control and requirements needed for the exercise type. The same personnel may
fulfill multiple roles, depending on available resources. The functional organizational structure
Homeland Security Exercise and Evaluation Program
3-8
shown in Figure 3.2 can vary to reflect the scope of the exercise and the resources and personnel
available to support the design and development, conduct, and evaluation of the exercise.
Figure 3.2: Example of a Functional Organizational Structure for an Exercise Planning Team
Table 3.6 describes the functions of an exercise planning team.
Table 3.6: Functions of an Exercise Planning Team
Functions Description
Exercise Director
Responsible for the strategic oversight and direction on the HSEEP Cycle for an
individual exercise.
Exercise Planning
Team Lead
Responsible for the management of the exercise planning team and coordinates
with the Exercise Director and senior leaders.
Safety/Security
R
esponsible to ensure that the exercise is conducted in a safe and secure
environment.
Homeland Security Exercise and Evaluation Program
3-9
Functions Description
Design and
Development
Responsible for compiling and developing all exercise background and
facilitation or control, providing technical or functional expertise for scenario
development, developing the simulation construct as applicable, and working
with the evaluation and resource/support functions to ensure all exercise design
and development needs are met.
Scenario
Responsible for developing a model or outline, either written or depicted by an
event timeline, of the simulated sequence of events that drive the player’s
discussion or actions.
Control
(Operations-Based
Exercise)
Responsible for the control structure needed for an operations-based exercise.
This includes the master control cell, individual venue control, and simulation
and modeling requirements. Will work with the evaluation component in
completing the C/E Handbook.
Facilitation
(Discussion-Based
Exercise)
Responsible for the facilitator guide for discussion-based exercises, once the
conduct and evaluation materials have been completed; managing the
recruitment and training of facilitators as necessary; and working with evaluation
planners to confirm the facilitator is properly directed regarding objectives and
areas of focus.
Modeling and
Simulation/GIS
Responsible for coordinating the use of modeling, simulation, and GIS for both
discussion- and operations-
based exercises. Ensures modeling is coordinated
with the scenario and ground truth elements and that simulation is coordinated
with the control element during exercise design and development.
Master Scenario
Events List (MSEL,
Operations-Based
Exercise)
Responsible for working with all other elements of exer
cise design and
development, as well as nonparticipating jurisdictions and subject-matter experts
(SMEs), to design and develop the MSEL timeline and events to properly drive
exercise play.
Evaluation
Responsible for the development of the overall exercise evaluation construct and
all evaluation documentation; assigns evaluation staffing in conjunction with the
exercise planning team. Collects and reviews the policies, plans, and procedures
to be examined during exercise conduct to finalize exercise objectives, match
capability targets to exercise objectives, and determine critical tasks to be
evaluated in order to complete exercise evaluation guides.
Evaluation Plan
Responsible for the development of the evaluation plan that provides guidance,
instructions, and structure on evaluating and observing during an exercise.
Exercise Evaluation
Guide
Responsible for the development of the Exercise Evaluation Guides (EEGs) used
for the exercise.
Resource Support
Responsible for logistics, administra
tion, facility, and finance support for
planning meetings, exercise documentation, and other planning considerations.
Facilities
Responsible for coordinating the needed venue locations for exercises, including
meeting rooms, conference centers, training l
ocations, and locations for
operations-based exercises.
Homeland Security Exercise and Evaluation Program
3-10
Functions Description
Administration
Responsible for meeting scheduling and administrative support, such as
distributing meeting invitations, read-
ahead materials, and meeting minutes;
providing and collecting sign-
in sheets; and note taking during planning
meetings.
Logistics
Responsible to ensure venues are adequately equipped with audiovisual
equipment, tables, and chairs and coordinates refreshments if required.
Exercise Design
The core components of design include clarifying the purpose, establishing the exercise scope,
setting objectives, developing evaluation parameters, creating a scenario, developing
documentation, and determining media and public affairs guidance. The exercise planning
meetings serve as the principal mechanism for completing the major steps of exercise design.
Exercise Purpose
The purpose of the exercise derives from a set of key factors determined from:
• A multi-year Integrated Preparedness Plan (IPP);
• Any existing plans, policies, and procedures;
• THIRA or other threat, hazard, or risk assessments;
• Any past exercise or real-world AAR/IPs; and
• Any grant or cooperative agreement requirements.
A review of the factors ensures the exercise builds and sustains a jurisdiction’s/organization’s
capabilities while taking prior circumstances into consideration during the exercise design process.
Once the review is complete, each exercise is designed to meet the intent of the exercises within
the Integrated Preparedness Plan (IPP).
Exercise Scope
Determining the exercise scope enables planners to “right-size” an exercise to meet the objectives
while staying within the resource and personnel constraints of the participating jurisdictions/
organizations.
Table 3.7: Elements that Define Scope
Element Description
Exercise Type
Exercise type is based on the exercise purpose. For instance, if the intent is to review
and discuss a new policy, plan, or set of procedures, a discussion-based exercise may
be appropriate. If the intent is to assess the responders’ ability to implement a plan,
policy, or set of procedures, an operations-based exercise may be appropriate.
Homeland Security Exercise and Evaluation Program
3-11
Element Description
Participation
Level
Participation level of jurisdictions/organizations provides information such as dates
and times of participation, number of personnel involved, positions of personnel
(operational, agency directors), available resources,
and the intended outcomes.
Participation level limitations may include scheduling conflicts, real-world incidents,
or other competing requirements. Consider simulation to alleviate the participation
level limitations. To ensure a successful exercise, the use of an Extent of Play
Agreement outlines the agreement to participate.
Exercise
Location
Exercise location is based on the exercise type. Discuss and decide suitable locations
for the exercise to determine the scope or define artificialities required to simulate real-
world incidents.
Exercise
Duration
Exercise duration is based on resources and exercise objectives. When selecting the
exercise duration, the planning team should determine how long the exercise will take
to address the exercise objectives effectively.
Other
Considerations
Other considerations may determine exercise activities that are useful to a jurisdiction/
organization. Clearly defining the exercise scope early in the design process helps
exercise planners keep the exercise to a manageable and realistic level.
Exercise Objectives
Based on direction from senior leaders, the exercise planning team selects one or more exercise
program priorities on which to focus an individual exercise. These priorities drive the development
of exercise objectives that a jurisdiction/organization wishes to achieve during an exercise. The
objectives are aligned to a common set of capabilities which enables:
• A systematic tracking of progress throughout exercise programs and cycles;
• A standardized exercise data collection informing preparedness assessments; and
• The fulfillment of grant or funding-specific reporting requirements.
Figure 3.3 shows how the exercise program priorities are developed and the relationship between
the exercise program priorities, exercise objectives, and capabilities.
Figure 3.3: Priorities, Objectives, and Capabilities
Exercise objectives should incorporate senior leaders’ intent; exercise participants’ plans, policies,
and procedures; operating environment; corrective actions from previous exercises and real-world
incidents; and desired outcomes. The exercise planning team should select a reasonable number of
objectives to facilitate effective scenario design, exercise conduct, and evaluation. An objective
should be specific, measurable, achievable, relevant, and time-bound (SMART), as shown in
Figure 3.4.
Homeland Security Exercise and Evaluation Program
3-12
Figure 3.4: SMART Guidelines for Exercise Objectives
Example of a SMART Objective
Demonstrate the ability of Central City staff to receive, develop, and disseminate a
public alert utilizing Central City’s notification system regarding a HAZMAT spill
within 15 minutes of initial notification, in accordance with the Communications
Annex in the Emergency Operations Plan.
Evaluation Parameters
The development of exercise evaluation parameters begins early in the exercise process and helps
to guide the development of the exercise scenario, discussion questions or event timeline, and
evaluation documentation. Once the exercise objectives are aligned to capabilities, the exercise
planning team identifies the linked capability targets and critical tasks. Chapter 5: Exercise
Evaluation provides additional details regarding capability targets and critical tasks.
Scenario
A scenario is a model or outline of the simulated sequence of events that drives participant
discussions or actions. The scenario can be a written narrative or depicted by an event timeline.
The scenario should be plausible, realistic, and challenging so participants can meet the objectives.
After developing the scope and objectives, the exercise planning team should select and develop
the scenario to enable an exercise to assess objectives and capabilities.
The elements of a scenario are:
• The conditions allowing players to demonstrate their ability to meet exercise objectives;
• The technical details necessary to accurately depict scenario conditions and events (for
example, the date and time of event and damage resulting from the event); and
• A general context or comprehensive narrative.
Homeland Security Exercise and Evaluation Program
3-13
Threat, Hazard, and Risk
The type of threat, hazard, or risk defines a scenario allowing for assessment of the objectives and
capabilities of the exercise. Threat, hazard, and risk identification originates from the
organization’s/jurisdiction’s THIRA or other threats, hazards, or risk assessments. Risk analyses
enable jurisdictions/organizations to identify potential events that stress preparedness capabilities.
Modeling and Simulation
Modeling and simulation bring versatility, cost savings, and fidelity to exercises when
incorporated into the development of the scenario and overall exercise design. A model is a
representation of a system at a point in time or space intended to expand an understanding of the
real system. Simulation is a method of implementing the performance of a model, or combination
of models, over time. Modeling and simulation may apply to situations where achieving reality is
difficult. The use of modeling and simulation realistically replicates variables such as disease
propagation, radiation, and chemical attacks.
Exercise Documentation
Exercise documentation provides the framework for exercise conduct and evaluation. The
documentation helps the planning team apply the HSEEP Cycle. Table 3.8 and Table 3.9 list the
most commonly used exercise design and development documents, identified by the exercise type
and relevant audience.
Table 3.8: Discussion-Based Exercise Documentation
Document Description Distribution
Situation
Manual
(SitMan)
A document that provides background information and serves as
the primary reference material for exercise participants (core
document for discussion-based exercises).
All
Participants
Player Handout
A document that supplements or replaces the Situation Manual and
presentation, providing players a quick-reference guide,
sometimes referred to as a “placemat.”
Players
Facilitator
Guide
A document that outlines instructions and key issues discussed and
utilized by the exercise facilitator to move participants through
exercise play.
Exercise
Facilitator(s)
Presentation
A multimedia display that provides background information for
participants during exercise conduct.
All
Participants
Exercise
Evaluation
Guide(s) (EEG)
A document that captures information specifically related to the
evaluation requirements developed by the exercise planning team.
The EEG provides evaluators with a standardized tool to guide
data collection and capture performance results.
Since each
jurisdiction/organization has unique targets and critical tasks,
EEGs are developed specific to their plans, policies, procedures,
and protocols.
Evaluators
Participant
Feedback Form
A document that provides a mechanism to collect input from all
participants following an exercise and supports the development
of an After-Action Report (AAR) and Improvement Plan (IP).
All
Participants
Homeland Security Exercise and Evaluation Program
3-14
Table 3.9: Operations-Based Exercise Documentation
Document Description Distribution
Exercise Plan
(ExPlan)
A document that provides general information to exercise
participants, including exercise objectives and scope, not including
the scenario information.
Players and
Observers
Player Handout
A document that supplements the Exercise Plan and provides
players with a quick-reference guide; sometimes referred to as “a
pocketsize trifold or bifold.”
Players
Ground Truth
A document comprised of the detailed elements of a scenario that
must remain consistent during exercise development and conduct
to ensure that realism is maintained,
and objectives can be
achieved.
All
Participants
Controller/
Evaluator (C/E)
Handbook
A
document that provides specific exercise information and
guidance for controllers and evaluators, may be a standalone
document or supplement to the Exercise Plan, and adjusted
accordingly.
Controllers
and
Evaluators
Master Scenario
Events List
(MSEL)
A document or system that is a chronological timeline of expected
actions and scripted events to be injected into exercise play by
controllers to generate or prompt player activity. It ensures
necessary events happen so that all objectives are met. Larger,
more complex exercises may also use
a procedural flow, which
differs from the MSEL in that it contains only expected player
actions or events. The MSEL links simulation to action, enhances
the exercise experience for players, and reflects an incident or
activity meant to prompt players to action.
Controllers,
Evaluators,
and Simulator
Extent of Play
Agreement
A document that is a formal written agreement outlining the
participating jurisdiction’s/organization’s level of play during an
exercise.
Exercise
Planning
Team
Exercise
Evaluation
Guide(s) (EEG)
A document that captures information specifically related to the
evaluation requirements developed by the exercise planning team.
The EEG provides evaluators with a standardized tool to guide
data collection and capture performance results.
Since each
jurisdiction/organization has unique targets and critical tasks,
EEGs are developed specific to their plans, policies, procedures,
and protocols.
Evaluators
Control Staff
Instructions
(COSIN)
A d
ocument for the Controller portion of the C/E Handbook for
large exercises that provides detailed guidelines for control and
simulation support and establishes a management structure for
these activities. This section provides guidance for controllers,
simulators, and evaluators on procedures and responsibilities for
exercise control, simulation, and support.
Controllers,
Simulators,
and Exercise
Support
Evaluation Plan
(EvalPlan)
A document for evaluators that provides guidance, instructions,
and structure on evaluating and observing during an exercise. It
also provides essential materials required to perform their specific
functions.
Evaluators
Homeland Security Exercise and Evaluation Program
3-15
Document Description Distribution
Participant
Feedback Form
A document that the exercise team uses to collect observed
strengths, areas for improvement, and input about exercise conduct
and logistics from participants.
All
Participants
Areas to consider when developing the exercise documentation are:
• Exercise Marking: Documents and messages should be clearly identified as exercise-
related products.
• Classification: Determine the importance of specific requirements for security.
Documents that are sensitive or classified need to be marked according to regulations or
guidance (for example, classified, For Official Use Only [FOUO], secret, etc.).
• Accessibility: Verify consideration and accessibility of presentations and documents (for
example, large print, compact disc, Braille), closed captioning or another form of text
display, or the provision of sign language interpreters.
• Distribution: Understanding the types of information and the dissemination of information
needs to be considered (for example, the printing and circulating classified scenario
details).
Table 3.10 describes other documentation used during an exercise where applicable.
Table 3.10: Other Exercise Documentation
Document Description Distribution
Table Tents
An item on each table identifying the functional area seated at that
table.
All Tables
Name Badge
An item that each participant wears to clearly identify their name
and agency/organization.
All
Participants
Liability
Waiver Form
A document that each actor should receive and sign prior to the start
of the exercise. Signing a waiver form releases liability for all
exercise planners and participants. Exercising entities should use
discretion when recruiting actors under the age of 18 years because
of additional challenges and concerns related to liability. If the
exercise requires volunteers younger than 18 years, parents or legal
guardians must sign their waiver forms.
Actors
Photo Consent
Form
A document for photographs prior to the exercise. Signing the form
releases permission for all exercise participants photographed,
allowing their photo to be used for informational and educational
purposes.
All
Participants
Weapons and
Safety Policy
A document for all exercises, where applicable, shall employ a
written weapons and safety policy in accordance with applicable
state or local laws and regulations. Exercise sponsors should
coordinate the application of a weapons and safety policy with the
appropriate safety and legal departments as necessary. All players
are expected to fully adhere to this policy; however, site security
personnel may continue to carry assigned weapons.
All
Participants
Homeland Security Exercise and Evaluation Program
3-16
Document Description Distribution
Symptomology
Card
An item containing the signs and symptoms that an exercise actor
will portray, as well as information helpful for players.
Actors and
Players
Press Release
A document that the exercise planning team develops for the media
and the public to inform them about the exercise.
Media and
Public
Planning for Media and Public Affairs
Media and public affairs can be important participants for an exercise. The exercise planning team
should work to incorporate media-related issues into exercise planning.
Press Release
Before an exercise, the exercise planning team may develop a written press release to disseminate
to media outlets, including web-based and social media outlets, observers, senior leaders, and other
very important persons (VIPs), as appropriate. The press release informs the media and the public
about general exercise information. The press release does not contain detailed scenario
information, such as the type of threat or hazard, nor contain information that might hinder meeting
exercise objectives if a participant reads the release.
The contents of a media or public information release include the following:
• An introduction, including sponsor and exercise program information;
• The exercise scope and objectives;
• Any general scenario information; and
• The participating agencies or disciplines.
Public Announcement
Public announcements should occur before any exercise involving public space or space viewable
to the public and help avoid public confusion. The announcement will also help the public avoid
congestion near the exercise site by providing suggestions for alternate routes. Examples of
announcements are local media, community alert and notification systems, and directly on signs
near the exercise site.
Media Policy
The agency or jurisdiction/organization sponsoring the exercise should decide whether to invite
media representatives to the exercise. If invited, media representatives conduct interviews with
exercise planners and participants prior to the exercise.
At discussion-based exercises, media representatives should not be present during the discussion
of any potentially sensitive information. Also, avoid the filming of exercise conduct, which may
hinder discussion or the flow of play.
Exercise Development
Exercise development involves planning for the critical elements of exercise logistics, facilitation,
control, and evaluation. Exercise logistics are important to create a smooth and seamless exercise.
Homeland Security Exercise and Evaluation Program
3-17
Discussion-Based Exercise Logistics Planning
Logistics for discussion-based exercises include facility and room, audio/visual requirements,
supplies, food, and refreshments, registration and badging, table and breakout identification,
parking, and media, public affairs, and VIPs.
Facility and Room
Meetings, briefings, and exercises should be conducted in facilities appropriate for the exercise
scope and attendance. Reserve facilities solely for exercise purposes, free from distractions, and
accessible to all participants.
Exercise planners should account for the following considerations:
• Ensure enough tables and chairs for every participant;
• Arrange tables to best suit the meeting or exercise (for example, U-shaped layout for
exercises requiring facilitation and participant interaction);
• A facility with room acoustics that encourage discussion;
• A facility with the accessibility of parking and restrooms for all participants, to include any
special needs requirements;
• A venue consistent with the number of attendees expected; the fire marshal office or similar
entity determines the maximum occupancy capacity, and this should not be exceeded; and
• Appropriate power, internet bandwidth, and cell phone reception and service to support
exercise play.
Audio/Visual Requirements
Identify audio/visual requirements early, including individuals assigned to ensure equipment is
properly functioning. Audio/visual equipment may include a projector, screens, microphones, and
speakers.
Supplies, Food, and Refreshments
Obtain supplies such as writing utensils, notepads, easels, copies of plans and procedures, name
badges, and any other necessary equipment before the exercise and provide to the participants. The
exercise planning team should consider, in accordance with applicable funding guidance or venue
policies, whether to provide food and refreshments for participants.
Registration and Badging
Participants should register upon arrival at the exercise venue. Participants should, at a minimum,
provide their name, jurisdiction/organization, telephone number, and e-mail address. The exercise
planning team retains copies of the sign-in sheets for follow-up correspondence. If appropriate for
security purposes, participants should wear a form of identification such as uniforms or badges
with name and jurisdiction/organization.
Homeland Security Exercise and Evaluation Program
3-18
Table and Breakout Identification
Table tents are used to identify participants by either organizational or functional areas. Before
exercise conduct, place table tents on each table to ensure that participants are sitting in the correct
locations. If additional breakout rooms are needed, place signage and table tents in those areas to
ensure that participants are identified.
Parking
Clearly label established parking areas for participants arriving in personally owned vehicles. If
needed, law enforcement personnel may direct vehicles to proper parking areas.
Media, Public Affairs, and VIPs
Media, public affairs, and VIP personnel should always have an escort. Identify a designated
location for the media prior to the start of the exercise.
Discussion-Based Exercise Facilitation Planning
Exercise facilitation is a method that uses a facilitator/presenter with an awareness of appropriate
plans, policies, and procedures. This facilitation typically occurs in a plenary session or in breakout
groups.
Staffing
The planning team identifies the number of facilitators/presenters needed for exercise conduct.
Facilitators/Presenters deliver scenario information and maintain a focused discussion. If
participants divide into groups for part of conduct, assign a table facilitator/presenter to each group.
Facilitation Structure
The facilitation structure is the framework that allows facilitators/presenters to coordinate amongst
one another. This structure becomes part of a larger organizational structure during conduct. The
Lead Facilitator/Presenter ensures that the discussions stay on track and objectives are being met.
If breakouts are needed to further enhance the exercise, additional facilitators/presenters may be
needed for each breakout session.
Configuration for the breakouts can be either by functional areas or organizational areas.
Functional areas are personnel that have the same subject matter expertise in a specific field (fire
and rescue, law enforcement, etc.). Organizational areas are personnel that have a particular
purpose (public safety, emergency management, etc.). Figure 3.5 shows an example of an exercise
facilitation structure.
Homeland Security Exercise and Evaluation Program
3-19
Figure 3.5: Example of a Discussion-Based Exercise Facilitation Structure
Table 3.11 describes the positions used in the exercise facilitation organization.
Table 3.11: Positions of an Exercise Facilitation Structure
Position Description
Lead Facilitator/
Presenter
An individual responsible for keeping a discussion focused on exercise
objectives and exploring all issues within the time allotted during discussion-
based exercises, or an individual designated to deliver information in a structured
setting.
Facilitator/
Presenter
Additional individual responsible for keeping a discussion focused on objectives
for each of the breakout groups as needed.
Prior to the exercise, the facilitator/presenter conducts a walkthrough of the exercise design, event
timeline, delivery methods, and facilitation techniques with all exercise facilitators/presenters,
evaluators/note-takers, and the Exercise Director.
Operations-Based Exercise Logistics Planning
Logistics for operations-based exercises include facilities and exercise areas; audio/visual
requirements; supplies, food, and refreshments; registration and badging; parking and
transportation; actors; and media.
Facilities and Exercise Areas
Operations-based exercises often have several areas for exercise conduct. Ensure any special needs
requirements are considered. Table 3.12 shows the facilities and exercise areas, descriptions, and
types.
Homeland Security Exercise and Evaluation Program
3-20
Table 3.12: Facilities and Exercise Areas, Descriptions, and Types
Area Description Type
Exercise Play
Area
A location where player activities and tasks are demonstrated during
an exercise.
Player Venue
Response
Route
A location for responding emergency units from the assembly area
to the staging area or other exercise sites during a response-focused
exercise.
Player Venue
Exercise
Assembly Area
A
location for participants and resources to gather and dispatch,
managed by controller(s).
Controller
Venue
Master Control
Cell
A location where overall coordination is managed between venue
control cells, simulation cells, and other control areas.
Controller
Venue
Venue Control
Cell
A location where controllers manage individual injects designed for
their relevant players.
Controller
Venue
Simulation Cell
A location from which controllers deliver messages representing
actions, activities, and conversations of an individual, agency, or
organization that is not participating in the exercise.
Controller
Venue
Emergency
Medical
Services Area
A location for real-world response support, such as treatment sector
areas, ambulance staging, and transportation coordination points.
Controller
Venue
Observer and
Media Area
A location where observers and real-world media representatives
can view the exercise but not interfere with exercise play.
Controller
Venue
Audio/Visual Requirements
Identify audio/visual requirements early, including individuals assigned to ensure equipment is
properly functioning.
Supplies, Food, and Refreshments
Exercise planners should not assume participants will bring necessary supplies with them. Obtain
any supplies (for example, vests, clipboards, signage, writing utensils, notepads, easels, copies of
plans and procedures, name badges, and any other necessary equipment) before the exercise
conduct and ensure they are provided to participants. The exercise planning team should consider,
in accordance with applicable funding guidance or venue policies, whether to provide food and
refreshments for participants. Hydration and relief stations are an important logistical aspect of an
operations-based exercise.
Registration and Badging
Participants should register upon arrival and for security purposes should wear a form of
identification such as uniforms or badges with name and jurisdiction/organization. Participants
should, at a minimum, provide their name, jurisdiction/organization, telephone number, and e-mail
address. The exercise planning team retains copies of the sign-in sheets for follow-up
correspondence.
Homeland Security Exercise and Evaluation Program
3-21
Parking and Transportation
Clearly label established parking areas for participants arriving in personally owned vehicles at all
venues. If needed, law enforcement personnel may direct vehicles to proper parking areas and
transportation to pre-position controllers, players, and actors in advance of the exercise.
Actors
Actors add realism and prompt players to react to a scenario event. Exercise planning team
members can recruit from various organizations.
8
Prior to the exercise, actors should receive the following:
• A liability waiver form for signature;
• Actor instructions, including information on when to arrive, where to report, and other
logistical details; and
• Symptomology cards, when applicable.
Media, Public Affairs, and VIPs
Media, public affairs, and VIP personnel should always have an escort. Identify a designated
location for the media prior to the start of the exercise.
Operations-Based Exercise Control Planning
Exercise control maintains exercise scope, pace, and integrity during conduct under safe and secure
conditions. Elements of exercise control include staffing, control structure, control cell, simulation
cell, MSEL, controller training, communications, and safety and security.
Staffing
The exercise planning team identifies the number of controllers needed during the exercise to
deliver and track information. As a guiding principle, place at least one controller at every venue
whenever possible to help ensure a safe exercise with proper security controls. Resource
constraints may make placing a controller at every site challenging; therefore, multitasking
personnel as both a controller and an evaluator is an option.
Control Structure
The control structure is the framework that allows controllers and simulators to communicate and
coordinate at multiple locations. The control structure ensures exercise play progresses to meet
exercise objectives and deliver and track exercise information. This structure becomes part of a
larger organizational structure during conduct. Figure 3.6 shows an example of a control structure
with multiple control cells.
8
Examples of organizations are local colleges and universities, medical and nursing schools, drama clubs, theaters, civic groups, emergency
response academies, Community Emergency Response Teams, and federal and state military units. Including volunteer actors from within the
access and functional needs population provides an opportunity to exercise in a variety of operational environments.
Homeland Security Exercise and Evaluation Program
3-22
Figure 3.6: Example of an Operations-Based Exercise Control Structure
Table 3.13 describes the positions used in an operations-based exercise control structure.
Table 3.13: Positions of an Operations-Based Exercise Control Structure
Position Description
Lead Controller
An individual that monitors exercise progression, communicates exercise
activities throughout all venues, and manages the control staff.
MSEL Manager
An individual that manages the timely and accurate delivery of injects and player
expected actions and decides on the need to release contingency injects.
Simulation Cell
Controller
An individual that delivers scenario messages representing actions, activities,
and conversations
of an individual, agency, or organization that is not
participating in the exercise.
Ground Truth
Advisor
An individual responsible for ensuring that the scenario details remain consistent
during exercise conduct.
Venue Controller
An individual that is responsible for setting up and operating a specific exercise
location. Venue Controllers manage exercise play and may prompt or initiate
certain players to ensure continuity and flow.
Observer/Media
Area Controller
An individual responsible to ensure that observers and the media stay in their
designated areas and do not interfere with the exercise.
Homeland Security Exercise and Evaluation Program
3-23
Position Description
Exercise Assembly
Area Controller
An individual responsible for the logistical organization of the exercise assembly
area, including placement locations for units entering the exercise assembly area,
release of dispatched units into the field, and coordination of routes and overall
safety within the assembly area.
Master Control Cell (MCC)
An MCC is a location where overall coordination is managed between venue control cells (VCCs),
simulations cells, and other control areas. An MCC compiles all the information into a common
operating picture for the exercise and ideally will contain a Point of Contact (POC) or a liaison
representing each participating jurisdiction/organization. If an exercise contains multiple
geographic locations, consider establishing several VCCs to communicate and coordinate through.
When an exercise requires the establishment of multiple VCCs, defining the roles and relationships
and the decision-making hierarchy is important. In smaller exercises, there may be only the MCC
for which all information is shared. In exercises involving both classified and unclassified
information, separate control cells with appropriate security firewalls should be set up to keep
classified and unclassified information separate.
Simulation Cell (SimCell)
The SimCell is a location from which simulation controllers deliver scenario messages
representing actions, activities, and conversations of an individual, agency, or jurisdiction/
organization that is not participating in the exercise. Depending on the type of exercise, the
SimCell may require a telephone, computer, e-mail account, radio, or other means of
communication. Moreover, if an exercise uses a SimCell to drive exercise play, a determination is
made on how to staff and integrate the SimCell the broader control structure.
Master Scenario Events List (MSEL)
A MSEL is a document or system that is a chronological timeline of expected actions and scripted
events to be injected into exercise play by controllers to generate or prompt player activity. It
ensures necessary events happen so that all objectives are met. Larger, more complex exercises
may also use a procedural flow, which differs from the MSEL in that it contains only expected
player actions or events. The MSEL links simulation to action, enhances the exercise experience
for players, and reflects an incident or activity meant to prompt players to action.
Additional descriptive event types can be added to the MSEL to support exercise play when
needed. These events are the foundation of the exercise and are the individual building blocks of
the MSEL. There are three types of events that make up the MSEL and support exercise play.
Table 3.14 shows the event types that can be used in a MSEL.
Table 3.14: MSEL Event Types
Event Types Description
Inject
A MSEL event introduced to a player by the control staff, representing non-
playing entities, to build the exercise environment based on the exercise
scenario and to drive operations-based exercise play.
Homeland Security Exercise and Evaluation Program
3-24
Event Types Description
Contingency Inject
A MSEL event introduced to a player by the control staff when a key player
expected action did not occur as planned,
to provide an additional
opportunity to meet exercise objectives.
Expected Action
A MSEL event that represents an anticipated action to be taken by a player
during the exercise.
Each MSEL entry should include the following:
• Event number;
• Designated scenario time;
• Event type;
• Inject mode;
• From (Non-playing entity delivered by the Control Staff);
• To (Intended player);
• Message;
• Expected participant response;
• Exercise objective; and
• Notes section (for controllers and evaluators to track actual events against those listed in
the MSEL, with special instructions for individual controllers and evaluators).
Scenario timelines listed in the MSEL should be as realistic as possible and based on input from
SMEs. If the activity occurs sooner than the MSEL writers anticipated, then controllers and
evaluators should note the time the activity occurred, but play should not be interrupted.
Typically, there is one MSEL per exercise, and it may be in a short format, long format, or both.
A short format lists the events in a single row in a spreadsheet and is a quick reference guide during
exercise play. A long format includes more detailed descriptions of expected actions and contains
scripted language for actors and simulators. The MSEL may be projected onto a large screen in a
control cell or simulation cell and shared across multiple venues during the exercise.
Controller Training
Controller training includes a summary of the exercise design, including the exercise objectives, a
MSEL review, delivery methods, venues, timeline, communications plan, and evaluation criteria.
Communications
The C/E Handbook or Control Staff Instructions (COSIN) provides a communication plan for
controllers and instructions on whom they will communicate with, what they need to communicate,
and how they will communicate. Communication can occur though many forms such as phone,
radio, e-mail, face-to-face, network system, or a combination. Depending on the method used, the
essential equipment should be provided to the controllers and the control cells.
Homeland Security Exercise and Evaluation Program
3-25
Safety and Security
Controllers also play an important role in ensuring that the exercise is a safe and secure
environment. In exercises involving potentially dangerous field play or the use of classified
materials, the control team designates a Safety and/or Security Controller as appropriate to focus
on those areas of control.
The planning team should consider the following to ensure a safe and secure environment:
• Appointing a Safety and/or Security Controller;
• Dedicating real-world medical (not involved in the exercise) and emergency procedures to
contact and deploy Emergency Medical Services;
• Outlining safety requirements and policies, to include other safety issues outside the scope
of the exercise (for example, weather, heat stress, hypothermia, etc.);
• Using local law enforcement to provide site security where appropriate; and
• Identifying and adhering to appropriate security standards to ensure that sensitive or
classified information is not compromised.
Planning for Exercise Evaluation
Planning and organization prior to an exercise are imperative to effective and successful exercise
evaluation. The exercise planning team identifies evaluation elements early in the exercise design
process following the development of the exercise objectives. An evaluation team determines the
appropriate structure, organizes the team, and develops a comprehensive plan to address the
evaluation of the exercise. Evaluation is further explained in Chapter 5: Exercise Evaluation.
Preparing for Exercise Conduct
The exercise planning team finalizes all aspects of the exercise to prepare for conduct. Preparations
include finalizing materials, rehearsing presentations and briefings, and training participants for
the exercise. Prior to the exercise, the appropriate personnel receive documentation and additional
instructions needed for conduct.
To prevent jeopardizing mission performance in response to real-world incidents, the exercise
planning team should develop a contingency process to halt, postpone, or cancel an exercise as
necessary. If the conduct of the exercise is put at risk, the Exercise Director and exercise planning
team should convene and determine the appropriate course of action. Following a decision, the
course of action should be communicated to all exercise planners, participants, and other key
stakeholders through formal communications mechanisms.
Homeland Security Exercise and Evaluation Program
4-1
4. Exercise Conduct
Overview
Exercise conduct involves activities such as preparing for exercise play, managing exercise play,
and conducting immediate exercise wrap-up activities. Exercise practitioners are encouraged to
apply and adapt the HSEEP doctrine during exercise conduct to meet specific needs. Throughout
the exercise conduct process, the engagement of senior leaders by practitioners confirms the
exercise is consistent with the original guidance and intent.
Discussion-Based Exercises
Exercise/Play Preparation
Members of the exercise planning team assigned to exercise setup should visit the exercise site to
arrange the room, test the audio/visual equipment, and discuss administrative and logistical issues.
Prior to exercise conduct, the exercise planning team delivers the necessary exercise materials and
equipment, which may include the following:
• Any Situational Manuals (SitMan) or other written materials for exercise participants;
• The presentations;
• The appropriate audio/visual equipment, including televisions, projectors, projection
screens, microphones, and speakers;
• The table tents for each table;
• The name tents for each participant;
• The badges identifying the role of each exercise participant;
• The sign-in sheets; and
• The Participant Feedback Forms.
A rehearsal of the exercise structure and presentations helps to ensure an understanding of
facilitator and evaluator roles and responsibilities, event timing, audio/visual equipment, and
location-specific details, including access and security. On the day of the exercise, planning team
members should arrive several hours before Start of Exercise (StartEx) to set up activities and
arrange for registration.
Exercise Play/Conduct
For a discussion-based exercise, conduct entails a facilitated discussion based on a scenario,
objectives, or goals. The Exercise Director provides opening statements to the participants and
welcomes them to the exercise. The Facilitator/Presenter then begins the discussion by presenting
the scenario and keeping the discussions on track. The Lead Evaluator is responsible to ensure that
all notes and observations are taken and provided back to the Exercise Director. If any additional
resource requirements are needed, the Resource Lead is responsible.
Homeland Security Exercise and Evaluation Program
4-2
If breakouts are needed to further enhance the exercise, additional facilitators/presenters,
evaluators/note takers may be added and will be responsible to their individual section leads.
Additional facilities, administration, or logistics personnel, if needed, will report to the Resource
Lead. Figure 4.1 shows the relationship of a discussion-based organizational structure.
Figure 4.1: Example of a Discussion-Based Control Structure
Table 4.1 describes the positions of a discussion-based exercise control organization.
Table 4.1: Positions of a Discussion-Based Exercise Control Structure
Position Description
Exercise Director
An individual that provides the strategic oversight and direction on the HSEEP
Cycle for an individual exercise.
Safety Controller
An individual responsible to ensure that the exercise is conducted in a safe
environment.
Lead Facilitator/
Presenter
An individual that oversees all facets of the facilitation process or the
presentation(s),
to include recruiting and assigning additional facilitators or
presenters.
Facilitator/
Presenter
An individual responsible for keeping a discussion focus
ed on exercise
objectives and exploring all issues within the time allotted during discussion-
based exercises, or an individual designated to deliver information in a structured
setting.
Lead Evaluator
An individual that oversees all facets of the evaluation process,
to include
recruiting, assigning, and training evaluators.
Homeland Security Exercise and Evaluation Program
4-3
Position Description
Evaluator
An individual chosen, based on their expertise in the functional areas, to observe
and collect exercise data and analyze results.
Note Taker (as
needed)
An individual that records what is said during breakout groups, interviews, or
hotwash discussions, allowing the facilitator, presenter, or interviewer to focus
on soliciting information and asking follow-up questions, and s
upports data
collection and data management throughout the evaluation process.
Resource Lead
An individual responsible for obtaining proper venues, equipment, and supplies
for exercise conduct, as well as providing support for media and VIP observers.
Facilities Lead
An individual responsible for managing exercise venue considerations, such as
set up, tear down, and table and breakout room assignments, as required.
Administration
Lead
An individual responsible for managing the registration process, printed
documents, sign-in sheets, and badges.
Logistics Lead
An individual responsible to ensure proper room function and set up for
audio/visual requirements, as well as obtaining necessary equipment, food, and
drinks; works with the facilities lead.
Conduct activities for a discussion-based exercise include the following considerations:
• Presentation is a crucial vehicle for conveying information to the players;
• Facilitators/presenters help evaluators/note takers collect useful data by keeping
discussions focused on exercise objectives, capabilities, capability targets, and critical
tasks;
• Facilitators/Presenters make sure all issues are explored within the time allotted; and
• Ensuring a safe and secure exercise environment.
All facilitators/presenters should take, and compile notes relevant to the groups’ discussions. If
multiple facilitators/presenters were used, the Lead Facilitator/Presenter will collect all notes from
each of the additional facilitators/presenters. This information will be used to generate the After-
Action Report/Improvement Plan (AAR/IP).
Operations-Based Exercises
Play Preparation
Depending on the scope of the exercise, the appropriate exercise planning team members should
begin event setup as many days before the exercise as necessary. A communication check is
important prior to the Start of Exercise (StartEx) to ensure all forms of communications are
working correctly.
Prior to exercise conduct, the exercise planning team delivers the necessary exercise materials and
equipment, which may include:
• Arranging the briefing rooms;
• Testing the audio/visual equipment;
Homeland Security Exercise and Evaluation Program
4-4
• Placing props and effects;
• Marking the appropriate exercise areas and the perimeters; and
• Checking for potential safety issues.
A rehearsal of the exercise structure is vital to ensure an understanding of controller and evaluator
responsibilities, transportation, event timing, audio/visual and communications equipment, and
location-specific details, including access and security. On the day of the exercise, all exercise
planning team members should arrive prior to Start of Exercise (StartEx) to complete any
remaining logistical or administrative items and set up registration.
Exercise Play/Conduct
During conduct of operations-based exercises, the Exercise Planning Team Leader often serves as
the Lead Controller. Controllers and evaluators report activities to the Lead Controller, who is
responsible for monitoring exercise progression, communicating exercise activities throughout all
venues, and managing the control staff. Figure 4.2 shows an example of an operations-based
exercise control organizational structure.
Figure 4.2: Example of an Operations-Based Exercise Control Structure
Table 4.2 4.2 describes the positions used in an operations-based exercise organization.
Homeland Security Exercise and Evaluation Program
4-5
Table 4.2: Positions of an Operations-Based Exercise Control Structure
Position Description
Exercise Director
An individual responsible for the strategic oversight and direction on the HSEEP
Cycle phases for an individual exercise.
Safety/Security
Controller
An individual responsible to ensure that the exercise is conducted in a safe and
secure environment.
Lead Controller
An individual that monitors exercise progression, communicates exercise
activities throughout all venues, and manages the control staff.
MSEL Manager
An individual that manages the timely and accurate delivery of injects and player
expected actions and decides on the need to release contingency injects.
Simulation Cell
Controller
An individual that delivers scenario messages representing actions, activities,
and conversations
of an individual, agency, or organization that is not
participating in the exercise.
Ground Truth
Advisor
An individual responsible for ensuring that the scenario details remain consistent
during exercise conduct.
Venue Controller
An individual that is responsible for setting up and operating a specific exercise
location. Venue Controllers manage exercise play and may prompt or initiate
certain players to ensure continuity and flow.
Lead Evaluator
An individual that oversees all facets of the evaluation process,
to include
recruiting, assigning, and training evaluators.
Venue Evaluator
An individual responsible for observing exercise activity/play in an assigned
exercise venue, area, or for a specific activity and collecting observations and
data.
Resource Lead
An individual responsible for obtaining proper venues, equipment, and supplies
for exercise conduct, as well as providing support for media and VIP observers.
Facilities Lead
An individual responsible to work closely with the logistics section to manage
exercise venue considerations, such as set up, tear down, scheduling, and traffic.
Administration
Lead
An individual responsible for managing the registration process, printed
documents, sign-in sheets, and badges.
Logistics Lead
An individual responsible to ensure proper room function and set up for
audio/visual requirements, as well as obtaining necessary equipment, food, and
drinks; works with the facilities lead.
Exercise Assembly
Area Controller
An individual responsible for the logistical organization of the exercise assembly
area, including placement locations for units entering the exercise assembly area,
release of dispatched units into the field, and coordination of routes and overall
safety within the assembly area.
Observer/Media
Area Controller
An individual responsible to ensure that observers and the media stay in their
designated areas and do not interfere with the exercise.
Homeland Security Exercise and Evaluation Program
4-6
Conduct activities for an operations-based exercise include:
• Ensuring a safe and secure exercise environment;
• Distributing exercise documents that outline roles, responsibilities, and a communication
plan for the control staff;
• Communicating between the SimCell, venue controllers, and MSEL manager to ensure
pace of the exercise play is appropriate;
• Ensuring the Exercise Assembly Area Controller remains in close communication with
other controllers throughout the exercise; and
• The Exercise Assembly Area Controller ensures that units are deployed based on the
timeline that is created by the planning team.
The Exercise Director and the planning team should have a contingency plan to pause, postpone,
or cancel in the event of a real-world emergency impeding the exercise.
Briefings
Before an exercise, briefings educate participants about their roles and responsibilities. By
scheduling separate briefings for senior leaders, controllers and evaluators, actors, players, and
observers, exercise planning team members can avoid giving extraneous material to different
groups. Table 4.3 describes the exercise brief types.
Table 4.3: Types of Exercise Briefs
Brief Description
Senior Leader
Brief
A briefing occurs during the design and development and before the conduct of an
exercise. The exercise planning team leader periodically consults with the senior
leaders within the exercise planning team to ensure the exercise aligns with leader
intent.
Controller
and Evaluator
Brief
A briefing is generally conducted before operations-based exercises. It begins with an
exercise overview and then reviews the exercise location and area, schedule of events,
scenario, control concept, controller and evaluator responsibilities, instructions on
completing EEGs
, and any miscellaneous information. Additional training for
evaluators may be conducted.
Actor Brief
A briefing before the start of the exercise, prior to the actors taking their positions. The
Actor Controller leads the actor briefing, a
meeting generally conducted before the
exercise, providing actors with an exercise overview, safety, real emergency
procedures, acting instructions, schedule, identification badges, and symptomatology
cards.
Player Brief
A briefing before the start of the exercise, for all players to address individual roles
and responsibilities, exercise parameters, safety, security badges, and any remaining
logistical exercise concerns or questions. Participant handouts and ExPlans or
SitMans, depending on the type of
exercise being conducted, are often distributed
during this briefing. Following the exercise, controllers ensure that appropriate players
attend the post-exercise hotwash in their respective functional areas.
Homeland Security Exercise and Evaluation Program
4-7
Brief Description
Observer
Brief
A briefing before the start of the exercise that occurs the day of an exercise and informs
observers and VIPs
about the exercise background, scenario, schedule of events,
observer limitations, and any other miscellaneous information. Often, observers are
unfamiliar with public safety procedures and have questions about the activities;
therefore, designating someone such as a
public information officer to answer
questions prevents observers from interrupting exercise participants.
Wrap-up Activities
Performing thorough exercise wrap-up ensures the collection of all relevant data to support
effective evaluation and improvement planning. Immediately after the exercise, the exercise
planning team members should conduct a short debrief to discuss the exercise and any issues or
concerns and propose improvements. Planners should collect exercise attendance lists, provide
copies to the exercise planning team leader, collect Participant Feedback Forms, and develop
debrief notes.
Player Hotwash
The player hotwash is a meeting that provides an opportunity to discuss exercise strengths and
areas for improvement immediately following the conduct of an exercise. An experienced
facilitator leads the hotwash and ensures the discussion remains constructive. The information
gathered during a hotwash contributes to the AAR/IP and any exercise suggestions can improve
future exercises. A hotwash also provides an opportunity for players to provide clarification and
time to fill-out Participant Feedback Forms, which helps to generate the AAR/IP.
Debrief
The Controller/Evaluator Debrief is a meeting for controllers, facilitators, and evaluators to
assemble after the exercise to discuss observations and provide information to create a shared
understanding of the exercise. The Controller/Evaluator Debrief allows each controller, facilitator,
and evaluator an opportunity to provide an overview of the functional area observed and discuss
the strengths and areas for improvement. The results captured during the debrief help develop the
AAR/IP. During the debrief, controllers, facilitators, and evaluators complete and submit a
Participant Feedback Form.
Homeland Security Exercise and Evaluation Program
5-1
5. E xercise Evaluation
Overview
Through exercise evaluation, jurisdictions/organizations assess the
capabilities needed to accomplish a mission, function, or objective.
Evaluation ties objectives to priorities, assesses the performance
of capability targets and critical tasks, and documents strengths
and areas for improvement. Evaluation connects the exercise to
Improvement Planning and feeds into the overall Integrated
Preparedness Cycle. Figure 5.1 shows exiting the HSEEP Cycle
and entering the Integrated Preparedness Cycle. This transition is
further explained in Chapter 6: Improvement Planning.
Effective evaluation should be integrated through the HSEEP
Cycle and involves:
• Planning for exercise evaluation;
• Observing the exercise and collecting exercise data during
exercise conduct;
• Analyzing collected data to identify strengths and areas for
improvement; and
• Reporting exercise outcomes in an After-Action Report (AAR).
Using a common approach to evaluation supports consistent and meaningful reporting of exercise
results.
Evaluation Planning
Evaluation planning begins during the initial planning phases of the exercise, as described in
Chapter 3: Exercise Design and Development. Identifying clear evaluation requirements early
in the planning process will ensure that the design, development, and conduct of the exercise best
support an effective evaluation. Exercise planners should collaborate to ensure a consistent
approach for evaluating an exercise. Also, engage senior leaders early in the evaluation planning
to identify any focus areas, potential concerns, and specific evaluation requirements. The
Evaluation Plan (EvalPlan) and/or Controller/Evaluator (C/E) Handbook are guidance documents
for evaluators that provide guidance, instructions, and structure on evaluating and observing during
an exercise.
Planning an exercise evaluation includes:
• Selecting the evaluation team requirements;
• Developing and finalizing the evaluation documentation and methodology; and
• Developing the Exercise Evaluation Guides (EEGs).
Figure 5.1: Exiting the HSEEP
Cycle and Entering the
Integrated Preparedness Cycle
Homeland Security Exercise and Evaluation Program
5-2
Evaluation Team
Once the exercise planning team defines evaluation requirements, the Lead Evaluator oversees the
recruiting, assigning, and training of evaluators.
The evaluation requirements should specify the following:
• The number of evaluators needed (for example, multiple locations or extended hours);
• The types of subject-matter expertise the evaluators should possess;
• The types of security clearance level (if required);
• The evaluator assignments; and
• The type of training or instruction needed prior to the exercise.
Team Composition
The Exercise Planning Team Lead should appoint a Lead Evaluator to oversee all facets of the
evaluation process. The Lead Evaluator participates fully as a member of the exercise planning
team and is familiar with the exercise objectives, mission areas, and capabilities associated with
the plans, policies, and procedures, and decision-making processes. Whenever possible,
evaluators/note takers should have experience and subject-matter expertise in their assigned
functional area.
The Lead Evaluator determines the structure of the exercise evaluation team based on exercise
scope, exercise objectives, associated capabilities, capability targets, and critical tasks. Exercises
that involve multiple jurisdictions/organizations and multiple venues should consider assigning
evaluators/note takers to each venue location. The exercise evaluation team becomes part of a
larger organizational structure during conduct. The evaluators/note takers at each of the venues
will support the Lead Evaluator at their assigned locations. Figure 5.2 shows an example of an
evaluation team organization.
Figure 5.2: Example of an Exercise Evaluation Team
Homeland Security Exercise and Evaluation Program
5-3
Table 5.1 Describes an Exercise Evaluation Team Organization.
Table 5.1: Positions of an Exercise Evaluation Team
Position Description
Lead Evaluator
An individual that oversees all facets of the evaluation process, to include
recruiting, assigning, and training of evaluator(s).
Evaluator
An individual chosen, based on their expertise in the functional areas, to observe
and collect exercise data and analyze results.
Note Taker (as
needed)
An individual that records what is said during breakout groups, interviews, or
hotwash discussions, allowing the facilitator, presenter, or interviewer to focus
on soliciting information and asking follow-up questions, and supports data
collection and data management throughout the evaluation process.
Training
Evaluator training should include guidance on using evaluation tools, observing exercise
discussions or operations, and data analysis.
Evaluator training should also include the following:
• General information about the exercise, including scope, objectives and aligned
capabilities, scenario, and schedule;
• Relevant evaluator documentation (for example, Situational Manual (SitMan), C/E
Handbook, evaluation tools); and
• Appropriate plans, policies, procedures, agreements, or other information.
Before exercise play begins, the Lead Evaluator should meet with all evaluators to verify roles,
responsibilities, and assignments and to provide any significant updates. The briefing provides an
opportunity for evaluators/note takers to ask questions and to ensure understanding of their roles
and responsibilities. The briefing may occur in conjunction with the Controller/Evaluator Brief to
include instructions for control staff on delivering the injects and evaluators/note takers completing
EEGs. Depending on the organization of the exercise, briefings may occur at more than one
exercise venue.
Evaluation Documentation
The exercise planning team and Lead Evaluator should determine the tools, data collection
methods, and documentation needed based on the level of complexity and scope. Table 5.2
provides the document, description, and distribution of evaluation documentation.
Table 5.2: Evaluation Documentation
Document Description Distribution
Facilitator/
Evaluator
Handbook
A document that provides specific exercise information and
guidance for facilitators/presenters and evaluators/note takers
for a discussion-based exercise.
Facilitators/Presenters
and Evaluators/Note
Takers
Homeland Security Exercise and Evaluation Program
5-4
Document Description Distribution
Controller/
Evaluator
Handbook
A document that provides specific exercise information and
guidance for controllers and evaluators/note takers, may be a
standalone document or supplement to the Exercise Plan, and
adjusted accordingly.
Controllers and
Evaluators/Note
Takers
Evaluation
Plan
A document for evaluators/note takers that provides guidance,
instructions, and structure on evaluating and observing an
exercise. It also provides essential materials required to perform
their specific functions.
Controllers and
Evaluators/Note
Takers
Exercise
Evaluation
Guide
(EEG)
A document that captures information specifically related to the
evaluation requirements developed by the exercise planning
team. The EEG provides evaluators/note takers wit
h a
standardized tool to guide data collection and capture
performance results.
Since each jurisdiction/organization has
unique targets and critical tasks, EEGs are developed specific to
their plans, policies, procedures, and protocols.
Evaluators
Participant
Feedback
Form
A document that the exercise team uses to collect observed
strengths, areas for improvement, and input about exercise
conduct and logistics from participants.
All Participants
Exercise Evaluation Guide (EEG) Development
EEGs provide a consistent tool to guide exercise observation and data collection. EEGs align to
exercise objectives and capabilities and list the relevant capability targets and critical tasks. Figure
5.3 shows the relationship of program management to the development of EEGs. Capability targets
are the performance thresholds established by the jurisdiction/organization for each capability.
Capability targets are composed of three parts: a critical task, an impact, and a timeframe.
Identified targets are part of a jurisdiction’s/organization’s Threat and Hazard Identification and
Risk Assessment (THIRA) or other threat and hazard identification or risk assessment processes.
Figure 5.3: Development of an Exercise Evaluation Guide (EEG)
Critical tasks are distinct specific actions needed to achieve the capability target. Critical tasks may
derive from the jurisdictional/organizational operations plans, standard operating procedures, or
discipline-specific standards. These tasks reflect the types of activities emergency managers
routinely plan for and are broadly applicable to a wide range of threats and hazards.
Capability targets include specific and measurable metrics that describe the capabilities.
Additionally, the capability targets establish a benchmark that the whole community can use to
Homeland Security Exercise and Evaluation Program
5-5
track progress over time such as assessing current response to threats and hazards and means of
improvement. Capability targets are composed of three parts and shown below in the example
provided:
• Critical Task, which represents the specific action needed to achieve the capability target;
• Impact, which represents the size of the capability requirement; and
• Timeframe, which represents the time in which the action must be performed.
The purpose of an EEG is to:
• Streamline data collection and provide a consistent process for assessing preparedness;
• Assist jurisdictions/organizations to map exercise results to exercise objectives,
capabilities, capability targets, and critical tasks for additional analysis and assessment;
and
• Support development of the AAR.
Exercises assess a jurisdiction’s/organization’s ability to meet its capability targets. Capability
targets, when incorporated into planning and used as exercise evaluation criteria, can measure how
effectively a plan meets the goal and how well the participants perform the tasks in the plan. The
after-action and continuous improvement process can validate current capabilities already in place
and describe areas for improvement. The after-action and continuous improvement process can
help the whole community identify how to close capability gaps and bring them closer to meeting
the target established.
Observation and Data Collection
Exercise observation and data collection can differ between discussion-based exercises and
operations-based exercises. Discussion-based exercises often focus on issues involving plans,
policies, and procedures. Evaluators/Note Takers collect observations from participants’
discussions. Operations-based exercises focus on issues affecting the operational conduct of
capabilities and critical tasks and implementation of plans, policies, and procedures.
Evaluators/Note Takers collect and record participant actions, which form the analytical basis for
determining the successful demonstration of critical tasks and capability targets.
Example of a Capability Target
Within (#) (time) of an incident, assess, contain, and begin clean up of hazardous material releases from (#)
HAZMAT release sites
.
Critical Tasks
Impact
Timeframe
Homeland Security Exercise and Evaluation Program
5-6
Observation
Exercise evaluators/note takers should observe exercise activity in a non-attribution environment,
in accordance with the EvalPlan. Evaluators/Note Takers observe many of the following:
• Activation or implementation of plans, policies, processes, and procedures, requests for
resources, use of mutual aid agreements, etc.;
• Roles and responsibilities and authorities of the government agencies, jurisdictions, and
private organizations;
• Pertinent decisions made or decision-making processes; and
• Information sharing with other agencies and the public.
Data Collection
Data collection is critical to providing an overall understanding of the exercise. Collecting data
using a variety of methods, tools, and techniques provides the information needed for continuous
improvement activities and resourcing decisions, rather than relying on assumptions. It supports
root cause analysis by providing a fact-based record of what actions were taken, what key decisions
were made, and the outcomes of those actions and decisions.
Evaluators and Note Takers should document relevant actions or discussions and retain their notes
and records to support the development of the AAR. As necessary, the Lead Evaluator may assign
evaluators/note takers to collect additional data, such as plans, policies, procedures, Incident
Action Plans, and other documentation associated with the exercise. Data collection does not only
come from evaluators, it can also come from observers, controllers, safety personnel, etc.
In addition to EEGs, evaluators can use a variety of data collection methods as part of their
evaluation plan, including direct observation, interviews and hotwashes, document reviews,
participant feedback forms, or surveys. Table 5.3 below outlines some considerations for each
collection method.
Table 5.3: Data Collection Methods
Method Considerations
Direct Observation ▪ Offers real-time data collection
▪ Requires large resources to observe multiple venues simultaneously
Documentation
Review
▪ Serves as authoritative record of activities
▪ Can be time-consuming and labor intensive to review and synthesize
Feedback Forms
and Surveys
▪ Easy to administer and provides firsthand information from participants
▪ Open-
ended feedback often varies in specificity and quality and offers
limited opportunity for follow-up
Interviews
(individual or
group)
▪ Enables in-depth discussions of key issues, often involving SMEs
▪ Can be difficult to schedule during or immediately following an exercise or
incident
Hotwash and
Debrief
▪ Enable group discussion with participants
▪ Led by an individual to ensure open, collaborative discussion
Homeland Security Exercise and Evaluation Program
5-7
Data Analysis
During data analysis, the exercise evaluation team first consolidates the data collected during the
exercise and determines whether participants performed critical tasks and met capability targets.
Evaluators consider participant performance against all targets to determine the overall ability to
perform capabilities. Additionally, the exercise evaluation team documents strengths and areas for
improvement over the course of exercise play. This provides the evaluators with not only what
happened, but why events happened.
In both discussion-based and operations-based exercises, evaluators identify issues by comparing
exercise objectives to actual performance. Through this comparison, evaluators identify which
capabilities (and their associated activities, performance measures, and tasks) were successfully
demonstrated in the exercise. They also identify which capabilities need improvement.
Evaluators should consider the following questions to identify issues:
• What happened? What was supposed to happen based on current plans, policies, and
procedures?
• Was there a difference? What was the impact?
• Were the consequences of the action (or inaction/decision) positive, negative, or neutral?
• Do plans, policies, and procedures support activities and associated tasks?
• Are players familiar with these documents?
• What are the strengths and areas of improvement to remedy deficiencies?
After evaluators identify discrepancies between what happened and what was supposed to happen
(the issues), they explore the source of these issues. Evaluators and exercise program managers
can use a variety of analysis techniques to support improvement planning for each individual
exercise as well as the overall exercise program, including:
• Data Synthesis;
• Event Reconstruction;
• Trend Analysis; and
• Root Cause Analysis.
Data Synthesis
Data synthesis is the process of compiling and reviewing data from multiple sources to identify
issues and support analysis. This phase enables analysts to collate and analyze large amounts of
information, identify potential issues requiring resolution, and examine the strength of evidence
(or remaining gaps) for each issue.
Event Reconstruction
Event reconstruction is consolidating information from multiple sources into a single, fact-based
account of what happened, including key decisions, actions, and communications. For each
decision, action, or communication, analysts try to identify the who, what, when, where, and how.
The reconstruction activity is not concerned with understanding the why of a specific action.
Homeland Security Exercise and Evaluation Program
5-8
Trend Analysis
Trend analysis identifies patterns in strengths, areas for improvement, and major changes in
observations over time. For example, a trend analysis on reoccurring trends of various
threat/hazard and capability topics guides planning, training, exercise design and development,
and future disaster-related efforts.
Root Cause Analysis
Root cause analysis focuses on identifying the most basic causal factor for why an expected action
did not occur or was not performed as expected. Determining the underlying reason behind an
identified issue allows the evaluator to direct improvement. Consider asking the following
questions to narrow down the issue of concern:
• Were the objectives for each critical task met? If not, what factors contributed to this result?
• Did discussions or activities suggest the critical tasks performed sufficiently to meet the
capability targets? If not, what were the resulting impacts or consequences?
• What improvements are required? Are other resources needed? If so, how will they be
obtained?
• What strengths are required to carry out those tasks? What decisions would need to be
made, and who would make them?
The importance of determining root cause is to understand that each step is a symptom (or effect)
of the next item and a cause of the previous item. There are several different analysis tools available
for determining root cause and Figure 5. provides one example.
Figure 5.4: Example of a Root Cause Analysis
After identifying issues and analyzing the data, evaluators develop recommendations for
improvement aimed at enhancing preparedness, which are then compiled into the draft AAR/IP.
After- Action Report (AAR)/Improvement Plan (IP)
The AAR/IP is a document that generally includes an exercise overview, analysis of capabilities,
and a list of corrective actions. The length, format, and development timeframe of the AAR/IP
depend on the exercise type and scope. These parameters should be determined by the exercise
planning team, based on the expectations of senior leaders. The AAR/IP should include an
overview of performance related to each exercise objective and associated capabilities.
The ability to communicate exercise evaluation results to stakeholders is crucial to the
improvement planning process. Time spent planning for data collection, collecting data, and
Homeland Security Exercise and Evaluation Program
5-9
conducting analysis supports the development of clear observations and recommendations for
inclusion in your AAR/IP.
The observations developed for the AAR/IP should be categorized as either strengths or areas for
improvement, as explained in Table 5.4: Categories of an AAR/IP Observation and the example
below. Observations do not have to be lengthy to be impactful. A strongly written observation
includes a clear and direct statement of the issue identified, a brief description of the analysis, and
the impact or result of the issue. A brief summary of the analysis completed can be included to
help stakeholders understand how the evaluation team arrived at the conclusion.
Table 5.4: Categories of an AAR/IP Observation
Observation Type Description
Strengths ▪ Actions that went exceptionally well given the circumstances
▪ The impact of positive performance on desired or expected outcomes
▪ Activities that yielded better results than could have been expected
Areas for
Improvement
▪ Outcomes did not meet expectations or intent
▪ The negative impact of actual performance on desired or expected outcomes
▪ The factors that contributed to the inability to meet critical tasks, capability
targets, or desired outcomes
Consider these tips when organizing your observations into the AAR/IP:
• Focus on issues that are critical to the success of a mission or represent a trend;
• Observations help guide corrective action planning by focusing time and resources on
issues that have the greatest impact on preparedness;
• If possible, include data on consequences and likelihood of reoccurrences or what would
happen if no action is taken; and
• A report should include the observations, but the After-Action Meeting (AAM) will give
leadership a chance to further organize the observations and order the development of
action plans.
The evaluation team develops and provides the draft AAR/IP to the Exercise Director, who
distributes the draft to participating jurisdictions/organizations. Senior leaders or their designees
review and confirm observations, strengths, and areas for improvement identified in the draft
AAR/IP.
Homeland Security Exercise and Evaluation Program
5-10
After- Action Meeting (AAM)
The purpose of the AAM is to serve as a forum to review the revised the draft AAR/IP. Participants
should seek to reach final consensus on strengths, areas for improvement, draft corrective actions,
concrete deadlines, and owners/assignees for implementation of corrective actions. Table 5.5
provides the meeting focus, discussion points, tools, outcomes, and follow-up actions for an AAM:
Table 5.5: After-Action Meeting
Element Considerations and Activities
Meeting Focus
▪
The AAM is an interactive session providing attendees the opportunity to
discuss and validate the analytical findings and corrective actions in the draft
AAR/IP
Discussion Points
▪ Review and revise the draft AAR/IP
▪ Discuss exercise results
▪ Identify areas for improvement
▪ Identify corrective actions
Tools
▪ Draft AAR
▪ Draft IP
Outcomes
▪ Final consensus on AAR/IP
▪ Consensus on draft corrective actions
▪ Proposed concrete deadlines for the implementation of corrective actions
▪ Consensus on the assigned corrective actions to preparedness stakeholders
Follow-up
▪ Distribute the finalized AAR/IP
▪ Track and report corrective actions
Participant jurisdictions/organizations are responsible for developing implementation processes
and timelines and keeping their senior leaders informed of the implementation status.
Improvement Plan (IP)
The IP includes all consolidated corrective actions. The IP may be an appendix to the AAR. The
AAR/IP is then final and distributed to exercise planners, participants, and other preparedness
stakeholders, as appropriate. Chapter 6: Improvement Planning describes the process in further
detail.
Homeland Security Exercise and Evaluation Program
6-1
6. Improvement Planning
Overview
Improvement Planning is a process by which the areas for
improvement from the exercise are turned into concrete,
measurable corrective actions that strengthen capabilities. In this
way, Improvement Planning activities can help shape a
jurisdiction’s/organization’s preparedness priorities and support
continuous improvement. The Improvement Planning activities
now transition from the HSEEP Cycle back to the Integrated
Preparedness Cycle.
Actions identified during Improvement Planning help to
strengthen elements of a jurisdiction’s/organization’s capability
to plan, organize/equip, train, and exercise. The transition is
shown in Figure 6.1.
Effective Improvement Planning serves as an important tool
throughout the Integrated Preparedness Cycle by:
• Prioritizing corrective actions identified from individual
exercises;
• Providing valuable input into strategy development and
program priorities;
• Initiating a review or new development of plans, polices, and procedures; and
• Identifying and obtaining needed training, equipment, and other resources.
SMART Corrective Actions
Specific, measurable, achievable, relevant, and time-bound (SMART) corrective actions are
actionable steps intended to resolve capability gaps and shortcomings identified in exercises.
Figure 6.2 depicts guidelines for developing SMART corrective actions.
Figure 6.1: Exiting the HSEEP
Cycle and Entering the
Integrated Preparedness Cycle
Homeland Security Exercise and Evaluation Program
6-2
Figure 6.2: SMART Guidelines for Corrective Actions
Example of a SMART Corrective Action
Central City will hold a seminar to address the confusion that exists within the
multiple organizations that respond to HAZMAT incidents within the city. The goals
of the seminar will focus on eliminating this confusion between the responding
organizations and will clarify the roles, responsibilities, and authorities of each.
Final planning and details for the seminar should be completed by September 1,
2020 (90 days), with identification of all necessary players to be invited by
September 15, 2020 (105 days).
During development of corrective actions, the jurisdiction’s/organization’s Point of Contact (POC)
assumes the responsibility for the identified issues. Lastly, the jurisdiction’s/organization’s POC
determines an initial list of appropriate corrective actions to resolve the identified issues.
The jurisdiction’s/organization’s POC may use the following questions to guide the discussion
when developing corrective actions:
• What changes to plans and procedures will improve performance?
• What changes to organizational structures will improve performance?
• What changes to management processes will improve performance?
• What changes to equipment or resources will improve performance?
• Was training provided and did the training improve performance?
Corrective Action Tracking and Implementation
Corrective actions captured in the AAR/IP should be tracked and continually reported on until
completion. These efforts are part of a more comprehensive continuous improvement process that
applies before, during, and after an exercise. Stakeholders should also ensure a system is in place
to validate previous corrective actions that were successfully implemented. Over time, corrective
actions effectively integrated back into the Integrated Preparedness Cycle will yield continuous
improvements.
Homeland Security Exercise and Evaluation Program
6-3
Improvement Planning to Support Continuous Improvement
Continuous improvement is a method in which capabilities are periodically examined to make sure
they are sufficient, accurate, and effective to handle the threats, hazards, and risks a
jurisdiction/organization may face. Identifying strengths, areas for improvement, and corrective
actions that result from exercises help jurisdictions/organizations build, sustain, and deliver
capabilities as part of a continuous improvement process.
Continuous improvement includes:
• A Consistent Approach: Jurisdictions/Organizations should employ a consistent
approach for continuous improvement-related activities across applicable mission areas—
Prevention, Protection, Mitigation, Response, and Recovery. A consistent approach
enables a shared understanding of terminology, functions, processes, and tools. The
approach also fosters continuous improvement-related interoperability and collaboration
across a jurisdiction’s/organization’s components.
• Supporting Preparedness: Jurisdictions/Organizations support the development and
sustainment of capabilities across the whole community, which contributes to supporting
preparedness. Continuous improvement activities also ensure that jurisdictions/
organizations can support assessments of preparedness in a timely, actionable, and
meaningful way.
• Effective Issue Resolution and Information Sharing: Through Improvement Planning,
jurisdictions/organizations complete continuous improvement action items at the lowest
level possible, while facilitating the sharing of strengths and areas for improvement.
Application of the above principles and the conduct of Improvement Planning ultimately support
the Integrated Preparedness Cycle. By continually examining the implementation of corrective
actions, jurisdictions/organizations can identify capability gaps and determine what corrective
actions require validation through exercises. Improvement Planning activities can help shape a
jurisdiction’s/organization’s preparedness priorities and support continuous improvement in the
building and sustaining of capabilities.
Homeland Security Exercise and Evaluation Program
Acronym-1
Acronyms
Abbreviation Description
AAM After-Action Meeting
AAR After-Action Report
AAR/IP After-Action Report/Improvement Plan
C&O Concept and Objectives
C/E Controller/Evaluator
CDP Center for Domestic Preparedness
COSIN Control Staff Instructions
EEG Exercise Evaluation Guide
EMI Emergency Management Institute
FEMA Federal Emergency Management Agency
FOUO For Official Use Only
FPM Final Planning Meeting
GIS Geographic Information System
HAZMAT Hazardous Materials
HSEEP Homeland Security Exercise and Evaluation Program
IP Improvement Plan
IPM Initial Planning Meeting
IPP Integrated Preparedness Plan
IPPW Integrated Preparedness Planning Workshop
MPM Midterm Planning Meeting
MSEL Master Scenario Events List
NPS National Preparedness System
NTED National Training and Education Division
POC Point of Contact
SMART Specific, Measurable, Achievable, Relevant, Time-bound
SME Subject-Matter Expert
THIRA Threat and Hazard Identification and Risk Assessment
VIP Very Important Person
Homeland Security Exercise and Evaluation Program
Glossary-1
Glossary of Terms
The table below provides a description for commonly used terms in HSEEP.
Term Description
A
Actor
An individual who simulates a specific role, such as a disaster casualty victim,
to add realism to an exercise.
Actor Brief
A briefing before the start of the exercise, prior to the actors taking their
positions. The Actor Controller leads the actor briefing, providing actors with
an exercise overview, safety, real emergency procedures, acting instructions,
schedule, identification badges, and symptomatology cards.
Actor Controller
An individual responsible for briefing, scheduling, and controlling the actors
used during an operations-based exercise.
Administration
Responsible for meeting scheduling and
administrative support, such as
distributing meeting invitations, read-ahead materials, and meeting minutes;
providing and collecting sign-
in sheets; and note taking during planning
meetings.
Administration Lead
An individual responsible for managing the
registration process, printed
documents, sign-in sheets, and badges.
After-Action Meeting
(AAM)
A meeting that serves as a forum to review the revised AAR and the draft IP.
Participants should seek to reach final consensus on strengths
, areas for
improvement, draft corrective actions, concrete deadlines, and owners/
assignees for implementation of corrective actions.
B
Best Practices
Peer-reviewed techniques, procedures, and solutions that prove successful and
are solidly grounded in actual experience in plans, operations, equipment,
training, and exercises.
C
Capabilities-Based
Planning
A process to build capabilities suitable for a wide range of threats and hazards
while working within an economic framework that necessitates prioritization
and choice. Capabilities-based planning is the basis for guidance such as the
National Preparedness Goal.
Capability
An ability to be planned, organized, equipped, trained, and exercised through
personnel or resources to achieve an intended target.
Capability Target
A performance threshold established by the jurisdiction/organization for each
capability.
Concept and
Objectives (C&O)
Meeting
A meeting that is the formal beginning of the exercise planning process. It is
held to identify the scope and exercise objectives. For less complex exercises
and for organizations with limited resources, the C&O Meeting can be
conducted in conjunction with the IPM.
Homeland Security Exercise and Evaluation Program
Glossary-2
Term Description
Contingency Inject
A MSEL event introduced to a player by the control staff when a key player
expected action did not occur as planned, to provide an additional opportunity
to meet exercise objectives.
Control Staff
Instructions (COSIN)
A document for the Controller portion of the C/E Handbook for large exercises
that provides detailed guidelines for control and simulation support and
establishes a management structure for these activities. This section provides
guidance for controllers, simulato
rs, and evaluators on procedures and
responsibilities for exercise control, simulation, and support.
Controller
An individual who manages exercise play and monitors
the pace of the
exercise. Controllers may prompt or initiate certain player actions to maintain
exercise pace and flow.
Controller/Evaluator
(C/E) Brief
A meeting before an operations-
based exercise that provides an exercise
overview and responsibilities
of controllers and evaluators, to include
instructions on delivering the MSEL and completing EEGs.
Controller/Evaluator
(C/E) Debrief
A meeting for controllers and evaluators to assemble after the exercise to
discuss observations and provide information to create a shared understanding
of the exercise.
Controller/Evaluator
(C/E) Handbook
A document that provides specific exercise information and guidance for
controllers and evaluators. It may be a standalone document or supplement to
the Exercise Plan and adjusted accordingly.
Core Capabilities
Distinct critical elements necessary to achieve the specific mission areas of
prevention, protection, mitigation, response, and recovery as outlined in the
National Preparedness Goal.
Corrective Action An item outlined in an IP that is intended to resolve areas for improvement.
Critical Tasks
An item where distinct specific actions are needed to achieve the capability
target.
D
Design and
Development
Responsible for
compiling and developing all exercise background and
facilitation or control, providing technical or functional expertise for scenario
development, developing the simulation construct as applicable, and working
with the evaluation and resource/support func
tions to ensure all exercise
design and development needs are met.
Drill
An operations-based exercise often employed to validate a single operation or
function.
E
Emergency Medical
Services Area
A location for real-world response support, such as treatment sector areas,
ambulance staging, and transportation coordination points.
End of Exercise
(EndEx)
An event that concludes an exercise.
Homeland Security Exercise and Evaluation Program
Glossary-3
Term Description
Evaluation
Responsible for the development of the overall exercise evaluation construct
and all evaluation documentation; assigns evaluation staffing in conjunction
with the exercise planning team. Collects and reviews the policies, plans, and
procedures to be examined during exercise conduct to finalize exercise
objectives, match capability targets
to exercise objectives, and determine
critical tasks to be evaluated to complete exercise evaluation guides.
Evaluation Plan
(EvalPlan)
A document for evaluators that provides guidance, instructions, and structure
on evaluating and observing during an exercise.
It also provides essential
material required to perform their specific functions.
Evaluator
An individual chosen based on their expertise in the functional areas to
observe and collect exercise data, and analyze results.
Exercise
An event or activity delivered through discussion or action to develop, assess,
or validate capabilities to achieve planned objectives.
Exercise Assembly
Area
A location for participants and resources to gather and dispatch, managed by
controller(s).
Exercise Assembly
Area Controller
An individual responsible for the logistical organization of the exercise
assembly area, including placement locations for units entering the exercise
assembly area, release of dispatched units into the field, and coordination of
routes and overall safety within the assembly area.
Exercise Director
An individual responsible for the
strategic oversight and direction on the
HSEEP Cycle phases for an individual exercise.
Exercise Evaluation
Guide (EEG)
A document that captures information specifically related to the evaluation
requirements developed by the exercise planning team. The EEG provides
evaluators with a standardized tool to guide data collection and capture
performance results. Since each jurisdiction/organization has unique targets
and critical tasks, EEGs are developed specific to their plans, policies,
procedures, and protocols.
Exercise Plan
(ExPlan)
An operations-based exercise document for players that includes an exercise
overview, objectives and align
ed capabilities, roles and responsibilities,
logistics, schedule, and communications plan.
Exercise Planning
Team
Individuals that are responsible for the design, development, conduct,
evaluation, and improvement planning for an individual exercise.
Exercise Planning
Team Lead
An individual responsible for the management of the exercise planning team
and coordinates with the Exercise Director and senior leaders.
Exercise Play Area
A location where player activities and tasks are demonstrated during an
exercise.
Exercise Play Rules
Items that identify the parameters that exercise participants follow during an
exercise.
Exercise Program
Manager
An individual responsible for overseeing and integrating preparedness
activities for a jurisdiction/organization over time
Homeland Security Exercise and Evaluation Program
Glossary-4
Term Description
Expected Action
A MSEL event representing an anticipated action to be taken by a player
during the exercise.
Extent of Play
Agreement
A document that is a formal written agreement outlining the participating
jurisdiction’s/organization’s level of play during an exercise.
F
Facilitated Discussion
A method that uses a facilitator with functional or subject-matter expertise and
awareness of appropriate plans and procedures for discussion-based exercises.
Facilities
Responsible for coordinating the needed venue locations for exercises
including meeting rooms, conference centers, training locations, and locations
for operations-based exercises.
Facilities Lead
An individual responsible to work closely with the logistics section to manage
exercise venue considerations, such as set up, tear down, scheduling, and
traffic.
Final Planning
Meeting (FPM)
A meeting that serves as the formal end of the exercise planning process. It is
held to finalize exercise documentation and logistics.
Full-Scale Exercise
An operations-based exercise that is typically the most complex and resource-
intensive of the exercise types and often involves multiple agencies,
jurisdictions/organizations, and real-time movement of resources.
Functional Exercise
An operations-based exercise designed to assess and evaluate capabilities and
functions while in a realistic, real-time environment; however, movement of
resources is usually simulated.
G
Game
A discussion-based exercise that is a structured form of play designed for
individuals or teams in a competitive or noncompetitive environment. It is an
event experienced by the players and gu
ided by clear rules, data, and
procedures for its execution.
Games are designed to depict an actual or
hypothetical situation to ensure that the participants make decisions and take
actions that would be plausible.
Games can be used to reinforce training,
stimulate team building, or enhance operational and tactical capabilities.
Ground Truth
A document comprised of the detailed elements of a scenario that must remain
consistent during exercise development and conduct to ensure that realism is
maintained, and objectives can be achieved.
Ground Truth
Advisor
An individual responsible for ensuring that the scenario details remain
consistent during exercise conduct.
H
Homeland Security
Exercise and
Evaluation Program
(HSEEP)
A document that provides a set of guiding principles for exercise programs, as
well as a common approach to exercise program management, design and
development, conduct, evaluation, and improvement planning.
Homeland Security Exercise and Evaluation Program
Glossary-5
Term Description
I
Improvement Plan
(IP)
A document that includes a consolidated list of corrective actions, responsible
parties, and a timeline for completion.
Initial Planning
Meeting (IPM)
A meeting that is held to refine the scope and exercise objectives. For less
complex exercises and
for organizations with limited resources, the C&O
Meeting can be conducted in conjunction with the IPM.
Inject
A Master Scenario Events List event introduced to a player by the control
staff, representing non-playing entities, to build the exercise environment
based on the exercise scenario and to drive operations-based exercise play.
Integrated
Preparedness Plan
(IPP)
A document for combing efforts across components of the Integrated
Preparedness Cycle to make sure that a jurisdiction/organization has the
capabilities to handle threats and hazards.
Integrated
Preparedness
Planning Workshop
(IPPW)
A meeting that establishes the strategy and structure for an exercise program
and preparedness efforts while setting the foundation for the planning,
conduct, and evaluation of individual exercises.
L
Lead Controller
An individual that monitors exercise progression, communicates exercise
activities throughout all venues, and manages the control staff.
Lead Evaluator
An individual that oversees all facets of the evaluation process, to include
recruiting, assigning, and training evaluators.
Lead
Facilitator/Presenter
An individual responsible for keeping a discussion focused on exercise
objectives and exploring all issues within the time allotted during discussion-
based exercises, or an individual designated to deliver information in a
structured setting.
Liability Waiver
Form
A document that each actor should receive and sign prior to the start of the
exercise. Signing a waiver form releases liability for all exercise planners and
participants. Exercising entities should use discretion when recruiting actors
under the age of 18 years because of additional challenges and concerns
related to liability. If the exercise requires volunteers younger than 18 years,
parents or legal guardians must sign their waiver forms.
Logistics
Responsible to ensure venues are adequately equipped with audio/visual
equipment, tables, and chairs and coordinates refreshments if required.
Logistics Lead
An
individual responsible to ensure proper room function and setup for
audio/visual requirements, as well as obtaining necessary equipment, food,
and drinks; works with the facilities lead.
M
Main Exercise Venue
A location designated as the primary
facility or geographic area where
participants gather to conduct an exercise.
Master Control Cell
A location where overall coordination is managed between venue control
cells, simulation cells, and other control areas.
Homeland Security Exercise and Evaluation Program
Glossary-6
Term Description
Master Scenario
Events List (MSEL)
A document or system that is a chronological timeline of expected actions and
scripted events to be injected into exercise play by controllers to generate or
prompt player activity. It ensures necessary events happen so that all
objectives are met. Larger, more complex exercises may also use a procedural
flow, which differs from the MSEL in that it contains only expected player
actions or events. The MSEL links simulation to action, enhances exercise
experience for players, and reflects an incident or activity meant to prompt
players to action.
Master Scenario
Events List (MSEL)
Manager
An individual that manages the timely and accurate delivery of injects and
player expected actions and decides on the need to release contingency injects.
Master Scenario
Events List (MSEL)
Meeting
A meeting for operations-based exercises that serves as a forum to build the
Master Scenario Events List in detail.
Midterm Planning
Meeting (MPM)
A meeting that serves as a forum to develop exercise scenario details and
timeline.
Mitigation
The capabilities necessary to reduce loss of life and property by lessening the
impact of disasters.
N
National
Preparedness Goal
(NPG)
A document that defines what it means for the whole community to be
prepared for all types of disasters and emergencies. The NPG describes 32
core capabilities that address the greatest risks to the nation.
National
Preparedness System
A method that outlines an organized process for everyone in the whole
community to move forward with preparedness activities and achieve the
National Preparedness Goal.
Note Taker
An individual that records what is said during breakout groups, interviews, or
hotwash discussions, allowing the facilitator, presenter, or interviewer to focus
on soliciting information and asking follow-up questions, and supports data
collection and data management throughout the evaluation process.
O
Objectives
An item with a distinct outcome that a jurisdiction/organization wishes to
achieve during an exercise. An objective should be: specific, measurable,
achievable, relevant, and time-bound (SMART).
Observer
An individual that does not directly participate in but may watch selected
segments of the exercise as it unfolds.
Observer Brief
A briefing before the start of the exercise that occurs on the day of an exercise
and informs observers and VIPs about the exercise background, scenario,
schedule of events, observer limitations, and
any other miscellaneous
information. Often, observers are unfamiliar with public safety procedures and
have questions about the activities; therefore, designating someone, such as a
public information officer,
to answer questions prevents observers from
interrupting exercise participants.
Homeland Security Exercise and Evaluation Program
Glossary-7
Term Description
Observer/Media Area
A location where observers and real-world media representatives can view the
exercise but not interfere with exercise play.
Observer/Media Area
Controller
An individual responsible to ensure that observers and the media stay in their
designated areas and do not interfere with the exercise.
P
Participant
A player, facilitator/controller, evaluator, simulator, actor, observer, VIP,
media, or other staff member involved in conducting an exercise.
Participant Feedback
Form
A document that the exercise team uses to collect observed strengths, areas
for improvement, and input about exercise conduct and logistics from
participants.
Pause Exercise
(PauseEx)
An event that suspends exercise play.
Photo Consent Form
A document for photographs prior to the exercise. Signing the form releases
permission for all exercise participants photographed, allowing their photo to
be used for informational and educational purposes.
Planning Meetings
Meetings that serve as a structured event or forum for completing the major
milestones of exercise design and development.
Player
An individual that has an active role in the exercise by either discussing or
performing a regular role and responsibility in response to a scenario.
Player Brief
A briefing for all players before the start of the exercise to address individual
roles and responsibilities, exercise parameters, safety, security badges, and
any remaining logistical exercise concerns or questions. Participant handouts
and ExPlans or SitMans, depending on the type of exercise being conducted,
are often distributed during this briefing. Following the exercise, controllers
ensure that appropriate players attend the post-
exercise hotwash in their
respective functional areas.
Player Handout
A document that supplements or replaces the Situation Manual and
presentation, providing players a quick-reference guide; sometimes referred
to as a “placemat.”
Player Hotwash
A meeting that provides an opportunity to discuss exercise strengths and areas
for improvement immediately following the conduct of an exercise.
Preparedness
The continuous cycle of planning, organizing, training, equipping, exercising,
evaluating, and taking corrective action to ensure effective coordination.
Presenter An individual designated to deliver information in a structured setting.
Press Release
A document that the exercise planning team develops for the media and the
public to inform them about the exercise.
Prevention
The capabilities necessary to avoid, prevent, or stop a threatened or actual act
of terrorism.
Program
Management
A process of overseeing and integrating preparedness activities for a
jurisdiction/organization over time.
Homeland Security Exercise and Evaluation Program
Glossary-8
Term Description
Props
Items that are nonfunctional replications of objects used during the exercise
that require certain actions from players.
Protection
The capabilities necessary to secure the homeland against acts of terrorism
and manmade or natural disasters.
R
Recovery
The capabilities necessary to assist communities affected by an incident to
recover effectively. Support for recovery ensures a continuum of care for
individuals to maintain and restore
health, safety, independence, and
livelihoods, especially those who experience financial, emotional, and
physical hardships.
Resource Lead
An individual responsible for obtaining proper venues, equipment, and
supplies for exercise conduct, as well as providing support for media and VIP
observers.
Resource/Support
Responsible for logistics, administration, facility, and finance support for
planning meetings, exercise documentation, and other planning
considerations.
Response
The capabilities necessary to save lives, protect property and the environment,
and meet basic human needs after an incident has occurred.
Response Route
A route for responding emergency units from the assembly area to the staging
area or other exercise sites during a response-focused exercise.
Rolling Summary
Report
A document that provides stakeholders with an analysis of issues, trends, and
key outcomes from all exercises conducted as part of the exercise program.
Root-Cause Analysis
A method used to trace the origin of an event back to earlier events and their
respective causes.
S
Safety/Security
Controller
An individual responsible to ensure that the exercise is conducted in a safe and
secure environment.
Scenario
A model or outline, either written or
depicted by an event timeline, of the
simulated sequence of events that drive the player’s discussion or actions.
Scope
An item that is defined by exercise type, participation level, exercise duration,
and exercise location that enables planners to meet exercise objectives while
staying within the resource and personnel constraints.
Seminar
A discussion-
based exercise that orients participants to or provides an
overview of authorities, strategies, plans, policies, procedures, protocols,
resources, concepts, and ideas.
Senior Leader
An individual that provide the overarching guidance and direction for the
exercise program and intent for individual exercises.
Homeland Security Exercise and Evaluation Program
Glossary-9
Term Description
Senior Leader Brief
A briefing during the design and development and before the conduct of an
exercise. The exercise planning team leader periodically consults with the
senior leaders on the exercise planning team to ensure the exercise aligns with
leader intent.
Simulation
A method of implementing the performance of a model, or combination of
models, over time.
Simulation Cell
(SimCell)
A location from which controllers deliver scenario messages representing
actions, activities, and conversations of an individual, agency, or organization
that is not participating in the exercise.
Simulation Cell
(SimCell) Controller
An individual that coordinates with the MSEL manager or may act as the
MSEL manager and directs simulators and actors.
Simulators
An individual that delivers scenario messages representing actions, activities,
and conversations
of an individual, agency, or organization that is not
participating in the exercise.
Situation Manual
(SitMan)
A document that provides background information and serves as the primary
reference material for exercise participants (core document for discussion-
based exercises).
Span of Control
A method that defines the ideal number of functions, people, or things for
which an individual or organization is responsible.
Special Effects
Items used for operations-based exercises that require trained and licensed
personnel and special permission for use.
Sponsor An individual or agency who funds and hosts the exercise.
Start of Exercise
(StartEx)
An event that marks the official beginning of an exercise.
Subject-Matter
Expert (SME)
An individual who has special skills and knowledge about a function or topic.
Symptomology Card
An item containing the signs and symptoms that an exercise actor will portray,
as well as information helpful for players.
T
Tabletop Exercise
(TTX)
A discussion-based exercise in response to a scenario intended to generate a
dialogue of various issues to facilitate a conceptual understanding, identify
strengths and areas for improvement, and/or achieve changes in perceptions
about plans, policies, or procedures.
Trusted Agent
An individual on the exercise planning team who is trusted not to reveal
exercise and scenario details t
o players or third parties before exercise
conduct.
V
Venue A location where the exercise conduct is taking place.
Venue Control Cell
A location where controllers manage individual injects designed for their
relevant players.
Homeland Security Exercise and Evaluation Program
Glossary-10
Term Description
Venue Controller
An individual that is responsible for setting up and operating a specific
exercise location. Venue Controllers manage exercise play and may prompt
or initiate certain players to ensure continuity and flow.
Venue Evaluator
An individual responsible for observing exercise activity/play in their assigned
exercise venue, area, or for a specific activity.
W
Weapons and Safety
Policy
A document for all exercises, where applicable, that shall employ a written
weapons and safety policy in accordance with applicable state or local laws
and regulations. Exercise sponsors should coordinate the application of a
weapons and safety policy with the appropriate safety and legal departments
as necessary. All players are expected to fully adhere to this policy; however,
site security personnel may continue to carry assigned weapons.
Whole Community
Individuals, to include families, businesses, faith-
based and community
organizations, nonprofit groups, schools and academia, media outlets, and all
levels of government who are involved in preparedness efforts.
Workshop
A discussion-
based exercise often employed to develop policy, plans, or
procedures.
Homeland Security Exercise and Evaluation Program
Glossary-11
This page is intentionally left blank.
