|
Applications of
Methods to Systems Engineering Phases:
Black denotes high
method/phase applicability, gray denotes medium applicability, and white
denotes low or no applicability
|
| |
|
Method |
Concept Definition |
Requirements Analysis |
Function Analysis |
Function Allocation |
Task Design |
Interface and Team Development |
Performance, Workload, and Training Estimation |
Requirements Review |
Personnel Selection |
Training Development |
Performance Assurance |
Problem Investigation |
Cognitive
Task
Analysis |
I.A.1 |
Applied Cognitive Task Analysis (ACTA) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.2 |
Critical Decision Method (CDM) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.3 |
PARI Method |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.4 |
Skill-Based CTA Framework |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.5 |
Decompose, Network, and Asses (DNA) Method |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.6 |
Task-Knowledge Structures (TKS) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.7 |
Goal-Directed Task Analysis (GDTA) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.8 |
Cognitive
Function Model (CFM) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.9 |
Cognitively Oriented Task Analysis (COTA) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.10 |
Hierarchical Task Analysis (HTA) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.11 |
Interacting Cognitive Subsystems (ICS) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.12 |
Knowledge Analysis and Documentation System (KADS) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.A.13 |
Team CTA Techniques |
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Method |
Concept Definition |
Requirements Analysis |
Function Analysis |
Function Allocation |
Task Design |
Interface and Team Development |
Performance, Workload, and Training Estimation |
Requirements Review |
Personnel Selection |
Training Development |
Performance Assurance |
Problem Investigation |
| Knowledge Elicitation |
I.B.1 |
Unstructured Interviews |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.2 |
Structured Interviews |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.3 |
Step Listing |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.4 |
Group Interview |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.5 |
Questionnaires |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.6 |
Teachback |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.7 |
Field Observations/Ethnographic Methods |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.8 |
Twenty Questions |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.9 |
Discourse/Conversation/Interaction Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.10 |
Activity Sampling |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.11 |
Think-Aloud Problem-Solving/Protocol Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.12 |
Retrospective/Aided Recall |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.13 |
Interruption Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.14 |
Shadowing Another |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.15 |
Shadowing Self |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.16 |
Simulators/Mockups |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.17 |
Exploratory Sequential Data Analysis (ESDA) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.18 |
Minimal Scenario Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.19 |
Critical Incident Technique (CIT) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.20 |
Cloze Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.21 |
Critiquing |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.22 |
Crystal Ball/Stumbling Block Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.23 |
Table-Top Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.24 |
Wizard of Oz Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.25 |
Decision Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.26 |
Rating and Sorting Tasks |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.27 |
Magnitude Estimation |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.28 |
Repertory Grid Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.29 |
P Sort |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.30 |
Q Sort |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.31 |
Hierarchical Sort |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.32 |
Cluster Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.33 |
Multidimensional Scaling (MDS) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.34 |
Likert Scale Elicitation |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.35 |
Structural Analysis Technique |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.36 |
Conceptual Graph Construction |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.37 |
Diagramming |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.38 |
Laddering |
|
|
|
|
|
|
|
|
|
|
|
|
| I.B.39 |
Influence Diagram Construction |
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Method |
Concept Definition |
Requirements Analysis |
Function Analysis |
Function Allocation |
Task Design |
Interface and Team Development |
Performance, Workload, and Training Estimation |
Requirements Review |
Personnel Selection |
Training Development |
Performance Assurance |
Problem Investigation |
Computational Cognitive
Modeling |
I.C.1 |
Keystroke Level Model (KLM) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.2 |
CMN-GOMS (Card Moran Newell GOMS) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.3 |
NGOMSL (Natural GOMS Language) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.4 |
CAT (Cognitive Analysis Tool) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.5 |
COGNET |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.6 |
COGENT |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.7 |
ACT-R (Atomic Component of Thought - Rational) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.8 |
Soar |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.9 |
EPIC (Executive-Process Interactive Control) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.10 |
Apex |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.11 |
MIDAS (Man Machine Integrated Design and Analysis System) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.12 |
SAMPLE (Situation Awareness Model for Pilot-in-the-Loop Evaluation) |
|
|
|
|
|
|
|
|
|
|
|
|
| I.C.13 |
OMAR (Operator Model ARchitecture) |
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Method |
Concept Definition |
Requirements Analysis |
Function Analysis |
Function Allocation |
Task Design |
Interface and Team Development |
Performance, Workload, and Training Estimation |
Requirements Review |
Personnel Selection |
Training Development |
Performance Assurance |
Problem Investigation |
Task
Analysis |
II.A.1 |
Behavioral Task Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| II.A.2 |
Operational Sequence Diagrams |
|
|
|
|
|
|
|
|
|
|
|
|
| II.A.3 |
Timeline Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| II.A.4 |
Operator Function Model (OFM) |
|
|
|
|
|
|
|
|
|
|
|
|
| II.A.5 |
Link Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
Computational
Task
Simulation |
II.B.1 |
IMPRINT (Improved Performance Research Integration Tool) |
|
|
|
|
|
|
|
|
|
|
|
|
| II.B.2 |
CART (Combat Automation Requirements Testbed) |
|
|
|
|
|
|
|
|
|
|
|
|
| II.B.3 |
Micro Saint (System Analysis of Integrated Network of Tasks) |
|
|
|
|
|
|
|
|
|
|
|
|
| II.B.4 |
WinCrew |
|
|
|
|
|
|
|
|
|
|
|
|
| II.B.5 |
IPME (Integrated Performance Modeling Environment) |
|
|
|
|
|
|
|
|
|
|
|
|
System
Evaluation
Methods |
III.A.1 |
Heuristic Evaluation |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.2 |
Walk-throughs/Cognitive Walk-throughs/Talk-throughs |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.3 |
Formal Usability Studies |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.4 |
Rapid Prototyping |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.5 |
Storyboarding |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.6 |
Interface Evaluation Surveys |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.7 |
Ergonomics Checklists |
|
|
|
|
|
|
|
|
|
|
|
|
| III.A.8 |
Contextual Inquiry |
|
|
|
|
|
|
|
|
|
|
|
|
| Descriptive Approaches |
III.B.1 |
Activity Theory |
|
|
|
|
|
|
|
|
|
|
|
|
| III.B.2 |
Situated
Cognition |
|
|
|
|
|
|
|
|
|
|
|
|
| III.B.3 |
Distributed Cognition |
|
|
|
|
|
|
|
|
|
|
|
|
| III.B.4 |
Naturalistic
Decision Making (NDM) |
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Method |
Concept Definition |
Requirements Analysis |
Function Analysis |
Function Allocation |
Task Design |
Interface and Team Development |
Performance, Workload, and Training Estimation |
Requirements Review |
Personnel Selection |
Training Development |
Performance Assurance |
Problem Investigation |
Human
Reliability
Analysis |
IV.1 |
Event Tree Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| IV.2 |
Fault Tree Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| IV.3 |
Failure Modes and Effects Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| IV.4 |
Barrier Analysis |
|
|
|
|
|
|
|
|
|
|
|
|
| IV.5 |
Hazard and Operability Analysis (HAZOP) |
|
|
|
|
|
|
|
|
|
|
|
|
| IV.6 |
Management Oversight Risk Tree (MORT) |
|
|
|
|
|
|
|
|
|
|
|
|
|