Home / Introduction / Motivation / Methods / Applications / More Information

Below are the various phases of the system engineering life-cycle where Cognitive Engineering methods may have a substantial impact.  Click each phase to see the objectives of the phase, the role of the Cognitive Engineer (e.g. a human factors engineer, a human-computer interaction specialist, or a systems engineer who is knowledgeable about human-centered design) in meeting those objectives, and the most applicable Cognitive Engineering methods.  Here are the phases:

Most Applicable Cognitive Engineering Methods

• Applied Cognitive Task Analysis (ACTA) and the Critical Decision Method can be used to uncover the challenging decisions made in the domain and identify areas where support is needed.

• Goal Directed Task Analysis can be used to uncover the information required to maintain situation awareness.

• Conceptual Knowledge Elicitation techniques, such as various Rating and Sorting tasks, can be used to determine the key concepts in the domain.

• Interviewing and observing techniques, especially Field Observations/Ethnographic Methods and Contextual Inquiry, can be used to uncover how workers interact and use existing systems, thus guiding the development of new systems. They can also be used to study the characteristics of the environment the system will operate in.

• Storyboarding and Rapid Prototyping methods can be used to give users a glimpse of proposed system functionality and get user feedback early in the design process.

• Missions and scenarios can be constructed to identify how the system is likely to be used and what it must be able to do.  Missions and scenarios that focus on important cognitive and decision making tasks can be devised based on analyses performed using methods including CWA, ACWA, ACTA, and Goal Directed Task Analysis.

• The work domain analysis phase of CWA and ACWA can help answer questions about why a new system should exist, what functions it should implement, and what physical devices are necessary. While time consuming to construct, a work domain analysis will be used throughout the system design process. Furthermore, it will not change as technology changes (since it defines the fundamental constraints of the work domain, which are technology independent), so it will also be helpful in designing future systems. The format of the work domain analysis used by ACWA can also highlight key decisions made in the domain, which can also be helpful in developing the system concept.

• ACTA and the Critical Decision Method can be used to uncover the information, cues, and strategies required to make key decisions.

• Goal-directed task analysis can be used to uncover the information required to maintain situation awareness.

• Hierarchical Task Analysis can be used to record system requirements and how they can be achieved, including the order in which tasks and subtasks must take place.

• Interviewing and observing techniques, such as Contextual Inquiry, can be used to determine how well existing systems support user requirements.

• The results of the work domain analysis phase of CWA and ACWA can be used to develop requirements, and to help put the system requirements in context by providing a framework for determining whether requirements relate to the purpose of the system or to the physical devices of the system. The work domain analysis phase can also help define hardware and software needs (e.g. models, databases, sensors, etc.). In addition, the ACWA functional abstraction hierarchy can show the key decisions that must be made and the information required to make those decisions.

• The ANALYSE phase of COADE can be used to specify the cognitive requirements that must be addressed in system design.

• The results from ACTA and goal-directed task analysis can be examined to assure the system functions meet the decision and information requirements.

• The Hierarchical Task Analysis method can be used to translate the function decomposition into tasks and supporting tasks.

• Operator Function Models (OFM’s) can be constructed to represent the operators’ functions in the system. The CFM technique can be used to determine which functions are cognitively demanded and should be pursued more deeply with Cognitive Task Analysis techniques.

• Task analysis techniques such as Operational Sequence Diagrams and timeline analysis can be used to represent the person’s function in the system.

• The work domain analysis phase of CWA and ACWA can be used to represent the functional decomposition of the system. The functional abstraction generated from ACWA in particular can be useful in assessing the key decision making activities and the information required to make those decisions.

• The Hierarchical Goal Analysis of the PCT-based approach can be used to conduct function analysis.

• Information flow analysis can be used to chart the flow of information and decision required to carry out the function of the system. Function flow analysis can be used to show the information flow relationships between system functions.

• Signal flow graph analysis can be identify how the input variables of the system are transformed into output variables.

• Cognitive Task Analysis techniques, especially ACTA and the Cognitive Function Model (CFM), can be used to identify and analyze system functions that have a high cognitive component and need to be allocated to people rather than machines.

• Results from a Hierarchical Task Analysis can guide function allocation decisions.

• The social organization and cooperation analysis phase of CWA can be used to describe actor roles, including the allocation of function and coordination structures. This phase is built upon the results from the strategies analysis phase of CWA. In addition, the work domain analysis phase of CWA and ACWA provides all of the functionality required to operate in the domain, which could help guide function allocation decisions. The work domain phase of CWA and ACWA help define the problem space, which must be done before function allocation decisions can be made.

• Function allocation techniques from the Human Factors community, such as Fitts list, can be applied.

• The Hierarchical Goal Analysis of the PCT-based approach can guide function allocation decisions.

• Cognitive Task Analysis methods are useful for analyzing tasks that are highly cognitive in nature, whereas behavioral task analysis techniques, especially Operational Sequence Diagrams and Timeline Analysis, are suited for analyzing tasks that are more procedural in nature.

• Computational Cognitive Modeling and Computational Task Simulation can be used to evaluate impacts on performance and workload with different task designs.

• Ergonomics checklists can be used to assure a proposed task design conforms to certain human factors standards.

• The control task analysis phase of CWA can assist in describing control tasks, and the strategies analysis phases of CWA can inform human-machine dialog design.

• The ANALYSE phase of COADE can be used to generate cognitive requirements through task analysis and cognitive task analysis activities.

• Results from Cognitive Task Analysis techniques, such as ACTA and the Critical Decision Method, are useful for designing interfaces that support the decision making strategies and information needs of the operator.

• Results from Cognitive Task Analysis can be useful in assessing teamwork situations. These methods, especially Goal-Directed Task Analysis, can provide insight into how teams make decision and how they develop a shared situation awareness. A team using a proposed system design can be assembled and team situation awareness can be measured, giving insight into how the team design might be improved.

• Results from interviewing and observing methods can be useful in informing the design of interfaces that meet user needs.

• Simulators/Mockups of a proposed interface can be constructed and presented to users.

• Computational Cognitive Modeling and Task Simulation techniques can be used to create surrogate users performing a task with a proposed interface, generating performance and workload estimates. The COGNET framework is particularly suited for modeling teamwork situations.

• Rapid prototyping and storyboarding can be used to get user feedback early on an iterate a proposed design quickly,

• Heuristic Evaluation, Walk-throughs/Talk-throughs/Cognitive Walk-Throughs, and Usability Studies can be used to iterate and improve proposed interfaces.

• Link Analysis can be used to optimize the placement of system components to minimize movement times and distances.

• The strategies analysis phases of CWA can inform human-machine dialog design. And the social organization and cooperation analysis phase of CWA can be used to describe actor roles, including the allocation of function and coordination structures. The work domain analysis of CWA and ACWA can also be used to determine major decisions and the information needed to make those decisions, providing insight into interface design.

• The DESIGN phase of COADE can be used to guide the development of user interfaces based on cognitive requirements addressed in the ANALYSE phase.

• Computational Cognitive Modeling and Computational Task Simulation techniques can be used to construct simulations of surrogate users to assess performance and workload demands with a proposed system design.

• Simulators can be used to gauge performance using actual users.

• Walk-throughs/Talk-throughs/Cognitive Walk-throughs can be conducted with a proposed system designs to get feedback from actual users.

• Usability studies can be used to assess deficiencies with proposed interfaces to assess how easy they are to learn and use.

• The work domain analysis phase of CWA and ACWA can be used to define the requirements of training simulators.

• The ANALYSE phase of COADE can be used to analyze the performance of behavioral and cognitive tasks, addressing ways in which performance is limited, which aspects of performance are most frequently limited, and whether it is feasible to remedy these limitations.

• Cognitive Task Analysis techniques, especially ACTA and GDTA, are useful in assessing whether the information needed to maintain situation awareness and make decisions are provided.

• Interviewing and observing and process tracing techniques can be used throughout the design process to get feedback from users as to how well a proposed deign meets their needs.

• Computational Cognitive Modeling and Computational Task Simulation Techniques can be used to construct simulations of surrogate users interacting with a proposed system design, thus providing performance and workload estimates of a current design.

• Heuristic Evaluation, Walk-throughs/Talk-throughs/Cognitive Walk-throughs, Usability Studies, and Interface Evaluation Surveys can be used throughout the design process to evaluate how easy a proposed design is to use and learn.

• The work domain analysis phase of CWA and ACWA can be used to evaluate alternative designs in terms of how well the technical solution supports the functional purposes of the work domain. And CWA also allows testing of whether the system under development supports the necessary control tasks, strategies, role allocation and coordination structures, and operators’ cognitive abilities.

• The EVALUATE phase of COADE can be used to verify whether the proposed system design actually addresses the requirements specified in the ANALYSE phase.

• ACTA and the Critical Decision Method, can be used to uncover difference between novice and expert performance.

• The PARI method can be used to used to develop a set of representative problems in the domain along with expert solutions to those problems.

• COTA can be used to develop assessments of job expertise and performance.

• The Skill-Based CTA framework can be used identify the hierarchy of skill needed to perform competently in a domain.

• The worker competencies analysis phase of CWA can identify the competencies that workers require to carry out the work of the proposed system. This is dependent upon the system functions that have been allocated to people, which stem from the results of the social organization and cooperation analysis phase.

• Results from a variety of CTA methods, especially ACTA, the Critical Decision Method, the Skill-Based framework, PARI, DNA, and COTA, can be used to uncover the skills and knowledge necessary to perform the cognitively challenging work of the system. They can also be used to highlight differences between expert and novice performance, thus examining how novices can more quickly be brought to the performance level of experts.

• Results from Knowledge Elicitation techniques can also be used to uncover the skills and knowledge necessary to perform the cognitively challenging work of the system. The specific techniques to apply depend largely on the type of knowledge to be elicited.

• Results from task analyses can be used to develop training for tasks that are less cognitive and more procedural in nature.

• Results from usability studies can be used to uncover areas where operator support and online help would be effective.

• The work domain analysis phase of CWA and ACWA can be used to define the requirements of training simulators. In addition, the work domain analysis allows training to be developed that focuses on satisfying the functional purposes of the work domain, rather than training that describes specific sequences of behaviors. Similarly, the control task analysis and strategies analysis phases can also guide training development

• Results from the ANALYSE phase of COADE, including the Behavior Model and the Cognitive Model, may guide training development.

• The PCT-based approach tracks the declarative and procedural knowledge needed by operators, which can inform training design.

• The Critical Decision Method and the Critical Incident Technique can be used to assess how well the system supports challenging situations that have previously arisen in the domain.

• Interviewing and observing techniques, such as Contextual Inquiry, can be used to evaluate how well a fielded system supports the needs of users.

• Human Reliability Analysis techniques, such as Fault trees, Event Trees, Failure Modes and Effects Analysis, and Hazard and Operability Studies, can be applied to identify potential errors, their likelihood of occurrence, and how they may be prevented or reduced. The Management Oversight and Risk Technique (MORT) can be used to address the adequacy of management safety structures.

• CWA and ACWA allow the testing of whether the system supports the necessary control tasks, strategies, role allocation and coordination structures, and operators’ cognitive capabilities. A work domain analysis can be useful in showing that a particular system is no longer competitive in meeting the functional requirements of the work domain and is ready to be replaced. In addition, control task analysis may be useful for revealing shortfalls in the relevance or effectiveness with which control tasks are carried out.

• The EVALUATE phase of COADE can be used to perform a summative evaluation of the system, addressing whether it actually accomplishes its goal, and whether users accept the system.

• The Critical Decision Method and Critical Incident Technique can be used to analyze the decision making activities and bases for situation assessment in non-routine incidents.

• Interviewing and observing techniques can also be used to aid in the investigation of accidents and incidents.

• Simulations of accidents and incidents can be constructed to recreate the conditions that existed.

• Usability Studies, Heuristic Evaluation, and Walk-throughs/Talk-throughs/Cognitive Walk-Throughs can be used to address human-computer interaction deficiencies that may contribute to accidents and errors.

• Human Reliability Analysis techniques, especially Management Oversight and Risk Trees (MORT) and Barrier and Work Safety Analysis, can be used to determine ways in which future accidents can be prevented.

• The results from the various phases of CWA and ACWA can shed light on the root causes of accidents and incidents in terms of whether the system failed to adequately support the operators.