How Do We Better Assess and Grade Decision Making and Adaptability in Those We Train?

Foch on Education

Developing an effective plan for assessing students is the most difficult challenge for courses applying Outcomes Based Training and Education OBTE and the Adaptive Course Model (ACM) as part of their Program of Instruction (POI). The reason for this is the tendency to seek easily quantifiable methods of assessment. In most course environments so much depends on a student’s class rank and the number of people a course graduates that there is often a burning desire to remove judgment from instructors in favor of a rigid, mathematical grading rubric. In short, there is a natural fear of subjectivity and a longing for the safety and presumed reliability of numbers and checklists. However, is it “safe” to quantify the intangibles of leadership and adaptability? How can one assign a number to creativity and initiative?

Despite these questions, one thing is certain: fill-in-the-blank and multiple choice tests are woefully inadequate for measuring our effectiveness in achieving the desired outcomes for our police training programs. Since the ability to memorize information is not a good indicator of one’s decision-making skills while in conflict, there is no reason to use this criterion as a basis for testing.

Instead, an adaptive course model based examination focuses on things that are much more difficult to quantify, placing a great deal of the responsibility and trust in the individual instructor. Some adaptive course model based tests are of the short-answer variety. These examinations place students in a specific tactical scenario and require them to make decisions. They must then explain the reasons behind their decision in writing. For example, a student might be told that he is the responder to an active shooter situation and must take action to stop the threat and render aid.

After being presented with information about the composition of the active shooter situation, a map, the nature of the threat, and the specifics of the mission and Commander’s Intent, the student would have to determine which course of action he will take and then explain why that course of action was selected. The instructor then assigns a grade on the approach used to solve the problem using the information at hand and how well the reasoning was communicated. This gets to the point of examining “how” the student is thinking, but not “what” he is thinking.

Another and more common, assessment technique that Adaptive Course Model instructors use is a “graded tactical decision exercise (TDE).” Just like the short-answer tests, these are scenario-based and require students to make decisions on the fly. This technique is virtually identical to a standard in-class TDE with the exception that students are required to write out or brief their solutions to their instructors who, in turn, grade those solutions. In many cases, these examinations require students to produce a concept sketch with short hand-written notes concerning exact guidance for individuals, their teams, the sequence of events, and most importantly the purpose behind various actions.

Regardless of the technique or format of the assessment, the tactical scenario must allow for multiple correct ways to solve the problem. For the assessment to be truly effective, students must have the freedom to actually make a decision on their own and formulate a plan rather than being forced to regurgitate a pre-determined template or school solution. If tests fail to allow room for creativity, students become focused on identifying the approved solution rather than thinking for themselves.

In order to permit freedom of thought, scenarios must have a significant amount of ambiguity. The situation must be such that one could reasonably interpret the available information in multiple ways. Of course, this does not preclude the existence of wrong answers.

Whether on an exam employing TDGs, or during training, teachers should use multiple tools to give officers continual and detailed evaluations that will allow the cadet to evolve, improve, and prepare for the graded field evaluations. During these tests, officers will be evaluated on their ability to lead, demonstrate adaptability and make intuitive decisions under varied conditions. Evaluation criteria should consider the following questions:

  • First and foremost, did the officer make a decision? And was it timely?
  • If so, did the officer effectively communicate it to officers?
  • Was the decision made in support of the commander’s intent (long-term contract), and mission (short-term contract)?
  • If not, was the officer’s solution based on changing conditions that made it a viable decision, even if it violated the original mission order, but nevertheless supported the commander’s intent?

“Guiding actions” intertwine with the department’s core values when evaluating police leadership, performance and potential. The stakes are high; the department needs to broaden its understanding of successful leadership from one that focuses almost entirely upon mission accomplishment; to one that includes long-term organizational health of the unit and its personnel alongside of mission accomplishment.

In other words, the Police Department’s current culture evaluates successful performance by determining whether an officer or leader accomplished a specific mission. The focus is on the “bottom line.” However, this method is shortsighted, and in the current leader paradigm, it can produce “performers” rather than leaders. Measuring an officer’s potential, on the other hand, allows for an assessment that incorporates an officer’s ability to develop teams as well as individuals, even in a classroom setting. This method can also include measurements of an officer’s loyalty, initiative and risk-taking.

To create problems that will properly demonstrate an officer’s potential, scenarios must be used that encourage officers in officer roles to take risks (that they can justify and explain) in accomplishing their mission. In the AAR, the commander should praise good performance of his peers (in the officer role), while accepting responsibility for their failure. The idea is that officers will eventually emulate this behavior over time, and begin to realize their potential.

Assessments should involve more than just training unit and officer observations of an officer’s level of adaptability. Performance evaluations also occur in the classroom. However, this does not imply that the use of traditional, Industrial Age testing techniques should be continued, because those techniques only reinforce rote memorization. These negative techniques include “true or false” questioning, “fill-in the blank” or “multiple-choice” examinations.

However, training unit likes to save time by using these linear evaluation techniques. They also provide quick feedback to the tested officer, the training unit, and the chain of command when utilized for reports and Power Point slides. But these teaching techniques cheat the officer because they focus on short-term results.

Since “knowledge” and “social judgment” is also part of the traits of adaptability, continual observations and evaluations of how a leader chooses to communicate decisions to officers or to inform the chain of command must occur. If leaders do not communicate decisions effectively to their officers or units, it makes no difference whether they are decisive or timely. This is why it is important that evaluations assess communication abilities, whether it is written or verbal.

Violations of the Commander’s Intent, unethical conduct, poor communication, or an unrealistic course of action all constitute an automatic failure. Additionally, if the student is unable to make a decision within the time and information constraints of the test, the student is assigned a failing grade.

On Commanders Intent: “The task describes in quantitative terms the who, what, where and when of the mission. The intent, on the other hand, describes the why behind the mission and, in qualitative terms, the end results we expect.” ~Major John F. Schmitt, USMCR, in Mastering Tactics

These automatic failure criteria are absolutely essential in communicating to students that they cannot achieve success in the class by going through the motions of employing a template or checklist to the problem.

The intentional ambiguity in the scenarios necessitates other efforts to keep everyone on the same path when it comes to grading. It is vital to ensure consistency across the board in this area without imposing an overly-restrictive grading scheme that would hinder freedom of judgment from the instructors. In order to effectively calibrate the grading criteria, all instructors must participate in a free exchange of ideas regarding the key concepts that are the focus of the upcoming assessment.

In ACM-based courses, these group discussions are referred to as Faculty Development (FD) sessions. Not all FD sessions focus on grading, but those that do begin with the instructors actually taking the test followed by an open discussion regarding the content of the exam and how to approach grading. At the end of this exchange, the Course Director compiles the applicable notes from the session into a short set of general guidelines. Because these guidelines are the product of a collective effort, they keep grading consistent among all instructors.

One of the essential principles of outcomes based training is the requirement to treat the trainee like an adult. This encourages them to take ownership of their development and training. Not surprisingly, students at all levels from entry level to senior executives respond accordingly. If the expectation is that they cannot be trusted to do anything without micro-management, then students will fail without extensive guidance.

However, if from the very beginning the expectation is that they must think on their own and take responsibility for their own training, they will almost always conduct themselves responsibly.

This post was adapted from Chapter 26 in The Adaptive Leadership Handbook Innovative Ways to Teach and Develop Your People which is available on Amazon.

Stay Oriented!

Fred