The Four Slices of Evaluation

We are all being evaluated. And we are all evaluators.
But how do we evaluate? By making two choices.

Will I evaluate qualitatively or quantitatively? 

Sometimes we evaluate ‘qualitatively’, by using words:

• She’s a lousy artist.
• They’re a funny group of people.
• What a smooth talker.
• It’s a very innovative company.
• Her daughter is smart.
• The cinematography is excellent.

Evaluating qualitatively can be difficult and controversial.
Think about how you and your friends disagree over music, movies, and food.

It gets even harder with talent.

Other times we evaluate ‘quantitatively’, by using numbers:

• He got a 90 on his test.
• He’s a billionaire.
• It’s the number one movie in America.
• She recorded two gold records.

Once we decide if we will evaluate qualitatively or quantitatively, we have a second decision to make.

Will I evaluate the output or the competencies that went into the output?

Sometimes, we will choose to evaluate output: A movie, a painting, a car, or a singing performance as a ‘whole’:

• It’s a good movie.
• Monet is a great artist.
• He is handsome.
• She sold 2 million records.

Or we can evaluate the competencies that created the output:

• The script was inventive, but the acting was ham-fisted.
• Monet’s use of color was ground-breaking.
• The figure skater’s double-axel is extraordinary.
• Stanfords student-to-teacher ratio is world-class.

From these two decisions – qualitative vs. quantitative and output vs. competencies – we can create The Four Slices of Evaluation, as seen in this example:

• Intuition: The qualitative evaluation of output (Stanford is a great college)
• Deliberation: The qualitative evaluation of competencies (It has excellent standard for faculty)
• Rubrics: The quantitative evaluation of competencies (It scored well on Business Week’s ‘Best Business School’ Scorecard)
• Metrics: The quantitative evaluation of output (Graduates earn 30% higher salaries)

On my drive into work, I typically use intuition to say, “The weather is good”.
I use metrics to say, “My gas tank is ¾ full” or “I’m driving 45 miles an hour”.
I might use intuition to say, “That driver sure uses his brakes a lot”.
I’m unlikely to use a rubric while driving – although I might use one from an auto magazine to make my decision about purchasing a car.

SMART as Hell Exercise:

1. Identify an evaluation you gave or received recently. 
2. Identify which of the Four Slices of Evaluation yours fell into?
3. Reflect – would a different slice have provided a different result?

Share your findings below.