You are here: Home - smart as hell - Evaluating Through Deliberation

Evaluating Through Deliberation

Posted by on February 6th, 2013 with 0 Comments

As we’ve discussed on this site, we all evaluatewe are all being evaluated, and we all invite evaluation.

While most of our assessments are ‘intuitive’, sometimes some of us put a little more care and thought into our evaluation.I call this ‘Deliberation’.

Deliberative evaluation is the qualitative assessment of criteria or competencies.
Common deliberative phrases include:

  • “He makes quick, accurate decisions that benefit the customer”.
  • “The brushstrokes are very dynamic and the colors are bold”.
  • “You were clear, concise, and logical in your presentation”.
  • “I have a throbbing pain in my neck”.
  • “The editing of that movie was a complete mismatch for the tone of the script”.

This deliberation, which I also sometimes call ‘judging’, has a number of positive and negative attributes.

Pros: It can be fast (though not always, as in the case of a trial jury). It often uses ‘expert’ analysis. It often relies on established criteria. It provides some level of depth and explanation. It facilitates learning.

Cons: It requires a judge. With complex evaluations, it can create ‘paralysis through analysis’. It can encourage an evaluation that ‘sees the trees, but not the forest’. It allows for, but doesn’t necessarily require transparency.

Deliberation is effective when there is time, the criteria is defined, and the judge is trusted. But it can be risky to use deliberation when the criteria are not clear, the judge is not skilled, or the process is not transparent.

Some very public examples of ‘Deliberative Evaluation’ being used include:

  • TripAdvisor.com: TripAdvisor is a website that aims to provide the “World’s most trusted travel advise”, by allowing guests to rate hotels, restaurants, and more. Unlke Yelp.com and others, TripAdvisor uses five criteria: value, rooms, location, cleanliness, and service to create consistency in their user reviews.
  • RogerEbert.com: I choose Roger Ebert, because he’s my favorite film critic, but you could use any film, music, or book critic you prefer. I have been reading Roger Ebert’s reviews for as long as I can remember. I know his tastes and his criteria – for example, his line, “It’s not what a movie is about, as much as it’s how it’s about it”. He’s thoughtful and consistent. That makes his evaluations very useful for me.
  • American Idol: Yesterday, I used American Idol as an example of ‘intuitive’ evaluation because of the phone-in voting from viewers – who typically lack deliberations skills and are subject to favoritism and bias. The use of a panel, however, adds deliberation to the program. Project Runway, Cooking shows and other programs that use consistent criteria or returning judges would also fit in this category.
  • Juries: Whether it’s at the Cannes Film Festival, or in your local courthouse, juries provide a form of deliberation.
  • Secret Deliberations: While these seem to be less and less common, there are still events where unknown people have unknown conversations around unknown criteria to decide who wins or loses; who ‘gets in’ or doesn’t. The NFL Football Hall of Fame is still run this way. The evaluators are not allowed to discuss what happened ‘behind closed doors’.

When your evaluation is likely to determine a reward or punishment, deliberation is preferable to intuition. If you have the time and skill, important, questions like “are you prepared for the trip/meeting?”, “is my paper ready for submission?”, “shall I take him/her home”, or “should we hire him/her?” should can be answered through deliberation.

SMART as Hell Exercise:

  1. List the evaluations you made yesterday.
  2. How many of those evaluations would be classified as ‘deliberation’ (qualitative assessment of criteria or competency)?
  3. How many of those evaluations carry a significant reward or punishment?
  4. Is there risk that your evaluation is not ‘robust’ enough, considering the stakes?
Share your findings below.

Share this:
Share this page via Email Share this page via Stumble Upon Share this page via Digg this Share this page via Facebook Share this page via Twitter

SMART as Hell

With a mission to help "Change Your World One Goal at a Time", SMART as Hell provides books, videos, coaching, and training programs for individuals, teams, organizations, coaches, and facilitators.

Download the SMART as Hell catalog

SMART as Hell TV

What are Photo Jolts?

A Graphic Guide to Writing SMART as Hell Goals

Cover - A Graphic Guide to Writing SMART as Hell Goals

A Graphic Guide to Writing SMART as Hell Goals

A Graphic Guide to Writing SMART as Hell Goals follows Mickey, a new employee, on his journey from SMART to SMART as Hell. In this fast paced comic book by award-winning author Glenn Hughes, you’ll watch Mickey rebound from a failed meeting with his boss Amy, by getting clarity around his

Buy Now

Buy This Book Online

Cover - A Graphic Guide to Writing SMART as Hell Goals

Find A Local Bookstore

,

Read More

SMART as Hell Advice

Front Cover for SMART as Hell Advice by Glenn Hughes

SMART as Hell Advice

Do your goals lack punch? Are you struggling for the motivation to realize your dreams? Do you sometimes wish you could get a nice pat on the back AND a good kick in the butt? SMART as Hell Advice is your antidote. In this book, award-winning goal expert Glenn Hughes gathers

Buy Now

Buy This Book Online

Front Cover for SMART as Hell Advice by Glenn Hughes

Find A Local Bookstore

,

Read More

Photo Jolts!

Photo Jolts! Award Winning eBook Cover

Photo Jolts!

Looking to increase clarity, creativity, and conversation?  Photo Jolts! shows you how to leverage the power of photographs to improve yourself, your team, or your organization. Master trainer Thiagi and SMART as Hell.com founder Glenn Hughes share 51 activities (with over 300 variations) that are tested and proven to help participants

Buy Now

Buy This Book Online

Photo Jolts! Award Winning eBook Cover

Find A Local Bookstore

,

Read More

Follow Us!

Archives