You are here: Home - smart as hell - Evaluating Through Rubrics

Evaluating Through Rubrics

Posted by on February 7th, 2013 with 0 Comments

Evaluation is a part of our lives: we all evaluatewe are all being evaluated, and we all invite evaluation.

When accuracy, transparency, and consistency are critical to an evaluation, we often struggle. There is, however, a very powerful – though underused – method for these types of evaluation.
These are called ‘Rubrics’.

Rubric evaluation is the quantitative assessment of criteria or competencies.
Common rubric phrases include:

  • “Kim Yu-na scored 5.0 on artistic interpretation and technical competence during her short program, though her degrees of difficulty were less than the Japanese skater”.
  • “Your slides had a ‘signal-to-noise’ ratio of 5:1. You received full scores for use of contrast, though you lost some points for irrelevant imagery”.
  • “Milton scored higher for their business school, because they have a higher teacher-to-student ration”.
  • “You lost points on your paper because you lacked the 5 credible references that were requested”.

Rubrics are often called scorecards or dashboards, although the three are quite different.

  • A rubric provides a ‘map’ to high scores by indicating that ‘x behavior will get y score’. For example, “Zero spelling errors will receive 10 points. One to three errors will receive 7 points. Four to six errors will receive 5 points. Seven to ten errors will receive 3 points. Eleven or more errors will receive 0 points.”
  • A scorecard simply provides a template for tracking scores. A bowling scorecard or golf scorecard is a good example.
  • A dash-board is a ‘real-time’ mechanism. It shows what’s happening right now. A car dashboard, for example, shows gas level, speed, engine temperature and more – right now. Otherwise the data would be useless. Which suggests to all you execs out there who love your jargon – there is NO SUCH THING as a quarterly dashboard.

Rubrics have a number of positive and negative attributes.

Pros: They reward behavior. They provide transparency of the scoring system, both to evaluate the performer and to evaluate the judge. They drive consistency in scoring. They provide actionable feedback.

Cons: They don’t necessarily reward results. They require judges. The criterial can be complex. They require a design investment of tie and/or money. They are only as good as the criteria.

Rubrics are extremely effective for measuring ‘soft skills’, when stakes are high, and when transparency and consistency are required.

Some public examples of ‘Rubrics Evaluation’ being used include:

  • Olympic Figure Skating: The Olympic Scoring Table for Women’s Figure Skating asks judges to score eight ‘planned elements’ and five ‘program elements’ in an effort to create consistency in scoring.
  • DrBeach.org: Dr. Stephen P. Leatherman, Professor and Director of the Laboratory for Coastal Research at Florida International University, has been rating beaches for 20 years. He uses 50 criteria, arranged in a rubric, to assess the beaches.
  • Nancy Duarte’s ‘Glance Meter’: As Nancy writes, “The test gives you a quantifiable way to test a slide’s viability as a glance medium by calculating a signal-to-noise ratio for individual slides.”
  • Business Weeks “Best B-Schools”: Business Week assembles this very popular llist from a multi-factor rubric.

The strength (and weakness) of a rubric is that it doesn’t rely on output. A couple million record sales is not proof that Britney Spears is a great singer. Sometimes the employee with the highest sales just got lucky – the customer opened up budget – while the best salesperson had bad luck.

Rubrics allow us to look at the competencies that create output – serving as a better indicator of skill and a better predictor of future performance.

SMART as Hell Exercise:

  1. List the evaluations you made yesterday.
  2. How many of those evaluations would be classified as ‘rubrics’ (quantitative assessment of criteria or competency)?
  3. How many of those evaluations are ‘soft’ and require consistency with transparency?
  4. Is there risk that your evaluation is not ‘robust’ enough, considering the stakes?
  5. Google your evaluation topic and see if you can find a rubric: for example, “photography rubric”, “customer service rubric”, or driving rubric”.
Share your findings below.

Share this:
Share this page via Email Share this page via Stumble Upon Share this page via Digg this Share this page via Facebook Share this page via Twitter

SMART as Hell

With a mission to help "Change Your World One Goal at a Time", SMART as Hell provides books, videos, coaching, and training programs for individuals, teams, organizations, coaches, and facilitators.

Download the SMART as Hell catalog

SMART as Hell TV

Rick Gilbert

A Graphic Guide to Writing SMART as Hell Goals

Cover - A Graphic Guide to Writing SMART as Hell Goals

A Graphic Guide to Writing SMART as Hell Goals

A Graphic Guide to Writing SMART as Hell Goals follows Mickey, a new employee, on his journey from SMART to SMART as Hell. In this fast paced comic book by award-winning author Glenn Hughes, you’ll watch Mickey rebound from a failed meeting with his boss Amy, by getting clarity around his

Buy Now

Buy This Book Online

Cover - A Graphic Guide to Writing SMART as Hell Goals

Find A Local Bookstore

,

Read More

SMART as Hell Advice

Front Cover for SMART as Hell Advice by Glenn Hughes

SMART as Hell Advice

Do your goals lack punch? Are you struggling for the motivation to realize your dreams? Do you sometimes wish you could get a nice pat on the back AND a good kick in the butt? SMART as Hell Advice is your antidote. In this book, award-winning goal expert Glenn Hughes gathers

Buy Now

Buy This Book Online

Front Cover for SMART as Hell Advice by Glenn Hughes

Find A Local Bookstore

,

Read More

Photo Jolts!

Photo Jolts! Award Winning eBook Cover

Photo Jolts!

Looking to increase clarity, creativity, and conversation?  Photo Jolts! shows you how to leverage the power of photographs to improve yourself, your team, or your organization. Master trainer Thiagi and SMART as Hell.com founder Glenn Hughes share 51 activities (with over 300 variations) that are tested and proven to help participants

Buy Now

Buy This Book Online

Photo Jolts! Award Winning eBook Cover

Find A Local Bookstore

,

Read More

Follow Us!

Archives