Scoring Criteria

Scoring Criteria

Oliver AI evaluates practice sessions using a structured scoring system. Understanding how scores are calculated helps managers configure effective evaluation criteria and helps reps understand what they need to improve.

Scoring Overview

Every practice session evaluation produces scores at three levels:

1. Criterion level -- Individual weighted factors within each behaviour
2. Behaviour level -- Aggregate score for each behaviour (1-5 scale)
3. Overall level -- Aggregate score across all evaluated behaviours

The 1-5 Scale

All behaviour scores use a five-point scale:

Score	Label	Meaning
1	Novice	Fundamental skill gaps; needs significant development
2	Developing	Emerging skill; inconsistent application
3	Competent	Solid baseline; meets expectations consistently
4	Proficient	Strong performance; exceeds expectations
5	Expert	Exceptional; can mentor others in this skill

A score of 3 represents competent performance. Most reps should aim to consistently score 3+ on all behaviours, with targeted development toward 4 and 5 on priority skills.

How Criteria Weights Work

Each behaviour has multiple evaluation criteria, each with a percentage weight:

Example: Objection Handling

Criterion	Weight	Description
Acknowledgment	20%	Did the rep acknowledge the objection?
Clarification	20%	Did they ask clarifying questions?
Value Response	30%	Did they respond with value, not just features?
Confirmation	15%	Did they confirm the objection was resolved?
Next Step	15%	Did they move the conversation forward?

The weights must add up to 100% for each behaviour. Higher-weighted criteria have more influence on the behaviour's overall score.

Score Calculation

The AI evaluates each criterion on the same 1-5 scale, then calculates the behaviour score as a weighted average:

Behaviour Score = Sum of (Criterion Score x Criterion Weight)

For example, if a rep scores:

• Acknowledgment: 4 (x 0.20 = 0.80)
• Clarification: 3 (x 0.20 = 0.60)
• Value Response: 4 (x 0.30 = 1.20)
• Confirmation: 3 (x 0.15 = 0.45)
• Next Step: 2 (x 0.15 = 0.30)

Behaviour Score = 0.80 + 0.60 + 1.20 + 0.45 + 0.30 = 3.35

This maps to a "Competent" level, approaching "Proficient."

Configuring Criteria

Managers can configure evaluation criteria through the behaviours system:

1. Navigate to Behaviours.
2. Select the behaviour you want to configure (or create a new one).
3. Define the criteria:
   • Criterion name -- What is being measured
   • Weight -- Relative importance (percentage)
   • Description -- What good performance looks like for this criterion
4. Define the rubric levels (1-5) with specific, observable criteria at each level.
5. Save the configuration.

Best Practices for Criteria Design

Make Criteria Observable

Each criterion should describe something the AI can identify in the transcript:

• Good: "Asked at least two open-ended questions about the customer's pain points"
• Poor: "Demonstrated genuine curiosity" (too subjective for AI evaluation)

Weight by Importance

Assign higher weights to the criteria most important for your sales process:

• Critical skills (like responding to objections with value) should be 25-30%.
• Supporting skills (like transition phrasing) can be 10-15%.

Keep Criteria Distinct

Avoid overlap between criteria. Each criterion should measure a different aspect of the behaviour:

• Overlapping: "Asked good questions" and "Used open-ended questions" (too similar)
• Distinct: "Open Questions" and "Active Listening" (clearly different skills)

Use Specific Rubric Descriptions

The rubric levels should describe concrete, observable behaviours:

• Level 1: "Did not acknowledge the objection or changed the subject"
• Level 3: "Acknowledged the objection and provided a relevant response"
• Level 5: "Proactively addressed potential concerns before the customer raised them"

Impact of Scoring Changes

When you modify scoring criteria:

• Future evaluations use the new criteria.
• Past evaluations retain their original scoring.
• Team members should be informed of changes so they can adjust their practice focus.

Warning: Changing criteria weights significantly can cause scores to shift. If you make major changes, communicate the rationale to your team and allow time for adjustment.

Interpreting Score Patterns

Pattern	What It Suggests	Recommended Action
All scores around 3	Consistent but not exceptional	Challenge the rep with harder scenarios
High variance (1s and 5s)	Inconsistent performance	Focus coaching on the low-scoring areas
Steady improvement	Effective practice habits	Continue the current approach
Plateau at 3-4	Comfortable but not pushing	Introduce advanced scenarios and new challenges
Declining scores	Possible fatigue or new challenges	Review recent sessions and provide supportive coaching