Why transcript quality and scale definitions drive scoring accuracy
Transcript dependency is the first rule for consistent sales reviews. According to Aircall, users can evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation. You must ensure your sales tool generates or imports transcripts for every call you want to score automatically. On Aircall, only calls with transcripts benefit from AI evaluation.
Scale definition is the second rule. According to Avoma, the scorecards are based upon a 5-point psychometric response scale in which responders specify their level of agreement to a statement. This 5-point structure defines the range of possible values for each question. When you configure your scorecard, you must align your questions with this specific response scale. You should review your existing scorecard questions to ensure they fit this psychometric framework.
Key takeaways for how to configure ai call scoring rules in sales tools
This transcript dependency is the first rule for standardizing reviews.
Scale definitions anchor the scoring logic. The scorecards are based upon a 5-point psychometric response scale in which responders specify their level of agreement to a statement, according to Avoma. Standardizing this specific scale prevents misinterpretation of rep performance.
Weight calibration ensures balanced assessment. You set the question weights to 30%, 20%, 40% and 10%, according to Gong. Assigning precise percentages allows for consistent mathematical comparison across different sales calls.
Bias mitigation requires strict thresholds. AI has a bias toward "Yes" so set a high bar, according to Outreach. Applying this rule to questions like "Did the rep identify the decision-making process?" prevents inflated positive scores.
Reference table for scorecard configuration across four tools
Eight rows below pair each scoring rule with the publisher quote that defines it. Each row names the publisher and cites the source where that rule is documented.
| Publisher | Rule Item | Specific Configuration Detail | Source Citation |
|---|---|---|---|
| Aircall | Transcript dependency | Users can evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation. | According to Aircall |
| Avoma | Scale normalization | The scorecards are based upon a 5-point psychometric response scale in which responders specify their level of agreement to a statement. | According to Avoma |
| Gong | Missing data handling | If the skipped question is a range question from 1 to 5, the value given to it in the overall score is 1. | According to Gong |
| Outreach | Bias mitigation | AI has a bias toward "Yes" so set a high bar. | According to Outreach |
| Avoma | Question specificity | Write a targeted, single objective question or a descriptive topic to be covered. | According to Avoma |
| Gong | Weight calibration | You set the question weights to 30%, 20%, 40% and 10%. | According to Gong |
| Avoma | Auto scoring triggers | Select the purposes that you want the AI Coaching Assistant to auto-score the conversations for against this scorecard. | According to Avoma |
| Aircall | Manual criteria | Call Scoring allows managers to create scorecards that combine AI evaluated questions and manually evaluated criteria to assess call performance. | According to Aircall |
Use this table to verify that your current scorecard setup aligns with the specific mechanical rules documented for each platform.
How to normalize 5-point scales and handle non-binary questions
Standardizing evaluation metrics requires mapping specific option counts to a uniform scale. For non-binary evaluation type of questions, the score will be prorated based on the number of options, according to Avoma. This proration logic ensures that questions with varying answer choices contribute proportionally to the final assessment.
Consider a specific scenario where a question presents three distinct options. In this case, if a question has three options with 1 being the worst and 3 being the best, the best option will be scored with a 5 and the worst will be scored with a 1, according to Avoma. This mapping converts the raw option index into a standardized five-point value. The system treats the lowest available option as the baseline and the highest available option as the maximum score.
When configuring your scorecard, identify every question that does not fit a simple yes-or-no format. For each of these items, determine the total number of selectable options. For non-binary questions, the score is prorated based on the number of options. The goal is to align all non-binary questions with the same five-point framework used for other evaluation types.
Do not assume that a three-option question automatically maps to a three-point scale. Instead, stretch the range to fit the standard five-point maximum. If your tool supports custom weighting, verify that the prorated values reflect the intended difficulty or importance of the question. Check that the highest option consistently receives the top score and the lowest option receives the bottom score.
Review your current question bank for any items with more than two options. For each item, calculate the score assigned to the middle options if applicable. Ensure that the gap between the worst and best scores remains consistent across all non-binary questions. This consistency prevents skewed averages that might result from uneven scaling.
Setting question weights and recalibrating to 100 percent
Assigning percentage weights to individual questions allows you to prioritize specific behaviors in your evaluation framework. In the "How to create and manage scorecards" documentation, the process involves defining these values directly for each question item. For example, you might set the question weights to 30%, 20%, 40% and 10% according to Gong. This distribution reflects the relative importance of each criterion within the total score.
Once you have entered your initial weight values, verify that the sum of all assigned percentages equals exactly 100%. If your manual adjustments result in a total that exceeds or falls short of this target, the system provides a specific correction tool. You can click Recalibrate to 100% to adjust automatically according to Gong. Click Recalibrate to 100% to adjust automatically.
When configuring your scorecard, ensure that the weight distribution aligns with your team's strategic priorities. Questions that represent critical deal-closing behaviors should typically carry higher percentages than those representing minor procedural steps. The automatic recalibration feature serves as a safety check to prevent configuration errors that could skew overall performance metrics. By using this tool, you maintain a consistent baseline for comparing rep performance across different call recordings.
Audit your current scorecard weights to ensure they reflect your latest sales priorities and that the total sums to 100% using the available automatic adjustment tools.
Mitigating AI bias and handling missing or skipped questions
AI evaluators are not neutral observers; they carry inherent tendencies that can skew results if you do not account for them. According to Outreach, the system has a bias toward "Yes" answers, so you must set a high bar for what constitutes a positive response. This tendency is particularly relevant for binary questions, such as "Did the rep identify the decision-making process?" where a vague "yes" might inflate performance metrics. To counteract this, draft questions that require specific evidence from the transcript rather than general impressions. Outreach gives the question "How well did the rep control the call flow?" By demanding concrete proof, you force the AI to look for actual behavioral markers rather than relying on its default positive lean.
When a rep skips a question or the transcript lacks the necessary context, the scoring system must assign a value to that gap. According to Gong, if a skipped question is a range question from 1 to 5, the value given to it in the overall score is 1. This means that skipping a complex evaluation question effectively penalizes the rep with the lowest possible score on that scale. If the skipped item is a yes/no question, the value given is 0, according to Gong. These specific values ensure that incomplete data does not artificially boost a rep’s average score. You cannot assume that all tools use the same normalization formula for skipped questions, so verify these specific values in your platform’s documentation.
Action step: Review your last ten scored calls to identify which questions were skipped and which binary questions received "yes" answers without supporting transcript evidence.
Triggering auto-scoring and managing manual evaluation overrides
Configuring the trigger points for automated evaluation requires understanding which subscription tiers support the feature and how to blend AI results with human judgment. Aircall's documentation states that AI Assist Standard does not support automated call scoring Aircall. For users on the Pro tier, specific features can be used with automated call scoring rules, exclusive to Pro Aircall. This distinction is critical for budget planning, as the ability to trigger auto-scoring is not available across all subscription levels for this publisher.
When the appropriate tier is active, the system allows for a hybrid evaluation model. Aircall explains that Call Scoring allows managers to create scorecards that combine AI evaluated questions and manually evaluated criteria to assess call performance Aircall. This structure means that a single scorecard can contain both automated checks and manual review items. Managers should design their scorecards to reflect this split, assigning specific questions to AI evaluation and others to human assessment. The platform does not disable manual scoring when AI scoring is active; rather, it integrates both methods within the same scorecard framework.
For teams using Avoma, the configuration process focuses on selecting specific conversation purposes. The platform instructs users to select the purposes that you want the AI Coaching Assistant to auto-score the conversations for against this scorecard Avoma. This step ensures that the AI only evaluates calls relevant to the chosen coaching goals. Once these purposes are selected, the system takes over the evaluation process. Avoma notes that the AI Coaching Assistant will automatically score the member with the scorecard and display the score Avoma. The AI Coaching Assistant will automatically score the member with the scorecard and display the score.
Action step: Check your current subscription tier to confirm access to automated scoring features, then define the specific conversation purposes for AI evaluation in your scorecard settings.
Related guides
FAQ: How to configure ai call scoring rules in sales tools
What happens to the overall score if a range question is skipped?
Gong assigns a value of 1 to a skipped range question from 1 to 5 in the overall score, according to Gong. This specific value applies when the question is skipped during the evaluation process.
How are non-binary question scores prorated?
For non-binary evaluation types, Avoma prorates the score based on the number of options, according to Avoma. If a question has three options, with 1 being the worst and 3 being the best, the best option is scored with a 5 and the worst is scored with a 1.
Can AI score calls that do not have a text transcript?
Aircall allows users to evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation, according to Aircall. Only calls with transcripts benefit from AI evaluation.
How should you handle AI bias toward positive answers in yes no questions?
Outreach notes that AI has a bias toward "Yes," so you should set a high bar for your questions, according to Outreach. An example question provided is "Did the rep identify the decision-making process?"
What is the difference between manual and automated call scoring availability?
Aircall's AI Assist Standard does not support automated call scoring, according to Aircall. This feature is exclusive to the Pro tier, distinguishing it from the Standard offering.