Role

Design lead

Responsibilities

E2E design, research, testing

Team

PM, AI & 3 eng

Duration

Mar—Oct 2024

AI Scorecards

Problem space:

Quality Assurance Scorecards serve as essential instruments for evaluating contact center agents during customer interactions. This process can be quite complex and time consuming, requiring QA analysts and supervisors to review A LOT of call recordings to verify that agents follow established protocols and effectively address customer inquiries.


Solution opportunity:
This is where AI comes into QA scorecards to boost speed and accuracy while minimizing time spent on mass call scoring.

Main experiences

Scorecard creation—Introduced templated AI questions to the creation flow to simplify and speed up the process.

Scorecard creation—Introduced templated AI questions to the creation flow to simplify and speed up the process.

Question set-up and Scorecard publishing—Adding more template AI questions made easy while also providing custom question creation, with advanced LLM transcript recognition for more accurate AI grading. With published scorecards, calls begin automatic grading immediately.

Question set-up and Scorecard publishing—Adding more template AI questions made easy while also providing custom question creation, with advanced LLM transcript recognition for more accurate AI grading. With published scorecards, calls begin automatic grading immediately.

AI grades in action in the background—With AI Scorecards, more than 95% of contact center are automatically scored. This reduces significant time and effort spent grading by supervisors and QA analysts while increasing QA. While AI grades automatically, grades can still be overridden by a human—and even disputed by agents (Dispute Management project coming soon).

AI grades in action in the background—With AI Scorecards, more than 95% of contact center are automatically scored. This reduces significant time and effort spent grading by supervisors and QA analysts while increasing QA. While AI grades automatically, grades can still be overridden by a human—and even disputed by agents (Dispute Management project coming soon).

Impact

This feature adoption increased by over 50% while net new customers that opted to buy this add-on increased by more than 35% continuing to be a key revenue driver for Dialpad.

Ai grades to date

3M+

...and counting. Since launching in October 2024, over 3M grades have been automated by Ai.

ARR since launch

$10M+

This feature adoption increased by over 50% while net new customers that opted to buy this add-on increased by more than 35% continuing to be a key revenue driver for Dialpad.

Ai grade increase

4k

September 2024

200k

October 2024

550k

November 2024

Our data shows the significant jump in total grades before and after integrating Ai into the Scorecard experience, month over month. Clearly displaying the impact this feature makes to Dialpad Support customers.

  • ARR since launch

    $10M+

    This feature adoption increased by over 50% while net new customers that opted to buy this add-on increased by more than 35% continuing to be a key revenue driver for Dialpad.

  • AI grades to date

    3 million+

    ...and counting. Since launching in October 2024, over 3M grades have been automated by AI.

  • AI grade increase

    4k

    September 2024

    200k

    October 2024

    550k

    November 2024

    Our data shows the significant jump in total grades before and after integrating AI into the Scorecard experience, month over month. Clearly displaying the impact this feature makes to Dialpad Support customers.

“this feature saves the manual work of answering individual questions each time and setting it up was so easy.”

Scott - InfoTrack, Australia.

All metric data has been rounded to protect confidential information.

Learnings

This project expanded my perspective on strategic thinking and prioritizing tasks. It can be daunting to face a PRD filled with numerous requirements that excite leadership.

This required a strong focus on design thinking and aligning our approach with the overall company strategy to deliver an effective solution on schedule, while minimizing the risk of losing new deals or customer renewals. Ultimately, we had to sacrifice some requirements due to time and resources, but we now have a clear path for the next phase of the project.

The insights gained from iterative learning and customer feedback were incredibly valuable, and I’m pleased that the solution I proposed was well-received by both stakeholders and the end users.

Contact

This is a preview of the final deliverables & impact—wanna see the full case study? Let's chat.

© All rights reserved 2025 - Paolo J Duarte

Contact

This is a preview of the final deliverables & impact—wanna see the full case study? Let's chat.

© All rights reserved 2025 - Paolo J Duarte

Contact

This is a preview of the final deliverables & impact—wanna see the full case study? Let's chat.

© All rights reserved 2025 - Paolo J Duarte