World's largest software provider and second-largest cloud provider 

Summary

Centific developed a method to systematically measure and rank harm within multiple versions of customers’ foundational models while tracking model iterations and parameters.

Challenge

To achieve organizational goals, the client’s leading foundational model development team needed a method of measuring the level of harm within different versions of the same foundational model.

Solution

Centific:

• Developed customized interfaces to facilitate the ranking of responses for harm grading, while tracking model iteration and parameters. 

• Created customized certification and training process for human experts to fully understand the definitions of harm. 
• Fine-tuned model versions based on the harm-grading teams’ feedback. 

• Created and curated a set of taxonomies aligned with the client's safety and governance policies. 

Results

This project resulted in substantial improvements to the safety and reliability of the client’s generative AI.