Expertise · Evaluation panels
Expert evaluation panels for AI models
Evaluation panels are a Kernavique Expertise service for AI labs, vertical-AI builders, and data and evaluation vendors: small panels of qualified practitioners who grade, rank and write rubrics against your brief. A panel is the right tool when one expert’s view is not enough and you need to know where qualified people agree and where they do not. Every panelist is vetted and conflict-checked.
Last updated
Evaluation panels
What you get
A panel matched to the question
Practitioners chosen for the domain and for a spread of practice, so the panel is not one firm’s view.
Independent first judgments
Each panelist grades or ranks on their own before seeing the others, so agreement means something.
Disagreement on the record
Where panelists differ, you see where and why, with their reasoning written down.
Rubrics written by practitioners
Panels can write or revise the rubric itself, so the criteria reflect how the profession judges the work.
How it runs
- BriefThe task, the domain and what each item must be judged on.
- Form the panelWe select and vet the practitioners and run conflict checks against your brief.
- CalibrateThe panel works a sample together against the rubric before the main work starts.
- Grade and reportIndependent judgments, then a record of agreement and disagreement with reasons.
When it fits
- Your model’s answers in a professional domain are judged by one reviewer, and you are not sure that reviewer is right.
- You need a rubric for legal, tax or clinical answers and want practitioners to write it.
- You are comparing model versions and need rankings that hold up when questioned.
Who it is for
- AI labs evaluating models in professional domains
- Vertical-AI builders in legal, tax, health and finance
- Data and evaluation vendors
Questions buyers ask
- How big is a panel?
- Large enough to show where qualified people disagree, small enough to stay consistent. We size it to the question at the brief.
- Can panels write rubrics as well as grade?
- Yes. Practitioners can draft a rubric from scratch, revise yours, or grade against it and tell you where it breaks.
- What happens when panelists disagree?
- You see it. Each disagreement is recorded with the panelists’ reasoning, so you can tell a hard item from a weak rubric or a weak answer.
- How do you handle confidentiality?
- Panelists are conflict-checked against your brief, and confidentiality terms are agreed before any material is shared. Each panel works within the professional-conduct rules that apply to its members.
Related services
- ExpertiseExpert sourcingNamed, vetted legal, finance, tax and clinical professionals for a defined evaluation or data program, introduced before they start.
- ExpertiseManaged expert teamsA team of vetted professionals with a lead, delivering per deliverable rather than per hour, managed through one relationship.
- ExpertiseProgram setupThe first stage of an Expertise engagement: we set the pool, vetting, deliverable units and conduct rules, then source the experts who work to them.