Model comparison guide

Compare AI models for your work

Start with a task you can check. A useful comparison shows what each model did, what needed fixing, and what it cost to reach an acceptable result.

Verified rankings are unavailable

Moyan AI does not currently have verified model rankings to publish. The previous leaderboard used AI-generated assessments, not measured benchmark results. Its scores and recommendations have been removed from this page.

This guide does not name a winning model or report live prices, licenses or performance measurements. Removing the leaderboard from this page does not stop existing background generation jobs.

Build a small, repeatable comparison

Define an acceptable result first. For an invoice extraction task, require every invoice ID and total to match the source, with missing fields marked as unknown. For a draft, define the audience, required facts and forbidden claims.
Prepare representative examples. Include ordinary inputs, an ambiguous case and a case with missing information. Use synthetic or appropriately authorized material. Keep expected answers separate from the prompt.
Keep the setup comparable. Give each candidate the same input and instructions. Record the exact model version, date, settings, enabled tools and output limits. Compare chat products separately from API configurations when their capabilities differ.
Check outputs against the original evidence. Count correct fields or passed requirements, note unsupported additions, and record failures. For code, run relevant tests in an isolated environment; plausible-looking code is not a passed test.
Repeat and record the work required. Try more than one run, keep unsuccessful attempts, and track retries and editing time. Report the sample size and variation rather than treating one good answer as typical.

Choose checks that fit the task

Research and summaries: Check whether each factual claim is supported by the supplied material. Open citations and verify that they support the claim. Record omissions and whether uncertainty is preserved.
Coding and structured data: Use expected outputs, edge cases and validation rules. Check that changes preserve existing behavior and that structured output parses without manual repairs.
Image, video and audio: Write a rubric for instruction adherence, consistency and defects before viewing results. Keep generation settings and review conditions consistent, and separate personal preference from observable errors.

Keep a comparison record

Copy these fields into your own notes for each candidate. Leave unmeasured fields as “not measured”; a missing measurement is not a zero.

Task, input set, expected result and acceptance rules
Model identifier, access method, test date, settings and tools
Prompt, raw output, number of runs and passed checks out of total checks
Failures, unsupported claims, retries and manual corrections
Observed response time and total time to an acceptable result
Actual usage cost, billing basis and any costs you did not measure
Unresolved limitations and the next case to test

Illustrative result: “8 of 10 fields matched in this synthetic invoice; two required correction.” This describes one trial. It does not establish an overall model score or predict performance on other documents.

Read evidence before choosing

For an external benchmark, look for the exact model version, dataset, test date, scoring rules, sample size and tool configuration. Check whether the evaluator publishes limitations and reproducible methods. Results from different setups may not be directly comparable.

Before adopting a model, check the provider’s current documentation for pricing, usage limits, data retention and the license for the exact version and access method. Do not infer privacy or permission to use a model commercially from an “open weights” label alone.

Choose based on your acceptance criteria and constraints. If no candidate meets them, keep a human review step or narrow the task. Save the comparison record so you can repeat it when the model or your workflow changes.

Use the AI Work Check to review an output