A new benchmark from Artificial Analysis offers a more practical way to compare AI models for a specialized task: building interactive 3D scenes that run in a web browser. Its AA-3D evaluation, announced on October 12, 2026, compares 21 models across 55 scene-building tasks and seven domains, from games and machines to landmarks, space, and natural phenomena.
The early result is not that one model dominates every use case. Claude Sonnet 5.5 leads the initial leaderboard, but its confidence interval overlaps with those of several other top models. Cost also changes the picture: a model that ranks slightly lower may be more attractive when repeated scene generation is part of the workflow.
This is a useful comparison for developers choosing a model for interactive prototypes, educational visualizations, browser-based demos, or small 3D experiences. It is not a general ranking of which AI assistant is best at every kind of coding.
What AA-3D actually tests
Each model receives the same written brief and builds a single web page using three.js in an offline sandbox. The agent can write code, render its work, inspect screenshots, test the page, and revise it. The environment includes tools such as a headless browser, Python, Blender, and optional libraries and assets.
The evaluation uses 55 tasks across seven domains:
- Landmarks and real places
- Machines
- Natural phenomena
- Living bodies
- Games, sports, and piloting
- Space
- Stories and fantasy
A scene must pass automated checks and a review before it is eligible for voting. Human evaluators then compare two scenes created for the same task without seeing which models produced them. They interact with both and choose the one they prefer. Artificial Analysis turns these blind pairwise votes into an Elo-style rating.
That makes AA-3D different from a coding benchmark that only checks whether code passes tests. It measures the quality of the resulting interactive experience, including whether the scene is compelling and usable to the people evaluating it.
The measured leaderboard
The launch results, dated October 12, 2026, put these models near the top:
| Model | AA-3D Elo | What the result means |
|---|---|---|
| Claude Sonnet 5.5 | 1194 ± 34 | Highest point estimate in the launch ranking |
| Claude Opus 5.5 | 1190 ± 35 | Effectively level with the leader within the reported uncertainty |
| GPT-6 Astra | 1165 ± 34 | Also overlaps with the leading models’ confidence intervals |
| Claude Fable 5.1 | 1131 ± 34 | In the next score band |
| GPT-6 Sol | 1123 ± 34 | Close to Claude Fable 5.1 in the reported intervals |
| GPT-6.1 Sol | 1114 ± 33 | Another model in the same broad score band |
| Muse Spark 1.3 | 1035 ± 32 | Seventh in the launch table |
| GPT-6 Luna | 1000 | The leaderboard’s fixed reference point |
| Kimi K3 | 982 ± 33 | Highest-ranked open-weights model in this launch table |
| MiMo-V2.6-Pro | 977 ± 32 | Close to Kimi K3 in the reported score range |
The plus/minus figures are the published 95% confidence-interval half-widths, not ordinary run-to-run score variation. Because the intervals for the top four overlap, the leaderboard does not establish a statistically clear winner among those models. Sonnet 5.5 has the highest point estimate, but treating it as definitively better than Opus 5.5 or GPT-6 Astra would overstate the evidence.
The same caution applies lower down the table. Kimi K3 and MiMo-V2.6-Pro are separated by only five Elo points, far less than their reported uncertainty. Their positions should not be interpreted as a decisive quality difference.
Quality is only half of the decision: cost matters
Artificial Analysis also reports the estimated model-token cost of generating a scene. In the launch data, GPT-6 Sol averaged about $3.46 per scene and was the least expensive among the five highest-ranked models. Claude Sonnet 5.5 cost about 4.4 times as much per scene in that comparison.
GPT-6 Luna is another notable trade-off: the benchmark reports an average cost of around $0.20 per scene, while it sits eighth on the Elo table. That does not make Luna the best option for every workflow, but it shows why the highest quality score and the lowest cost are different selection criteria.
These figures should be read as benchmark-specific estimates, not guaranteed prices for every user. They depend on the evaluated run, token usage, and the pricing inputs used by the benchmark. A real project may also incur engineering, hosting, asset, and review costs that are not captured by a model-token estimate.
For a developer who needs the strongest possible result for a showcase scene, the top-ranked models are sensible candidates to test first. For a product that generates hundreds of scenes, the cheaper models may be more practical—provided their output quality meets the project’s requirements.
What appears to separate stronger models?
The benchmark’s analysis suggests that the agent’s workflow matters, not just the amount of code it writes. Higher-ranked models tended to inspect their own rendered output earlier and begin browser testing sooner. Across models, earlier visual inspection was strongly associated with higher Elo, while simply producing more output tokens did not show a clear relationship with score.
That is a correlation, not proof that early screenshot inspection alone causes better results. Stronger models may also be better at deciding what to inspect, identifying problems, and making useful revisions. Still, the finding offers a practical lesson for anyone building an AI coding workflow: a model that can check its own output and iterate may be more valuable than one that generates a large amount of code without validating the result.
What AA-3D does—and does not—prove
AA-3D is more informative than a marketing claim because it publishes a defined task set, a ranking method, confidence intervals, and details about the environment. Artificial Analysis says every model receives the same briefs, tools, and limits, and that the evaluation is run independently. Its methodology also describes eligibility checks and quality checks for voting sessions.
There are important limits:
- Human preference is not the same as technical correctness. A scene that wins a preference vote may be more engaging without being the most accurate, accessible, maintainable, or suitable for production.
- The scope is narrow by design. The benchmark focuses on interactive browser-based 3D scenes. It does not rank models for ordinary web development, backend engineering, image generation, or all other coding tasks.
- The ranking is relative. Elo indicates how often scenes are preferred against other scenes in this benchmark; it is not a percentage score or a guarantee of success on a new brief.
- Confidence intervals matter. The leading models’ intervals overlap, so small differences in point estimates should not be marketed as a decisive victory.
- Cost is workload-dependent. The reported cost per scene reflects the evaluated agent runs and model pricing assumptions, not every cost a team will face in production.
The benchmark is produced by Artificial Analysis, the organization publishing the results. Its detailed methodology is available publicly, which helps readers inspect the approach, but the leaderboard should still be treated as one independent evaluation rather than a universal verdict.
Which model should you choose?
Use the AA-3D results as a shortlist, then test the models against your own scene briefs.
- For quality-first prototyping: start by comparing Claude Sonnet 5.5, Claude Opus 5.5, and GPT-6 Astra. The launch data does not show a clear statistical winner among the top group.
- For balancing quality and cost: include GPT-6 Sol in a hands-on test. Its reported cost is substantially lower than Sonnet 5.5’s while its score remains in the upper group.
- For very low-cost experiments: consider GPT-6 Luna, but verify that its output meets your quality threshold before using it at scale.
- For open-weights options: Kimi K3 and MiMo-V2.6-Pro are the strongest open-weights entries in the launch table. Test them directly if deployment control or hosting flexibility matters to you.
A practical evaluation should use several representative briefs, the same acceptance criteria, and a record of both model cost and the amount of manual repair required. Include load time, interaction behavior, browser errors, accessibility, and whether the scene actually communicates the intended idea. Those checks may change which model is best for your project.
The bottom line
AA-3D provides fresh, task-specific evidence that model choice matters when generating interactive 3D scenes for the browser. Claude Sonnet 5.5 leads the initial Elo table, but the top four are statistically tied within their reported confidence intervals. GPT-6 Sol looks comparatively cost-efficient among the top five, while GPT-6 Luna offers a much cheaper starting point with a lower ranking.
The best decision is not simply to select the first row. Choose a model based on the quality your task requires, the cost of repeated generation, and how much repair the output needs. AA-3D is a useful starting point for that decision—not proof that any one model is best at all AI coding.
Sources
- Artificial Analysis: Announcing AA-3D — launch results, cost comparisons, and the analysis of model behavior.
- AA-3D leaderboard and evaluation details — current ranking, score intervals, task coverage, and cost metrics.
- AA-3D benchmarking methodology — how scenes are generated, checked, compared, and scored.
- Stirrup, the open-source agent harness — the tool environment used for the benchmark.
Scores and costs in this article reflect the AA-3D launch data published on October 12, 2026. Check the live leaderboard for updates before making a purchasing or deployment decision.