GPT-6.1 Terra vs Claude & Gemini: Design Test

When you ask FontVibe for a design, a language model does the designer's job before any image is drawn. It reads the brief, proposes three design directions, and then writes the instructions the image model renders. GPT-6.1 Terra came out on October 26, 2026, so I tested whether it does that job better than GPT-5.6 Terra, the model FontVibe uses now, and against six other language models from OpenAI, Anthropic and Google.

Every model got the same 12 briefs, three times each. The image model was the same for every design: GPT Image 2.5 Sunburst at 2K and Medium quality, the model that came second in my Qwen Image 3.0 test and my Grok Imagine 2.0 test. The only thing that changed from one design to the next was which language model played the designer. 287 designs were scored blind, with the model names hidden.

The results

Scores are out of 10 and averaged over every design that finished. Claude Opus 5.5 is not ranked: it finished designs on only 8 of the 12 briefs, mostly the easier ones, so its average is not comparable with the others (how it compares on the same briefs). The 95% confidence intervals come from resampling the briefs, because the three runs of a brief share one brief and one judging pass. Two models count as different only when the interval of their brief-by-brief difference stays clear of zero; all 28 comparisons are in the appendix. Cost per design is the median language-model cost of one design (the proposal plus the image instructions) at each provider's public price on October 30, 2026; the image itself costs the same for every model and is not included. Every score per job is in the appendix.

Four results stand out.

Which language model plays the designer makes a difference, but on quality alone most models could not be told apart. With the same image model and settings, average scores ranged from 7.08 to 7.97. Of the 28 pairs of models, 5 differ clearly when compared brief by brief: GPT-6.1 Terra scored lower than GPT-5.6 Terra, GPT-6 Luna and Claude Opus 5.5, and GPT-6.1 Sol lower than GPT-6 Luna and Claude Opus 5.5. The other 23 pairs are ties in this test.

Put quality, cost, speed and reliability together and the gaps open up. Models in the same quality group differed more than 30-fold in cost per design (GPT-6 Luna $0.005, Claude Sonnet 5.5 $0.18). The time to propose and write the image instructions differed about twofold (about 41 seconds for GPT-5.6 Terra and Claude Sonnet 5.5, about 87 for GPT-6.1 Sol). And the models failed for different reasons (details). Taken together, several models lose on at least one count without gaining on another: GPT-6.1 Sol cost the same as GPT-5.6 Terra and could not be told apart on quality, but took twice as long; Gemini 3.8 Flash cost eight times as much as GPT-6 Luna, could not be told apart on quality and hit 8 provider errors; GPT-6.1 Terra cost more than GPT-6 Luna and scored clearly lower.

No model wins everywhere; each suits different work. Claude Opus 5.5 had the highest average, 7.97, but from only 18 designs on 8 briefs. In the main run its proposals, far longer than any other model's, were cut off 17 times by the 6,600-token limit FontVibe set for its own production use, and one more run failed when its image instructions came back broken. Rerun with the limit doubled, it finished every time; over all 36 designs it scored clearly higher than GPT-5.6 Terra, Claude Sonnet 5.5, Gemini 3.8 Flash and both GPT-6.1 models, and could not be told apart from GPT-6 Luna or Gemini 4 Argon (details). It is also the slowest and the most expensive: a median of $0.41 per design, or $0.72 per finished design once failed runs are counted. It suits complex design work that is not in a hurry, where its edge is worth the price. GPT-6 Luna was in the top quality group, finished every run and cost the least, which makes it the best value and suits high volume. GPT-5.6 Terra was fast and balanced, which suits real-time use where someone is waiting for the result.

Invented text is a measurable defect, and it varied 14-fold between models. GPT-6.1 Terra added an average of 0.17 lines per design that the brief never gave; Gemini 3.8 Flash added 2.36: founding years, slogans, service lists and fake figures. Invented text was the most common reason a design lost points on information, so whichever model you use, the pipeline needs a copy check.

GPT-6.1 Terra was the starting point of this test, so a word on it. In the same pipeline, with the same prompts, it was more conservative than GPT-5.6 Terra. Compared brief by brief, it scored 0.50 points lower (95% CI −0.81 to −0.21) and lower on 9 of the 12 briefs. It lost on lettering and finish (−0.93) and on appeal (−1.38, lower on all 12 briefs); its legibility was slightly higher (+0.47), within the margin of error. Its designs had the least invented text and were the plainest; 17 of its 36 only partly matched the mood the brief asked for. This describes how it reads a brief, chooses a direction and writes image instructions, not how it does on other tasks.

Cost, speed and reliability

Design quality is only part of choosing a model. The same runs also measured what each design costs, how long it takes and how often it fails.

Times are the language model's own time for each step, over the runs that finished. Rendering the image took a median of 20 seconds on top of that, the same for every model. Costs use each provider's public price on October 30, 2026, cover both language-model steps and leave out the image, which costs the same whichever model designed it. "Spend per 1,000 finished designs" counts everything spent, including failed runs and reruns, divided by the designs that finished. "Finished on the first try" counts runs that produced an image without a rerun; the two GPT-5.6 Terra and two GPT-6.1 Sol runs that FontVibe's front desk stopped to ask a question are counted as not finishing on the first try, although the model was not at fault. Internal retries are the times FontVibe's pipeline retried the same model inside one request after a failed attempt.

Cost. GPT-6 Luna was the cheapest by far: about $5 per 1,000 finished designs, against $93 for GPT-5.6 Terra and $721 for Claude Opus 5.5, whose failed runs cost almost as much as its finished ones. GPT-6.1 Terra cost less than half as much as GPT-5.6 Terra. Claude Opus 5.5 was the most expensive, partly because it wrote the longest proposals: its median proposal reached the 6,600-token limit.

Speed. Claude Sonnet 5.5 proposed fastest, with a median of 27 seconds, and GPT-5.6 Terra was close at 29 seconds. GPT-6.1 Sol and Claude Opus 5.5 took over a minute. Gemini 4 Argon's median was 40 seconds, but its slowest 10% took 73 seconds or more, the widest spread of any model.

Reliability. GPT-6.1 Terra, GPT-6 Luna, Claude Sonnet 5.5 and Gemini 4 Argon finished all 36 runs on the first try. Gemini 3.8 Flash's own output never failed, but Google's service returned 8 errors (rate limits and timeouts) across 7 of its runs, all of which finished on a rerun. Claude Opus 5.5 failed 18 times: 17 proposals cut off at the output limit and one set of image instructions that came back broken. Most of its 19 internal retries were second attempts after a cut-off proposal that were cut off again.

How I tested

The pipeline

This test measures one job: being the designer. In FontVibe's design mode, a request goes through three steps:

1. Propose. A language model reads the brief and proposes three design directions, each with a title, the exact text to print, colours, a layout and a choice of lettering style.
2. Write the image instructions. For the first direction, a language model writes the full instructions for the image model.
3. Render. An image model draws it.

The tested model did steps 1 and 2. Step 3 was always GPT Image 2.5 Sunburst at 2K and Medium quality. Every request went through FontVibe's production pipeline, with the same system prompts, the same output format and the same reasoning-effort setting (medium) for every model. No model got its own prompt or its own settings. The first direction was always the one rendered; I did not pick directions.

The eight models

The 12 briefs, in four layers

A designer can go wrong in four different ways, so the briefs come in four layers of three, one layer for each way.

The briefs cover six languages (English, Chinese, Spanish, Portuguese, German and Japanese), the six languages of FontVibe's interface, and every one asks for text on the design. Eight of the 12 briefs were first used in an internal test of three OpenAI models in September 2026. The company in the hiring poster and every name and address in the briefs are made up. The full brief for each job is printed in its section below.

Runs and failures

Each model ran each brief three times: 36 designs per model, 288 in all. When a design failed, the cause decided what happened next:

• The model's own output failed (cut off at the production token limit, or an empty or broken proposal): counted as a failure and not rerun. 19 designs failed this way: 18 from Claude Opus 5.5 (17 proposals cut off at the limit, 1 set of broken image instructions) and 1 empty proposal from GPT-5.6 Terra.
• The provider's service failed (rate limits, server errors, timeouts): rerun. This happened 8 times, all with Gemini 3.8 Flash.
• FontVibe's own front desk asked a question first. Before a request reaches the designer, a separate fixed model checks whether anything is missing. For the nightclub brief it sometimes asked "almost invisible or readable from 50 metres?" instead of passing the brief on. That is not the tested model's doing, so those 4 runs were rerun, and the rerun's design is the one scored. Every attempt is listed in the open dataset.

All 287 designs (269 from the main runs and 18 from the Opus reruns below) were generated on October 30, 2026, on FontVibe's production site with an official account, and are public on FontVibe.

Scoring

Each design was scored against FontVibe's benchmark scoring standard, written before the results were tallied. It builds on the rubric from my test of seven AI design tools and the two image-model tests. The judge plays the client who wrote the brief and judges in this order: is it the thing I asked for; can I use it without fixing anything; is it the feeling I wanted; are the details well made; does it make me say "wow" at first sight.

The overall score is calculated, not judged: 0.20 × legibility + 0.35 × lettering + 0.25 × information and layout + 0.20 × appeal.

One rule decided many scores, so here it is in full. The question for every line of text was: would the client have to change this before using it, and did the client ever say it? Restating what the brief already says ("COFFEE ROASTERS" for a coffee roastery) and ordinary greetings ("Happy birthday") were accepted. Facts, figures, promises and slogans that the client never gave were each a deduction, because the client would either have to delete them or publish something untrue without noticing. Placeholders printed as is, such as {{fecha}}, and changes to how the client wrote something, such as "7 de la noche" becoming "7:00 p. m.", were deductions too.

Who scored. The scores come from AI judges running Claude Opus 5.5, one judge per group of three briefs, each looking at every original image in a brief before scoring any of them. The images had random IDs, and the judges did not know which model made which design. One of the eight tested models, Claude Opus 5.5, is the same model as the judges. Keep that in mind when reading its scores: whether the judges favour Opus's own designs was not tested. The blind pick by a person is the only human check in this test.

Job by job

Each grid shows the first design that finished for each model, so no design was picked by hand. The caption gives that design's score and the model's average over its three runs of the job.

Layer 1: Everyday requests

Job 1: Coffee roastery wordmark

A logo-style title for my new coffee roastery called EMBER & OAK

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Claude Opus 5.5 scored highest, 8.38 on average: it burned the name into wood, with the edges of each letter scorched. GPT-5.6 Terra (8.13) and GPT-6 Luna (8.05) were close behind. GPT-6.1 Terra (6.62) and GPT-6.1 Sol (6.43) set the name in clean but ordinary type, which the judge marked as only partly matching the brief's "ember and oak". Claude Sonnet 5.5's design invented a founding year and a city, "EST. 2019" and "AUSTIN", which the client never gave.

Job 2: Dinosaur birthday poster

给我做一张生日海报，寿星叫小雨，今年 8 岁，喜欢恐龙 (A birthday poster: the birthday girl is Xiaoyu, she is turning 8 and she loves dinosaurs.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

The OpenAI models did best here. GPT-6.1 Sol scored 8.73, and GPT-6.1 Terra 8.43, its best job, with a dinosaur tail growing out of the character 雨 in Xiaoyu's name. Gemini 4 Argon came last, 6.48: a photorealistic jungle and a "Jurassic Expedition · Level 8" party that the client never mentioned, which the judge marked as only partly matching a birthday for an eight-year-old. For balloon lettering like GPT-6.1 Sol's, see the bubble letter generator. Gemini 3.8 Flash's first design invented a date, a time and a venue for a party that does not exist, and the date had already passed.

Job 3: Graduation party poster

Un cartel para la fiesta de graduación de secundaria, dice PROMOCIÓN 2026 (A poster for a high-school graduation party that says PROMOCIÓN 2026.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Every model wrote PROMOCIÓN with its accent. Gemini 4 Argon (8.40) and Gemini 3.8 Flash (8.28) scored highest with gold lettering and laurels, though one of Argon's designs added a formal dress code, and two designs in this job showed champagne glasses at a school party. This brief had no date or place, and many designs left an empty area at the bottom where those details would go, or printed placeholders such as {{fecha}} there.

Layer 2: Getting the information right

Job 4: YouTube thumbnail

YouTube thumbnail text for a video about losing 30 pounds, title THE SHIFT

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Claude Sonnet 5.5 scored highest, 8.20, contrasting a clay "THE" with a mirror-finish "SHIFT" to show the change. GPT-6 Luna scored 7.97. Claude Opus 5.5 came last, 6.53: its first design offset the halves of each letter, which reads as a printing error. Gemini 4 Argon and Gemini 3.8 Flash filled the background with gym slogans the client never wrote.

Job 5: Hiring poster with seven facts

招聘海报：标题「一起把账算明白」，副标题「马来西亚 AI 公司，招两位财务」，要写上公司名 Northwind Labs Sdn Bhd、月薪 RM6000-9000、地点吉隆坡 Bangsar、投递邮箱 jobs@northwindlabs.my、截止 10 月 15 日。标题最大，邮箱最小。 (A hiring poster with a title, a subtitle, the company name, the salary, the location, the email address and the deadline. Title largest, email smallest.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

This job separated the models most clearly. Claude Sonnet 5.5 scored 7.92 with a ledger-paper design and a red stamp that kept all seven facts and the size order. Gemini 3.8 Flash scored lowest, 6.12: its first design filled the poster with English slogans, a logo and dashboards of made-up figures, and changed the client's subtitle. Claude Opus 5.5 failed all three runs at the production token limit; with the limit doubled it made the best design of the job, 8.95, with a similar ink-on-paper title, a seal and a fountain pen.

Job 6: Coffee cup sleeve

Uma capa para o copo de café para viagem da minha cafeteria em Belo Horizonte, a Grão de Ouro. Tem que ter o slogan "Café de verdade, sem pressa" e "Desde 1998". (A sleeve for my café's takeaway cups, with the name, the slogan and "since 1998".)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Claude Opus 5.5 scored 8.97 on the two runs that finished, with gold-foil serif lettering and a coffee-bean O. GPT-5.6 Terra (8.30) and Claude Sonnet 5.5 (8.38) followed. Gemini 3.8 Flash's first design drew the tilde in GRÃO wrongly, so the name reads closer to "GRÁO". Gemini 4 Argon added claims such as "100% ARÁBICA" that the café never made. For gold-foil lettering on its own, try the gold text generator.

Layer 3: Design judgement

Job 7: A nightclub wordmark with contradictory requirements

I want an extremely minimal, quiet, almost invisible wordmark for my nightclub — but it also has to scream across a dark street from 50 metres away and feel expensive. The name is VOID.

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

The strongest designs resolved the contradiction with backlighting: black letters on a black wall that disappear up close and glow at night. Gemini 4 Argon (8.13), Gemini 3.8 Flash (8.05) and Claude Opus 5.5 (7.88) did this best. GPT-6.1 Terra came last, 5.48: its first design is flat white bold type on black, which shouts but is neither quiet nor expensive. GPT-5.6 Terra scored 5.90. For lit signage, the storefront sign maker starts from glowing shop signs.

Job 8: Scientific journal cover

Cover art for a peer-reviewed paper on cryo-EM structural determination of amyloid fibrils. Title: CONFORMATIONAL POLYMORPHISM. It has to look like it belongs on the cover of Nature, not like a startup pitch deck.

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

GPT-6.1 Sol (7.65), GPT-5.6 Terra (7.47) and GPT-6 Luna (7.43) scored highest with restrained serif titles and glassy fibrils. Several designs drew the fibrils as a ladder-like double helix, which looks like DNA, a scientific error in a cover about amyloid. Claude Sonnet 5.5 invented a volume, issue and date. Claude Opus 5.5 failed all three runs at the production limit.

Job 9: Tattoo studio sign for two audiences

A shop sign for my tattoo studio called INKWELL. It should still feel welcoming to my grandmother's generation.

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Invented text was everywhere in this job: taglines, service lists, "Walk-ins welcome", and two made-up founding years. Claude Opus 5.5 scored highest, 8.25, and GPT-6 Luna 7.95. Gemini 4 Argon came last, 6.37: its first design used a decorative capital I that reads as a J, so the name looks like "JNKWELL". The designs that printed only the name and "Tattoo Studio" were accurate but often plain.

Layer 4: Language and culture

Job 10: Japanese shaved-ice stall sign

夏祭りの屋台の看板を作ってください。「かき氷」と書いて、昭和レトロな駄菓子屋の雰囲気で。提灯の灯りみたいな温かさがほしい。 (A summer-festival stall sign that says かき氷, with a Showa-retro candy-shop mood and the warmth of lantern light.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Every model wrote かき氷 correctly. Claude Opus 5.5 (8.55), Gemini 4 Argon (8.38) and GPT-5.6 Terra (8.27) scored highest with dyed-cloth banners and enamel signs. Most designs added a short line such as "ひんやり" (nice and cold) or a list of syrup flavours, which counted against them. Two designs printed {{price}} and one invented a price.

Job 11: German bakery opening sign

Ein Schild für die Neueröffnung unserer Bäckerei „Großmanns Backstube" in Leipzig. Öffnungszeiten: Mo–Sa 6:30–18 Uhr, So 7–12 Uhr. Gemütlich, aber nicht kitschig. (An opening sign for our bakery with its name and opening hours. Cosy, but not kitschy.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Every model spelled Großmanns with its ß. Claude Sonnet 5.5 scored highest, 8.03, with weathered, outlined lettering on wood. GPT-6.1 Terra (6.32) and GPT-6.1 Sol (6.62) set the sign in flat digital type with no warmth. All three of Claude Sonnet 5.5's designs added slogans the bakery never wrote, such as "Frisch gebacken seit dem ersten Morgen" (freshly baked since the first morning), and so did Gemini 3.8 Flash's. Claude Opus 5.5 failed all three runs at the production limit.

Job 12: Día de Muertos invitation

Invitación para la reunión familiar de Día de Muertos en casa de la abuela Rosa. Domingo 1 de noviembre de 2026, 7 de la noche, Calle Hidalgo 45, Oaxaca. Algo cálido y respetuoso, no de disfraz. (An invitation to the family Day of the Dead gathering at grandma Rosa's house, with the date, time and address. Warm and respectful, not a costume party.)

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

Use as Reference

No design turned the gathering into a Halloween costume party, and every one kept the date and address. Nearly all of them changed "7 de la noche" to "7:00 p. m.". GPT-5.6 Terra scored highest, 8.22. A subtler problem showed up in several designs: a portrait of an elderly woman placed at the centre of the altar, right next to "abuela Rosa", which reads as if Rosa herself had died. Claude Opus 5.5 failed all three runs at the production limit.

Claude Opus 5.5 with a bigger budget

FontVibe limits a design proposal to 6,600 output tokens, thinking included. Claude Opus 5.5 wrote longer proposals than any other model. Its proposals reached that limit in 19 of its 36 runs: 17 were cut off and failed, and 2 still finished. One more run failed for a different reason: the proposal was under the limit, but the image instructions in the second step came back as broken output. Among the other models, only one GPT-6 Luna proposal reached the limit, and it finished.

To see what Opus does when it is not cut off, I reran only its 18 failed runs with both output limits doubled and nothing else changed. All 18 finished. 13 of them used more than 6,600 tokens for the proposal (median 7,286, longest 8,890), and the median proposal took 76 seconds. The 18 designs scored 8.10, and all 36 Opus designs together scored 8.04 (95% CI 7.70–8.30). Compared brief by brief with the other models on all 12 briefs, those 36 designs scored clearly higher than GPT-5.6 Terra (+0.46), Claude Sonnet 5.5 (+0.60), GPT-6.1 Sol (+0.68), Gemini 3.8 Flash (+0.71) and GPT-6.1 Terra (+0.96), and could not be told apart from GPT-6 Luna (+0.35, 95% CI −0.06 to +0.71) or Gemini 4 Argon (+0.48, −0.05 to +0.98).

These reruns are not in the main ranking, because no other model got a larger budget. The 6,600-token limit is one FontVibe set for its own production use, not a limit of the model. The reruns show that with enough budget Opus finishes every time and its quality holds up, at the price of higher cost and a longer wait. Its main-table average comes from 18 designs on 8 briefs, and most of the briefs it failed were the layer 2 to 4 briefs with more text to plan, so that average leans on the easier briefs; that is why it is not ranked there.

A blind pick by someone who was not the client

FontVibe's founder looked at one design per model for each job, in random order with the names hidden, and picked the one he would use. His 12 picks: Gemini 3.8 Flash 4, Claude Sonnet 5.5 3, GPT-5.6 Terra 2, Gemini 4 Argon 2 and Claude Opus 5.5 1. He never picked GPT-6 Luna or either GPT-6.1 model.

Among the eight designs he saw for each job, his pick was the judges' top-scoring one in 2 of 12 jobs; it was never the top-scoring design of the job overall. The difference has one clear source: 8 of his 12 picks contain text the brief never gave. In lettering and appeal his picks scored 7 to 9 almost every time, which shows the judges also saw those designs as visually strong; the picks lost their points on information.

He reads the gap this way: he was not the client for any of these briefs, so he picked on first impression. A client who needs the design looks at the feeling first, then at the details, and treats wrong or invented text as a hard error. That is why the rubric scores carry the ranking, and this pick is a note. It also shows the risk for anyone who picks on first impression: the design that grabs your attention first is often the one that added text you did not write.

Each model, briefly

• Claude Opus 5.5: the best lettering and appeal, and few invented lines (0.22 per design). The longest proposals and the longest thinking, so it needs a wider output limit than FontVibe's production one; the slowest and by far the most expensive. Suits complex work that is not in a hurry.
• GPT-6 Luna: the best value: in the top quality group, finished every run, high legibility, few invented lines, and the cheapest by a wide margin. Its lettering is good rather than memorable.
• GPT-5.6 Terra: fast and balanced. Strong in layer 4 and in most everyday briefs, weakest in layer 3, where it often answered a hard brief with plain type.
• Gemini 4 Argon: some of the richest materials in the test, and 1.56 invented lines per design.
• Claude Sonnet 5.5: the fastest proposals and strong lettering, and the most placeholders printed as is (5 designs).
• GPT-6.1 Sol: the most legible designs and few invented lines, but plain lettering, and slow at both steps (about 87 seconds together).
• Gemini 3.8 Flash: visually strong, the most invented text (2.36 lines per design) and the only model with provider errors (8).
• GPT-6.1 Terra: the cleanest text and the plainest designs.

Limits of this test

• The judges run on Claude Opus 5.5, which is also one of the tested models. The images were anonymous, but the judges and one contestant share a model, and whether they favour its designs was not tested.
• Each brief was scored once, by one judge in one pass. There was no repeat scoring and no second judge.
• Claude Opus 5.5's main score comes from 18 designs on 8 briefs, mostly the easier ones.
• One image model rendered every design. A different image model could change how much the designer's instructions matter.
• Each model ran each brief three times, and each run rendered only the first of its three proposed directions. With 12 briefs, most differences between models are too small to tell apart; 28 pairs were compared with no correction for multiple comparisons.
• The scoring standard covers designs with text. Images without text were not tested.
• All models used FontVibe's production prompts and limits. Another app's prompts or limits could favour different models.
• The results describe these model versions on October 30, 2026.

Every design is public on FontVibe and every score has a written reason. The briefs, all proposals, image instructions, scores and reasons are in the open dataset AI Design Benchmark (CC BY 4.0), with every attempt and a script that recomputes every table in this post. FontVibe's production prompts are not published; researchers who need them can ask at hello@fontvibe.ai.

Appendix: every score

Average overall score per model and job over the runs that finished (number of finished runs in brackets when fewer than 3).

All pairwise comparisons

Each pair is compared on the briefs where both models finished designs: the average difference of their per-brief scores (first model minus second), with a 95% confidence interval. When the interval includes zero, the two cannot be told apart in this test.

The scoring standard

The judges' instructions, in brief: you are the client who wrote the brief and will use this design. Judge in this order: whether it is the thing you asked for; whether every piece of text can be used as it is; whether it has the feeling you wanted, stated or reasonably implied; how well the lettering, materials, textures, lines and colours are made; and whether it impresses at first sight. Each of the four scores covers its own question, and a problem is deducted only once, under the score it belongs to.