AI Shouldn't Get Your Language Wrong

If you read Arabic, Japanese, Chinese, English, Spanish, Portuguese or German as a native speaker, we'd like your help. AI image models now write text in all of these languages, and few people who actually read them are checking the result. We publish every image and every judgement in an open dataset, and we need native readers to tell us which ones are right.

A batch is about 30 images and takes about 10 minutes. Your answers go into the dataset under the name you choose. To join, write to [email protected] and tell us which language you read natively.

What AI gets wrong

AI now writes for the whole world: posters, signs, greeting cards, slides, even Arabic calligraphy. But image models have practised only a handful of scripts properly. In our test of five image models, every model scored above 90% on short English, German, Spanish, Portuguese and Chinese words.

Arabic is where they split. Out of 57 to 59 first attempts each, GPT Image wrote 53 words correctly, Nano Banana 55, Grok Imagine 42, Seedream 34 of 54, and Qwen-Image 8 of 59. Japanese splits less, but three of the five models still missed 7 to 14 of 59 words.

The mistakes are small. A dot sits in the wrong place, two letters join the wrong way, a kana gains a dakuten (ばくはつ becomes ばくばつ). At a glance the image looks like real writing. One dot can turn an Arabic word into a different word, and only someone who reads Arabic will see it.

Why it takes a native reader

Every image in our test is first read by an AI reviewer, zoomed in for Arabic, Chinese and Japanese. Then a person goes through the ones most likely to be misjudged, including every Arabic image. That person is one of us, and we are not native readers of most of these languages.

In November 2026, a native Arabic speaker reviewed all 291 Arabic first attempts from our test in batch AR-001. They saw the word each model was asked to write, but not the model name or our verdict. Compared with our own call (the person's, or the AI's where no person had looked):

41 of these were images where our AI reviewer and we had disagreed, so we had left them out of the accuracy table. The native reader settled 34 of them. Where we had agreed, they overturned 5 of the 243 they were sure about. Where they were sure, their call is now the verdict.

The two images below show why. Both models got the same Aztec font prompt and were asked to write شمس (sun). We passed the first image and failed the second. The native reader called it the other way round.

Heavy styling makes it harder. On a pichação wall tag, both of our checks failed this ليل (night) from Grok Imagine. The native reader read it as correct.

An AI that reads Arabic poorly and a person who does not read it natively make the same kind of mistake: they judge the shapes, not the word. A native reader judges the word. The same holds for every other language on the list.

What your review does

We test one script at a time and publish every image and every judgement in the AIFont text-rendering dataset on GitHub, also archived on Zenodo, under CC BY 4.0. Each record keeps every reviewer's call separately, so anyone can weigh them. The accuracy table is recomputed from the dataset.

Model makers rarely catch these errors themselves. A public record of which model got which word wrong, checked by people who read the language, gives them evidence they can see, can't ignore, and can fix.

Every judgement you make speaks for everyone who shares your language. It goes in under the credit you pick: your real name, a pseudonym, or anonymous.

How a review batch works

• One language per batch, about 30 images, about 10 minutes. In AR-001, the reviewer spent 11 seconds per image on average.
• For each image you see the word the model was asked to write. You answer three things: is it spelled correctly; if not, what is wrong and what the image actually says; is the text a real word.
• The model name and earlier verdicts are hidden, so your call is independent.
• Answers save as drafts as you go. Nothing counts until you sign and submit the batch.
• Several people can review the same batch separately, so answers can be checked against each other.

Batches are open now for Arabic, Japanese, Chinese, English, Spanish, Portuguese and German. Arabic has the most images waiting. Anyone can browse them at aifont.ai/native-review.

How to join

Write to [email protected] and tell us which language you read natively. We set up a reviewer account for that language, and you can take any open batch in it. We hope you'll do it as a contribution; if you would need to be paid for your time, say so in the same email and we'll talk.

Reviewers are invited for now. In the dataset, "native speaker" has to mean something, so we add each reviewer by hand.

If you don't want an account, you can still correct a single image: open an issue on the dataset repository with the record id and what the image actually says.

If you know someone who reads one of these languages, send them this page. One reviewer per language already makes a difference.