🌠🌧️🫂

A language model that actually speaks emoji.

Tell it anything and it replies the way a friend would in a chat: with one to three emoji, and nothing else. It doesn’t translate your words into emoji. It answers you.

Download the model Read the code

Phrasebook

Real replies from the released model. It samples, so the same message can get a different answer. Ask again to see another one.

How it talks

🎉🥳: One to three emoji. Every reply, whatever you send: a question, a rant, a request for code, a whole essay.
🙊🙊🙊: One emoji is said three times. When a single emoji is the whole answer, the reply wrapper repeats it for emphasis.
🤐: It never switches to words. Asking it to, or telling it to ignore its instructions, gets an emoji too.
💬: One message at a time. It answers your latest message and doesn’t remember earlier ones.

How well it works

Measured on 666 prompts: held-out chat messages plus 140 hand-written ones that try hard to get words out of it.

Replies that are 1–3 emoji only	666 / 666
On the first try, before any resample	98.9%
The same prompts on base Qwen3-1.7B	0%
Different emoji in its training data	1,915
Reply length, 1 / 2 / 3 emoji	19 / 57 / 23%
Preferred by Claude Opus over its own starting point	53.0%

Run it

Download emoji-qwen3-1.7b-q8_0.gguf (1.8 GB) from the latest release into the repo’s deploy folder.

With Ollama

cd deploy
ollama create emoji-bot -f Modelfile
ollama run emoji-bot "my flight got cancelled"

With Python on a GPU

pip install -r requirements.txt
python emoji_chat.py

Ollama and llama.cpp return the model’s raw reply. To get the full behaviour, check it with is_valid_reply and repeat a single emoji with finalize_reply, both in emoji_spec.py. For llama.cpp, deploy/emoji.gbnf restricts decoding to emoji.

How it was built

Conversations. Claude Opus wrote about 11,600 varied messages in Chinese and English, across 40 topics and 6 writing styles, and replied to each with emoji.
Fine-tuning. LoRA on Qwen3-1.7B with thinking turned off, trained only on the replies.
A bigger vocabulary. For 1,397 emoji that were rare or missing, Opus wrote conversations where that emoji is the right reply. No emoji may make up more than 2% of the data.
Preferences. The model drafted four replies per message, Opus scored them, and DPO taught the model to prefer the better one, comparing only replies of the same length.
Packaging. Merged, wrapped with a format check and resample, and exported to GGUF.

What didn’t work

Reinforcement learning against a learned reward model collapsed in under 200 steps: the model discovered that 🥹🫂 scored well on almost anything. The root cause was the judge. Opus prefers 🫂, and it also rates three-emoji replies above one-emoji ones, so both habits leaked into anything trained on its scores. Balancing the preference pairs per emoji and per length fixed it. The README has the details.