Our family group chat is a river of emoji. My contributions are mostly words; the younger generation answers in little pictures. What none of us think about while typing is that each of those little pictures costs several times the storage of a letter.
For decades of English-language computing, one character really did equal one byte, and it felt like a law of nature. But paste in a paragraph of Korean, a few emoji, or even a curly quotation mark, and the byte count quietly outruns the character count. The reason is encoding: the rulebook that maps human characters onto the numbers computers actually store.
The one-byte era and its garbled aftermath
Early computers standardized on ASCII, which assigns numbers 0 through 127 to English letters, digits, and punctuation. A byte holds values up to 255, so one character fit in one byte with room left over.
That spare room became the problem. Different regions filled the upper 128 slots with their own accented letters and symbols, producing dozens of incompatible code pages. Open a file with the wrong one and é turns into gibberish — anyone who used computers in the nineties remembers that particular flavor of garbage text.
UTF-8: one system, variable prices
Unicode ended the chaos by giving every character in every writing system its own number, called a code point — well over 100,000 are assigned. UTF-8, the encoding that now dominates the web, stores them at variable length:
- Basic English letters, digits, punctuation: 1 byte
- Most accented Latin letters, Greek, Cyrillic, Arabic, Hebrew: 2 bytes
- Chinese, Japanese, Korean characters: usually 3 bytes
- Emoji and rarer symbols: typically 4 bytes
The quietly brilliant part is backward compatibility: a pure-ASCII file is already valid UTF-8, byte for byte. That single design decision is a big reason UTF-8 conquered the internet without a war.
Where the byte math sneaks up on you
The same 100 “characters” can be 100 bytes in English, around 300 in Korean, or 400 as emoji. Systems that count bytes rather than characters — database columns, SMS messages, some web forms — cut off non-English text sooner than expected. If you have ever watched a form reject a message that looked well under the limit, you have met this bug in the wild.
Emoji push it further. A single family emoji can be several code points joined together, totaling 20 bytes or more for what looks like one symbol. The counter says you have 20 characters left; the underlying limit may firmly disagree.
Treat “one character equals one byte” as a historical accident of English computing, and expect anything past plain ASCII to cost two, three, or four bytes apiece.
Loading comments…