Inconsistent emoji display (emoji font ordering issue)

[wolfbeast]

Description

Gitea's Web UI explicitly lists fonts to be used for emoji characters. The order of this causes inconsistency by placing browser-supplied fonts (in particular Twemoji Mozilla) last, and specifying various system-supplied fonts (which are not updated nearly as often) before them. Since emoji rendering occurs in a first-available order, any older/system emoji that are defined will take precedence over browser-supplied fonts, resulting in inconsistent emoji display by having older unicode emoji be used if defined in system fonts but then switching to newer ones if it hits an undefined and switches to the less prioritized emoji font.
This is particularly noticable with country flags where older fonts often don't have flags but a two-letter country code as letters, e.g. 🇨🇶 (flag: Sark) 🇸🇪 (flag:Sweden). Putting likely-more-up-to-date font face definitions first will solve this issue.

Gitea Version

1.23.6+16-ga40e15a116

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

current order

Twemoji to front

Git Version

No response

Operating System

Windows 10 22H2

How are you running Gitea?

self-hosted, Linux binary

Database

MySQL/MariaDB

[wxiaoguang]

Just FYI: GitHub also explicilty lists the font families like this: -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"

---

I don't have enough understanding of this issue or how browser rendering engine works, what's the proper way to handle emoji font families?

[wolfbeast]

Github displays the same issue.
Fonts inside browsers generally work in a "first declared" order. If a font contains a glyph for a character in unicode, then it will display it. if it doesn't, then the next font in the font list is tried. system fonts are usually not updated unless a new version of the OS is installed (and then again frozen until the next, etc.) -- Emoji in particular are updated/added to often, and browsers will keep their internal fonts (if supplied like twemoji mozilla) up to date more often. Those fonts should have precedence so you don't get a mix of emoji.

In fact, if websites would not declare emoji fonts specifically, browsers will generally use their internal list of preferred fonts and orders (which might actually work better!) but if you specify an emoji font order explicitly, then what you should do is: browser-supplied fonts first, then specific known system emoji fonts, then generic fallback.
So, in the case of Gitea's font list:
--fonts-emoji: "Twemoji Mozilla", "Apple Color Emoji", "Segoe UI Emoji", "Noto Color Emoji";
This will make all emoji supplied by Twemoji Mozilla consistently rendered and up-to-date, and only those not covered by that font will be rendered by the fonts after it.
Currently, the order has it last, which causes it to try the Apple font first (if existing) the Segoe UI (if existing) then Noto (if existing) and only then the most complete/recent Twemoji one. Meaning older unicode ones and often B&W versions are picked in favor of newer/color ones if they are present in the prioritized fonts.
I don't know if Chrome and/or Edge come with specific embedded emoji fonts or not, but if they do, they should also go before system fonts.

[wxiaoguang]

Some questions:

• Is Twemoji Mozilla the de-facto standard for all emoji characters? Do Chrome and other browsers have it builtin, or does modern OS have it builtin?
• Putting it before Apple Color Emoji, would it cause inconsistency on macOS or not?

[wolfbeast]

1. Twemoji Mozilla is used in Firefox, Pale Moon, Basilisk and other UXP browsers. As said I don't know if Blink/Safari webkit has/uses a similar embedded font or not. If not, then that entry will just be ignored (and be the same as it already is right now)
2. I don't think so. As said OS supplied fonts are often less complete, and certainly not regularly updated (or at all) like browser fonts that try to stay abreast of Unicode releases. If you think emoji consistency with the OS on Mac is more important (even if less complete coverage... so inconsistent inside the browser) then you could keep that as the first entry, but I'd personally always try to keep things consistent within the web UI even if they would be different than in other applications.

[wxiaoguang]

-> Fix various frontend trivial problems #34263

[silverwind]

By ordering Twemoji Mozilla to the front, users on Apple OS would see that font instead of Apple's, if they have Twemoji installed in the OS or browser, right? I think we should make it so each user gets their OS or browser's preferred emoji font.

I wonder if we even have to specify emoji fonts as part of font-face or if that can be removed entirely. As I understand, browsers will render the right font, even when the emoji fonts are not explicitely listed in font-face, e.g. <html>😆</html> should render fine.

[wolfbeast]

Yeah I already noted above that if you don't specify an emoji font explicitly, you leave it up to browsers (which tend to make sane choices, or have a mechanism for the user to choose what they prefer). You could leave it out altogether, for that behaviour.

But, if you do make it explicit (like Gitea currently does), then you need to pick an order that makes sense. For the Apple situation, as I said, if Apple is placed at the front then it will use Apple OS emoji glyphs up to the point where the OS font doesn't cover a specific emoji, then it would fall back to the other specified fonts. This would lead to inconsistency in emoji inside the Gitea web UI, but would be more consistent with other Apple apps on the system (preferring Apple OS brand styling, but only for those covered by the OS font, which will vary greatly depending on which MacOS version is in use). It would make the most sense to minimize that inconsistency inside the Gitea Web UI IMHO, meaning the most complete font should preferably be first in the list.
Twemoji is the most complete set, AFAIK.

[silverwind]

I think not specifying any emoji fonts will be the best course of action. I don't recall why exactly Gitea has these specified. I think it's mostly because GitHub is/was also doing that, but I agree it's best to let the browser decide.