"Dafont.com hosts a vast collection of fonts, featuring a forum for users seeking unidentified fonts. It surpasses Google Fonts in comprehensiveness and variety."
"I set up a live benchmark to evaluate LLMs on their ability to identify fonts from images that the community hasn't identified yet, ensuring no prior exposure."
"Evaluating only two LLMs, gpt-4o-mini and gemini-2.5-flash-preview-05-20, I provided images, thread titles, and descriptions to assist in accurate font identification."
"Performance is assessed using a top-k accuracy metric, which checks if the correct font appears within the LLM's first k guesses, allowing for leniency."
Dafont.com offers a large array of fonts, and includes a forum where users seek help identifying unknown fonts. A live benchmark was set up to assess LLMs in identifying fonts from images not previously identified by the community, alleviating issues with benchmark contamination. Two specific LLMs, gpt-4o-mini and gemini-2.5-flash-preview-05-20, were evaluated with image uploads along with thread titles and descriptions for improved identification accuracy. Results showed diminished performance levels for the LLMs within this task.
Read at Max Halford
Unable to calculate read time
Collection
[
|
...
]