Count unique words in a text.
The type-token ratio — unique words divided by total words — is the standard measure of lexical diversity, and its main quirk is that it falls as text lengthens. A 100-word passage might score 70 percent while a 10,000-word document on the same subject scores 20, simply because function words repeat. That makes the ratio meaningful only between texts of similar length. Words used exactly once, known as hapax legomena, are a more robust signal of vocabulary breadth. Case folding is offered as an option because whether Apple and apple are one word or two depends on whether you are analysing style or entities.
Type-token ratio
Ratio % = unique words / total words x 100
Hapax count
Number of distinct words appearing exactly once in the text
Because vocabulary saturates while length does not. Function words like the and of repeat constantly, so every additional thousand words adds far more tokens than new types. Comparing the ratio across texts of very different lengths is therefore meaningless.
They indicate genuine vocabulary breadth rather than an artefact of length. A high proportion of single-use words suggests varied, specific language; a low proportion suggests the text circles a small core vocabulary. It is a more stable comparison across lengths than the raw ratio.