Work out embedding dimension choice instantly with clear inputs, formula shown and shareable results.
An embedding table holds cardinality times dimension parameters, so the dimension choice directly sets memory and the number of weights that must be learned per category. Common heuristics grow the dimension sub-linearly with cardinality — the fast.ai rule uses 1.6 x c^0.56 capped at 600 — because rare categories cannot support many parameters no matter how large the vocabulary.
Embedding sizing
fast.ai: d = min(cap, round(1.6 x c^0.56)); parameters = c x d; bytes = parameters x 4 in fp32
No. Beyond the point where each category has enough examples to fit its vector, extra dimensions add memory and overfitting without improving accuracy.
Bucket them into a shared unknown token or hash them. Giving a category seen twice its own 128-dimensional vector guarantees noise.