Estimate an appropriate embedding dimension size for a given vocabulary or dataset.
Embedding dimension should be large enough to capture semantic relationships but not so large that it wastes compute and memory. A common heuristic is dimension ≈ vocab^0.25 × constant. Standard choices are 256, 384, 768, or 1536 for most applications.
Dimension Heuristic
dim ≈ vocab_size^0.25 × 16 (clamped to practical range)
GPU hardware is optimized for dimensions that are powers of 2 or multiples of 64/128. Non-aligned dimensions waste memory bandwidth and CUDA core utilization.
OpenAI text-embedding-3-small uses 1536, Cohere uses 1024, and sentence-transformers models commonly use 384 or 768 dimensions.