Jaccard index

Jaccard index, also Tanimoto coefficient, measures similarity between two sets A and B as ratio of intersection to union cardinalities:

J(A, B) = |A ∩ B| / |A ∪ B|

Range: [0, 1]. J = 0 if A, B disjoint, J = 1 if A = B. Introduced by Paul Jaccard (1901) for biogeographic similarity between alpine flora regions. Now standard in NLP, bioinformatics, recommender systems, CV.

Definition and formula

• |A ∩ B| = number of common elements.
• |A ∪ B| = total distinct elements.
• J(A, B) = |A ∩ B| / |A ∪ B|.
• Jaccard distance: d_J = 1 - J(A, B), range [0, 1].

Example: A = {a, b, c, d}, B = {c, d, e, f}. A ∩ B = {c, d}, A ∪ B = {a, b, c, d, e, f}. J = 2/6 = 0.333.

Binary version (Tanimoto)

For binary vectors (0/1) of length n, with n_11, n_10, n_01, n_00 counts:

J = n_11 / (n_11 + n_10 + n_01)

Excludes n_00 (both 0). Important for sparse data: two users not interacting with same product are NOT similar. Also called Tanimoto coefficient.

Sorensen-Dice coefficient

Popular variant: D(A, B) = 2 |A ∩ B| / (|A| + |B|). Range [0, 1].

• Jaccard: |A ∩ B| / |A ∪ B|.
• Sorensen-Dice: 2 |A ∩ B| / (|A| + |B|).
• Relation: D = 2J / (1 + J).
• Dice gives more weight to intersection.

Jaccard distance as metric

d_J = 1 - J(A, B) IS genuine metric (Levandowsky-Winter 1971): satisfies positivity, identity, symmetry, triangle. Note: Sorensen-Dice 1 - D NOT metric. Jaccard preferred when metric structure needed.

Example: document comparison

Tokenized texts as word sets. Documents with same key terms have high J, regardless of frequency. n-gram variant: sets of length-n substrings.

Weighted Jaccard (Ruzicka)

For non-negative integer vectors:

J_w(a, b) = sum min(a_i, b_i) / sum max(a_i, b_i)

Generalization to vectors. With binary values, recovers binary Jaccard. Used in ecology, NLP frequencies, CV histograms.

Applications

• NLP: document similarity via token or n-gram overlap.
• Recommender systems: similar users via product sets.
• Bioinformatics: genome similarity, biodiversity.
• CV: bounding box IoU - Intersection over Union.
• Plagiarism detection: shingle-based (Broder 1997).
• Network analysis: node similarity via shared neighbors.
• Cheminformatics: molecule fingerprint similarity.
• Market basket analysis.

IoU in computer vision

Intersection over Union: Jaccard for bounding boxes in object detection. IoU = Area(box_pred ∩ box_true) / Area(box_pred ∪ box_true). Standard threshold IoU >= 0.5 for true positive. COCO, ImageNet, Pascal VOC standard. Drives YOLO, Faster R-CNN losses.

MinHash and LSH

For huge sets, MinHash (Broder 1997): random hash permutation, take min hash as signature. Pr[h_min(A) = h_min(B)] = J(A, B). Estimates Jaccard in O(k) instead of O(|A| + |B|). LSH groups similar sets sub-linearly. Big data standard: Web duplicate detection, near-duplicate documents.

Limits

• Sensitive to granularity (n-gram size, tokenization).
• Doesn't consider order or frequency (basic version).
• Sparse: dominated by rare terms.
• Dense: dominated by common terms.
• Weighted Ruzicka: not metric in general.

History

Paul Jaccard (1901): introduces index for alpine flora distribution. Tanimoto (1957): cheminformatics redefinition. Sorensen (1948), Dice (1945): ratio variant. Levandowsky-Winter (1971): prove triangle of d_J. Broder (1997): MinHash for Web. Recommender systems 90s-2000s.

Botanical origins: Paul Jaccard and biodiversity

The Jaccard index was born not in computer science but in botany. In 1901 the Swiss botanist Paul Jaccard, studying the flora of the Alps, sought a way to quantify how much two regions shared the same plant species. He thus introduced the ratio between the number of common species and the total number of species present in at least one of the two areas, giving rise to one of the longest-lived and most used similarity measures in science.

In ecology the index is still fundamental for measuring beta-diversity, that is, the turnover of species between different habitats: a high value indicates similar communities, a low one very different communities. The same idea has extended to modern genomics, where sets of expressed genes or microbial species in different samples are compared. Comparing the gut microbiome composition of two individuals, for example, reduces to computing how much their sets of bacteria overlap, exactly as Jaccard did with alpine flowers over a century ago.

Why the Jaccard distance is a true metric

Unlike the cosine distance, which does not satisfy the triangle inequality, the Jaccard distance, defined as 1 minus the Jaccard index, is a genuine metric: it respects positivity, symmetry and the triangle inequality. This means it behaves like a true geometric distance, in which the direct path between two sets is never longer than one passing through a third set. It is a valuable property, because it guarantees consistency in clustering and nearest-neighbour search algorithms.

The proof has an elegant link with probability and the MinHash technique: the Jaccard index between two sets is exactly the probability that a random permutation of their elements assigns the same minimum to both. This probabilistic interpretation not only makes the distance a metric, but allows it to be estimated quickly on huge sets without comparing them element by element, the principle behind deduplicating billions of web pages and near-duplicate detection at massive scale.

Updated July 2026