MinHash
v0.1.0Added Sep 26, 2026Near-duplicate detection with MinHash signatures, bit-for-bit compatible with datasketch. Estimates Jaccard similarity between token sets in constant space, for deduplicating RAG corpora and training data.
cargo install kura-rs
kura add minhashCopies 2 source files into your project. You own the code from then on.
If another tool already installs a kura command, run cargo install kura-rs --bin kura-rs and use kura-rs add instead.
use crate::parts::minhash::MinHasher;
// Same num_perm and seed as datasketch.MinHash() → identical signatures
let hasher = MinHasher::new(128, 1);
let a = hasher.signature("the quick brown fox jumps".split_whitespace());
let b = hasher.signature("the quick brown fox leaps".split_whitespace());
println!("estimated jaccard = {:.3}", a.jaccard(&b));Checked against datasketch 2.0.0 (Python) using differential testing: the same inputs go to both implementations and the outputs are compared.
kura-rs (Rust) against datasketch across input sizes. Faster at every measured input size, up to 19× faster.
Show data table
| Input size | kura-rs (Rust) | datasketch | Speedup |
|---|---|---|---|
| 10 | 0 ms | 0.01 ms | 19× |
| 100 | 0 ms | 0.04 ms | 9.9× |
| 1,000 | 0.04 ms | 0.37 ms | 8.8× |
| 10,000 | 0.42 ms | 3.73 ms | 9.0× |
| 100,000 | 4.14 ms | 37 ms | 8.9× |
Input size: tokens per document (num_perm = 128).
Measured on Apple M4 Pro, macOS 27.0 (arm64). kura-rs with rustc 1.98.1; reference on Python 3.14.6, numpy 2.5.3.
Files
- parts/minhash/mod.rs
- parts/minhash/permutation.rs
Crate dependencies
- sha10.11