distkit
Distkit (DISTributed systems KIT) is a Rust library of distributed systems primitives backed by Redis. It gives you four counter types (strict, lax, strict instance-aware, and lax instance-aware) for global consistency, it also includes sliding-window rate limiting via the trypema.
Overview
Distkit ships four distributed counter implementations and re-exports a rate limiter (trypema).
Counter types
The counter types are what they sound like. They are used to hold counters across a distributed system. The counters are in two categories, the normal counters (StrictCounter, LaxCounter) and the instance aware counters. The normal counters behave like normal counters and can be useful for counting events across a shared key or namespace, while an instance aware counter keeps track of the counter each instance (or server) holds, if the instance dies, then the count it holds would be subtracted from the cumulative total. If the server is marked as dead because of, say, a network interruption, when the instance comes back online the values are recovered and synchronized with the global cumulative.
There is also a concept of epochs for the instance aware counters. An epoch is more like the version number of era the counters are in. If the epoch changes, it means we have moved on to start from zero. During recovery, an instance might attempt recovery, the recovery is only done if the last recorded epoch is still the same as the global epoch. So, recoveries are safe. Here is a table that highlights general features in each counter type:
| StrictCounter | LaxCounter | StrictInstanceAwareCounter | LaxInstanceAwareCounter | |
|---|---|---|---|---|
| Consistency | Immediate | Eventual | Immediate | Eventual |
inc latency | Redis round-trip | Sub-microsecond | Redis round-trip | Sub-microsecond |
| Redis I/O | Every call | Batched | Every call | Batched |
| Per-instance tracking | No | No | Yes | Yes |
| Dead-instance cleanup | No | No | Yes | Yes |
| Feature flag | counter (default) | counter (default) | instance-aware-counter | instance-aware-counter |
| Use case | Billing, inventory | High-throughput analytics | Live connection counts | Live connection counts |
StrictCounter
Every call is a single Redis round-trip executing an atomic Lua script. The stored value is always authoritative with no local state to go stale. Use this when accuracy is non-negotiable (billing, inventory, seat counts).
[dependencies]
distkit = "0.2"
use distkit::{RedisKey, counter::{StrictCounter, CounterOptions, CounterTrait}};
let client = redis::Client::open("redis://127.0.0.1/")?;
let conn = client.get_connection_manager().await?;
let prefix = RedisKey::try_from("my_app".to_string())?;
let counter = StrictCounter::new(CounterOptions::new(prefix, conn));
let key = RedisKey::try_from("orders".to_string())?;
counter.inc(&key, 1).await?; // HINCRBY via Lua
counter.set(&key, 100).await?; // HSET via Lua
let total = counter.get(&key).await?; // HGET
counter.del(&key).await?; // HDEL, returns old value
counter.clear().await?; // DEL on the hash
LaxCounter
When you can allow a small lag between writes and reads, use the LaxCounter. It would be significantly faster. The main caveat is that we get an eventual consistency instead of immediate consistency the StrictCounter offers.
use distkit::{RedisKey, counter::{LaxCounter, CounterOptions, CounterTrait}};
let counter = LaxCounter::new(CounterOptions::new(prefix, conn));
let key = RedisKey::try_from("impressions".to_string())?;
counter.inc(&key, 1).await?; // local atomic add, sub-microsecond
let val = counter.get(&key).await?; // reads local state, no Redis hit
Instance-aware counters
As mentioned above, the instance aware counters are a bit different from the normal counters. They track the counter on each instance and allow you to read the cumulative total. And they manage instance deaths and recovery.
[dependencies]
distkit = { version = "0.2", features = ["instance-aware-counter"] }
StrictInstanceAwareCounter
This is the StrictCounter with instance awareness. Every call is immediately consistent.
use distkit::icounter::{
InstanceAwareCounterTrait,
StrictInstanceAwareCounter, StrictInstanceAwareCounterOptions,
};
use distkit::RedisKey;
let counter = StrictInstanceAwareCounter::new(
StrictInstanceAwareCounterOptions::new(prefix, conn),
);
let key = RedisKey::try_from("connections".to_string())?;
// All methods return (cumulative, this_instance_value).
let (total, mine) = counter.inc(&key, 5).await?;
let (total, mine) = counter.dec(&key, 2).await?;
let (total, mine) = counter.get(&key).await?;
// Adjust only this instance's slice (no epoch bump).
let (total, mine) = counter.set_on_instance(&key, 10).await?;
// Coordinate a global reset across all instances (bumps epoch).
let (total, mine) = counter.set(&key, 100).await?;
// Remove only this instance's contribution.
let (total, removed) = counter.del_on_instance(&key).await?;
// Delete the key globally and bump the epoch.
let (old_total, _) = counter.del(&key).await?;
LaxInstanceAwareCounter
The LaxInstanceAwareCounter is a buffered wrapper around StrictInstanceAwareCounter. It allowed for a small lag between writes and reads and batches the writes when necessary. This allowed for eventual consistency but at a high throughput since we are not paying the network price for each operation.
use distkit::icounter::{
InstanceAwareCounterTrait,
LaxInstanceAwareCounter, LaxInstanceAwareCounterOptions,
};
use distkit::RedisKey;
use std::time::Duration;
let counter = LaxInstanceAwareCounter::new(LaxInstanceAwareCounterOptions {
prefix,
connection_manager: conn,
dead_instance_threshold_ms: 30_000,
flush_interval: Duration::from_millis(20),
allowed_lag: Duration::from_millis(20),
});
let key = RedisKey::try_from("connections".to_string())?;
// Warm path: returns local estimate with no Redis call.
let (local_total, mine) = counter.inc(&key, 1).await?;
let (local_total, mine) = counter.dec(&key, 1).await?;
let (total, mine) = counter.get(&key).await?;
Dead-instance cleanup
Each instance writes a liveness heartbeat to Redis on every flush. If a process silently dies, surviving instances detect the expired heartbeat and remove the orphaned contribution automatically the next time any of them touches the same key.
// Two separate processes / server instances:
let server_a = StrictInstanceAwareCounter::new(opts(conn1));
let server_b = StrictInstanceAwareCounter::new(opts(conn2));
server_a.inc(&key, 10).await?; // cumulative = 10
server_b.inc(&key, 5).await?; // cumulative = 15
// server_a goes offline.
// After dead_instance_threshold_ms (default 30 s), server_b's next call
// removes server_a's contribution automatically.
let (total, _) = server_b.get(&key).await?; // total = 5 once cleaned up
Rate limiting
Enable the trypema feature to access sliding-window rate limiting. distkit re-exports the entire trypema crate under distkit::trypema:
To get more details on trypema, you can see the project overview, the api documentation or the rust documentation
[dependencies]
distkit = { version = "0.2", features = ["trypema"] }
Three providers are available: local (in-process, sub-microsecond), Redis (distributed, atomic Lua enforcement), and hybrid (local fast-path with periodic Redis sync). Two strategies are available: absolute (binary allow/reject sliding window) and suppressed (probabilistic degradation that ramps rejection probability near capacity).
use distkit::trypema::{RateLimit, RateLimitDecision, RateLimiter, RateLimiterOptions};
// ... build rl: Arc<RateLimiter> at startup ...
let rate = RateLimit::try_from(10.0)?; // 10 requests per second
match rl.local().absolute().inc("user_123", &rate, 1) {
RateLimitDecision::Allowed => { /* proceed */ }
RateLimitDecision::Rejected { retry_after_ms, .. } => {
eprintln!("Rate limited, retry in {retry_after_ms} ms");
}
_ => {}
}
Highlights
::div
class: "grid grid-cols-2 sm:grid-cols-4 gap-4 md:gap-6"
:::content-card
class: "col-span-2"
Architecture
::::div
class: "text-xs text-dune-600"
- Four counter types across two feature flags
- Strict variants: atomic Lua scripts on every call
- Lax variants: in-memory
DashMapbuffer with background flush task - Instance-aware variants: UUID-namespaced Redis hash fields
- All backed by
redis::aio::ConnectionManager
:::: :::
:::content-card
class: "col-span-2"
Consistency model
::::div
class: "text-xs text-dune-600"
- Strict counters: immediately consistent, no local state
- Lax counters: eventual consistency, configurable lag (default 20 ms)
- Instance-aware: per-key epoch prevents double-counting after
set/del set_on_instanceadjusts only this instance's slice without touching the epoch
:::: :::
:::content-card
class: "col-span-2"
Warm-path performance
::::div
class: "text-xs text-dune-600"
LaxCounter::incandLaxInstanceAwareCounter::inc: sub-microsecond, pure in-memory atomicsLaxInstanceAwareCounter::getandset_on_instance: also in-memory, no Redis round-trip- Flushes are batched into Redis pipelines on a configurable interval
- Background flush task drops automatically when the counter is dropped
:::: :::
:::content-card
class: "col-span-2"
Dead-instance cleanup
::::div
class: "text-xs text-dune-600"
- Each instance writes a heartbeat to
<prefix>:liveness:<uuid>on every flush - Surviving instances scan liveness keys and remove expired contributions automatically
- Configurable threshold via
dead_instance_threshold_ms(default 30 s) - No manual cleanup needed; works correctly after crashes and ungraceful shutdowns
:::: :::
:::content-card
class: "col-span-2"
Safety & correctness
::::div
class: "text-xs text-dune-600"
#![forbid(unsafe_code)]: zero unsafe blocks in the library- No panics in library code; all fallible paths return
DistkitError - 57 runnable doc-test examples, each isolated to a unique Redis key prefix
- Criterion benchmarks for all four counter types across all operations
:::: :::
:::content-card
class: "col-span-2"
Feature flags
::::div
class: "text-xs text-dune-600"
counter(default):StrictCounterandLaxCounterinstance-aware-counter:StrictInstanceAwareCounterandLaxInstanceAwareCountertrypema: sliding-window rate limiting via the trypema crate- All flags are independent and composable
:::: :::
::
Other Projects
Payaza Web SDK
A JavaScript Web SDK that simplifies integrating Payaza checkout on web applications. Built as part of my role at Payaza Africa.
Actix Web Starter Template
Production-ready Rust/Actix Web REST API starter with RBAC auth, SeaORM/PostgreSQL, Kafka-based email, and Docker tooling.
Canvas Random Floating Circle
Pet project animating randomly floating circles on HTML Canvas with simple drift and easing.