Skip to content

xtr_rate_limiter

Limits on how often anything may happen: calls, jobs, logins, messages.

A :class:~xtr_rate_limiter.rate_limiter_factory_interface.RateLimiterFactoryInterface is one configured limit; it hands out a :class:~xtr_rate_limiter.limiter_interface.LimiterInterface per key — a user, an address, an account — which answers whether a hit may go ahead:

factory = RateLimiterFactory(
    "api",
    LimiterConfig("sliding_window", limit=100, interval="1 minute"),
    InMemoryStorage(),
)

limit = await factory.create(client_ip).consume()
if not limit.is_accepted():
    ...  # refuse, or `await limit.wait()`

Three policies count hits — a fixed window, a sliding window, a token bucket — and a fourth counts nothing. State lives in a :class:~xtr_rate_limiter.storage.StorageInterface under a lock, or on a Redis server that counts each hit atomically (:class:~xtr_rate_limiter.redis.RedisRateLimiterFactory).

Every call that touches a limiter is awaited.

AUTO_LOCK module-attribute

AUTO_LOCK: Final = 'auto'

Lock through the lock bundle's default factory when active; within this process otherwise.

CACHE_STORAGE module-attribute

CACHE_STORAGE: Final = 'cache'

Keep states in a cache pool: :attr:LimiterConfig.cache_pool.

DEFAULT_CACHE_POOL module-attribute

DEFAULT_CACHE_POOL: Final = 'rate_limiter'

The pool states are kept in: the cache's application pool, under a namespace of its own.

IN_MEMORY_STORAGE module-attribute

IN_MEMORY_STORAGE: Final = 'in-memory'

Keep states in this process only.

Policy

Policy = Literal['fixed_window', 'sliding_window', 'token_bucket', 'no_limit', 'compound']

How a limiter counts hits.

CompoundLimiter

CompoundLimiter(limiters: Sequence[LimiterInterface])

Bases: LimiterInterface

Consumes from each limiter in turn, stopping at the first that refuses.

Say 10 hits a minute and 1 000 a day: both apply to every hit. The answer is the refusal when one refuses, and otherwise the limit closest to running out.

The limiters consulted before a refusal have spent their tokens; they are not given back. Only :meth:consume is offered — reserving on several limiters would book tokens on some while another refuses.

Combine limiters, consulted in order.

Raises:

reserve async

reserve(tokens: int = 1, max_time: float | None = None) -> Reservation

Refuse: a compound limiter cannot reserve.

Raises:

consume async

consume(tokens: int = 1) -> RateLimit

Consume tokens from every limiter until one refuses.

reset async

reset() -> None

Reset every limiter.

CompoundRateLimiterFactory

CompoundRateLimiterFactory(factories: Mapping[str, RateLimiterFactoryInterface], keys: Mapping[str, str] | None = None)

Bases: RateLimiterFactoryInterface

Hands out a :class:~xtr_rate_limiter.compound_limiter.CompoundLimiter per key.

per_user = CompoundRateLimiterFactory(
    {"burst": burst_factory, "daily": daily_factory, "global": global_factory},
    keys={"global": "everyone"},
)

Combine factories, consulted in order.

Parameters:

Raises:

create

create(key: str | None = None) -> LimiterInterface

Return a limiter combining one limiter of every factory.

RateLimitExceededEvent dataclass

RateLimitExceededEvent(rate_limit: RateLimit, limiter_name: str | None = None, key: str | None = None)

Bases: Event

A caller turned a hit away because a limiter denied it.

A denied hit is not a rejection by itself — a caller may wait, retry or carry on — so limiters never dispatch this. Whoever turns the hit away does, such as a web route answering "too many requests".

Attributes:

  • rate_limit (RateLimit) –

    The limit that denied the hit.

  • limiter_name (str | None) –

    The configured limiter's name, when known.

  • key (str | None) –

    The key that was consumed, when known, in whatever form the caller built it.

InvalidArgumentError

InvalidArgumentError(reason: str)

Bases: RateLimiterError, ValueError

A limiter, a factory or a storage was given an argument it cannot work with.

Raised where the argument is given — a limit below one, more tokens than a limiter can ever hold, a policy option that belongs to another policy — rather than on the first hit.

Also a :class:ValueError, so code that already guards its configuration with except ValueError keeps working without learning a new exception.

Attributes:

  • reason (str) –

    What is wrong with the argument.

Record what is wrong with the argument.

InvalidIntervalError

InvalidIntervalError(interval: object, reason: str)

Bases: InvalidArgumentError

An interval could not be read, or is not long enough to count anything in.

Attributes:

  • interval (object) –

    The interval as it was given.

Record the interval and what is wrong with it.

MaxWaitDurationExceededError

MaxWaitDurationExceededError(wait_duration: float, max_time: float, rate_limit: RateLimit)

Bases: RateLimiterError

A reservation would have to wait longer than its caller agreed to.

Nothing was reserved: the limiter's state is left as it was.

Attributes:

  • wait_duration (float) –

    The seconds the tokens would have taken to be free.

  • max_time (float) –

    The seconds the caller agreed to wait.

  • rate_limit (RateLimit) –

    The limit as it stands, rejected.

Record how long the wait would have been, and the limit it was measured against.

RateLimiterError

Bases: Exception

Base class for every error raised by this library.

Catch this to handle anything limiting can go wrong with; catch a subclass to handle one cause. Every subclass carries the data a caller needs as typed attributes and composes its own message from them.

RateLimiterStorageError

RateLimiterStorageError(reason: str)

Bases: RateLimiterError

The server keeping a limiter's state failed to answer.

Raised rather than answering as if the limiter were empty: whether a limit fails open or closed is the caller's decision, not the storage's.

Attributes:

  • reason (str) –

    What the server said, or why it could not be reached.

Record why the state could not be read or written.

RateLimitExceededError

RateLimitExceededError(rate_limit: RateLimit)

Bases: RateLimiterError

A hit the caller insisted on was not accepted.

Raised by :meth:RateLimit.ensure_accepted <xtr_rate_limiter.rate_limit.RateLimit.ensure_accepted>, for code that would rather fail than branch on the answer.

Attributes:

  • rate_limit (RateLimit) –

    The rejected limit.

Record the rejected limit.

retry_after property

retry_after: datetime

When the tokens asked for become available.

remaining_tokens property

remaining_tokens: int

How many tokens are left.

limit property

limit: int

How many tokens the limiter holds when full.

ReserveNotSupportedError

ReserveNotSupportedError(limiter: str)

Bases: RateLimiterError

A limiter was asked to reserve, which it cannot do.

A compound limiter answers only :meth:consume: reserving on several limiters would book tokens on some of them while another refuses.

Attributes:

  • limiter (str) –

    The name of the limiter's class.

Record which limiter refused.

LimiterConfig dataclass

LimiterConfig(policy: Policy, limit: int | None = None, interval: IntervalLike | None = None, rate: Rate | None = None, anchor_at: datetime | str | None = None, limiters: Sequence[str] = (), keys: Mapping[str, str] = _no_keys(), storage: str | Reference = CACHE_STORAGE, cache_pool: str = DEFAULT_CACHE_POOL, lock: str | None = AUTO_LOCK)

Configuration for a single named limiter.

policy decides which of the other fields apply:

  • "fixed_window" — at most limit hits per window of interval, opening on the first hit, or following a calendar from anchor_at;
  • "sliding_window" — at most limit hits in any span of interval;
  • "token_bucket" — bursts of up to limit hits, refilled at rate;
  • "no_limit" — every hit accepted, nothing kept;
  • "compound" — every one of limiters applies to each hit.
LimiterConfig("sliding_window", limit=100, interval="1 minute")
LimiterConfig("token_bucket", limit=10, rate=Rate("1 minute", amount=2))
LimiterConfig("fixed_window", limit=5, interval="15 minutes", storage=env("REDIS_DSN"))
LimiterConfig("compound", limiters=["api", "uploads"])

A factory reads how the limiter counts; the bundle also reads storage, cache_pool and lock, and builds the storage and lock factory it hands the factory.

Attributes:

  • policy (Policy) –

    How the limiter counts hits.

  • limit (int | None) –

    The hits a window takes, or the tokens a bucket holds.

  • interval (IntervalLike | None) –

    How long a window lasts: "15 minutes", "1 hour 30 minutes", or a :class:~datetime.timedelta.

  • rate (Rate | None) –

    How fast a bucket refills.

  • anchor_at (datetime | str | None) –

    Align fixed windows to a calendar starting at this instant — ISO 8601, read as UTC without a zone — instead of opening one on the first hit. Requires an interval of at least a month.

  • limiters (Sequence[str]) –

    The configured limiters a compound limiter combines, by name.

  • keys (Mapping[str, str]) –

    A key to use for some of the combined limiters instead of the one the compound limiter is created with — {"global": "all"} to share one count across every caller.

  • storage (str | Reference) –

    "cache" for a cache pool, "in-memory" for this process only, a Redis DSN ("redis://host:6379") to count atomically on the server with no lock, or a :class:~xtr_dependency_injection.Reference to a :class:~xtr_rate_limiter.storage.StorageInterface or a Redis client the container provides. Also env(...).

  • cache_pool (str) –

    The cache pool a "cache" storage keeps states in.

  • lock (str | None) –

    The lock around each read and write of a state: "auto", a lock resource's name, or None to lock within this process only. Unused in Redis, which needs none.

anchor

anchor() -> datetime | None

Return :attr:anchor_at as a timezone-aware datetime; None when there is none.

Raises:

parsed_interval

parsed_interval() -> Interval

Return :attr:interval, read.

Raises:

LimiterInterface

Bases: Protocol

Counts the hits of one key — one user, one address, one account — against a limit.

A limiter comes from a :class:~xtr_rate_limiter.rate_limiter_factory_interface.RateLimiterFactoryInterface, one per key. Two ways to spend tokens:

  • :meth:consume decides now: the tokens are granted, or they are not and nothing is spent.
  • :meth:reserve books the tokens for the moment they become available, which may lie ahead, and says when that is.

Spending zero tokens changes nothing and reports the limit as it stands.

reserve async

reserve(tokens: int = 1, max_time: float | None = None) -> Reservation

Book tokens for the moment they become available.

Parameters:

  • tokens (int, default: 1 ) –

    How many tokens to book.

  • max_time (float | None, default: None ) –

    The most seconds the caller will wait; any wait when None.

Raises:

consume async

consume(tokens: int = 1) -> RateLimit

Spend tokens now if they are available; spend nothing otherwise.

Raises:

reset async

reset() -> None

Forget every hit counted for this key.

LimiterStateInterface

Bases: Protocol

The state of one limiter for one key, as a storage keeps it.

A state is a plain object a storage may copy, pickle or hand back as it is. It knows the moment it is no longer worth keeping, so a storage lets it go then.

id property

id: str

The limiter's id and key, which the state is stored under.

expires_at property

expires_at: float | None

When the state can be forgotten, in seconds since the epoch; None for never.

FixedWindowLimiter

FixedWindowLimiter(state_id: str, limit: int, interval: Interval, storage: StorageInterface, lock: LimiterLock, *, anchor_at: datetime | None = None, clock: ClockInterface | None = None)

Bases: StoredLimiter

At most limit hits per window of interval.

By default a window opens on the first hit and lasts interval. With anchor_at, windows follow a calendar instead: every interval from that instant — the first of each month, say — whenever the hits arrive. The anchor keeps its zone, so a local midnight stays a local midnight across a clock change, and may lie in the past or the future.

A window counts hits simply, so a burst at the end of one window and another at the start of the next can reach twice the limit in a short span; a sliding window smooths that out.

A window covers [start, start + interval), so the instant one interval past its start already belongs to the next window.

Limit state_id to limit hits per interval.

Raises:

  • InvalidArgumentError –

    When limit is below one, or anchor_at is given with an interval shorter than a month.

NoLimiter

NoLimiter(clock: ClockInterface | None = None)

Bases: LimiterInterface

Accepts every hit and keeps nothing.

For code that expects a limiter where no limit should apply — a limit turned off in one environment, a trusted caller.

Report instants by clock; the clock in force when None.

reserve async

reserve(tokens: int = 1, max_time: float | None = None) -> Reservation

Grant tokens at once.

consume async

consume(tokens: int = 1) -> RateLimit

Grant tokens.

reset async

reset() -> None

Do nothing: nothing was counted.

Rate dataclass

Rate(interval: IntervalLike, amount: int = 1)

amount tokens back in the bucket every interval.

Rate("10 seconds", amount=5)
Rate.per_minute(30)

Attributes:

  • interval (IntervalLike) –

    How often the bucket is topped up: "15 minutes", or a :class:~datetime.timedelta.

  • amount (int) –

    How many tokens each top-up adds.

per_second classmethod

per_second(amount: int = 1) -> Rate

Return amount tokens a second.

per_minute classmethod

per_minute(amount: int = 1) -> Rate

Return amount tokens a minute.

per_hour classmethod

per_hour(amount: int = 1) -> Rate

Return amount tokens an hour.

per_day classmethod

per_day(amount: int = 1) -> Rate

Return amount tokens a day.

per_month classmethod

per_month(amount: int = 1) -> Rate

Return amount tokens a month.

per_year classmethod

per_year(amount: int = 1) -> Rate

Return amount tokens a year.

cycle_from

cycle_from(timestamp: float) -> float

Return how many seconds one top-up takes, starting at timestamp.

SlidingWindowLimiter

SlidingWindowLimiter(state_id: str, limit: int, interval: Interval, storage: StorageInterface, lock: LimiterLock, *, clock: ClockInterface | None = None)

Bases: StoredLimiter

At most limit hits in any span of interval, as estimated by a sliding window.

Where a fixed window lets a burst at the end of one window and another at the start of the next through together, a sliding window still counts the first burst while it lies within the last interval.

Limit state_id to limit hits in any interval.

Raises:

TokenBucketLimiter

TokenBucketLimiter(state_id: str, max_burst: int, rate: Rate, storage: StorageInterface, lock: LimiterLock, *, clock: ClockInterface | None = None)

Bases: StoredLimiter

Bursts of up to max_burst hits, then a steady rate.

Each hit takes a token from a bucket holding at most max_burst; the rate puts tokens back. An idle caller can burst, a busy one settles to the rate.

Limit state_id to bursts of max_burst, refilled at rate.

Raises:

RateLimit

RateLimit(remaining_tokens: int, retry_after: datetime, accepted: bool, limit: int, reset_at: datetime | None = None, *, clock: ClockInterface | None = None)

The state of a limit, as one consumption left it.

Two instants are told apart. :attr:retry_after is when the tokens this caller asked for become available; :attr:reset_at is when the limiter is back to holding every token — None for a limiter that has no such moment.

limit = await limiter.consume()
if not limit.is_accepted():
    await limit.wait()

Describe a limit.

Parameters:

  • remaining_tokens (int) –

    How many tokens are left.

  • retry_after (datetime) –

    When the tokens asked for become available.

  • accepted (bool) –

    Whether the consumption was accepted.

  • limit (int) –

    How many tokens the limiter holds when full.

  • reset_at (datetime | None, default: None ) –

    When the limiter holds every token again, if it ever does.

  • clock (ClockInterface | None, default: None ) –

    What :meth:wait measures against; the clock in force when None.

remaining_tokens property

remaining_tokens: int

How many tokens are left.

retry_after property

retry_after: datetime

When the tokens asked for become available.

limit property

limit: int

How many tokens the limiter holds when full.

reset_at property

reset_at: datetime | None

When the limiter holds every token again; None when it has no such moment.

is_accepted

is_accepted() -> bool

Tell whether the tokens asked for were granted.

ensure_accepted

ensure_accepted() -> Self

Return this limit, or raise when it was not accepted.

Raises:

wait async

wait() -> None

Wait until :attr:retry_after, without blocking the event loop.

RateLimiterBuilder

RateLimiterBuilder(storage: StorageInterface, lock_factory: LockFactory | None = None, *, clock: ClockInterface | None = None)

Builds limits in code, all kept in one storage.

For a limit decided at runtime — per tenant, per plan — rather than written in configuration:

factory = builder.token_bucket(f"tenant-{tenant.id}", limit=tenant.burst, interval="1 second")
limit = await factory.create(user_id).consume()

Keep every limiter's state in storage, locked through lock_factory.

sliding_window

sliding_window(limiter_id: str, limit: int, interval: IntervalLike) -> RateLimiterFactoryInterface

Return at most limit hits in any span of interval.

fixed_window

fixed_window(limiter_id: str, limit: int, interval: IntervalLike, anchor_at: datetime | str | None = None) -> RateLimiterFactoryInterface

Return at most limit hits per window of interval, aligned to anchor_at.

token_bucket

token_bucket(limiter_id: str, limit: int, interval: IntervalLike, amount: int = 1) -> RateLimiterFactoryInterface

Return bursts of up to limit hits, amount tokens back every interval.

compound

compound(*factories: RateLimiterFactoryInterface) -> RateLimiterFactoryInterface

Return every one of factories applied to each hit.

noop

noop() -> RateLimiterFactoryInterface

Return no limit at all.

RateLimiterFactory

RateLimiterFactory(limiter_id: str, config: LimiterConfig, storage: StorageInterface, lock_factory: LockFactory | None = None, *, clock: ClockInterface | None = None)

Bases: RateLimiterFactoryInterface

Hands out limiters for one limit, keeping their state in storage.

Each limiter's state is kept under "<length>:<id>:<key>", the length being the limit's name's, so no two name-and-key pairs ever meet in one state. Its read and write happen under a lock on that name, taken from lock_factory — so processes sharing the storage also share the lock — or held within this process when there is none.

factory = RateLimiterFactory(
    "api",
    LimiterConfig("sliding_window", limit=100, interval="1 minute"),
    InMemoryStorage(),
)
limit = await factory.create(client_ip).consume()

Build limiters counting as config says, named limiter_id.

config's storage and lock fields are the bundle's to read; here the storage and the lock factory are given.

Raises:

  • InvalidArgumentError –

    When config is a compound limiter — that is a :class:~xtr_rate_limiter.CompoundRateLimiterFactory.

create

create(key: str | None = None) -> LimiterInterface

Return the limiter counting the hits of key.

RateLimiterFactoryInterface

Bases: Protocol

One configured limit, handing out a limiter for each key it is asked about.

limiter = factory.create(f"user-{user.id}")
(await limiter.consume()).ensure_accepted()

create

create(key: str | None = None) -> LimiterInterface

Return the limiter counting the hits of key.

Limiters for the same key share their count, whichever factory call made them. None is one key like any other: a limit shared by every caller.

Reservation

Reservation(time_to_act: float, rate_limit: RateLimit, *, clock: ClockInterface | None = None)

Tokens a limiter has booked, to be used at :attr:time_to_act.

The booking is made when the reservation is: from then on the limiter counts those tokens as spent, so later callers are served after this one.

reservation = await limiter.reserve(tokens=1, max_time=5)
await reservation.wait()
await call_the_api()

Describe a reservation.

Parameters:

  • time_to_act (float) –

    When the tokens may be used, in seconds since the epoch.

  • rate_limit (RateLimit) –

    The limit as the reservation left it.

  • clock (ClockInterface | None, default: None ) –

    What waiting is measured against; the clock in force when None.

time_to_act property

time_to_act: float

When the tokens may be used, in seconds since the epoch.

rate_limit property

rate_limit: RateLimit

The limit as the reservation left it.

wait_duration

wait_duration() -> float

Return how many seconds are left until :attr:time_to_act; zero once it passed.

wait async

wait() -> None

Wait until :attr:time_to_act, without blocking the event loop.

CacheStorage

CacheStorage(pool: CacheItemPoolInterface)

Bases: StorageInterface

Keeps states in a cache pool, each expiring with its state.

States are stored under "rl." and a hash of their id, so any key a caller limits by — an address, an email — fits the pool's rules for keys and no other user of the pool can reach a limiter's count by accident.

A cache never fails loudly: a pool that cannot reach its backend reads as empty and drops what it is given. A limiter on such a pool therefore lets every hit through while the backend is down — it fails open. Keep limits that must hold under that failure in Redis instead, where a failure is an error.

A state goes in and comes back as an object, which a pool encodes with pickle by default: reading one runs code the stored bytes name. On a backend anything outside the application can write to — a Redis server shared with another tenant — give the pool a :class:~xtr_cache.marshaller.SodiumMarshaller, which refuses bytes it cannot authenticate before they ever reach pickle.

Keep states in pool.

save async

save(state: LimiterStateInterface) -> None

Store state, expiring when it does.

fetch async

fetch(state_id: str) -> LimiterStateInterface | None

Return the state stored under state_id; None for anything else.

delete async

delete(state_id: str) -> None

Remove the state stored under state_id.

InMemoryStorage

InMemoryStorage(clock: ClockInterface | None = None)

Bases: StorageInterface

Keeps states in a dictionary of this process, each until it expires.

A state is copied in and out, so a limiter changing the state it fetched changes nothing here until it saves it, and a state saved is a snapshot of the moment it was. The copy is made object to object, so nothing a caller handed over is ever serialized or read back as code. Nothing is shared with another process: each counts its own hits.

Every thousand saves the expired states are swept out, so a process limiting by client address does not keep one state per address it ever saw.

Keep states, expiring them by clock; the clock in force when None.

save async

save(state: LimiterStateInterface) -> None

Keep a copy of state until it expires.

fetch async

fetch(state_id: str) -> LimiterStateInterface | None

Return a copy of the state kept under state_id, unless it expired.

delete async

delete(state_id: str) -> None

Forget the state kept under state_id.

StorageInterface

Bases: Protocol

Keeps limiter states by id: fetch one, save one, delete one.

A storage is deliberately plain — nothing here is atomic. A limiter keeps two callers from interleaving by holding a lock on the id around each fetch and save, so a storage only has to keep what it is given and let it go once its :attr:~LimiterStateInterface.expires_at passes.

save async

save(state: LimiterStateInterface) -> None

Keep state under its id until it expires.

fetch async

fetch(state_id: str) -> LimiterStateInterface | None

Return the state kept under state_id; None when there is none.

delete async

delete(state_id: str) -> None

Forget the state kept under state_id; an id holding nothing is fine.