Skip to content

xtr_cache_contracts

The caching contract: what code that caches depends on, and nothing more.

Code that caches should not decide where values go. It takes a :class:CacheInterface and hands it a key and the function that computes the value; the application decides whether memory, files or a server sit behind it — so depending on this package costs a library nothing but the contract.

async def load_profile(item: ItemInterface) -> Profile:
    item.expires_after(3600)
    return await profiles.fetch(user_id)


profile = await cache.get(f"profile.{user_id}", load_profile)

Three levels, most code needing only the first:

  • :class:CacheInterface — fetch-or-compute, and delete. :class:TagAwareCacheInterface adds invalidation by tag.
  • :class:CacheItemPoolInterface — the items underneath: hits told apart from misses, reads of several keys at once, batched writes. What a backend implements; :class:CacheMixin builds :class:CacheInterface on it.
  • :class:NamespacedPoolInterface — a view of a pool confined to a sub-namespace.

Every call that reaches a backend is awaited. Backends and everything that acts on stored values — adapters, serialisation, stampede locks, the bundle — are xtr-cache, which implements this contract and re-exports it, so the two are never two different objects.

RESERVED_CHARACTERS module-attribute

RESERVED_CHARACTERS: Final = '{}()/\\@:'

Characters no key and no tag may contain.

Pools use them for their own structure — namespace separators above all — so a key holding one could reach into another key's space.

No key or tag may hold a non-printable character either (anything :py:meth:str.isprintable rejects): a pool builds its own internal keys — a tag's version among them — with control characters, so a key that could hold one could name them.

Callback

Callback = Callable[[ItemInterface], Awaitable[T]]

Computes the value for an item, on a miss.

Receives the item the value is for, so it can set how long the value lives (:meth:~xtr_cache_contracts.item_interface.ItemInterface.expires_after) or tag it (:meth:~xtr_cache_contracts.item_interface.ItemInterface.tag), and returns the value. Asynchronous, because computing a value worth caching is usually I/O.

Any async def taking the item fits, and so does an object with an async def __call__ taking it. Raising stores nothing: the error reaches the caller of :meth:~xtr_cache_contracts.cache_interface.CacheInterface.get as it was raised.

async def load_profile(item: ItemInterface) -> Profile:
    item.expires_after(3600)
    return await profiles.fetch(user_id)

CacheInterface

Bases: Protocol

Covers most caching needs in two calls: fetch-or-compute, and forget.

The type code that caches should depend on. Instead of checking for a value, computing it on a miss and saving it — three steps, with a race between each — a caller hands :meth:get the function that computes the value, and the cache decides when to call it:

profile = await cache.get(f"profile.{user_id}", load_profile)

Deciding in one place is what lets an implementation protect a backend from a stampede — many callers missing the same key at once, all computing it — by computing it once and sharing the result, or by refreshing a value shortly before it expires.

get async

get(key: str, callback: Callback[T], /, *, beta: float | None = None, metadata: Metadata | None = None) -> T

Return the value cached under key, computing and saving it on a miss.

On a miss callback is awaited with the item for key; what it returns is saved and returned. A value that cannot be saved is still returned — the cache is an optimisation, not a dependency — and reported through metadata.

On a hit the stored value is returned as the type callback would have returned: keep one key for one type.

Parameters:

  • key (str) –

    The key the value is stored under.

  • callback (Callback[T]) –

    Computes the value on a miss.

  • beta (float | None, default: None ) –

    How eagerly to recompute a value before it expires. The chance grows as expiry nears, and faster the larger beta is. 0 disables early recomputation; math.inf forces recomputation now. None lets the implementation choose, 1.0 being the usual choice.

  • metadata (Metadata | None, default: None ) –

    A mapping to fill with the metadata of the value — see :class:~xtr_cache_contracts.metadata.Metadata — plus save_failed when a computed value could not be saved. The value's own keys are overwritten and save_failed is reset each call; any other key already in the mapping is left untouched.

Returns:

  • T –

    The cached value, or the one callback computed.

Raises:

delete async

delete(key: str) -> bool

Remove the value under key; a key holding nothing is fine.

Returns:

  • bool –

    False when the backend failed; True otherwise.

Raises:

CacheItemPoolInterface

Bases: Protocol

The item-level view of a cache, for code that needs more than fetch-or-compute.

Most code should depend on :class:~xtr_cache_contracts.cache_interface.CacheInterface. This is the level below it: a hit told apart from a miss, several keys read in one round trip, writes batched until :meth:commit. It is also the level a backend implements — :class:~xtr_cache_contracts.cache_mixin.CacheMixin builds the fetch-or-compute contract on top of it.

The rules:

  • A key is a non-empty string without any of :data:~xtr_cache_contracts.item_interface.RESERVED_CHARACTERS. Letters, digits, _ and . up to 64 characters work everywhere; a pool may accept more. A key it refuses raises :class:~xtr_cache_contracts.exception.InvalidArgumentError.
  • A backend failure never raises: the call returns False, or reads as a miss, and the implementation logs it. Code that caches keeps working when the cache does not.
  • A pool saves items of the kind it hands out, and refuses any other by returning False. Pools of one implementation may take each other's items, which is how one pool fills another.

get_item async

get_item(key: str) -> ItemInterface

Return the item for key, a hit or a miss.

Raises:

get_items async

get_items(keys: Iterable[str]) -> Mapping[str, ItemInterface]

Return an item for each of keys, hits and misses alike, in the order asked.

Raises:

has_item async

has_item(key: str) -> bool

Tell whether the pool holds a value for key.

Do not follow it with :meth:get_item: the value may expire in between. Read the item and ask it :meth:~xtr_cache_contracts.item_interface.ItemInterface.is_hit instead.

Raises:

clear async

clear(prefix: str = '') -> bool

Remove every item whose key starts with prefix; every item, by default.

Items saved as deferred and not yet committed are dropped too.

Returns:

  • bool –

    False when the backend failed; True otherwise.

delete_item async

delete_item(key: str) -> bool

Remove the item under key; a key holding nothing is fine.

Returns:

  • bool –

    False when the backend failed; True otherwise.

Raises:

delete_items async

delete_items(keys: Iterable[str]) -> bool

Remove the items under keys.

Returns:

  • bool –

    False when the backend failed for any of them; True

  • bool –

    otherwise.

Raises:

save async

save(item: ItemInterface) -> bool

Store item now.

Returns:

  • bool –

    False when the backend failed, or when item is of a kind

  • bool –

    the pool does not store; True otherwise.

save_deferred async

save_deferred(item: ItemInterface) -> bool

Queue item to be stored by the next :meth:commit.

Reading a queued item's key from this pool commits the queue first, so the pool never answers with a value older than one it was given. Nothing commits the queue on its own when the pool is discarded: call :meth:commit.

Returns:

  • bool –

    False when item is of a kind the pool does not store;

  • bool –

    True otherwise.

commit async

commit() -> bool

Store every item queued by :meth:save_deferred.

The queue is emptied whatever happens: an item the backend failed to store is dropped, not kept for a later commit. A cache may lose a write; the next read computes the value again.

Returns:

  • bool –

    False when the backend failed for any of them; True

  • bool –

    otherwise.

CacheMixin

Bases: CacheInterface, ABC

Implements :class:~xtr_cache_contracts.cache_interface.CacheInterface on a pool.

A class that implements :class:~xtr_cache_contracts.cache_item_pool_interface.CacheItemPoolInterface derives from this too and gets :meth:get and :meth:delete built on its own :meth:get_item, :meth:save and :meth:delete_item. List it after the pool's interfaces, so the pool's own methods are the ones found.

Early recomputation: a hit whose metadata says when it expires and how long it took to compute may be recomputed before it expires, with a chance that grows as expiry nears. Under load, one caller refreshes the value while the others still read the old one, rather than all of them missing at the same moment. A value stored without that metadata is only recomputed once it has expired. The time is read from the clock in force (:func:xtr_clock.now), so freezing it in a test freezes this too.

:meth:get is a template an implementation adjusts through three steps rather than rewrites: :meth:_compute — how a missing value is computed and saved, where concurrent misses can be made to share one computation or wait on a lock; :meth:_now — which clock is read; and :meth:_on_elected — what to do when a hit is elected for early recomputation, such as logging it.

get_item abstractmethod async

get_item(key: str) -> ItemInterface

Return the item for key, a hit or a miss.

save abstractmethod async

save(item: ItemInterface) -> bool

Store item now; False when that failed.

delete_item abstractmethod async

delete_item(key: str) -> bool

Remove the item under key; False when that failed.

get async

get(key: str, callback: Callback[T], /, *, beta: float | None = None, metadata: Metadata | None = None) -> T

Return the value under key, computing and saving it on a miss.

Raises:

delete async

delete(key: str) -> bool

Remove the value under key; False when the backend failed.

CacheError

Bases: Exception

Base class for every error raised through the caching contract.

Catch this to handle anything caching can go wrong with, whichever package raised it; catch a subclass to handle one cause. Every subclass carries the data a caller needs as typed attributes and composes its own message from them.

A backend failing is not one of those causes: a cache is an optimisation, so a pool reports a backend it cannot reach by returning False rather than by raising. What does raise is a mistake in the calling code — a key no pool accepts, a tag on an item that cannot carry one.

InvalidArgumentError

InvalidArgumentError(reason: str)

Bases: CacheError, ValueError

A cache was given a key, a tag or an option it cannot work with.

An empty key, one holding a reserved character, a negative beta. Raised where the argument is given, before anything reaches a backend.

Also a :class:ValueError, so code that already guards its input with except ValueError keeps working without learning a new exception.

Attributes:

  • reason (str) –

    What is wrong with the argument.

Record what is wrong with the argument.

ItemInterface

Bases: Protocol

A cache entry as a pool hands it out: looked up, maybe found, ready to save.

A pool returns an item for every key it is asked about, found or not; :meth:is_hit tells which. That is what lets None be a value like any other rather than a stand-in for "not cached".

An item is a plain object in memory. Changing it — :meth:set, :meth:expires_after, :meth:tag — reaches the backend only once the item is handed back to the pool it came from, through :meth:CacheItemPoolInterface.save <xtr_cache_contracts.cache_item_pool_interface.CacheItemPoolInterface.save> or, inside :meth:CacheInterface.get <xtr_cache_contracts.cache_interface.CacheInterface.get>, by the pool itself. The mutators return the item, so calls chain.

key property

key: str

The key the item is stored under.

metadata property

metadata: Metadata

What the pool stored alongside the value; empty when it stored nothing.

get

get() -> object

Return the value; None when the item is not a hit.

Typed object because a pool holds values of any type: narrow it where the type is known. :meth:CacheInterface.get <xtr_cache_contracts.cache_interface.CacheInterface.get> does that for you, through the type its callback returns.

is_hit

is_hit() -> bool

Tell whether the pool had a value for the key when the item was looked up.

set

set(value: object) -> Self

Set the value to save; the item is not saved until the pool is told to.

expires_at

expires_at(expiration: datetime | None) -> Self

Expire the item at expiration; None falls back to the pool's default.

expires_after

expires_after(ttl: float | timedelta | None) -> Self

Expire the item ttl from now, in seconds or as a duration.

A lifetime of zero or less expires the item at once: saving it removes whatever the pool held under its key. None falls back to the pool's default.

tag

tag(tags: str | Iterable[str]) -> Self

Add one tag, or several, to invalidate the item by later.

Tags follow the same rules as keys.

Raises:

  • InvalidArgumentError –

    When a tag is empty, holds a reserved character, or holds a non-printable character.

  • CacheError –

    When the item comes from a pool that cannot store tags.

Metadata

Bases: TypedDict

Facts about a cached value, every one of them optional.

A pool that records none returns an empty mapping. The keys a pool fills in are what makes early recomputation possible: knowing when a value expires and how long it took to compute is enough to refresh it shortly before it runs out, instead of making every caller wait once it has.

Attributes:

  • expiry (float) –

    When the value expires, as a Unix timestamp in seconds. Wall clock rather than monotonic, because the value is shared with other processes and other machines.

  • ctime (int) –

    How long the value took to compute, in milliseconds.

  • tags (Sequence[str]) –

    The tags the value was saved with.

  • save_failed (bool) –

    Set by :meth:CacheInterface.get <xtr_cache_contracts.cache_interface.CacheInterface.get> when the value it computed could not be saved. Never stored, so never in :attr:ItemInterface.metadata <xtr_cache_contracts.item_interface.ItemInterface.metadata>.

NamespacedPoolInterface

Bases: Protocol

Confines keys to a sub-namespace, so a group of them can be dropped at once.

Keys written through the returned pool are prefixed with the backend's own namespace separator, so clearing that namespace invalidates them together without listing them. Tags ignore sub-namespaces: invalidating a tag reaches a value whichever sub-namespace it was saved in.

with_sub_namespace

with_sub_namespace(namespace: str) -> Self

Return a pool whose keys live under namespace, inside this pool's namespace.

This pool is left as it was; the returned one is new.

Raises:

TagAwareCacheInterface

Bases: CacheInterface, Protocol

A cache that can drop every value carrying a tag, without knowing their keys.

A value is tagged when it is computed, through the item its callback receives:

async def load_invoice(item: ItemInterface) -> Invoice:
    item.tag(f"customer.{customer_id}")
    return await invoices.fetch(invoice_id)

Invalidating customer.42 later drops every invoice cached for that customer, whatever it was cached under.

invalidate_tags async

invalidate_tags(tags: Iterable[str]) -> bool

Invalidate every value tagged with any of tags.

An implementation built on an item pool does not invalidate items saved as deferred and not yet committed; they are committed as usual. That lets a caller replace old tagged values with new ones without a window where neither is cached.

Returns:

  • bool –

    False when the backend failed; True otherwise.

Raises: