xtr_lock ¶
Exclusive and shared locks around resources, in whatever store fits.
A :class:~xtr_lock.lock_factory.LockFactory built around a store hands out
:class:~xtr_lock.lock.Lock objects, one per resource. The store decides
who else a lock excludes: tasks in this process
(:class:~xtr_lock.store.InMemoryStore), processes on this machine
(:class:~xtr_lock.store.FlockStore), anything that reaches a Redis server
(:class:~xtr_lock.store.RedisStore), or a quorum of several stores
(:class:~xtr_lock.store.CombinedStore).
Every call that touches a store is awaited.
DEFAULT_TTL
module-attribute
¶
Five minutes: how long a lock is expected to be held unless told otherwise.
BlockingSharedLockStoreInterface ¶
Bases: SharedLockStoreInterface, Protocol
A store that is told when a writer lets go, instead of asking over and over.
Without it, a blocking :meth:~xtr_lock.lock.Lock.acquire_read retries
:meth:save_read every tenth of a second.
wait_and_save_read
async
¶
Wait until no writer holds the lock, then take it for reading for key.
BlockingStoreInterface ¶
Bases: PersistingStoreInterface, Protocol
A store that is told when a lock comes free, instead of asking over and over.
Without it, a blocking :meth:~xtr_lock.lock.Lock.acquire retries
:meth:save every tenth of a second.
wait_and_save
async
¶
Wait until the lock is free, then take it for key.
InvalidArgumentError ¶
Bases: LockError, ValueError
A lock, a store or a factory was given an argument it cannot work with.
Raised where the argument is given — a lock directory that cannot be written, a connection no store knows, a refresh with no duration — rather than on the first attempt to lock.
Also a :class:ValueError, so code that already guards its configuration
with except ValueError keeps working without learning a new exception.
Attributes:
-
reason(str) –What is wrong with the argument.
Record what is wrong with the argument.
InvalidTtlError ¶
Bases: InvalidArgumentError
A store was given a lifetime it cannot keep a lock for.
A store whose locks expire on their own needs a lifetime greater than zero: a lock that expires the moment it is taken protects nothing.
Attributes:
-
ttl(float) –The lifetime, in seconds, that was refused.
Record the lifetime that was refused.
LockAcquiringError ¶
Bases: LockError, RuntimeError
Acquiring a lock, or extending one, failed for a reason other than contention.
The store could not be reached, or answered with an error. The underlying
error is chained as __cause__. Contention is not a failure of this
kind: a lock someone else holds is
:class:~xtr_lock.exception.LockConflictedError, and a lock whose
lifetime ran out before the store confirmed it is
:class:~xtr_lock.exception.LockExpiredError.
Attributes:
-
resource(str) –The resource the lock is for.
-
reason(str) –What was being attempted when it failed.
Record the resource and what was being attempted.
LockConflictedError ¶
Bases: LockError, RuntimeError
Someone else holds the lock.
The one signal a store gives for contention. A non-blocking
:meth:~xtr_lock.lock.Lock.acquire turns it into False; a blocking
one waits it out; :meth:~xtr_lock.lock.Lock.refresh lets it through,
since a lock that can no longer be extended has been lost.
Attributes:
-
resource(str) –The resource someone else holds.
Record the resource someone else holds.
LockError ¶
Bases: Exception
Base class for every error raised by this library.
Catch this to handle anything locking can go wrong with; catch a subclass to handle one cause. Every subclass carries the data a caller needs as typed attributes and composes its own message from them.
LockExpiredError ¶
Bases: LockError, RuntimeError
A lock's lifetime ran out before the store confirmed it.
Storing the lock took longer than the lock was meant to live — a slow network, several stores in a row — so by the time it was stored it had already expired. The store has been asked to forget it again.
Attributes:
-
resource(str) –The resource the lock is for.
Record the resource whose lock expired.
LockReleasingError ¶
Bases: LockError, RuntimeError
Releasing a lock failed, or the lock was still there afterwards.
A store's delete can fail outright — the underlying error is then
chained as __cause__ — or appear to succeed while the lock remains,
which is checked for rather than trusted.
Attributes:
Record the resource and what went wrong.
LockStorageError ¶
Bases: LockError, RuntimeError
The storage behind a store failed.
Not a question of who holds the lock: the lock file could not be opened,
the server answered a command with an error. Whatever the storage said is
kept in :attr:reason, and its own error, when there is one, is chained
as __cause__.
Attributes:
-
reason(str) –What the storage reported.
Record what the storage reported.
UnserializableKeyError ¶
Bases: LockError, RuntimeError
A key holding something that cannot leave the process was pickled.
Some stores keep an operating system handle on the key — an open file descriptor, for one — which means nothing to another process. Once such a store has touched a key, the key refuses to be pickled rather than arrive somewhere else claiming a lock it cannot hold.
Attributes:
-
resource(str) –The resource the key is for.
Record the resource the key is for.
Key ¶
Names a resource and carries whatever a store needs to remember about holding it.
Two keys for the same resource are two different holders: a store tells them apart by the state it stashes on each — a random token, an open file — never by the resource name alone. Reusing the same key is therefore how a holder says "this is still me".
The key also tracks when the lock it stands for expires. Every store that
sets a lifetime can only shorten it (:meth:reduce_lifetime), so a lock
held in several places lives no longer than the shortest-lived of them.
A lifetime is a duration, so it is measured on a monotonic clock unless
another is given: setting the system time back cannot keep a lock alive.
Name resource.
Parameters:
-
resource(str) –What the lock protects. Any string; stores that need a narrower alphabet derive one from it.
-
clock(ClockInterface | None, default:None) –Where the lifetime is measured.
Nonecounts on a monotonic clock; hand in a frozen one to test expiry.
has_state ¶
Tell whether a store has stashed something under state_key.
set_state ¶
Stash state under state_key, replacing what was there.
Each store uses its own class as state_key, so two stores working
on the same key never overwrite each other.
remove_state ¶
Forget what was stashed under state_key; nothing there is fine.
get_state ¶
mark_unserializable ¶
Refuse to be pickled from now on.
Called by a store that keeps something on the key only this process can use. There is no way back: the thing may be gone, but the key cannot know that.
reset_lifetime ¶
Forget the expiry, so the next store to set one starts from nothing.
reduce_lifetime ¶
Expire in ttl seconds, unless the key already expires sooner.
Only ever shortens: a lock several stores hold expires with the first of them to let it go.
get_remaining_lifetime ¶
Return the seconds left before expiry, negative once past, None when unset.
Lock ¶
Lock(key: Key, store: PersistingStoreInterface, ttl: float | None = None, *, clock: ClockInterface | None = None)
Bases: LoggerAware, SharedLockInterface
Takes, extends and lets go of the lock a :class:~xtr_lock.key.Key names, in a store.
Build one through :class:~xtr_lock.lock_factory.LockFactory rather than
by hand, so the store, the clock and the logger come from one place.
Asked to wait, a lock uses the store's own way of waiting when it has one, and otherwise retries every tenth of a second, give or take ten milliseconds so that two waiters do not retry in step.
Given a lifetime, a lock sets it the moment it is taken, and a lock whose lifetime ran out while it was being stored is let go again and reported rather than returned as held.
The key is the holder. Every lock made for one key — this one, shared between tasks, or another made from the same key — acts for that one holder: acquiring what the holder already holds succeeds at once, and one release lets go for all of them. What they do to the store takes turns, so two tasks never interleave half-done operations on one holder. To exclude another task, give it a lock of its own.
An acquire that is cancelled — by a timeout, say — takes back whatever the store may have stored for it, unless the holder already held the lock before it started.
async with lock: waits for the lock and lets it go on the way out;
a lock is never let go just because it is no longer referenced.
A lock cannot be pickled: it holds a store, and possibly an open file, that mean nothing anywhere else.
Lock the resource key names, in store.
Parameters:
-
key(Key) –The holder's identity. Reuse it to act as the same holder.
-
store(PersistingStoreInterface) –Where the lock is kept.
-
ttl(float | None, default:None) –The longest the lock is expected to be held, in seconds;
Noneor zero sets no lifetime. -
clock(ClockInterface | None, default:None) –What a retrying wait sleeps on.
Nonesleeps on the clock in force.
acquire
async
¶
Take the lock for writing; see :meth:LockInterface.acquire.
acquire_read
async
¶
Take the lock for reading; see :meth:SharedLockInterface.acquire_read.
refresh
async
¶
Push the expiry back; see :meth:LockInterface.refresh.
is_acquired
async
¶
Ask the store, and remember the answer; see :meth:LockInterface.is_acquired.
release
async
¶
Let the lock go, and check it is gone; see :meth:LockInterface.release.
is_expired ¶
Tell whether the lifetime ran out; see :meth:LockInterface.is_expired.
get_remaining_lifetime ¶
Return the seconds left; see :meth:LockInterface.get_remaining_lifetime.
LockFactory ¶
Bases: LoggerAware
Makes :class:~xtr_lock.lock.Lock objects on one store.
The one object an application usually needs: build it once around a
store, then ask it for a lock per resource. Every lock it makes logs
through the logger it was given with :meth:set_logger.
Make locks kept in store.
Parameters:
-
store(PersistingStoreInterface) –Where every lock this factory makes is kept.
-
clock(ClockInterface | None, default:None) –What lifetimes are measured and waits sleep on.
Nonereads the clock in force.
create_lock ¶
Make a lock on resource, as a new holder.
Parameters:
-
resource(str) –What the lock protects.
-
ttl(float | None, default:DEFAULT_TTL) –The longest the lock is expected to be held, in seconds.
Nonesets no lifetime, so a store whose locks expire keeps its own default.
create_lock_from_key ¶
Make a lock for key, as the holder that key already is.
Parameters:
-
key(Key) –The holder to act as, typically one kept from an earlier lock.
-
ttl(float | None, default:DEFAULT_TTL) –The longest the lock is expected to be held, in seconds.
LockInterface ¶
Bases: Protocol
An exclusive lock on one resource.
Depend on this rather than on :class:~xtr_lock.lock.Lock, and switching
locking off becomes a matter of handing in a
:class:~xtr_lock.no_lock.NoLock.
acquire
async
¶
Take the lock.
Parameters:
-
blocking(bool, default:False) –Wait until the lock is free instead of giving up at once.
Returns:
-
bool–Trueonce the lock is held;Falsewhen it is held by -
bool–someone else and
blockingis false.
Raises:
-
LockConflictedError–When waiting was asked for but the store gave up.
-
LockExpiredError–When the lifetime the lock carries ran out while the store was taking it. The lock is let go again first, so nothing is held when this arrives.
-
LockAcquiringError–When the store failed for any other reason.
refresh
async
¶
Push the expiry back to ttl seconds from now.
Parameters:
-
ttl(float | None, default:None) –The new lifetime;
Nonereuses the one the lock was made with.
Raises:
-
InvalidArgumentError–When no lifetime is given here or on the lock.
-
LockConflictedError–When someone else holds the lock now.
-
LockExpiredError–When the lifetime ran out while the store was pushing the expiry back, which leaves the lock let go rather than extended.
-
LockAcquiringError–When the store failed for any other reason.
is_acquired
async
¶
Ask the store whether this lock is still held by its holder.
release
async
¶
Let the lock go.
Raises:
-
LockReleasingError–When the store failed to let it go, or still holds it afterwards.
get_remaining_lifetime ¶
Return the seconds left, negative once past, None when no lifetime is set.
NoLock ¶
Bases: SharedLockInterface
Says yes to everything and locks nothing.
Hand it to code that takes a :class:~xtr_lock.lock_interface.LockInterface
to switch locking off there without a branch in that code. Every acquire
succeeds, the lock always reports itself held and never expires, and
refreshing or releasing it does nothing.
PersistingStoreInterface ¶
Bases: Protocol
Where locks are kept.
The store-side contract: a :class:~xtr_lock.lock.Lock decides when to
lock, a store decides how. A store that can also wait, or share, adds
the matching interface; :class:~xtr_lock.lock.Lock looks for them and
uses the best one available.
save
async
¶
Take the lock for key, without waiting.
Saving a key that already holds the lock is not an error.
Raises:
-
LockConflictedError–When another key holds the lock.
delete
async
¶
Let go of the lock key holds; a key holding nothing is fine.
put_off_expiration
async
¶
Keep the lock key holds for another ttl seconds.
A store whose locks never expire does nothing.
Raises:
-
LockConflictedError–When
keyno longer holds the lock.
SharedLockInterface ¶
Bases: LockInterface, Protocol
A lock with a shared, read-only mode next to the exclusive one.
Any number of holders may read at once; a writer excludes everyone. A sole reader may become the writer, and a writer may step down to a reader, without letting go in between.
acquire_read
async
¶
Take the lock for reading.
Falls back to the exclusive lock when the store cannot share.
Parameters:
-
blocking(bool, default:False) –Wait until the lock can be shared instead of giving up at once.
Returns:
Raises:
-
LockConflictedError–When waiting was asked for but the store gave up.
-
LockAcquiringError–When the store failed for any other reason.
SharedLockStoreInterface ¶
Bases: PersistingStoreInterface, Protocol
A store with a shared mode next to the exclusive one.
:meth:save on a key already reading promotes it to the writer, when it
is the only reader; :meth:save_read on the writer demotes it.
save_read
async
¶
Take the lock for reading for key, without waiting.
Raises:
-
LockConflictedError–When another key holds the lock for writing.
CombinedStore ¶
CombinedStore(stores: Iterable[PersistingStoreInterface], strategy: StrategyInterface, *, clock: ClockInterface | None = None)
Bases: LoggerAware, SharedLockStoreInterface, ExpiringStoreMixin
Holds each lock in every one of several stores, and lets a strategy decide.
With independent servers and a
:class:~xtr_lock.strategy.ConsensusStrategy, a lock survives any
minority of them failing; with a
:class:~xtr_lock.strategy.UnanimousStrategy, it is held only where all
of them hold it. The stores are asked in order, and asking stops as soon
as the strategy can no longer be met. A lock that falls short is taken
back out of every store before the conflict is reported.
A lifetime is a deadline, not a duration handed to each store: each store gets what is left of it when its turn comes, so the lock never outlives the moment it was meant to end.
There has to be at least one store. A combination of none would hand every strategy a count of nothing out of nothing, which every strategy reads as met: a store that reports every lock taken and keeps none. That is refused where it is built rather than discovered the first time two holders run at once.
Reading falls back to writing on any store that cannot share.
:meth:close closes every store that has something to close, such as a
Redis store that opened its own connection.
Keep every lock in each of stores.
Parameters:
-
stores(Iterable[PersistingStoreInterface]) –The stores to keep each lock in, asked in this order.
-
strategy(StrategyInterface) –How many of them must agree.
-
clock(ClockInterface | None, default:None) –What the lifetime deadline is measured on.
Nonecounts on a monotonic clock.
Raises:
-
InvalidArgumentError–When
storesis empty, or when something in it is not a store.
save
async
¶
Take the lock for writing in every store, until the strategy is met or cannot be.
save_read
async
¶
Take the lock for reading in every store, for writing where a store cannot share.
put_off_expiration
async
¶
Extend the lock in every store, each up to the same deadline ttl seconds away.
exists
async
¶
Ask the stores in turn, until the strategy is met or cannot be.
close
async
¶
Close every store that has something to close, and leave the rest alone.
Raises:
-
ExceptionGroup–What the stores that failed to close raised, once every store has been asked.
ExpiringStoreMixin ¶
For a store that sets a lifetime on the locks it keeps.
Storing a lock takes time — a round trip, several stores in a row — and
that time comes out of the lock's lifetime. A store calls
:meth:_check_not_expired once it is done: a lock that expired on the way
is taken back out of the store rather than reported as held.
List it after the store's interfaces, so the store's own :meth:delete
is the one found.
FlockStore ¶
Bases: BlockingStoreInterface, BlockingSharedLockStoreInterface
Holds each lock as an advisory lock on a file of its own.
Every resource maps to one file under the lock directory, and every key
opens its own descriptor on it, so two keys conflict whether they live in
two processes or in the same one. The operating system lets go of the
lock when the descriptor is closed — on :meth:delete, or when the
process ends, however it ends — so a crashed holder never leaves a lock
behind. Nothing else expires: a lifetime given to the lock is ignored.
A key dropped while it holds a lock has its descriptor closed when it is collected, which lets go of the lock: the descriptor is the store's to tidy, and a worker that loses keys must not run out of descriptors. That is no release to rely on — release the lock.
What a key holds is kept under this store, not under the class: two stores on two directories are two locks, and one key may hold both.
Waiting for a lock tries again without blocking, sooner at first and then every tenth of a second, sleeping on the event loop in between: no thread is ever parked in the system call, and a wait that is cancelled holds nothing. Waiters are not served in the order they arrived — the operating system does not promise that either.
The locks are advisory and local: they bind only processes that use this store on the same machine, and a directory on a network file system may not honour them at all. POSIX only.
The lock directory belongs to one user: without a directory of its own,
the store keeps its files in one made for this user alone under the
temporary directory, where anyone could otherwise create or hold a lock
file first; a directory it is given must be owned by this user too, and a
directory it creates is made private (0o700). The owner opens it up —
the directory's and the files' permissions — to share it with others: a
lock file is created readable and writable by everyone, and never opened
through a symbolic link.
Keep lock files in lock_path, created when missing.
Parameters:
-
lock_path(str | PathLike[str] | None, default:None) –The directory for the lock files.
Noneuses one private to this user in the system's temporary directory. -
clock(ClockInterface | None, default:None) –What a wait sleeps on between attempts.
Nonesleeps on the clock in force.
Raises:
-
InvalidArgumentError–When the directory cannot be created or written to, it belongs to another user, or the platform has no file locks.
wait_and_save_read
async
¶
Wait until no writer holds the lock, then take it for reading.
put_off_expiration
async
¶
Check key still holds the lock; a file lock is held until it is released.
Raises:
-
LockConflictedError–When
keydoes not hold the lock.
delete
async
¶
Let go of the lock and close the descriptor; a key holding nothing is fine.
file_for ¶
Return the file that holds the lock for key's resource.
The name keeps up to fifty characters of the resource, anything
outside A-Za-z0-9._- folded to -, and adds a short digest so
two resources that fold alike still get two files.
InMemoryStore ¶
Bases: SharedLockStoreInterface
Keeps locks in a dictionary, for coordinating tasks inside one process.
A holder is told apart by a random token the store stashes on its key the first time it sees it. Nothing ever expires here: a lock is held until it is released, whatever lifetime it was given.
Every method completes without awaiting anything, so on one event loop no two calls interleave. The store is not meant to be shared between threads.
Start with no locks.
save_read
async
¶
Take the lock for reading; the writer may demote itself.
put_off_expiration
async
¶
Check key still holds the lock; a lock in memory is held until it is released.
Raises:
-
LockConflictedError–When
keydoes not hold the lock.
NullStore ¶
Bases: BlockingStoreInterface, BlockingSharedLockStoreInterface
Accepts every lock at once and remembers none of them.
Switches locking off at the store: every save succeeds without waiting,
and :meth:exists always answers False, so a lock on it never reports
itself held. To switch locking off where a lock is handed in instead, use
:class:~xtr_lock.no_lock.NoLock.
wait_and_save
async
¶
Do nothing: the lock is free at once, so there is nothing to wait for.
RedisStore ¶
RedisStore(redis: Redis, initial_ttl: float = 300.0, *, prefix: str = '', clock: ClockInterface | None = None)
Bases: SharedLockStoreInterface, ExpiringStoreMixin
Keeps locks on a Redis server, with a lifetime the server enforces.
Each resource is a sorted set named after it: one member per holder, scored with the moment its hold expires, plus a marker member while the holder writes. Every change is one script, so it happens whole on the server. The server's own clock decides expiry when scripts may read it; otherwise this process's clock is sent along.
A lock is stored with :attr:initial_ttl of life, which the lock's own
lifetime then replaces. A lock is never held forever: a holder that dies
loses it when the lifetime runs out.
The Redis key is the resource, after :attr:prefix — set one when the
database is shared with anything else.
Works with Valkey too. One server, not a cluster.
Keep locks on the server redis talks to.
The client stays the caller's to close.
Parameters:
-
redis(Redis) –An asyncio client.
-
initial_ttl(float, default:300.0) –How long a lock lives once stored, before the lock's own lifetime is set, in seconds.
-
prefix(str, default:'') –Put in front of every resource to make its Redis key.
-
clock(ClockInterface | None, default:None) –What this process sends as the time, when the server cannot read its own.
Nonereads the clock in force.
Raises:
-
InvalidTtlError–When
initial_ttlis not strictly positive.
MISSING_CLIENT
class-attribute
¶
What a Redis DSN is refused with when the client library is not installed.
initial_ttl
property
¶
How long a lock lives once stored, before the lock's own lifetime is set.
from_url
classmethod
¶
from_url(dsn: str, initial_ttl: float = 300.0, *, clock: ClockInterface | None = None) -> RedisStore
Connect to the server dsn names, on first use, and own the connection.
Close the store with :meth:close when done.
Parameters:
-
dsn(str) –redis://,rediss://,unix://, orvalkey://andvalkeys://, which read as the first two. Aprefixquery option (redis://host?prefix=app:) sets :attr:prefix; every other option goes to the client. -
initial_ttl(float, default:300.0) –See :meth:
__init__. -
clock(ClockInterface | None, default:None) –See :meth:
__init__.
Raises:
-
InvalidArgumentError–When the scheme is not one of those, or the
redisextra is not installed.
create_connection
staticmethod
¶
Return an asyncio client for dsn that connects on first use.
A prefix option in the DSN is this store's, not the client's, and
is left out.
Raises:
-
InvalidArgumentError–When the scheme is not a Redis one, or the
redisextra is not installed.
close
async
¶
Close the connection, when :meth:from_url opened it; otherwise do nothing.
save_read
async
¶
Take the lock for reading; the writer may demote itself.
put_off_expiration
async
¶
Keep the lock for another ttl seconds.
delete
async
¶
Let go of whatever key holds; a key holding nothing is fine.
exists
async
¶
Tell whether key holds the lock, for reading or writing, and has not expired.
StoreFactory ¶
Turns what a configuration names into a store.
| Given | Store |
|---|---|
"flock" |
:class:FlockStore in the system's temporary directory |
"flock:///var/lock/app" |
:class:FlockStore in that directory |
"in-memory" |
a new :class:InMemoryStore |
"null" |
:class:NullStore |
"redis://…", "rediss://…", "unix://…" |
:class:RedisStore owning a connection |
"valkey://…", "valkeys://…" |
the same, read as redis / rediss |
| an asyncio Redis client | :class:RedisStore on that client |
create_store
staticmethod
¶
Build the store connection names.
Raises:
-
InvalidArgumentError–When no store serves
connection. The message names its scheme or type only, never credentials.
validate
staticmethod
¶
Refuse a DSN no store serves, without building a store or connecting to anything.
Raises:
-
InvalidArgumentError–When no store serves
connection, or it needs an extra that is not installed. The message names its scheme only, never credentials.
ConsensusStrategy ¶
Bases: StrategyInterface
Holds a lock once a strict majority of the stores do.
Survives the loss of any minority of stores, which is what makes several independent servers safer than one.
StrategyInterface ¶
Bases: Protocol
Decides whether a lock held in some of several stores is held.
UnanimousStrategy ¶
Bases: StrategyInterface
Holds a lock only when every store does; one refusal is enough to give up.