xtr_orm ¶
Databases for async applications: engines, sessions and versioned schema migrations.
A :class:~xtr_orm.migrations.Migrator runs one database's revisions, a
:class:~xtr_orm.database.DatabaseManager creates and drops the database
itself, and a :class:ConnectionRegistry names every connection an
application has. The bundle in :mod:xtr_orm.bundle builds all of them, and
the engines and sessions, from one configuration — and a library bringing its
own tables hands them over as a :class:MappingDriverInterface.
DEFAULT_CONNECTION
module-attribute
¶
The name of the connection used when none is named.
ConnectionRegistry ¶
Every connection by name: its engine, its migrator and its database manager.
Each is given as the object itself, or as an async function building it — called only when that piece is first asked for, so a command touching one connection opens nothing of another's.
A connection may also say how to reach the session of the current unit of work — the one every handler of a message shares — and how to close it, releasing every connection it holds; the message bus middleware use both.
registry = ConnectionRegistry()
registry.register(
"default",
engine=engine,
migrator=Migrator(engine, MigrationsConfig(directory="migrations")),
database=DatabaseManager(url),
)
Hold connections, answering for default when none is named.
register ¶
register(name: str, *, engine: AsyncEngine | Callable[[], Awaitable[AsyncEngine]], migrator: Migrator | Callable[[], Awaitable[Migrator]], database: DatabaseManager | Callable[[], Awaitable[DatabaseManager]], session: Callable[[], Awaitable[AsyncSession]] | None = None, close: Callable[[], Awaitable[None]] | None = None, in_use: Callable[[], bool] | None = None) -> None
Add the connection name, or replace it.
Parameters:
-
name(str) –What the connection is known by.
-
engine(AsyncEngine | Callable[[], Awaitable[AsyncEngine]]) –Its engine, or what builds it.
-
migrator(Migrator | Callable[[], Awaitable[Migrator]]) –Its migrator, or what builds it.
-
database(DatabaseManager | Callable[[], Awaitable[DatabaseManager]]) –Its database manager, or what builds it.
-
session(Callable[[], Awaitable[AsyncSession]] | None, default:None) –What returns the session of the unit of work under way; without it, :meth:
sessionis refused. -
close(Callable[[], Awaitable[None]] | None, default:None) –What closes the connection; its engine is disposed when left out.
-
in_use(Callable[[], bool] | None, default:None) –Whether the connection is in use — its engine built; without it, the connection always is. See :meth:
in_useand :meth:close.
Raises:
-
InvalidArgumentError–When
nameis empty.
in_use ¶
The names of the connections in use, in the order registered.
What the message bus middleware act on unless given names: a connection nothing has used has no session to check and no pool to close, and opening it to find out would read its configuration.
has ¶
Whether the connection name — the default one when None — exists.
engine
async
¶
Return the engine of the connection name, the default one when None.
Raises:
-
UnknownConnectionError–When no connection has that name.
migrator
async
¶
Return the migrator of the connection name, the default one when None.
Raises:
-
UnknownConnectionError–When no connection has that name.
database
async
¶
Return the database manager of the connection name, the default one when None.
Raises:
-
UnknownConnectionError–When no connection has that name.
session
async
¶
Return the session of the current unit of work on the connection name.
Raises:
-
UnknownConnectionError–When no connection has that name.
-
SessionUnavailableError–When the connection was registered without a way to reach one.
close
async
¶
Close the connection name: release every database connection its pools hold.
The engine stays usable; its next use connects again.
A connection nothing has used — one whose in_use says so — is left
alone: it holds nothing to release, and building its engine to dispose
it would read its configuration and connect.
Raises:
-
UnknownConnectionError–When no connection has that name.
DatabaseManager ¶
Creates and drops the database a connection URL names, on its server.
A database cannot be created from a connection to itself, so the server
is reached through its maintenance database — postgres on
PostgreSQL, master on SQL Server, none on MySQL and MariaDB — with
every statement committed as it runs. A SQLite database is its file.
manager = DatabaseManager("postgresql+asyncpg://app:secret@db/shop")
await manager.create(if_not_exists=True)
Manage the database url names, connecting with connect_args.
Raises:
-
InvalidArgumentError–When the URL names no database on a server.
exists
async
¶
Whether the database exists.
Raises:
-
UnsupportedDatabaseError–For a server this library cannot ask.
create
async
¶
Create the database; return whether it was created.
Parameters:
-
if_not_exists(bool, default:False) –Leave an existing database alone and return
False, rather than let the server refuse.
Raises:
-
UnsupportedDatabaseError–For a server this library cannot ask.
drop
async
¶
Drop the database; return whether it was dropped.
Parameters:
-
if_exists(bool, default:False) –Do nothing for a database that does not exist and return
False, rather than let the server refuse.
Raises:
-
MissingDatabaseError–When the SQLite file is not there and
if_existswas not asked for. -
UnsupportedDatabaseError–For a server this library cannot ask.
InvalidArgumentError ¶
Bases: OrmError, ValueError
A configuration or a call was given something the library cannot work with.
Raised where the value is given — a connection without a name, a version table without a name — rather than when a database is first reached.
Also a :class:ValueError, so code that already guards its configuration
with except ValueError keeps working without learning a new exception.
Attributes:
-
reason(str) –What is wrong with the value.
Record what is wrong with the value.
MigrationError ¶
Bases: OrmError
A migration operation was refused, or its revisions could not be read.
Raised before anything is changed whenever it can be decided up front: a version that no revision answers to, a revision run out of order, a diff against a database that is behind its revisions. Nothing is written when it is raised from a check; a failing revision rolls back what the database lets it.
Attributes:
-
reason(str) –Why the operation was refused.
Record why the operation was refused.
MissingDatabaseError ¶
Bases: OrmError, FileNotFoundError
A database was dropped that is not there — a SQLite file that does not exist.
Also a :class:FileNotFoundError, as the missing file it stands for is.
Ask for if_exists to leave a database that does not exist alone and be
told so instead.
Attributes:
-
database(str) –The database's name — for SQLite, its file.
-
operation(str) –What was asked, such as
"drop".
Record which database was asked for what.
OrmError ¶
Bases: Exception
Base class for every error raised by this library.
Catch this to handle anything the library can refuse; catch a subclass to handle one cause. Every subclass carries the data a caller needs as typed attributes and composes its own message from them.
SessionUnavailableError ¶
Bases: OrmError
The session of the current unit of work was asked for where there is none.
A connection reaches the session everything in a unit of work shares — a message's handlers, say — only inside one, and only when it was registered with a way to.
Attributes:
Record which connection was asked, and why it has no session to give.
UnknownConnectionError ¶
Bases: OrmError, LookupError
A connection was asked for by a name nothing registered.
Also a :class:LookupError, as a missing key in any other mapping is.
Attributes:
Record the name asked for and the ones that exist.
UnsupportedDatabaseError ¶
Bases: OrmError
A database server was asked for something this library cannot do on it.
Creating or dropping a database takes a statement of the server's own; a server whose statement is not known here is refused rather than sent a guess.
Attributes:
-
backend(str) –The kind of server the URL names, such as
"oracle". -
operation(str) –What was asked, such as
"create".
Record which server was asked for what.
MappingDriverInterface ¶
Bases: Protocol
Tables a library builds once its own configuration is read.
A library shipping tables an application names — a prefix, which of them it keeps, which column type an identifier gets — cannot declare them when it is imported: it knows them only once configured. A driver is that build behind one call, so the tables can join the definitions of whichever connection the application picked.
load_metadata ¶
Add this driver's tables to metadata, and return the names added.
Must be idempotent. The definitions a connection migrates towards
are often shared for the whole process — the ones advanced-alchemy
registers per bind key are — so a driver may be loaded into the same
MetaData twice: by a second kernel in one test run, or by a second
library mapping the same table. Skip every name already in
metadata.tables and leave it out of what is returned, so the
second load changes nothing and says so.
The argument is passed positionally, so an implementation may name it whatever reads best.
AvailableMigration
dataclass
¶
AvailableMigration(version: str, description: str, down_versions: tuple[str, ...], path: str, is_head: bool)
A revision file, as it was found in the migrations directory.
Attributes:
-
version(str) –The revision's identifier.
-
description(str) –The first line of its message.
-
down_versions(tuple[str, ...]) –The revisions it follows; empty for a first revision, two or more for one merging branches.
-
path(str) –Where its file is.
-
is_head(bool) –Whether no revision follows it.
ExecutedMigration
dataclass
¶
ExecutedMigration(version: str, executed_at: datetime | None = None, execution_time: float | None = None)
A revision the database counts as applied.
Applied means recorded in the version table, preceding a revision that is, or recorded in the history table. When and how long come from the history table: a revision applied before it existed has neither, and one only marked as applied has no duration.
Attributes:
-
version(str) –The revision's identifier.
-
executed_at(datetime | None) –When it was applied, by the database's clock — with a time zone where the column keeps one (PostgreSQL), naive elsewhere: UTC on SQLite, the session's time zone on MySQL.
-
execution_time(float | None) –How long it took, in seconds.
ExecutionResult
dataclass
¶
MigrationPlan
dataclass
¶
MigrationsConfig
dataclass
¶
MigrationsConfig(directory: str = 'migrations', version_table: str = 'alembic_version', version_table_schema: str | None = None, history_table: str = 'alembic_version_history', file_template: str = '{year}{month}{day}{hour}{minute}{second}_{rev}_{slug}', truncate_slug_length: int = 40, timezone: str | None = None, template_directory: str | None = None, recursive_version_locations: bool = False, post_write_hooks: Sequence[Mapping[str, str]] = (), transaction_per_migration: bool = False, transactional_ddl: bool | None = None, compare_type: bool = False, compare_server_default: bool = False, render_as_batch: bool = False, include_schemas: bool = False, include_name: Callable[..., bool] | None = None, include_object: Callable[..., bool] | None = None, render_item: Callable[..., str | bool] | None = None, process_revision_directives: Callable[..., None] | None = None, user_module_prefix: str | None = None, context_options: Mapping[str, object] = _no_options())
Where a database's revisions live, how new ones are written, and how runs are recorded.
Revisions are ordinary migration files, one per change, in
:attr:directory. The database keeps its current revisions in
:attr:version_table, and a row per revision ever applied — when, and how
long it took — in :attr:history_table.
MigrationsConfig(
directory="%kernel.project_dir%/migrations",
version_table_schema="meta",
render_as_batch=True, # SQLite cannot alter a table in place
)
Attributes:
-
directory(str) –Where revision files are read from and written to; created when the first one is written. A relative path is relative to the working directory — under a kernel, start it with
%kernel.project_dir%. -
version_table(str) –The table holding the database's current revisions.
-
version_table_schema(str | None) –The schema of both tables; the connection's default when
None. -
history_table(str) –The table recording every revision applied, when it ran and how long it took.
-
file_template(str) –How a new revision's file is named.
{rev},{slug},{epoch},{year},{month},{day},{hour},{minute}and{second}are filled in; the default sorts files by the time they were written. -
truncate_slug_length(int) –The longest the message part of a file name gets.
-
timezone(str | None) –The time zone a revision's creation date is written in; local time when
None. -
template_directory(str | None) –A directory holding a
script.py.makothat new revisions are rendered from; the package's own whenNone. -
recursive_version_locations(bool) –Read revisions from subdirectories of :attr:
directorytoo. -
post_write_hooks(Sequence[Mapping[str, str]]) –Programs run on every file written, such as a formatter — each a mapping with a
type("console_scripts","exec"or"module") and that type's options. -
transaction_per_migration(bool) –Commit after each revision rather than once after all of them.
-
transactional_ddl(bool | None) –Whether schema changes run inside the transaction; the database's own answer when
None. -
compare_type(bool) –Compare column types when generating a diff. Off by default: a type the database stores as another — a UUID kept as bytes on SQLite — would otherwise differ in every diff.
-
compare_server_default(bool) –Compare server defaults when generating a diff.
-
render_as_batch(bool) –Write table changes as copy-and-move batches, which SQLite needs for anything but adding a column.
-
include_schemas(bool) –Compare every schema when generating a diff, not only the default one.
-
include_name(Callable[..., bool] | None) –Decides, by name, which schemas and tables a diff reads from the database.
-
include_object(Callable[..., bool] | None) –Decides which tables, columns, indexes and constraints a diff considers.
-
render_item(Callable[..., str | bool] | None) –Renders a type or a construct a diff writes, before the default rendering.
-
process_revision_directives(Callable[..., None] | None) –Adjusts a revision about to be written.
-
user_module_prefix(str | None) –Written before a custom type's class name; the type's own module — imported by the revision — when
None. -
context_options(Mapping[str, object]) –Any other migration context option, passed as it is.
alembic_file_template ¶
Return :attr:file_template in the form the revision writer reads.
MigrationStatus
dataclass
¶
MigrationStatus(current: tuple[str, ...], latest: tuple[str, ...], previous: tuple[str, ...], next: tuple[str, ...], available: tuple[AvailableMigration, ...], executed: tuple[ExecutedMigration, ...], new: tuple[AvailableMigration, ...], executed_unavailable: tuple[ExecutedMigration, ...])
Where a database stands against its revision files, read in one go.
Attributes:
-
current(tuple[str, ...]) –The revisions the version table holds; empty when nothing is applied.
-
latest(tuple[str, ...]) –The revisions no other follows.
-
previous(tuple[str, ...]) –The revisions the current ones follow.
-
next(tuple[str, ...]) –The revisions that could run next, their predecessors all applied.
-
available(tuple[AvailableMigration, ...]) –Every revision file, earliest first.
-
executed(tuple[ExecutedMigration, ...]) –Every revision counted as applied — those with a file, earliest first, then those without. A revision is applied when the version table holds it or one following it, or the history table records it.
-
new(tuple[AvailableMigration, ...]) –The revisions with a file that are not applied.
-
executed_unavailable(tuple[ExecutedMigration, ...]) –The revisions recorded as applied that have no file.
Migrator ¶
Migrator(engine: AsyncEngine, config: MigrationsConfig | None = None, metadata: MetaData | Sequence[MetaData] | None = None, *, name: str = 'default')
Bases: LoggerAware
Reads, plans, runs and records one database's revisions.
Revisions are ordinary migration files in the configured directory. The database keeps the revisions it is at in the version table, and a row per revision applied in the history table; every change to one is made to the other in the same transaction.
Every method opens its own connection from the engine and returns once it
is closed. What reads files only — :meth:generate — opens none.
migrator = Migrator(engine, MigrationsConfig(directory="migrations"), Base.metadata)
await migrator.migrate() # to the latest revisions
status = await migrator.status()
Manage the revisions config describes on engine's database.
Parameters:
-
engine(AsyncEngine) –The database the revisions apply to.
-
config(MigrationsConfig | None, default:None) –Where the revisions are and how they are recorded.
-
metadata(MetaData | Sequence[MetaData] | None, default:None) –The table definitions a diff compares the database with; a diff is refused without them.
-
name(str, default:'default') –What reports call this database.
status
async
¶
Read where the database stands against the revision files.
plan
async
¶
Return the revisions :meth:migrate would run to reach target, without running them.
Raises:
-
MigrationError–When the target is unknown or cannot be reached.
migrate
async
¶
Run every revision between where the database is and target.
target is a revision — a full or partial id, or a label — a
relative one (+1, -2, ae10+1), or one of latest,
first (before every revision), next, prev and current.
A revision already applied is migrated down to; one that is not, up
to.
Raises:
-
MigrationError–When the target is unknown or cannot be reached.
migrate_sql
async
¶
Return the SQL :meth:migrate would run, recording included, without running it.
Raises:
-
MigrationError–When the target is unknown or cannot be reached.
execute
async
¶
execute(versions: Sequence[str], direction: Direction = Direction.UP) -> tuple[ExecutionResult, ...]
Run exactly versions, in order, each one way — and nothing before or after them.
Going up, a revision must not be applied and must follow only applied ones; going down, it must be applied and followed by none that is. Each version is checked against the ones before it, so a revision and the one following it may be run together.
Raises:
-
MigrationError–When a version is unknown, or out of order.
plan_for_versions
async
¶
plan_for_versions(versions: Sequence[str], direction: Direction = Direction.UP) -> tuple[MigrationPlan, ...]
Return what :meth:execute would run for versions, without running it.
Each version is resolved to the revision it names — a partial id to
the full one — and checked as :meth:execute checks it.
Raises:
-
MigrationError–When a version is unknown, or out of order.
execute_sql
async
¶
Return the SQL :meth:execute would run, recording included, without running it.
Raises:
-
MigrationError–When a version is unknown, or out of order.
generate ¶
Write an empty revision following the latest one, and return it.
Raises:
-
MigrationError–When there are several latest revisions to follow.
diff
async
¶
diff(message: str | None = None, *, allow_empty: bool = False, from_empty_schema: bool = False) -> AvailableMigration | None
Write a revision bringing the database to the table definitions, and return it.
The database is compared with the table definitions, and what differs
becomes the revision's upgrade. Nothing is written when nothing
differs, unless allow_empty.
Parameters:
-
message(str | None, default:None) –The revision's message.
-
allow_empty(bool, default:False) –Write a revision even when nothing differs.
-
from_empty_schema(bool, default:False) –Write every table, as if the database were empty.
Raises:
-
MigrationError–With no table definitions to compare with, or when the database is not at the latest revisions.
dump_schema
async
¶
Write a first revision creating every table the database has, and return it.
For a database that predates its revisions: the revision is written,
not applied — record it as applied with :meth:rollup.
Parameters:
-
message(str | None, default:None) –The revision's message.
-
table_filters(Sequence[str], default:()) –Regular expressions; only the tables whose name one of them finds are written. Every table with none.
Raises:
-
MigrationError–When revisions exist already, or the database has no table to write.
add_versions
async
¶
Mark versions applied without running them, in order, and return them.
Each must not be applied and must follow only applied revisions, as
with :meth:execute.
Raises:
-
MigrationError–When a version is unknown, applied, or out of order.
add_all_versions
async
¶
Mark every revision not yet applied as applied, without running any; return them.
delete_versions
async
¶
Mark versions not applied without running them, in order, and return them.
A version with a file must be applied and followed by no applied
revision, as with :meth:execute. A version without one — a revision
applied whose file is gone — is simply forgotten.
Raises:
-
MigrationError–When a version is not applied, or out of order.
delete_all_versions
async
¶
Mark every revision not applied, without running any.
rollup
async
¶
Forget every applied revision, then mark the single remaining one applied.
For after squashing every revision into one: the database is left exactly as it is, recorded as being at that one revision.
Raises:
-
MigrationError–When there is no revision, or more than one.