Skip to content

xtr_orm

Databases for async applications: engines, sessions and versioned schema migrations.

A :class:~xtr_orm.migrations.Migrator runs one database's revisions, a :class:~xtr_orm.database.DatabaseManager creates and drops the database itself, and a :class:ConnectionRegistry names every connection an application has. The bundle in :mod:xtr_orm.bundle builds all of them, and the engines and sessions, from one configuration — and a library bringing its own tables hands them over as a :class:MappingDriverInterface.

DEFAULT_CONNECTION module-attribute

DEFAULT_CONNECTION: Final = 'default'

The name of the connection used when none is named.

ConnectionRegistry

ConnectionRegistry(default: str = DEFAULT_CONNECTION)

Every connection by name: its engine, its migrator and its database manager.

Each is given as the object itself, or as an async function building it — called only when that piece is first asked for, so a command touching one connection opens nothing of another's.

A connection may also say how to reach the session of the current unit of work — the one every handler of a message shares — and how to close it, releasing every connection it holds; the message bus middleware use both.

registry = ConnectionRegistry()
registry.register(
    "default",
    engine=engine,
    migrator=Migrator(engine, MigrationsConfig(directory="migrations")),
    database=DatabaseManager(url),
)

Hold connections, answering for default when none is named.

default property

default: str

The name of the connection used when none is named.

register

register(name: str, *, engine: AsyncEngine | Callable[[], Awaitable[AsyncEngine]], migrator: Migrator | Callable[[], Awaitable[Migrator]], database: DatabaseManager | Callable[[], Awaitable[DatabaseManager]], session: Callable[[], Awaitable[AsyncSession]] | None = None, close: Callable[[], Awaitable[None]] | None = None, in_use: Callable[[], bool] | None = None) -> None

Add the connection name, or replace it.

Parameters:

  • name (str) –

    What the connection is known by.

  • engine (AsyncEngine | Callable[[], Awaitable[AsyncEngine]]) –

    Its engine, or what builds it.

  • migrator (Migrator | Callable[[], Awaitable[Migrator]]) –

    Its migrator, or what builds it.

  • database (DatabaseManager | Callable[[], Awaitable[DatabaseManager]]) –

    Its database manager, or what builds it.

  • session (Callable[[], Awaitable[AsyncSession]] | None, default: None ) –

    What returns the session of the unit of work under way; without it, :meth:session is refused.

  • close (Callable[[], Awaitable[None]] | None, default: None ) –

    What closes the connection; its engine is disposed when left out.

  • in_use (Callable[[], bool] | None, default: None ) –

    Whether the connection is in use — its engine built; without it, the connection always is. See :meth:in_use and :meth:close.

Raises:

names

names() -> tuple[str, ...]

Every connection's name, in the order registered.

in_use

in_use() -> tuple[str, ...]

The names of the connections in use, in the order registered.

What the message bus middleware act on unless given names: a connection nothing has used has no session to check and no pool to close, and opening it to find out would read its configuration.

has

has(name: str | None = None) -> bool

Whether the connection name — the default one when None — exists.

engine async

engine(name: str | None = None) -> AsyncEngine

Return the engine of the connection name, the default one when None.

Raises:

migrator async

migrator(name: str | None = None) -> Migrator

Return the migrator of the connection name, the default one when None.

Raises:

database async

database(name: str | None = None) -> DatabaseManager

Return the database manager of the connection name, the default one when None.

Raises:

session async

session(name: str | None = None) -> AsyncSession

Return the session of the current unit of work on the connection name.

Raises:

close async

close(name: str | None = None) -> None

Close the connection name: release every database connection its pools hold.

The engine stays usable; its next use connects again.

A connection nothing has used — one whose in_use says so — is left alone: it holds nothing to release, and building its engine to dispose it would read its configuration and connect.

Raises:

DatabaseManager

DatabaseManager(url: str | URL, *, connect_args: Mapping[str, object] | None = None)

Creates and drops the database a connection URL names, on its server.

A database cannot be created from a connection to itself, so the server is reached through its maintenance database — postgres on PostgreSQL, master on SQL Server, none on MySQL and MariaDB — with every statement committed as it runs. A SQLite database is its file.

manager = DatabaseManager("postgresql+asyncpg://app:secret@db/shop")
await manager.create(if_not_exists=True)

Manage the database url names, connecting with connect_args.

Raises:

backend property

backend: str

The kind of server, such as "postgresql" or "sqlite".

database property

database: str

The database's name — for SQLite, its file.

safe_url property

safe_url: str

The URL with its password hidden, fit for a message.

exists async

exists() -> bool

Whether the database exists.

Raises:

create async

create(*, if_not_exists: bool = False) -> bool

Create the database; return whether it was created.

Parameters:

  • if_not_exists (bool, default: False ) –

    Leave an existing database alone and return False, rather than let the server refuse.

Raises:

drop async

drop(*, if_exists: bool = False) -> bool

Drop the database; return whether it was dropped.

Parameters:

  • if_exists (bool, default: False ) –

    Do nothing for a database that does not exist and return False, rather than let the server refuse.

Raises:

InvalidArgumentError

InvalidArgumentError(reason: str)

Bases: OrmError, ValueError

A configuration or a call was given something the library cannot work with.

Raised where the value is given — a connection without a name, a version table without a name — rather than when a database is first reached.

Also a :class:ValueError, so code that already guards its configuration with except ValueError keeps working without learning a new exception.

Attributes:

  • reason (str) –

    What is wrong with the value.

Record what is wrong with the value.

MigrationError

MigrationError(reason: str)

Bases: OrmError

A migration operation was refused, or its revisions could not be read.

Raised before anything is changed whenever it can be decided up front: a version that no revision answers to, a revision run out of order, a diff against a database that is behind its revisions. Nothing is written when it is raised from a check; a failing revision rolls back what the database lets it.

Attributes:

  • reason (str) –

    Why the operation was refused.

Record why the operation was refused.

MissingDatabaseError

MissingDatabaseError(database: str, operation: str)

Bases: OrmError, FileNotFoundError

A database was dropped that is not there — a SQLite file that does not exist.

Also a :class:FileNotFoundError, as the missing file it stands for is. Ask for if_exists to leave a database that does not exist alone and be told so instead.

Attributes:

  • database (str) –

    The database's name — for SQLite, its file.

  • operation (str) –

    What was asked, such as "drop".

Record which database was asked for what.

OrmError

Bases: Exception

Base class for every error raised by this library.

Catch this to handle anything the library can refuse; catch a subclass to handle one cause. Every subclass carries the data a caller needs as typed attributes and composes its own message from them.

SessionUnavailableError

SessionUnavailableError(connection: str, reason: str)

Bases: OrmError

The session of the current unit of work was asked for where there is none.

A connection reaches the session everything in a unit of work shares — a message's handlers, say — only inside one, and only when it was registered with a way to.

Attributes:

  • connection (str) –

    The connection the session was asked of.

  • reason (str) –

    Why there is none.

Record which connection was asked, and why it has no session to give.

UnknownConnectionError

UnknownConnectionError(name: str, known: tuple[str, ...])

Bases: OrmError, LookupError

A connection was asked for by a name nothing registered.

Also a :class:LookupError, as a missing key in any other mapping is.

Attributes:

  • name (str) –

    The name asked for.

  • known (tuple[str, ...]) –

    Every name that is registered.

Record the name asked for and the ones that exist.

UnsupportedDatabaseError

UnsupportedDatabaseError(backend: str, operation: str)

Bases: OrmError

A database server was asked for something this library cannot do on it.

Creating or dropping a database takes a statement of the server's own; a server whose statement is not known here is refused rather than sent a guess.

Attributes:

  • backend (str) –

    The kind of server the URL names, such as "oracle".

  • operation (str) –

    What was asked, such as "create".

Record which server was asked for what.

MappingDriverInterface

Bases: Protocol

Tables a library builds once its own configuration is read.

A library shipping tables an application names — a prefix, which of them it keeps, which column type an identifier gets — cannot declare them when it is imported: it knows them only once configured. A driver is that build behind one call, so the tables can join the definitions of whichever connection the application picked.

load_metadata

load_metadata(metadata: MetaData) -> tuple[str, ...]

Add this driver's tables to metadata, and return the names added.

Must be idempotent. The definitions a connection migrates towards are often shared for the whole process — the ones advanced-alchemy registers per bind key are — so a driver may be loaded into the same MetaData twice: by a second kernel in one test run, or by a second library mapping the same table. Skip every name already in metadata.tables and leave it out of what is returned, so the second load changes nothing and says so.

The argument is passed positionally, so an implementation may name it whatever reads best.

AvailableMigration dataclass

AvailableMigration(version: str, description: str, down_versions: tuple[str, ...], path: str, is_head: bool)

A revision file, as it was found in the migrations directory.

Attributes:

  • version (str) –

    The revision's identifier.

  • description (str) –

    The first line of its message.

  • down_versions (tuple[str, ...]) –

    The revisions it follows; empty for a first revision, two or more for one merging branches.

  • path (str) –

    Where its file is.

  • is_head (bool) –

    Whether no revision follows it.

Direction

Bases: Enum

Which way a revision runs: its upgrade, or its downgrade.

ExecutedMigration dataclass

ExecutedMigration(version: str, executed_at: datetime | None = None, execution_time: float | None = None)

A revision the database counts as applied.

Applied means recorded in the version table, preceding a revision that is, or recorded in the history table. When and how long come from the history table: a revision applied before it existed has neither, and one only marked as applied has no duration.

Attributes:

  • version (str) –

    The revision's identifier.

  • executed_at (datetime | None) –

    When it was applied, by the database's clock — with a time zone where the column keeps one (PostgreSQL), naive elsewhere: UTC on SQLite, the session's time zone on MySQL.

  • execution_time (float | None) –

    How long it took, in seconds.

ExecutionResult dataclass

ExecutionResult(version: str, direction: Direction, duration: float)

One revision a migration ran, and how long it took.

Attributes:

  • version (str) –

    The revision's identifier.

  • direction (Direction) –

    Whether its upgrade or its downgrade ran.

  • duration (float) –

    How long it took, in seconds.

MigrationPlan dataclass

MigrationPlan(version: str, direction: Direction, description: str)

One revision a migration would run, and which way.

Attributes:

  • version (str) –

    The revision's identifier.

  • direction (Direction) –

    Whether its upgrade or its downgrade runs.

  • description (str) –

    The first line of its message.

MigrationsConfig dataclass

MigrationsConfig(directory: str = 'migrations', version_table: str = 'alembic_version', version_table_schema: str | None = None, history_table: str = 'alembic_version_history', file_template: str = '{year}{month}{day}{hour}{minute}{second}_{rev}_{slug}', truncate_slug_length: int = 40, timezone: str | None = None, template_directory: str | None = None, recursive_version_locations: bool = False, post_write_hooks: Sequence[Mapping[str, str]] = (), transaction_per_migration: bool = False, transactional_ddl: bool | None = None, compare_type: bool = False, compare_server_default: bool = False, render_as_batch: bool = False, include_schemas: bool = False, include_name: Callable[..., bool] | None = None, include_object: Callable[..., bool] | None = None, render_item: Callable[..., str | bool] | None = None, process_revision_directives: Callable[..., None] | None = None, user_module_prefix: str | None = None, context_options: Mapping[str, object] = _no_options())

Where a database's revisions live, how new ones are written, and how runs are recorded.

Revisions are ordinary migration files, one per change, in :attr:directory. The database keeps its current revisions in :attr:version_table, and a row per revision ever applied — when, and how long it took — in :attr:history_table.

MigrationsConfig(
    directory="%kernel.project_dir%/migrations",
    version_table_schema="meta",
    render_as_batch=True,  # SQLite cannot alter a table in place
)

Attributes:

  • directory (str) –

    Where revision files are read from and written to; created when the first one is written. A relative path is relative to the working directory — under a kernel, start it with %kernel.project_dir%.

  • version_table (str) –

    The table holding the database's current revisions.

  • version_table_schema (str | None) –

    The schema of both tables; the connection's default when None.

  • history_table (str) –

    The table recording every revision applied, when it ran and how long it took.

  • file_template (str) –

    How a new revision's file is named. {rev}, {slug}, {epoch}, {year}, {month}, {day}, {hour}, {minute} and {second} are filled in; the default sorts files by the time they were written.

  • truncate_slug_length (int) –

    The longest the message part of a file name gets.

  • timezone (str | None) –

    The time zone a revision's creation date is written in; local time when None.

  • template_directory (str | None) –

    A directory holding a script.py.mako that new revisions are rendered from; the package's own when None.

  • recursive_version_locations (bool) –

    Read revisions from subdirectories of :attr:directory too.

  • post_write_hooks (Sequence[Mapping[str, str]]) –

    Programs run on every file written, such as a formatter — each a mapping with a type ("console_scripts", "exec" or "module") and that type's options.

  • transaction_per_migration (bool) –

    Commit after each revision rather than once after all of them.

  • transactional_ddl (bool | None) –

    Whether schema changes run inside the transaction; the database's own answer when None.

  • compare_type (bool) –

    Compare column types when generating a diff. Off by default: a type the database stores as another — a UUID kept as bytes on SQLite — would otherwise differ in every diff.

  • compare_server_default (bool) –

    Compare server defaults when generating a diff.

  • render_as_batch (bool) –

    Write table changes as copy-and-move batches, which SQLite needs for anything but adding a column.

  • include_schemas (bool) –

    Compare every schema when generating a diff, not only the default one.

  • include_name (Callable[..., bool] | None) –

    Decides, by name, which schemas and tables a diff reads from the database.

  • include_object (Callable[..., bool] | None) –

    Decides which tables, columns, indexes and constraints a diff considers.

  • render_item (Callable[..., str | bool] | None) –

    Renders a type or a construct a diff writes, before the default rendering.

  • process_revision_directives (Callable[..., None] | None) –

    Adjusts a revision about to be written.

  • user_module_prefix (str | None) –

    Written before a custom type's class name; the type's own module — imported by the revision — when None.

  • context_options (Mapping[str, object]) –

    Any other migration context option, passed as it is.

alembic_file_template

alembic_file_template() -> str

Return :attr:file_template in the form the revision writer reads.

MigrationStatus dataclass

MigrationStatus(current: tuple[str, ...], latest: tuple[str, ...], previous: tuple[str, ...], next: tuple[str, ...], available: tuple[AvailableMigration, ...], executed: tuple[ExecutedMigration, ...], new: tuple[AvailableMigration, ...], executed_unavailable: tuple[ExecutedMigration, ...])

Where a database stands against its revision files, read in one go.

Attributes:

  • current (tuple[str, ...]) –

    The revisions the version table holds; empty when nothing is applied.

  • latest (tuple[str, ...]) –

    The revisions no other follows.

  • previous (tuple[str, ...]) –

    The revisions the current ones follow.

  • next (tuple[str, ...]) –

    The revisions that could run next, their predecessors all applied.

  • available (tuple[AvailableMigration, ...]) –

    Every revision file, earliest first.

  • executed (tuple[ExecutedMigration, ...]) –

    Every revision counted as applied — those with a file, earliest first, then those without. A revision is applied when the version table holds it or one following it, or the history table records it.

  • new (tuple[AvailableMigration, ...]) –

    The revisions with a file that are not applied.

  • executed_unavailable (tuple[ExecutedMigration, ...]) –

    The revisions recorded as applied that have no file.

is_up_to_date property

is_up_to_date: bool

Whether every revision with a file is applied.

Migrator

Migrator(engine: AsyncEngine, config: MigrationsConfig | None = None, metadata: MetaData | Sequence[MetaData] | None = None, *, name: str = 'default')

Bases: LoggerAware

Reads, plans, runs and records one database's revisions.

Revisions are ordinary migration files in the configured directory. The database keeps the revisions it is at in the version table, and a row per revision applied in the history table; every change to one is made to the other in the same transaction.

Every method opens its own connection from the engine and returns once it is closed. What reads files only — :meth:generate — opens none.

migrator = Migrator(engine, MigrationsConfig(directory="migrations"), Base.metadata)

await migrator.migrate()  # to the latest revisions
status = await migrator.status()

Manage the revisions config describes on engine's database.

Parameters:

  • engine (AsyncEngine) –

    The database the revisions apply to.

  • config (MigrationsConfig | None, default: None ) –

    Where the revisions are and how they are recorded.

  • metadata (MetaData | Sequence[MetaData] | None, default: None ) –

    The table definitions a diff compares the database with; a diff is refused without them.

  • name (str, default: 'default' ) –

    What reports call this database.

name property

name: str

What reports call this database.

engine property

engine: AsyncEngine

The database the revisions apply to.

config property

config: MigrationsConfig

Where the revisions are and how they are recorded.

status async

status() -> MigrationStatus

Read where the database stands against the revision files.

plan async

plan(target: str = 'latest') -> tuple[MigrationPlan, ...]

Return the revisions :meth:migrate would run to reach target, without running them.

Raises:

migrate async

migrate(target: str = 'latest') -> tuple[ExecutionResult, ...]

Run every revision between where the database is and target.

target is a revision — a full or partial id, or a label — a relative one (+1, -2, ae10+1), or one of latest, first (before every revision), next, prev and current. A revision already applied is migrated down to; one that is not, up to.

Raises:

migrate_sql async

migrate_sql(target: str = 'latest') -> str

Return the SQL :meth:migrate would run, recording included, without running it.

Raises:

execute async

execute(versions: Sequence[str], direction: Direction = Direction.UP) -> tuple[ExecutionResult, ...]

Run exactly versions, in order, each one way — and nothing before or after them.

Going up, a revision must not be applied and must follow only applied ones; going down, it must be applied and followed by none that is. Each version is checked against the ones before it, so a revision and the one following it may be run together.

Raises:

plan_for_versions async

plan_for_versions(versions: Sequence[str], direction: Direction = Direction.UP) -> tuple[MigrationPlan, ...]

Return what :meth:execute would run for versions, without running it.

Each version is resolved to the revision it names — a partial id to the full one — and checked as :meth:execute checks it.

Raises:

execute_sql async

execute_sql(versions: Sequence[str], direction: Direction = Direction.UP) -> str

Return the SQL :meth:execute would run, recording included, without running it.

Raises:

generate

generate(message: str | None = None) -> AvailableMigration

Write an empty revision following the latest one, and return it.

Raises:

  • MigrationError –

    When there are several latest revisions to follow.

diff async

diff(message: str | None = None, *, allow_empty: bool = False, from_empty_schema: bool = False) -> AvailableMigration | None

Write a revision bringing the database to the table definitions, and return it.

The database is compared with the table definitions, and what differs becomes the revision's upgrade. Nothing is written when nothing differs, unless allow_empty.

Parameters:

  • message (str | None, default: None ) –

    The revision's message.

  • allow_empty (bool, default: False ) –

    Write a revision even when nothing differs.

  • from_empty_schema (bool, default: False ) –

    Write every table, as if the database were empty.

Raises:

  • MigrationError –

    With no table definitions to compare with, or when the database is not at the latest revisions.

dump_schema async

dump_schema(message: str | None = None, *, table_filters: Sequence[str] = ()) -> AvailableMigration

Write a first revision creating every table the database has, and return it.

For a database that predates its revisions: the revision is written, not applied — record it as applied with :meth:rollup.

Parameters:

  • message (str | None, default: None ) –

    The revision's message.

  • table_filters (Sequence[str], default: () ) –

    Regular expressions; only the tables whose name one of them finds are written. Every table with none.

Raises:

  • MigrationError –

    When revisions exist already, or the database has no table to write.

add_versions async

add_versions(versions: Sequence[str]) -> tuple[str, ...]

Mark versions applied without running them, in order, and return them.

Each must not be applied and must follow only applied revisions, as with :meth:execute.

Raises:

  • MigrationError –

    When a version is unknown, applied, or out of order.

add_all_versions async

add_all_versions() -> tuple[str, ...]

Mark every revision not yet applied as applied, without running any; return them.

delete_versions async

delete_versions(versions: Sequence[str]) -> tuple[str, ...]

Mark versions not applied without running them, in order, and return them.

A version with a file must be applied and followed by no applied revision, as with :meth:execute. A version without one — a revision applied whose file is gone — is simply forgotten.

Raises:

delete_all_versions async

delete_all_versions() -> None

Mark every revision not applied, without running any.

rollup async

rollup() -> AvailableMigration

Forget every applied revision, then mark the single remaining one applied.

For after squashing every revision into one: the database is left exactly as it is, recorded as being at that one revision.

Raises: