Skip to content

xtr_http_client

An async HTTP client: typed options in, a response read on demand out.

One interface with clients behind it — one that opens real connections and streams what comes back, one that answers from a script a test wrote, five that wrap another and add what every application writes again (a set of defaults per service, a refusal to reach inwards, a request made again while that is worth doing, a pace requests are held to, and a record of every request made), and :class:HttpClient, the one call that hands one over so nothing else has to name it.

A request returns once the status line and headers have arrived, whatever the status; the body is read when the caller asks for it, as a whole or piece by piece.

HttpClient

One call that hands back a client, so nothing has to name an implementation.

Code that calls out depends on the interface. Code that builds a client has to pick something, and picking is a decision worth making in one place: this is that place.

from xtr_http_client import HttpClient

client = HttpClient.create(base_uri="https://example.com/", timeout=5)

It is not a client itself and holds nothing; there is nothing to keep an instance of.

create staticmethod

create(max_host_connections: int = 6, **default_options: Unpack[RequestOptions]) -> HttpClientInterface

Return a client whose requests start from default_options.

Parameters:

  • max_host_connections (int, default: 6 ) –

    How many connections one pool may hold at once.

  • **default_options (Unpack[RequestOptions], default: {} ) –

    What every request the client makes starts from, over the contract's own defaults.

Returns:

Raises:

  • InvalidArgumentError –

    When an option is not one the client knows.

HttpClientMixin

HttpClientMixin(default_options: RequestOptions | None = None)

What a client refuses, and the options it starts every request from.

Mixed into every client here, so all of them keep their defaults the same way and refuse the same requests — which is what lets one be swapped for another without a caller noticing. A client is then only the part that opens connections.

Its own state is the one thing a client needs from it: the options every request starts from. __init__ takes them so a subclass can hand over what it was built with and get back the merged defaults.

The checks are exposed one by one because a client makes them one at a time: the method is known before the options are read, and a client preparing its own defaults has no method to check at all.

Start from the contract's defaults, with default_options over them.

Raises:

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

A copy of this client whose defaults have options over them.

A copy rather than a change, so a caller can narrow a client for one job — a header, a timeout, a base URI — without the client everyone else holds changing under them.

Raises:

merge_default_options classmethod

merge_default_options(options: Mapping[str, object], default_options: Mapping[str, object]) -> dict[str, object]

Merge what a request asked for over what the client always asks for.

See :func:xtr_http_client.internal.option_merge.merge_default_options.

validate_options

validate_options(options: Mapping[str, object]) -> None

Refuse an option this client does not know, naming the closest it does.

See :func:xtr_http_client.internal.option_merge.validate_options.

validate_method staticmethod

validate_method(method: str) -> None

Refuse a method no server would read as the one that was meant.

Raises:

  • InvalidArgumentError –

    When the method is empty, or is anything other than uppercase ASCII letters.

validate_auth staticmethod

validate_auth(options: Mapping[str, object]) -> None

Refuse credentials this client cannot send, or two sets of them.

Raises:

  • InvalidArgumentError –

    When auth_basic or auth_bearer is not of a shape credentials can be read from, or both are set.

validate_body staticmethod

validate_body(options: Mapping[str, object]) -> None

Refuse a request that says what to send twice.

A JSON document and a body are both the whole body, so a request carrying each has not said which one to send.

Raises:

HttpxHttpClient

HttpxHttpClient(default_options: RequestOptions | None = None, max_host_connections: int = 6)

Bases: HttpClientMixin, HttpClientInterface, ResetInterface

Sends requests over real connections, and hands back answers as they arrive.

A request returns once the status line and the headers are in, whatever the status; the body is read off the connection when the caller asks for it, and the connection is held until the body is done with or the response is left.

The request is authoritative and the options fill what it leaves out — a header it already carries is the one sent, credentials apply only where it has no Authorization of its own — which is what lets one set of client-wide defaults sit under requests that each know their own mind. What each part of an outgoing request is assembled from is in :mod:xtr_http_client.internal.httpx_request, and the redirect chain an exchange may turn into is followed in :mod:xtr_http_client.internal.httpx_redirect; what is here is the exchange itself, under its deadline.

Connections are pooled by how they connect rather than by host: a proxy, a trust store, a protocol version and the deadlines all decide what a connection is, so each combination of them gets a pool, and max_host_connections is each pool's ceiling across every host it reaches.

A host a request pins to an address through resolve stays pinned for the requests after it, in the pool it went through; :meth:reset forgets every pin, and keeps the connections open until the client is closed.

async with HttpxHttpClient(max_host_connections=4) as client:
    response = await client.request("GET", "https://example.com/orders")
    orders = await response.to_array()

Redirects are followed here rather than by the backend, so that an Authorization header is dropped the moment a chain leaves the host it was meant for, and comes back only when the chain returns to it.

Start from default_options, opening nothing until a request is made.

Parameters:

  • default_options (RequestOptions | None, default: None ) –

    What every request this client makes starts from, over the contract's own defaults.

  • max_host_connections (int, default: 6 ) –

    How many connections one pool may hold at once.

Raises:

  • InvalidArgumentError –

    When a default option is not one this client knows.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponse

Make a request for method and url, and return once its head is in.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made.

Returns:

  • HttpResponse –

    The response, with its status and headers already in.

Raises:

  • TransportTimeoutError –

    When no answer came in time.

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came.

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponse

Send request, the options filling what it leaves out.

Parameters:

Returns:

  • HttpResponse –

    The response, with its status and headers already in.

Raises:

  • TransportTimeoutError –

    When no answer came in time.

  • InvalidArgumentError –

    For an invalid method, URL or option, and for a body given both with the request and as an option.

  • TransportError –

    When no answer came.

reset async

reset() -> None

Keep everything: the connections are how the next unit gets anywhere.

A request's resolve pins its own request only, so there is nothing a unit of work left behind to forget.

aclose async

aclose() -> None

Close every pool, and with it every connection this client holds.

MockHttpClient

MockHttpClient(response_factory: ResponseFactory | None = None, base_uri: str | None = 'https://example.com')

Bases: HttpClientMixin, HttpClientInterface, ResetInterface

Answers every request from what a test wrote out, and opens no connection.

It is the same client as any other here — the options it refuses, the defaults it starts from and the way a request and its options are merged are all the ones a real transfer uses — with the transfer replaced by a script. So a test exercises the code that calls out, down to the options it passed, without a server to point it at.

client = MockHttpClient([MockResponse(b"first"), MockResponse(b"second")])
async with await client.request("GET", "/orders") as response:
    assert await response.get_content() == b"first"
assert client.get_requests_count() == 1

What the answers come from is whatever a test finds easiest to write:

  • nothing, and every request is answered with an empty 200;
  • one :class:MockResponse <xtr_http_client.response.mock_response.MockResponse>, which answers one request, and a second request is a mistake, reported as the script running out;
  • an iterable of them, one per request, in order — a request past the end of it is a mistake and is reported as one, since a test that makes more requests than it wrote answers for is no longer testing what it says;
  • a callable, which is handed the method, the URL and the merged options and says what to answer with — the way to assert on the request that was made, and to answer differently depending on it. It may be async.

An iterable may hold callables too, so one request in a sequence can be answered that way while the rest are written out.

Nothing is followed and nothing is resolved: a 3xx is the answer, and a relative URL is resolved against base_uri — which starts at https://example.com so a test can say /orders and mean something.

Start from response_factory, which nothing is taken from until asked.

Parameters:

  • response_factory (ResponseFactory | None, default: None ) –

    Where the answers come from.

  • base_uri (str | None, default: 'https://example.com' ) –

    What a relative URL is resolved against. None leaves a relative URL unresolvable, which is then refused.

set_response_factory

set_response_factory(response_factory: ResponseFactory | None = None) -> None

Answer from response_factory from now on.

What was left of the previous one is dropped, and the count of requests already made is kept: a test that changes its mind about the answers has not unmade the requests.

Parameters:

  • response_factory (ResponseFactory | None, default: None ) –

    Where the answers come from.

get_requests_count

get_requests_count() -> int

Return how many requests this client has been asked to make.

reset async

reset() -> None

Forget how many requests were made, leaving the answers as they are.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> MockResponse

Answer a request for method and url from the script.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request would have been made.

Returns:

  • MockResponse –

    The scripted answer, issued for this request.

Raises:

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> MockResponse

Answer request from the script, the options filling what it leaves out.

Parameters:

Returns:

  • MockResponse –

    The scripted answer, issued for this request.

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option, and for a body given both with the request and as an option.

  • TransportError –

    When the script says the transfer fails, has no answer left to give, or gives something that is not an answer.

NoPrivateNetworkHttpClient

NoPrivateNetworkHttpClient(client: HttpClientInterface, subnets: str | Sequence[str] | None = None, allow_list: str | Sequence[str] = (), resolver: Callable[[str, int], Awaitable[Sequence[str]]] | None = None)

Bases: HttpClientInterface, ResetInterface

Wraps a client and refuses every request that leads inwards.

A URL that came from outside is data, so without this a request is a way to make this process reach an address only this process can see: another service on the same network, an admin port, a metadata endpoint handing out credentials.

client = NoPrivateNetworkHttpClient(inner)
response = await client.request("GET", "https://example.com/avatar")

The check runs on what the host resolves to, not on how it is written, and one refused address refuses the whole request — whichever of them the connection would pick, nothing is sent. The request is then pinned to the address that was checked, through the resolve option, so a name whose records change between the check and the connection reaches what was vetted or nothing at all.

Redirects are followed here rather than by the wrapped client, because a hop nobody checked would walk straight past the guard: each one is resolved and checked like the request that started the chain.

What a host resolves to is remembered for one request's chain of hops, so hops back to the same host ask the resolver once; the next request asks again, so no answer outlives the request that asked for it.

The networks it refuses come from xtr-http-foundation, which is not a dependency of this package: install it beside it to use this client.

Guard client, refusing whichever networks subnets names.

Parameters:

  • client (HttpClientInterface) –

    The client every vetted request is handed to.

  • subnets (str | Sequence[str] | None, default: None ) –

    The networks to refuse, in CIDR notation or as single addresses, replacing the default. None refuses the networks only this one can reach.

  • allow_list (str | Sequence[str], default: () ) –

    Networks or addresses exempt from the refusal, which is how one internal receiver is opened without opening the network it sits in. Addresses only, never host names: the check runs on resolved addresses, and a name would itself have to be resolved, and could point anywhere by then.

  • resolver (Callable[[str, int], Awaitable[Sequence[str]]] | None, default: None ) –

    What every address of a host and port is asked of. None asks the platform's own resolver.

Raises:

  • LogicError –

    When xtr-http-foundation is not installed, since there is then no way to tell which addresses to refuse.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request for method and url, once where it leads is vetted.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made.

Returns:

Raises:

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send request, vetting every host the chain reaches.

Parameters:

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option, and for a URL that is relative with no base_uri to resolve it.

  • TransportError –

    When a host is, or resolves to, a refused address, and when no answer came.

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a copy around a wrapped client whose defaults have options over them.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the wrapped client of the copy.

Returns:

  • Self –

    A new client, guarding the same networks.

Raises:

reset async

reset() -> None

Reset the wrapped client; this one keeps nothing between requests.

HttpResponse

HttpResponse(message: Response, body: ResponseBody, info: dict[str, object], on_cancel: Callable[[], Awaitable[None]] | None = None)

Bases: HttpResponseInterface

A response whose head has arrived and whose body is read when it is asked for.

Three parts are held apart because they arrive apart: the message — the status line and the headers, which came in whole — the body, which is still coming off the transfer, and the info, which the transfer goes on writing to as it does.

A with_* copy therefore copies the message alone. The body is the one transfer's and cannot be had twice, and the info is that transfer's own record, so every copy reads the one mapping and sees whatever has been written to it since.

Hold the parts of a response that arrived apart.

Parameters:

  • message (Response) –

    The status line and the headers, as they came in.

  • body (ResponseBody) –

    The body, taken off the transfer as it is asked for.

  • info (dict[str, object]) –

    What the transfer knows about itself. Kept rather than copied, because the transfer is still writing to it.

  • on_cancel (Callable[[], Awaitable[None]] | None, default: None ) –

    What abandoning the transfer takes, when there is anything: the client that opened it knows, a response does not.

get_status_code

get_status_code() -> int

Return the status code the server answered with.

Returns:

  • int –

    The three-digit code off the status line.

get_reason_phrase

get_reason_phrase() -> str

Return the reason phrase that goes with the status code.

Returns:

  • str –

    The phrase, or an empty string when the status line carried none.

with_status

with_status(code: int, reason_phrase: str = '') -> HttpResponse

Return a copy carrying code and, optionally, reason_phrase.

Parameters:

  • code (int) –

    The status code to set.

  • reason_phrase (str, default: '' ) –

    The phrase to go with code.

Returns:

  • HttpResponse –

    A new response; this one is left as it was.

get_protocol_version

get_protocol_version() -> str

Return the HTTP version the answer came back on.

Returns:

  • str –

    The version number alone, such as "1.1".

with_protocol_version

with_protocol_version(version: str) -> HttpResponse

Return a copy carrying version.

Parameters:

  • version (str) –

    The HTTP version number alone.

Returns:

  • HttpResponse –

    A new response; this one is left as it was.

get_headers

get_headers() -> Mapping[str, Sequence[str]]

Return every header, each name mapped to all of its values.

Returns:

has_header

has_header(name: str) -> bool

Tell whether a header arrived, matching name without case.

Parameters:

  • name (str) –

    The header name, in any case.

Returns:

  • bool –

    True when some header name matches it.

get_header

get_header(name: str) -> Sequence[str]

Return every value recorded under a header, matched without case.

Parameters:

  • name (str) –

    The header name, in any case.

Returns:

  • Sequence[str] –

    Its values in order, or an empty sequence when it did not arrive.

get_header_line

get_header_line(name: str) -> str

Return the values of a header joined with commas.

Parameters:

  • name (str) –

    The header name, in any case.

Returns:

  • str –

    The values joined with commas, or an empty string.

with_header

with_header(name: str, value: str | Sequence[str]) -> HttpResponse

Return a copy whose name header holds value and nothing else.

Parameters:

  • name (str) –

    The header name, in any case.

  • value (str | Sequence[str]) –

    One value, or every value the header is to hold.

Returns:

  • HttpResponse –

    A new response; this one is left as it was.

with_added_header

with_added_header(name: str, value: str | Sequence[str]) -> HttpResponse

Return a copy with value appended to the name header.

Parameters:

  • name (str) –

    The header name, in any case.

  • value (str | Sequence[str]) –

    One value, or every value to append.

Returns:

  • HttpResponse –

    A new response; this one is left as it was.

without_header

without_header(name: str) -> HttpResponse

Return a copy with the name header gone.

Parameters:

  • name (str) –

    The header name, in any case.

Returns:

  • HttpResponse –

    A new response; this one is left as it was.

get_body

get_body() -> StreamInterface

Return the body, which reads off the transfer as it is read.

Returns:

with_body

with_body(body: StreamInterface) -> HttpResponse

Return a copy reading its body from body instead.

Parameters:

  • body (StreamInterface) –

    The stream the copy is to read its body from, in place of the one coming off the transfer.

Returns:

get_info

get_info(key: str | None = None) -> object

Return what the transfer knows about itself, as it stands.

The mapping handed back is a copy, so reading the info of a transfer still running cannot be disturbed by the transfer writing to it, and nothing a caller does to it reaches the transfer.

Parameters:

  • key (str | None, default: None ) –

    One entry to look up. None asks for all of them.

Returns:

  • object –

    Every entry when key is None; otherwise the value under

  • object –

    key, or None when there is no such entry.

cancel async

cancel() -> None

Abandon the transfer and let go of what it was holding.

The record is written before anything is let go of, so a reader that finds the body closed can be told why it is closed rather than only that it is.

get_content async

get_content(throw: bool = True) -> bytes

Read the whole body, waiting for the rest of it to arrive.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than answered with the body that came with it.

Returns:

  • bytes –

    Every byte of the body, empty when there was none.

Raises:

to_array async

to_array(throw: bool = True) -> dict[str, object]

Read the whole body and decode it as a JSON object.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than answered with the body that came with it.

Returns:

Raises:

JsonMockResponse

JsonMockResponse(body: object = None, info: Mapping[str, object] | None = None)

Bases: MockResponse

A written answer carrying a value encoded as JSON, labelled as JSON.

The one thing a JSON answer adds to a scripted one: the body is a value rather than bytes, and the content type says what it is — unless the script named one of its own, which a test does to answer with a problem document.

client = MockHttpClient(JsonMockResponse({"id": 7}, {"http_code": 201}))
async with await client.request("GET", "/orders") as response:
    assert await response.to_array() == {"id": 7}

Write out an answer carrying body as JSON.

Parameters:

  • body (object, default: None ) –

    Any value JSON can be made of, encoded as it stands. None, the default, is not encoded: it answers an empty list, so an answer nobody wrote out is still a document. To send null, write the bytes with :class:MockResponse instead.

  • info (Mapping[str, object] | None, default: None ) –

    What the transfer is to claim about itself, as :class:MockResponse takes it. A content-type among its response_headers is left alone; one is added when there is none.

Raises:

from_file classmethod

from_file(path: str | PathLike[str], info: Mapping[str, object] | None = None) -> Self

Write out an answer carrying the JSON document a file holds.

The file is read now rather than when the response is issued, so a test that names the wrong file is told so where it named it.

Parameters:

Returns:

  • Self –

    The response, carrying the document the file holds.

Raises:

MockResponse

MockResponse(body: bytes | str | Iterable[bytes] = b'', info: Mapping[str, object] | None = None)

Bases: HttpResponseInterface

An answer written before the question, which a client hands back as its own.

A test says what it wants to come back — a body, a status, headers, or an error instead of an answer — and :class:MockHttpClient <xtr_http_client.mock_http_client.MockHttpClient> answers with it. Nothing is sent and nothing is opened.

client = MockHttpClient(MockResponse(b'{"id": 7}', {"http_code": 201}))
async with await client.request("GET", "/orders") as response:
    assert await response.to_array() == {"id": 7}

A written response is only half a response: the other half — the method, the URL and the options it answers — is the client's, and is filled in when the client issues it. Until then there is nothing to read, because what came back is not yet an answer to anything; reading one raises :class:InvalidArgumentError <xtr_http_client_contracts.exception.invalid_argument_error.InvalidArgumentError>.

Once issued, it is an ordinary response of this package: the status, headers, body, copies and errors are :class:HttpResponse <xtr_http_client.response.http_response.HttpResponse>'s, over a body made of the scripted pieces, so a test reads a scripted answer exactly as it reads a real one.

Write out an answer.

Parameters:

  • body (bytes | str | Iterable[bytes], default: b'' ) –

    What the body is to be. Bytes or text are the whole of it; an iterable of bytes is handed over a piece at a time, which is how a test reads a body that arrives in pieces.

  • info (Mapping[str, object] | None, default: None ) –

    What the transfer is to claim about itself — http_code (200 when left out), response_headers, and error for a transfer that is to fail rather than answer. Any other entry is carried through to get_info() untouched.

issue

issue(method: str, url: str, options: Mapping[str, object]) -> Self

Make this written answer the answer to a request that was just made.

What a client does to a response before handing it over: the request side is filled in, and the body and the record of the transfer come into being. Called by :class:MockHttpClient <xtr_http_client.mock_http_client.MockHttpClient>.

Parameters:

  • method (str) –

    The method the request was made with.

  • url (str) –

    The URL it was made to, resolved.

  • options (Mapping[str, object]) –

    The options it was made with, merged.

Returns:

  • Self –

    This response, now readable.

Raises:

  • TransportError –

    When the script says the transfer fails rather than answers. Nothing is opened in that case, so there is nothing left to let go of.

get_request_method

get_request_method() -> str

The method of the request this answered, empty until it answered one.

get_request_url

get_request_url() -> str

The URL of the request this answered, empty until it answered one.

get_request_options

get_request_options() -> Mapping[str, object]

The options the request was made with, merged as the client sent them.

get_status_code

get_status_code() -> int

Return the status the script says the answer came back with.

get_reason_phrase

get_reason_phrase() -> str

Return the reason phrase that goes with the status.

with_status

with_status(code: int, reason_phrase: str = '') -> Self

Return a copy carrying code and, optionally, reason_phrase.

get_protocol_version

get_protocol_version() -> str

Return the HTTP version the script says the answer came back on.

with_protocol_version

with_protocol_version(version: str) -> Self

Return a copy carrying version.

get_headers

get_headers() -> Mapping[str, Sequence[str]]

Return every header, each name mapped to all of its values.

has_header

has_header(name: str) -> bool

Tell whether a header is there, matching name without case.

get_header

get_header(name: str) -> Sequence[str]

Return every value recorded under a header, matched without case.

get_header_line

get_header_line(name: str) -> str

Return the values of a header joined with commas.

with_header

with_header(name: str, value: str | Sequence[str]) -> Self

Return a copy whose name header holds value and nothing else.

with_added_header

with_added_header(name: str, value: str | Sequence[str]) -> Self

Return a copy with value appended to the name header.

without_header

without_header(name: str) -> Self

Return a copy with the name header gone.

get_body

get_body() -> StreamInterface

Return the body, over the pieces the script was written with.

with_body

with_body(body: StreamInterface) -> Self

Return a copy reading its body from body instead.

get_info

get_info(key: str | None = None) -> object

Return what the transfer knows about itself, as it stands.

Readable before the response is issued, where it is the script as it was written — which is what lets a cancelled response say so even though it never answered anything.

Parameters:

  • key (str | None, default: None ) –

    One entry to look up. None asks for all of them.

Returns:

  • object –

    Every entry when key is None; otherwise the value under

  • object –

    key, or None when there is no such entry.

cancel async

cancel() -> None

Abandon the transfer and let go of what it was holding.

get_content async

get_content(throw: bool = True) -> bytes

Read the whole body, as the script hands it over.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than answered with the body that came with it.

Returns:

  • bytes –

    Every byte of the body, empty when the script wrote none.

Raises:

  • InvalidArgumentError –

    When the response has not been issued.

  • TransportError –

    When the body cannot be read.

  • RedirectionError –

    On a 3xx, when throw.

  • ClientError –

    On a 4xx, when throw.

  • ServerError –

    On a 5xx, when throw.

to_array async

to_array(throw: bool = True) -> dict[str, object]

Read the whole body and decode it as a JSON object.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than answered with the body that came with it.

Returns:

Raises:

  • InvalidArgumentError –

    When the response has not been issued.

  • JsonError –

    When the body is empty, is not JSON, or is JSON that is not an object.

  • TransportError –

    When the body cannot be read.

  • RedirectionError –

    On a 3xx, when throw.

  • ClientError –

    On a 4xx, when throw.

  • ServerError –

    On a 5xx, when throw.

GenericRetryStrategy

GenericRetryStrategy(status_codes: _StatusCodes = DEFAULT_RETRY_STATUS_CODES, delay_ms: int = 1000, multiplier: float = 2.0, max_delay_ms: int = 0, jitter: float = 0.1, random: Callable[[float, float], float] = uniform, *, max_retry_after_ms: int = _MAX_RETRY_AFTER_MS)

Bases: RetryStrategyInterface

Retries a status code on the list, after a delay that grows each time.

The decision is the status code and, for the codes where it matters, the method: a 503 is the server saying "not now", so asking again is the right thing whatever was asked; a 500 after a POST may have changed something already, so sending it again is not.

strategy = GenericRetryStrategy([500, 502, 503], delay_ms=500)
client = RetryableHttpClient(inner, strategy, max_retries=2)

The delay starts at delay_ms and is multiplied by multiplier for every attempt already made, so the server is given more room each time rather than being asked again as fast as it refused. jitter spreads that over a window around it, which is what keeps a thousand clients that failed together from coming back together.

Retry what status_codes lists, backing off between attempts.

Parameters:

  • status_codes (_StatusCodes, default: DEFAULT_RETRY_STATUS_CODES ) –

    What to retry. A sequence of codes is every method for each of them; a mapping says, per code, which methods — None for all of them. 0 is the entry for an attempt nothing came back from.

  • delay_ms (int, default: 1000 ) –

    How long to wait before the second attempt, and what the growing delay starts from.

  • multiplier (float, default: 2.0 ) –

    What the delay is multiplied by per attempt already made. 1 waits the same every time.

  • max_delay_ms (int, default: 0 ) –

    The longest wait to allow. 0 sets no ceiling.

  • jitter (float, default: 0.1 ) –

    How much of the delay is randomness, from 0 — none — to 1, where the wait lands anywhere from nothing to twice the delay.

  • random (Callable[[float, float], float], default: uniform ) –

    Where a number between two bounds comes from. Only worth passing to make a test's delays repeatable.

  • max_retry_after_ms (int, default: _MAX_RETRY_AFTER_MS ) –

    The longest wait a server's Retry-After may ask for, when max_delay_ms sets no ceiling of its own: a minute by default, so an answer cannot park the caller for as long as it likes.

Raises:

  • InvalidArgumentError –

    When a delay is negative, the multiplier is below one, or the jitter is outside 0 to 1; each of them is a strategy that would wait for a length of time nobody meant.

should_retry async

should_retry(context: RetryContext, content: bytes | None, error: TransportError | None) -> bool | None

Say whether the attempt's status code is one on the list.

Never None: the body says nothing about a status code, so this strategy never needs it to decide.

Parameters:

  • context (RetryContext) –

    The attempt, whose status code and method are read.

  • content (bytes | None) –

    Ignored.

  • error (TransportError | None) –

    What went wrong when nothing came back, which sends the decision to the 0 entry.

Returns:

  • bool | None –

    Whether to try again.

get_delay

get_delay(context: RetryContext, content: bytes | None, error: TransportError | None) -> int

Return the delay for the next attempt: the server's Retry-After, or a growing one.

A wait the answer asks for is obeyed up to max_delay_ms — or up to max_retry_after_ms when that sets no ceiling.

Parameters:

  • context (RetryContext) –

    The attempt just made, whose retry_count is the power the multiplier is raised to.

  • content (bytes | None) –

    Ignored.

  • error (TransportError | None) –

    Ignored.

Returns:

  • int –

    The wait in whole milliseconds, never above max_delay_ms

  • int –

    unless that is 0.

RetryContext dataclass

RetryContext(method: str, url: str, options: RequestOptions, retry_count: int, response: HttpResponseInterface | None)

What one attempt was made of, and what came back from it.

Handed to a :class:RetryStrategyInterface <xtr_http_client.retry.retry_strategy_interface.RetryStrategyInterface> so it can decide on the attempt rather than on the request alone: the same 503 is worth repeating on the first try and not on the fourth, and a method that may be sent twice is not every method.

The request side is carried as it was sent — the URL resolved against the base URI this attempt used, the options merged — so a strategy reading them reads what actually went out.

Attributes:

  • method (str) –

    The HTTP method the attempt was made with.

  • url (str) –

    The URL it was made to, as it was given to the wrapped client.

  • options (RequestOptions) –

    The options it was made with, this attempt's base_uri among them.

  • retry_count (int) –

    How many attempts came before this one, so 0 is the first.

  • response (HttpResponseInterface | None) –

    What came back, or None when nothing did and the attempt ended in a transport error.

get_status_code

get_status_code() -> int

Return the status the attempt was answered with.

Returns:

  • int –

    The three-digit code, or 0 when nothing was answered — which

  • int –

    is the same thing a response says before its status line arrives,

  • int –

    so a strategy has one number to read either way.

get_info

get_info(key: str | None = None) -> object

Return what the attempt's transfer knows about itself.

Parameters:

  • key (str | None, default: None ) –

    One entry to look up. None asks for all of them.

Returns:

  • object –

    What the response reports; with nothing to report — an attempt

  • object –

    that ended in a transport error — every entry is empty and one

  • object –

    entry is None.

RetryStrategyInterface

Bases: Protocol

The two questions a retrying client asks itself after an attempt.

Whether to try again is one decision, how long to wait before doing so is another, and they are asked separately because a strategy that answers the first often has nothing of its own to say about the second. :class:RetryableHttpClient <xtr_http_client.retryable_http_client.RetryableHttpClient> asks both.

should_retry async

should_retry(context: RetryContext, content: bytes | None, error: TransportError | None) -> bool | None

Say whether the attempt in context is worth making again.

None is the third answer: "not without the body". The client then reads the whole body and asks again with it — once. Answering None a second time is a strategy that cannot decide, and is reported as such rather than taken for a "no".

Parameters:

  • context (RetryContext) –

    The attempt, and whatever came back from it.

  • content (bytes | None) –

    The whole body, once the client has read it because this method asked for it. None the first time round, and whenever the attempt ended in a transport error, where there is no body to read.

  • error (TransportError | None) –

    What went wrong when nothing came back, or None when something did.

Returns:

  • bool | None –

    True to try again, False to hand back what came back, or

  • bool | None –

    None to be asked again with the body.

get_delay

get_delay(context: RetryContext, content: bytes | None, error: TransportError | None) -> int

Return how long to wait before the next attempt, in milliseconds.

Only asked when the answer has not already been given by the server: a Retry-After header is obeyed over this.

Parameters:

  • context (RetryContext) –

    The attempt just made, whose retry_count is what a backing-off delay grows with.

  • content (bytes | None) –

    The body, when it was read to make the decision.

  • error (TransportError | None) –

    What went wrong when nothing came back.

Returns:

  • int –

    The wait in milliseconds; 0 to try again at once.

RetryableHttpClient

RetryableHttpClient(client: HttpClientInterface, strategy: RetryStrategyInterface | None = None, max_retries: int = 3, logger: LoggerInterface | None = None, sleep: Callable[[float], Awaitable[None]] = asyncio.sleep)

Bases: HttpClientInterface, ResetInterface

Wraps a client and makes the request again while a strategy says to.

A server that answered 503, or a connection that never came up, has not answered the question that was asked — and often answers it on the next try. Written at the call site that is a loop, a counter and a delay around every request; here it is the client, so the code calling out is unchanged and the decision lives in one place.

client = RetryableHttpClient(inner, max_retries=2)
async with await client.request("GET", "https://example.com/orders") as response:
    orders = await response.to_array()

What is retried, and how long is waited between tries, is the :class:RetryStrategyInterface <xtr_http_client.retry.retry_strategy_interface.RetryStrategyInterface>'s — :class:GenericRetryStrategy <xtr_http_client.retry.generic_retry_strategy.GenericRetryStrategy> by default, which retries the status codes worth retrying and backs off between attempts. A Retry-After header wins over its delay: a server saying when to come back is better informed than a client guessing.

Two things bound it. max_retries — which a request may lower or raise through the max_retries option — is how many further attempts are allowed, so the request is made at most that many times plus one; and a body that cannot be produced twice is sent once, because a second attempt would send a request that is not the one that failed.

The base_uri option may name several, and then each attempt moves one along it — the way a client is pointed at a second host rather than at the host that has just refused. The response handed back reports how many tries it took, under the retry_count entry of get_info().

Retry what client sends, as often as max_retries allows.

Parameters:

  • client (HttpClientInterface) –

    The client every attempt is handed to.

  • strategy (RetryStrategyInterface | None, default: None ) –

    What decides whether an attempt is worth repeating. None takes :class:GenericRetryStrategy <xtr_http_client.retry.generic_retry_strategy.GenericRetryStrategy> as it comes.

  • max_retries (int, default: 3 ) –

    How many further attempts a request may have. Zero or less is a client that passes everything straight through.

  • logger (LoggerInterface | None, default: None ) –

    Where each retry is reported, at info level. None reports none.

  • sleep (Callable[[float], Awaitable[None]], default: sleep ) –

    What waiting between attempts is done with. Only worth passing to make a test's waits instant.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request for method and url, again while that is worth doing.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to a base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made. max_retries is this client's own and is not passed on.

Returns:

Raises:

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send request, again while that is worth doing.

Parameters:

  • request (RequestInterface) –

    The request to send. Every attempt sends this very request, its body rewound first.

  • **options (Unpack[RequestOptions], default: {} ) –

    What the request leaves out. max_retries is this client's own and is not passed on.

Returns:

Raises:

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a copy around a wrapped client whose defaults have options over them.

base_uri and max_retries stay here rather than going on: a list of base URIs is this client's to walk, and how many attempts are allowed is not something the client behind it can act on.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the wrapped client of the copy.

Returns:

  • Self –

    A new client, retrying by the same strategy; this one is left as

  • Self –

    it was.

Raises:

reset async

reset() -> None

Reset the wrapped client.

Nothing here outlives a request — how many attempts are allowed and what decides them are configuration — so there is only the client behind to pass this on to.

ScopingHttpClient

ScopingHttpClient(client: HttpClientInterface, default_options_by_regexp: Mapping[str, RequestOptions], default_regexp: str | None = None)

Bases: HttpClientInterface, ResetInterface

Wraps a client and gives each request the options its URL falls under.

One application talks to several services, and each wants its own credentials, headers and deadlines. Written at the call sites, that is the same four options repeated wherever a service is reached, and one place that forgot them. Here it is said once: a scope is a regular expression the start of a URL is matched against, and the options every request landing in it starts from.

client = ScopingHttpClient(
    inner,
    {r"https://api\.example\.com/": {"auth_bearer": "t0ken"}},
)
response = await client.request("GET", "https://api.example.com/orders")

What the request itself asked for always wins — a scope fills in what was left out, it never overrides. headers is the one option merged rather than replaced, name by name, so a scope-wide Accept survives a request setting a Content-Type of its own.

With a default_regexp, that scope's base_uri also resolves a relative URL, which is what makes :meth:for_base_uri read as "this client is that service".

Scope client, matching a URL against each regexp in turn.

Parameters:

  • client (HttpClientInterface) –

    The client every vetted request is handed to.

  • default_options_by_regexp (Mapping[str, RequestOptions]) –

    Each scope's pattern mapped to the options a request falling in it starts from. The first pattern matching the start of a URL is the one that applies, so order them from the most specific to the least.

  • default_regexp (str | None, default: None ) –

    Which of those scopes a relative URL belongs to, and whose base_uri resolves it. None leaves a relative URL to the wrapped client.

Raises:

  • InvalidArgumentError –

    When default_regexp names a scope that is not in default_options_by_regexp — a default scope with no options is a client whose relative URLs quietly resolve against nothing.

for_base_uri classmethod

for_base_uri(client: HttpClientInterface, base_uri: str, default_options: RequestOptions | None = None, regexp: str | None = None) -> Self

Scope client to one service, reached at base_uri.

Parameters:

  • client (HttpClientInterface) –

    The client every request is handed to.

  • base_uri (str) –

    What a relative URL resolves against, and — unless regexp says otherwise — what a URL must start with to be in scope.

  • default_options (RequestOptions | None, default: None ) –

    What a request in scope starts from.

  • regexp (str | None, default: None ) –

    The scope's pattern, for a service whose URLs are not all under one base.

Returns:

  • Self –

    A client scoped to that one base URI.

for_base_uris classmethod

for_base_uris(client: HttpClientInterface, base_uris: Sequence[str], default_options: RequestOptions | None = None, regexp: str | None = None) -> Self

Scope client to one service reachable at several base URIs.

The scope matches all of them, so the options keep applying whichever one a request is resolved against; the first is what a relative URL resolves against.

Parameters:

  • client (HttpClientInterface) –

    The client every request is handed to.

  • base_uris (Sequence[str]) –

    Where the service answers, the first being the default.

  • default_options (RequestOptions | None, default: None ) –

    What a request in scope starts from.

  • regexp (str | None, default: None ) –

    The scope's pattern, replacing the one built from them.

Returns:

  • Self –

    A client scoped to all of those base URIs.

Raises:

  • InvalidArgumentError –

    When no base URI is given, since a scope over nothing would match nothing and resolve nothing.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request for method and url, under the scope it falls in.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to a base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made; the scope fills in what is left out.

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came.

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send request under the scope the URI it carries falls in.

Parameters:

  • request (RequestInterface) –

    The request to send; it goes on carrying the URL this client resolved for it.

  • **options (Unpack[RequestOptions], default: {} ) –

    What the request leaves out; the scope fills in what both leave out.

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came.

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a copy around a wrapped client whose defaults have options over them.

The scopes are kept as they are: they say what belongs to which service, which is not something a caller narrowing a client for one job is changing.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the wrapped client of the copy.

Returns:

  • Self –

    A new client; this one is left as it was.

Raises:

reset async

reset() -> None

Reset the wrapped client, if it holds state between units of work.

The scopes are configuration, not state, so they are kept.

ThrottlingHttpClient

ThrottlingHttpClient(client: HttpClientInterface, limiter: LimiterInterface)

Bases: HttpClientInterface, ResetInterface

Wraps a client and lets each request go only once the rate allows it.

An API that answers a hundred calls a minute answers the hundred and first with a refusal, and a caller that keeps asking is throttled by the service it is calling rather than by itself. Written at the call site that is a counter and a sleep around every request; here it is the client, so the code calling out is unchanged and the pace is kept in one place — including between several callers, when the limiter they share counts where they can all see it.

client = ThrottlingHttpClient(inner, limiter)
async with await client.request("GET", "https://api.example.com/orders") as response:
    orders = await response.to_array()

A token is booked before each request, and the request waits for the moment that booking is due. Booking first and waiting after is what makes the pace hold under concurrency: the limiter counts the token as spent as soon as it is reserved, so requests leave in the order they were asked for, each at its own turn, rather than all at once as a limiter merely asked whether it may go ahead would allow.

The limiter is the pace, so a copy made by :meth:with_options shares it: narrowing a client for one job hands out another way of making requests, not another allowance to make them.

Pace what client sends by limiter.

Parameters:

  • client (HttpClientInterface) –

    The client every request is handed to, once it is due.

  • limiter (LimiterInterface) –

    What a token is booked from before each request. Which requests share a pace is decided by which of them are given the same limiter — one per host, one per account, one for the whole application.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request for method and url, once a token is due.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to a base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made.

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came.

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send request, once a token is due.

Parameters:

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came.

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a copy around a wrapped client whose defaults have options over them.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the wrapped client of the copy.

Returns:

  • Self –

    A new client, paced by this one's limiter, so requests made

  • Self –

    through either count against the same rate; this one is left as

  • Self –

    it was.

Raises:

  • InvalidArgumentError –

    When an option is invalid.

reset async

reset() -> None

Reset the wrapped client, keeping the pace.

The rate is shared on purpose — by every unit of work and every copy made with :meth:with_options — so the end of one unit forgets none of what was counted against it.

TraceableHttpClient

TraceableHttpClient(client: HttpClientInterface, disabled: Callable[[], bool] | None = None, recorder: Callable[[TracedRequest], None] | None = None)

Bases: HttpClientInterface, ResetInterface

Wraps a client and writes down every request made through it.

"Which calls did this go on to make, with what, and what came back" is asked of a running application constantly — by a profiler, by a test asserting on a call it has no other sight of, by whoever is reading a log that stops short of the call. Answered at the call site that is logging around every request; here it is the client, so the code calling out is unchanged and the record is one list.

client = TraceableHttpClient(inner)
async with await client.request("GET", "https://example.com/orders") as response:
    orders = await response.to_array()

traced = client.get_traced_requests()
assert traced[0]["method"] == "GET"
assert traced[0]["content"] == b'{"orders": []}'

A record is added the moment a request is made and filled in as that request goes on: info when the response arrives — or the error alone, when nothing arrives and the error is raised on — and content when the caller reads the body. So a trace read while a request is still being answered says how far it has got, rather than nothing until it is over.

The bodies are the costly half of a record and the half most likely to carry a credential, so extra={"trace_content": False} leaves them out: the body and json that were sent are dropped from options, and content is False rather than the bytes. Nothing is recorded at all for a request made while disabled says so, which is how a profiler that collects per request stops collecting between them.

The record is the point, so a copy made by :meth:with_options shares it: narrowing a client for one job hands out another way of making requests, all of them written into the one list.

The record keeps the last :attr:MAX_TRACES requests, so a client that is never reset does not grow without bound, and it never keeps a credential as it was sent.

A recorder is handed each record the moment it is written — the live record, filled in afterwards like the one in the list — so whoever keeps records elsewhere, a profiler keeping each request's own, sees them as they are made without reading the list. Copies hand theirs to it too.

Write down what client is asked to send, unless disabled says not to.

Parameters:

  • client (HttpClientInterface) –

    The client every request is handed to.

  • disabled (Callable[[], bool] | None, default: None ) –

    Asked before each request whether to record it. None records every one.

  • recorder (Callable[[TracedRequest], None] | None, default: None ) –

    Handed every record as it is written. None hands them to nobody.

MAX_TRACES class-attribute instance-attribute

MAX_TRACES: Final[int] = 1000

How many requests the record keeps, the oldest dropped first.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request for method and url, and write it down.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to a base_uri.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made.

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came, which is written down before it is raised on.

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send request, and write it down.

Parameters:

  • request (RequestInterface) –

    The request to send, whose method and URI are what the record names it by.

  • **options (Unpack[RequestOptions], default: {} ) –

    What the request leaves out.

Returns:

Raises:

  • InvalidArgumentError –

    For an invalid method, URL or option.

  • TransportError –

    When no answer came, which is written down before it is raised on.

get_traced_requests

get_traced_requests() -> list[TracedRequest]

Return the record of every request made through this client so far.

Returns:

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a copy around a wrapped client whose defaults have options over them.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the wrapped client of the copy.

Returns:

  • Self –

    A new client writing into this one's record, so a request made

  • Self –

    through either is traced in one place; this one is left as it was.

Raises:

  • InvalidArgumentError –

    When an option is invalid.

reset async

reset() -> None

Forget every request written down, and reset the wrapped client.

The record goes for every client sharing it, copies included: there is one record, and this ends the unit of work it was kept for.

TracedRequest

Bases: TypedDict

The record of one request made through a tracing client.

Attributes:

  • method (str) –

    The HTTP method the request was made with.

  • url (str) –

    The URL it was made to, as it was asked for.

  • options (RequestOptions) –

    The options it was made with, as they were given — without body and json when extra.trace_content turned the bodies off, and with every credential (auth_basic, auth_bearer, an Authorization, Cookie or Proxy-Authorization header) written as "***".

  • info (Mapping[str, object]) –

    What the transfer knew about itself when the response arrived. Empty until it arrives, and carrying error alone when nothing does.

  • content (bytes | bool | None) –

    Every byte of the body, once the caller has read it; None while nobody has, and False when extra.trace_content turned the bodies off.