xtr_http_client ¶
An async HTTP client: typed options in, a response read on demand out.
One interface with clients behind it — one that opens real connections and
streams what comes back, one that answers from a script a test wrote, five
that wrap another and add what every application writes again (a set of
defaults per service, a refusal to reach inwards, a request made again while
that is worth doing, a pace requests are held to, and a record of every
request made), and :class:HttpClient, the one call that hands one over so
nothing else has to name it.
A request returns once the status line and headers have arrived, whatever the status; the body is read when the caller asks for it, as a whole or piece by piece.
HttpClient ¶
One call that hands back a client, so nothing has to name an implementation.
Code that calls out depends on the interface. Code that builds a client has to pick something, and picking is a decision worth making in one place: this is that place.
from xtr_http_client import HttpClient
client = HttpClient.create(base_uri="https://example.com/", timeout=5)
It is not a client itself and holds nothing; there is nothing to keep an instance of.
create
staticmethod
¶
create(max_host_connections: int = 6, **default_options: Unpack[RequestOptions]) -> HttpClientInterface
Return a client whose requests start from default_options.
Parameters:
-
max_host_connections(int, default:6) –How many connections one pool may hold at once.
-
**default_options(Unpack[RequestOptions], default:{}) –What every request the client makes starts from, over the contract's own defaults.
Returns:
-
HttpClientInterface–A client, typed as the interface: what is behind it is this
-
HttpClientInterface–package's decision, not the caller's.
Raises:
-
InvalidArgumentError–When an option is not one the client knows.
HttpClientMixin ¶
What a client refuses, and the options it starts every request from.
Mixed into every client here, so all of them keep their defaults the same way and refuse the same requests — which is what lets one be swapped for another without a caller noticing. A client is then only the part that opens connections.
Its own state is the one thing a client needs from it: the options every
request starts from. __init__ takes them so a subclass can hand over
what it was built with and get back the merged defaults.
The checks are exposed one by one because a client makes them one at a time: the method is known before the options are read, and a client preparing its own defaults has no method to check at all.
Start from the contract's defaults, with default_options over them.
Raises:
-
InvalidArgumentError–When an option is not one this client knows.
with_options ¶
A copy of this client whose defaults have options over them.
A copy rather than a change, so a caller can narrow a client for one job — a header, a timeout, a base URI — without the client everyone else holds changing under them.
Raises:
-
InvalidArgumentError–When an option is not one this client knows.
merge_default_options
classmethod
¶
merge_default_options(options: Mapping[str, object], default_options: Mapping[str, object]) -> dict[str, object]
Merge what a request asked for over what the client always asks for.
See :func:xtr_http_client.internal.option_merge.merge_default_options.
validate_options ¶
Refuse an option this client does not know, naming the closest it does.
See :func:xtr_http_client.internal.option_merge.validate_options.
validate_method
staticmethod
¶
Refuse a method no server would read as the one that was meant.
Raises:
-
InvalidArgumentError–When the method is empty, or is anything other than uppercase ASCII letters.
validate_auth
staticmethod
¶
Refuse credentials this client cannot send, or two sets of them.
Raises:
-
InvalidArgumentError–When
auth_basicorauth_beareris not of a shape credentials can be read from, or both are set.
validate_body
staticmethod
¶
Refuse a request that says what to send twice.
A JSON document and a body are both the whole body, so a request carrying each has not said which one to send.
Raises:
-
InvalidArgumentError–When
jsonandbodyare both set.
HttpxHttpClient ¶
Bases: HttpClientMixin, HttpClientInterface, ResetInterface
Sends requests over real connections, and hands back answers as they arrive.
A request returns once the status line and the headers are in, whatever the status; the body is read off the connection when the caller asks for it, and the connection is held until the body is done with or the response is left.
The request is authoritative and the options fill what it leaves out — a
header it already carries is the one sent, credentials apply only where it
has no Authorization of its own — which is what lets one set of
client-wide defaults sit under requests that each know their own mind.
What each part of an outgoing request is assembled from is in
:mod:xtr_http_client.internal.httpx_request, and the redirect chain an
exchange may turn into is followed in
:mod:xtr_http_client.internal.httpx_redirect; what is here is the
exchange itself, under its deadline.
Connections are pooled by how they connect rather than by host: a proxy,
a trust store, a protocol version and the deadlines all decide what a
connection is, so each combination of them gets a pool, and
max_host_connections is each pool's ceiling across every host it
reaches.
A host a request pins to an address through resolve stays pinned for
the requests after it, in the pool it went through; :meth:reset forgets
every pin, and keeps the connections open until the client is closed.
async with HttpxHttpClient(max_host_connections=4) as client:
response = await client.request("GET", "https://example.com/orders")
orders = await response.to_array()
Redirects are followed here rather than by the backend, so that an
Authorization header is dropped the moment a chain leaves the host it
was meant for, and comes back only when the chain returns to it.
Start from default_options, opening nothing until a request is made.
Parameters:
-
default_options(RequestOptions | None, default:None) –What every request this client makes starts from, over the contract's own defaults.
-
max_host_connections(int, default:6) –How many connections one pool may hold at once.
Raises:
-
InvalidArgumentError–When a default option is not one this client knows.
request
async
¶
Make a request for method and url, and return once its head is in.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made.
Returns:
-
HttpResponse–The response, with its status and headers already in.
Raises:
-
TransportTimeoutError–When no answer came in time.
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came.
send_request
async
¶
Send request, the options filling what it leaves out.
Parameters:
-
request(RequestInterface) –The request to send.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
Returns:
-
HttpResponse–The response, with its status and headers already in.
Raises:
-
TransportTimeoutError–When no answer came in time.
-
InvalidArgumentError–For an invalid method, URL or option, and for a body given both with the request and as an option.
-
TransportError–When no answer came.
reset
async
¶
Keep everything: the connections are how the next unit gets anywhere.
A request's resolve pins its own request only, so there is nothing
a unit of work left behind to forget.
MockHttpClient ¶
MockHttpClient(response_factory: ResponseFactory | None = None, base_uri: str | None = 'https://example.com')
Bases: HttpClientMixin, HttpClientInterface, ResetInterface
Answers every request from what a test wrote out, and opens no connection.
It is the same client as any other here — the options it refuses, the defaults it starts from and the way a request and its options are merged are all the ones a real transfer uses — with the transfer replaced by a script. So a test exercises the code that calls out, down to the options it passed, without a server to point it at.
client = MockHttpClient([MockResponse(b"first"), MockResponse(b"second")])
async with await client.request("GET", "/orders") as response:
assert await response.get_content() == b"first"
assert client.get_requests_count() == 1
What the answers come from is whatever a test finds easiest to write:
- nothing, and every request is answered with an empty 200;
- one :class:
MockResponse <xtr_http_client.response.mock_response.MockResponse>, which answers one request, and a second request is a mistake, reported as the script running out; - an iterable of them, one per request, in order — a request past the end of it is a mistake and is reported as one, since a test that makes more requests than it wrote answers for is no longer testing what it says;
- a callable, which is handed the method, the URL and the merged options and says what to answer with — the way to assert on the request that was made, and to answer differently depending on it. It may be async.
An iterable may hold callables too, so one request in a sequence can be answered that way while the rest are written out.
Nothing is followed and nothing is resolved: a 3xx is the answer, and a
relative URL is resolved against base_uri — which starts at
https://example.com so a test can say /orders and mean something.
Start from response_factory, which nothing is taken from until asked.
Parameters:
-
response_factory(ResponseFactory | None, default:None) –Where the answers come from.
-
base_uri(str | None, default:'https://example.com') –What a relative URL is resolved against.
Noneleaves a relative URL unresolvable, which is then refused.
set_response_factory ¶
Answer from response_factory from now on.
What was left of the previous one is dropped, and the count of requests already made is kept: a test that changes its mind about the answers has not unmade the requests.
Parameters:
-
response_factory(ResponseFactory | None, default:None) –Where the answers come from.
get_requests_count ¶
Return how many requests this client has been asked to make.
request
async
¶
Answer a request for method and url from the script.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request would have been made.
Returns:
-
MockResponse–The scripted answer, issued for this request.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When the script says the transfer fails, or has no answer left to give.
send_request
async
¶
Answer request from the script, the options filling what it leaves out.
Parameters:
-
request(RequestInterface) –The request that would have been sent.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
Returns:
-
MockResponse–The scripted answer, issued for this request.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option, and for a body given both with the request and as an option.
-
TransportError–When the script says the transfer fails, has no answer left to give, or gives something that is not an answer.
NoPrivateNetworkHttpClient ¶
NoPrivateNetworkHttpClient(client: HttpClientInterface, subnets: str | Sequence[str] | None = None, allow_list: str | Sequence[str] = (), resolver: Callable[[str, int], Awaitable[Sequence[str]]] | None = None)
Bases: HttpClientInterface, ResetInterface
Wraps a client and refuses every request that leads inwards.
A URL that came from outside is data, so without this a request is a way to make this process reach an address only this process can see: another service on the same network, an admin port, a metadata endpoint handing out credentials.
client = NoPrivateNetworkHttpClient(inner)
response = await client.request("GET", "https://example.com/avatar")
The check runs on what the host resolves to, not on how it is written,
and one refused address refuses the whole request — whichever of them the
connection would pick, nothing is sent. The request is then pinned to the
address that was checked, through the resolve option, so a name whose
records change between the check and the connection reaches what was
vetted or nothing at all.
Redirects are followed here rather than by the wrapped client, because a hop nobody checked would walk straight past the guard: each one is resolved and checked like the request that started the chain.
What a host resolves to is remembered for one request's chain of hops, so hops back to the same host ask the resolver once; the next request asks again, so no answer outlives the request that asked for it.
The networks it refuses come from xtr-http-foundation, which is not a dependency of this package: install it beside it to use this client.
Guard client, refusing whichever networks subnets names.
Parameters:
-
client(HttpClientInterface) –The client every vetted request is handed to.
-
subnets(str | Sequence[str] | None, default:None) –The networks to refuse, in CIDR notation or as single addresses, replacing the default.
Nonerefuses the networks only this one can reach. -
allow_list(str | Sequence[str], default:()) –Networks or addresses exempt from the refusal, which is how one internal receiver is opened without opening the network it sits in. Addresses only, never host names: the check runs on resolved addresses, and a name would itself have to be resolved, and could point anywhere by then.
-
resolver(Callable[[str, int], Awaitable[Sequence[str]]] | None, default:None) –What every address of a host and port is asked of.
Noneasks the platform's own resolver.
Raises:
-
LogicError–When xtr-http-foundation is not installed, since there is then no way to tell which addresses to refuse.
request
async
¶
Make a request for method and url, once where it leads is vetted.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers at the end of the chain.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When the host is, or resolves to, a refused address, and when no answer came.
send_request
async
¶
Send request, vetting every host the chain reaches.
Parameters:
-
request(RequestInterface) –The request to send.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers at the end of the chain.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option, and for a URL that is relative with no
base_urito resolve it. -
TransportError–When a host is, or resolves to, a refused address, and when no answer came.
with_options ¶
Return a copy around a wrapped client whose defaults have options over them.
Parameters:
-
**options(Unpack[RequestOptions], default:{}) –The defaults of the wrapped client of the copy.
Returns:
-
Self–A new client, guarding the same networks.
Raises:
-
InvalidArgumentError–When an option is invalid.
HttpResponse ¶
HttpResponse(message: Response, body: ResponseBody, info: dict[str, object], on_cancel: Callable[[], Awaitable[None]] | None = None)
Bases: HttpResponseInterface
A response whose head has arrived and whose body is read when it is asked for.
Three parts are held apart because they arrive apart: the message — the status line and the headers, which came in whole — the body, which is still coming off the transfer, and the info, which the transfer goes on writing to as it does.
A with_* copy therefore copies the message alone. The body is the one
transfer's and cannot be had twice, and the info is that transfer's own
record, so every copy reads the one mapping and sees whatever has been
written to it since.
Hold the parts of a response that arrived apart.
Parameters:
-
message(Response) –The status line and the headers, as they came in.
-
body(ResponseBody) –The body, taken off the transfer as it is asked for.
-
info(dict[str, object]) –What the transfer knows about itself. Kept rather than copied, because the transfer is still writing to it.
-
on_cancel(Callable[[], Awaitable[None]] | None, default:None) –What abandoning the transfer takes, when there is anything: the client that opened it knows, a response does not.
get_status_code ¶
Return the status code the server answered with.
Returns:
-
int–The three-digit code off the status line.
get_reason_phrase ¶
Return the reason phrase that goes with the status code.
Returns:
-
str–The phrase, or an empty string when the status line carried none.
with_status ¶
Return a copy carrying code and, optionally, reason_phrase.
Parameters:
-
code(int) –The status code to set.
-
reason_phrase(str, default:'') –The phrase to go with
code.
Returns:
-
HttpResponse–A new response; this one is left as it was.
get_protocol_version ¶
Return the HTTP version the answer came back on.
Returns:
-
str–The version number alone, such as
"1.1".
with_protocol_version ¶
Return a copy carrying version.
Parameters:
-
version(str) –The HTTP version number alone.
Returns:
-
HttpResponse–A new response; this one is left as it was.
get_headers ¶
has_header ¶
get_header ¶
get_header_line ¶
with_header ¶
Return a copy whose name header holds value and nothing else.
Parameters:
-
name(str) –The header name, in any case.
-
value(str | Sequence[str]) –One value, or every value the header is to hold.
Returns:
-
HttpResponse–A new response; this one is left as it was.
with_added_header ¶
Return a copy with value appended to the name header.
Parameters:
-
name(str) –The header name, in any case.
-
value(str | Sequence[str]) –One value, or every value to append.
Returns:
-
HttpResponse–A new response; this one is left as it was.
without_header ¶
Return a copy with the name header gone.
Parameters:
-
name(str) –The header name, in any case.
Returns:
-
HttpResponse–A new response; this one is left as it was.
get_body ¶
Return the body, which reads off the transfer as it is read.
Returns:
-
StreamInterface–The one body of this transfer, shared by every copy of the
-
StreamInterface–response.
with_body ¶
Return a copy reading its body from body instead.
Parameters:
-
body(StreamInterface) –The stream the copy is to read its body from, in place of the one coming off the transfer.
Returns:
-
HttpResponse–A new response; this one is left as it was, still over the
-
HttpResponse–transfer's own body.
get_info ¶
Return what the transfer knows about itself, as it stands.
The mapping handed back is a copy, so reading the info of a transfer still running cannot be disturbed by the transfer writing to it, and nothing a caller does to it reaches the transfer.
Parameters:
-
key(str | None, default:None) –One entry to look up.
Noneasks for all of them.
Returns:
cancel
async
¶
Abandon the transfer and let go of what it was holding.
The record is written before anything is let go of, so a reader that finds the body closed can be told why it is closed rather than only that it is.
get_content
async
¶
Read the whole body, waiting for the rest of it to arrive.
Parameters:
-
throw(bool, default:True) –Whether an unsuccessful status is to be raised rather than answered with the body that came with it.
Returns:
-
bytes–Every byte of the body, empty when there was none.
Raises:
-
TransportError–When the body cannot be read — the transfer broke, or it was cancelled.
-
RedirectionError–On a 3xx, when
throw. -
ClientError–On a 4xx, when
throw. -
ServerError–On a 5xx, when
throw.
to_array
async
¶
Read the whole body and decode it as a JSON object.
Parameters:
-
throw(bool, default:True) –Whether an unsuccessful status is to be raised rather than answered with the body that came with it.
Returns:
Raises:
-
JsonError–When the body is empty, is not JSON, or is JSON that is not an object.
-
TransportError–When the body cannot be read.
-
RedirectionError–On a 3xx, when
throw. -
ClientError–On a 4xx, when
throw. -
ServerError–On a 5xx, when
throw.
JsonMockResponse ¶
Bases: MockResponse
A written answer carrying a value encoded as JSON, labelled as JSON.
The one thing a JSON answer adds to a scripted one: the body is a value rather than bytes, and the content type says what it is — unless the script named one of its own, which a test does to answer with a problem document.
client = MockHttpClient(JsonMockResponse({"id": 7}, {"http_code": 201}))
async with await client.request("GET", "/orders") as response:
assert await response.to_array() == {"id": 7}
Write out an answer carrying body as JSON.
Parameters:
-
body(object, default:None) –Any value JSON can be made of, encoded as it stands.
None, the default, is not encoded: it answers an empty list, so an answer nobody wrote out is still a document. To sendnull, write the bytes with :class:MockResponseinstead. -
info(Mapping[str, object] | None, default:None) –What the transfer is to claim about itself, as :class:
MockResponsetakes it. Acontent-typeamong itsresponse_headersis left alone; one is added when there is none.
Raises:
-
InvalidArgumentError–When the value cannot be encoded as JSON.
from_file
classmethod
¶
Write out an answer carrying the JSON document a file holds.
The file is read now rather than when the response is issued, so a test that names the wrong file is told so where it named it.
Parameters:
-
path(str | PathLike[str]) –The file to read.
-
info(Mapping[str, object] | None, default:None) –What the transfer is to claim about itself.
Returns:
-
Self–The response, carrying the document the file holds.
Raises:
-
InvalidArgumentError–When there is no such file, or what it holds is not JSON.
MockResponse ¶
Bases: HttpResponseInterface
An answer written before the question, which a client hands back as its own.
A test says what it wants to come back — a body, a status, headers, or an
error instead of an answer — and :class:MockHttpClient
<xtr_http_client.mock_http_client.MockHttpClient> answers with it. Nothing
is sent and nothing is opened.
client = MockHttpClient(MockResponse(b'{"id": 7}', {"http_code": 201}))
async with await client.request("GET", "/orders") as response:
assert await response.to_array() == {"id": 7}
A written response is only half a response: the other half — the method,
the URL and the options it answers — is the client's, and is filled in when
the client issues it. Until then there is nothing to read, because what
came back is not yet an answer to anything; reading one raises
:class:InvalidArgumentError
<xtr_http_client_contracts.exception.invalid_argument_error.InvalidArgumentError>.
Once issued, it is an ordinary response of this package: the status,
headers, body, copies and errors are :class:HttpResponse
<xtr_http_client.response.http_response.HttpResponse>'s, over a body made
of the scripted pieces, so a test reads a scripted answer exactly as it
reads a real one.
Write out an answer.
Parameters:
-
body(bytes | str | Iterable[bytes], default:b'') –What the body is to be. Bytes or text are the whole of it; an iterable of bytes is handed over a piece at a time, which is how a test reads a body that arrives in pieces.
-
info(Mapping[str, object] | None, default:None) –What the transfer is to claim about itself —
http_code(200when left out),response_headers, anderrorfor a transfer that is to fail rather than answer. Any other entry is carried through toget_info()untouched.
issue ¶
Make this written answer the answer to a request that was just made.
What a client does to a response before handing it over: the request
side is filled in, and the body and the record of the transfer come
into being. Called by :class:MockHttpClient
<xtr_http_client.mock_http_client.MockHttpClient>.
Parameters:
-
method(str) –The method the request was made with.
-
url(str) –The URL it was made to, resolved.
-
options(Mapping[str, object]) –The options it was made with, merged.
Returns:
-
Self–This response, now readable.
Raises:
-
TransportError–When the script says the transfer fails rather than answers. Nothing is opened in that case, so there is nothing left to let go of.
get_request_method ¶
The method of the request this answered, empty until it answered one.
get_request_url ¶
The URL of the request this answered, empty until it answered one.
get_request_options ¶
The options the request was made with, merged as the client sent them.
get_status_code ¶
Return the status the script says the answer came back with.
with_status ¶
Return a copy carrying code and, optionally, reason_phrase.
get_protocol_version ¶
Return the HTTP version the script says the answer came back on.
get_headers ¶
Return every header, each name mapped to all of its values.
has_header ¶
Tell whether a header is there, matching name without case.
get_header ¶
Return every value recorded under a header, matched without case.
get_header_line ¶
Return the values of a header joined with commas.
with_header ¶
Return a copy whose name header holds value and nothing else.
with_added_header ¶
Return a copy with value appended to the name header.
get_body ¶
Return the body, over the pieces the script was written with.
with_body ¶
Return a copy reading its body from body instead.
get_info ¶
Return what the transfer knows about itself, as it stands.
Readable before the response is issued, where it is the script as it was written — which is what lets a cancelled response say so even though it never answered anything.
Parameters:
-
key(str | None, default:None) –One entry to look up.
Noneasks for all of them.
Returns:
get_content
async
¶
Read the whole body, as the script hands it over.
Parameters:
-
throw(bool, default:True) –Whether an unsuccessful status is to be raised rather than answered with the body that came with it.
Returns:
-
bytes–Every byte of the body, empty when the script wrote none.
Raises:
-
InvalidArgumentError–When the response has not been issued.
-
TransportError–When the body cannot be read.
-
RedirectionError–On a 3xx, when
throw. -
ClientError–On a 4xx, when
throw. -
ServerError–On a 5xx, when
throw.
to_array
async
¶
Read the whole body and decode it as a JSON object.
Parameters:
-
throw(bool, default:True) –Whether an unsuccessful status is to be raised rather than answered with the body that came with it.
Returns:
Raises:
-
InvalidArgumentError–When the response has not been issued.
-
JsonError–When the body is empty, is not JSON, or is JSON that is not an object.
-
TransportError–When the body cannot be read.
-
RedirectionError–On a 3xx, when
throw. -
ClientError–On a 4xx, when
throw. -
ServerError–On a 5xx, when
throw.
GenericRetryStrategy ¶
GenericRetryStrategy(status_codes: _StatusCodes = DEFAULT_RETRY_STATUS_CODES, delay_ms: int = 1000, multiplier: float = 2.0, max_delay_ms: int = 0, jitter: float = 0.1, random: Callable[[float, float], float] = uniform, *, max_retry_after_ms: int = _MAX_RETRY_AFTER_MS)
Bases: RetryStrategyInterface
Retries a status code on the list, after a delay that grows each time.
The decision is the status code and, for the codes where it matters, the
method: a 503 is the server saying "not now", so asking again is the
right thing whatever was asked; a 500 after a POST may have changed
something already, so sending it again is not.
strategy = GenericRetryStrategy([500, 502, 503], delay_ms=500)
client = RetryableHttpClient(inner, strategy, max_retries=2)
The delay starts at delay_ms and is multiplied by multiplier for
every attempt already made, so the server is given more room each time
rather than being asked again as fast as it refused. jitter spreads
that over a window around it, which is what keeps a thousand clients that
failed together from coming back together.
Retry what status_codes lists, backing off between attempts.
Parameters:
-
status_codes(_StatusCodes, default:DEFAULT_RETRY_STATUS_CODES) –What to retry. A sequence of codes is every method for each of them; a mapping says, per code, which methods —
Nonefor all of them.0is the entry for an attempt nothing came back from. -
delay_ms(int, default:1000) –How long to wait before the second attempt, and what the growing delay starts from.
-
multiplier(float, default:2.0) –What the delay is multiplied by per attempt already made.
1waits the same every time. -
max_delay_ms(int, default:0) –The longest wait to allow.
0sets no ceiling. -
jitter(float, default:0.1) –How much of the delay is randomness, from
0— none — to1, where the wait lands anywhere from nothing to twice the delay. -
random(Callable[[float, float], float], default:uniform) –Where a number between two bounds comes from. Only worth passing to make a test's delays repeatable.
-
max_retry_after_ms(int, default:_MAX_RETRY_AFTER_MS) –The longest wait a server's
Retry-Aftermay ask for, whenmax_delay_mssets no ceiling of its own: a minute by default, so an answer cannot park the caller for as long as it likes.
Raises:
-
InvalidArgumentError–When a delay is negative, the multiplier is below one, or the jitter is outside
0to1; each of them is a strategy that would wait for a length of time nobody meant.
should_retry
async
¶
should_retry(context: RetryContext, content: bytes | None, error: TransportError | None) -> bool | None
Say whether the attempt's status code is one on the list.
Never None: the body says nothing about a status code, so this
strategy never needs it to decide.
Parameters:
-
context(RetryContext) –The attempt, whose status code and method are read.
-
content(bytes | None) –Ignored.
-
error(TransportError | None) –What went wrong when nothing came back, which sends the decision to the
0entry.
Returns:
-
bool | None–Whether to try again.
get_delay ¶
Return the delay for the next attempt: the server's Retry-After, or a growing one.
A wait the answer asks for is obeyed up to max_delay_ms — or up
to max_retry_after_ms when that sets no ceiling.
Parameters:
-
context(RetryContext) –The attempt just made, whose
retry_countis the power the multiplier is raised to. -
content(bytes | None) –Ignored.
-
error(TransportError | None) –Ignored.
Returns:
RetryContext
dataclass
¶
RetryContext(method: str, url: str, options: RequestOptions, retry_count: int, response: HttpResponseInterface | None)
What one attempt was made of, and what came back from it.
Handed to a :class:RetryStrategyInterface
<xtr_http_client.retry.retry_strategy_interface.RetryStrategyInterface>
so it can decide on the attempt rather than on the request alone: the same
503 is worth repeating on the first try and not on the fourth, and a
method that may be sent twice is not every method.
The request side is carried as it was sent — the URL resolved against the base URI this attempt used, the options merged — so a strategy reading them reads what actually went out.
Attributes:
-
method(str) –The HTTP method the attempt was made with.
-
url(str) –The URL it was made to, as it was given to the wrapped client.
-
options(RequestOptions) –The options it was made with, this attempt's
base_uriamong them. -
retry_count(int) –How many attempts came before this one, so
0is the first. -
response(HttpResponseInterface | None) –What came back, or
Nonewhen nothing did and the attempt ended in a transport error.
RetryStrategyInterface ¶
Bases: Protocol
The two questions a retrying client asks itself after an attempt.
Whether to try again is one decision, how long to wait before doing so is
another, and they are asked separately because a strategy that answers the
first often has nothing of its own to say about the second.
:class:RetryableHttpClient
<xtr_http_client.retryable_http_client.RetryableHttpClient> asks both.
should_retry
async
¶
should_retry(context: RetryContext, content: bytes | None, error: TransportError | None) -> bool | None
Say whether the attempt in context is worth making again.
None is the third answer: "not without the body". The client then
reads the whole body and asks again with it — once. Answering None
a second time is a strategy that cannot decide, and is reported as
such rather than taken for a "no".
Parameters:
-
context(RetryContext) –The attempt, and whatever came back from it.
-
content(bytes | None) –The whole body, once the client has read it because this method asked for it.
Nonethe first time round, and whenever the attempt ended in a transport error, where there is no body to read. -
error(TransportError | None) –What went wrong when nothing came back, or
Nonewhen something did.
Returns:
get_delay ¶
Return how long to wait before the next attempt, in milliseconds.
Only asked when the answer has not already been given by the server: a
Retry-After header is obeyed over this.
Parameters:
-
context(RetryContext) –The attempt just made, whose
retry_countis what a backing-off delay grows with. -
content(bytes | None) –The body, when it was read to make the decision.
-
error(TransportError | None) –What went wrong when nothing came back.
Returns:
-
int–The wait in milliseconds;
0to try again at once.
RetryableHttpClient ¶
RetryableHttpClient(client: HttpClientInterface, strategy: RetryStrategyInterface | None = None, max_retries: int = 3, logger: LoggerInterface | None = None, sleep: Callable[[float], Awaitable[None]] = asyncio.sleep)
Bases: HttpClientInterface, ResetInterface
Wraps a client and makes the request again while a strategy says to.
A server that answered 503, or a connection that never came up, has not answered the question that was asked — and often answers it on the next try. Written at the call site that is a loop, a counter and a delay around every request; here it is the client, so the code calling out is unchanged and the decision lives in one place.
client = RetryableHttpClient(inner, max_retries=2)
async with await client.request("GET", "https://example.com/orders") as response:
orders = await response.to_array()
What is retried, and how long is waited between tries, is the
:class:RetryStrategyInterface
<xtr_http_client.retry.retry_strategy_interface.RetryStrategyInterface>'s
— :class:GenericRetryStrategy
<xtr_http_client.retry.generic_retry_strategy.GenericRetryStrategy> by
default, which retries the status codes worth retrying and backs off
between attempts. A Retry-After header wins over its delay: a server
saying when to come back is better informed than a client guessing.
Two things bound it. max_retries — which a request may lower or raise
through the max_retries option — is how many further attempts are
allowed, so the request is made at most that many times plus one; and a
body that cannot be produced twice is sent once, because a second attempt
would send a request that is not the one that failed.
The base_uri option may name several, and then each attempt moves one
along it — the way a client is pointed at a second host rather than at the
host that has just refused. The response handed back reports how many
tries it took, under the retry_count entry of get_info().
Retry what client sends, as often as max_retries allows.
Parameters:
-
client(HttpClientInterface) –The client every attempt is handed to.
-
strategy(RetryStrategyInterface | None, default:None) –What decides whether an attempt is worth repeating.
Nonetakes :class:GenericRetryStrategy <xtr_http_client.retry.generic_retry_strategy.GenericRetryStrategy>as it comes. -
max_retries(int, default:3) –How many further attempts a request may have. Zero or less is a client that passes everything straight through.
-
logger(LoggerInterface | None, default:None) –Where each retry is reported, at info level.
Nonereports none. -
sleep(Callable[[float], Awaitable[None]], default:sleep) –What waiting between attempts is done with. Only worth passing to make a test's waits instant.
request
async
¶
Make a request for method and url, again while that is worth doing.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to a
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made.
max_retriesis this client's own and is not passed on.
Returns:
-
HttpResponseInterface–What the last attempt answered.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option, and when the strategy will not decide.
-
TransportError–When the last attempt brought back no answer.
send_request
async
¶
Send request, again while that is worth doing.
Parameters:
-
request(RequestInterface) –The request to send. Every attempt sends this very request, its body rewound first.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
max_retriesis this client's own and is not passed on.
Returns:
-
HttpResponseInterface–What the last attempt answered.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option, and when the strategy will not decide.
-
TransportError–When the last attempt brought back no answer.
with_options ¶
Return a copy around a wrapped client whose defaults have options over them.
base_uri and max_retries stay here rather than going on: a
list of base URIs is this client's to walk, and how many attempts are
allowed is not something the client behind it can act on.
Parameters:
-
**options(Unpack[RequestOptions], default:{}) –The defaults of the wrapped client of the copy.
Returns:
Raises:
-
InvalidArgumentError–When an option is invalid.
reset
async
¶
Reset the wrapped client.
Nothing here outlives a request — how many attempts are allowed and what decides them are configuration — so there is only the client behind to pass this on to.
ScopingHttpClient ¶
ScopingHttpClient(client: HttpClientInterface, default_options_by_regexp: Mapping[str, RequestOptions], default_regexp: str | None = None)
Bases: HttpClientInterface, ResetInterface
Wraps a client and gives each request the options its URL falls under.
One application talks to several services, and each wants its own credentials, headers and deadlines. Written at the call sites, that is the same four options repeated wherever a service is reached, and one place that forgot them. Here it is said once: a scope is a regular expression the start of a URL is matched against, and the options every request landing in it starts from.
client = ScopingHttpClient(
inner,
{r"https://api\.example\.com/": {"auth_bearer": "t0ken"}},
)
response = await client.request("GET", "https://api.example.com/orders")
What the request itself asked for always wins — a scope fills in what was
left out, it never overrides. headers is the one option merged rather
than replaced, name by name, so a scope-wide Accept survives a request
setting a Content-Type of its own.
With a default_regexp, that scope's base_uri also resolves a
relative URL, which is what makes :meth:for_base_uri read as "this
client is that service".
Scope client, matching a URL against each regexp in turn.
Parameters:
-
client(HttpClientInterface) –The client every vetted request is handed to.
-
default_options_by_regexp(Mapping[str, RequestOptions]) –Each scope's pattern mapped to the options a request falling in it starts from. The first pattern matching the start of a URL is the one that applies, so order them from the most specific to the least.
-
default_regexp(str | None, default:None) –Which of those scopes a relative URL belongs to, and whose
base_uriresolves it.Noneleaves a relative URL to the wrapped client.
Raises:
-
InvalidArgumentError–When
default_regexpnames a scope that is not indefault_options_by_regexp— a default scope with no options is a client whose relative URLs quietly resolve against nothing.
for_base_uri
classmethod
¶
for_base_uri(client: HttpClientInterface, base_uri: str, default_options: RequestOptions | None = None, regexp: str | None = None) -> Self
Scope client to one service, reached at base_uri.
Parameters:
-
client(HttpClientInterface) –The client every request is handed to.
-
base_uri(str) –What a relative URL resolves against, and — unless
regexpsays otherwise — what a URL must start with to be in scope. -
default_options(RequestOptions | None, default:None) –What a request in scope starts from.
-
regexp(str | None, default:None) –The scope's pattern, for a service whose URLs are not all under one base.
Returns:
-
Self–A client scoped to that one base URI.
for_base_uris
classmethod
¶
for_base_uris(client: HttpClientInterface, base_uris: Sequence[str], default_options: RequestOptions | None = None, regexp: str | None = None) -> Self
Scope client to one service reachable at several base URIs.
The scope matches all of them, so the options keep applying whichever one a request is resolved against; the first is what a relative URL resolves against.
Parameters:
-
client(HttpClientInterface) –The client every request is handed to.
-
base_uris(Sequence[str]) –Where the service answers, the first being the default.
-
default_options(RequestOptions | None, default:None) –What a request in scope starts from.
-
regexp(str | None, default:None) –The scope's pattern, replacing the one built from them.
Returns:
-
Self–A client scoped to all of those base URIs.
Raises:
-
InvalidArgumentError–When no base URI is given, since a scope over nothing would match nothing and resolve nothing.
request
async
¶
Make a request for method and url, under the scope it falls in.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to a
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made; the scope fills in what is left out.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came.
send_request
async
¶
Send request under the scope the URI it carries falls in.
Parameters:
-
request(RequestInterface) –The request to send; it goes on carrying the URL this client resolved for it.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out; the scope fills in what both leave out.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came.
with_options ¶
Return a copy around a wrapped client whose defaults have options over them.
The scopes are kept as they are: they say what belongs to which service, which is not something a caller narrowing a client for one job is changing.
Parameters:
-
**options(Unpack[RequestOptions], default:{}) –The defaults of the wrapped client of the copy.
Returns:
-
Self–A new client; this one is left as it was.
Raises:
-
InvalidArgumentError–When an option is invalid.
reset
async
¶
Reset the wrapped client, if it holds state between units of work.
The scopes are configuration, not state, so they are kept.
ThrottlingHttpClient ¶
Bases: HttpClientInterface, ResetInterface
Wraps a client and lets each request go only once the rate allows it.
An API that answers a hundred calls a minute answers the hundred and first with a refusal, and a caller that keeps asking is throttled by the service it is calling rather than by itself. Written at the call site that is a counter and a sleep around every request; here it is the client, so the code calling out is unchanged and the pace is kept in one place — including between several callers, when the limiter they share counts where they can all see it.
client = ThrottlingHttpClient(inner, limiter)
async with await client.request("GET", "https://api.example.com/orders") as response:
orders = await response.to_array()
A token is booked before each request, and the request waits for the moment that booking is due. Booking first and waiting after is what makes the pace hold under concurrency: the limiter counts the token as spent as soon as it is reserved, so requests leave in the order they were asked for, each at its own turn, rather than all at once as a limiter merely asked whether it may go ahead would allow.
The limiter is the pace, so a copy made by :meth:with_options shares it:
narrowing a client for one job hands out another way of making requests,
not another allowance to make them.
Pace what client sends by limiter.
Parameters:
-
client(HttpClientInterface) –The client every request is handed to, once it is due.
-
limiter(LimiterInterface) –What a token is booked from before each request. Which requests share a pace is decided by which of them are given the same limiter — one per host, one per account, one for the whole application.
request
async
¶
Make a request for method and url, once a token is due.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to a
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came.
send_request
async
¶
Send request, once a token is due.
Parameters:
-
request(RequestInterface) –The request to send.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
Returns:
-
HttpResponseInterface–Whatever the wrapped client answers.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came.
with_options ¶
Return a copy around a wrapped client whose defaults have options over them.
Parameters:
-
**options(Unpack[RequestOptions], default:{}) –The defaults of the wrapped client of the copy.
Returns:
-
Self–A new client, paced by this one's limiter, so requests made
-
Self–through either count against the same rate; this one is left as
-
Self–it was.
Raises:
-
InvalidArgumentError–When an option is invalid.
reset
async
¶
Reset the wrapped client, keeping the pace.
The rate is shared on purpose — by every unit of work and every copy
made with :meth:with_options — so the end of one unit forgets none
of what was counted against it.
TraceableHttpClient ¶
TraceableHttpClient(client: HttpClientInterface, disabled: Callable[[], bool] | None = None, recorder: Callable[[TracedRequest], None] | None = None)
Bases: HttpClientInterface, ResetInterface
Wraps a client and writes down every request made through it.
"Which calls did this go on to make, with what, and what came back" is asked of a running application constantly — by a profiler, by a test asserting on a call it has no other sight of, by whoever is reading a log that stops short of the call. Answered at the call site that is logging around every request; here it is the client, so the code calling out is unchanged and the record is one list.
client = TraceableHttpClient(inner)
async with await client.request("GET", "https://example.com/orders") as response:
orders = await response.to_array()
traced = client.get_traced_requests()
assert traced[0]["method"] == "GET"
assert traced[0]["content"] == b'{"orders": []}'
A record is added the moment a request is made and filled in as that
request goes on: info when the response arrives — or the error alone,
when nothing arrives and the error is raised on — and content when the
caller reads the body. So a trace read while a request is still being
answered says how far it has got, rather than nothing until it is over.
The bodies are the costly half of a record and the half most likely to
carry a credential, so extra={"trace_content": False} leaves them out:
the body and json that were sent are dropped from options, and
content is False rather than the bytes. Nothing is recorded at all
for a request made while disabled says so, which is how a profiler
that collects per request stops collecting between them.
The record is the point, so a copy made by :meth:with_options shares it:
narrowing a client for one job hands out another way of making requests,
all of them written into the one list.
The record keeps the last :attr:MAX_TRACES requests, so a client that
is never reset does not grow without bound, and it never keeps a
credential as it was sent.
A recorder is handed each record the moment it is written — the live
record, filled in afterwards like the one in the list — so whoever keeps
records elsewhere, a profiler keeping each request's own, sees them as they
are made without reading the list. Copies hand theirs to it too.
Write down what client is asked to send, unless disabled says not to.
Parameters:
-
client(HttpClientInterface) –The client every request is handed to.
-
disabled(Callable[[], bool] | None, default:None) –Asked before each request whether to record it.
Nonerecords every one. -
recorder(Callable[[TracedRequest], None] | None, default:None) –Handed every record as it is written.
Nonehands them to nobody.
MAX_TRACES
class-attribute
instance-attribute
¶
How many requests the record keeps, the oldest dropped first.
request
async
¶
Make a request for method and url, and write it down.
Parameters:
-
method(str) –The HTTP method, in upper case.
-
url(str) –The URL to request, absolute or relative to a
base_uri. -
**options(Unpack[RequestOptions], default:{}) –How the request is to be made.
Returns:
-
HttpResponseInterface–What the wrapped client answers, reading its body into the record
-
HttpResponseInterface–as the caller reads it.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came, which is written down before it is raised on.
send_request
async
¶
Send request, and write it down.
Parameters:
-
request(RequestInterface) –The request to send, whose method and URI are what the record names it by.
-
**options(Unpack[RequestOptions], default:{}) –What the request leaves out.
Returns:
-
HttpResponseInterface–What the wrapped client answers, reading its body into the record
-
HttpResponseInterface–as the caller reads it.
Raises:
-
InvalidArgumentError–For an invalid method, URL or option.
-
TransportError–When no answer came, which is written down before it is raised on.
get_traced_requests ¶
Return the record of every request made through this client so far.
Returns:
-
list[TracedRequest]–A list of its own, so taking the record does not hand over the
-
list[TracedRequest]–one the client goes on appending to. The records in it are the
-
list[TracedRequest]–live ones: a request still being answered fills in the record
-
list[TracedRequest]–this list holds.
with_options ¶
Return a copy around a wrapped client whose defaults have options over them.
Parameters:
-
**options(Unpack[RequestOptions], default:{}) –The defaults of the wrapped client of the copy.
Returns:
-
Self–A new client writing into this one's record, so a request made
-
Self–through either is traced in one place; this one is left as it was.
Raises:
-
InvalidArgumentError–When an option is invalid.
reset
async
¶
Forget every request written down, and reset the wrapped client.
The record goes for every client sharing it, copies included: there is one record, and this ends the unit of work it was kept for.
TracedRequest ¶
Bases: TypedDict
The record of one request made through a tracing client.
Attributes:
-
method(str) –The HTTP method the request was made with.
-
url(str) –The URL it was made to, as it was asked for.
-
options(RequestOptions) –The options it was made with, as they were given — without
bodyandjsonwhenextra.trace_contentturned the bodies off, and with every credential (auth_basic,auth_bearer, anAuthorization,CookieorProxy-Authorizationheader) written as"***". -
info(Mapping[str, object]) –What the transfer knew about itself when the response arrived. Empty until it arrives, and carrying
erroralone when nothing does. -
content(bytes | bool | None) –Every byte of the body, once the caller has read it;
Nonewhile nobody has, andFalsewhenextra.trace_contentturned the bodies off.