Skip to content

xtr_http_client_contracts

The async HTTP client contract: what code that calls out depends on, and nothing more.

A library that sends an HTTP request should not decide what sends it. It takes the client interface, asks for a method and a URL — or hands over a message request — and gets back a response whose status line and headers have arrived, its body read when, and only when, the caller wants it.

What is here is only what appears in that contract: the client, the response it answers with, the options a request takes, the errors any of them raises, and a conformance suite an implementation can run against itself.

OPTIONS_DEFAULTS module-attribute

OPTIONS_DEFAULTS: Final[RequestOptions] = {'auth_basic': None, 'auth_bearer': None, 'query': {}, 'headers': {}, 'body': b'', 'json': None, 'user_data': None, 'max_redirects': 20, 'http_version': None, 'base_uri': None, 'timeout': None, 'max_duration': 0.0, 'max_connect_duration': 0.0, 'resolve': {}, 'proxy': None, 'verify_peer': True, 'verify_host': True, 'cafile': None, 'capath': None, 'local_cert': None, 'local_pk': None, 'passphrase': None, 'max_retries': 3, 'extra': {'use_persistent_connections': False}}

Every option the contract names, mapped to what it means to leave it out.

A client merges what a request was given over its own defaults, and those over this, so every option always has a value by the time a request is made. Nothing here is an instruction — it is the absence of one, which is why the defaults name no host, no credentials and no deadline.

The entries worth reading twice: timeout of None is not "no timeout" but 60 seconds; max_duration and max_connect_duration of zero are "no limit".

The mapping is shared, so merge into a copy rather than writing to it.

ClientError

ClientError(response: HttpResponseInterface, reason: str)

Bases: HttpError

A 4xx status: the server understood the request and would not serve it.

Retrying the same request unchanged gets the same answer, so this is the one range worth telling apart — a 401 asks for credentials, a 404 says there is nothing there, a 429 says to slow down.

DecodingError

DecodingError(reason: str)

Bases: HttpClientError, ValueError

A body could not be turned into the shape that was asked for.

What :meth:HttpResponseInterface.to_array <xtr_http_client_contracts.http_response_interface.HttpResponseInterface.to_array> raises when the body is empty, is not JSON, or is JSON that is not an object. The transfer succeeded, so this is about the payload, not the connection.

Also a :class:ValueError, so code that already guards what it parses with except ValueError keeps working without learning a new exception.

Attributes:

  • reason (str) –

    What is wrong with the body, naming the URL it came from.

Record what is wrong with the body.

HttpClientError

Bases: Exception

Base class for every error raised through the HTTP client contract.

Catch this to handle anything calling out can go wrong with, whichever client is behind the interface; catch a subclass to handle one cause. Every subclass carries what a caller needs as typed attributes and composes its message from them.

A server answering is not a cause: a request that came back with a 404 or a 503 succeeded as a transfer, so request() returns the response and nothing is raised until someone reads its content and asks to be told. What raises on its own is the transfer failing, a body that will not decode, and an option no client here understands.

HttpError

HttpError(response: HttpResponseInterface, reason: str)

Bases: HttpClientError

The status code says the request did not get what it asked for.

Never raised by making the request: a status is an answer, not a failure of the transfer, so it reaches the caller as a response. This is raised where the caller asks to be told about it — :meth:HttpResponseInterface.get_content <xtr_http_client_contracts.http_response_interface.HttpResponseInterface.get_content> or :meth:HttpResponseInterface.to_array <xtr_http_client_contracts.http_response_interface.HttpResponseInterface.to_array> with throw.

Catch this to handle any unsuccessful status; :class:RedirectionError <...redirection_error.RedirectionError>, :class:ClientError <...client_error.ClientError> and :class:ServerError <...server_error.ServerError> are the three ranges, and they are siblings, so catching one leaves the others alone.

The response comes along because it is the only place the answer is left: the status code, the headers and the body, which is kept.

Attributes:

  • response (HttpResponseInterface) –

    The response that was answered with.

  • reason (str) –

    What the status means for this request, naming the code and the URL. The message is this, composed by whoever raised it — the body of an error document is the most useful thing to put there and reading it is awaited, which a constructor cannot do.

Record the response the status came on, and what it means.

InvalidArgumentError

InvalidArgumentError(reason: str)

Bases: TransportError, ValueError

The request was refused before it was sent, for something it asked for.

An unknown option name, an option whose value the client cannot honour, a method or URL it cannot make a request with, or a body given both with the request and as an option. Nothing reached the network, so there is no response, and the same call with a corrected argument would go through.

Also a :class:ValueError, so code that already guards the arguments it passes with except ValueError keeps working without learning a new exception.

JsonError

JsonError(reason: str)

Bases: DecodingError

The body is JSON, but not the object :meth:HttpResponseInterface.to_array promises.

A JSON array, string, number or null where an object was asked for. The body was read in full and parsed, so the failure is the shape of the payload, not the transfer.

Also a :class:DecodingError, so code that already catches a body it could not decode keeps catching this one.

RedirectionError

RedirectionError(response: HttpResponseInterface, reason: str)

Bases: HttpError

A 3xx status that was not followed.

Raised once max_redirects has been reached, or straight away when following was turned off with max_redirects of zero or less — a redirect that is followed never reaches the caller, since what comes back is the response from the last hop.

Where the answer points is in the redirect_url entry of :meth:HttpResponseInterface.get_info <xtr_http_client_contracts.http_response_interface.HttpResponseInterface.get_info>.

ServerError

ServerError(response: HttpResponseInterface, reason: str)

Bases: HttpError

A 5xx status: the request may have been fine, the server was not.

The far side is at fault, so the same request may well succeed later. Nothing here retries; what to do about it is the caller's to decide.

TransportError

TransportError(reason: str)

Bases: HttpClientError

Nothing usable came back, or the request could never be made.

A host that will not resolve, a connection refused or reset, a body that stopped half way, a certificate the client would not accept — and the one case that never touches the network: an option the client does not support. A client reports that by raising rather than by ignoring it, so an option that was spelled wrong, or that this client cannot honour, cannot pass for one it quietly obeyed.

It is also what reading a response raises once it has been cancelled.

Attributes:

  • reason (str) –

    What went wrong, naming the URL it went wrong for.

Record what went wrong.

TransportTimeoutError

TransportTimeoutError(reason: str)

Bases: TransportError, TimeoutError

The transfer went quiet for longer than it was allowed to.

Raised when the idle timeout passes with nothing coming in, when the whole transfer outlasts max_duration, or when connecting outlasts max_connect_duration.

Also a :class:TimeoutError, which is what the standard library raises and what asyncio.timeout turns a deadline into, so code that already gives up on a timeout keeps working without learning a new exception.

HttpClientInterface

Bases: Protocol

Sends HTTP requests and hands back responses.

A library that calls out takes this interface, and the application decides what is behind it — a real transfer, a scoped client, a guard, or a client answering from memory in a test.

response = await client.request("GET", "https://example.com/orders")
orders = await response.to_array()

Several requests run at once by being made from tasks of one task group.

An option name the contract does not list is refused with :class:InvalidArgumentError <xtr_http_client_contracts.exception.invalid_argument_error.InvalidArgumentError>: Unsupported option "foo" passed to "<client class>", did you mean "..."?. An option that was spelled wrong cannot pass for one that was obeyed.

request async

request(method: str, url: str, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Make a request, and return once the status line and the headers are in.

The same as :meth:send_request with a message request carrying method and url, no headers and an empty body.

It never raises because of the status: a 404 or a 503 is an answer, and comes back as a response.

Parameters:

  • method (str) –

    The HTTP method, in upper case.

  • url (str) –

    The URL to request, absolute or relative to the base_uri option.

  • **options (Unpack[RequestOptions], default: {} ) –

    How the request is to be made; anything left out falls back to this client's defaults.

Returns:

Raises:

send_request async

send_request(request: RequestInterface, **options: Unpack[RequestOptions]) -> HttpResponseInterface

Send a message request, the options filling what it leaves out.

The request is authoritative:

  • a relative URI is resolved against base_uri;
  • query is merged into the URI;
  • a header from headers is sent only when the request has none of that name;
  • auth_basic and auth_bearer apply only when the request has no Authorization header;
  • body or json with a request whose body is not empty is refused: Define either the request body or the "body" or "json" option, setting both is not supported..

It returns, and never raises because of the status, as :meth:request does.

Parameters:

  • request (RequestInterface) –

    The request to send.

  • **options (Unpack[RequestOptions], default: {} ) –

    What the request leaves out; anything left out of both falls back to this client's defaults.

Returns:

Raises:

with_options

with_options(**options: Unpack[RequestOptions]) -> Self

Return a client whose requests start from these options.

Parameters:

  • **options (Unpack[RequestOptions], default: {} ) –

    The defaults of the client returned, merged over this client's own, so what is left out is kept.

Returns:

  • Self –

    A new client; this one is left as it was.

Raises:

HttpResponseInterface

Bases: ResponseInterface, Protocol

An HTTP response whose body is read off the network when it is asked for.

It is a message response first: the status code and reason phrase, the protocol version, the headers, the body as a stream and the with_* copies are the message contract's, so code typed against the plain response interface takes this one as it is. A copy keeps the transfer info and the very same body.

What a client adds:

  • A status is an answer, not a failure. The response is handed back once the status line and the headers are in, whatever the status. :meth:get_content and :meth:to_array raise for a 3xx, 4xx or 5xx when throw is left on, and return the body when it is turned off.
  • The body is kept. It is read off the network on demand and kept — in a temporary file past 2 MiB — so it can be read again; get_body() is an async stream over that kept body.
  • It holds a connection until it is done with. Leaving an async with block closes the body, which frees the connection; :meth:cancel does the same mid-transfer.

get_info

get_info(key: str | None = None) -> object

Return what the transfer knows about itself, as it stands.

Never raises and never waits, so it can be read while the transfer is going on.

These entries are always there:

  • canceled (bool) — whether :meth:cancel was called.
  • error (str | None) — what went wrong, once something has.
  • http_code (int) — the last status code, or 0 before one arrived.
  • http_method (str) — the method of the last request made.
  • redirect_count (int) — how many redirects were followed.
  • redirect_url (str | None) — where a 3xx points, resolved, when it was not followed.
  • response_headers (list[str]) — the raw status and header lines of every hop.
  • start_time (float) — when the request was sent.
  • total_time (float) — how long the transfer has taken.
  • url (str) — the URL of the last hop.
  • user_data — the user_data option, or None.

primary_ip and retry_count are there when they are known.

Parameters:

  • key (str | None, default: None ) –

    One entry to look up. None asks for all of them.

Returns:

  • object –

    A mapping of every entry when key is None; otherwise the

  • object –

    value under key, or None when there is no such entry.

cancel async

cancel() -> None

Abandon the transfer and let go of what it was holding.

Afterwards get_info("canceled") reads True and get_info("error") reads Response has been canceled., and reading the body raises :class:TransportError <xtr_http_client_contracts.exception.transport_error.TransportError> with that message.

get_content async

get_content(throw: bool = True) -> bytes

Read the whole body, waiting for the rest of it to arrive.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than returned. The message is HTTP <code> returned for "<url>"., or the title and detail of the body when it is a JSON problem document (RFC 7807, or its hydra form).

Returns:

  • bytes –

    Every byte of the body, empty when there was none.

Raises:

to_array async

to_array(throw: bool = True) -> dict[str, object]

Read the whole body and decode it as a JSON object.

Parameters:

  • throw (bool, default: True ) –

    Whether an unsuccessful status is to be raised rather than returned, as in :meth:get_content.

Returns:

Raises:

RequestOptions

Bases: TypedDict

How a request is to be made, beyond its method and URL.

Every option is optional, and an option left out is one nobody asked about: the client fills it in from its own defaults, which start at :data:OPTIONS_DEFAULTS <xtr_http_client_contracts.options_defaults.OPTIONS_DEFAULTS>. The same keys are what :meth:HttpClientInterface.with_options <xtr_http_client_contracts.http_client_interface.HttpClientInterface.with_options> takes, where they become the defaults of the client it returns.

A name not listed here is refused. extra carries what one client understands and another has no use for, so entries in it may be ignored.

Attributes:

  • auth_basic (str | tuple[str] | tuple[str, str] | None) –

    Credentials for basic authentication: "user:password", or the user alone, or the two as a tuple. None sends none.

  • auth_bearer (str | None) –

    A token to send as a bearer credential. None sends none.

  • query (Mapping[str, str | int | float | Sequence[str | int | float]]) –

    Parameters to merge into the URL's query string. A scalar is one value; a sequence repeats the name once per item. None, booleans and nested mappings are not accepted. They win over a parameter of the same name already in the URL.

  • headers (Mapping[str, str | Sequence[str]]) –

    Headers to send, a mapping of name to one value or several.

  • body (bytes | str | Mapping[str, object] | AsyncIterable[bytes]) –

    What to send as the request body. Bytes or text go as they are; a mapping is sent as an HTML form; an async iterable of bytes is sent as it is produced, which is how a body too large to hold is uploaded. A mapping holding a binary file object among its values — what open(path, "rb") returns, or an io.BytesIO — is sent as multipart/form-data instead: each plain value as a part of its own, each file as a part whose file name is the last component of the file's name (unknown when it has none) and whose content type is guessed from that name by mimetypes.guess_type, else application/octet-stream. A Content-Type: multipart/form-data; boundary=... header set by the caller keeps its boundary; any other content type is replaced by one naming the boundary the client picks. The file is read synchronously while the body is sent, which suits uploads of a modest size. In such a body every other value is a string, bytes, a number, a boolean (sent as true or false), None or a list of them, one part per item; a nested mapping or list is refused with an InvalidArgumentError naming the field — encode it, as JSON say, first. A mapping without a file stays url-encoded.

  • json (object) –

    A value to send as a JSON body. The client encodes it and sets a JSON content type unless headers already names one. Setting both this and body is refused.

  • user_data (object) –

    Anything the caller wants to find again on the response, under the user_data entry of get_info(). Never read by the client.

  • max_redirects (int) –

    How many redirects to follow. Zero or less follows none, which leaves the 3xx to be answered with.

  • http_version (str | None) –

    The HTTP version to ask for — "1.0", "1.1", "2" or "2.0". None leaves the client to use the best version the connection offers.

  • base_uri (str | Sequence[str] | None) –

    The URI a relative URL is resolved against. A sequence of them is only for a client that retries, which moves along it from one attempt to the next.

  • timeout (float | None) –

    How long the transfer may go quiet for, in seconds. None is 60 seconds.

  • max_duration (float) –

    How long the whole request and response may take, in seconds. Zero is unlimited.

  • max_connect_duration (float) –

    How long resolving, connecting and the TLS handshake may take, in seconds. Zero is unlimited.

  • resolve (Mapping[str, str]) –

    Host to IP address, used instead of asking a resolver. This is what pins a request to an address that has already been checked, so that nothing can change under it between the check and the connection. It pins the request it is given with only.

  • proxy (str | None) –

    The proxy to go through, as a URL.

  • verify_peer (bool) –

    Whether to check that the peer's certificate is trusted.

  • verify_host (bool) –

    Whether to check that the certificate is for the host being asked for.

  • cafile (str | None) –

    A file of certificates to trust instead of the defaults.

  • capath (str | None) –

    A directory of certificates to trust instead of the defaults.

  • local_cert (str | None) –

    A client certificate to present.

  • local_pk (str | None) –

    The private key of local_cert, when it is kept apart.

  • passphrase (str | None) –

    The passphrase of local_pk.

  • max_retries (int) –

    How many times a client that retries may try the request again. A client that does not retry ignores it.

  • extra (Mapping[str, object]) –

    Additional options a client may ignore when it does not support them, unlike the regular options it must refuse: use_persistent_connections says whether to use persistent connections where supported, trace_content whether a tracing client keeps the bodies.