ds_protocol_http_py_lib.utils.http ================================== .. py:module:: ds_protocol_http_py_lib.utils.http .. autoapi-nested-parse:: **File:** ``__init__.py`` **Region:** ``ds_protocol_http_py_lib/utils/http`` HTTP utility subpackage. This file exists to ensure tools (like Sphinx AutoAPI) treat this directory as a proper Python package, so intra-package relative imports resolve correctly. Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ds_protocol_http_py_lib/utils/http/config/index /autoapi/ds_protocol_http_py_lib/utils/http/provider/index /autoapi/ds_protocol_http_py_lib/utils/http/token_bucket/index Classes ------- .. autoapisummary:: ds_protocol_http_py_lib.utils.http.HttpConfig ds_protocol_http_py_lib.utils.http.RetryConfig ds_protocol_http_py_lib.utils.http.Http ds_protocol_http_py_lib.utils.http.TokenBucket Package Contents ---------------- .. py:class:: HttpConfig Configuration for the HTTP client. - headers: applied to all requests (overridable per call) - timeout_seconds: connect+read timeout seconds (or (connect, read) per call) - user_agent: user agent for all requests - pool_maxsize: maximum number of connections in the pool - pool_connections: maximum number of connections in the pool - raise_for_status: raise for status for all requests - retry: RetryConfig .. py:attribute:: headers :type: collections.abc.Mapping[str, str] .. py:attribute:: timeout_seconds :type: int | float :value: 10 .. py:attribute:: user_agent :type: str :value: 'Http/1.0' .. py:attribute:: pool_maxsize :type: int :value: 32 .. py:attribute:: pool_connections :type: int :value: 10 .. py:attribute:: retry :type: RetryConfig .. py:class:: RetryConfig Retry policy (urllib3 Retry via requests). - total: max attempts (includes first request) - backoff_factor: sleep = factor * (2 ** (retry_num - 1)) - status_forcelist: statuses that trigger retry - allowed_methods: methods eligible for retry - respect_retry_after_header: honor Retry-After on 429/503 .. py:attribute:: total :type: int :value: 3 .. py:attribute:: backoff_factor :type: float :value: 0.2 .. py:attribute:: status_forcelist :type: tuple[int, Ellipsis] :value: (429, 500, 502, 503, 504) .. py:attribute:: allowed_methods :type: tuple[str, Ellipsis] :value: ('GET', 'POST', 'PUT', 'DELETE', 'PATCH') .. py:attribute:: raise_on_status :type: bool :value: False .. py:attribute:: respect_retry_after_header :type: bool :value: True .. py:class:: Http(*, config: ds_protocol_http_py_lib.utils.http.config.HttpConfig | None = None, bucket: ds_protocol_http_py_lib.utils.http.token_bucket.TokenBucket | None = None, session: requests.Session | None = None) Minimal synchronous HTTP client with: - requests.Session + urllib3.Retry (429/5xx, backoff, Retry-After) - optional TokenBucket for simple RPS throttling - context-managed lifetime - tiny API: request/get/post/close .. py:attribute:: _cfg .. py:attribute:: _bucket .. py:attribute:: _session .. py:method:: _build_session() -> requests.Session Build the session. :returns: The session. :rtype: requests.Session .. py:method:: _response_info(response: requests.Response) -> dict[str, Any] Get information about a response. Extracts safe metadata only (no request/response bodies). :param response: The HTTP response object to extract info from. :returns: Dictionary containing the response information. :rtype: dict[str, Any] .. py:property:: session :type: requests.Session Get the underlying requests session for direct use. Allows direct access to session properties like headers. :returns: The requests session. :rtype: requests.Session .. rubric:: Example >>> http = Http() >>> http.session.headers.update({"Authorization": "Bearer token"}) .. py:method:: __enter__() -> Http .. py:method:: __exit__(exc_type: Any, exc: Any, tb: Any) -> None .. py:method:: close() -> None .. py:method:: request(method: str, url: str, **kwargs: Any) -> requests.Response Send an HTTP request with rate limiting, retry logic, and comprehensive logging. :param method: HTTP method (GET, POST, PUT, DELETE, etc.). :param url: Target URL for the request. :param \*\*kwargs: Additional keyword arguments passed to requests (timeout, headers, data, etc.). :returns: The HTTP response object. :rtype: requests.Response :raises requests.HTTPError: If the response status code indicates an error. :raises requests.RequestException: For other request-related errors. .. rubric:: Example >>> with Http() as client: ... response = client.request('GET', 'https://api.example.com/data', timeout=30) ... data = response.json() .. py:method:: get(url: str, **kwargs: Any) -> requests.Response Send a GET request with enhanced logging. :param url: Target URL for the GET request. :param \*\*kwargs: Additional keyword arguments passed to requests. :returns: The HTTP response object. :rtype: requests.Response .. rubric:: Example >>> with Http() as client: ... response = client.get("https://api.example.com/data") .. py:method:: post(url: str, **kwargs: Any) -> requests.Response Send a POST request with enhanced logging. :param url: Target URL for the POST request. :param \*\*kwargs: Additional keyword arguments passed to requests. :returns: The HTTP response object. :rtype: requests.Response .. rubric:: Example >>> with Http() as client: ... response = client.post("https://api.example.com/data", json={"key": "value"}) .. py:method:: put(url: str, **kwargs: Any) -> requests.Response Send a PUT request with enhanced logging. :param url: Target URL for the PUT request. :param \*\*kwargs: Additional keyword arguments passed to requests. :returns: The HTTP response object. :rtype: requests.Response .. rubric:: Example >>> with Http() as client: ... response = client.put("https://api.example.com/data/1", json={"key": "value"}) .. py:method:: delete(url: str, **kwargs: Any) -> requests.Response Send a DELETE request with enhanced logging. :param url: Target URL for the DELETE request. :param \*\*kwargs: Additional keyword arguments passed to requests. :returns: The HTTP response object. :rtype: requests.Response .. rubric:: Example >>> with Http() as client: ... response = client.delete("https://api.example.com/data/1") .. py:class:: TokenBucket(rps: float = 10.0, capacity: int = 20) Token Bucket Rate Limiter Implements the classic token bucket algorithm for controlling request rates in threading environments. Each request consumes one token from the bucket. If no tokens are available, the request waits until tokens are replenished based on the configured rate. The bucket starts full and refills continuously at the specified RPS rate. This allows for burst traffic (up to the bucket capacity) while maintaining the overall rate limit. :param rps: Target requests per second rate. Determines token refill rate. :param capacity: Maximum number of tokens the bucket can hold. Defaults to 2x RPS. :return: None .. rubric:: Example # Limit to 10 requests per second with burst capacity of 20 limiter = TokenBucket(rps=10.0, capacity=20) # Acquire permission for a request limiter.acquire() # Make your HTTP request here .. py:attribute:: rps .. py:attribute:: capacity :value: 20 .. py:attribute:: tokens .. py:attribute:: last .. py:attribute:: _lock .. py:method:: acquire() -> None Acquire a token from the bucket, waiting if necessary. :return: None .. py:method:: available() -> float Return the current available token count *without mutating* internal state. This computes how many tokens would be available if we refilled based on elapsed time, but it does not update `tokens` or `last`.