ds_protocol_http_py_lib ======================= .. py:module:: ds_protocol_http_py_lib .. autoapi-nested-parse:: A Python package from the ds-protocol library collection. **File:** ``__init__.py`` **Region:** ``ds-protocol-http-py-lib`` Example: .. code-block:: python from ds_protocol_http_py_lib import __version__ print(f"Package version: {__version__}") Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ds_protocol_http_py_lib/dataset/index /autoapi/ds_protocol_http_py_lib/enums/index /autoapi/ds_protocol_http_py_lib/linked_service/index /autoapi/ds_protocol_http_py_lib/models/index /autoapi/ds_protocol_http_py_lib/utils/index Attributes ---------- .. autoapisummary:: ds_protocol_http_py_lib.__version__ Classes ------- .. autoapisummary:: ds_protocol_http_py_lib.HttpDataset ds_protocol_http_py_lib.HttpDatasetSettings ds_protocol_http_py_lib.HttpLinkedService ds_protocol_http_py_lib.HttpLinkedServiceSettings Package Contents ---------------- .. py:data:: __version__ .. py:class:: HttpDataset Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`HttpLinkedServiceType`\ , :py:obj:`HttpDatasetSettingsType`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer`\ ], :py:obj:`Generic`\ [\ :py:obj:`HttpLinkedServiceType`\ , :py:obj:`HttpDatasetSettingsType`\ ] Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents. The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame. .. py:attribute:: linked_service :type: HttpLinkedServiceType .. py:attribute:: settings :type: HttpDatasetSettingsType .. py:attribute:: serializer :type: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None .. py:attribute:: deserializer :type: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None .. py:property:: type :type: ds_protocol_http_py_lib.enums.ResourceType Get the type of the dataset. .. py:method:: _resolve_url() -> str Resolve the URL by substituting any path parameters. .. py:method:: create() -> None Create data at the specified endpoint. :param kwargs: Additional keyword arguments to pass to the request. :raises AuthenticationError: If the authentication fails. :raises AuthorizationError: If the authorization fails. :raises ConnectionError: If the connection fails. :raises CreateError: If the create error occurs. .. py:method:: read() -> None Read data from the specified endpoint. :param kwargs: Additional keyword arguments to pass to the request. :raises AuthenticationError: If the authentication fails. :raises AuthorizationError: If the authorization fails. :raises ConnectionError: If the connection fails. :raises ReadError: If the read error occurs. .. py:method:: delete() -> NoReturn Delete entity using http. .. py:method:: update() -> NoReturn Update entity using http. .. py:method:: rename() -> NoReturn Rename entity using http. .. py:method:: upsert() -> NoReturn Upsert entity using http. .. py:method:: purge() -> NoReturn Purge entity using http. .. py:method:: list() -> NoReturn List entity using http. .. py:method:: _map_files(files: collections.abc.Sequence[ds_protocol_http_py_lib.models.Files] | None) -> Any Convert typed `Files` descriptors into `requests` compatible `files=...`. `HttpDatasetSettings.files` is expected to already be deserialized into the correct typed model, so this method focuses purely on the `requests` shape conversion. .. py:method:: close() -> None Close the dataset. .. py:class:: HttpDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings for HTTP dataset. .. py:attribute:: method :type: ds_protocol_http_py_lib.enums.HttpMethod The HTTP method to use. .. py:attribute:: url :type: str The URL to send the request to. .. py:attribute:: data :type: Any | None :value: None The data to send with the request. .. py:attribute:: json :type: dict[str, Any] | None :value: None The JSON data to send with the request. .. py:attribute:: params :type: dict[str, Any] | None :value: None The parameters to send with the request. .. py:attribute:: files :type: list[ds_protocol_http_py_lib.models.Files] | None :value: None The multipart files to send with the request. .. py:attribute:: headers :type: dict[str, Any] | None :value: None The headers to send with the request. .. py:attribute:: path_params :type: dict[str, Any] | None :value: None Path parameters to interpolate into the URL template using {param} syntax. .. rubric:: Example url="https://api.example.com/documents/{document_guid}/original" path_params={"document_guid": "abc123"} # → https://api.example.com/documents/abc123/original .. py:class:: HttpLinkedService Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.linked_service.LinkedService`\ [\ :py:obj:`HttpLinkedServiceSettingsType`\ ], :py:obj:`Generic`\ [\ :py:obj:`HttpLinkedServiceSettingsType`\ ] The class is used to connect with HTTP API. .. py:attribute:: settings :type: HttpLinkedServiceSettingsType .. py:attribute:: _session :type: ds_protocol_http_py_lib.utils.http.provider.Http | None :value: None .. py:attribute:: _http :type: ds_protocol_http_py_lib.utils.http.provider.Http | None :value: None .. py:method:: __post_init__() -> None .. py:property:: type :type: ds_protocol_http_py_lib.enums.ResourceType Get the type of the linked service. :returns: ResourceType .. py:property:: connection :type: ds_protocol_http_py_lib.utils.http.provider.Http Get the session. :returns: The session. :rtype: Http .. py:method:: _init_http() -> ds_protocol_http_py_lib.utils.http.provider.Http Initialize the Http client instance with HttpConfig and TokenBucket. Creates an Http instance with: - HttpConfig using headers from the linked service settings - TokenBucket with rate limiting (10 requests per second, capacity of 20) Subclasses can override this method to customize the entire Http initialization, including custom HttpConfig, TokenBucket, or other Http parameters. :returns: The initialized Http client instance. :rtype: Http .. py:method:: _fetch_user_token(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> str Fetch a user token from the token endpoint using the Http provider. :param http: The Http instance to use for the request. :returns: The user token. :rtype: str :raises LinkedServiceException: If bearer settings are missing. :raises AuthenticationError: If the token is missing in the response. .. py:method:: _fetch_oauth2_token(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> str Fetch an OAuth2 token from the token endpoint using the Http provider. :param http: The Http instance to use for the request. :returns: The OAuth2 token. :rtype: str :raises LinkedServiceException: If OAuth2 settings are missing. :raises AuthenticationError: If the token is missing in the response. .. py:method:: _configure_bearer_auth(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure Bearer authentication. Fetches a user token via `_fetch_user_token` and sets the session's Authorization header. :param http: The Http client instance to configure. .. py:method:: _configure_oauth2_auth(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure OAuth2 (client credentials) authentication. Fetches an OAuth2 token via `_fetch_oauth2_token` and sets the session's Authorization header. :param http: The Http client instance to configure. .. py:method:: _configure_basic_auth(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure HTTP Basic authentication. Uses the basic auth settings to construct a base64-encoded `username:password` token and sets the session's Authorization header. :param http: The Http client instance to configure. :raises LinkedServiceException: If basic auth settings are missing. .. py:method:: _configure_apikey_auth(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure API key authentication. Updates the session headers with the configured API key name/value. :param http: The Http client instance to configure. :raises LinkedServiceException: If API key settings are missing. .. py:method:: _configure_custom_auth(http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure custom authentication. Calls the configured token endpoint and extracts an access token from the JSON response using common token key names. The resulting token is stored in the session Authorization header. :param http: The Http client instance to configure. :raises AuthenticationError: If the token is missing in the response. :raises LinkedServiceException: If custom auth settings are missing. .. py:method:: _configure_noauth(_http: ds_protocol_http_py_lib.utils.http.provider.Http) -> None Configure no authentication. This is a no-op handler used to keep the auth dispatch table fully typed. :param _http: The Http client instance to configure. .. py:method:: connect() -> None Connect to the HTTP API and configure authentication. Initializes the Http client instance if not already initialized. Configures authentication based on the auth_type. Merges configured headers into the session, then applies auth configuration. Header precedence: - `settings.headers` are applied first - the selected auth handler may override headers (especially `Authorization`) :returns: The session is configured. :rtype: None :raises AuthenticationError: If the authentication fails. :raises LinkedServiceException: If the auth_type is unsupported. .. py:method:: test_connection() -> tuple[bool, str] Test the connection to the HTTP API. :returns: A tuple containing a boolean indicating success and a string message. :rtype: tuple[bool, str] .. py:method:: close() -> None Close the linked service. .. py:class:: HttpLinkedServiceSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.linked_service.LinkedServiceSettings` Settings for HTTP linked service connections. Provide the appropriate auth settings object based on your auth_type: - ``AuthType.API_KEY`` → ``api_key`` - ``AuthType.BASIC`` → ``basic`` - ``AuthType.BEARER`` → ``bearer`` - ``AuthType.OAUTH2`` → ``oauth2`` - ``AuthType.CUSTOM`` → ``custom`` - ``AuthType.NO_AUTH`` → (no auth settings needed) .. rubric:: Example >>> settings = HttpLinkedServiceSettings( ... host="api.example.com", ... auth_type=AuthType.OAUTH2, ... oauth2=OAuth2AuthSettings( ... token_endpoint="https://auth.example.com/token", ... client_id="my-client", ... client_secret="secret", ... ), ... ) .. py:attribute:: host :type: str The API host (e.g., 'api.example.com'). .. py:attribute:: auth_type :type: ds_protocol_http_py_lib.enums.AuthType The authentication type to use. .. py:attribute:: schema :type: str :value: 'https' URL scheme ('http' or 'https'). .. py:attribute:: port :type: int | None :value: None Optional port number. .. py:attribute:: headers :type: dict[str, str] | None :value: None Additional headers to include with all requests. .. py:attribute:: api_key :type: ApiKeyAuthSettings | None :value: None Settings for API Key authentication. Required when auth_type=AuthType.API_KEY. .. py:attribute:: basic :type: BasicAuthSettings | None :value: None Settings for Basic authentication. Required when auth_type=AuthType.BASIC. .. py:attribute:: bearer :type: BearerAuthSettings | None :value: None Settings for Bearer token authentication. Required when auth_type=AuthType.BEARER. .. py:attribute:: oauth2 :type: OAuth2AuthSettings | None :value: None Settings for OAuth2 client credentials authentication. Required when auth_type=AuthType.OAUTH2. .. py:attribute:: custom :type: CustomAuthSettings | None :value: None Settings for custom token authentication. Required when auth_type=AuthType.CUSTOM.