ds_protocol_http_py_lib.dataset

File: __init__.py Region: ds_protocol_http_py_lib/dataset

HTTP Dataset

This module implements a dataset for HTTP APIs.

Example

>>> dataset = HttpDataset(
...     id=uuid.uuid4(),
...     name="example::dataset",
...     version="1.0.0",
...     deserializer=PandasDeserializer(format=DatasetStorageFormatType.JSON),
...     serializer=PandasSerializer(format=DatasetStorageFormatType.JSON),
...     settings=HttpDatasetSettings(
...         url="https://api.example.com/data",
...         method=HttpMethod.GET,
...     ),
...     linked_service=HttpLinkedService(
...         id=uuid.uuid4(),
...         name="example::linked_service",
...         version="1.0.0",
...         settings=HttpLinkedServiceSettings(
...             host="https://api.example.com",
...             auth_type=AuthType.OAUTH2,
...             oauth2=OAuth2AuthSettings(
...                 token_endpoint="https://api.example.com/token",
...                 client_id="my-client",
...                 client_secret="secret",
...             ),
...         ),
...     ),
... )
>>> dataset.read()
>>> data = dataset.output

Submodules

• ds_protocol_http_py_lib.dataset.http

Classes

HttpDataset	Tabular dataset object which identifies data within a data store,
HttpDatasetSettings	Settings for HTTP dataset.

Package Contents

class ds_protocol_http_py_lib.dataset.HttpDataset

Bases: ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset[HttpLinkedServiceType, HttpDatasetSettingsType, ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer, ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer], Generic[HttpLinkedServiceType, HttpDatasetSettingsType]

Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents.

The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame.

linked_service: HttpLinkedServiceType

settings: HttpDatasetSettingsType

serializer: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None

deserializer: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None

property type: ds_protocol_http_py_lib.enums.ResourceType

Get the type of the dataset.

_resolve_url() → str

Resolve the URL by substituting any path parameters.

create() → None

Create data at the specified endpoint.

Parameters:

kwargs – Additional keyword arguments to pass to the request.

Raises:

• AuthenticationError – If the authentication fails.

• AuthorizationError – If the authorization fails.

• ConnectionError – If the connection fails.

• CreateError – If the create error occurs.

read() → None

Read data from the specified endpoint.

Parameters:

kwargs – Additional keyword arguments to pass to the request.

Raises:

• AuthenticationError – If the authentication fails.

• AuthorizationError – If the authorization fails.

• ConnectionError – If the connection fails.

• ReadError – If the read error occurs.

delete() → NoReturn

Delete entity using http.

update() → NoReturn

Update entity using http.

rename() → NoReturn

Rename entity using http.

upsert() → NoReturn

Upsert entity using http.

purge() → NoReturn

Purge entity using http.

list() → NoReturn

List entity using http.

_map_files(files: collections.abc.Sequence[ds_protocol_http_py_lib.models.Files] | None) → Any

Convert typed Files descriptors into requests compatible files=….

HttpDatasetSettings.files is expected to already be deserialized into the correct typed model, so this method focuses purely on the requests shape conversion.

close() → None

Close the dataset.

class ds_protocol_http_py_lib.dataset.HttpDatasetSettings

Bases: ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings

Settings for HTTP dataset.

method: ds_protocol_http_py_lib.enums.HttpMethod

The HTTP method to use.

url: str

The URL to send the request to.

data: Any | None = None

The data to send with the request.

json: dict[str, Any] | None = None

The JSON data to send with the request.

params: dict[str, Any] | None = None

The parameters to send with the request.

files: list[ds_protocol_http_py_lib.models.Files] | None = None

The multipart files to send with the request.

headers: dict[str, Any] | None = None

The headers to send with the request.

path_params: dict[str, Any] | None = None

Path parameters to interpolate into the URL template using {param} syntax.

Example

url=”https://api.example.com/documents/{document_guid}/original” path_params={“document_guid”: “abc123”} # → https://api.example.com/documents/abc123/original