ds_resource_plugin_py_lib.common.serde.deserialize.awswrangler

File: awswrangler.py Region: ds_resource_plugin_py_lib/common/serde/deserialize

Description

Deserialize a value into a pandas DataFrame using awswrangler.

Example

import boto3

from ds_resource_plugin_py_lib.common.serde.deserialize.awswrangler import AwsWranglerDeserializer
from ds_resource_plugin_py_lib.common.resource.dataset.storage_format import DatasetStorageFormatType

boto3_session = boto3.Session()
deserializer = AwsWranglerDeserializer(format=DatasetStorageFormatType.PARQUET)

df = deserializer("s3://my-bucket/path/to/data.parquet", boto3_session=boto3_session)

Attributes

logger

Classes

AwsWranglerDeserializer

Extensible class to deserialize dataset content.

Module Contents

ds_resource_plugin_py_lib.common.serde.deserialize.awswrangler.logger

class ds_resource_plugin_py_lib.common.serde.deserialize.awswrangler.AwsWranglerDeserializer

Bases: ds_resource_plugin_py_lib.common.serde.deserialize.base.DataDeserializer

Extensible class to deserialize dataset content.

Not supposed to be used directly, but to be subclassed.

format: ds_resource_plugin_py_lib.common.resource.dataset.storage_format.DatasetStorageFormatType

kwargs: dict[str, Any]

__call__(value: Any, **kwargs: Any) → pandas.DataFrame

Deserialize a value into a pandas DataFrame. :param value: The value to deserialize. :param **kwargs: Additional keyword arguments.

Returns:

A pandas DataFrame.