ds_provider_grasp_py_lib.dataset.file

File: file.py Region: ds_provider_grasp_py_lib/dataset/file

Grasp File Dataset

This module implements a generic GRASP file dataset. The dataset can read and create files via grasp API.

Attributes

logger
GraspFileDatasetSettingsType
HttpLinkedServiceType

Classes

CreateSettings	Settings for create operations.
ReadSettings	Settings for read operations.
GraspFileDatasetSettings	Settings for Grasp file dataset operations.
GraspFileDataset	Tabular dataset object which identifies data within a data store,

Module Contents

ds_provider_grasp_py_lib.dataset.file.logger

class ds_provider_grasp_py_lib.dataset.file.CreateSettings

Settings for create operations.

acl: dict[str, Any] | None

description: str | None = None

file_path: str | None = None

metadata: dict[str, Any] | None

status: str | None = 'active'

tags: dict[str, Any] | None

version: str | None = '1.0.0'

content: io.BytesIO | None = None

File content to be uploaded.

class ds_provider_grasp_py_lib.dataset.file.ReadSettings

Settings for read operations.

download_file: bool = True

limit: int = 500

offset: int = 0

order_by: str | None = None

tags: dict[str, str] | None

meta: dict[str, str] | None

id: str | None = None

file_path: str | None = None

created_at_gte: str | None = None

modified_at_gte: str | None = None

created_at_lte: str | None = None

modified_at_lte: str | None = None

status: str | None = None

class ds_provider_grasp_py_lib.dataset.file.GraspFileDatasetSettings

Bases: ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings

Settings for Grasp file dataset operations.

url: str | None = 'https://grasp-daas.com/api/file-dev/v2/file/'

create: CreateSettings

read: ReadSettings

ds_provider_grasp_py_lib.dataset.file.GraspFileDatasetSettingsType

ds_provider_grasp_py_lib.dataset.file.HttpLinkedServiceType

class ds_provider_grasp_py_lib.dataset.file.GraspFileDataset

Bases: ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset[HttpLinkedServiceType, GraspFileDatasetSettingsType, ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer, ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer], Generic[HttpLinkedServiceType, GraspFileDatasetSettingsType]

Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents.

The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame.

linked_service: HttpLinkedServiceType

settings: GraspFileDatasetSettingsType

property type: ds_provider_grasp_py_lib.enums.ResourceType

Get the type of the dataset.

_details(**extra: Any) → dict[str, Any]

_base_url() → str

_read_params() → dict[str, Any]

Build query parameters for file listing from read settings.

create() → None

Write the content of the dataset to the file. :return: None

read() → None

Read the file from the file_uri. :return: None

update() → NoReturn

Update existing rows in the target matched by identity columns defined in self.settings. Atomic. Must not insert new rows.

Raises:

• UpdateError – If the operation fails.

• NotSupportedError – If the provider does not support update.

See also

Full contract: docs/DATASET_CONTRACT.md – update()

upsert() → NoReturn

Insert rows that do not exist, update rows that do, matched by identity columns defined in self.settings. Atomic.

Raises:

• UpsertError – If the operation fails.

• NotSupportedError – If the provider does not support upsert.

See also

Full contract: docs/DATASET_CONTRACT.md – upsert()

delete() → NoReturn

Remove specific rows from the target matched by identity columns defined in self.settings. Atomic. Idempotent.

Raises:

• DeleteError – If the operation fails.

• NotSupportedError – If the provider does not support delete.

See also

Full contract: docs/DATASET_CONTRACT.md – delete()

purge() → NoReturn

Remove all content from the target. self.input is not used. Atomic. Idempotent.

Raises:

• PurgeError – If the operation fails.

• NotSupportedError – If the provider does not support purge.

See also

Full contract: docs/DATASET_CONTRACT.md – purge()

list() → NoReturn

Discover available resources and populate self.output with a DataFrame of resources and their metadata. Idempotent.

Raises:

• ListError – If the operation fails.

• NotSupportedError – If the provider does not support listing.

See also

Full contract: docs/DATASET_CONTRACT.md – list()

rename() → NoReturn

Rename the resource in the backend. Atomic. Not idempotent.

Raises:

• RenameError – If the operation fails.

• NotSupportedError – If the provider does not support renaming.

See also

Full contract: docs/DATASET_CONTRACT.md – rename()

close() → None

Close the dataset.

_create_metadata() → dict[str, Any]

Create the metadata for the file. :return: Dict

_upload_file_content(metadata: dict[str, Any]) → dict[str, Any]

Upload the file content to the file. :return: Dict