ds_provider_grasp_py_lib.dataset.file ===================================== .. py:module:: ds_provider_grasp_py_lib.dataset.file .. autoapi-nested-parse:: **File:** ``file.py`` **Region:** ``ds_provider_grasp_py_lib/dataset/file`` Grasp File Dataset This module implements a generic GRASP file dataset. The dataset can read and create files via grasp API. Attributes ---------- .. autoapisummary:: ds_provider_grasp_py_lib.dataset.file.logger ds_provider_grasp_py_lib.dataset.file.GraspFileDatasetSettingsType ds_provider_grasp_py_lib.dataset.file.HttpLinkedServiceType Classes ------- .. autoapisummary:: ds_provider_grasp_py_lib.dataset.file.CreateSettings ds_provider_grasp_py_lib.dataset.file.ReadSettings ds_provider_grasp_py_lib.dataset.file.GraspFileDatasetSettings ds_provider_grasp_py_lib.dataset.file.GraspFileDataset Module Contents --------------- .. py:data:: logger .. py:class:: CreateSettings Settings for create operations. .. py:attribute:: acl :type: dict[str, Any] | None .. py:attribute:: description :type: str | None :value: None .. py:attribute:: file_path :type: str | None :value: None .. py:attribute:: metadata :type: dict[str, Any] | None .. py:attribute:: status :type: str | None :value: 'active' .. py:attribute:: tags :type: dict[str, Any] | None .. py:attribute:: version :type: str | None :value: '1.0.0' .. py:attribute:: content :type: io.BytesIO | None :value: None File content to be uploaded. .. py:class:: ReadSettings Settings for read operations. .. py:attribute:: download_file :type: bool :value: True .. py:attribute:: limit :type: int :value: 500 .. py:attribute:: offset :type: int :value: 0 .. py:attribute:: order_by :type: str | None :value: None .. py:attribute:: tags :type: dict[str, str] | None .. py:attribute:: meta :type: dict[str, str] | None .. py:attribute:: id :type: str | None :value: None .. py:attribute:: file_path :type: str | None :value: None .. py:attribute:: created_at_gte :type: str | None :value: None .. py:attribute:: modified_at_gte :type: str | None :value: None .. py:attribute:: created_at_lte :type: str | None :value: None .. py:attribute:: modified_at_lte :type: str | None :value: None .. py:attribute:: status :type: str | None :value: None .. py:class:: GraspFileDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings for Grasp file dataset operations. .. py:attribute:: url :type: str | None :value: 'https://grasp-daas.com/api/file-dev/v2/file/' .. py:attribute:: create :type: CreateSettings .. py:attribute:: read :type: ReadSettings .. py:data:: GraspFileDatasetSettingsType .. py:data:: HttpLinkedServiceType .. py:class:: GraspFileDataset Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`HttpLinkedServiceType`\ , :py:obj:`GraspFileDatasetSettingsType`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer`\ ], :py:obj:`Generic`\ [\ :py:obj:`HttpLinkedServiceType`\ , :py:obj:`GraspFileDatasetSettingsType`\ ] Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents. The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame. .. py:attribute:: linked_service :type: HttpLinkedServiceType .. py:attribute:: settings :type: GraspFileDatasetSettingsType .. py:property:: type :type: ds_provider_grasp_py_lib.enums.ResourceType Get the type of the dataset. .. py:method:: _details(**extra: Any) -> dict[str, Any] .. py:method:: _base_url() -> str .. py:method:: _read_params() -> dict[str, Any] Build query parameters for file listing from read settings. .. py:method:: create() -> None Write the content of the dataset to the file. :return: None .. py:method:: read() -> None Read the file from the file_uri. :return: None .. py:method:: update() -> NoReturn Update existing rows in the target matched by identity columns defined in ``self.settings``. Atomic. Must not insert new rows. :raises UpdateError: If the operation fails. :raises NotSupportedError: If the provider does not support update. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``update()`` .. py:method:: upsert() -> NoReturn Insert rows that do not exist, update rows that do, matched by identity columns defined in ``self.settings``. Atomic. :raises UpsertError: If the operation fails. :raises NotSupportedError: If the provider does not support upsert. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``upsert()`` .. py:method:: delete() -> NoReturn Remove specific rows from the target matched by identity columns defined in ``self.settings``. Atomic. Idempotent. :raises DeleteError: If the operation fails. :raises NotSupportedError: If the provider does not support delete. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``delete()`` .. py:method:: purge() -> NoReturn Remove all content from the target. ``self.input`` is not used. Atomic. Idempotent. :raises PurgeError: If the operation fails. :raises NotSupportedError: If the provider does not support purge. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``purge()`` .. py:method:: list() -> NoReturn Discover available resources and populate ``self.output`` with a DataFrame of resources and their metadata. Idempotent. :raises ListError: If the operation fails. :raises NotSupportedError: If the provider does not support listing. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``list()`` .. py:method:: rename() -> NoReturn Rename the resource in the backend. Atomic. Not idempotent. :raises RenameError: If the operation fails. :raises NotSupportedError: If the provider does not support renaming. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``rename()`` .. py:method:: close() -> None Close the dataset. .. py:method:: _create_metadata() -> dict[str, Any] Create the metadata for the file. :return: Dict .. py:method:: _upload_file_content(metadata: dict[str, Any]) -> dict[str, Any] Upload the file content to the file. :return: Dict