ds_provider_azure_py_lib.dataset.table ====================================== .. py:module:: ds_provider_azure_py_lib.dataset.table .. autoapi-nested-parse:: **File:** ``table.py`` **Region:** ``ds_provider_azure_py_lib/dataset/table`` Azure Dataset - Table Storage This module implements a dataset for Azure Table Storage, allowing for CRUD operations on table entities using pandas DataFrames for data representation. .. rubric:: Example >>> azure_table = AzureTable( ... settings=AzureTableDatasetSettings( ... table_name="users", ... partition_key="partition_key", ... row_key="row_key", ... query_filter="additional query filter", ... delete_table=False, ... ), ... linked_service=AzureLinkedService( ... settings=AzureLinkedServiceSettings( ... account_name="account name", ... access_key="access key" ... ), ... id=uuid.uuid4(), ... name="testazurepackage", ... version="0.0.1", ... description="testazurepackage", ... ), ... id=uuid.uuid4(), ... name="testazurepackage", ... version="0.0.1", ... description="testazurepackage" ... ) >>> azure_table.read() >>> table_data = azure_table.output Attributes ---------- .. autoapisummary:: ds_provider_azure_py_lib.dataset.table.logger ds_provider_azure_py_lib.dataset.table.TransactionEntry ds_provider_azure_py_lib.dataset.table.AzureTableDatasetSettingsType ds_provider_azure_py_lib.dataset.table.AzureLinkedServiceType Classes ------- .. autoapisummary:: ds_provider_azure_py_lib.dataset.table.ReadSettings ds_provider_azure_py_lib.dataset.table.PurgeSettings ds_provider_azure_py_lib.dataset.table.AzureTableDatasetSettings ds_provider_azure_py_lib.dataset.table.AzureTable Module Contents --------------- .. py:data:: logger .. py:data:: TransactionEntry .. py:class:: ReadSettings Settings specific to the read() operation. These settings only apply when reading data from the database and do not affect other operations like: create(), delete(), update(), or rename(). .. py:attribute:: query_filter :type: str | None :value: None An OData-compliant string to filter the entities returned by the read() operation. If None, no filter is applied and all entities are returned. Example: "PartitionKey eq '{self.partition_key}' and RowKey eq '{self.row_key}'" .. py:class:: PurgeSettings Settings specific to the purge() operation. These settings only apply when deleting data from the database and do not affect other operations like: create(), read(), update(), or rename(). .. py:attribute:: delete_table :type: bool :value: False If True, the entire table will be deleted when purge() is called. If False, only the table content will be deleted. .. py:class:: AzureTableDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings for Azure Table Storage dataset operations. The `read` settings contains read-specific configuration that only applies to the read() operation, not to create(), delete(), update(), etc. .. py:attribute:: table_name :type: str .. py:attribute:: purge :type: PurgeSettings Purge-specific settings. Only applies to the purge() operation. .. py:attribute:: read :type: ReadSettings Read-specific settings. Only applies to the read() operation. By default, read() will use read without filter. .. py:data:: AzureTableDatasetSettingsType .. py:data:: AzureLinkedServiceType .. py:class:: AzureTable Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`AzureLinkedServiceType`\ , :py:obj:`AzureTableDatasetSettingsType`\ , :py:obj:`ds_provider_azure_py_lib.serde.AzureTableSerializer`\ , :py:obj:`ds_provider_azure_py_lib.serde.AzureTableDeserializer`\ ], :py:obj:`Generic`\ [\ :py:obj:`AzureLinkedServiceType`\ , :py:obj:`AzureTableDatasetSettingsType`\ ] Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents. The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame. .. py:attribute:: linked_service :type: AzureLinkedServiceType .. py:attribute:: settings :type: AzureTableDatasetSettingsType .. py:method:: __post_init__() -> None .. py:property:: type :type: ds_provider_azure_py_lib.enums.ResourceType Get the type of the Dataset. :returns: ResourceType .. py:method:: _prepare_content(content: pandas.DataFrame) -> dict[str, Any] Ensure that the content is provided and is in the correct format. :param content: The content to prepare. :type content: pd.DataFrame :returns: The prepared content. :rtype: dict :raises DatasetException: If the content is not a DataFrame, is empty, or does not contain required columns. .. py:method:: _get_table_client() -> azure.data.tables.TableClient Return a TableClient for the currently configured table. :returns: TableClient .. py:method:: _build_transaction_from_input(operation: str, params: collections.abc.Mapping[str, Any] | None = None) -> list[TransactionEntry] Build a list of transaction entries from self.input. operation: operation name as expected by TableClient.submit_transaction, e.g. "create", "upsert", "delete" :param operation: The operation to perform. :type operation: str :param params: optional params dict passed as third item in tuple (when required) e.g. {"mode": UpdateMode.REPLACE} :returns: list[TransactionEntry] :raises CreateError: If there is an error preparing content for creation. :raises UpdateError: If there is an error preparing content for update. :raises DeleteError: If there is an error preparing content for deletion. :raises DatasetException: If there is a general error preparing content. .. py:method:: _submit_transaction(transaction: collections.abc.Iterable[TransactionEntry], error_cls: type[ds_resource_plugin_py_lib.common.resource.dataset.errors.DatasetException]) -> None Submit transaction and map TableTransactionError to provided error_type. :param transaction: The transaction to submit. :type transaction: Iterable[TransactionEntry] :param error_cls: The exception class to raise on error. :type error_cls: builtins.type[DatasetException] :raises error_cls: An error submitting the transaction. .. py:method:: _delete_table() -> None Deletes the entire table from Azure Table Storage. :returns: None :raises DeleteError: If the table could not be deleted. .. py:method:: _create_table() -> None Creates a table in Azure Table Storage if it does not exist. :returns: None :raises CreateError: If the table could not be created due to an error other than it already existing. .. py:method:: read(**_kwargs: Any) -> None Read Azure Table Storage dataset. :param _kwargs: Additional keyword arguments :returns: None :raises ReadError: If there is an error reading from Azure Table Storage. .. py:method:: create(**_kwargs: Any) -> None Create an entity in Azure Table Storage. :returns: None :raises CreateError: If the entity could not be created. .. py:method:: update(**_kwargs: Any) -> None Update an entity in Azure Table Storage. :returns: None .. py:method:: delete(**_kwargs: Any) -> None Delete specific entities from Azure Table Storage. Only entities specified in `self.input` are deleted, matched by PartitionKey and RowKey. :param _kwargs: Additional keyword arguments :returns: None :raises DeleteError: If there is an error deleting from Azure Table Storage. .. py:method:: rename() -> NoReturn Rename the resource in the backend. Atomic. Not idempotent. :raises RenameError: If the operation fails. :raises NotSupportedError: If the provider does not support renaming. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``rename()`` .. py:method:: close() -> None No need to close the linked service. Just to comply with the interface. :returns: None .. py:method:: list() -> NoReturn Discover available resources and populate ``self.output`` with a DataFrame of resources and their metadata. Idempotent. :raises ListError: If the operation fails. :raises NotSupportedError: If the provider does not support listing. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``list()`` .. py:method:: purge(**_kwargs: Any) -> None Purge all entities from the table or drop the entire table. If `delete_table=True` in settings, deletes the entire table. Otherwise, deletes all entities from the table, leaving it empty. :returns: None :raises DeleteError: If there is an error purging from Azure Table Storage. .. py:method:: upsert(**_kwargs: Any) -> None Insert rows that do not exist, update rows that do, matched by identity columns defined in ``self.settings``. Atomic. :raises UpsertError: If the operation fails. :raises NotSupportedError: If the provider does not support upsert. .. seealso:: Full contract: ``docs/DATASET_CONTRACT.md`` -- ``upsert()`` .. py:method:: get_details() -> dict[str, Any] Get details about the dataset. :returns: dict[str, Any]