ds_provider_powerofficego_py_lib.dataset

File: __init__.py Region: ds_provider_powerofficego_py_lib/dataset

This module contains dataset-related classes and functions for the PowerOfficeGo provider.

Submodules

• ds_provider_powerofficego_py_lib.dataset.powerofficego

Classes

PowerOfficeGoDataset	PowerOfficeGoDataset represents a dataset for the PowerOfficeGo provider.
PowerOfficeGoDatasetSettings	Settings for PowerOfficeGo dataset.
ReadSettings	Settings for reading from PowerOfficeGo dataset.

Package Contents

class ds_provider_powerofficego_py_lib.dataset.PowerOfficeGoDataset

Bases: ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset[PowerOfficeGoLinkedServiceType, PowerOfficeGoDatasetSettingsType, ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer, ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer], Generic[PowerOfficeGoLinkedServiceType, PowerOfficeGoDatasetSettingsType]

PowerOfficeGoDataset represents a dataset for the PowerOfficeGo provider.

linked_service

The linked service used to connect to PowerOfficeGo.

Type:

PowerOfficeGoLinkedServiceType

settings

The settings for the dataset, including data product and read settings.

Type:

PowerOfficeGoDatasetSettingsType

serializer

The serializer used for the dataset. Defaults to JSON format.

Type:

PandasSerializer | None

deserializer

The deserializer used for the dataset. Defaults to JSON format.

Type:

PandasDeserializer | None

linked_service: PowerOfficeGoLinkedServiceType

settings: PowerOfficeGoDatasetSettingsType

serializer: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None

deserializer: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None

property type: ds_provider_powerofficego_py_lib.enums.ResourceType

Get the type of the dataset.

property supports_checkpoint: bool

Whether this provider supports incremental loads via self.checkpoint.

The checkpoint is a dictionary that tracks pagination and incremental state:

• On a full load, self.checkpoint is expected to be empty ({}) or None. In this case, read() starts from page 1.

• After each successfully read page, self.checkpoint is updated with at least {"last_page": page, ...}.

• If incremental loading is possible (i.e., the data contains a lastChangedDateTimeOffset field), the checkpoint will also include an incremental key with the latest observed value:

  {"incremental": {"last_modified_date": ...}}.

• On a subsequent run, if self.checkpoint contains a "last_page" entry, read() resumes from last_page + 1 and continues fetching data from the PowerOfficeGo API.

• If self.checkpoint contains an incremental key, the loader will use the stored last_modified_date to filter for new/changed records.

This allows consumers to perform incremental loads by persisting and reusing the checkpoint between executions, avoiding re-reading pages that were already processed successfully. The checkpoint structure is designed to support both paginated and incremental (watermark-based) loading.

Returns:

True if checkpointing is supported, False otherwise.

Return type:

bool

read() → None

Read data from PowerOfficeGo API from requested endpoint.

Raises:

ReadError – If there is an error during the read operation.

create() → None

Insert all rows in self.input into the target as a single atomic transaction. Must not delete, update, or overwrite existing data.

Raises:

• CreateError – If the operation fails.

• NotSupportedError – If the provider does not support create.

See also

Full contract: docs/DATASET_CONTRACT.md – create()

delete() → None

Remove specific rows from the target matched by identity columns defined in self.settings. Atomic. Idempotent.

Raises:

• DeleteError – If the operation fails.

• NotSupportedError – If the provider does not support delete.

See also

Full contract: docs/DATASET_CONTRACT.md – delete()

update() → None

Update existing rows in the target matched by identity columns defined in self.settings. Atomic. Must not insert new rows.

Raises:

• UpdateError – If the operation fails.

• NotSupportedError – If the provider does not support update.

See also

Full contract: docs/DATASET_CONTRACT.md – update()

rename() → None

Rename the resource in the backend. Atomic. Not idempotent.

Raises:

• RenameError – If the operation fails.

• NotSupportedError – If the provider does not support renaming.

See also

Full contract: docs/DATASET_CONTRACT.md – rename()

list() → None

Discover available resources and populate self.output with a DataFrame of resources and their metadata. Idempotent.

Raises:

• ListError – If the operation fails.

• NotSupportedError – If the provider does not support listing.

See also

Full contract: docs/DATASET_CONTRACT.md – list()

upsert() → None

Insert rows that do not exist, update rows that do, matched by identity columns defined in self.settings. Atomic.

Raises:

• UpsertError – If the operation fails.

• NotSupportedError – If the provider does not support upsert.

See also

Full contract: docs/DATASET_CONTRACT.md – upsert()

purge() → None

Remove all content from the target. self.input is not used. Atomic. Idempotent.

Raises:

• PurgeError – If the operation fails.

• NotSupportedError – If the provider does not support purge.

See also

Full contract: docs/DATASET_CONTRACT.md – purge()

_fetch_data(session: Any) → None

Fetch data from PowerOfficeGo API using the provided session.

Parameters:

session (Any) – The session object to use for making API requests.

Raises:

ReadError – If there is an error during the data fetching process.

static _parse_iso8601_timestamp(value: str) → datetime.datetime

Parse an ISO-8601 timestamp string to an aware UTC datetime for comparison. Handles up to 7 digits of fractional seconds by truncating to 6 (Python max).

_greatest_time_field_value(values: collections.abc.Sequence[Any]) → Any

Return the original value that sorts last by parsed UTC datetime.

greatest_incremental_value(values: collections.abc.Sequence[Any], *, kind: str) → Any | None

Return the greatest watermark among observed values for the given strategy.

Parameters:

• values – Non-empty sequence of observed watermark candidates (nulls should be excluded by callers).

• kind – Incremental strategy from metadata (e.g. time_field).

Returns:

The winning original value from values, or None when values is empty.

Raises:

• InvalidIncrementalWatermarkException – When time_field values are not strings or parsing fails.

• UnsupportedIncrementalKindException – When kind is not supported.

_build_checkpoint(last_page: int | None, last_modified_date: str | None) → dict[str, Any]

Build a checkpoint dictionary to track the last successfully read page.

Include last_modified_date from read settings.

Parameters:

• last_page (int | None) – The last page number that was successfully read.

• last_modified_date (str | None) – The last modified date to include in the checkpoint.

Returns:

A checkpoint dictionary containing the last page information.

Return type:

dict[str, Any]

_build_params(page: int, last_modified_date: str | None) → dict[str, Any]

Build the query parameters for the PowerOfficeGo API request based on the dataset settings and pagination.

Parameters:

• page (int) – The page number to fetch.

• last_modified_date (str | None) – The last modified date to filter results.

Returns:

A dictionary of query parameters for the API request.

Return type:

dict[str, Any]

_build_url() → str

Build the URL for the PowerOfficeGo API request based on the dataset settings.

Returns:

The constructed URL for the API request.

Return type:

str

close() → None

Release any resources held by the dataset.

Connection lifecycle is managed by the linked service.

class ds_provider_powerofficego_py_lib.dataset.PowerOfficeGoDatasetSettings

Bases: ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings

Settings for PowerOfficeGo dataset.

data_product

Data product to read from PowerOfficeGo API. Required. Must match product name as used in DS Config.

Type:

str

read

Settings for reading from PowerOfficeGo dataset. Contains pagination and filtering options.

Type:

ReadSettings

data_product: str

Data product to read from PowerOfficeGo API. Required. Must match product name as used in DS Config.

read: ReadSettings

Settings for reading from PowerOfficeGo dataset.

class ds_provider_powerofficego_py_lib.dataset.ReadSettings

Bases: ds_common_serde_py_lib.Serializable

Settings for reading from PowerOfficeGo dataset.

page_size

Number of records to read per page. Default is 20,000.

Type:

int

fields

List of fields to include in the response. Optional.

Type:

list[str] | None

filters

Additional filters for the API request. Optional.

Type:

dict[str, Any] | None

page_size: int = 20000

Number of records to read per page. Default is 20,000.

fields: list[str] | None = None

List of fields to include in the response. Optional.

filters: dict[str, Any] | None = None

Additional filters for the API request. Optional.