ds_provider_mock_py_lib.dataset =============================== .. py:module:: ds_provider_mock_py_lib.dataset .. autoapi-nested-parse:: **File:** ``__init__.py`` **Region:** ``ds_provider_mock_py_lib/dataset`` Mock Dataset This module implements a read-only synthetic dataset for e2e testing. .. rubric:: Example >>> from uuid import uuid4 >>> from ds_provider_mock_py_lib.dataset import MockDataset, MockDatasetSettings >>> from ds_provider_mock_py_lib.linked_service import MockLinkedService, MockLinkedServiceSettings >>> linked_service = MockLinkedService( ... id=uuid4(), ... name="mock-ls", ... version="1.0.0", ... settings=MockLinkedServiceSettings(), ... ) >>> dataset = MockDataset( ... id=uuid4(), ... name="mock-ds", ... version="1.0.0", ... linked_service=linked_service, ... settings=MockDatasetSettings(row_count=5), ... ) >>> linked_service.connect() >>> dataset.read() >>> data = dataset.output Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ds_provider_mock_py_lib/dataset/engines/index /autoapi/ds_provider_mock_py_lib/dataset/mock/index /autoapi/ds_provider_mock_py_lib/dataset/settings/index Classes ------- .. autoapisummary:: ds_provider_mock_py_lib.dataset.MockDataset ds_provider_mock_py_lib.dataset.MockColumn ds_provider_mock_py_lib.dataset.MockDatasetSettings Package Contents ---------------- .. py:class:: MockDataset Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`ds_provider_mock_py_lib.linked_service.mock.MockLinkedService`\ , :py:obj:`ds_provider_mock_py_lib.dataset.settings.MockDatasetSettings`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer`\ ] Read-only tabular dataset that emits deterministic synthetic rows. .. py:attribute:: linked_service :type: ds_provider_mock_py_lib.linked_service.mock.MockLinkedService .. py:attribute:: settings :type: ds_provider_mock_py_lib.dataset.settings.MockDatasetSettings .. py:attribute:: serializer :type: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None .. py:attribute:: deserializer :type: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None .. py:method:: __post_init__() -> None Ensure serializer and deserializer are always available. .. py:property:: supports_checkpoint :type: bool Whether this dataset supports incremental loads via ``self.checkpoint``. :returns: Always ``True`` for the mock provider. :rtype: bool .. py:property:: type :type: ds_provider_mock_py_lib.enums.ResourceType Get the type of the dataset. :returns: ResourceType .. py:method:: read() -> None Read the current mock batch into ``self.output``. An empty checkpoint performs a full load (batch 0). A populated incremental watermark reads the next change batch. A pagination cursor resumes an in-flight batch after failure. :raises ReadError: If the mock backend fails or settings are invalid for read. :raises ConnectionError: If dataset-level injection is configured as a connection error. .. py:method:: create() -> NoReturn Create is not supported by this dataset. .. py:method:: update() -> NoReturn Update is not supported by this dataset. .. py:method:: upsert() -> NoReturn Upsert is not supported by this dataset. .. py:method:: delete() -> NoReturn Delete is not supported by this dataset. .. py:method:: purge() -> NoReturn Purge is not supported by this dataset. .. py:method:: rename() -> NoReturn Rename is not supported by this dataset. .. py:method:: list() -> NoReturn List is not supported by this dataset. .. py:method:: close() -> None Close the dataset and underlying linked service. .. py:method:: _unsupported(method: str) -> NoReturn Raise ``NotSupportedError`` for a read-only method. :param method: Dataset method name. :raises NotSupportedError: Always. .. py:class:: MockColumn Bases: :py:obj:`ds_common_serde_py_lib.Serializable` One synthetic column in a mock dataset. .. py:attribute:: name :type: str Column name emitted in ``self.output``. .. py:attribute:: kind :type: ds_provider_mock_py_lib.enums.ColumnKind Value generator used for this column. .. py:attribute:: value :type: Any :value: None Payload for ``constant`` and ``enum``. A scalar is emitted on every row when ``kind`` is ``constant``. A non-empty list is sampled when ``kind`` is ``enum``. .. py:attribute:: prefix :type: str :value: '' Prefix used when ``kind`` is ``text``. .. py:attribute:: low :type: int :value: 0 Inclusive lower bound for random numeric kinds. .. py:attribute:: high :type: int :value: 1000 Exclusive upper bound for random numeric kinds. .. py:attribute:: null_every :type: int | None :value: None When set, every Nth id is ``None`` (stable across batches). .. py:class:: MockDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings that define mock read scope and injected failure behaviour. .. py:attribute:: columns :type: list[MockColumn] Synthetic columns included in every emitted row. .. py:attribute:: row_count :type: int :value: 100 Rows in the full load (batch 0, all inserts). .. py:attribute:: incremental_insert_count :type: int :value: 0 New unique primary keys delivered in each incremental batch. .. py:attribute:: incremental_update_count :type: int :value: 0 Existing primary keys re-delivered with changed column values (row hash changes). .. py:attribute:: incremental_noop_count :type: int :value: 0 Existing primary keys re-delivered with an identical row (same hash). .. py:attribute:: page_size :type: int | None :value: None Page size. ``None`` emits the whole batch in one page. .. py:attribute:: seed :type: int :value: 42 Determinism seed for id selection and random cell values. .. py:attribute:: page_delay_ms :type: int :value: 0 Artificial delay applied after each successful page. .. py:attribute:: op_column :type: str | None :value: None Optional label column for intended row kind. Gold merge uses primary key plus row hash, not this column. Leave unset so the label cannot change the hash of a noop row. Counts are always in ``operation.metadata["ops"]``. .. py:attribute:: modified_at_column :type: str :value: '_modified_at' Column that stores the deterministic modified-at timestamp. .. py:attribute:: raise_on_page :type: int | None :value: None 1-based page number within the current batch that should fail. ``None`` disables. .. py:attribute:: raise_as :type: ds_provider_mock_py_lib.enums.RaiseAs Contract exception class used when ``raise_on_page`` matches. .. py:attribute:: raise_error :type: ds_provider_mock_py_lib.models.MockError Error spec used when ``raise_on_page`` matches.