ds_provider_mock_py_lib ======================= .. py:module:: ds_provider_mock_py_lib .. autoapi-nested-parse:: **File:** ``__init__.py`` **Region:** ``ds-provider-mock-py-lib`` Description ----------- Mock provider for e2e pipeline testing. Read-only, fully deterministic, deterministic change batches for gold merge via primary key and row hash. .. rubric:: Example .. code-block:: python from uuid import uuid4 from ds_provider_mock_py_lib import ( MockDataset, MockDatasetSettings, MockLinkedService, MockLinkedServiceSettings, ) linked_service = MockLinkedService( id=uuid4(), name="mock-ls", version="1.0.0", settings=MockLinkedServiceSettings(), ) dataset = MockDataset( id=uuid4(), name="mock-ds", version="1.0.0", linked_service=linked_service, settings=MockDatasetSettings(row_count=10), ) linked_service.connect() dataset.read() print(len(dataset.output.index)) Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ds_provider_mock_py_lib/dataset/index /autoapi/ds_provider_mock_py_lib/enums/index /autoapi/ds_provider_mock_py_lib/errors/index /autoapi/ds_provider_mock_py_lib/linked_service/index /autoapi/ds_provider_mock_py_lib/models/index Attributes ---------- .. autoapisummary:: ds_provider_mock_py_lib.__version__ Classes ------- .. autoapisummary:: ds_provider_mock_py_lib.MockColumn ds_provider_mock_py_lib.MockDataset ds_provider_mock_py_lib.MockDatasetSettings ds_provider_mock_py_lib.MockLinkedService ds_provider_mock_py_lib.MockLinkedServiceSettings Package Contents ---------------- .. py:class:: MockColumn Bases: :py:obj:`ds_common_serde_py_lib.Serializable` One synthetic column in a mock dataset. .. py:attribute:: name :type: str Column name emitted in ``self.output``. .. py:attribute:: kind :type: ds_provider_mock_py_lib.enums.ColumnKind Value generator used for this column. .. py:attribute:: value :type: Any :value: None Payload for ``constant`` and ``enum``. A scalar is emitted on every row when ``kind`` is ``constant``. A non-empty list is sampled when ``kind`` is ``enum``. .. py:attribute:: prefix :type: str :value: '' Prefix used when ``kind`` is ``text``. .. py:attribute:: low :type: int :value: 0 Inclusive lower bound for random numeric kinds. .. py:attribute:: high :type: int :value: 1000 Exclusive upper bound for random numeric kinds. .. py:attribute:: null_every :type: int | None :value: None When set, every Nth id is ``None`` (stable across batches). .. py:class:: MockDataset Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`ds_provider_mock_py_lib.linked_service.mock.MockLinkedService`\ , :py:obj:`ds_provider_mock_py_lib.dataset.settings.MockDatasetSettings`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer`\ ] Read-only tabular dataset that emits deterministic synthetic rows. .. py:attribute:: linked_service :type: ds_provider_mock_py_lib.linked_service.mock.MockLinkedService .. py:attribute:: settings :type: ds_provider_mock_py_lib.dataset.settings.MockDatasetSettings .. py:attribute:: serializer :type: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None .. py:attribute:: deserializer :type: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None .. py:method:: __post_init__() -> None Ensure serializer and deserializer are always available. .. py:property:: supports_checkpoint :type: bool Whether this dataset supports incremental loads via ``self.checkpoint``. :returns: Always ``True`` for the mock provider. :rtype: bool .. py:property:: type :type: ds_provider_mock_py_lib.enums.ResourceType Get the type of the dataset. :returns: ResourceType .. py:method:: read() -> None Read the current mock batch into ``self.output``. An empty checkpoint performs a full load (batch 0). A populated incremental watermark reads the next change batch. A pagination cursor resumes an in-flight batch after failure. :raises ReadError: If the mock backend fails or settings are invalid for read. :raises ConnectionError: If dataset-level injection is configured as a connection error. .. py:method:: create() -> NoReturn Create is not supported by this dataset. .. py:method:: update() -> NoReturn Update is not supported by this dataset. .. py:method:: upsert() -> NoReturn Upsert is not supported by this dataset. .. py:method:: delete() -> NoReturn Delete is not supported by this dataset. .. py:method:: purge() -> NoReturn Purge is not supported by this dataset. .. py:method:: rename() -> NoReturn Rename is not supported by this dataset. .. py:method:: list() -> NoReturn List is not supported by this dataset. .. py:method:: close() -> None Close the dataset and underlying linked service. .. py:method:: _unsupported(method: str) -> NoReturn Raise ``NotSupportedError`` for a read-only method. :param method: Dataset method name. :raises NotSupportedError: Always. .. py:class:: MockDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings that define mock read scope and injected failure behaviour. .. py:attribute:: columns :type: list[MockColumn] Synthetic columns included in every emitted row. .. py:attribute:: row_count :type: int :value: 100 Rows in the full load (batch 0, all inserts). .. py:attribute:: incremental_insert_count :type: int :value: 0 New unique primary keys delivered in each incremental batch. .. py:attribute:: incremental_update_count :type: int :value: 0 Existing primary keys re-delivered with changed column values (row hash changes). .. py:attribute:: incremental_noop_count :type: int :value: 0 Existing primary keys re-delivered with an identical row (same hash). .. py:attribute:: page_size :type: int | None :value: None Page size. ``None`` emits the whole batch in one page. .. py:attribute:: seed :type: int :value: 42 Determinism seed for id selection and random cell values. .. py:attribute:: page_delay_ms :type: int :value: 0 Artificial delay applied after each successful page. .. py:attribute:: op_column :type: str | None :value: None Optional label column for intended row kind. Gold merge uses primary key plus row hash, not this column. Leave unset so the label cannot change the hash of a noop row. Counts are always in ``operation.metadata["ops"]``. .. py:attribute:: modified_at_column :type: str :value: '_modified_at' Column that stores the deterministic modified-at timestamp. .. py:attribute:: raise_on_page :type: int | None :value: None 1-based page number within the current batch that should fail. ``None`` disables. .. py:attribute:: raise_as :type: ds_provider_mock_py_lib.enums.RaiseAs Contract exception class used when ``raise_on_page`` matches. .. py:attribute:: raise_error :type: ds_provider_mock_py_lib.models.MockError Error spec used when ``raise_on_page`` matches. .. py:class:: MockLinkedService Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.linked_service.LinkedService`\ [\ :py:obj:`MockLinkedServiceSettings`\ ] In-memory linked service with tunable connect, health, and request failures. .. py:attribute:: settings :type: MockLinkedServiceSettings .. py:attribute:: _connection :type: MockBackend | None :value: None .. py:attribute:: _connect_attempts :type: int :value: 0 .. py:property:: type :type: ds_provider_mock_py_lib.enums.ResourceType Get the type of the linked service. :returns: ResourceType .. py:property:: connection :type: MockBackend Return the mock backend established by ``connect()``. :returns: The connected backend handle. :rtype: MockBackend :raises ConnectionError: If ``connect()`` has not been called. .. py:method:: connect() -> None Establish a connection to the mock backend. :raises ConnectionError: When connect is configured to fail or is still in the transient-failure window. :raises AuthenticationError: When ``connect_behaviour`` is ``authentication_error``. :raises AuthorizationError: When ``connect_behaviour`` is ``authorization_error``. .. py:method:: test_connection() -> tuple[bool, str] Verify mock backend health without raising on failure. :returns: ``(True, "")`` on success, otherwise ``(False, reason)``. :rtype: tuple[bool, str] .. py:method:: close() -> None Release the mock backend handle. Safe to call repeatedly. .. py:class:: MockLinkedServiceSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.linked_service.LinkedServiceSettings` Settings required to connect to the in-memory mock backend. .. py:attribute:: connect_behaviour :type: ds_provider_mock_py_lib.enums.ConnectBehaviour What ``connect()`` does after optional transient failures. .. py:attribute:: connect_delay_ms :type: int :value: 0 Artificial delay applied during ``connect()``. .. py:attribute:: connect_fail_first_n :type: int :value: 0 First N ``connect()`` calls raise ``ConnectionError``, then succeed. .. py:attribute:: connect_error :type: ds_provider_mock_py_lib.models.MockError Error spec used when ``connect()`` is configured to fail. .. py:attribute:: test_connection_ok :type: bool :value: True When ``False``, ``test_connection()`` returns a failed health check. .. py:attribute:: test_connection_message :type: str :value: '' Reason returned when ``test_connection_ok`` is ``False``. .. py:attribute:: latency_ms :type: int :value: 0 Artificial delay applied to every ``connection.request()`` call. .. py:attribute:: fail_on_call :type: int | None :value: None Raise ``MockBackendError`` on the Nth ``request()`` call. ``None`` disables. .. py:attribute:: request_error :type: ds_provider_mock_py_lib.models.MockError Error spec used when ``fail_on_call`` matches. .. py:attribute:: drop_after_calls :type: int | None :value: None Every ``request()`` after N calls raises. ``None`` disables. .. py:attribute:: drop_error :type: ds_provider_mock_py_lib.models.MockError Error spec used when the connection is dropped after N calls. .. py:data:: __version__