ds_provider_tripletex_py_lib.dataset.tripletex ============================================== .. py:module:: ds_provider_tripletex_py_lib.dataset.tripletex .. autoapi-nested-parse:: **File:** ``tripletex.py`` **Region:** ``ds_provider_tripletex_py_lib/dataset/tripletex`` Read-only dataset for the Tripletex v2 API. Products are read via ``from``/``count`` offset pagination, or rolling monthly date windows for ``ledger/posting``/``balance_sheet`` (see ``read_info.py``). Most products cache via ``versionDigest``/``If-None-Match``; a few (``customer``, ``supplier``, their ``_lite`` variants) instead use Tripletex's ``changedSince`` filter. All write methods raise ``NotSupportedError``. .. rubric:: Example >>> dataset = TripletexDataset( ... settings=TripletexDatasetSettings(product_name=TripletexProductName.CUSTOMER), ... linked_service=TripletexLinkedService( ... settings=TripletexLinkedServiceSettings( ... consumer_token="my-consumer-token", ... employee_token="my-employee-token", ... ), ... ), ... ) >>> dataset.linked_service.connect() >>> dataset.read() >>> data = dataset.output Attributes ---------- .. autoapisummary:: ds_provider_tripletex_py_lib.dataset.tripletex.logger ds_provider_tripletex_py_lib.dataset.tripletex.TripletexDatasetSettingsType ds_provider_tripletex_py_lib.dataset.tripletex.TripletexLinkedServiceType Classes ------- .. autoapisummary:: ds_provider_tripletex_py_lib.dataset.tripletex.TripletexReadSettings ds_provider_tripletex_py_lib.dataset.tripletex.TripletexDatasetSettings ds_provider_tripletex_py_lib.dataset.tripletex._ReadContext ds_provider_tripletex_py_lib.dataset.tripletex.TripletexDataset Functions --------- .. autoapisummary:: ds_provider_tripletex_py_lib.dataset.tripletex._explode_column ds_provider_tripletex_py_lib.dataset.tripletex._build_fields_param ds_provider_tripletex_py_lib.dataset.tripletex._request_key ds_provider_tripletex_py_lib.dataset.tripletex._add_years ds_provider_tripletex_py_lib.dataset.tripletex._period_generator Module Contents --------------- .. py:data:: logger .. py:class:: TripletexReadSettings Bases: :py:obj:`ds_common_serde_py_lib.Serializable` Settings for Tripletex read operations. .. py:attribute:: count :type: int :value: 5000 Page size for ``from``/``count`` offset pagination. .. py:attribute:: fields :type: list[Any] | None :value: None Field selector. Required with ``path``; ignored with ``product_name``. Each item is a field name (``str``), or a one-key dict expanding a nested object/sub-collection, e.g. ``{"account": ["id"]}``. .. rubric:: Example >>> TripletexReadSettings(path="custom/thing", fields=["id", "name", {"account": ["id"]}]) # -> fields=id,name,account(id) .. py:attribute:: params :type: dict[str, Any] | None :value: None Additional query parameters merged into every request (e.g. ``isInactive``). Not for ``dateFrom``/``dateTo`` on ``date_window``-paginated products (``ledger_posting``, ``balance_sheet``) -- those are auto-generated per window and would override any values passed here. .. py:attribute:: path :type: str | None :value: None Custom API path, for reading an endpoint not in the packaged product catalog. Set together with ``fields`` and ``pagination`` instead of ``TripletexDatasetSettings.product_name``. Exactly one of ``product_name``/``path`` may be set -- both raises ``ReadError``. .. rubric:: Example >>> TripletexReadSettings(path="custom/thing", fields=["id"], pagination=PaginationKind.OFFSET) .. py:attribute:: pagination :type: ds_provider_tripletex_py_lib.enums.PaginationKind | None :value: None Required when ``path`` is set. Ignored (must be unset) when ``product_name`` is set. .. py:attribute:: changed_since :type: bool | None :value: None Whether ``path`` supports Tripletex's ``changedSince`` filter. Required when ``path`` is set -- no safe default, since silently assuming ``False`` could mask never having checked. Ignored (must be unset) when ``product_name`` is set (fixed by its own metadata). .. py:attribute:: date_from :type: str | None :value: None Start of a date-windowed read (e.g. ``ledger_posting``), in years back from today. A relative year offset, e.g. ``"3"`` or ``"-3"`` (sign doesn't matter). Defaults to 3 years back when unset. Only valid for :attr:`PaginationKind.DATE_WINDOW` reads -- otherwise raises ``ReadError``. .. rubric:: Example >>> TripletexReadSettings(date_from="3") .. py:attribute:: explode_columns :type: list[str] | None :value: None Field names that are a list of objects, exploded into one row per element. Only meaningful with ``path`` -- ignored (must be unset) when ``product_name`` is set, since a packaged product's explode columns are fixed by its own metadata. See ``read_info.py`` for why this differs from a plain nested object (which flattens in place, no explode needed). .. py:class:: TripletexDatasetSettings Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings` Settings for the Tripletex dataset. .. py:attribute:: product_name :type: ds_provider_tripletex_py_lib.enums.TripletexProductName | None :value: None Tripletex product to read. Exactly one of ``product_name``/``read.path`` may be set. .. py:attribute:: read :type: TripletexReadSettings Settings for ``read()``. .. py:data:: TripletexDatasetSettingsType .. py:data:: TripletexLinkedServiceType .. py:class:: _ReadContext Mutable, per-call state threaded through one ``read()`` call's pagination. Only holds values that aren't otherwise reachable via ``self`` on ``TripletexDataset``: ``url``/``changed_since`` come from the resolved ``read_info``, a local in ``_execute_read`` rather than stored on the instance; ``checksums``/``watermarks``/``records`` are local, mutable copies the readers write into (``self.checkpoint``/``self.output`` aren't updated until after the reader returns). Everything else the readers need (``self.linked_service.connection``, ``self.settings.read.params``, ``self.settings.read.count``) is read directly via ``self``, same as ``self.settings.read.count`` already is. .. py:attribute:: url :type: str .. py:attribute:: fields_param :type: str .. py:attribute:: checksums :type: dict[str, str] .. py:attribute:: watermarks :type: dict[str, str] .. py:attribute:: records :type: list[dict[str, Any]] .. py:attribute:: changed_since :type: bool .. py:class:: TripletexDataset Bases: :py:obj:`ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset`\ [\ :py:obj:`TripletexLinkedServiceType`\ , :py:obj:`TripletexDatasetSettingsType`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer`\ , :py:obj:`ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer`\ ], :py:obj:`Generic`\ [\ :py:obj:`TripletexLinkedServiceType`\ , :py:obj:`TripletexDatasetSettingsType`\ ] Read-only tabular dataset for Tripletex products. Exposed (caller-configurable): ``id``, ``name``, ``description``, ``version``, ``settings``, ``linked_service``, ``serializer``, ``deserializer``, ``checkpoint``. ``input``/``output`` (inherited from ``TabularDataset``) are excluded from serialization -- they hold runtime read results, not configuration -- though still constructible, matching the base class. No field on this class is fully internal (``init=False``). .. py:attribute:: linked_service :type: TripletexLinkedServiceType .. py:attribute:: settings :type: TripletexDatasetSettingsType .. py:method:: __post_init__() -> None Fill in a default serializer/deserializer if the caller didn't provide one. .. py:property:: type :type: ds_provider_tripletex_py_lib.enums.ResourceType Return the dataset resource type. .. py:property:: supports_checkpoint :type: bool Whether this dataset supports incremental loads via ``self.checkpoint``. Checkpoint holds ``{"checksums": {...}, "watermarks": {...}}`` -- stored ``versionDigest``/``changedSince`` values per request. An empty checkpoint (``{}``) means a full load. :returns: Always ``True``. :rtype: bool .. py:method:: _resolve_reader(pagination: ds_provider_tripletex_py_lib.enums.PaginationKind) -> collections.abc.Callable[[_ReadContext], None] Return the read implementation for the given pagination kind. :param pagination: The resolved endpoint's pagination kind. :returns: ``_read_date_windowed`` for ``DATE_WINDOW``, otherwise ``_read_paginated``. :rtype: Callable[[_ReadContext], None] .. py:method:: _resolve_product_name(product_name: str) -> ds_provider_tripletex_py_lib.enums.TripletexProductName Resolve a raw ``settings.product_name`` value into a ``TripletexProductName``. :param product_name: The raw product name to validate. :returns: The resolved product. :rtype: TripletexProductName :raises ReadError: If not a recognized :class:`TripletexProductName` value. .. py:method:: _resolve_packaged_read_info(product_name: str) -> ds_provider_tripletex_py_lib.read_info.ReadInfo Resolve read info for a packaged product, wrapping metadata failures as ``ReadError``. :param product_name: The raw ``settings.product_name`` value to resolve. :returns: The packaged product's read info. :rtype: ReadInfo :raises ReadError: If not a recognized product, or its packaged read metadata is missing or malformed. .. py:method:: _resolve_read_info() -> ds_provider_tripletex_py_lib.read_info.ReadInfo Resolve the endpoint to read: a packaged product, or a custom path. ``settings.product_name`` is resolved via the packaged asset catalog if set; otherwise ``settings.read.path``/``fields``/``pagination`` are used. See ``_validate_read_settings`` for the combination rules. :returns: The fully resolved endpoint to read. :rtype: ReadInfo :raises ReadError: If the settings combination is invalid, or the resolved product/metadata is unrecognized/malformed. .. py:method:: _resolve_read_info_from_settings() -> ds_provider_tripletex_py_lib.read_info.ReadInfo Build a ``ReadInfo`` directly from ``settings.read``, for a custom (non-packaged) endpoint. Only called when ``settings.product_name`` is unset. Raises immediately on a missing value, so every field passed to ``ReadInfo`` is already guaranteed present. :returns: The endpoint built from ``settings.read``. :rtype: ReadInfo :raises ReadError: If ``path``/``fields``/``pagination``/``changed_since`` aren't all set alongside ``product_name`` being unset. .. py:method:: _validate_read_settings(read_info: ds_provider_tripletex_py_lib.read_info.ReadInfo) -> None Validate ``settings.product_name``/``settings.read`` against the resolved ``read_info``. :param read_info: The resolved endpoint to read. :raises ReadError: If any ``read.*`` setting is set alongside ``product_name`` (ignored there), ``read.date_from`` is set but pagination isn't :attr:`PaginationKind.DATE_WINDOW`, or ``changed_since`` is combined with date-windowed pagination (``_read_date_windowed`` never emits ``changedSince``). .. py:method:: read() -> None Read all rows for the configured product and assign them to ``self.output``. See ``_resolve_read_info`` for how the endpoint is resolved. :raises AuthenticationError: If authentication fails. :raises AuthorizationError: If authorization fails. :raises ConnectionError: If the transport cannot reach Tripletex. :raises ReadError: If neither a known product nor a custom path is configured, or the read fails. .. py:method:: _execute_read(read_info: ds_provider_tripletex_py_lib.read_info.ReadInfo, fields_param: str) -> None Run the resolved read against Tripletex and populate ``self.output``/``self.checkpoint``. :param read_info: The endpoint to read, resolved by ``_resolve_read_info``. :param fields_param: The rendered ``fields`` query parameter value. :raises AuthenticationError: If authentication fails. :raises AuthorizationError: If authorization fails. :raises ConnectionError: If the transport cannot reach Tripletex. :raises ReadError: If the read fails for any other reason. .. py:method:: _read_paginated(ctx: _ReadContext) -> None Fetch every page of a ``from``/``count`` offset-paginated product into ``ctx.records``. Dispatches to ``_read_paginated_by_changed_since`` (products supporting Tripletex's ``changedSince`` filter) or ``_read_paginated_by_digest`` (``versionDigest``/``If-None-Match``, every other product). .. py:method:: _read_paginated_by_changed_since(ctx: _ReadContext) -> None Fetch every page of a ``changedSince``-capable product into ``ctx.records``. Tripletex's ``changedSince`` param (format ``YYYY-MM-DDThh:mm:ssZ``) filters server-side to only rows changed since that timestamp. The next run's watermark is captured as this run's start time (so concurrent changes aren't missed) and only stored in ``ctx.watermarks`` once every page succeeds. .. py:method:: _read_paginated_by_digest(ctx: _ReadContext) -> None Fetch every page of an offset-paginated product using ``versionDigest``/``If-None-Match``. See ``_fetch_pages_by_digest`` for the mechanism. .. py:method:: _fetch_pages_by_digest(ctx: _ReadContext, *, request_key: str, extra_query_params: dict[str, Any] | None = None) -> None Page through ``ctx.url`` via ``versionDigest``/``If-None-Match``, caching under ``request_key``. Exits on ``304`` (unchanged); falls back to ``"magic-value"`` since ``versionDigest`` can be JSON ``null``. :param ctx: Read context to append ``records`` into and cache ``checksums`` on. :param request_key: Cache key identifying this request shape. :param extra_query_params: Extra query params beyond ``from``/``count``/``fields``. .. py:method:: _resolve_date_from(*, date_to: datetime.date) -> datetime.date Resolve ``settings.read.date_from`` into an absolute start date. A relative year offset from ``date_to`` (e.g. ``"3"``/``"-3"``, sign ignored). Defaults to 3 years back when unset. :param date_to: The read window's end date (today). :returns: The resolved start date. :rtype: date :raises ReadError: If ``settings.read.date_from`` is not a valid integer. .. py:method:: _read_date_windowed(ctx: _ReadContext) -> None Fetch a date-windowed product (``ledger/posting``) across rolling monthly windows. Tripletex requires a bounded ``dateFrom``/``dateTo`` per request, so this walks the read window in monthly steps, offset-paginating and digest-caching within each. ``ctx.watermarks``/``ctx.changed_since`` are never touched here -- no date-windowed product supports ``changedSince`` -- but ``ctx`` is still accepted so the calling convention matches ``_read_paginated``. :raises ReadError: If ``settings.read.date_from`` is not a valid integer. .. py:method:: create() -> NoReturn Create is not supported by the Tripletex provider. .. py:method:: update() -> NoReturn Update is not supported by the Tripletex provider. .. py:method:: delete() -> NoReturn Delete is not supported by the Tripletex provider. .. py:method:: upsert() -> NoReturn Upsert is not supported by the Tripletex provider. .. py:method:: rename() -> NoReturn Rename is not supported by the Tripletex provider. .. py:method:: purge() -> NoReturn Purge is not supported by the Tripletex provider. .. py:method:: list() -> NoReturn List is not supported by the Tripletex provider. .. py:method:: close() -> None Close the linked-service connection. .. py:function:: _explode_column(df: pandas.DataFrame, column: str) -> pandas.DataFrame Explode a list-of-objects column (e.g. ``bankAccountPresentation``) into one row per element. :param df: DataFrame with ``column`` still holding raw list values. :param column: Name of the column to explode. :returns: ``df`` with ``column`` replaced by one row per element. :rtype: pd.DataFrame :raises ReadError: If ``column`` is missing from a non-empty ``df`` (likely a packaging bug), or a non-null value in ``column`` isn't a ``list``. A fully empty ``df`` is a no-op instead. .. py:function:: _build_fields_param(fields: list[Any]) -> str Build a Tripletex ``fields`` query parameter value from a field selector list. Each item is a field name (``str``), or a one-key dict -- ``{"group": [...]}`` -- expanding to ``group(...)``. :param fields: Field selector list, e.g. ``["id", {"account": ["id"]}]``. :returns: Comma-separated Tripletex field expression, e.g. ``"id,account(id)"``. :rtype: str .. py:function:: _request_key(fields_param: str, *, date_from: str | None = None, date_to: str | None = None, extra_params: dict[str, Any] | None = None) -> str Derive a stable cache key for the ``versionDigest``/``changedSince`` checkpoint. Two requests share a cache entry only if they'd return the same records: different ``fields``, ``params``, or date window each get their own key, but different pages (``from``/``count``) of the same query share one. :returns: A SHA-256 hex digest identifying this request shape. :rtype: str .. py:function:: _add_years(reference: datetime.date, years: int) -> datetime.date Shift ``reference`` by a number of calendar years, positive or negative. Falls back to February 28th when shifting a leap day (Feb 29th) into a non-leap year. :param reference: The date to shift. :param years: Number of years to shift by; negative shifts into the past. :returns: ``reference`` shifted by ``years`` calendar years. :rtype: date .. py:function:: _period_generator(*, date_from: datetime.date, date_to: datetime.date) -> collections.abc.Iterator[tuple[datetime.date, datetime.date]] Generate monthly periods used to page the ``ledger/posting`` endpoint. :param date_from: Inclusive date to start generating periods from. :param date_to: Exclusive date to stop generating periods at. :Yields: *tuple[date, date]* -- Each period as ``(period_from, period_to)``, where ``period_from`` is inclusive and ``period_to`` is exclusive. Callers wanting to include a specific day (e.g. today) must pass a ``date_to`` one day past it -- see ``_read_date_windowed``.