ds_fetch_py_sdk =============== .. py:module:: ds_fetch_py_sdk .. autoapi-nested-parse:: AI Commons Fetch: read-only SQL over your tenant data lake. Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ds_fetch_py_sdk/auth/index /autoapi/ds_fetch_py_sdk/client/index /autoapi/ds_fetch_py_sdk/env/index /autoapi/ds_fetch_py_sdk/errors/index /autoapi/ds_fetch_py_sdk/models/index Package Contents ---------------- .. py:class:: BearerToken(token: str) Bases: :py:obj:`httpx.Auth` A platform JWT you already have. .. py:method:: auth_flow(request: httpx.Request) -> collections.abc.Generator[httpx.Request, httpx.Response, None] Execute the authentication flow. To dispatch a request, `yield` it: ``` yield request ``` The client will `.send()` the response back into the flow generator. You can access it like so: ``` response = yield request ``` A `return` (or reaching the end of the generator) will result in the client returning the last response obtained from the server. You can dispatch as many requests as is necessary. .. py:class:: BrowserLogin(*, client_id: str | None = None, port: int = 8976, cache: pathlib.Path | bool = True, open_browser: bool = True, on_url: collections.abc.Callable[[str], None] | None = None, timeout: float = 300, **kwargs: Any) Bases: :py:obj:`OAuth` Interactive sign-in in the browser, including enterprise SSO. Uses authorization code with PKCE on a loopback redirect. Without a `client_id` the SDK registers a public client (RFC 7591) and caches its id with the session. `on_url` receives the sign-in URL, for when the browser cannot be opened; it defaults to writing it to stderr. .. py:class:: ClientCredentials(client_id: str, client_secret: str, **kwargs: Any) Bases: :py:obj:`OAuth` Machine-to-machine sign-in with a registered client id and secret. .. py:class:: OAuth(*, scope: str = 'openid read', **kwargs: Any) Bases: :py:obj:`TokenAuth` OAuth 2.1 against the identity server, with endpoints from RFC 8414 metadata. .. py:property:: metadata :type: dict[str, Any] The identity server's RFC 8414 metadata, checked to be its own and to use HTTPS. .. py:class:: PasswordLogin(email: str | None = None, password: str | None = None, *, mfa_code: str | collections.abc.Callable[[int], str] | None = None, cache: pathlib.Path | bool = True, **kwargs: Any) Bases: :py:obj:`TokenAuth` Sign in with email and password at the identity server's `/auth/login/`. When the account has MFA, `mfa_code` supplies the TOTP code: a string, or a callable that receives the number of digits and returns the code. The session is renewed with its refresh token; once that expires, signing in again needs the password. .. py:class:: TokenAuth(*, issuer: str | None = None, cache: pathlib.Path | bool = False) Bases: :py:obj:`httpx.Auth` A session with the identity server: acquires, caches and renews tokens, and retries once on 401. Sessions are cached per environment in `default_cache()`, or at `cache`; `cache=False` keeps them in memory. .. py:method:: login() -> None Sign in now, replacing any cached session. .. py:method:: logout() -> None Forget the session, in memory and in the cache. .. py:method:: auth_flow(request: httpx.Request) -> collections.abc.Generator[httpx.Request, httpx.Response, None] Execute the authentication flow. To dispatch a request, `yield` it: ``` yield request ``` The client will `.send()` the response back into the flow generator. You can access it like so: ``` response = yield request ``` A `return` (or reaching the end of the generator) will result in the client returning the last response obtained from the server. You can dispatch as many requests as is necessary. .. py:function:: cached_session(env: ds_fetch_py_sdk.env.Environment, cache: pathlib.Path | None = None) -> TokenAuth | None The cached sign-in for `env`, as the auth that created it, or None. .. py:data:: MAX_LIMIT :value: 500 The largest dataset page the API serves; it lowers anything above this. .. py:class:: Fetch(env: str | ds_fetch_py_sdk.env.Environment | None = None, *, token: str | None = None, auth: httpx.Auth | None = None, language: str = 'en', timeout: float = 30.0, retries: int = 3, sql_tool: str = 'text_to_sql', transport: httpx.BaseTransport | None = None) Client for AI Commons Fetch: read-only SQL over your tenant data lake. Without `token` or `auth`, credentials come from `$AIC_TOKEN`; then `$AIC_CLIENT_ID` and `$AIC_CLIENT_SECRET`; then `$AIC_EMAIL` and `$AIC_PASSWORD`; then the cached session; then browser sign-in. with Fetch("dev") as fetch: result = fetch.query('SELECT count(*) FROM gold.""') .. py:method:: query(sql: str, *, fresh: bool = False) -> ds_fetch_py_sdk.models.QueryResult Run one SELECT. `fresh` reads S3 as it is now, bypassing the index and cache. .. py:method:: validate(sql: str, *, fresh: bool = False) -> ds_fetch_py_sdk.models.Validation Check one SELECT as `query` would, without reading data. Invalid SQL raises `QueryError`. .. py:method:: sql(question: str, *, datasets: collections.abc.Sequence[str] = ()) -> str Translate a natural-language question to SQL with ds-tools. .. py:method:: ask(question: str, *, datasets: collections.abc.Sequence[str] = (), fresh: bool = False) -> ds_fetch_py_sdk.models.QueryResult Answer a natural-language question: translate it to SQL, then run it. .. py:method:: datasets(*, names: collections.abc.Sequence[str] = (), layers: collections.abc.Sequence[Layer] = (), limit: int = 100, fresh: bool = False) -> collections.abc.Iterator[ds_fetch_py_sdk.models.Dataset] Every dataset and pipeline run you are granted, across pages; only `names` and `layers` if given. Names that do not exist or are not granted are left out. `limit` is the page size, held to 1…`MAX_LIMIT`; the server describes every dataset on a page within its timeout, so large pages risk a 504. .. py:method:: dataset(name: str, *, fresh: bool = False) -> ds_fetch_py_sdk.models.Dataset One dataset by name, e.g. `gold.` or `silver./`. .. py:class:: Tools(client: Fetch) ds-tools catalogue and invocation, under the same sign-in as Fetch. .. py:method:: invoke(tool_id: str, input: dict[str, Any], *, timeout_ms: int = 0) -> ds_fetch_py_sdk.models.ToolResult Run a tool. Retries reuse the same `tool_use_id`, so they are idempotent. .. py:class:: Environment Where the APIs and the identity server live. `resource` is the RFC 8707 resource id tokens are minted for. Local uses dev's identity server and resource, so a dev sign-in works against a locally running service. .. py:exception:: AICError Bases: :py:obj:`Exception` Base for every error raised by this SDK. .. py:exception:: APIError(status: int, code: str, message: str, *, request_id: str | None = None, retry_after: int = 0, recoverable: bool = False) Bases: :py:obj:`AICError` The API answered with an error. .. py:exception:: AuthError Bases: :py:obj:`AICError` Sign-in or token acquisition failed. .. py:exception:: NetworkError Bases: :py:obj:`AICError` The API could not be reached, or did not answer in time, after retries. .. py:exception:: QueryError(code: str, message: str, details: list[ds_fetch_py_sdk.models.FieldError], *, request_id: str | None) Bases: :py:obj:`APIError` The SQL was rejected (HTTP 422). Fix the query; retrying will not help. .. py:class:: QueryResult Result of one SELECT. Values are JSON-encoded per DuckDB type (see the API docs). .. py:attribute:: truncated :type: bool The 100,000-row or 8 MiB inline limit cut the result. .. py:attribute:: cache :type: str | None :value: None `HIT`, `MISS` or `SHARED`. .. py:method:: to_pandas() -> pandas.DataFrame Requires the `pandas` extra. .. py:class:: Validation SQL that would run, checked through binding without reading data. .. py:attribute:: sql :type: str The canonical SQL. .. py:attribute:: datasets :type: list[str] The datasets the SQL reads. .. py:attribute:: columns :type: list[Column] The result columns a query would return.