ds_provider_postgresql_py_lib¶

File: __init__.py Region: ds-provider-postgresql-py-lib

Description¶

A Python package from the ds-provider-postgresql-py-lib library.

Example

from ds_provider_postgresql_py_lib import __version__

print(f"Package version: {__version__}")

Submodules¶

Attributes¶

__version__

Classes¶

`PostgreSQLDataset`	Tabular dataset object which identifies data within a data store,
`PostgreSQLDatasetSettings`	Settings for PostgreSQL dataset operations.
`PostgreSQLLinkedService`	The class is used to connect with PostgreSQL database.
`PostgreSQLLinkedServiceSettings`	The object containing the PostgreSQL linked service settings.

Package Contents¶

class ds_provider_postgresql_py_lib.PostgreSQLDataset[source]¶

Bases: ds_resource_plugin_py_lib.common.resource.dataset.TabularDataset[PostgreSQLLinkedServiceType, PostgreSQLDatasetSettingsType, ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer, ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer], Generic[PostgreSQLLinkedServiceType, PostgreSQLDatasetSettingsType]

Tabular dataset object which identifies data within a data store, such as table/csv/json/parquet/parquetdataset/ and other documents.

The input of the dataset is a pandas DataFrame. The output of the dataset is a pandas DataFrame.

linked_service: PostgreSQLLinkedServiceType¶

settings: PostgreSQLDatasetSettingsType¶

serializer: ds_resource_plugin_py_lib.common.serde.serialize.PandasSerializer | None¶

deserializer: ds_resource_plugin_py_lib.common.serde.deserialize.PandasDeserializer | None¶

property type: ds_provider_postgresql_py_lib.enums.ResourceType¶

Get the type of the dataset.

Returns:: The dataset resource type.
Return type:: ResourceType

create(**_kwargs: Any) → None[source]¶

Create/write data to the configured table.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: CreateError – If writing data fails.

read(**_kwargs: Any) → None[source]¶

Read rows from the configured table into self.output.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: ReadError – If reading data fails.

delete(**_kwargs: Any) → None[source]¶

Delete rows matching configured identity columns.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: DeleteError – If deleting rows fails.

update(**_kwargs: Any) → None[source]¶

Update rows matching configured identity columns.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: UpdateError – If updating rows fails.

upsert(**_kwargs: Any) → None[source]¶

Insert or update rows using PostgreSQL ON CONFLICT semantics.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: UpsertError – If upserting rows fails.

purge(**_kwargs: Any) → None[source]¶

Purge table contents or drop the table.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: PurgeError – If purging table data fails.

list(**_kwargs: Any) → None[source]¶

List operation is not supported for this provider.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: NotSupportedError – Always, as list is not supported.

rename(**_kwargs: Any) → None[source]¶

Rename operation is not supported for this provider.

Parameters:: _kwargs – Additional keyword arguments for interface compatibility.
Returns:: None
Raises:: NotSupportedError – Always, as rename is not supported.

close() → None[source]¶

Close the dataset and underlying linked service.

Returns:: None

_output_from_empty_input() → pandas.DataFrame[source]¶

Build a consistent empty-operation output while preserving input schema.

Returns:: Empty dataframe or a schema-preserving input copy.
Return type:: pd.DataFrame

_get_table() → sqlalchemy.Table[source]¶

Get the reflected SQLAlchemy table for configured schema and table.

Returns:: Reflected table object.
Return type:: Table

_build_table_from_input(content: pandas.DataFrame) → sqlalchemy.Table[source]¶

Build a SQLAlchemy Table definition from input DataFrame dtypes.

Parameters:: content – Input DataFrame to build the table from.
Returns:: SQLAlchemy Table definition.
Return type:: Table

_resolve_create_primary_key_columns(content: pandas.DataFrame) → collections.abc.Sequence[str] | None[source]¶

Resolve and validate create-time primary key columns.

Parameters:: content – Input DataFrame used for table creation.
Returns:: Primary key columns for new table creation.
Return type:: Sequence[str] | None
Raises:: ValidationError – If primary_key is enabled but columns are invalid.

_copy_into_table(conn: Any, table: sqlalchemy.Table, content: pandas.DataFrame) → None[source]¶: Insert rows using PostgreSQL COPY.

_validate_columns(table: sqlalchemy.Table, column_names: collections.abc.Sequence[str]) → None[source]¶

Validate that all requested columns exist in the reflected table.

Parameters:

table – Reflected SQLAlchemy table.
column_names – Column names to validate.

Returns:

None

Raises:

ValidationError – If one or more columns do not exist in the table.

_validate_read_settings() → None[source]¶

Validate read settings before query construction.

Returns:: None
Raises:: ValidationError – If limit or order direction is invalid.

_build_select_columns(table: sqlalchemy.Table) → sqlalchemy.sql.Select[Any][source]¶

Build a SELECT statement for configured columns or all columns.

Parameters:: table – Reflected SQLAlchemy table.
Returns:: SELECT statement with chosen columns.
Return type:: Select[Any]
Raises:: ValidationError – If any selected column does not exist.

_build_filters(stmt: sqlalchemy.sql.Select[Any], table: sqlalchemy.Table) → sqlalchemy.sql.Select[Any][source]¶

Apply equality filters from read settings to the SELECT statement.

Parameters:

stmt – Current SELECT statement.
table – Reflected SQLAlchemy table.

Returns:

SELECT statement with WHERE conditions applied.

Return type:

Select[Any]

Raises:

ValidationError – If any filter column does not exist.

_build_order_by(stmt: sqlalchemy.sql.Select[Any], table: sqlalchemy.Table) → sqlalchemy.sql.Select[Any][source]¶

Apply ORDER BY clauses from read settings to the SELECT statement.

Parameters:

stmt – Current SELECT statement.
table – Reflected SQLAlchemy table.

Returns:

SELECT statement with ORDER BY applied.

Return type:

Select[Any]

Raises:

ValidationError – If any order-by column does not exist.

class ds_provider_postgresql_py_lib.PostgreSQLDatasetSettings[source]¶

Bases: ds_resource_plugin_py_lib.common.resource.dataset.DatasetSettings

Settings for PostgreSQL dataset operations.

The read settings contains read-specific configuration that only applies to the read() operation, not to create(), delete(), update(), etc.

schema: str = 'public'¶: Schema for dataset operations.

table: str¶: Table for dataset operations.

read: ReadSettings¶: Settings for read().

create: CreateSettings¶: Settings for create().

update: UpdateSettings | None = None¶: Settings for update().

upsert: UpsertSettings | None = None¶: Settings for upsert().

delete: DeleteSettings | None = None¶: Settings for delete().

purge: PurgeSettings¶: Settings for purge().

class ds_provider_postgresql_py_lib.PostgreSQLLinkedService[source]¶

Bases: ds_resource_plugin_py_lib.common.resource.linked_service.LinkedService[PostgreSQLLinkedServiceSettingsType], Generic[PostgreSQLLinkedServiceSettingsType]

The class is used to connect with PostgreSQL database.

settings: PostgreSQLLinkedServiceSettingsType¶

_engine: sqlalchemy.Engine | None = None¶: The SQLAlchemy engine instance with connection pool.

property type: ds_provider_postgresql_py_lib.enums.ResourceType¶

Get the type of the linked service.

Returns:: ResourceType

property engine: sqlalchemy.Engine | None¶

Get the SQLAlchemy engine instance.

Returns:: The engine if initialized, None otherwise.
Return type:: Engine | None

property connection: sqlalchemy.Engine¶

Get the established backend connection object.

Returns:: The initialized SQLAlchemy engine.
Return type:: Engine
Raises:: ConnectionError – If connect() has not been called successfully.

property pool: sqlalchemy.pool.Pool | None¶

Get the connection pool from the engine.

The pool is automatically created by SQLAlchemy when create_engine() is called with pool parameters. All connections (via engine.connect(), engine.begin(), etc.) automatically use this pool.

Returns:: The connection pool if the engine is initialized, None otherwise.
Return type:: Pool | None

connect() → None[source]¶

Connect to the PostgreSQL database and create a connection pool.

Returns:: None

test_connection() → tuple[bool, str][source]¶

Test the connection to the PostgreSQL database.

Returns:: A tuple containing a boolean indicating success and a string message.
Return type:: tuple[bool, str]

_is_authentication_error(error: Exception) → bool[source]¶

Detect authentication failures from SQLAlchemy/driver exceptions.

Parameters:: error – The exception to check.
Returns:: True if the error is an authentication error, False otherwise.
Return type:: bool

close() → None[source]¶: Close the linked service.

class ds_provider_postgresql_py_lib.PostgreSQLLinkedServiceSettings[source]¶

Bases: ds_resource_plugin_py_lib.common.resource.linked_service.LinkedServiceSettings

The object containing the PostgreSQL linked service settings.

uri: str¶

PostgreSQL connection URI.

Format: postgresql://[user[:password]@][host][:port][/database][?param1=value1&param2=value2]

Examples

postgresql://user:password@localhost:5432/mydb postgresql://user:password@localhost:5432/mydb?sslmode=require postgresql://user@localhost/mydb postgresql://localhost/mydb

pool_size: int = 5¶: The size of the connection pool. Defaults to 5.

max_overflow: int = 10¶: The maximum overflow connections allowed. Defaults to 10.

pool_timeout: int = 30¶: The timeout in seconds for getting a connection from the pool. Defaults to 30.

pool_recycle: int = 3600¶: The time in seconds after which a connection is recycled. Defaults to 3600.

ds_provider_postgresql_py_lib.__version__¶