neosqlite.collection.temporary_table_aggregation.manager module

class neosqlite.collection.temporary_table_aggregation.manager.DeterministicTempTableManager(pipeline_id: str)[source]

Bases: object

Manager for deterministic temporary table names.

This class generates unique but deterministic temporary table names based on pipeline stages and a pipeline ID. It ensures that the same pipeline stage will always generate the same table name within the same pipeline execution, which is useful for caching and optimization purposes.

__init__(pipeline_id: str)[source]

Initialize the DeterministicTempTableManager with a pipeline ID for generating unique table names.

Parameters:

pipeline_id (str) – A unique identifier for the pipeline, used to ensure table names are deterministic and unique across different pipeline executions.

make_temp_table_name(stage: dict[str, Any], name_suffix: str = '') → str[source]

Generate a deterministic temporary table name based on the pipeline stage and pipeline ID.

This method creates a unique but deterministic name for a temporary table by: 1. Creating a canonical representation of the stage 2. Hashing the stage to create a short, unique suffix 3. Combining the pipeline ID, stage type, and hash to form a base name 4. Ensuring uniqueness by tracking name usage within the pipeline

Parameters:

  • stage (dict[str, Any]) – The pipeline stage dictionary used to generate the table name

  • name_suffix (str, optional) – An additional suffix to append to the table name. Defaults to “”.

Returns:

A deterministic temporary table name unique to this stage and

pipeline

Return type:

str

neosqlite.collection.temporary_table_aggregation.manager.aggregation_pipeline_context(db_connection, pipeline_id: str | None = None)[source]

Context manager for temporary aggregation tables with automatic cleanup.

This context manager provides a clean and safe way to work with temporary tables during aggregation pipeline processing. It handles:

  1. Creating a savepoint for atomicity of the entire pipeline

  2. Generating deterministic temporary table names

  3. Providing a function to create temporary tables with proper naming

  4. Automatic cleanup of all temporary tables and savepoint on exit

The context manager supports both new deterministic naming (using stage dictionaries) and backward compatibility (using string suffixes) for temporary tables.

Parameters:

  • db_connection – The database connection object

  • pipeline_id (str | None) – A unique identifier for the pipeline. If None, a default ID is generated for backward compatibility.

Yields:

Callable –

A function to create temporary tables with the signature:

create_temp_table(stage_or_suffix, query, params=None, name_suffix=””)

Where: - stage_or_suffix: Either a stage dict (new approach) or string

(backward compatibility)

  • query: The SQL query to populate the temporary table

  • params: Optional parameters for the SQL query

  • name_suffix: Optional suffix for backward compatibility naming

Raises:

Exception – Any exception that occurs during pipeline processing is re-raised after cleanup operations