Ctrl+K

hipscat_import.soap.arguments

hipscat_import.soap.arguments#

Module Contents#

Classes#

SoapArguments

Data class for holding source-object association arguments
class SoapArguments[source]#

Bases: hipscat_import.runtime_arguments.RuntimeArguments

Data class for holding source-object association arguments

object_catalog_dir: str = ''[source]#
object_id_column: str = ''[source]#
object_storage_options: Dict[Any, Any] | None[source]#

optional dictionary of abstract filesystem credentials for the OBJECT catalog.

source_catalog_dir: str = ''[source]#
source_object_id_column: str = ''[source]#
source_id_column: str = ''[source]#
source_storage_options: Dict[Any, Any] | None[source]#

optional dictionary of abstract filesystem credentials for the SOURCE catalog.

resume: bool = True[source]#

if there are existing intermediate resume files, should we read those and continue to run the pipeline where we left off

write_leaf_files: bool = False[source]#

Should we also write out leaf parquet files (e.g. Norder/Dir/Npix.parquet) that represent the full association table

compute_partition_size: int = 1000000000[source]#
__post_init__()[source]#
_check_arguments()[source]#
to_catalog_info(total_rows) → hipscat.catalog.association_catalog.association_catalog.AssociationCatalogInfo[source]#

Catalog-type-specific dataset info.

additional_runtime_provenance_info() → dict[source]#

Any additional runtime args to be included in provenance info from subclasses

Contents