hipscat_import.margin_cache.margin_cache_map_reduce

Module Contents

Functions

map_pixel_shards(partition_file, ...)	Creates margin cache shards from a source partition file.
_to_pixel_shard(data, margin_threshold, output_path, ...)	Do boundary checking for the cached partition and then output remaining data.
reduce_margin_shards(intermediate_directory, ...)	Reduce all partition pixel directories into a single file

map_pixel_shards(partition_file, input_storage_options, margin_pairs, margin_threshold, output_path, margin_order, ra_column, dec_column)

Creates margin cache shards from a source partition file.

_to_pixel_shard(data, margin_threshold, output_path, ra_column, dec_column)

Do boundary checking for the cached partition and then output remaining data.

reduce_margin_shards(intermediate_directory, output_path, output_storage_options, partition_order, partition_pixel, original_catalog_metadata, input_storage_options)

Reduce all partition pixel directories into a single file