Index

A

auto_select_profile() (dask_setup.config_manager.ConfigManager method)

B

benchmark_config() (in module dask_setup.benchmark)

BenchmarkResult (class in dask_setup.benchmark) best() (dask_setup.benchmark.ScalingResult method)

C

chunk_impact() (in module dask_setup.benchmark) chunk_sizes (dask_setup.benchmark.ChunkImpactResult property) chunk_specs (dask_setup.benchmark.ChunkImpactResult attribute) ChunkImpactResult (class in dask_setup.benchmark) ChunkRecommendation (class in dask_setup.xarray) cleanup() (dask_setup.multinode.SharedTempDir method) cluster_report() (in module dask_setup.reporting)

ClusterReport (class in dask_setup.reporting) ClusterSetupError (class in dask_setup.error_handling) compression (dask_setup.parquet.ParquetRecommendation attribute) ConfigManager (class in dask_setup.config_manager) ConfigurationValidationError (class in dask_setup.error_handling) configure_logging() (in module dask_setup.logging) create_profile_interactively() (dask_setup.config_manager.ConfigManager method) create_user_friendly_error() (in module dask_setup.error_handling)

D

dask_setup.exceptions module DaskClientContext (class in dask_setup.client) DaskSetupConfig (class in dask_setup.config) DaskSetupError delete_profile() (dask_setup.config_manager.ConfigManager method)

DependencyError (class in dask_setup.error_handling) detect_cluster_mode() (in module dask_setup.multinode) detect_format() (dask_setup.io_patterns.KerchunkOptimizer method) (dask_setup.io_patterns.NetCDFOptimizer method) (dask_setup.io_patterns.ZarrOptimizer method) (dask_setup.io_patterns.ZarrV3Optimizer method) detect_storage_format() (in module dask_setup.io_patterns)

E

efficiencies (dask_setup.benchmark.ScalingResult attribute) EnhancedDaskSetupError (class in dask_setup.error_handling) ensure_config_dir() (dask_setup.config_manager.ConfigManager method) ErrorContext (class in dask_setup.error_handling)

errors (dask_setup.benchmark.BenchmarkResult attribute) estimated_partition_mb (dask_setup.parquet.ParquetRecommendation attribute) extra (dask_setup.benchmark.BenchmarkResult attribute) (dask_setup.io_patterns.IORecommendation attribute) (dask_setup.parquet.ParquetRecommendation attribute)

F

format_exception_chain() (in module dask_setup.error_handling)

from_dict() (dask_setup.config.DaskSetupConfig class method)

G

generate_pbs_script() (in module dask_setup.multinode) generate_slurm_script() (in module dask_setup.multinode) get_diagnostic_info() (dask_setup.error_handling.EnhancedDaskSetupError method) get_environment_summary() (dask_setup.error_handling.ErrorContext method)

get_environment_type() (in module dask_setup.environment) get_logger() (in module dask_setup.logging) get_profile() (dask_setup.config_manager.ConfigManager method) get_profile_schema() (dask_setup.config_manager.ConfigManager static method) get_setup_client_kwargs() (dask_setup.config.DaskSetupConfig method)

I

import_profile_from_url() (dask_setup.config_manager.ConfigManager method) infer_workload_type() (in module dask_setup.workload) InsufficientResourcesError

InvalidConfigurationError IORecommendation (class in dask_setup.io_patterns) is_jupyter() (in module dask_setup.environment)

K

KerchunkOptimizer (class in dask_setup.io_patterns)

L

list_profiles() (dask_setup.config_manager.ConfigManager method)

load_profile_from_file() (dask_setup.config_manager.ConfigManager method)

M

memory_per_worker_gib (dask_setup.reporting.ClusterReport attribute) MemoryTuneResult (class in dask_setup.tune) merge_overrides() (dask_setup.config.DaskSetupConfig method)

merge_with() (dask_setup.config.DaskSetupConfig method) module dask_setup.exceptions MultiNodeConfig (class in dask_setup.multinode)

N

n_tasks (dask_setup.benchmark.BenchmarkResult attribute) n_workers (dask_setup.benchmark.BenchmarkResult attribute) name (dask_setup.benchmark.BenchmarkResult attribute)

NetCDFOptimizer (class in dask_setup.io_patterns) new_spill (dask_setup.tune.MemoryTuneResult attribute) new_target (dask_setup.tune.MemoryTuneResult attribute)

O

old_spill (dask_setup.tune.MemoryTuneResult attribute) old_target (dask_setup.tune.MemoryTuneResult attribute) optimal() (dask_setup.benchmark.ChunkImpactResult method) optimize_chunks() (dask_setup.io_patterns.KerchunkOptimizer method) (dask_setup.io_patterns.NetCDFOptimizer method) (dask_setup.io_patterns.ZarrOptimizer method) (dask_setup.io_patterns.ZarrV3Optimizer method)

optimize_compression() (dask_setup.io_patterns.KerchunkOptimizer method) (dask_setup.io_patterns.NetCDFOptimizer method) (dask_setup.io_patterns.ZarrOptimizer method) (dask_setup.io_patterns.ZarrV3Optimizer method) optimize_storage_options() (dask_setup.io_patterns.KerchunkOptimizer method) (dask_setup.io_patterns.NetCDFOptimizer method) (dask_setup.io_patterns.ZarrOptimizer method) (dask_setup.io_patterns.ZarrV3Optimizer method)

P

ParquetRecommendation (class in dask_setup.parquet) peak_memory_gib (dask_setup.benchmark.BenchmarkResult attribute) (dask_setup.reporting.ClusterReport property)

plot() (dask_setup.benchmark.ChunkImpactResult method) (dask_setup.benchmark.ScalingResult method)

R

rationale (dask_setup.tune.MemoryTuneResult attribute) rechunk_dataset() (in module dask_setup.rechunk) recommend_chunks() (in module dask_setup.xarray) recommend_io_chunks() (in module dask_setup.io_patterns) recommend_parquet_chunks() (in module dask_setup.parquet) recommended_chunks (dask_setup.benchmark.ChunkImpactResult attribute) register_worker_callbacks() (in module dask_setup.callbacks)

resolved_path (dask_setup.multinode.SharedTempDir attribute) ResourceConstraintError (class in dask_setup.error_handling) ResourceDetectionError results (dask_setup.benchmark.ChunkImpactResult attribute) (dask_setup.benchmark.ScalingResult attribute) rows_per_partition (dask_setup.parquet.ParquetRecommendation attribute) run_synthetic_benchmark() (in module dask_setup.benchmark)

S

save_profile() (dask_setup.config_manager.ConfigManager method) scaling_analysis() (in module dask_setup.benchmark) ScalingResult (class in dask_setup.benchmark) setup_dask_client() (in module dask_setup.client) setup_interactive_cluster() (in module dask_setup.multinode) setup_pbs_cluster() (in module dask_setup.multinode) setup_slurm_cluster() (in module dask_setup.multinode) SharedTempDir (class in dask_setup.multinode) speedups (dask_setup.benchmark.ScalingResult attribute)

spill_gib (dask_setup.benchmark.BenchmarkResult attribute) spill_gib_observed (dask_setup.tune.MemoryTuneResult attribute) storage_options (dask_setup.parquet.ParquetRecommendation attribute) StorageConfigurationError (class in dask_setup.error_handling) strategy (dask_setup.tune.MemoryTuneResult attribute) summary() (dask_setup.benchmark.ScalingResult method) (dask_setup.parquet.ParquetRecommendation method) (dask_setup.tune.MemoryTuneResult method) summary_line() (dask_setup.benchmark.BenchmarkResult method) (dask_setup.reporting.ClusterReport method)

T

tasks_per_second (dask_setup.benchmark.BenchmarkResult attribute) to_dataframe() (dask_setup.benchmark.ChunkImpactResult method) (dask_setup.benchmark.ScalingResult method) to_dict() (dask_setup.benchmark.BenchmarkResult method) (dask_setup.config.DaskSetupConfig method) (dask_setup.multinode.MultiNodeConfig method) (dask_setup.reporting.ClusterReport method)

total_cores_per_job (dask_setup.multinode.MultiNodeConfig property) total_mem_gb_per_job (dask_setup.multinode.MultiNodeConfig property) total_memory_gib (dask_setup.reporting.ClusterReport property) total_spill_gib (dask_setup.reporting.ClusterReport attribute) total_tasks (dask_setup.reporting.ClusterReport attribute) tune_memory_thresholds() (in module dask_setup.tune)

V

validate() (dask_setup.config.DaskSetupConfig method) validate_against_environment() (dask_setup.config.DaskSetupConfig method)

validate_chunks() (in module dask_setup.xarray) validate_profile() (dask_setup.config_manager.ConfigManager method)

W

wall_time_seconds (dask_setup.benchmark.BenchmarkResult attribute) (dask_setup.reporting.ClusterReport attribute) wall_time_std (dask_setup.benchmark.BenchmarkResult attribute) wall_time_str (dask_setup.reporting.ClusterReport property)

wall_times (dask_setup.benchmark.ScalingResult property) warnings (dask_setup.parquet.ParquetRecommendation attribute) worker_counts (dask_setup.benchmark.ScalingResult attribute) workers_updated (dask_setup.tune.MemoryTuneResult attribute)

Z

ZarrOptimizer (class in dask_setup.io_patterns)

ZarrV3Optimizer (class in dask_setup.io_patterns)