Source symbols: evals/old/hotpot_qa_24_2025/src/create_benchmark_summary_json.py

Deterministic, value-free symbol metadata extracted from public source. Implementations and literal values are not copied into this page.

• Revision: main@90b4acaac937dc1c0aeffaead8b707c896ebf3db · 2026-07-22T09:03:55Z
• Declarations: 22
• Referenced calls: 49

Declared symbols

• validate_csv_exists — def at line 15; references exists, print, endswith, read_csv, FileNotFoundError, read_summary_dataframe, validate_required_columns, ValueError, join, get_benchmark_analysis_path, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores
• read_summary_dataframe — def at line 29; references read_csv, print, validate_csv_exists, FileNotFoundError, validate_required_columns, ValueError, join, get_benchmark_analysis_path, exists, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores, round
• validate_required_columns — def at line 40; references print, validate_csv_exists, FileNotFoundError, read_summary_dataframe, ValueError, join, get_benchmark_analysis_path, exists, read_csv, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores, round
• load_cross_benchmark_data — def at line 60; references print, validate_csv_exists, FileNotFoundError, read_summary_dataframe, validate_required_columns, ValueError, join, get_benchmark_analysis_path, exists, read_csv, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores
• get_benchmark_analysis_path — def at line 79; references join, exists, FileNotFoundError, read_csv, print, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores, round, bootstrap_confidence_interval
• load_aggregate_metrics — def at line 85; references get_benchmark_analysis_path, join, exists, FileNotFoundError, read_csv, print, range, choice, len, append, mean, percentile, flatten, isnan, tolist, load_all_run_scores, round, bootstrap_confidence_interval
• bootstrap_confidence_interval — def at line 102; references range, choice, len, append, mean, percentile, get_benchmark_analysis_path, join, exists, FileNotFoundError, read_csv, flatten, isnan, print, tolist, load_all_run_scores, round
• load_all_run_scores — def at line 125; references get_benchmark_analysis_path, join, exists, FileNotFoundError, read_csv, flatten, isnan, print, tolist, round, mean, bootstrap_confidence_interval
• process_single_metric_with_bootstrap — def at line 149; references print, load_all_run_scores, round, mean, bootstrap_confidence_interval
• process_single_benchmark — def at line 179; references print
• extract_confidence_intervals — def at line 209; references print
• map_metric_names — def at line 229
• create_metric_entry — def at line 240; references map_metric_names, print
• format_benchmark_entry — def at line 249; references print
• validate_benchmark_folder — def at line 276; references get_benchmark_analysis_path, exists, print, join, append, load_cross_benchmark_data, iterrows, len, get
• handle_processing_errors — def at line 306; references print, join, load_cross_benchmark_data, iterrows, len, validate_benchmark_folder, get
• process_all_benchmarks — def at line 312; references print, join, load_cross_benchmark_data, iterrows, len, validate_benchmark_folder, get
• validate_output_data — def at line 392; references print, enumerate, keys, endswith
• format_json_output — def at line 422; references dumps, print
• create_output_directory — def at line 433; references dirname, exists, makedirs, print
• save_benchmark_summary_json — def at line 441; references print, validate_output_data, ValueError, create_output_directory, format_json_output, open, write, getsize
• main — def at line 470; references print, exists, load_cross_benchmark_data, len, iterrows, process_all_benchmarks, save_benchmark_summary_json

File-level call references

• exists — line 17
• print — line 18
• endswith — line 21
• read_csv — line 32
• validate_csv_exists — line 65
• FileNotFoundError — line 66
• read_summary_dataframe — line 69
• validate_required_columns — line 72
• ValueError — line 73
• join — line 81
• get_benchmark_analysis_path — line 87
• range — line 108
• choice — line 110
• len — line 110
• append — line 112
• mean — line 112
• percentile — line 119
• flatten — line 136
• isnan — line 138
• tolist — line 143
• load_all_run_scores — line 156
• round — line 160
• bootstrap_confidence_interval — line 167
• process_single_metric_with_bootstrap — line 197
• items — line 216
• get — line 237
• map_metric_names — line 244
• create_metric_entry — line 260
• load_cross_benchmark_data — line 318
• iterrows — line 328
• validate_benchmark_folder — line 339
• process_single_benchmark — line 354
• extract_confidence_intervals — line 361
• format_benchmark_entry — line 367
• handle_processing_errors — line 379
• enumerate — line 400
• keys — line 408
• dumps — line 425
• dirname — line 435
• makedirs — line 437
• validate_output_data — line 446
• create_output_directory — line 450
• format_json_output — line 453
• open — line 457
• write — line 458
• getsize — line 462
• process_all_benchmarks — line 521
• save_benchmark_summary_json — line 528
• main — line 539

Source: evals/old/hotpot_qa_24_2025/src/create_benchmark_summary_json.py