Source symbols: scripts/infer_bench.py

Deterministic, value-free symbol metadata extracted from public source. Implementations and literal values are not copied into this page.

• Revision: master@92d63d4e8bb4df75c3b71618f31ddde2378b2bcd · 2026-07-04T18:18:25Z
• Declarations: 4
• Referenced calls: 48

Declared symbols

• weight_bytes — def at line 51; references sum, numel, element_size, parameters, get_device, reset_peak_memory_stats, synchronize, generate, perf_counter, next, append, max_memory_allocated, dict, encode, len, ArgumentParser, add_argument, parse_args, autodetect_device_type, compute_init
• bench_generate — def at line 55; references get_device, reset_peak_memory_stats, synchronize, generate, perf_counter, next, append, max_memory_allocated, dict, encode, len, ArgumentParser, add_argument, parse_args, autodetect_device_type, compute_init, load_model, Engine, min, print
• build_prompt — def at line 84; references encode, len, ArgumentParser, add_argument, parse_args, autodetect_device_type, compute_init, load_model, Engine, min, print, get_device_name, get_peak_bandwidth, get_peak_flops, get_device_properties, weight_bytes, sum, numel, parameters, kv_bytes_per_token
• main — def at line 97; references ArgumentParser, add_argument, parse_args, autodetect_device_type, compute_init, load_model, Engine, min, print, build_prompt, get_device_name, get_peak_bandwidth, get_peak_flops, get_device_properties, weight_bytes, sum, numel, parameters, kv_bytes_per_token, kv_read_bytes

File-level call references

• sum — line 53
• numel — line 53
• element_size — line 53
• parameters — line 53
• get_device — line 57
• reset_peak_memory_stats — line 58
• synchronize — line 59
• generate — line 61
• perf_counter — line 65
• next — line 66
• append — line 79
• max_memory_allocated — line 81
• dict — line 82
• encode — line 91
• len — line 92
• ArgumentParser — line 98
• add_argument — line 99
• parse_args — line 106
• autodetect_device_type — line 108
• compute_init — line 110
• load_model — line 113
• Engine — line 115
• min — line 119
• print — line 121
• build_prompt — line 122
• get_device_name — line 126
• get_peak_bandwidth — line 127
• get_peak_flops — line 128
• get_device_properties — line 129
• weight_bytes — line 130
• kv_bytes_per_token — line 132
• kv_read_bytes — line 134
• int — line 138
• str — line 147
• replace — line 147
• get — line 148
• join — line 149
• sorted — line 149
• items — line 149
• float — line 164
• estimate_decode_flops — line 173
• round — line 174
• bench_generate — line 185
• estimate_prefill_flops — line 188
• split — line 201
• dumps — line 241
• compute_cleanup — line 243
• main — line 246

Source: scripts/infer_bench.py