Source symbols: src/transformers/generation/continuous_batching/scheduler.py

Deterministic, value-free symbol metadata extracted from public source. Implementations and literal values are not copied into this page.

• Revision: main@9a0fe3f5dd36ffe1888133f09eb03f1eb14b8a6e · 2026-07-20T12:47:48Z
• Declarations: 23
• Referenced calls: 33

Declared symbols

• Scheduler — class at line 22; references Lock, ValueError, reset
• __init__ — def at line 29; references Lock, ValueError, reset
• reset — def at line 49
• add_waiting_request — def at line 61; references append
• schedule_batch — def at line 67
• has_pending_requests — def at line 77; references bool, len
• finish_request — def at line 81; references free_blocks, pop
• get_active_request_static_outputs — def at line 88
• set_request_cancellation — def at line 94; references add
• clear_cancelled_requests — def at line 99; references pop, append, remove, free_blocks, set
• request_is_cancelled — def at line 116
• _allocate_blocks_if_needed — def at line 122; references current_len, allocate_blocks, blocks_needed, append
• _infer_request_tokens — def at line 145; references search_prefix_match, add, min, len
• _schedule_request — def at line 170; references len, append, add
• _process_candidates — def at line 208; references get_num_free_blocks, debug, _infer_request_tokens, min, len, current_len, not, _allocate_blocks_if_needed, info, _schedule_request, append, FutureRequestState, pop, add, deque, popleft, super, __init__, values, extend
• _get_waiting_candidates — def at line 306; references deque, append, popleft, super, __init__, values, extend, set, _process_candidates, _cleanup_waiting_queue
• _cleanup_waiting_queue — def at line 323; references deque, super, __init__, values, append, extend, _get_waiting_candidates, set, _process_candidates
• FIFOScheduler — class at line 331; references super, __init__, values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue
• __init__ — def at line 335; references super, values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue
• schedule_batch — def at line 341; references values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue, super, __init__
• PrefillFirstScheduler — class at line 380; references super, __init__, values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue
• __init__ — def at line 385; references super, values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue
• schedule_batch — def at line 391; references values, append, extend, _get_waiting_candidates, set, _process_candidates, _cleanup_waiting_queue

File-level call references

• Lock — line 37
• ValueError — line 40
• reset — line 47
• deque — line 53
• set — line 54
• append — line 64
• bool — line 79
• len — line 79
• free_blocks — line 85
• pop — line 86
• add — line 97
• remove — line 111
• current_len — line 130
• allocate_blocks — line 134
• blocks_needed — line 139
• search_prefix_match — line 150
• min — line 158
• get_num_free_blocks — line 230
• debug — line 234
• _infer_request_tokens — line 240
• not — line 252
• _allocate_blocks_if_needed — line 256
• info — line 264
• _schedule_request — line 269
• FutureRequestState — line 290
• popleft — line 318
• super — line 339
• __init__ — line 339
• values — line 347
• extend — line 355
• _get_waiting_candidates — line 355
• _process_candidates — line 360
• _cleanup_waiting_queue — line 369

Source: src/transformers/generation/continuous_batching/scheduler.py