Source symbols: src/transformers/modeling_flash_attention_utils.py
Deterministic, value-free symbol metadata extracted from public source. Implementations and literal values are not copied into this page.
- Revision: main@9a0fe3f5dd36ffe1888133f09eb03f1eb14b8a6e · 2026-07-20T12:47:48Z
- Declarations: 18
- Referenced calls: 64
Declared symbols
flash_attn_supports_top_left_mask— def at line 43; referencesis_flash_attn_2_available,is_flash_attn_3_available,is_flash_attn_4_available,is_npu_fa2_top_left_aligned_causal_mask,is_torch_npu_available,is_torch_xpu_available,is_rocm_platformis_flash_attn_available— def at line 53; referencesis_flash_attn_4_available,is_flash_attn_3_available,is_flash_attn_2_available,is_torch_npu_available,is_torch_xpu_available,is_rocm_platform_lazy_imports— def at line 143; referencesis_flash_attn_2_available,is_flash_attn_3_available,is_flash_attn_4_available,split_attention_implementation,is_torch_npu_available,get,load_and_register_attn_kernel,getattr,hasattr,ValueError,warning,signature_lazy_define_process_function— def at line 224; referencessignaturelazy_import_flash_attention— def at line 248; referencesValueError,_lazy_imports,_lazy_define_process_function,reshape,sum,nonzero,flatten,max,item,pad,cumsum,_index_first_axis,zeros,view,_get_unpad_data,arange,squeeze,unpad_input_funclazy_import_paged_flash_attention— def at line 275; referenceslazy_import_flash_attention,reshape,sum,nonzero,flatten,max,item,pad,cumsum,_index_first_axis,zeros,view,_get_unpad_data,arange,squeeze,unpad_input_func_index_first_axis— def at line 287; referencesreshape,sum,nonzero,flatten,max,item,pad,cumsum,zeros,view,_get_unpad_data,arange,squeeze,unpad_input_func_unpad_input— def at line 299; referencessum,nonzero,flatten,max,item,pad,cumsum,_index_first_axis,zeros,view,_get_unpad_data,arange,squeeze,unpad_input_func_pad_input— def at line 332; referenceszeros,view,sum,nonzero,flatten,max,item,pad,cumsum,_get_unpad_data,_index_first_axis,arange,squeeze,unpad_input_func_get_unpad_data— def at line 351; referencessum,nonzero,flatten,max,item,pad,cumsum,_index_first_axis,arange,squeeze,unpad_input_func_upad_input— def at line 379; references_get_unpad_data,_index_first_axis,arange,squeeze,unpad_input_funcprepare_fa_kwargs_from_position_ids— def at line 458_prepare_from_posids— def at line 496; referencescontiguous,view,size,prepare_fa_kwargs_from_position_ids,arange,min,and,abs,sum,bool,warning_once,not_is_packed_sequence— def at line 534; referencesarange,min,and,abs,sum,bool,warning_once,notfa_peft_integration_check— def at line 550; referenceswarning_once,notFlashAttentionKwargs— class at line 568; referencesnot_process_flash_attention_kwargs— def at line 589; referencesnot_flash_attention_forward— def at line 692; referenceslazy_import_flash_attention,fa_peft_integration_check,partial,size,_is_packed_sequence,all,_upad_input,str,clone,flash_varlen_fn,flash_kwargs,isinstance,pad_fn,_prepare_from_posids,reshape,view,flash_fn
File-level call references
import— line 24get_logger— line 39is_flash_attn_2_available— line 44is_flash_attn_3_available— line 44is_flash_attn_4_available— line 44is_npu_fa2_top_left_aligned_causal_mask— line 49is_torch_npu_available— line 58is_torch_xpu_available— line 59is_rocm_platform— line 68find_spec— line 83replace— line 84get— line 84split_attention_implementation— line 162load_and_register_attn_kernel— line 190getattr— line 194hasattr— line 201ValueError— line 204warning— line 209signature— line 234partial— line 245_lazy_imports— line 265_lazy_define_process_function— line 270lazy_import_flash_attention— line 281reshape— line 295sum— line 316nonzero— line 318flatten— line 318max— line 320item— line 320pad— line 321cumsum— line 321_index_first_axis— line 324zeros— line 346view— line 348_get_unpad_data— line 420arange— line 438squeeze— line 442unpad_input_func— line 446cat— line 479tensor— line 482size— line 482diff— line 490contiguous— line 525prepare_fa_kwargs_from_position_ids— line 529min— line 545and— line 547abs— line 547bool— line 547warning_once— line 563not— line 641getenv— line 657isinstance— line 678is_tracing— line 678fa_peft_integration_check— line 738_is_packed_sequence— line 764all— line 765_upad_input— line 771str— line 777clone— line 778flash_varlen_fn— line 780flash_kwargs— line 786pad_fn— line 791_prepare_from_posids— line 796flash_fn— line 824
Source: src/transformers/modeling_flash_attention_utils.py