| Loop id | Source Location | Source Function | Level | Max Thread Time / Walltime gcc_5 (%) | Exclusive Coverage gcc_5 (%) | Inclusive Coverage gcc_5 (%) | Max Exclusive Time Over Threads gcc_5 (s) | Max Inclusive Time Over Threads gcc_5 (s) | Exclusive Time w.r.t. Wall Time gcc_5 (s) | Inclusive Time w.r.t. Wall Time gcc_5 (s) | Nb Threads gcc_5 | Vectorization Ratio (%) | Vector Length Use (%) | Speedup If No Scalar Integer | Speedup If FP Vectorized | Speedup If Fully Vectorized | Speedup If Perfect Load Balancing gcc_5 | Stride 0 | Stride 1 | Stride n | Stride Unknown | Stride Indirect | Array Access Efficiency |
|---|
| 1945 | libggml-cpu.so - quants.c:979-1000 [...] | ggml_vec_dot_q8_0_q8_0 | Innermost | 62.91 | 65.85 | 65.85 | 8.09 | 8.09 | 7.96 | 7.96 | 96 | NA | NA | NA | NA | NA | 1.04 | NA | NA | NA | NA | NA | 0.00 |
| 1943 | libggml-cpu.so - quants.c:908-1128 [...] | ggml_vec_dot_q8_0_q8_0 | Outermost | 3.23 | 2.67 | 68.56 | 0.42 | 8.40 | 0.32 | 8.29 | 96 | NA | NA | NA | NA | NA | 1.32 | NA | NA | NA | NA | NA | 0.00 |
| 59 | libggml-cpu.so - ggml-cpu.c:2879-2898 [...] | ggml_graph_compute_thread | Innermost | 0.58 | 0.24 | 0.24 | 0.08 | 0.08 | 0.03 | 0.03 | 93 | 0 | 55.26 | 1 | 1 | 1.29 | 2.59 | NA | NA | NA | NA | NA | 0.00 |
| 712 | libggml-cpu.so - vec.cpp:231-262 | ggml_vec_dot_f16 | Single | 0.78 | 0.20 | 0.20 | 0.10 | 0.10 | 0.02 | 0.02 | 32 | 100 | 100 | 1 | 1 | 1 | 1.42 | 0 | 0 | 0 | 16 | 0 | 50.00 |
| 1493 | libggml-cpu.so - ops.cpp:8816-8880 [...] | ggml_compute_forward_flash_attn_ext | InBetween | 0.74 | 0.19 | 0.33 | 0.09 | 0.17 | 0.02 | 0.04 | 32 | 6.63 | 41.74 | 2.96 | 1.06 | 1.29 | 1.44 | NA | NA | NA | NA | NA | 0.00 |
| 47 | libggml-cpu.so - ggml-cpu.c:1164-1198 [...] | ggml_compute_forward_mul_mat | InBetween | 0.35 | 0.15 | 0.20 | 0.05 | 0.05 | 0.02 | 0.02 | 92 | 0 | 53.87 | 1 | 1 | 1.71 | 2.41 | NA | NA | NA | NA | NA | 0.00 |
| 60 | libggml-cpu.so - ggml-cpu.c:1664-2898 [...] | ggml_graph_compute_thread | Outermost | 0.31 | 0.15 | 0.39 | 0.04 | 0.10 | 0.02 | 0.05 | 91 | 0 | 38.64 | 1 | 1 | 3.53 | 2.18 | NA | NA | NA | NA | NA | 0.00 |
| 1495 | libggml-cpu.so - vec.h:411-458 | ggml_compute_forward_flash_attn_ext | Innermost | 0.78 | 0.14 | 0.14 | 0.10 | 0.10 | 0.02 | 0.02 | 32 | 96.97 | 100 | 1.24 | 1 | 1 | 1.99 | 1 | 0 | 0 | 16 | 0 | 52.94 |
| 1925 | libggml-cpu.so - ggml-impl.h:354-404 [...] | quantize_row_q8_0 | Single | 0.31 | 0.10 | 0.10 | 0.04 | 0.04 | 0.01 | 0.01 | 86 | 32.41 | 43.85 | 1.07 | 1.01 | 1.53 | 2.95 | 1 | 2 | 0 | 1 | 0 | 87.50 |
| 726 | libggml-cpu.so - ops.cpp:6238-6245 | ggml_rope_cache_init(float, float, float const*, float*, long, float, float, float*, float, float) | Innermost | 0.19 | 0.05 | 0.05 | 0.03 | 0.03 | 0.01 | 0.01 | 69 | 0 | 34.25 | 1 | 2.03 | 1.66 | 3.11 | 0 | 0 | 0 | 1 | 0.75 | 31.25 |
| 1946 | libggml-cpu.so - quants.c:910-929 [...] | ggml_vec_dot_q8_0_q8_0 | Innermost | 0.08 | 0.05 | 0.05 | 0.01 | 0.01 | 0.01 | 0.01 | 83 | 72.5 | 74.38 | 1.17 | 1 | 1.19 | 1.57 | 0 | 2 | 0 | 0 | 0 | 100.00 |
| 53 | libggml-cpu.so - ggml-cpu.c:1193-1194 | ggml_compute_forward_mul_mat | Innermost | 0.19 | 0.05 | 0.05 | 0.03 | 0.03 | 0.01 | 0.01 | 60 | 0 | 47.92 | 1 | 1 | 1.73 | 2.91 | 1 | 0 | 0 | 0 | 0 | 100.00 |
| 725 | libggml-cpu.so - ops.cpp:6211-6231 [...] | rope_yarn(float, float, float*, long, float, float, float*, float*) | Single | 0.19 | 0.04 | 0.04 | 0.03 | 0.03 | 0.00 | 0.00 | 50 | 7.41 | 34.26 | 2.71 | 2.71 | 1.58 | 2.94 | NA | NA | NA | NA | NA | 0.00 |
| 51 | libggml-cpu.so - ggml-cpu.c:1125-1395 [...] | ggml_compute_forward_mul_mat | Innermost | 0.12 | 0.03 | 0.03 | 0.01 | 0.01 | 0.00 | 0.00 | 47 | 0 | 58.82 | 1 | 1 | 1 | 2.2 | NA | NA | NA | NA | NA | 0.00 |
| 714 | libggml-cpu.so - vec.cpp:385-387 [...] | ggml_vec_swiglu_f32 | Single | 2.33 | 0.03 | 0.03 | 0.30 | 0.30 | 0.00 | 0.00 | 2 | 91.43 | 97.32 | 1 | 1 | 1.03 | 1.97 | 0 | 0 | 0 | 3 | 0 | 50.00 |
| 49 | libggml-cpu.so - ggml-cpu.c:1132-1198 [...] | ggml_compute_forward_mul_mat | Innermost | 0.08 | 0.02 | 0.02 | 0.01 | 0.01 | 0.00 | 0.00 | 39 | 0 | 46.64 | 1 | 1 | 2 | 1.73 | NA | NA | NA | NA | NA | 0.00 |
| 55 | libggml-cpu.so - ggml-cpu.c:1290-1297 | ggml_compute_forward_mul_mat | InBetween | 0.12 | 0.02 | 0.02 | 0.02 | 0.02 | 0.00 | 0.00 | 28 | 0 | 54.59 | 1 | 1 | 1.98 | 2.4 | 1 | 0 | 0 | 1 | 0 | 75.00 |
| 4 | libggml-cpu.so - ggml-cpu.c:3228-3229 [...] | ggml_cpu_fp32_to_fp16 | Single | 0.23 | 0.01 | 0.01 | 0.03 | 0.03 | 0.00 | 0.00 | 13 | 82.05 | 93.27 | 1 | 1 | 1 | 3.71 | 0 | 0 | 0 | 3 | 0 | 50.00 |
| 362 | libggml-cpu.so - binary-ops.cpp:10-32 [...] | ggml_compute_forward_add_non_quantized | Innermost | 0.70 | 0.01 | 0.01 | 0.09 | 0.09 | 0.00 | 0.00 | 1 | 0 | 25 | 1 | 1 | 4 | 1 | 0 | 0 | 3 | 0 | 0 | 75.00 |
| 710 | libggml-cpu.so - vec.cpp:224-337 [...] | ggml_vec_dot_f16 | Outermost | 0.08 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 0.00 | 12 | 41.67 | 76.82 | 1.55 | 1 | 1.2 | 1.6 | NA | NA | NA | NA | NA | 0.00 |
| 440 | libggml-cpu.so - binary-ops.cpp:18-32 [...] | ggml_compute_forward_mul | Innermost | 0.54 | 0.01 | 0.01 | 0.07 | 0.07 | 0.00 | 0.00 | 1 | 0 | 25 | 1 | 1 | 4 | 1 | 0 | 0 | 3 | 0 | 0 | 75.00 |
| 742 | libggml-cpu.so - ops.cpp:6446-6457 | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | Innermost | 0.08 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 0.00 | 11 | 2.56 | 25.64 | 1 | 1 | 3.92 | 1.69 | 0 | 0 | 4 | 4 | 0 | 62.50 |
| 732 | libggml-cpu.so - ops.cpp:6389-6475 [...] | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | InBetween | 0.08 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 0.00 | 11 | 0 | 32.83 | 1.71 | 1.32 | 1.73 | 1.83 | NA | NA | NA | NA | NA | 0.00 |
| 50 | libggml-cpu.so - ggml-cpu.c:1125-1397 [...] | ggml_compute_forward_mul_mat | Outermost | 0.04 | 0.00 | 0.25 | 0.01 | 0.06 | 0.00 | 0.03 | 10 | NA | NA | NA | NA | NA | 1 | NA | NA | NA | NA | NA | 0.00 |
| 4227 | exec - sampling.cpp:125-126 | common_sampler_sample(common_sampler*, llama_context*, int, bool) | Innermost | 0.39 | 0.00 | 0.00 | 0.05 | 0.05 | 0.00 | 0.00 | 1 | 0 | 25 | 2.25 | 1 | 4 | 1 | 0 | 0 | 3 | 1 | 0 | 68.75 |
| 1490 | libggml-cpu.so - ops.cpp:8777-8938 [...] | ggml_compute_forward_flash_attn_ext | InBetween | 0.04 | 0.00 | 0.01 | 0.01 | 0.01 | 0.00 | 0.00 | 9 | 8.76 | 49.45 | 40.28 | 1.01 | 1.61 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 735 | libggml-cpu.so - ops.cpp:6389-6490 [...] | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | Outermost | 0.08 | 0.00 | 0.02 | 0.01 | 0.02 | 0.00 | 0.00 | 7 | 6.25 | 36.46 | 1.42 | 1.36 | 1.92 | 1.75 | NA | NA | NA | NA | NA | 0.00 |
| 1124 | libggml-cpu.so - ops.cpp:4325-4326 | ggml_compute_forward_rms_norm | Innermost | 0.27 | 0.00 | 0.00 | 0.04 | 0.04 | 0.00 | 0.00 | 1 | 100 | 82.5 | 1 | 1 | 1.16 | 1 | 0 | 0 | 0 | 1 | 0 | 50.00 |
| 15 | libggml-base.so - ggml.c:1367-1380 | ggml_is_contiguous_n | Innermost | 0.04 | 0.00 | 0.00 | 0.01 | 0.01 | 0.00 | 0.00 | 4 | 0 | 51.92 | 1 | 1 | 1 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 1499 | libggml-cpu.so - vec.h:646-653 | ggml_compute_forward_flash_attn_ext | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 100 | 100 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 2 | 0 | 50.00 |
| 733 | libggml-cpu.so - ops.cpp:6389-6407 [...] | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | Innermost | 0.04 | 0.00 | 0.00 | 0.01 | 0.01 | 0.00 | 0.00 | 4 | 0 | 44.94 | 5.21 | 1 | 1.64 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 1128 | libggml-cpu.so - vec.h:646-653 | ggml_compute_forward_rms_norm | Innermost | 0.16 | 0.00 | 0.00 | 0.02 | 0.02 | 0.00 | 0.00 | 1 | 100 | 100 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 2 | 0 | 50.00 |
| 86 | libggml-cpu.so - ggml-cpu.cpp:61-64 | ggml_backend_cpu_get_extra_buffer_types() | Single | 0.04 | 0.00 | 0.00 | 0.01 | 0.01 | 0.00 | 0.00 | 3 | 0 | 100 | 1 | 1 | 1 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3907 | libllama.so - basic_string.h:1077-3760 [...] | std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned char>, s... | Single | 0.08 | 0.00 | 0.00 | 0.01 | 0.01 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | 0.67 | 0 | 0 | 2 | 0 | 61.11 |
| 56 | libggml-cpu.so - ggml-cpu.c:1289-1297 | ggml_compute_forward_mul_mat | Outermost | 0.04 | 0.00 | 0.02 | 0.01 | 0.01 | 0.00 | 0.00 | 2 | 0 | 56.52 | 1 | 1 | 1.13 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 789 | libggml-cpu.so - ops.cpp:709-716 | ggml_compute_forward_dup | InBetween | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 0 | 48.18 | 1 | 1 | 1.62 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 1496 | libggml-cpu.so - vec.h:710-717 | ggml_compute_forward_flash_attn_ext | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 100 | 100 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 2 | 0 | 50.00 |
| 14 | libggml-base.so - ggml.c:1363-1380 | ggml_is_contiguous_n | Outermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 0 | 53.85 | 1 | 1 | 2 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3130 | libllama.so - stl_algo.h:1594-1595 [...] | llama_token_data_array_partial_sort_inplace(llama_token_data_array*, int) | Innermost | 0.08 | 0.00 | 0.00 | 0.01 | 0.01 | 0.00 | 0.00 | 1 | 0 | 30.83 | 2.17 | 1 | 3.06 | 1 | 1.5 | 0.5 | 0.5 | 0 | 0 | 93.75 |
| 3749 | libllama.so - hashtable.h:2627-2644 [...] | std::_Hashtable<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::pair<std::pai... | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 4 | 1 | 40.00 |
| 739 | libggml-cpu.so - ops.cpp:6429-6479 [...] | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | InBetween | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 32.08 | 1.66 | 1.27 | 1.98 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3569 | libllama.so - char_traits.h:381-381 [...] | llama_vocab::impl::load(llama_model_loader&, LLM_KV const&) | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 10.53 | 38.49 | 3.3 | 1 | 5.18 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 1880 | libllama.so - llama-kv-cells.h:125-129 [...] | llama_kv_cache::prepare(std::vector<llama_ubatch, std::allocator<llama_ubatch> > const&) | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 35 | 1 | 1 | 2.67 | 1 | 0 | 0 | 0 | 0 | 5 | 0.00 |
| 1917 | libllama.so - llama-kv-cache.cpp:487-511 [...] | llama_kv_cache::init_batch(llama_batch_allocr&, unsigned int, bool) | InBetween | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 17.22 | 54.72 | 6.48 | 1 | 1 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3495 | libllama.so - hashtable_policy.h:387-2058 [...] | llama_vocab::~llama_vocab() | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 2 | 0 | 50.00 |
| 3601 | libllama.so - llama-vocab.cpp:2160-2301 [...] | llama_vocab::impl::load(llama_model_loader&, LLM_KV const&) | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 13.77 | 39.67 | 3.04 | 1 | 3.65 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3580 | libllama.so - basic_string.h:218-1084 [...] | llama_vocab::impl::load(llama_model_loader&, LLM_KV const&) | Innermost | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 45.34 | 1 | 1 | 1.36 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3391 | libllama.so - basic_string.h:1077-3760 [...] | std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::alloca... | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | 0.67 | 0 | 0 | 2 | 0 | 61.11 |
| 22 | libggml-base.so - ggml.c:1444-1448 | ggml_are_same_shape | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 43.75 | 1 | 1 | 3 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 1125 | libggml-cpu.so - ops.cpp:4319-4343 [...] | ggml_compute_forward_rms_norm | InBetween | 0.04 | 0.00 | 0.01 | 0.00 | 0.06 | 0.00 | 0.00 | 1 | 10.42 | 58.98 | 5.82 | 1.04 | 1.39 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3392 | libllama.so - basic_string.h:1077-3760 [...] | std::_Hashtable<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::pair<std::pai... | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | NA | NA | NA | NA | NA | 0.00 |
| 3675 | libllama.so - hashtable_policy.h:387-2058 [...] | std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::alloca... | Single | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 0 | 50 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 1.5 | 0 | 50.00 |
| 3745 | libllama.so - hashtable.h:2627-2644 [...] | std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::alloca... | InBetween | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | NA | NA | 1 | NA | NA | 1 | NA | NA | NA | NA | NA | 0.00 |
