| Loop id | Source Location | Source Function | Level | Max Thread Time / Walltime 1x6 (%) | Max Thread Time / Walltime 1x72 (%) | Max Thread Time / Walltime 1x96 (%) | Max Thread Time / Walltime 1x120 (%) | Max Thread Time / Walltime 1x128 (%) | Max Thread Time / Walltime 1x144 (%) | Max Thread Time / Walltime 1x168 (%) | Max Thread Time / Walltime 1x192 (%) | Exclusive Coverage 1x6 (%) | Exclusive Coverage 1x72 (%) | Exclusive Coverage 1x96 (%) | Exclusive Coverage 1x120 (%) | Exclusive Coverage 1x128 (%) | Exclusive Coverage 1x144 (%) | Exclusive Coverage 1x168 (%) | Exclusive Coverage 1x192 (%) | Inclusive Coverage 1x6 (%) | Inclusive Coverage 1x72 (%) | Inclusive Coverage 1x96 (%) | Inclusive Coverage 1x120 (%) | Inclusive Coverage 1x128 (%) | Inclusive Coverage 1x144 (%) | Inclusive Coverage 1x168 (%) | Inclusive Coverage 1x192 (%) | Max Exclusive Time Over Threads 1x6 (s) | Max Exclusive Time Over Threads 1x72 (s) | Max Exclusive Time Over Threads 1x96 (s) | Max Exclusive Time Over Threads 1x120 (s) | Max Exclusive Time Over Threads 1x128 (s) | Max Exclusive Time Over Threads 1x144 (s) | Max Exclusive Time Over Threads 1x168 (s) | Max Exclusive Time Over Threads 1x192 (s) | Max Inclusive Time Over Threads 1x6 (s) | Max Inclusive Time Over Threads 1x72 (s) | Max Inclusive Time Over Threads 1x96 (s) | Max Inclusive Time Over Threads 1x120 (s) | Max Inclusive Time Over Threads 1x128 (s) | Max Inclusive Time Over Threads 1x144 (s) | Max Inclusive Time Over Threads 1x168 (s) | Max Inclusive Time Over Threads 1x192 (s) | Exclusive Time w.r.t. Wall Time 1x6 (s) | Exclusive Time w.r.t. Wall Time 1x72 (s) | Exclusive Time w.r.t. Wall Time 1x96 (s) | Exclusive Time w.r.t. Wall Time 1x120 (s) | Exclusive Time w.r.t. Wall Time 1x128 (s) | Exclusive Time w.r.t. Wall Time 1x144 (s) | Exclusive Time w.r.t. Wall Time 1x168 (s) | Exclusive Time w.r.t. Wall Time 1x192 (s) | Inclusive Time w.r.t. Wall Time 1x6 (s) | Inclusive Time w.r.t. Wall Time 1x72 (s) | Inclusive Time w.r.t. Wall Time 1x96 (s) | Inclusive Time w.r.t. Wall Time 1x120 (s) | Inclusive Time w.r.t. Wall Time 1x128 (s) | Inclusive Time w.r.t. Wall Time 1x144 (s) | Inclusive Time w.r.t. Wall Time 1x168 (s) | Inclusive Time w.r.t. Wall Time 1x192 (s) | Nb Threads 1x6 | Nb Threads 1x72 | Nb Threads 1x96 | Nb Threads 1x120 | Nb Threads 1x128 | Nb Threads 1x144 | Nb Threads 1x168 | Nb Threads 1x192 | GFLOPS 1x6 | GFLOPS 1x72 | GFLOPS 1x96 | GFLOPS 1x120 | GFLOPS 1x128 | GFLOPS 1x144 | GFLOPS 1x168 | GFLOPS 1x192 | Vectorization Ratio (%) | Vector Length Use (%) | Speedup If No Scalar Integer | Speedup If FP Vectorized | Speedup If Fully Vectorized | Speedup If Perfect Load Balancing 1x6 | Speedup If Perfect Load Balancing 1x72 | Speedup If Perfect Load Balancing 1x96 | Speedup If Perfect Load Balancing 1x120 | Speedup If Perfect Load Balancing 1x128 | Speedup If Perfect Load Balancing 1x144 | Speedup If Perfect Load Balancing 1x168 | Speedup If Perfect Load Balancing 1x192 | Stride 0 | Stride 1 | Stride n | Stride Unknown | Stride Indirect | Array Access Efficiency | (1x6) Efficiency | (1x6) Potential Speed-Up (%) | (1x72) Efficiency | (1x72) Potential Speed-Up (%) | (1x96) Efficiency | (1x96) Potential Speed-Up (%) | (1x120) Efficiency | (1x120) Potential Speed-Up (%) | (1x128) Efficiency | (1x128) Potential Speed-Up (%) | (1x144) Efficiency | (1x144) Potential Speed-Up (%) | (1x168) Efficiency | (1x168) Potential Speed-Up (%) | (1x192) Efficiency | (1x192) Potential Speed-Up (%) |
|---|
| 3017 | libggml-cpu.so - sgemm.cpp:144-464 [...] | void (anonymous namespace)::tinyBLAS<16, float __vector(16), float __vector(16), unsigned short, unsigned short, float>::gemm<4, 6, 4>(long, long, long) | Innermost | 2.40 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 3.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 3.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.30 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.30 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 191.86 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | NA | NA | NA | NA | NA | 1.21 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NA | NA | NA | NA | NA | 0.00 | 1 | 0 | | | | | | | | | | | | | | |
| 3018 | libggml-cpu.so - sgemm.cpp:144-464 [...] | void (anonymous namespace)::tinyBLAS<16, float __vector(16), float __vector(16), unsigned short, unsigned short, float>::gemm<4, 6, 4>(long, long, long) | InBetween | 0.96 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.15 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4.29 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.52 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.54 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.42 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.55 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 172.35 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 76.08 | 29.63 | 1.06 | 1.49 | 2.56 | 1.31 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 0 | 50.00 | 1 | 0 | | | | | | | | | | | | | | |
| 3325 | libggml-cpu.so - quants.c:298-355 [...] | quantize_row_q8_0 | Single | 3.26 | 3.51 | 3.64 | 3.69 | 3.51 | 3.68 | 3.24 | 3.27 | 0.85 | 0.08 | 0.06 | 0.05 | 0.04 | 0.04 | 0.03 | 0.03 | 0.85 | 0.08 | 0.06 | 0.05 | 0.04 | 0.04 | 0.03 | 0.03 | 1.76 | 1.74 | 1.74 | 1.75 | 1.71 | 1.79 | 1.62 | 1.26 | 1.76 | 1.74 | 1.74 | 1.75 | 1.71 | 1.79 | 1.62 | 1.26 | 0.31 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.31 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 17.39 | 220.32 | 291.82 | 363.08 | 397.86 | 428.10 | 551.69 | 813.27 | 60.7 | 29.66 | 1 | 1.31 | 2.68 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | 1.05 | -0 | 1.05 | -0 | 1.04 | -0 | 1.07 | -0 | 1.02 | -0 | 1.13 | -0 | 1.45 | 0 |
| 2078 | libggml-cpu.so - vec.h:89-89 | ggml_compute_forward_soft_max | Innermost | 0.62 | 0.20 | 0.22 | 0.18 | 0.17 | 0.20 | 0.25 | 0.18 | 0.76 | 0.11 | 0.16 | 0.14 | 0.13 | 0.14 | 0.17 | 0.11 | 0.76 | 0.11 | 0.16 | 0.14 | 0.13 | 0.14 | 0.17 | 0.11 | 0.34 | 0.10 | 0.11 | 0.09 | 0.08 | 0.10 | 0.13 | 0.07 | 0.34 | 0.10 | 0.11 | 0.09 | 0.08 | 0.10 | 0.13 | 0.07 | 0.27 | 0.03 | 0.05 | 0.04 | 0.04 | 0.04 | 0.05 | 0.02 | 0.27 | 0.03 | 0.05 | 0.04 | 0.04 | 0.04 | 0.05 | 0.02 | 6 | 72 | 96 | 120 | 128 | 144 | 168 | 190 | 2.34 | 18.53 | 10.29 | 13.12 | 13.75 | 10.95 | 8.69 | 19.65 | 100 | 50 | 1 | 1 | 2 | 1.29 | 2.93 | 2.22 | 2.13 | 2.22 | 2.37 | 2.37 | 2.92 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.66 | 0.04 | 0.36 | 0.1 | 0.34 | 0.09 | 0.33 | 0.08 | 0.27 | 0.1 | 0.18 | 0.14 | 0.36 | 0.07 |
| 2064 | libggml-cpu.so - vec.h:1444-1445 | ggml_compute_forward_soft_max | Innermost | 0.54 | 0.10 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.10 | 0.64 | 0.07 | 0.05 | 0.04 | 0.04 | 0.03 | 0.03 | 0.03 | 0.64 | 0.07 | 0.05 | 0.04 | 0.04 | 0.03 | 0.03 | 0.03 | 0.29 | 0.05 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.04 | 0.29 | 0.05 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.04 | 0.23 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.23 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 6 | 72 | 96 | 120 | 128 | 144 | 167 | 192 | 10.11 | 95.46 | 148.08 | 162.48 | 189.18 | 191.24 | 199.96 | 312.03 | 0 | 6.25 | 1 | 1 | 16 | 1.32 | 2.35 | 2.52 | 3.18 | 3.25 | 2.84 | 3.17 | 5.77 | 0 | 1 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.89 | 0.01 | 0.9 | 0.01 | 1.04 | -0 | 0.99 | 0 | 0.9 | 0 | 0.87 | 0 | 1.03 | -0 |
| 386 | libggml-cpu.so - mmq.cpp:303-1392 [...] | void parallel_for<(anonymous namespace)::convert_B_packed_format<block_q8_0, 32>(void*, block_q8_0 const*, int, int)::{lambda(int, int)#1}>(int, (anonymous namespace)::convert_B_packed_format<block_q8_0, 32>(void*, block_q8_0 const*, int,... | Innermost | 0.48 | 0.13 | 0.13 | 0.11 | 0.10 | 0.12 | 0.10 | 0.10 | 0.54 | 0.11 | 0.09 | 0.08 | 0.08 | 0.07 | 0.05 | 0.08 | 0.54 | 0.11 | 0.09 | 0.08 | 0.08 | 0.07 | 0.05 | 0.08 | 0.26 | 0.06 | 0.06 | 0.05 | 0.05 | 0.06 | 0.05 | 0.04 | 0.26 | 0.06 | 0.06 | 0.05 | 0.05 | 0.06 | 0.05 | 0.04 | 0.20 | 0.03 | 0.03 | 0.02 | 0.03 | 0.02 | 0.02 | 0.02 | 0.20 | 0.03 | 0.03 | 0.02 | 0.03 | 0.02 | 0.02 | 0.02 | 6 | 69 | 90 | 112 | 128 | 128 | 147 | 175 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 90.91 | 38.76 | 1.47 | 1 | 1.41 | 1.4 | 1.9 | 2.03 | 2.08 | 1.95 | 2.39 | 2.56 | 2.3 | 23 | 0 | 0 | 9 | 0 | 85.94 | 1 | 0 | 0.49 | 0.05 | 0.44 | 0.05 | 0.43 | 0.04 | 0.35 | 0.05 | 0.36 | 0.05 | 0.4 | 0.03 | 0.38 | 0.05 |
| 6 | libggml-cpu.so - ggml-cpu.c:3204-3207 | ggml_cpu_fp32_to_fp16 | Single | 0.50 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.51 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.51 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.27 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.27 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.18 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.18 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.10 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 100 | 100 | 1 | 1 | 1 | 1.54 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | | | | | | | | | | | | | | |
| 913 | libggml-cpu.so - binary-ops.cpp:18-32 [...] | ggml_compute_forward_mul | Innermost | 0.34 | 0.11 | 0.07 | 0.09 | 0.07 | 0.07 | 0.07 | 0.09 | 0.41 | 0.06 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.04 | 0.41 | 0.06 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.04 | 0.19 | 0.05 | 0.04 | 0.05 | 0.04 | 0.04 | 0.04 | 0.04 | 0.19 | 0.05 | 0.04 | 0.05 | 0.04 | 0.04 | 0.04 | 0.04 | 0.15 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.15 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 6 | 64 | 86 | 103 | 128 | 128 | 128 | 171 | 3.50 | 26.73 | 39.51 | 44.59 | 43.58 | 47.72 | 71.69 | 60.73 | 0 | 6.25 | 1 | 1.06 | 16 | 1.32 | 2.7 | 2.43 | 3.31 | 3.13 | 2.94 | 2.94 | 3.68 | 0 | 3 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.67 | 0.02 | 0.7 | 0.01 | 0.62 | 0.01 | 0.61 | 0.01 | 0.57 | 0.01 | 0.57 | 0.01 | 0.54 | 0.02 |
| 1303 | libggml-cpu.so - vec.h:1084-1101 [...] | ggml_vec_soft_max_f32 | Single | 0.44 | 0.08 | 0.07 | 0.07 | 0.05 | 0.05 | 0.06 | 0.05 | 0.40 | 0.04 | 0.03 | 0.03 | 0.02 | 0.02 | 0.02 | 0.02 | 0.40 | 0.04 | 0.03 | 0.03 | 0.02 | 0.02 | 0.02 | 0.02 | 0.24 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.03 | 0.02 | 0.24 | 0.04 | 0.04 | 0.04 | 0.03 | 0.03 | 0.03 | 0.02 | 0.14 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 0.14 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 6 | 72 | 96 | 120 | 128 | 144 | 168 | 192 | 148.14 | 1900.53 | 2186.13 | 2619.12 | 3127.13 | 3109.82 | 3231.27 | 5215.90 | 88.58 | 75.12 | 1.05 | 1.05 | 1.23 | 1.76 | 3.62 | 3.54 | 4.57 | 3.58 | 3.67 | 4.54 | 5.09 | 0.5 | 0 | 0 | 2 | 0 | 58.33 | 1 | 0 | 1.07 | -0 | 0.9 | 0 | 0.93 | 0 | 0.95 | 0 | 0.87 | 0 | 0.77 | 0 | 1.13 | -0 |
| 1299 | libggml-cpu.so - vec.h:1084-1115 [...] | ggml_vec_swiglu_f32 | Single | 0.31 | 0.08 | 0.09 | 0.12 | 0.14 | 0.08 | 0.10 | 0.09 | 0.37 | 0.04 | 0.05 | 0.04 | 0.07 | 0.04 | 0.04 | 0.06 | 0.37 | 0.04 | 0.05 | 0.04 | 0.07 | 0.04 | 0.04 | 0.06 | 0.17 | 0.04 | 0.05 | 0.05 | 0.07 | 0.04 | 0.05 | 0.04 | 0.17 | 0.04 | 0.05 | 0.05 | 0.07 | 0.04 | 0.05 | 0.04 | 0.13 | 0.01 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 0.13 | 0.01 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 6 | 64 | 86 | 103 | 128 | 128 | 128 | 171 | 143.26 | 1397.20 | 1362.90 | 1575.09 | 916.89 | 1461.19 | 1638.38 | 1410.76 | 98 | 98.13 | 1 | 1 | 1 | 1.33 | 2.67 | 2.97 | 4.06 | 3.35 | 2.81 | 3.3 | 2.28 | 0.5 | 0 | 0 | 3 | 0 | 56.25 | 1 | 0 | 0.83 | 0.01 | 0.61 | 0.02 | 0.57 | 0.02 | 0.3 | 0.05 | 0.44 | 0.02 | 0.41 | 0.02 | 0.3 | 0.05 |
| 815 | libggml-cpu.so - binary-ops.cpp:10-32 [...] | ggml_compute_forward_add_non_quantized | Innermost | 0.31 | 0.09 | 0.08 | 0.07 | 0.10 | 0.12 | 0.09 | 0.12 | 0.32 | 0.05 | 0.05 | 0.04 | 0.07 | 0.06 | 0.04 | 0.06 | 0.32 | 0.05 | 0.05 | 0.04 | 0.07 | 0.06 | 0.04 | 0.06 | 0.17 | 0.05 | 0.04 | 0.04 | 0.05 | 0.06 | 0.05 | 0.05 | 0.17 | 0.05 | 0.04 | 0.04 | 0.05 | 0.06 | 0.05 | 0.05 | 0.12 | 0.02 | 0.01 | 0.01 | 0.02 | 0.02 | 0.01 | 0.01 | 0.12 | 0.02 | 0.01 | 0.01 | 0.02 | 0.02 | 0.01 | 0.01 | 6 | 64 | 86 | 103 | 128 | 128 | 126 | 170 | 4.49 | 33.12 | 36.63 | 41.66 | 23.19 | 27.29 | 45.15 | 42.18 | 0 | 6.25 | 1 | 1.06 | 16 | 1.5 | 2.67 | 2.49 | 2.42 | 2.32 | 2.86 | 2.57 | 3.44 | 0 | 3 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.63 | 0.02 | 0.5 | 0.02 | 0.46 | 0.02 | 0.25 | 0.05 | 0.26 | 0.05 | 0.31 | 0.03 | 0.31 | 0.04 |
| 2075 | libggml-cpu.so - vec.h:677-682 | ggml_compute_forward_soft_max | Innermost | 0.29 | 0.07 | 0.09 | 0.11 | 0.08 | 0.07 | 0.10 | 0.08 | 0.32 | 0.05 | 0.06 | 0.05 | 0.04 | 0.04 | 0.05 | 0.03 | 0.32 | 0.05 | 0.06 | 0.05 | 0.04 | 0.04 | 0.05 | 0.03 | 0.16 | 0.04 | 0.04 | 0.05 | 0.04 | 0.04 | 0.05 | 0.03 | 0.16 | 0.04 | 0.04 | 0.05 | 0.04 | 0.04 | 0.05 | 0.03 | 0.12 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.12 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 6 | 70 | 94 | 117 | 117 | 129 | 152 | 152 | 3.96 | 42.90 | 28.20 | 40.18 | 36.62 | 42.30 | 36.92 | 104.15 | 100 | 100 | 1 | 1 | 1 | 1.42 | 2.29 | 2.59 | 3.45 | 2.93 | 2.75 | 3.06 | 3.91 | 0 | 1 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.63 | 0.02 | 0.42 | 0.03 | 0.4 | 0.03 | 0.43 | 0.02 | 0.42 | 0.02 | 0.27 | 0.03 | 0.59 | 0.01 |
| 1820 | libggml-cpu.so - ops.cpp:4325-4326 | ggml_compute_forward_rms_norm | Innermost | 0.25 | 0.10 | 0.10 | 0.11 | 0.10 | 0.08 | 0.08 | 0.10 | 0.29 | 0.08 | 0.07 | 0.07 | 0.07 | 0.05 | 0.05 | 0.06 | 0.29 | 0.08 | 0.07 | 0.07 | 0.07 | 0.05 | 0.05 | 0.06 | 0.13 | 0.05 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.04 | 0.13 | 0.05 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.04 | 0.10 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.01 | 0.10 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.01 | 6 | 72 | 96 | 120 | 128 | 144 | 168 | 192 | 10.12 | 41.87 | 48.21 | 48.16 | 49.77 | 61.48 | 59.67 | 64.81 | 100 | 31.25 | 1 | 1.38 | 2.93 | 1.36 | 2.14 | 2.41 | 2.44 | 2.34 | 2.4 | 2.6 | 2.94 | 0 | 1 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.37 | 0.05 | 0.31 | 0.05 | 0.25 | 0.05 | 0.23 | 0.05 | 0.26 | 0.04 | 0.24 | 0.04 | 0.24 | 0.05 |
| 2128 | libggml-cpu.so - ops.cpp:6446-6457 | ggml_compute_forward_rope_f32(ggml_compute_params const*, ggml_tensor*, bool) | Innermost | 0.22 | 0.05 | 0.05 | 0.08 | 0.06 | 0.06 | 0.09 | 0.06 | 0.20 | 0.02 | 0.03 | 0.02 | 0.03 | 0.03 | 0.02 | 0.02 | 0.20 | 0.02 | 0.03 | 0.02 | 0.03 | 0.03 | 0.02 | 0.02 | 0.12 | 0.02 | 0.03 | 0.04 | 0.03 | 0.03 | 0.05 | 0.03 | 0.12 | 0.02 | 0.03 | 0.04 | 0.03 | 0.03 | 0.05 | 0.03 | 0.07 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 0.07 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.00 | 6 | 72 | 96 | 120 | 128 | 143 | 163 | 191 | 13.15 | 142.12 | 107.00 | 130.34 | 114.91 | 117.83 | 126.10 | 227.50 | 0 | 6.25 | 1 | 1.5 | 16 | 1.78 | 3.83 | 3.06 | 5.71 | 3.67 | 3.88 | 5.62 | 6.2 | 0 | 2 | 0 | 2 | 0 | 75.00 | 1 | 0 | 0.89 | 0 | 0.54 | 0.01 | 0.5 | 0.01 | 0.4 | 0.02 | 0.38 | 0.02 | 0.32 | 0.02 | 0.55 | 0.01 |
| 2069 | libggml-cpu.so - ops.cpp:5915-5916 | ggml_compute_forward_soft_max | Innermost | 0.28 | 0.06 | 0.08 | 0.04 | 0.04 | 0.03 | 0.05 | 0.03 | 0.19 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.19 | 0.02 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.15 | 0.03 | 0.04 | 0.02 | 0.02 | 0.02 | 0.03 | 0.01 | 0.15 | 0.03 | 0.04 | 0.02 | 0.02 | 0.02 | 0.03 | 0.01 | 0.07 | 0.01 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.07 | 0.01 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 6 | 72 | 96 | 120 | 127 | 144 | 168 | 164 | 18.22 | 275.02 | 217.14 | 443.68 | 387.55 | 509.20 | 440.91 | 928.64 | 100 | 50 | 1 | 1.12 | 2 | 2.25 | 5.76 | 6.98 | 4.95 | 4.88 | 4.8 | 6.94 | 5.75 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | 1.11 | -0 | 0.75 | 0 | 0.86 | 0 | 0.8 | 0 | 0.92 | 0 | 0.69 | 0 | 1.46 | -0 |
| 1 | libggml-cpu.so - ggml-cpu.c:3215-3218 | ggml_cpu_fp32_to_fp16 | Single | 0.16 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.01 | 0.01 | 0.15 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.15 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.08 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.08 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 5 | 3 | 5 | 13 | 21 | 31 | 10 | 15 | 0.01 | 3.54 | 0.00 | 0.00 | 1.99 | 1.96 | 0.00 | 0.00 | 100 | 25 | 1.17 | 1 | 4 | 1.42 | 1.5 | 1 | 1 | 2.21 | 2.14 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | 31.1 | 0 | 12.45 | 0 | 4.79 | 0 | 3.28 | 0 | 2.15 | -0 | 6.24 | -0 | 4.16 | 0 |
| 3015 | libggml-cpu.so - sgemm.cpp:144-407 [...] | void (anonymous namespace)::tinyBLAS<16, float __vector(16), float __vector(16), unsigned short, unsigned short, float>::gemm<4, 6, 4>(long, long, long) | Innermost | 0.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.08 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.08 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 175.85 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | NA | NA | NA | NA | NA | 1.55 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NA | NA | NA | NA | NA | 0.00 | 1 | 0 | | | | | | | | | | | | | | |
| 712 | libggml-cpu.so - mmq.cpp:2068-2078 | ggml_backend_amx_mul_mat(ggml_compute_params const*, ggml_tensor*)::$_2::operator()(int, int) const::{lambda()#1}::operator()() const | Single | 0.12 | 0.07 | 0.08 | 0.07 | 0.10 | 0.08 | 0.12 | 0.06 | 0.14 | 0.05 | 0.05 | 0.04 | 0.05 | 0.04 | 0.04 | 0.04 | 0.14 | 0.05 | 0.05 | 0.04 | 0.05 | 0.04 | 0.04 | 0.04 | 0.06 | 0.04 | 0.04 | 0.04 | 0.05 | 0.04 | 0.06 | 0.03 | 0.06 | 0.04 | 0.04 | 0.04 | 0.05 | 0.04 | 0.06 | 0.03 | 0.05 | 0.01 | 0.02 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 0.05 | 0.01 | 0.02 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 6 | 70 | 91 | 112 | 124 | 137 | 150 | 158 | 9.76 | 26.41 | 19.03 | 28.57 | 24.70 | 24.20 | 36.90 | 39.04 | NA | NA | NA | NA | NA | 1.39 | 2.38 | 2.55 | 2.63 | 3.03 | 2.91 | 4.81 | 2.61 | NA | NA | NA | NA | NA | 0.00 | 1 | 0 | 0.28 | 0.03 | 0.2 | 0.04 | 0.2 | 0.03 | 0.14 | 0.04 | 0.15 | 0.04 | 0.16 | 0.03 | 0.19 | 0.03 |
| 4 | libggml-cpu.so - ggml-impl.h:346-404 [...] | ggml_cpu_fp32_to_fp16 | Single | 0.09 | 1.35 | 1.34 | 1.23 | 0.21 | 0.25 | 0.22 | 0.23 | 0.10 | 1.06 | 0.59 | 0.23 | 0.13 | 0.12 | 0.11 | 0.13 | 0.10 | 1.06 | 0.59 | 0.23 | 0.13 | 0.12 | 0.11 | 0.13 | 0.05 | 0.67 | 0.64 | 0.58 | 0.10 | 0.12 | 0.11 | 0.09 | 0.05 | 0.67 | 0.64 | 0.58 | 0.10 | 0.12 | 0.11 | 0.09 | 0.04 | 0.34 | 0.18 | 0.07 | 0.04 | 0.04 | 0.03 | 0.03 | 0.04 | 0.34 | 0.18 | 0.07 | 0.04 | 0.04 | 0.03 | 0.03 | 6 | 72 | 96 | 120 | 128 | 137 | 142 | 155 | 2.60 | 0.84 | 1.14 | 1.42 | 2.09 | 2.67 | 1.14 | 1.86 | 9.09 | 8.52 | 1.46 | 3.75 | 13.88 | 1.46 | 2.01 | 3.66 | 8.76 | 2.48 | 3.02 | 2.69 | 2.73 | 0 | 2 | 0 | 0 | 0 | 100.00 | 1 | 0 | 0.01 | 1.05 | 0.01 | 0.58 | 0.03 | 0.22 | 0.04 | 0.13 | 0.04 | 0.12 | 0.04 | 0.11 | 0.04 | 0.12 |
| 97 | libggml-cpu.so - ggml-cpu.c:1291-1297 | ggml_compute_forward_mul_mat | Innermost | 0.01 | 1.37 | 1.81 | 1.70 | 1.59 | 1.61 | 1.69 | 1.59 | 0.01 | 0.31 | 1.01 | 1.34 | 1.40 | 1.19 | 1.02 | 1.16 | 0.01 | 0.31 | 1.01 | 1.34 | 1.40 | 1.19 | 1.02 | 1.16 | 0.01 | 0.68 | 0.87 | 0.81 | 0.78 | 0.78 | 0.85 | 0.61 | 0.01 | 0.68 | 0.87 | 0.81 | 0.78 | 0.78 | 0.85 | 0.61 | 0.00 | 0.10 | 0.31 | 0.40 | 0.43 | 0.37 | 0.33 | 0.25 | 0.00 | 0.10 | 0.31 | 0.40 | 0.43 | 0.37 | 0.33 | 0.25 | 6 | 72 | 96 | 120 | 128 | 142 | 164 | 187 | 3.23 | 2.06 | 0.93 | 0.99 | 0.94 | 1.16 | 1.08 | 1.74 | 0 | 11.38 | 1 | 1 | 2.46 | 2 | 6.99 | 2.88 | 2.04 | 1.81 | 2.14 | 2.56 | 2.46 | NA | NA | NA | NA | NA | 0.00 | 1 | 0 | 0 | 0.31 | 0 | 1.01 | 0 | 1.34 | 0 | 1.4 | 0 | 1.19 | 0 | 1.02 | 0 | 1.16 |
| 3066 | libggml-cpu.so - sgemm.cpp:144-464 [...] | void (anonymous namespace)::tinyBLAS<16, float __vector(16), float __vector(16), unsigned short, unsigned short, float>::gemm<4, 6, 2>(long, long, long) | InBetween | 0.00 | 0.14 | 0.14 | 0.13 | 0.09 | 0.09 | 0.09 | 0.13 | 0.00 | 0.11 | 0.08 | 0.07 | 0.06 | 0.06 | 0.05 | 0.06 | 0.00 | 1.13 | 0.94 | 0.70 | 0.57 | 0.53 | 0.51 | 0.39 | 0.00 | 0.07 | 0.06 | 0.06 | 0.05 | 0.04 | 0.05 | 0.05 | 0.00 | 0.48 | 0.42 | 0.33 | 0.28 | 0.26 | 0.28 | 0.14 | 0.00 | 0.03 | 0.03 | 0.02 | 0.02 | 0.02 | 0.02 | 0.01 | 0.00 | 0.36 | 0.29 | 0.21 | 0.18 | 0.16 | 0.16 | 0.08 | 0 | 72 | 96 | 120 | 128 | 144 | 168 | 192 | 0.00 | 2223.48 | 3052.76 | 3492.72 | 3849.60 | 4319.66 | 4560.01 | 5584.75 | 76.38 | 29.7 | 1.06 | 1.49 | 2.53 | 0 | 2.1 | 2.64 | 2.86 | 2.38 | 2.65 | 2.81 | 3.86 | 0 | 0 | 0 | 2 | 0 | 50.00 | | | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 |
| 3065 | libggml-cpu.so - sgemm.cpp:144-464 [...] | void (anonymous namespace)::tinyBLAS<16, float __vector(16), float __vector(16), unsigned short, unsigned short, float>::gemm<4, 6, 2>(long, long, long) | Innermost | 0.00 | 0.92 | 0.79 | 0.64 | 0.50 | 0.46 | 0.48 | 0.31 | 0.00 | 1.02 | 0.86 | 0.63 | 0.51 | 0.47 | 0.46 | 0.33 | 0.00 | 1.02 | 0.86 | 0.63 | 0.51 | 0.47 | 0.46 | 0.33 | 0.00 | 0.46 | 0.38 | 0.30 | 0.25 | 0.23 | 0.24 | 0.12 | 0.00 | 0.46 | 0.38 | 0.30 | 0.25 | 0.23 | 0.24 | 0.12 | 0.00 | 0.32 | 0.26 | 0.19 | 0.16 | 0.14 | 0.15 | 0.07 | 0.00 | 0.32 | 0.26 | 0.19 | 0.16 | 0.14 | 0.15 | 0.07 | 0 | 72 | 96 | 120 | 128 | 144 | 168 | 192 | 0.00 | 661.20 | 817.94 | 1142.07 | 1363.30 | 1482.56 | 1453.18 | 3101.69 | NA | NA | NA | NA | NA | 0 | 1.43 | 1.48 | 1.64 | 1.57 | 1.57 | 1.65 | 1.74 | NA | NA | NA | NA | NA | 0.00 | | | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 |
