You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 16:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2s.osadl.org (updated Tue Aug 26, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350860,0irq/25-eth10-21swapper/306:39:493
3080726463,0sleep00-21swapper/006:38:290
3081324334,0sleep10-21swapper/106:38:341
3085523728,0sleep20-21swapper/206:39:032
8225070,0irq/24-0000:00:572-21cat10:27:522
8225060,0irq/24-0000:00:27756-21latency_hist07:52:522
339960,0migration/33837-21sh12:10:443
339960,0migration/330853-21sh11:07:133
339960,0migration/329227-21sh11:24:183
339960,0migration/319174-21kthreadcore08:33:103
339960,0migration/312647-21sh08:51:203
339960,0migration/311694-21sh11:53:263
339960,0migration/310751-21kthreadcore08:03:093
312579965,0cyclictest4754-21kworker/1:011:20:511
312579965,0cyclictest3032-21kworker/1:210:41:431
312579965,0cyclictest3032-21kworker/1:209:43:051
312579965,0cyclictest3032-21kworker/1:209:23:311
312579960,0cyclictest0-21swapper/111:16:401
312569960,0cyclictest0-21swapper/010:25:530
28415060,0irq/27-eth0-tx-19552-21runrttasks11:58:172
28415060,0irq/27-eth0-tx-0-21swapper/212:03:382
28415060,0irq/27-eth0-tx-0-21swapper/211:42:352
28415060,0irq/27-eth0-tx-0-21swapper/211:33:552
28415060,0irq/27-eth0-tx-0-21swapper/211:30:122
28415060,0irq/27-eth0-tx-0-21swapper/211:21:502
28415060,0irq/27-eth0-tx-0-21swapper/210:40:222
28415060,0irq/27-eth0-tx-0-21swapper/210:17:182
28415060,0irq/27-eth0-tx-0-21swapper/210:08:062
28415060,0irq/27-eth0-tx-0-21swapper/209:22:232
28415060,0irq/27-eth0-tx-0-21swapper/209:17:002
28415060,0irq/27-eth0-tx-0-21swapper/209:08:132
28415060,0irq/27-eth0-tx-0-21swapper/208:43:372
28415060,0irq/27-eth0-tx-0-21swapper/008:13:070
28415060,0irq/27-eth0-tx-0-21swapper/007:43:050
28405060,0irq/26-eth0-rx-18807-21memory11:58:103
28405060,0irq/26-eth0-rx-16148-21sh10:37:393
249960,0watchdog/20-21swapper/211:48:102
249960,0watchdog/20-21swapper/209:47:142
149960,0migration/03579-21sh12:09:570
149960,0migration/016041-21sh10:37:140
149960,0migration/013595-21timerandwakeup10:53:150
339950,0migration/37155-21cpuspeed_turbos11:13:043
339950,0migration/35350-21taskset06:45:253
339950,0migration/35141-21kthreadcore09:48:123
339950,0migration/34633-21taskset08:46:223
339950,0migration/3454-21munin-node07:08:153
339950,0migration/33315-21munin-node08:23:043
339950,0migration/332358-21latency_hist07:07:523
339950,0migration/327339-21kthreadcore10:43:113
339950,0migration/321432-21timerandwakeup10:58:163
339950,0migration/31962-21munin-node07:33:073
339950,0migration/31955-21needreboot08:18:103
339950,0migration/319478-21taskset09:14:373
339950,0migration/31934-21taskset07:57:103
339950,0migration/318361-21kthreadcore08:08:093
339950,0migration/315023-21munin-node07:43:093
339950,0migration/313863-21munin-node07:38:173
339950,0migration/311943-21taskset06:49:203
312639950,0cyclictest0-21swapper/209:02:532
312579955,0cyclictest4754-21kworker/1:012:08:071
312579955,0cyclictest4754-21kworker/1:012:03:251
312579955,0cyclictest4754-21kworker/1:011:58:111
312579955,0cyclictest4754-21kworker/1:011:56:191
312579955,0cyclictest4754-21kworker/1:011:47:511
312579955,0cyclictest4754-21kworker/1:011:43:491
312579955,0cyclictest4754-21kworker/1:011:38:051
312579955,0cyclictest4754-21kworker/1:011:33:291
312579955,0cyclictest4754-21kworker/1:011:29:131
312579955,0cyclictest4754-21kworker/1:011:23:171
312579955,0cyclictest3032-21kworker/1:211:08:111
312579955,0cyclictest3032-21kworker/1:211:03:151
312579955,0cyclictest3032-21kworker/1:211:00:211
312579955,0cyclictest3032-21kworker/1:210:54:071
312579955,0cyclictest3032-21kworker/1:210:50:371
312579955,0cyclictest3032-21kworker/1:210:35:431
312579955,0cyclictest3032-21kworker/1:210:30:231
312579955,0cyclictest3032-21kworker/1:210:24:391
312579955,0cyclictest3032-21kworker/1:210:18:371
312579955,0cyclictest3032-21kworker/1:210:14:151
312579955,0cyclictest3032-21kworker/1:210:07:551
312579955,0cyclictest3032-21kworker/1:210:03:451
312579955,0cyclictest3032-21kworker/1:209:58:191
312579955,0cyclictest3032-21kworker/1:209:48:211
312579955,0cyclictest3032-21kworker/1:209:38:091
312579955,0cyclictest3032-21kworker/1:209:33:011
312579955,0cyclictest3032-21kworker/1:209:28:191
312579955,0cyclictest3032-21kworker/1:209:18:091
312579955,0cyclictest3032-21kworker/1:209:16:371
312579955,0cyclictest3032-21kworker/1:209:08:511
312579955,0cyclictest3032-21kworker/1:209:05:591
312579955,0cyclictest3032-21kworker/1:208:57:531
312579955,0cyclictest3032-21kworker/1:208:53:111
312579955,0cyclictest3032-21kworker/1:208:49:591
312579955,0cyclictest3032-21kworker/1:208:44:591
312579955,0cyclictest3032-21kworker/1:208:38:141
312579955,0cyclictest3032-21kworker/1:208:35:561
312579955,0cyclictest3032-21kworker/1:208:28:581
312579955,0cyclictest3032-21kworker/1:208:23:381
312579955,0cyclictest3032-21kworker/1:208:17:561
312579955,0cyclictest3032-21kworker/1:208:17:481
312579955,0cyclictest3032-21kworker/1:208:08:321
312579955,0cyclictest3032-21kworker/1:208:02:581
312579955,0cyclictest3032-21kworker/1:207:58:241
312579955,0cyclictest24605-21kworker/1:107:55:061
312579955,0cyclictest24605-21kworker/1:107:48:201
312579955,0cyclictest24605-21kworker/1:107:44:401
312579955,0cyclictest24605-21kworker/1:107:38:081
312579955,0cyclictest24605-21kworker/1:107:33:081
312579955,0cyclictest24605-21kworker/1:107:28:581
312579955,0cyclictest24605-21kworker/1:107:23:341
312579955,0cyclictest24605-21kworker/1:107:18:561
312579955,0cyclictest24605-21kworker/1:107:04:001
312579955,0cyclictest24605-21kworker/1:106:59:241
312579955,0cyclictest24605-21kworker/1:106:53:121
312579955,0cyclictest24605-21kworker/1:106:49:401
312579955,0cyclictest24605-21kworker/1:106:43:221
312579954,0cyclictest3032-21kworker/1:209:54:491
312579950,0cyclictest0-21swapper/107:13:051
312569955,0cyclictest0-21swapper/011:29:250
312569955,0cyclictest0-21swapper/011:07:580
28415050,0irq/27-eth0-tx-9419-21gltestperf10:33:092
28415050,0irq/27-eth0-tx-5807-21munin-node06:48:080
28415050,0irq/27-eth0-tx-4041-21ssh09:28:032
28415050,0irq/27-eth0-tx-3428-21df09:48:082
28415050,0irq/27-eth0-tx-31469-21munin-node06:43:120
28415050,0irq/27-eth0-tx-2240-21munin-node07:58:080
28415050,0irq/27-eth0-tx-19723-21munin-node09:38:152
28415050,0irq/27-eth0-tx-17438-21munin-node10:18:072
28415050,0irq/27-eth0-tx-0-21swapper/212:08:292
28415050,0irq/27-eth0-tx-0-21swapper/211:25:332
28415050,0irq/27-eth0-tx-0-21swapper/210:49:232
28415050,0irq/27-eth0-tx-0-21swapper/210:03:062
28415050,0irq/27-eth0-tx-0-21swapper/210:01:142
28415050,0irq/27-eth0-tx-0-21swapper/208:50:292
28415050,0irq/27-eth0-tx-0-21swapper/008:03:120
28415050,0irq/27-eth0-tx-0-21swapper/007:13:040
28405050,0irq/26-eth0-rx-8431-21grep10:53:063
28405050,0irq/26-eth0-rx-6018-21ssh10:48:213
28405050,0irq/26-eth0-rx-32155-21kthreadcore11:28:093
28405050,0irq/26-eth0-rx-22010-21irqstats09:18:083
28405050,0irq/26-eth0-rx-18764-21ssh09:35:433
28405050,0irq/26-eth0-rx-15433-21ntp_kernel_pll_08:28:133
28405050,0irq/26-eth0-rx-0-21swapper/311:50:443
28405050,0irq/26-eth0-rx-0-21swapper/311:18:203
28405050,0irq/26-eth0-rx-0-21swapper/310:28:103
28405050,0irq/26-eth0-rx-0-21swapper/310:14:243
28405050,0irq/26-eth0-rx-0-21swapper/310:12:303
28405050,0irq/26-eth0-rx-0-21swapper/310:03:103
28405050,0irq/26-eth0-rx-0-21swapper/309:30:553
28405050,0irq/26-eth0-rx-0-21swapper/309:23:043
259950,0migration/28870-21kthreadcore11:53:112
259950,0migration/28371-21fschecks_time10:53:062
259950,0migration/26591-21munin-node07:33:112
259950,0migration/225196-21latency_hist10:22:522
259950,0migration/223346-21sh11:02:582
259950,0migration/223088-21irqcore11:43:122
259950,0migration/214602-21munin-node07:18:082
259950,0migration/214429-21munin-node08:03:122
249950,0watchdog/20-21swapper/211:13:062
249950,0watchdog/20-21swapper/211:10:142
249950,0watchdog/20-21swapper/210:59:502
249950,0watchdog/20-21swapper/210:42:542
249950,0watchdog/20-21swapper/209:53:342
249950,0watchdog/20-21swapper/209:34:462
249950,0watchdog/20-21swapper/209:24:462
249950,0watchdog/20-21swapper/208:59:582
249950,0watchdog/20-21swapper/208:53:102
249950,0watchdog/20-21swapper/208:36:462
249950,0watchdog/20-21swapper/208:01:182
249950,0watchdog/20-21swapper/207:25:062
249950,0watchdog/20-21swapper/207:03:102
249950,0watchdog/20-21swapper/206:51:222
169950,0watchdog/125372-21iostat_ios10:43:101
149950,0migration/0774-21taskset07:31:130
149950,0migration/07072-21kthreadcore08:23:100
149950,0migration/04824-21kthreadcore10:08:130
149950,0migration/030331-21kthreadcore07:03:090
149950,0migration/029893-21munin-node08:18:060
149950,0migration/029276-21switchtime11:03:160
149950,0migration/02885-21taskset09:25:070
149950,0migration/028739-21sh12:07:070
149950,0migration/028178-21kthreadcore10:43:120
149950,0migration/026952-21munin-node08:58:110
149950,0migration/024503-21kthreadcore07:23:090
149950,0migration/021201-21idleruntime-cro08:57:520
149950,0migration/020076-21kthreadcore10:38:110
149950,0migration/018696-21kthreadcore11:58:090
149950,0migration/018678-21processes09:13:150
149950,0migration/016239-21sh09:53:120
149950,0migration/014743-21latency_hist11:17:510
149950,0migration/012829-21ksmtuned11:34:480
149950,0migration/012553-21kthreadcore10:13:110
149950,0migration/012211-21kthreadcore07:38:120
149950,0migration/01187-21taskset07:32:580
149950,0migration/01123-21kthreadcore11:48:100
339940,0migration/327865-21latency_hist07:52:523
339940,0migration/324722-21kthreadcore08:58:093
339940,0migration/323368-21kthreadcore09:58:103
312639943,0cyclictest7720-21awk07:13:072
312639943,0cyclictest16628-21awk08:08:082
312639943,0cyclictest10875-21awk08:28:082
312579944,0cyclictest24605-21kworker/1:107:07:541
312569944,0cyclictest0-21swapper/011:24:030
312569944,0cyclictest0-21swapper/011:13:200
312569944,0cyclictest0-21swapper/010:59:370
312569944,0cyclictest0-21swapper/008:43:050
312569944,0cyclictest0-21swapper/008:11:240
312569943,0cyclictest685-21awk10:48:100
312569943,0cyclictest6445-21snmpd09:36:340
312569943,0cyclictest3425-21lldpd10:03:580
312569943,0cyclictest3425-21lldpd09:28:280
312569943,0cyclictest23075-21tail11:43:120
312569943,0cyclictest21665-21awk09:18:070
312569943,0cyclictest20058-21awk06:58:070
28415040,0irq/27-eth0-tx-0-21swapper/008:36:260
28415040,0irq/27-eth0-tx-0-21swapper/007:18:140
28405040,0irq/26-eth0-rx-6925-21uniq11:33:053
28405040,0irq/26-eth0-rx-32372-21munin-node11:08:103
28405040,0irq/26-eth0-rx-27076-21aten_r1power_vo07:28:053
28405040,0irq/26-eth0-rx-0-21swapper/312:03:013
28405040,0irq/26-eth0-rx-0-21swapper/311:44:033
28405040,0irq/26-eth0-rx-0-21swapper/311:38:403
28405040,0irq/26-eth0-rx-0-21swapper/310:38:093
28405040,0irq/26-eth0-rx-0-21swapper/310:24:043
28405040,0irq/26-eth0-rx-0-21swapper/309:53:063
28405040,0irq/26-eth0-rx-0-21swapper/309:43:153
28405040,0irq/26-eth0-rx-0-21swapper/309:40:203
28405040,0irq/26-eth0-rx-0-21swapper/309:08:283
28405040,0irq/26-eth0-rx-0-21swapper/308:53:553
28405040,0irq/26-eth0-rx-0-21swapper/307:58:033
259940,0migration/23905-21fschecks_time08:23:072
259940,0migration/230342-21kthreadcore08:18:082
259940,0migration/217366-21kthreadcore07:43:132
259940,0migration/213961-21kthreadcore06:53:122
249940,0watchdog/227758-21netstat08:13:142
249940,0watchdog/20-21swapper/208:38:102
249940,0watchdog/20-21swapper/207:56:222
249940,0watchdog/20-21swapper/207:38:102
249940,0watchdog/20-21swapper/207:28:062
249940,0watchdog/20-21swapper/207:08:182
249940,0watchdog/20-21swapper/207:02:422
249940,0watchdog/20-21swapper/206:44:032
149940,0migration/06519-21sh11:53:090
149940,0migration/03848-21gltestperf09:48:090
149940,0migration/028444-21kthreadcore08:38:140
149940,0migration/027999-21aten_r1power_cu07:53:030
149940,0migration/025612-21kthreadcore07:48:120
149940,0migration/018536-21uptime06:53:190
149940,0migration/01665-21kthreadcore09:03:150
149940,0migration/016508-21kthreadcore11:38:090
312569933,0cyclictest10972-21awk08:28:090
312569932,0cyclictest6477-21kthreadcore09:08:090
312569932,0cyclictest6445-21snmpd10:18:040
312569932,0cyclictest6349-21awk08:48:080
312569930,0cyclictest24051-21ntp_states09:58:120
312569930,0cyclictest1496-21sh09:43:470
28405030,0irq/26-eth0-rx-0-21swapper/310:17:573
28405030,0irq/26-eth0-rx-0-21swapper/309:02:563
28405030,0irq/26-eth0-rx-0-21swapper/308:38:373
28405030,0irq/26-eth0-rx-0-21swapper/308:13:073
28405030,0irq/26-eth0-rx-0-21swapper/307:18:043
28405030,0irq/26-eth0-rx-0-21swapper/307:15:343
28405030,0irq/26-eth0-rx-0-21swapper/306:59:113
28405030,0irq/26-eth0-rx-0-21swapper/306:53:063
312569921,0cyclictest584-21wc10:27:520
312569921,0cyclictest19456-21latency_hist09:37:520
312569921,0cyclictest0-21swapper/007:08:040
28405020,0irq/26-eth0-rx-0-21swapper/307:22:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional