You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-17 - 14:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Nov 17, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30428180,0sleep20-21swapper/206:33:542
304450780,0irq/25-eth10-21swapper/306:37:203
64324132,0sleep10-21swapper/106:37:451
46623524,0sleep00-21swapper/006:35:440
339970,0migration/39104-21kthreadcore09:43:083
7259960,0cyclictest29696-21rm09:57:220
339960,0migration/326721-21sh11:57:063
339960,0migration/318652-21sh11:08:513
339960,0migration/318549-21sh11:51:273
339960,0migration/314025-21hwlatdetect10:28:053
339960,0migration/311367-21sh10:44:253
339960,0migration/310638-21diskmemload11:03:423
29725060,0irq/27-eth0-tx-24997-21munin-node08:33:140
29715060,0irq/26-eth0-rx-15554-21df09:08:043
259960,0migration/225746-21timerandwakeup11:33:152
149960,0migration/07264-21sh09:02:550
149960,0migration/026031-21taskset11:55:030
149960,0migration/011001-21sh11:26:200
8215050,0irq/24-0000:00:29952-21cat09:57:522
8215050,0irq/24-0000:00:0-21swapper/208:07:522
7279955,0cyclictest0-21swapper/211:12:452
7279950,0cyclictest0-21swapper/209:12:472
7269955,0cyclictest31405-21diskmemload11:16:531
7269955,0cyclictest0-21swapper/111:12:471
7269955,0cyclictest0-21swapper/110:18:051
7269955,0cyclictest0-21swapper/110:13:121
7269955,0cyclictest0-21swapper/108:47:161
7269950,0cyclictest0-21swapper/111:52:121
7269950,0cyclictest0-21swapper/110:50:521
7269950,0cyclictest0-21swapper/110:50:521
7269950,0cyclictest0-21swapper/109:18:201
7259955,0cyclictest0-21swapper/008:51:460
7259950,0cyclictest0-21swapper/010:55:040
7259950,0cyclictest0-21swapper/009:15:440
4150,0ktimersoftd/00-21swapper/010:14:270
339950,0migration/39432-21munin-node11:23:113
339950,0migration/38889-21kthreadcore10:23:103
339950,0migration/37361-21kthreadcore10:03:083
339950,0migration/37050-21kthreadcore08:18:083
339950,0migration/35120-21kthreadcore11:43:073
339950,0migration/34786-21timerwakeupswit09:18:153
339950,0migration/33257-21needreboot09:58:113
339950,0migration/33095-21aten_r1power_po07:28:033
339950,0migration/329259-21fschecks_time07:23:063
339950,0migration/326833-21diskmemload11:35:503
339950,0migration/326783-21sh11:14:253
339950,0migration/322853-21aten_r1power_po09:33:043
339950,0migration/322521-21cpuspeed_turbos09:53:033
339950,0migration/321564-21latency_hist10:12:523
339950,0migration/321488-21taskset07:14:223
339950,0migration/319247-21if_eth1.208:28:053
339950,0migration/318001-21ksmtuned08:27:123
339950,0migration/317650-21kthreadcore08:48:093
339950,0migration/312706-21munin-node08:43:113
339950,0migration/312151-21latency09:23:093
339950,0migration/31052-21munin-node06:38:043
29725050,0irq/27-eth0-tx-3623-21ssh10:40:042
29725050,0irq/27-eth0-tx-31617-21ssh08:37:530
29725050,0irq/27-eth0-tx-27787-21munin-node11:38:072
29725050,0irq/27-eth0-tx-23158-21latency_hist08:52:522
29725050,0irq/27-eth0-tx-21190-21munin-node10:33:062
29725050,0irq/27-eth0-tx-1968-21munin-node07:03:050
29725050,0irq/27-eth0-tx-19520-21cpu11:53:042
29725050,0irq/27-eth0-tx-14769-21munin-node09:28:132
29725050,0irq/27-eth0-tx-13216-21munin-node10:28:052
29725050,0irq/27-eth0-tx-11896-21vmstat10:03:152
29725050,0irq/27-eth0-tx-11796-21munin-node11:28:092
29725050,0irq/27-eth0-tx-11403-21ssh11:27:312
29725050,0irq/27-eth0-tx-0-21swapper/210:52:562
29725050,0irq/27-eth0-tx-0-21swapper/210:51:002
29725050,0irq/27-eth0-tx-0-21swapper/210:51:002
29725050,0irq/27-eth0-tx-0-21swapper/210:24:182
29725050,0irq/27-eth0-tx-0-21swapper/210:02:342
29725050,0irq/27-eth0-tx-0-21swapper/209:18:042
29725050,0irq/27-eth0-tx-0-21swapper/208:58:592
29715050,0irq/26-eth0-rx-7472-21ssh09:03:043
29715050,0irq/26-eth0-rx-29795-21taskset08:53:503
29715050,0irq/26-eth0-rx-29528-21idleruntime10:58:063
29715050,0irq/26-eth0-rx-27786-21cut07:43:103
29715050,0irq/26-eth0-rx-2297-21munin-node07:03:043
29715050,0irq/26-eth0-rx-21754-21fschecks_time06:53:063
29715050,0irq/26-eth0-rx-20213-21ssh10:52:053
29715050,0irq/26-eth0-rx-20213-21ssh10:52:053
29715050,0irq/26-eth0-rx-17576-21python11:28:133
29715050,0irq/26-eth0-rx-15273-21kthreadcore10:08:063
29715050,0irq/26-eth0-rx-0-21swapper/310:38:133
29715050,0irq/26-eth0-rx-0-21swapper/310:18:073
29715050,0irq/26-eth0-rx-0-21swapper/309:51:093
29715050,0irq/26-eth0-rx-0-21swapper/309:38:133
29715050,0irq/26-eth0-rx-0-21swapper/309:28:143
29715050,0irq/26-eth0-rx-0-21swapper/308:58:043
29715050,0irq/26-eth0-rx-0-21swapper/308:40:383
27150,0ktimersoftd/20-21swapper/211:01:342
27150,0ktimersoftd/20-21swapper/210:14:272
259950,0migration/25237-21df_abs10:43:032
259950,0migration/23923-21ntp_offset09:38:132
259950,0migration/230645-21munin-node08:13:042
259950,0migration/221972-21idleruntime-cro09:52:522
259950,0migration/216014-21fschecks_time08:48:052
259950,0migration/215109-21kthreadcore10:08:062
259950,0migration/210577-21latency_hist12:07:512
249950,0watchdog/213957-21kthreadcore07:33:112
179950,0migration/13943-21vmstat09:58:161
179950,0migration/120545-21kthreadcore08:28:071
179950,0migration/119761-21missed_timers06:48:101
149950,0migration/09650-21sh11:43:410
149950,0migration/06640-21sh09:01:520
149950,0migration/06111-21irqrtprio10:43:060
149950,0migration/032639-21mdstat11:58:100
149950,0migration/032404-21kthreadcore09:58:090
149950,0migration/030979-21kthreadcore07:23:090
149950,0migration/030057-21kthreadcore11:38:090
149950,0migration/028998-21taskset06:58:070
149950,0migration/026846-21timerandwakeup06:53:150
149950,0migration/02598-21sh10:58:350
149950,0migration/025720-21timerandwakeup11:33:140
149950,0migration/020764-21kthreadcore07:38:100
149950,0migration/018046-21kthreadcore09:28:110
149950,0migration/01783-21kthreadcore09:18:090
149950,0migration/017769-21needreboot11:08:080
149950,0migration/015870-21kthreadcore07:58:110
149950,0migration/015533-21taskset07:13:040
149950,0migration/015076-21taskset08:47:430
149950,0migration/014891-21fschecks_count06:48:050
149950,0migration/013737-21kthreadcore08:23:090
149950,0migration/013716-21kthreadcore11:28:080
149950,0migration/010270-21kthreadcore06:43:080
8215040,0irq/24-0000:00:14159-21idleruntime-cro06:47:532
7279943,0cyclictest4848-21awk07:53:062
7279943,0cyclictest3556-21lldpd09:03:192
7279943,0cyclictest3556-21lldpd08:34:192
7279943,0cyclictest3556-21lldpd07:29:502
7279943,0cyclictest3514-21snmpd11:05:542
7279942,0cyclictest26654-21cut09:13:092
7269944,0cyclictest0-21swapper/109:44:301
7269943,0cyclictest6680-21awk10:03:061
7269943,0cyclictest4725-21awk11:23:061
7269943,0cyclictest3556-21lldpd11:44:511
7269943,0cyclictest3556-21lldpd09:04:491
7269943,0cyclictest3556-21lldpd07:54:201
7269943,0cyclictest3556-21lldpd07:44:201
7269943,0cyclictest3556-21lldpd07:34:491
7269943,0cyclictest3514-21snmpd08:01:531
7269943,0cyclictest32503-21awk08:38:061
7269943,0cyclictest31552-21awk09:38:071
7269943,0cyclictest30569-21awk07:48:071
7269943,0cyclictest2962-21awk07:03:071
7269943,0cyclictest28674-21awk06:58:061
7269943,0cyclictest18221-21awk08:03:081
7269943,0cyclictest17358-21awk07:38:081
7269943,0cyclictest16256-21awk08:48:071
7269943,0cyclictest16077-21awk09:08:061
7259944,0cyclictest0-21swapper/012:06:010
7259944,0cyclictest0-21swapper/010:19:160
7259944,0cyclictest0-21swapper/010:08:430
7259943,0cyclictest3241-21irqbalance11:05:100
7259943,0cyclictest3241-21irqbalance10:34:400
7259943,0cyclictest3241-21irqbalance09:35:110
7259943,0cyclictest3241-21irqbalance09:25:410
7259943,0cyclictest3241-21irqbalance08:18:110
7259943,0cyclictest1893-21awk06:38:080
7259943,0cyclictest15109-21awk09:48:060
339940,0migration/32858-21aten_r1power_en12:03:033
339940,0migration/327752-21diskmemload11:37:573
339940,0migration/327592-21df11:58:043
339940,0migration/323803-21kthreadcore07:18:103
29725040,0irq/27-eth0-tx-7137-21idleruntime09:43:062
29725040,0irq/27-eth0-tx-3393-21ssh10:18:152
29725040,0irq/27-eth0-tx-0-21swapper/211:59:412
29725040,0irq/27-eth0-tx-0-21swapper/211:48:282
29725040,0irq/27-eth0-tx-0-21swapper/211:18:062
29725040,0irq/27-eth0-tx-0-21swapper/211:13:392
29725040,0irq/27-eth0-tx-0-21swapper/209:34:352
29725040,0irq/27-eth0-tx-0-21swapper/209:23:082
29725040,0irq/27-eth0-tx-0-21swapper/208:43:032
29725040,0irq/27-eth0-tx-0-21swapper/208:39:152
29725040,0irq/27-eth0-tx-0-21swapper/008:28:010
29725040,0irq/27-eth0-tx-0-21swapper/008:03:060
29725040,0irq/27-eth0-tx-0-21swapper/007:53:110
29725040,0irq/27-eth0-tx-0-21swapper/007:43:080
29725040,0irq/27-eth0-tx-0-21swapper/007:28:140
29725040,0irq/27-eth0-tx-0-21swapper/007:08:050
29715040,0irq/26-eth0-rx-0-21swapper/311:19:043
29715040,0irq/26-eth0-rx-0-21swapper/310:33:033
29715040,0irq/26-eth0-rx-0-21swapper/309:13:033
29715040,0irq/26-eth0-rx-0-21swapper/308:02:123
29715040,0irq/26-eth0-rx-0-21swapper/307:53:043
29715040,0irq/26-eth0-rx-0-21swapper/306:48:093
259940,0migration/27522-21kthreadcore11:43:082
259940,0migration/25434-21fschecks_count08:18:052
259940,0migration/229904-21cpuspeed_turbos07:48:042
259940,0migration/228745-21kthreadcore08:08:102
259940,0migration/227316-21munin-node07:18:112
259940,0migration/22676-21runrttasks07:27:152
259940,0migration/224293-21irqrtprio07:43:092
259940,0migration/21891-21latency_hist07:02:532
259940,0migration/216394-21aten_r1power_cu07:38:042
259940,0migration/212746-21kthreadcore07:58:102
249940,0watchdog/221066-21munin-node06:53:072
179940,0migration/17517-21kthreadcore08:18:081
179940,0migration/16686-21latency_hist09:22:521
179940,0migration/1577-21kthreadcore11:38:101
179940,0migration/132342-21gltestperf08:58:051
179940,0migration/124998-21munin-node08:33:031
179940,0migration/121242-21aten_r1power_en10:33:031
179940,0migration/11855-21kthreadcore10:38:091
179940,0migration/11522-21munin-node11:18:121
179940,0migration/114210-21ksmtuned07:08:121
179940,0migration/113817-21kthreadcore11:28:081
179940,0migration/111903-21cpuspeed_turbos08:23:041
179940,0migration/11017-21kthreadcore07:23:101
169940,0watchdog/16336-21grep11:03:101
169940,0watchdog/15951-21gltestperf10:23:061
169940,0watchdog/131405-21diskmemload11:34:541
169940,0watchdog/131405-21diskmemload08:53:271
169940,0watchdog/126806-21tr07:18:111
169940,0watchdog/115157-21ssh09:48:061
169940,0watchdog/112053-21ssh10:46:301
169940,0watchdog/10-21swapper/111:58:101
169940,0watchdog/10-21swapper/110:58:061
169940,0watchdog/10-21swapper/110:32:381
169940,0watchdog/10-21swapper/110:08:021
169940,0watchdog/10-21swapper/108:13:111
169940,0watchdog/10-21swapper/106:53:071
149940,0migration/06903-21irqcore10:03:070
149940,0migration/02597-21sendmail_mailst10:38:120
149940,0migration/016990-21kthreadcore10:28:080
149940,0migration/010707-21missed_timers10:23:110
7279932,0cyclictest9589-21awk07:08:072
7279932,0cyclictest3556-21lldpd07:15:202
7279932,0cyclictest12677-21ps08:23:082
7269933,0cyclictest3556-21lldpd09:53:501
7269932,0cyclictest8713-21sensors07:28:131
7269932,0cyclictest8641-21awk06:43:061
7269932,0cyclictest6735-21sensors06:38:141
7269932,0cyclictest21056-21tr10:53:031
7269932,0cyclictest20753-21needreboot07:13:111
7269931,0cyclictest5983-21kthreadcore12:03:091
7259933,0cyclictest7415-21kworker/0:211:12:550
7259933,0cyclictest7415-21kworker/0:210:48:010
7259933,0cyclictest7415-21kworker/0:210:48:010
7259933,0cyclictest7415-21kworker/0:209:45:180
7259933,0cyclictest7415-21kworker/0:209:08:320
7259933,0cyclictest7415-21kworker/0:208:52:540
7259933,0cyclictest7415-21kworker/0:208:08:060
7259933,0cyclictest7415-21kworker/0:207:47:560
7259933,0cyclictest7415-21kworker/0:207:32:560
7259933,0cyclictest7415-21kworker/0:207:17:580
7259932,0cyclictest29198-21awk11:18:060
7259932,0cyclictest2832-21kthreadcore09:38:100
7259930,0cyclictest0-21swapper/011:48:080
7259930,0cyclictest0-21swapper/008:13:040
29715030,0irq/26-eth0-rx-0-21swapper/310:54:133
29715030,0irq/26-eth0-rx-0-21swapper/308:33:033
29715030,0irq/26-eth0-rx-0-21swapper/308:08:043
29715030,0irq/26-eth0-rx-0-21swapper/308:03:053
29715030,0irq/26-eth0-rx-0-21swapper/307:48:123
29715030,0irq/26-eth0-rx-0-21swapper/307:38:043
29715030,0irq/26-eth0-rx-0-21swapper/307:33:053
29715030,0irq/26-eth0-rx-0-21swapper/307:09:413
29715030,0irq/26-eth0-rx-0-21swapper/306:58:023
29715030,0irq/26-eth0-rx-0-21swapper/306:43:023
249930,0watchdog/20-21swapper/208:28:072
249930,0watchdog/20-21swapper/206:58:072
179930,0migration/115837-21irqcore09:28:091
169930,0watchdog/10-21swapper/111:53:141
169930,0watchdog/10-21swapper/109:32:541
169930,0watchdog/10-21swapper/109:13:031
169930,0watchdog/10-21swapper/108:10:311
7279921,0cyclictest0-21swapper/206:48:002
7279920,0cyclictest924-21cut06:37:532
29715020,0irq/26-eth0-rx-0-21swapper/308:13:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional