You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 01:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Thu Jun 04, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
630224233,0sleep20-21swapper/218:51:502
640723524,0sleep00-21swapper/018:53:040
606623525,0sleep30-21swapper/318:49:153
619123324,0sleep10-21swapper/118:50:331
65419965,0cyclictest0-21swapper/222:44:002
65419960,0cyclictest29811-21diskmemload23:09:132
65419960,0cyclictest0-21swapper/200:14:012
65409966,0cyclictest0-21swapper/123:23:581
65399966,0cyclictest0-21swapper/022:33:590
4160,0ktimersoftd/00-21swapper/023:28:420
339960,0migration/330599-21sh22:56:563
339960,0migration/322911-21sh22:52:513
339960,0migration/314114-21sh23:47:473
29725060,0irq/27-eth0-tx-716-21kthreadcore20:53:570
29725060,0irq/27-eth0-tx-0-21swapper/223:50:032
29715060,0irq/26-eth0-rx-25297-21interrupts21:33:583
29715060,0irq/26-eth0-rx-11222-21kthreadcore21:44:003
29715060,0irq/26-eth0-rx-0-21swapper/321:54:523
149960,0migration/028322-21timerandwakeup00:14:040
149960,0migration/024301-21sh21:33:540
149960,0migration/014563-21runrttasks22:46:520
8215050,0irq/24-0000:00:7810-21sadc20:23:432
8215050,0irq/24-0000:00:21210-21munin-plugin-st19:13:432
65419955,0cyclictest0-21swapper/222:16:532
65419950,0cyclictest0-21swapper/221:53:042
65419950,0cyclictest0-21swapper/221:12:432
65409955,0cyclictest13003-21sensors23:04:031
65409954,0cyclictest24161-21latency_hist21:13:431
65409950,0cyclictest0-21swapper/121:07:071
65399950,0cyclictest0-21swapper/022:29:150
339950,0migration/3376-21fschecks_time21:58:563
339950,0migration/330657-21kthreadcore19:23:553
339950,0migration/329801-21timerandwakeup22:14:043
339950,0migration/329603-21timerandwakeup22:34:053
339950,0migration/328607-21kthreadcore20:03:553
339950,0migration/323677-21kthreadcore19:13:553
339950,0migration/322562-21latency_hist23:53:413
339950,0migration/320828-21taskset19:09:283
339950,0migration/318775-21munin-node19:53:513
339950,0migration/318246-21kthreadcore19:03:563
339950,0migration/310633-21kthreadcore00:03:593
29725050,0irq/27-eth0-tx-9373-21irqstats21:43:592
29725050,0irq/27-eth0-tx-8529-21ps23:03:572
29725050,0irq/27-eth0-tx-4230-21ssh20:55:132
29725050,0irq/27-eth0-tx-28900-21kthreadcore21:54:002
29725050,0irq/27-eth0-tx-21509-21switchtime22:29:042
29725050,0irq/27-eth0-tx-0-21swapper/222:54:242
29725050,0irq/27-eth0-tx-0-21swapper/222:50:412
29725050,0irq/27-eth0-tx-0-21swapper/221:22:182
29715050,0irq/26-eth0-rx-7693-21munin-node22:43:543
29715050,0irq/26-eth0-rx-6675-21crond23:23:413
29715050,0irq/26-eth0-rx-6307-40yum-updatesd-he22:41:223
29715050,0irq/26-eth0-rx-5344-21smartctl22:19:023
29715050,0irq/26-eth0-rx-4477-21ssh00:19:543
29715050,0irq/26-eth0-rx-28900-21port_sshpriv23:14:013
29715050,0irq/26-eth0-rx-25851-21kthreadcore21:13:583
29715050,0irq/26-eth0-rx-22999-21cpuspeed_turbos23:53:543
29715050,0irq/26-eth0-rx-22263-21ssh22:09:463
29715050,0irq/26-eth0-rx-21753-21taskset23:10:163
29715050,0irq/26-eth0-rx-12040-21munin-node18:54:013
29715050,0irq/26-eth0-rx-11878-21hwlatdetect19:43:523
29715050,0irq/26-eth0-rx-10375-21kthreadcore23:03:583
29715050,0irq/26-eth0-rx-0-21swapper/323:40:123
29715050,0irq/26-eth0-rx-0-21swapper/323:33:583
29715050,0irq/26-eth0-rx-0-21swapper/322:59:203
29715050,0irq/26-eth0-rx-0-21swapper/322:29:093
29715050,0irq/26-eth0-rx-0-21swapper/322:24:183
29715050,0irq/26-eth0-rx-0-21swapper/322:03:513
29715050,0irq/26-eth0-rx-0-21swapper/321:53:323
29715050,0irq/26-eth0-rx-0-21swapper/321:38:473
29715050,0irq/26-eth0-rx-0-21swapper/321:28:583
29715050,0irq/26-eth0-rx-0-21swapper/321:28:583
29715050,0irq/26-eth0-rx-0-21swapper/321:24:403
29715050,0irq/26-eth0-rx-0-21swapper/321:08:483
29715050,0irq/26-eth0-rx-0-21swapper/321:05:263
29715050,0irq/26-eth0-rx-0-21swapper/320:24:303
29715050,0irq/26-eth0-rx-0-21swapper/320:01:333
259950,0migration/28934-21taskset21:03:532
259950,0migration/25575-21taskset23:20:312
259950,0migration/231641-21latency_hist22:38:422
259950,0migration/222532-21sh22:10:332
259950,0migration/214403-21aten_r1power_cu00:08:522
249950,0watchdog/213496-21ssh23:25:082
179950,0migration/16032-21sh22:20:001
179950,0migration/131895-21taskset22:58:541
179950,0migration/125684-21kthreadcore19:13:561
179950,0migration/123199-21taskset22:53:401
179950,0migration/121094-21missed_timers22:29:001
179950,0migration/114449-21idleruntime-cro23:48:411
149950,0migration/09435-21kthreadcore22:23:580
149950,0migration/07352-21fschecks_time18:53:560
149950,0migration/06953-21sh22:01:490
149950,0migration/06441-21taskset21:00:180
149950,0migration/05523-21sh23:41:170
149950,0migration/03895-21kthreadcore22:58:590
149950,0migration/031235-21taskset22:17:340
149950,0migration/029542-21taskset20:50:530
149950,0migration/027207-21kthreadcore21:13:580
149950,0migration/026602-21kthreadcore21:54:000
149950,0migration/026077-21kthreadcore22:53:580
149950,0migration/025620-21latency_hist20:03:430
149950,0migration/023083-21aten_r1power_fr23:33:520
149950,0migration/022876-21sh21:50:540
149950,0migration/022759-21sh21:09:590
149950,0migration/02244-21kthreadcore20:13:550
149950,0migration/020537-21taskset19:08:420
149950,0migration/017316-21kthreadcore00:08:580
149950,0migration/015952-21aten_r1power_en22:08:520
149950,0migration/013868-21munin-node21:04:010
149950,0migration/011445-21kthreadcore21:23:580
149950,0migration/010485-21kthreadcore23:03:580
8215040,0irq/24-0000:00:14723-21latency_hist23:28:412
65419944,0cyclictest0-21swapper/200:04:522
65419943,0cyclictest9128-21awk21:23:562
65419943,0cyclictest8945-21awk20:23:532
65419943,0cyclictest779-21awk21:38:572
65419943,0cyclictest3556-21lldpd20:46:262
65419943,0cyclictest3556-21lldpd20:33:552
65419943,0cyclictest3556-21lldpd20:30:252
65419943,0cyclictest3556-21lldpd19:40:232
65419943,0cyclictest3556-21lldpd19:23:542
65419943,0cyclictest3556-21lldpd19:00:542
65419943,0cyclictest3482-21sadc19:33:432
65419943,0cyclictest31294-21awk00:18:562
65419943,0cyclictest25307-21awk21:13:562
65419943,0cyclictest0-21swapper/223:53:572
65409944,0cyclictest0-21swapper/122:56:231
65409944,0cyclictest0-21swapper/120:33:521
65409943,0cyclictest8374-21awk22:43:571
65409943,0cyclictest4707-21awk19:33:531
65409943,0cyclictest3556-21lldpd21:47:571
65409943,0cyclictest3514-21snmpd23:57:051
65409943,0cyclictest3514-21snmpd21:01:351
65409943,0cyclictest3514-21snmpd20:39:351
65409943,0cyclictest3514-21snmpd19:10:491
65409943,0cyclictest3514-21snmpd19:01:191
65409943,0cyclictest3241-21irqbalance22:39:501
65409943,0cyclictest3241-21irqbalance22:34:501
65409943,0cyclictest3241-21irqbalance00:23:401
65409943,0cyclictest3241-21irqbalance00:09:401
65409943,0cyclictest320-21sadc20:13:431
65409943,0cyclictest1557-21awk20:13:541
65399944,0cyclictest0-21swapper/023:27:170
65399944,0cyclictest0-21swapper/023:23:390
65399944,0cyclictest0-21swapper/023:13:520
65399943,0cyclictest8969-21awk20:23:530
65399943,0cyclictest8852-21awk22:03:580
65399943,0cyclictest7587-21awk23:43:540
65399943,0cyclictest3514-21snmpd22:38:500
65399943,0cyclictest3514-21snmpd20:47:050
65399943,0cyclictest3241-21irqbalance23:58:500
65399943,0cyclictest3241-21irqbalance22:23:300
65399943,0cyclictest3241-21irqbalance00:23:200
65399943,0cyclictest16171-21awk23:08:550
65399942,0cyclictest16954-21kthreadcore22:48:570
339940,0migration/35168-21kthreadcore20:13:563
29725040,0irq/27-eth0-tx-7862-21ssh22:03:482
29725040,0irq/27-eth0-tx-24167-21ssh22:33:562
29725040,0irq/27-eth0-tx-16844-21fschecks_time21:28:542
29725040,0irq/27-eth0-tx-16844-21fschecks_time21:28:542
29725040,0irq/27-eth0-tx-0-21swapper/222:58:422
29725040,0irq/27-eth0-tx-0-21swapper/222:24:072
29725040,0irq/27-eth0-tx-0-21swapper/222:18:432
29725040,0irq/27-eth0-tx-0-21swapper/221:58:502
29725040,0irq/27-eth0-tx-0-21swapper/221:34:052
29725040,0irq/27-eth0-tx-0-21swapper/220:58:582
29725040,0irq/27-eth0-tx-0-21swapper/020:06:260
29715040,0irq/26-eth0-rx-0-21swapper/323:59:173
29715040,0irq/26-eth0-rx-0-21swapper/323:30:243
29715040,0irq/26-eth0-rx-0-21swapper/323:24:193
29715040,0irq/26-eth0-rx-0-21swapper/321:21:123
29715040,0irq/26-eth0-rx-0-21swapper/320:58:513
29715040,0irq/26-eth0-rx-0-21swapper/320:55:573
29715040,0irq/26-eth0-rx-0-21swapper/319:33:553
29715040,0irq/26-eth0-rx-0-21swapper/300:13:563
29715040,0irq/26-eth0-rx-0-21swapper/300:08:483
259940,0migration/28323-21kthreadcore23:43:562
259940,0migration/22770-21kthreadcore23:38:582
259940,0migration/226378-21fschecks_time20:03:522
259940,0migration/226334-21kthreadcore19:13:562
259940,0migration/224649-21kthreadcore23:13:582
259940,0migration/215471-21kthreadcore19:03:552
259940,0migration/213253-21latency_hist18:58:432
249940,0watchdog/26887-21seq20:15:412
249940,0watchdog/2229512sleep223:33:442
249940,0watchdog/20-21swapper/223:59:002
249940,0watchdog/20-21swapper/219:33:572
19140,0ktimersoftd/10-21swapper/100:03:231
179940,0migration/17093-21runrttasks18:53:551
179940,0migration/132761-21cpuspeed_turbos21:18:551
179940,0migration/132008-21munin-node23:38:561
179940,0migration/128961-21kthreadcore21:34:011
179940,0migration/128207-21taskset19:20:161
179940,0migration/127007-21kthreadcore22:13:591
179940,0migration/126208-21fschecks_count20:03:521
179940,0migration/124881-21kthreadcore23:33:581
179940,0migration/118443-21aten_r1power_en19:53:501
179940,0migration/117008-21gltestperf21:28:551
179940,0migration/117008-21gltestperf21:28:551
179940,0migration/116834-21kthreadcore23:48:591
179940,0migration/116511-21fschecks_time22:08:541
179940,0migration/113772-21kthreadcore19:43:551
179940,0migration/113285-21latency20:23:561
179940,0migration/110777-21taskset19:42:581
179940,0migration/11062-21kthreadcore23:18:591
169940,0watchdog/17731-21ssh21:43:321
169940,0watchdog/1349-21grep20:53:571
169940,0watchdog/131774-21ssh21:17:251
169940,0watchdog/131264-21ssh21:56:561
169940,0watchdog/128391-21expr20:44:011
169940,0watchdog/116104-21ssh21:48:481
169940,0watchdog/10-21swapper/122:04:001
169940,0watchdog/10-21swapper/121:59:481
149940,0migration/06352-21sh00:03:460
149940,0migration/019741-21kthreadcore23:49:000
149940,0migration/015623-21kthreadcore19:43:550
149940,0migration/015325-21sh21:47:110
65419933,0cyclictest0-21swapper/220:48:452
65419932,0cyclictest3556-21lldpd20:39:252
65419932,0cyclictest3556-21lldpd20:10:262
65419932,0cyclictest3556-21lldpd20:02:562
65419932,0cyclictest12060-21awk19:43:532
65409932,0cyclictest8519-21awk22:23:561
65409932,0cyclictest3556-21lldpd19:50:251
65409932,0cyclictest3514-21snmpd23:10:051
65409932,0cyclictest3514-21snmpd20:32:351
65409932,0cyclictest3514-21snmpd19:25:041
65399933,0cyclictest20993-21kworker/0:023:53:430
65399933,0cyclictest15040-21kworker/0:121:38:460
65399933,0cyclictest15040-21kworker/0:121:28:440
65399933,0cyclictest15040-21kworker/0:121:28:440
65399933,0cyclictest15040-21kworker/0:121:18:440
65399933,0cyclictest15040-21kworker/0:120:38:480
65399933,0cyclictest15040-21kworker/0:120:33:500
65399933,0cyclictest15040-21kworker/0:120:28:440
65399933,0cyclictest15040-21kworker/0:120:18:440
65399933,0cyclictest15040-21kworker/0:120:08:440
65399933,0cyclictest15040-21kworker/0:119:54:000
65399933,0cyclictest15040-21kworker/0:119:48:440
65399933,0cyclictest15040-21kworker/0:119:33:480
65399933,0cyclictest15040-21kworker/0:119:28:440
65399933,0cyclictest15040-21kworker/0:119:23:440
65399933,0cyclictest15040-21kworker/0:119:19:180
65399933,0cyclictest15040-21kworker/0:119:13:440
65399933,0cyclictest15040-21kworker/0:119:08:440
65399933,0cyclictest15040-21kworker/0:118:58:480
65399932,0cyclictest15040-21kworker/0:119:38:440
339930,0migration/310452-21runrttasks19:39:123
29715030,0irq/26-eth0-rx-0-21swapper/320:53:423
29715030,0irq/26-eth0-rx-0-21swapper/320:45:383
29715030,0irq/26-eth0-rx-0-21swapper/320:33:343
29715030,0irq/26-eth0-rx-0-21swapper/319:21:173
29715030,0irq/26-eth0-rx-0-21swapper/319:00:553
259930,0migration/221538-21kthreadcore19:53:552
249930,0watchdog/20-21swapper/219:19:132
179930,0migration/17942-21latency_hist20:23:431
179930,0migration/16358-21sh00:03:471
179930,0migration/125377-21runrttasks20:02:361
169930,0watchdog/129811-21diskmemload00:16:001
169930,0watchdog/10-21swapper/123:28:521
169930,0watchdog/10-21swapper/123:14:001
169930,0watchdog/10-21swapper/121:27:081
65409920,0cyclictest0-21swapper/119:28:431
29715020,0irq/26-eth0-rx-0-21swapper/320:38:503
29715020,0irq/26-eth0-rx-0-21swapper/320:33:583
29715020,0irq/26-eth0-rx-0-21swapper/320:19:333
29715020,0irq/26-eth0-rx-0-21swapper/320:09:013
29715020,0irq/26-eth0-rx-0-21swapper/319:48:583
29715020,0irq/26-eth0-rx-0-21swapper/319:29:563
249920,0watchdog/20-21swapper/219:48:452
169920,0watchdog/10-21swapper/120:49:051
169920,0watchdog/10-21swapper/119:03:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional