You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 10:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon May 25, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281623626,0sleep30-21swapper/318:51:293
282023523,0sleep10-21swapper/118:51:311
306423324,0sleep20-21swapper/218:54:162
277022622,0sleep00-21swapper/018:50:540
31679970,0cyclictest13576-21kthreadcore19:24:562
339960,0migration/38640-21sh21:37:153
339960,0migration/331229-21mdstat21:49:593
339960,0migration/325123-21sh22:49:403
31679966,0cyclictest0-21swapper/221:19:042
31679965,0cyclictest0-21swapper/223:41:362
31679961,0cyclictest0-21swapper/221:11:592
31679960,0cyclictest0-21swapper/200:19:292
31659966,0cyclictest0-21swapper/022:08:380
31659960,0cyclictest6620-21unixbench_singl23:35:010
31659960,0cyclictest1176-21diskmemload22:52:560
31659960,0cyclictest0-21swapper/021:04:180
29725060,0irq/27-eth0-tx-2408-21if_eth1.220:54:570
29715060,0irq/26-eth0-rx-29309-21cut00:09:543
29715060,0irq/26-eth0-rx-29309-21cut00:09:543
29715060,0irq/26-eth0-rx-26148-21hostname22:29:543
29715060,0irq/26-eth0-rx-17072-21grep23:44:533
29715060,0irq/26-eth0-rx-1157-21if_eth1.323:54:543
29715060,0irq/26-eth0-rx-0-21swapper/323:52:473
149960,0migration/08034-21sh22:37:270
149960,0migration/031001-21sh23:30:090
149960,0migration/025491-21sh21:49:320
149960,0migration/018169-21kthreadcore23:44:560
4150,0ktimersoftd/00-21swapper/000:19:290
339950,0migration/39287-21taskset21:18:263
339950,0migration/37611-21taskset22:15:393
339950,0migration/33489-21kthreadcore20:54:583
339950,0migration/332687-21latency_hist20:29:403
339950,0migration/330298-21needreboot23:29:573
339950,0migration/328163-21kthreadcore19:59:573
339950,0migration/326223-21switchtime20:45:023
339950,0migration/323502-21aten_r1power_po19:09:513
339950,0migration/322532-21timerandwakeup19:05:023
339950,0migration/322428-21missed_timers23:04:573
339950,0migration/320233-21kthreadcore00:04:563
339950,0migration/317470-21munin-node22:44:533
339950,0migration/314925-21latency21:59:573
339950,0migration/31374-21latency_hist21:54:393
31679953,0cyclictest8990-21aten_r1power_en22:59:502
31679953,0cyclictest8755-21ssh21:58:082
31679953,0cyclictest5964-21kthreadcore22:14:572
31679953,0cyclictest4788-21fschecks_count19:19:542
31679953,0cyclictest32747-21sadc23:14:392
31679953,0cyclictest3179-21kthreadcore22:34:582
31679953,0cyclictest3101-21kthreadcore21:34:572
31679953,0cyclictest28206-21kthreadcore21:49:562
31679953,0cyclictest27497-21kthreadcore00:09:532
31679953,0cyclictest27497-21kthreadcore00:09:532
31679953,0cyclictest27336-21kthreadcore22:49:542
31679953,0cyclictest27192-21kthreadcore19:09:572
31679953,0cyclictest27038-21hwlatdetect20:24:542
31679953,0cyclictest26484-21kthreadcore19:34:572
31679953,0cyclictest26370-21hddtemp_smartct22:09:532
31679953,0cyclictest25987-21irqcore23:29:542
31679953,0cyclictest25788-21fschecks_count22:29:532
31679953,0cyclictest22979-21missed_timers19:29:572
31679953,0cyclictest22738-21timerandwakeup23:25:022
31679953,0cyclictest21896-21kthreadcore23:04:562
31679953,0cyclictest20156-21kthreadcore22:04:562
31679953,0cyclictest19712-21kthreadcore23:44:572
31679953,0cyclictest19335-21fschecks_time19:54:532
31679953,0cyclictest18810-21ps21:44:552
31679953,0cyclictest1809-21if_eth1.322:54:542
31679953,0cyclictest1776-21kthreadcore23:54:552
31679953,0cyclictest16963-21ssh22:24:252
31679953,0cyclictest15306-21ssh23:21:142
31679953,0cyclictest14239-21kernelversion20:14:562
31679953,0cyclictest1417-21kthreadcore20:04:562
31679953,0cyclictest14123-21kthreadcore19:49:582
31679953,0cyclictest13187-21kthreadcore21:59:562
31679953,0cyclictest12831-21kthreadcore00:19:582
31679953,0cyclictest1176-21diskmemload00:04:422
31679953,0cyclictest11690-21kthreadcore18:59:562
31679953,0cyclictest1167-21crond20:54:392
31679952,0cyclictest9070-21munin-node22:39:582
31679952,0cyclictest7670-21ssh23:37:532
31669955,0cyclictest1176-21diskmemload21:27:131
31669955,0cyclictest0-21swapper/122:45:251
31669955,0cyclictest0-21swapper/120:57:441
31659955,0cyclictest0-21swapper/023:15:190
31659955,0cyclictest0-21swapper/021:41:590
31659952,0cyclictest27017-21kworker/0:120:09:550
31659950,0cyclictest0-21swapper/023:00:300
31659950,0cyclictest0-21swapper/021:08:530
29725050,0irq/27-eth0-tx-30993-21munin-node19:39:580
29725050,0irq/27-eth0-tx-12942-21munin-node20:14:540
29725050,0irq/27-eth0-tx-0-21swapper/019:44:540
29715050,0irq/26-eth0-rx-6559-21runrttasks21:25:203
29715050,0irq/26-eth0-rx-25738-21aten_r1power_vo22:09:513
29715050,0irq/26-eth0-rx-23489-21processes21:05:003
29715050,0irq/26-eth0-rx-22575-21find23:25:003
29715050,0irq/26-eth0-rx-17911-21aten_r1power_vo21:44:503
29715050,0irq/26-eth0-rx-10346-21gltestperf20:59:543
29715050,0irq/26-eth0-rx-0-21swapper/323:34:493
29715050,0irq/26-eth0-rx-0-21swapper/322:39:433
29715050,0irq/26-eth0-rx-0-21swapper/322:19:483
29715050,0irq/26-eth0-rx-0-21swapper/322:06:573
29715050,0irq/26-eth0-rx-0-21swapper/321:19:403
29715050,0irq/26-eth0-rx-0-21swapper/300:24:293
29715050,0irq/26-eth0-rx-0-21swapper/300:16:433
19150,0ktimersoftd/10-21swapper/122:40:041
19150,0ktimersoftd/10-21swapper/121:11:561
179950,0migration/19909-21fschecks_time22:19:531
179950,0migration/19524-21latency_hist21:39:391
179950,0migration/126635-21fschecks_time21:29:511
179950,0migration/122759-21missed_timers22:24:561
179950,0migration/121675-21kthreadcore22:04:571
179950,0migration/120331-21kthreadcore19:04:571
179950,0migration/111323-21kthreadcore21:59:561
169950,0watchdog/16509-21grep23:35:011
169950,0watchdog/126423-21kthreadcore00:09:531
169950,0watchdog/126423-21kthreadcore00:09:531
149950,0migration/09210-21sh21:18:120
149950,0migration/08574-21kthreadcore18:54:580
149950,0migration/0508-21sh21:12:050
149950,0migration/04748-21diskmemload21:34:570
149950,0migration/03501-21kthreadcore22:54:550
149950,0migration/030908-21latency22:09:570
149950,0migration/027882-21kthreadcore21:49:560
149950,0migration/02688-21kthreadcore20:04:560
149950,0migration/02608-21kthreadcore22:14:560
149950,0migration/024190-21taskset22:27:050
149950,0migration/022438-21switchtime19:05:010
149950,0migration/020948-21kthreadcore19:29:560
149950,0migration/015036-21irqrtprio20:39:550
149950,0migration/014645-21taskset00:21:240
149950,0migration/01394-21latency_hist21:34:390
149950,0migration/012997-21kthreadcore23:39:560
149950,0migration/011665-21kthreadcore18:59:560
149950,0migration/011392-21kthreadcore21:19:570
149950,0migration/010790-21irqrtprio21:59:550
149950,0migration/010379-21kthreadcore23:19:580
339940,0migration/325080-21open_inodes20:20:013
339940,0migration/318361-21taskset19:54:173
339940,0migration/314790-21kthreadcore20:14:563
339940,0migration/31407-21kthreadcore19:14:573
339940,0migration/314029-21kthreadcore23:19:593
339940,0migration/310740-21diskmemload21:39:543
339940,0migration/310694-21taskset19:23:313
31679943,0cyclictest7229-21awk20:34:512
31679943,0cyclictest635-21cpuspeed_turbos20:29:522
31679943,0cyclictest6289-21aten_r1power_cu20:09:502
31679943,0cyclictest554-21ssh23:14:482
31679943,0cyclictest30952-21latency_hist19:39:402
31679943,0cyclictest25314-21aten_r1power_cu19:59:512
31679943,0cyclictest20753-21cpuspeed_turbos20:44:522
31679943,0cyclictest20389-21smartctl20:19:542
31679943,0cyclictest18273-21fschecks_count21:04:522
31679943,0cyclictest1802-21df_abs20:54:542
31679943,0cyclictest13838-21ps20:39:412
31679943,0cyclictest1176-21diskmemload23:49:532
31679943,0cyclictest1176-21diskmemload22:44:422
31679942,0cyclictest9895-21latency_hist21:19:392
31679942,0cyclictest9645-21latency_hist21:39:392
31679942,0cyclictest9530-21latency_hist20:59:402
31679942,0cyclictest9136-21hwlatdetect23:59:542
31679942,0cyclictest5210-21latency_hist19:44:402
31679942,0cyclictest4106-21fschecks_time18:54:542
31679942,0cyclictest29999-21latency_hist19:14:402
31679942,0cyclictest27043-21irqcore21:29:542
31679942,0cyclictest18815-21if_err_eth021:24:532
31679942,0cyclictest17258-21latency_hist22:24:392
31679942,0cyclictest16675-21latency_hist19:04:402
31669944,0cyclictest0-21swapper/123:52:251
31669944,0cyclictest0-21swapper/123:40:361
31669944,0cyclictest0-21swapper/121:54:301
31669943,0cyclictest9835-21awk22:59:531
31669943,0cyclictest9757-21awk23:19:571
31669943,0cyclictest6123-21awk19:44:531
31669943,0cyclictest3556-21lldpd23:24:531
31669943,0cyclictest3556-21lldpd23:10:531
31669943,0cyclictest3556-21lldpd20:31:521
31669943,0cyclictest3556-21lldpd19:49:521
31669943,0cyclictest3556-21lldpd19:37:221
31669943,0cyclictest3556-21lldpd19:12:221
31669943,0cyclictest3556-21lldpd18:54:521
31669943,0cyclictest3556-21lldpd00:19:231
31669943,0cyclictest3556-21lldpd00:04:531
31669943,0cyclictest3241-21irqbalance23:14:571
31669943,0cyclictest26057-21awk22:49:521
31669943,0cyclictest25817-21awk23:29:541
31669943,0cyclictest18649-21awk21:44:541
31669943,0cyclictest18553-21awk19:29:541
31669943,0cyclictest1853-21awk22:34:561
31669943,0cyclictest10630-21awk21:39:541
31659944,0cyclictest0-21swapper/023:09:480
31659944,0cyclictest0-21swapper/022:49:090
31659944,0cyclictest0-21swapper/019:20:060
31659943,0cyclictest9928-21awk22:39:520
31659943,0cyclictest3514-21snmpd23:26:320
31659943,0cyclictest19593-21awk19:54:540
29725040,0irq/27-eth0-tx-6559-21runrttasks20:24:510
29725040,0irq/27-eth0-tx-30040-21munin-node19:14:580
29725040,0irq/27-eth0-tx-0-21swapper/020:34:590
29725040,0irq/27-eth0-tx-0-21swapper/019:59:520
29715040,0irq/26-eth0-rx-4074-21ntpq19:40:003
29715040,0irq/26-eth0-rx-10869-21kthreadcore22:59:563
29715040,0irq/26-eth0-rx-0-21swapper/323:40:453
29715040,0irq/26-eth0-rx-0-21swapper/323:16:003
29715040,0irq/26-eth0-rx-0-21swapper/323:10:373
29715040,0irq/26-eth0-rx-0-21swapper/322:54:453
29715040,0irq/26-eth0-rx-0-21swapper/322:35:033
29715040,0irq/26-eth0-rx-0-21swapper/322:24:533
29715040,0irq/26-eth0-rx-0-21swapper/321:29:553
29715040,0irq/26-eth0-rx-0-21swapper/321:10:153
29715040,0irq/26-eth0-rx-0-21swapper/319:24:523
29715040,0irq/26-eth0-rx-0-21swapper/300:00:413
179940,0migration/17564-21taskset22:56:301
179940,0migration/132699-21runrttasks22:33:411
179940,0migration/132234-21kthreadcore19:14:571
179940,0migration/127145-21if_eth020:24:551
179940,0migration/119533-21kthreadcore23:04:551
179940,0migration/118331-21fschecks_count21:04:521
179940,0migration/116324-21mdstat19:24:571
179940,0migration/11477-21aten_r1power_en21:54:501
179940,0migration/112802-21latency_hist20:14:401
169940,0watchdog/19696-21ssh21:19:381
169940,0watchdog/13300-21cut19:39:581
169940,0watchdog/132618-21aten2.4-expect23:54:491
169940,0watchdog/130263-21cut20:49:581
169940,0watchdog/122894-21grep20:19:581
169940,0watchdog/121506-21kthreadcore23:44:571
169940,0watchdog/115084-21ssh00:01:491
169940,0watchdog/114662-21kthreadcore21:19:581
169940,0watchdog/10-21swapper/120:59:581
169940,0watchdog/10-21swapper/119:20:021
149940,0migration/031615-21sh22:30:320
149940,0migration/027384-21kthreadcore19:34:580
149940,0migration/026266-21kthreadcore00:09:530
149940,0migration/026266-21kthreadcore00:09:530
149940,0migration/019966-21kthreadcore21:24:550
149940,0migration/01538-21kthreadcore23:54:550
149940,0migration/013996-21munin-node00:00:010
31669932,0cyclictest8177-21awk20:34:551
31669932,0cyclictest3556-21lldpd00:19:531
31669932,0cyclictest29850-21kthreadcore19:59:571
31669932,0cyclictest21341-21awk20:44:541
31669932,0cyclictest14806-21awk20:39:541
31669932,0cyclictest13811-21awk20:14:541
31659933,0cyclictest27017-21kworker/0:123:04:580
31659933,0cyclictest27017-21kworker/0:122:19:450
31659933,0cyclictest27017-21kworker/0:121:54:470
31659933,0cyclictest27017-21kworker/0:120:44:470
31659933,0cyclictest27017-21kworker/0:120:29:410
31659933,0cyclictest27017-21kworker/0:119:49:450
31659933,0cyclictest27017-21kworker/0:119:24:410
31659933,0cyclictest27017-21kworker/0:119:09:410
31659933,0cyclictest25629-21kworker/0:223:49:540
31659933,0cyclictest25629-21kworker/0:200:04:400
31659930,0cyclictest27318-21telnet20:49:520
31659930,0cyclictest0-21swapper/020:19:570
29715030,0irq/26-eth0-rx-0-21swapper/320:39:503
29715030,0irq/26-eth0-rx-0-21swapper/320:29:523
29715030,0irq/26-eth0-rx-0-21swapper/320:04:513
29715030,0irq/26-eth0-rx-0-21swapper/319:54:573
29715030,0irq/26-eth0-rx-0-21swapper/319:44:553
29715030,0irq/26-eth0-rx-0-21swapper/319:34:523
29715030,0irq/26-eth0-rx-0-21swapper/319:02:463
29715030,0irq/26-eth0-rx-0-21swapper/318:54:523
179930,0migration/131931-21runrttasks20:04:421
179930,0migration/125753-21sh22:09:511
169930,0watchdog/10-21swapper/122:14:581
169930,0watchdog/10-21swapper/119:55:421
169930,0watchdog/10-21swapper/118:59:541
35120,0ktimersoftd/318145-21fschecks_count19:29:523
29715020,0irq/26-eth0-rx-0-21swapper/320:49:523
29715020,0irq/26-eth0-rx-0-21swapper/320:35:283
29715020,0irq/26-eth0-rx-0-21swapper/320:09:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional