You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 09:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Sep 08, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191800,0ktimersoftd/118629-21perf18:44:241
2510324030,0sleep20-21swapper/218:43:312
2506923726,0sleep30-21swapper/318:43:043
2498123221,0sleep00-21swapper/018:42:010
252669970,0cyclictest30579-21fschecks_time22:09:471
8215060,0irq/24-0000:00:13178-21munin-plugin-st22:19:322
339960,0migration/36746-21munin-node21:34:423
339960,0migration/3632-21sh00:11:273
339960,0migration/36004-21taskset20:51:303
339960,0migration/311857-21sh22:15:353
339960,0migration/310103-21munin-node22:54:543
29715060,0irq/26-eth0-rx-19422-21munin-node00:04:503
29715060,0irq/26-eth0-rx-0-21swapper/300:00:203
252669960,0cyclictest30321-21rm20:46:421
252649960,0cyclictest0-21swapper/022:19:220
149960,0migration/014509-21sh20:57:380
4150,0ktimersoftd/00-21swapper/000:07:260
339950,0migration/39431-21kthreadcore20:54:523
339950,0migration/38273-21kthreadcore18:54:483
339950,0migration/35546-21fschecks_time22:34:463
339950,0migration/33610-21latency_hist23:14:323
339950,0migration/331219-21munin-node22:09:493
339950,0migration/329266-21kthreadcore19:09:513
339950,0migration/326144-21taskset19:08:473
339950,0migration/324414-21kthreadcore22:44:543
339950,0migration/320789-21pmu-power21:19:533
339950,0migration/318116-21kthreadcore22:39:513
339950,0migration/313795-21runrttasks21:16:083
29725050,0irq/27-eth0-tx-7448-21munin-node20:54:532
29725050,0irq/27-eth0-tx-3514-21snmpd19:39:530
29725050,0irq/27-eth0-tx-29598-21switchtime20:44:560
29725050,0irq/27-eth0-tx-28941-21ssh22:07:232
29725050,0irq/27-eth0-tx-22996-21ssh21:03:432
29725050,0irq/27-eth0-tx-13892-21iostat_ios22:39:492
29725050,0irq/27-eth0-tx-10578-21munin-node23:59:532
29725050,0irq/27-eth0-tx-0-21swapper/222:51:042
29725050,0irq/27-eth0-tx-0-21swapper/222:29:502
29725050,0irq/27-eth0-tx-0-21swapper/222:01:122
29725050,0irq/27-eth0-tx-0-21swapper/221:54:412
29725050,0irq/27-eth0-tx-0-21swapper/020:19:480
29725050,0irq/27-eth0-tx-0-21swapper/020:14:490
29725050,0irq/27-eth0-tx-0-21swapper/019:37:020
29725050,0irq/27-eth0-tx-0-21swapper/019:14:420
29725050,0irq/27-eth0-tx-0-21swapper/018:49:370
29715050,0irq/26-eth0-rx-6230-21kthreadcore23:54:503
29715050,0irq/26-eth0-rx-4884-21munin-node20:29:483
29715050,0irq/26-eth0-rx-2284-21sshd20:02:503
29715050,0irq/26-eth0-rx-22779-21kthreadcore23:44:533
29715050,0irq/26-eth0-rx-16993-21kthreadcore21:59:503
29715050,0irq/26-eth0-rx-14556-21kthreadcore22:59:503
29715050,0irq/26-eth0-rx-0-21swapper/323:49:463
29715050,0irq/26-eth0-rx-0-21swapper/323:29:583
29715050,0irq/26-eth0-rx-0-21swapper/323:24:533
29715050,0irq/26-eth0-rx-0-21swapper/323:22:053
29715050,0irq/26-eth0-rx-0-21swapper/323:11:243
29715050,0irq/26-eth0-rx-0-21swapper/323:04:483
29715050,0irq/26-eth0-rx-0-21swapper/322:49:413
29715050,0irq/26-eth0-rx-0-21swapper/322:29:503
29715050,0irq/26-eth0-rx-0-21swapper/322:19:503
29715050,0irq/26-eth0-rx-0-21swapper/322:06:203
29715050,0irq/26-eth0-rx-0-21swapper/321:54:453
29715050,0irq/26-eth0-rx-0-21swapper/321:41:373
29715050,0irq/26-eth0-rx-0-21swapper/321:33:493
29715050,0irq/26-eth0-rx-0-21swapper/321:10:353
29715050,0irq/26-eth0-rx-0-21swapper/321:06:273
29715050,0irq/26-eth0-rx-0-21swapper/320:48:213
29715050,0irq/26-eth0-rx-0-21swapper/318:44:513
27150,0ktimersoftd/20-21swapper/223:06:542
27150,0ktimersoftd/20-21swapper/222:38:592
252679950,0cyclictest0-21swapper/221:19:422
252669955,0cyclictest23527-21diskmemload21:31:001
252669955,0cyclictest0-21swapper/122:54:521
252669955,0cyclictest0-21swapper/122:50:371
252669955,0cyclictest0-21swapper/122:21:081
252669955,0cyclictest0-21swapper/120:04:551
252669950,0cyclictest0-21swapper/122:49:121
252669950,0cyclictest0-21swapper/122:16:521
252669950,0cyclictest0-21swapper/121:39:121
252669950,0cyclictest0-21swapper/120:59:561
252649955,0cyclictest0-21swapper/023:07:560
252649955,0cyclictest0-21swapper/021:51:000
252649955,0cyclictest0-21swapper/021:15:030
259950,0migration/27352-21aten_r1power_po21:14:452
259950,0migration/26778-21sh20:53:292
259950,0migration/26709-21kthreadcore22:54:492
259950,0migration/26228-21sh21:34:102
259950,0migration/229790-21kthreadcore23:29:492
259950,0migration/229704-21sh21:47:572
259950,0migration/22626-21kthreadcore21:49:502
259950,0migration/222381-21kthreadcore23:44:532
259950,0migration/221601-21idleruntime22:44:512
259950,0migration/211928-21hwlatdetect23:39:452
249950,0watchdog/220700-21ssh22:23:272
179950,0migration/12840-21timerwakeupswit22:29:561
169950,0watchdog/122763-21cut20:19:431
169950,0watchdog/115273-21fschecks_time21:39:471
149950,0migration/09512-21munin-node20:09:460
149950,0migration/07916-21kthreadcore22:54:490
149950,0migration/07244-21kthreadcore23:34:490
149950,0migration/06679-21aten_r1power_en21:34:420
149950,0migration/06400-21kthreadcore22:34:490
149950,0migration/04504-21taskset22:12:240
149950,0migration/027834-21kthreadcore19:09:500
149950,0migration/02669-21kthreadcore21:29:500
149950,0migration/026106-21ksmtuned23:26:000
149950,0migration/025994-21kthreadcore19:29:520
149950,0migration/02385-21latency_hist23:54:320
149950,0migration/021893-21sh21:21:430
149950,0migration/02066-21kthreadcore22:29:510
149950,0migration/019239-21aten2.4-expect23:44:470
149950,0migration/019031-21kthreadcore19:49:510
149950,0migration/019031-21kthreadcore19:49:510
149950,0migration/01455-21munin-node23:09:530
149950,0migration/014315-21timerandwakeup19:44:560
149950,0migration/013376-21kthreadcore20:34:520
149950,0migration/012614-21kthreadcore19:19:510
149950,0migration/01109-21taskset23:51:110
149950,0migration/010002-21taskset20:32:410
339940,0migration/34589-21sh21:50:553
339940,0migration/34498-21kthreadcore19:39:513
339940,0migration/33723-21fschecks_time23:34:453
339940,0migration/331334-21kthreadcore20:24:483
339940,0migration/326706-21kthreadcore20:19:493
339940,0migration/323503-21if_eth1.221:44:503
339940,0migration/322582-21kthreadcore19:29:513
339940,0migration/315285-21irqrtprio19:24:513
339940,0migration/313839-21taskset19:24:203
339940,0migration/313825-21fschecks_time18:59:473
29725040,0irq/27-eth0-tx-23610-21ssh21:04:432
29725040,0irq/27-eth0-tx-23527-21diskmemload23:15:112
29725040,0irq/27-eth0-tx-23527-21diskmemload22:24:392
29725040,0irq/27-eth0-tx-12258-21ssh22:59:432
29725040,0irq/27-eth0-tx-11626-21ssh23:19:402
29725040,0irq/27-eth0-tx-0-21swapper/223:55:562
29725040,0irq/27-eth0-tx-0-21swapper/223:09:432
29725040,0irq/27-eth0-tx-0-21swapper/222:09:492
29725040,0irq/27-eth0-tx-0-21swapper/221:34:432
29725040,0irq/27-eth0-tx-0-21swapper/221:24:462
29725040,0irq/27-eth0-tx-0-21swapper/221:09:352
29725040,0irq/27-eth0-tx-0-21swapper/200:10:012
29725040,0irq/27-eth0-tx-0-21swapper/019:24:570
29715040,0irq/26-eth0-rx-18404-21kthreadcore20:39:523
29715040,0irq/26-eth0-rx-0-21swapper/323:40:573
29715040,0irq/26-eth0-rx-0-21swapper/322:24:483
29715040,0irq/26-eth0-rx-0-21swapper/321:24:433
29715040,0irq/26-eth0-rx-0-21swapper/320:59:453
29715040,0irq/26-eth0-rx-0-21swapper/320:04:503
29715040,0irq/26-eth0-rx-0-21swapper/319:34:503
29715040,0irq/26-eth0-rx-0-21swapper/318:49:373
252679943,0cyclictest3556-21lldpd20:36:592
252679943,0cyclictest3556-21lldpd20:15:292
252679943,0cyclictest3556-21lldpd19:46:592
252679943,0cyclictest3556-21lldpd19:35:292
252679943,0cyclictest3556-21lldpd19:14:592
252679943,0cyclictest3556-21lldpd19:01:282
252679943,0cyclictest27290-21awk19:09:482
252679943,0cyclictest2661-21awk19:39:492
252679943,0cyclictest20489-21awk23:24:512
252679943,0cyclictest10314-21awk20:09:492
252669944,0cyclictest0-21swapper/120:56:001
252669944,0cyclictest0-21swapper/118:59:481
252669943,0cyclictest9496-21awk19:44:481
252669943,0cyclictest8172-21awk21:14:481
252669943,0cyclictest3556-21lldpd23:40:021
252669943,0cyclictest3556-21lldpd23:20:021
252669943,0cyclictest3514-21snmpd19:29:381
252669943,0cyclictest3514-21snmpd18:57:381
252669943,0cyclictest3241-21irqbalance21:48:111
252669943,0cyclictest3241-21irqbalance20:35:211
252669943,0cyclictest3241-21irqbalance20:26:111
252669943,0cyclictest3241-21irqbalance20:15:111
252669943,0cyclictest3241-21irqbalance19:35:121
252669943,0cyclictest3241-21irqbalance19:25:021
252669943,0cyclictest3241-21irqbalance19:19:521
252669943,0cyclictest3241-21irqbalance19:15:121
252669943,0cyclictest3241-21irqbalance19:09:421
252669943,0cyclictest3241-21irqbalance19:04:521
252669943,0cyclictest31362-21awk21:49:471
252669943,0cyclictest22676-21awk19:54:471
252669943,0cyclictest13765-21awk22:39:481
252669943,0cyclictest10284-21awk20:09:491
252669940,0cyclictest0-21swapper/100:05:301
252649944,0cyclictest0-21swapper/021:48:090
252649944,0cyclictest0-21swapper/020:28:040
252649943,0cyclictest3514-21snmpd23:43:230
252649943,0cyclictest3241-21irqbalance23:30:310
252649943,0cyclictest3241-21irqbalance23:19:410
252649943,0cyclictest3241-21irqbalance22:25:110
252649943,0cyclictest3241-21irqbalance22:23:410
252649943,0cyclictest3241-21irqbalance22:04:510
252649943,0cyclictest3241-21irqbalance21:58:210
252649943,0cyclictest29358-21awk19:59:480
252649943,0cyclictest24503-21awk21:04:510
252649943,0cyclictest15040-21kworker/0:122:50:340
252649943,0cyclictest15040-21kworker/0:118:54:430
252649943,0cyclictest13734-21awk22:39:480
259940,0migration/27224-21kthreadcore20:29:492
259940,0migration/26928-21kthreadcore23:34:492
259940,0migration/232347-21kthreadcore19:59:502
259940,0migration/225703-21kthreadcore20:44:502
259940,0migration/221579-21kthreadcore00:04:522
259940,0migration/221542-21runrttasks19:54:252
259940,0migration/221542-21runrttasks19:54:252
259940,0migration/218210-21irqcore20:39:512
259940,0migration/215225-21irqcore19:24:502
259940,0migration/213337-21munin-node19:19:562
249940,0watchdog/26559-21runrttasks19:06:522
249940,0watchdog/231148-21find18:44:562
249940,0watchdog/227701-21unixbench_multi19:54:552
249940,0watchdog/223681-21munin-node20:19:472
249940,0watchdog/219115-21kthreadcore21:39:512
249940,0watchdog/20-21swapper/223:49:542
249940,0watchdog/20-21swapper/219:29:512
179940,0migration/19823-21runrttasks20:30:351
179940,0migration/18744-21kthreadcore23:14:501
179940,0migration/13202-21kthreadcore19:39:511
179940,0migration/130564-21kthreadcore19:59:501
179940,0migration/130230-21kthreadcore23:09:481
179940,0migration/129187-21open_files21:04:561
179940,0migration/119301-21taskset23:02:381
179940,0migration/11318-21kthreadcore20:49:501
179940,0migration/110688-21munin-node22:34:541
169940,0watchdog/131404-21kthreadcore00:09:501
169940,0watchdog/126432-21kthreadcore22:24:551
169940,0watchdog/125622-21ssh23:46:101
169940,0watchdog/125433-21swap23:24:591
169940,0watchdog/123115-21ksmtuned20:39:591
169940,0watchdog/120229-21aten_r1power_cu23:04:431
169940,0watchdog/118778-21kthreadcore19:49:511
169940,0watchdog/118778-21kthreadcore19:49:511
169940,0watchdog/118497-21kthreadcore21:59:511
169940,0watchdog/11625-21ssh23:30:591
169940,0watchdog/115596-21tr23:59:501
169940,0watchdog/110053-21ssh23:37:061
169940,0watchdog/10-21swapper/121:19:511
169940,0watchdog/10-21swapper/121:09:511
149940,0migration/029059-21kthreadcore00:09:500
149940,0migration/028876-21sensors_temp21:24:530
149940,0migration/025192-21kthreadcore19:54:500
149940,0migration/017988-21processes22:59:540
149940,0migration/014470-21fschecks_count21:59:460
149940,0migration/014331-21irqrtprio18:59:490
29725030,0irq/27-eth0-tx-2774-21aten2.4-expect20:04:440
29725030,0irq/27-eth0-tx-0-21swapper/018:44:460
29715030,0irq/26-eth0-rx-0-21swapper/320:34:573
29715030,0irq/26-eth0-rx-0-21swapper/320:09:463
29715030,0irq/26-eth0-rx-0-21swapper/319:54:443
29715030,0irq/26-eth0-rx-0-21swapper/319:49:463
29715030,0irq/26-eth0-rx-0-21swapper/319:49:463
29715030,0irq/26-eth0-rx-0-21swapper/319:18:423
252679932,0cyclictest571-21awk18:49:482
252679932,0cyclictest3556-21lldpd20:08:292
252679930,0cyclictest30105-21perl20:24:452
252669932,0cyclictest27268-21cut23:49:441
252669932,0cyclictest23569-21tune2fs21:24:451
252669932,0cyclictest13137-21ssh21:57:421
252669930,0cyclictest5769-21grep23:54:491
252669930,0cyclictest32565-21diskstats18:49:461
252649933,0cyclictest15040-21kworker/0:123:15:440
252649933,0cyclictest15040-21kworker/0:121:39:420
252649933,0cyclictest15040-21kworker/0:121:09:360
252649933,0cyclictest15040-21kworker/0:120:59:420
252649933,0cyclictest15040-21kworker/0:120:39:520
252649932,0cyclictest3514-21snmpd19:04:400
252649932,0cyclictest2957-21tr20:49:510
252649932,0cyclictest11524-21awk23:59:480
252649930,0cyclictest21433-21perl22:44:500
249930,0watchdog/20-21swapper/218:54:522
169930,0watchdog/126166-21cut18:44:481
169930,0watchdog/10-21swapper/122:04:431
29715020,0irq/26-eth0-rx-0-21swapper/320:14:353
29715020,0irq/26-eth0-rx-0-21swapper/319:44:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional