You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 04:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Mar 24, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:56:223
2928923526,0sleep20-21swapper/218:56:262
2936323424,0sleep10-21swapper/118:57:171
2944023221,0sleep00-21swapper/018:58:080
297119999,0cyclictest0-21swapper/221:01:012
339960,0migration/39360-21sh23:49:523
339960,0migration/325845-21sh23:40:063
339960,0migration/320693-21irqrtprio20:05:493
339960,0migration/318785-21irqrtprio19:15:473
339960,0migration/318084-21sh22:53:173
29725060,0irq/27-eth0-tx-7118-21munin-node20:20:550
29725060,0irq/27-eth0-tx-17297-21grep22:10:512
297119960,0cyclictest0-21swapper/219:10:482
297109960,0cyclictest0-21swapper/121:19:211
29715060,0irq/26-eth0-rx-8491-21memory22:45:463
29715060,0irq/26-eth0-rx-27010-21df23:20:433
149960,0migration/08763-21sh00:09:120
149960,0migration/08737-21taskset23:48:010
149960,0migration/05465-21unixbench_singl19:50:560
149960,0migration/032074-21sh00:22:230
149960,0migration/027368-21sh22:20:190
149960,0migration/024753-21sh23:58:040
8215050,0irq/24-0000:00:666-21latency_hist00:25:292
8215050,0irq/24-0000:00:27287-21cat22:40:312
8215050,0irq/24-0000:00:25925-21munin-run20:10:312
8215050,0irq/24-0000:00:0-21swapper/223:35:302
4150,0ktimersoftd/00-21swapper/019:02:550
35150,0ktimersoftd/30-21swapper/300:16:273
339950,0migration/37068-21timerwakeupswit00:25:553
339950,0migration/34269-21munin-node19:25:543
339950,0migration/33614-21cpu22:25:423
339950,0migration/330857-21kthreadcore23:00:463
339950,0migration/330309-21munin-node19:00:413
339950,0migration/330164-21kthreadcore22:20:453
339950,0migration/327681-21kthreadcore00:00:483
339950,0migration/326739-21missed_timers20:55:503
339950,0migration/323246-21kthreadcore21:35:483
339950,0migration/320159-21aten_r1power_fr21:55:403
339950,0migration/319854-21latency_hist21:55:313
339950,0migration/318848-21if_eth023:55:473
339950,0migration/316648-21kthreadcore20:50:483
339950,0migration/316124-21munin-node23:30:523
339950,0migration/311391-21taskset22:10:193
339950,0migration/311354-21kthreadcore19:55:483
339950,0migration/310010-21aten2.4-expect23:50:403
29725050,0irq/27-eth0-tx-9979-21ssh22:06:102
29725050,0irq/27-eth0-tx-7062sleep223:21:502
29725050,0irq/27-eth0-tx-4404-21ssh21:25:392
29725050,0irq/27-eth0-tx-4128-21munin-node21:45:482
29725050,0irq/27-eth0-tx-30922-21munin-node19:25:500
29725050,0irq/27-eth0-tx-19516-21munin-node20:05:450
29725050,0irq/27-eth0-tx-0-21swapper/223:02:542
29725050,0irq/27-eth0-tx-0-21swapper/222:31:412
29725050,0irq/27-eth0-tx-0-21swapper/221:44:532
29725050,0irq/27-eth0-tx-0-21swapper/221:15:452
29725050,0irq/27-eth0-tx-0-21swapper/221:14:572
29725050,0irq/27-eth0-tx-0-21swapper/021:00:590
29725050,0irq/27-eth0-tx-0-21swapper/019:10:510
297119955,0cyclictest0-21swapper/222:27:152
297119950,0cyclictest20106-21ssh21:35:132
297119950,0cyclictest0-21swapper/223:29:502
297119950,0cyclictest0-21swapper/221:36:002
297119950,0cyclictest0-21swapper/220:12:552
297109955,0cyclictest0-21swapper/123:05:461
297109955,0cyclictest0-21swapper/123:05:461
297109955,0cyclictest0-21swapper/121:45:501
297109955,0cyclictest0-21swapper/121:10:411
297109955,0cyclictest0-21swapper/120:05:531
297109950,0cyclictest0-21swapper/123:56:411
297109950,0cyclictest0-21swapper/122:32:401
29715050,0irq/26-eth0-rx-9752-21port_sshpriv21:45:513
29715050,0irq/26-eth0-rx-4832-21runrttasks20:40:503
29715050,0irq/26-eth0-rx-3132-21fschecks_count23:05:433
29715050,0irq/26-eth0-rx-3132-21fschecks_count23:05:433
29715050,0irq/26-eth0-rx-30303-21tr22:40:483
29715050,0irq/26-eth0-rx-2595-21taskset21:41:593
29715050,0irq/26-eth0-rx-10312-21ssh21:06:223
29715050,0irq/26-eth0-rx-0-21swapper/323:17:303
29715050,0irq/26-eth0-rx-0-21swapper/323:17:303
29715050,0irq/26-eth0-rx-0-21swapper/322:56:123
29715050,0irq/26-eth0-rx-0-21swapper/322:35:503
29715050,0irq/26-eth0-rx-0-21swapper/322:30:473
29715050,0irq/26-eth0-rx-0-21swapper/322:15:053
29715050,0irq/26-eth0-rx-0-21swapper/321:31:443
29715050,0irq/26-eth0-rx-0-21swapper/321:22:423
29715050,0irq/26-eth0-rx-0-21swapper/321:15:533
29715050,0irq/26-eth0-rx-0-21swapper/321:00:503
297099955,0cyclictest0-21swapper/019:30:410
297099950,0cyclictest0-21swapper/023:25:170
27150,0ktimersoftd/20-21swapper/219:02:552
27150,0ktimersoftd/20-21swapper/200:09:342
259950,0migration/26310-21kthreadcore21:05:482
259950,0migration/229976-21kthreadcore21:20:482
259950,0migration/219540-21idleruntime23:15:432
259950,0migration/219540-21idleruntime23:15:432
259950,0migration/219332-21latency_hist22:35:312
259950,0migration/218756-21idleruntime23:55:462
259950,0migration/216644-21sh00:13:532
249950,0watchdog/25192-21cat22:45:442
249950,0watchdog/228468-21forks22:00:452
249950,0watchdog/225963-21sh22:16:132
249950,0watchdog/21088-21perl20:15:452
249950,0watchdog/20-21swapper/220:20:492
179950,0migration/17106-21munin-node20:40:551
179950,0migration/13606-21kthreadcore23:45:471
179950,0migration/1329-21switchtime20:35:551
179950,0migration/116756-21pmu-power22:50:521
179950,0migration/113489-21kthreadcore22:10:481
179950,0migration/110879-21latency_hist19:10:311
169950,0watchdog/1982-21tr21:20:491
169950,0watchdog/16170-21if_eth1.319:30:451
169950,0watchdog/131656-21grep19:25:451
169950,0watchdog/126959-21perl23:40:441
169950,0watchdog/11210-21kthreadcore19:00:451
149950,0migration/09855-21sh22:47:410
149950,0migration/09555-21kthreadcore20:45:480
149950,0migration/09444-21taskset00:10:340
149950,0migration/09120-21runrttasks23:27:410
149950,0migration/06755-21kthreadcore21:05:480
149950,0migration/03363-21kthreadcore20:15:490
149950,0migration/032293-21sh00:01:490
149950,0migration/030820-21kthreadcore21:20:480
149950,0migration/030397-21taskset19:22:110
149950,0migration/02822-21kthreadcore00:25:490
149950,0migration/028191-21aten_r1power_vo22:00:430
149950,0migration/028015-21cpuspeed_turbos22:20:400
149950,0migration/027637-21fschecks_count23:00:420
149950,0migration/026419-21aten_r1power_vo23:40:420
149950,0migration/023613-21taskset19:15:580
149950,0migration/023460-21kthreadcore22:35:510
149950,0migration/021923-21kthreadcore23:35:470
149950,0migration/020507-21kthreadcore23:15:460
149950,0migration/020507-21kthreadcore23:15:460
149950,0migration/018899-21kthreadcore20:25:500
149950,0migration/015784-21irqcore20:50:480
149950,0migration/014063-21kthreadcore23:30:480
149950,0migration/013078-21fschecks_time21:30:440
149950,0migration/012437-21taskset19:58:170
149950,0migration/012325-21kthreadcore23:10:480
149950,0migration/011902-21taskset19:35:320
339940,0migration/39472-21kthreadcore20:45:483
339940,0migration/330279-21kthreadcore20:35:503
339940,0migration/329740-21open_files19:20:503
339940,0migration/325183-21latency_hist19:45:313
339940,0migration/324854-21kthreadcore20:30:463
339940,0migration/324567-21kthreadcore22:15:483
339940,0migration/319402-21if_err_eth019:40:453
339940,0migration/314554-21kthreadcore20:00:493
339940,0migration/314011-21kthreadcore21:10:483
29725040,0irq/27-eth0-tx-0-21swapper/223:53:312
29725040,0irq/27-eth0-tx-0-21swapper/223:38:592
29725040,0irq/27-eth0-tx-0-21swapper/223:08:112
29725040,0irq/27-eth0-tx-0-21swapper/223:08:112
29725040,0irq/27-eth0-tx-0-21swapper/222:55:352
29725040,0irq/27-eth0-tx-0-21swapper/221:55:422
29725040,0irq/27-eth0-tx-0-21swapper/200:20:482
29725040,0irq/27-eth0-tx-0-21swapper/200:00:372
29725040,0irq/27-eth0-tx-0-21swapper/020:35:480
29725040,0irq/27-eth0-tx-0-21swapper/020:30:490
297119943,0cyclictest3514-21snmpd20:47:012
297119943,0cyclictest3514-21snmpd19:43:012
297119943,0cyclictest3514-21snmpd19:07:012
297119943,0cyclictest28634-21awk22:20:432
297119943,0cyclictest27075-21hwlatdetect23:40:452
297119943,0cyclictest26036-21awk19:45:462
297119943,0cyclictest2176-21awk20:40:472
297109944,0cyclictest0-21swapper/123:15:331
297109944,0cyclictest0-21swapper/123:15:331
297109944,0cyclictest0-21swapper/122:39:171
297109943,0cyclictest3514-21snmpd22:26:011
297109943,0cyclictest3514-21snmpd22:08:311
297109943,0cyclictest3514-21snmpd21:50:311
297109943,0cyclictest3514-21snmpd19:47:011
297109943,0cyclictest29049-21awk22:00:471
297109943,0cyclictest21322-21awk20:30:431
297109943,0cyclictest20960-21awk21:55:441
297109943,0cyclictest18590-21awk19:15:461
297109943,0cyclictest15663-21awk20:50:471
297109943,0cyclictest13387-21awk21:30:451
29715040,0irq/26-eth0-rx-0-21swapper/323:43:343
29715040,0irq/26-eth0-rx-0-21swapper/323:25:413
29715040,0irq/26-eth0-rx-0-21swapper/323:12:313
29715040,0irq/26-eth0-rx-0-21swapper/322:00:403
29715040,0irq/26-eth0-rx-0-21swapper/321:25:433
29715040,0irq/26-eth0-rx-0-21swapper/319:50:443
29715040,0irq/26-eth0-rx-0-21swapper/319:10:473
29715040,0irq/26-eth0-rx-0-21swapper/300:20:533
29715040,0irq/26-eth0-rx-0-21swapper/300:10:473
29715040,0irq/26-eth0-rx-0-21swapper/300:05:333
297099944,0cyclictest0-21swapper/022:29:000
297099943,0cyclictest3241-21irqbalance21:12:090
297099943,0cyclictest21380-21awk21:35:460
297099943,0cyclictest20089-21awk22:55:460
297099943,0cyclictest12207-21sadc21:30:310
297099943,0cyclictest10839-21kworker/0:221:45:540
259940,0migration/231830-21kthreadcore20:35:502
259940,0migration/228253-21kthreadcore19:20:472
259940,0migration/222398-21kthreadcore19:15:482
259940,0migration/214495-21kthreadcore21:50:472
259940,0migration/211200-21aten_r1power_fr22:50:422
249940,0watchdog/231659-21gltestperf19:25:452
249940,0watchdog/217814-21sed00:15:442
249940,0watchdog/216329-21cut20:00:492
249940,0watchdog/215084-21kernelversion20:25:492
249940,0watchdog/211349-21perl23:10:442
249940,0watchdog/20-21swapper/223:45:322
249940,0watchdog/20-21swapper/220:50:492
249940,0watchdog/20-21swapper/220:30:412
249940,0watchdog/20-21swapper/219:35:452
179940,0migration/1985-21sh00:05:041
179940,0migration/19020-21kthreadcore21:05:491
179940,0migration/16170-21latency_hist19:55:311
179940,0migration/15486-21irqcore21:25:461
179940,0migration/15251-21kthreadcore20:15:491
179940,0migration/13727-21kthreadcore00:05:481
179940,0migration/129158-21kthreadcore19:20:471
179940,0migration/128488-21kthreadcore23:00:451
179940,0migration/128041-21kthreadcore23:20:471
179940,0migration/127344-21runrttasks22:40:311
179940,0migration/125246-21ksmtuned22:55:541
179940,0migration/123657-21kthreadcore19:40:481
179940,0migration/12320-21aten_r1power_en23:25:401
179940,0migration/123019-21kthreadcore22:15:481
179940,0migration/122955-21kthreadcore20:55:481
179940,0migration/121222-21gltestperf21:35:451
179940,0migration/112824-21kthreadcore23:30:481
179940,0migration/110829-21munin-node23:50:441
169940,0watchdog/18797-21processes22:45:481
169940,0watchdog/16559-21runrttasks00:27:441
169940,0watchdog/15007-21hddtemp_smartct19:05:451
169940,0watchdog/132691-21ssh00:24:121
169940,0watchdog/129034-21kthreadcore22:20:451
169940,0watchdog/11862-21python21:40:531
169940,0watchdog/118286-21ssh23:35:321
169940,0watchdog/116164-21ssh00:12:281
169940,0watchdog/115040-21cut19:35:491
169940,0watchdog/111173-21kthreadcore20:45:491
169940,0watchdog/10-21swapper/123:10:481
169940,0watchdog/10-21swapper/120:20:491
169940,0watchdog/10-21swapper/119:50:491
169940,0watchdog/10-21swapper/100:15:441
149940,0migration/06568-21kthreadcore20:40:510
149940,0migration/028746-21kthreadcore22:40:470
149940,0migration/016085-21kthreadcore22:10:480
339930,0migration/310579-21runrttasks19:09:263
297119932,0cyclictest6410-21cpuspeed_turbos19:55:422
297119932,0cyclictest6133-21awk19:30:452
297119932,0cyclictest4348-21cut19:50:492
297109932,0cyclictest3514-21snmpd20:25:311
297109932,0cyclictest3514-21snmpd20:10:461
297109932,0cyclictest29071-21awk21:00:481
29715030,0irq/26-eth0-rx-0-21swapper/320:25:473
29715030,0irq/26-eth0-rx-0-21swapper/320:25:293
29715030,0irq/26-eth0-rx-0-21swapper/320:15:433
297099933,0cyclictest20204-21kworker/0:022:50:340
297099933,0cyclictest20204-21kworker/0:000:15:510
297099933,0cyclictest10839-21kworker/0:222:30:320
297099933,0cyclictest10839-21kworker/0:222:05:360
297099933,0cyclictest10839-21kworker/0:221:55:360
297099933,0cyclictest10839-21kworker/0:221:50:460
297099933,0cyclictest10839-21kworker/0:221:41:300
297099933,0cyclictest10839-21kworker/0:221:15:440
297099933,0cyclictest10839-21kworker/0:220:55:380
297099933,0cyclictest10839-21kworker/0:220:10:360
297099933,0cyclictest10839-21kworker/0:220:00:340
297099933,0cyclictest10839-21kworker/0:219:40:360
297099932,0cyclictest9922-21hostname23:50:400
297099932,0cyclictest8533-21sensors23:05:520
297099932,0cyclictest8533-21sensors23:05:520
297099932,0cyclictest30587-21needreboot19:45:490
297099930,0cyclictest4246-21munin-node19:05:450
249930,0watchdog/20-21swapper/220:56:012
169930,0watchdog/10-21swapper/120:00:451
35120,0ktimersoftd/326241-21aten2.4-expect20:10:433
29715020,0irq/26-eth0-rx-0-21swapper/319:35:353
29715020,0irq/26-eth0-rx-0-21swapper/319:30:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional