You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 04:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Apr 13, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450530,0irq/25-eth10-21swapper/318:55:213
304450530,0irq/25-eth10-21swapper/318:55:213
1979624738,0sleep10-21swapper/118:56:441
1979624738,0sleep10-21swapper/118:56:441
1964724026,0sleep00-21swapper/018:54:590
1964724026,0sleep00-21swapper/018:54:590
1985823724,0sleep20-21swapper/218:57:252
1985823724,0sleep20-21swapper/218:57:252
339960,0migration/37283-21sh00:10:533
339960,0migration/327603-21sh22:03:543
339960,0migration/324876-21sh21:00:213
339960,0migration/323001-21missed_timers22:58:563
339960,0migration/316734-21sh21:54:433
339960,0migration/316395-21sh22:14:243
339960,0migration/31052-21latency_hist19:08:393
259960,0migration/217815-21sh22:38:462
200329960,0cyclictest0-21swapper/122:28:461
200319965,0cyclictest23439-21tail19:48:550
200319960,0cyclictest0-21swapper/021:03:570
19160,0ktimersoftd/118079-21diskmemload23:25:501
149960,0migration/0645-21sh23:48:460
149960,0migration/031143-21switchtime23:24:010
149960,0migration/025742-21latency_hist22:43:380
149960,0migration/025263-21taskset21:20:060
149960,0migration/023920-21sh22:59:400
149960,0migration/023448-21sh00:00:260
149960,0migration/015874-21sh00:17:380
8215050,0irq/24-0000:00:24822-21latency_hist23:43:372
8215050,0irq/24-0000:00:1102-21python19:28:592
35150,0ktimersoftd/30-21swapper/322:56:223
339950,0migration/38211-21runrttasks22:50:323
339950,0migration/36888-21munin-node23:08:583
339950,0migration/35434-21kthreadcore21:28:553
339950,0migration/35193-21irqrtprio20:23:553
339950,0migration/35044-21kthreadcore19:33:573
339950,0migration/3501-21taskset23:07:423
339950,0migration/331799-21munin-node20:43:543
339950,0migration/330818-21taskset00:24:273
339950,0migration/330161-21fschecks_count20:18:533
339950,0migration/329235-21kthreadcore21:43:553
339950,0migration/327029-21irqrtprio22:43:533
339950,0migration/326413-21kthreadcore00:03:533
339950,0migration/326385-21cpuspeed_turbos22:23:493
339950,0migration/326370-21kthreadcore19:23:553
339950,0migration/325603-21missed_timers18:58:573
339950,0migration/322192-21kthreadcore21:58:553
339950,0migration/321879-21kthreadcore20:33:553
339950,0migration/321308-21kthreadcore21:38:563
339950,0migration/320784-21missed_timers19:43:563
339950,0migration/315017-21kthreadcore22:33:553
339950,0migration/313004-21kthreadcore23:53:553
339950,0migration/312924-21kthreadcore20:28:543
339950,0migration/311894-21cpuspeed_turbos20:53:503
339950,0migration/310579-21sh21:33:493
29725050,0irq/27-eth0-tx-8524-21munin-node00:13:542
29725050,0irq/27-eth0-tx-6559-21runrttasks21:48:552
29725050,0irq/27-eth0-tx-3542-21lldpd22:49:382
29725050,0irq/27-eth0-tx-3514-21snmpd19:38:560
29725050,0irq/27-eth0-tx-32003-21ssh23:04:542
29725050,0irq/27-eth0-tx-26790-21diskstats21:03:522
29725050,0irq/27-eth0-tx-26594-21munin-node21:43:522
29725050,0irq/27-eth0-tx-24819-21munin-node00:04:002
29725050,0irq/27-eth0-tx-2466-21munin-node21:28:562
29725050,0irq/27-eth0-tx-18714-21munin-node21:18:522
29725050,0irq/27-eth0-tx-17690-21ssh21:57:312
29725050,0irq/27-eth0-tx-16813-21munin-node23:58:482
29725050,0irq/27-eth0-tx-1160-21munin-node23:09:002
29725050,0irq/27-eth0-tx-0-21swapper/223:30:462
29725050,0irq/27-eth0-tx-0-21swapper/222:47:062
29725050,0irq/27-eth0-tx-0-21swapper/221:38:532
29725050,0irq/27-eth0-tx-0-21swapper/020:38:560
29715050,0irq/26-eth0-rx-648-21ssh23:48:463
29715050,0irq/26-eth0-rx-32655-21ssh23:27:303
29715050,0irq/26-eth0-rx-27392-21interrupts21:03:553
29715050,0irq/26-eth0-rx-20537-21kthreadcore23:18:543
29715050,0irq/26-eth0-rx-20520-21kthreadcore22:38:563
29715050,0irq/26-eth0-rx-14955-21sshd19:40:063
29715050,0irq/26-eth0-rx-0-21swapper/323:59:203
29715050,0irq/26-eth0-rx-0-21swapper/323:44:413
29715050,0irq/26-eth0-rx-0-21swapper/323:38:563
29715050,0irq/26-eth0-rx-0-21swapper/322:08:563
29715050,0irq/26-eth0-rx-0-21swapper/321:50:593
29715050,0irq/26-eth0-rx-0-21swapper/321:18:493
29715050,0irq/26-eth0-rx-0-21swapper/321:16:513
259950,0migration/225278-21latency_hist23:23:372
259950,0migration/222195-21munin-node19:23:532
259950,0migration/220263-21kthreadcore19:43:552
259950,0migration/218143-21aten_r1power_cu22:18:492
259950,0migration/216243-21sh22:14:012
259950,0migration/211857-21munin-node21:33:542
259950,0migration/211240-21kthreadcore23:13:532
249950,0watchdog/26674-21ntp_offset19:08:572
249950,0watchdog/230334-21kthreadcore21:23:562
249950,0watchdog/214527-21cat23:33:562
200329955,0cyclictest0-21swapper/122:36:201
200329950,0cyclictest0-21swapper/123:49:381
200329950,0cyclictest0-21swapper/122:49:341
200319954,0cyclictest3787-21dump-pmu-power20:23:380
179950,0migration/12900-21hwlatdetect19:33:541
179950,0migration/125302-21kthreadcore20:13:551
179950,0migration/124923-21sh22:20:521
179950,0migration/116486-21sh00:18:451
179950,0migration/115643-21unixbench_singl22:54:021
179950,0migration/111455-21kthreadcore00:13:551
179950,0migration/110558-21latency_hist20:28:381
179950,0migration/110551-21fschecks_count22:13:501
169950,0watchdog/15715-21cut23:28:561
169950,0watchdog/110597-21if_err_eth020:03:521
149950,0migration/08888-21sh22:31:460
149950,0migration/0772-21needreboot19:28:550
149950,0migration/05767-21kthreadcore00:08:570
149950,0migration/05109-21kthreadcore19:33:570
149950,0migration/04942-21aten_r1power_cu20:48:490
149950,0migration/04911-21if_eth020:23:540
149950,0migration/03691-21kthreadcore21:08:560
149950,0migration/031473-21sh23:44:380
149950,0migration/029481-21kthreadcore21:23:560
149950,0migration/028091-21kthreadcore22:03:540
149950,0migration/026671-21kthreadcore20:13:550
149950,0migration/025612-21hwlatdetect00:03:500
149950,0migration/025438-21sh21:41:050
149950,0migration/023709-21taskset23:19:370
149950,0migration/022880-21latency_hist20:13:380
149950,0migration/02088-21aten_r1power_fr22:08:500
149950,0migration/02032-21latency_hist21:48:380
149950,0migration/019560-21kthreadcore23:38:560
149950,0migration/019476-21irqcore20:58:540
149950,0migration/019232-21munin-node21:58:520
149950,0migration/018592-21irqrtprio20:33:540
149950,0migration/016109-21fschecks_time19:43:510
149950,0migration/016009-21taskset23:15:370
149950,0migration/015847-21taskset20:05:010
149950,0migration/015119-21kthreadcore20:28:550
149950,0migration/013441-21kthreadcore21:13:550
149950,0migration/011797-21munin-node20:53:500
149950,0migration/010349-21sh22:13:490
339940,0migration/37508-21sh23:29:583
339940,0migration/332374-21kthreadcore19:28:543
339940,0migration/326148-21kthreadcore19:48:563
339940,0migration/323037-21latency_hist20:13:383
339940,0migration/322191-21kthreadcore22:18:553
29725040,0irq/27-eth0-tx-8456-21ssh21:09:042
29725040,0irq/27-eth0-tx-26917-21munin-node19:03:540
29725040,0irq/27-eth0-tx-20400-21telnet18:58:510
29725040,0irq/27-eth0-tx-1964-21ssh22:08:412
29725040,0irq/27-eth0-tx-19396-21ssh22:58:542
29725040,0irq/27-eth0-tx-17490-21munin-node23:38:522
29725040,0irq/27-eth0-tx-0-21swapper/222:53:532
29725040,0irq/27-eth0-tx-0-21swapper/222:28:512
29725040,0irq/27-eth0-tx-0-21swapper/222:23:422
29725040,0irq/27-eth0-tx-0-21swapper/221:58:422
29725040,0irq/27-eth0-tx-0-21swapper/221:00:292
29725040,0irq/27-eth0-tx-0-21swapper/200:23:412
29725040,0irq/27-eth0-tx-0-21swapper/200:18:452
29725040,0irq/27-eth0-tx-0-21swapper/200:08:482
29725040,0irq/27-eth0-tx-0-21swapper/019:23:370
29715040,0irq/26-eth0-rx-0-21swapper/323:35:513
29715040,0irq/26-eth0-rx-0-21swapper/323:14:203
29715040,0irq/26-eth0-rx-0-21swapper/322:29:123
29715040,0irq/26-eth0-rx-0-21swapper/321:23:503
29715040,0irq/26-eth0-rx-0-21swapper/321:08:563
29715040,0irq/26-eth0-rx-0-21swapper/320:48:493
29715040,0irq/26-eth0-rx-0-21swapper/320:08:563
29715040,0irq/26-eth0-rx-0-21swapper/319:03:553
29715040,0irq/26-eth0-rx-0-21swapper/300:18:493
29715040,0irq/26-eth0-rx-0-21swapper/300:18:203
259940,0migration/2670-21kthreadcore20:43:552
259940,0migration/26241-21kthreadcore19:33:572
259940,0migration/23643-21kthreadcore23:48:552
259940,0migration/227178-21cpuspeed_turbos19:03:502
259940,0migration/227017-21kthreadcore20:13:552
259940,0migration/223722-21kthreadcore18:58:562
259940,0migration/222619-21fschecks_count19:48:522
259940,0migration/220374-21timerwakeupswit19:19:002
259940,0migration/220053-21kthreadcore23:18:542
259940,0migration/214295-21turbostat19:18:392
259940,0migration/212228-21kthreadcore20:03:552
259940,0migration/210777-21aten_r1power_fr20:28:492
249940,0watchdog/23862-21hwlatdetect19:58:522
249940,0watchdog/20-21swapper/219:38:562
200349943,0cyclictest4895-21awk20:23:532
200349943,0cyclictest3514-21snmpd20:50:042
200349943,0cyclictest3514-21snmpd19:56:482
200349943,0cyclictest3241-21irqbalance23:53:462
200349943,0cyclictest30642-21awk20:18:552
200349943,0cyclictest27344-21awk22:03:522
200349943,0cyclictest18333-21awk20:33:532
200349943,0cyclictest17331-21awk20:08:542
200349943,0cyclictest12612-21awk20:53:542
200349943,0cyclictest11318-21awk21:13:522
200349943,0cyclictest10658-21awk22:33:522
200329944,0cyclictest0-21swapper/123:57:311
200329944,0cyclictest0-21swapper/122:59:491
200329944,0cyclictest0-21swapper/122:12:031
200329944,0cyclictest0-21swapper/121:56:171
200329944,0cyclictest0-21swapper/119:04:071
200329944,0cyclictest0-21swapper/100:07:181
200329943,0cyclictest8864-21awk19:13:531
200329943,0cyclictest3556-21lldpd21:44:371
200329943,0cyclictest3556-21lldpd00:10:091
200329943,0cyclictest3514-21snmpd20:30:041
200329943,0cyclictest3514-21snmpd20:09:031
200329943,0cyclictest3514-21snmpd19:39:031
200329943,0cyclictest3064-21awk21:48:521
200329943,0cyclictest27819-21awk21:23:541
200329943,0cyclictest27114-21awk22:23:521
200329943,0cyclictest25980-21awk23:43:521
200329943,0cyclictest22205-21irqcore19:23:531
200329943,0cyclictest21197-21awk18:58:541
200329943,0cyclictest19223-21awk20:58:521
200329943,0cyclictest11284-21awk21:13:521
200329943,0cyclictest10206-21awk23:13:511
200329942,0cyclictest18117-21irqcore23:38:551
200319943,0cyclictest3514-21snmpd19:18:320
200319943,0cyclictest3241-21irqbalance23:38:260
200319943,0cyclictest2941-21kworker/0:121:28:510
200319943,0cyclictest2060-21awk23:08:520
200319943,0cyclictest1971-21awk19:08:510
179940,0migration/16079-21kthreadcore21:28:551
179940,0migration/13754-21kthreadcore23:08:541
179940,0migration/129402-21kthreadcore19:28:531
179940,0migration/127661-21kthreadcore20:38:571
179940,0migration/127366-21kthreadcore23:03:571
179940,0migration/126627-21kthreadcore00:23:551
179940,0migration/123039-21kthreadcore21:58:551
179940,0migration/119786-21kthreadcore22:38:561
179940,0migration/118703-21kthreadcore19:18:551
179940,0migration/118055-21gltestperf20:33:521
179940,0migration/115717-21cpu19:43:501
179940,0migration/111707-21kthreadcore23:33:541
169940,0watchdog/129479-21kthreadcore21:03:561
169940,0watchdog/127429-21memory19:48:561
169940,0watchdog/11315-21ssh22:07:281
169940,0watchdog/10-21swapper/122:45:521
169940,0watchdog/10-21swapper/121:38:561
169940,0watchdog/10-21swapper/121:18:441
169940,0watchdog/10-21swapper/121:08:561
169940,0watchdog/10-21swapper/120:18:561
169940,0watchdog/10-21swapper/119:08:571
149940,0migration/08582-21sh21:49:260
149940,0migration/020990-21kthreadcore22:18:550
149940,0migration/017301-21runrttasks21:35:500
149940,0migration/015605-21runrttasks23:55:560
149940,0migration/011381-21irqcore21:53:540
149940,0migration/011071-21irqrtprio22:33:540
149940,0migration/010107-21fschecks_count22:53:520
339930,0migration/324024-21munin-node20:38:513
29715030,0irq/26-eth0-rx-0-21swapper/319:58:403
29715030,0irq/26-eth0-rx-0-21swapper/319:19:113
29715030,0irq/26-eth0-rx-0-21swapper/319:13:493
200349932,0cyclictest3514-21snmpd20:38:492
200329932,0cyclictest31296-21grep20:43:521
200329932,0cyclictest18293-21awk23:18:521
200329932,0cyclictest17836-21irqcore23:58:531
200329932,0cyclictest15176-21tr21:33:551
200329931,0cyclictest1545-21cut19:53:551
200319933,0cyclictest2941-21kworker/0:123:28:390
200319933,0cyclictest2941-21kworker/0:123:03:410
200319933,0cyclictest2941-21kworker/0:122:48:450
200319933,0cyclictest2941-21kworker/0:122:38:390
200319933,0cyclictest2941-21kworker/0:122:23:410
200319933,0cyclictest25883-21kworker/0:000:23:390
200319933,0cyclictest25883-21kworker/0:000:18:470
200319933,0cyclictest23119-21kworker/0:020:43:390
200319933,0cyclictest23119-21kworker/0:019:58:400
200319933,0cyclictest23119-21kworker/0:019:53:520
200319933,0cyclictest23119-21kworker/0:019:23:420
169930,0watchdog/10-21swapper/119:58:521
329920,0watchdog/315732-21uptime20:04:013
29715020,0irq/26-eth0-rx-0-21swapper/319:53:523
200329921,0cyclictest3241-21irqbalance20:48:461
200329921,0cyclictest11425-21latency_hist20:53:381
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional