You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 05:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat May 30, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450560,0irq/25-eth10-21swapper/318:53:003
1810823526,0sleep20-21swapper/218:51:542
1813723424,0sleep10-21swapper/118:52:171
1811123423,0sleep00-21swapper/018:51:560
339960,0migration/323629-21sh23:38:063
339960,0migration/322108-21missed_timers22:54:283
339960,0migration/315707-21sh21:50:343
29715060,0irq/26-eth0-rx-7975-21sh21:06:453
29715060,0irq/26-eth0-rx-22227-21python23:14:313
29715060,0irq/26-eth0-rx-15512-21unixbench_singl21:29:343
29715060,0irq/26-eth0-rx-10681-21kthreadcore23:09:273
183759960,0cyclictest0-21swapper/221:31:342
183759960,0cyclictest0-21swapper/220:56:432
149960,0migration/07452-21sh00:08:460
149960,0migration/022765-21sh23:35:400
149960,0migration/015872-21sh00:14:140
149960,0migration/015692-21sh21:50:320
8215050,0irq/24-0000:00:27888-21awk20:14:112
8215050,0irq/24-0000:00:25041-21latency_hist21:59:112
4150,0ktimersoftd/00-21swapper/023:29:350
35150,0ktimersoftd/30-21swapper/323:22:053
339950,0migration/39041-21latency_hist20:24:113
339950,0migration/331557-21taskset21:59:513
339950,0migration/33038-21taskset20:43:323
339950,0migration/330313-21latency20:59:283
339950,0migration/329184-21kthreadcore21:39:273
339950,0migration/328577-21taskset20:35:543
339950,0migration/327624-21fschecks_count19:49:253
339950,0migration/326823-21munin-node20:09:293
339950,0migration/325892-21cpuspeed_turbos21:19:243
339950,0migration/325106-21idleruntime23:39:243
339950,0migration/324806-21aten_r1power_fr22:59:193
339950,0migration/32348-21kthreadcore19:04:273
339950,0migration/322659-21latency21:14:273
339950,0migration/320960-21kthreadcore23:54:313
339950,0migration/320233-21kthreadcore21:34:273
339950,0migration/319670-21kthreadcore22:14:263
339950,0migration/314504-21needreboot22:09:283
339950,0migration/31421-21switchtime20:14:333
339950,0migration/314023-21timerandwakeup23:49:343
339950,0migration/312998-21munin-node19:14:223
29725050,0irq/27-eth0-tx-30674-21ntpq22:19:312
29725050,0irq/27-eth0-tx-29093-21munin-node20:39:270
29725050,0irq/27-eth0-tx-25777-21cpuspeed_turbos21:39:222
29725050,0irq/27-eth0-tx-24280-21df00:19:232
29725050,0irq/27-eth0-tx-17580-21kthreadcore00:14:272
29725050,0irq/27-eth0-tx-16232-21munin-node23:34:272
29725050,0irq/27-eth0-tx-1010-21ssh21:44:152
29725050,0irq/27-eth0-tx-0-21swapper/223:13:352
29725050,0irq/27-eth0-tx-0-21swapper/223:03:032
29725050,0irq/27-eth0-tx-0-21swapper/222:01:542
29725050,0irq/27-eth0-tx-0-21swapper/221:37:032
29725050,0irq/27-eth0-tx-0-21swapper/020:54:250
29725050,0irq/27-eth0-tx-0-21swapper/020:49:250
29725050,0irq/27-eth0-tx-0-21swapper/020:48:040
29725050,0irq/27-eth0-tx-0-21swapper/020:34:040
29725050,0irq/27-eth0-tx-0-21swapper/020:19:210
29725050,0irq/27-eth0-tx-0-21swapper/019:10:050
29725050,0irq/27-eth0-tx-0-21swapper/018:58:040
29715050,0irq/26-eth0-rx-32414-21cpuspeed_turbos00:04:223
29715050,0irq/26-eth0-rx-32319-21munin-node23:44:213
29715050,0irq/26-eth0-rx-31622-21taskset00:03:293
29715050,0irq/26-eth0-rx-30995-21sh22:19:343
29715050,0irq/26-eth0-rx-27181-21kthreadcore22:39:263
29715050,0irq/26-eth0-rx-23435-21ssh21:54:353
29715050,0irq/26-eth0-rx-18849-21cut18:54:233
29715050,0irq/26-eth0-rx-16975-21cut22:34:213
29715050,0irq/26-eth0-rx-1064-21ssh23:24:263
29715050,0irq/26-eth0-rx-0-21swapper/323:06:113
29715050,0irq/26-eth0-rx-0-21swapper/322:52:403
29715050,0irq/26-eth0-rx-0-21swapper/322:44:253
29715050,0irq/26-eth0-rx-0-21swapper/322:04:183
29715050,0irq/26-eth0-rx-0-21swapper/320:54:233
29715050,0irq/26-eth0-rx-0-21swapper/300:14:273
29715050,0irq/26-eth0-rx-0-21swapper/300:09:293
259950,0migration/24401-21munin-node20:44:262
259950,0migration/216730-21sh21:13:192
259950,0migration/21533-21kthreadcore19:04:262
249950,0watchdog/214953-21unixbench_singl22:29:332
249950,0watchdog/20-21swapper/223:50:012
19150,0ktimersoftd/10-21swapper/123:22:051
183759955,0cyclictest16541-21diskmemload23:39:162
183759955,0cyclictest0-21swapper/222:49:462
183759955,0cyclictest0-21swapper/222:26:352
183759954,0cyclictest1481-21awk22:44:252
183759954,0cyclictest1102-21ps23:44:272
183759954,0cyclictest10278-21tr20:24:272
183759950,0cyclictest0-21swapper/220:41:352
183759950,0cyclictest0-21swapper/200:05:072
183759950,0cyclictest0-21swapper/200:03:422
183749955,0cyclictest0-21swapper/121:58:361
183749955,0cyclictest0-21swapper/121:01:231
183749955,0cyclictest0-21swapper/119:42:331
183749954,0cyclictest3241-21irqbalance22:05:191
183749950,0cyclictest0-21swapper/123:07:021
183749950,0cyclictest0-21swapper/122:41:391
183749950,0cyclictest0-21swapper/122:16:541
183749950,0cyclictest0-21swapper/100:11:021
183739954,0cyclictest1192-21sadc19:54:110
179950,0migration/1883-21aten_r1power_fr21:04:221
179950,0migration/113803-21munin-node19:59:321
169950,0watchdog/122630-21kthreadcore19:44:251
149950,0migration/08946-21runrttasks21:28:350
149950,0migration/08631-21sh22:08:450
149950,0migration/07694-21taskset22:26:360
149950,0migration/05707-21ksmtuned19:06:330
149950,0migration/04897-21kthreadcore23:44:280
149950,0migration/03475-21kthreadcore19:54:270
149950,0migration/030344-21sh23:59:480
149950,0migration/028840-21kthreadcore00:19:280
149950,0migration/02796-21kthreadcore19:29:270
149950,0migration/027527-21kthreadcore22:59:250
149950,0migration/025201-21if_eth023:39:250
149950,0migration/024238-21latency_hist23:19:100
149950,0migration/019626-21kthreadcore22:14:260
149950,0migration/01671-21fschecks_time21:44:240
149950,0migration/016366-21cpuspeed_turbos23:54:250
149950,0migration/016299-21kthreadcore19:14:270
149950,0migration/015638-21turbostat.cron00:14:100
149950,0migration/014551-21fschecks_time19:39:240
149950,0migration/014143-21kthreadcore20:24:280
149950,0migration/013508-21munin-node23:49:300
149950,0migration/012654-21kthreadcore22:09:260
149950,0migration/011775-21kthreadcore21:09:280
149950,0migration/010777-21kthreadcore22:29:260
149950,0migration/010156-21gltestperf21:29:240
339940,0migration/35663-21kthreadcore20:44:273
339940,0migration/319905-21taskset19:40:083
339940,0migration/316327-21kthreadcore20:04:273
339940,0migration/312018-21timerwakeupswit19:09:323
339940,0migration/310920-21hwlatdetect20:49:253
339940,0migration/310796-21kthreadcore22:29:263
29725040,0irq/27-eth0-tx-0-21swapper/223:28:222
29725040,0irq/27-eth0-tx-0-21swapper/223:04:262
29725040,0irq/27-eth0-tx-0-21swapper/222:34:242
29725040,0irq/27-eth0-tx-0-21swapper/222:15:222
29725040,0irq/27-eth0-tx-0-21swapper/222:04:202
29725040,0irq/27-eth0-tx-0-21swapper/221:49:252
29725040,0irq/27-eth0-tx-0-21swapper/221:25:522
29725040,0irq/27-eth0-tx-0-21swapper/221:04:342
29725040,0irq/27-eth0-tx-0-21swapper/200:09:232
29725040,0irq/27-eth0-tx-0-21swapper/019:34:220
29725040,0irq/27-eth0-tx-0-21swapper/018:59:220
29715040,0irq/26-eth0-rx-0-21swapper/323:29:353
29715040,0irq/26-eth0-rx-0-21swapper/322:25:403
29715040,0irq/26-eth0-rx-0-21swapper/321:44:303
29715040,0irq/26-eth0-rx-0-21swapper/321:27:183
29715040,0irq/26-eth0-rx-0-21swapper/321:09:523
29715040,0irq/26-eth0-rx-0-21swapper/320:00:593
29715040,0irq/26-eth0-rx-0-21swapper/319:44:123
29715040,0irq/26-eth0-rx-0-21swapper/300:19:343
259940,0migration/29969-21kthreadcore19:34:282
259940,0migration/24587-21kthreadcore19:54:272
259940,0migration/2384-21runrttasks19:29:142
259940,0migration/231047-21missed_timers21:19:302
259940,0migration/226643-21fschecks_count19:24:232
259940,0migration/225684-21irqrtprio23:19:262
259940,0migration/222551-21aten_r1power_vo20:34:222
259940,0migration/220813-21kthreadcore18:54:282
259940,0migration/220147-21runrttasks19:42:552
259940,0migration/217569-21kthreadcore20:04:272
249940,0watchdog/231425-21ssh22:40:332
249940,0watchdog/230472-21munin-node20:59:292
249940,0watchdog/216129-21munin-node23:54:282
249940,0watchdog/20-21swapper/223:15:052
183759943,0cyclictest9898-21awk22:09:232
183759943,0cyclictest9165-21awk23:29:252
183759943,0cyclictest3556-21lldpd19:13:042
183759943,0cyclictest3514-21snmpd20:50:582
183759943,0cyclictest3514-21snmpd20:15:432
183759943,0cyclictest3514-21snmpd20:01:582
183759943,0cyclictest3514-21snmpd19:21:282
183759943,0cyclictest3514-21snmpd19:15:132
183759943,0cyclictest3241-21irqbalance19:44:302
183759943,0cyclictest28166-21awk19:49:272
183759943,0cyclictest18276-21awk21:14:242
183759943,0cyclictest17596-21awk22:54:242
183749944,0cyclictest0-21swapper/123:34:561
183749944,0cyclictest0-21swapper/123:13:011
183749944,0cyclictest0-21swapper/122:56:281
183749944,0cyclictest0-21swapper/121:42:411
183749943,0cyclictest7181-21awk19:09:251
183749943,0cyclictest3556-21lldpd23:40:361
183749943,0cyclictest3556-21lldpd23:15:361
183749943,0cyclictest3556-21lldpd23:04:061
183749943,0cyclictest3556-21lldpd22:35:361
183749943,0cyclictest3556-21lldpd22:28:361
183749943,0cyclictest3556-21lldpd21:49:361
183749943,0cyclictest3556-21lldpd19:54:341
183749943,0cyclictest3556-21lldpd19:22:041
183749943,0cyclictest3514-21snmpd20:26:131
183749943,0cyclictest3514-21snmpd18:56:571
183749943,0cyclictest3241-21irqbalance22:03:591
183749943,0cyclictest3241-21irqbalance21:32:191
183749943,0cyclictest3241-21irqbalance21:25:491
183749943,0cyclictest27882-21sadc20:14:111
183749943,0cyclictest24896-21awk00:19:251
183749943,0cyclictest16811-21awk00:14:251
183749943,0cyclictest16534-21sadc20:54:111
183749943,0cyclictest15571-21grep20:04:241
183749943,0cyclictest10018-21awk22:09:241
183739943,0cyclictest5342-21kworker/0:022:50:240
183739943,0cyclictest5342-21kworker/0:022:45:280
183739943,0cyclictest3556-21lldpd23:27:060
183739943,0cyclictest3556-21lldpd22:43:060
183739943,0cyclictest3241-21irqbalance21:20:490
183739943,0cyclictest21374-21awk19:44:240
183739943,0cyclictest18218-21awk21:54:250
183739943,0cyclictest17853-21awk22:34:250
183739943,0cyclictest15698-21awk20:04:250
179940,0migration/1695-21idleruntime23:44:251
179940,0migration/130959-21kthreadcore20:39:271
179940,0migration/129503-21kthreadcore19:49:291
179940,0migration/128634-21kthreadcore23:59:261
179940,0migration/128099-21latency_hist20:14:111
179940,0migration/127243-21sh21:19:291
179940,0migration/125345-21aten_r1power_po18:59:211
179940,0migration/118917-21kthreadcore21:34:271
179940,0migration/115677-21taskset23:53:431
179940,0migration/112633-21kthreadcore22:49:301
179940,0migration/11041-21sh21:44:201
169940,0watchdog/18947-21munin-node22:29:211
169940,0watchdog/16879-21ssh00:07:041
169940,0watchdog/15955-21sh23:24:331
169940,0watchdog/127880-21ntp_offset20:34:291
169940,0watchdog/114998-21smartctl21:09:331
169940,0watchdog/10-21swapper/122:19:251
149940,0migration/031918-21runrttasks22:21:290
149940,0migration/026434-21kthreadcore20:34:270
149940,0migration/024053-21runrttasks21:15:570
149940,0migration/020969-21kthreadcore21:34:270
149940,0migration/019477-21latency_hist19:19:120
149940,0migration/010841-21kthreadcore23:09:270
339930,0migration/38345-21unixbench-2d20:19:333
339930,0migration/32744-21kthreadcore19:29:273
29725030,0irq/27-eth0-tx-0-21swapper/020:14:220
29725030,0irq/27-eth0-tx-0-21swapper/019:24:210
29715030,0irq/26-eth0-rx-0-21swapper/320:29:333
29715030,0irq/26-eth0-rx-0-21swapper/319:34:303
29715030,0irq/26-eth0-rx-0-21swapper/319:23:293
29715030,0irq/26-eth0-rx-0-21swapper/318:59:233
249930,0watchdog/20-21swapper/220:19:132
249930,0watchdog/20-21swapper/218:59:222
183759932,0cyclictest3556-21lldpd20:29:342
183749932,0cyclictest7159-21cut19:34:121
183749932,0cyclictest3241-21irqbalance23:54:191
183749932,0cyclictest26289-21basename19:24:221
183749932,0cyclictest16667-21awk20:29:261
183749932,0cyclictest13724-21awk19:14:251
183749932,0cyclictest1103-21grep19:29:241
183749932,0cyclictest10560-21cut23:29:261
183739933,0cyclictest5342-21kworker/0:023:19:140
183739933,0cyclictest5342-21kworker/0:023:04:360
183739933,0cyclictest5342-21kworker/0:022:54:200
183739933,0cyclictest5342-21kworker/0:021:39:280
183739933,0cyclictest5342-21kworker/0:021:04:180
183739933,0cyclictest5342-21kworker/0:020:59:220
183739933,0cyclictest5342-21kworker/0:020:09:250
183739932,0cyclictest5342-21kworker/0:021:59:160
183739930,0cyclictest0-21swapper/020:01:330
169930,0watchdog/10-21swapper/122:44:291
169930,0watchdog/10-21swapper/121:15:171
169930,0watchdog/10-21swapper/120:54:251
169930,0watchdog/10-21swapper/120:19:131
169930,0watchdog/10-21swapper/119:04:221
29715020,0irq/26-eth0-rx-0-21swapper/319:54:223
29715020,0irq/26-eth0-rx-0-21swapper/319:24:323
183749921,0cyclictest3384-21aten_r1power_cu20:44:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional