You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 03:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Apr 19, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:53:233
2944623425,0sleep20-21swapper/218:56:022
2929423026,0sleep10-21swapper/118:54:171
2950222824,0sleep00-21swapper/018:56:400
4160,0ktimersoftd/00-21swapper/023:38:240
35160,0ktimersoftd/30-21swapper/323:38:243
339960,0migration/32594-21sh22:01:113
29715060,0irq/26-eth0-rx-737-21kthreadcore18:58:213
29715060,0irq/26-eth0-rx-28974-21irqstats22:18:183
29715060,0irq/26-eth0-rx-27411-21turbostat22:18:043
29715060,0irq/26-eth0-rx-26194-21aten2.4-expect20:08:143
29715060,0irq/26-eth0-rx-19844-21diskstats22:33:163
296939960,0cyclictest0-21swapper/221:33:512
296939960,0cyclictest0-21swapper/221:07:032
296939960,0cyclictest0-21swapper/219:23:282
296929960,0cyclictest0-21swapper/121:28:171
296929960,0cyclictest0-21swapper/119:48:191
296919965,0cyclictest0-21swapper/023:01:500
296919963,0cyclictest18314-21munin-run23:33:030
259960,0migration/29873-21timerwakeupswit21:43:262
179960,0migration/126355-21sh22:35:421
149960,0migration/032709-21sh23:59:010
149960,0migration/02954-21sh21:01:480
149960,0migration/026041-21sh00:18:060
149960,0migration/011123-21sh22:07:180
339950,0migration/36682-21kthreadcore22:43:223
339950,0migration/35069-21kthreadcore00:03:193
339950,0migration/35026-21kthreadcore20:38:213
339950,0migration/32911-21kthreadcore20:13:203
339950,0migration/328756-21df_abs21:18:163
339950,0migration/328296-21aten_r1power_po21:38:153
339950,0migration/32829-21cpuspeed_turbos23:23:153
339950,0migration/327956-21latency_hist20:58:043
339950,0migration/326512-21kthreadcore20:53:223
339950,0migration/324245-21timerandwakeup23:53:253
339950,0migration/323202-21kthreadcore21:33:213
339950,0migration/322890-21kthreadcore20:03:203
339950,0migration/31616-21kthreadcore19:48:223
339950,0migration/314776-21kthreadcore23:48:213
339950,0migration/314493-21hddtemp_smartct20:23:193
339950,0migration/314300-21kthreadcore19:58:193
339950,0migration/314197-21kthreadcore21:48:213
339950,0migration/311001-21taskset22:06:583
29725050,0irq/27-eth0-tx-28419-21munin-node21:18:192
29725050,0irq/27-eth0-tx-2199-21ssh23:02:082
29725050,0irq/27-eth0-tx-12252-21munin-node21:28:252
29725050,0irq/27-eth0-tx-0-21swapper/223:23:582
29725050,0irq/27-eth0-tx-0-21swapper/222:18:132
29725050,0irq/27-eth0-tx-0-21swapper/222:15:402
29725050,0irq/27-eth0-tx-0-21swapper/222:05:322
29725050,0irq/27-eth0-tx-0-21swapper/221:57:072
29725050,0irq/27-eth0-tx-0-21swapper/221:23:192
29725050,0irq/27-eth0-tx-0-21swapper/221:09:112
29725050,0irq/27-eth0-tx-0-21swapper/200:08:212
29725050,0irq/27-eth0-tx-0-21swapper/020:41:580
29725050,0irq/27-eth0-tx-0-21swapper/019:13:060
29715050,0irq/26-eth0-rx-9966-21rt-features21:23:253
29715050,0irq/26-eth0-rx-8390-21ntpdc23:03:223
29715050,0irq/26-eth0-rx-625-21timerandwakeup22:58:263
29715050,0irq/26-eth0-rx-386-21ssh23:59:263
29715050,0irq/26-eth0-rx-21994-21kthreadcore23:13:213
29715050,0irq/26-eth0-rx-20292-21df_abs21:53:173
29715050,0irq/26-eth0-rx-17792-21yum22:08:293
29715050,0irq/26-eth0-rx-17489-21ssh22:48:553
29715050,0irq/26-eth0-rx-15680-21kthreadcore23:28:213
29715050,0irq/26-eth0-rx-12113-21kthreadcore00:08:193
29715050,0irq/26-eth0-rx-10411-21ssh21:44:323
29715050,0irq/26-eth0-rx-0-21swapper/323:45:493
29715050,0irq/26-eth0-rx-0-21swapper/323:11:423
29715050,0irq/26-eth0-rx-0-21swapper/322:23:213
29715050,0irq/26-eth0-rx-0-21swapper/321:14:453
29715050,0irq/26-eth0-rx-0-21swapper/321:08:043
29715050,0irq/26-eth0-rx-0-21swapper/321:03:463
29715050,0irq/26-eth0-rx-0-21swapper/319:38:213
296939955,0cyclictest0-21swapper/223:53:132
296939955,0cyclictest0-21swapper/222:02:042
296929955,0cyclictest0-21swapper/123:58:221
296929955,0cyclictest0-21swapper/123:49:421
296929955,0cyclictest0-21swapper/123:25:351
296929955,0cyclictest0-21swapper/123:16:541
296929955,0cyclictest0-21swapper/122:22:121
296929955,0cyclictest0-21swapper/119:03:331
296929950,0cyclictest0-21swapper/122:57:211
296929950,0cyclictest0-21swapper/120:58:101
296919950,0cyclictest0-21swapper/022:38:110
296919950,0cyclictest0-21swapper/021:36:140
27150,0ktimersoftd/20-21swapper/223:38:242
27150,0ktimersoftd/20-21swapper/221:01:222
259950,0migration/27660-21fschecks_time20:18:152
259950,0migration/27501-21kthreadcore19:53:222
259950,0migration/26800-21kthreadcore22:43:222
259950,0migration/227002-21fschecks_time23:58:162
259950,0migration/226685-21sh21:14:222
259950,0migration/223896-21sh23:33:192
259950,0migration/212500-21sh22:48:212
259950,0migration/211841-21aten_r1power_cu19:33:152
249950,0watchdog/28380-21tr22:23:222
249950,0watchdog/23227-21sh21:42:062
19150,0ktimersoftd/10-21swapper/123:42:591
179950,0migration/15511-21kthreadcore23:43:221
179950,0migration/126714-21sh21:14:251
179950,0migration/11473-21sh22:39:171
179950,0migration/113256-21kthreadcore23:08:211
169950,0watchdog/14250-21sort22:03:181
169950,0watchdog/113596-21cat22:48:221
149950,0migration/09593-21sendmail_mailst21:03:250
149950,0migration/08803-21munin-node22:23:220
149950,0migration/05649-21kthreadcore19:03:210
149950,0migration/04346-21fschecks_count21:43:170
149950,0migration/04330-21irqrtprio22:43:200
149950,0migration/03341-21fschecks_count23:03:160
149950,0migration/03242-21switchtime18:58:260
149950,0migration/032148-21taskset20:08:460
149950,0migration/031755-21kthreadcore21:58:200
149950,0migration/03099-21fschecks_time23:23:160
149950,0migration/03056-21kthreadcore19:48:230
149950,0migration/028827-21fschecks_time21:38:170
149950,0migration/027914-21munin-node20:33:190
149950,0migration/021139-21kthreadcore22:33:200
149950,0migration/02046-21aten_r1power_fr00:03:130
149950,0migration/020227-21kthreadcore23:13:200
149950,0migration/019736-21kthreadcore23:53:190
149950,0migration/018953-21taskset20:00:060
149950,0migration/018280-21taskset23:12:090
149950,0migration/017290-21kthreadcore21:28:210
149950,0migration/01622-21runrttasks00:23:040
149950,0migration/01559-21taskset22:18:510
149950,0migration/015191-21kthreadcore00:08:200
149950,0migration/013845-21kthreadcore19:33:210
149950,0migration/012458-21needreboot20:18:210
149950,0migration/011571-21taskset19:32:520
149950,0migration/010262-21kthreadcore20:43:220
4140,0ktimersoftd/00-21swapper/021:24:150
339940,0migration/33127-21kthreadcore00:23:193
339940,0migration/329289-21kthreadcore19:43:203
339940,0migration/329140-21kthreadcore00:18:213
339940,0migration/326451-21taskset20:29:073
339940,0migration/314680-21taskset20:48:083
339940,0migration/310676-21pmu-power19:28:243
29725040,0irq/27-eth0-tx-32755-21munin-node20:13:280
29725040,0irq/27-eth0-tx-2328-21fschecks_time00:23:162
29725040,0irq/27-eth0-tx-20217-21munin-node22:33:172
29725040,0irq/27-eth0-tx-12180-21idleruntime22:28:162
29725040,0irq/27-eth0-tx-0-21swapper/223:18:182
29725040,0irq/27-eth0-tx-0-21swapper/223:13:152
29725040,0irq/27-eth0-tx-0-21swapper/222:53:102
29725040,0irq/27-eth0-tx-0-21swapper/222:38:152
29725040,0irq/27-eth0-tx-0-21swapper/221:48:452
29725040,0irq/27-eth0-tx-0-21swapper/200:19:352
29725040,0irq/27-eth0-tx-0-21swapper/200:13:062
29725040,0irq/27-eth0-tx-0-21swapper/200:03:182
29725040,0irq/27-eth0-tx-0-21swapper/020:23:180
29725040,0irq/27-eth0-tx-0-21swapper/019:43:120
29725040,0irq/27-eth0-tx-0-21swapper/019:18:070
29715040,0irq/26-eth0-rx-0-21swapper/323:33:123
29715040,0irq/26-eth0-rx-0-21swapper/323:19:523
29715040,0irq/26-eth0-rx-0-21swapper/322:53:413
29715040,0irq/26-eth0-rx-0-21swapper/322:38:143
29715040,0irq/26-eth0-rx-0-21swapper/322:28:383
29715040,0irq/26-eth0-rx-0-21swapper/321:28:093
29715040,0irq/26-eth0-rx-0-21swapper/320:33:203
29715040,0irq/26-eth0-rx-0-21swapper/320:19:333
29715040,0irq/26-eth0-rx-0-21swapper/319:33:163
29715040,0irq/26-eth0-rx-0-21swapper/319:13:213
29715040,0irq/26-eth0-rx-0-21swapper/300:14:193
296939943,0cyclictest32722-21awk19:48:212
296939943,0cyclictest22220-21awk20:53:192
296939943,0cyclictest2167-21awk20:38:182
296939943,0cyclictest19434-21awk19:38:192
296939943,0cyclictest13682-21awk19:58:182
296939943,0cyclictest12570-21awk22:08:192
296939943,0cyclictest1249-21awk20:13:182
296939940,0cyclictest0-21swapper/220:49:042
296929943,0cyclictest4748-21awk21:43:181
296929943,0cyclictest4649-21awk21:03:181
296929943,0cyclictest4126-21awk22:43:201
296929943,0cyclictest3556-21lldpd23:30:331
296929943,0cyclictest3556-21lldpd20:39:031
296929943,0cyclictest3556-21lldpd20:29:331
296929943,0cyclictest3556-21lldpd20:19:331
296929943,0cyclictest1322-21awk20:13:191
296929943,0cyclictest12555-21awk22:08:191
296919943,0cyclictest3556-21lldpd23:45:340
296919943,0cyclictest3556-21lldpd19:12:030
296919943,0cyclictest3514-21snmpd21:18:200
296919943,0cyclictest3514-21snmpd21:08:200
296919943,0cyclictest3241-21irqbalance22:32:220
296919943,0cyclictest3241-21irqbalance22:08:220
296919943,0cyclictest3241-21irqbalance21:57:220
296919943,0cyclictest27898-21sort23:18:190
296919943,0cyclictest19494-21awk23:33:160
259940,0migration/24500-21cpuspeed_turbos19:03:172
259940,0migration/24086-21irqrtprio23:03:192
259940,0migration/226951-21latency_hist20:33:042
259940,0migration/213831-21kthreadcore23:28:202
259940,0migration/212079-21kthreadcore23:48:202
249940,0watchdog/25940-21idleruntime19:28:182
249940,0watchdog/25114-21cat23:43:212
249940,0watchdog/226295-21munin-node20:08:142
249940,0watchdog/215060-21cut23:08:222
249940,0watchdog/20-21swapper/220:43:222
249940,0watchdog/20-21swapper/219:08:272
179940,0migration/18793-21taskset00:26:031
179940,0migration/18690-21sh22:23:221
179940,0migration/1739-21taskset20:34:571
179940,0migration/16512-21kthreadcore23:03:201
179940,0migration/126712-21fschecks_time20:08:161
179940,0migration/125495-21sh00:17:111
179940,0migration/12534-21diskmemload22:01:001
179940,0migration/12424-21kthreadcore18:58:221
179940,0migration/12294-21latency_hist23:23:041
179940,0migration/119920-21kthreadcore19:13:211
179940,0migration/118658-21kthreadcore20:23:231
179940,0migration/115065-21kthreadcore22:28:191
179940,0migration/115031-21munin-node21:48:211
179940,0migration/114591-21kthreadcore19:33:211
169940,0watchdog/14794-21taskset19:26:541
169940,0watchdog/132531-21munin-node22:58:221
169940,0watchdog/126012-21ssh22:14:061
169940,0watchdog/124715-21cut21:53:221
169940,0watchdog/12334-21ssh21:18:501
169940,0watchdog/119426-21ssh21:11:381
169940,0watchdog/118384-21aten_r1power_en23:53:131
169940,0watchdog/116600-21swap19:08:271
169940,0watchdog/111744-21kthreadcore20:43:221
169940,0watchdog/110027-21ssh21:23:261
149940,0migration/025729-21missed_timers20:28:210
149940,0migration/024927-21turbostat19:43:040
149940,0migration/023810-21kthreadcore22:13:210
149940,0migration/021434-21kthreadcore22:53:200
149940,0migration/015877-21kthreadcore22:48:220
29715030,0irq/26-eth0-rx-0-21swapper/320:43:163
29715030,0irq/26-eth0-rx-0-21swapper/319:55:143
29715030,0irq/26-eth0-rx-0-21swapper/319:23:153
29715030,0irq/26-eth0-rx-0-21swapper/319:20:233
29715030,0irq/26-eth0-rx-0-21swapper/319:08:163
296939932,0cyclictest30784-21awk18:58:192
296939932,0cyclictest25212-21irqcore19:18:212
296939932,0cyclictest20330-21cat20:28:042
296929933,0cyclictest0-21swapper/119:28:181
296929932,0cyclictest3514-21snmpd20:54:501
296929932,0cyclictest3514-21snmpd19:20:051
296929932,0cyclictest29142-21if_eth1.221:38:181
296929932,0cyclictest26060-21awk19:43:181
296929932,0cyclictest24166-21sensors19:38:251
296929932,0cyclictest20354-21awk20:03:181
296929932,0cyclictest19452-21awk23:33:161
296929932,0cyclictest15834-21sensors00:08:251
296919933,0cyclictest28231-21kworker/0:223:48:170
296919933,0cyclictest28231-21kworker/0:221:48:150
296919933,0cyclictest28231-21kworker/0:221:13:130
296919933,0cyclictest28231-21kworker/0:220:53:090
296919933,0cyclictest28231-21kworker/0:220:48:050
296919933,0cyclictest28231-21kworker/0:220:03:070
296919933,0cyclictest28231-21kworker/0:219:53:060
296919933,0cyclictest28231-21kworker/0:219:23:060
296919932,0cyclictest3556-21lldpd00:13:040
249930,0watchdog/20-21swapper/220:23:222
249930,0watchdog/20-21swapper/219:44:022
249930,0watchdog/20-21swapper/219:13:072
169930,0watchdog/19278-21kthreadcore19:53:221
169930,0watchdog/118177-21cut20:48:181
169930,0watchdog/10-21swapper/121:33:261
169930,0watchdog/10-21swapper/100:18:171
169930,0watchdog/10-21swapper/100:04:051
339920,0migration/35208-21iostat_ios19:03:203
296929920,0cyclictest12861-21sh19:58:141
249920,0watchdog/20-21swapper/220:03:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional