You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 18:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Sep 10, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450640,0irq/25-eth10-21swapper/306:40:413
193023826,0sleep20-21swapper/206:43:322
2806523524,0sleep00-21swapper/006:39:260
165723228,0sleep10-21swapper/106:40:231
339960,0migration/36603-21sh09:05:383
339960,0migration/320141-21munin-node10:14:363
339960,0migration/32013-21sh12:05:323
29715060,0irq/26-eth0-rx-4025-21diskstats12:09:313
29715060,0irq/26-eth0-rx-31358-21cut11:24:343
29715060,0irq/26-eth0-rx-12419-21python10:09:423
20689966,0cyclictest32570-21diskmemload09:57:262
20689960,0cyclictest0-21swapper/209:00:412
20679966,0cyclictest0-21swapper/111:34:371
8215050,0irq/24-0000:00:19746-21munin-run11:59:172
339950,0migration/34349-21sh11:07:513
339950,0migration/325367-21munin-node09:19:343
339950,0migration/324701-21irqcore09:39:343
339950,0migration/322967-21taskset07:23:213
339950,0migration/322572-21cpuspeed_turbos06:59:293
339950,0migration/321660-21aten_r1power_fr10:59:283
339950,0migration/321454-21timerandwakeup09:34:413
339950,0migration/321344-21munin-node11:39:333
339950,0migration/320672-21fschecks_time11:59:323
339950,0migration/320666-21munin-node08:34:353
339950,0migration/318876-21munin-node07:44:353
339950,0migration/316031-21kthreadcore10:54:343
339950,0migration/315985-21kthreadcore10:34:343
339950,0migration/315948-21latency_hist09:14:193
339950,0migration/313087-21ksmtuned11:14:263
339950,0migration/311001-21kthreadcore08:24:373
339950,0migration/310636-21taskset07:38:133
339950,0migration/310504-21kthreadcore08:49:353
29725050,0irq/27-eth0-tx-27942-21sh12:04:252
29725050,0irq/27-eth0-tx-26971-21ssh11:40:292
29725050,0irq/27-eth0-tx-20023-21munin-node11:59:342
29725050,0irq/27-eth0-tx-18323-21sh11:34:402
29725050,0irq/27-eth0-tx-14592-21ssh09:53:562
29725050,0irq/27-eth0-tx-0-21swapper/211:11:002
29725050,0irq/27-eth0-tx-0-21swapper/211:00:232
29725050,0irq/27-eth0-tx-0-21swapper/210:07:572
29725050,0irq/27-eth0-tx-0-21swapper/209:36:252
29725050,0irq/27-eth0-tx-0-21swapper/209:16:462
29715050,0irq/26-eth0-rx-9334-21munin-node09:09:323
29715050,0irq/26-eth0-rx-5911-21kthreadcore11:29:353
29715050,0irq/26-eth0-rx-5353-21basename07:59:303
29715050,0irq/26-eth0-rx-28867-21df_abs11:44:293
29715050,0irq/26-eth0-rx-28002-21timerandwakeup10:39:403
29715050,0irq/26-eth0-rx-18749-21cpu08:09:303
29715050,0irq/26-eth0-rx-16935-21fschecks_time08:54:313
29715050,0irq/26-eth0-rx-12276-21munin-node11:54:293
29715050,0irq/26-eth0-rx-1124-21grep10:44:383
29715050,0irq/26-eth0-rx-10888-21rt-features11:09:373
29715050,0irq/26-eth0-rx-0-21swapper/310:27:093
29715050,0irq/26-eth0-rx-0-21swapper/310:19:383
29715050,0irq/26-eth0-rx-0-21swapper/309:44:343
29715050,0irq/26-eth0-rx-0-21swapper/309:25:023
29715050,0irq/26-eth0-rx-0-21swapper/309:00:223
29715050,0irq/26-eth0-rx-0-21swapper/307:54:373
27150,0ktimersoftd/20-21swapper/209:59:422
259950,0migration/230424-21kthreadcore11:44:332
259950,0migration/229614-21unixbench_singl09:39:412
259950,0migration/216190-21kthreadcore10:34:342
259950,0migration/212718-21latency_hist08:29:192
20689955,0cyclictest0-21swapper/211:24:462
20689955,0cyclictest0-21swapper/211:19:152
20689955,0cyclictest0-21swapper/210:43:282
20689955,0cyclictest0-21swapper/207:09:412
20689950,0cyclictest0-21swapper/211:22:342
20689950,0cyclictest0-21swapper/210:57:212
20689950,0cyclictest0-21swapper/210:50:242
20689950,0cyclictest0-21swapper/207:54:282
20689950,0cyclictest0-21swapper/206:56:262
20679955,0cyclictest0-21swapper/112:00:471
20679955,0cyclictest0-21swapper/110:21:081
20679955,0cyclictest0-21swapper/109:37:001
20679955,0cyclictest0-21swapper/109:00:541
20679955,0cyclictest0-21swapper/107:44:351
20679954,0cyclictest3556-21lldpd11:52:241
20679954,0cyclictest3556-21lldpd10:29:251
20679950,0cyclictest0-21swapper/112:11:301
20669955,0cyclictest0-21swapper/009:59:420
20669955,0cyclictest0-21swapper/008:47:080
20669955,0cyclictest0-21swapper/006:44:380
20669954,0cyclictest0-21swapper/011:34:340
20669953,0cyclictest31827-21awk10:04:330
20669952,0cyclictest3694-21sh11:49:000
20669952,0cyclictest26391-21expr11:39:410
20669952,0cyclictest23944-21diskstats09:39:310
20669950,0cyclictest0-21swapper/010:37:110
20669950,0cyclictest0-21swapper/008:09:370
19150,0ktimersoftd/10-21swapper/111:41:521
19150,0ktimersoftd/10-21swapper/109:59:421
179950,0migration/16576-21taskset09:26:581
179950,0migration/132381-21taskset08:42:201
179950,0migration/125482-21kthreadcore10:59:351
179950,0migration/125033-21munin-node08:34:391
179950,0migration/114248-21taskset09:52:521
179950,0migration/112832-21kthreadcore09:29:371
179950,0migration/110287-21kthreadcore08:49:351
169950,0watchdog/15215-21aten_r1power_po11:09:281
8215040,0irq/24-0000:00:0-21swapper/208:14:192
339940,0migration/36404-21cpuspeed_turbos10:29:293
339940,0migration/35565-21runrttasks08:19:583
339940,0migration/330425-21kthreadcore07:04:353
339940,0migration/327271-21kthreadcore07:24:363
339940,0migration/326058-21if_err_eth008:14:343
339940,0migration/322723-21kthreadcore11:19:343
339940,0migration/317134-21kthreadcore09:54:343
339940,0migration/311984-21kthreadcore07:14:353
29725040,0irq/27-eth0-tx-16366-21ssh08:54:252
29725040,0irq/27-eth0-tx-0-21swapper/212:09:322
29725040,0irq/27-eth0-tx-0-21swapper/211:49:272
29725040,0irq/27-eth0-tx-0-21swapper/211:29:552
29725040,0irq/27-eth0-tx-0-21swapper/211:04:272
29725040,0irq/27-eth0-tx-0-21swapper/210:19:212
29725040,0irq/27-eth0-tx-0-21swapper/209:44:252
29725040,0irq/27-eth0-tx-0-21swapper/209:09:282
29725040,0irq/27-eth0-tx-0-21swapper/208:46:022
29725040,0irq/27-eth0-tx-0-21swapper/007:24:310
29715040,0irq/26-eth0-rx-26259-21aten_r1power_fr08:39:303
29715040,0irq/26-eth0-rx-19086-40yum-updatesd-he11:35:563
29715040,0irq/26-eth0-rx-0-21swapper/311:49:363
29715040,0irq/26-eth0-rx-0-21swapper/310:51:063
29715040,0irq/26-eth0-rx-0-21swapper/310:04:193
29715040,0irq/26-eth0-rx-0-21swapper/309:59:193
29715040,0irq/26-eth0-rx-0-21swapper/309:49:443
29715040,0irq/26-eth0-rx-0-21swapper/309:29:463
29715040,0irq/26-eth0-rx-0-21swapper/308:46:313
29715040,0irq/26-eth0-rx-0-21swapper/308:29:303
259940,0migration/26938-21hddtemp_smartct10:29:312
259940,0migration/26733-21aten_r1power_fr10:09:312
259940,0migration/24375-21kthreadcore09:04:352
259940,0migration/24132-21kthreadcore08:19:352
259940,0migration/24027-21switchtime10:24:402
259940,0migration/23062-21kthreadcore09:24:352
259940,0migration/229095-21munin-node07:04:302
259940,0migration/224393-21cpuspeed_turbos09:19:302
259940,0migration/224251-21runrttasks07:49:262
259940,0migration/210956-21switchtime07:59:412
249940,0watchdog/27894-21kthreadcore07:34:372
249940,0watchdog/215004-21users06:49:412
20689943,0cyclictest3556-21lldpd10:44:252
20689943,0cyclictest3556-21lldpd09:29:242
20689943,0cyclictest3556-21lldpd08:49:242
20689943,0cyclictest3556-21lldpd08:40:532
20689943,0cyclictest3556-21lldpd08:30:232
20689943,0cyclictest3556-21lldpd08:11:542
20689943,0cyclictest3556-21lldpd07:40:532
20689943,0cyclictest3556-21lldpd07:27:542
20689943,0cyclictest3556-21lldpd07:19:542
20689943,0cyclictest3514-21snmpd08:04:362
20689943,0cyclictest30832-21awk07:29:332
20689943,0cyclictest23176-21awk06:59:322
20689943,0cyclictest20380-21awk08:34:342
20689943,0cyclictest18460-21awk07:44:342
20689943,0cyclictest15480-21awk10:14:322
20679943,0cyclictest9758-21sadc07:14:191
20679943,0cyclictest5252-21awk07:34:351
20679943,0cyclictest3556-21lldpd11:54:551
20679943,0cyclictest3556-21lldpd10:49:251
20679943,0cyclictest3556-21lldpd10:16:541
20679943,0cyclictest3556-21lldpd10:13:541
20679943,0cyclictest3556-21lldpd08:23:541
20679943,0cyclictest3556-21lldpd08:17:241
20679943,0cyclictest3556-21lldpd07:20:241
20679943,0cyclictest32394-21awk09:44:321
20679943,0cyclictest29843-21awk11:24:321
20679943,0cyclictest15999-21awk09:54:331
20679943,0cyclictest1260-21ps12:04:381
20669944,0cyclictest0-21swapper/009:44:220
20669943,0cyclictest8604-21awk09:29:340
20669943,0cyclictest7514-21awk10:09:350
20669943,0cyclictest3241-21irqbalance12:10:560
20669943,0cyclictest3241-21irqbalance12:04:260
20669943,0cyclictest3241-21irqbalance11:14:260
20669943,0cyclictest3241-21irqbalance11:09:460
20669943,0cyclictest3241-21irqbalance09:24:270
20669943,0cyclictest3241-21irqbalance09:02:170
20669943,0cyclictest3241-21irqbalance07:14:270
20669943,0cyclictest3241-21irqbalance07:09:270
20669943,0cyclictest3241-21irqbalance06:49:280
20669943,0cyclictest26200-21awk08:14:350
20669943,0cyclictest17607-21munin-node07:44:310
20669943,0cyclictest14524-21awk10:54:320
20669943,0cyclictest10222-21users11:49:390
20669942,0cyclictest9158-21iostat09:09:310
20669942,0cyclictest897-21munin-node09:04:320
20669942,0cyclictest6426-21grep08:24:320
20669942,0cyclictest6364-21perl10:49:320
20669942,0cyclictest5211-21awk07:59:250
20669942,0cyclictest5165-21sendmail_mailst08:19:380
20669942,0cyclictest4623-21rm10:47:010
20669942,0cyclictest4541-21sh11:29:230
20669942,0cyclictest4060-21munin-node07:54:370
20669942,0cyclictest31137-21sed07:29:340
20669942,0cyclictest30988-21tune2fs10:24:310
20669942,0cyclictest29927-21diskstats11:04:350
20669942,0cyclictest29364-21diskstats11:24:300
20669942,0cyclictest25530-21irqstats07:49:340
20669942,0cyclictest24652-21perl09:19:310
20669942,0cyclictest23579-21iostat_ios10:19:340
20669942,0cyclictest22494-21diskstats10:39:300
20669942,0cyclictest22394-21aten2.4-expect06:59:290
20669942,0cyclictest22277-21perl10:59:310
20669942,0cyclictest21786-21grep08:54:360
20669942,0cyclictest21348-21sensors_volt06:54:410
20669942,0cyclictest21068-21aten2.4-expect11:19:280
20669942,0cyclictest20180-21aten2.4-expect11:59:300
20669942,0cyclictest17943-21irqstats07:19:360
20669942,0cyclictest17097-21iostat_ios09:14:340
20669942,0cyclictest16255-21perl09:34:310
20669942,0cyclictest13966-21ntpq08:49:380
20669942,0cyclictest13346-21sh09:50:090
20669942,0cyclictest12893-21aten2.4-expect08:29:290
20669942,0cyclictest12452-21df_inode11:54:300
20669942,0cyclictest12302-21yum10:29:420
20669942,0cyclictest12248-21irqstats07:39:350
20669942,0cyclictest12215-21df_abs08:04:300
20669941,0cyclictest19498-21munin-node08:34:350
20669941,0cyclictest15408-21diskstats09:54:300
20669941,0cyclictest14986-21diskstats10:14:300
20669941,0cyclictest0-21swapper/007:04:300
179940,0migration/19045-21kthreadcore07:59:361
179940,0migration/18666-21runrttasks06:47:491
179940,0migration/16582-21kthreadcore11:29:351
179940,0migration/131178-21kthreadcore11:04:381
179940,0migration/129752-21sh09:39:571
179940,0migration/128905-21kthreadcore09:19:351
179940,0migration/128121-21kthreadcore07:49:351
179940,0migration/122754-21munin-node06:59:301
179940,0migration/122385-21kthreadcore11:19:341
179940,0migration/12175-21kthreadcore09:04:351
179940,0migration/121446-21timerandwakeup06:54:411
179940,0migration/120731-21kthreadcore08:09:351
179940,0migration/120221-21taskset10:56:061
179940,0migration/118657-21kthreadcore09:14:351
179940,0migration/118555-21switchtime08:29:401
179940,0migration/117413-21kthreadcore11:14:341
179940,0migration/115140-21taskset06:51:051
179940,0migration/114410-21cpuspeed_turbos10:34:301
179940,0migration/113865-21kthreadcore08:04:341
179940,0migration/111548-21fschecks_time07:39:321
179940,0migration/110006-21kthreadcore09:09:321
169940,0watchdog/1692-21df08:44:331
169940,0watchdog/130031-21hostname07:29:291
169940,0watchdog/128662-21missed_timers07:24:371
169940,0watchdog/110951-21awk07:14:331
169940,0watchdog/10-21swapper/111:49:121
169940,0watchdog/10-21swapper/110:39:321
169940,0watchdog/10-21swapper/108:24:331
339930,0migration/317579-21munin-node08:04:393
29715030,0irq/26-eth0-rx-0-21swapper/307:52:303
29715030,0irq/26-eth0-rx-0-21swapper/307:42:023
29715030,0irq/26-eth0-rx-0-21swapper/307:29:393
29715030,0irq/26-eth0-rx-0-21swapper/306:54:303
29715030,0irq/26-eth0-rx-0-21swapper/306:44:313
259930,0migration/23215-21if_eth006:44:332
249930,0watchdog/20-21swapper/207:14:332
20679932,0cyclictest31761-21awk07:54:321
20669931,0cyclictest4431-21aten_r1power_vo07:34:310
20669931,0cyclictest26186-21basename08:39:290
179930,0migration/14003-21runrttasks10:45:071
169930,0watchdog/10-21swapper/110:24:361
169930,0watchdog/10-21swapper/110:06:491
169930,0watchdog/10-21swapper/108:55:211
169930,0watchdog/10-21swapper/107:06:171
29715020,0irq/26-eth0-rx-0-21swapper/307:09:193
29715020,0irq/26-eth0-rx-0-21swapper/306:49:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional