You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 20:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 28, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
824323624,0sleep00-21swapper/006:54:040
819723627,0sleep30-21swapper/306:53:353
841923424,0sleep20-21swapper/206:56:062
840023424,0sleep10-21swapper/106:55:571
85839960,0cyclictest0-21swapper/212:15:162
85819960,0cyclictest0-21swapper/009:50:180
4160,0ktimersoftd/06559-21runrttasks09:28:230
339960,0migration/328509-21munin-node10:47:313
339960,0migration/322695-21sh09:45:463
339960,0migration/316031-21irqcore10:22:263
339960,0migration/313932-21sh08:59:253
29715060,0irq/26-eth0-rx-29118-21sh11:29:563
29715060,0irq/26-eth0-rx-28923-21sh12:10:443
29715060,0irq/26-eth0-rx-15194-21iostat_ios11:22:273
85839955,0cyclictest6928-21diskmemload10:05:312
85839955,0cyclictest15381-21irqrtprio11:22:282
85839955,0cyclictest0-21swapper/211:16:472
85839955,0cyclictest0-21swapper/210:25:422
85839955,0cyclictest0-21swapper/210:10:342
85839953,0cyclictest3241-21irqbalance12:21:482
85839950,0cyclictest23063-21ssh09:26:062
85839950,0cyclictest0-21swapper/210:21:552
85839950,0cyclictest0-21swapper/209:56:132
85819955,0cyclictest0-21swapper/011:59:440
85819955,0cyclictest0-21swapper/010:31:360
85819950,0cyclictest0-21swapper/012:13:270
85819950,0cyclictest0-21swapper/012:08:530
85819950,0cyclictest0-21swapper/011:30:530
85819950,0cyclictest0-21swapper/010:02:170
339950,0migration/39958-21kthreadcore09:37:273
339950,0migration/38645-21munin-node07:17:313
339950,0migration/36919-21latency_hist10:17:093
339950,0migration/35704-21aten_r1power_po11:57:203
339950,0migration/35477-21kthreadcore07:42:263
339950,0migration/3533-21kthreadcore12:12:263
339950,0migration/34240-21taskset11:33:263
339950,0migration/332628-21kthreadcore07:12:283
339950,0migration/331870-21timerandwakeup08:47:333
339950,0migration/330906-21latency_hist10:32:093
339950,0migration/327961-21timerandwakeup07:07:333
339950,0migration/325852-21kthreadcore11:07:283
339950,0migration/325825-21kthreadcore07:32:273
339950,0migration/324435-21irqcore09:07:263
339950,0migration/31801-21kthreadcore09:52:283
339950,0migration/317806-21kthreadcore07:02:253
339950,0migration/315762-21kthreadcore08:37:293
339950,0migration/315557-21taskset11:02:273
339950,0migration/314616-21taskset12:02:223
339950,0migration/314325-21runrttasks10:00:193
339950,0migration/314008-21taskset10:40:563
339950,0migration/312271-21munin-node10:57:313
29725050,0irq/27-eth0-tx-8797-21munin-node06:57:310
29725050,0irq/27-eth0-tx-20617-21load09:02:282
29725050,0irq/27-eth0-tx-20617-21load09:02:282
29715050,0irq/26-eth0-rx-5659-21grep12:17:223
29715050,0irq/26-eth0-rx-32743-21iostat_ios09:32:253
29715050,0irq/26-eth0-rx-21578-21taskset11:46:593
29715050,0irq/26-eth0-rx-21276-21pmu-power09:22:303
29715050,0irq/26-eth0-rx-16782-21cut10:02:263
29715050,0irq/26-eth0-rx-14360-21pmu-power06:57:343
29715050,0irq/26-eth0-rx-13073-21ssh11:40:403
29715050,0irq/26-eth0-rx-12895-21latency09:17:293
29715050,0irq/26-eth0-rx-11770-21sendmail_mailqu08:32:323
29715050,0irq/26-eth0-rx-0-21swapper/312:22:273
29715050,0irq/26-eth0-rx-0-21swapper/311:12:473
29715050,0irq/26-eth0-rx-0-21swapper/310:52:443
29715050,0irq/26-eth0-rx-0-21swapper/310:27:213
29715050,0irq/26-eth0-rx-0-21swapper/310:11:043
29715050,0irq/26-eth0-rx-0-21swapper/309:03:443
29715050,0irq/26-eth0-rx-0-21swapper/309:03:443
259950,0migration/225549-21kthreadcore07:32:272
249950,0watchdog/20-21swapper/211:28:322
179950,0migration/17973-21hwlatdetect08:57:241
179950,0migration/15998-21latency_hist11:17:091
179950,0migration/15952-21latency_hist08:32:111
179950,0migration/126835-21kthreadcore09:27:281
179950,0migration/126032-21latency_hist08:47:111
179950,0migration/125917-21kthreadcore07:32:271
179950,0migration/123042-21kthreadcore12:07:251
179950,0migration/117842-21kthreadcore09:02:271
179950,0migration/117842-21kthreadcore09:02:271
179950,0migration/112758-21irqcore08:12:261
179950,0migration/112485-21kthreadcore07:47:271
169950,0watchdog/18546-21kthreadcore12:17:281
169950,0watchdog/131700-21grep10:32:241
149950,0migration/09891-21kthreadcore10:37:290
149950,0migration/07663-21fschecks_time10:17:230
149950,0migration/04797-21kthreadcore07:17:280
149950,0migration/04788-21munin-node09:52:320
149950,0migration/04204-21sh11:33:190
149950,0migration/023507-21aten_r1power_vo09:07:210
149950,0migration/021036-21pmu-power07:02:280
149950,0migration/020951-21kthreadcore07:52:280
149950,0migration/014746-21sh10:01:320
149950,0migration/013975-21fschecks_count12:22:230
85839943,0cyclictest9650-21awk06:57:282
85839943,0cyclictest8354-21awk09:17:262
85839943,0cyclictest6989-21awk08:32:252
85839943,0cyclictest6698-21awk11:37:232
85839943,0cyclictest5953-21awk08:07:262
85839943,0cyclictest3556-21lldpd12:07:152
85839943,0cyclictest3556-21lldpd07:10:442
85839943,0cyclictest3514-21snmpd07:03:142
85839943,0cyclictest32653-21awk08:27:242
85839943,0cyclictest32609-21ps10:12:252
85839943,0cyclictest3241-21irqbalance08:12:192
85839943,0cyclictest3241-21irqbalance07:47:292
85839943,0cyclictest3241-21irqbalance07:43:292
85839943,0cyclictest31593-21awk08:02:242
85839943,0cyclictest27078-21awk08:47:252
85839943,0cyclictest17377-21awk07:27:252
85839943,0cyclictest15445-21awk11:02:262
85839940,0cyclictest0-21swapper/210:27:262
85829943,0cyclictest8286-21awk09:37:241
85829943,0cyclictest3556-21lldpd12:02:151
85829943,0cyclictest3556-21lldpd11:14:451
85829943,0cyclictest3556-21lldpd07:31:451
85829943,0cyclictest23674-21awk10:47:241
85829943,0cyclictest16358-21awk07:02:231
85829940,0cyclictest12407-21ssh11:59:211
85829940,0cyclictest0-21swapper/111:47:221
85819944,0cyclictest0-21swapper/010:16:280
85819944,0cyclictest0-21swapper/009:37:130
85819944,0cyclictest0-21swapper/008:37:270
85819943,0cyclictest7169-21awk11:17:250
85819943,0cyclictest6301-21awk12:17:250
85819943,0cyclictest5301-21tr08:07:230
85819943,0cyclictest5051-21awk07:42:250
85819943,0cyclictest3556-21lldpd11:47:150
85819943,0cyclictest3556-21lldpd10:09:440
85819943,0cyclictest32720-21awk09:32:250
85819943,0cyclictest32629-21awk08:27:240
85819943,0cyclictest3241-21irqbalance11:37:180
85819943,0cyclictest3241-21irqbalance11:12:480
85819943,0cyclictest3241-21irqbalance11:07:580
85819943,0cyclictest3241-21irqbalance11:02:180
85819943,0cyclictest3241-21irqbalance10:59:380
85819943,0cyclictest3241-21irqbalance10:47:280
85819943,0cyclictest3241-21irqbalance10:42:180
85819943,0cyclictest3241-21irqbalance08:52:590
85819943,0cyclictest3241-21irqbalance08:42:190
85819943,0cyclictest3241-21irqbalance07:27:190
85819943,0cyclictest3241-21irqbalance07:22:190
85819943,0cyclictest3241-21irqbalance07:12:190
85819943,0cyclictest30606-21awk07:37:270
85819943,0cyclictest14859-21awk11:42:250
85819943,0cyclictest14496-21awk12:02:210
85819940,0cyclictest26099-21munin-node08:47:230
339940,0migration/39403-21kthreadcore08:07:283
339940,0migration/33579-21kthreadcore08:52:283
339940,0migration/328600-21sh11:49:063
339940,0migration/319524-21aten_r1power_fr08:42:223
339940,0migration/311181-21kthreadcore07:22:283
29725040,0irq/27-eth0-tx-6949-21ssh09:36:402
29725040,0irq/27-eth0-tx-6928-21diskmemload11:32:162
29725040,0irq/27-eth0-tx-29476-21awk09:27:322
29725040,0irq/27-eth0-tx-21416-21ssh10:44:192
29725040,0irq/27-eth0-tx-15452-21munin-node09:42:212
29725040,0irq/27-eth0-tx-13526-21ssh08:58:202
29725040,0irq/27-eth0-tx-0-21swapper/211:52:212
29725040,0irq/27-eth0-tx-0-21swapper/211:18:102
29725040,0irq/27-eth0-tx-0-21swapper/211:00:002
29725040,0irq/27-eth0-tx-0-21swapper/210:52:122
29725040,0irq/27-eth0-tx-0-21swapper/209:48:032
29725040,0irq/27-eth0-tx-0-21swapper/209:37:222
29725040,0irq/27-eth0-tx-0-21swapper/008:57:170
29725040,0irq/27-eth0-tx-0-21swapper/008:12:330
29725040,0irq/27-eth0-tx-0-21swapper/007:57:240
29725040,0irq/27-eth0-tx-0-21swapper/007:32:220
29725040,0irq/27-eth0-tx-0-21swapper/007:07:320
29715040,0irq/26-eth0-rx-0-21swapper/311:52:223
29715040,0irq/26-eth0-rx-0-21swapper/311:17:153
29715040,0irq/26-eth0-rx-0-21swapper/310:42:313
29715040,0irq/26-eth0-rx-0-21swapper/310:12:113
29715040,0irq/26-eth0-rx-0-21swapper/309:50:353
29715040,0irq/26-eth0-rx-0-21swapper/309:28:143
29715040,0irq/26-eth0-rx-0-21swapper/309:12:163
29715040,0irq/26-eth0-rx-0-21swapper/308:17:263
29715040,0irq/26-eth0-rx-0-21swapper/308:12:453
259940,0migration/28943-21kthreadcore09:57:272
259940,0migration/28433-21kthreadcore10:37:282
259940,0migration/23536-21runrttasks07:38:542
259940,0migration/229425-21fschecks_time07:12:252
259940,0migration/226723-21kthreadcore08:22:272
259940,0migration/224726-21kthreadcore10:47:262
259940,0migration/217903-21fschecks_count07:52:242
259940,0migration/216178-21kthreadcore12:22:272
259940,0migration/216030-21runrttasks07:24:372
249940,0watchdog/25587-21ssh09:13:042
249940,0watchdog/23530-21kthreadcore08:52:282
249940,0watchdog/231702-21tail10:32:242
249940,0watchdog/219080-21kthreadcore11:42:282
249940,0watchdog/214972-21kthreadcore12:02:242
249940,0watchdog/20-21swapper/211:57:242
249940,0watchdog/20-21swapper/211:47:282
249940,0watchdog/20-21swapper/211:07:242
249940,0watchdog/20-21swapper/209:07:202
249940,0watchdog/20-21swapper/207:57:292
179940,0migration/18604-21kthreadcore10:37:281
179940,0migration/130788-21irqcore12:12:251
179940,0migration/130745-21hwlatdetect11:52:241
179940,0migration/129404-21munin-node07:12:251
179940,0migration/127686-21kthreadcore09:47:281
179940,0migration/126977-21kthreadcore07:07:281
179940,0migration/125090-21aten_r1power_cu08:22:211
179940,0migration/124788-21kthreadcore10:27:261
179940,0migration/122040-21kthreadcore07:52:281
179940,0migration/118414-21kthreadcore09:42:261
179940,0migration/116646-21kthreadcore10:22:271
179940,0migration/115839-21kthreadcore11:22:281
179940,0migration/11337-21kthreadcore09:32:261
169940,0watchdog/1945-21fschecks_time08:52:241
169940,0watchdog/16928-21diskmemload09:57:561
169940,0watchdog/16897-21ssh09:16:241
169940,0watchdog/15745-21ssh09:53:561
169940,0watchdog/14100-21telnet07:42:211
169940,0watchdog/13721-21fschecks_time07:17:251
169940,0watchdog/13514-21snmpd07:59:011
169940,0watchdog/131553-21ssh10:52:241
169940,0watchdog/12901-21port_sshpriv07:37:331
169940,0watchdog/123624-21kthreadcore08:42:281
169940,0watchdog/123358-21munin-node09:07:201
169940,0watchdog/119078-21kthreadcore11:42:281
169940,0watchdog/117864-21grep12:22:281
169940,0watchdog/110609-21gltestperf07:22:251
169940,0watchdog/110080-21kthreadcore08:07:291
169940,0watchdog/10-21swapper/111:27:281
169940,0watchdog/10-21swapper/110:57:321
149940,0migration/08805-21munin-node09:17:280
149940,0migration/03634-21kthreadcore09:12:270
149940,0migration/032480-21kthreadcore10:32:270
149940,0migration/029972-21aten_r1power_en11:52:210
149940,0migration/021731-21unixbench_singl09:22:330
149940,0migration/019110-21kthreadcore11:22:290
149940,0migration/018928-21kthreadcore09:42:260
85839932,0cyclictest3556-21lldpd08:37:152
85839932,0cyclictest3556-21lldpd08:17:452
85829932,0cyclictest9598-21ls06:57:281
85829932,0cyclictest7378-21irqstats08:32:271
85829932,0cyclictest6715-21awk11:37:241
85829932,0cyclictest4326-21grep08:02:331
85829932,0cyclictest3556-21lldpd11:34:451
85829932,0cyclictest3556-21lldpd10:17:151
85829932,0cyclictest23555-21cut10:07:201
85829932,0cyclictest16737-21kthreadcore10:42:271
85829932,0cyclictest15775-21basename09:22:211
85829932,0cyclictest13664-21awk08:37:261
85819932,0cyclictest31628-21awk10:52:250
85819932,0cyclictest214912sleep009:02:570
85819932,0cyclictest214912sleep009:02:570
85819932,0cyclictest16161-21needreboot07:47:300
85819930,0cyclictest31345-21perl08:02:230
85819930,0cyclictest25071-21munin-node08:22:260
85819930,0cyclictest15389-21diskstats10:22:230
85819930,0cyclictest0-21swapper/008:17:240
29725030,0irq/27-eth0-tx-0-21swapper/008:32:210
29715030,0irq/26-eth0-rx-6559-21runrttasks07:37:213
29715030,0irq/26-eth0-rx-0-21swapper/308:27:213
29715030,0irq/26-eth0-rx-0-21swapper/308:22:223
29715030,0irq/26-eth0-rx-0-21swapper/307:52:223
29715030,0irq/26-eth0-rx-0-21swapper/307:51:173
29715030,0irq/26-eth0-rx-0-21swapper/307:27:223
249930,0watchdog/20-21swapper/208:42:242
249930,0watchdog/20-21swapper/207:17:252
179930,0migration/117920-21kthreadcore10:02:261
169930,0watchdog/19939-21kthreadcore09:17:281
169930,0watchdog/131346-21munin-node10:12:201
169930,0watchdog/125967-21kthreadcore11:07:281
169930,0watchdog/116828-21kthreadcore11:02:281
29715020,0irq/26-eth0-rx-0-21swapper/308:02:253
29715020,0irq/26-eth0-rx-0-21swapper/307:57:213
169920,0watchdog/10-21swapper/108:17:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional