You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 22:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Jun 28, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351610,0ktimersoftd/320158-21perf06:51:133
2660723524,0sleep00-21swapper/006:49:450
2664723422,0sleep10-21swapper/106:50:141
2059823324,0sleep20-21swapper/206:46:352
339960,0migration/38288-21sh09:00:173
339960,0migration/332419-21sh10:56:193
339960,0migration/324036-21sh10:29:403
339960,0migration/323227-21sh11:08:413
339960,0migration/315786-21sh09:03:123
339960,0migration/313990-21unixbench_singl07:01:453
29715060,0irq/26-eth0-rx-22823-21sshd12:09:463
29715060,0irq/26-eth0-rx-17820-21cut07:06:403
29715060,0irq/26-eth0-rx-0-21swapper/311:28:483
268179960,0cyclictest0-21swapper/007:16:390
149960,0migration/07396-21idleruntime-cro12:01:210
8215050,0irq/24-0000:00:30707-21perf08:31:222
339950,0migration/39658-21fschecks_time07:26:353
339950,0migration/3827-21munin-node11:36:363
339950,0migration/35794-21kthreadcore08:11:403
339950,0migration/35105-21kthreadcore07:21:393
339950,0migration/33022-21kthreadcore10:56:383
339950,0migration/325940-21kthreadcore11:11:393
339950,0migration/32301-21munin-node06:56:383
339950,0migration/322527-21munin-node10:06:423
339950,0migration/321644-21munin-node07:31:453
339950,0migration/317237-21runrttasks09:46:313
339950,0migration/316355-21taskset10:25:313
339950,0migration/314139-21sh12:02:393
339950,0migration/313694-21kthreadcore08:16:443
339950,0migration/310916-21munin-node08:36:453
29725050,0irq/27-eth0-tx-930-21munin-node09:56:332
29725050,0irq/27-eth0-tx-24280-21ssh09:49:252
29725050,0irq/27-eth0-tx-23649-21munin-node12:11:342
29725050,0irq/27-eth0-tx-23066-21python09:06:442
29725050,0irq/27-eth0-tx-20400-21tr10:46:392
29725050,0irq/27-eth0-tx-16196-21munin-node11:26:392
29725050,0irq/27-eth0-tx-15301-21ssh09:02:022
29725050,0irq/27-eth0-tx-13494-21kthreadcore10:01:382
29725050,0irq/27-eth0-tx-1290-21munin-node06:56:350
29725050,0irq/27-eth0-tx-1259-21munin-node09:16:422
29725050,0irq/27-eth0-tx-1061-21munin-node09:36:392
29725050,0irq/27-eth0-tx-0-21swapper/211:41:382
29725050,0irq/27-eth0-tx-0-21swapper/211:16:382
29725050,0irq/27-eth0-tx-0-21swapper/209:53:162
29725050,0irq/27-eth0-tx-0-21swapper/209:27:102
29725050,0irq/27-eth0-tx-0-21swapper/209:15:022
29725050,0irq/27-eth0-tx-0-21swapper/208:57:592
29725050,0irq/27-eth0-tx-0-21swapper/008:51:460
29725050,0irq/27-eth0-tx-0-21swapper/008:21:450
29715050,0irq/26-eth0-rx-8853-21if_eth111:41:363
29715050,0irq/26-eth0-rx-30831-21processes10:11:423
29715050,0irq/26-eth0-rx-28454-21kthreadcore06:51:403
29715050,0irq/26-eth0-rx-27862-21kthreadcore10:31:393
29715050,0irq/26-eth0-rx-2491-21grep11:16:373
29715050,0irq/26-eth0-rx-24402-21munin-run10:51:213
29715050,0irq/26-eth0-rx-17860-21grep09:26:353
29715050,0irq/26-eth0-rx-16767-21ssh10:06:103
29715050,0irq/26-eth0-rx-1550-21fschecks_count09:36:363
29715050,0irq/26-eth0-rx-14187-21ntpdc10:41:403
29715050,0irq/26-eth0-rx-0-21swapper/312:16:393
29715050,0irq/26-eth0-rx-0-21swapper/312:11:353
29715050,0irq/26-eth0-rx-0-21swapper/311:56:363
29715050,0irq/26-eth0-rx-0-21swapper/311:24:213
29715050,0irq/26-eth0-rx-0-21swapper/311:01:493
29715050,0irq/26-eth0-rx-0-21swapper/309:57:173
29715050,0irq/26-eth0-rx-0-21swapper/309:54:483
29715050,0irq/26-eth0-rx-0-21swapper/309:31:383
29715050,0irq/26-eth0-rx-0-21swapper/309:06:583
29715050,0irq/26-eth0-rx-0-21swapper/308:54:573
29715050,0irq/26-eth0-rx-0-21swapper/308:01:223
29715050,0irq/26-eth0-rx-0-21swapper/307:46:353
268189955,0cyclictest0-21swapper/112:10:111
268189955,0cyclictest0-21swapper/112:04:111
268189955,0cyclictest0-21swapper/111:12:191
268189950,0cyclictest0-21swapper/111:54:531
268189950,0cyclictest0-21swapper/110:52:271
268189950,0cyclictest0-21swapper/110:14:371
268189950,0cyclictest0-21swapper/109:40:521
268179955,0cyclictest0-21swapper/009:19:180
268179954,0cyclictest3556-21lldpd09:46:100
268179950,0cyclictest0-21swapper/011:19:360
268179950,0cyclictest0-21swapper/011:11:530
259950,0migration/28482-21idleruntime12:01:332
259950,0migration/229555-21munin-node11:51:412
259950,0migration/227955-21kthreadcore11:31:372
259950,0migration/217194-21munin-node11:06:352
249950,0watchdog/227090-21kthreadcore11:11:402
19150,0ktimersoftd/10-21swapper/111:29:041
179950,0migration/18490-21sh10:21:061
179950,0migration/132561-21munin-node11:36:321
179950,0migration/132151-21sh09:12:591
179950,0migration/13213-21kthreadcore08:31:371
179950,0migration/130866-21kthreadcore06:51:411
179950,0migration/130180-21kthreadcore10:31:401
179950,0migration/123859-21sh11:10:331
179950,0migration/119311-21gltestperf08:46:351
179950,0migration/118559-21munin-node09:26:371
179950,0migration/116077-21aten_r1power_vo11:46:331
179950,0migration/114689-21proc_pri09:01:411
179950,0migration/110016-21irqcore07:26:371
149950,0migration/09558-21if_eth010:41:360
149950,0migration/06859-21sh12:20:410
149950,0migration/06111-21kthreadcore07:46:380
149950,0migration/031059-21kthreadcore06:51:410
149950,0migration/029794-21munin-node11:51:440
149950,0migration/027350-21kthreadcore11:31:370
149950,0migration/027054-21kthreadcore10:51:390
149950,0migration/025866-21kthreadcore08:26:390
149950,0migration/025654-21kthreadcore07:36:390
149950,0migration/025587-21munin-node09:11:300
149950,0migration/025541-21munin-node09:31:320
149950,0migration/024836-21runrttasks09:30:290
149950,0migration/023361-21taskset10:48:240
149950,0migration/022441-21sh11:47:460
149950,0migration/018202-21kthreadcore07:06:410
149950,0migration/017675-21kthreadcore11:26:380
149950,0migration/016348-21latency_hist11:06:220
149950,0migration/015585-21taskset09:02:440
149950,0migration/015119-21taskset07:27:530
149950,0migration/014759-21switchtime10:21:430
149950,0migration/014256-210anacron11:42:220
149950,0migration/010819-21kthreadcore09:21:380
9175240,0sleep210492-21kthreadcore11:21:372
8215040,0irq/24-0000:00:28742-21latency_hist07:41:222
339940,0migration/34462-21timerwakeupswit08:31:423
339940,0migration/330240-21cpuspeed_turbos08:06:343
339940,0migration/329660-21kthreadcore07:16:383
339940,0migration/324783-21kthreadcore08:01:383
339940,0migration/320853-21kthreadcore08:46:383
339940,0migration/317972-21kthreadcore11:46:393
339940,0migration/316332-21taskset07:55:223
339940,0migration/312492-21fschecks_time08:41:363
29725040,0irq/27-eth0-tx-513-21telnet10:36:302
29725040,0irq/27-eth0-tx-31695-21ssh09:32:082
29725040,0irq/27-eth0-tx-24999-21diskmemload08:52:302
29725040,0irq/27-eth0-tx-0-21swapper/212:16:382
29725040,0irq/27-eth0-tx-0-21swapper/212:08:332
29725040,0irq/27-eth0-tx-0-21swapper/211:56:352
29725040,0irq/27-eth0-tx-0-21swapper/211:01:372
29725040,0irq/27-eth0-tx-0-21swapper/210:51:392
29725040,0irq/27-eth0-tx-0-21swapper/210:42:092
29725040,0irq/27-eth0-tx-0-21swapper/210:26:312
29725040,0irq/27-eth0-tx-0-21swapper/210:21:262
29725040,0irq/27-eth0-tx-0-21swapper/210:11:242
29725040,0irq/27-eth0-tx-0-21swapper/209:21:302
29725040,0irq/27-eth0-tx-0-21swapper/008:31:310
29715040,0irq/26-eth0-rx-0-21swapper/311:52:163
29715040,0irq/26-eth0-rx-0-21swapper/311:32:523
29715040,0irq/26-eth0-rx-0-21swapper/310:37:313
29715040,0irq/26-eth0-rx-0-21swapper/310:18:043
29715040,0irq/26-eth0-rx-0-21swapper/309:41:333
29715040,0irq/26-eth0-rx-0-21swapper/309:23:143
29715040,0irq/26-eth0-rx-0-21swapper/309:19:493
29715040,0irq/26-eth0-rx-0-21swapper/309:11:483
29715040,0irq/26-eth0-rx-0-21swapper/307:36:423
268209943,0cyclictest3556-21lldpd10:06:402
268209943,0cyclictest3514-21snmpd07:13:212
268209943,0cyclictest3514-21snmpd07:06:362
268209943,0cyclictest23195-21awk07:36:372
268209943,0cyclictest1661-21awk10:16:372
268209940,0cyclictest9180-21munin-node09:41:382
268189944,0cyclictest0-21swapper/112:13:101
268189944,0cyclictest0-21swapper/111:41:291
268189944,0cyclictest0-21swapper/108:51:411
268189943,0cyclictest9492-21awk10:41:351
268189943,0cyclictest9297-21awk11:01:351
268189943,0cyclictest3556-21lldpd11:21:411
268189943,0cyclictest32637-21awk12:16:361
268189943,0cyclictest26400-21awk09:31:361
268189943,0cyclictest24602-21kthreadcore08:01:381
268189943,0cyclictest10308-21awk09:21:371
268189943,0cyclictest0-21swapper/106:56:451
268179944,0cyclictest0-21swapper/012:02:480
268179944,0cyclictest0-21swapper/008:59:360
268179943,0cyclictest9328-21awk11:01:350
268179943,0cyclictest3556-21lldpd10:33:100
268179943,0cyclictest3241-21irqbalance08:12:210
268179943,0cyclictest3241-21irqbalance08:08:010
268179943,0cyclictest3241-21irqbalance08:02:110
268179943,0cyclictest3241-21irqbalance07:56:410
268179943,0cyclictest3241-21irqbalance07:51:510
268179943,0cyclictest3241-21irqbalance07:42:210
268179943,0cyclictest3241-21irqbalance07:21:510
268179943,0cyclictest3241-21irqbalance07:11:410
268179943,0cyclictest26050-21awk10:11:360
268179943,0cyclictest17953-21grep09:46:350
268179943,0cyclictest16565-21awk07:31:370
268179943,0cyclictest1479-21awk10:36:340
268179943,0cyclictest11111-21basename08:16:380
259940,0migration/2331-21fschecks_time11:36:332
259940,0migration/226091-21irqcore10:31:382
259940,0migration/224723-21runrttasks08:48:202
259940,0migration/222451-21timerwakeupswit07:56:442
259940,0migration/215983-21munin-node07:31:342
259940,0migration/215819-21kthreadcore08:41:402
259940,0migration/211866-21kthreadcore07:51:392
259940,0migration/211542-21kthreadcore07:01:392
249940,0watchdog/28592-21chrt07:24:252
249940,0watchdog/25659-21diskstats08:36:362
249940,0watchdog/24764-21fschecks_count08:11:362
249940,0watchdog/24232-21grep08:31:402
249940,0watchdog/232755-21aten_r1power_en10:56:322
249940,0watchdog/231238-21kthreadcore06:51:412
249940,0watchdog/228735-21netstat08:01:402
249940,0watchdog/220932-21kthreadcore11:46:392
249940,0watchdog/215866-21tr08:16:442
249940,0watchdog/20-21swapper/208:21:362
249940,0watchdog/20-21swapper/208:06:362
179940,0migration/19717-21kthreadcore07:01:391
179940,0migration/18574-21taskset07:46:391
179940,0migration/17470-21taskset09:57:341
179940,0migration/16583-21sensors_fan08:56:431
179940,0migration/1635-21fschecks_time11:16:341
179940,0migration/14675-21kthreadcore10:56:381
179940,0migration/132620-21fschecks_time11:56:341
179940,0migration/123344-21kthreadcore07:11:391
179940,0migration/121240-21kthreadcore09:06:401
179940,0migration/120768-21kthreadcore07:31:391
179940,0migration/11842-21fschecks_time09:16:351
179940,0migration/118397-21kthreadcore07:06:411
179940,0migration/11839-21latency07:41:391
179940,0migration/116036-21taskset07:51:511
179940,0migration/114127-21kthreadcore08:41:401
179940,0migration/110963-21kthreadcore10:21:381
179940,0migration/110799-21kthreadcore10:01:371
169940,0watchdog/17404-21kthreadcore08:11:401
169940,0watchdog/13031-21sed07:21:371
169940,0watchdog/129971-21munin-node08:06:361
169940,0watchdog/128062-21switchtime07:36:441
169940,0watchdog/122708-21switchtime10:26:441
169940,0watchdog/122548-21tail07:56:441
169940,0watchdog/118371-21expr08:21:361
169940,0watchdog/117489-21idleruntime10:46:361
169940,0watchdog/10-21swapper/110:36:361
169940,0watchdog/10-21swapper/109:49:121
149940,0migration/0960-21latency_hist09:36:220
149940,0migration/06272-21sh11:37:260
149940,0migration/06074-21idleruntime08:36:380
149940,0migration/028171-21kthreadcore09:51:380
149940,0migration/024740-21sh09:10:020
149940,0migration/022032-21kthreadcore10:26:390
149940,0migration/021079-21missed_timers12:06:400
149940,0migration/013695-21kthreadcore08:41:400
149940,0migration/011676-21kthreadcore11:21:380
29725030,0irq/27-eth0-tx-0-21swapper/007:01:340
29715030,0irq/26-eth0-rx-0-21swapper/308:26:343
29715030,0irq/26-eth0-rx-0-21swapper/308:21:353
29715030,0irq/26-eth0-rx-0-21swapper/307:41:323
29715030,0irq/26-eth0-rx-0-21swapper/307:11:343
268209932,0cyclictest2277-21awk06:56:382
268209932,0cyclictest1995-21sadc07:21:222
268209930,0cyclictest0-21swapper/207:46:382
268189932,0cyclictest28638-21grep11:31:371
268189932,0cyclictest26133-21awk09:51:361
268179933,0cyclictest3556-21lldpd10:57:400
268179932,0cyclictest5125-21grep09:56:380
268179932,0cyclictest3514-21snmpd10:02:060
268179932,0cyclictest29809-21vmstat12:11:430
268179932,0cyclictest1677-21awk10:16:370
268179930,0cyclictest19292-21perl08:46:350
259930,0migration/215219-21runrttasks07:29:042
179930,0migration/129075-21irqcore07:16:381
169930,0watchdog/124999-21diskmemload09:41:361
169930,0watchdog/10-21swapper/110:06:281
169930,0watchdog/10-21swapper/108:36:401
169930,0watchdog/10-21swapper/108:16:401
268179920,0cyclictest0-21swapper/010:06:220
169920,0watchdog/10-21swapper/108:26:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional