You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 18:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Dec 30, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3089927170,0sleep10-21swapper/107:05:141
304450650,0irq/25-eth10-21swapper/307:05:423
3080423826,0sleep20-21swapper/207:04:072
3098423622,0sleep00-21swapper/007:06:130
339960,0migration/38086-21sh12:05:403
339960,0migration/37250-21sh09:32:223
339960,0migration/328098-21sh10:59:573
339960,0migration/319884-21munin-node11:13:453
339960,0migration/313604-21sh11:12:333
339960,0migration/313076-21latency08:03:443
312409960,0cyclictest0-21swapper/212:07:512
312409960,0cyclictest0-21swapper/210:42:332
179960,0migration/131780-21ntp_offset12:18:421
8215050,0irq/24-0000:00:31390-21awk09:28:222
339950,0migration/3906-21latency_hist09:48:223
339950,0migration/38296-21unixbench_multi12:23:453
339950,0migration/38124-21kthreadcore07:38:393
339950,0migration/36831-21kthreadcore08:48:413
339950,0migration/3599-21missed_timers08:18:413
339950,0migration/35451-21kthreadcore10:28:393
339950,0migration/34812-21kthreadcore10:08:413
339950,0migration/332528-21cpuspeed_turbos11:43:343
339950,0migration/331513-21kthreadcore12:18:403
339950,0migration/331393-21taskset11:42:093
339950,0migration/325728-21munin-node07:48:433
339950,0migration/323060-21runrttasks11:37:093
339950,0migration/319896-21taskset07:48:193
339950,0migration/317465-21hwlatdetect09:38:373
339950,0migration/317293-21needreboot10:33:403
339950,0migration/315586-21aten_r1power_vo08:33:363
339950,0migration/315084-21munin-node12:08:433
339950,0migration/314982-21latency08:53:383
339950,0migration/311874-21munin-node09:13:463
339950,0migration/311461-21rt-features07:13:443
339950,0migration/311021-21df_abs12:28:343
339950,0migration/311021-21df_abs12:28:343
312409955,0cyclictest0-21swapper/211:51:172
312409955,0cyclictest0-21swapper/210:18:542
312409954,0cyclictest19897-21sadc10:38:222
312409950,0cyclictest0-21swapper/212:01:232
312409950,0cyclictest0-21swapper/209:44:512
312399955,0cyclictest0-21swapper/107:58:451
312399950,0cyclictest0-21swapper/110:11:301
312389955,0cyclictest0-21swapper/009:41:510
312389954,0cyclictest27122-21awk12:18:370
312389954,0cyclictest12268-21munin-run07:18:220
312389953,0cyclictest646-21sadc09:48:220
312389953,0cyclictest602-21awk11:43:360
312389953,0cyclictest17549-21awk11:53:370
312389953,0cyclictest0-21swapper/010:09:080
312389952,0cyclictest8680-21diskstats11:48:350
312389952,0cyclictest3865-21diskstats10:28:350
312389952,0cyclictest27220-21diskstats07:53:350
312389952,0cyclictest24308-21irqstats09:23:390
312389952,0cyclictest24305-21ssh09:59:280
312389952,0cyclictest21514-21diskstats08:13:360
312389952,0cyclictest2035-21iostat_ios09:48:390
29725050,0irq/27-eth0-tx-4899-21tr10:28:392
29725050,0irq/27-eth0-tx-3194-21munin-node12:23:372
29725050,0irq/27-eth0-tx-26733-21cut11:18:392
29725050,0irq/27-eth0-tx-2361-21ssh10:43:552
29725050,0irq/27-eth0-tx-13492-21munin-node07:43:472
29725050,0irq/27-eth0-tx-12627-21kthreadcore09:53:392
29725050,0irq/27-eth0-tx-0-21swapper/211:57:582
29725050,0irq/27-eth0-tx-0-21swapper/210:24:512
29725050,0irq/27-eth0-tx-0-21swapper/210:14:452
29725050,0irq/27-eth0-tx-0-21swapper/210:10:242
29725050,0irq/27-eth0-tx-0-21swapper/209:51:202
29725050,0irq/27-eth0-tx-0-21swapper/209:33:252
29725050,0irq/27-eth0-tx-0-21swapper/209:28:352
29725050,0irq/27-eth0-tx-0-21swapper/209:21:002
29715050,0irq/26-eth0-rx-690-21kthreadcore11:23:383
29715050,0irq/26-eth0-rx-31475-210anacron11:59:223
29715050,0irq/26-eth0-rx-30035-21kthreadcore10:23:413
29715050,0irq/26-eth0-rx-24743-21ssh12:15:343
29715050,0irq/26-eth0-rx-19624-21cut12:33:343
29715050,0irq/26-eth0-rx-19624-21cut12:33:343
29715050,0irq/26-eth0-rx-15943-21ssh09:54:323
29715050,0irq/26-eth0-rx-14893-21ssh09:35:123
29715050,0irq/26-eth0-rx-13137-21grep11:48:403
29715050,0irq/26-eth0-rx-12278-21ssh10:52:203
29715050,0irq/26-eth0-rx-0-21swapper/311:55:493
29715050,0irq/26-eth0-rx-0-21swapper/311:28:363
29715050,0irq/26-eth0-rx-0-21swapper/311:20:333
29715050,0irq/26-eth0-rx-0-21swapper/311:07:533
29715050,0irq/26-eth0-rx-0-21swapper/310:55:283
29715050,0irq/26-eth0-rx-0-21swapper/310:13:333
29715050,0irq/26-eth0-rx-0-21swapper/310:06:143
29715050,0irq/26-eth0-rx-0-21swapper/310:00:113
29715050,0irq/26-eth0-rx-0-21swapper/309:43:393
29715050,0irq/26-eth0-rx-0-21swapper/309:26:153
29715050,0irq/26-eth0-rx-0-21swapper/309:18:543
27150,0ktimersoftd/20-21swapper/210:50:452
259950,0migration/26047-21sh11:26:442
259950,0migration/24479-21kthreadcore07:58:412
259950,0migration/231887-21kthreadcore09:08:392
259950,0migration/220939-21kthreadcore12:13:392
259950,0migration/218423-21kthreadcore08:58:392
259950,0migration/214952-21fschecks_count08:08:362
259950,0migration/212029-21kthreadcore08:28:402
249950,0watchdog/26528-21hwlatdetect09:13:362
249950,0watchdog/230055-21kthreadcore08:43:402
179950,0migration/15824-21kthreadcore12:23:391
179950,0migration/15519-21latency_hist09:13:221
179950,0migration/14465-21latency_hist10:48:221
179950,0migration/126872-21kthreadcore11:58:391
179950,0migration/126754-21fschecks_count10:03:381
179950,0migration/126463-21taskset10:39:041
179950,0migration/124287-21kthreadcore11:18:381
179950,0migration/122010-21latency_hist08:38:221
179950,0migration/11867-21munin-node12:03:371
169950,0watchdog/11125-21cat09:28:401
169950,0watchdog/10-21swapper/108:33:401
8215040,0irq/24-0000:00:5542-21netstat07:33:422
339940,0migration/39524-21runrttasks08:28:373
339940,0migration/332068-21taskset07:31:113
339940,0migration/328329-21taskset08:41:203
339940,0migration/31767-21kthreadcore07:08:393
339940,0migration/313000-21fschecks_count07:18:343
312409943,0cyclictest7974-21awk11:28:372
312409943,0cyclictest6819-21munin-node07:38:352
312409943,0cyclictest3556-21lldpd12:28:402
312409943,0cyclictest3556-21lldpd12:28:402
312409943,0cyclictest3556-21lldpd08:48:402
312409943,0cyclictest3556-21lldpd08:03:402
312409943,0cyclictest3556-21lldpd07:55:402
312409943,0cyclictest3556-21lldpd07:52:392
312409943,0cyclictest30689-21awk11:03:372
312409943,0cyclictest28584-21awk08:18:372
312409943,0cyclictest24057-21awk09:03:382
312409943,0cyclictest18985-21munin-node12:33:432
312409943,0cyclictest18985-21munin-node12:33:432
312399944,0cyclictest0-21swapper/112:28:401
312399944,0cyclictest0-21swapper/112:28:401
312399943,0cyclictest9729-21awk08:28:381
312399943,0cyclictest3880-21awk08:48:381
312399943,0cyclictest3514-21snmpd09:59:561
312399943,0cyclictest3514-21snmpd09:52:561
312399943,0cyclictest3514-21snmpd09:35:551
312399943,0cyclictest3514-21snmpd09:16:551
312399943,0cyclictest3514-21snmpd07:54:551
312399943,0cyclictest3514-21snmpd07:51:551
312399943,0cyclictest3514-21snmpd07:34:551
312399943,0cyclictest3514-21snmpd07:26:101
312399943,0cyclictest3514-21snmpd07:20:401
312399943,0cyclictest3514-21snmpd07:13:551
312399943,0cyclictest3241-21irqbalance11:43:301
312399943,0cyclictest3241-21irqbalance11:18:101
312399943,0cyclictest32341-21awk07:08:371
312399943,0cyclictest23945-21awk09:03:381
312399943,0cyclictest23246-21ssh11:54:461
312399943,0cyclictest19928-21awk10:18:371
312399943,0cyclictest17541-21awk09:38:381
312399943,0cyclictest17315-21awk08:58:371
312399943,0cyclictest16344-21awk11:33:361
312399943,0cyclictest12806-21awk10:33:371
312399943,0cyclictest10604-21awk08:53:351
312389943,0cyclictest3241-21irqbalance09:08:310
312389943,0cyclictest3241-21irqbalance07:28:310
312389943,0cyclictest2160-21ps12:23:320
312389943,0cyclictest11666-21tail12:28:370
312389943,0cyclictest11666-21tail12:28:370
312389942,0cyclictest9830-21aten_r1power_fr08:53:320
312389942,0cyclictest9588-21aten2.4-expect09:53:330
312389942,0cyclictest9573-21aten2.4-expect12:08:340
312389942,0cyclictest8685-21latency_hist08:28:220
312389942,0cyclictest8240-21egrep09:33:340
312389942,0cyclictest8142-21df08:03:390
312389942,0cyclictest7369-21expr11:28:350
312389942,0cyclictest7211-21df_inode07:38:350
312389942,0cyclictest6251-21df_inode11:08:340
312389942,0cyclictest5826-21aten2.4-expect09:13:340
312389942,0cyclictest4920-21df10:48:350
312389942,0cyclictest3411-21diskstats08:48:370
312389942,0cyclictest32654-21aten2.4-expect07:33:340
312389942,0cyclictest31942-21sort09:28:360
312389942,0cyclictest31773-21cut07:08:350
312389942,0cyclictest31513-21fschecks_count11:23:350
312389942,0cyclictest29811-21ssh11:03:310
312389942,0cyclictest28776-21aten2.4-expect08:43:340
312389942,0cyclictest28291-21port_sshpriv11:18:420
312389942,0cyclictest27912-21egrep08:18:340
312389942,0cyclictest27378-21sh10:23:310
312389942,0cyclictest26172-21sh10:03:240
312389942,0cyclictest25042-21ssh11:58:320
312389942,0cyclictest24289-21diskstats11:38:350
312389942,0cyclictest2319-21df08:23:350
312389942,0cyclictest23099-21cut09:03:340
312389942,0cyclictest22736-21perl08:38:370
312389942,0cyclictest21375-21sh10:58:300
312389942,0cyclictest20154-21sh10:38:280
312389942,0cyclictest19455-21df_inode10:18:350
312389942,0cyclictest19320-21cut12:33:330
312389942,0cyclictest19320-21cut12:33:330
312389942,0cyclictest19238-21sh07:23:340
312389942,0cyclictest18456-21perl12:13:350
312389942,0cyclictest16707-21df08:58:350
312389942,0cyclictest15860-21diskstats11:33:340
312389942,0cyclictest15556-21munin-node08:33:360
312389942,0cyclictest14686-21cpuspeed_turbos08:08:350
312389942,0cyclictest1444-21diskstats07:58:360
312389942,0cyclictest14290-21ssh09:18:300
312389942,0cyclictest14198-21sh11:13:270
312389942,0cyclictest1384-21df_abs12:03:360
312389942,0cyclictest13552-21telnet07:43:340
312389942,0cyclictest13178-21cut10:53:350
312389942,0cyclictest12568-21munin-node07:18:330
312389942,0cyclictest12208-21munin-node10:33:340
312389942,0cyclictest10996-21df10:13:330
312389941,0cyclictest28970-21diskstats10:43:350
312389941,0cyclictest20805-21perl07:48:360
29725040,0irq/27-eth0-tx-9020-21kthreadcore11:08:392
29725040,0irq/27-eth0-tx-16635-21ssh09:38:302
29725040,0irq/27-eth0-tx-0-21swapper/212:18:352
29725040,0irq/27-eth0-tx-0-21swapper/212:08:292
29725040,0irq/27-eth0-tx-0-21swapper/211:43:242
29725040,0irq/27-eth0-tx-0-21swapper/211:33:232
29725040,0irq/27-eth0-tx-0-21swapper/210:53:222
29725040,0irq/27-eth0-tx-0-21swapper/210:03:232
29725040,0irq/27-eth0-tx-0-21swapper/209:58:362
29715040,0irq/26-eth0-rx-0-21swapper/310:43:383
29715040,0irq/26-eth0-rx-0-21swapper/310:38:423
29715040,0irq/26-eth0-rx-0-21swapper/310:19:003
29715040,0irq/26-eth0-rx-0-21swapper/309:08:423
259940,0migration/232005-21fschecks_time07:08:362
259940,0migration/224190-21kthreadcore08:13:412
259940,0migration/223951-21aten_r1power_fr11:38:342
259940,0migration/221651-21kthreadcore07:23:392
259940,0migration/218876-21kthreadcore11:13:422
259940,0migration/215767-21taskset08:54:032
249940,0watchdog/25806-21munin-node07:13:452
249940,0watchdog/225319-21tr10:58:402
249940,0watchdog/212602-21aten_r1power_po07:18:332
249940,0watchdog/20-21swapper/208:38:402
249940,0watchdog/20-21swapper/208:33:402
179940,0migration/19899-21kthreadcore11:28:391
179940,0migration/17151-21df_abs07:38:351
179940,0migration/16780-21idleruntime-cro11:28:221
179940,0migration/132698-21kthreadcore10:43:401
179940,0migration/132314-21kthreadcore11:03:391
179940,0migration/131466-21munin-node07:28:451
179940,0migration/129217-21kthreadcore10:23:411
179940,0migration/123689-21kthreadcore12:33:381
179940,0migration/123689-21kthreadcore12:33:381
179940,0migration/120934-21kthreadcore12:13:391
179940,0migration/120524-21sh09:18:481
179940,0migration/113475-21taskset07:43:261
179940,0migration/110649-21kthreadcore11:08:391
179940,0migration/110353-21kthreadcore08:03:431
169940,0watchdog/19257-21sensors_volt10:28:441
169940,0watchdog/129479-21diskmemload11:53:161
169940,0watchdog/129479-21diskmemload09:53:281
169940,0watchdog/123649-21kthreadcore08:13:411
169940,0watchdog/122010-21tune2fs10:58:361
169940,0watchdog/119563-21kthreadcore08:08:411
169940,0watchdog/116795-21users10:13:441
169940,0watchdog/10-21swapper/111:38:401
169940,0watchdog/10-21swapper/110:53:401
169940,0watchdog/10-21swapper/109:23:401
169940,0watchdog/10-21swapper/108:23:371
35130,0ktimersoftd/30-21swapper/308:10:453
29725030,0irq/27-eth0-tx-0-21swapper/207:28:362
29715030,0irq/26-eth0-rx-0-21swapper/309:03:343
29715030,0irq/26-eth0-rx-0-21swapper/309:02:403
29715030,0irq/26-eth0-rx-0-21swapper/308:43:463
29715030,0irq/26-eth0-rx-0-21swapper/308:13:353
29715030,0irq/26-eth0-rx-0-21swapper/308:02:393
29715030,0irq/26-eth0-rx-0-21swapper/307:54:523
29715030,0irq/26-eth0-rx-0-21swapper/307:33:343
29715030,0irq/26-eth0-rx-0-21swapper/307:23:343
249930,0watchdog/20-21swapper/208:23:372
179930,0migration/129172-21fschecks_count08:43:371
169930,0watchdog/10-21swapper/112:08:401
169930,0watchdog/10-21swapper/109:43:281
339920,0migration/33012-21iostat_ios08:23:383
169920,0watchdog/10-21swapper/108:18:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional