You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 00:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Dec 27, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3001628685,0sleep20-21swapper/207:03:532
3034628580,0sleep00-21swapper/007:03:550
304450660,0irq/25-eth10-21swapper/307:06:593
392523324,0sleep10-21swapper/107:07:411
149970,0migration/031857-21sh10:21:000
40819960,0cyclictest0-21swapper/110:31:161
40809960,0cyclictest0-21swapper/012:34:210
40809960,0cyclictest0-21swapper/012:22:120
339960,0migration/36407-21sh12:19:183
339960,0migration/35395-21sh12:00:003
339960,0migration/332648-21sh10:22:543
339960,0migration/329952-21sh09:42:453
339960,0migration/321033-21missed_timers12:08:563
339960,0migration/317296-21sh10:50:113
339960,0migration/317288-21missed_timers07:38:583
339960,0migration/317206-21sh09:14:193
339960,0migration/313557-21sh11:47:323
29715060,0irq/26-eth0-rx-9766-21diskstats10:28:523
29715060,0irq/26-eth0-rx-8640-21unixbench-2d09:09:023
29715060,0irq/26-eth0-rx-788-21sh10:23:463
29715060,0irq/26-eth0-rx-29436-21ssh11:38:243
149960,0migration/03988-21sh11:22:510
149960,0migration/029059-21sh11:54:220
149960,0migration/019722-21sh11:13:120
149960,0migration/017417-21sh10:33:270
149960,0migration/014055-21sh12:05:360
149960,0migration/012222-21sh11:44:210
40829950,0cyclictest0-21swapper/211:56:262
40829950,0cyclictest0-21swapper/211:50:452
40829950,0cyclictest0-21swapper/211:41:272
40829950,0cyclictest0-21swapper/210:01:302
40819955,0cyclictest0-21swapper/112:37:141
40819955,0cyclictest0-21swapper/111:04:321
40819955,0cyclictest0-21swapper/110:28:251
40819955,0cyclictest0-21swapper/109:23:381
40819950,0cyclictest0-21swapper/112:07:451
40819950,0cyclictest0-21swapper/111:41:061
40819950,0cyclictest0-21swapper/110:07:181
40819950,0cyclictest0-21swapper/109:58:191
40809955,0cyclictest0-21swapper/010:59:070
40809955,0cyclictest0-21swapper/008:39:070
40809950,0cyclictest0-21swapper/011:30:010
40809950,0cyclictest0-21swapper/010:56:170
40809950,0cyclictest0-21swapper/009:43:080
4150,0ktimersoftd/00-21swapper/009:53:130
339950,0migration/36531-21kthreadcore11:03:553
339950,0migration/35611-21if_eth1.211:23:553
339950,0migration/34588-21latency_hist09:28:393
339950,0migration/332645-21needreboot08:38:583
339950,0migration/330610-21kthreadcore09:03:553
339950,0migration/330482-21fschecks_time11:38:513
339950,0migration/325608-21kthreadcore08:58:573
339950,0migration/323983-21kthreadcore10:53:573
339950,0migration/323653-21cpu09:58:513
339950,0migration/32275-21munin-node10:43:503
339950,0migration/321473-21munin-node08:33:523
339950,0migration/321297-21kthreadcore08:08:573
339950,0migration/319088-21kthreadcore09:53:573
339950,0migration/318892-21kthreadcore08:53:583
339950,0migration/314315-21kthreadcore07:13:593
339950,0migration/314135-21munin-node12:23:583
29725050,0irq/27-eth0-tx-23259-21irqcore09:38:572
29725050,0irq/27-eth0-tx-19064-21ntp_offset09:33:592
29715050,0irq/26-eth0-rx-8751-21grep07:58:573
29715050,0irq/26-eth0-rx-32354-21forks10:03:543
29715050,0irq/26-eth0-rx-28724-21ssh11:18:523
29715050,0irq/26-eth0-rx-28044-21uniq10:58:543
29715050,0irq/26-eth0-rx-18319-21ssh11:09:513
29715050,0irq/26-eth0-rx-0-21swapper/312:33:383
29715050,0irq/26-eth0-rx-0-21swapper/312:30:503
29715050,0irq/26-eth0-rx-0-21swapper/312:13:573
29715050,0irq/26-eth0-rx-0-21swapper/311:49:573
29715050,0irq/26-eth0-rx-0-21swapper/311:30:183
29715050,0irq/26-eth0-rx-0-21swapper/311:17:513
29715050,0irq/26-eth0-rx-0-21swapper/310:41:213
29715050,0irq/26-eth0-rx-0-21swapper/310:37:333
29715050,0irq/26-eth0-rx-0-21swapper/310:13:563
29715050,0irq/26-eth0-rx-0-21swapper/309:49:113
29715050,0irq/26-eth0-rx-0-21swapper/309:44:133
29715050,0irq/26-eth0-rx-0-21swapper/309:34:193
29715050,0irq/26-eth0-rx-0-21swapper/309:27:583
29715050,0irq/26-eth0-rx-0-21swapper/309:21:073
249950,0watchdog/20-21swapper/207:18:582
19150,0ktimersoftd/10-21swapper/110:19:571
179950,0migration/117469-21kthreadcore12:08:551
179950,0migration/114000-21sh09:51:421
169950,0watchdog/123809-21ssh12:30:481
169950,0watchdog/10-21swapper/109:43:571
149950,0migration/08527-21taskset12:23:460
149950,0migration/06800-21taskset08:20:500
149950,0migration/05864-21irqcore09:28:550
149950,0migration/04797-21kthreadcore10:43:550
149950,0migration/04469-21kthreadcore10:03:580
149950,0migration/030251-21kthreadcore09:23:550
149950,0migration/028224-21latency_hist09:03:390
149950,0migration/027852-21kthreadcore08:13:560
149950,0migration/027304-21kthreadcore07:23:570
149950,0migration/026118-21sh11:14:040
149950,0migration/025398-21kthreadcore09:58:560
149950,0migration/023779-21kthreadcore08:58:560
149950,0migration/021933-21kthreadcore08:08:570
149950,0migration/021198-21kthreadcore09:18:560
149950,0migration/021193-21munin-node12:08:580
149950,0migration/020837-21kthreadcore10:13:570
149950,0migration/018687-21kthreadcore12:28:540
149950,0migration/018588-21kthreadcore08:53:580
149950,0migration/017621-21sendmail_mailqu07:39:010
149950,0migration/015966-21idleruntime09:53:540
149950,0migration/013983-21fschecks_count09:33:520
149950,0migration/013132-21timerwakeupswit08:24:020
149950,0migration/013038-21kthreadcore09:13:560
149950,0migration/01214-21kthreadcore09:43:570
149950,0migration/011495-21idleruntime10:48:530
8215040,0irq/24-0000:00:18450-21cat07:43:402
40829944,0cyclictest0-21swapper/209:24:362
40829943,0cyclictest7572-21awk09:48:532
40829943,0cyclictest4294-21awk11:03:522
40829943,0cyclictest3556-21lldpd08:51:162
40829943,0cyclictest352-21awk10:03:552
40829943,0cyclictest3514-21snmpd12:04:092
40829943,0cyclictest3514-21snmpd10:19:542
40829943,0cyclictest3514-21snmpd08:39:232
40829943,0cyclictest3514-21snmpd08:06:532
40829943,0cyclictest3514-21snmpd07:28:522
40829943,0cyclictest3514-21snmpd07:25:522
40829943,0cyclictest3514-21snmpd07:14:522
40829943,0cyclictest3241-21irqbalance12:34:472
40829943,0cyclictest3241-21irqbalance09:14:072
40829943,0cyclictest2354-21awk08:43:542
40829943,0cyclictest22594-21awk08:58:552
40829943,0cyclictest20565-21awk09:18:552
40829943,0cyclictest19931-21awk10:53:542
40829943,0cyclictest17405-21awk10:13:552
40829943,0cyclictest1699-21awk10:23:542
40829943,0cyclictest16025-21awk09:53:552
40829943,0cyclictest12701-21awk11:08:542
40829943,0cyclictest11730-21sadc07:38:402
40829940,0cyclictest5914-21munin-node11:44:042
40819944,0cyclictest0-21swapper/111:54:491
40819944,0cyclictest0-21swapper/111:44:441
40819943,0cyclictest5659-21awk09:28:541
40819943,0cyclictest3514-21snmpd08:26:081
40819943,0cyclictest3514-21snmpd08:23:081
40819943,0cyclictest3514-21snmpd07:31:071
40819943,0cyclictest3514-21snmpd07:20:071
40819943,0cyclictest3514-21snmpd07:09:221
40819943,0cyclictest3514-21snmpd07:09:221
40819943,0cyclictest3241-21irqbalance11:14:171
40819943,0cyclictest3241-21irqbalance09:09:471
40819943,0cyclictest29162-21awk09:03:531
40819943,0cyclictest28113-21awk08:38:541
40819943,0cyclictest27061-21awk10:38:531
40819943,0cyclictest25194-21awk07:23:551
40819943,0cyclictest24986-21awk12:13:551
40819943,0cyclictest16623-21sadc12:28:381
40819943,0cyclictest12816-21awk07:38:541
40819943,0cyclictest11820-21awk09:13:541
40809944,0cyclictest0-21swapper/007:58:360
40809943,0cyclictest25031-21sadc07:48:390
40809943,0cyclictest18754-21awk10:33:540
40809943,0cyclictest1773-21awk10:23:540
40809943,0cyclictest15299-21awk11:48:530
40809940,0cyclictest0-21swapper/007:49:580
339940,0migration/39079-21kthreadcore07:33:563
339940,0migration/34808-21kthreadcore07:53:573
339940,0migration/329985-21switchtime07:24:023
339940,0migration/328029-21kthreadcore11:53:563
339940,0migration/318758-21irqcore07:18:573
339940,0migration/31753-21irqrtprio08:18:553
339940,0migration/311643-21kthreadcore08:48:563
339940,0migration/310412-21taskset07:09:483
339940,0migration/310412-21taskset07:09:483
29725040,0irq/27-eth0-tx-31900-21gltestperf11:58:522
29725040,0irq/27-eth0-tx-29405-21ssh12:13:572
29725040,0irq/27-eth0-tx-28278-21munin-node09:03:572
29725040,0irq/27-eth0-tx-23248-21ssh12:29:302
29725040,0irq/27-eth0-tx-2302-21diskmemload10:33:392
29725040,0irq/27-eth0-tx-2110-21ssh10:43:412
29725040,0irq/27-eth0-tx-0-21swapper/212:23:532
29725040,0irq/27-eth0-tx-0-21swapper/212:19:162
29725040,0irq/27-eth0-tx-0-21swapper/212:08:522
29725040,0irq/27-eth0-tx-0-21swapper/211:28:532
29725040,0irq/27-eth0-tx-0-21swapper/211:18:412
29725040,0irq/27-eth0-tx-0-21swapper/211:13:582
29725040,0irq/27-eth0-tx-0-21swapper/210:49:212
29725040,0irq/27-eth0-tx-0-21swapper/210:39:402
29725040,0irq/27-eth0-tx-0-21swapper/210:08:462
29725040,0irq/27-eth0-tx-0-21swapper/209:30:502
29725040,0irq/27-eth0-tx-0-21swapper/209:08:562
29725040,0irq/27-eth0-tx-0-21swapper/208:24:022
29715040,0irq/26-eth0-rx-0-21swapper/312:03:533
29715040,0irq/26-eth0-rx-0-21swapper/310:08:423
29715040,0irq/26-eth0-rx-0-21swapper/307:48:573
259940,0migration/24808-21fschecks_count07:08:532
259940,0migration/24808-21fschecks_count07:08:532
259940,0migration/227801-21kthreadcore08:13:562
259940,0migration/22741-21kthreadcore09:43:572
259940,0migration/224677-21runrttasks07:44:372
259940,0migration/221336-21latency_hist11:33:382
259940,0migration/215766-21idleruntime08:53:552
259940,0migration/210025-21fschecks_time10:28:532
249940,0watchdog/232751-21idleruntime07:53:532
249940,0watchdog/226029-21hddtemp_smartct07:48:532
249940,0watchdog/220939-21kthreadcore08:08:572
249940,0watchdog/21264-21ssh10:59:252
249940,0watchdog/211316-21kthreadcore07:58:572
249940,0watchdog/20-21swapper/211:23:572
179940,0migration/16285-21taskset07:33:541
179940,0migration/132366-21taskset08:14:501
179940,0migration/132105-21cpu07:53:511
179940,0migration/130593-21missed_timers07:48:581
179940,0migration/129342-21taskset11:38:101
179940,0migration/127584-21unixbench_singl08:59:031
179940,0migration/127401-21aten_r1power_cu10:58:511
179940,0migration/124283-21taskset10:34:521
179940,0migration/116528-21munin-node07:14:021
179940,0migration/114371-21aten_r1power_en11:48:491
179940,0migration/112091-21cpuspeed_turbos11:08:511
179940,0migration/111007-21kthreadcore07:58:571
179940,0migration/110703-21kthreadcore08:48:561
169940,0watchdog/130582-21ssh10:01:091
169940,0watchdog/129735-21irqcore11:18:571
169940,0watchdog/128479-21ssh09:39:251
169940,0watchdog/12302-21diskmemload10:13:571
169940,0watchdog/120032-21diskstats08:08:531
169940,0watchdog/116258-21cut08:53:571
169940,0watchdog/10-21swapper/111:23:571
169940,0watchdog/10-21swapper/110:53:571
169940,0watchdog/10-21swapper/109:35:131
169940,0watchdog/10-21swapper/109:24:331
169940,0watchdog/10-21swapper/108:43:571
169940,0watchdog/10-21swapper/108:03:571
169940,0watchdog/10-21swapper/107:43:581
149940,0migration/09354-21timerandwakeup11:04:000
149940,0migration/0684-21kthreadcore11:58:550
149940,0migration/03391-21kthreadcore07:28:570
149940,0migration/027768-21kthreadcore12:13:570
149940,0migration/026812-21latency11:33:570
149940,0migration/02622-21kthreadcore11:38:560
149940,0migration/013787-21kthreadcore07:13:580
40829930,0cyclictest0-21swapper/208:33:512
40819932,0cyclictest3556-21lldpd11:58:481
40819932,0cyclictest3514-21snmpd11:28:541
40819930,0cyclictest7961-21port_sshpriv10:43:591
40809933,0cyclictest0-21swapper/008:28:430
40809933,0cyclictest0-21swapper/007:10:590
40809933,0cyclictest0-21swapper/007:10:590
40809932,0cyclictest9781-21kthreadcore10:08:570
40809932,0cyclictest3514-21snmpd10:41:240
40809932,0cyclictest3514-21snmpd08:06:240
40809932,0cyclictest3514-21snmpd07:18:520
40809932,0cyclictest2302-21diskmemload11:26:340
40809932,0cyclictest11952-21sensors07:59:020
40809930,0cyclictest0-21swapper/008:49:590
29725030,0irq/27-eth0-tx-0-21swapper/208:18:562
29715030,0irq/26-eth0-rx-0-21swapper/308:43:573
29715030,0irq/26-eth0-rx-0-21swapper/308:23:503
29715030,0irq/26-eth0-rx-0-21swapper/308:18:393
29715030,0irq/26-eth0-rx-0-21swapper/308:03:533
29715030,0irq/26-eth0-rx-0-21swapper/307:43:523
29715030,0irq/26-eth0-rx-0-21swapper/307:28:503
249930,0watchdog/20-21swapper/208:29:012
179930,0migration/111210-21fschecks_count10:48:521
169930,0watchdog/119604-21sendmail_mailst08:29:011
169930,0watchdog/10-21swapper/112:19:121
169930,0watchdog/10-21swapper/110:08:571
169930,0watchdog/10-21swapper/108:33:571
40809921,0cyclictest41ktimersoftd/007:33:400
40809921,0cyclictest0-21swapper/008:43:500
40809921,0cyclictest0-21swapper/008:33:390
40809920,0cyclictest0-21swapper/009:08:430
339920,0migration/314938-21iostat_ios08:28:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional