You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 22:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 16, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2598426564,0sleep20-21swapper/206:56:492
2576523829,0sleep30-21swapper/306:54:143
2589123723,0sleep00-21swapper/006:55:430
2600923425,0sleep10-21swapper/106:57:041
4160,0ktimersoftd/00-21swapper/011:29:570
35160,0ktimersoftd/30-21swapper/311:29:573
339960,0migration/331012-21sh10:21:023
339960,0migration/329324-21sh12:00:203
339960,0migration/323718-21sh10:38:263
339960,0migration/323197-21sh09:55:453
339960,0migration/315180-21sh10:11:393
339960,0migration/314217-21sh11:11:283
29715060,0irq/26-eth0-rx-30187-21sh11:00:263
261879966,0cyclictest0-21swapper/210:27:532
261879960,0cyclictest0-21swapper/210:42:522
261879960,0cyclictest0-21swapper/207:13:392
261859966,0cyclictest24339-21diskmemload09:24:171
261859960,0cyclictest24339-21diskmemload09:59:361
261859960,0cyclictest0-21swapper/111:56:351
261859960,0cyclictest0-21swapper/111:24:441
261849965,0cyclictest0-21swapper/010:51:070
149960,0migration/030692-21sh09:59:000
8215050,0irq/24-0000:00:30042-21latency_hist08:38:182
339950,0migration/39308-21kthreadcore10:48:353
339950,0migration/35914-21kthreadcore07:28:343
339950,0migration/331110-21fschecks_count12:03:303
339950,0migration/330412-21munin-node08:38:303
339950,0migration/327719-21kthreadcore08:58:353
339950,0migration/323673-21fschecks_count11:18:313
339950,0migration/323092-21taskset09:13:583
339950,0migration/322339-21taskset12:18:243
339950,0migration/321225-21taskset07:40:293
339950,0migration/317353-21taskset08:49:343
339950,0migration/317313-21kthreadcore11:53:353
339950,0migration/316073-21irqrtprio11:33:343
339950,0migration/315925-21latency_hist08:03:183
339950,0migration/314774-21unixbench_singl09:08:413
339950,0migration/312797-21kthreadcore08:23:363
339950,0migration/31064-21munin-node09:03:323
339950,0migration/310247-21kthreadcore09:48:343
29725050,0irq/27-eth0-tx-9356-21munin-node07:58:310
29725050,0irq/27-eth0-tx-8653-21munin-node09:08:302
29725050,0irq/27-eth0-tx-7809-21ssh09:06:552
29725050,0irq/27-eth0-tx-32378-21munin-node10:03:392
29725050,0irq/27-eth0-tx-29525-21ssh12:21:342
29725050,0irq/27-eth0-tx-24995-21ssh08:58:310
29725050,0irq/27-eth0-tx-23924-21kthreadcore11:58:322
29725050,0irq/27-eth0-tx-16009-21munin-node08:03:400
29725050,0irq/27-eth0-tx-14794-21ssh11:33:192
29725050,0irq/27-eth0-tx-0-21swapper/211:48:322
29725050,0irq/27-eth0-tx-0-21swapper/210:50:302
29725050,0irq/27-eth0-tx-0-21swapper/210:43:242
29725050,0irq/27-eth0-tx-0-21swapper/210:21:302
29725050,0irq/27-eth0-tx-0-21swapper/210:08:332
29725050,0irq/27-eth0-tx-0-21swapper/209:35:062
29725050,0irq/27-eth0-tx-0-21swapper/008:53:290
29725050,0irq/27-eth0-tx-0-21swapper/008:30:090
29725050,0irq/27-eth0-tx-0-21swapper/007:05:430
29715050,0irq/26-eth0-rx-6691-21ssh10:05:193
29715050,0irq/26-eth0-rx-4129-21users12:23:423
29715050,0irq/26-eth0-rx-3881-21memory11:43:363
29715050,0irq/26-eth0-rx-30219-21munin-node09:58:403
29715050,0irq/26-eth0-rx-21595-21ssh11:14:383
29715050,0irq/26-eth0-rx-1493-21kthreadcore11:23:363
29715050,0irq/26-eth0-rx-1137-21grep10:23:363
29715050,0irq/26-eth0-rx-0-21swapper/311:04:073
29715050,0irq/26-eth0-rx-0-21swapper/310:54:013
29715050,0irq/26-eth0-rx-0-21swapper/310:37:043
29715050,0irq/26-eth0-rx-0-21swapper/310:30:093
29715050,0irq/26-eth0-rx-0-21swapper/310:13:353
29715050,0irq/26-eth0-rx-0-21swapper/309:43:343
261879955,0cyclictest0-21swapper/211:03:122
261879955,0cyclictest0-21swapper/210:53:352
261879955,0cyclictest0-21swapper/209:17:352
261879950,0cyclictest0-21swapper/211:56:582
261879950,0cyclictest0-21swapper/211:14:432
261859955,0cyclictest24339-21diskmemload11:00:561
261859955,0cyclictest0-21swapper/112:10:181
261859955,0cyclictest0-21swapper/109:49:501
261859955,0cyclictest0-21swapper/109:37:041
261859955,0cyclictest0-21swapper/109:06:571
261859950,0cyclictest0-21swapper/112:21:491
261859950,0cyclictest0-21swapper/111:49:111
261859950,0cyclictest0-21swapper/111:39:341
261859950,0cyclictest0-21swapper/111:12:271
261859950,0cyclictest0-21swapper/110:27:501
261849955,0cyclictest0-21swapper/011:20:260
261849955,0cyclictest0-21swapper/010:39:180
261849950,0cyclictest0-21swapper/010:38:110
261849950,0cyclictest0-21swapper/010:15:100
261849950,0cyclictest0-21swapper/010:05:240
261849950,0cyclictest0-21swapper/009:31:110
259950,0migration/2534-21rt-features07:23:402
259950,0migration/231390-21sh09:00:302
259950,0migration/230984-21kthreadcore07:48:342
259950,0migration/229234-21latency_hist08:13:182
259950,0migration/228066-21kthreadcore09:38:332
259950,0migration/226998-21missed_timers07:43:372
259950,0migration/222840-21sh10:15:432
259950,0migration/217334-21kthreadcore12:13:322
259950,0migration/215011-21latency_hist07:38:182
259950,0migration/214730-21sh09:49:152
259950,0migration/213036-21sh11:28:502
259950,0migration/212511-21kthreadcore07:08:352
19150,0ktimersoftd/10-21swapper/111:29:571
179950,0migration/19869-21kthreadcore10:28:341
179950,0migration/18243-21latency_hist09:48:181
179950,0migration/126701-21cpuspeed_turbos06:58:291
179950,0migration/114047-21kthreadcore07:58:361
169950,0watchdog/128608-21ls11:18:361
149950,0migration/09370-21kthreadcore11:08:360
149950,0migration/0809-21fschecks_count09:03:310
149950,0migration/05404-21kthreadcore07:53:360
149950,0migration/03288-21kthreadcore10:23:360
149950,0migration/031909-21aten_r1power_vo10:43:310
149950,0migration/02847-21kthreadcore12:23:360
149950,0migration/023976-21kthreadcore08:08:340
149950,0migration/021650-21sh11:14:470
149950,0migration/021095-21processes10:53:390
149950,0migration/016657-21kthreadcore11:33:350
149950,0migration/013879-21kthreadcore08:23:360
149950,0migration/013410-21taskset12:11:220
339940,0migration/36383-21kthreadcore08:18:363
339940,0migration/320734-21kthreadcore08:53:353
339940,0migration/319266-21kthreadcore09:33:343
339940,0migration/316336-21kthreadcore07:13:333
339940,0migration/31451-21hwlatdetect07:03:333
29725040,0irq/27-eth0-tx-4529-21ssh12:04:052
29725040,0irq/27-eth0-tx-3568-21munin-node08:18:370
29725040,0irq/27-eth0-tx-0-21swapper/212:23:492
29725040,0irq/27-eth0-tx-0-21swapper/211:43:192
29725040,0irq/27-eth0-tx-0-21swapper/211:23:342
29725040,0irq/27-eth0-tx-0-21swapper/211:18:272
29725040,0irq/27-eth0-tx-0-21swapper/211:03:362
29725040,0irq/27-eth0-tx-0-21swapper/210:34:502
29725040,0irq/27-eth0-tx-0-21swapper/210:29:122
29725040,0irq/27-eth0-tx-0-21swapper/209:58:452
29725040,0irq/27-eth0-tx-0-21swapper/209:28:342
29725040,0irq/27-eth0-tx-0-21swapper/008:33:300
29715040,0irq/26-eth0-rx-0-21swapper/312:14:103
29715040,0irq/26-eth0-rx-0-21swapper/312:09:053
29715040,0irq/26-eth0-rx-0-21swapper/311:48:353
29715040,0irq/26-eth0-rx-0-21swapper/311:38:233
29715040,0irq/26-eth0-rx-0-21swapper/310:44:063
29715040,0irq/26-eth0-rx-0-21swapper/309:38:333
29715040,0irq/26-eth0-rx-0-21swapper/309:28:373
29715040,0irq/26-eth0-rx-0-21swapper/309:23:213
29715040,0irq/26-eth0-rx-0-21swapper/309:19:143
29715040,0irq/26-eth0-rx-0-21swapper/307:53:313
29715040,0irq/26-eth0-rx-0-21swapper/306:58:373
261879944,0cyclictest0-21swapper/211:38:502
261879943,0cyclictest7227-21awk12:08:332
261879943,0cyclictest5457-21awk08:43:322
261879943,0cyclictest3514-21snmpd08:48:252
261879943,0cyclictest3514-21snmpd08:22:252
261879943,0cyclictest3514-21snmpd08:11:252
261879943,0cyclictest3514-21snmpd08:06:402
261879943,0cyclictest3514-21snmpd07:08:102
261879943,0cyclictest3241-21irqbalance11:08:262
261879943,0cyclictest1557-21awk09:23:342
261859943,0cyclictest3556-21lldpd12:25:381
261859943,0cyclictest3556-21lldpd10:22:091
261859943,0cyclictest3556-21lldpd10:09:391
261859943,0cyclictest3556-21lldpd08:41:391
261859943,0cyclictest3556-21lldpd08:09:071
261859943,0cyclictest3514-21snmpd10:42:101
261859943,0cyclictest3304-21sadc08:18:181
261859943,0cyclictest3241-21irqbalance12:03:361
261859943,0cyclictest3241-21irqbalance12:01:561
261859943,0cyclictest3241-21irqbalance11:05:061
261859943,0cyclictest25560-21awk08:58:331
261859943,0cyclictest21606-21awk07:18:321
261859943,0cyclictest17801-21awk08:28:321
261859943,0cyclictest16084-21awk11:13:341
261859943,0cyclictest1529-21awk07:03:341
261859940,0cyclictest0-21swapper/110:44:011
261849944,0cyclictest0-21swapper/009:19:310
261849943,0cyclictest8923-21awk10:08:310
261849943,0cyclictest8229-21awk07:08:320
261849943,0cyclictest7518-21awk11:48:330
261849943,0cyclictest3241-21irqbalance11:58:260
261849943,0cyclictest3241-21irqbalance11:55:060
261849943,0cyclictest3241-21irqbalance11:41:460
261849943,0cyclictest3241-21irqbalance11:23:560
261849943,0cyclictest3241-21irqbalance10:58:460
261849943,0cyclictest3241-21irqbalance09:49:060
261849943,0cyclictest3241-21irqbalance09:43:360
261849943,0cyclictest3241-21irqbalance08:48:270
261849943,0cyclictest3241-21irqbalance07:43:370
261849943,0cyclictest3241-21irqbalance07:34:170
261849943,0cyclictest3241-21irqbalance07:18:570
261849943,0cyclictest3241-21irqbalance07:14:170
261849943,0cyclictest3241-21irqbalance06:59:070
261849943,0cyclictest31725-21awk11:43:320
261849943,0cyclictest25600-21awk09:38:300
261849943,0cyclictest17821-21awk09:13:320
261849943,0cyclictest17774-21awk09:33:330
261849943,0cyclictest16013-21awk07:38:330
259940,0migration/29772-21kthreadcore07:33:342
259940,0migration/228295-21kthreadcore09:18:342
259940,0migration/224334-21sh08:58:182
259940,0migration/218899-21kthreadcore08:28:342
259940,0migration/215977-21smart_sda08:23:402
249940,0watchdog/26017-21grep07:53:372
249940,0watchdog/20-21swapper/209:53:322
249940,0watchdog/20-21swapper/209:43:322
249940,0watchdog/20-21swapper/207:58:332
179940,0migration/16517-21kthreadcore08:43:341
179940,0migration/14083-21fschecks_count08:18:321
179940,0migration/1383-21fschecks_count10:03:311
179940,0migration/130955-21aten_r1power_po11:43:291
179940,0migration/127122-21kthreadcore09:38:321
179940,0migration/123716-21aten_r1power_po08:33:301
179940,0migration/122710-21switchtime09:13:401
179940,0migration/121640-21kthreadcore08:53:361
179940,0migration/118823-21kthreadcore10:33:361
179940,0migration/118031-21kthreadcore08:03:341
179940,0migration/116293-21idleruntime10:53:331
16716240,0sleep119921-21kthreadcore07:38:351
169940,0watchdog/122262-21fschecks_time07:43:331
169940,0watchdog/116926-21if_eth010:13:321
169940,0watchdog/115639-21ssh09:30:441
169940,0watchdog/110616-21kthreadcore10:48:361
169940,0watchdog/10-21swapper/109:53:321
169940,0watchdog/10-21swapper/107:23:331
149940,0migration/07964-21kthreadcore08:43:340
149940,0migration/0540-21kthreadcore11:03:340
149940,0migration/029552-21kthreadcore07:23:360
149940,0migration/029259-21latency_hist08:13:180
149940,0migration/02846-21irqrtprio07:28:330
149940,0migration/027722-21latency12:18:350
149940,0migration/02582-21kthreadcore08:38:360
149940,0migration/018641-21kthreadcore09:53:360
149940,0migration/01360-21taskset07:48:360
149940,0migration/013139-21kthreadcore09:08:350
149940,0migration/012392-21kthreadcore10:28:350
29715030,0irq/26-eth0-rx-0-21swapper/308:44:083
29715030,0irq/26-eth0-rx-0-21swapper/308:35:173
29715030,0irq/26-eth0-rx-0-21swapper/308:13:293
29715030,0irq/26-eth0-rx-0-21swapper/308:08:293
29715030,0irq/26-eth0-rx-0-21swapper/307:50:473
29715030,0irq/26-eth0-rx-0-21swapper/307:46:493
29715030,0irq/26-eth0-rx-0-21swapper/307:33:283
29715030,0irq/26-eth0-rx-0-21swapper/307:18:293
29715030,0irq/26-eth0-rx-0-21swapper/307:08:283
261879933,0cyclictest0-21swapper/206:58:502
261879932,0cyclictest21207-21fschecks_count07:18:312
261879930,0cyclictest0-21swapper/208:38:312
261859932,0cyclictest7377-21expr07:28:381
261859932,0cyclictest25835-21awk09:18:321
261859932,0cyclictest16755-21kthreadcore11:33:351
261859932,0cyclictest14953-21awk07:13:321
261859932,0cyclictest14039-21sensors07:33:391
261849932,0cyclictest688-21basename09:23:300
261849932,0cyclictest3514-21snmpd12:16:410
261849932,0cyclictest29823-21seq10:18:400
261849930,0cyclictest31027-21diskstats12:03:290
169930,0watchdog/124339-21diskmemload12:14:041
169930,0watchdog/10-21swapper/109:08:321
169930,0watchdog/10-21swapper/108:48:321
169930,0watchdog/10-21swapper/108:23:321
169930,0watchdog/10-21swapper/107:53:371
169930,0watchdog/10-21swapper/107:48:291
169930,0watchdog/10-21swapper/107:08:291
339920,0migration/327779-21diskstats07:23:323
29715020,0irq/26-eth0-rx-0-21swapper/308:28:293
29715020,0irq/26-eth0-rx-0-21swapper/307:58:383
261879921,0cyclictest1543-21sed07:28:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional