You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 02:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon May 18, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3276624536,0sleep10-21swapper/118:51:261
59223521,0sleep00-21swapper/018:54:490
304450350,0irq/25-eth10-21swapper/318:50:203
58923324,0sleep20-21swapper/218:54:472
7109976,0cyclictest0-21swapper/023:05:470
29715070,0irq/26-eth0-rx-0-21swapper/322:36:343
7119966,0cyclictest31394-21diskmemload23:06:401
7119960,0cyclictest0-21swapper/122:40:011
7109960,0cyclictest0-21swapper/022:05:240
339960,0migration/332609-21irqcore22:15:363
339960,0migration/331484-21sh21:34:513
339960,0migration/329892-21sh22:52:323
339960,0migration/328545-21timerandwakeup23:30:443
339960,0migration/323822-21aten_r1power_po21:50:313
339960,0migration/31383-21munin-node18:55:333
339960,0migration/312608-21rt-features23:00:413
29725060,0irq/27-eth0-tx-15365-21munin-node19:30:410
29715060,0irq/26-eth0-rx-3653-21grep22:55:383
29715060,0irq/26-eth0-rx-15832-21fschecks_count22:25:333
179960,0migration/120103-21sendmail_mailqu23:45:421
149960,0migration/05422-21sh00:19:560
149960,0migration/021801-21sh22:47:180
149960,0migration/021628-21sh00:09:370
149960,0migration/014230-21sh23:04:160
7159950,0cyclictest0-21swapper/221:10:002
7119955,0cyclictest31394-21diskmemload00:12:341
7119955,0cyclictest0-21swapper/123:51:451
7119955,0cyclictest0-21swapper/122:44:541
7119955,0cyclictest0-21swapper/122:30:151
7119955,0cyclictest0-21swapper/122:01:051
7119950,0cyclictest0-21swapper/122:16:221
7119950,0cyclictest0-21swapper/121:01:481
7119950,0cyclictest0-21swapper/100:04:121
7109955,0cyclictest0-21swapper/023:51:400
7109955,0cyclictest0-21swapper/023:38:060
7109954,0cyclictest609-21tr21:15:350
7109950,0cyclictest0-21swapper/000:02:130
4150,0ktimersoftd/00-21swapper/021:40:580
339950,0migration/38072-21munin-node19:20:453
339950,0migration/3600-21kthreadcore19:40:373
339950,0migration/35951-21taskset21:56:483
339950,0migration/3422-21kthreadcore20:05:393
339950,0migration/331133-21munin-node20:30:343
339950,0migration/331133-21munin-node20:30:343
339950,0migration/329267-21latency20:25:383
339950,0migration/324253-21irqrtprio20:00:373
339950,0migration/323531-21taskset20:22:263
339950,0migration/322457-21aten_r1power_fr23:50:303
339950,0migration/321447-21diskmemload00:09:063
339950,0migration/32046-21kthreadcore23:55:373
339950,0migration/319479-21kthreadcore19:55:363
339950,0migration/318438-21aten_r1power_vo20:45:313
339950,0migration/316990-21kthreadcore19:30:373
339950,0migration/316429-21kthreadcore19:05:373
339950,0migration/316040-21fschecks_count22:05:333
339950,0migration/315642-21fschecks_count22:45:343
339950,0migration/313074-21munin-node22:20:383
339950,0migration/312460-21kthreadcore19:00:403
29725050,0irq/27-eth0-tx-8825-21ssh21:20:352
29725050,0irq/27-eth0-tx-23968-21munin-node21:30:332
29725050,0irq/27-eth0-tx-1960-21munin-node19:20:390
29725050,0irq/27-eth0-tx-14360-21ssh23:04:382
29725050,0irq/27-eth0-tx-0-21swapper/223:36:472
29725050,0irq/27-eth0-tx-0-21swapper/223:08:502
29725050,0irq/27-eth0-tx-0-21swapper/020:55:340
29725050,0irq/27-eth0-tx-0-21swapper/020:54:590
29725050,0irq/27-eth0-tx-0-21swapper/020:10:390
29725050,0irq/27-eth0-tx-0-21swapper/019:45:390
29725050,0irq/27-eth0-tx-0-21swapper/019:38:160
29715050,0irq/26-eth0-rx-5849-21uptime21:15:433
29715050,0irq/26-eth0-rx-32331-21kthreadcore23:35:363
29715050,0irq/26-eth0-rx-2957-21cut21:35:373
29715050,0irq/26-eth0-rx-2950-21aten_r1power_cu19:45:303
29715050,0irq/26-eth0-rx-29121-21ssh23:10:573
29715050,0irq/26-eth0-rx-28556-21ssh00:11:293
29715050,0irq/26-eth0-rx-25082-21sed21:10:353
29715050,0irq/26-eth0-rx-14964-21ssh21:23:243
29715050,0irq/26-eth0-rx-13021-40yum-updatesd-he23:42:173
29715050,0irq/26-eth0-rx-0-21swapper/323:46:073
29715050,0irq/26-eth0-rx-0-21swapper/323:28:303
29715050,0irq/26-eth0-rx-0-21swapper/323:15:363
29715050,0irq/26-eth0-rx-0-21swapper/323:05:373
29715050,0irq/26-eth0-rx-0-21swapper/322:43:133
29715050,0irq/26-eth0-rx-0-21swapper/322:30:373
29715050,0irq/26-eth0-rx-0-21swapper/321:46:353
29715050,0irq/26-eth0-rx-0-21swapper/321:44:063
29715050,0irq/26-eth0-rx-0-21swapper/320:58:383
29715050,0irq/26-eth0-rx-0-21swapper/320:50:383
29715050,0irq/26-eth0-rx-0-21swapper/300:23:273
29715050,0irq/26-eth0-rx-0-21swapper/300:02:463
259950,0migration/27877-21irqrtprio23:20:352
259950,0migration/26236-21sh00:20:302
259950,0migration/25222-21munin-node21:55:422
259950,0migration/231134-21latency_hist22:35:192
259950,0migration/21779-21kthreadcore21:35:372
259950,0migration/217372-21kthreadcore21:25:362
249950,0watchdog/211497-21smartctl20:15:342
249950,0watchdog/20-21swapper/220:05:342
179950,0migration/17406-21missed_timers19:20:391
179950,0migration/16731-21latency_hist23:00:191
179950,0migration/1469-21kthreadcore23:15:351
179950,0migration/128557-21timerandwakeup23:30:441
179950,0migration/113763-21unixbench_singl21:40:421
169950,0watchdog/129145-21kthreadcore21:30:381
169950,0watchdog/119503-21cut19:05:381
149950,0migration/0898-21kthreadcore20:05:390
149950,0migration/07301-21latency_hist22:00:190
149950,0migration/06586-21gltestperf00:20:320
149950,0migration/05710-21gltestperf20:35:330
149950,0migration/03325-21kthreadcore21:35:370
149950,0migration/02913-21kthreadcore22:55:380
149950,0migration/027914-21kthreadcore22:10:370
149950,0migration/026829-21kthreadcore23:10:370
149950,0migration/026195-21kthreadcore21:50:360
149950,0migration/025372-21kthreadcore20:25:360
149950,0migration/025102-21irqcore21:30:360
149950,0migration/020533-21taskset19:07:230
149950,0migration/014939-21sh21:23:190
149950,0migration/014904-21cpuspeed_turbos23:25:300
149950,0migration/014204-21sh22:02:210
149950,0migration/012474-21idleruntime20:40:320
149950,0migration/011023-21kthreadcore23:40:350
7159943,0cyclictest9581-21awk19:25:352
7159943,0cyclictest7339-21awk23:40:332
7159943,0cyclictest3556-21lldpd19:03:592
7159943,0cyclictest3556-21lldpd00:00:282
7159943,0cyclictest3514-21snmpd23:10:472
7159943,0cyclictest3514-21snmpd22:10:322
7159943,0cyclictest3514-21snmpd21:10:322
7159943,0cyclictest3514-21snmpd20:36:462
7159943,0cyclictest3514-21snmpd20:35:012
7159943,0cyclictest3514-21snmpd20:35:012
7159943,0cyclictest3514-21snmpd20:29:012
7159943,0cyclictest3514-21snmpd19:55:312
7159943,0cyclictest3514-21snmpd19:46:312
7159943,0cyclictest3514-21snmpd19:31:312
7159943,0cyclictest3241-21irqbalance23:15:372
7159943,0cyclictest32049-21awk22:55:352
7159943,0cyclictest23549-21awk23:30:352
7159943,0cyclictest22926-21awk19:35:352
7159943,0cyclictest1930-21awk18:55:352
7159943,0cyclictest15350-21awk23:45:362
7159940,0cyclictest23719-21munin-node21:50:332
7159940,0cyclictest0-21swapper/220:55:302
7119943,0cyclictest8156-21awk22:20:311
7119943,0cyclictest459-21awk21:35:351
7119943,0cyclictest3556-21lldpd22:10:591
7119943,0cyclictest3556-21lldpd21:59:291
7119943,0cyclictest3556-21lldpd21:47:291
7119943,0cyclictest3556-21lldpd21:12:591
7119943,0cyclictest3556-21lldpd20:48:581
7119943,0cyclictest3556-21lldpd20:03:291
7119943,0cyclictest3556-21lldpd18:56:291
7119943,0cyclictest28621-21awk19:15:341
7119943,0cyclictest16130-21awk22:45:361
7119940,0cyclictest0-21swapper/123:42:181
7109944,0cyclictest0-21swapper/023:21:330
7109944,0cyclictest0-21swapper/022:24:280
7109943,0cyclictest3556-21lldpd19:40:290
7109943,0cyclictest3514-21snmpd23:15:320
7109943,0cyclictest3241-21irqbalance23:56:370
7109943,0cyclictest3241-21irqbalance23:46:170
7109943,0cyclictest3241-21irqbalance23:32:570
7109943,0cyclictest3241-21irqbalance22:45:070
7109943,0cyclictest3241-21irqbalance22:17:480
7109943,0cyclictest3241-21irqbalance19:10:380
7109943,0cyclictest18696-21kworker/0:022:30:410
7109943,0cyclictest0-21swapper/019:15:310
4140,0ktimersoftd/023553-21munin-run21:10:190
339940,0migration/326088-21kthreadcore22:10:363
339940,0migration/321807-21diskmemload21:05:523
29725040,0irq/27-eth0-tx-7579-21ssh21:40:202
29725040,0irq/27-eth0-tx-15948-21ssh22:05:332
29725040,0irq/27-eth0-tx-15127-21ssh22:45:312
29725040,0irq/27-eth0-tx-0-21swapper/223:56:132
29725040,0irq/27-eth0-tx-0-21swapper/223:51:262
29725040,0irq/27-eth0-tx-0-21swapper/222:50:222
29725040,0irq/27-eth0-tx-0-21swapper/222:40:262
29725040,0irq/27-eth0-tx-0-21swapper/222:26:032
29725040,0irq/27-eth0-tx-0-21swapper/222:20:302
29725040,0irq/27-eth0-tx-0-21swapper/222:15:272
29725040,0irq/27-eth0-tx-0-21swapper/222:00:312
29725040,0irq/27-eth0-tx-0-21swapper/221:45:292
29725040,0irq/27-eth0-tx-0-21swapper/221:01:322
29725040,0irq/27-eth0-tx-0-21swapper/200:15:412
29725040,0irq/27-eth0-tx-0-21swapper/200:05:252
29725040,0irq/27-eth0-tx-0-21swapper/020:30:340
29725040,0irq/27-eth0-tx-0-21swapper/020:30:340
29725040,0irq/27-eth0-tx-0-21swapper/020:00:310
29725040,0irq/27-eth0-tx-0-21swapper/019:55:410
29725040,0irq/27-eth0-tx-0-21swapper/019:50:330
29725040,0irq/27-eth0-tx-0-21swapper/019:25:320
29725040,0irq/27-eth0-tx-0-21swapper/018:55:430
29715040,0irq/26-eth0-rx-0-21swapper/323:20:523
29715040,0irq/26-eth0-rx-0-21swapper/322:03:273
29715040,0irq/26-eth0-rx-0-21swapper/321:28:463
29715040,0irq/26-eth0-rx-0-21swapper/321:00:413
29715040,0irq/26-eth0-rx-0-21swapper/320:10:313
29715040,0irq/26-eth0-rx-0-21swapper/300:17:173
259940,0migration/23821-21kthreadcore22:35:372
259940,0migration/231691-21kthreadcore19:40:372
259940,0migration/227167-21runrttasks19:12:092
259940,0migration/224640-21runrttasks20:49:202
259940,0migration/223996-21if_eth1.220:00:362
259940,0migration/222873-21gltestperf00:10:322
259940,0migration/219719-21missed_timers19:05:392
259940,0migration/218519-21kthreadcore23:25:362
259940,0migration/217815-21fschecks_count20:20:342
259940,0migration/212589-21kthreadcore19:50:382
249940,0watchdog/20-21swapper/221:15:342
249940,0watchdog/20-21swapper/220:50:382
249940,0watchdog/20-21swapper/219:20:382
179940,0migration/1735-21kthreadcore23:55:361
179940,0migration/16273-21taskset20:58:061
179940,0migration/124363-21fschecks_time21:50:321
179940,0migration/124070-21aten_r1power_en20:25:311
179940,0migration/12278-21munin-node19:40:441
179940,0migration/122744-21taskset21:08:211
179940,0migration/121980-21latency_hist19:35:201
179940,0migration/119731-21kthreadcore21:25:371
179940,0migration/116314-21latency_hist19:55:201
179940,0migration/115973-21kthreadcore20:40:361
179940,0migration/115134-21if_eth1.300:05:341
179940,0migration/114517-21latency_hist23:25:191
169940,0watchdog/18602-21fschecks_time21:20:341
169940,0watchdog/16559-21runrttasks23:11:291
169940,0watchdog/16057-21sh21:16:141
169940,0watchdog/131540-21cut00:15:371
169940,0watchdog/131163-21fschecks_count23:35:331
169940,0watchdog/129638-21ssh22:31:131
169940,0watchdog/127734-21kthreadcore20:50:381
169940,0watchdog/1271632sleep119:12:061
169940,0watchdog/115786-21fschecks_count19:30:341
169940,0watchdog/11391-21cut20:30:381
169940,0watchdog/11391-21cut20:30:381
169940,0watchdog/112253-21cut19:50:381
169940,0watchdog/112096-21kthreadcore23:00:371
169940,0watchdog/10-21swapper/122:50:371
169940,0watchdog/10-21swapper/122:05:331
169940,0watchdog/10-21swapper/120:20:341
149940,0migration/028175-21kthreadcore22:50:380
149940,0migration/013536-21kthreadcore20:15:370
149940,0migration/011406-21kthreadcore21:00:410
7159932,0cyclictest3514-21snmpd20:10:312
7119932,0cyclictest9596-21awk19:25:351
7119932,0cyclictest7344-21grep23:20:321
7119932,0cyclictest5670-21kthreadcore20:10:361
7119932,0cyclictest3556-21lldpd20:35:291
7119932,0cyclictest3556-21lldpd19:00:591
7119932,0cyclictest15337-21kthreadcore20:15:381
7119931,0cyclictest0-21swapper/100:20:391
7109933,0cyclictest18696-21kworker/0:022:35:360
7109933,0cyclictest18696-21kworker/0:022:27:130
7109933,0cyclictest18696-21kworker/0:021:25:210
7109933,0cyclictest18696-21kworker/0:021:10:330
7109933,0cyclictest18696-21kworker/0:020:45:230
7109933,0cyclictest18696-21kworker/0:020:20:230
7109933,0cyclictest18696-21kworker/0:019:00:290
7109933,0cyclictest18696-21kworker/0:000:10:400
7109932,0cyclictest3241-21irqbalance21:45:280
29715030,0irq/26-eth0-rx-0-21swapper/320:35:313
29715030,0irq/26-eth0-rx-0-21swapper/319:52:513
29715030,0irq/26-eth0-rx-0-21swapper/319:39:163
29715030,0irq/26-eth0-rx-0-21swapper/319:25:323
29715030,0irq/26-eth0-rx-0-21swapper/319:15:313
29715030,0irq/26-eth0-rx-0-21swapper/319:12:213
259930,0migration/216152-21kthreadcore20:40:362
169930,0watchdog/129966-21cut20:05:341
7159920,0cyclictest0-21swapper/219:15:202
7119920,0cyclictest0-21swapper/119:45:311
29715020,0irq/26-eth0-rx-0-21swapper/320:41:023
29715020,0irq/26-eth0-rx-0-21swapper/320:15:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional