You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 12:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Thu May 28, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271520,0ktimersoftd/29966-21perf18:54:122
1018323728,0sleep10-21swapper/118:49:321
304450360,0irq/25-eth10-21swapper/318:51:593
1620423322,0sleep00-21swapper/018:50:390
339960,0migration/35957-21sh21:26:273
339960,0migration/330085-21sh22:01:503
339960,0migration/322413-21sh21:37:223
339960,0migration/321203-21sh23:37:183
339960,0migration/313526-21sh21:10:113
29725060,0irq/27-eth0-tx-23459-21munin-node18:59:370
29725060,0irq/27-eth0-tx-17719-21munin-node19:19:390
29715060,0irq/26-eth0-rx-6676-21perl23:49:353
29715060,0irq/26-eth0-rx-26695-21kthreadcore21:19:413
29715060,0irq/26-eth0-rx-17991-21irqrtprio18:54:383
249960,0watchdog/20-21swapper/222:04:442
165999960,0cyclictest0-21swapper/222:58:152
165999960,0cyclictest0-21swapper/221:14:412
165989966,0cyclictest14615-21diskmemload23:41:521
165989966,0cyclictest0-21swapper/121:54:391
165989960,0cyclictest0-21swapper/123:58:061
165989960,0cyclictest0-21swapper/123:58:061
149960,0migration/030819-21sh21:43:260
149960,0migration/014528-21sh21:12:450
149960,0migration/014089-21sh22:12:450
8215050,0irq/24-0000:00:30155-21ls23:24:222
8215050,0irq/24-0000:00:13916-21latency_hist23:34:222
8215050,0irq/24-0000:00:12876-21cat20:04:222
4150,0ktimersoftd/00-21swapper/023:54:050
339950,0migration/35969-21diskmemload21:46:483
339950,0migration/34870-21taskset22:24:493
339950,0migration/33813-21timerandwakeup23:44:463
339950,0migration/332455-21irqrtprio22:04:373
339950,0migration/331656-21irqcore23:04:363
339950,0migration/330858-21aten_r1power_fr22:44:323
339950,0migration/32619-21munin-node20:44:393
339950,0migration/325775-21fschecks_count19:49:353
339950,0migration/323818-21kthreadcore20:34:393
339950,0migration/322471-21munin-node23:19:333
339950,0migration/320916-21kthreadcore19:44:393
339950,0migration/318056-21df_abs19:19:353
339950,0migration/313987-21kthreadcore19:39:383
339950,0migration/312681-21sh22:49:523
29725050,0irq/27-eth0-tx-7003-21ssh23:29:362
29725050,0irq/27-eth0-tx-6942-21cut00:09:372
29725050,0irq/27-eth0-tx-21977-21ssh22:37:532
29725050,0irq/27-eth0-tx-20467-21head22:14:402
29725050,0irq/27-eth0-tx-18844-21kthreadcore23:54:392
29725050,0irq/27-eth0-tx-18844-21kthreadcore23:54:392
29725050,0irq/27-eth0-tx-15218-21munin-node21:54:442
29725050,0irq/27-eth0-tx-13239-21ssh22:51:282
29725050,0irq/27-eth0-tx-0-21swapper/223:39:332
29725050,0irq/27-eth0-tx-0-21swapper/223:11:282
29725050,0irq/27-eth0-tx-0-21swapper/222:11:242
29725050,0irq/27-eth0-tx-0-21swapper/200:14:342
29725050,0irq/27-eth0-tx-0-21swapper/020:14:460
29725050,0irq/27-eth0-tx-0-21swapper/019:39:330
29715050,0irq/26-eth0-rx-8062-21gltestperf21:29:353
29715050,0irq/26-eth0-rx-26671-21kthreadcore21:39:393
29715050,0irq/26-eth0-rx-23230-21cpu20:59:343
29715050,0irq/26-eth0-rx-20628-21proc_pri22:14:413
29715050,0irq/26-eth0-rx-15945-21grep23:54:383
29715050,0irq/26-eth0-rx-15945-21grep23:54:383
29715050,0irq/26-eth0-rx-0-21swapper/323:44:043
29715050,0irq/26-eth0-rx-0-21swapper/323:29:443
29715050,0irq/26-eth0-rx-0-21swapper/323:24:353
29715050,0irq/26-eth0-rx-0-21swapper/323:02:003
29715050,0irq/26-eth0-rx-0-21swapper/322:56:283
29715050,0irq/26-eth0-rx-0-21swapper/322:34:103
29715050,0irq/26-eth0-rx-0-21swapper/322:14:203
29715050,0irq/26-eth0-rx-0-21swapper/321:57:183
29715050,0irq/26-eth0-rx-0-21swapper/321:17:473
29715050,0irq/26-eth0-rx-0-21swapper/321:04:283
29715050,0irq/26-eth0-rx-0-21swapper/320:57:073
29715050,0irq/26-eth0-rx-0-21swapper/320:04:383
29715050,0irq/26-eth0-rx-0-21swapper/319:14:353
29715050,0irq/26-eth0-rx-0-21swapper/300:04:383
259950,0migration/26256-21sh22:28:492
259950,0migration/25867-21sh23:09:002
259950,0migration/225966-21kthreadcore21:39:392
249950,0watchdog/231657-21aten_r1power_po21:24:332
249950,0watchdog/230719-21ssh22:44:242
19150,0ktimersoftd/10-21swapper/123:03:521
19150,0ktimersoftd/10-21swapper/121:47:431
179950,0migration/16098-21hwlatdetect19:34:351
179950,0migration/15713-21taskset22:27:151
179950,0migration/122020-21taskset23:59:301
179950,0migration/120123-21taskset00:15:051
179950,0migration/113699-21idleruntime-cro20:29:221
165999955,0cyclictest82150irq/24-0000:00:21:20:042
165999955,0cyclictest0-21swapper/223:46:212
165999955,0cyclictest0-21swapper/222:31:562
165989955,0cyclictest0-21swapper/123:32:241
165989955,0cyclictest0-21swapper/123:20:351
165989955,0cyclictest0-21swapper/122:16:061
165989955,0cyclictest0-21swapper/121:52:551
165989955,0cyclictest0-21swapper/121:40:281
165989950,0cyclictest0-21swapper/122:55:121
165989950,0cyclictest0-21swapper/121:30:041
165989950,0cyclictest0-21swapper/121:10:311
165979950,0cyclictest0-21swapper/023:05:030
165979950,0cyclictest0-21swapper/021:37:140
169950,0watchdog/124909-21kthreadcore21:19:411
149950,0migration/08121-21kthreadcore23:09:370
149950,0migration/05636-21sh22:06:300
149950,0migration/0549-21timerwakeupswit20:39:440
149950,0migration/04993-21missed_timers19:54:410
149950,0migration/04675-21kthreadcore20:19:390
149950,0migration/04464-21taskset23:25:300
149950,0migration/032481-21kthreadcore19:29:380
149950,0migration/032458-21kthreadcore23:44:390
149950,0migration/032045-21kthreadcore00:04:370
149950,0migration/031794-21kthreadcore19:04:380
149950,0migration/027996-21kthreadcore21:59:400
149950,0migration/027520-21latency23:39:400
149950,0migration/024061-21kthreadcore23:59:390
149950,0migration/02368-21kthreadcore21:44:380
149950,0migration/023227-21idleruntime23:19:360
149950,0migration/020992-21taskset00:17:370
149950,0migration/020469-21latency_hist20:34:220
149950,0migration/016053-21idleruntime21:54:350
149950,0migration/015002-21aten_r1power_vo20:54:340
149950,0migration/014477-21latency_hist22:54:220
149950,0migration/012628-21taskset20:02:370
149950,0migration/010329-21munin-node19:09:450
339940,0migration/36975-21kthreadcore19:34:383
339940,0migration/330544-21fschecks_count19:04:353
339940,0migration/324818-21kthreadcore00:19:393
339940,0migration/324344-21taskset19:24:293
339940,0migration/32355-21kthreadcore19:29:393
339940,0migration/323202-21diskmemload22:39:353
339940,0migration/322291-21kthreadcore20:09:373
339940,0migration/312036-21sh19:59:443
29725040,0irq/27-eth0-tx-8660-21kthreadcore21:09:372
29725040,0irq/27-eth0-tx-6702-21gltestperf23:49:352
29725040,0irq/27-eth0-tx-23011-21munin-node20:59:342
29725040,0irq/27-eth0-tx-22725-21ssh22:39:242
29725040,0irq/27-eth0-tx-21242-21ssh20:55:292
29725040,0irq/27-eth0-tx-15304-21ssh21:34:322
29725040,0irq/27-eth0-tx-13448-21ssh21:50:062
29725040,0irq/27-eth0-tx-0-21swapper/223:59:332
29725040,0irq/27-eth0-tx-0-21swapper/223:24:262
29725040,0irq/27-eth0-tx-0-21swapper/223:14:422
29725040,0irq/27-eth0-tx-0-21swapper/222:20:092
29725040,0irq/27-eth0-tx-0-21swapper/221:04:322
29725040,0irq/27-eth0-tx-0-21swapper/200:04:292
29725040,0irq/27-eth0-tx-0-21swapper/020:49:370
29715040,0irq/26-eth0-rx-12978-21smartctl21:49:453
29715040,0irq/26-eth0-rx-0-21swapper/323:14:343
29715040,0irq/26-eth0-rx-0-21swapper/323:10:483
29715040,0irq/26-eth0-rx-0-21swapper/322:35:323
29715040,0irq/26-eth0-rx-0-21swapper/322:19:313
29715040,0irq/26-eth0-rx-0-21swapper/320:39:353
29715040,0irq/26-eth0-rx-0-21swapper/300:17:183
29715040,0irq/26-eth0-rx-0-21swapper/300:10:563
29715040,0irq/26-eth0-rx-0-21swapper/300:01:233
259940,0migration/27802-21fschecks_count21:29:352
259940,0migration/26575-21kthreadcore20:44:412
259940,0migration/227703-21switchtime00:19:452
259940,0migration/224595-21irqcore18:59:392
259940,0migration/222472-21kthreadcore20:34:392
259940,0migration/216552-21kthreadcore20:29:382
259940,0migration/212591-21kthreadcore19:14:402
249940,0watchdog/230086-21munin-node19:04:332
249940,0watchdog/20-21swapper/222:59:362
179940,0migration/19126-21kthreadcore23:09:381
179940,0migration/15873-21runrttasks19:56:511
179940,0migration/14699-21kthreadcore20:19:391
179940,0migration/131057-21fschecks_count23:04:331
179940,0migration/125992-21kthreadcore18:59:401
179940,0migration/125838-21fschecks_count19:49:351
179940,0migration/124792-21runrttasks19:45:421
179940,0migration/124006-21if_eth1.320:59:371
179940,0migration/121167-21rt-features21:14:431
179940,0migration/119208-21kthreadcore22:34:391
179940,0migration/115516-21kthreadcore19:39:381
179940,0migration/115440-21kthreadcore19:14:411
179940,0migration/113124-21taskset22:30:291
179940,0migration/113114-21aten_r1power_fr20:04:331
179940,0migration/112641-21runrttasks20:02:471
165999943,0cyclictest3514-21snmpd20:28:042
165999943,0cyclictest3514-21snmpd19:53:192
165999943,0cyclictest3514-21snmpd19:34:192
165999943,0cyclictest3514-21snmpd19:29:192
165999943,0cyclictest3241-21irqbalance21:44:312
165999943,0cyclictest3241-21irqbalance20:49:412
165999943,0cyclictest24180-21awk21:59:372
165999943,0cyclictest24117-21sadc19:24:222
165999943,0cyclictest20658-21awk20:09:362
165999943,0cyclictest17644-21awk18:54:372
165999943,0cyclictest12903-21awk19:39:362
165989944,0cyclictest0-21swapper/122:41:191
165989944,0cyclictest0-21swapper/120:54:261
165989943,0cyclictest3514-21snmpd22:12:491
165989943,0cyclictest3514-21snmpd22:00:041
165989943,0cyclictest3514-21snmpd00:20:191
165989943,0cyclictest3514-21snmpd00:13:041
165989943,0cyclictest3241-21irqbalance23:53:001
165989943,0cyclictest3241-21irqbalance20:09:411
165989943,0cyclictest31871-21awk19:29:371
165989943,0cyclictest31635-21awk22:44:361
165989943,0cyclictest31036-21awk00:04:361
165989943,0cyclictest27193-21awk20:14:381
165989943,0cyclictest17674-21awk18:54:371
165989943,0cyclictest14748-21awk20:29:361
165979944,0cyclictest0-21swapper/021:49:230
165979944,0cyclictest0-21swapper/021:05:230
165979944,0cyclictest0-21swapper/000:22:280
165979943,0cyclictest7652-21awk22:29:360
165979943,0cyclictest6672-21awk00:09:360
165979943,0cyclictest5307-21munin-node19:34:340
165979943,0cyclictest3556-21lldpd19:47:410
165979943,0cyclictest3514-21snmpd20:06:190
165979943,0cyclictest3241-21irqbalance23:17:300
165979943,0cyclictest3241-21irqbalance22:38:400
165979943,0cyclictest3241-21irqbalance21:29:310
165979943,0cyclictest24485-21awk21:19:380
165979943,0cyclictest2283-21awk20:44:370
165979941,0cyclictest0-21swapper/020:09:320
169940,0watchdog/18255-21aten_r1power_en20:49:331
169940,0watchdog/16016-21ssh21:26:361
169940,0watchdog/130130-21aten_r1power_en19:04:331
169940,0watchdog/122382-21chrt21:37:161
169940,0watchdog/115339-21iwlist23:14:361
169940,0watchdog/113331-21ssh22:51:441
169940,0watchdog/10-21swapper/123:39:201
169940,0watchdog/10-21swapper/122:04:401
149940,0migration/04249-21switchtime22:44:440
149940,0migration/03664-21kthreadcore21:24:390
149940,0migration/030427-21runrttasks21:03:030
149940,0migration/021614-21sh22:16:100
339930,0migration/310489-21runrttasks19:10:143
29725030,0irq/27-eth0-tx-0-21swapper/020:24:330
29715030,0irq/26-eth0-rx-0-21swapper/320:51:313
29715030,0irq/26-eth0-rx-0-21swapper/320:29:413
29715030,0irq/26-eth0-rx-0-21swapper/320:24:353
29715030,0irq/26-eth0-rx-0-21swapper/320:20:253
29715030,0irq/26-eth0-rx-0-21swapper/320:19:183
29715030,0irq/26-eth0-rx-0-21swapper/319:54:343
249930,0watchdog/20-21swapper/220:19:412
249930,0watchdog/20-21swapper/219:34:372
165999932,0cyclictest5334-21sensors19:54:442
165999932,0cyclictest3514-21snmpd20:41:042
165999932,0cyclictest18659-21munin-node19:44:452
165999932,0cyclictest13113-21aten2.4-expect20:04:332
165999930,0cyclictest27008-21perl20:14:362
165989932,0cyclictest9041-21grep19:09:391
165989932,0cyclictest3514-21snmpd20:44:341
165989932,0cyclictest3241-21irqbalance20:34:311
165989932,0cyclictest32126-21awk21:04:361
165989932,0cyclictest25022-21tune2fs19:24:351
165989931,0cyclictest27147-21latency_hist20:39:221
165979933,0cyclictest32041-21kworker/0:023:54:230
165979933,0cyclictest32041-21kworker/0:023:54:230
165979933,0cyclictest32041-21kworker/0:023:34:230
165979933,0cyclictest32041-21kworker/0:023:29:290
165979933,0cyclictest32041-21kworker/0:022:59:310
165979933,0cyclictest32041-21kworker/0:022:49:250
165979933,0cyclictest32041-21kworker/0:022:39:230
165979933,0cyclictest32041-21kworker/0:022:19:310
165979933,0cyclictest32041-21kworker/0:021:14:300
165979933,0cyclictest2227-21kworker/0:120:34:360
165979933,0cyclictest2227-21kworker/0:119:49:340
165979933,0cyclictest2227-21kworker/0:119:24:440
165979933,0cyclictest2227-21kworker/0:119:14:260
165979933,0cyclictest2227-21kworker/0:118:54:320
165979932,0cyclictest31923-21awk22:24:360
169930,0watchdog/10-21swapper/123:44:361
169930,0watchdog/10-21swapper/123:24:361
169930,0watchdog/10-21swapper/122:20:281
169930,0watchdog/10-21swapper/119:19:371
29715020,0irq/26-eth0-rx-0-21swapper/318:59:313
249920,0watchdog/20-21swapper/219:09:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional