You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 05:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun May 17, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450540,0irq/25-eth10-21swapper/318:53:403
3124723622,0sleep00-21swapper/018:52:060
3152823423,0sleep10-21swapper/118:55:191
3132123425,0sleep20-21swapper/218:52:562
339960,0migration/34427-21sh21:36:423
339960,0migration/33509-21sh23:38:593
339960,0migration/329109-21irqcore20:05:413
339960,0migration/32857-21sh00:19:183
339960,0migration/328247-21sh22:52:493
339960,0migration/326997-21munin-node22:30:423
339960,0migration/320860-21sh21:26:523
339960,0migration/312533-21sh23:04:423
339960,0migration/310534-21sh00:23:063
316109960,0cyclictest0-21swapper/022:52:050
29715060,0irq/26-eth0-rx-13203-21if_eth023:45:373
149960,0migration/029051-21latency_hist20:30:250
8215050,0irq/24-0000:00:15535-21crond20:20:252
339950,0migration/38500-21kthreadcore22:00:423
339950,0migration/38224-21kthreadcore21:20:423
339950,0migration/37511-21kthreadcore21:00:443
339950,0migration/36873-21kthreadcore00:00:423
339950,0migration/3645-21taskset19:41:013
339950,0migration/332320-21taskset19:18:163
339950,0migration/331246-21irqrtprio20:55:423
339950,0migration/330802-21kthreadcore23:55:413
339950,0migration/329869-21latency_hist21:55:253
339950,0migration/329314-21sh21:13:063
339950,0migration/320816-21munin-node19:35:383
339950,0migration/31540-21kthreadcore23:15:433
339950,0migration/313739-21munin-node19:50:463
316129955,0cyclictest0-21swapper/220:56:422
316129954,0cyclictest23212-21awk21:30:392
316129950,0cyclictest0-21swapper/223:15:362
316129950,0cyclictest0-21swapper/223:01:152
316119955,0cyclictest0-21swapper/123:59:201
316119955,0cyclictest0-21swapper/123:43:061
316119955,0cyclictest0-21swapper/123:25:451
316119955,0cyclictest0-21swapper/121:02:261
316119955,0cyclictest0-21swapper/119:25:291
316119954,0cyclictest20634-21cut19:10:431
316119954,0cyclictest0-21swapper/121:52:441
316119950,0cyclictest0-21swapper/121:46:441
316119950,0cyclictest0-21swapper/121:13:471
316119950,0cyclictest0-21swapper/121:06:131
316119950,0cyclictest0-21swapper/100:19:211
316109955,0cyclictest0-21swapper/023:55:470
316109955,0cyclictest0-21swapper/023:33:520
316109955,0cyclictest0-21swapper/023:00:450
316109955,0cyclictest0-21swapper/022:38:320
316109950,0cyclictest0-21swapper/021:59:160
316109950,0cyclictest0-21swapper/021:39:240
316109950,0cyclictest0-21swapper/021:32:570
29725050,0irq/27-eth0-tx-7118-21munin-node19:25:400
29725050,0irq/27-eth0-tx-6677-21kthreadcore22:40:422
29725050,0irq/27-eth0-tx-3093-21ssh23:58:502
29725050,0irq/27-eth0-tx-30051-21munin-node20:55:410
29725050,0irq/27-eth0-tx-29886-21munin-node22:15:412
29725050,0irq/27-eth0-tx-26559-21ssh23:51:082
29725050,0irq/27-eth0-tx-14911-21kthreadcore22:25:432
29725050,0irq/27-eth0-tx-11913-21ssh22:21:252
29725050,0irq/27-eth0-tx-10780-21grep21:20:422
29725050,0irq/27-eth0-tx-0-21swapper/223:50:242
29725050,0irq/27-eth0-tx-0-21swapper/223:40:352
29725050,0irq/27-eth0-tx-0-21swapper/223:38:082
29725050,0irq/27-eth0-tx-0-21swapper/221:14:262
29725050,0irq/27-eth0-tx-0-21swapper/221:02:502
29725050,0irq/27-eth0-tx-0-21swapper/200:10:402
29725050,0irq/27-eth0-tx-0-21swapper/020:43:510
29715050,0irq/26-eth0-rx-6172-21ssh22:40:403
29715050,0irq/26-eth0-rx-3764-21ssh22:16:073
29715050,0irq/26-eth0-rx-3122-21memory21:55:433
29715050,0irq/26-eth0-rx-27171-21ssh23:31:493
29715050,0irq/26-eth0-rx-1979-21kthreadcore22:35:433
29715050,0irq/26-eth0-rx-19691-21proc_pri21:45:453
29715050,0irq/26-eth0-rx-0-21swapper/323:50:443
29715050,0irq/26-eth0-rx-0-21swapper/323:40:363
29715050,0irq/26-eth0-rx-0-21swapper/323:29:323
29715050,0irq/26-eth0-rx-0-21swapper/323:20:423
29715050,0irq/26-eth0-rx-0-21swapper/323:09:343
29715050,0irq/26-eth0-rx-0-21swapper/322:58:223
29715050,0irq/26-eth0-rx-0-21swapper/322:22:173
29715050,0irq/26-eth0-rx-0-21swapper/321:40:373
29715050,0irq/26-eth0-rx-0-21swapper/321:30:433
29715050,0irq/26-eth0-rx-0-21swapper/321:15:383
29715050,0irq/26-eth0-rx-0-21swapper/321:06:023
29715050,0irq/26-eth0-rx-0-21swapper/319:25:493
29715050,0irq/26-eth0-rx-0-21swapper/319:05:363
259950,0migration/28227-21aten_r1power_vo19:50:382
259950,0migration/26776-21kthreadcore23:20:422
259950,0migration/25045-21kthreadcore00:20:412
259950,0migration/23731-21munin-node21:35:442
259950,0migration/231884-21kthreadcore21:55:412
259950,0migration/227661-21fschecks_time19:40:382
259950,0migration/226059-21latency_hist19:15:252
259950,0migration/224417-21kthreadcore20:25:422
259950,0migration/222079-21munin-node23:10:402
259950,0migration/221440-21runrttasks21:28:332
249950,0watchdog/2538-21kthreadcore22:35:432
19150,0ktimersoftd/10-21swapper/122:55:381
179950,0migration/19342-21cpuspeed_turbos20:15:381
179950,0migration/129397-21munin-node20:05:421
179950,0migration/126079-21munin-node23:30:431
179950,0migration/120350-21proc_pri19:55:441
179950,0migration/115377-21munin-node20:40:451
179950,0migration/113667-21cpuspeed_turbos22:45:371
149950,0migration/06494-21kthreadcore00:20:410
149950,0migration/05486-21aten_r1power_vo22:40:370
149950,0migration/04793-21sh22:19:070
149950,0migration/032213-21taskset19:17:010
149950,0migration/029043-21sh23:15:310
149950,0migration/026258-21taskset00:11:180
149950,0migration/025172-21switchtime19:10:490
149950,0migration/024638-21kthreadcore20:00:410
149950,0migration/023949-21kthreadcore22:10:440
149950,0migration/023224-21idleruntime20:25:400
149950,0migration/020292-21taskset23:50:270
149950,0migration/019554-21kthreadcore20:20:410
149950,0migration/014286-21irqrtprio23:05:400
149950,0migration/014157-21cpuspeed_turbos22:05:370
149950,0migration/013130-21sh21:03:360
8215040,0irq/24-0000:00:3853-21cat00:00:242
8215040,0irq/24-0000:00:21256-21cat20:00:252
339940,0migration/36490-21kthreadcore19:45:423
339940,0migration/330092-21iostat20:30:413
339940,0migration/328481-21diskmemload22:12:113
339940,0migration/325679-21kthreadcore20:00:423
339940,0migration/321802-21kthreadcore00:10:433
316129943,0cyclictest3556-21lldpd19:37:212
316129943,0cyclictest3556-21lldpd19:22:502
316129943,0cyclictest32674-21awk18:55:412
316129943,0cyclictest30050-21awk20:30:402
316129943,0cyclictest28948-21awk20:05:402
316129943,0cyclictest17584-21awk20:45:402
316129943,0cyclictest16634-21awk20:20:392
316119943,0cyclictest3556-21lldpd20:46:211
316119943,0cyclictest3556-21lldpd19:47:501
316119943,0cyclictest3556-21lldpd19:15:501
316119943,0cyclictest3241-21irqbalance20:35:441
316119943,0cyclictest31040-21awk20:55:411
316119943,0cyclictest30000-21awk23:15:401
316119943,0cyclictest24282-21awk20:50:411
316119943,0cyclictest21188-21awk23:50:381
316119943,0cyclictest13771-21awk19:05:411
316109943,0cyclictest6737-21awk22:00:400
316109943,0cyclictest3556-21lldpd23:23:210
316109943,0cyclictest3556-21lldpd22:31:500
316109943,0cyclictest3556-21lldpd22:28:510
316109943,0cyclictest29585-21awk23:35:400
316109943,0cyclictest17554-21awk20:45:400
316109943,0cyclictest15190-21awk21:25:400
316109943,0cyclictest13689-21awk23:25:400
316109943,0cyclictest1207-21awk19:20:400
316109943,0cyclictest11998-21cut00:05:350
316109943,0cyclictest1097-21sadc19:45:250
316109943,0cyclictest10076-21awk20:15:410
29725040,0irq/27-eth0-tx-8965-21kthreadcore21:40:432
29725040,0irq/27-eth0-tx-27151-21ssh23:31:462
29725040,0irq/27-eth0-tx-22093-21munin-node21:50:452
29725040,0irq/27-eth0-tx-14793-21expr21:45:382
29725040,0irq/27-eth0-tx-13029-21ssh23:05:292
29725040,0irq/27-eth0-tx-12720-21munin-node23:25:432
29725040,0irq/27-eth0-tx-11994-21ssh22:00:542
29725040,0irq/27-eth0-tx-0-21swapper/222:55:422
29725040,0irq/27-eth0-tx-0-21swapper/222:50:352
29725040,0irq/27-eth0-tx-0-21swapper/222:05:422
29725040,0irq/27-eth0-tx-0-21swapper/221:15:362
29725040,0irq/27-eth0-tx-0-21swapper/221:05:302
29725040,0irq/27-eth0-tx-0-21swapper/200:15:372
29725040,0irq/27-eth0-tx-0-21swapper/020:39:500
29725040,0irq/27-eth0-tx-0-21swapper/020:13:500
29725040,0irq/27-eth0-tx-0-21swapper/020:00:210
29725040,0irq/27-eth0-tx-0-21swapper/019:45:400
29725040,0irq/27-eth0-tx-0-21swapper/019:35:160
29715040,0irq/26-eth0-rx-21282-21munin-node23:10:363
29715040,0irq/26-eth0-rx-0-21swapper/322:47:253
29715040,0irq/26-eth0-rx-0-21swapper/322:25:373
29715040,0irq/26-eth0-rx-0-21swapper/322:05:453
29715040,0irq/26-eth0-rx-0-21swapper/320:45:373
29715040,0irq/26-eth0-rx-0-21swapper/319:57:223
29715040,0irq/26-eth0-rx-0-21swapper/300:06:153
259940,0migration/29045-21kthreadcore19:00:432
259940,0migration/224256-21kthreadcore22:30:412
259940,0migration/22278-21munin-node19:45:402
259940,0migration/216583-21kthreadcore19:30:422
259940,0migration/212132-21munin-node00:05:362
249940,0watchdog/227462-21kthreadcore20:50:432
249940,0watchdog/227438-21ntp_offset22:10:472
249940,0watchdog/22354-21aten_r1power_cu20:10:362
249940,0watchdog/213158-21kthreadcore20:40:432
249940,0watchdog/20-21swapper/222:45:432
249940,0watchdog/20-21swapper/219:10:402
179940,0migration/1931-21fschecks_time19:20:381
179940,0migration/18692-21kthreadcore00:00:431
179940,0migration/18433-21kthreadcore22:00:421
179940,0migration/17706-21kthreadcore19:00:421
179940,0migration/16664-21kernelversion22:40:421
179940,0migration/15470-21kthreadcore00:20:411
179940,0migration/13870-21timerandwakeup21:35:451
179940,0migration/130442-21taskset21:15:271
179940,0migration/128638-21taskset20:27:541
179940,0migration/124394-21kthreadcore21:30:421
179940,0migration/120973-21latency_hist23:10:251
179940,0migration/120357-21kthreadcore20:20:421
179940,0migration/117131-21kthreadcore21:25:421
179940,0migration/117039-21kthreadcore22:25:431
179940,0migration/113882-21aten_r1power_po19:30:361
179940,0migration/110218-21kthreadcore22:20:421
179940,0migration/110072-21kthreadcore19:50:421
169940,0watchdog/18730-21chrt20:14:041
169940,0watchdog/129887-21aten_r1power_cu22:15:351
169940,0watchdog/128764-21ssh22:33:471
169940,0watchdog/126265-21ksmtuned00:11:191
169940,0watchdog/125718-21memory19:35:441
169940,0watchdog/122399-21if_eth1.320:00:401
169940,0watchdog/121520-21aten_r1power_po22:50:351
169940,0watchdog/119295-21ls22:05:431
169940,0watchdog/118852-21ssh00:08:031
169940,0watchdog/111100-21kthreadcore21:40:431
169940,0watchdog/10-21swapper/122:35:431
169940,0watchdog/10-21swapper/122:10:471
149940,0migration/04326-21taskset22:59:130
149940,0migration/03927-21taskset00:00:290
149940,0migration/028885-21runrttasks21:11:500
149940,0migration/027301-21taskset23:11:030
149940,0migration/02064-21taskset20:10:040
4130,0ktimersoftd/012753-21sh21:21:430
316129932,0cyclictest7954-21awk19:25:402
316129932,0cyclictest3924-21fschecks_time20:35:372
316129932,0cyclictest3556-21lldpd19:09:502
316119933,0cyclictest5911-21awk23:00:411
316119932,0cyclictest27910-21awk19:40:401
316119932,0cyclictest2199-21timerwakeupswit23:35:481
316119932,0cyclictest10485-21sensors23:20:461
316119931,0cyclictest0-21swapper/123:05:421
316109932,0cyclictest6941-21awk21:40:400
316109932,0cyclictest4370-21aten_r1power_fr23:40:370
316109932,0cyclictest3556-21lldpd22:20:510
316109932,0cyclictest3556-21lldpd19:06:210
316109932,0cyclictest3514-21snmpd21:16:400
316109932,0cyclictest334-21awk18:55:420
316109932,0cyclictest3241-21irqbalance20:50:440
316109932,0cyclictest21040-21sh21:48:140
316109932,0cyclictest18949-21ntpq22:45:450
316109932,0cyclictest1288-21sensors00:15:460
316109930,0cyclictest6129-21munin-node19:00:410
316109930,0cyclictest0-21swapper/019:35:430
29725030,0irq/27-eth0-tx-0-21swapper/019:50:500
29715030,0irq/26-eth0-rx-0-21swapper/320:52:153
29715030,0irq/26-eth0-rx-0-21swapper/320:40:383
29715030,0irq/26-eth0-rx-0-21swapper/320:20:363
29715030,0irq/26-eth0-rx-0-21swapper/320:15:373
29715030,0irq/26-eth0-rx-0-21swapper/320:10:493
29715030,0irq/26-eth0-rx-0-21swapper/319:10:383
29715030,0irq/26-eth0-rx-0-21swapper/319:00:383
29715030,0irq/26-eth0-rx-0-21swapper/318:57:453
249930,0watchdog/20-21swapper/220:00:402
179930,0migration/130271-21aten_r1power_vo21:55:361
179930,0migration/112568-21runrttasks21:21:111
169930,0watchdog/10-21swapper/123:45:351
169930,0watchdog/10-21swapper/118:55:401
316109921,0cyclictest0-21swapper/021:50:270
316109921,0cyclictest0-21swapper/021:05:270
316109920,0cyclictest12218-21wc23:45:250
29715020,0irq/26-eth0-rx-0-21swapper/320:36:403
29715020,0irq/26-eth0-rx-0-21swapper/320:26:073
29715020,0irq/26-eth0-rx-0-21swapper/319:30:293
29715020,0irq/26-eth0-rx-0-21swapper/319:20:473
169920,0watchdog/10-21swapper/120:30:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional