You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 01:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Wed Apr 15, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450470,0irq/25-eth10-21swapper/318:57:133
2935424536,0sleep10-21swapper/118:53:501
2991223824,0sleep00-21swapper/018:56:170
2376123525,0sleep20-21swapper/218:53:392
301689983,0cyclictest32191-21awk19:23:422
301689973,0cyclictest761-21awk19:48:412
301689973,0cyclictest5754-21awk21:23:412
301689973,0cyclictest3514-21snmpd21:14:312
301689973,0cyclictest21150-21awk22:33:422
301689973,0cyclictest20017-21awk23:53:432
301689973,0cyclictest14201-21awk19:58:412
301689972,0cyclictest29555-21awk21:58:382
301689972,0cyclictest27967-21munin-node22:58:392
301689972,0cyclictest12556-21awk23:08:412
301689972,0cyclictest12357-21sadc21:08:272
301689970,0cyclictest2720-21munin-node00:03:472
301689970,0cyclictest18774-21munin-node00:13:432
301689970,0cyclictest0-21swapper/200:18:452
339960,0migration/38108-21fschecks_count20:18:383
339960,0migration/325780-21sh23:55:213
339960,0migration/310423-21ntp_offset21:23:463
301689962,0cyclictest6559-21runrttasks21:34:132
301689962,0cyclictest3514-21snmpd20:07:302
301689962,0cyclictest3282-21kthreadcore20:13:442
301689962,0cyclictest27363-21sadc23:18:262
301689962,0cyclictest10383-21sensors19:03:502
301689962,0cyclictest0-21swapper/222:23:322
301689961,0cyclictest0-21swapper/221:11:162
301689960,0cyclictest3738-21sh21:21:092
301689960,0cyclictest17724-21python22:48:492
301689960,0cyclictest12143-21kthreadcore00:08:432
301689960,0cyclictest10824-21ssh21:44:052
301689960,0cyclictest0-21swapper/223:49:502
301689960,0cyclictest0-21swapper/223:33:102
301689960,0cyclictest0-21swapper/223:24:402
301689960,0cyclictest0-21swapper/223:20:242
301689960,0cyclictest0-21swapper/222:09:572
301689960,0cyclictest0-21swapper/221:38:512
301689960,0cyclictest0-21swapper/200:23:502
301669960,0cyclictest0-21swapper/021:33:470
29715060,0irq/26-eth0-rx-8400-21grep20:43:273
29715060,0irq/26-eth0-rx-29409-21hddtemp_smartct20:58:413
29715060,0irq/26-eth0-rx-22908-21kthreadcore19:38:433
29715060,0irq/26-eth0-rx-16704-21grep21:48:433
149960,0migration/09718-21sh00:06:060
339950,0migration/3739-21kthreadcore22:38:443
339950,0migration/33393-21kthreadcore19:23:453
339950,0migration/32990-21kthreadcore19:48:443
339950,0migration/329255-21kthreadcore19:18:453
339950,0migration/328114-21taskset20:55:333
339950,0migration/326565-21proc_pri21:33:483
339950,0migration/32488-21kthreadcore18:58:433
339950,0migration/320977-21kthreadcore23:13:423
339950,0migration/320302-21munin-node22:53:383
339950,0migration/319480-21kthreadcore20:23:453
339950,0migration/319222-21sh21:10:023
339950,0migration/318364-21taskset19:34:043
339950,0migration/317809-21kthreadcore20:48:443
339950,0migration/315370-21kthreadcore23:48:443
339950,0migration/314271-21kthreadcore19:08:443
339950,0migration/314131-21irqrtprio21:28:423
339950,0migration/31392-21taskset23:18:573
339950,0migration/313225-21munin-node22:08:413
339950,0migration/312284-21if_eth123:28:413
301689952,0cyclictest7408-21kthreadcore19:28:442
301689952,0cyclictest3556-21lldpd20:28:542
301689951,0cyclictest23091-21kthreadcore22:53:432
301689950,0cyclictest9612-21kthreadcore20:18:422
301689950,0cyclictest6272-21kthreadcore22:43:432
301689950,0cyclictest4954-21kthreadcore23:03:442
301689950,0cyclictest32218-21kthreadcore23:38:412
301689950,0cyclictest30184-21cut23:58:442
301689950,0cyclictest20476-21munin-node22:13:372
301689950,0cyclictest19219-21latency_hist23:33:262
301689950,0cyclictest0-21swapper/223:43:572
301689950,0cyclictest0-21swapper/222:40:332
301689950,0cyclictest0-21swapper/222:28:262
301689950,0cyclictest0-21swapper/222:18:382
301689950,0cyclictest0-21swapper/222:03:492
301689950,0cyclictest0-21swapper/221:53:542
301689950,0cyclictest0-21swapper/221:48:462
301689950,0cyclictest0-21swapper/221:29:072
301689950,0cyclictest0-21swapper/220:59:192
301689950,0cyclictest0-21swapper/220:41:312
301689950,0cyclictest0-21swapper/220:33:452
301689950,0cyclictest0-21swapper/220:08:462
301689950,0cyclictest0-21swapper/219:53:452
301669955,0cyclictest0-21swapper/023:40:230
301669955,0cyclictest0-21swapper/023:24:180
301669955,0cyclictest0-21swapper/022:58:370
301669955,0cyclictest0-21swapper/021:16:170
301669955,0cyclictest0-21swapper/000:01:210
301669950,0cyclictest0-21swapper/023:29:120
301669950,0cyclictest0-21swapper/023:22:250
301669950,0cyclictest0-21swapper/023:09:100
301669950,0cyclictest0-21swapper/021:39:280
301669950,0cyclictest0-21swapper/021:22:260
301669950,0cyclictest0-21swapper/019:05:450
301669950,0cyclictest0-21swapper/000:23:540
29725050,0irq/27-eth0-tx-14326-21munin-node20:23:460
29725050,0irq/27-eth0-tx-0-21swapper/020:53:480
29725050,0irq/27-eth0-tx-0-21swapper/018:58:390
29715050,0irq/26-eth0-rx-9373-21tr21:43:443
29715050,0irq/26-eth0-rx-7535-21kthreadcore21:03:463
29715050,0irq/26-eth0-rx-31167-21kthreadcore21:38:443
29715050,0irq/26-eth0-rx-28746-21kthreadcore19:43:443
29715050,0irq/26-eth0-rx-26223-21timerwakeupswit22:33:513
29715050,0irq/26-eth0-rx-21804-21kthreadcore22:13:433
29715050,0irq/26-eth0-rx-16222-21grep19:58:433
29715050,0irq/26-eth0-rx-16023-21kthreadcore22:28:443
29715050,0irq/26-eth0-rx-10051-21sed22:23:493
29715050,0irq/26-eth0-rx-0-21swapper/323:44:323
29715050,0irq/26-eth0-rx-0-21swapper/323:42:403
29715050,0irq/26-eth0-rx-0-21swapper/323:33:383
29715050,0irq/26-eth0-rx-0-21swapper/323:01:323
29715050,0irq/26-eth0-rx-0-21swapper/322:03:133
29715050,0irq/26-eth0-rx-0-21swapper/321:53:493
29715050,0irq/26-eth0-rx-0-21swapper/321:14:493
29715050,0irq/26-eth0-rx-0-21swapper/300:18:393
29715050,0irq/26-eth0-rx-0-21swapper/300:00:283
179950,0migration/19632-21missed_timers22:23:441
179950,0migration/1945-21rt-features23:18:471
179950,0migration/18932-21timerandwakeup23:43:491
179950,0migration/131860-21kthreadcore21:18:451
179950,0migration/130461-21aten_r1power_fr18:58:371
179950,0migration/13031-21taskset21:39:591
179950,0migration/125593-21taskset00:15:451
179950,0migration/120826-21sh21:33:261
179950,0migration/112850-21sh21:28:281
179950,0migration/111075-21perf23:28:261
149950,0migration/03335-21taskset21:01:260
149950,0migration/030313-21sendmail_mailqu19:18:480
149950,0migration/029369-21kthreadcore00:18:430
149950,0migration/020523-21latency_hist21:53:260
149950,0migration/018165-21taskset23:51:340
149950,0migration/018031-21ntp_offset22:08:460
149950,0migration/017173-21kthreadcore19:58:430
149950,0migration/013445-21kthreadcore19:08:440
339940,0migration/39163-21kthreadcore22:43:443
339940,0migration/38049-21kthreadcore22:03:443
339940,0migration/36434-21ksmtuned19:28:423
339940,0migration/332712-21kthreadcore21:18:463
339940,0migration/33156-21kthreadcore20:13:443
339940,0migration/328160-21kthreadcore20:08:433
339940,0migration/322670-21kthreadcore20:28:443
339940,0migration/314988-21kthreadcore23:08:443
301689943,0cyclictest3556-21lldpd20:53:552
301689943,0cyclictest3556-21lldpd20:49:252
301689943,0cyclictest3514-21snmpd20:44:012
301689943,0cyclictest3241-21irqbalance19:41:252
301689943,0cyclictest3241-21irqbalance19:21:152
301689943,0cyclictest31218-21awk18:58:402
301689943,0cyclictest13287-21awk19:33:432
301689941,0cyclictest26485-21if_err_eth1.219:43:412
301679943,0cyclictest6596-21awk19:28:431
301679943,0cyclictest5345-21awk22:03:421
301679943,0cyclictest3514-21snmpd23:55:031
301679943,0cyclictest3514-21snmpd23:00:021
301679943,0cyclictest3514-21snmpd22:30:471
301679943,0cyclictest3514-21snmpd21:44:461
301679943,0cyclictest3514-21snmpd20:08:461
301679943,0cyclictest3514-21snmpd00:06:481
301679943,0cyclictest3422-21awk00:23:391
301679943,0cyclictest3241-21irqbalance23:09:141
301679943,0cyclictest3241-21irqbalance21:12:551
301679943,0cyclictest3241-21irqbalance21:03:451
301679943,0cyclictest2848-21awk20:38:421
301679943,0cyclictest21176-21awk22:33:431
301679943,0cyclictest20315-21awk23:33:421
301679943,0cyclictest20308-21df22:53:381
301679943,0cyclictest13534-21awk21:48:411
301679943,0cyclictest13410-21awk22:08:421
301679943,0cyclictest11684-21awk00:08:411
301669944,0cyclictest0-21swapper/020:43:300
301669943,0cyclictest870-21awk19:48:420
301669943,0cyclictest5487-21awk21:43:420
301669943,0cyclictest3241-21irqbalance23:03:350
301669943,0cyclictest3241-21irqbalance22:55:350
301669943,0cyclictest3241-21irqbalance22:23:350
301669943,0cyclictest3241-21irqbalance22:00:350
301669943,0cyclictest3241-21irqbalance21:23:350
301669943,0cyclictest3241-21irqbalance21:09:250
301669943,0cyclictest3241-21irqbalance20:03:450
301669943,0cyclictest3241-21irqbalance00:14:540
301669943,0cyclictest26484-21awk19:43:410
301669943,0cyclictest21257-21awk22:33:430
301669943,0cyclictest20096-21awk23:53:440
301669943,0cyclictest1641-21sadc20:38:270
301669943,0cyclictest16227-21awk20:48:430
301669940,0cyclictest0-21swapper/020:14:010
29725040,0irq/27-eth0-tx-3524-21sshd19:53:350
29715040,0irq/26-eth0-rx-0-21swapper/323:23:373
29715040,0irq/26-eth0-rx-0-21swapper/323:08:213
29715040,0irq/26-eth0-rx-0-21swapper/322:48:353
29715040,0irq/26-eth0-rx-0-21swapper/322:18:393
29715040,0irq/26-eth0-rx-0-21swapper/300:24:143
29715040,0irq/26-eth0-rx-0-21swapper/300:13:373
29715040,0irq/26-eth0-rx-0-21swapper/300:09:513
29715040,0irq/26-eth0-rx-0-21swapper/300:03:333
179940,0migration/19180-21fschecks_time20:43:401
179940,0migration/18370-21kthreadcore19:03:451
179940,0migration/17247-21fschecks_time19:53:401
179940,0migration/15316-21fschecks_count21:23:401
179940,0migration/14136-21kthreadcore20:13:451
179940,0migration/130568-21kthreadcore20:58:441
179940,0migration/130454-21timerandwakeup19:18:491
179940,0migration/130182-21kthreadcore22:18:441
179940,0migration/129515-21kthreadcore23:58:441
179940,0migration/128945-21fschecks_time22:38:401
179940,0migration/127425-21aten_r1power_fr23:38:351
179940,0migration/119873-21cpu23:13:381
179940,0migration/117716-21taskset19:12:401
179940,0migration/112349-21taskset19:33:381
179940,0migration/110957-21kthreadcore20:18:421
169940,0watchdog/14745-21ls22:43:401
169940,0watchdog/132574-21sensors00:18:481
169940,0watchdog/129682-21kthreadcore20:33:451
169940,0watchdog/126251-21missed_timers21:13:451
169940,0watchdog/124156-21kthreadcore20:53:451
169940,0watchdog/123417-21kthreadcore20:03:451
169940,0watchdog/117818-21ssh23:29:481
169940,0watchdog/115804-21kthreadcore23:48:441
169940,0watchdog/10-21swapper/123:03:321
169940,0watchdog/10-21swapper/121:58:411
149940,0migration/09384-21kthreadcore19:28:450
149940,0migration/05127-21kthreadcore20:38:440
149940,0migration/032471-21kthreadcore22:38:440
149940,0migration/031387-21kthreadcore22:18:440
149940,0migration/028713-21kthreadcore20:08:430
149940,0migration/019782-21aten_r1power_po23:13:370
149940,0migration/016768-21unixbench-2d00:08:490
149940,0migration/016356-21kthreadcore21:28:430
149940,0migration/013209-21if_eth1.222:28:420
149940,0migration/012995-21missed_timers20:18:440
339930,0migration/321983-21kthreadcore20:03:453
301689932,0cyclictest18965-21awk19:13:422
301679933,0cyclictest0-21swapper/122:14:291
301679932,0cyclictest444-21tune2fs19:48:401
301679932,0cyclictest32206-21awk19:23:421
301679931,0cyclictest0-21swapper/122:48:431
301669933,0cyclictest4050-21kworker/0:223:43:370
301669933,0cyclictest4050-21kworker/0:223:33:530
301669933,0cyclictest12095-21kworker/0:022:43:340
301669933,0cyclictest12095-21kworker/0:022:13:280
301669933,0cyclictest12095-21kworker/0:022:03:400
301669933,0cyclictest12095-21kworker/0:021:03:280
301669933,0cyclictest12095-21kworker/0:020:28:340
301669933,0cyclictest12095-21kworker/0:019:38:300
301669933,0cyclictest12095-21kworker/0:019:33:280
301669933,0cyclictest12095-21kworker/0:019:23:280
301669933,0cyclictest12095-21kworker/0:019:13:300
301669932,0cyclictest13510-21awk21:48:410
301669932,0cyclictest12863-21awk22:48:410
29715030,0irq/26-eth0-rx-0-21swapper/320:43:383
29715030,0irq/26-eth0-rx-0-21swapper/319:53:383
29715030,0irq/26-eth0-rx-0-21swapper/319:13:393
249930,0watchdog/20-21swapper/220:23:412
249930,0watchdog/20-21swapper/219:08:452
179930,0migration/131007-21missed_timers19:43:451
169930,0watchdog/127409-21ssh21:55:371
169930,0watchdog/10-21swapper/120:48:451
169930,0watchdog/10-21swapper/120:28:451
169930,0watchdog/10-21swapper/120:23:411
339920,0migration/35083-21diskstats19:03:413
329920,0watchdog/332403-21kthreadcore20:33:463
301679920,0cyclictest0-21swapper/119:38:371
169920,0watchdog/10-21swapper/119:58:291
169920,0watchdog/10-21swapper/119:13:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional