You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 11:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Jul 09, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:46:373
1249823322,0sleep00-21swapper/018:46:200
1284023223,0sleep20-21swapper/218:50:182
1277223122,0sleep10-21swapper/118:49:291
339960,0migration/34127-21kthreadcore23:45:373
339960,0migration/328253-21munin-node21:20:343
339960,0migration/326225-21sh23:59:053
339960,0migration/325661-21sh23:17:013
339960,0migration/323644-21kthreadcore21:35:373
339960,0migration/32133-21sh23:04:063
339960,0migration/31878-21munin-node19:05:373
339960,0migration/310622-21sh22:49:453
29715060,0irq/26-eth0-rx-31870-21grep21:40:393
29715060,0irq/26-eth0-rx-27902-21diskstats22:20:343
29715060,0irq/26-eth0-rx-24769-21python23:35:423
29715060,0irq/26-eth0-rx-20300-21iostat20:55:383
129169960,0cyclictest0-21swapper/223:00:572
8215050,0irq/24-0000:00:27379-21cat22:20:212
339950,0migration/38762-21timerandwakeup23:25:433
339950,0migration/37601-21timerandwakeup19:30:443
339950,0migration/3619-21munin-node21:00:423
339950,0migration/33846-21fschecks_count22:05:343
339950,0migration/332672-21kthreadcore22:40:393
339950,0migration/32728-21munin-node00:05:323
339950,0migration/324223-21kthreadcore19:20:393
339950,0migration/322863-21aten2.4-expect20:10:343
339950,0migration/322286-21fschecks_count19:45:353
339950,0migration/322179-21kthreadcore22:15:383
339950,0migration/321417-21kthreadcore18:55:393
339950,0migration/320406-21irqcore22:35:383
339950,0migration/319919-21cpuspeed_turbos21:55:323
339950,0migration/319451-21cpuspeed_turbos22:55:343
339950,0migration/319128-21munin-node18:50:463
339950,0migration/318101-21runrttasks20:53:083
339950,0migration/317342-21munin-node20:30:343
339950,0migration/317255-21taskset23:51:253
339950,0migration/316210-21kthreadcore22:50:383
339950,0migration/316013-21kthreadcore00:10:393
339950,0migration/313501-21munin-node19:10:463
339950,0migration/311682-21aten_r1power_en21:30:323
29725050,0irq/27-eth0-tx-8992-21ssh23:25:452
29725050,0irq/27-eth0-tx-3514-21snmpd21:35:382
29725050,0irq/27-eth0-tx-3144-21munin-node22:25:412
29725050,0irq/27-eth0-tx-27842-21ssh23:40:352
29725050,0irq/27-eth0-tx-26551-21ssh22:38:522
29725050,0irq/27-eth0-tx-19624-21munin-node21:55:352
29725050,0irq/27-eth0-tx-18385-21ssh22:53:592
29725050,0irq/27-eth0-tx-17401-21rt-features21:30:432
29725050,0irq/27-eth0-tx-12717-21iostat21:10:382
29725050,0irq/27-eth0-tx-11575-21munin-node21:50:322
29725050,0irq/27-eth0-tx-11011-21munin-node23:10:432
29725050,0irq/27-eth0-tx-0-21swapper/223:55:382
29725050,0irq/27-eth0-tx-0-21swapper/223:47:322
29725050,0irq/27-eth0-tx-0-21swapper/223:34:192
29725050,0irq/27-eth0-tx-0-21swapper/222:32:202
29725050,0irq/27-eth0-tx-0-21swapper/020:39:190
29725050,0irq/27-eth0-tx-0-21swapper/019:05:320
29725050,0irq/27-eth0-tx-0-21swapper/018:50:470
29715050,0irq/26-eth0-rx-31563-21tr19:25:373
29715050,0irq/26-eth0-rx-29140-21grep23:20:393
29715050,0irq/26-eth0-rx-17891-21sshd22:11:393
29715050,0irq/26-eth0-rx-16721-21python23:30:433
29715050,0irq/26-eth0-rx-12375-21hwlatdetect21:50:363
29715050,0irq/26-eth0-rx-11733-21gltestperf23:10:343
29715050,0irq/26-eth0-rx-0-21swapper/323:44:203
29715050,0irq/26-eth0-rx-0-21swapper/321:25:443
29715050,0irq/26-eth0-rx-0-21swapper/321:12:263
29715050,0irq/26-eth0-rx-0-21swapper/300:00:323
259950,0migration/231672-21kthreadcore22:40:382
259950,0migration/228263-21munin-node21:00:362
259950,0migration/227601-21aten_r1power_po19:25:312
259950,0migration/222253-21kthreadcore19:20:382
259950,0migration/220293-21munin-node22:15:352
259950,0migration/219525-21cpuspeed_turbos20:55:342
259950,0migration/218692-21sh20:54:402
259950,0migration/218292-21kthreadcore18:50:402
249950,0watchdog/230409-21grep23:20:392
179950,0migration/18836-21munin-node21:05:391
179950,0migration/16811-21kthreadcore21:45:381
179950,0migration/125266-21timerwakeupswit22:35:451
179950,0migration/123922-21kthreadcore23:55:371
179950,0migration/120813-21kthreadcore20:55:391
179950,0migration/113783-21kthreadcore22:30:391
179950,0migration/112850-21missed_timers19:10:411
169950,0watchdog/124560-21if_err_eth1.220:35:361
149950,0migration/03426-21aten_r1power_fr21:05:320
149950,0migration/02984-21kthreadcore20:40:390
149950,0migration/029416-21kthreadcore21:40:380
149950,0migration/02753-21sh22:04:430
149950,0migration/026615-21sh20:58:480
149950,0migration/026121-21sh23:38:330
149950,0migration/021152-21kthreadcore23:15:370
149950,0migration/018566-21taskset21:12:360
149950,0migration/015099-21latency_hist19:40:220
149950,0migration/013595-21kthreadcore20:50:380
149950,0migration/011736-21idleruntime23:30:350
149950,0migration/011148-21cpuspeed_turbos23:50:320
149950,0migration/010780-21sh00:10:290
129169955,0cyclictest0-21swapper/221:15:472
129169955,0cyclictest0-21swapper/200:05:452
129169950,0cyclictest0-21swapper/223:52:592
129169950,0cyclictest0-21swapper/223:16:052
129159950,0cyclictest11134-21diskmemload23:48:591
129159950,0cyclictest0-21swapper/121:34:311
129139955,0cyclictest0-21swapper/022:42:350
129139955,0cyclictest0-21swapper/022:25:430
129139955,0cyclictest0-21swapper/021:38:160
129139955,0cyclictest0-21swapper/020:05:330
129139954,0cyclictest3514-21snmpd20:01:290
129139950,0cyclictest0-21swapper/023:42:490
129139950,0cyclictest0-21swapper/021:19:490
339940,0migration/33282-21missed_timers20:40:403
339940,0migration/332131-21kthreadcore19:50:383
339940,0migration/316472-21kthreadcore19:15:363
339940,0migration/314288-21kthreadcore20:25:393
29725040,0irq/27-eth0-tx-22783-21aten_r1power_en20:10:330
29725040,0irq/27-eth0-tx-11134-21diskmemload21:20:282
29725040,0irq/27-eth0-tx-0-21swapper/223:05:442
29725040,0irq/27-eth0-tx-0-21swapper/222:45:282
29725040,0irq/27-eth0-tx-0-21swapper/222:00:382
29725040,0irq/27-eth0-tx-0-21swapper/221:45:272
29725040,0irq/27-eth0-tx-0-21swapper/221:26:012
29725040,0irq/27-eth0-tx-0-21swapper/221:06:322
29725040,0irq/27-eth0-tx-0-21swapper/200:15:342
29725040,0irq/27-eth0-tx-0-21swapper/200:00:342
29725040,0irq/27-eth0-tx-0-21swapper/020:25:430
29725040,0irq/27-eth0-tx-0-21swapper/019:50:320
29725040,0irq/27-eth0-tx-0-21swapper/019:30:350
29715040,0irq/26-eth0-rx-29991-40yum-updatesd-he20:39:273
29715040,0irq/26-eth0-rx-0-21swapper/323:08:083
29715040,0irq/26-eth0-rx-0-21swapper/322:32:273
29715040,0irq/26-eth0-rx-0-21swapper/322:25:523
29715040,0irq/26-eth0-rx-0-21swapper/322:00:533
29715040,0irq/26-eth0-rx-0-21swapper/321:45:453
29715040,0irq/26-eth0-rx-0-21swapper/321:15:373
29715040,0irq/26-eth0-rx-0-21swapper/321:06:383
29715040,0irq/26-eth0-rx-0-21swapper/320:15:383
29715040,0irq/26-eth0-rx-0-21swapper/319:00:323
29715040,0irq/26-eth0-rx-0-21swapper/300:15:333
259940,0migration/28833-21kthreadcore19:10:392
259940,0migration/27069-21kthreadcore20:45:372
259940,0migration/26571-21switchtime19:05:442
259940,0migration/23770-21fschecks_count22:05:342
259940,0migration/224585-21munin-node23:35:402
259940,0migration/22356-21fschecks_time19:30:342
259940,0migration/220442-21munin-node19:40:422
259940,0migration/211981-21fschecks_time22:10:352
259940,0migration/210740-21runrttasks00:10:242
249940,0watchdog/22787-21aten_r1power_fr19:55:322
249940,0watchdog/225470-21python18:55:442
249940,0watchdog/223871-21tr22:55:392
249940,0watchdog/214659-21fschecks_count19:15:322
179940,0migration/19014-21missed_timers20:20:381
179940,0migration/18803-21unixbench_multi00:05:451
179940,0migration/18339-21kthreadcore21:25:391
179940,0migration/131639-21kthreadcore20:15:381
179940,0migration/130516-21kthreadcore22:00:371
179940,0migration/129451-21munin-node19:50:361
179940,0migration/127531-21df_abs23:20:341
179940,0migration/127370-21idleruntime-cro22:00:211
179940,0migration/127022-21taskset19:22:491
179940,0migration/125174-21taskset00:16:161
179940,0migration/122238-21kthreadcore21:35:371
179940,0migration/110892-21sh23:50:311
169940,0watchdog/130688-21grep00:00:391
169940,0watchdog/126735-21ssh22:59:511
169940,0watchdog/118933-21ssh23:35:231
169940,0watchdog/118002-21ssh21:51:321
169940,0watchdog/117876-21if_err_eth1.220:30:361
169940,0watchdog/114670-21fschecks_count19:15:321
169940,0watchdog/10-21swapper/123:29:071
169940,0watchdog/10-21swapper/119:55:321
149940,0migration/09811-21kthreadcore20:45:380
149940,0migration/07405-21kthreadcore22:45:390
149940,0migration/05532-21kthreadcore21:45:380
149940,0migration/03458-21fschecks_time23:25:340
149940,0migration/03294-21fschecks_time19:55:340
149940,0migration/028846-21kthreadcore22:20:380
149940,0migration/02735-21taskset21:24:000
149940,0migration/026062-21taskset22:17:080
149940,0migration/021994-21kthreadcore22:55:390
149940,0migration/020705-21kthreadcore23:55:350
149940,0migration/018373-21runrttasks22:13:040
149940,0migration/018141-21taskset23:13:310
149940,0migration/015298-21irqcore19:15:350
149940,0migration/013691-21kthreadcore22:50:370
149940,0migration/011980-21kthreadcore19:10:400
149940,0migration/01146-21kthreadcore20:15:390
149940,0migration/01090-21taskset23:01:080
129169943,0cyclictest3556-21lldpd21:40:332
129169943,0cyclictest3556-21lldpd20:38:332
129169943,0cyclictest3514-21snmpd20:40:302
129169943,0cyclictest3514-21snmpd20:30:302
129169943,0cyclictest3514-21snmpd20:16:292
129169943,0cyclictest3514-21snmpd20:11:442
129169943,0cyclictest3514-21snmpd20:00:292
129169943,0cyclictest3514-21snmpd19:37:292
129169943,0cyclictest3241-21irqbalance19:00:412
129159944,0cyclictest0-21swapper/122:20:331
129159944,0cyclictest0-21swapper/121:20:291
129159944,0cyclictest0-21swapper/121:01:521
129159943,0cyclictest4049-21awk22:25:351
129159943,0cyclictest3556-21lldpd23:41:341
129159943,0cyclictest3556-21lldpd23:16:331
129159943,0cyclictest3556-21lldpd23:10:331
129159943,0cyclictest3556-21lldpd22:50:341
129159943,0cyclictest3556-21lldpd22:47:031
129159943,0cyclictest3556-21lldpd22:44:031
129159943,0cyclictest3556-21lldpd22:07:341
129159943,0cyclictest3556-21lldpd21:19:041
129159943,0cyclictest3556-21lldpd21:10:341
129159943,0cyclictest3556-21lldpd20:41:021
129159943,0cyclictest3556-21lldpd19:48:021
129159943,0cyclictest3556-21lldpd19:37:011
129159943,0cyclictest3556-21lldpd18:50:301
129159943,0cyclictest3556-21lldpd00:10:341
129159943,0cyclictest28641-21awk21:40:361
129159943,0cyclictest15376-21tr19:40:351
129159943,0cyclictest12276-21awk20:50:361
129159943,0cyclictest11238-21awk20:25:371
129159943,0cyclictest10331-21awk20:00:361
129159940,0cyclictest0-21swapper/123:33:421
129159940,0cyclictest0-21swapper/122:10:461
129139944,0cyclictest11134-21diskmemload21:50:240
129139943,0cyclictest4596-21awk20:20:350
129139943,0cyclictest4479-21awk21:25:370
129139943,0cyclictest3787-21awk23:05:340
129139943,0cyclictest3514-21snmpd21:32:000
129139943,0cyclictest3241-21irqbalance19:46:500
129139943,0cyclictest3241-21irqbalance19:35:300
129139943,0cyclictest3241-21irqbalance19:29:300
129139943,0cyclictest3241-21irqbalance00:19:090
129139943,0cyclictest3241-21irqbalance00:02:490
129139943,0cyclictest28072-21awk23:20:370
129139943,0cyclictest26220-21sadc19:00:220
129139943,0cyclictest17974-21awk20:30:370
339930,0migration/310745-21runrttasks20:46:053
29715030,0irq/26-eth0-rx-0-21swapper/320:01:293
29715030,0irq/26-eth0-rx-0-21swapper/319:55:353
29715030,0irq/26-eth0-rx-0-21swapper/319:40:353
29715030,0irq/26-eth0-rx-0-21swapper/319:37:063
179930,0migration/17382-21proc_pri19:30:411
169930,0watchdog/10-21swapper/123:05:271
169930,0watchdog/10-21swapper/122:15:321
169930,0watchdog/10-21swapper/120:45:401
169930,0watchdog/10-21swapper/118:55:441
129169933,0cyclictest11262-21awk20:25:372
129169932,0cyclictest3556-21lldpd20:21:332
129169932,0cyclictest3514-21snmpd19:45:442
129169932,0cyclictest28845-21df_inode19:50:342
129169932,0cyclictest17063-21awk20:05:362
129159932,0cyclictest3556-21lldpd20:09:031
129159932,0cyclictest31878-21kthreadcore23:00:381
129159932,0cyclictest27308-21awk19:00:361
129159932,0cyclictest1673-21awk19:05:361
129139933,0cyclictest3150-21kworker/0:022:30:370
129139933,0cyclictest3150-21kworker/0:021:55:430
129139933,0cyclictest3150-21kworker/0:021:01:490
129139932,0cyclictest3786-21awk23:45:360
129139932,0cyclictest3514-21snmpd22:05:310
129139932,0cyclictest2582-21sh00:05:250
129139932,0cyclictest21836-21awk19:20:370
129139932,0cyclictest20324-21awk22:35:370
129139930,0cyclictest26853-21diskstats19:00:340
29715020,0irq/26-eth0-rx-0-21swapper/320:20:223
29715020,0irq/26-eth0-rx-0-21swapper/320:05:333
169920,0watchdog/10-21swapper/120:10:281
169920,0watchdog/10-21swapper/119:25:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional