You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 07:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Apr 04, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3020523827,0sleep20-21swapper/218:57:402
3027923624,0sleep00-21swapper/018:58:300
3000923626,0sleep30-21swapper/318:55:213
3001223526,0sleep10-21swapper/118:55:231
339960,0migration/38342-21sh00:25:193
339960,0migration/34434-21sh21:24:053
339960,0migration/3365-21sh23:59:493
339960,0migration/310460-21sh23:28:513
304359960,0cyclictest28580-21diskmemload22:57:031
304359960,0cyclictest0-21swapper/122:22:121
29725060,0irq/27-eth0-tx-14480-21munin-node20:24:420
29725060,0irq/27-eth0-tx-11868-21fschecks_time23:09:412
29715060,0irq/26-eth0-rx-32260-21kthreadcore22:39:463
149960,0migration/09372-21sh23:46:320
149960,0migration/03578-21sh21:02:410
149960,0migration/01682-21sh23:00:450
149960,0migration/011868-21sh21:08:100
8215050,0irq/24-0000:00:1817-21sadc20:39:282
35150,0ktimersoftd/30-21swapper/322:54:333
339950,0migration/39431-21munin-node22:24:483
339950,0migration/38885-21diskmemload00:05:483
339950,0migration/37642-21kthreadcore21:44:463
339950,0migration/34612-21kernelversion23:04:433
339950,0migration/34223-21aten_r1power_en22:04:393
339950,0migration/33554-21kthreadcore20:14:453
339950,0migration/332604-21timerandwakeup20:09:523
339950,0migration/331941-21taskset19:45:203
339950,0migration/3315-21cpuspeed_turbos19:49:413
339950,0migration/328575-21munin-node22:19:413
339950,0migration/326842-21munin-node20:29:503
339950,0migration/326167-21idleruntime-cro00:19:283
339950,0migration/321913-21taskset20:54:103
339950,0migration/321149-21cpuspeed_turbos21:34:403
339950,0migration/318976-21taskset23:34:353
339950,0migration/317243-21latency22:29:473
339950,0migration/316935-21kthreadcore20:24:463
339950,0migration/316812-21munin-node23:49:513
339950,0migration/315418-21kthreadcore21:29:463
339950,0migration/314326-21hwlatdetect19:59:423
339950,0migration/313283-21kthreadcore23:29:453
304369955,0cyclictest0-21swapper/223:29:272
304369950,0cyclictest0-21swapper/223:34:022
304369950,0cyclictest0-21swapper/221:32:282
304369950,0cyclictest0-21swapper/200:22:352
304359955,0cyclictest28580-21diskmemload23:45:551
304359955,0cyclictest0-21swapper/123:34:531
304359955,0cyclictest0-21swapper/121:59:301
304359955,0cyclictest0-21swapper/119:44:521
304359950,0cyclictest0-21swapper/123:58:321
304359950,0cyclictest0-21swapper/123:15:581
304359950,0cyclictest0-21swapper/123:07:181
304359950,0cyclictest0-21swapper/122:35:081
304359950,0cyclictest0-21swapper/100:22:391
304349955,0cyclictest0-21swapper/022:20:490
304349950,0cyclictest0-21swapper/023:13:570
304349950,0cyclictest0-21swapper/021:41:520
304349950,0cyclictest0-21swapper/021:35:430
304349950,0cyclictest0-21swapper/000:20:190
304349950,0cyclictest0-21swapper/000:16:510
29725050,0irq/27-eth0-tx-9267-40yum-updatesd-he20:44:410
29725050,0irq/27-eth0-tx-5988-21sshd19:29:410
29725050,0irq/27-eth0-tx-5988-21sshd19:29:410
29725050,0irq/27-eth0-tx-4551-21munin-node21:04:522
29725050,0irq/27-eth0-tx-4279-21forks22:24:412
29725050,0irq/27-eth0-tx-22176-21munin-node20:54:520
29725050,0irq/27-eth0-tx-19688-21munin-node22:54:442
29725050,0irq/27-eth0-tx-12552-21munin-node21:49:412
29725050,0irq/27-eth0-tx-12294-21diskstats22:29:422
29725050,0irq/27-eth0-tx-10443-21munin-node00:09:522
29725050,0irq/27-eth0-tx-0-21swapper/019:09:500
29715050,0irq/26-eth0-rx-29169-21cpuspeed_turbos21:39:403
29715050,0irq/26-eth0-rx-25421-21kthreadcore21:14:473
29715050,0irq/26-eth0-rx-21121-21if_err_eth1.322:14:423
29715050,0irq/26-eth0-rx-20533-21ssh21:54:343
29715050,0irq/26-eth0-rx-20005-21aten_r1power_po22:34:403
29715050,0irq/26-eth0-rx-19220-21latency_hist23:14:283
29715050,0irq/26-eth0-rx-16214-21python00:09:513
29715050,0irq/26-eth0-rx-12017-21fschecks_time23:09:413
29715050,0irq/26-eth0-rx-1185-21ssh23:41:113
29715050,0irq/26-eth0-rx-10711-21ssh21:24:503
29715050,0irq/26-eth0-rx-0-21swapper/322:44:473
29715050,0irq/26-eth0-rx-0-21swapper/322:09:453
29715050,0irq/26-eth0-rx-0-21swapper/321:59:563
29715050,0irq/26-eth0-rx-0-21swapper/321:52:053
29715050,0irq/26-eth0-rx-0-21swapper/321:10:373
29715050,0irq/26-eth0-rx-0-21swapper/321:05:083
29715050,0irq/26-eth0-rx-0-21swapper/320:49:203
29715050,0irq/26-eth0-rx-0-21swapper/320:19:463
29715050,0irq/26-eth0-rx-0-21swapper/319:34:503
29715050,0irq/26-eth0-rx-0-21swapper/319:19:413
29715050,0irq/26-eth0-rx-0-21swapper/319:04:403
29715050,0irq/26-eth0-rx-0-21swapper/300:22:213
27150,0ktimersoftd/20-21swapper/223:05:012
259950,0migration/26851-21munin-node19:29:442
259950,0migration/26851-21munin-node19:29:442
259950,0migration/24973-21timerandwakeup19:24:512
259950,0migration/231554-21kthreadcore22:59:442
259950,0migration/227603-21ntp_offset20:54:482
259950,0migration/222034-21kthreadcore23:34:452
259950,0migration/21045-21sh23:19:562
249950,0watchdog/232480-21cut21:19:472
19150,0ktimersoftd/10-21swapper/100:15:521
179950,0migration/131503-21kthreadcore20:59:451
179950,0migration/129617-21gltestperf21:39:411
179950,0migration/126628-21kthreadcore20:54:461
179950,0migration/120330-21latency_hist21:54:281
169950,0watchdog/112373-21tail19:09:431
149950,0migration/06740-21kthreadcore23:24:440
149950,0migration/06005-21timerandwakeup19:49:510
149950,0migration/05750-21kthreadcore00:24:460
149950,0migration/05266-21irqcore22:04:430
149950,0migration/04723-21irqcore22:44:450
149950,0migration/04082-21timerandwakeup18:59:520
149950,0migration/04040-21kthreadcore20:14:450
149950,0migration/032653-21timerwakeupswit20:09:520
149950,0migration/032040-21kthreadcore20:34:450
149950,0migration/030874-21kthreadcore23:19:450
149950,0migration/03038-21runrttasks22:43:500
149950,0migration/028827-21latency_hist21:19:280
149950,0migration/026558-21sh23:18:120
149950,0migration/026351-21fschecks_count19:44:410
149950,0migration/024990-21sh23:55:190
149950,0migration/02298-21pmu-power21:19:500
149950,0migration/021828-21kthreadcore19:39:460
149950,0migration/019919-21munin-node22:34:390
149950,0migration/018874-21taskset19:38:300
149950,0migration/018236-21sh23:53:550
149950,0migration/017968-21missed_timers21:49:470
149950,0migration/014865-21kthreadcore00:09:470
149950,0migration/013853-21cpuspeed_turbos19:59:400
149950,0migration/013782-21munin-node21:29:430
149950,0migration/012327-21latency_hist21:49:280
149950,0migration/012206-21cpuspeed_turbos22:29:410
149950,0migration/010893-21timerwakeupswit21:24:500
339940,0migration/3768-21ntp_states22:59:473
339940,0migration/332629-21latency23:19:463
339940,0migration/332544-21kthreadcore18:59:463
339940,0migration/322285-21kthreadcore19:39:463
339940,0migration/31672-21taskset20:37:493
339940,0migration/313422-21kthreadcore19:09:473
304369943,0cyclictest5835-21awk19:04:432
304369943,0cyclictest3556-21lldpd20:49:502
304369943,0cyclictest3556-21lldpd19:03:202
304369943,0cyclictest3514-21snmpd20:26:582
304369943,0cyclictest3514-21snmpd20:18:572
304369943,0cyclictest3514-21snmpd20:07:572
304369943,0cyclictest3514-21snmpd19:51:582
304369943,0cyclictest3514-21snmpd19:43:282
304369943,0cyclictest3514-21snmpd19:37:282
304369943,0cyclictest29839-21awk21:39:432
304369943,0cyclictest27821-21awk23:39:432
304369943,0cyclictest19136-21awk19:14:422
304359943,0cyclictest5844-21awk21:24:431
304359943,0cyclictest5765-21awk19:04:431
304359943,0cyclictest3960-21df22:44:421
304359943,0cyclictest3556-21lldpd21:45:501
304359943,0cyclictest3556-21lldpd21:36:501
304359943,0cyclictest3556-21lldpd19:36:501
304359943,0cyclictest3556-21lldpd00:10:501
304359943,0cyclictest3361-21awk00:04:431
304359943,0cyclictest2921-21awk20:39:431
304359943,0cyclictest21765-21awk21:14:451
304359943,0cyclictest16323-21awk20:49:431
304359943,0cyclictest12846-21awk22:29:441
304359940,0cyclictest0-21swapper/119:19:571
304349944,0cyclictest0-21swapper/022:49:400
304349944,0cyclictest0-21swapper/022:10:340
304349944,0cyclictest0-21swapper/021:11:550
304349944,0cyclictest0-21swapper/000:06:080
304349943,0cyclictest32535-21awk19:24:440
304349943,0cyclictest3053-21awk20:39:440
304349943,0cyclictest28279-21sadc21:59:280
304349943,0cyclictest25830-21awk19:19:430
304349943,0cyclictest20556-21awk22:54:440
304349943,0cyclictest19121-21awk19:14:420
304349943,0cyclictest16394-21awk20:49:430
29725040,0irq/27-eth0-tx-29247-21fschecks_time21:59:432
29725040,0irq/27-eth0-tx-29021-21idleruntime22:19:432
29725040,0irq/27-eth0-tx-2731-21ssh21:00:352
29725040,0irq/27-eth0-tx-20658-21cut21:14:282
29725040,0irq/27-eth0-tx-11644-21telnet22:49:392
29725040,0irq/27-eth0-tx-0-21swapper/223:59:372
29725040,0irq/27-eth0-tx-0-21swapper/223:44:462
29725040,0irq/27-eth0-tx-0-21swapper/222:44:412
29725040,0irq/27-eth0-tx-0-21swapper/222:40:312
29725040,0irq/27-eth0-tx-0-21swapper/222:34:302
29725040,0irq/27-eth0-tx-0-21swapper/222:14:352
29725040,0irq/27-eth0-tx-0-21swapper/222:09:472
29725040,0irq/27-eth0-tx-0-21swapper/221:54:302
29725040,0irq/27-eth0-tx-0-21swapper/221:45:242
29725040,0irq/27-eth0-tx-0-21swapper/221:25:512
29725040,0irq/27-eth0-tx-0-21swapper/221:09:442
29725040,0irq/27-eth0-tx-0-21swapper/200:04:312
29715040,0irq/26-eth0-rx-17249-21open_inodes22:49:483
29715040,0irq/26-eth0-rx-0-21swapper/323:54:343
29715040,0irq/26-eth0-rx-0-21swapper/323:44:333
29715040,0irq/26-eth0-rx-0-21swapper/320:59:453
259940,0migration/26633-21kthreadcore22:04:442
259940,0migration/231123-21latency19:44:462
259940,0migration/229447-21kthreadcore20:09:462
259940,0migration/22557-21fschecks_count20:39:422
259940,0migration/220964-21aten_r1power_fr21:34:392
259940,0migration/220142-21kthreadcore00:14:442
259940,0migration/219937-21fschecks_time23:14:412
259940,0migration/216166-21kthreadcore19:59:452
259940,0migration/214148-21missed_timers20:44:462
249940,0watchdog/223327-21kthreadcore23:54:462
249940,0watchdog/212374-21head19:09:432
249940,0watchdog/20-21swapper/223:49:422
249940,0watchdog/20-21swapper/220:29:432
249940,0watchdog/20-21swapper/219:54:472
249940,0watchdog/20-21swapper/200:24:462
179940,0migration/19610-21kthreadcore20:19:451
179940,0migration/15990-21kthreadcore23:24:441
179940,0migration/15566-21kthreadcore22:24:451
179940,0migration/14844-21cpuspeed_turbos21:04:411
179940,0migration/14502-21kthreadcore00:24:461
179940,0migration/130791-21aten_r1power_vo18:59:411
179940,0migration/127008-21cpuspeed_turbos23:59:381
179940,0migration/125105-21kthreadcore22:14:451
179940,0migration/122621-21kthreadcore20:29:451
179940,0migration/121168-21fschecks_time21:54:421
179940,0migration/119909-21kthreadcore19:14:441
179940,0migration/119044-21munin-node19:59:481
179940,0migration/11761-21taskset23:22:021
179940,0migration/113552-21kernelversion22:09:461
169940,0watchdog/130107-21kthreadcore23:39:461
169940,0watchdog/127856-21ssh22:39:351
169940,0watchdog/113708-21munin-node21:29:431
169940,0watchdog/111008-21aten_r1power_en23:29:381
169940,0watchdog/10-21swapper/123:49:421
169940,0watchdog/10-21swapper/121:19:471
169940,0watchdog/10-21swapper/119:54:471
149940,0migration/08706-21kthreadcore19:04:460
149940,0migration/023819-21kthreadcore20:29:450
339930,0migration/32202-21kthreadcore19:24:453
304369933,0cyclictest14249-21idleruntime-cro20:24:282
304369932,0cyclictest3514-21snmpd19:20:282
304359932,0cyclictest994-21awk19:49:431
304359932,0cyclictest3556-21lldpd21:09:501
304359932,0cyclictest3556-21lldpd19:24:501
304359932,0cyclictest28365-21tr22:59:411
304359932,0cyclictest27665-21awk20:09:441
304359932,0cyclictest15319-21awk20:24:431
304359932,0cyclictest12512-21if_err_eth1.322:49:431
304359932,0cyclictest11582-21sensors19:29:501
304359932,0cyclictest11582-21sensors19:29:501
304359932,0cyclictest11389-21aten_r1power_fr23:09:391
304359932,0cyclictest10777-21kthreadcore20:44:441
304359930,0cyclictest1705-21perl20:14:421
304349933,0cyclictest29652-21kworker/0:123:59:300
304349933,0cyclictest29652-21kworker/0:123:39:320
304349933,0cyclictest29652-21kworker/0:123:34:300
304349933,0cyclictest29652-21kworker/0:123:29:300
304349933,0cyclictest29652-21kworker/0:123:04:300
304349933,0cyclictest29652-21kworker/0:122:24:360
304349933,0cyclictest29652-21kworker/0:122:14:320
304349933,0cyclictest29652-21kworker/0:121:59:320
304349933,0cyclictest11548-21kworker/0:020:19:300
304349933,0cyclictest11548-21kworker/0:020:04:300
304349933,0cyclictest11548-21kworker/0:019:54:430
29715030,0irq/26-eth0-rx-0-21swapper/320:39:563
29715030,0irq/26-eth0-rx-0-21swapper/319:54:543
29715030,0irq/26-eth0-rx-0-21swapper/319:29:403
29715030,0irq/26-eth0-rx-0-21swapper/319:29:403
29715030,0irq/26-eth0-rx-0-21swapper/319:17:263
179930,0migration/17868-21kthreadcore22:04:451
179930,0migration/125423-21runrttasks19:41:561
169930,0watchdog/10-21swapper/120:34:391
339920,0migration/320592-21diskstats20:04:413
29715020,0irq/26-eth0-rx-0-21swapper/320:54:393
169920,0watchdog/10-21swapper/120:04:391
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional