You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 01:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue May 26, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450830,0irq/25-eth10-21swapper/318:51:073
172023627,0sleep10-21swapper/118:52:541
175023425,0sleep20-21swapper/218:53:152
165222622,0sleep00-21swapper/018:52:050
19349999,0cyclictest14556-21sensors18:59:572
35170,0ktimersoftd/30-21swapper/320:14:513
4160,0ktimersoftd/00-21swapper/021:53:370
339960,0migration/37906-21sh22:17:553
339960,0migration/37063-21munin-node20:34:513
339960,0migration/332109-21sh21:51:593
339960,0migration/329643-21munin-node00:09:563
339960,0migration/324963-21sh21:28:443
339960,0migration/323358-21sh23:48:213
339960,0migration/323072-21sh22:25:353
339960,0migration/322110-21sh00:05:173
339960,0migration/316000-21taskset21:21:203
339960,0migration/315746-21sh21:41:063
339960,0migration/314354-21mdstat21:59:513
339960,0migration/314233-21sh23:40:033
339960,0migration/31013-21sh21:14:433
29725060,0irq/27-eth0-tx-10852-21munin-node19:49:560
29715060,0irq/26-eth0-rx-9150-21df_inode22:19:453
29715060,0irq/26-eth0-rx-5121-21cut22:34:513
29715060,0irq/26-eth0-rx-30658-21pmu-power22:09:553
29715060,0irq/26-eth0-rx-30658-21pmu-power22:09:553
29715060,0irq/26-eth0-rx-24468-21diskstats23:49:463
27160,0ktimersoftd/20-21swapper/220:14:512
259960,0migration/215197-21sh00:24:042
19349960,0cyclictest0-21swapper/220:44:512
19339960,0cyclictest9516-21kthreadcore23:19:491
19319960,0cyclictest0-21swapper/023:44:460
149960,0migration/07320-21sh23:58:580
149960,0migration/031670-21runrttasks20:04:490
149960,0migration/031588-21sh22:11:080
149960,0migration/031588-21sh22:11:080
149960,0migration/030612-21sh23:51:070
149960,0migration/024152-21sh23:09:330
149960,0migration/018018-21irqrtprio22:24:500
8215050,0irq/24-0000:00:24880-21latency_hist20:24:342
8215050,0irq/24-0000:00:0-21swapper/219:24:342
339950,0migration/38324-21taskset18:55:243
339950,0migration/36781-21taskset23:57:263
339950,0migration/3478-21fschecks_time23:34:463
339950,0migration/329624-21munin-node19:14:473
339950,0migration/325404-21irqcore23:29:503
339950,0migration/325205-21irqcore19:59:513
339950,0migration/324040-21taskset23:09:133
339950,0migration/32245-21kthreadcore21:54:503
339950,0migration/320824-21kthreadcore20:19:513
339950,0migration/318217-21kthreadcore23:24:493
339950,0migration/317163-21latency_hist21:44:343
339950,0migration/316990-21taskset19:51:053
339950,0migration/315359-21runrttasks23:22:533
339950,0migration/312861-21munin-node20:39:453
339950,0migration/312780-21kthreadcore00:19:513
29725050,0irq/27-eth0-tx-8920-21munin-node18:59:560
29725050,0irq/27-eth0-tx-8749-21munin-node22:19:482
29725050,0irq/27-eth0-tx-864-21munin-node21:54:532
29725050,0irq/27-eth0-tx-7807-21munin-node23:59:462
29725050,0irq/27-eth0-tx-6559-21runrttasks22:11:072
29725050,0irq/27-eth0-tx-6559-21runrttasks22:11:072
29725050,0irq/27-eth0-tx-32314-21sh23:34:442
29725050,0irq/27-eth0-tx-31866-21munin-node00:14:542
29725050,0irq/27-eth0-tx-26969-21grep23:29:512
29725050,0irq/27-eth0-tx-21826-21swap23:44:572
29725050,0irq/27-eth0-tx-1022-21munin-node21:14:542
29725050,0irq/27-eth0-tx-0-21swapper/223:52:372
29725050,0irq/27-eth0-tx-0-21swapper/222:53:272
29725050,0irq/27-eth0-tx-0-21swapper/221:45:522
29725050,0irq/27-eth0-tx-0-21swapper/221:21:252
29725050,0irq/27-eth0-tx-0-21swapper/020:39:090
29725050,0irq/27-eth0-tx-0-21swapper/020:24:230
29725050,0irq/27-eth0-tx-0-21swapper/019:55:390
29715050,0irq/26-eth0-rx-4790-21kthreadcore23:14:503
29715050,0irq/26-eth0-rx-4157-21python20:04:563
29715050,0irq/26-eth0-rx-387-21if_eth1.300:14:483
29715050,0irq/26-eth0-rx-31196-21ssh23:11:413
29715050,0irq/26-eth0-rx-28649-21ssh21:09:523
29715050,0irq/26-eth0-rx-26476-21if_eth121:29:493
29715050,0irq/26-eth0-rx-26455-21cut22:29:513
29715050,0irq/26-eth0-rx-20016-21grep22:04:513
29715050,0irq/26-eth0-rx-13579-21munin-node23:59:563
29715050,0irq/26-eth0-rx-11838-21kthreadcore20:59:523
29715050,0irq/26-eth0-rx-0-21swapper/322:54:493
29715050,0irq/26-eth0-rx-0-21swapper/320:55:343
29715050,0irq/26-eth0-rx-0-21swapper/320:44:473
259950,0migration/27772-21pmu-power18:54:542
259950,0migration/27063-21taskset23:58:142
259950,0migration/227779-21kthreadcore22:29:512
259950,0migration/226783-21fschecks_time20:49:482
259950,0migration/224113-21sh23:09:272
259950,0migration/223643-21sh22:27:112
259950,0migration/222926-21taskset00:07:382
259950,0migration/216852-21latency_hist21:04:342
259950,0migration/216622-21aten2.4-expect22:44:442
259950,0migration/216385-21latency_hist19:29:342
259950,0migration/215320-21runrttasks23:02:112
259950,0migration/215149-21sh22:00:042
249950,0watchdog/232334-21ssh21:31:522
249950,0watchdog/214214-21sed23:19:562
19349955,0cyclictest0-21swapper/222:35:032
19349955,0cyclictest0-21swapper/219:44:472
19349950,0cyclictest0-21swapper/221:36:432
19349950,0cyclictest0-21swapper/221:01:062
19339955,0cyclictest32516-21diskmemload22:47:301
19339955,0cyclictest0-21swapper/123:59:521
19339955,0cyclictest0-21swapper/122:06:501
19339955,0cyclictest0-21swapper/100:12:571
19339954,0cyclictest3241-21irqbalance20:45:221
19339950,0cyclictest0-21swapper/123:53:531
19339950,0cyclictest0-21swapper/123:15:151
19339950,0cyclictest0-21swapper/122:28:351
19319954,0cyclictest16128-21sadc23:04:330
19319950,0cyclictest0-21swapper/022:20:070
19319950,0cyclictest0-21swapper/000:09:500
179950,0migration/19682-21fschecks_time21:59:461
179950,0migration/16006-21latency_hist20:34:341
179950,0migration/131452-21sh23:32:581
179950,0migration/123149-21pmu-power21:24:541
179950,0migration/122294-21timerandwakeup23:04:561
179950,0migration/120153-21kthreadcore19:04:511
179950,0migration/116837-21gltestperf23:44:471
169950,0watchdog/131604-21ls20:04:481
149950,0migration/08420-21cpuspeed_turbos23:19:450
149950,0migration/032095-21munin-node00:14:440
149950,0migration/031776-21switchtime20:49:570
149950,0migration/02965-21kthreadcore20:29:510
149950,0migration/02945-21kthreadcore21:54:500
149950,0migration/028063-21kthreadcore22:29:510
149950,0migration/024818-21needreboot20:44:530
149950,0migration/023736-21sh22:06:390
149950,0migration/022475-21cpuspeed_turbos19:09:470
149950,0migration/02022-21kthreadcore22:14:500
149950,0migration/019235-21kthreadcore21:44:500
149950,0migration/018356-21kthreadcore19:04:500
149950,0migration/017721-21kthreadcore23:24:490
149950,0migration/01728-21munin-node22:34:500
149950,0migration/01672-21kthreadcore23:14:490
149950,0migration/016542-21gltestperf00:04:470
149950,0migration/01629-21runrttasks20:54:490
149950,0migration/01607-21fschecks_time21:14:460
339940,0migration/35072-21sendmail_mailtr20:29:563
339940,0migration/318514-21kthreadcore19:29:503
29725040,0irq/27-eth0-tx-17497-21ssh21:24:372
29725040,0irq/27-eth0-tx-0-21swapper/223:24:502
29725040,0irq/27-eth0-tx-0-21swapper/223:09:332
29725040,0irq/27-eth0-tx-0-21swapper/222:05:382
29725040,0irq/27-eth0-tx-0-21swapper/221:49:382
29725040,0irq/27-eth0-tx-0-21swapper/221:09:372
29725040,0irq/27-eth0-tx-0-21swapper/220:55:322
29725040,0irq/27-eth0-tx-0-21swapper/019:24:440
29715040,0irq/26-eth0-rx-0-21swapper/322:59:443
29715040,0irq/26-eth0-rx-0-21swapper/322:49:373
29715040,0irq/26-eth0-rx-0-21swapper/322:44:453
29715040,0irq/26-eth0-rx-0-21swapper/322:40:453
29715040,0irq/26-eth0-rx-0-21swapper/321:34:503
29715040,0irq/26-eth0-rx-0-21swapper/321:05:073
29715040,0irq/26-eth0-rx-0-21swapper/319:04:453
259940,0migration/28850-21kthreadcore20:34:522
259940,0migration/23023-21kthreadcore20:29:512
259940,0migration/22980-21kthreadcore20:04:522
259940,0migration/225512-21kthreadcore00:09:512
259940,0migration/221138-21kthreadcore19:54:512
259940,0migration/21465-21idleruntime22:14:482
259940,0migration/211345-21kthreadcore23:39:512
249940,0watchdog/28634-21ssh22:39:442
249940,0watchdog/211328-21kthreadcore21:39:522
19349943,0cyclictest3556-21lldpd23:17:102
19349943,0cyclictest3556-21lldpd20:41:392
19349943,0cyclictest3556-21lldpd20:09:392
19349943,0cyclictest3556-21lldpd19:39:392
19349943,0cyclictest3556-21lldpd19:36:392
19349943,0cyclictest3556-21lldpd19:33:392
19349943,0cyclictest3556-21lldpd19:14:382
19349943,0cyclictest3556-21lldpd19:09:382
19349943,0cyclictest3556-21lldpd19:06:382
19349943,0cyclictest24876-21sadc20:24:342
19349943,0cyclictest11604-21gltestperf19:49:482
19349943,0cyclictest1074-21awk22:54:472
19339944,0cyclictest0-21swapper/123:10:411
19339944,0cyclictest0-21swapper/122:58:231
19339944,0cyclictest0-21swapper/122:50:391
19339943,0cyclictest386-21awk00:14:481
19339943,0cyclictest3556-21lldpd23:55:101
19339943,0cyclictest3556-21lldpd21:31:091
19339943,0cyclictest3556-21lldpd20:55:091
19339943,0cyclictest3556-21lldpd20:31:091
19339943,0cyclictest3556-21lldpd20:13:091
19339943,0cyclictest3556-21lldpd19:34:391
19339943,0cyclictest3556-21lldpd19:11:081
19339943,0cyclictest3556-21lldpd18:59:381
19339943,0cyclictest3556-21lldpd00:05:111
19339943,0cyclictest3241-21irqbalance20:50:221
19339943,0cyclictest3241-21irqbalance20:17:421
19339943,0cyclictest30712-21awk19:39:481
19339943,0cyclictest29731-21awk19:14:481
19339943,0cyclictest26130-21awk21:09:501
19339943,0cyclictest25707-21awk22:29:491
19339943,0cyclictest1303-21awk22:34:481
19319944,0cyclictest0-21swapper/022:54:290
19319944,0cyclictest0-21swapper/000:22:370
19319943,0cyclictest9869-21awk21:59:470
19319943,0cyclictest9554-21awk20:59:490
19319943,0cyclictest771-21awk23:34:480
19319943,0cyclictest26402-21awk21:29:480
19319943,0cyclictest25954-21tail21:09:490
19319943,0cyclictest2145-21awk21:34:490
19319943,0cyclictest18438-21awk21:24:480
19319943,0cyclictest16283-21sadc22:44:340
179940,0migration/19843-21munin-node19:44:561
179940,0migration/131178-21vmstat22:09:581
179940,0migration/131178-21vmstat22:09:581
179940,0migration/130071-21unixbench_multi19:59:591
179940,0migration/129872-21kthreadcore20:24:501
179940,0migration/123718-21latency20:19:521
179940,0migration/118798-21kthreadcore23:24:491
179940,0migration/118595-21kthreadcore19:29:501
179940,0migration/118506-21kthreadcore21:04:511
179940,0migration/117816-21fschecks_time21:44:461
179940,0migration/11507-21sh21:14:461
179940,0migration/112253-21diskmemload22:59:511
179940,0migration/111252-21kthreadcore22:39:491
179940,0migration/110953-21kthreadcore23:39:511
169940,0watchdog/19307-21ssh21:19:401
169940,0watchdog/17673-21sh21:35:521
169940,0watchdog/1325362sleep121:53:121
169940,0watchdog/115855-21ssh22:22:481
169940,0watchdog/11468-21idleruntime22:14:481
169940,0watchdog/111327-21kthreadcore21:39:521
169940,0watchdog/10-21swapper/123:34:471
169940,0watchdog/10-21swapper/121:54:481
169940,0watchdog/10-21swapper/100:22:591
149940,0migration/07944-21kthreadcore19:44:520
149940,0migration/06994-21sh22:56:260
149940,0migration/06974-21kthreadcore19:19:500
149940,0migration/030818-21kthreadcore19:14:500
149940,0migration/021315-21kthreadcore22:44:510
149940,0migration/018574-21timerwakeupswit20:39:560
149940,0migration/013124-21kthreadcore21:39:530
29715030,0irq/26-eth0-rx-0-21swapper/320:10:483
29715030,0irq/26-eth0-rx-0-21swapper/319:58:213
29715030,0irq/26-eth0-rx-0-21swapper/319:39:453
29715030,0irq/26-eth0-rx-0-21swapper/319:35:513
29715030,0irq/26-eth0-rx-0-21swapper/319:19:453
29715030,0irq/26-eth0-rx-0-21swapper/319:11:173
19339932,0cyclictest3556-21lldpd20:59:391
19339932,0cyclictest3556-21lldpd19:29:081
19339932,0cyclictest3013-21awk18:54:481
19339932,0cyclictest18265-21expr19:54:481
19339932,0cyclictest13615-21awk20:39:481
19339930,0cyclictest13940-21tr19:49:521
19319933,0cyclictest22975-21kworker/0:023:59:370
19319933,0cyclictest22975-21kworker/0:023:39:390
19319933,0cyclictest22975-21kworker/0:023:09:370
19319933,0cyclictest22975-21kworker/0:021:19:370
19319933,0cyclictest22975-21kworker/0:021:04:430
19319933,0cyclictest22975-21kworker/0:020:24:410
19319933,0cyclictest22975-21kworker/0:020:14:390
19319933,0cyclictest22975-21kworker/0:020:09:370
19319933,0cyclictest22975-21kworker/0:019:59:370
19319933,0cyclictest22975-21kworker/0:019:39:400
19319933,0cyclictest22975-21kworker/0:019:34:380
19319933,0cyclictest22975-21kworker/0:019:29:360
19319933,0cyclictest12146-21kworker/0:218:54:400
19319932,0cyclictest25111-21awk23:29:480
169930,0watchdog/10-21swapper/119:19:521
339920,0migration/326502-21diskstats20:49:473
29715020,0irq/26-eth0-rx-0-21swapper/320:24:433
29715020,0irq/26-eth0-rx-0-21swapper/319:46:023
29715020,0irq/26-eth0-rx-0-21swapper/319:24:473
29715020,0irq/26-eth0-rx-0-21swapper/318:59:473
19349920,0cyclictest0-21swapper/219:59:342
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional