You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 12:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Aug 30, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450710,0irq/25-eth10-21swapper/318:42:233
1447623624,0sleep20-21swapper/218:43:222
1441823523,0sleep00-21swapper/018:42:360
1438823223,0sleep10-21swapper/118:42:211
147229970,0cyclictest22523-21ssh00:12:531
8215060,0irq/24-0000:00:3779-21munin-run21:40:242
339960,0migration/36999-21kthreadcore21:00:403
339960,0migration/327868-21sh20:52:173
339960,0migration/327584-21sh21:12:043
339960,0migration/325468-21sh22:54:363
339960,0migration/323270-21sh00:15:133
339960,0migration/319618-21munin-node22:30:393
339960,0migration/315121-21sh23:25:463
29725060,0irq/27-eth0-tx-13221-21munin-node20:45:400
29715060,0irq/26-eth0-rx-18298-21cut20:45:433
29715060,0irq/26-eth0-rx-0-21swapper/322:16:493
147239960,0cyclictest0-21swapper/223:12:342
149960,0migration/07225-21sh23:42:340
149960,0migration/03301-21sh21:39:060
149960,0migration/027364-21sh20:50:570
149960,0migration/015506-21sh23:26:540
149960,0migration/012224-21cpuspeed_turbos21:45:370
8215050,0irq/24-0000:00:31934-21latency_hist00:00:242
8215050,0irq/24-0000:00:10097-21cat22:45:242
339950,0migration/34131-21munin-node20:10:413
339950,0migration/328283-21switchtime19:15:473
339950,0migration/328117-21latency_hist18:55:253
339950,0migration/327278-21kthreadcore23:55:403
339950,0migration/326342-21idleruntime20:30:373
339950,0migration/324914-21aten_r1power_en23:35:353
339950,0migration/321776-21kthreadcore19:35:413
339950,0migration/32061-21latency_hist22:40:243
339950,0migration/320200-21munin-node22:10:393
339950,0migration/319292-21kthreadcore23:50:423
339950,0migration/317397-21taskset19:57:573
339950,0migration/315656-21gltestperf18:45:383
339950,0migration/313363-21kthreadcore19:05:433
339950,0migration/311385-21kthreadcore20:40:433
29725050,0irq/27-eth0-tx-8427-21munin-node23:45:372
29725050,0irq/27-eth0-tx-3505-21ssh21:18:082
29725050,0irq/27-eth0-tx-28246-21munin-node18:55:360
29725050,0irq/27-eth0-tx-16363-21munin-node23:50:462
29725050,0irq/27-eth0-tx-1533-21munin-node23:00:452
29725050,0irq/27-eth0-tx-12171-21munin-node20:20:430
29725050,0irq/27-eth0-tx-11192-21cut23:05:402
29725050,0irq/27-eth0-tx-11165-21kthreadcore23:25:392
29725050,0irq/27-eth0-tx-0-21swapper/223:30:542
29725050,0irq/27-eth0-tx-0-21swapper/223:24:162
29725050,0irq/27-eth0-tx-0-21swapper/222:55:432
29725050,0irq/27-eth0-tx-0-21swapper/222:20:432
29725050,0irq/27-eth0-tx-0-21swapper/221:06:572
29725050,0irq/27-eth0-tx-0-21swapper/019:40:490
29725050,0irq/27-eth0-tx-0-21swapper/018:47:290
29715050,0irq/26-eth0-rx-941-21ssh22:37:293
29715050,0irq/26-eth0-rx-8236-21kthreadcore22:00:423
29715050,0irq/26-eth0-rx-8230-21ssh23:02:183
29715050,0irq/26-eth0-rx-5180-21grep23:40:423
29715050,0irq/26-eth0-rx-3125-21grep19:00:383
29715050,0irq/26-eth0-rx-20038-21kthreadcore23:30:423
29715050,0irq/26-eth0-rx-18542-21ps23:10:403
29715050,0irq/26-eth0-rx-1819-21python21:35:463
29715050,0irq/26-eth0-rx-12010-21cut22:45:403
29715050,0irq/26-eth0-rx-0-21swapper/323:24:463
29715050,0irq/26-eth0-rx-0-21swapper/322:58:553
29715050,0irq/26-eth0-rx-0-21swapper/322:27:343
29715050,0irq/26-eth0-rx-0-21swapper/321:55:343
29715050,0irq/26-eth0-rx-0-21swapper/321:44:073
29715050,0irq/26-eth0-rx-0-21swapper/321:25:493
29715050,0irq/26-eth0-rx-0-21swapper/321:22:403
29715050,0irq/26-eth0-rx-0-21swapper/321:16:283
29715050,0irq/26-eth0-rx-0-21swapper/300:08:233
29715050,0irq/26-eth0-rx-0-21swapper/300:00:443
259950,0migration/25380-21munin-node21:20:402
259950,0migration/230880-21taskset23:57:132
259950,0migration/229972-21open_files20:05:442
259950,0migration/228636-21sh21:15:162
259950,0migration/221988-21kthreadcore21:30:412
259950,0migration/218558-21unixbench-2d21:25:472
259950,0migration/216565-21idleruntime00:10:382
259950,0migration/214558-21kthreadcore20:45:412
259950,0migration/213942-21sh21:45:412
259950,0migration/210631-21kthreadcore00:05:412
19150,0ktimersoftd/10-21swapper/120:15:461
179950,0migration/130563-21kthreadcore18:55:411
179950,0migration/122283-21fschecks_time18:50:391
179950,0migration/112939-21kthreadcore19:30:411
179950,0migration/110832-21kthreadcore23:45:381
169950,0watchdog/124641-21kthreadcore21:50:421
169950,0watchdog/117994-21grep20:45:431
147239955,0cyclictest0-21swapper/222:36:002
147239955,0cyclictest0-21swapper/219:05:412
147239950,0cyclictest0-21swapper/221:54:042
147229955,0cyclictest0-21swapper/122:47:541
147229955,0cyclictest0-21swapper/122:15:161
147229955,0cyclictest0-21swapper/121:44:511
147229955,0cyclictest0-21swapper/121:09:131
147229955,0cyclictest0-21swapper/100:07:311
147229954,0cyclictest28986-21sh19:20:251
147229950,0cyclictest0-21swapper/122:43:201
147229950,0cyclictest0-21swapper/122:38:461
147229950,0cyclictest0-21swapper/120:20:491
149950,0migration/09722-21taskset22:01:180
149950,0migration/0927-21irqrtprio20:35:410
149950,0migration/08545-21runrttasks19:00:480
149950,0migration/08020-21kthreadcore20:40:420
149950,0migration/05266-21munin-node19:50:380
149950,0migration/031786-21kthreadcore21:15:420
149950,0migration/029999-21kthreadcore23:35:410
149950,0migration/029660-21kthreadcore22:35:400
149950,0migration/029248-21kthreadcore22:15:420
149950,0migration/025670-21munin-node21:50:470
149950,0migration/024862-21kthreadcore19:15:420
149950,0migration/023992-21kthreadcore22:10:410
149950,0migration/023614-21taskset23:32:280
149950,0migration/022351-21gltestperf18:50:390
149950,0migration/021876-21kthreadcore21:30:410
149950,0migration/018077-21kthreadcore21:05:430
149950,0migration/016635-21ntp_kernel_pll_19:55:430
149950,0migration/015968-21cpu00:10:360
149950,0migration/01347-21munin-node23:20:380
149950,0migration/011155-21kthreadcore19:05:420
149950,0migration/010942-21aten_r1power_vo22:25:360
149950,0migration/010747-21irqrtprio23:05:390
149950,0migration/010237-21kthreadcore20:15:410
8215040,0irq/24-0000:00:0-21swapper/222:30:242
4140,0ktimersoftd/00-21swapper/021:40:270
339940,0migration/329190-21kthreadcore20:05:423
339940,0migration/32919-21kthreadcore19:45:413
339940,0migration/329169-21diskmemload20:55:353
339940,0migration/322455-21kthreadcore20:00:423
339940,0migration/317857-21taskset20:20:473
339940,0migration/31650-21timerwakeupswit18:55:473
339940,0migration/311805-21fschecks_count22:05:373
29725040,0irq/27-eth0-tx-29447-21cpuspeed_turbos20:55:372
29725040,0irq/27-eth0-tx-24848-21ssh23:35:322
29725040,0irq/27-eth0-tx-0-21swapper/222:40:372
29725040,0irq/27-eth0-tx-0-21swapper/222:25:262
29725040,0irq/27-eth0-tx-0-21swapper/222:00:292
29725040,0irq/27-eth0-tx-0-21swapper/220:50:252
29725040,0irq/27-eth0-tx-0-21swapper/020:25:470
29715040,0irq/26-eth0-rx-12678-21fschecks_time21:45:383
29715040,0irq/26-eth0-rx-0-21swapper/323:45:383
29715040,0irq/26-eth0-rx-0-21swapper/323:15:343
29715040,0irq/26-eth0-rx-0-21swapper/323:05:333
29715040,0irq/26-eth0-rx-0-21swapper/322:21:593
29715040,0irq/26-eth0-rx-0-21swapper/321:50:363
29715040,0irq/26-eth0-rx-0-21swapper/321:30:383
29715040,0irq/26-eth0-rx-0-21swapper/321:05:533
29715040,0irq/26-eth0-rx-0-21swapper/320:35:433
29715040,0irq/26-eth0-rx-0-21swapper/320:25:253
259940,0migration/2700-21kthreadcore20:10:392
259940,0migration/230424-21kthreadcore20:30:402
259940,0migration/228824-21taskset21:55:392
259940,0migration/224622-21latency22:10:422
259940,0migration/222844-21kthreadcore22:50:432
259940,0migration/221514-21latency_hist18:50:252
259940,0migration/220187-21kthreadcore20:25:412
259940,0migration/218345-21kthreadcore19:10:412
259940,0migration/217201-21kthreadcore18:45:412
259940,0migration/216233-21kthreadcore19:55:412
249940,0watchdog/24380-21if_err_eth119:25:392
249940,0watchdog/225372-21ssh23:15:302
249940,0watchdog/210249-21ssh21:40:582
249940,0watchdog/20-21swapper/223:40:422
179940,0migration/17140-21kthreadcore22:20:431
179940,0migration/126992-21kthreadcore23:35:401
179940,0migration/126667-21fschecks_time22:55:381
179940,0migration/12579-21kthreadcore23:40:411
179940,0migration/124742-21cpuspeed_turbos20:05:371
179940,0migration/124631-21kthreadcore21:10:421
179940,0migration/123465-21latency_hist19:40:251
179940,0migration/119414-21munin-node22:30:381
179940,0migration/119401-21kthreadcore20:00:411
179940,0migration/114095-21kthreadcore19:55:411
179940,0migration/112665-21taskset20:42:061
169940,0watchdog/19852-21perl19:05:391
169940,0watchdog/17391-21cut21:20:421
169940,0watchdog/129264-21grep22:15:421
169940,0watchdog/12904-21ps21:16:261
169940,0watchdog/12865-21chrt19:21:071
169940,0watchdog/125690-21taskset20:30:351
169940,0watchdog/125499-21kthreadcore21:30:421
169940,0watchdog/122445-21cut23:10:421
169940,0watchdog/121220-21kthreadcore22:50:421
169940,0watchdog/120012-21grep23:50:421
169940,0watchdog/115717-21ssh23:27:341
169940,0watchdog/110146-21sh22:02:341
169940,0watchdog/10-21swapper/123:30:421
169940,0watchdog/10-21swapper/123:20:381
169940,0watchdog/10-21swapper/123:00:421
169940,0watchdog/10-21swapper/121:35:461
147239943,0cyclictest5719-21awk21:00:382
147239943,0cyclictest3556-21lldpd20:45:022
147239943,0cyclictest3556-21lldpd20:36:312
147239943,0cyclictest3556-21lldpd20:24:312
147239943,0cyclictest3556-21lldpd19:34:312
147239943,0cyclictest3556-21lldpd19:21:302
147239943,0cyclictest3556-21lldpd19:01:312
147239943,0cyclictest29201-21awk18:55:402
147239943,0cyclictest27217-21munin-node22:15:382
147239943,0cyclictest12315-21awk22:05:392
147229944,0cyclictest0-21swapper/121:58:151
147229943,0cyclictest655-21awk20:35:401
147229943,0cyclictest5766-21awk21:00:381
147229943,0cyclictest3556-21lldpd23:05:331
147229943,0cyclictest3556-21lldpd20:25:311
147229943,0cyclictest3556-21lldpd20:13:011
147229943,0cyclictest3556-21lldpd19:46:001
147229943,0cyclictest3556-21lldpd19:04:301
147229943,0cyclictest3556-21lldpd18:47:001
147229943,0cyclictest17805-21awk19:35:381
147229943,0cyclictest13415-21awk21:25:391
147229943,0cyclictest12989-21awk21:45:401
147219943,0cyclictest8699-21awk00:05:380
147219943,0cyclictest3514-21snmpd22:41:350
147219943,0cyclictest3241-21irqbalance23:46:420
147219943,0cyclictest3241-21irqbalance22:55:420
147219943,0cyclictest31934-21sh00:00:230
147219943,0cyclictest30664-21kworker/0:221:04:260
147219943,0cyclictest2402-21awk23:00:390
147219943,0cyclictest18715-21awk20:00:390
147219943,0cyclictest17702-21awk19:35:380
147219943,0cyclictest12233-21awk22:05:390
147219943,0cyclictest11100-21awk22:45:380
149940,0migration/05651-21irqcore21:20:410
149940,0migration/031650-21kthreadcore21:55:410
149940,0migration/030141-21sendmail_mailst20:05:460
149940,0migration/028967-21kthreadcore20:30:400
149940,0migration/027768-21runrttasks21:12:380
149940,0migration/025843-21fschecks_count23:15:370
149940,0migration/02487-21rt-features20:55:460
149940,0migration/023434-21kthreadcore22:30:420
149940,0migration/017649-21kthreadcore19:10:410
8215030,0irq/24-0000:00:0-21swapper/219:40:462
29725030,0irq/27-eth0-tx-0-21swapper/019:25:360
29725030,0irq/27-eth0-tx-0-21swapper/019:20:380
29715030,0irq/26-eth0-rx-0-21swapper/320:15:353
29715030,0irq/26-eth0-rx-0-21swapper/319:50:353
29715030,0irq/26-eth0-rx-0-21swapper/319:40:363
29715030,0irq/26-eth0-rx-0-21swapper/319:30:353
29715030,0irq/26-eth0-rx-0-21swapper/319:25:353
29715030,0irq/26-eth0-rx-0-21swapper/319:20:363
29715030,0irq/26-eth0-rx-0-21swapper/319:10:433
249930,0watchdog/20-21swapper/220:15:392
179930,0migration/132200-21runrttasks00:00:331
169930,0watchdog/111615-21munin-node22:25:381
169930,0watchdog/10-21swapper/123:55:261
169930,0watchdog/10-21swapper/123:15:381
169930,0watchdog/10-21swapper/122:05:341
169930,0watchdog/10-21swapper/120:55:431
169930,0watchdog/10-21swapper/120:51:271
147239932,0cyclictest28500-21grep19:15:482
147239932,0cyclictest22499-21sensors19:35:442
147239932,0cyclictest18827-21awk20:00:402
147239930,0cyclictest4505-21munin-node19:50:452
147239930,0cyclictest2586-21cut19:45:412
147229932,0cyclictest4505-21munin-node19:50:371
147229932,0cyclictest4460-21awk19:25:391
147219932,0cyclictest8975-21sed22:20:490
147219932,0cyclictest19642-21kthreadcore23:50:420
147219932,0cyclictest18411-21awk23:10:390
147219932,0cyclictest15016-21tr19:30:410
147219930,0cyclictest5269-21memory00:00:420
147219930,0cyclictest30177-21munin-node19:45:400
147219930,0cyclictest15244-21grep21:25:410
149930,0migration/018062-21sh22:50:260
169920,0watchdog/10-21swapper/119:10:311
147219920,0cyclictest0-21swapper/020:10:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional