You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 14:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Sun Nov 16, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
214231421110,2sleep30-21swapper/320:30:523
2036845992822,5cyclictest2136627-21sh20:30:000
2036845992721,5cyclictest2480862-1atop00:00:010
2036845992721,5cyclictest2480862-1atop00:00:010
2036845992713,13cyclictest1702-21in:imuxsock23:28:280
2036845992614,5cyclictest2318066-21kthreadcore22:25:420
2036845992519,2cyclictest2025-21snmpd22:20:010
2036845992511,6cyclictest0-21swapper/023:00:010
2036848992118,2cyclictest2445685-21awk23:40:011
2036848992114,5cyclictest260-21systemd-journal00:29:551
2036848992114,2cyclictest11systemd21:45:011
203684599219,7cyclictest0-21swapper/022:00:370
203684599214,2cyclictest191rcu_preempt21:20:010
2036845992114,6cyclictest2251214-21kthreadcore21:45:440
2036845992113,7cyclictest0-21swapper/000:20:010
2036845992110,5cyclictest2310642-21kthreadcore22:20:430
2036845992110,4cyclictest191rcu_preempt22:32:270
2036849992017,1cyclictest0-21swapper/220:00:012
2036848992017,2cyclictest2228812-21sh21:35:011
203684599209,1cyclictest191rcu_preempt21:40:490
2036845992017,2cyclictest1700-21systemd-logind22:40:010
2036845992014,5cyclictest2295218-21grep22:10:450
2036845992011,4cyclictest2469309-21sensors23:50:470
2036845992011,4cyclictest2043639-21grep19:15:020
2036845992011,3cyclictest16-21ksoftirqd/021:40:000
2036845992011,3cyclictest16-21ksoftirqd/021:40:000
2036849991917,1cyclictest0-21swapper/221:40:012
2036849991917,1cyclictest0-21swapper/221:40:012
2036849991916,1cyclictest0-21swapper/223:05:002
2036848991918,1cyclictest0-21swapper/100:00:341
203684599199,6cyclictest2445957-21/usr/sbin/munin23:40:010
203684599199,6cyclictest2112421-21kworker/u16:1-events_unbound23:21:230
203684599199,5cyclictest0-21swapper/021:25:430
203684599197,4cyclictest2271462-21sh21:58:330
203684599196,3cyclictest2112421-21kworker/u16:1+rpciod21:53:330
2036845991916,2cyclictest2463267-21apt-key23:50:000
2036845991911,3cyclictest191rcu_preempt23:31:110
203684599190,7cyclictest2490766-21grep00:05:380
302487291181,3phc2sys0-21swapper/322:25:113
302487291180,2phc2sys0-21swapper/323:07:233
2036848991815,2cyclictest2472186-21grep23:55:001
2036848991815,2cyclictest2460682-21vmstat23:45:491
2036848991815,2cyclictest2237642-21ls21:40:001
2036848991815,2cyclictest2237642-21ls21:40:001
2036848991814,3cyclictest2103312-21kthreadcore20:00:411
2036848991812,2cyclictest2337881-21kthreadcore22:35:431
203684599188,6cyclictest0-21swapper/023:15:360
203684599188,3cyclictest191rcu_preempt21:14:490
203684599188,3cyclictest191rcu_preempt00:13:160
203684599187,3cyclictest191rcu_preempt22:50:160
203684599187,1cyclictest191rcu_preempt00:23:540
203684599187,1cyclictest191rcu_preempt00:23:540
203684599184,7cyclictest0-21swapper/021:30:350
203684599184,6cyclictest16-21ksoftirqd/022:05:420
2036845991816,1cyclictest8351-21rtkit-daemon00:25:080
2036845991815,2cyclictest2550102-21apt-key00:40:000
2036845991814,1cyclictest491rcuog/221:20:350
2036845991812,5cyclictest2482661-21irqrtprio00:00:400
302487291171,2phc2sys0-21swapper/300:25:553
302487291170,3phc2sys0-21swapper/321:18:593
302487291170,2phc2sys0-21swapper/321:30:053
302487291170,2phc2sys0-21swapper/321:21:083
2036849991715,1cyclictest0-21swapper/221:00:012
2036849991715,1cyclictest0-21swapper/221:00:012
2036849991714,1cyclictest0-21swapper/221:50:002
2036849991714,1cyclictest0-21swapper/220:20:012
2036848991714,2cyclictest2536227-21kthreadcore00:30:471
2036848991714,2cyclictest2436979-21perf23:35:011
2036848991714,2cyclictest2329204-21kthreadcore22:30:441
2036848991714,2cyclictest2324278-21latency_hist22:30:001
2036848991714,2cyclictest2250876-21kthreadcore21:45:441
2036848991714,2cyclictest2043770-21kthreadcore19:15:021
2036848991714,2cyclictest0-21swapper/122:45:011
2036848991713,3cyclictest2283008-21kthreadcore22:05:411
2036848991712,4cyclictest2033-21lldpd19:35:491
2036848991710,5cyclictest1803-21gdbus00:20:011
203684599178,4cyclictest1706620-21kworker/u16:2+events_unbound19:15:380
203684599178,3cyclictest191rcu_preempt23:00:480
203684599177,1cyclictest191rcu_preempt22:40:530
203684599175,1cyclictest191rcu_preempt23:14:310
2036845991714,2cyclictest2106501-21cut20:05:020
2036845991714,2cyclictest1700-21systemd-logind19:35:010
203684599170,6cyclictest2351057-21aten_repower_fr22:45:350
203684599170,6cyclictest2076566-21expr19:40:380
835298160,2rtkit-daemon0-21swapper/120:53:341
835298160,1rtkit-daemon0-21swapper/120:30:321
302487291162,2phc2sys0-21swapper/320:50:423
302487291161,4phc2sys0-21swapper/319:35:363
302487291160,2phc2sys0-21swapper/323:58:073
302487291160,2phc2sys0-21swapper/323:58:073
302487291160,2phc2sys0-21swapper/323:50:413
302487291160,2phc2sys0-21swapper/322:39:583
302487291160,2phc2sys0-21swapper/300:36:053
302487291160,1phc2sys0-21swapper/320:14:553
2036849991613,1cyclictest0-21swapper/222:38:432
2036849991613,1cyclictest0-21swapper/200:20:012
2036849991612,2cyclictest0-21swapper/223:54:382
2036849991612,2cyclictest0-21swapper/222:45:422
2036849991612,2cyclictest0-21swapper/219:25:452
2036848991614,1cyclictest2263713-21ls21:55:001
2036848991613,2cyclictest8351-21rtkit-daemon00:22:511
2036848991613,2cyclictest8351-21rtkit-daemon00:22:511
2036848991613,2cyclictest2500237-21kthreadcore00:10:451
2036848991613,2cyclictest2411131-21sh23:20:011
2036848991613,2cyclictest2124687-21latency_hist20:20:011
2036848991613,2cyclictest2036841-21cyclictest20:00:011
2036848991613,2cyclictest1635-21dbus-daemon22:20:001
2036848991612,3cyclictest2253982-21kworker/u16:2+nfsiod21:56:121
203684599167,7cyclictest16-21ksoftirqd/000:30:390
203684599167,5cyclictest0-21swapper/023:05:450
203684599165,5cyclictest0-21swapper/019:25:150
203684599163,2cyclictest191rcu_preempt19:55:490
2036845991613,2cyclictest2446866-21cpuspeed_turbos23:40:360
2036845991613,2cyclictest241rcuc/020:40:010
2036845991613,2cyclictest2167172-21cut20:55:010
2036845991612,3cyclictest1652-1atopacctd20:20:420
835298150,1rtkit-daemon0-21swapper/122:10:061
302487291158,2phc2sys0-21swapper/322:50:123
302487291150,2phc2sys0-21swapper/323:11:063
302487291150,2phc2sys0-21swapper/321:29:093
2036849991512,1cyclictest0-21swapper/223:20:002
2036849991512,1cyclictest0-21swapper/222:40:422
2036849991512,1cyclictest0-21swapper/221:55:012
2036849991512,1cyclictest0-21swapper/221:40:102
2036849991512,1cyclictest0-21swapper/221:05:432
2036849991512,1cyclictest0-21swapper/220:55:002
2036849991512,1cyclictest0-21swapper/220:35:002
2036849991512,1cyclictest0-21swapper/219:35:022
2036849991512,1cyclictest0-21swapper/200:36:362
2036849991511,2cyclictest0-21swapper/222:03:032
203684999150,8cyclictest0-21swapper/221:30:352
203684899158,2cyclictest0-21swapper/123:45:011
203684899157,3cyclictest0-21swapper/121:20:341
203684899156,1cyclictest11systemd19:30:011
2036848991512,2cyclictest2543183-21sed00:35:421
2036848991512,2cyclictest2394274-21acpi23:10:341
2036848991512,2cyclictest2186035-21head21:10:341
2036848991512,2cyclictest2173238-21latency_hist21:00:011
2036848991512,2cyclictest2173238-21latency_hist21:00:011
2036848991512,2cyclictest2082121-21latency_hist19:45:011
2036848991512,2cyclictest2036841-21cyclictest19:20:411
2036848991511,3cyclictest2196189-21kthreadcore21:15:451
2036848991511,3cyclictest2116001-21kthreadcore20:10:441
203684599159,3cyclictest2137128-21aten2.4-expect20:30:400
203684599153,5cyclictest2088388-21acpi19:50:340
2036845991513,1cyclictest2036841-21cyclictest21:00:410
2036845991512,2cyclictest2161040-21latency_hist20:50:010
2036845991512,2cyclictest2112421-21kworker/u16:1-events_unbound20:45:020
2036845991512,2cyclictest0-21swapper/020:20:010
2036845991510,1cyclictest191rcu_preempt19:20:420
302487291143,2phc2sys0-21swapper/321:40:113
302487291141,6phc2sys0-21swapper/300:00:343
302487291140,3phc2sys0-21swapper/319:25:383
302487291140,2phc2sys0-21swapper/322:43:533
302487291140,2phc2sys0-21swapper/321:10:283
302487291140,2phc2sys0-21swapper/300:20:353
302487291140,2phc2sys0-21swapper/300:20:353
302487291140,2phc2sys0-21swapper/300:11:053
2036849991412,1cyclictest0-21swapper/223:35:022
2036849991412,1cyclictest0-21swapper/222:00:012
2036849991412,1cyclictest0-21swapper/221:30:012
2036849991412,1cyclictest0-21swapper/219:40:012
2036849991412,1cyclictest0-21swapper/219:15:022
2036849991411,1cyclictest0-21swapper/223:49:342
2036849991411,1cyclictest0-21swapper/223:40:422
2036849991411,1cyclictest0-21swapper/223:25:442
2036849991411,1cyclictest0-21swapper/223:20:392
2036849991411,1cyclictest0-21swapper/223:10:372
2036849991411,1cyclictest0-21swapper/222:11:492
2036849991411,1cyclictest0-21swapper/222:05:482
2036849991411,1cyclictest0-21swapper/221:10:402
2036849991411,1cyclictest0-21swapper/220:45:022
2036849991411,1cyclictest0-21swapper/219:45:402
2036849991411,1cyclictest0-21swapper/200:30:432
2036849991411,1cyclictest0-21swapper/200:05:372
2036849991410,2cyclictest0-21swapper/222:20:442
203684899149,1cyclictest1106-21hwrng22:23:221
2036848991412,1cyclictest8351-21rtkit-daemon20:20:511
2036848991411,2cyclictest260-21systemd-journal00:00:011
2036848991411,2cyclictest260-21systemd-journal00:00:011
2036848991411,2cyclictest2378262-21kthreadcore23:00:431
2036848991411,2cyclictest2368915-21sh22:55:401
2036848991411,2cyclictest2351642-21cut22:45:391
2036848991411,2cyclictest2276300-21kthreadcore22:00:451
2036848991411,2cyclictest2213396-21kthreadcore21:25:421
2036848991411,2cyclictest2178350-21processes21:00:451
2036848991411,2cyclictest2107331-21gltestperf20:05:391
2036848991410,3cyclictest2363190-21kthreadcore22:50:441
2036848991410,3cyclictest2144415-21kthreadcore20:35:431
2036845991411,2cyclictest0-21swapper/021:10:010
2036845991410,3cyclictest2083632-21kthreadcore19:45:430
835298130,4rtkit-daemon2030979-21/usr/sbin/munin19:05:500
302487291133,3phc2sys0-21swapper/322:10:413
302487291131,3phc2sys0-21swapper/300:30:363
302487291130,4phc2sys0-21swapper/323:17:103
302487291130,2phc2sys0-21swapper/323:46:563
302487291130,2phc2sys0-21swapper/323:24:113
302487291130,2phc2sys0-21swapper/322:33:493
302487291130,2phc2sys0-21swapper/321:55:103
24901052131,7sleep30-21swapper/300:05:353
2036849991311,1cyclictest0-21swapper/223:35:502
2036849991311,1cyclictest0-21swapper/222:15:352
2036849991311,1cyclictest0-21swapper/221:00:372
2036849991311,1cyclictest0-21swapper/220:00:392
2036849991311,1cyclictest0-21swapper/219:20:422
2036849991310,1cyclictest0-21swapper/223:05:342
2036849991310,1cyclictest0-21swapper/222:55:352
2036849991310,1cyclictest0-21swapper/222:53:172
2036849991310,1cyclictest0-21swapper/222:25:432
2036849991310,1cyclictest0-21swapper/220:45:452
2036849991310,1cyclictest0-21swapper/220:05:432
2036849991310,1cyclictest0-21swapper/219:50:422
2036849991310,1cyclictest0-21swapper/219:40:402
203684899139,3cyclictest2086031-21kthreadcore19:45:441
203684899138,4cyclictest2036841-21cyclictest20:25:391
203684899137,5cyclictest2421721-21cut23:25:441
2036848991311,1cyclictest2036841-21cyclictest21:09:341
2036848991310,2cyclictest2416666-21proc_pri23:20:461
2036848991310,2cyclictest2386944-21kthreadcore23:05:391
2036848991310,2cyclictest2150431-21kernelversion20:40:421
2036848991310,2cyclictest2093472-21threads19:50:461
2036848991310,2cyclictest2068828-21phc2sys19:30:441
2036845991311,1cyclictest2168994-21kthreadcore20:55:450
2036845991311,1cyclictest2168994-21kthreadcore20:55:450
2036845991311,1cyclictest2106635-21aten_repower_cu20:05:350
2036845991310,2cyclictest2114755-21kthreadcore20:10:430
2036845991310,2cyclictest2074420-21kthreadcore19:35:440
302487291121,6phc2sys0-21swapper/321:50:353
302487291120,3phc2sys0-21swapper/321:45:363
302487291120,2phc2sys0-21swapper/323:33:583
302487291120,2phc2sys0-21swapper/319:40:293
302487291120,1phc2sys0-21swapper/323:02:373
203684999129,1cyclictest0-21swapper/221:20:422
203684999129,1cyclictest0-21swapper/220:10:452
203684999129,1cyclictest0-21swapper/219:10:012
203684999129,1cyclictest0-21swapper/200:25:132
203684999129,1cyclictest0-21swapper/200:20:422
203684999129,1cyclictest0-21swapper/200:20:422
2036849991210,1cyclictest0-21swapper/223:55:412
2036849991210,1cyclictest0-21swapper/223:55:412
2036849991210,1cyclictest0-21swapper/222:30:432
2036849991210,1cyclictest0-21swapper/221:15:452
2036849991210,1cyclictest0-21swapper/219:15:032
2036849991210,1cyclictest0-21swapper/200:00:402
203684899129,2cyclictest2156299-21irqcore20:45:421
203684899129,2cyclictest2043584-21kthreadcore19:15:031
203684899123,1cyclictest170550irq/121-eno100:05:431
302487291112,2phc2sys0-21swapper/322:18:233
302487291111,4phc2sys0-21swapper/319:30:343
302487291111,2phc2sys0-21swapper/320:24:083
302487291110,3phc2sys0-21swapper/322:55:303
302487291110,2phc2sys0-21swapper/320:45:363
302487291110,1phc2sys0-21swapper/320:05:053
203684999118,1cyclictest0-21swapper/220:25:422
203684999118,1cyclictest0-21swapper/220:20:432
203684999115,3cyclictest0-21swapper/200:10:382
203684999110,8cyclictest0-21swapper/220:35:362
302487291101,3phc2sys0-21swapper/300:15:453
302487291101,2phc2sys0-21swapper/323:29:363
302487291101,2phc2sys0-21swapper/322:45:523
302487291101,2phc2sys0-21swapper/322:02:193
302487291101,2phc2sys0-21swapper/321:10:023
302487291101,2phc2sys0-21swapper/321:00:363
302487291100,4phc2sys0-21swapper/319:45:363
302487291100,3phc2sys0-21swapper/323:41:513
302487291100,3phc2sys0-21swapper/323:38:193
302487291100,3phc2sys0-21swapper/322:20:393
302487291100,3phc2sys0-21swapper/320:40:353
302487291100,3phc2sys0-21swapper/319:24:563
302487291100,2phc2sys0-21swapper/322:05:483
302487291100,2phc2sys0-21swapper/321:39:233
302487291100,2phc2sys0-21swapper/321:39:233
302487291100,2phc2sys0-21swapper/319:50:183
302487291100,2phc2sys0-21swapper/319:15:233
203684899104,5cyclictest1635-21dbus-daemon19:10:011
30248729191,2phc2sys0-21swapper/320:35:023
30248729191,2phc2sys0-21swapper/319:05:383
30248729190,2phc2sys0-21swapper/320:55:473
30248729190,2phc2sys0-21swapper/320:55:473
30248729190,2phc2sys0-21swapper/320:00:323
30248729190,2phc2sys0-21swapper/319:55:393
30248729190,2phc2sys0-21swapper/319:10:393
30248729180,2phc2sys0-21swapper/320:25:093
30248729180,2phc2sys0-21swapper/320:15:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional