You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 19:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sun Nov 23, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
180601426151,7sleep60-21swapper/615:20:556
180599525948,8sleep50-21swapper/515:20:385
1806539995654,2cyclictest2188305-21kworker/u16:1+events_unbound18:33:275
180590025622,29sleep70-21swapper/715:19:207
1806542995553,2cyclictest1830363-21kworker/u16:2+flush-8:015:33:276
1806544995348,4cyclictest2386776-21latency_hist20:08:267
180597525242,7sleep40-21swapper/415:20:214
1806542995147,3cyclictest2021704-21kworker/u16:1+flush-8:017:08:276
1806544994944,4cyclictest2316163-21latency_hist19:33:277
1806544994843,4cyclictest2123829-21latency_hist17:58:277
1806544994843,4cyclictest2123829-21latency_hist17:58:277
180654499480,46cyclictest1850831-21latency_hist15:43:277
180654499480,46cyclictest1850831-21latency_hist15:43:277
1806539994845,2cyclictest1860859-21kworker/u16:0+flush-8:016:03:285
1806539994644,2cyclictest2188305-21kworker/u16:1+flush-8:019:13:265
1806544994439,4cyclictest2052970-21latency_hist17:23:277
1806539994442,2cyclictest1880128-21kworker/u16:4+events_unbound16:28:275
1806539994442,2cyclictest1880128-21kworker/u16:4+events_unbound16:28:275
180654499420,40cyclictest2336436-21cat19:43:277
1806542994139,2cyclictest2315960-21kworker/u16:4+flush-8:019:38:276
1806542993836,2cyclictest2265324-21kworker/u16:3+flush-8:020:18:276
1806539993632,3cyclictest2184403-21kworker/u16:0+events_unbound19:13:435
1806539993533,2cyclictest1951728-21kworker/u16:2+events_unbound16:53:415
1806544993432,1cyclictest0-21swapper/719:13:397
180654499340,32cyclictest2103585-21latency_hist17:48:277
1806542993431,2cyclictest2265324-21kworker/u16:3+events_unbound19:58:406
1806542993430,3cyclictest2188305-21kworker/u16:1+flush-8:019:13:406
180654499330,31cyclictest1921598-21latency_hist16:18:277
1806542993331,2cyclictest1860859-21kworker/u16:0+events_unbound16:03:446
1806542993330,2cyclictest1871416-21kworker/u16:1+flush-8:015:53:416
1806539993230,2cyclictest2021704-21kworker/u16:1+events_unbound18:03:495
1806539993229,2cyclictest2188305-21kworker/u16:1+flush-8:019:23:405
1806539993229,2cyclictest2188305-21kworker/u16:1+flush-8:019:23:405
1806544993130,0cyclictest0-21swapper/719:48:487
1806544993126,4cyclictest2235282-21latency_hist18:53:277
1806544993126,4cyclictest1840729-21latency_hist15:38:277
1806539993126,4cyclictest2205862-21diskstats18:38:415
1806539993126,4cyclictest2205862-21diskstats18:38:415
180654499300,28cyclictest589235-21systemd-journal20:13:197
180654499300,28cyclictest589235-21systemd-journal20:13:197
1806542993027,2cyclictest2315960-21kworker/u16:4+events_unbound20:28:406
1806544992928,0cyclictest0-21swapper/716:34:017
1806539992926,3cyclictest2315960-21kworker/u16:4+flush-8:020:23:275
1806544992827,0cyclictest0-21swapper/716:58:437
1806534992826,2cyclictest2134327-21kworker/u16:5+flush-8:018:08:414
1806539992724,2cyclictest2188305-21kworker/u16:1+events_unbound18:48:545
1806539992724,2cyclictest1880128-21kworker/u16:4+flush-8:017:03:425
1806539992724,2cyclictest1880128-21kworker/u16:4+flush-8:016:53:235
1806534992724,2cyclictest2315960-21kworker/u16:4+events_unbound20:08:404
1806534992724,2cyclictest2315960-21kworker/u16:4+events_unbound20:08:404
1806534992723,3cyclictest2315960-21kworker/u16:4+flush-8:020:33:534
1806544992620,5cyclictest589235-21systemd-journal18:32:377
180654499260,25cyclictest2184543-21latency_hist18:28:277
1806542992624,2cyclictest1901693-21kworker/u16:2+flush-8:016:13:426
1806542992623,2cyclictest2427156-21kworker/u16:1+events_unbound20:48:426
1806539992624,2cyclictest2265324-21kworker/u16:3+events_unbound19:33:435
1806539992623,2cyclictest2154239-21kworker/u16:3+flush-8:018:18:425
1806544992523,1cyclictest0-21swapper/715:28:477
1806542992523,2cyclictest1961824-21kworker/u16:1+flush-8:016:36:446
1806539992523,2cyclictest2042605-21kworker/u16:2+flush-8:017:33:565
1806539992522,2cyclictest2306087-21kworker/u16:2+flush-8:019:45:175
1806539992521,3cyclictest2184403-21kworker/u16:0+flush-8:019:21:095
1806539992520,4cyclictest2246293-21diskstats18:58:395
1806544992421,1cyclictest0-21swapper/720:33:477
1806539992422,2cyclictest2306087-21kworker/u16:2+flush-8:019:58:495
1806539992422,2cyclictest1830363-21kworker/u16:2+events_unbound15:38:395
1806539992422,2cyclictest1830363-21kworker/u16:2+events_unbound15:38:395
1806544992321,1cyclictest0-21swapper/718:08:397
1806544992318,4cyclictest1871030-21latency_hist15:53:287
180654499230,21cyclictest2063081-21latency_hist17:28:277
1806539992321,2cyclictest2427156-21kworker/u16:1+events_unbound20:43:435
1806539992321,2cyclictest2154575-21kworker/u16:4+events_unbound18:33:595
1806539992321,2cyclictest2064709-21kworker/u16:3+events_unbound17:53:575
1806539992321,2cyclictest2064709-21kworker/u16:3+events_unbound17:53:575
1806539992320,3cyclictest2188305-21kworker/u16:1+flush-8:019:28:565
1806539992320,2cyclictest2031826-21kworker/u16:5+flush-8:017:43:405
1806542992220,2cyclictest2154239-21kworker/u16:3+flush-8:018:33:426
1806542992219,2cyclictest2265324-21kworker/u16:3+flush-8:019:18:446
1806539992220,2cyclictest346995-21kworker/u16:7+flush-8:015:28:495
1806539992219,3cyclictest2416949-21kworker/u16:0+flush-8:020:32:165
1806539992219,3cyclictest2306087-21kworker/u16:2+flush-8:019:48:525
1806544992120,0cyclictest0-21swapper/718:53:427
1806544992119,1cyclictest0-21swapper/719:03:467
1806544992115,5cyclictest2093476-21latency_hist17:43:277
1806542992119,2cyclictest2315960-21kworker/u16:4+events_unbound20:33:546
1806539992119,2cyclictest2427156-21kworker/u16:1+events_unbound20:38:465
1806539992119,2cyclictest2315960-21kworker/u16:4+flush-8:019:53:575
1806539992119,2cyclictest2134327-21kworker/u16:5+flush-8:018:08:405
1806539992119,2cyclictest2042605-21kworker/u16:2+flush-8:017:31:185
1806539992119,2cyclictest1931433-21kworker/u16:0+flush-8:016:43:535
1806539992118,2cyclictest2315960-21kworker/u16:4+events_unbound20:03:455
1806534992119,2cyclictest2188305-21kworker/u16:1+events_unbound18:48:564
1806544992019,0cyclictest0-21swapper/718:43:577
1806544992018,1cyclictest0-21swapper/715:53:487
180654499200,18cyclictest589235-21systemd-journal19:26:397
180654499200,18cyclictest589235-21systemd-journal19:26:397
1806542992017,2cyclictest2427156-21kworker/u16:1+flush-8:020:28:276
1806542992017,2cyclictest2154239-21kworker/u16:3+flush-8:018:18:186
1806542992017,2cyclictest1931433-21kworker/u16:0+events_unbound16:28:506
1806542992016,3cyclictest2188305-21kworker/u16:1+flush-8:018:53:446
1806539992016,3cyclictest2194211-21kworker/u16:2+flush-8:019:03:555
1806539992015,4cyclictest1962975-21tune2fs16:38:425
1806544991918,0cyclictest0-21swapper/720:43:487
1806544991918,0cyclictest0-21swapper/716:18:417
1806542991917,2cyclictest2265324-21kworker/u16:3+events_unbound20:03:516
1806542991917,2cyclictest2265324-21kworker/u16:3+events_unbound19:08:566
1806542991917,2cyclictest2042605-21kworker/u16:2+flush-8:017:18:436
1806542991916,2cyclictest1982205-21kworker/u16:3+flush-8:017:08:516
1806542991916,2cyclictest1982205-21kworker/u16:3+flush-8:017:08:516
1806539991917,2cyclictest2427156-21kworker/u16:1+events_unbound20:38:225
1806539991917,2cyclictest1880128-21kworker/u16:4+flush-8:016:58:495
1806539991917,2cyclictest1850699-21kworker/u16:3+flush-8:015:48:495
1806534991915,3cyclictest2265324-21kworker/u16:3+flush-8:019:23:444
1806534991915,3cyclictest2265324-21kworker/u16:3+flush-8:019:23:444
1806542991816,2cyclictest2457882-21kworker/u16:0+events_unbound20:43:566
1806542991816,2cyclictest1739885-21kworker/u16:1+flush-8:015:23:416
1806539991815,3cyclictest1880128-21kworker/u16:4+flush-8:016:18:275
1806539991815,2cyclictest2064709-21kworker/u16:3+flush-8:017:53:265
20272332170,0sleep50-21swapper/517:08:465
20272332170,0sleep50-21swapper/517:08:465
1806544991715,1cyclictest0-21swapper/720:38:497
1806544991714,2cyclictest4175659-21snmpd19:12:087
1806544991712,4cyclictest1911418-21latency_hist16:13:277
1806542991715,2cyclictest2184403-21kworker/u16:0+events_unbound19:03:416
1806542991715,2cyclictest1931433-21kworker/u16:0+flush-8:016:43:506
1806539991715,2cyclictest2153904-21kworker/u16:2+flush-8:018:13:425
1806539991715,2cyclictest1850699-21kworker/u16:3+flush-8:016:08:435
1806544991615,0cyclictest0-21swapper/717:08:447
1806544991615,0cyclictest0-21swapper/717:08:447
180654499160,14cyclictest0-21swapper/719:33:517
1806542991614,2cyclictest346995-21kworker/u16:7+events_unbound15:33:556
1806542991614,2cyclictest2315960-21kworker/u16:4+events_unbound19:53:516
1806542991614,2cyclictest2315960-21kworker/u16:4+events_unbound19:43:416
1806542991614,2cyclictest2021704-21kworker/u16:1+events_unbound18:08:526
1806542991614,2cyclictest1850699-21kworker/u16:3+flush-8:015:48:536
1806542991613,2cyclictest2021704-21kworker/u16:1+flush-8:017:44:006
1806539991614,2cyclictest2416949-21kworker/u16:0+flush-8:020:23:555
1806539991614,2cyclictest1901693-21kworker/u16:2+events_unbound16:18:485
1806539991614,2cyclictest1880128-21kworker/u16:4+flush-8:016:33:425
1806539991613,2cyclictest2306087-21kworker/u16:2+events_unbound20:08:385
1806539991613,2cyclictest2306087-21kworker/u16:2+events_unbound20:08:385
1806539991612,3cyclictest2427156-21kworker/u16:1+flush-8:020:48:445
180654499157,7cyclictest0-21swapper/717:52:567
1806544991515,0cyclictest0-21swapper/718:38:447
1806544991515,0cyclictest0-21swapper/718:38:447
1806544991514,0cyclictest0-21swapper/715:43:437
1806544991510,4cyclictest2376710-21latency_hist20:03:267
1806542991512,3cyclictest2184403-21kworker/u16:0+flush-8:018:30:076
1806542991511,3cyclictest2306087-21kworker/u16:2+flush-8:019:48:566
1806539991511,3cyclictest1830363-21kworker/u16:2+flush-8:015:43:585
180653499150,2cyclictest2448509-21perl20:38:394
1806544991414,0cyclictest0-21swapper/717:13:567
1806544991412,1cyclictest0-21swapper/720:13:507
180654499140,1cyclictest0-21swapper/716:28:527
180654499140,1cyclictest0-21swapper/716:03:537
180654499140,13cyclictest589235-21systemd-journal16:42:547
180654499140,13cyclictest0-21swapper/719:43:517
1806542991412,2cyclictest2031826-21kworker/u16:5+events_unbound17:28:506
1806542991412,2cyclictest2021704-21kworker/u16:1+events_unbound17:13:576
1806542991412,1cyclictest0-21swapper/617:23:486
1806534991410,3cyclictest2306087-21kworker/u16:2+events_unbound19:53:414
180654499139,3cyclictest2417441-21/usr/sbin/munin20:23:577
180654499137,5cyclictest2133940-21latency_hist18:03:277
1806544991310,2cyclictest0-21swapper/716:53:287
180654499130,12cyclictest2067426-21kthreadcore17:28:457
180654499130,11cyclictest2255509-21munin-run19:03:277
180654299139,3cyclictest2031826-21kworker/u16:5+flush-8:017:33:596
1806542991310,2cyclictest2154239-21kworker/u16:3+flush-8:018:40:136
1806542991310,2cyclictest2154239-21kworker/u16:3+flush-8:018:40:136
1806539991311,2cyclictest2154239-21kworker/u16:3+flush-8:018:43:475
1806534991312,1cyclictest0-21swapper/416:48:484
1806534991311,2cyclictest1871416-21kworker/u16:1+events_unbound16:03:534
1806534991310,2cyclictest2188305-21kworker/u16:1+flush-8:019:28:524
180653499130,1cyclictest0-21swapper/415:39:544
180653499130,1cyclictest0-21swapper/415:39:544
180654499128,3cyclictest96-21ksoftirqd/717:33:407
180654499128,3cyclictest4175659-21snmpd18:04:157
180654499126,5cyclictest589235-21systemd-journal19:23:127
1806544991212,0cyclictest0-21swapper/717:03:517
1806544991211,0cyclictest0-21swapper/715:23:457
1806544991210,1cyclictest0-21swapper/720:18:457
1806544991210,1cyclictest0-21swapper/716:23:467
1806544991210,1cyclictest0-21swapper/716:23:467
1806542991210,2cyclictest2194211-21kworker/u16:2+events_unbound18:48:426
1806542991210,2cyclictest2134327-21kworker/u16:5+events_unbound18:18:406
1806542991210,2cyclictest1830363-21kworker/u16:2+flush-8:015:38:426
1806542991210,2cyclictest1830363-21kworker/u16:2+flush-8:015:38:426
1806539991211,1cyclictest0-21swapper/518:55:425
1806539991211,1cyclictest0-21swapper/518:01:005
1806539991211,1cyclictest0-21swapper/517:39:345
1806539991210,2cyclictest1931433-21kworker/u16:0+events_unbound16:28:415
180653999120,12cyclictest0-21swapper/518:24:015
1806534991211,1cyclictest0-21swapper/415:23:314
1806534991210,2cyclictest2124754-21kworker/u16:4+events_unbound17:58:564
1806534991210,2cyclictest1739885-21kworker/u16:1+flush-8:015:28:304
180653499120,12cyclictest0-21swapper/417:04:434
180654499118,2cyclictest4175659-21snmpd18:34:167
180654499117,3cyclictest4175659-21snmpd18:16:287
180654499110,10cyclictest0-21swapper/719:53:487
180654299119,2cyclictest2154239-21kworker/u16:3+events_unbound18:43:536
180654299119,2cyclictest1871416-21kworker/u16:1+events_unbound15:58:506
1806542991110,1cyclictest0-21swapper/620:42:266
1806542991110,1cyclictest0-21swapper/620:18:286
1806542991110,1cyclictest0-21swapper/618:59:396
1806542991110,1cyclictest0-21swapper/618:04:006
1806542991110,1cyclictest0-21swapper/616:52:356
180654299110,11cyclictest0-21swapper/620:11:416
180654299110,11cyclictest0-21swapper/620:11:416
180654299110,11cyclictest0-21swapper/619:41:066
180654299110,11cyclictest0-21swapper/619:30:336
180654299110,11cyclictest0-21swapper/619:26:466
180654299110,11cyclictest0-21swapper/619:26:466
180654299110,11cyclictest0-21swapper/616:23:166
180654299110,0cyclictest0-21swapper/617:54:516
180654299110,0cyclictest0-21swapper/617:54:516
180653999119,2cyclictest1871416-21kworker/u16:1+flush-8:016:03:515
180653499118,2cyclictest2338206-21kthreadcore19:43:444
180653499117,3cyclictest2134327-21kworker/u16:5+flush-8:018:23:504
1806534991110,1cyclictest0-21swapper/420:05:424
1806534991110,1cyclictest0-21swapper/419:13:504
1806534991110,1cyclictest0-21swapper/419:05:384
1806534991110,1cyclictest0-21swapper/418:55:044
1806534991110,1cyclictest0-21swapper/418:14:334
1806534991110,1cyclictest0-21swapper/417:51:424
1806534991110,1cyclictest0-21swapper/417:21:264
1806534991110,1cyclictest0-21swapper/416:55:354
1806534991110,1cyclictest0-21swapper/416:09:424
180653499110,11cyclictest0-21swapper/420:29:404
180653499110,11cyclictest0-21swapper/419:01:514
180653499110,11cyclictest0-21swapper/418:22:074
180653499110,11cyclictest0-21swapper/415:59:564
180653499110,0cyclictest0-21swapper/417:25:134
180653499110,0cyclictest0-21swapper/416:18:414
180654499107,2cyclictest948-21NetworkManager16:52:117
180654499103,6cyclictest0-21swapper/720:31:247
180654499100,9cyclictest4175659-21snmpd20:49:537
180654499100,8cyclictest4175659-21snmpd18:18:407
180654499100,10cyclictest96-21ksoftirqd/715:58:507
180654299109,1cyclictest0-21swapper/618:25:176
180654299109,1cyclictest0-21swapper/617:38:566
180654299108,2cyclictest1880128-21kworker/u16:4+flush-8:016:23:536
180654299108,2cyclictest1880128-21kworker/u16:4+flush-8:016:23:536
180654299108,2cyclictest1830363-21kworker/u16:2+events_unbound15:43:446
180654299106,3cyclictest2003594-21cat16:58:446
180654299100,9cyclictest0-21swapper/616:53:436
180654299100,0cyclictest0-21swapper/617:59:256
180653999109,1cyclictest0-21swapper/519:39:515
180653999109,1cyclictest0-21swapper/517:20:295
180653999109,1cyclictest0-21swapper/517:15:075
180653999109,1cyclictest0-21swapper/515:36:365
180653999108,2cyclictest2315960-21kworker/u16:4+flush-8:020:13:575
180653499109,1cyclictest0-21swapper/417:13:524
180653499109,1cyclictest0-21swapper/417:09:184
180653499109,1cyclictest0-21swapper/417:09:184
180653499109,1cyclictest0-21swapper/416:26:154
180653499109,1cyclictest0-21swapper/416:26:154
180653499108,2cyclictest1931433-21kworker/u16:0+events_unbound16:43:554
180653499107,3cyclictest2042605-21kworker/u16:2+flush-8:017:38:574
180653499100,10cyclictest0-21swapper/416:17:544
18065449996,2cyclictest0-21swapper/716:43:287
18065399998,1cyclictest0-21swapper/515:57:345
18065399996,2cyclictest1815281-21cut15:23:505
18065399994,4cyclictest2054149-21cat17:23:425
18065349998,1cyclictest0-21swapper/420:52:504
18065349998,1cyclictest0-21swapper/420:14:414
18065349998,1cyclictest0-21swapper/419:50:534
18065349998,1cyclictest0-21swapper/419:09:254
18065349998,1cyclictest0-21swapper/416:30:024
18065349997,1cyclictest0-21swapper/416:38:444
18065349996,2cyclictest2414951-21kthreadcore20:18:484
18065349995,3cyclictest2094669-21cat17:43:434
18065429987,1cyclictest0-21swapper/617:50:176
18065349987,1cyclictest0-21swapper/418:35:134
18065349987,1cyclictest0-21swapper/415:55:314
18065349986,1cyclictest2417657-21cpuspeed_turbos20:23:414
18065349986,1cyclictest0-21swapper/418:43:444
18065349985,2cyclictest0-21swapper/417:53:464
18065349985,2cyclictest0-21swapper/417:53:464
18065349984,3cyclictest2320670-21kthreadcore19:33:454
18065349984,3cyclictest1831125-21tr15:33:384
18065349983,4cyclictest2371524-21cut19:58:464
18065349983,4cyclictest2327245-21cut19:38:404
18065349983,4cyclictest2286294-21cpu19:18:394
18065349983,4cyclictest2211854-21cut18:38:484
18065349983,4cyclictest2211854-21cut18:38:484
18065349983,4cyclictest2190105-21tr18:28:504
18065349983,4cyclictest2139168-21cut18:03:514
18065349983,4cyclictest1952813-21cat16:33:414
18065429975,1cyclictest0-21swapper/616:38:446
18065429972,4cyclictest1901971-21expr16:08:426
18065349975,1cyclictest0-21swapper/416:58:464
18065349975,1cyclictest0-21swapper/415:48:444
18065349974,2cyclictest2074159-21fschecks_count17:33:404
18065349973,3cyclictest2458162-21cpuspeed_turbos20:43:414
18065349972,4cyclictest2063744-21expr17:28:394
18065349972,4cyclictest1851681-21expr15:43:404
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional