You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-02 - 17:21
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sun Nov 02, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
222834425949,7sleep60-21swapper/615:21:236
222858825746,7sleep40-21swapper/415:24:424
2228901995349,3cyclictest2218505-21kworker/u16:3+flush-8:016:14:515
2228901995249,2cyclictest2394308-21kworker/u16:3+flush-8:016:44:515
2228906994742,4cyclictest2677113-21latency_hist19:04:507
2228901994745,2cyclictest2354396-21kworker/u16:5+events_unbound17:20:065
2228901994543,2cyclictest2808198-21kworker/u16:3+events_unbound20:18:545
2228901994542,2cyclictest2565952-21kworker/u16:3+flush-8:018:14:545
2228899994543,2cyclictest2354396-21kworker/u16:5+flush-8:016:30:214
2228899994441,2cyclictest2424371-21kworker/u16:0+flush-8:017:10:174
2228901994341,2cyclictest2444900-21kworker/u16:4+flush-8:017:30:205
2228906994140,0cyclictest0-21swapper/718:40:067
222890699410,39cyclictest2858914-21latency_hist20:34:507
2228901994138,2cyclictest2444900-21kworker/u16:4+flush-8:018:05:145
2228899994138,3cyclictest2343770-21kworker/u16:1+flush-8:016:40:194
2228899994038,2cyclictest2525376-21kworker/u16:1+events_unbound19:06:334
22274792403,31sleep70-21swapper/715:20:207
2765589390,32rtkit-daemon0-21swapper/515:20:225
2228901993937,2cyclictest2787409-21kworker/u16:5+events_unbound20:27:315
2228901993935,3cyclictest2252811-21kworker/u16:4+flush-8:015:31:435
2228899993836,2cyclictest2218505-21kworker/u16:3+flush-8:016:13:324
2228906993732,4cyclictest2788174-21latency_hist19:59:517
2228901993735,2cyclictest2444900-21kworker/u16:4+events_unbound17:53:225
2228901993735,2cyclictest2273336-21kworker/u16:1+events_unbound16:04:315
2228899993735,2cyclictest2787409-21kworker/u16:5+flush-8:020:04:314
2228899993735,2cyclictest2323472-21kworker/u16:0+events_unbound16:22:334
2228901993634,2cyclictest2394308-21kworker/u16:3+events_unbound17:00:265
2228899993634,2cyclictest2475178-21kworker/u16:1+events_unbound17:28:314
2228901993533,2cyclictest2738890-21kworker/u16:0+events_unbound20:05:155
2228901993532,2cyclictest2343770-21kworker/u16:1+events_unbound16:30:185
2228899993533,2cyclictest2738890-21kworker/u16:0+events_unbound19:48:324
2228899993533,2cyclictest2384159-21kworker/u16:2+events_unbound17:18:314
2228899993532,2cyclictest2424371-21kworker/u16:0+events_unbound17:20:184
2228901993432,2cyclictest2616376-21kworker/u16:3+events_unbound18:32:315
2228901993431,2cyclictest2384159-21kworker/u16:2+events_unbound17:39:315
2228899993432,2cyclictest2702124-21kworker/u16:2+events_unbound19:58:594
2228899993432,2cyclictest2323472-21kworker/u16:0+flush-8:016:16:314
2228901993331,2cyclictest2768388-21kworker/u16:1+flush-8:019:53:315
2228899993331,2cyclictest2808198-21kworker/u16:3+flush-8:020:15:164
2228899993331,2cyclictest2702124-21kworker/u16:2+events_unbound19:43:104
2228899993331,2cyclictest2505401-21kworker/u16:0+events_unbound18:05:314
222890699320,30cyclictest2616595-21latency_hist18:34:517
2228905993230,2cyclictest2354396-21kworker/u16:5+events_unbound16:45:196
2228901993230,2cyclictest2616382-21kworker/u16:5+flush-8:019:14:455
2228899993230,2cyclictest2808198-21kworker/u16:3+events_unbound20:54:004
2228899993230,2cyclictest2616382-21kworker/u16:5+flush-8:019:17:514
2228899993230,2cyclictest2616382-21kworker/u16:5+flush-8:018:56:104
2228899993230,2cyclictest2616382-21kworker/u16:5+events_unbound18:53:144
2228899993230,2cyclictest2238601-21kworker/u16:2+events_unbound15:31:274
2228901993129,2cyclictest2787409-21kworker/u16:5+events_unbound20:03:535
2228901993129,2cyclictest2616382-21kworker/u16:5+flush-8:019:00:205
2228901993129,2cyclictest2303782-21kworker/u16:2+flush-8:016:15:225
2228901993129,2cyclictest2252811-21kworker/u16:4+flush-8:015:44:215
2228901993128,2cyclictest2424371-21kworker/u16:0+events_unbound17:04:585
2228899993129,2cyclictest2525376-21kworker/u16:1+flush-8:018:33:394
2228899993129,2cyclictest2354396-21kworker/u16:5+events_unbound16:47:324
2228899993129,2cyclictest2238601-21kworker/u16:2+events_unbound15:36:344
2228901993028,2cyclictest2616382-21kworker/u16:5+events_unbound19:32:315
2228901993028,2cyclictest2424371-21kworker/u16:0+events_unbound17:10:165
2228899993028,2cyclictest2545692-21kworker/u16:3+events_unbound18:00:514
2228899993028,2cyclictest2444900-21kworker/u16:4+flush-8:017:48:254
2228899993028,2cyclictest2444900-21kworker/u16:4+events_unbound17:57:324
2228899993028,2cyclictest2384159-21kworker/u16:2+events_unbound17:42:294
2228899993028,2cyclictest2354396-21kworker/u16:5+flush-8:016:39:394
2228899993028,2cyclictest2354396-21kworker/u16:5+events_unbound17:04:034
2228899993028,2cyclictest1517054-21kworker/u16:1+flush-8:015:29:514
2228906992924,4cyclictest2606468-21latency_hist18:29:517
2228901992927,2cyclictest2879043-21kworker/u16:1+flush-8:020:50:315
2228901992927,2cyclictest2839050-21kworker/u16:4+events_unbound20:31:535
2228901992927,2cyclictest2303782-21kworker/u16:2+events_unbound16:06:145
2228899992927,2cyclictest2828306-21kworker/u16:2+flush-8:020:38:294
2228899992927,2cyclictest2828306-21kworker/u16:2+events_unbound20:22:504
2228899992927,2cyclictest2616376-21kworker/u16:3+flush-8:018:35:314
2228899992927,2cyclictest2525376-21kworker/u16:1+flush-8:018:14:334
2228899992927,2cyclictest2505401-21kworker/u16:0+events_unbound17:52:334
2228899992927,2cyclictest2444514-21kworker/u16:1+events_unbound17:08:154
2228899992927,2cyclictest2303782-21kworker/u16:2+events_unbound16:06:344
2228899992927,2cyclictest2273336-21kworker/u16:1+flush-8:015:59:434
2228899992926,2cyclictest2616382-21kworker/u16:5+events_unbound19:34:514
2228901992826,2cyclictest2808198-21kworker/u16:3+events_unbound20:44:325
2228901992826,2cyclictest2627893-21kworker/u16:0+events_unbound18:43:315
2228901992826,2cyclictest2525376-21kworker/u16:1+events_unbound18:53:315
2228899992826,2cyclictest2798633-21kworker/u16:1+flush-8:020:05:244
2228899992826,2cyclictest2787409-21kworker/u16:5+events_unbound20:29:134
2228899992826,2cyclictest2768388-21kworker/u16:1+events_unbound19:51:504
2228899992826,2cyclictest2596040-21kworker/u16:4+flush-8:019:24:104
2228899992826,2cyclictest2384159-21kworker/u16:2+events_unbound16:57:264
2228899992826,2cyclictest2354396-21kworker/u16:5+events_unbound16:53:224
2228899992825,2cyclictest2627893-21kworker/u16:0+flush-8:018:44:554
2228906992726,0cyclictest0-21swapper/718:05:137
2228906992722,4cyclictest2869010-21latency_hist20:39:507
2228906992722,4cyclictest2313770-21sed16:04:517
2228905992724,3cyclictest2616382-21kworker/u16:5+flush-8:019:00:076
2228901992726,1cyclictest2849703-21kworker/u16:0+events_unbound20:49:135
2228901992725,2cyclictest2303782-21kworker/u16:2+events_unbound16:24:325
2228899992725,2cyclictest2839050-21kworker/u16:4+flush-8:020:32:054
2228899992725,2cyclictest2616382-21kworker/u16:5+flush-8:018:41:094
2228899992725,2cyclictest2495068-21kworker/u16:3+flush-8:017:35:304
2228899992724,2cyclictest2849703-21kworker/u16:0+flush-8:020:39:514
2228899992724,2cyclictest2616382-21kworker/u16:5+events_unbound19:30:004
2228906992625,0cyclictest0-21swapper/720:11:537
2228906992625,0cyclictest0-21swapper/717:12:537
2228906992624,1cyclictest0-21swapper/719:35:157
2228906992620,5cyclictest589235-21systemd-journal19:44:517
2228906992620,5cyclictest589235-21systemd-journal19:15:067
222890699260,1cyclictest0-21swapper/719:05:157
2228901992624,2cyclictest2828306-21kworker/u16:2+flush-8:020:38:345
2228901992624,2cyclictest2555832-21kworker/u16:2+events_unbound18:27:325
2228901992624,2cyclictest2545692-21kworker/u16:3+events_unbound18:02:395
2228901992624,2cyclictest2495068-21kworker/u16:3+events_unbound17:40:085
2228901992624,2cyclictest2343770-21kworker/u16:1+events_unbound16:28:335
2228901992623,2cyclictest2768388-21kworker/u16:1+flush-8:019:54:535
2228899992624,2cyclictest2839050-21kworker/u16:4+flush-8:020:45:544
2228899992624,2cyclictest2555832-21kworker/u16:2+events_unbound18:19:024
2228899992624,2cyclictest2525376-21kworker/u16:1+events_unbound18:21:034
2228899992624,2cyclictest2273336-21kworker/u16:1+events_unbound16:01:194
2228899992624,2cyclictest2218505-21kworker/u16:3+events_unbound15:42:314
2228906992520,4cyclictest2273303-21latency_hist15:44:527
2228901992523,2cyclictest2702124-21kworker/u16:2+flush-8:019:46:315
2228901992523,2cyclictest2596040-21kworker/u16:4+events_unbound19:26:315
2228901992523,2cyclictest2354396-21kworker/u16:5+events_unbound16:55:315
2228901992523,2cyclictest2354396-21kworker/u16:5+events_unbound16:52:315
2228899992523,2cyclictest2677035-21kworker/u16:0+events_unbound19:11:544
2228899992523,2cyclictest2525376-21kworker/u16:1+events_unbound18:26:104
2228899992523,2cyclictest2303782-21kworker/u16:2+flush-8:016:27:014
2228899992523,2cyclictest2252811-21kworker/u16:4+flush-8:015:46:474
2228906992423,0cyclictest0-21swapper/715:54:017
2228901992422,2cyclictest2738890-21kworker/u16:0+events_unbound20:13:335
2228901992422,2cyclictest2717886-21kworker/u16:3+flush-8:019:40:265
2228901992422,2cyclictest2555832-21kworker/u16:2+events_unbound18:10:345
2228901992421,2cyclictest2702124-21kworker/u16:2+flush-8:019:19:545
2228901992420,3cyclictest1269054-21kworker/u16:7+events_unbound15:45:215
2228899992422,2cyclictest2596040-21kworker/u16:4+flush-8:019:29:464
2228899992422,2cyclictest2525376-21kworker/u16:1+events_unbound19:01:304
2228906992322,0cyclictest0-21swapper/719:20:327
2228906992318,4cyclictest2394389-21latency_hist16:44:517
2228906992316,5cyclictest589235-21systemd-journal16:29:527
2228901992321,2cyclictest2702124-21kworker/u16:2+events_unbound19:18:405
2228901992321,2cyclictest2525376-21kworker/u16:1+events_unbound18:57:325
2228901992321,2cyclictest2475178-21kworker/u16:1+events_unbound17:28:255
2228899992321,2cyclictest2787409-21kworker/u16:5+events_unbound20:13:004
2228906992217,4cyclictest2495400-21latency_hist17:34:517
2228901992220,2cyclictest2717886-21kworker/u16:3+events_unbound19:38:015
2228901992220,2cyclictest2596040-21kworker/u16:4+events_unbound18:47:305
2228901992220,2cyclictest2444900-21kworker/u16:4+events_unbound17:58:315
2228899992220,2cyclictest2218505-21kworker/u16:3+events_unbound15:52:314
2228906992120,0cyclictest0-21swapper/718:58:587
2228906992120,0cyclictest0-21swapper/718:19:217
2228906992120,0cyclictest0-21swapper/717:45:467
2228906992120,0cyclictest0-21swapper/715:58:267
2228906992116,0cyclictest0-21swapper/719:32:587
2228901992119,2cyclictest2808198-21kworker/u16:3+events_unbound20:23:255
2228901992119,2cyclictest2495068-21kworker/u16:3+events_unbound17:45:155
2228901992119,2cyclictest2333656-21kworker/u16:4+events_unbound16:36:145
2228901992119,2cyclictest2218505-21kworker/u16:3+flush-8:015:53:135
2228901992118,2cyclictest1269054-21kworker/u16:7+events_unbound15:25:165
2228899992118,2cyclictest2384159-21kworker/u16:2+events_unbound17:29:534
2228906992019,0cyclictest0-21swapper/719:11:207
2228906992019,0cyclictest0-21swapper/717:04:067
2228906992019,0cyclictest0-21swapper/716:26:477
2228906992019,0cyclictest0-21swapper/715:28:217
2228901992018,2cyclictest2616376-21kworker/u16:3+events_unbound18:35:255
2228901992018,2cyclictest2252811-21kworker/u16:4+flush-8:015:36:445
2228906991918,0cyclictest0-21swapper/719:27:107
2228906991918,0cyclictest0-21swapper/717:26:577
2228906991918,0cyclictest0-21swapper/717:24:067
2228906991918,0cyclictest0-21swapper/716:50:207
2228906991817,0cyclictest0-21swapper/720:48:037
2228906991817,0cyclictest0-21swapper/720:16:077
2228906991817,0cyclictest0-21swapper/718:11:427
2228906991817,0cyclictest0-21swapper/717:53:397
2228906991817,0cyclictest0-21swapper/717:15:167
2228906991817,0cyclictest0-21swapper/716:58:307
222890699180,17cyclictest0-21swapper/717:40:177
222890699179,7cyclictest27904-21gmain18:45:057
2228906991716,0cyclictest0-21swapper/718:52:407
2228906991715,1cyclictest0-21swapper/715:35:117
2228901991715,2cyclictest2354396-21kworker/u16:5+events_unbound17:15:545
2228906991615,0cyclictest0-21swapper/720:19:547
2228906991613,2cyclictest4175659-21snmpd20:27:307
2228906991611,4cyclictest589235-21systemd-journal16:07:417
222890699160,14cyclictest0-21swapper/719:50:177
2228905991615,1cyclictest0-21swapper/619:40:196
222890599161,2cyclictest0-21swapper/619:10:156
2228906991515,0cyclictest0-21swapper/720:06:047
2228906991514,0cyclictest0-21swapper/719:41:197
2228906991512,2cyclictest0-21swapper/715:30:137
222890699150,14cyclictest4175659-21snmpd17:37:267
222890599150,1cyclictest2239326-21sensors15:29:526
2228901991513,2cyclictest2525376-21kworker/u16:1+events_unbound18:20:185
2228906991414,0cyclictest0-21swapper/716:24:407
2228906991413,0cyclictest0-21swapper/718:01:497
2228906991412,1cyclictest0-21swapper/720:40:127
2228906991410,3cyclictest2283385-21munin-run15:49:507
2228905991412,2cyclictest2596040-21kworker/u16:4+flush-8:019:59:506
222890599140,7cyclictest0-21swapper/620:35:066
222890599140,14cyclictest0-21swapper/618:20:086
222890599140,14cyclictest0-21swapper/616:18:356
222890599140,14cyclictest0-21swapper/615:53:036
2228901991412,2cyclictest2616382-21kworker/u16:5+events_unbound19:08:095
222890699137,5cyclictest2899306-21latency_hist20:54:517
2228906991313,0cyclictest0-21swapper/717:55:207
2228906991312,0cyclictest0-21swapper/716:46:227
2228906991312,0cyclictest0-21swapper/716:36:087
2228906991312,0cyclictest0-21swapper/716:15:417
222890599138,4cyclictest2842639-21grep20:25:096
222890599138,4cyclictest2297028-21cut15:55:126
222890599134,8cyclictest2530115-21kthreadcore17:50:096
222890599133,7cyclictest2459328-21grep17:15:096
222890599133,7cyclictest2325769-21grep16:10:106
2228905991311,2cyclictest0-21swapper/616:33:526
2228905991310,2cyclictest2897623-21kthreadcore20:50:126
222890599130,13cyclictest0-21swapper/620:15:506
2228901991311,2cyclictest2354396-21kworker/u16:5+events_unbound16:46:515
222890699129,2cyclictest2595391-21ntpq18:20:167
2228906991210,1cyclictest0-21swapper/718:34:567
222890699120,10cyclictest96-21ksoftirqd/716:10:067
222890599129,2cyclictest2738691-21forks19:35:046
222890599129,2cyclictest2698517-21iostat19:15:076
222890599129,2cyclictest2682241-21cut19:05:106
222890599129,2cyclictest2316009-21kthreadcore16:05:086
222890599128,3cyclictest2313608-21chrt16:04:256
222890599127,4cyclictest2426342-21kthreadcore17:00:116
222890599126,5cyclictest0-21swapper/616:35:466
222890599125,6cyclictest2790918-21grep20:00:116
222890599124,7cyclictest2876771-21grep20:40:126
222890599124,7cyclictest2406126-21chrt16:50:086
222890599124,7cyclictest2358177-21cut16:25:116
222890599124,7cyclictest2353832-21chrt16:22:366
222890599124,4cyclictest27115131cat19:20:096
222890599122,9cyclictest2538426-21kthreadcore17:55:096
222890599122,7cyclictest0-21swapper/620:05:086
2228905991211,1cyclictest0-21swapper/619:51:526
2228905991211,1cyclictest0-21swapper/618:15:056
2228905991211,1cyclictest0-21swapper/617:09:026
2228905991211,1cyclictest0-21swapper/616:42:426
222890599120,1cyclictest0-21swapper/620:12:226
2228901991210,2cyclictest2263124-21kworker/u16:0+events_unbound15:56:305
222890699117,3cyclictest96-21ksoftirqd/720:00:107
222890699110,10cyclictest4175659-21snmpd17:07:447
222890599117,3cyclictest2603172-21cut18:25:126
222890599115,5cyclictest2729187-21cut19:30:086
222890599115,5cyclictest2415051-21cpuspeed16:55:036
222890599114,6cyclictest2855995-21cut20:30:136
222890599114,6cyclictest2648430-21kernelversion18:50:086
222890599114,6cyclictest2643698-21cut18:45:136
222890599114,6cyclictest2629466-21cat18:40:106
222890599114,6cyclictest2618255-21grep18:35:106
222890599114,6cyclictest2608898-21grep18:30:096
222890599114,6cyclictest2556855-21cstates18:05:036
222890599114,6cyclictest2519123-21cut17:45:096
222890599114,6cyclictest2491348-21grep17:30:126
222890599114,6cyclictest2476850-21grep17:25:086
222890599114,6cyclictest2474834-21chrt17:20:296
222890599114,6cyclictest2274078-21expr15:45:056
222890599114,6cyclictest2268643-21cut15:40:106
222890599114,6cyclictest2256500-21cut15:35:106
222890599114,6cyclictest2229271-21grep15:25:046
222890599114,5cyclictest2657826-21df18:55:066
2228905991110,1cyclictest0-21swapper/619:26:206
2228905991110,1cyclictest0-21swapper/617:13:266
222890599110,11cyclictest0-21swapper/620:46:446
222890599110,11cyclictest0-21swapper/618:13:216
222890599110,11cyclictest0-21swapper/617:41:586
222890599104,5cyclictest2829128-21expr20:20:016
222890599104,5cyclictest2759105-21if_p4p119:45:066
222890599104,5cyclictest2546891-21fschecks_time18:00:086
222890599103,6cyclictest2496531-21gltestperf17:35:066
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional