You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-17 - 01:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot2.osadl.org (updated Fri Oct 17, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
69081821000,1sleep7690819-21cut05:21:247
44055626656,7sleep40-21swapper/403:23:514
44064225521,29sleep70-21swapper/703:24:187
441331995045,4cyclictest451907-21latency_hist03:30:587
44063624838,7sleep50-21swapper/503:24:165
441331994745,1cyclictest997490-21sh08:00:577
441323994745,2cyclictest510413-21kworker/u16:0+events_unbound04:01:305
441323994745,2cyclictest510413-21kworker/u16:0+events_unbound04:01:305
44133199460,44cyclictest963888-21ps07:41:277
441323994644,2cyclictest727460-21kworker/u16:0+events_unbound06:31:145
441327994543,2cyclictest846693-21kworker/u16:1+flush-8:007:16:146
441331994440,2cyclictest792391-21ntpq06:21:277
441327994442,2cyclictest1018535-21kworker/u16:0+events_unbound08:46:246
441327994341,2cyclictest986110-21kworker/u16:4+flush-8:007:56:136
441323994340,2cyclictest986110-21kworker/u16:4+events_unbound08:41:145
441327994240,2cyclictest523199-21kworker/u16:1+events_unbound04:06:256
441327994139,2cyclictest727460-21kworker/u16:0+flush-8:006:16:266
441327994139,2cyclictest727460-21kworker/u16:0+flush-8:006:16:266
441327994139,2cyclictest3924602-21kworker/u16:9+events_unbound04:40:536
441323994139,2cyclictest727460-21kworker/u16:0+flush-8:007:06:125
441327994038,2cyclictest846693-21kworker/u16:1+events_unbound06:56:296
441327994038,2cyclictest451717-21kworker/u16:4+events_unbound05:26:216
2765589400,32rtkit-daemon0-21swapper/603:21:156
441327993937,2cyclictest727460-21kworker/u16:0+flush-8:006:26:266
441327993937,2cyclictest1076863-21kworker/u16:6+flush-8:008:40:376
441327993836,2cyclictest986110-21kworker/u16:4+flush-8:008:04:366
441327993836,2cyclictest825623-21kworker/u16:2+events_unbound07:36:296
441323993836,2cyclictest825623-21kworker/u16:2+flush-8:007:36:125
441323993836,2cyclictest825623-21kworker/u16:2+flush-8:007:26:235
441323993835,2cyclictest996845-21kworker/u16:1+flush-8:008:26:175
441327993735,2cyclictest985871-21kworker/u16:3+flush-8:008:06:156
441327993735,2cyclictest825623-21kworker/u16:2+events_unbound07:45:366
441327993735,2cyclictest727460-21kworker/u16:0+flush-8:005:51:206
441327993735,2cyclictest607988-21kworker/u16:1+events_unbound04:57:456
441327993735,2cyclictest510413-21kworker/u16:0+flush-8:004:26:286
441327993734,3cyclictest825623-21kworker/u16:2+flush-8:007:51:246
441323993735,2cyclictest650744-21kworker/u16:5+flush-8:006:26:225
441323993735,2cyclictest1071763-21kworker/u16:3+events_unbound08:50:445
441327993634,2cyclictest986110-21kworker/u16:4+flush-8:008:51:176
441327993634,2cyclictest1093271-21kworker/u16:1+flush-8:008:42:566
441327993533,2cyclictest909956-21kworker/u16:3+flush-8:007:26:246
441327993533,2cyclictest727460-21kworker/u16:0+events_unbound07:04:206
441327993533,2cyclictest682998-21kworker/u16:1+events_unbound06:07:206
441327993533,2cyclictest585602-21kworker/u16:2+events_unbound04:45:216
441327993533,2cyclictest498700-21kworker/u16:3+flush-8:003:51:586
441323993533,2cyclictest650744-21kworker/u16:5+events_unbound06:20:085
441323993533,2cyclictest650744-21kworker/u16:5+events_unbound06:20:085
441323993533,2cyclictest596793-21kworker/u16:0+events_unbound05:29:135
441323993533,2cyclictest3924602-21kworker/u16:9+events_unbound04:39:335
441331993429,4cyclictest954553-21latency_hist07:40:567
441331993429,4cyclictest826032-21latency_hist06:40:577
441327993432,2cyclictest825623-21kworker/u16:2+flush-8:007:48:566
441327993432,2cyclictest690231-21kworker/u16:2+events_unbound05:34:286
441327993432,2cyclictest690231-21kworker/u16:2+events_unbound05:34:286
441327993432,2cyclictest1050295-21kworker/u16:3+events_unbound08:25:206
441327993432,2cyclictest1029400-21kworker/u16:2+events_unbound08:16:156
441323993432,2cyclictest825623-21kworker/u16:2+flush-8:007:51:465
441323993432,2cyclictest825623-21kworker/u16:2+flush-8:007:20:405
441323993432,2cyclictest727460-21kworker/u16:0+events_unbound06:40:085
441323993432,2cyclictest1050295-21kworker/u16:3+events_unbound08:24:495
441327993331,2cyclictest510413-21kworker/u16:0+flush-8:004:01:276
441327993331,2cyclictest510413-21kworker/u16:0+flush-8:004:01:276
441327993331,2cyclictest510413-21kworker/u16:0+events_unbound04:33:206
441327993331,2cyclictest451717-21kworker/u16:4+flush-8:005:19:206
441327993331,2cyclictest3924602-21kworker/u16:9+flush-8:003:36:216
441327993331,2cyclictest3924602-21kworker/u16:9+events_unbound03:32:546
441323993331,2cyclictest932577-21kworker/u16:4+events_unbound07:44:085
441323993331,2cyclictest909956-21kworker/u16:3+flush-8:007:34:405
441323993331,2cyclictest825623-21kworker/u16:2+events_unbound07:03:335
441323993331,2cyclictest825623-21kworker/u16:2+events_unbound06:55:145
441323993331,2cyclictest825623-21kworker/u16:2+events_unbound06:45:485
441323993331,2cyclictest596793-21kworker/u16:0+events_unbound04:42:255
441323993331,2cyclictest451685-21kworker/u16:2+events_unbound03:31:425
441323993331,2cyclictest429936-21kworker/u16:0+events_unbound03:30:025
441323993331,2cyclictest1006835-21kworker/u16:2+events_unbound08:06:405
441331993227,4cyclictest933133-21latency_hist07:30:577
441331993227,4cyclictest589235-21systemd-journal06:44:287
441331993224,6cyclictest589235-21systemd-journal05:35:577
441331993224,6cyclictest589235-21systemd-journal05:35:577
441327993230,2cyclictest996845-21kworker/u16:1+events_unbound08:32:206
441327993230,2cyclictest791111-21kworker/u16:4+events_unbound06:47:206
441327993230,2cyclictest650744-21kworker/u16:5+events_unbound06:01:326
441327993230,2cyclictest554282-21kworker/u16:2+flush-8:004:21:166
441327993230,2cyclictest429936-21kworker/u16:0+events_unbound03:28:096
441327993229,2cyclictest715889-21kworker/u16:3+flush-8:005:47:566
441323993230,2cyclictest986110-21kworker/u16:4+flush-8:008:04:405
441323993230,2cyclictest986110-21kworker/u16:4+events_unbound08:32:595
441323993230,2cyclictest985871-21kworker/u16:3+events_unbound07:56:215
441323993230,2cyclictest909956-21kworker/u16:3+events_unbound07:25:535
441323993230,2cyclictest791111-21kworker/u16:4+flush-8:006:46:225
441323993230,2cyclictest760025-21kworker/u16:2+events_unbound06:15:445
441323993230,2cyclictest554282-21kworker/u16:2+events_unbound04:21:075
441323993230,2cyclictest510413-21kworker/u16:0+flush-8:004:26:145
441323993230,2cyclictest451717-21kworker/u16:4+flush-8:005:02:415
441323993230,2cyclictest3924602-21kworker/u16:9+events_unbound03:46:295
441323993230,2cyclictest1106069-21kworker/u16:2+events_unbound08:52:445
441323993229,2cyclictest596793-21kworker/u16:0+events_unbound05:38:265
441331993129,1cyclictest628952-21awk04:55:577
441327993129,2cyclictest912167-21kworker/u16:5+flush-8:007:21:236
441327993129,2cyclictest889462-21kworker/u16:4+events_unbound07:14:066
441327993129,2cyclictest690231-21kworker/u16:2+events_unbound05:40:456
441327993129,2cyclictest682998-21kworker/u16:1+events_unbound05:43:496
441327993129,2cyclictest650744-21kworker/u16:5+flush-8:006:32:136
441327993129,2cyclictest650744-21kworker/u16:5+events_unbound05:15:076
441323993129,2cyclictest727460-21kworker/u16:0+events_unbound06:09:505
441323993129,2cyclictest715889-21kworker/u16:3+events_unbound05:55:305
441323993129,2cyclictest635481-21kworker/u16:3+events_unbound05:08:305
441323993129,2cyclictest596793-21kworker/u16:0+flush-8:005:22:255
441323993129,2cyclictest596793-21kworker/u16:0+events_unbound05:20:295
441323993129,2cyclictest3924602-21kworker/u16:9+events_unbound04:32:295
441327993028,2cyclictest727460-21kworker/u16:0+flush-8:006:40:576
441327993028,2cyclictest585602-21kworker/u16:2+events_unbound04:55:256
441327993028,2cyclictest523199-21kworker/u16:1+events_unbound04:12:216
441327993028,2cyclictest451717-21kworker/u16:4+events_unbound03:47:466
441327993027,2cyclictest985871-21kworker/u16:3+flush-8:008:11:136
441323993028,2cyclictest985871-21kworker/u16:3+flush-8:008:14:415
441323993028,2cyclictest727460-21kworker/u16:0+events_unbound06:01:065
441323993028,2cyclictest650744-21kworker/u16:5+flush-8:005:58:335
441323993028,2cyclictest607988-21kworker/u16:1+events_unbound04:52:255
441323993028,2cyclictest585602-21kworker/u16:2+events_unbound04:46:125
441323993028,2cyclictest585602-21kworker/u16:2+events_unbound04:46:125
441323993028,2cyclictest498700-21kworker/u16:3+flush-8:004:08:215
441323993028,2cyclictest467128-21kworker/u16:0+events_unbound03:41:135
441323993028,2cyclictest451717-21kworker/u16:4+events_unbound05:11:525
441323993028,2cyclictest451685-21kworker/u16:2+events_unbound03:55:465
441323993028,2cyclictest451685-21kworker/u16:2+events_unbound03:37:515
441323993028,2cyclictest3924602-21kworker/u16:9+flush-8:004:12:415
441323993028,2cyclictest1076863-21kworker/u16:6+events_unbound08:40:215
441323993027,2cyclictest451717-21kworker/u16:4+events_unbound05:31:015
441323993027,2cyclictest451717-21kworker/u16:4+events_unbound05:31:015
8052672290,1chrt805269-21cpuspeed_turbos06:31:117
441327992927,2cyclictest909956-21kworker/u16:3+events_unbound07:32:406
441327992927,2cyclictest782388-21kworker/u16:3+events_unbound06:23:136
441327992927,2cyclictest715889-21kworker/u16:3+flush-8:005:56:326
441327992927,2cyclictest451685-21kworker/u16:2+events_unbound03:58:016
441327992927,2cyclictest3924602-21kworker/u16:9+flush-8:003:45:296
441327992927,2cyclictest1018535-21kworker/u16:0+events_unbound08:26:506
441323992927,2cyclictest727460-21kworker/u16:0+events_unbound05:47:145
441323992927,2cyclictest682998-21kworker/u16:1+flush-8:005:42:465
441323992927,2cyclictest1018535-21kworker/u16:0+events_unbound08:20:335
441327992826,2cyclictest596793-21kworker/u16:0+events_unbound05:02:336
441327992826,2cyclictest585602-21kworker/u16:2+events_unbound04:49:176
441327992826,2cyclictest585602-21kworker/u16:2+events_unbound04:49:176
441327992826,2cyclictest523199-21kworker/u16:1+flush-8:004:16:206
441323992826,2cyclictest954189-21kworker/u16:0+flush-8:007:47:565
441323992826,2cyclictest846693-21kworker/u16:1+flush-8:007:15:135
441323992826,2cyclictest782388-21kworker/u16:3+events_unbound06:23:495
441323992825,2cyclictest451685-21kworker/u16:2+flush-8:003:56:075
441331992726,0cyclictest0-21swapper/708:31:067
441331992723,3cyclictest543152-21awk04:15:577
441327992725,2cyclictest846693-21kworker/u16:1+events_unbound06:52:306
441327992725,2cyclictest650744-21kworker/u16:5+events_unbound05:23:336
441327992724,2cyclictest791111-21kworker/u16:4+events_unbound06:41:006
441323992725,2cyclictest3924602-21kworker/u16:9+flush-8:004:18:235
441331992621,4cyclictest589235-21systemd-journal06:54:357
441331992620,5cyclictest716144-21latency_hist05:40:577
441331992620,5cyclictest1051053-21latency_hist08:25:567
441327992624,2cyclictest857979-21kworker/u16:3+flush-8:007:06:326
441327992624,2cyclictest596793-21kworker/u16:0+events_unbound05:07:326
441331992524,0cyclictest0-21swapper/705:01:277
441331992524,0cyclictest0-21swapper/703:49:337
441331992519,5cyclictest589235-21systemd-journal04:20:587
441323992523,2cyclictest857979-21kworker/u16:3+events_unbound07:00:395
441323992523,2cyclictest607988-21kworker/u16:1+flush-8:004:56:255
7429052240,1chrt742906-21kthreadcore05:56:177
441331992419,4cyclictest589235-21systemd-journal07:04:367
441321992422,2cyclictest585602-21kworker/u16:2+flush-8:004:51:264
441331992322,0cyclictest0-21swapper/706:13:367
441331992317,5cyclictest986729-21latency_hist07:55:577
44133199230,21cyclictest616771-21sh04:46:237
44133199230,21cyclictest616771-21sh04:46:237
441327992321,2cyclictest682998-21kworker/u16:1+flush-8:006:11:326
441321992321,2cyclictest727460-21kworker/u16:0+events_unbound06:16:154
441321992321,2cyclictest727460-21kworker/u16:0+events_unbound06:16:154
5097452220,1chrt509746-21sed03:56:307
441321992119,2cyclictest996845-21kworker/u16:1+events_unbound08:01:244
441321992119,2cyclictest954189-21kworker/u16:0+flush-8:007:51:124
441321992119,2cyclictest727460-21kworker/u16:0+flush-8:006:26:124
441321992119,2cyclictest596793-21kworker/u16:0+flush-8:004:57:064
441321992119,2cyclictest451717-21kworker/u16:4+flush-8:004:11:124
44133199200,18cyclictest969143-21cut07:46:177
441321992018,2cyclictest727460-21kworker/u16:0+events_unbound05:49:314
441321992018,2cyclictest451685-21kworker/u16:2+events_unbound03:55:524
441331991918,0cyclictest0-21swapper/705:09:437
441321991917,2cyclictest986110-21kworker/u16:4+flush-8:007:56:274
441321991917,2cyclictest889462-21kworker/u16:4+flush-8:007:20:524
441321991917,2cyclictest596793-21kworker/u16:0+flush-8:004:46:244
441321991917,2cyclictest596793-21kworker/u16:0+flush-8:004:46:244
441321991917,2cyclictest451717-21kworker/u16:4+flush-8:004:01:124
441321991917,2cyclictest451717-21kworker/u16:4+flush-8:004:01:124
441321991816,2cyclictest986110-21kworker/u16:4+flush-8:008:51:534
441321991816,2cyclictest986110-21kworker/u16:4+flush-8:008:13:374
441321991816,2cyclictest954189-21kworker/u16:0+events_unbound07:42:224
441321991816,2cyclictest846693-21kworker/u16:1+events_unbound06:55:364
441321991816,2cyclictest825623-21kworker/u16:2+events_unbound07:01:434
441321991816,2cyclictest760025-21kworker/u16:2+flush-8:006:10:534
441321991816,2cyclictest727460-21kworker/u16:0+events_unbound06:00:374
441321991816,2cyclictest682998-21kworker/u16:1+events_unbound05:21:264
441321991816,2cyclictest451717-21kworker/u16:4+events_unbound03:41:304
441321991815,2cyclictest690231-21kworker/u16:2+events_unbound05:41:084
441331991716,0cyclictest0-21swapper/707:10:367
441331991714,2cyclictest0-21swapper/708:50:577
441321991715,2cyclictest909956-21kworker/u16:3+events_unbound07:21:304
441321991715,2cyclictest635481-21kworker/u16:3+events_unbound05:01:224
441321991715,2cyclictest1093271-21kworker/u16:1+flush-8:008:49:404
441331991616,0cyclictest0-21swapper/704:59:087
441331991613,2cyclictest0-21swapper/707:16:247
441321991614,2cyclictest954189-21kworker/u16:0+events_unbound07:46:464
441321991614,2cyclictest857979-21kworker/u16:3+flush-8:006:57:414
441321991614,2cyclictest846693-21kworker/u16:1+events_unbound07:10:514
441321991614,2cyclictest727460-21kworker/u16:0+events_unbound06:21:264
441321991614,2cyclictest650744-21kworker/u16:5+events_unbound06:40:434
441321991614,2cyclictest635481-21kworker/u16:3+events_unbound05:06:154
441321991614,2cyclictest488819-21kworker/u16:1+events_unbound03:50:164
441321991614,2cyclictest3924602-21kworker/u16:9+flush-8:003:38:084
44133199159,5cyclictest500132-21latency_hist03:55:577
441331991512,2cyclictest0-21swapper/708:40:577
441331991512,2cyclictest0-21swapper/708:00:577
441331991510,3cyclictest516772-21grep04:01:217
441331991510,3cyclictest516772-21grep04:01:217
44133199150,14cyclictest1313-21snmpd08:20:547
441321991513,2cyclictest986110-21kworker/u16:4+events_unbound08:18:374
441321991513,2cyclictest909956-21kworker/u16:3+events_unbound07:36:564
441321991513,2cyclictest715889-21kworker/u16:3+events_unbound05:40:024
441321991513,2cyclictest451717-21kworker/u16:4+flush-8:005:17:564
441321991513,2cyclictest451717-21kworker/u16:4+flush-8:005:12:204
441321991513,2cyclictest451685-21kworker/u16:2+flush-8:003:31:484
441321991513,2cyclictest1018535-21kworker/u16:0+flush-8:008:21:244
441321991512,2cyclictest760025-21kworker/u16:2+flush-8:006:11:014
44133199148,5cyclictest682616-21latency_hist05:20:577
44133199147,6cyclictest27924-21gmain08:11:127
441331991411,2cyclictest0-21swapper/706:30:577
441331991411,2cyclictest0-21swapper/704:16:107
441321991413,1cyclictest0-21swapper/404:16:154
441321991412,2cyclictest986110-21kworker/u16:4+events_unbound08:41:164
441321991412,2cyclictest727460-21kworker/u16:0+flush-8:007:15:034
441321991412,2cyclictest727460-21kworker/u16:0+events_unbound06:44:574
441321991412,2cyclictest727460-21kworker/u16:0+events_unbound05:54:364
441321991412,2cyclictest682998-21kworker/u16:1+events_unbound05:32:524
441321991412,2cyclictest682998-21kworker/u16:1+events_unbound05:32:524
441321991412,2cyclictest585602-21kworker/u16:2+events_unbound04:45:384
44133199139,3cyclictest575750-21/usr/sbin/munin04:31:177
44133199138,4cyclictest717352-21cat05:41:097
44133199136,6cyclictest0-21swapper/708:10:357
441331991313,0cyclictest0-21swapper/705:14:537
44133199131,10cyclictest781672-21ntp_states06:16:267
44133199131,10cyclictest781672-21ntp_states06:16:267
441331991310,2cyclictest599310-21cut04:41:177
441331991310,2cyclictest1313-21snmpd08:30:457
441331991310,2cyclictest0-21swapper/706:06:207
44133199130,5cyclictest457952-21cstates03:36:077
44133199130,4cyclictest861415-21cut06:56:197
44133199130,11cyclictest838866-21kthreadcore06:46:197
441321991311,2cyclictest986110-21kworker/u16:4+events_unbound08:06:254
441321991311,2cyclictest846693-21kworker/u16:1+flush-8:006:50:084
441321991311,2cyclictest825623-21kworker/u16:2+events_unbound07:27:144
441321991311,2cyclictest650744-21kworker/u16:5+flush-8:006:33:454
441321991311,2cyclictest650744-21kworker/u16:5+events_unbound05:28:404
441321991311,2cyclictest451717-21kworker/u16:4+flush-8:004:23:534
441321991311,2cyclictest451717-21kworker/u16:4+events_unbound03:56:164
441321991311,2cyclictest429936-21kworker/u16:0+flush-8:003:27:184
441321991311,2cyclictest3924602-21kworker/u16:9+events_unbound04:38:174
441321991311,2cyclictest1076863-21kworker/u16:6+flush-8:008:37:494
441321991310,2cyclictest996845-21kworker/u16:1+events_unbound08:31:144
44133199129,2cyclictest565050-21/usr/sbin/munin04:26:267
44133199129,2cyclictest468397-21/usr/sbin/munin03:41:157
44133199129,2cyclictest1313-21snmpd06:03:007
44133199128,3cyclictest522107-21grep04:06:087
44133199128,3cyclictest1313-21snmpd03:32:087
44133199128,2cyclictest0-21swapper/707:31:137
44133199120,2cyclictest728623-21/usr/sbin/munin05:51:177
44133199120,11cyclictest891428-21cat07:11:137
441321991210,2cyclictest996845-21kworker/u16:1+events_unbound08:30:334
441321991210,2cyclictest650744-21kworker/u16:5+events_unbound06:05:564
441321991210,2cyclictest510413-21kworker/u16:0+flush-8:004:26:534
441321991210,2cyclictest498700-21kworker/u16:3+events_unbound04:08:094
44133199116,4cyclictest696374-21cut05:26:157
44133199115,5cyclictest920896-21ntpq07:21:257
44133199113,7cyclictest1103924-21minicom08:46:297
44133199110,10cyclictest1076238-21kthreadcore08:36:217
44132199119,2cyclictest909956-21kworker/u16:3+events_unbound07:34:064
44132199119,2cyclictest510413-21kworker/u16:0+events_unbound04:32:074
44133199107,2cyclictest587000-21taskset04:36:117
44133199105,4cyclictest727584-21lockdev05:47:437
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional