You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 10:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Fri Dec 05, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
45807726048,8sleep60-21swapper/603:17:486
46748125948,7sleep40-21swapper/403:19:154
46761925644,8sleep50-21swapper/503:21:085
468083995451,2cyclictest562510-21kworker/u16:2+flush-8:004:28:084
468090995351,2cyclictest976387-21kworker/u16:1+flush-8:007:52:416
468084995250,2cyclictest643721-21kworker/u16:4+flush-8:005:18:085
468083994846,2cyclictest417162-21kworker/u16:3+events_unbound04:04:094
468083994745,2cyclictest845815-21kworker/u16:4+events_unbound07:10:374
468083994644,2cyclictest602295-21kworker/u16:1+events_unbound04:44:094
468084994543,2cyclictest512260-21kworker/u16:0+events_unbound03:51:095
468084994543,2cyclictest512260-21kworker/u16:0+events_unbound03:51:095
468084994441,2cyclictest744393-21kworker/u16:2+flush-8:005:57:415
4674842439,30sleep70-21swapper/703:19:187
468083994240,2cyclictest714623-21kworker/u16:5+events_unbound05:51:094
468083994240,2cyclictest522417-21kworker/u16:2+events_unbound03:49:084
468083994240,2cyclictest522417-21kworker/u16:2+events_unbound03:49:084
468093994136,4cyclictest589235-21systemd-journal06:01:527
468083994138,2cyclictest744393-21kworker/u16:2+events_unbound06:32:444
468084994038,2cyclictest804662-21kworker/u16:3+events_unbound06:51:105
468083994038,2cyclictest744393-21kworker/u16:2+events_unbound06:05:094
468083994038,2cyclictest542583-21kworker/u16:0+events_unbound04:01:094
468084993937,2cyclictest704533-21kworker/u16:1+events_unbound05:34:095
468084993936,2cyclictest562510-21kworker/u16:2+flush-8:004:12:585
468083993937,2cyclictest895798-21kworker/u16:0+flush-8:006:51:564
468083993937,2cyclictest845815-21kworker/u16:4+events_unbound07:13:414
468084993836,2cyclictest917220-21kworker/u16:2+events_unbound07:29:445
468083993836,2cyclictest643721-21kworker/u16:4+events_unbound05:04:094
468083993836,2cyclictest417162-21kworker/u16:3+events_unbound03:44:094
468084993735,2cyclictest562510-21kworker/u16:2+events_unbound04:22:085
468083993735,2cyclictest1017159-21kworker/u16:4+events_unbound08:09:094
468084993634,2cyclictest976387-21kworker/u16:1+events_unbound07:39:095
468084993634,2cyclictest594180-21kworker/u16:0+events_unbound04:30:535
468084993634,2cyclictest3937713-21kworker/u16:4+flush-8:003:40:095
468084993634,2cyclictest1006800-21kworker/u16:2+events_unbound08:02:095
468083993634,2cyclictest673790-21kworker/u16:0+events_unbound05:32:294
468084993533,2cyclictest845815-21kworker/u16:4+events_unbound06:41:105
468084993533,2cyclictest804662-21kworker/u16:3+events_unbound06:46:115
468084993533,2cyclictest602295-21kworker/u16:1+events_unbound05:05:135
468084993533,2cyclictest417162-21kworker/u16:3+events_unbound03:55:415
468084993532,2cyclictest996675-21kworker/u16:3+flush-8:007:43:085
468084993532,2cyclictest895798-21kworker/u16:0+flush-8:007:12:525
468083993533,2cyclictest602295-21kworker/u16:1+events_unbound05:09:374
468083993533,2cyclictest3937713-21kworker/u16:4+events_unbound03:25:004
468093993433,0cyclictest0-21swapper/706:14:097
46809399340,32cyclictest673897-21latency_hist05:02:427
468090993432,2cyclictest714623-21kworker/u16:5+events_unbound05:23:086
468084993432,2cyclictest895798-21kworker/u16:0+events_unbound07:35:525
468084993432,2cyclictest804662-21kworker/u16:3+flush-8:007:00:365
468084993432,2cyclictest673790-21kworker/u16:0+events_unbound05:16:515
468084993432,2cyclictest1006800-21kworker/u16:2+events_unbound08:15:095
468084993431,2cyclictest804662-21kworker/u16:3+events_unbound06:27:455
468083993432,2cyclictest714623-21kworker/u16:5+events_unbound05:24:524
468083993431,2cyclictest804662-21kworker/u16:3+flush-8:006:18:084
468083993431,2cyclictest1106907-21kworker/u16:3+flush-8:008:38:134
468093993328,4cyclictest492142-21latency_hist03:32:427
468090993331,2cyclictest1017159-21kworker/u16:4+flush-8:008:13:006
468090993329,3cyclictest1057382-21kworker/u16:0+flush-8:008:33:006
468084993331,2cyclictest643721-21kworker/u16:4+events_unbound05:11:135
468084993331,2cyclictest602295-21kworker/u16:1+events_unbound04:55:525
468084993331,2cyclictest542583-21kworker/u16:0+events_unbound04:10:035
468084993331,2cyclictest1017159-21kworker/u16:4+events_unbound08:22:425
468084993331,2cyclictest1006800-21kworker/u16:2+events_unbound08:07:105
468083993331,2cyclictest895798-21kworker/u16:0+flush-8:006:53:124
468083993331,2cyclictest845815-21kworker/u16:4+flush-8:006:40:324
468084993230,2cyclictest917220-21kworker/u16:2+events_unbound07:06:525
468084993230,2cyclictest895798-21kworker/u16:0+flush-8:006:55:515
468084993230,2cyclictest895798-21kworker/u16:0+events_unbound07:21:105
468084993230,2cyclictest744393-21kworker/u16:2+events_unbound06:07:105
468084993230,2cyclictest744393-21kworker/u16:2+events_unbound05:41:445
468084993230,2cyclictest714623-21kworker/u16:5+events_unbound05:25:135
468084993230,2cyclictest1017159-21kworker/u16:4+events_unbound07:56:525
468083993230,2cyclictest845815-21kworker/u16:4+events_unbound07:18:164
468083993230,2cyclictest804662-21kworker/u16:3+flush-8:006:09:514
468083993230,2cyclictest417162-21kworker/u16:3+events_unbound03:40:524
468083993230,2cyclictest1017159-21kworker/u16:4+events_unbound08:47:104
468083993229,2cyclictest1017159-21kworker/u16:4+events_unbound07:57:434
468090993128,2cyclictest917220-21kworker/u16:2+flush-8:007:17:576
468084993129,2cyclictest996675-21kworker/u16:3+events_unbound08:11:535
468084993129,2cyclictest845408-21kworker/u16:1+flush-8:006:37:105
468084993129,2cyclictest804662-21kworker/u16:3+events_unbound06:09:115
468084993129,2cyclictest1073989-21kworker/u16:1+events_unbound08:19:525
468084993129,2cyclictest1057382-21kworker/u16:0+events_unbound08:48:325
468084993129,2cyclictest1017159-21kworker/u16:4+events_unbound08:46:525
468084993128,2cyclictest845815-21kworker/u16:4+events_unbound07:07:435
468084993128,2cyclictest744393-21kworker/u16:2+flush-8:005:48:095
468083993129,2cyclictest1057382-21kworker/u16:0+events_unbound08:30:334
468093993028,1cyclictest0-21swapper/707:03:047
468093993026,3cyclictest576600-21grep04:13:027
468084993028,2cyclictest673790-21kworker/u16:0+events_unbound06:20:085
468084993028,2cyclictest412076-21kworker/u16:1+events_unbound03:31:015
468083993028,2cyclictest643721-21kworker/u16:4+events_unbound05:21:134
468083993027,2cyclictest744835-21kworker/u16:3+flush-8:005:37:524
468083993027,2cyclictest744393-21kworker/u16:2+flush-8:005:57:444
468083992927,2cyclictest804662-21kworker/u16:3+events_unbound06:31:094
468083992927,2cyclictest1006800-21kworker/u16:2+events_unbound08:15:524
468083992926,2cyclictest417162-21kworker/u16:3+flush-8:003:52:494
468090992826,2cyclictest412076-21kworker/u16:1+events_unbound03:48:056
468090992826,2cyclictest412076-21kworker/u16:1+events_unbound03:48:056
468090992825,2cyclictest845815-21kworker/u16:4+flush-8:006:48:096
468084992826,2cyclictest845815-21kworker/u16:4+events_unbound07:26:135
468084992826,2cyclictest744393-21kworker/u16:2+events_unbound05:47:145
468084992826,2cyclictest602295-21kworker/u16:1+events_unbound04:40:105
468084992826,2cyclictest417162-21kworker/u16:3+events_unbound04:25:545
468083992826,2cyclictest845815-21kworker/u16:4+events_unbound07:06:214
468083992826,2cyclictest804662-21kworker/u16:3+events_unbound06:58:304
468083992826,2cyclictest602295-21kworker/u16:1+events_unbound04:52:134
468084992725,2cyclictest804662-21kworker/u16:3+flush-8:006:15:135
468083992725,2cyclictest996675-21kworker/u16:3+events_unbound07:42:094
468083992725,2cyclictest562510-21kworker/u16:2+events_unbound04:26:474
468083992725,2cyclictest417162-21kworker/u16:3+events_unbound03:32:414
468083992725,2cyclictest1006800-21kworker/u16:2+flush-8:008:05:564
468093992625,0cyclictest0-21swapper/704:29:257
468090992624,2cyclictest895798-21kworker/u16:0+flush-8:006:56:516
468090992623,2cyclictest714623-21kworker/u16:5+flush-8:005:47:596
468084992624,2cyclictest794703-21kworker/u16:1+events_unbound06:00:535
468084992624,2cyclictest1016906-21kworker/u16:0+events_unbound07:52:095
468084992623,2cyclictest1073989-21kworker/u16:1+flush-8:008:27:515
468083992624,2cyclictest976387-21kworker/u16:1+flush-8:007:43:074
468083992624,2cyclictest744393-21kworker/u16:2+flush-8:005:56:284
468093992524,0cyclictest0-21swapper/703:32:457
468090992523,2cyclictest673790-21kworker/u16:0+flush-8:005:03:016
468090992523,2cyclictest1017159-21kworker/u16:4+events_unbound08:22:586
468084992523,2cyclictest633379-21kworker/u16:3+events_unbound04:50:285
468083992523,2cyclictest744393-21kworker/u16:2+events_unbound05:36:334
468083992523,2cyclictest417162-21kworker/u16:3+flush-8:004:08:314
468083992523,2cyclictest3937713-21kworker/u16:4+events_unbound03:33:534
468083992523,2cyclictest1106907-21kworker/u16:3+flush-8:008:36:334
468093992423,0cyclictest0-21swapper/704:18:167
468093992423,0cyclictest0-21swapper/703:49:207
468093992423,0cyclictest0-21swapper/703:49:207
468090992421,2cyclictest673790-21kworker/u16:0+flush-8:006:22:416
468084992422,2cyclictest714623-21kworker/u16:5+events_unbound05:30:245
468084992422,2cyclictest673790-21kworker/u16:0+events_unbound05:01:445
468084992422,2cyclictest542583-21kworker/u16:0+flush-8:004:05:535
468084992422,2cyclictest1017159-21kworker/u16:4+events_unbound08:40:525
468083992423,1cyclictest917220-21kworker/u16:2+events_unbound07:30:124
468083992422,2cyclictest1017159-21kworker/u16:4+events_unbound08:24:124
468093992322,0cyclictest0-21swapper/706:44:557
468093992322,0cyclictest0-21swapper/706:06:087
468093992322,0cyclictest0-21swapper/705:49:177
468093992322,0cyclictest0-21swapper/705:08:117
468093992322,0cyclictest0-21swapper/703:45:047
468093992321,1cyclictest0-21swapper/707:23:107
468093992321,1cyclictest0-21swapper/704:23:087
468084992321,2cyclictest623316-21kworker/u16:2+events_unbound04:44:425
468084992321,2cyclictest512260-21kworker/u16:0+events_unbound03:43:455
468083992321,2cyclictest996675-21kworker/u16:3+events_unbound07:49:524
468083992321,2cyclictest744393-21kworker/u16:2+flush-8:006:26:104
468083992320,2cyclictest1006800-21kworker/u16:2+flush-8:008:17:514
468093992221,0cyclictest0-21swapper/708:22:527
468093992221,0cyclictest0-21swapper/707:58:277
468093992221,0cyclictest0-21swapper/706:32:047
468093992217,4cyclictest1057418-21latency_hist08:12:417
468090992220,2cyclictest744393-21kworker/u16:2+flush-8:006:08:126
468090992220,2cyclictest633379-21kworker/u16:3+flush-8:004:52:576
468090992220,2cyclictest562510-21kworker/u16:2+flush-8:004:18:086
468084992220,2cyclictest594180-21kworker/u16:0+flush-8:004:36:285
468084992220,2cyclictest594180-21kworker/u16:0+flush-8:004:36:285
468084992220,2cyclictest542583-21kworker/u16:0+events_unbound04:01:425
468083992220,2cyclictest895798-21kworker/u16:0+events_unbound07:27:104
468083992220,2cyclictest633379-21kworker/u16:3+events_unbound04:59:104
468083992220,2cyclictest623316-21kworker/u16:2+events_unbound04:42:084
468083992220,2cyclictest417162-21kworker/u16:3+events_unbound04:16:474
468093992120,0cyclictest0-21swapper/707:53:397
468093992120,0cyclictest0-21swapper/707:38:397
468090992119,2cyclictest542683-21kworker/u16:1+flush-8:004:02:576
468083992119,2cyclictest917220-21kworker/u16:2+events_unbound07:33:084
468083992119,2cyclictest602295-21kworker/u16:1+events_unbound04:52:524
468083992118,2cyclictest673790-21kworker/u16:0+events_unbound05:12:454
468093992020,0cyclictest0-21swapper/705:45:257
468093992019,0cyclictest0-21swapper/704:55:447
468093992019,0cyclictest0-21swapper/704:50:577
468093992019,0cyclictest0-21swapper/703:38:417
468090992018,2cyclictest594180-21kworker/u16:0+flush-8:004:37:586
468090992017,2cyclictest845815-21kworker/u16:4+events_unbound07:23:046
468084992018,2cyclictest1006800-21kworker/u16:2+flush-8:008:33:535
468084992017,2cyclictest3937713-21kworker/u16:4+flush-8:003:22:515
468093991918,0cyclictest0-21swapper/708:12:527
468093991918,0cyclictest0-21swapper/707:49:577
468093991918,0cyclictest0-21swapper/707:32:527
468093991918,0cyclictest0-21swapper/703:57:447
468090991917,2cyclictest594180-21kworker/u16:0+events_unbound04:33:106
468090991917,2cyclictest594180-21kworker/u16:0+events_unbound04:33:106
468090991917,2cyclictest417162-21kworker/u16:3+flush-8:003:27:476
468090991916,2cyclictest1006800-21kworker/u16:2+flush-8:008:02:396
468084991917,2cyclictest673790-21kworker/u16:0+events_unbound06:23:455
468093991818,0cyclictest0-21swapper/708:42:097
468093991817,0cyclictest0-21swapper/708:32:187
468093991817,0cyclictest0-21swapper/706:53:107
468093991817,0cyclictest0-21swapper/706:41:017
468093991817,0cyclictest0-21swapper/706:18:337
468093991817,0cyclictest0-21swapper/703:25:107
46809399180,16cyclictest956448-21latency_hist07:22:417
468090991816,2cyclictest895798-21kworker/u16:0+flush-8:007:05:166
468090991816,2cyclictest542583-21kworker/u16:0+events_unbound04:08:076
6371732170,1sleep70-21swapper/704:43:067
468093991716,0cyclictest0-21swapper/705:31:337
468093991716,0cyclictest0-21swapper/704:07:187
468093991715,1cyclictest0-21swapper/707:13:227
468090991715,2cyclictest714623-21kworker/u16:5+events_unbound05:43:006
468084991715,2cyclictest412076-21kworker/u16:1+events_unbound03:35:335
468083991715,2cyclictest1016906-21kworker/u16:0+events_unbound07:56:424
46809399168,7cyclictest27177-21gmain06:58:077
468093991615,0cyclictest0-21swapper/708:51:347
468093991615,0cyclictest0-21swapper/706:50:397
468093991615,0cyclictest0-21swapper/706:24:347
468093991615,0cyclictest0-21swapper/705:18:327
468093991615,0cyclictest0-21swapper/705:03:327
468093991611,4cyclictest589235-21systemd-journal08:35:457
468090991614,2cyclictest845408-21kworker/u16:1+flush-8:006:28:046
468090991614,2cyclictest1057382-21kworker/u16:0+flush-8:008:42:416
468090991613,2cyclictest714623-21kworker/u16:5+flush-8:005:40:406
468083991614,2cyclictest562510-21kworker/u16:2+flush-8:004:18:314
46809399159,5cyclictest936283-21latency_hist07:12:417
46809399159,5cyclictest589235-21systemd-journal08:21:097
468093991514,0cyclictest0-21swapper/703:56:517
468093991510,4cyclictest589235-21systemd-journal08:42:417
468090991513,2cyclictest1106907-21kworker/u16:3+flush-8:008:43:046
468090991512,2cyclictest804662-21kworker/u16:3+flush-8:006:26:366
468083991513,2cyclictest804662-21kworker/u16:3+flush-8:006:12:574
468093991413,0cyclictest0-21swapper/705:55:587
468093991413,0cyclictest0-21swapper/704:37:577
468093991412,1cyclictest0-21swapper/705:35:577
46809399140,1cyclictest0-21swapper/707:28:077
46809099140,14cyclictest0-21swapper/603:38:026
468083991412,2cyclictest714623-21kworker/u16:5+events_unbound05:43:024
468083991412,2cyclictest1017159-21kworker/u16:4+events_unbound08:50:004
46809399139,3cyclictest96-21ksoftirqd/706:07:597
468093991313,0cyclictest0-21swapper/707:47:197
468093991312,0cyclictest0-21swapper/705:15:337
468093991310,2cyclictest754836-21latency_hist05:42:427
46809099139,3cyclictest512260-21kworker/u16:0+events_unbound03:43:106
468090991311,2cyclictest673790-21kworker/u16:0+flush-8:005:58:106
468090991310,3cyclictest562510-21kworker/u16:2+flush-8:004:22:576
46809099130,13cyclictest0-21swapper/608:48:076
468083991311,2cyclictest594180-21kworker/u16:0+flush-8:004:33:074
468083991311,2cyclictest594180-21kworker/u16:0+flush-8:004:33:074
468093991211,0cyclictest0-21swapper/706:33:307
468093991211,0cyclictest0-21swapper/705:22:507
468093991211,0cyclictest0-21swapper/704:33:197
468093991211,0cyclictest0-21swapper/704:33:197
468090991210,2cyclictest895798-21kworker/u16:0+flush-8:007:27:566
468090991210,2cyclictest714623-21kworker/u16:5+flush-8:005:28:086
468090991210,2cyclictest633379-21kworker/u16:3+flush-8:005:08:086
468090991210,2cyclictest417162-21kworker/u16:3+flush-8:003:58:086
468083991211,1cyclictest0-21swapper/406:44:144
46809399110,10cyclictest1127-21ntpd04:10:387
46809099119,2cyclictest845815-21kworker/u16:4+flush-8:007:07:586
46809099118,2cyclictest417162-21kworker/u16:3+flush-8:004:31:526
46809099117,3cyclictest996675-21kworker/u16:3+flush-8:008:03:076
468090991110,1cyclictest0-21swapper/606:42:286
468090991110,1cyclictest0-21swapper/606:34:456
468090991110,1cyclictest0-21swapper/604:17:176
468090991110,1cyclictest0-21swapper/603:35:306
46809099110,11cyclictest0-21swapper/606:14:536
46809099110,11cyclictest0-21swapper/604:58:446
46809099110,0cyclictest0-21swapper/604:51:016
46809399107,2cyclictest4175659-21snmpd08:06:177
46809099109,1cyclictest0-21swapper/608:08:046
46809099109,1cyclictest0-21swapper/607:45:136
46809099109,1cyclictest0-21swapper/607:40:106
46809099109,1cyclictest0-21swapper/607:35:086
46809099109,1cyclictest0-21swapper/605:55:116
46809099109,1cyclictest0-21swapper/603:55:326
46809099108,2cyclictest845408-21kworker/u16:1+flush-8:006:46:366
46809099107,2cyclictest1073989-21kworker/u16:1+flush-8:008:28:036
4680909998,1cyclictest0-21swapper/607:54:316
4680909997,2cyclictest1057382-21kworker/u16:0+flush-8:008:18:076
4680909987,1cyclictest0-21swapper/605:37:126
4680909986,2cyclictest845408-21kworker/u16:1+events_unbound06:57:576
4680909983,4cyclictest984965-21wsdd05:14:126
4680909983,4cyclictest938610-21cut07:13:006
4680909983,4cyclictest636327-21grep04:43:066
4680909983,4cyclictest469184-21sed03:22:586
4680909980,7cyclictest706267-21kthreadcore05:18:026
4680909973,3cyclictest796390-21cut06:02:576
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional