You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-16 - 01:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Wed Jul 16, 2025 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27096922700,1sleep50-21swapper/507:32:105
221132326555,7sleep60-21swapper/603:29:276
221137925848,7sleep50-21swapper/503:30:145
221149425240,8sleep40-21swapper/403:31:494
22113242439,30sleep70-21swapper/703:29:287
2211819993533,2cyclictest2221083-21kworker/u16:4+events_unbound05:02:114
221182899340,32cyclictest2287753-21latency_hist04:07:117
221182899320,30cyclictest2246629-21latency_hist03:47:117
221182899280,26cyclictest2637616-21latency_hist06:57:117
2211822992725,2cyclictest2699030-21kworker/u16:0+flush-8:008:39:415
2211819992523,2cyclictest1466972-21kworker/u16:5+flush-8:003:32:304
221182899240,22cyclictest643-21systemd-journal06:12:117
2211819992321,2cyclictest2740163-21kworker/u16:1+flush-8:008:32:104
2211822992220,2cyclictest2221083-21kworker/u16:4+events_unbound04:42:295
2211828992119,1cyclictest2678727-21awk07:17:107
2211822992119,2cyclictest2221083-21kworker/u16:4+flush-8:004:12:425
221182899200,19cyclictest2606749-21latency_hist06:42:107
221182899200,18cyclictest2709614-21latency_hist07:32:107
2211822992018,2cyclictest2400737-21kworker/u16:2+flush-8:006:07:425
221182899190,17cyclictest1313-21snmpd05:44:557
2211823991915,3cyclictest2213194-21cat03:32:286
2211819991917,2cyclictest2411058-21kworker/u16:3+events_unbound05:22:334
2211819991916,3cyclictest2221083-21kworker/u16:4+flush-8:004:37:114
2211819991916,2cyclictest2853394-21users08:37:454
2211822991816,2cyclictest2729829-21kworker/u16:4+events_unbound08:02:385
2211822991814,3cyclictest2596231-21kworker/u16:4+flush-8:007:02:365
2211822991814,3cyclictest2523512-21kworker/u16:3+flush-8:006:42:285
2211822991814,3cyclictest2523512-21kworker/u16:3+flush-8:006:42:285
2211819991816,2cyclictest2699030-21kworker/u16:0+events_unbound07:37:414
2211819991815,2cyclictest2842529-21ntpq08:32:384
2211828991716,0cyclictest0-21swapper/707:47:347
221182899170,16cyclictest643-21systemd-journal05:02:057
221182899170,16cyclictest2575931-21latency_hist06:27:117
221182399170,17cyclictest0-21swapper/606:27:366
2211822991715,2cyclictest2596231-21kworker/u16:4+events_unbound07:07:285
2211819991716,1cyclictest0-21swapper/408:47:594
2211819991715,2cyclictest2493392-21kworker/u16:0+flush-8:005:52:294
2211819991715,2cyclictest2493392-21kworker/u16:0+flush-8:005:52:294
2211819991715,2cyclictest2482428-21kworker/u16:4+flush-8:005:52:104
2211819991715,2cyclictest2145064-21kworker/4:204:12:334
2211823991613,2cyclictest2805496-21tr08:17:306
2211823991613,2cyclictest2805496-21tr08:17:306
221182399160,2cyclictest2605702-21sed06:37:336
2211819991615,1cyclictest0-21swapper/409:01:184
2211819991615,1cyclictest0-21swapper/408:55:094
2211819991615,1cyclictest0-21swapper/408:42:524
2211819991615,1cyclictest0-21swapper/408:23:254
2211819991615,1cyclictest0-21swapper/408:17:164
2211819991615,1cyclictest0-21swapper/408:17:164
2211819991615,1cyclictest0-21swapper/408:10:064
2211819991615,1cyclictest0-21swapper/408:04:594
2211819991615,1cyclictest0-21swapper/407:54:444
2211819991615,1cyclictest0-21swapper/407:45:314
2211819991615,1cyclictest0-21swapper/407:33:144
2211819991615,1cyclictest0-21swapper/407:29:084
2211819991615,1cyclictest0-21swapper/407:24:014
2211819991615,1cyclictest0-21swapper/405:27:174
2211819991615,1cyclictest0-21swapper/405:19:064
2211819991615,1cyclictest0-21swapper/405:13:594
2211819991615,1cyclictest0-21swapper/405:03:444
2211819991615,1cyclictest0-21swapper/404:52:294
2211819991615,1cyclictest0-21swapper/404:49:244
2211819991615,1cyclictest0-21swapper/404:42:144
2211819991615,1cyclictest0-21swapper/404:28:564
2211819991615,1cyclictest0-21swapper/404:17:404
2211819991615,1cyclictest0-21swapper/404:17:404
2211819991615,1cyclictest0-21swapper/404:07:254
2211819991615,1cyclictest0-21swapper/404:03:204
2211819991615,1cyclictest0-21swapper/403:56:104
2211819991615,1cyclictest0-21swapper/403:44:544
2211819991615,1cyclictest0-21swapper/403:38:454
2211819991614,2cyclictest2523512-21kworker/u16:3+flush-8:006:52:324
2211828991514,0cyclictest0-21swapper/707:20:427
2211828991514,0cyclictest0-21swapper/707:02:347
2211828991513,1cyclictest0-21swapper/705:07:337
2211828991513,1cyclictest0-21swapper/705:07:337
221182899150,13cyclictest2267925-21sendmail03:57:127
221182399150,1cyclictest0-21swapper/606:17:286
2211819991514,1cyclictest0-21swapper/407:58:504
2211819991512,3cyclictest2523512-21kworker/u16:3+events_unbound06:22:404
2211819991512,2cyclictest2400737-21kworker/u16:2+events_unbound06:37:304
2211828991413,0cyclictest0-21swapper/707:07:417
2211828991412,1cyclictest0-21swapper/706:02:327
221182899140,4cyclictest2792002-21/usr/sbin/munin08:12:107
221182899140,1cyclictest0-21swapper/704:22:387
221182899140,13cyclictest1313-21snmpd09:02:037
221182899140,13cyclictest1313-21snmpd04:32:067
2211823991411,2cyclictest0-21swapper/606:02:336
2211822991411,2cyclictest0-21swapper/505:17:295
221182299140,13cyclictest0-21swapper/503:52:145
221181999142,5cyclictest0-21swapper/405:07:294
221181999142,5cyclictest0-21swapper/405:07:294
2211819991413,1cyclictest0-21swapper/404:40:114
2211828991312,0cyclictest0-21swapper/708:32:307
2211828991312,0cyclictest0-21swapper/704:37:297
2211828991312,0cyclictest0-21swapper/703:37:267
2211828991311,1cyclictest2483352-21munin-run05:42:107
2211828991311,1cyclictest0-21swapper/708:42:267
2211828991311,1cyclictest0-21swapper/708:17:277
2211828991311,1cyclictest0-21swapper/708:17:277
2211828991311,1cyclictest0-21swapper/707:42:307
2211828991311,1cyclictest0-21swapper/707:32:237
221182899130,12cyclictest96-21ksoftirqd/708:37:447
221182899130,11cyclictest1313-21snmpd06:36:457
221182899130,11cyclictest1313-21snmpd04:56:367
221182399130,1cyclictest0-21swapper/608:49:586
2211822991312,1cyclictest0-21swapper/506:56:415
2211822991312,1cyclictest0-21swapper/505:29:495
2211822991312,1cyclictest0-21swapper/505:03:205
2211822991312,1cyclictest0-21swapper/503:57:355
2211822991311,2cyclictest2575603-21kworker/u16:0+events_unbound06:47:305
2211822991311,2cyclictest2534957-21kworker/u16:1+events_unbound06:12:425
221182299130,1cyclictest0-21swapper/507:49:105
221182299130,13cyclictest0-21swapper/507:14:205
2211819991312,1cyclictest0-21swapper/404:00:124
2211819991312,1cyclictest0-21swapper/403:47:584
221181999130,13cyclictest0-21swapper/406:58:494
221181999130,13cyclictest0-21swapper/406:29:454
221181999130,13cyclictest0-21swapper/405:36:224
221182899126,5cyclictest27763-21gvfs-afc-volume04:17:287
221182899126,5cyclictest27763-21gvfs-afc-volume04:17:287
221182899125,6cyclictest27642-21gmain06:47:417
221182899124,7cyclictest27904-21gmain04:32:257
2211828991212,0cyclictest0-21swapper/708:02:317
2211828991212,0cyclictest0-21swapper/705:27:317
2211828991212,0cyclictest0-21swapper/703:32:317
2211828991211,1cyclictest0-21swapper/708:52:327
2211828991211,0cyclictest0-21swapper/707:57:357
2211828991211,0cyclictest0-21swapper/707:41:217
2211828991211,0cyclictest0-21swapper/706:59:437
2211828991211,0cyclictest0-21swapper/705:47:507
2211828991211,0cyclictest0-21swapper/705:32:287
2211828991211,0cyclictest0-21swapper/703:52:177
2211828991210,1cyclictest0-21swapper/706:12:267
2211828991210,1cyclictest0-21swapper/705:12:287
2211828991210,1cyclictest0-21swapper/704:12:337
2211828991210,1cyclictest0-21swapper/704:07:317
221182899120,1cyclictest0-21swapper/705:02:397
221182899120,11cyclictest1313-21snmpd04:43:067
221182899120,10cyclictest1313-21snmpd08:12:267
221182899120,10cyclictest1313-21snmpd05:54:057
221182899120,10cyclictest1313-21snmpd05:54:057
221182399124,7cyclictest0-21swapper/607:07:286
2211823991211,1cyclictest0-21swapper/606:09:206
2211823991211,1cyclictest0-21swapper/603:43:126
221182399120,1cyclictest0-21swapper/606:56:286
221182399120,1cyclictest0-21swapper/605:37:106
221182399120,1cyclictest0-21swapper/604:22:466
2211822991211,1cyclictest0-21swapper/508:29:035
2211822991211,1cyclictest0-21swapper/508:00:505
2211822991211,1cyclictest0-21swapper/505:52:505
2211822991211,1cyclictest0-21swapper/505:52:505
2211822991211,1cyclictest0-21swapper/504:33:235
221182299120,1cyclictest0-21swapper/504:28:205
221182299120,1cyclictest0-21swapper/504:11:095
221182299120,12cyclictest0-21swapper/507:01:055
221182299120,12cyclictest0-21swapper/503:34:165
2211819991211,1cyclictest0-21swapper/408:13:104
2211819991211,1cyclictest0-21swapper/404:23:484
221181999120,1cyclictest0-21swapper/406:36:174
221181999120,12cyclictest0-21swapper/407:19:474
221181999120,12cyclictest0-21swapper/407:15:224
221181999120,12cyclictest0-21swapper/407:10:574
221181999120,12cyclictest0-21swapper/406:43:044
221181999120,12cyclictest0-21swapper/406:43:044
221181999120,12cyclictest0-21swapper/406:03:084
221181999120,12cyclictest0-21swapper/405:58:364
221181999120,12cyclictest0-21swapper/405:38:334
221182899119,1cyclictest2761134-21basename07:57:117
221182899119,1cyclictest0-21swapper/705:57:297
221182899115,5cyclictest27763-21gvfs-afc-volume06:27:127
221182899114,6cyclictest27642-21gmain06:42:217
221182899114,6cyclictest27642-21gmain06:42:217
221182899112,2cyclictest2441847-21sshd-session05:18:477
2211828991110,0cyclictest0-21swapper/707:22:327
221182899110,9cyclictest0-21swapper/708:47:237
221182899110,1cyclictest0-21swapper/708:22:357
221182899110,10cyclictest2832216-21ntp_offset08:27:507
2211823991110,1cyclictest0-21swapper/607:56:516
2211823991110,1cyclictest0-21swapper/607:21:326
2211823991110,1cyclictest0-21swapper/605:20:566
2211823991110,1cyclictest0-21swapper/604:36:386
2211823991110,1cyclictest0-21swapper/604:28:456
2211823991110,1cyclictest0-21swapper/604:21:496
2211823991110,1cyclictest0-21swapper/604:21:496
2211823991110,1cyclictest0-21swapper/604:12:406
2211823991110,1cyclictest0-21swapper/603:56:266
221182399110,1cyclictest0-21swapper/605:27:246
221182399110,1cyclictest0-21swapper/604:47:216
221182399110,11cyclictest0-21swapper/609:02:076
221182399110,11cyclictest0-21swapper/608:44:566
221182399110,11cyclictest0-21swapper/608:31:516
221182399110,11cyclictest0-21swapper/607:26:546
221182399110,11cyclictest0-21swapper/606:59:476
221182299114,6cyclictest0-21swapper/506:02:285
2211822991110,1cyclictest0-21swapper/508:53:295
2211822991110,1cyclictest0-21swapper/508:49:245
2211822991110,1cyclictest0-21swapper/507:40:305
2211822991110,1cyclictest0-21swapper/506:21:225
2211822991110,1cyclictest0-21swapper/505:22:155
2211822991110,1cyclictest0-21swapper/504:50:055
2211822991110,1cyclictest0-21swapper/503:44:115
221182299110,1cyclictest0-21swapper/508:34:065
221182299110,11cyclictest0-21swapper/508:45:185
221182299110,11cyclictest0-21swapper/504:59:525
221182299110,11cyclictest0-21swapper/504:26:175
221182299110,11cyclictest0-21swapper/504:20:275
221182299110,11cyclictest0-21swapper/504:20:275
221182299110,10cyclictest0-21swapper/504:52:295
221181999117,3cyclictest2546342-21tune2fs06:12:284
2211819991110,1cyclictest0-21swapper/406:47:474
221182899108,1cyclictest0-21swapper/704:47:257
221182899100,3cyclictest1313-21snmpd06:17:337
221182899100,1cyclictest0-21swapper/705:22:337
221182899100,1cyclictest0-21swapper/703:47:397
221182399109,1cyclictest0-21swapper/608:10:156
221182399109,1cyclictest0-21swapper/605:57:596
221182399109,1cyclictest0-21swapper/603:52:016
221182399105,4cyclictest2618999-21grep06:47:316
221182399100,1cyclictest0-21swapper/608:12:376
221182399100,1cyclictest0-21swapper/607:12:326
221182399100,10cyclictest0-21swapper/605:07:266
221182399100,10cyclictest0-21swapper/605:07:266
221182399100,10cyclictest0-21swapper/604:37:376
221182299109,1cyclictest0-21swapper/508:57:355
221182299109,1cyclictest0-21swapper/508:25:545
221182299109,1cyclictest0-21swapper/508:17:335
221182299109,1cyclictest0-21swapper/508:17:335
221182299109,1cyclictest0-21swapper/508:11:145
221182299109,1cyclictest0-21swapper/507:52:295
221182299109,1cyclictest0-21swapper/507:19:515
221182299109,1cyclictest0-21swapper/505:58:405
221182299109,1cyclictest0-21swapper/505:36:265
221182299109,1cyclictest0-21swapper/505:16:445
221182299109,1cyclictest0-21swapper/505:09:005
221182299109,1cyclictest0-21swapper/505:09:005
221182299100,9cyclictest0-21swapper/506:32:325
221181999109,1cyclictest0-21swapper/407:47:344
221181999109,1cyclictest0-21swapper/405:46:564
22118239998,1cyclictest0-21swapper/608:33:546
22118239998,1cyclictest0-21swapper/607:50:326
22118239998,1cyclictest0-21swapper/607:45:116
22118239998,1cyclictest0-21swapper/605:54:316
22118239998,1cyclictest0-21swapper/605:54:316
22118239998,1cyclictest0-21swapper/605:23:086
22118239994,4cyclictest2710926-21tune2fs07:32:246
22118239993,4cyclictest2381237-21tr04:52:266
22118229998,1cyclictest0-21swapper/508:14:245
22118229998,1cyclictest0-21swapper/507:42:425
22118229998,1cyclictest0-21swapper/505:47:005
22118229998,1cyclictest0-21swapper/505:41:105
22118229998,1cyclictest0-21swapper/504:40:195
22118229997,2cyclictest2699030-21kworker/u16:0+flush-8:007:32:415
22118199997,2cyclictest2446645-21kworker/u16:5+flush-8:006:07:304
22118199995,3cyclictest2649234-21tune2fs07:02:244
22118239987,1cyclictest0-21swapper/608:24:456
22118239987,1cyclictest0-21swapper/607:29:066
22118239987,1cyclictest0-21swapper/606:22:446
22118239987,1cyclictest0-21swapper/605:46:286
22118239987,1cyclictest0-21swapper/605:12:166
22118239985,2cyclictest0-21swapper/604:02:306
22118239984,3cyclictest2879713-21cut08:52:326
22118239984,3cyclictest2394302-21cat04:57:306
22118239984,3cyclictest2268444-21expr03:57:246
22118239983,4cyclictest2849603-21cut08:37:356
22118239983,4cyclictest2763347-21cut07:57:286
22118239983,4cyclictest2654375-21kthreadcore07:02:306
22118239983,4cyclictest2590136-21cut06:32:356
22118239983,4cyclictest2410071-21missed_timers05:02:376
22118239980,7cyclictest0-21swapper/607:37:286
22118239980,1cyclictest0-21swapper/606:12:176
22118229987,1cyclictest0-21swapper/503:47:495
22118229983,4cyclictest2578984-21cut06:27:305
22118229983,4cyclictest2568540-21grep06:22:295
22118229983,4cyclictest2231120-21cut03:37:335
22118199986,1cyclictest0-21swapper/406:17:294
22118239975,1cyclictest0-21swapper/608:02:256
22118239975,1cyclictest0-21swapper/606:42:296
22118239975,1cyclictest0-21swapper/606:42:296
22118239974,2cyclictest2288591-21cpuspeed_turbos04:07:256
22118239974,2cyclictest2223126-21timerwakeupswit03:37:126
22118239973,3cyclictest2496106-21tr05:47:356
22118239973,3cyclictest2475113-21kthreadcore05:37:296
22118239971,2cyclictest681rcuog/404:42:346
22118229973,3cyclictest2494906-21entropy05:47:315
22118229973,3cyclictest2278431-21tr04:02:245
22118229972,4cyclictest2691513-21cut07:22:305
22118229972,4cyclictest2597784-21fschecks_count06:37:255
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional