You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 00:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Sat Jul 05, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30687932920,1sleep70-21swapper/716:38:137
2925617996763,3cyclictest2905544-21kworker/u16:4+events_unbound15:42:065
2925617996763,3cyclictest2905544-21kworker/u16:4+events_unbound15:42:065
2925617995957,2cyclictest2874960-21kworker/u16:3+flush-8:015:45:325
292512025948,8sleep40-21swapper/415:30:564
292512025948,8sleep40-21swapper/415:30:564
292509425242,7sleep50-21swapper/515:30:335
292509425242,7sleep50-21swapper/515:30:335
2925617994543,2cyclictest3402683-21kworker/u16:1+events_unbound19:38:115
2765589410,33rtkit-daemon0-21swapper/615:28:536
2765589410,33rtkit-daemon0-21swapper/615:28:536
2925617994038,2cyclictest3515123-21kworker/u16:2+events_unbound20:18:075
2925617994038,2cyclictest3042321-21kworker/u16:2+events_unbound16:38:085
29163672403,31sleep70-21swapper/715:28:107
29163672403,31sleep70-21swapper/715:28:107
2925617993937,2cyclictest3227450-21kworker/u16:3+events_unbound18:53:085
2925612993935,3cyclictest2905544-21kworker/u16:4+events_unbound15:42:024
2925612993935,3cyclictest2905544-21kworker/u16:4+events_unbound15:42:024
30085052340,0chrt0-21swapper/516:08:145
2925624993432,2cyclictest3042321-21kworker/u16:2+events_unbound16:25:276
2925617993432,2cyclictest3309723-21kworker/u16:0+events_unbound18:38:115
2925617993331,2cyclictest3380965-21kworker/u16:0+events_unbound19:23:105
292562799320,31cyclictest3453849-21latency_hist19:47:527
2925624993230,2cyclictest2905544-21kworker/u16:4+events_unbound15:42:166
2925624993230,2cyclictest2905544-21kworker/u16:4+events_unbound15:42:166
2925617993230,2cyclictest3535681-21kworker/u16:4+flush-8:020:33:085
292562799310,29cyclictest3227773-21latency_hist17:57:537
2925624992927,2cyclictest2949767-21kworker/u16:5+flush-8:016:07:536
2925624992927,2cyclictest2949767-21kworker/u16:5+flush-8:016:07:536
292562799270,25cyclictest3402461-21latency_hist19:22:537
34495372260,0chrt0-21swapper/619:43:136
2925617992523,2cyclictest3535681-21kworker/u16:4+flush-8:020:38:235
2925617992421,2cyclictest2874960-21kworker/u16:3+flush-8:015:52:545
2925617992321,2cyclictest3402683-21kworker/u16:1+flush-8:019:58:195
2925617992321,2cyclictest3402683-21kworker/u16:1+flush-8:019:58:195
2925617992321,2cyclictest3350154-21kworker/u16:1+flush-8:019:07:475
2925617992220,2cyclictest3402683-21kworker/u16:1+flush-8:019:57:475
2925617992220,2cyclictest3402683-21kworker/u16:1+flush-8:019:52:475
2925617992220,2cyclictest3258559-21kworker/u16:2+flush-8:018:12:495
2925617992220,2cyclictest3177058-21kworker/u16:0+flush-8:017:38:085
2925617992220,2cyclictest3177058-21kworker/u16:0+flush-8:017:38:085
2925617992220,2cyclictest2874960-21kworker/u16:3+flush-8:017:12:485
2925612992219,2cyclictest3197165-21kworker/u16:1+flush-8:018:03:124
2925617992119,2cyclictest3227450-21kworker/u16:3+events_unbound19:13:105
2925617992119,2cyclictest2971422-21kworker/u16:0+events_unbound16:03:545
2925617992119,2cyclictest2971422-21kworker/u16:0+events_unbound16:03:545
2925612992117,3cyclictest3402683-21kworker/u16:1+flush-8:019:48:114
2925627992019,0cyclictest0-21swapper/719:58:107
2925627992019,0cyclictest0-21swapper/719:58:107
2925617992018,2cyclictest3380965-21kworker/u16:0+events_unbound19:33:085
2925612992018,2cyclictest2874960-21kworker/u16:3+flush-8:017:08:084
2925627991919,0cyclictest0-21swapper/718:46:047
2925627991918,0cyclictest0-21swapper/718:38:277
2925627991918,0cyclictest0-21swapper/717:48:287
292562799190,17cyclictest3566831-21latency_hist20:42:527
292562799190,17cyclictest3464114-21latency_hist19:52:527
292562799190,17cyclictest3146209-21exim417:17:547
2925617991916,2cyclictest3177058-21kworker/u16:0+flush-8:017:33:035
342296721816,1sleep70-21swapper/719:32:337
2925627991818,0cyclictest0-21swapper/715:42:007
2925627991818,0cyclictest0-21swapper/715:42:007
2925627991817,0cyclictest0-21swapper/720:49:157
2925627991817,0cyclictest0-21swapper/719:07:157
2925627991817,0cyclictest0-21swapper/718:53:047
292562799180,16cyclictest643-21systemd-journal16:02:537
292562799180,16cyclictest1313-21snmpd17:06:437
292562799180,16cyclictest1313-21snmpd16:08:217
2925624991815,3cyclictest3587332-21kworker/u16:2+flush-8:020:53:236
2925617991816,2cyclictest3535681-21kworker/u16:4+flush-8:020:53:265
2925617991816,2cyclictest3402683-21kworker/u16:1+flush-8:019:43:155
2925617991816,2cyclictest3309723-21kworker/u16:0+flush-8:018:37:485
2925617991816,2cyclictest3227450-21kworker/u16:3+flush-8:018:27:485
2925617991816,2cyclictest3227450-21kworker/u16:3+flush-8:018:27:485
2925617991816,2cyclictest3062262-21kworker/u16:4+events_unbound16:57:485
2925617991816,2cyclictest2874960-21kworker/u16:3+events_unbound16:23:115
2925617991815,2cyclictest3380965-21kworker/u16:0+events_unbound20:58:075
2925612991816,2cyclictest3104769-21kworker/u16:2+events_unbound17:48:114
2925627991716,0cyclictest0-21swapper/719:39:087
2925627991716,0cyclictest0-21swapper/719:25:157
2925627991716,0cyclictest0-21swapper/718:16:537
2925627991716,0cyclictest0-21swapper/717:12:167
2925617991715,2cyclictest3485355-21kworker/u16:3+events_unbound20:28:155
2925617991715,2cyclictest3463891-21kworker/u16:2+events_unbound20:03:165
2925617991715,2cyclictest3104769-21kworker/u16:2+flush-8:017:52:485
2925617991715,2cyclictest2990274-21kworker/u16:1+events_unbound17:03:165
2925617991715,2cyclictest2874960-21kworker/u16:3+flush-8:016:13:115
2925612991715,2cyclictest3268530-21kworker/u16:4+events_unbound18:23:104
2925612991715,2cyclictest3268530-21kworker/u16:4+events_unbound18:23:104
292562799163,11cyclictest2925601-21cyclictest16:03:197
292562799163,11cyclictest2925601-21cyclictest16:03:197
2925627991615,0cyclictest0-21swapper/718:59:377
2925627991615,0cyclictest0-21swapper/716:13:287
2925627991613,2cyclictest3240014-21kthreadcore18:03:127
2925627991613,2cyclictest3166027-21awk17:27:527
2925612991614,2cyclictest3320283-21kworker/u16:2+events_unbound18:58:114
2925612991613,2cyclictest3197165-21kworker/u16:1+flush-8:018:22:534
292561299160,1cyclictest0-21swapper/418:33:204
2925627991514,0cyclictest0-21swapper/720:04:527
2925627991514,0cyclictest0-21swapper/719:12:177
2925627991514,0cyclictest0-21swapper/716:32:047
292562799150,14cyclictest3515421-21latency_hist20:17:537
2925624991512,2cyclictest3168375-21kthreadcore17:28:136
2925617991513,2cyclictest3380965-21kworker/u16:0+events_unbound19:28:135
2925617991513,2cyclictest2971422-21kworker/u16:0+flush-8:015:58:285
292561299150,6cyclictest0-21swapper/415:58:094
292562799144,9cyclictest2925719-21cyclictest16:53:197
292562799140,2cyclictest2950845-21PK-Backend15:45:237
292562799140,1cyclictest0-21swapper/716:48:207
2925624991413,1cyclictest0-21swapper/618:37:546
2925624991412,2cyclictest2949766-21kworker/u16:2+events_unbound15:45:306
2925624991411,3cyclictest3535681-21kworker/u16:4+flush-8:020:33:206
292562499140,1cyclictest0-21swapper/615:33:186
292562499140,14cyclictest0-21swapper/617:18:086
2925617991413,1cyclictest0-21swapper/519:18:165
2925617991412,2cyclictest3485355-21kworker/u16:3+events_unbound20:23:165
2925617991412,2cyclictest3380965-21kworker/u16:0+flush-8:020:45:025
2925617991412,2cyclictest3227450-21kworker/u16:3+flush-8:017:57:485
2925617991412,2cyclictest3227450-21kworker/u16:3+events_unbound19:09:535
2925617991412,2cyclictest3197165-21kworker/u16:1+flush-8:017:43:175
2925617991412,2cyclictest3062262-21kworker/u16:4+events_unbound16:43:205
2925617991412,2cyclictest2874960-21kworker/u16:3+events_unbound17:24:545
292561799140,14cyclictest0-21swapper/515:48:245
2925612991413,1cyclictest0-21swapper/415:43:134
292561299140,13cyclictest0-21swapper/420:33:074
292561299140,13cyclictest0-21swapper/417:13:114
292562799134,8cyclictest2925719-21cyclictest18:28:177
292562799134,7cyclictest2925719-21cyclictest17:28:127
292562799133,9cyclictest2925601-21cyclictest20:58:117
2925627991313,0cyclictest0-21swapper/719:16:207
2925627991312,0cyclictest0-21swapper/717:41:047
2925627991312,0cyclictest0-21swapper/717:41:047
2925627991312,0cyclictest0-21swapper/717:33:187
2925627991311,1cyclictest3063201-21turbostat16:37:527
292562799130,12cyclictest3258615-21latency_hist18:12:537
292562799130,11cyclictest643-21systemd-journal18:52:537
292562799130,11cyclictest0-21swapper/716:23:187
292562499139,3cyclictest3239558-21cat18:03:106
2925624991312,1cyclictest0-21swapper/619:42:416
2925624991311,1cyclictest0-21swapper/617:23:106
292561799138,4cyclictest3136733-21cat17:13:115
2925617991311,2cyclictest3268530-21kworker/u16:4+events_unbound18:18:195
2925617991311,2cyclictest3227450-21kworker/u16:3+flush-8:018:03:035
2925617991311,2cyclictest3197165-21kworker/u16:1+flush-8:018:02:485
2925617991311,2cyclictest3134271-21kworker/u16:4+flush-8:017:18:175
2925612991310,2cyclictest3248073-21kworker/u16:0+events_unbound18:13:084
292562799126,5cyclictest27177-21gmain18:18:237
292562799124,0cyclictest0-21swapper/717:43:077
292562799123,8cyclictest2925719-21cyclictest20:08:127
292562799120,11cyclictest96-21ksoftirqd/716:43:237
292562799120,11cyclictest1313-21snmpd20:56:347
292562799120,11cyclictest1313-21snmpd20:45:167
292562799120,11cyclictest1313-21snmpd20:19:487
292562799120,10cyclictest2935724-21latency_hist15:37:547
292562499129,2cyclictest3104769-21kworker/u16:2+flush-8:017:43:086
292562499128,3cyclictest3054459-21cat16:33:106
2925624991211,1cyclictest0-21swapper/618:28:196
2925624991211,1cyclictest0-21swapper/618:17:536
2925624991211,1cyclictest0-21swapper/616:38:526
2925624991211,1cyclictest0-21swapper/616:20:356
2925624991210,2cyclictest3485355-21kworker/u16:3+flush-8:020:43:076
292562499120,1cyclictest0-21swapper/620:14:136
292562499120,1cyclictest0-21swapper/619:21:156
292562499120,1cyclictest0-21swapper/617:59:356
2925617991210,2cyclictest3320283-21kworker/u16:2+events_unbound18:43:085
2925617991210,2cyclictest2990274-21kworker/u16:1+events_unbound16:18:195
292561799120,1cyclictest0-21swapper/520:08:595
292561299124,8cyclictest0-21swapper/415:56:084
2925612991211,1cyclictest0-21swapper/416:42:114
292561299120,1cyclictest0-21swapper/419:46:004
292561299120,1cyclictest0-21swapper/417:03:474
292561299120,12cyclictest0-21swapper/420:50:284
292561299120,12cyclictest0-21swapper/420:45:264
292561299120,12cyclictest0-21swapper/420:26:024
292561299120,12cyclictest0-21swapper/419:22:024
292562799114,7cyclictest2925601-21cyclictest20:33:187
292562799112,8cyclictest0-21swapper/717:58:137
292562799111,9cyclictest2925719-21cyclictest17:16:587
292562799111,1cyclictest2972600-21sshd-session15:53:117
292562799110,4cyclictest3535884-21sshd-session20:27:157
292562799110,11cyclictest951ktimers/720:28:197
292562799110,10cyclictest96-21ksoftirqd/716:58:187
292562799110,10cyclictest1313-21snmpd19:55:457
292562799110,10cyclictest1313-21snmpd19:32:597
292562799110,10cyclictest1313-21snmpd16:20:457
292562499119,2cyclictest3062262-21kworker/u16:4+events_unbound16:52:556
2925624991110,1cyclictest0-21swapper/620:48:456
2925624991110,1cyclictest0-21swapper/620:23:316
2925624991110,1cyclictest0-21swapper/619:35:176
2925624991110,1cyclictest0-21swapper/619:31:586
2925624991110,1cyclictest0-21swapper/619:07:046
2925624991110,1cyclictest0-21swapper/618:57:556
2925624991110,1cyclictest0-21swapper/618:23:426
2925624991110,1cyclictest0-21swapper/618:23:426
2925624991110,1cyclictest0-21swapper/618:21:116
2925624991110,1cyclictest0-21swapper/617:41:286
2925624991110,1cyclictest0-21swapper/617:41:286
2925624991110,1cyclictest0-21swapper/617:15:366
2925624991110,1cyclictest0-21swapper/616:52:456
292562499110,1cyclictest0-21swapper/620:01:466
292562499110,1cyclictest0-21swapper/620:01:466
292562499110,1cyclictest0-21swapper/619:55:096
292562499110,11cyclictest0-21swapper/620:21:476
292562499110,11cyclictest0-21swapper/618:35:226
292562499110,11cyclictest0-21swapper/618:09:316
292562499110,11cyclictest0-21swapper/617:48:526
292562499110,11cyclictest0-21swapper/617:02:126
292562499110,11cyclictest0-21swapper/616:12:426
292562499110,11cyclictest0-21swapper/615:56:006
292561799119,2cyclictest3227450-21kworker/u16:3+flush-8:018:58:175
292561799119,2cyclictest3227450-21kworker/u16:3+events_unbound18:13:225
292561799119,2cyclictest3083852-21kworker/u16:0+flush-8:016:51:045
292561799119,2cyclictest2874960-21kworker/u16:3+flush-8:016:31:165
292561799119,2cyclictest2867992-21kworker/u16:1+events_unbound15:33:075
2925617991110,1cyclictest0-21swapper/520:14:495
2925617991110,1cyclictest0-21swapper/517:02:395
292561799110,5cyclictest0-21swapper/518:48:105
292561799110,11cyclictest0-21swapper/518:29:125
292561299116,4cyclictest3465637-21cat19:53:104
2925612991110,1cyclictest0-21swapper/421:02:464
2925612991110,1cyclictest0-21swapper/420:40:234
2925612991110,1cyclictest0-21swapper/418:56:114
2925612991110,1cyclictest0-21swapper/418:38:414
2925612991110,1cyclictest0-21swapper/418:01:574
2925612991110,1cyclictest0-21swapper/417:39:434
2925612991110,1cyclictest0-21swapper/417:39:434
2925612991110,1cyclictest0-21swapper/417:35:374
2925612991110,1cyclictest0-21swapper/416:58:444
2925612991110,1cyclictest0-21swapper/416:53:324
2925612991110,1cyclictest0-21swapper/415:35:494
292561299110,11cyclictest0-21swapper/420:04:174
292561299110,11cyclictest0-21swapper/419:32:454
292561299110,11cyclictest0-21swapper/416:33:314
292562799104,5cyclictest27763-21gvfs-afc-volume15:48:177
292562799100,1cyclictest0-21swapper/718:33:087
292562799100,1cyclictest0-21swapper/718:23:117
292562799100,1cyclictest0-21swapper/718:23:117
292562499109,1cyclictest0-21swapper/620:10:546
292562499109,1cyclictest0-21swapper/620:04:176
292562499109,1cyclictest0-21swapper/619:51:506
292562499108,2cyclictest2874960-21kworker/u16:3+flush-8:017:08:246
292562499100,4cyclictest0-21swapper/619:16:076
292561799108,1cyclictest0-21swapper/516:33:165
292561799107,2cyclictest3380965-21kworker/u16:0+events_unbound20:47:555
292561799105,4cyclictest3167542-21cat17:28:115
292561299109,1cyclictest0-21swapper/418:11:064
292561299109,1cyclictest0-21swapper/416:18:414
292561299106,3cyclictest3516958-21cat20:18:084
292561299100,9cyclictest0-21swapper/416:23:104
29256249998,1cyclictest3013082-21fschecks_count16:13:106
29256249998,1cyclictest0-21swapper/621:01:216
29256249998,1cyclictest0-21swapper/617:56:176
29256249998,1cyclictest0-21swapper/616:47:336
29256249998,1cyclictest0-21swapper/615:51:356
29256249994,5cyclictest0-21swapper/617:33:096
29256129998,1cyclictest0-21swapper/419:42:414
29256129998,1cyclictest0-21swapper/417:19:424
29256129998,1cyclictest0-21swapper/416:28:184
29256129998,1cyclictest0-21swapper/416:04:404
29256129998,1cyclictest0-21swapper/416:04:404
29256129997,2cyclictest2874960-21kworker/u16:3+flush-8:017:23:194
29256129994,4cyclictest3432327-21ntpq19:33:204
29256129994,4cyclictest3366137-21cut19:03:134
29256129990,8cyclictest0-21swapper/416:43:104
29256129990,1cyclictest0-21swapper/420:28:224
29256249987,1cyclictest0-21swapper/619:23:466
29256249983,4cyclictest3565641-21sed20:38:156
29256249983,4cyclictest3116314-21sed17:03:136
29256249980,8cyclictest0-21swapper/618:53:086
29256129987,1cyclictest0-21swapper/419:24:334
29256129986,1cyclictest0-21swapper/419:58:084
29256129986,1cyclictest0-21swapper/419:58:084
29256129986,1cyclictest0-21swapper/417:43:134
29256129986,1cyclictest0-21swapper/415:48:124
29256129984,3cyclictest3496455-21ip_192.168.115.20:08:124
29256129984,3cyclictest3373019-21tune2fs19:08:074
29256129984,3cyclictest3084534-21/usr/sbin/munin16:48:114
29256129983,4cyclictest3507710-21tr20:13:134
29256129983,4cyclictest3386779-21cut19:13:144
29256129983,4cyclictest3323745-21cut18:43:124
29256129983,4cyclictest3294580-21cut18:28:134
29256129983,4cyclictest3016247-21cut16:13:144
29256129983,4cyclictest3003869-21cat16:08:114
29256249975,1cyclictest0-21swapper/618:48:146
29256249975,1cyclictest0-21swapper/615:58:126
29256249974,2cyclictest3537915-21kthreadcore20:28:116
29256249974,2cyclictest3045271-21kthreadcore16:28:126
29256249973,3cyclictest3374842-21cat19:08:116
29256249973,3cyclictest3321139-21cat18:43:066
29256129975,1cyclictest0-21swapper/417:53:074
29256129973,3cyclictest3588706-21tune2fs20:53:114
29256129972,4cyclictest3331351-21cpu18:48:074
29256129970,2cyclictest0-21swapper/417:28:084
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional