You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 02:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Dec 03, 2025 00:50:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3132029083,4sleep60-21swapper/619:05:156
113127954,19sleep40-21swapper/419:08:034
108027837,29sleep10-21swapper/119:07:191
107727844,30sleep70-21swapper/719:07:167
103227643,29sleep00-21swapper/019:06:380
118727339,29sleep30-21swapper/319:08:503
108327339,30sleep20-21swapper/219:07:192
115027234,6sleep50-21swapper/519:08:205
1711995453,0cyclictest0-21swapper/223:05:222
1711995410,44cyclictest0-21swapper/200:10:062
124552540,0sleep60-21swapper/622:44:536
1712995351,1cyclictest0-21swapper/300:20:093
1712995310,0cyclictest0-21swapper/321:20:223
119432510,0sleep10-21swapper/121:25:451
1716995010,40cyclictest0-21swapper/700:20:417
1710995050,0cyclictest0-21swapper/121:55:221
75382490,0sleep40-21swapper/421:40:064
1713994747,0cyclictest0-21swapper/423:35:044
1709994747,0cyclictest0-21swapper/021:35:200
1709994747,0cyclictest0-21swapper/000:05:150
1713994644,2cyclictest0-21swapper/423:45:004
1713994644,1cyclictest0-21swapper/400:16:204
1712994645,1cyclictest0-21swapper/300:16:203
1712994645,0cyclictest0-21swapper/323:35:283
1711994645,0cyclictest0-21swapper/200:05:192
1711994610,36cyclictest0-21swapper/200:38:232
1709994646,0cyclictest0-21swapper/023:57:550
116442460,0sleep60-21swapper/622:12:506
116442460,0sleep60-21swapper/622:12:506
96062450,0sleep10-21swapper/100:01:021
1714994544,1cyclictest0-21swapper/500:24:015
1714994543,1cyclictest0-21swapper/523:06:555
1712994544,0cyclictest0-21swapper/323:26:103
1709994545,0cyclictest0-21swapper/000:23:510
1709994544,0cyclictest0-21swapper/000:10:160
1716994444,0cyclictest0-21swapper/723:44:237
1716994437,6cyclictest0-21swapper/700:10:107
1715994443,1cyclictest0-21swapper/600:07:456
1715994442,1cyclictest0-21swapper/623:37:236
1714994443,1cyclictest0-21swapper/500:15:225
1714994443,1cyclictest0-21swapper/500:01:555
1712994444,0cyclictest0-21swapper/323:48:573
171199440,0cyclictest0-21swapper/200:32:382
171199440,0cyclictest0-21swapper/200:32:372
1710994444,0cyclictest0-21swapper/100:36:231
170999440,44cyclictest0-21swapper/000:16:070
170999440,43cyclictest0-21swapper/023:44:550
1716994343,0cyclictest0-21swapper/723:26:197
1716994310,33cyclictest0-21swapper/700:17:547
171499430,0cyclictest0-21swapper/523:30:235
1713994341,2cyclictest0-21swapper/400:29:264
171399430,42cyclictest0-21swapper/400:20:104
1710994342,0cyclictest0-21swapper/100:30:211
1710994342,0cyclictest0-21swapper/100:30:211
1709994310,25cyclictest121rcu_preempt23:45:170
1716994237,5cyclictest0-21swapper/700:33:257
1716994237,5cyclictest0-21swapper/700:33:247
1715994241,0cyclictest0-21swapper/600:20:096
1714994240,1cyclictest0-21swapper/523:16:395
1714994240,1cyclictest0-21swapper/500:26:415
1714994210,0cyclictest0-21swapper/523:59:405
171499420,40cyclictest0-21swapper/522:55:215
1713994241,0cyclictest0-21swapper/400:30:324
1713994241,0cyclictest0-21swapper/400:30:314
1712994241,0cyclictest0-21swapper/300:33:463
1712994241,0cyclictest0-21swapper/300:33:453
1712994210,32cyclictest0-21swapper/300:05:513
171199420,0cyclictest0-21swapper/221:05:132
1714994140,0cyclictest0-21swapper/523:23:285
1712994141,0cyclictest0-21swapper/300:27:293
1712994140,1cyclictest0-21swapper/323:43:083
1710994141,0cyclictest0-21swapper/121:10:131
1710994140,0cyclictest0-21swapper/120:15:001
1710994140,0cyclictest0-21swapper/100:05:301
171099410,0cyclictest0-21swapper/119:30:121
1716994040,0cyclictest0-21swapper/722:20:167
1716994040,0cyclictest0-21swapper/700:26:287
1716994040,0cyclictest0-21swapper/700:05:087
1714994038,1cyclictest0-21swapper/500:05:205
1713994040,0cyclictest0-21swapper/423:05:004
1710994040,0cyclictest0-21swapper/123:15:261
1710994040,0cyclictest0-21swapper/100:15:081
1710994039,1cyclictest9313-21diskmemload22:55:001
1709994040,0cyclictest0-21swapper/023:54:070
1709994040,0cyclictest0-21swapper/022:15:180
1709994040,0cyclictest0-21swapper/000:25:430
1714993937,1cyclictest0-21swapper/523:25:145
1714993910,28cyclictest0-21swapper/521:50:185
1713993939,0cyclictest0-21swapper/423:43:084
1711993937,1cyclictest0-21swapper/221:10:132
1716993838,0cyclictest0-21swapper/700:04:237
1716993810,28cyclictest0-21swapper/700:37:387
1715993837,0cyclictest0-21swapper/619:20:156
1714993836,1cyclictest0-21swapper/500:37:505
1711993838,0cyclictest0-21swapper/219:30:132
1710993838,0cyclictest0-21swapper/123:35:001
1710993837,0cyclictest0-21swapper/121:20:211
1710993837,0cyclictest0-21swapper/100:11:161
1709993832,5cyclictest0-21swapper/022:20:010
1715993737,0cyclictest0-21swapper/600:35:336
1715993736,0cyclictest0-21swapper/623:48:526
171599370,0cyclictest0-21swapper/600:19:476
1713993737,0cyclictest0-21swapper/423:10:224
1712993737,0cyclictest0-21swapper/323:18:073
1712993737,0cyclictest0-21swapper/320:30:133
1712993736,1cyclictest0-21swapper/300:10:113
171299370,2cyclictest0-21swapper/322:45:013
1711993737,0cyclictest0-21swapper/200:29:252
1711993737,0cyclictest0-21swapper/200:02:472
1711993736,1cyclictest0-21swapper/220:15:012
1711993736,0cyclictest0-21swapper/223:15:262
1710993737,0cyclictest0-21swapper/119:45:111
1709993737,0cyclictest0-21swapper/022:45:180
1709993737,0cyclictest0-21swapper/000:35:230
1715993634,1cyclictest0-21swapper/623:26:096
1714993635,0cyclictest0-21swapper/523:54:185
1713993636,0cyclictest0-21swapper/420:30:134
1713993634,1cyclictest32469-21apt-get20:00:124
171399360,35cyclictest0-21swapper/419:50:154
1712993636,0cyclictest0-21swapper/300:36:013
1711993636,0cyclictest0-21swapper/200:15:072
1711993635,0cyclictest962-21perf22:54:592
1711993610,25cyclictest0-21swapper/223:47:262
1710993635,0cyclictest0-21swapper/123:37:231
1710993634,1cyclictest0-21swapper/100:20:101
1710993610,26cyclictest0-21swapper/100:26:461
1709993636,0cyclictest0-21swapper/000:34:590
1709993636,0cyclictest0-21swapper/000:34:590
1716993535,0cyclictest0-21swapper/723:50:157
1716993535,0cyclictest0-21swapper/721:35:117
1715993535,0cyclictest0-21swapper/621:55:166
1714993534,1cyclictest0-21swapper/523:41:385
1713993535,0cyclictest0-21swapper/400:05:214
171399350,3cyclictest25913-21grep22:50:214
171399350,35cyclictest0-21swapper/423:18:064
1712993534,0cyclictest0-21swapper/320:00:123
171299350,0cyclictest0-21swapper/319:50:143
1711993534,0cyclictest0-21swapper/223:35:002
171199350,0cyclictest0-21swapper/221:45:152
1710993535,0cyclictest0-21swapper/123:50:221
1709993535,0cyclictest0-21swapper/022:55:130
170999350,0cyclictest0-21swapper/022:05:140
1715993434,0cyclictest0-21swapper/623:00:196
1715993433,0cyclictest0-21swapper/600:32:516
1715993433,0cyclictest0-21swapper/600:32:516
1714993433,0cyclictest0-21swapper/523:47:455
1714993433,0cyclictest0-21swapper/500:10:015
1714993432,1cyclictest0-21swapper/522:10:165
1714993432,1cyclictest0-21swapper/522:10:165
171499340,33cyclictest0-21swapper/522:55:005
1712993432,1cyclictest9313-21diskmemload23:10:213
1711993434,0cyclictest0-21swapper/223:58:332
1711993433,1cyclictest0-21swapper/221:20:222
1711993433,1cyclictest0-21swapper/219:45:112
1709993434,0cyclictest0-21swapper/023:04:310
1709993434,0cyclictest0-21swapper/022:50:170
1709993433,0cyclictest0-21swapper/019:15:120
1715993331,1cyclictest0-21swapper/600:27:296
171599330,32cyclictest0-21swapper/621:35:166
1712993331,1cyclictest0-21swapper/323:30:233
1709993333,0cyclictest0-21swapper/023:30:220
1709993333,0cyclictest0-21swapper/021:25:140
1709993333,0cyclictest0-21swapper/000:03:170
1716993232,0cyclictest0-21swapper/722:35:187
1716993231,1cyclictest0-21swapper/720:45:137
1715993230,1cyclictest0-21swapper/623:43:096
171599320,32cyclictest0-21swapper/600:12:056
171499320,0cyclictest0-21swapper/500:32:255
171499320,0cyclictest0-21swapper/500:32:245
1713993232,0cyclictest0-21swapper/423:30:224
1713993232,0cyclictest0-21swapper/400:36:184
1712993230,1cyclictest0-21swapper/323:59:413
1712993230,1cyclictest0-21swapper/300:02:253
1711993230,1cyclictest0-21swapper/223:35:482
1711993230,1cyclictest0-21swapper/200:20:092
1710993232,0cyclictest0-21swapper/123:20:011
1710993230,1cyclictest8580-21apt-get19:20:151
1709993232,0cyclictest0-21swapper/023:20:140
1709993210,19cyclictest121rcu_preempt23:18:250
1709993210,18cyclictest0-21swapper/019:10:220
1713993130,1cyclictest0-21swapper/423:59:414
1713993130,0cyclictest0-21swapper/400:14:394
1713993129,1cyclictest0-21swapper/423:06:544
1709993131,0cyclictest0-21swapper/021:15:110
1709993131,0cyclictest0-21swapper/021:15:110
1709993125,5cyclictest121rcu_preempt21:05:170
1709993110,18cyclictest0-21swapper/021:48:220
1716993029,1cyclictest0-21swapper/723:15:117
1715993030,0cyclictest0-21swapper/623:05:226
1715993028,1cyclictest0-21swapper/620:30:126
1714993029,1cyclictest0-21swapper/523:00:145
1714993029,1cyclictest0-21swapper/519:20:145
171499300,0cyclictest0-21swapper/522:25:165
1713993030,0cyclictest0-21swapper/400:00:324
1713993028,1cyclictest0-21swapper/423:53:294
171399300,0cyclictest0-21swapper/422:40:014
1712993028,1cyclictest0-21swapper/322:55:203
1710993029,1cyclictest0-21swapper/123:48:521
1710993010,20cyclictest0-21swapper/123:55:121
1709993030,0cyclictest0-21swapper/023:27:300
1709993025,5cyclictest0-21swapper/023:07:340
1709993023,5cyclictest121rcu_preempt22:42:360
1709993011,17cyclictest121rcu_preempt23:39:110
1709993011,17cyclictest121rcu_preempt21:53:150
1709993010,18cyclictest121rcu_preempt23:14:050
1709993010,18cyclictest121rcu_preempt22:02:020
1709993010,18cyclictest121rcu_preempt20:10:220
1709993010,17cyclictest121rcu_preempt21:44:180
1709993010,15cyclictest121rcu_preempt21:24:390
1716992929,0cyclictest0-21swapper/723:56:447
1716992929,0cyclictest0-21swapper/723:38:167
1716992929,0cyclictest0-21swapper/723:00:177
1716992929,0cyclictest0-21swapper/722:25:127
1715992929,0cyclictest0-21swapper/623:19:166
1715992928,1cyclictest9313-21diskmemload23:10:236
1715992928,1cyclictest0-21swapper/622:50:206
1715992928,0cyclictest0-21swapper/600:00:136
1712992929,0cyclictest0-21swapper/323:53:303
1712992929,0cyclictest0-21swapper/323:06:543
171199290,1cyclictest0-21swapper/223:41:382
1709992929,0cyclictest0-21swapper/022:25:150
1709992919,8cyclictest121rcu_preempt21:12:340
1709992910,17cyclictest121rcu_preempt22:38:270
1709992910,17cyclictest121rcu_preempt21:59:410
1716992828,0cyclictest0-21swapper/723:48:377
1714992827,0cyclictest0-21swapper/521:15:205
1714992827,0cyclictest0-21swapper/521:15:205
1709992810,17cyclictest121rcu_preempt22:32:320
1709992810,16cyclictest121rcu_preempt21:31:490
1709992810,13cyclictest121rcu_preempt22:10:140
1709992810,13cyclictest121rcu_preempt22:10:140
1716992727,0cyclictest0-21swapper/723:10:157
1715992727,0cyclictest0-21swapper/619:50:156
1715992725,1cyclictest0-21swapper/620:00:136
1714992727,0cyclictest0-21swapper/523:36:395
1713992727,0cyclictest3775-21ls22:55:214
1713992726,1cyclictest0-21swapper/423:23:284
1712992726,1cyclictest0-21swapper/323:23:293
1710992727,0cyclictest0-21swapper/123:26:091
1710992727,0cyclictest0-21swapper/123:00:191
1709992710,17cyclictest0-21swapper/020:30:140
1709992710,16cyclictest0-21swapper/020:50:000
1709992710,16cyclictest0-21swapper/020:35:220
1716992626,0cyclictest0-21swapper/723:20:167
1716992610,0cyclictest0-21swapper/722:14:377
1716992610,0cyclictest0-21swapper/722:14:377
1714992626,0cyclictest0-21swapper/522:05:195
1712992624,1cyclictest0-21swapper/323:05:003
1711992626,0cyclictest9313-21diskmemload22:55:202
1710992626,0cyclictest0-21swapper/123:05:181
1710992626,0cyclictest0-21swapper/121:35:161
1709992610,15cyclictest121rcu_preempt19:20:180
1709992610,15cyclictest0-21swapper/019:50:160
1709992610,12cyclictest0-21swapper/020:15:190
1715992524,1cyclictest9313-21diskmemload23:30:216
1714992523,1cyclictest0-21swapper/521:00:005
1713992525,0cyclictest0-21swapper/422:00:114
1712992525,0cyclictest0-21swapper/321:50:173
1712992523,1cyclictest0-21swapper/322:10:203
1712992523,1cyclictest0-21swapper/322:10:203
1711992525,0cyclictest0-21swapper/220:40:122
1711992525,0cyclictest0-21swapper/219:20:152
1711992524,1cyclictest0-21swapper/221:55:162
1710992525,0cyclictest0-21swapper/123:10:171
1710992525,0cyclictest0-21swapper/120:50:141
1710992523,1cyclictest0-21swapper/123:43:091
1709992510,15cyclictest0-21swapper/020:40:200
1709992510,15cyclictest0-21swapper/019:45:010
1716992424,0cyclictest0-21swapper/721:55:127
1716992424,0cyclictest0-21swapper/719:40:147
1715992424,0cyclictest0-21swapper/623:51:206
1713992424,0cyclictest0-21swapper/421:50:184
1711992422,2cyclictest1589-21snmpd23:27:522
1710992424,0cyclictest0-21swapper/122:35:221
1709992410,13cyclictest0-21swapper/020:55:120
1716992323,0cyclictest8864-21apt-get21:25:137
1716992323,0cyclictest0-21swapper/721:20:147
1716992322,0cyclictest23507-21apt-get19:45:107
1716992310,3cyclictest0-21swapper/722:40:317
1715992321,1cyclictest0-21swapper/623:59:406
1715992310,13cyclictest0-21swapper/620:51:086
1714992323,0cyclictest0-21swapper/522:45:195
1714992319,3cyclictest23113-21nscd22:18:425
1714992310,6cyclictest0-21swapper/521:32:515
1713992322,1cyclictest0-21swapper/423:25:154
1713992322,1cyclictest0-21swapper/422:10:214
1713992322,1cyclictest0-21swapper/422:10:214
1713992320,3cyclictest4712-21munin-node21:15:094
1713992320,3cyclictest4712-21munin-node21:15:094
1713992310,7cyclictest0-21swapper/420:45:234
1712992323,0cyclictest0-21swapper/321:40:233
1711992323,0cyclictest0-21swapper/223:00:192
1710992321,1cyclictest0-21swapper/120:30:121
1710992310,6cyclictest0-21swapper/122:30:031
1709992318,3cyclictest121rcu_preempt19:35:180
1709992310,5cyclictest0-21swapper/019:30:200
1709992310,13cyclictest0-21swapper/020:25:000
1709992310,12cyclictest0-21swapper/021:00:170
1709992310,12cyclictest0-21swapper/019:25:140
1716992222,0cyclictest0-21swapper/722:55:167
1716992210,6cyclictest0-21swapper/722:09:217
1716992210,3cyclictest0-21swapper/722:30:167
1714992222,0cyclictest0-21swapper/522:20:015
1714992222,0cyclictest0-21swapper/522:00:215
1714992210,4cyclictest0-21swapper/520:10:125
1712992221,0cyclictest0-21swapper/322:54:593
1711992222,0cyclictest0-21swapper/221:35:162
1711992218,4cyclictest687-21nscd23:20:542
1711992210,5cyclictest0-21swapper/221:50:152
171199220,22cyclictest0-21swapper/222:05:122
1710992221,1cyclictest12677-21perf22:45:001
1710992210,5cyclictest0-21swapper/119:25:171
1710992210,0cyclictest0-21swapper/122:04:041
1709992210,8cyclictest0-21swapper/019:45:140
1709992210,7cyclictest0-21swapper/020:25:230
2075922110,0sleep20-21swapper/222:48:342
1716992120,1cyclictest0-21swapper/723:05:217
1716992111,3cyclictest0-21swapper/719:25:177
1716992110,6cyclictest0-21swapper/719:15:217
1715992121,0cyclictest0-21swapper/622:34:496
1715992110,7cyclictest0-21swapper/619:30:216
1715992110,5cyclictest0-21swapper/623:23:196
1715992110,2cyclictest0-21swapper/620:45:176
1714992121,0cyclictest0-21swapper/521:45:215
1714992118,2cyclictest20830-21nscd21:55:025
1712992118,1cyclictest0-21swapper/321:47:573
1712992114,6cyclictest1590-21runrttasks22:15:013
1712992110,0cyclictest0-21swapper/321:34:453
1711992121,0cyclictest0-21swapper/223:10:172
1709992110,11cyclictest0-21swapper/020:00:130
1709992110,10cyclictest0-21swapper/020:50:200
1716992010,3cyclictest0-21swapper/720:55:177
1716992010,0cyclictest0-21swapper/721:10:437
1715992018,1cyclictest0-21swapper/622:55:206
1715992010,4cyclictest0-21swapper/620:10:226
1714992020,0cyclictest0-21swapper/521:00:145
1714992010,5cyclictest0-21swapper/523:11:555
1714992010,4cyclictest0-21swapper/519:25:165
1713992013,3cyclictest71450irq/122-eth022:46:164
1713992013,3cyclictest1707-21cyclictest19:15:214
1713992010,5cyclictest0-21swapper/422:25:314
1713992010,4cyclictest0-21swapper/420:05:124
1713992010,2cyclictest0-21swapper/421:38:154
1713992010,1cyclictest0-21swapper/420:50:134
1712992019,0cyclictest0-21swapper/322:25:153
1712992010,7cyclictest0-21swapper/322:34:273
1712992010,6cyclictest0-21swapper/321:35:103
1712992010,5cyclictest0-21swapper/321:55:123
1712992010,5cyclictest0-21swapper/320:25:233
1711992020,0cyclictest0-21swapper/222:35:222
1711992020,0cyclictest0-21swapper/220:50:142
1711992010,5cyclictest0-21swapper/219:40:192
1710992020,0cyclictest0-21swapper/119:50:151
1710992012,2cyclictest0-21swapper/122:15:161
1710992010,6cyclictest658-21ptp4l21:53:021
1710992010,5cyclictest0-21swapper/122:07:151
1710992010,5cyclictest0-21swapper/119:55:171
1710992010,5cyclictest0-21swapper/119:55:171
1710992010,2cyclictest0-21swapper/122:48:201
1710992010,2cyclictest0-21swapper/122:11:281
1710992010,2cyclictest0-21swapper/122:11:281
1710992010,2cyclictest0-21swapper/120:10:141
1710992010,10cyclictest0-21swapper/121:41:351
1709992010,8cyclictest121rcu_preempt19:55:210
1709992010,8cyclictest121rcu_preempt19:55:210
1716991919,0cyclictest19827-21kworker/u16:121:47:547
1716991919,0cyclictest0-21swapper/721:40:137
1716991917,2cyclictest677-21nscd22:03:437
1716991910,4cyclictest0-21swapper/722:15:087
1716991910,4cyclictest0-21swapper/721:16:537
1716991910,4cyclictest0-21swapper/721:16:537
1716991910,2cyclictest0-21swapper/720:20:237
1715991913,4cyclictest765-21gmain22:02:416
1715991911,7cyclictest658-21ptp4l21:54:146
1715991910,7cyclictest0-21swapper/622:35:146
1715991910,5cyclictest0-21swapper/620:35:186
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional