You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 09:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Thu Dec 04, 2025 00:50:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
449829287,2sleep30-21swapper/319:05:203
425229287,2sleep10-21swapper/119:05:181
566929157,30sleep40-21swapper/419:09:064
550328570,11sleep70-21swapper/719:06:457
565328248,29sleep60-21swapper/619:08:536
409228240,26sleep00-21swapper/019:05:170
569528147,30sleep20-21swapper/219:09:292
541427744,29sleep50-21swapper/519:05:305
6170995810,0cyclictest0-21swapper/700:02:257
6166995550,2cyclictest121rcu_preempt00:29:133
63912540,0sleep70-21swapper/700:18:147
6169995252,0cyclictest0-21swapper/600:08:496
6168995252,0cyclictest0-21swapper/523:42:345
6166995251,0cyclictest0-21swapper/300:16:053
30642510,0sleep30-21swapper/323:44:563
6166995048,1cyclictest0-21swapper/323:35:183
6163995010,0cyclictest0-21swapper/000:37:240
116482500,0sleep50-21swapper/523:48:505
6168994949,0cyclictest0-21swapper/500:14:375
6167994910,39cyclictest0-21swapper/423:45:034
6165994947,1cyclictest0-21swapper/200:30:242
6168994848,0cyclictest0-21swapper/522:05:155
6166994847,1cyclictest0-21swapper/323:31:103
6166994847,0cyclictest0-21swapper/321:35:163
6165994846,1cyclictest33-21ksoftirqd/200:29:132
22292480,0sleep10-21swapper/121:51:001
134092480,0sleep00-21swapper/021:25:120
6170994747,0cyclictest0-21swapper/723:39:067
6169994747,0cyclictest0-21swapper/623:57:566
35342470,0sleep10-21swapper/120:50:141
206572470,0sleep50-21swapper/523:05:105
6169994646,0cyclictest0-21swapper/622:25:146
6164994646,0cyclictest0-21swapper/100:33:351
214992460,0sleep30-21swapper/322:49:313
6167994544,0cyclictest0-21swapper/400:17:054
6163994545,0cyclictest0-21swapper/023:40:240
6163994545,0cyclictest0-21swapper/019:35:010
6163994545,0cyclictest0-21swapper/000:07:340
6165994442,1cyclictest0-21swapper/223:35:192
176212440,0sleep20-21swapper/221:26:432
6170994343,0cyclictest0-21swapper/700:33:387
616899430,1cyclictest4462-21ls21:20:185
6167994342,1cyclictest0-21swapper/420:30:114
6166994342,0cyclictest0-21swapper/322:25:133
6166994310,32cyclictest0-21swapper/323:50:363
6165994342,0cyclictest0-21swapper/200:20:222
6163994343,0cyclictest0-21swapper/022:05:150
6163994342,0cyclictest0-21swapper/000:10:200
6163994310,33cyclictest0-21swapper/023:55:170
6163994310,33cyclictest0-21swapper/020:10:120
107392430,0sleep00-21swapper/023:48:130
6170994242,0cyclictest0-21swapper/723:25:157
6170994242,0cyclictest0-21swapper/723:15:127
6169994242,0cyclictest0-21swapper/619:10:116
6168994242,0cyclictest0-21swapper/500:08:495
6168994241,0cyclictest0-21swapper/520:15:115
6168994241,0cyclictest0-21swapper/500:16:515
616799420,0cyclictest0-21swapper/423:52:484
6165994241,0cyclictest0-21swapper/200:38:272
6165994240,1cyclictest2339-21ssh21:35:172
6164994210,0cyclictest0-21swapper/121:10:121
616499420,41cyclictest0-21swapper/123:40:141
616499420,41cyclictest0-21swapper/100:15:481
6163994242,0cyclictest0-21swapper/000:23:260
6169994140,0cyclictest0-21swapper/623:42:336
6168994110,30cyclictest0-21swapper/500:25:465
6167994141,0cyclictest0-21swapper/423:01:434
6166994141,0cyclictest0-21swapper/323:48:173
6164994141,0cyclictest0-21swapper/123:23:421
6163994141,0cyclictest0-21swapper/000:04:300
6170994040,0cyclictest0-21swapper/722:10:157
6169994040,0cyclictest0-21swapper/600:02:106
6167994040,0cyclictest0-21swapper/423:05:194
6167994040,0cyclictest0-21swapper/400:36:504
6165994039,0cyclictest0-21swapper/200:13:422
6164994040,0cyclictest0-21swapper/100:10:211
6163994040,0cyclictest0-21swapper/021:30:160
6169993938,0cyclictest0-21swapper/600:30:246
6167993939,0cyclictest0-21swapper/423:38:524
616799390,39cyclictest0-21swapper/400:12:274
6166993939,0cyclictest0-21swapper/323:14:373
6166993939,0cyclictest0-21swapper/319:35:013
6166993938,1cyclictest0-21swapper/323:55:123
6166993937,1cyclictest0-21swapper/323:00:223
6163993939,0cyclictest0-21swapper/021:20:170
6170993838,0cyclictest0-21swapper/723:47:367
6170993838,0cyclictest0-21swapper/723:21:447
6170993810,27cyclictest0-21swapper/721:10:117
617099380,37cyclictest0-21swapper/700:10:217
6169993838,0cyclictest0-21swapper/623:20:216
6169993837,0cyclictest0-21swapper/623:30:196
6168993838,0cyclictest0-21swapper/523:57:565
6168993838,0cyclictest0-21swapper/521:30:175
6168993837,0cyclictest0-21swapper/522:25:145
616799380,38cyclictest0-21swapper/420:20:164
6166993836,1cyclictest0-21swapper/323:05:223
6165993836,1cyclictest0-21swapper/222:25:122
6170993737,0cyclictest0-21swapper/700:38:177
6169993737,0cyclictest0-21swapper/623:52:556
6169993737,0cyclictest0-21swapper/623:12:216
6169993737,0cyclictest0-21swapper/600:23:226
6169993736,1cyclictest0-21swapper/600:29:136
6169993736,0cyclictest0-21swapper/622:55:216
6168993737,0cyclictest0-21swapper/500:03:035
6167993737,0cyclictest0-21swapper/422:55:174
6166993737,0cyclictest0-21swapper/300:07:533
6166993735,1cyclictest0-21swapper/300:35:183
6165993735,1cyclictest0-21swapper/223:48:172
6163993737,0cyclictest0-21swapper/000:16:510
6163993736,0cyclictest0-21swapper/023:08:010
6166993636,0cyclictest0-21swapper/300:11:003
6164993636,0cyclictest0-21swapper/100:26:171
6163993636,0cyclictest0-21swapper/022:10:140
6163993636,0cyclictest0-21swapper/020:15:120
6163993610,26cyclictest0-21swapper/000:25:040
6170993535,0cyclictest0-21swapper/723:50:527
6169993535,0cyclictest0-21swapper/622:47:316
6169993535,0cyclictest0-21swapper/600:16:066
6168993535,0cyclictest0-21swapper/522:15:195
6167993535,0cyclictest0-21swapper/423:30:144
616799350,35cyclictest0-21swapper/423:23:224
6166993533,1cyclictest0-21swapper/322:35:163
616599355,1cyclictest121rcu_preempt21:30:132
6165993534,0cyclictest0-21swapper/223:10:202
6165993510,0cyclictest0-21swapper/221:50:002
6164993535,0cyclictest0-21swapper/123:35:141
6163993535,0cyclictest0-21swapper/000:33:530
6170993434,0cyclictest0-21swapper/723:30:147
6170993434,0cyclictest0-21swapper/722:45:117
6169993434,0cyclictest0-21swapper/623:49:466
6169993434,0cyclictest0-21swapper/600:35:406
6169993432,1cyclictest0-21swapper/623:35:196
6168993434,0cyclictest0-21swapper/523:23:145
6166993434,0cyclictest0-21swapper/319:30:123
6166993434,0cyclictest0-21swapper/300:33:463
6165993433,1cyclictest0-21swapper/219:35:012
6165993432,1cyclictest99-21kswapd023:55:112
6164993434,0cyclictest0-21swapper/123:57:381
6163993434,0cyclictest0-21swapper/020:00:120
6163993434,0cyclictest0-21swapper/020:00:120
6169993331,1cyclictest0-21swapper/621:35:176
6168993332,0cyclictest0-21swapper/519:10:115
6167993333,0cyclictest0-21swapper/423:58:414
6167993333,0cyclictest0-21swapper/423:25:204
6167993332,1cyclictest0-21swapper/400:04:254
616799330,33cyclictest0-21swapper/400:21:264
6166993333,0cyclictest0-21swapper/300:24:103
6165993331,1cyclictest0-21swapper/223:00:212
6164993333,0cyclictest0-21swapper/123:28:441
6164993333,0cyclictest0-21swapper/100:38:131
6164993333,0cyclictest0-21swapper/100:01:311
6164993332,0cyclictest0-21swapper/120:10:111
6163993310,0cyclictest0-21swapper/023:22:270
6169993230,1cyclictest0-21swapper/600:13:426
616899320,31cyclictest0-21swapper/522:55:175
6167993232,0cyclictest0-21swapper/400:25:224
6166993231,0cyclictest0-21swapper/322:20:113
6166993231,0cyclictest0-21swapper/321:50:173
6165993230,1cyclictest0-21swapper/223:05:212
6164993232,0cyclictest0-21swapper/122:45:001
6164993231,0cyclictest0-21swapper/123:45:151
6170993131,0cyclictest0-21swapper/722:55:167
616999310,0cyclictest0-21swapper/619:55:136
6167993131,0cyclictest0-21swapper/422:30:134
6167993131,0cyclictest0-21swapper/420:25:114
6167993131,0cyclictest0-21swapper/400:31:454
6167993131,0cyclictest0-21swapper/400:05:174
6167993129,1cyclictest29105-21apt-get22:05:144
6165993129,1cyclictest0-21swapper/222:55:132
6163993131,0cyclictest0-21swapper/022:25:140
6170993030,0cyclictest0-21swapper/723:56:367
6170993030,0cyclictest0-21swapper/700:27:207
6170993030,0cyclictest0-21swapper/700:20:277
6170993029,0cyclictest0-21swapper/722:00:117
6169993030,0cyclictest0-21swapper/623:15:516
6169993027,0cyclictest0-21swapper/621:30:136
6168993030,0cyclictest0-21swapper/500:24:355
6168993028,1cyclictest0-21swapper/520:50:125
6167993030,0cyclictest0-21swapper/421:45:154
6166993030,0cyclictest0-21swapper/323:27:033
6166993029,0cyclictest0-21swapper/321:45:133
616699300,27cyclictest17173-21sh22:30:213
6164993030,0cyclictest0-21swapper/100:20:491
6164993029,0cyclictest14197-21diskmemload00:09:151
6164993010,20cyclictest0-21swapper/122:50:211
6163993030,0cyclictest0-21swapper/023:26:400
6163993030,0cyclictest0-21swapper/022:15:190
6170992928,0cyclictest0-21swapper/723:40:147
6168992929,0cyclictest0-21swapper/500:35:505
6168992928,1cyclictest0-21swapper/523:30:205
6166992928,0cyclictest0-21swapper/300:00:583
6165992929,0cyclictest0-21swapper/223:44:052
6165992929,0cyclictest0-21swapper/200:16:432
6165992928,1cyclictest0-21swapper/219:30:122
6165992928,0cyclictest0-21swapper/200:07:392
6164992929,0cyclictest0-21swapper/121:40:111
6168992828,0cyclictest0-21swapper/523:52:555
6168992828,0cyclictest0-21swapper/523:37:305
6168992828,0cyclictest0-21swapper/500:30:255
6165992827,1cyclictest0-21swapper/222:35:162
616599280,27cyclictest0-21swapper/222:40:192
6163992828,0cyclictest0-21swapper/022:55:170
6163992828,0cyclictest0-21swapper/021:50:010
6168992727,0cyclictest0-21swapper/523:12:215
6167992727,0cyclictest0-21swapper/419:40:114
6167992727,0cyclictest0-21swapper/419:40:114
6165992726,0cyclictest0-21swapper/222:45:012
6163992727,0cyclictest0-21swapper/019:10:110
6170992626,0cyclictest0-21swapper/723:00:237
6168992626,0cyclictest0-21swapper/522:47:325
6167992626,0cyclictest0-21swapper/423:15:204
6167992626,0cyclictest0-21swapper/421:15:004
6167992626,0cyclictest0-21swapper/420:55:134
6167992626,0cyclictest0-21swapper/420:40:134
616799260,26cyclictest0-21swapper/421:15:134
6170992525,0cyclictest0-21swapper/700:08:267
6169992524,1cyclictest0-21swapper/619:35:016
6167992525,0cyclictest0-21swapper/421:35:214
6166992525,0cyclictest0-21swapper/322:00:003
6166992525,0cyclictest0-21swapper/320:35:023
6166992522,3cyclictest0-21swapper/322:40:193
6165992525,0cyclictest0-21swapper/223:27:032
6164992525,0cyclictest0-21swapper/123:51:341
6164992525,0cyclictest0-21swapper/122:35:121
6164992525,0cyclictest0-21swapper/121:15:231
6170992424,0cyclictest0-21swapper/722:20:017
6169992422,1cyclictest0-21swapper/623:00:216
6168992424,0cyclictest0-21swapper/523:18:545
6167992424,0cyclictest0-21swapper/421:25:214
6166992424,0cyclictest0-21swapper/321:55:153
6166992423,1cyclictest15650irq/16-i801_smb22:16:543
6166992423,0cyclictest0-21swapper/319:45:113
6166992411,12cyclictest15650irq/16-i801_smb23:15:093
6165992424,0cyclictest0-21swapper/222:20:112
6165992424,0cyclictest0-21swapper/221:50:172
6165992424,0cyclictest0-21swapper/221:20:132
6165992422,1cyclictest33-21ksoftirqd/219:13:592
6164992424,0cyclictest0-21swapper/121:30:161
6163992423,1cyclictest0-21swapper/023:30:190
6168992323,0cyclictest0-21swapper/523:26:215
6168992323,0cyclictest0-21swapper/519:55:125
6168992322,0cyclictest0-21swapper/521:25:215
6167992322,0cyclictest22394-21apt-get19:35:124
6166992323,0cyclictest0-21swapper/322:10:193
6166992316,3cyclictest121rcu_preempt21:14:213
6166992314,3cyclictest121rcu_preempt21:40:153
6165992323,0cyclictest0-21swapper/223:30:252
6165992322,1cyclictest0-21swapper/200:00:582
6165992315,5cyclictest33-21ksoftirqd/222:30:162
6164992323,0cyclictest0-21swapper/123:14:001
6164992323,0cyclictest0-21swapper/122:45:011
6164992310,7cyclictest0-21swapper/120:05:181
6163992323,0cyclictest0-21swapper/023:52:550
6163992323,0cyclictest0-21swapper/023:37:300
6163992323,0cyclictest0-21swapper/020:50:120
47422310,0sleep70-21swapper/722:06:207
6170992210,8cyclictest0-21swapper/721:47:287
6170992210,0cyclictest0-21swapper/721:41:357
6169992222,0cyclictest0-21swapper/620:35:126
6168992221,1cyclictest1589-21snmpd21:58:515
6168992221,0cyclictest0-21swapper/521:35:165
6168992221,0cyclictest0-21swapper/520:05:145
6168992210,3cyclictest0-21swapper/521:15:515
6167992222,0cyclictest0-21swapper/422:25:154
6167992222,0cyclictest0-21swapper/421:50:174
6166992222,0cyclictest0-21swapper/323:20:173
6166992214,7cyclictest15650irq/16-i801_smb21:33:183
6165992221,1cyclictest33-21ksoftirqd/221:00:002
6165992210,3cyclictest0-21swapper/220:40:222
6164992222,0cyclictest0-21swapper/123:05:211
616499220,0cyclictest0-21swapper/119:25:101
6163992222,0cyclictest0-21swapper/023:12:210
1211622210,0sleep70-21swapper/721:24:487
6170992110,7cyclictest0-21swapper/720:20:197
616999212,1cyclictest121rcu_preempt21:40:116
6169992121,0cyclictest0-21swapper/621:45:146
6169992110,2cyclictest0-21swapper/622:50:306
6169992110,2cyclictest0-21swapper/621:20:226
6168992110,4cyclictest0-21swapper/522:50:215
6168992110,4cyclictest0-21swapper/521:10:065
6167992120,1cyclictest9112-21munin-node20:01:094
6167992120,1cyclictest9112-21munin-node20:01:094
6167992116,5cyclictest20830-21nscd22:16:434
616799210,0cyclictest0-21swapper/423:14:354
6166992116,3cyclictest0-21swapper/322:55:203
6166992114,4cyclictest121rcu_preempt22:53:073
6166992113,6cyclictest121rcu_preempt21:20:573
6166992110,6cyclictest0-21swapper/321:25:203
6165992115,4cyclictest121rcu_preempt20:00:142
6165992115,4cyclictest121rcu_preempt20:00:132
6165992110,3cyclictest0-21swapper/222:09:512
6164992110,8cyclictest0-21swapper/123:31:361
6164992110,3cyclictest0-21swapper/121:35:161
6163992121,0cyclictest0-21swapper/022:47:310
356022110,0sleep10-21swapper/122:08:221
2452622110,0sleep60-21swapper/620:30:196
6170992010,6cyclictest0-21swapper/722:44:047
6170992010,1cyclictest0-21swapper/721:50:127
6169992020,0cyclictest0-21swapper/623:05:216
6168992018,1cyclictest0-21swapper/521:50:005
6168992015,4cyclictest0-21swapper/522:20:285
6168992015,3cyclictest6161-21cyclictest22:34:525
6168992010,2cyclictest0-21swapper/522:13:215
6168992010,2cyclictest0-21swapper/520:30:165
6168992010,0cyclictest0-21swapper/522:01:165
6167992019,1cyclictest765-21gmain22:42:084
6167992010,4cyclictest0-21swapper/419:30:124
6166992017,2cyclictest0-21swapper/320:05:153
6166992017,1cyclictest0-21swapper/320:10:213
6166992010,7cyclictest0-21swapper/321:17:023
6166992010,5cyclictest0-21swapper/322:05:023
6166992010,0cyclictest121rcu_preempt19:50:193
6165992019,1cyclictest0-21swapper/220:35:022
6165992018,1cyclictest14047-21apt-get23:50:122
6165992017,2cyclictest12152-21hddtemp_smartct21:40:142
6165992010,9cyclictest0-21swapper/223:20:172
6165992010,3cyclictest0-21swapper/222:10:302
6165992010,3cyclictest0-21swapper/220:15:202
6164992010,5cyclictest0-21swapper/121:55:151
6164992010,5cyclictest0-21swapper/120:40:181
6164992010,4cyclictest0-21swapper/121:20:221
6164992010,2cyclictest0-21swapper/123:17:161
6163992020,0cyclictest0-21swapper/022:42:160
6163992019,1cyclictest1937-21apt-get19:55:120
6163992010,4cyclictest0-21swapper/022:52:220
6163992010,2cyclictest0-21swapper/022:31:550
6163992010,2cyclictest0-21swapper/021:43:320
6170991910,5cyclictest0-21swapper/720:15:157
6170991910,5cyclictest0-21swapper/719:55:177
6170991910,4cyclictest0-21swapper/722:50:027
6170991910,4cyclictest0-21swapper/722:15:247
6170991910,3cyclictest0-21swapper/722:29:287
6170991910,3cyclictest0-21swapper/721:59:407
6170991910,2cyclictest0-21swapper/722:38:577
6170991910,2cyclictest0-21swapper/721:15:057
6169991919,0cyclictest0-21swapper/622:40:196
6169991919,0cyclictest0-21swapper/621:55:156
6169991916,2cyclictest3754-21nscd22:08:446
6169991910,6cyclictest0-21swapper/622:24:276
6169991910,4cyclictest0-21swapper/620:00:206
6169991910,4cyclictest0-21swapper/620:00:206
6169991910,3cyclictest121rcu_preempt22:15:236
6169991910,2cyclictest0-21swapper/619:15:216
6168991919,0cyclictest0-21swapper/520:35:125
6168991919,0cyclictest0-21swapper/520:25:105
6168991910,0cyclictest0-21swapper/521:41:325
6167991919,0cyclictest0-21swapper/421:00:124
6167991914,1cyclictest6161-21cyclictest22:46:244
6167991910,5cyclictest0-21swapper/420:45:154
6167991910,3cyclictest0-21swapper/422:00:584
616699199,10cyclictest0-21swapper/321:00:223
6166991916,2cyclictest8920-21latency_hist19:15:013
6166991915,4cyclictest0-21swapper/320:55:143
6166991910,4cyclictest0-21swapper/320:15:113
616599198,9cyclictest33-21ksoftirqd/223:19:112
6165991919,0cyclictest0-21swapper/222:00:002
6165991918,1cyclictest321ktimersoftd/222:54:592
6165991917,1cyclictest33-21ksoftirqd/219:25:002
6165991917,1cyclictest0-21swapper/219:45:122
6165991913,4cyclictest33-21ksoftirqd/219:50:192
6165991910,5cyclictest0-21swapper/220:50:172
6165991910,3cyclictest0-21swapper/222:16:032
6164991910,6cyclictest0-21swapper/121:45:221
6164991910,4cyclictest0-21swapper/121:26:311
6164991910,4cyclictest0-21swapper/119:10:151
6163991918,1cyclictest12012-21sh23:16:060
6163991914,4cyclictest1660-21ntpd21:48:360
6163991913,5cyclictest1590-21runrttasks22:38:140
6163991910,3cyclictest0-21swapper/019:50:120
6163991910,3cyclictest0-21swapper/019:40:160
6163991910,3cyclictest0-21swapper/019:40:160
6170991818,0cyclictest0-21swapper/720:45:117
6170991818,0cyclictest0-21swapper/719:30:127
6170991812,3cyclictest0-21swapper/721:26:127
6170991810,8cyclictest0-21swapper/723:07:167
6170991810,6cyclictest0-21swapper/719:25:107
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional