You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Sat Jul 12, 2025 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31102210156,42sleep00-21swapper/019:09:180
28302202164,25sleep30-21swapper/319:05:383
31172201162,26sleep20-21swapper/219:09:242
28682199160,26sleep10-21swapper/119:06:091
218572700,0sleep10-21swapper/123:10:001
316742650,0sleep00-21swapper/021:13:360
3270992323,0cyclictest28-21ksoftirqd/221:24:492
3270991817,1cyclictest28-21ksoftirqd/220:40:162
3270991817,1cyclictest28-21ksoftirqd/219:55:162
3270991717,0cyclictest28-21ksoftirqd/221:45:142
3270991716,1cyclictest28-21ksoftirqd/222:10:172
3270991716,1cyclictest28-21ksoftirqd/200:05:142
3270991716,0cyclictest28-21ksoftirqd/222:45:152
3270991616,0cyclictest28-21ksoftirqd/223:08:512
3270991616,0cyclictest28-21ksoftirqd/221:32:012
3270991616,0cyclictest28-21ksoftirqd/221:29:042
3270991616,0cyclictest28-21ksoftirqd/200:22:152
3270991615,1cyclictest28-21ksoftirqd/221:14:562
3270991615,1cyclictest28-21ksoftirqd/219:50:212
3270991615,1cyclictest28-21ksoftirqd/200:29:512
3270991614,1cyclictest28-21ksoftirqd/223:10:202
3270991515,0cyclictest28-21ksoftirqd/222:39:002
3270991514,1cyclictest28-21ksoftirqd/220:55:202
3270991414,0cyclictest28-21ksoftirqd/223:51:082
3270991414,0cyclictest28-21ksoftirqd/223:46:052
3270991414,0cyclictest28-21ksoftirqd/223:41:262
3270991414,0cyclictest28-21ksoftirqd/222:30:042
3270991414,0cyclictest28-21ksoftirqd/222:24:152
3270991414,0cyclictest28-21ksoftirqd/222:19:482
3270991414,0cyclictest28-21ksoftirqd/221:00:182
3270991413,1cyclictest28-21ksoftirqd/220:50:002
3270991412,1cyclictest28-21ksoftirqd/223:20:002
327099140,6cyclictest28-21ksoftirqd/220:30:112
3254991413,1cyclictest9-21ksoftirqd/023:25:210
3254991413,1cyclictest9-21ksoftirqd/021:40:000
3275991312,1cyclictest35-21ksoftirqd/321:10:003
327099138,3cyclictest28-21ksoftirqd/223:22:152
3270991313,0cyclictest28-21ksoftirqd/223:28:192
3270991313,0cyclictest28-21ksoftirqd/223:03:472
3270991313,0cyclictest28-21ksoftirqd/222:07:592
3270991313,0cyclictest28-21ksoftirqd/222:00:262
3270991313,0cyclictest28-21ksoftirqd/221:52:572
3270991313,0cyclictest28-21ksoftirqd/221:38:182
3270991313,0cyclictest28-21ksoftirqd/221:16:252
3270991313,0cyclictest28-21ksoftirqd/200:10:022
3270991312,1cyclictest28-21ksoftirqd/219:21:142
3270991312,1cyclictest28-21ksoftirqd/219:10:172
3270991312,1cyclictest28-21ksoftirqd/200:35:072
325499139,3cyclictest9-21ksoftirqd/000:25:000
3254991312,1cyclictest9-21ksoftirqd/023:35:180
3254991311,1cyclictest1107-21ssh21:57:230
327099129,2cyclictest28-21ksoftirqd/220:35:122
327099127,4cyclictest28-21ksoftirqd/221:05:122
327099126,2cyclictest28-21ksoftirqd/220:00:172
3270991212,0cyclictest28-21ksoftirqd/222:55:152
3270991212,0cyclictest28-21ksoftirqd/222:53:032
3270991212,0cyclictest28-21ksoftirqd/220:15:142
3270991212,0cyclictest28-21ksoftirqd/219:35:142
3270991212,0cyclictest28-21ksoftirqd/200:30:132
3270991211,1cyclictest28-21ksoftirqd/221:55:182
3270991211,1cyclictest28-21ksoftirqd/200:17:132
3270991211,1cyclictest28-21ksoftirqd/200:00:132
3270991211,1cyclictest28-21ksoftirqd/200:00:132
327099120,1cyclictest0-21swapper/222:44:592
325499129,1cyclictest9-21ksoftirqd/019:55:000
3254991211,1cyclictest9-21ksoftirqd/023:50:160
3254991211,1cyclictest9-21ksoftirqd/022:05:140
3254991211,1cyclictest9-21ksoftirqd/020:55:140
3254991210,1cyclictest9-21ksoftirqd/022:10:220
3275991110,1cyclictest35-21ksoftirqd/323:45:193
327099119,1cyclictest28-21ksoftirqd/223:30:152
327099119,1cyclictest28-21ksoftirqd/221:40:212
327099119,1cyclictest28-21ksoftirqd/219:45:202
327099119,1cyclictest28-21ksoftirqd/219:45:002
327099119,1cyclictest28-21ksoftirqd/219:30:002
3270991111,0cyclictest28-21ksoftirqd/223:35:232
3270991111,0cyclictest28-21ksoftirqd/222:26:142
3270991110,1cyclictest28-21ksoftirqd/223:55:212
3270991110,1cyclictest28-21ksoftirqd/220:55:002
3270991110,1cyclictest28-21ksoftirqd/220:25:212
3270991110,0cyclictest28-21ksoftirqd/220:15:002
3270991110,0cyclictest28-21ksoftirqd/220:15:002
325499119,1cyclictest9-21ksoftirqd/021:50:230
325499119,1cyclictest9-21ksoftirqd/021:20:190
325499119,1cyclictest9-21ksoftirqd/020:00:000
325499119,1cyclictest9-21ksoftirqd/019:40:160
3254991110,1cyclictest9-21ksoftirqd/022:40:150
3254991110,1cyclictest9-21ksoftirqd/000:10:130
3254991110,0cyclictest9-21ksoftirqd/020:00:170
48552100,0sleep30-21swapper/323:30:133
327599109,1cyclictest35-21ksoftirqd/322:00:003
327599109,1cyclictest35-21ksoftirqd/300:10:013
327599109,1cyclictest35-21ksoftirqd/300:10:013
327599108,1cyclictest35-21ksoftirqd/322:40:143
327599108,1cyclictest35-21ksoftirqd/320:05:003
3275991010,0cyclictest35-21ksoftirqd/322:00:153
327599100,5cyclictest254-21systemd-journal23:11:093
327099109,1cyclictest28-21ksoftirqd/219:30:122
327099108,1cyclictest28-21ksoftirqd/220:20:172
327099108,1cyclictest28-21ksoftirqd/220:05:162
327099108,1cyclictest28-21ksoftirqd/219:15:012
325499109,1cyclictest9-21ksoftirqd/023:45:000
325499109,1cyclictest9-21ksoftirqd/023:05:110
325499109,1cyclictest9-21ksoftirqd/021:19:450
325499109,1cyclictest9-21ksoftirqd/020:25:010
325499109,1cyclictest9-21ksoftirqd/000:25:210
325499109,1cyclictest10488-21ssh21:27:160
325499108,1cyclictest9-21ksoftirqd/020:15:000
325499108,1cyclictest9-21ksoftirqd/020:15:000
3254991010,0cyclictest9-21ksoftirqd/023:55:130
3254991010,0cyclictest9-21ksoftirqd/023:23:300
3254991010,0cyclictest9-21ksoftirqd/000:01:060
3254991010,0cyclictest9-21ksoftirqd/000:01:060
316402100,0sleep10-21swapper/122:39:331
314482100,0sleep028327-21diskmemload22:39:000
263732100,0sleep328327-21diskmemload23:59:103
32759999,0cyclictest35-21ksoftirqd/321:20:203
32759998,1cyclictest35-21ksoftirqd/300:05:003
32759998,1cyclictest35-21ksoftirqd/300:05:003
32549999,0cyclictest9-21ksoftirqd/023:45:140
32549999,0cyclictest9-21ksoftirqd/022:20:170
32549999,0cyclictest9-21ksoftirqd/021:40:130
32549999,0cyclictest9-21ksoftirqd/021:30:260
32549999,0cyclictest9-21ksoftirqd/000:05:210
32549998,1cyclictest9-21ksoftirqd/023:30:130
32549998,1cyclictest9-21ksoftirqd/022:50:180
32549998,1cyclictest9-21ksoftirqd/022:30:120
32549998,1cyclictest9-21ksoftirqd/021:05:120
32549998,1cyclictest9-21ksoftirqd/020:50:150
32549998,1cyclictest9-21ksoftirqd/020:25:210
32549998,1cyclictest9-21ksoftirqd/019:20:130
32549998,1cyclictest9-21ksoftirqd/019:10:170
32549998,1cyclictest9-21ksoftirqd/000:30:200
32549997,1cyclictest9-21ksoftirqd/022:55:220
32549997,1cyclictest9-21ksoftirqd/020:45:160
32549996,1cyclictest9-21ksoftirqd/023:14:550
32549994,5cyclictest9-21ksoftirqd/021:45:200
32549993,2cyclictest0-21swapper/020:30:110
32549993,0cyclictest9-21ksoftirqd/000:35:230
32759988,0cyclictest35-21ksoftirqd/323:05:143
32759988,0cyclictest35-21ksoftirqd/322:20:003
32759988,0cyclictest35-21ksoftirqd/321:00:163
32759988,0cyclictest35-21ksoftirqd/300:28:323
32759987,1cyclictest35-21ksoftirqd/323:40:123
32759987,1cyclictest35-21ksoftirqd/321:40:143
32759987,1cyclictest35-21ksoftirqd/321:35:223
32759987,1cyclictest35-21ksoftirqd/319:30:133
32759987,1cyclictest35-21ksoftirqd/300:15:163
32759986,1cyclictest35-21ksoftirqd/322:50:233
32759981,4cyclictest35-21ksoftirqd/319:50:003
32639980,6cyclictest0-21swapper/121:25:151
32549988,0cyclictest9-21ksoftirqd/022:25:210
32549988,0cyclictest9-21ksoftirqd/022:15:210
32549988,0cyclictest9-21ksoftirqd/022:00:210
32549987,1cyclictest9-21ksoftirqd/023:15:120
32549987,1cyclictest9-21ksoftirqd/020:40:160
32549987,1cyclictest9-21ksoftirqd/019:15:200
32549987,1cyclictest16535-21ssh23:00:480
32549985,3cyclictest9-21ksoftirqd/000:18:330
32549980,7cyclictest6417-21ssh22:48:470
32759977,0cyclictest35-21ksoftirqd/321:45:193
32759977,0cyclictest35-21ksoftirqd/321:30:133
32759976,1cyclictest35-21ksoftirqd/322:45:143
32759976,1cyclictest35-21ksoftirqd/322:10:003
32759976,1cyclictest35-21ksoftirqd/320:10:003
32759976,1cyclictest35-21ksoftirqd/320:00:003
32759976,1cyclictest35-21ksoftirqd/319:25:013
32759976,1cyclictest17740-21snmpd22:14:033
32759976,1cyclictest17740-21snmpd21:19:413
32759975,1cyclictest4392-21ssh23:29:583
32759975,1cyclictest31426-21ssh22:38:563
32759975,1cyclictest17740-21snmpd23:35:203
32759975,1cyclictest17740-21snmpd22:29:363
32759975,1cyclictest17740-21snmpd19:50:383
32759975,1cyclictest13164-21nfs4_client20:35:183
32759971,2cyclictest0-21swapper/321:54:193
32759971,1cyclictest0-21swapper/322:55:143
32759970,6cyclictest16518-21fschecks_count19:35:153
32759970,4cyclictest20732-21/usr/sbin/munin23:50:203
32759970,4cyclictest0-21swapper/322:24:133
32759970,1cyclictest0-21swapper/300:35:593
32639971,6cyclictest0-21swapper/122:32:571
32639971,6cyclictest0-21swapper/121:02:041
32639971,6cyclictest0-21swapper/119:21:011
32639971,3cyclictest0-21swapper/119:45:191
32639970,4cyclictest0-21swapper/123:45:381
32639970,4cyclictest0-21swapper/121:20:451
32639970,4cyclictest0-21swapper/100:01:341
32639970,4cyclictest0-21swapper/100:01:341
32639970,2cyclictest0-21swapper/122:02:461
32639970,2cyclictest0-21swapper/120:00:131
32639970,1cyclictest0-21swapper/123:16:131
32639970,1cyclictest0-21swapper/121:47:141
32639970,1cyclictest0-21swapper/100:10:221
32639970,1cyclictest0-21swapper/100:08:191
32549976,1cyclictest9-21ksoftirqd/020:35:120
32549976,1cyclictest9-21ksoftirqd/019:25:120
32549975,1cyclictest31268-21ntp_states20:05:200
32549974,2cyclictest13581-21/usr/sbin/munin19:30:140
32549970,6cyclictest23855-21/usr/sbin/munin21:00:160
32549970,6cyclictest16942-21memory19:35:170
32759966,0cyclictest35-21ksoftirqd/323:20:143
32759966,0cyclictest35-21ksoftirqd/320:35:003
32759965,1cyclictest35-21ksoftirqd/322:30:153
32759965,1cyclictest35-21ksoftirqd/320:15:133
32759965,1cyclictest35-21ksoftirqd/319:15:013
32759965,1cyclictest28938-21ssh23:19:203
32759965,1cyclictest28623-21ssh21:10:123
32759965,1cyclictest17331-21ssh23:03:023
32759964,1cyclictest9536-21/usr/sbin/munin00:20:203
32759964,1cyclictest35-21ksoftirqd/320:50:143
32759964,1cyclictest35-21ksoftirqd/320:40:153
32759964,1cyclictest0-21swapper/319:10:183
32759961,3cyclictest16774-21apt-get20:45:133
32759960,2cyclictest0-21swapper/321:29:123
32759960,2cyclictest0-21swapper/300:30:533
32639963,1cyclictest0-21swapper/121:38:341
32639961,5cyclictest0-21swapper/122:42:341
32639961,1cyclictest0-21swapper/120:55:111
32639960,6cyclictest0-21swapper/123:12:131
32639960,6cyclictest0-21swapper/122:53:131
32639960,6cyclictest0-21swapper/100:25:161
32639960,6cyclictest0-21swapper/100:21:411
32639960,5cyclictest0-21swapper/122:05:161
32639960,5cyclictest0-21swapper/121:15:131
32639960,5cyclictest0-21swapper/100:35:141
32639960,5cyclictest0-21swapper/100:32:541
32639960,4cyclictest0-21swapper/121:50:221
32639960,4cyclictest0-21swapper/121:43:331
32639960,4cyclictest0-21swapper/120:45:181
32639960,4cyclictest0-21swapper/120:30:001
32639960,3cyclictest0-21swapper/123:20:401
32639960,3cyclictest0-21swapper/122:56:331
32639960,3cyclictest0-21swapper/122:28:121
32639960,3cyclictest0-21swapper/122:18:111
32639960,3cyclictest0-21swapper/121:30:011
32639960,2cyclictest0-21swapper/123:55:191
32639960,2cyclictest0-21swapper/123:05:001
32639960,1cyclictest0-21swapper/121:10:161
32639960,1cyclictest0-21swapper/120:50:121
32639960,1cyclictest0-21swapper/119:57:321
32639960,0cyclictest0-21swapper/123:31:181
32549960,6cyclictest0-21swapper/020:15:050
32549960,3cyclictest0-21swapper/019:45:160
32759954,1cyclictest35-21ksoftirqd/320:15:003
32759954,1cyclictest35-21ksoftirqd/320:15:003
32759954,1cyclictest22830-21systemctl20:55:193
32759953,1cyclictest7285-21aten2.4-expect20:25:133
32759953,1cyclictest11729-21df19:25:153
32759950,2cyclictest0-21swapper/319:40:153
32639955,0cyclictest0-21swapper/122:45:251
32639955,0cyclictest0-21swapper/119:51:341
32639955,0cyclictest0-21swapper/119:40:061
32639953,1cyclictest0-21swapper/100:15:061
32639952,2cyclictest0-21swapper/120:35:121
32639951,2cyclictest0-21swapper/122:20:121
32639950,5cyclictest0-21swapper/121:55:571
32639950,5cyclictest0-21swapper/119:27:361
32639950,4cyclictest0-21swapper/120:29:331
32639950,4cyclictest0-21swapper/120:10:161
32639950,4cyclictest0-21swapper/120:10:161
32639950,3cyclictest0-21swapper/123:27:511
32639950,2cyclictest0-21swapper/123:53:531
32639950,2cyclictest0-21swapper/123:38:381
32639950,2cyclictest0-21swapper/122:10:401
32639950,1cyclictest0-21swapper/123:41:471
32639950,1cyclictest0-21swapper/120:40:131
32639950,0cyclictest0-21swapper/120:07:331
32639950,0cyclictest0-21swapper/119:15:141
32759942,1cyclictest4976-21cpu20:20:163
32639944,0cyclictest0-21swapper/121:06:341
32639944,0cyclictest0-21swapper/119:30:341
32639943,1cyclictest17819-21chrt19:37:051
32639941,0cyclictest0-21swapper/120:20:201
32639931,1cyclictest0-21swapper/120:15:111
32639930,1cyclictest0-21swapper/119:10:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional