You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 06:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Sun Nov 16, 2025 00:45:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
136691220162,47phc2sys0-21swapper/319:05:473
15112206172,22sleep20-21swapper/219:06:422
14612204171,22sleep00-21swapper/019:06:040
15572203169,23sleep10-21swapper/119:07:131
43321840,8sleep1208199cyclictest20:10:261
3147221640,7sleep1208199cyclictest21:15:261
208099580,21cyclictest27422-21irqstats22:15:200
208099570,27cyclictest10072-21tr22:45:250
136591530,36ptp4l0-21swapper/319:10:013
51032520,2sleep20-21swapper/221:30:162
208099486,18cyclictest12089-21grep19:30:140
36732470,2sleep30-21swapper/321:25:233
136591440,0ptp4l401ktimersoftd/323:01:163
136591440,0ptp4l401ktimersoftd/322:51:133
136591430,1ptp4l401ktimersoftd/320:15:343
136591430,1ptp4l401ktimersoftd/319:25:053
136591420,0ptp4l401ktimersoftd/323:10:033
136591420,0ptp4l401ktimersoftd/321:45:293
136591410,0ptp4l401ktimersoftd/320:40:243
136591400,0ptp4l401ktimersoftd/321:25:013
136591400,0ptp4l401ktimersoftd/320:45:273
2081993933,4cyclictest19696-21cron00:15:001
136591390,9ptp4l401ktimersoftd/319:55:283
208099380,19cyclictest10016-21/usr/sbin/munin23:55:230
208299371,35cyclictest17639-21kworker/2:000:33:462
208199371,33cyclictest3094-21kworker/1:022:55:241
136591360,0ptp4l401ktimersoftd/320:35:223
208199354,19cyclictest3117-21kworker/1:020:09:131
208199350,22cyclictest18432-21kworker/1:022:12:211
208099350,21cyclictest14366-21/usr/sbin/munin21:50:260
136591350,0ptp4l401ktimersoftd/319:30:043
2083993328,3cyclictest41-21ksoftirqd/320:22:193
2083993327,4cyclictest41-21ksoftirqd/319:51:443
2083993327,4cyclictest41-21ksoftirqd/300:26:403
2082993327,4cyclictest33-21ksoftirqd/222:29:012
2081993327,4cyclictest25-21ksoftirqd/120:47:301
2081993326,4cyclictest25-21ksoftirqd/121:25:011
2080993328,3cyclictest9-21ksoftirqd/023:05:110
2080993327,4cyclictest9-21ksoftirqd/023:34:170
2080993327,4cyclictest9-21ksoftirqd/023:34:170
2080993327,4cyclictest9-21ksoftirqd/000:13:050
2080993327,4cyclictest9-21ksoftirqd/000:06:020
136591330,0ptp4l401ktimersoftd/323:05:163
136591330,0ptp4l401ktimersoftd/319:15:173
2083993228,2cyclictest136591ptp4l21:40:323
2083993226,4cyclictest41-21ksoftirqd/300:15:573
2082993227,3cyclictest33-21ksoftirqd/200:05:122
2082993226,4cyclictest33-21ksoftirqd/221:59:232
2082993226,4cyclictest33-21ksoftirqd/220:27:332
2082993226,4cyclictest33-21ksoftirqd/219:58:042
2082993226,4cyclictest33-21ksoftirqd/200:22:372
2082993226,4cyclictest33-21ksoftirqd/200:01:112
208199328,21cyclictest20081-21kworker/1:221:00:121
2081993226,4cyclictest25-21ksoftirqd/123:01:201
208199320,31cyclictest0-21swapper/120:25:251
2080993227,3cyclictest9-21ksoftirqd/023:25:150
2080993226,4cyclictest9-21ksoftirqd/022:00:580
2080993226,4cyclictest9-21ksoftirqd/020:30:390
2080993212,17cyclictest10188-21ls21:40:200
208099321,17cyclictest0-21swapper/020:50:190
208099320,17cyclictest0-21swapper/022:50:140
2083993125,4cyclictest41-21ksoftirqd/322:43:183
2083993125,4cyclictest41-21ksoftirqd/322:20:433
2083993125,4cyclictest41-21ksoftirqd/322:13:133
2083993125,4cyclictest41-21ksoftirqd/322:01:143
2083993125,4cyclictest41-21ksoftirqd/321:35:373
2083993125,4cyclictest41-21ksoftirqd/321:33:493
2083993125,4cyclictest41-21ksoftirqd/320:51:493
2082993127,3cyclictest33-21ksoftirqd/223:36:472
2082993127,3cyclictest33-21ksoftirqd/222:05:192
2082993127,3cyclictest33-21ksoftirqd/221:38:482
2082993126,3cyclictest33-21ksoftirqd/219:18:232
2082993125,4cyclictest33-21ksoftirqd/223:46:072
2082993125,4cyclictest33-21ksoftirqd/223:40:412
2082993125,4cyclictest33-21ksoftirqd/223:22:402
2082993125,4cyclictest33-21ksoftirqd/220:41:282
2082993125,4cyclictest33-21ksoftirqd/200:18:202
2082993125,4cyclictest33-21ksoftirqd/200:12:382
2082993117,2cyclictest33-21ksoftirqd/200:37:182
2081993125,4cyclictest25-21ksoftirqd/123:27:291
2081993125,4cyclictest25-21ksoftirqd/119:21:131
2081993125,4cyclictest25-21ksoftirqd/100:34:121
2081993125,4cyclictest25-21ksoftirqd/100:06:411
2081993125,4cyclictest25-21ksoftirqd/100:03:511
2081993115,3cyclictest25-21ksoftirqd/119:55:151
2080993125,4cyclictest9-21ksoftirqd/023:44:160
2080993125,4cyclictest9-21ksoftirqd/021:30:380
2080993125,4cyclictest9-21ksoftirqd/019:26:580
2080993125,4cyclictest9-21ksoftirqd/000:00:050
2080993113,15cyclictest27462-21latency_hist23:25:000
208099310,18cyclictest23229-21cat19:50:250
136591310,0ptp4l401ktimersoftd/323:28:203
136591310,0ptp4l401ktimersoftd/322:45:223
2083993025,3cyclictest41-21ksoftirqd/322:55:173
2083993024,4cyclictest41-21ksoftirqd/323:18:493
2083993024,4cyclictest41-21ksoftirqd/320:02:233
2082993025,3cyclictest33-21ksoftirqd/220:58:352
2082993024,4cyclictest33-21ksoftirqd/223:18:392
2082993024,4cyclictest33-21ksoftirqd/223:12:292
2082993024,4cyclictest33-21ksoftirqd/223:00:462
2082993024,4cyclictest33-21ksoftirqd/222:31:252
2082993024,4cyclictest33-21ksoftirqd/222:13:522
2082993024,4cyclictest33-21ksoftirqd/219:34:102
2082993024,4cyclictest33-21ksoftirqd/219:12:022
2082993016,12cyclictest33-21ksoftirqd/219:50:202
2081993026,3cyclictest25-21ksoftirqd/122:32:461
2081993025,3cyclictest25-21ksoftirqd/120:20:561
2081993025,3cyclictest25-21ksoftirqd/100:29:581
2081993024,4cyclictest25-21ksoftirqd/123:39:331
2081993024,4cyclictest25-21ksoftirqd/122:52:501
2081993024,4cyclictest25-21ksoftirqd/122:00:521
2081993024,4cyclictest25-21ksoftirqd/120:57:441
2081993024,4cyclictest25-21ksoftirqd/120:55:001
2081993024,4cyclictest25-21ksoftirqd/100:17:441
2081993023,5cyclictest25-21ksoftirqd/121:39:451
208199302,25cyclictest8221-21kworker/1:120:35:151
208099303,2cyclictest121rcu_preempt00:18:510
2080993024,4cyclictest9-21ksoftirqd/023:54:230
2080993024,4cyclictest9-21ksoftirqd/023:47:170
2080993024,4cyclictest9-21ksoftirqd/023:39:310
2080993024,4cyclictest9-21ksoftirqd/021:37:360
2080993024,4cyclictest9-21ksoftirqd/020:03:270
2080993024,4cyclictest9-21ksoftirqd/000:31:060
2080993011,16cyclictest7424-21sh19:20:170
2080993011,16cyclictest32232-21sed20:10:220
2080993011,16cyclictest20588-21cpuspeed23:10:140
208099300,16cyclictest0-21swapper/020:58:470
136591300,0ptp4l401ktimersoftd/300:25:013
2083992924,3cyclictest41-21ksoftirqd/323:47:533
2083992924,3cyclictest41-21ksoftirqd/322:26:283
2083992924,3cyclictest41-21ksoftirqd/322:07:093
2083992924,3cyclictest41-21ksoftirqd/319:41:143
2083992923,4cyclictest41-21ksoftirqd/319:23:243
2083992923,4cyclictest41-21ksoftirqd/300:33:383
2082992924,3cyclictest33-21ksoftirqd/221:43:242
2082992924,3cyclictest33-21ksoftirqd/220:47:452
2082992924,3cyclictest33-21ksoftirqd/220:18:052
2082992923,4cyclictest33-21ksoftirqd/222:15:012
2082992923,4cyclictest33-21ksoftirqd/222:00:402
2082992923,4cyclictest33-21ksoftirqd/220:24:122
2081992924,3cyclictest25-21ksoftirqd/123:48:581
2081992924,3cyclictest25-21ksoftirqd/122:43:031
2081992924,3cyclictest25-21ksoftirqd/121:40:141
2081992923,4cyclictest25-21ksoftirqd/123:16:181
2081992923,4cyclictest25-21ksoftirqd/119:17:041
2081992923,3cyclictest25-21ksoftirqd/121:11:221
208199290,27cyclictest3094-21kworker/1:022:47:331
2080992924,3cyclictest9-21ksoftirqd/021:49:190
2080992923,4cyclictest9-21ksoftirqd/019:18:060
2080992912,15cyclictest19530-21cpuspeed_turbos19:45:160
2080992911,16cyclictest24499-21sed21:00:280
2080992911,15cyclictest29794-21sed22:20:220
136591290,0ptp4l401ktimersoftd/323:52:313
136591290,0ptp4l401ktimersoftd/322:35:573
136591290,0ptp4l401ktimersoftd/320:30:333
136591290,0ptp4l401ktimersoftd/320:14:213
136591290,0ptp4l401ktimersoftd/319:37:183
2083992823,3cyclictest41-21ksoftirqd/321:05:363
2083992823,3cyclictest41-21ksoftirqd/300:01:133
2083992822,4cyclictest41-21ksoftirqd/321:16:093
2082992823,3cyclictest33-21ksoftirqd/222:58:302
2082992823,3cyclictest33-21ksoftirqd/222:53:002
2082992823,3cyclictest33-21ksoftirqd/222:44:592
2082992823,3cyclictest33-21ksoftirqd/221:21:542
2082992822,4cyclictest33-21ksoftirqd/221:48:452
2081992823,3cyclictest25-21ksoftirqd/120:31:081
2081992822,4cyclictest25-21ksoftirqd/123:50:351
2081992822,4cyclictest25-21ksoftirqd/123:22:161
2081992822,4cyclictest25-21ksoftirqd/120:15:401
2081992822,4cyclictest25-21ksoftirqd/119:46:241
2081992822,4cyclictest25-21ksoftirqd/119:37:491
2081992821,5cyclictest25-21ksoftirqd/122:27:551
2081992821,4cyclictest25-21ksoftirqd/100:23:051
208199280,27cyclictest3117-21kworker/1:020:04:211
208199280,27cyclictest18432-21kworker/1:022:09:411
208099289,16cyclictest31029-21sed00:35:290
2080992823,3cyclictest9-21ksoftirqd/021:24:510
2080992823,3cyclictest9-21ksoftirqd/019:42:100
2080992822,4cyclictest9-21ksoftirqd/021:28:160
2080992822,4cyclictest9-21ksoftirqd/020:49:180
2080992822,4cyclictest9-21ksoftirqd/019:37:090
208099281,25cyclictest1630-21cpuspeed_turbos20:15:170
208099281,24cyclictest25482-21cut21:05:150
2083992722,3cyclictest41-21ksoftirqd/323:55:463
2083992722,3cyclictest41-21ksoftirqd/322:17:033
2083992722,3cyclictest41-21ksoftirqd/321:12:003
2083992722,3cyclictest41-21ksoftirqd/321:02:163
2083992721,4cyclictest41-21ksoftirqd/300:39:073
2082992723,3cyclictest33-21ksoftirqd/219:28:482
2082992722,3cyclictest33-21ksoftirqd/219:36:122
2082992722,3cyclictest33-21ksoftirqd/200:28:102
2082992721,4cyclictest33-21ksoftirqd/221:01:392
2082992721,4cyclictest33-21ksoftirqd/220:10:122
2081992722,3cyclictest25-21ksoftirqd/119:53:171
2081992722,3cyclictest25-21ksoftirqd/119:42:431
2081992721,4cyclictest25-21ksoftirqd/123:41:091
2081992721,4cyclictest25-21ksoftirqd/122:23:211
2081992721,4cyclictest25-21ksoftirqd/122:00:001
2081992721,4cyclictest25-21ksoftirqd/121:05:021
2081992721,4cyclictest25-21ksoftirqd/119:33:401
2081992721,3cyclictest25-21ksoftirqd/100:35:071
208199270,15cyclictest3094-21kworker/1:023:10:281
2080992722,3cyclictest9-21ksoftirqd/022:55:160
2080992721,4cyclictest9-21ksoftirqd/021:57:100
2080992721,4cyclictest9-21ksoftirqd/020:28:080
2080992721,4cyclictest9-21ksoftirqd/020:09:220
2080992721,4cyclictest9-21ksoftirqd/019:12:580
2080992710,15cyclictest11049-21/usr/sbin/munin20:35:210
136591270,0ptp4l401ktimersoftd/322:34:113
63092260,5sleep1208199cyclictest21:30:261
2083992621,3cyclictest41-21ksoftirqd/323:44:433
2083992621,3cyclictest41-21ksoftirqd/319:49:043
2083992621,3cyclictest41-21ksoftirqd/300:07:113
2083992620,4cyclictest41-21ksoftirqd/320:25:103
2082992621,3cyclictest33-21ksoftirqd/223:26:322
2082992621,3cyclictest33-21ksoftirqd/219:22:182
2082992620,4cyclictest33-21ksoftirqd/223:57:242
2082992620,4cyclictest33-21ksoftirqd/223:50:442
2081992621,3cyclictest25-21ksoftirqd/122:16:221
2081992621,3cyclictest25-21ksoftirqd/121:47:301
2081992621,3cyclictest25-21ksoftirqd/119:10:561
208099269,15cyclictest13549-21grep20:40:150
208099268,15cyclictest31665-21fschecks_count22:25:180
208099268,15cyclictest24581-21fschecks_count19:55:190
208099267,16cyclictest276992chrt21:10:040
208099267,16cyclictest24000-21cpuspeed_turbos22:10:140
208099261,22cyclictest226-21systemd-journal20:20:110
2080992611,12cyclictest23626-21ps00:20:240
208099260,22cyclictest31517-21/usr/sbin/munin21:15:260
136591260,0ptp4l401ktimersoftd/323:33:273
136591260,0ptp4l401ktimersoftd/323:33:273
2083992519,4cyclictest41-21ksoftirqd/300:13:093
2083992518,4cyclictest41-21ksoftirqd/320:55:263
2082992520,3cyclictest33-21ksoftirqd/221:15:212
2082992520,3cyclictest33-21ksoftirqd/220:08:392
2082992519,4cyclictest33-21ksoftirqd/223:31:462
2082992519,4cyclictest33-21ksoftirqd/223:31:462
2082992519,4cyclictest33-21ksoftirqd/222:37:592
2082992519,4cyclictest33-21ksoftirqd/221:28:462
2082992519,4cyclictest33-21ksoftirqd/219:46:052
2081992520,3cyclictest25-21ksoftirqd/119:27:221
208199250,23cyclictest8221-21kworker/1:120:43:051
208099253,17cyclictest3702-21apt-get22:35:140
2080992520,3cyclictest9-21ksoftirqd/023:04:110
208099251,22cyclictest7184-21ls22:40:220
208099251,22cyclictest3460-21idleruntime-cro22:35:010
208099251,21cyclictest23387-21fschecks_time23:15:170
208099251,19cyclictest24708-21apt-get00:25:140
208099250,22cyclictest21881-21/usr/sbin/munin22:05:150
2083992418,4cyclictest41-21ksoftirqd/323:38:423
2082992419,3cyclictest33-21ksoftirqd/221:05:212
2082992419,3cyclictest33-21ksoftirqd/220:30:122
2081992419,3cyclictest25-21ksoftirqd/123:30:121
2081992419,3cyclictest25-21ksoftirqd/123:30:121
2081992418,4cyclictest25-21ksoftirqd/122:38:431
136591240,0ptp4l401ktimersoftd/323:20:143
136591240,0ptp4l401ktimersoftd/321:50:123
136591240,0ptp4l401ktimersoftd/320:05:183
2082992319,3cyclictest33-21ksoftirqd/223:05:242
2082992318,3cyclictest33-21ksoftirqd/222:22:042
2082992318,3cyclictest33-21ksoftirqd/220:53:412
2082992318,3cyclictest33-21ksoftirqd/220:38:122
2082992318,3cyclictest33-21ksoftirqd/219:43:052
2082992317,4cyclictest33-21ksoftirqd/221:10:332
2081992318,3cyclictest25-21ksoftirqd/123:59:011
208199231,21cyclictest2567-21kworker/1:121:28:451
136591230,0ptp4l401ktimersoftd/321:55:153
2082992217,3cyclictest33-21ksoftirqd/222:45:552
2082992217,3cyclictest33-21ksoftirqd/221:54:002
2082992217,3cyclictest33-21ksoftirqd/220:03:232
2081992217,3cyclictest25-21ksoftirqd/121:50:321
2081992116,3cyclictest25-21ksoftirqd/123:06:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional