You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 05:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot4.osadl.org (updated Thu Sep 18, 2025 00:45:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
286352207174,22sleep20-21swapper/219:09:182
134791204171,22phc2sys0-21swapper/319:08:053
283892202168,22sleep10-21swapper/119:06:151
286332201166,23sleep00-21swapper/019:09:160
151612620,5sleep215163-21unixbench_singl00:10:302
2898599531,16cyclictest25277-21/usr/sbin/munin00:30:303
28982994438,4cyclictest21951-21idleruntime-cro21:09:590
28985994336,4cyclictest31224-21cron19:15:013
134991430,0ptp4l401ktimersoftd/323:28:443
28984994237,3cyclictest6071-21sshd19:26:452
28983994237,3cyclictest25-21ksoftirqd/120:40:001
28983994235,4cyclictest20084-21cron22:15:011
28983994235,4cyclictest20084-21cron22:15:011
28983994136,3cyclictest25-21ksoftirqd/123:20:001
28983994136,3cyclictest25-21ksoftirqd/122:44:591
134991410,0ptp4l401ktimersoftd/321:04:183
28983994036,2cyclictest25-21ksoftirqd/122:34:591
28983994035,2cyclictest25-21ksoftirqd/100:35:001
2898399402,35cyclictest12485-21sh22:00:001
134991400,0ptp4l401ktimersoftd/322:30:203
134991400,0ptp4l401ktimersoftd/320:30:223
28983993935,2cyclictest16378-21latency_hist19:50:011
28985993832,4cyclictest41-21ksoftirqd/320:02:013
28985993814,3cyclictest41-21ksoftirqd/300:29:393
28983993833,3cyclictest25-21ksoftirqd/123:54:591
28983993832,4cyclictest25-21ksoftirqd/120:02:521
2898399380,35cyclictest13734-21cron19:45:001
28982993833,3cyclictest9-21ksoftirqd/021:19:100
28985993731,4cyclictest41-21ksoftirqd/321:57:073
28985993731,4cyclictest41-21ksoftirqd/300:20:013
28985993731,3cyclictest41-21ksoftirqd/319:58:543
28983993731,4cyclictest25-21ksoftirqd/121:43:551
28983993731,4cyclictest25-21ksoftirqd/121:02:501
28983993731,3cyclictest25-21ksoftirqd/120:25:191
28983993730,4cyclictest25-21ksoftirqd/121:36:401
28983993727,7cyclictest25-21ksoftirqd/121:00:011
28982993731,4cyclictest9-21ksoftirqd/023:16:560
28982993730,4cyclictest9-21ksoftirqd/020:56:180
134991370,1ptp4l391rcuc/319:15:033
28985993631,3cyclictest41-21ksoftirqd/323:44:433
28985993630,4cyclictest41-21ksoftirqd/320:51:573
28985993630,4cyclictest41-21ksoftirqd/320:35:393
28985993630,4cyclictest41-21ksoftirqd/300:37:513
28985993629,4cyclictest41-21ksoftirqd/322:23:153
28985993629,4cyclictest41-21ksoftirqd/300:06:223
28984993631,3cyclictest33-21ksoftirqd/221:55:002
28984993630,4cyclictest33-21ksoftirqd/223:45:422
28983993632,2cyclictest25-21ksoftirqd/100:24:591
28983993631,3cyclictest25-21ksoftirqd/123:15:001
28983993631,3cyclictest25-21ksoftirqd/119:35:181
28983993630,4cyclictest25-21ksoftirqd/122:57:391
28983993630,4cyclictest25-21ksoftirqd/121:54:291
28983993630,4cyclictest25-21ksoftirqd/120:41:191
28983993630,4cyclictest25-21ksoftirqd/119:13:261
28982993631,3cyclictest9-21ksoftirqd/021:42:130
28982993630,4cyclictest9-21ksoftirqd/000:36:200
28985993529,4cyclictest41-21ksoftirqd/323:03:393
28985993529,4cyclictest41-21ksoftirqd/321:53:263
28985993529,4cyclictest41-21ksoftirqd/321:27:233
28985993529,4cyclictest41-21ksoftirqd/319:48:303
28985993529,4cyclictest41-21ksoftirqd/300:21:023
28984993530,3cyclictest33-21ksoftirqd/222:37:442
28984993529,4cyclictest33-21ksoftirqd/223:39:052
28984993529,4cyclictest33-21ksoftirqd/223:39:052
28984993529,4cyclictest33-21ksoftirqd/221:26:252
28984993529,4cyclictest33-21ksoftirqd/220:48:432
28984993529,4cyclictest33-21ksoftirqd/220:24:312
28984993529,4cyclictest33-21ksoftirqd/220:04:582
28984993529,4cyclictest33-21ksoftirqd/219:21:002
28984993529,4cyclictest33-21ksoftirqd/219:13:552
28983993531,2cyclictest25-21ksoftirqd/122:50:001
28983993530,3cyclictest25-21ksoftirqd/123:45:201
28983993530,3cyclictest25-21ksoftirqd/121:20:181
28983993530,3cyclictest25-21ksoftirqd/121:11:411
28983993530,3cyclictest25-21ksoftirqd/119:55:141
28983993530,3cyclictest25-21ksoftirqd/119:50:301
28983993530,3cyclictest25-21ksoftirqd/119:16:421
28983993529,4cyclictest25-21ksoftirqd/120:51:541
28983993529,3cyclictest25-21ksoftirqd/123:35:151
28983993529,3cyclictest25-21ksoftirqd/123:35:151
28982993531,2cyclictest4222-21sshd21:37:120
28982993529,4cyclictest9-21ksoftirqd/023:06:300
28982993529,4cyclictest9-21ksoftirqd/023:06:290
28982993529,4cyclictest9-21ksoftirqd/022:55:340
28982993529,4cyclictest9-21ksoftirqd/022:39:460
28982993529,4cyclictest9-21ksoftirqd/020:42:550
28985993429,3cyclictest41-21ksoftirqd/322:36:153
28985993429,3cyclictest41-21ksoftirqd/322:28:213
28985993429,3cyclictest41-21ksoftirqd/300:10:113
28985993428,4cyclictest41-21ksoftirqd/322:41:573
28985993428,4cyclictest41-21ksoftirqd/320:06:233
28985993428,4cyclictest41-21ksoftirqd/319:31:163
28985993427,5cyclictest41-21ksoftirqd/321:14:343
2898499349,22cyclictest1280-21runrttasks00:35:232
28984993429,3cyclictest9163-21sshd21:47:142
28984993429,3cyclictest33-21ksoftirqd/223:08:212
28984993429,3cyclictest33-21ksoftirqd/223:08:212
28984993429,3cyclictest33-21ksoftirqd/222:44:012
28983993430,3cyclictest25-21ksoftirqd/121:10:001
28983993430,3cyclictest25-21ksoftirqd/100:15:231
28983993429,3cyclictest25-21ksoftirqd/122:35:201
28983993429,3cyclictest25-21ksoftirqd/121:50:011
28983993429,3cyclictest25-21ksoftirqd/120:15:201
28983993428,4cyclictest25-21ksoftirqd/123:25:041
28983993428,4cyclictest25-21ksoftirqd/122:09:371
28983993428,4cyclictest25-21ksoftirqd/121:17:301
28983993428,4cyclictest25-21ksoftirqd/120:48:371
28983993428,4cyclictest25-21ksoftirqd/119:33:031
28983993428,4cyclictest25-21ksoftirqd/100:07:441
28982993429,3cyclictest9-21ksoftirqd/023:32:530
28982993428,4cyclictest9-21ksoftirqd/023:52:590
28982993428,4cyclictest9-21ksoftirqd/023:39:350
28982993428,4cyclictest9-21ksoftirqd/023:39:350
28982993428,4cyclictest9-21ksoftirqd/022:52:290
28982993428,4cyclictest9-21ksoftirqd/022:34:530
28982993428,4cyclictest9-21ksoftirqd/022:03:430
28982993428,4cyclictest9-21ksoftirqd/021:22:510
28982993428,4cyclictest9-21ksoftirqd/000:00:020
28982993427,4cyclictest9-21ksoftirqd/020:25:000
134991340,0ptp4l401ktimersoftd/323:09:553
134991340,0ptp4l401ktimersoftd/323:09:553
28985993328,3cyclictest41-21ksoftirqd/322:55:533
28985993328,2cyclictest41-21ksoftirqd/320:20:203
28985993327,4cyclictest41-21ksoftirqd/323:45:523
28985993327,4cyclictest41-21ksoftirqd/320:17:273
28985993327,4cyclictest41-21ksoftirqd/319:24:003
28985993327,3cyclictest41-21ksoftirqd/323:56:203
28985993326,4cyclictest41-21ksoftirqd/322:10:003
2898499336,26cyclictest0-21swapper/220:55:272
28984993328,3cyclictest33-21ksoftirqd/223:35:002
28984993327,4cyclictest33-21ksoftirqd/223:27:212
28984993327,4cyclictest33-21ksoftirqd/222:11:312
28984993327,4cyclictest33-21ksoftirqd/222:11:312
28984993327,4cyclictest33-21ksoftirqd/220:34:592
28984993327,4cyclictest33-21ksoftirqd/219:50:412
28984993327,4cyclictest33-21ksoftirqd/219:49:572
28984993327,4cyclictest33-21ksoftirqd/219:34:532
28983993328,3cyclictest25-21ksoftirqd/123:40:211
28983993328,3cyclictest25-21ksoftirqd/121:30:201
28983993328,3cyclictest25-21ksoftirqd/121:27:531
28983993328,3cyclictest25-21ksoftirqd/120:31:061
28983993328,3cyclictest25-21ksoftirqd/119:30:001
28983993328,3cyclictest25-21ksoftirqd/119:23:331
28983993328,3cyclictest25-21ksoftirqd/100:40:001
28983993327,4cyclictest25-21ksoftirqd/120:08:491
28983993326,4cyclictest25-21ksoftirqd/122:25:581
28982993328,3cyclictest9-21ksoftirqd/021:54:480
28982993328,3cyclictest9-21ksoftirqd/020:54:110
28982993327,4cyclictest9-21ksoftirqd/023:44:240
28982993327,4cyclictest9-21ksoftirqd/023:04:330
28982993327,4cyclictest9-21ksoftirqd/020:03:380
28982993327,4cyclictest9-21ksoftirqd/000:34:390
28982993327,4cyclictest9-21ksoftirqd/000:13:170
28985993227,3cyclictest41-21ksoftirqd/323:12:393
28985993227,3cyclictest41-21ksoftirqd/322:51:513
28985993227,3cyclictest41-21ksoftirqd/320:57:433
28985993226,4cyclictest41-21ksoftirqd/322:46:313
28985993226,4cyclictest41-21ksoftirqd/321:46:053
28985993226,4cyclictest41-21ksoftirqd/321:20:573
28985993226,4cyclictest41-21ksoftirqd/319:29:073
28985993226,4cyclictest41-21ksoftirqd/300:03:333
28985993225,5cyclictest41-21ksoftirqd/322:18:263
28984993227,3cyclictest33-21ksoftirqd/223:12:442
28984993227,3cyclictest33-21ksoftirqd/221:40:382
28984993227,3cyclictest33-21ksoftirqd/221:07:002
28984993226,4cyclictest33-21ksoftirqd/222:31:182
28984993226,4cyclictest33-21ksoftirqd/221:21:172
28984993226,4cyclictest33-21ksoftirqd/219:38:142
28984993226,4cyclictest33-21ksoftirqd/219:18:152
28984993226,4cyclictest33-21ksoftirqd/200:32:572
28984993226,4cyclictest33-21ksoftirqd/200:27:152
28984993226,4cyclictest33-21ksoftirqd/200:03:322
28983993227,3cyclictest25-21ksoftirqd/123:20:201
28983993227,3cyclictest25-21ksoftirqd/123:10:001
28983993227,3cyclictest25-21ksoftirqd/123:10:001
28983993227,3cyclictest25-21ksoftirqd/122:00:541
28983993226,4cyclictest25-21ksoftirqd/123:32:541
28983993226,4cyclictest25-21ksoftirqd/120:14:391
28983993226,4cyclictest25-21ksoftirqd/120:14:391
28983993226,3cyclictest25-21ksoftirqd/122:20:141
28982993227,3cyclictest9-21ksoftirqd/022:10:360
28982993227,3cyclictest9-21ksoftirqd/022:10:360
28982993227,3cyclictest9-21ksoftirqd/021:03:120
28982993227,3cyclictest9-21ksoftirqd/020:34:310
28982993227,3cyclictest9-21ksoftirqd/019:21:340
28982993226,4cyclictest9-21ksoftirqd/023:47:530
28982993226,4cyclictest9-21ksoftirqd/023:13:560
28982993226,4cyclictest9-21ksoftirqd/022:27:580
28982993226,4cyclictest9-21ksoftirqd/021:55:480
28982993226,4cyclictest9-21ksoftirqd/021:27:170
28985993126,3cyclictest41-21ksoftirqd/322:11:053
28985993126,3cyclictest41-21ksoftirqd/322:11:053
28985993126,3cyclictest41-21ksoftirqd/320:13:093
28985993126,3cyclictest41-21ksoftirqd/320:13:093
28985993125,4cyclictest41-21ksoftirqd/321:44:573
28984993126,3cyclictest33-21ksoftirqd/219:57:422
28984993125,4cyclictest33-21ksoftirqd/222:21:062
28984993125,4cyclictest33-21ksoftirqd/221:19:162
28984993125,4cyclictest33-21ksoftirqd/220:52:512
28984993125,4cyclictest33-21ksoftirqd/220:40:372
28984993124,5cyclictest33-21ksoftirqd/221:37:122
28984993117,12cyclictest8771-21kworker/2:020:05:272
28983993126,3cyclictest25-21ksoftirqd/123:03:371
28982993126,3cyclictest9-21ksoftirqd/020:37:380
28982993126,3cyclictest9-21ksoftirqd/019:58:090
28982993126,3cyclictest9-21ksoftirqd/019:40:090
28982993125,4cyclictest9-21ksoftirqd/023:25:330
28982993125,4cyclictest9-21ksoftirqd/020:05:520
2898599304,6cyclictest41-21ksoftirqd/321:35:313
28985993026,3cyclictest41-21ksoftirqd/321:30:433
28985993025,3cyclictest41-21ksoftirqd/323:24:583
28985993025,3cyclictest41-21ksoftirqd/323:17:393
28985993025,3cyclictest41-21ksoftirqd/322:01:363
28985993025,3cyclictest41-21ksoftirqd/320:26:563
28985993024,4cyclictest41-21ksoftirqd/320:46:383
28985993014,4cyclictest41-21ksoftirqd/323:39:573
28985993014,4cyclictest41-21ksoftirqd/323:39:573
28984993025,3cyclictest33-21ksoftirqd/223:52:432
28984993025,3cyclictest33-21ksoftirqd/220:17:392
28984993025,3cyclictest33-21ksoftirqd/219:43:042
28984993024,4cyclictest33-21ksoftirqd/221:32:192
28983993025,3cyclictest25-21ksoftirqd/122:15:431
28983993025,3cyclictest25-21ksoftirqd/100:15:011
28983993024,4cyclictest25-21ksoftirqd/100:27:341
28983993024,3cyclictest25-21ksoftirqd/123:58:371
28983993023,5cyclictest25-21ksoftirqd/122:55:011
28982993025,3cyclictest9-21ksoftirqd/022:42:580
28982993025,3cyclictest9-21ksoftirqd/022:20:030
28982993024,4cyclictest9-21ksoftirqd/020:14:550
28982993024,4cyclictest9-21ksoftirqd/020:14:550
28982993024,4cyclictest9-21ksoftirqd/019:11:200
28982993024,3cyclictest9-21ksoftirqd/019:16:570
113122300,4sleep011311-21tr19:40:010
28985992924,3cyclictest41-21ksoftirqd/319:36:443
28985992923,4cyclictest41-21ksoftirqd/323:31:223
28984992924,3cyclictest33-21ksoftirqd/222:56:262
28984992924,3cyclictest33-21ksoftirqd/222:19:552
28984992924,3cyclictest33-21ksoftirqd/221:04:052
28984992924,3cyclictest33-21ksoftirqd/220:29:222
28984992924,3cyclictest33-21ksoftirqd/200:23:502
28984992924,3cyclictest33-21ksoftirqd/200:18:542
28984992923,4cyclictest33-21ksoftirqd/223:43:022
28984992923,4cyclictest33-21ksoftirqd/222:07:092
28983992924,3cyclictest25-21ksoftirqd/120:20:111
28983992923,4cyclictest25-21ksoftirqd/100:00:421
28982992924,3cyclictest9-21ksoftirqd/021:47:460
28982992924,3cyclictest9-21ksoftirqd/021:12:510
28982992924,3cyclictest9-21ksoftirqd/019:27:060
28982992923,4cyclictest9-21ksoftirqd/000:25:530
28985992823,3cyclictest41-21ksoftirqd/319:41:573
2898499286,19cyclictest5313-21apt-get22:50:112
28984992823,3cyclictest33-21ksoftirqd/223:01:052
28984992823,3cyclictest33-21ksoftirqd/222:27:062
28984992822,4cyclictest33-21ksoftirqd/221:57:132
2898499281,13cyclictest22032-21ls23:20:272
28982992823,3cyclictest9-21ksoftirqd/000:06:030
28982992822,4cyclictest9-21ksoftirqd/020:25:410
28982992822,4cyclictest9-21ksoftirqd/020:17:440
28982992822,4cyclictest9-21ksoftirqd/000:16:460
28985992722,3cyclictest41-21ksoftirqd/323:52:523
2898499272,4cyclictest5536-21apt-get23:55:132
28984992722,3cyclictest33-21ksoftirqd/222:45:022
28984992722,3cyclictest33-21ksoftirqd/220:38:302
28982992722,3cyclictest9-21ksoftirqd/000:20:550
28982992721,4cyclictest9-21ksoftirqd/019:33:270
36998260,2rtkit-daemon368-21rtkit-daemon21:13:442
28985992620,4cyclictest41-21ksoftirqd/320:44:303
28984992621,3cyclictest33-21ksoftirqd/222:03:382
28984992620,4cyclictest33-21ksoftirqd/200:08:462
28984992619,5cyclictest33-21ksoftirqd/223:15:422
28982992620,4cyclictest9-21ksoftirqd/019:48:260
28985992520,3cyclictest41-21ksoftirqd/321:15:353
28985992520,3cyclictest41-21ksoftirqd/321:07:383
28982992520,3cyclictest9-21ksoftirqd/022:17:500
28982992520,3cyclictest9-21ksoftirqd/022:06:590
28982992418,4cyclictest9-21ksoftirqd/023:55:120
2898499230,22cyclictest0-21swapper/220:10:402
2898499230,22cyclictest0-21swapper/220:10:402
28982992318,3cyclictest9-21ksoftirqd/022:45:510
28982992318,3cyclictest9-21ksoftirqd/021:34:140
28982992318,3cyclictest9-21ksoftirqd/020:45:560
134991230,0ptp4l401ktimersoftd/319:50:173
28982992117,3cyclictest9-21ksoftirqd/019:51:270
28982992015,3cyclictest22586-21sh23:24:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional