You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-17 - 09:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot3.osadl.org (updated Mon Mar 17, 2025 00:45:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
284622233168,22sleep10-21swapper/119:07:051
137191207172,23phc2sys0-21swapper/319:09:283
285252206141,22sleep00-21swapper/019:07:530
284702204170,23sleep20-21swapper/219:07:112
1371918363,10phc2sys0-21swapper/319:10:003
136991650,2ptp4l401ktimersoftd/320:05:233
136991580,1ptp4l401ktimersoftd/321:55:183
285812530,4sleep328583-21latency_hist20:20:013
2898399460,44cyclictest0-21swapper/123:10:261
2898399460,1cyclictest0-21swapper/122:15:271
2898399440,2cyclictest0-21swapper/120:40:261
136991430,0ptp4l401ktimersoftd/322:32:033
28985994236,3cyclictest41-21ksoftirqd/300:00:003
136991420,0ptp4l401ktimersoftd/323:20:223
136991420,0ptp4l401ktimersoftd/320:22:183
136991420,0ptp4l401ktimersoftd/319:31:523
59298410,9rtkit-daemon0-21swapper/121:23:351
59298410,8rtkit-daemon0-21swapper/121:43:451
136991410,1ptp4l401ktimersoftd/321:03:013
136991410,0ptp4l401ktimersoftd/320:10:153
132691400,2getstats0-21swapper/320:02:273
132691400,1getstats0-21swapper/319:57:263
28985993934,3cyclictest41-21ksoftirqd/323:10:003
28985993934,3cyclictest41-21ksoftirqd/300:15:003
28985993933,3cyclictest41-21ksoftirqd/323:00:013
28983993933,3cyclictest22854-21cron00:00:001
2898399390,37cyclictest0-21swapper/122:50:161
136991390,1ptp4l401ktimersoftd/300:18:333
136991390,0ptp4l401ktimersoftd/319:50:103
59298380,7rtkit-daemon0-21swapper/121:46:551
28985993833,2cyclictest41-21ksoftirqd/321:19:593
28985993831,4cyclictest41-21ksoftirqd/300:10:003
28985993825,10cyclictest41-21ksoftirqd/319:15:223
28983993832,4cyclictest25-21ksoftirqd/119:20:001
2898399380,36cyclictest0-21swapper/121:10:361
59298370,6rtkit-daemon6237-21wc21:55:151
28984993732,3cyclictest33-21ksoftirqd/221:12:242
59298360,7rtkit-daemon0-21swapper/121:34:061
28985993631,3cyclictest41-21ksoftirqd/321:14:593
28985993631,3cyclictest41-21ksoftirqd/321:00:013
28985993631,3cyclictest41-21ksoftirqd/320:34:593
28985993630,3cyclictest41-21ksoftirqd/322:30:003
28985993630,3cyclictest41-21ksoftirqd/322:30:003
28985993630,3cyclictest41-21ksoftirqd/321:25:003
28985993625,10cyclictest41-21ksoftirqd/300:20:283
28984993631,3cyclictest33-21ksoftirqd/219:57:132
28984993630,4cyclictest33-21ksoftirqd/220:55:112
28984993630,4cyclictest33-21ksoftirqd/220:08:192
136991360,0ptp4l401ktimersoftd/323:35:203
28985993532,2cyclictest41-21ksoftirqd/319:25:003
28985993530,3cyclictest41-21ksoftirqd/323:02:413
28985993530,3cyclictest41-21ksoftirqd/322:35:123
28985993530,3cyclictest41-21ksoftirqd/321:05:143
28985993530,3cyclictest41-21ksoftirqd/320:25:123
28985993530,3cyclictest41-21ksoftirqd/319:40:013
28985993529,3cyclictest41-21ksoftirqd/322:15:263
28985993529,3cyclictest41-21ksoftirqd/320:45:143
28985993528,4cyclictest41-21ksoftirqd/321:55:013
28984993529,4cyclictest33-21ksoftirqd/220:37:172
28984993529,4cyclictest33-21ksoftirqd/220:29:262
28984993529,4cyclictest33-21ksoftirqd/219:33:412
28983993529,4cyclictest25-21ksoftirqd/122:56:121
28983993528,4cyclictest25-21ksoftirqd/123:50:261
28983993528,4cyclictest25-21ksoftirqd/123:06:541
28983993528,4cyclictest25-21ksoftirqd/122:14:531
28983993527,5cyclictest25-21ksoftirqd/122:23:541
2898299353,3cyclictest121rcu_preempt19:11:230
28982993529,4cyclictest9-21ksoftirqd/022:26:070
28982993529,4cyclictest9-21ksoftirqd/022:26:060
28982993529,4cyclictest9-21ksoftirqd/020:10:380
59298340,2rtkit-daemon0-21swapper/023:11:430
28985993429,3cyclictest41-21ksoftirqd/323:35:003
28985993429,3cyclictest41-21ksoftirqd/322:42:093
28985993428,4cyclictest41-21ksoftirqd/323:14:473
28984993429,3cyclictest33-21ksoftirqd/222:10:072
28984993429,3cyclictest33-21ksoftirqd/200:38:472
28984993429,3cyclictest33-21ksoftirqd/200:11:102
28984993428,4cyclictest33-21ksoftirqd/223:28:182
28984993428,4cyclictest33-21ksoftirqd/223:12:422
28984993428,4cyclictest33-21ksoftirqd/222:24:592
28984993428,4cyclictest33-21ksoftirqd/219:43:452
28984993428,3cyclictest33-21ksoftirqd/220:22:192
28984993427,4cyclictest33-21ksoftirqd/219:50:032
28983993428,4cyclictest25-21ksoftirqd/120:06:231
28983993428,4cyclictest25-21ksoftirqd/119:29:281
28983993427,5cyclictest25-21ksoftirqd/123:32:461
28983993427,5cyclictest25-21ksoftirqd/121:38:461
28983993427,4cyclictest25-21ksoftirqd/120:25:071
28983993427,4cyclictest25-21ksoftirqd/100:21:241
28982993429,3cyclictest9-21ksoftirqd/022:09:210
28982993427,4cyclictest9-21ksoftirqd/022:47:160
136991340,0ptp4l401ktimersoftd/319:49:023
28985993329,3cyclictest41-21ksoftirqd/321:40:203
28985993329,2cyclictest41-21ksoftirqd/300:05:003
28985993328,3cyclictest41-21ksoftirqd/321:50:003
28985993328,3cyclictest41-21ksoftirqd/320:45:003
28985993328,3cyclictest41-21ksoftirqd/300:40:013
28985993327,4cyclictest41-21ksoftirqd/322:50:443
28985993327,3cyclictest41-21ksoftirqd/323:45:003
28985993327,3cyclictest41-21ksoftirqd/323:20:013
28984993329,3cyclictest33-21ksoftirqd/223:04:552
28984993328,3cyclictest33-21ksoftirqd/223:06:292
28984993328,3cyclictest33-21ksoftirqd/219:38:552
28984993327,4cyclictest33-21ksoftirqd/223:42:422
28984993327,4cyclictest33-21ksoftirqd/221:46:412
28984993327,4cyclictest33-21ksoftirqd/221:44:552
28984993327,4cyclictest33-21ksoftirqd/221:39:442
28984993327,4cyclictest33-21ksoftirqd/219:25:472
28983993327,4cyclictest25-21ksoftirqd/123:47:341
28983993327,4cyclictest25-21ksoftirqd/123:43:411
28983993327,4cyclictest25-21ksoftirqd/120:39:351
28983993327,4cyclictest25-21ksoftirqd/119:57:111
28983993326,4cyclictest25-21ksoftirqd/121:51:241
28983993326,4cyclictest25-21ksoftirqd/121:29:221
28983993326,4cyclictest25-21ksoftirqd/121:29:221
28983993326,4cyclictest25-21ksoftirqd/120:14:171
28982993328,3cyclictest9-21ksoftirqd/021:45:210
28982993327,4cyclictest9-21ksoftirqd/019:20:360
28982993326,4cyclictest9-21ksoftirqd/023:41:120
28982993310,20cyclictest23667-21sed00:00:150
28985993228,2cyclictest41-21ksoftirqd/322:49:593
28985993227,3cyclictest41-21ksoftirqd/322:25:003
28985993227,3cyclictest41-21ksoftirqd/322:15:013
28985993227,3cyclictest41-21ksoftirqd/321:40:013
28985993227,3cyclictest41-21ksoftirqd/320:35:253
28984993227,3cyclictest33-21ksoftirqd/223:39:052
28984993226,4cyclictest33-21ksoftirqd/222:04:332
28984993226,4cyclictest33-21ksoftirqd/220:48:072
28984993226,3cyclictest33-21ksoftirqd/221:05:172
28983993226,4cyclictest25-21ksoftirqd/123:02:341
28983993226,4cyclictest25-21ksoftirqd/100:27:301
28983993226,3cyclictest25-21ksoftirqd/123:27:171
28982993228,3cyclictest9-21ksoftirqd/021:24:550
28982993227,3cyclictest9-21ksoftirqd/000:05:140
28982993226,4cyclictest9-21ksoftirqd/023:59:380
28982993226,4cyclictest9-21ksoftirqd/023:05:290
28982993226,4cyclictest9-21ksoftirqd/022:54:020
28982993226,4cyclictest9-21ksoftirqd/022:30:520
28982993226,4cyclictest9-21ksoftirqd/022:16:110
28982993226,4cyclictest9-21ksoftirqd/021:17:340
28982993226,4cyclictest9-21ksoftirqd/021:11:040
28982993226,4cyclictest9-21ksoftirqd/021:04:220
28982993226,4cyclictest9-21ksoftirqd/020:59:360
28982993226,4cyclictest9-21ksoftirqd/019:45:480
28982993225,5cyclictest9-21ksoftirqd/021:33:340
28982993211,18cyclictest7459-21grep20:40:260
28982993210,19cyclictest3854-21cat21:50:100
28982993210,19cyclictest27723-21seq21:26:320
28982993210,19cyclictest27723-21seq21:26:320
136991320,0ptp4l401ktimersoftd/322:09:273
136991320,0ptp4l401ktimersoftd/321:34:323
28985993128,2cyclictest41-21ksoftirqd/323:29:593
28985993125,4cyclictest41-21ksoftirqd/321:25:403
28985993125,4cyclictest41-21ksoftirqd/321:25:393
28985993125,4cyclictest41-21ksoftirqd/319:28:263
28985993119,3cyclictest41-21ksoftirqd/319:40:233
28984993126,3cyclictest33-21ksoftirqd/220:42:552
28984993126,3cyclictest33-21ksoftirqd/219:21:142
28984993126,3cyclictest33-21ksoftirqd/200:20:002
28984993125,4cyclictest33-21ksoftirqd/223:56:542
28984993125,4cyclictest33-21ksoftirqd/221:18:332
28984993125,4cyclictest33-21ksoftirqd/219:45:542
28984993123,5cyclictest33-21ksoftirqd/222:44:472
28983993126,3cyclictest25-21ksoftirqd/122:36:451
28983993126,3cyclictest25-21ksoftirqd/121:09:511
28983993126,3cyclictest25-21ksoftirqd/100:05:001
28983993125,4cyclictest25-21ksoftirqd/123:21:271
28983993125,4cyclictest25-21ksoftirqd/122:41:591
28983993125,4cyclictest25-21ksoftirqd/122:26:071
28983993125,4cyclictest25-21ksoftirqd/122:26:071
28983993125,4cyclictest25-21ksoftirqd/122:08:531
28983993125,4cyclictest25-21ksoftirqd/120:30:011
28983993125,4cyclictest25-21ksoftirqd/120:04:101
28982993126,3cyclictest9-21ksoftirqd/023:24:150
28982993126,3cyclictest9-21ksoftirqd/020:38:100
28982993126,3cyclictest9-21ksoftirqd/020:29:530
28982993126,3cyclictest9-21ksoftirqd/000:23:470
28982993125,4cyclictest9-21ksoftirqd/022:42:380
28982993125,4cyclictest9-21ksoftirqd/022:10:290
28982993125,4cyclictest9-21ksoftirqd/020:52:340
28982993114,14cyclictest9-21ksoftirqd/000:35:120
28982993113,16cyclictest21500-21/usr/sbin/munin20:05:250
2898299311,28cyclictest20004-21ls23:50:220
28982993110,19cyclictest27640-21tr20:15:240
28985993027,2cyclictest41-21ksoftirqd/322:00:203
28985993025,3cyclictest41-21ksoftirqd/300:30:273
28985993024,4cyclictest41-21ksoftirqd/323:49:213
28985993024,3cyclictest41-21ksoftirqd/323:50:003
28984993025,3cyclictest33-21ksoftirqd/222:09:112
28984993025,3cyclictest33-21ksoftirqd/220:17:122
28984993025,3cyclictest33-21ksoftirqd/200:10:002
28984993024,4cyclictest33-21ksoftirqd/222:55:112
28984993024,4cyclictest33-21ksoftirqd/222:46:562
28984993024,4cyclictest33-21ksoftirqd/222:28:332
28984993024,4cyclictest33-21ksoftirqd/222:28:322
28984993024,4cyclictest33-21ksoftirqd/221:57:322
28984993024,4cyclictest33-21ksoftirqd/220:10:402
28984993023,4cyclictest33-21ksoftirqd/221:24:512
28984993023,4cyclictest33-21ksoftirqd/219:12:482
28983993025,3cyclictest25-21ksoftirqd/120:46:381
28983993024,4cyclictest25-21ksoftirqd/119:45:181
28983993023,4cyclictest25-21ksoftirqd/119:40:071
28983993023,4cyclictest25-21ksoftirqd/100:40:001
2898299309,18cyclictest11990-21grep23:30:220
28982993025,3cyclictest9-21ksoftirqd/022:39:050
28982993025,3cyclictest9-21ksoftirqd/019:44:220
28982993025,3cyclictest9-21ksoftirqd/000:25:430
28982993025,3cyclictest9-21ksoftirqd/000:13:280
28982993024,4cyclictest9-21ksoftirqd/023:03:080
28982993024,4cyclictest9-21ksoftirqd/022:56:420
28982993024,4cyclictest9-21ksoftirqd/022:23:490
28982993024,4cyclictest9-21ksoftirqd/019:15:510
28982993024,3cyclictest9-21ksoftirqd/000:34:550
28982993024,3cyclictest9-21ksoftirqd/000:19:170
28982993014,14cyclictest18314-21fschecks_time21:05:140
28982993012,15cyclictest8965-21cat19:35:140
28985992924,3cyclictest41-21ksoftirqd/320:50:183
28985992924,3cyclictest41-21ksoftirqd/300:30:003
28984992924,3cyclictest33-21ksoftirqd/200:25:532
28984992923,4cyclictest33-21ksoftirqd/222:38:072
28984992923,4cyclictest33-21ksoftirqd/222:16:152
28983992923,3cyclictest25-21ksoftirqd/100:30:481
28983992922,4cyclictest25-21ksoftirqd/122:31:161
28982992924,3cyclictest9-21ksoftirqd/023:47:530
28982992924,3cyclictest9-21ksoftirqd/021:35:110
59298280,2rtkit-daemon0-21swapper/120:15:161
28984992823,3cyclictest33-21ksoftirqd/222:50:152
28984992823,3cyclictest33-21ksoftirqd/222:31:342
28984992823,3cyclictest33-21ksoftirqd/221:33:532
28984992823,3cyclictest33-21ksoftirqd/220:02:462
28984992823,3cyclictest33-21ksoftirqd/219:15:552
28983992823,3cyclictest25-21ksoftirqd/123:37:241
28983992822,4cyclictest25-21ksoftirqd/121:01:431
28983992821,4cyclictest25-21ksoftirqd/119:51:451
2898299288,17cyclictest2031-21grep20:30:240
28982992823,3cyclictest9-21ksoftirqd/023:16:280
28982992823,3cyclictest9-21ksoftirqd/022:00:580
28982992823,3cyclictest9-21ksoftirqd/019:28:090
28982992821,5cyclictest9-21ksoftirqd/019:54:140
28982992811,15cyclictest8148-21/usr/sbin/munin20:45:210
28984992722,3cyclictest33-21ksoftirqd/221:51:422
28984992722,3cyclictest33-21ksoftirqd/220:32:172
28984992722,3cyclictest33-21ksoftirqd/200:24:242
28984992721,4cyclictest33-21ksoftirqd/221:04:082
28984992721,4cyclictest33-21ksoftirqd/200:00:552
28983992722,3cyclictest25-21ksoftirqd/120:53:431
2898299274,4cyclictest14891-21latency_hist23:40:000
2898299274,3cyclictest0-21swapper/020:20:230
28982992721,4cyclictest9-21ksoftirqd/021:40:120
28982992721,4cyclictest9-21ksoftirqd/019:56:150
28982992711,13cyclictest19069-21/usr/sbin/munin20:00:130
28984992621,3cyclictest33-21ksoftirqd/223:51:532
28984992621,3cyclictest33-21ksoftirqd/200:30:532
28984992619,4cyclictest33-21ksoftirqd/221:27:012
28984992619,4cyclictest33-21ksoftirqd/221:27:012
28983992621,3cyclictest25-21ksoftirqd/100:14:261
28983992620,4cyclictest25-21ksoftirqd/122:47:571
28983992620,4cyclictest25-21ksoftirqd/119:14:431
28983992620,4cyclictest25-21ksoftirqd/100:16:471
28983992620,3cyclictest25-21ksoftirqd/119:30:151
28983992620,3cyclictest25-21ksoftirqd/100:08:451
2898299265,4cyclictest99432chrt23:28:240
28982992621,3cyclictest9-21ksoftirqd/021:56:330
28982992621,3cyclictest9-21ksoftirqd/019:32:110
28984992520,3cyclictest33-21ksoftirqd/223:47:462
28984992520,3cyclictest33-21ksoftirqd/223:22:342
28983992520,3cyclictest25-21ksoftirqd/123:18:211
28983992519,3cyclictest25-21ksoftirqd/121:19:541
28983992519,3cyclictest25-21ksoftirqd/120:22:301
59298240,2rtkit-daemon0-21swapper/122:04:041
2898499244,17cyclictest12584-21cat23:35:002
28984992419,3cyclictest33-21ksoftirqd/220:50:012
2898499233,17cyclictest6871-21latency_hist23:20:002
28983992217,3cyclictest25-21ksoftirqd/120:57:131
28983992014,4cyclictest25-21ksoftirqd/119:38:351
28983991712,3cyclictest3522-21sshd19:22:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional