You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 06:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot4.osadl.org (updated Fri May 09, 2025 00:45:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292652203171,21sleep10-21swapper/119:09:461
134791201167,23phc2sys0-21swapper/319:05:433
290552200166,22sleep20-21swapper/219:07:012
289452199165,22sleep00-21swapper/019:05:340
292882161122,13sleep10-21swapper/119:10:021
134991730,1ptp4l0-21swapper/322:17:363
118372690,6sleep22958299cyclictest21:50:002
235352670,3sleep20-21swapper/222:10:262
29583996432,4cyclictest41-21ksoftirqd/319:35:003
77372630,5sleep20-21swapper/223:50:152
134991580,1ptp4l401ktimersoftd/321:00:493
130691570,1getstats0-21swapper/320:55:063
130691570,1getstats0-21swapper/320:55:063
134991530,1ptp4l0-21swapper/321:44:343
29583995121,3cyclictest41-21ksoftirqd/319:55:213
29583995121,3cyclictest41-21ksoftirqd/319:55:213
29583995023,6cyclictest41-21ksoftirqd/320:10:013
29583995021,4cyclictest41-21ksoftirqd/300:25:223
29583995018,7cyclictest41-21ksoftirqd/321:20:003
29583994918,3cyclictest41-21ksoftirqd/321:40:013
29583994916,4cyclictest41-21ksoftirqd/300:06:513
29583994913,8cyclictest41-21ksoftirqd/319:35:143
29583994816,3cyclictest41-21ksoftirqd/323:30:013
29583994815,3cyclictest41-21ksoftirqd/319:25:003
29581994843,3cyclictest9521-21sh20:40:011
29583994723,6cyclictest41-21ksoftirqd/320:25:213
29583994723,6cyclictest41-21ksoftirqd/320:25:213
29583994717,6cyclictest41-21ksoftirqd/322:15:013
29583994714,4cyclictest41-21ksoftirqd/323:35:183
29583994615,8cyclictest41-21ksoftirqd/320:10:013
29581994640,3cyclictest227-21systemd-journal00:01:511
134991460,1ptp4l401ktimersoftd/300:12:583
2958399443,8cyclictest41-21ksoftirqd/319:10:143
29583994416,4cyclictest41-21ksoftirqd/322:40:203
29583994416,3cyclictest41-21ksoftirqd/322:55:203
29583994415,4cyclictest41-21ksoftirqd/320:40:153
29583994414,4cyclictest41-21ksoftirqd/321:29:593
29583994411,3cyclictest41-21ksoftirqd/321:15:003
29583994410,3cyclictest41-21ksoftirqd/319:20:003
2958399438,4cyclictest41-21ksoftirqd/323:10:243
29583994338,2cyclictest41-21ksoftirqd/323:35:013
29583994313,4cyclictest41-21ksoftirqd/319:45:273
29581994336,4cyclictest16935-21cron20:55:001
134991430,0ptp4l401ktimersoftd/321:45:093
2958399428,4cyclictest41-21ksoftirqd/300:00:213
2958399425,9cyclictest41-21ksoftirqd/323:45:153
29583994215,7cyclictest41-21ksoftirqd/323:20:003
29583994215,3cyclictest41-21ksoftirqd/300:15:283
29583994214,7cyclictest41-21ksoftirqd/323:20:243
29583994213,4cyclictest41-21ksoftirqd/320:00:183
29582994214,26cyclictest0-21swapper/220:50:272
134991420,0ptp4l401ktimersoftd/320:30:473
36998410,8rtkit-daemon0-21swapper/200:20:222
2958399418,3cyclictest41-21ksoftirqd/322:30:123
2958399412,8cyclictest41-21ksoftirqd/321:22:563
29583994115,7cyclictest41-21ksoftirqd/321:05:163
29583994113,3cyclictest41-21ksoftirqd/322:20:123
29583994111,3cyclictest41-21ksoftirqd/323:00:283
29583994111,3cyclictest41-21ksoftirqd/320:50:003
134991410,0ptp4l401ktimersoftd/320:16:553
2958399406,8cyclictest41-21ksoftirqd/323:55:143
2958399405,4cyclictest41-21ksoftirqd/323:55:013
29583994033,4cyclictest41-21ksoftirqd/322:30:013
29583994029,3cyclictest41-21ksoftirqd/319:40:143
29583994011,8cyclictest41-21ksoftirqd/300:39:593
29582994012,27cyclictest0-21swapper/219:50:272
29582994011,27cyclictest0-21swapper/223:25:372
2958399396,8cyclictest41-21ksoftirqd/321:30:223
29583993910,4cyclictest41-21ksoftirqd/323:05:563
130691390,1getstats0-21swapper/320:50:063
130691390,1getstats0-21swapper/320:35:063
130691390,1getstats0-21swapper/319:50:073
2958399385,7cyclictest41-21ksoftirqd/321:55:173
2958399385,7cyclictest41-21ksoftirqd/321:55:173
2958399383,32cyclictest2899-21irqrtprio22:35:203
29583993828,3cyclictest41-21ksoftirqd/322:00:263
29582993815,21cyclictest0-21swapper/223:55:282
29582993812,25cyclictest0-21swapper/221:15:282
29581993831,4cyclictest25-21ksoftirqd/100:10:131
2958399377,4cyclictest41-21ksoftirqd/319:25:193
2958399377,4cyclictest41-21ksoftirqd/319:25:193
29583993733,3cyclictest41-21ksoftirqd/322:45:153
29583993711,6cyclictest41-21ksoftirqd/300:30:143
29582993731,4cyclictest33-21ksoftirqd/222:19:282
29582993730,5cyclictest33-21ksoftirqd/200:19:272
29583993632,2cyclictest41-21ksoftirqd/300:20:173
29583993631,3cyclictest41-21ksoftirqd/321:55:003
29583993630,4cyclictest41-21ksoftirqd/323:44:023
29583993629,4cyclictest41-21ksoftirqd/322:54:203
29582993630,4cyclictest33-21ksoftirqd/222:00:352
29582993630,4cyclictest33-21ksoftirqd/221:14:052
29582993630,3cyclictest33-21ksoftirqd/220:10:002
29582993629,5cyclictest33-21ksoftirqd/223:10:072
29581993630,4cyclictest25-21ksoftirqd/123:36:181
29581993630,4cyclictest25-21ksoftirqd/123:31:201
29581993630,4cyclictest25-21ksoftirqd/119:31:381
29581993629,5cyclictest25-21ksoftirqd/120:35:001
29579993630,4cyclictest9-21ksoftirqd/023:18:070
29579993630,4cyclictest9-21ksoftirqd/021:08:500
29579993629,4cyclictest9-21ksoftirqd/022:25:200
29579993621,5cyclictest9-21ksoftirqd/020:19:470
2958399355,8cyclictest41-21ksoftirqd/320:20:113
29583993532,2cyclictest41-21ksoftirqd/322:05:273
29582993530,3cyclictest33-21ksoftirqd/220:19:332
29582993529,4cyclictest33-21ksoftirqd/220:25:472
29582993529,4cyclictest33-21ksoftirqd/220:25:472
29582993529,4cyclictest33-21ksoftirqd/219:46:062
29581993529,4cyclictest25-21ksoftirqd/123:56:361
29581993529,4cyclictest25-21ksoftirqd/123:26:391
29581993529,4cyclictest25-21ksoftirqd/122:44:071
29581993529,4cyclictest25-21ksoftirqd/121:18:501
29581993529,4cyclictest25-21ksoftirqd/120:13:391
29581993529,4cyclictest25-21ksoftirqd/100:36:061
29579993529,4cyclictest9-21ksoftirqd/022:44:170
29579993529,4cyclictest9-21ksoftirqd/021:53:340
29579993529,4cyclictest9-21ksoftirqd/021:26:230
29579993529,4cyclictest9-21ksoftirqd/020:26:580
29579993529,4cyclictest9-21ksoftirqd/020:26:580
2958299347,26cyclictest0-21swapper/221:35:282
2958299346,26cyclictest0-21swapper/220:40:172
29582993428,4cyclictest33-21ksoftirqd/223:08:112
29582993428,4cyclictest33-21ksoftirqd/222:29:352
29582993428,4cyclictest33-21ksoftirqd/222:05:122
29582993428,4cyclictest33-21ksoftirqd/219:41:262
29582993428,4cyclictest33-21ksoftirqd/219:32:332
29582993428,4cyclictest33-21ksoftirqd/200:04:542
29582993427,4cyclictest33-21ksoftirqd/223:49:432
2958299341,16cyclictest7091-21latency_hist20:35:002
2958299340,3cyclictest17122-21latency_hist00:10:002
29581993428,4cyclictest25-21ksoftirqd/121:53:281
29581993428,4cyclictest25-21ksoftirqd/121:39:321
29581993428,4cyclictest25-21ksoftirqd/121:24:551
29581993428,4cyclictest25-21ksoftirqd/121:10:431
29581993428,4cyclictest25-21ksoftirqd/120:59:561
29581993428,4cyclictest25-21ksoftirqd/120:59:561
29581993427,5cyclictest25-21ksoftirqd/100:16:101
29579993428,4cyclictest9-21ksoftirqd/023:27:290
29579993428,4cyclictest9-21ksoftirqd/023:09:570
29579993428,4cyclictest9-21ksoftirqd/021:41:590
29579993428,4cyclictest9-21ksoftirqd/021:17:360
29579993428,4cyclictest9-21ksoftirqd/020:51:570
29579993428,4cyclictest9-21ksoftirqd/020:34:080
29579993428,4cyclictest9-21ksoftirqd/020:08:520
29582993327,4cyclictest33-21ksoftirqd/223:16:042
29582993327,4cyclictest33-21ksoftirqd/223:02:452
29582993327,4cyclictest33-21ksoftirqd/222:56:232
29582993327,4cyclictest33-21ksoftirqd/222:51:022
29582993327,4cyclictest33-21ksoftirqd/221:54:362
29582993327,4cyclictest33-21ksoftirqd/221:00:412
29582993327,4cyclictest33-21ksoftirqd/219:23:192
29582993327,4cyclictest33-21ksoftirqd/219:12:102
29582993326,4cyclictest33-21ksoftirqd/220:50:002
29581993328,3cyclictest26971-21latency_hist20:10:001
29581993327,4cyclictest25-21ksoftirqd/123:40:591
29581993327,4cyclictest25-21ksoftirqd/120:23:501
29581993327,4cyclictest25-21ksoftirqd/119:20:191
29581993327,4cyclictest25-21ksoftirqd/100:23:081
2958199330,3cyclictest13672-21kworker/1:120:02:471
29579993327,4cyclictest9-21ksoftirqd/023:49:130
29579993327,4cyclictest9-21ksoftirqd/023:42:510
29579993327,4cyclictest9-21ksoftirqd/022:56:090
29579993327,4cyclictest9-21ksoftirqd/022:34:380
29579993327,4cyclictest9-21ksoftirqd/022:19:260
29579993327,4cyclictest9-21ksoftirqd/021:00:510
29579993327,4cyclictest9-21ksoftirqd/020:48:440
29579993327,4cyclictest9-21ksoftirqd/000:11:540
29579993326,5cyclictest9-21ksoftirqd/000:06:170
29582993226,4cyclictest33-21ksoftirqd/221:40:532
29582993226,4cyclictest33-21ksoftirqd/221:21:562
29582993226,3cyclictest33-21ksoftirqd/221:06:052
29582993210,3cyclictest1751-21seq19:17:572
29581993227,3cyclictest25-21ksoftirqd/122:21:311
29581993226,4cyclictest25-21ksoftirqd/123:51:511
29581993226,4cyclictest25-21ksoftirqd/122:07:241
29581993226,4cyclictest25-21ksoftirqd/122:04:351
29581993226,4cyclictest25-21ksoftirqd/119:50:371
29581993226,4cyclictest25-21ksoftirqd/100:31:411
2958199321,2cyclictest0-21swapper/121:00:271
29581993212,18cyclictest0-21swapper/123:12:271
29579993226,4cyclictest9-21ksoftirqd/022:06:150
29579993226,4cyclictest9-21ksoftirqd/019:39:520
29579993226,4cyclictest9-21ksoftirqd/019:30:300
29579993226,4cyclictest9-21ksoftirqd/000:19:260
29579993225,4cyclictest9-21ksoftirqd/021:23:420
29582993126,3cyclictest33-21ksoftirqd/222:42:582
29582993126,3cyclictest33-21ksoftirqd/200:31:372
29582993112,3cyclictest14544-21/usr/sbin/munin21:55:232
29582993112,3cyclictest14544-21/usr/sbin/munin21:55:232
29582993111,2cyclictest1797-21latency_hist21:30:002
29581993126,3cyclictest25-21ksoftirqd/100:09:121
29581993125,4cyclictest25-21ksoftirqd/122:46:271
29581993125,4cyclictest25-21ksoftirqd/120:46:321
29581993125,4cyclictest25-21ksoftirqd/119:39:491
29581993125,4cyclictest25-21ksoftirqd/100:28:361
29579993126,3cyclictest9-21ksoftirqd/021:57:520
29579993126,3cyclictest9-21ksoftirqd/021:57:520
29579993126,3cyclictest9-21ksoftirqd/020:41:380
29579993126,3cyclictest9-21ksoftirqd/020:38:210
29579993126,3cyclictest9-21ksoftirqd/020:10:120
29579993126,3cyclictest9-21ksoftirqd/019:41:240
29579993125,4cyclictest9-21ksoftirqd/023:34:090
29579993125,4cyclictest9-21ksoftirqd/021:36:540
29579993125,4cyclictest9-21ksoftirqd/020:56:220
29579993125,4cyclictest9-21ksoftirqd/020:56:220
29579993125,4cyclictest9-21ksoftirqd/019:51:230
29579993125,4cyclictest9-21ksoftirqd/000:30:330
29579993125,3cyclictest9-21ksoftirqd/020:21:520
29582993025,3cyclictest33-21ksoftirqd/222:33:012
29582993025,3cyclictest33-21ksoftirqd/222:20:492
29582993024,4cyclictest33-21ksoftirqd/223:38:242
29582993024,4cyclictest33-21ksoftirqd/222:48:072
29582993024,4cyclictest33-21ksoftirqd/222:36:082
29582993024,4cyclictest33-21ksoftirqd/220:14:122
29582993024,4cyclictest33-21ksoftirqd/220:03:132
29582993024,3cyclictest33-21ksoftirqd/200:25:122
29581993025,3cyclictest25-21ksoftirqd/121:48:361
29581993025,3cyclictest25-21ksoftirqd/120:17:001
29579993023,4cyclictest9-21ksoftirqd/019:45:580
29582992924,3cyclictest33-21ksoftirqd/223:41:592
29582992923,4cyclictest33-21ksoftirqd/220:35:062
29582992923,4cyclictest33-21ksoftirqd/220:21:592
29582992913,3cyclictest24846-21/usr/sbin/munin23:20:222
29581992924,3cyclictest25-21ksoftirqd/122:55:221
29581992923,4cyclictest25-21ksoftirqd/122:36:481
29581992923,4cyclictest25-21ksoftirqd/122:13:291
29581992923,4cyclictest25-21ksoftirqd/119:19:031
29581992915,12cyclictest0-21swapper/123:15:071
29579992924,3cyclictest9-21ksoftirqd/022:36:180
29579992923,4cyclictest9-21ksoftirqd/023:51:580
29579992923,4cyclictest9-21ksoftirqd/023:12:500
29579992923,4cyclictest9-21ksoftirqd/022:20:350
29579992923,4cyclictest9-21ksoftirqd/021:49:380
29579992923,4cyclictest9-21ksoftirqd/000:29:040
29579992923,4cyclictest9-21ksoftirqd/000:02:520
2958299285,3cyclictest0-21swapper/220:55:152
2958299285,3cyclictest0-21swapper/220:55:152
29582992823,3cyclictest33-21ksoftirqd/223:31:282
29582992822,4cyclictest33-21ksoftirqd/219:25:272
29582992822,4cyclictest33-21ksoftirqd/219:25:262
2958299280,14cyclictest10803-21phc2sys19:35:232
2958199287,19cyclictest0-21swapper/122:51:271
29581992824,3cyclictest25-21ksoftirqd/121:43:281
29581992823,3cyclictest25-21ksoftirqd/121:30:371
29581992822,4cyclictest25-21ksoftirqd/121:06:301
29581992822,4cyclictest25-21ksoftirqd/120:28:191
29581992822,4cyclictest25-21ksoftirqd/120:28:191
29581992822,4cyclictest25-21ksoftirqd/119:45:061
29579992823,3cyclictest9-21ksoftirqd/023:23:000
29579992823,3cyclictest9-21ksoftirqd/020:03:370
29579992823,3cyclictest9-21ksoftirqd/019:29:440
29579992823,3cyclictest9-21ksoftirqd/019:29:430
29579992822,4cyclictest9-21ksoftirqd/023:58:070
29579992822,4cyclictest9-21ksoftirqd/022:50:440
29579992822,4cyclictest9-21ksoftirqd/021:31:200
29582992714,11cyclictest0-21swapper/219:55:172
29582992714,11cyclictest0-21swapper/219:55:172
29581992722,3cyclictest25-21ksoftirqd/123:21:261
29579992721,4cyclictest9-21ksoftirqd/023:00:380
29582992612,2cyclictest0-21swapper/221:33:112
29581992621,3cyclictest25-21ksoftirqd/123:45:301
29581992621,3cyclictest25-21ksoftirqd/120:40:101
29581992621,3cyclictest25-21ksoftirqd/119:55:011
29581992621,3cyclictest25-21ksoftirqd/119:55:001
29581992620,4cyclictest25-21ksoftirqd/119:27:131
29581992620,4cyclictest25-21ksoftirqd/119:27:131
29579992621,3cyclictest9-21ksoftirqd/022:00:130
29579992621,3cyclictest9-21ksoftirqd/019:13:070
29579992620,4cyclictest9-21ksoftirqd/022:11:480
2958299254,18cyclictest29610-21apt-get00:35:122
2958199255,18cyclictest0-21swapper/122:34:071
29581992520,3cyclictest25-21ksoftirqd/121:57:411
29581992520,3cyclictest25-21ksoftirqd/121:57:411
29579992520,3cyclictest9-21ksoftirqd/022:49:030
29579992520,3cyclictest9-21ksoftirqd/021:14:350
29579992519,4cyclictest9-21ksoftirqd/019:23:110
29579992519,4cyclictest9-21ksoftirqd/000:22:300
29581992419,3cyclictest25-21ksoftirqd/123:05:531
29581992418,4cyclictest25-21ksoftirqd/122:25:051
29579992419,3cyclictest9-21ksoftirqd/000:38:540
29579992418,4cyclictest9-21ksoftirqd/019:56:170
29579992418,4cyclictest9-21ksoftirqd/019:56:170
29582992317,4cyclictest33-21ksoftirqd/200:13:132
29581992319,3cyclictest25-21ksoftirqd/123:01:241
29581992318,3cyclictest25-21ksoftirqd/121:29:411
29579992015,3cyclictest9-21ksoftirqd/023:37:530
29581991813,3cyclictest25-21ksoftirqd/122:18:111
29579991813,3cyclictest1230-21timerandwakeup19:15:260
29581991712,3cyclictest1280-21runrttasks19:40:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional