You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 04:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Fri Dec 05, 2025 00:45:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40652208175,22sleep30-21swapper/319:09:383
38982205171,22sleep20-21swapper/219:07:282
38972205171,23sleep10-21swapper/119:07:271
37412202168,23sleep00-21swapper/019:05:330
2253121740,5sleep2438799cyclictest20:55:152
678321410,8sleep2438799cyclictest21:30:172
438799825,75cyclictest13401-21kworker/2:000:10:242
93982650,3sleep00-21swapper/019:20:010
93982650,3sleep00-21swapper/019:20:010
221902570,2sleep022194-21irqrtprio19:45:200
136591570,50ptp4l0-21swapper/319:10:003
136591560,1ptp4l391rcuc/321:40:243
438799553,17cyclictest19501-21df_abs19:40:142
4388995411,11cyclictest32700-21chrt00:36:523
4388995012,10cyclictest31419-21chrt21:11:453
438899489,10cyclictest7790-21snmpd22:49:003
4388994810,11cyclictest28230-21chrt22:14:073
4388994810,11cyclictest28230-21chrt22:14:063
438799486,6cyclictest28970-21cut00:30:192
4387994611,5cyclictest33-21ksoftirqd/221:38:262
438799458,7cyclictest1820-21cat21:20:012
438799447,7cyclictest26004-21cat22:10:002
136591440,0ptp4l401ktimersoftd/322:20:413
136591440,0ptp4l401ktimersoftd/300:29:593
438899439,11cyclictest30959-21latency_hist20:05:013
4388994311,18cyclictest25368-21cat23:15:193
136591430,0ptp4l401ktimersoftd/323:59:243
51698420,8rtkit-daemon0-21swapper/122:46:011
438799425,6cyclictest31411-21fschecks_time22:20:172
136591420,0ptp4l401ktimersoftd/321:21:173
136591420,0ptp4l401ktimersoftd/319:55:163
4388994114,11cyclictest22843-21grep00:15:273
4388994114,11cyclictest22843-21grep00:15:273
438899411,11cyclictest24383-21seq23:13:383
438799418,5cyclictest33-21ksoftirqd/223:48:542
438899407,16cyclictest7790-21snmpd20:30:323
4388994011,17cyclictest226-21systemd-journal20:35:133
136591400,0ptp4l401ktimersoftd/321:00:343
136591400,0ptp4l401ktimersoftd/320:05:153
133391400,1getstats31478-1kworker/3:0H21:49:263
4388993910,19cyclictest226-21systemd-journal19:35:043
438799397,16cyclictest22523-21cat22:00:212
133391390,1getstats31478-1kworker/3:0H22:04:263
438899389,17cyclictest4268-21/usr/sbin/munin23:40:203
438899389,11cyclictest25328-21cat19:50:253
4388993812,14cyclictest226-21systemd-journal21:55:303
4388993812,12cyclictest9958-21awk20:25:293
136591380,0ptp4l401ktimersoftd/320:53:273
136591380,0ptp4l401ktimersoftd/319:40:043
438899379,12cyclictest19633-21sshd23:03:283
438799371,11cyclictest24237-21perl00:20:192
133391370,1getstats31478-1kworker/3:0H22:09:263
133391370,1getstats31478-1kworker/3:0H21:54:263
438899369,18cyclictest28623-21ls23:20:263
438899369,11cyclictest10486-21sensors23:50:243
438899367,17cyclictest0-21swapper/320:10:143
438899359,17cyclictest1981-21cat22:25:233
438899359,16cyclictest3922-21cat22:30:203
438899357,17cyclictest0-21swapper/319:20:033
438899357,17cyclictest0-21swapper/319:20:033
438899356,10cyclictest0-21swapper/322:45:003
438799354,28cyclictest20935-21kworker/2:100:15:122
438799354,28cyclictest20935-21kworker/2:100:15:122
4387993510,24cyclictest21548-21kworker/2:123:30:352
438899348,14cyclictest31938-21apt-get21:15:133
438899347,17cyclictest0-21swapper/323:25:213
438899338,11cyclictest23581-21sshd19:49:253
438899338,10cyclictest6698-21/usr/sbin/munin23:45:203
438899336,17cyclictest0-21swapper/323:05:253
4388993327,4cyclictest41-21ksoftirqd/320:24:393
438799336,25cyclictest863-21kworker/2:022:30:352
4387993326,4cyclictest33-21ksoftirqd/222:14:262
4387993326,4cyclictest33-21ksoftirqd/222:14:262
4387993326,4cyclictest33-21ksoftirqd/220:00:152
438799331,31cyclictest32546-21kworker/2:119:10:052
438799331,31cyclictest13401-21kworker/2:023:59:282
4386993327,4cyclictest25-21ksoftirqd/119:51:151
4386993327,3cyclictest25-21ksoftirqd/121:20:131
4385993327,4cyclictest9-21ksoftirqd/022:26:080
4385993327,4cyclictest9-21ksoftirqd/022:06:220
4385993327,4cyclictest9-21ksoftirqd/021:31:360
438899328,16cyclictest3382-21/usr/sbin/munin20:15:273
438899328,15cyclictest4934-21if_enp1s021:25:203
438899328,14cyclictest16749-21ls22:55:283
4388993227,3cyclictest41-21ksoftirqd/321:05:143
4387993226,4cyclictest33-21ksoftirqd/222:26:342
4386993227,3cyclictest25-21ksoftirqd/121:43:081
4386993226,4cyclictest25-21ksoftirqd/122:17:331
4386993226,4cyclictest25-21ksoftirqd/119:11:071
4385993227,3cyclictest9-21ksoftirqd/022:10:150
4385993227,3cyclictest9-21ksoftirqd/022:10:150
4385993227,3cyclictest9-21ksoftirqd/019:36:300
4385993226,4cyclictest9-21ksoftirqd/023:41:580
4385993226,4cyclictest9-21ksoftirqd/023:39:130
4385993226,4cyclictest9-21ksoftirqd/022:32:020
4385993226,4cyclictest9-21ksoftirqd/022:24:350
4385993226,4cyclictest9-21ksoftirqd/021:55:420
4385993226,4cyclictest9-21ksoftirqd/021:02:390
4385993226,4cyclictest9-21ksoftirqd/019:26:150
136591320,1ptp4l391rcuc/300:06:433
438899318,10cyclictest0-21swapper/319:25:133
438899310,19cyclictest0-21swapper/320:45:133
4387993125,4cyclictest33-21ksoftirqd/220:08:052
4387993110,5cyclictest33-21ksoftirqd/223:03:312
4386993126,3cyclictest25-21ksoftirqd/121:20:001
4386993126,3cyclictest25-21ksoftirqd/120:10:211
4386993125,4cyclictest25-21ksoftirqd/123:22:051
4386993125,4cyclictest25-21ksoftirqd/100:27:401
4386993125,4cyclictest25-21ksoftirqd/100:21:231
4386993124,5cyclictest25-21ksoftirqd/100:15:371
4386993124,5cyclictest25-21ksoftirqd/100:15:371
4385993126,3cyclictest9-21ksoftirqd/019:33:410
4385993125,4cyclictest9-21ksoftirqd/023:16:580
4385993125,4cyclictest9-21ksoftirqd/021:45:430
4385993125,4cyclictest9-21ksoftirqd/020:26:330
4385993125,4cyclictest9-21ksoftirqd/019:11:180
438899308,7cyclictest14175-21apt-get19:30:113
438899300,10cyclictest7790-21snmpd23:36:583
4387993025,3cyclictest33-21ksoftirqd/222:55:402
4387993024,4cyclictest33-21ksoftirqd/223:08:222
4387993024,4cyclictest33-21ksoftirqd/219:37:202
438799301,5cyclictest0-21swapper/222:35:262
4386993026,3cyclictest25-21ksoftirqd/123:14:271
4386993026,2cyclictest25-21ksoftirqd/122:39:591
4386993025,3cyclictest25-21ksoftirqd/123:39:471
4386993025,3cyclictest25-21ksoftirqd/121:06:151
4386993025,3cyclictest25-21ksoftirqd/120:18:521
4386993025,3cyclictest25-21ksoftirqd/100:05:201
4386993024,4cyclictest25-21ksoftirqd/123:27:271
4386993024,4cyclictest25-21ksoftirqd/122:42:001
4386993024,4cyclictest25-21ksoftirqd/121:56:361
4386993024,4cyclictest25-21ksoftirqd/121:12:501
4386993024,4cyclictest25-21ksoftirqd/120:43:471
4386993024,4cyclictest25-21ksoftirqd/100:33:011
4386993024,4cyclictest25-21ksoftirqd/100:12:121
4386993024,4cyclictest25-21ksoftirqd/100:02:251
4385993025,3cyclictest9-21ksoftirqd/023:00:010
4385993025,3cyclictest9-21ksoftirqd/020:57:020
4385993024,4cyclictest9-21ksoftirqd/022:50:470
4385993024,4cyclictest9-21ksoftirqd/020:41:440
4388992923,4cyclictest41-21ksoftirqd/323:30:053
4388992923,4cyclictest41-21ksoftirqd/322:17:233
4388992923,4cyclictest41-21ksoftirqd/320:57:333
4387992924,3cyclictest33-21ksoftirqd/223:38:012
4387992924,3cyclictest33-21ksoftirqd/222:53:102
4387992924,3cyclictest33-21ksoftirqd/219:35:002
4387992923,4cyclictest33-21ksoftirqd/221:43:322
4387992923,4cyclictest33-21ksoftirqd/219:26:282
4387992923,4cyclictest33-21ksoftirqd/219:21:432
4387992923,4cyclictest33-21ksoftirqd/219:21:422
438799292,17cyclictest226-21systemd-journal00:05:512
438799290,18cyclictest21548-21kworker/2:123:25:242
4386992923,4cyclictest25-21ksoftirqd/123:55:351
4386992923,4cyclictest25-21ksoftirqd/121:33:411
4386992923,4cyclictest25-21ksoftirqd/121:00:161
4386992923,4cyclictest25-21ksoftirqd/120:56:141
4386992923,4cyclictest25-21ksoftirqd/120:47:481
4386992923,4cyclictest25-21ksoftirqd/120:23:091
4386992922,5cyclictest25-21ksoftirqd/122:32:141
4385992924,3cyclictest9-21ksoftirqd/023:11:560
4385992924,3cyclictest9-21ksoftirqd/022:15:420
4385992924,3cyclictest9-21ksoftirqd/021:10:580
4385992923,4cyclictest9-21ksoftirqd/023:52:410
4385992923,4cyclictest9-21ksoftirqd/022:42:090
4385992923,4cyclictest9-21ksoftirqd/020:48:390
4385992923,4cyclictest9-21ksoftirqd/020:20:470
4385992923,4cyclictest9-21ksoftirqd/020:09:560
4385992923,4cyclictest9-21ksoftirqd/020:00:170
4388992823,3cyclictest41-21ksoftirqd/320:42:483
4388992816,9cyclictest11266-21cron21:40:003
4388992810,15cyclictest15010-21latency_hist22:55:013
4387992825,2cyclictest321ktimersoftd/221:45:242
4387992823,3cyclictest33-21ksoftirqd/220:26:452
4387992823,3cyclictest33-21ksoftirqd/219:15:112
4387992822,4cyclictest33-21ksoftirqd/223:19:252
4387992822,4cyclictest33-21ksoftirqd/223:13:352
4387992822,4cyclictest33-21ksoftirqd/221:54:232
4387992822,4cyclictest33-21ksoftirqd/221:22:512
4387992822,4cyclictest33-21ksoftirqd/220:31:552
4387992822,4cyclictest33-21ksoftirqd/200:01:132
4387992822,3cyclictest33-21ksoftirqd/223:40:072
438799282,16cyclictest12954-21apt-get20:35:122
438799282,11cyclictest9425-21df_abs23:50:152
4387992811,5cyclictest33-21ksoftirqd/220:54:292
4386992823,3cyclictest25-21ksoftirqd/123:34:241
4386992823,3cyclictest25-21ksoftirqd/100:35:491
4386992822,4cyclictest25-21ksoftirqd/123:45:471
4386992822,4cyclictest25-21ksoftirqd/122:10:401
4386992822,4cyclictest25-21ksoftirqd/122:10:401
4386992822,4cyclictest25-21ksoftirqd/120:36:001
4386992822,4cyclictest25-21ksoftirqd/120:06:281
4386992822,4cyclictest25-21ksoftirqd/119:47:461
4386992816,11cyclictest0-21swapper/119:21:151
4386992816,11cyclictest0-21swapper/119:21:151
4385992823,3cyclictest9-21ksoftirqd/023:46:290
4385992823,3cyclictest9-21ksoftirqd/019:17:450
4385992822,4cyclictest9-21ksoftirqd/022:45:580
4385992822,4cyclictest9-21ksoftirqd/021:51:130
4385992822,4cyclictest9-21ksoftirqd/020:50:120
4385992822,4cyclictest9-21ksoftirqd/020:37:550
4385992821,5cyclictest9-21ksoftirqd/023:57:590
438899279,15cyclictest7572-21irqrtprio19:15:183
4388992721,4cyclictest41-21ksoftirqd/321:30:513
4388992714,10cyclictest13890-21sort00:00:163
438799274,12cyclictest20935-21kworker/2:100:25:172
4387992722,3cyclictest33-21ksoftirqd/221:08:572
4387992722,3cyclictest33-21ksoftirqd/221:01:002
438799272,23cyclictest32546-21kworker/2:119:51:132
4387992721,4cyclictest33-21ksoftirqd/223:21:172
4387992721,4cyclictest33-21ksoftirqd/220:44:002
4387992721,4cyclictest33-21ksoftirqd/219:55:502
4387992720,4cyclictest33-21ksoftirqd/220:49:532
438799270,3cyclictest30467-21kworker/2:220:15:252
438799270,18cyclictest4661-21df_abs21:25:172
4386992723,2cyclictest25-21ksoftirqd/122:02:091
4386992722,3cyclictest25-21ksoftirqd/122:57:141
4386992722,3cyclictest25-21ksoftirqd/119:35:501
4385992722,3cyclictest9-21ksoftirqd/021:41:220
4385992722,3cyclictest9-21ksoftirqd/000:33:410
4385992722,3cyclictest9-21ksoftirqd/000:20:570
4385992722,3cyclictest9-21ksoftirqd/000:01:460
4385992721,4cyclictest9-21ksoftirqd/023:01:060
4385992721,4cyclictest9-21ksoftirqd/021:37:090
4388992615,8cyclictest7431-21grep22:35:283
4388992615,7cyclictest28642-21cpuspeed_turbos00:30:153
4387992621,3cyclictest33-21ksoftirqd/221:13:492
4386992621,3cyclictest25-21ksoftirqd/122:50:291
4386992621,3cyclictest25-21ksoftirqd/120:30:261
4385992621,3cyclictest9-21ksoftirqd/000:38:380
4385992619,4cyclictest21260-21cron19:45:000
51698250,2rtkit-daemon0-21swapper/000:10:350
4388992515,7cyclictest19035-21sort00:10:153
4388992514,7cyclictest23895-21cpuspeed_turbos00:20:143
438799251,23cyclictest32546-21kworker/2:119:46:052
4386992519,4cyclictest25-21ksoftirqd/122:28:101
4386992517,5cyclictest13460-21grep19:25:291
4385992521,3cyclictest9-21ksoftirqd/023:25:190
4385992521,3cyclictest9-21ksoftirqd/020:14:250
4385992520,3cyclictest9-21ksoftirqd/023:20:150
4385992520,3cyclictest9-21ksoftirqd/019:55:150
4385992520,3cyclictest9-21ksoftirqd/019:54:040
4385992519,4cyclictest9-21ksoftirqd/021:17:160
4385992519,4cyclictest9-21ksoftirqd/000:15:390
4385992519,4cyclictest9-21ksoftirqd/000:15:390
1550122518,3sleep125-21ksoftirqd/121:45:301
438799245,5cyclictest33-21ksoftirqd/200:36:122
438799243,20cyclictest18965-21kworker/2:222:17:252
438799242,21cyclictest863-21kworker/2:022:41:172
4387992419,3cyclictest33-21ksoftirqd/220:24:292
4387992419,3cyclictest33-21ksoftirqd/220:11:262
4386992419,3cyclictest25-21ksoftirqd/123:41:381
4386992418,4cyclictest25-21ksoftirqd/123:51:251
4386992418,3cyclictest25-21ksoftirqd/120:25:141
4385992419,3cyclictest9-21ksoftirqd/021:20:500
4385992419,3cyclictest9-21ksoftirqd/020:15:490
4385992418,4cyclictest9-21ksoftirqd/021:29:150
4387992317,4cyclictest33-21ksoftirqd/222:46:442
438799231,21cyclictest18965-21kworker/2:221:55:342
4386992318,3cyclictest25-21ksoftirqd/120:05:001
4386992317,4cyclictest25-21ksoftirqd/119:31:101
4385992318,3cyclictest9-21ksoftirqd/023:34:200
4385992318,3cyclictest9-21ksoftirqd/000:25:140
51698220,2rtkit-daemon0-21swapper/120:52:471
4386992218,2cyclictest25-21ksoftirqd/121:36:451
4386992217,3cyclictest25-21ksoftirqd/122:05:141
4386992217,3cyclictest25-21ksoftirqd/121:50:181
4386992217,3cyclictest25-21ksoftirqd/119:40:181
4386992117,3cyclictest25-21ksoftirqd/123:15:201
4386992116,3cyclictest25-21ksoftirqd/119:20:001
4385992115,4cyclictest9-21ksoftirqd/023:07:240
4386992015,3cyclictest25-21ksoftirqd/119:55:211
4385992015,3cyclictest9-21ksoftirqd/021:05:220
4385992013,4cyclictest21733-21df22:00:130
4386991914,3cyclictest4518-21grep21:25:171
4386991914,3cyclictest25-21ksoftirqd/123:07:591
4386991914,3cyclictest25-21ksoftirqd/122:21:021
4386991913,4cyclictest25-21ksoftirqd/123:00:121
4385991812,4cyclictest17906-21threads00:05:260
4385991713,2cyclictest6272-21hddtemp_smartct22:35:180
4385991711,3cyclictest10593-21apt-get20:30:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional