You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 10:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Sun Jul 13, 2025 00:45:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137191228168,50phc2sys0-21swapper/319:06:553
114702207173,22sleep10-21swapper/119:06:571
114712205172,22sleep20-21swapper/219:06:582
115702200167,22sleep00-21swapper/019:08:090
2606521310,3sleep10-21swapper/123:00:211
1200899792,75cyclictest19187-21kworker/1:219:46:471
1200899680,48cyclictest10415-21kworker/1:123:16:081
136991600,1ptp4l401ktimersoftd/319:36:243
136991600,1ptp4l401ktimersoftd/319:36:243
129772600,3sleep312958-21systemctl20:19:593
1371915737,10phc2sys0-21swapper/319:10:013
167092540,4sleep21200999cyclictest20:25:212
44172520,2sleep00-21swapper/022:15:230
131982510,2sleep10-21swapper/123:40:241
12007995011,27cyclictest22554-21seq20:38:190
3712470,1sleep00-21swapper/023:15:160
12009994631,12cyclictest33-21ksoftirqd/200:04:592
326522440,1sleep20-21swapper/223:15:142
12009994429,12cyclictest33-21ksoftirqd/222:25:312
12009994338,3cyclictest33-21ksoftirqd/219:20:002
12009994330,11cyclictest33-21ksoftirqd/223:05:002
12007994338,3cyclictest12946-21idleruntime-cro20:20:000
136991420,1ptp4l401ktimersoftd/321:48:223
136991420,0ptp4l401ktimersoftd/300:14:563
12009994112,4cyclictest33-21ksoftirqd/221:25:142
136991400,0ptp4l401ktimersoftd/320:50:143
1200999406,3cyclictest33-21ksoftirqd/200:25:132
1200999405,6cyclictest33-21ksoftirqd/223:40:012
12009994035,3cyclictest33-21ksoftirqd/200:40:002
12009994033,4cyclictest4591-21cron21:10:002
1200899404,35cyclictest19187-21kworker/1:219:36:471
1200899404,35cyclictest19187-21kworker/1:219:36:461
1200899404,35cyclictest19187-21kworker/1:219:34:151
136991390,16ptp4l401ktimersoftd/319:54:213
12009993934,3cyclictest33-21ksoftirqd/222:45:002
12009993934,3cyclictest33-21ksoftirqd/222:35:002
12009993934,3cyclictest33-21ksoftirqd/222:15:012
12008993932,4cyclictest22408-21cron22:55:011
124042380,4sleep312403-21tail22:30:313
1200999384,3cyclictest33-21ksoftirqd/222:45:192
1200999384,3cyclictest33-21ksoftirqd/222:45:192
1200999383,4cyclictest33-21ksoftirqd/221:49:592
12009993834,2cyclictest33-21ksoftirqd/221:05:002
12009993833,3cyclictest33-21ksoftirqd/223:14:592
12009993832,3cyclictest33-21ksoftirqd/222:25:002
12009993832,3cyclictest33-21ksoftirqd/220:20:012
12009993832,3cyclictest33-21ksoftirqd/220:10:172
12009993827,9cyclictest33-21ksoftirqd/223:45:322
136991370,0ptp4l401ktimersoftd/300:00:273
12010993732,3cyclictest41-21ksoftirqd/322:15:303
12010993732,3cyclictest41-21ksoftirqd/320:45:003
12010993732,3cyclictest41-21ksoftirqd/300:30:003
12010993730,5cyclictest41-21ksoftirqd/300:37:273
1200999377,2cyclictest33-21ksoftirqd/220:45:162
12009993733,2cyclictest33-21ksoftirqd/221:14:592
12009993732,3cyclictest33-21ksoftirqd/219:50:212
12009993732,3cyclictest33-21ksoftirqd/219:35:162
12009993732,3cyclictest33-21ksoftirqd/219:35:162
12009993731,4cyclictest33-21ksoftirqd/220:00:052
12009993731,3cyclictest33-21ksoftirqd/223:05:152
12009993731,3cyclictest33-21ksoftirqd/222:10:012
12009993731,3cyclictest33-21ksoftirqd/219:20:182
12009993710,4cyclictest33-21ksoftirqd/222:35:302
12008993731,4cyclictest25-21ksoftirqd/120:42:331
12008993714,22cyclictest27812-21kworker/1:120:55:461
12010993632,2cyclictest41-21ksoftirqd/323:54:593
12010993631,3cyclictest41-21ksoftirqd/322:39:593
12010993630,4cyclictest41-21ksoftirqd/320:27:593
12010993630,3cyclictest41-21ksoftirqd/300:05:243
1200999363,6cyclictest33-21ksoftirqd/200:30:222
12009993632,3cyclictest33-21ksoftirqd/220:10:002
12009993631,3cyclictest33-21ksoftirqd/222:55:012
12009993631,3cyclictest33-21ksoftirqd/222:00:172
12009993631,3cyclictest33-21ksoftirqd/222:00:172
12009993631,3cyclictest33-21ksoftirqd/221:35:152
12009993631,3cyclictest33-21ksoftirqd/221:20:212
12009993631,3cyclictest33-21ksoftirqd/220:45:002
12009993628,6cyclictest33-21ksoftirqd/200:21:182
12009993624,4cyclictest33-21ksoftirqd/219:40:202
1200999361,2cyclictest121rcu_preempt00:05:302
12008993630,4cyclictest25-21ksoftirqd/122:56:281
12008993629,5cyclictest25-21ksoftirqd/121:01:311
12008993629,5cyclictest25-21ksoftirqd/120:34:101
136991350,0ptp4l401ktimersoftd/320:46:253
12010993530,3cyclictest41-21ksoftirqd/322:55:013
12010993530,3cyclictest41-21ksoftirqd/320:35:013
12010993530,3cyclictest41-21ksoftirqd/319:30:003
12010993528,5cyclictest41-21ksoftirqd/320:10:503
12010993526,4cyclictest41-21ksoftirqd/322:03:103
12010993526,4cyclictest41-21ksoftirqd/322:03:093
1200999357,2cyclictest33-21ksoftirqd/220:30:192
12009993531,2cyclictest33-21ksoftirqd/223:29:592
12009993530,3cyclictest33-21ksoftirqd/221:55:282
12009993530,3cyclictest33-21ksoftirqd/221:54:592
12009993530,3cyclictest33-21ksoftirqd/219:45:232
12009993530,3cyclictest33-21ksoftirqd/219:30:202
12009993530,3cyclictest33-21ksoftirqd/219:25:232
12009993529,4cyclictest33-21ksoftirqd/221:15:012
12009993528,4cyclictest33-21ksoftirqd/223:58:202
12008993529,4cyclictest25-21ksoftirqd/100:16:281
12008993528,4cyclictest25-21ksoftirqd/120:28:531
12007993530,3cyclictest9-21ksoftirqd/023:46:380
12007993529,4cyclictest9-21ksoftirqd/000:07:430
12007993528,5cyclictest9-21ksoftirqd/019:51:130
12010993429,3cyclictest41-21ksoftirqd/323:40:203
12010993429,3cyclictest41-21ksoftirqd/322:10:193
12010993428,4cyclictest41-21ksoftirqd/321:22:593
12010993428,4cyclictest41-21ksoftirqd/320:04:123
12010993428,4cyclictest41-21ksoftirqd/319:31:223
12010993428,3cyclictest41-21ksoftirqd/323:35:093
12010993428,3cyclictest41-21ksoftirqd/321:30:003
1201099340,4cyclictest28497-21awk19:45:003
1200999346,3cyclictest33-21ksoftirqd/221:30:142
12009993430,3cyclictest33-21ksoftirqd/220:20:032
12009993430,2cyclictest33-21ksoftirqd/223:55:012
12009993428,3cyclictest33-21ksoftirqd/222:18:122
12009993428,3cyclictest33-21ksoftirqd/219:10:302
12009993427,4cyclictest33-21ksoftirqd/220:50:042
12009993426,3cyclictest33-21ksoftirqd/223:40:132
1200999341,4cyclictest33-21ksoftirqd/221:40:232
1200999341,3cyclictest121rcu_preempt20:55:232
12008993429,3cyclictest25-21ksoftirqd/119:41:391
12008993428,4cyclictest25-21ksoftirqd/122:32:531
12008993428,4cyclictest25-21ksoftirqd/121:25:371
12008993428,4cyclictest25-21ksoftirqd/121:15:551
12008993428,4cyclictest25-21ksoftirqd/119:24:221
12007993428,4cyclictest9-21ksoftirqd/022:26:300
12007993428,4cyclictest9-21ksoftirqd/019:33:240
12007993416,15cyclictest22047-21sh00:00:160
12010993329,3cyclictest41-21ksoftirqd/320:20:263
12010993327,4cyclictest41-21ksoftirqd/323:29:203
1201099330,5cyclictest18033-21ptp4l-jitter21:35:263
12009993328,3cyclictest33-21ksoftirqd/200:10:272
12009993327,3cyclictest33-21ksoftirqd/219:55:162
12009993327,3cyclictest33-21ksoftirqd/200:15:162
1200999331,4cyclictest33-21ksoftirqd/223:25:002
1200899333,28cyclictest0-21swapper/100:00:371
12008993328,3cyclictest25-21ksoftirqd/123:49:181
12008993327,4cyclictest25-21ksoftirqd/121:12:461
12008993327,4cyclictest25-21ksoftirqd/120:11:361
12008993327,4cyclictest25-21ksoftirqd/119:18:241
12008993327,4cyclictest25-21ksoftirqd/100:26:101
12008993326,5cyclictest25-21ksoftirqd/120:50:031
12008993316,3cyclictest25-21ksoftirqd/123:50:121
12007993327,4cyclictest9-21ksoftirqd/023:37:470
12007993327,4cyclictest9-21ksoftirqd/023:20:090
12007993327,4cyclictest9-21ksoftirqd/022:56:140
12007993327,4cyclictest9-21ksoftirqd/022:48:330
12007993327,4cyclictest9-21ksoftirqd/022:48:330
12007993327,4cyclictest9-21ksoftirqd/019:11:210
136991320,0ptp4l401ktimersoftd/320:07:183
12010993228,3cyclictest41-21ksoftirqd/321:30:183
12010993227,3cyclictest41-21ksoftirqd/323:10:263
12010993227,3cyclictest41-21ksoftirqd/323:06:413
12010993227,3cyclictest41-21ksoftirqd/322:40:183
12010993227,3cyclictest41-21ksoftirqd/321:00:233
12010993226,4cyclictest41-21ksoftirqd/322:23:013
12010993226,3cyclictest41-21ksoftirqd/322:55:223
12010993226,3cyclictest41-21ksoftirqd/321:06:273
12009993228,2cyclictest33-21ksoftirqd/223:00:002
12009993226,4cyclictest33-21ksoftirqd/220:38:082
1200999321,3cyclictest33-21ksoftirqd/223:30:192
12008993226,4cyclictest25-21ksoftirqd/123:29:381
12008993226,4cyclictest25-21ksoftirqd/120:48:391
12008993226,4cyclictest25-21ksoftirqd/119:14:311
12008993226,3cyclictest25-21ksoftirqd/123:25:001
12008993225,5cyclictest25-21ksoftirqd/123:05:431
1200799329,20cyclictest10624-21date20:15:000
12007993227,3cyclictest9-21ksoftirqd/023:58:330
12007993226,4cyclictest9-21ksoftirqd/023:41:410
12007993226,4cyclictest9-21ksoftirqd/022:20:140
12007993226,4cyclictest9-21ksoftirqd/022:01:550
12007993226,4cyclictest9-21ksoftirqd/022:01:550
12007993226,4cyclictest9-21ksoftirqd/020:22:500
12007993225,5cyclictest9-21ksoftirqd/000:13:570
1200799320,3cyclictest20178-21/usr/sbin/munin21:40:240
136991310,0ptp4l401ktimersoftd/323:55:063
136991310,0ptp4l401ktimersoftd/319:15:323
136991310,0ptp4l401ktimersoftd/300:15:323
12010993127,3cyclictest41-21ksoftirqd/323:15:203
12010993127,3cyclictest41-21ksoftirqd/300:30:163
12010993126,3cyclictest41-21ksoftirqd/321:50:233
12010993126,3cyclictest41-21ksoftirqd/321:45:013
12010993125,4cyclictest41-21ksoftirqd/323:21:183
12010993125,4cyclictest41-21ksoftirqd/321:16:333
12010993125,3cyclictest41-21ksoftirqd/322:05:143
12010993125,3cyclictest41-21ksoftirqd/322:00:003
12010993118,3cyclictest41-21ksoftirqd/319:20:173
12008993126,3cyclictest25-21ksoftirqd/100:30:131
12008993125,4cyclictest25-21ksoftirqd/122:42:271
12008993125,4cyclictest25-21ksoftirqd/122:15:441
12008993125,4cyclictest25-21ksoftirqd/122:01:441
12008993125,4cyclictest25-21ksoftirqd/122:01:441
12008993125,4cyclictest25-21ksoftirqd/120:09:231
12008993125,4cyclictest25-21ksoftirqd/120:02:491
12008993125,4cyclictest25-21ksoftirqd/119:51:181
12008993125,4cyclictest25-21ksoftirqd/100:35:451
12008993124,5cyclictest25-21ksoftirqd/121:51:131
12007993126,3cyclictest9-21ksoftirqd/020:02:350
12007993126,3cyclictest9-21ksoftirqd/000:35:310
12007993125,4cyclictest9-21ksoftirqd/023:51:280
12007993125,4cyclictest9-21ksoftirqd/023:26:430
12007993125,4cyclictest9-21ksoftirqd/021:46:230
12007993125,4cyclictest9-21ksoftirqd/019:23:570
12007993111,17cyclictest6866-21cat20:05:220
12007993111,17cyclictest30493-21tune2fs23:10:150
136991300,0ptp4l401ktimersoftd/322:46:133
136991300,0ptp4l401ktimersoftd/322:46:133
1201099303,5cyclictest121rcu_preempt23:45:303
12010993025,3cyclictest41-21ksoftirqd/323:35:013
12010993025,3cyclictest41-21ksoftirqd/323:01:373
12010993025,3cyclictest41-21ksoftirqd/322:30:003
12010993025,3cyclictest41-21ksoftirqd/300:20:233
12010993024,4cyclictest41-21ksoftirqd/321:11:573
12010993024,4cyclictest41-21ksoftirqd/319:56:383
12010993024,3cyclictest41-21ksoftirqd/320:55:213
12008993024,4cyclictest25-21ksoftirqd/122:48:331
12008993024,4cyclictest25-21ksoftirqd/122:48:331
12008993024,4cyclictest25-21ksoftirqd/121:44:311
12008993024,4cyclictest25-21ksoftirqd/121:06:461
12008993024,4cyclictest25-21ksoftirqd/100:06:171
12008993023,5cyclictest25-21ksoftirqd/122:24:591
12008993023,4cyclictest25-21ksoftirqd/123:15:001
1200799308,19cyclictest13361-21run-parts21:25:250
12007993025,3cyclictest9-21ksoftirqd/022:43:540
12007993025,3cyclictest9-21ksoftirqd/020:27:320
12007993024,4cyclictest9-21ksoftirqd/022:06:050
12007993024,4cyclictest9-21ksoftirqd/019:44:470
12007993012,15cyclictest15783-21sed21:30:250
12007993011,16cyclictest20837-21cat19:25:270
12007993011,16cyclictest10948-21cat22:30:160
12007993010,18cyclictest29748-21wc19:45:230
12008992924,3cyclictest25-21ksoftirqd/122:11:171
12008992923,4cyclictest25-21ksoftirqd/123:59:501
12008992923,4cyclictest25-21ksoftirqd/121:58:191
12008992923,4cyclictest25-21ksoftirqd/121:22:411
12008992923,4cyclictest25-21ksoftirqd/100:24:501
12008992923,4cyclictest25-21ksoftirqd/100:11:171
1200799299,17cyclictest8146-21cut21:15:190
12007992924,3cyclictest9-21ksoftirqd/023:03:230
12007992910,16cyclictest31113-21latency_hist00:20:000
12007992910,16cyclictest1156-21cat22:10:140
12010992822,4cyclictest41-21ksoftirqd/320:35:283
12008992822,4cyclictest25-21ksoftirqd/122:25:031
12008992822,4cyclictest25-21ksoftirqd/120:23:021
12008992822,4cyclictest25-21ksoftirqd/120:19:581
12008992822,4cyclictest25-21ksoftirqd/119:29:001
1200799286,3cyclictest3957-21expr00:30:170
1200799285,3cyclictest14723-21timerwakeupswit22:35:300
12007992823,3cyclictest9-21ksoftirqd/021:56:070
12007992822,4cyclictest9-21ksoftirqd/023:05:530
12007992822,4cyclictest9-21ksoftirqd/020:44:070
12007992822,4cyclictest9-21ksoftirqd/000:29:500
12007992813,12cyclictest30516-21fschecks_time20:55:180
1200799280,4cyclictest25604-21cstates20:45:160
1200799280,3cyclictest29812-21taskset20:55:000
12008992722,3cyclictest25-21ksoftirqd/119:59:211
1200799277,18cyclictest18932chrt21:01:340
1200799276,18cyclictest18960-21irqrtprio20:30:200
1200799274,6cyclictest0-21swapper/021:05:160
12007992722,3cyclictest9-21ksoftirqd/019:19:320
12007992721,4cyclictest9-21ksoftirqd/019:37:130
12007992721,4cyclictest9-21ksoftirqd/019:37:130
12008992621,3cyclictest25-21ksoftirqd/122:07:201
1200799269,14cyclictest10700-21needreboot21:20:240
1200799265,3cyclictest2619-21wc19:55:280
1200799260,3cyclictest8176-21meminfo23:30:230
1200799260,3cyclictest5226-21diskstats21:10:160
136991250,1ptp4l401ktimersoftd/319:47:083
12008992519,4cyclictest25-21ksoftirqd/123:31:501
12008992519,4cyclictest25-21ksoftirqd/121:47:041
1200799257,16cyclictest21942-21unixbench_multi22:50:310
1200799256,17cyclictest25100-21phc2sys-jitter21:50:260
12008992419,3cyclictest25-21ksoftirqd/121:36:051
1200799248,14cyclictest1297-21runrttasks00:22:270
12007992410,12cyclictest17334-21if_err_enp4s021:35:200
12008992116,3cyclictest25-21ksoftirqd/120:39:181
12008992015,3cyclictest25-21ksoftirqd/121:33:021
12008991813,3cyclictest10712-21phc2sys23:35:251
12008991711,3cyclictest13761-21interrupts22:35:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional