You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 18:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Tue Aug 26, 2025 12:45:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137391220192,19phc2sys0-21swapper/307:09:593
137391220192,19phc2sys0-21swapper/307:09:583
304722207174,22sleep10-21swapper/107:08:251
304722207174,22sleep10-21swapper/107:08:241
303432206173,22sleep00-21swapper/007:06:430
303432206173,22sleep00-21swapper/007:06:430
304952203170,22sleep20-21swapper/207:08:422
304952203170,22sleep20-21swapper/207:08:422
1399321260,7sleep23089099cyclictest07:40:182
1373918763,16phc2sys0-21swapper/307:10:013
137291650,1ptp4l391rcuc/309:35:293
3089199618,14cyclictest0-21swapper/308:00:203
258792600,6sleep125883-21fschecks_count10:20:171
279952590,2sleep10-21swapper/109:15:251
3089199583,21cyclictest19047-21fschecks_count12:20:183
173352540,2sleep10-21swapper/108:55:161
137291540,1ptp4l391rcuc/312:12:383
137291540,1ptp4l391rcuc/310:34:093
3089099510,11cyclictest5401-21mailstats09:35:252
3089199500,22cyclictest153942sleep307:44:173
30891994712,24cyclictest6416-21/usr/sbin/munin11:55:243
30891994712,24cyclictest6416-21/usr/sbin/munin11:55:243
3089099470,24cyclictest10843-21cat07:35:002
3089099470,12cyclictest9145-21grep08:35:272
3089199450,21cyclictest13576-21awk09:55:173
3089099440,13cyclictest28712-21grep08:10:232
137291440,0ptp4l401ktimersoftd/311:38:273
136602440,2sleep00-21swapper/008:45:240
3089099430,16cyclictest31437-21smartctl07:10:162
3089199420,20cyclictest27621-21latency11:30:223
3089099420,7cyclictest829-21iostat10:35:222
3089099420,12cyclictest12560-21awk12:05:232
30889994236,3cyclictest25-21ksoftirqd/108:00:001
30889994226,3cyclictest25-21ksoftirqd/112:39:591
137291420,0ptp4l401ktimersoftd/310:03:063
137291420,0ptp4l401ktimersoftd/309:08:103
30891994136,3cyclictest14502-21latency_hist08:50:003
137291410,0ptp4l401ktimersoftd/307:30:233
3089199400,20cyclictest26451-21cat10:20:203
30889994035,3cyclictest25-21ksoftirqd/107:39:591
3089199395,17cyclictest218-21systemd-journal11:43:283
3089199390,21cyclictest218-21systemd-journal08:59:293
30889993934,3cyclictest25-21ksoftirqd/109:55:011
30889993934,3cyclictest25-21ksoftirqd/108:40:001
3089199381,26cyclictest10838-21munin-node09:31:193
3088999386,4cyclictest25-21ksoftirqd/111:05:161
3088999384,3cyclictest25-21ksoftirqd/108:40:191
30889993834,2cyclictest231rcuc/109:35:241
30889993833,3cyclictest25-21ksoftirqd/111:45:011
30889993832,3cyclictest25-21ksoftirqd/108:10:001
30889993825,10cyclictest25-21ksoftirqd/107:20:011
30888993833,3cyclictest9-21ksoftirqd/012:40:000
42098370,4rtkit-daemon2494-21gnome-shell09:42:590
3089199370,27cyclictest12892-21sshd09:50:283
3089099379,8cyclictest11526-21ls08:40:272
3088999379,4cyclictest25-21ksoftirqd/109:10:171
3088999375,3cyclictest25-21ksoftirqd/107:40:191
30889993733,2cyclictest25-21ksoftirqd/109:50:001
30889993732,3cyclictest25-21ksoftirqd/110:30:001
30889993732,3cyclictest25-21ksoftirqd/108:35:011
30889993732,2cyclictest25-21ksoftirqd/112:30:001
30889993731,3cyclictest25-21ksoftirqd/111:15:011
30889993731,3cyclictest25-21ksoftirqd/110:10:151
30888993732,3cyclictest9-21ksoftirqd/008:00:010
30888993731,4cyclictest9-21ksoftirqd/008:54:430
30891993630,4cyclictest41-21ksoftirqd/312:27:163
30891993630,4cyclictest41-21ksoftirqd/311:08:453
30891993630,4cyclictest41-21ksoftirqd/310:44:183
30891993630,3cyclictest41-21ksoftirqd/307:18:593
3089199362,20cyclictest4676-21sshd08:29:283
3089199360,23cyclictest24889-21taskset10:16:023
3089199360,22cyclictest7075-21seq10:49:133
30890993630,4cyclictest33-21ksoftirqd/209:55:002
3089099360,35cyclictest7967-21kworker/2:111:05:292
3088999364,9cyclictest25-21ksoftirqd/110:09:591
30889993632,3cyclictest25-21ksoftirqd/108:50:001
30889993632,2cyclictest25-21ksoftirqd/111:25:011
30889993631,3cyclictest25-21ksoftirqd/112:00:201
30889993631,3cyclictest25-21ksoftirqd/112:00:191
30889993631,3cyclictest25-21ksoftirqd/111:50:281
30889993631,3cyclictest25-21ksoftirqd/111:20:011
30889993631,3cyclictest25-21ksoftirqd/110:45:011
30889993630,4cyclictest25-21ksoftirqd/111:34:301
30889993630,3cyclictest25-21ksoftirqd/107:29:591
30891993530,3cyclictest41-21ksoftirqd/309:49:283
30891993529,4cyclictest41-21ksoftirqd/311:10:463
30891993529,4cyclictest41-21ksoftirqd/309:42:593
3089199350,22cyclictest22492-21taskset07:59:033
3089199350,21cyclictest218-21systemd-journal08:38:293
3089199350,21cyclictest2043-21sshd08:24:273
3089199350,21cyclictest1928-21hddtemp_smartct11:45:213
3089099350,11cyclictest19600-21sed12:20:212
3088999354,2cyclictest121rcu_preempt10:50:281
3088999354,2cyclictest121rcu_preempt10:50:281
30889993531,3cyclictest25-21ksoftirqd/110:05:001
30889993531,2cyclictest32105-21systemctl08:20:011
30889993531,2cyclictest32105-21systemctl08:20:001
30889993530,3cyclictest25-21ksoftirqd/108:25:141
30889993530,3cyclictest25-21ksoftirqd/107:34:591
30889993529,4cyclictest25-21ksoftirqd/112:30:551
30889993529,4cyclictest25-21ksoftirqd/110:35:371
30889993529,3cyclictest25-21ksoftirqd/111:40:001
30889993529,3cyclictest25-21ksoftirqd/111:30:001
3088999350,3cyclictest121rcu_preempt09:20:191
30888993530,3cyclictest9-21ksoftirqd/008:20:120
30888993529,4cyclictest9-21ksoftirqd/011:04:160
30891993429,3cyclictest41-21ksoftirqd/309:28:293
30891993428,4cyclictest41-21ksoftirqd/309:18:173
30891993428,4cyclictest41-21ksoftirqd/309:13:463
30891993428,4cyclictest41-21ksoftirqd/307:38:073
3089199341,20cyclictest30898-21irqstats08:15:223
3089199341,20cyclictest30898-21irqstats08:15:223
3089199340,21cyclictest14052-21sshd11:01:293
3089199340,17cyclictest0-21swapper/308:30:243
30890993429,3cyclictest33-21ksoftirqd/209:20:132
30890993428,4cyclictest33-21ksoftirqd/210:17:442
3089099341,20cyclictest0-21swapper/209:15:142
3089099340,11cyclictest20136-21cat09:00:182
3088999346,2cyclictest25-21ksoftirqd/108:10:211
30889993430,2cyclictest25-21ksoftirqd/110:55:141
30889993430,2cyclictest25-21ksoftirqd/108:55:001
30889993429,3cyclictest25-21ksoftirqd/108:20:191
30889993429,3cyclictest25-21ksoftirqd/107:10:201
30889993428,4cyclictest25-21ksoftirqd/112:15:001
30889993428,4cyclictest25-21ksoftirqd/110:45:451
30889993428,4cyclictest25-21ksoftirqd/110:32:241
30889993428,4cyclictest25-21ksoftirqd/109:27:411
30889993428,4cyclictest25-21ksoftirqd/109:02:501
30889993428,3cyclictest25-21ksoftirqd/112:15:191
30889993428,3cyclictest25-21ksoftirqd/110:15:191
30888993428,4cyclictest9-21ksoftirqd/012:27:080
30888993428,4cyclictest9-21ksoftirqd/011:24:540
30888993428,4cyclictest9-21ksoftirqd/010:40:450
30888993428,4cyclictest9-21ksoftirqd/010:26:300
30888993428,4cyclictest9-21ksoftirqd/010:06:560
30888993428,4cyclictest9-21ksoftirqd/009:55:560
30888993428,4cyclictest9-21ksoftirqd/009:48:230
30888993428,4cyclictest9-21ksoftirqd/007:34:070
30888993428,4cyclictest9-21ksoftirqd/007:23:440
30891993327,4cyclictest41-21ksoftirqd/309:02:003
30891993327,4cyclictest41-21ksoftirqd/308:12:573
30891993327,4cyclictest41-21ksoftirqd/307:53:363
30891993327,4cyclictest41-21ksoftirqd/307:25:593
3089199330,24cyclictest0-21swapper/312:15:293
3089199330,18cyclictest0-21swapper/310:25:153
30890993328,3cyclictest33-21ksoftirqd/208:58:442
30890993327,4cyclictest33-21ksoftirqd/212:27:232
30890993327,4cyclictest33-21ksoftirqd/212:18:002
30890993327,4cyclictest33-21ksoftirqd/211:13:332
30890993327,4cyclictest33-21ksoftirqd/210:34:562
30890993327,4cyclictest33-21ksoftirqd/208:08:502
30890993327,4cyclictest33-21ksoftirqd/207:51:412
30890993326,4cyclictest33-21ksoftirqd/207:49:202
3089099330,11cyclictest25596-21ptp4l11:25:242
3088999336,3cyclictest25-21ksoftirqd/109:30:281
3088999335,2cyclictest121rcu_preempt11:45:201
30889993329,2cyclictest25-21ksoftirqd/109:40:141
30889993328,3cyclictest25-21ksoftirqd/112:25:021
30889993328,3cyclictest25-21ksoftirqd/107:50:201
30889993327,4cyclictest25-21ksoftirqd/107:49:571
30889993327,4cyclictest13528-21cron12:10:001
3088999332,2cyclictest25-21ksoftirqd/109:05:311
30888993327,4cyclictest9-21ksoftirqd/012:17:380
30888993327,4cyclictest9-21ksoftirqd/008:42:310
30888993326,4cyclictest9-21ksoftirqd/008:57:100
30891993227,3cyclictest41-21ksoftirqd/308:52:353
3089099328,8cyclictest1314-21snmpd11:04:092
30890993227,3cyclictest33-21ksoftirqd/208:25:112
30890993226,4cyclictest33-21ksoftirqd/211:48:032
30890993226,4cyclictest33-21ksoftirqd/210:51:032
30890993226,4cyclictest33-21ksoftirqd/210:51:022
30890993226,4cyclictest33-21ksoftirqd/209:07:452
30890993226,4cyclictest33-21ksoftirqd/208:47:512
3089099320,11cyclictest27789-21seq12:39:282
30889993227,3cyclictest25-21ksoftirqd/111:55:261
30889993227,3cyclictest25-21ksoftirqd/111:55:261
30889993227,3cyclictest25-21ksoftirqd/107:20:201
3088999321,3cyclictest25-21ksoftirqd/108:00:151
30888993227,3cyclictest9-21ksoftirqd/009:03:360
30888993226,4cyclictest9-21ksoftirqd/012:22:190
30888993226,4cyclictest9-21ksoftirqd/011:56:270
30888993226,4cyclictest9-21ksoftirqd/011:56:270
30888993226,4cyclictest9-21ksoftirqd/011:48:450
30888993226,4cyclictest9-21ksoftirqd/011:44:440
30888993226,4cyclictest9-21ksoftirqd/011:32:080
30888993226,4cyclictest9-21ksoftirqd/010:52:170
30888993226,4cyclictest9-21ksoftirqd/010:52:160
30888993226,4cyclictest9-21ksoftirqd/009:37:030
30888993226,4cyclictest9-21ksoftirqd/007:48:300
30891993126,3cyclictest41-21ksoftirqd/311:25:053
30891993126,3cyclictest41-21ksoftirqd/310:52:043
30891993126,3cyclictest41-21ksoftirqd/310:52:043
3089199310,17cyclictest3108-21apt-get07:20:023
30890993126,3cyclictest33-21ksoftirqd/209:47:472
30890993125,4cyclictest33-21ksoftirqd/212:34:502
3089099311,20cyclictest0-21swapper/208:50:132
3089099310,19cyclictest0-21swapper/210:40:132
3089099310,11cyclictest20653-21gzip11:15:252
30889993125,4cyclictest25-21ksoftirqd/109:56:541
30888993126,3cyclictest9-21ksoftirqd/012:14:140
30888993126,3cyclictest9-21ksoftirqd/010:14:460
30888993125,4cyclictest9-21ksoftirqd/012:09:310
30888993125,4cyclictest9-21ksoftirqd/010:36:440
30888993125,4cyclictest9-21ksoftirqd/009:31:040
30888993125,4cyclictest9-21ksoftirqd/009:18:100
30888993125,4cyclictest9-21ksoftirqd/008:14:460
30888993125,4cyclictest9-21ksoftirqd/007:36:520
30891993025,3cyclictest41-21ksoftirqd/311:52:093
30891993025,3cyclictest41-21ksoftirqd/311:17:483
30891993025,3cyclictest41-21ksoftirqd/310:13:213
30891993024,4cyclictest41-21ksoftirqd/312:38:593
3089199300,13cyclictest0-21swapper/310:55:263
30890993025,3cyclictest33-21ksoftirqd/211:34:452
30890993025,3cyclictest33-21ksoftirqd/208:32:512
30890993024,4cyclictest33-21ksoftirqd/211:21:072
30890993024,4cyclictest33-21ksoftirqd/210:22:102
30890993024,4cyclictest33-21ksoftirqd/210:05:422
30890993024,4cyclictest33-21ksoftirqd/207:28:432
30890993023,5cyclictest33-21ksoftirqd/211:38:122
3089099302,27cyclictest9876-21kworker/2:108:02:382
3089099302,15cyclictest30739-21cut08:15:222
3089099302,15cyclictest30739-21cut08:15:222
3088999300,9cyclictest25-21ksoftirqd/111:00:201
30888993026,3cyclictest9-21ksoftirqd/011:00:010
30888993025,3cyclictest9-21ksoftirqd/011:15:580
30888993025,3cyclictest9-21ksoftirqd/010:30:180
30888993025,3cyclictest9-21ksoftirqd/008:35:010
30888993025,3cyclictest9-21ksoftirqd/007:43:170
30888993025,3cyclictest9-21ksoftirqd/007:14:130
30888993024,4cyclictest9-21ksoftirqd/012:01:090
30888993024,4cyclictest9-21ksoftirqd/012:01:090
30888993024,4cyclictest9-21ksoftirqd/008:29:420
30888993022,6cyclictest9-21ksoftirqd/011:25:460
30891992923,4cyclictest41-21ksoftirqd/310:08:453
30891992923,4cyclictest41-21ksoftirqd/307:45:573
30891992914,12cyclictest23838-21cron11:25:013
30890992925,3cyclictest33-21ksoftirqd/208:24:282
30890992924,3cyclictest33-21ksoftirqd/212:14:372
30890992924,3cyclictest33-21ksoftirqd/207:57:122
30890992923,4cyclictest33-21ksoftirqd/209:57:582
30890992923,4cyclictest33-21ksoftirqd/209:40:382
30890992923,4cyclictest33-21ksoftirqd/207:20:412
3089099292,26cyclictest11859-21kworker/2:209:27:492
3089099291,27cyclictest13976-21kworker/2:110:29:032
30888992924,3cyclictest9-21ksoftirqd/010:15:140
30888992924,3cyclictest9-21ksoftirqd/008:16:280
30888992924,3cyclictest9-21ksoftirqd/008:16:270
30888992924,3cyclictest9-21ksoftirqd/007:53:200
30888992923,4cyclictest9-21ksoftirqd/011:10:340
30888992923,4cyclictest9-21ksoftirqd/009:09:220
30891992823,3cyclictest41-21ksoftirqd/312:06:193
30891992823,3cyclictest41-21ksoftirqd/309:24:153
30891992822,3cyclictest41-21ksoftirqd/312:02:363
30891992822,3cyclictest41-21ksoftirqd/312:02:363
3089199282,22cyclictest9947-21apt-get08:40:123
30890992823,3cyclictest33-21ksoftirqd/211:44:572
30890992823,3cyclictest33-21ksoftirqd/210:57:532
30890992823,3cyclictest33-21ksoftirqd/210:47:222
30890992823,3cyclictest33-21ksoftirqd/210:02:552
30890992810,8cyclictest9876-21kworker/2:107:39:342
3089099280,18cyclictest26387-21latency_hist09:15:012
30888992823,3cyclictest9-21ksoftirqd/011:55:000
30888992823,3cyclictest9-21ksoftirqd/010:23:360
3089199278,16cyclictest218-21systemd-journal10:39:053
3089199272,19cyclictest23349-21apt-get12:30:173
30890992722,3cyclictest33-21ksoftirqd/211:56:392
30890992722,3cyclictest33-21ksoftirqd/211:56:392
30890992721,4cyclictest33-21ksoftirqd/212:03:282
30890992721,4cyclictest33-21ksoftirqd/212:03:282
3089099270,8cyclictest0-21swapper/207:15:202
30888992722,3cyclictest9-21ksoftirqd/009:14:550
30888992721,4cyclictest9-21ksoftirqd/010:03:420
30888992721,4cyclictest9-21ksoftirqd/007:28:090
3089199260,22cyclictest26980-21vmstat08:05:313
30890992621,3cyclictest33-21ksoftirqd/209:34:142
30888992621,3cyclictest9-21ksoftirqd/011:05:480
30888992620,4cyclictest9-21ksoftirqd/009:22:180
30890992520,3cyclictest33-21ksoftirqd/210:11:452
3089099250,22cyclictest59472chrt11:52:042
30888992520,3cyclictest9-21ksoftirqd/012:31:180
30888992519,4cyclictest9-21ksoftirqd/010:49:080
42098240,2rtkit-daemon0-21swapper/008:33:340
30888992418,4cyclictest9-21ksoftirqd/007:17:450
30888992318,3cyclictest9-21ksoftirqd/008:02:390
30888992116,3cyclictest9-21ksoftirqd/008:05:440
30888992114,4cyclictest9-21ksoftirqd/009:30:000
30888991813,3cyclictest29563-21hwlatdetect11:35:190
30888991712,3cyclictest9-21ksoftirqd/009:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional