You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 22:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot2.osadl.org (updated Fri Nov 21, 2025 12:45:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
254812208172,25sleep00-21swapper/007:08:480
252462208173,23sleep20-21swapper/207:05:442
253602206170,25sleep30-21swapper/307:07:143
252752206172,23sleep10-21swapper/107:06:071
136591640,8ptp4l401ktimersoftd/308:58:303
2587599629,16cyclictest3459-21sed09:45:010
2587599628,17cyclictest17713-21/usr/sbin/munin10:15:210
2587599618,17cyclictest6985-21sed11:00:000
2587599609,16cyclictest6106-21df_abs12:05:160
25877995927,6cyclictest33-21ksoftirqd/210:50:292
2587699590,22cyclictest13029-21dpkg07:50:131
25877995825,4cyclictest33-21ksoftirqd/210:35:132
2587699580,22cyclictest5529-21dpkg07:35:001
2587599588,15cyclictest14674-21sed12:20:280
2587599579,17cyclictest8614-21systemctl11:00:260
2587599579,15cyclictest30647-21/usr/sbin/munin07:20:220
25877995624,6cyclictest33-21ksoftirqd/210:10:212
2587599558,15cyclictest12030-21grep11:10:110
25877995324,6cyclictest33-21ksoftirqd/210:05:162
25877995323,6cyclictest33-21ksoftirqd/210:40:272
25877995323,6cyclictest33-21ksoftirqd/210:25:152
25877995323,6cyclictest33-21ksoftirqd/210:20:132
2587699530,24cyclictest9654-21swap09:55:251
2587599539,16cyclictest20007-21irqrtprio09:10:180
143562530,2sleep20-21swapper/209:00:122
143562530,2sleep20-21swapper/209:00:122
25877995227,3cyclictest33-21ksoftirqd/210:00:172
25877995225,4cyclictest33-21ksoftirqd/207:15:272
25877995222,6cyclictest33-21ksoftirqd/210:55:262
25877995221,7cyclictest33-21ksoftirqd/210:35:002
2587599529,12cyclictest10642-21/usr/sbin/munin07:45:130
2587599527,16cyclictest0-21swapper/009:15:170
2587599527,10cyclictest24824-21cat10:30:000
2587799515,2cyclictest33-21ksoftirqd/211:15:122
25877995123,3cyclictest33-21ksoftirqd/211:05:242
25877995120,4cyclictest33-21ksoftirqd/208:50:272
25877995119,6cyclictest33-21ksoftirqd/210:45:162
25877995117,4cyclictest33-21ksoftirqd/208:40:162
25877995116,7cyclictest33-21ksoftirqd/208:20:152
25877995115,6cyclictest33-21ksoftirqd/209:20:232
2587599519,7cyclictest30755-21irqstats10:40:220
2587599519,13cyclictest21534-21python10:20:240
136591510,1ptp4l401ktimersoftd/310:58:423
25877995018,6cyclictest33-21ksoftirqd/210:15:242
25877994925,6cyclictest33-21ksoftirqd/212:00:002
25877994924,7cyclictest33-21ksoftirqd/211:44:592
25877994924,7cyclictest33-21ksoftirqd/211:15:002
25877994918,6cyclictest33-21ksoftirqd/208:00:212
25877994917,6cyclictest33-21ksoftirqd/211:35:262
25877994916,4cyclictest33-21ksoftirqd/209:45:222
2587599499,14cyclictest20757-21hwlatdetect12:35:150
2587599499,13cyclictest739-21irqstats10:45:200
2587599498,17cyclictest27131-21pluginstate07:10:230
2587599498,15cyclictest27285-21taskset08:20:140
25875994910,12cyclictest15646-21df10:10:140
25877994822,7cyclictest33-21ksoftirqd/212:09:032
25877994821,4cyclictest33-21ksoftirqd/208:15:212
25877994821,3cyclictest33-21ksoftirqd/209:08:382
25877994819,12cyclictest33-21ksoftirqd/207:50:192
25877994818,4cyclictest33-21ksoftirqd/209:50:182
25877994816,4cyclictest33-21ksoftirqd/211:20:252
2587699481,25cyclictest2784-21unixbench_multi09:40:291
2587699481,25cyclictest2784-21unixbench_multi09:40:291
2587699480,23cyclictest16188-21chrt11:16:311
2587599489,17cyclictest26207-21/usr/sbin/munin10:30:260
2587599489,12cyclictest28855-21python10:35:240
2587599488,8cyclictest23890-21cpu11:35:130
2587599486,9cyclictest0-21swapper/008:40:260
25877994721,3cyclictest33-21ksoftirqd/212:20:202
25877994720,6cyclictest33-21ksoftirqd/209:30:172
25877994717,6cyclictest33-21ksoftirqd/208:10:172
25877994717,4cyclictest33-21ksoftirqd/211:50:232
25877994715,4cyclictest33-21ksoftirqd/209:15:172
25877994715,3cyclictest33-21ksoftirqd/212:30:282
2587599479,12cyclictest13642-21iostat_ios10:05:200
188082470,1sleep10-21swapper/112:30:221
94682460,1sleep30-21swapper/312:10:213
25877994621,7cyclictest33-21ksoftirqd/208:55:272
25877994621,7cyclictest33-21ksoftirqd/207:55:262
25877994619,7cyclictest33-21ksoftirqd/212:15:192
25877994619,3cyclictest33-21ksoftirqd/211:25:182
25877994618,3cyclictest33-21ksoftirqd/208:35:012
25877994617,4cyclictest33-21ksoftirqd/209:25:172
25877994615,4cyclictest33-21ksoftirqd/207:20:142
25877994614,4cyclictest33-21ksoftirqd/208:25:112
25877994611,5cyclictest33-21ksoftirqd/207:35:172
25877994521,7cyclictest33-21ksoftirqd/212:35:172
25877994521,6cyclictest33-21ksoftirqd/209:35:222
25877994518,7cyclictest33-21ksoftirqd/209:10:152
25877994518,7cyclictest33-21ksoftirqd/208:35:172
25877994517,4cyclictest33-21ksoftirqd/208:05:162
25877994517,4cyclictest33-21ksoftirqd/207:30:012
25877994516,6cyclictest33-21ksoftirqd/207:10:272
25877994514,6cyclictest33-21ksoftirqd/209:40:182
25877994514,6cyclictest33-21ksoftirqd/209:40:182
25877994512,6cyclictest33-21ksoftirqd/209:55:132
25877994511,5cyclictest33-21ksoftirqd/212:00:162
2587699455,21cyclictest27189-21latency11:40:211
25877994420,7cyclictest33-21ksoftirqd/211:45:282
25877994419,3cyclictest33-21ksoftirqd/207:30:142
25877994418,7cyclictest33-21ksoftirqd/212:15:012
25877994418,7cyclictest33-21ksoftirqd/211:30:162
25877994417,4cyclictest33-21ksoftirqd/208:45:122
25877994411,6cyclictest33-21ksoftirqd/212:25:202
25877994333,3cyclictest33-21ksoftirqd/207:45:012
25877994317,7cyclictest33-21ksoftirqd/211:00:212
25877994311,5cyclictest33-21ksoftirqd/207:45:012
2587699410,24cyclictest30238-21chrt07:18:111
2587699410,24cyclictest17055-21perl09:05:201
136591410,0ptp4l401ktimersoftd/311:57:043
136591410,0ptp4l401ktimersoftd/310:35:033
2587699401,24cyclictest8355-21gdbus12:10:011
2587699401,23cyclictest23683-21phc2sys10:25:231
2587699401,23cyclictest18658-21sed08:00:191
2587699400,23cyclictest29970-21tr11:45:261
2587599407,8cyclictest0-21swapper/011:58:480
2587599407,15cyclictest29513-21ps07:15:230
2587599406,15cyclictest0-21swapper/011:25:000
2587599399,16cyclictest0-21swapper/007:35:220
136591380,0ptp4l401ktimersoftd/310:29:123
2587699370,21cyclictest2180-21expr07:25:221
2587599377,15cyclictest2259-21gnome-shell11:27:580
25878993631,3cyclictest41-21ksoftirqd/310:24:363
2587599365,17cyclictest0-21swapper/012:10:200
25875993610,9cyclictest4945-21cat09:45:250
2587699350,22cyclictest12482-21sshd07:46:201
2587599354,18cyclictest0-21swapper/007:25:170
136591350,6ptp4l41-21ksoftirqd/312:05:173
2587699340,1cyclictest0-21swapper/110:57:381
2587599348,7cyclictest9525-21ps09:55:220
2587599346,17cyclictest0-21swapper/011:15:180
2587599346,17cyclictest0-21swapper/008:25:180
2587599345,19cyclictest0-21swapper/011:30:190
2587599345,17cyclictest0-21swapper/009:50:120
2587599345,15cyclictest0-21swapper/008:00:190
25875993410,15cyclictest11709-21latency12:15:210
25878993327,4cyclictest41-21ksoftirqd/311:02:023
25878993327,4cyclictest41-21ksoftirqd/309:12:053
25878993327,4cyclictest41-21ksoftirqd/308:32:263
2587699339,21cyclictest25288-21fschecks_count10:30:191
25876993327,4cyclictest25-21ksoftirqd/108:45:031
2587599336,17cyclictest0-21swapper/009:00:250
2587599336,17cyclictest0-21swapper/009:00:250
2587599335,18cyclictest0-21swapper/008:35:220
2587599335,17cyclictest0-21swapper/010:50:150
2587599334,18cyclictest0-21swapper/008:15:210
2587599334,17cyclictest0-21swapper/007:30:180
25875993327,4cyclictest9-21ksoftirqd/009:24:270
136591330,0ptp4l401ktimersoftd/307:35:003
25878993227,3cyclictest9176-21cron08:49:593
25878993226,4cyclictest41-21ksoftirqd/310:46:003
25878993226,4cyclictest41-21ksoftirqd/308:53:283
25878993226,4cyclictest41-21ksoftirqd/308:29:423
25878993226,4cyclictest41-21ksoftirqd/308:24:243
25878993226,4cyclictest41-21ksoftirqd/307:52:313
25878993226,4cyclictest41-21ksoftirqd/307:17:123
25876993227,3cyclictest25-21ksoftirqd/111:11:581
25876993226,4cyclictest25-21ksoftirqd/112:09:071
25876993226,4cyclictest25-21ksoftirqd/111:04:331
25876993226,4cyclictest25-21ksoftirqd/110:02:071
25876993226,4cyclictest25-21ksoftirqd/107:14:291
2587699320,3cyclictest13996-21sed10:05:211
2587599327,16cyclictest2259-21gnome-shell11:06:580
2587599326,17cyclictest0-21swapper/008:50:220
2587599326,16cyclictest0-21swapper/012:25:200
2587599325,17cyclictest0-21swapper/008:45:290
2587599325,16cyclictest0-21swapper/011:40:170
2587599325,16cyclictest0-21swapper/009:40:220
2587599325,16cyclictest0-21swapper/008:55:120
2587599325,16cyclictest0-21swapper/007:40:150
2587599324,17cyclictest0-21swapper/010:00:260
25878993126,3cyclictest41-21ksoftirqd/310:03:013
25878993126,3cyclictest41-21ksoftirqd/309:16:313
25878993126,3cyclictest41-21ksoftirqd/307:31:193
25878993125,4cyclictest41-21ksoftirqd/312:32:563
25878993125,4cyclictest41-21ksoftirqd/311:41:543
25878993125,4cyclictest41-21ksoftirqd/307:48:053
25878993123,6cyclictest41-21ksoftirqd/309:41:423
25878993123,6cyclictest41-21ksoftirqd/309:41:423
2587699314,23cyclictest2118-21latency_hist08:35:011
25876993126,3cyclictest25-21ksoftirqd/109:17:191
25876993125,4cyclictest25-21ksoftirqd/109:48:301
2587699312,23cyclictest2259-21gnome-shell10:52:581
2587699311,25cyclictest2259-21gnome-shell10:46:581
25876993111,17cyclictest11493-21sed12:15:191
2587699310,1cyclictest0-21swapper/111:52:261
2587599315,16cyclictest0-21swapper/012:00:120
2587599315,15cyclictest0-21swapper/012:30:160
2587599315,15cyclictest0-21swapper/007:50:260
2587599314,17cyclictest0-21swapper/008:30:180
2587599314,16cyclictest0-21swapper/008:05:160
136591310,0ptp4l401ktimersoftd/310:15:183
136591310,0ptp4l401ktimersoftd/307:25:113
25878993025,3cyclictest41-21ksoftirqd/308:08:303
25878993024,4cyclictest41-21ksoftirqd/311:37:413
25878993024,4cyclictest41-21ksoftirqd/311:06:593
25878993024,4cyclictest41-21ksoftirqd/310:10:343
25878993024,4cyclictest41-21ksoftirqd/309:09:523
25878993024,4cyclictest41-21ksoftirqd/308:03:413
2587699309,18cyclictest30283-21grep10:40:181
2587699306,20cyclictest5719-21latency_hist12:05:001
2587699303,22cyclictest15390-21apt-get07:55:121
25876993024,4cyclictest25-21ksoftirqd/110:16:581
25876993024,4cyclictest25-21ksoftirqd/109:26:401
2587699302,24cyclictest29856-21apt-get08:25:131
2587699302,23cyclictest9619-21apt-get11:05:141
25876993014,14cyclictest9252-21open_files07:40:261
25876993012,16cyclictest26158-21sed09:24:591
25876993010,17cyclictest20443-21sed10:20:141
2587699300,2cyclictest0-21swapper/109:35:171
2587699300,28cyclictest13950-21cat09:00:001
2587699300,1cyclictest0-21swapper/109:00:141
2587699300,1cyclictest0-21swapper/109:00:141
2587699300,1cyclictest0-21swapper/107:30:131
2587599305,17cyclictest0-21swapper/009:05:190
2587599305,16cyclictest0-21swapper/011:45:130
2587599305,16cyclictest0-21swapper/009:35:110
2587599304,16cyclictest0-21swapper/009:30:130
136591300,0ptp4l401ktimersoftd/310:40:273
136591300,0ptp4l401ktimersoftd/310:08:293
25878992924,3cyclictest41-21ksoftirqd/312:37:583
25878992924,3cyclictest41-21ksoftirqd/307:11:203
25878992923,4cyclictest41-21ksoftirqd/312:17:393
25878992923,4cyclictest41-21ksoftirqd/307:43:323
2587699295,21cyclictest16910-21tr11:20:161
2587699294,21cyclictest19294-21cpuspeed_turbos09:10:111
25876992923,4cyclictest25-21ksoftirqd/112:21:511
2587699292,23cyclictest1889-21Xorg11:26:591
25876992911,15cyclictest21402-21nfs4_client12:35:211
25876992910,16cyclictest24357-21sed11:35:171
2587699290,2cyclictest24868-21/usr/sbin/munin08:15:141
2587699290,26cyclictest29511-21apt-key08:25:001
2587699290,1cyclictest0-21swapper/112:25:181
2587599295,16cyclictest0-21swapper/007:55:590
136591290,0ptp4l401ktimersoftd/309:45:583
136591290,0ptp4l401ktimersoftd/308:38:523
25878992822,4cyclictest41-21ksoftirqd/312:28:443
25878992822,4cyclictest41-21ksoftirqd/311:51:173
25878992822,4cyclictest41-21ksoftirqd/308:17:103
2587699289,16cyclictest6133-21expr09:50:141
2587699284,21cyclictest5421-21irqrtprio08:40:191
2587699284,21cyclictest21303-21open_files08:05:181
25876992823,3cyclictest25-21ksoftirqd/107:21:511
2587699282,20cyclictest9426-21apt-get08:50:131
2587699281,24cyclictest30371-21users09:30:271
2587699281,23cyclictest27500-21cpuspeed10:35:131
2587699280,1cyclictest0-21swapper/108:35:171
136591280,0ptp4l401ktimersoftd/309:55:143
136591280,0ptp4l401ktimersoftd/308:11:423
25878992722,3cyclictest41-21ksoftirqd/311:19:523
25878992722,3cyclictest41-21ksoftirqd/309:52:153
25878992722,3cyclictest41-21ksoftirqd/307:23:103
25878992722,3cyclictest30697-21sshd09:34:273
25878992721,4cyclictest41-21ksoftirqd/312:00:063
25878992721,4cyclictest41-21ksoftirqd/311:14:253
25878992720,3cyclictest41-21ksoftirqd/309:25:123
2587699274,21cyclictest22464-21latency11:30:191
2587699272,20cyclictest971-21apt-get11:55:111
2587699271,22cyclictest16388-21memory10:10:191
2587699270,1cyclictest0-21swapper/108:13:011
136591270,0ptp4l401ktimersoftd/312:20:563
25878992621,3cyclictest41-21ksoftirqd/310:30:143
25878992621,3cyclictest41-21ksoftirqd/309:39:183
25878992621,3cyclictest41-21ksoftirqd/308:42:013
25878992621,3cyclictest41-21ksoftirqd/307:58:163
25878992620,4cyclictest41-21ksoftirqd/311:33:033
25875992620,4cyclictest9-21ksoftirqd/009:29:590
25878992520,3cyclictest41-21ksoftirqd/309:00:233
25878992520,3cyclictest41-21ksoftirqd/309:00:233
25875992519,4cyclictest9-21ksoftirqd/011:54:380
136591240,0ptp4l401ktimersoftd/310:50:293
136591240,0ptp4l401ktimersoftd/309:20:193
25875992312,8cyclictest23013-21/usr/sbin/munin08:10:150
136591230,1ptp4l401ktimersoftd/311:28:373
136591230,0ptp4l401ktimersoftd/311:45:223
136591230,0ptp4l401ktimersoftd/311:20:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional