You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 20:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Sat Nov 29, 2025 12:45:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32402213181,21sleep20-21swapper/207:05:592
137191212178,22phc2sys0-21swapper/307:07:273
33482202169,22sleep10-21swapper/107:07:211
34642199166,22sleep00-21swapper/007:08:540
59298770,13rtkit-daemon0-21swapper/307:30:253
385499599,11cyclictest1989-21grep10:25:251
385499598,11cyclictest2192-21uniq11:35:171
385499596,20cyclictest8444-21hwlatdetect10:40:171
27432590,2sleep20-21swapper/208:15:142
145612580,6sleep2385599cyclictest09:45:202
3854995710,10cyclictest30722-21/usr/sbin/munin09:10:271
385499569,13cyclictest7834-21cat09:30:261
385499568,11cyclictest19250-21cat12:10:181
3854995610,12cyclictest18417-21/usr/sbin/munin08:45:241
385499557,11cyclictest18229-21taskset12:08:111
385499557,11cyclictest12659-21mailstats11:55:261
385499547,11cyclictest6614-21df_abs07:15:161
385499547,11cyclictest6614-21df_abs07:15:151
136991540,0ptp4l401ktimersoftd/307:10:113
136991530,1ptp4l401ktimersoftd/311:19:143
385499526,9cyclictest12954-21sed09:40:271
385499526,11cyclictest30372-21gpgv12:35:001
385499517,10cyclictest412-21cron10:24:591
131202510,3sleep113122-21fschecks_time10:50:181
3855995036,12cyclictest33-21ksoftirqd/209:19:592
385499507,12cyclictest9287-21/usr/sbin/munin11:50:161
385499506,10cyclictest17851-21sshd07:39:401
59298490,2rtkit-daemon0-21swapper/307:43:063
385499496,11cyclictest18835-21grep11:00:261
385499495,8cyclictest756-21smartctl08:10:191
3854994916,31cyclictest27867-21kworker/1:212:24:081
3855994841,4cyclictest8397-21turbostat07:20:002
3855994841,4cyclictest8397-21turbostat07:20:002
385499488,11cyclictest7127-21seq08:23:571
385499476,11cyclictest30773-21dpkg12:35:121
385499474,11cyclictest30752-21mailstats11:25:281
132691470,1getstats0-21swapper/308:30:153
3855994631,9cyclictest33-21ksoftirqd/209:20:252
3855994628,16cyclictest33-21ksoftirqd/211:30:002
385499466,12cyclictest15031-21mailstats09:45:251
3855994528,15cyclictest33-21ksoftirqd/209:44:592
385499456,12cyclictest29224-21awk08:00:271
385399452,6cyclictest9-21ksoftirqd/009:49:590
59298440,2rtkit-daemon0-21swapper/311:26:163
3855994418,7cyclictest33-21ksoftirqd/212:15:212
3855994416,4cyclictest33-21ksoftirqd/211:35:292
3855994413,9cyclictest33-21ksoftirqd/208:40:162
3855994413,4cyclictest33-21ksoftirqd/207:50:202
385499447,11cyclictest223-21systemd-journal11:40:451
385499447,11cyclictest12127-21/usr/sbin/munin07:25:221
385499447,11cyclictest10596-21chrt07:22:531
385399445,4cyclictest9-21ksoftirqd/007:50:260
385599435,2cyclictest121rcu_preempt12:05:202
385599430,3cyclictest121rcu_preempt09:00:252
385499437,9cyclictest20604-21chrt09:55:571
385499435,11cyclictest9355-21fschecks_time09:35:181
385499434,9cyclictest0-21swapper/109:07:541
385399436,5cyclictest9-21ksoftirqd/012:05:160
385399434,5cyclictest9-21ksoftirqd/007:27:420
385399432,6cyclictest9-21ksoftirqd/010:58:460
385599428,4cyclictest33-21ksoftirqd/211:00:172
385599428,3cyclictest33-21ksoftirqd/207:20:272
3855994237,3cyclictest33-21ksoftirqd/211:00:002
3855994237,3cyclictest33-21ksoftirqd/209:40:012
3855994237,3cyclictest33-21ksoftirqd/208:25:002
3855994222,3cyclictest33-21ksoftirqd/212:10:212
3855994211,4cyclictest33-21ksoftirqd/208:45:282
385499428,10cyclictest26479-21taskset11:16:571
385499427,11cyclictest32375-21iostat11:30:191
385499427,11cyclictest32081-21seq08:08:341
385499420,39cyclictest31600-21kworker/1:110:30:271
385399426,4cyclictest9-21ksoftirqd/008:00:300
385399424,5cyclictest9-21ksoftirqd/009:10:440
385399422,5cyclictest9-21ksoftirqd/008:50:460
385399422,5cyclictest9-21ksoftirqd/008:36:220
385399422,5cyclictest9-21ksoftirqd/007:21:120
59298410,9rtkit-daemon0-21swapper/311:49:263
59298410,8rtkit-daemon0-21swapper/211:40:562
385599416,2cyclictest121rcu_preempt09:25:262
3855994134,4cyclictest1297-21runrttasks09:10:402
3855994128,11cyclictest311rcuc/211:25:002
3855994110,4cyclictest33-21ksoftirqd/210:10:202
385499415,11cyclictest6228-21latency_hist07:15:001
3854994110,10cyclictest20929-21uname11:05:221
385399412,6cyclictest9-21ksoftirqd/010:40:250
385399412,5cyclictest9-21ksoftirqd/011:45:490
385399412,5cyclictest9-21ksoftirqd/011:34:000
385399411,5cyclictest9-21ksoftirqd/008:05:160
385399410,5cyclictest9-21ksoftirqd/009:05:270
136991410,0ptp4l401ktimersoftd/310:25:253
385599406,2cyclictest33-21ksoftirqd/209:30:172
385599406,2cyclictest33-21ksoftirqd/208:25:182
385599406,2cyclictest33-21ksoftirqd/208:05:182
385599405,3cyclictest33-21ksoftirqd/211:15:262
385599404,4cyclictest33-21ksoftirqd/210:30:132
3855994034,3cyclictest1297-21runrttasks10:37:292
385499406,11cyclictest8004-21cat11:45:281
385499403,36cyclictest27867-21kworker/1:211:21:001
385399402,6cyclictest9-21ksoftirqd/009:55:250
385399401,5cyclictest9-21ksoftirqd/011:15:210
136991400,0ptp4l401ktimersoftd/312:20:163
385599397,3cyclictest33-21ksoftirqd/210:05:192
385599397,2cyclictest33-21ksoftirqd/210:00:282
385599396,6cyclictest33-21ksoftirqd/208:10:212
3855993935,3cyclictest33-21ksoftirqd/209:05:162
3855993934,3cyclictest33-21ksoftirqd/211:30:182
3855993926,10cyclictest33-21ksoftirqd/207:40:282
3855993912,7cyclictest33-21ksoftirqd/210:25:272
3855993912,7cyclictest33-21ksoftirqd/207:45:272
385399393,5cyclictest9-21ksoftirqd/008:21:570
385399391,5cyclictest9-21ksoftirqd/009:40:220
136991390,0ptp4l401ktimersoftd/310:35:243
3856993831,4cyclictest41-21ksoftirqd/310:11:593
385599389,4cyclictest33-21ksoftirqd/208:30:182
385599387,5cyclictest33-21ksoftirqd/207:55:232
385599386,3cyclictest33-21ksoftirqd/209:55:252
385599384,6cyclictest33-21ksoftirqd/211:55:202
385599384,3cyclictest121rcu_preempt10:20:272
3855993833,3cyclictest33-21ksoftirqd/212:30:002
3855993833,3cyclictest33-21ksoftirqd/207:35:002
3855993832,4cyclictest33-21ksoftirqd/207:25:552
385599382,3cyclictest121rcu_preempt11:05:252
385599381,6cyclictest33-21ksoftirqd/207:10:172
3855993813,7cyclictest33-21ksoftirqd/208:50:172
385599381,2cyclictest121rcu_preempt07:35:292
385499387,16cyclictest27675-21chrt12:26:421
385499386,13cyclictest13532-21chrt08:35:221
385399385,5cyclictest9-21ksoftirqd/010:30:250
385399384,5cyclictest9-21ksoftirqd/011:40:230
385399381,7cyclictest9-21ksoftirqd/009:00:280
136991380,1ptp4l401ktimersoftd/312:19:563
59298370,3rtkit-daemon29280-21ls12:30:223
3856993731,4cyclictest41-21ksoftirqd/311:33:283
385599377,3cyclictest33-21ksoftirqd/208:00:232
3855993731,3cyclictest33-21ksoftirqd/211:45:302
3855993712,7cyclictest33-21ksoftirqd/209:50:162
385499376,29cyclictest31600-21kworker/1:110:35:591
385399372,6cyclictest9-21ksoftirqd/009:35:010
385399372,5cyclictest9-21ksoftirqd/009:28:570
385399372,5cyclictest9-21ksoftirqd/007:49:440
385399371,6cyclictest9-21ksoftirqd/011:56:540
385399371,6cyclictest9-21ksoftirqd/007:10:250
385399371,4cyclictest9-21ksoftirqd/009:51:590
132691370,1getstats0-21swapper/308:35:153
3856993631,3cyclictest41-21ksoftirqd/308:04:003
3856993630,4cyclictest41-21ksoftirqd/311:53:263
385599369,7cyclictest33-21ksoftirqd/210:40:202
385599368,4cyclictest33-21ksoftirqd/211:10:212
385599364,2cyclictest121rcu_preempt10:45:152
3855993631,3cyclictest33-21ksoftirqd/212:35:152
385599362,3cyclictest121rcu_preempt10:15:172
3855993611,6cyclictest33-21ksoftirqd/208:35:202
3855993610,8cyclictest33-21ksoftirqd/212:20:142
3855993610,7cyclictest33-21ksoftirqd/211:50:252
3855993610,7cyclictest33-21ksoftirqd/208:55:232
385499367,27cyclictest10755-21kworker/1:208:41:391
385399363,5cyclictest9-21ksoftirqd/010:25:230
385399363,4cyclictest9-21ksoftirqd/012:20:130
385399361,6cyclictest9-21ksoftirqd/012:30:220
385399361,5cyclictest0-21swapper/010:00:020
136991360,0ptp4l401ktimersoftd/309:34:593
3856993529,4cyclictest41-21ksoftirqd/312:09:583
3856993529,4cyclictest41-21ksoftirqd/309:10:213
3856993529,4cyclictest41-21ksoftirqd/308:29:193
385599358,7cyclictest33-21ksoftirqd/212:30:252
3855993529,4cyclictest33-21ksoftirqd/210:50:142
385499358,9cyclictest31457-21taskset09:15:031
385499358,12cyclictest25997-21dpkg10:10:131
385499355,18cyclictest22175-21apt-get11:10:141
3854993512,21cyclictest10755-21kworker/1:208:25:271
385399352,6cyclictest9-21ksoftirqd/008:10:170
385399352,5cyclictest9-21ksoftirqd/008:30:240
3853993519,9cyclictest9-21ksoftirqd/012:19:330
3856993429,3cyclictest12432-21sh10:50:003
3856993428,4cyclictest41-21ksoftirqd/309:59:173
3856993428,4cyclictest41-21ksoftirqd/309:25:593
3856993428,4cyclictest41-21ksoftirqd/308:58:233
3856993428,4cyclictest41-21ksoftirqd/308:47:243
385499348,8cyclictest3286-21users09:20:281
385499346,10cyclictest23141-21taskset10:02:521
3854993429,3cyclictest25-21ksoftirqd/108:16:241
385399344,4cyclictest9-21ksoftirqd/010:05:280
3853993428,4cyclictest9-21ksoftirqd/012:04:210
3853993428,4cyclictest9-21ksoftirqd/010:46:310
3853993428,4cyclictest9-21ksoftirqd/008:46:290
1233723420,4sleep341-21ksoftirqd/307:25:253
3856993328,3cyclictest41-21ksoftirqd/312:25:233
3856993328,3cyclictest41-21ksoftirqd/310:15:333
3856993327,4cyclictest41-21ksoftirqd/311:58:563
3856993327,4cyclictest41-21ksoftirqd/311:42:593
3856993327,4cyclictest41-21ksoftirqd/310:31:063
3856993326,5cyclictest41-21ksoftirqd/308:20:443
3856993326,4cyclictest41-21ksoftirqd/312:04:593
3855993329,2cyclictest33-21ksoftirqd/212:00:202
385499336,16cyclictest25783-21mailstats09:00:261
385499335,9cyclictest20649-21missed_timers08:50:231
385499335,17cyclictest0-21swapper/109:25:181
3854993327,4cyclictest25-21ksoftirqd/108:55:121
3853993327,4cyclictest9-21ksoftirqd/011:24:350
3853993327,4cyclictest9-21ksoftirqd/009:30:050
3853993327,4cyclictest9-21ksoftirqd/008:17:520
3853993326,5cyclictest9-21ksoftirqd/008:44:120
385399332,4cyclictest9-21ksoftirqd/012:27:000
385399331,5cyclictest9-21ksoftirqd/011:51:380
136991330,0ptp4l401ktimersoftd/309:08:223
3856993227,3cyclictest41-21ksoftirqd/312:13:043
3856993227,3cyclictest41-21ksoftirqd/310:23:523
3856993227,3cyclictest41-21ksoftirqd/309:51:593
3856993227,3cyclictest41-21ksoftirqd/309:02:133
3856993226,4cyclictest41-21ksoftirqd/308:51:213
3856993226,4cyclictest41-21ksoftirqd/308:40:383
3856993226,4cyclictest41-21ksoftirqd/308:06:483
3856993226,4cyclictest41-21ksoftirqd/307:48:033
385499327,11cyclictest13848-21dpkg12:00:021
385499326,16cyclictest20982-21/usr/sbin/munin12:15:251
3854993226,4cyclictest25-21ksoftirqd/110:45:271
3854993226,4cyclictest25-21ksoftirqd/107:59:121
3854993226,4cyclictest25-21ksoftirqd/107:53:061
3854993215,15cyclictest10755-21kworker/1:208:34:571
385399326,5cyclictest9-21ksoftirqd/007:15:160
385399326,5cyclictest9-21ksoftirqd/007:15:160
3853993226,4cyclictest9-21ksoftirqd/010:22:080
3853993226,4cyclictest9-21ksoftirqd/010:16:510
3853993226,4cyclictest9-21ksoftirqd/009:24:430
3853993226,4cyclictest9-21ksoftirqd/009:19:500
3853993226,3cyclictest19182-21sshd11:00:300
3853993225,4cyclictest9-21ksoftirqd/007:59:510
385399321,5cyclictest9-21ksoftirqd/010:36:560
136991320,1ptp4l401ktimersoftd/311:00:153
3856993126,3cyclictest41-21ksoftirqd/309:47:343
3856993125,4cyclictest41-21ksoftirqd/307:54:453
385499314,17cyclictest0-21swapper/107:40:191
3853993125,4cyclictest9-21ksoftirqd/011:36:300
385399311,5cyclictest9-21ksoftirqd/010:10:150
3856993025,3cyclictest41-21ksoftirqd/312:35:503
3856993024,4cyclictest41-21ksoftirqd/311:35:453
3856993024,4cyclictest41-21ksoftirqd/311:24:463
3856993024,4cyclictest41-21ksoftirqd/311:10:583
3856993024,4cyclictest41-21ksoftirqd/308:15:103
3856993022,5cyclictest41-21ksoftirqd/308:00:003
3853993025,3cyclictest9-21ksoftirqd/008:59:430
3853993024,4cyclictest9-21ksoftirqd/011:14:240
3853993024,4cyclictest9-21ksoftirqd/007:42:140
3856992924,3cyclictest41-21ksoftirqd/310:01:543
3856992924,3cyclictest41-21ksoftirqd/309:37:033
3856992923,4cyclictest41-21ksoftirqd/308:12:503
3854992924,3cyclictest25-21ksoftirqd/107:49:521
3854992923,4cyclictest25-21ksoftirqd/110:59:161
385399294,4cyclictest9-21ksoftirqd/012:35:130
3853992923,4cyclictest9-21ksoftirqd/011:07:260
3853992923,4cyclictest9-21ksoftirqd/010:51:170
385399290,5cyclictest9-21ksoftirqd/011:25:190
136991290,0ptp4l401ktimersoftd/310:54:563
136991290,0ptp4l401ktimersoftd/309:16:083
3856992822,4cyclictest41-21ksoftirqd/311:07:163
3856992822,4cyclictest41-21ksoftirqd/310:05:513
3856992822,4cyclictest41-21ksoftirqd/307:35:593
3854992822,4cyclictest25-21ksoftirqd/109:51:081
385399285,5cyclictest9-21ksoftirqd/007:35:180
385399283,4cyclictest9-21ksoftirqd/012:10:220
136991280,0ptp4l401ktimersoftd/309:24:393
3854992710,14cyclictest142601irqrtprio07:30:181
136991260,0ptp4l401ktimersoftd/310:55:143
59298250,2rtkit-daemon0-21swapper/310:44:353
3854992521,3cyclictest25-21ksoftirqd/110:15:221
385399256,4cyclictest9-21ksoftirqd/007:30:220
385399242,6cyclictest9-21ksoftirqd/008:25:170
136991240,1ptp4l401ktimersoftd/307:21:473
136991240,0ptp4l401ktimersoftd/309:40:123
136991240,0ptp4l401ktimersoftd/307:15:273
136991240,0ptp4l401ktimersoftd/307:15:273
385499230,9cyclictest0-21swapper/110:05:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional