You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 14:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Sun Jul 13, 2025 00:45:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
314442209176,22sleep20-21swapper/219:06:242
313972205172,22sleep00-21swapper/019:05:460
137391205172,22phc2sys0-21swapper/319:06:153
314132204170,22sleep10-21swapper/119:06:001
317302145121,13sleep00-21swapper/019:10:000
1973521400,2sleep10-21swapper/121:00:001
61582640,2sleep30-21swapper/322:45:173
127202560,2sleep10-21swapper/119:35:231
32022995516,3cyclictest9-21ksoftirqd/022:25:300
32022995318,5cyclictest9-21ksoftirqd/019:55:000
32022995215,9cyclictest9-21ksoftirqd/019:30:230
127742520,2sleep10-21swapper/100:05:301
32022995121,4cyclictest9-21ksoftirqd/019:15:180
32022995120,9cyclictest9-21ksoftirqd/021:55:280
32022995114,7cyclictest9-21ksoftirqd/020:05:280
220142510,1sleep00-21swapper/022:10:200
32022995016,8cyclictest9-21ksoftirqd/023:15:260
32024994930,16cyclictest33-21ksoftirqd/222:20:012
32022994914,5cyclictest9-21ksoftirqd/020:35:250
3202299489,7cyclictest9-21ksoftirqd/021:10:140
32022994814,3cyclictest9-21ksoftirqd/023:50:150
132591480,2getstats0-21swapper/300:34:423
32022994712,6cyclictest9-21ksoftirqd/020:55:300
32022994712,5cyclictest9-21ksoftirqd/022:20:280
32022994712,4cyclictest9-21ksoftirqd/023:55:170
3202499462,7cyclictest2779-21irqrtprio21:30:192
32024994614,4cyclictest33-21ksoftirqd/222:10:012
32023994632,4cyclictest25-21ksoftirqd/119:30:001
32023994618,27cyclictest0-21swapper/119:10:291
32022994617,9cyclictest9-21ksoftirqd/023:00:140
32022994617,3cyclictest9-21ksoftirqd/000:00:220
32022994613,3cyclictest9-21ksoftirqd/000:25:160
32022994611,6cyclictest9-21ksoftirqd/020:40:240
32022994611,5cyclictest9-21ksoftirqd/020:00:150
32022994610,5cyclictest9-21ksoftirqd/022:40:120
260532460,2sleep20-21swapper/222:20:162
137291460,1ptp4l0-21swapper/320:36:103
32025994526,16cyclictest41-21ksoftirqd/322:44:493
3202299459,5cyclictest9-21ksoftirqd/000:20:390
32022994532,3cyclictest9-21ksoftirqd/019:55:190
32022994514,7cyclictest9-21ksoftirqd/023:20:210
32022994513,6cyclictest9-21ksoftirqd/020:45:240
32022994513,6cyclictest9-21ksoftirqd/020:25:150
32022994511,2cyclictest121rcu_preempt22:53:050
299562450,1sleep00-21swapper/020:10:290
3202399440,3cyclictest31524-21apt-get21:25:011
3202299449,5cyclictest9-21ksoftirqd/021:00:290
3202299448,5cyclictest9-21ksoftirqd/020:30:250
32022994439,3cyclictest9-21ksoftirqd/023:40:000
32022994415,3cyclictest9-21ksoftirqd/021:25:150
32022994414,3cyclictest9-21ksoftirqd/000:15:000
32022994413,6cyclictest9-21ksoftirqd/021:30:200
32022994412,6cyclictest9-21ksoftirqd/020:15:210
32022994410,7cyclictest9-21ksoftirqd/021:50:280
32022994410,7cyclictest9-21ksoftirqd/021:50:280
32024994329,3cyclictest33-21ksoftirqd/221:05:152
32024994325,9cyclictest33-21ksoftirqd/220:00:002
3202299439,6cyclictest9-21ksoftirqd/022:05:280
3202299439,4cyclictest9-21ksoftirqd/021:15:130
3202299438,5cyclictest9-21ksoftirqd/021:40:170
3202299438,5cyclictest9-21ksoftirqd/021:35:280
3202299438,5cyclictest9-21ksoftirqd/020:20:180
3202299438,5cyclictest9-21ksoftirqd/019:40:220
32022994313,6cyclictest9-21ksoftirqd/021:05:220
32024994236,3cyclictest33-21ksoftirqd/219:40:002
32024994220,3cyclictest33-21ksoftirqd/219:25:272
3202399421,3cyclictest4896-21tune2fs19:20:191
3202299429,4cyclictest9-21ksoftirqd/022:15:170
32022994237,3cyclictest9-21ksoftirqd/019:40:000
32022994225,4cyclictest9-21ksoftirqd/019:25:250
3202299422,3cyclictest23071-21kworker/0:221:20:180
32022994216,6cyclictest9-21ksoftirqd/022:30:300
32022994210,3cyclictest9-21ksoftirqd/000:35:220
137291420,0ptp4l401ktimersoftd/322:13:333
3202499415,3cyclictest33-21ksoftirqd/221:20:162
32023994136,3cyclictest25-21ksoftirqd/120:19:591
3202399413,3cyclictest121rcu_preempt21:50:011
3202299419,5cyclictest9-21ksoftirqd/023:45:170
3202299418,5cyclictest9-21ksoftirqd/000:05:210
32022994112,3cyclictest9-21ksoftirqd/021:45:260
32022994111,4cyclictest9-21ksoftirqd/022:55:170
32022994110,6cyclictest9-21ksoftirqd/023:30:200
42098400,2rtkit-daemon0-21swapper/220:10:282
32024994036,3cyclictest33-21ksoftirqd/220:20:012
32023994036,3cyclictest25-21ksoftirqd/121:20:011
32023994011,3cyclictest25-21ksoftirqd/122:00:251
132691400,1getstats0-21swapper/321:40:013
32025993914,22cyclictest10838-21munin-node00:35:133
3202499397,7cyclictest33-21ksoftirqd/221:55:002
3202499397,7cyclictest33-21ksoftirqd/221:55:002
32023993935,2cyclictest25-21ksoftirqd/122:25:001
3202299396,5cyclictest9-21ksoftirqd/022:35:310
32022993934,3cyclictest9-21ksoftirqd/023:14:590
32022993934,3cyclictest9-21ksoftirqd/000:20:000
32022993933,3cyclictest9-21ksoftirqd/019:25:010
3202299392,4cyclictest121rcu_preempt23:05:240
32022993924,4cyclictest9-21ksoftirqd/022:00:190
137291390,13ptp4l401ktimersoftd/320:20:133
3202499388,4cyclictest33-21ksoftirqd/222:10:162
32024993834,3cyclictest33-21ksoftirqd/219:25:012
3202499383,3cyclictest121rcu_preempt00:15:232
32024993833,3cyclictest33-21ksoftirqd/219:35:012
32023993833,3cyclictest25-21ksoftirqd/100:40:001
32023993832,3cyclictest25-21ksoftirqd/119:50:001
32023993811,3cyclictest25-21ksoftirqd/123:25:011
3202299380,3cyclictest121rcu_preempt22:45:290
32025993731,4cyclictest41-21ksoftirqd/319:10:153
32025993731,4cyclictest41-21ksoftirqd/300:22:133
32025993730,4cyclictest41-21ksoftirqd/319:37:113
3202499374,5cyclictest121rcu_preempt19:45:182
32024993732,3cyclictest33-21ksoftirqd/220:25:012
3202499371,5cyclictest33-21ksoftirqd/223:25:172
3202499371,4cyclictest33-21ksoftirqd/219:55:002
32024993710,7cyclictest33-21ksoftirqd/220:40:152
32023993732,3cyclictest25-21ksoftirqd/122:45:001
32023993731,4cyclictest25-21ksoftirqd/120:35:581
32023993731,3cyclictest25-21ksoftirqd/100:30:001
3202399371,10cyclictest25-21ksoftirqd/100:10:281
32022993732,2cyclictest9-21ksoftirqd/023:40:210
32022993731,3cyclictest9-21ksoftirqd/023:25:220
3202299370,3cyclictest121rcu_preempt20:50:160
3202299370,3cyclictest121rcu_preempt20:50:150
132691370,3getstats3202599cyclictest19:35:003
132691370,2getstats3202599cyclictest20:10:003
32025993630,4cyclictest41-21ksoftirqd/322:57:563
32025993630,3cyclictest41-21ksoftirqd/322:38:183
3202499366,3cyclictest33-21ksoftirqd/221:40:232
3202499363,4cyclictest33-21ksoftirqd/220:02:322
32024993632,3cyclictest33-21ksoftirqd/219:14:592
32024993631,3cyclictest33-21ksoftirqd/223:40:292
32024993630,3cyclictest33-21ksoftirqd/200:05:162
32024993629,5cyclictest33-21ksoftirqd/223:48:542
3202499362,3cyclictest121rcu_preempt23:35:312
3202399367,2cyclictest121rcu_preempt23:40:291
32023993633,2cyclictest231rcuc/121:30:241
32023993618,8cyclictest25-21ksoftirqd/121:40:221
3202399361,5cyclictest121rcu_preempt20:30:241
32022993630,3cyclictest9-21ksoftirqd/000:30:160
32025993529,4cyclictest41-21ksoftirqd/323:23:573
32025993529,4cyclictest41-21ksoftirqd/322:03:003
32025993529,4cyclictest41-21ksoftirqd/321:32:153
32025993529,4cyclictest41-21ksoftirqd/319:54:243
32025993529,4cyclictest41-21ksoftirqd/300:25:323
32025993528,5cyclictest41-21ksoftirqd/322:27:203
32025993528,4cyclictest41-21ksoftirqd/323:13:423
32025993513,2cyclictest18675-21/usr/sbin/munin22:05:313
32025993511,3cyclictest6286-21latency_hist23:55:003
32025993510,4cyclictest1285-21runrttasks21:00:153
3202499356,5cyclictest33-21ksoftirqd/222:45:282
3202499356,3cyclictest33-21ksoftirqd/223:05:212
3202499355,4cyclictest33-21ksoftirqd/220:55:272
3202499355,3cyclictest33-21ksoftirqd/223:30:222
3202499354,4cyclictest33-21ksoftirqd/220:35:142
3202499354,2cyclictest33-21ksoftirqd/219:40:272
3202499353,6cyclictest33-21ksoftirqd/222:50:202
32024993531,3cyclictest33-21ksoftirqd/223:50:272
32024993530,3cyclictest33-21ksoftirqd/221:40:002
32024993530,3cyclictest33-21ksoftirqd/200:14:592
32024993530,2cyclictest33-21ksoftirqd/221:00:172
3202499351,4cyclictest33-21ksoftirqd/223:20:272
3202499351,4cyclictest121rcu_preempt23:10:152
3202499350,7cyclictest33-21ksoftirqd/222:35:182
3202499350,2cyclictest121rcu_preempt20:46:382
32023993532,2cyclictest25-21ksoftirqd/120:25:001
32023993530,3cyclictest25-21ksoftirqd/122:10:131
32023993530,3cyclictest25-21ksoftirqd/120:45:001
32023993523,10cyclictest25-21ksoftirqd/123:45:291
32023993523,10cyclictest25-21ksoftirqd/121:50:251
32023993523,10cyclictest25-21ksoftirqd/121:50:251
32023993522,10cyclictest25-21ksoftirqd/119:15:231
32022993531,3cyclictest9-21ksoftirqd/019:45:200
3202599349,3cyclictest16770-21grep23:05:303
32025993428,4cyclictest41-21ksoftirqd/322:30:053
32025993428,4cyclictest41-21ksoftirqd/320:27:453
32025993428,4cyclictest41-21ksoftirqd/319:47:583
3202499343,4cyclictest121rcu_preempt21:10:182
3202499343,2cyclictest121rcu_preempt19:15:212
32024993431,2cyclictest33-21ksoftirqd/223:00:002
32024993430,3cyclictest33-21ksoftirqd/223:00:202
32024993428,4cyclictest33-21ksoftirqd/223:57:492
32024993426,5cyclictest33-21ksoftirqd/200:30:392
32024993423,2cyclictest33-21ksoftirqd/220:25:152
32024993419,5cyclictest33-21ksoftirqd/200:30:002
32024993416,8cyclictest33-21ksoftirqd/222:45:002
32024993414,9cyclictest33-21ksoftirqd/222:00:002
3202499341,3cyclictest33-21ksoftirqd/220:05:252
3202499341,2cyclictest121rcu_preempt00:00:132
32023993430,3cyclictest25-21ksoftirqd/122:05:221
32023993430,2cyclictest25-21ksoftirqd/120:10:011
32023993429,3cyclictest25-21ksoftirqd/123:55:151
32023993429,3cyclictest25-21ksoftirqd/123:05:161
32023993429,3cyclictest25-21ksoftirqd/121:58:111
32023993429,3cyclictest25-21ksoftirqd/120:51:421
32023993429,3cyclictest25-21ksoftirqd/120:51:421
32023993429,3cyclictest25-21ksoftirqd/120:30:001
32023993429,3cyclictest25-21ksoftirqd/119:40:231
32023993428,4cyclictest25-21ksoftirqd/123:38:341
32023993428,4cyclictest25-21ksoftirqd/123:11:071
32023993428,4cyclictest25-21ksoftirqd/121:07:471
32023993428,4cyclictest25-21ksoftirqd/119:57:071
3202399342,4cyclictest18606-21latency_hist00:20:011
3202399342,3cyclictest121rcu_preempt19:32:511
3202399341,4cyclictest31887-21grep22:30:281
42098330,16rtkit-daemon0-21swapper/222:00:282
3202599339,3cyclictest11517-21cat21:50:013
32025993328,3cyclictest41-21ksoftirqd/321:57:033
32025993327,4cyclictest41-21ksoftirqd/323:47:593
32025993327,4cyclictest41-21ksoftirqd/320:49:243
32025993327,4cyclictest41-21ksoftirqd/319:16:373
32025993327,4cyclictest41-21ksoftirqd/300:15:123
32025993327,4cyclictest41-21ksoftirqd/300:09:053
32025993314,17cyclictest29779-21fschecks_time23:35:193
32025993310,3cyclictest12371-21sed21:50:243
32025993310,3cyclictest12371-21sed21:50:243
32024993329,3cyclictest33-21ksoftirqd/200:25:002
32024993328,3cyclictest33-21ksoftirqd/221:45:142
32024993327,4cyclictest33-21ksoftirqd/221:15:582
32024993327,4cyclictest33-21ksoftirqd/200:39:382
3202399333,4cyclictest1854-21cat21:30:001
3202399333,4cyclictest10490-21ls00:00:261
32023993328,3cyclictest25-21ksoftirqd/122:55:001
32023993328,3cyclictest25-21ksoftirqd/122:44:591
32023993328,3cyclictest25-21ksoftirqd/120:14:591
32023993327,4cyclictest25-21ksoftirqd/122:35:401
32023993327,3cyclictest25-21ksoftirqd/119:50:161
3202399332,6cyclictest25-21ksoftirqd/121:10:281
32023993319,3cyclictest25-21ksoftirqd/100:25:001
3202399331,5cyclictest121rcu_preempt21:00:191
137291330,1ptp4l401ktimersoftd/323:41:223
32025993227,3cyclictest41-21ksoftirqd/321:25:013
32025993227,3cyclictest41-21ksoftirqd/321:07:333
32025993226,4cyclictest41-21ksoftirqd/323:30:063
32025993226,4cyclictest41-21ksoftirqd/319:59:013
32025993210,3cyclictest5983-21switchtime19:20:283
3202499324,4cyclictest33-21ksoftirqd/221:25:262
32024993227,3cyclictest33-21ksoftirqd/223:15:502
32024993227,3cyclictest33-21ksoftirqd/220:50:162
32024993227,3cyclictest33-21ksoftirqd/220:50:162
32024993226,3cyclictest33-21ksoftirqd/222:30:192
32024993225,4cyclictest33-21ksoftirqd/220:30:122
32023993227,3cyclictest25-21ksoftirqd/122:18:511
32023993227,3cyclictest25-21ksoftirqd/120:05:011
32023993226,4cyclictest25-21ksoftirqd/123:34:581
32023993226,4cyclictest25-21ksoftirqd/123:29:491
32023993226,4cyclictest25-21ksoftirqd/122:29:151
32023993222,9cyclictest25-21ksoftirqd/122:55:271
137291320,9ptp4l401ktimersoftd/321:14:393
3202599319,3cyclictest10208-21ptp4l00:00:243
32025993127,2cyclictest137291ptp4l20:15:083
32025993126,3cyclictest41-21ksoftirqd/323:56:503
32025993126,3cyclictest41-21ksoftirqd/320:41:473
32025993126,3cyclictest41-21ksoftirqd/320:13:243
32025993125,4cyclictest41-21ksoftirqd/322:18:453
32025993125,4cyclictest41-21ksoftirqd/320:56:373
32025993125,3cyclictest41-21ksoftirqd/322:55:003
32024993126,3cyclictest33-21ksoftirqd/222:25:182
3202399313,3cyclictest121rcu_preempt20:45:231
32023993127,3cyclictest25-21ksoftirqd/121:39:591
32023993126,3cyclictest25-21ksoftirqd/123:00:201
32023993125,3cyclictest25-21ksoftirqd/123:55:001
32023993116,3cyclictest25-21ksoftirqd/123:15:151
137291310,9ptp4l0-21swapper/300:13:323
137291310,8ptp4l0-21swapper/323:26:063
32025993024,3cyclictest41-21ksoftirqd/321:27:323
3202599301,5cyclictest5610-21apt-get20:30:133
3202599301,5cyclictest1314-21snmpd20:50:133
3202599301,5cyclictest1314-21snmpd20:50:133
32025993012,2cyclictest1285-21runrttasks22:22:513
32025992924,3cyclictest41-21ksoftirqd/321:16:453
32025992923,4cyclictest41-21ksoftirqd/323:16:553
32025992910,2cyclictest1285-21runrttasks20:03:023
32025992823,3cyclictest41-21ksoftirqd/319:40:593
32025992722,3cyclictest41-21ksoftirqd/321:37:453
3202399260,6cyclictest121rcu_preempt00:30:161
137291240,0ptp4l401ktimersoftd/323:00:303
137291230,0ptp4l401ktimersoftd/319:25:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional