You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-05 - 21:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Sat Jul 05, 2025 12:45:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15992231166,22sleep30-21swapper/307:05:313
19382206172,22sleep10-21swapper/107:08:471
18042205172,22sleep20-21swapper/207:07:072
18352202169,22sleep00-21swapper/007:07:280
413921480,6sleep1233399cyclictest11:40:241
1373918363,11phc2sys0-21swapper/307:10:013
270782710,2sleep00-21swapper/009:09:590
297472690,6sleep0233299cyclictest08:05:200
13852680,2sleep20-21swapper/209:20:282
2335995927,2cyclictest33-21ksoftirqd/209:50:192
179942590,6sleep2233599cyclictest12:10:192
179942590,6sleep2233599cyclictest12:10:192
233272580,2sleep10-21swapper/111:15:181
137291560,1ptp4l391rcuc/307:20:213
233299550,11cyclictest18702-21/usr/sbin/munin11:05:200
137291550,1ptp4l391rcuc/308:20:293
233299541,10cyclictest22754-21ls09:00:130
233299540,19cyclictest0-21swapper/012:20:160
233299540,18cyclictest0-21swapper/011:45:000
233299540,15cyclictest0-21swapper/010:05:210
271802530,1sleep00-21swapper/012:30:180
233299530,9cyclictest29418-21/usr/sbin/munin12:35:160
233299520,9cyclictest0-21swapper/011:10:210
233299510,11cyclictest26890-21cut08:00:180
233299500,12cyclictest16859-21/usr/sbin/munin07:40:180
2335994922,3cyclictest33-21ksoftirqd/208:50:012
2335994918,5cyclictest33-21ksoftirqd/209:55:152
2335994917,6cyclictest33-21ksoftirqd/210:19:592
233599484,7cyclictest23593-21kworker/2:108:14:592
2335994812,6cyclictest33-21ksoftirqd/209:20:002
233299480,16cyclictest23903-21cut11:15:220
233299470,17cyclictest5275-21/usr/sbin/munin11:45:200
233299470,10cyclictest8549-21grep07:20:280
295712460,2sleep329577-21irqrtprio08:05:203
2335994623,3cyclictest33-21ksoftirqd/212:05:012
2335994616,7cyclictest33-21ksoftirqd/211:05:202
233299460,17cyclictest0-21swapper/009:50:250
233299460,11cyclictest8669-21grep09:35:260
2335994520,7cyclictest33-21ksoftirqd/207:50:262
2335994518,7cyclictest33-21ksoftirqd/207:15:012
2335994518,6cyclictest33-21ksoftirqd/211:35:162
233299450,8cyclictest6125-21gzip07:15:260
233299450,17cyclictest0-21swapper/011:50:260
233299450,10cyclictest19364-21cpuspeed_turbos07:45:150
2335994431,10cyclictest33-21ksoftirqd/211:25:172
2335994417,4cyclictest33-21ksoftirqd/211:30:252
233299440,17cyclictest19151-21mailstats08:50:270
233299440,15cyclictest19367-21/usr/sbin/munin10:00:210
233599438,5cyclictest33-21ksoftirqd/211:50:202
233599438,10cyclictest33-21ksoftirqd/210:30:012
2335994331,10cyclictest33-21ksoftirqd/209:35:222
2335994319,7cyclictest33-21ksoftirqd/211:45:172
2335994315,7cyclictest33-21ksoftirqd/207:55:292
2335994313,6cyclictest33-21ksoftirqd/208:55:132
2335994312,6cyclictest33-21ksoftirqd/212:05:262
2335994312,5cyclictest33-21ksoftirqd/210:50:192
233599430,3cyclictest121rcu_preempt09:30:002
233299430,8cyclictest1655-21ps11:35:250
2336994231,6cyclictest41-21ksoftirqd/310:25:143
233599429,7cyclictest33-21ksoftirqd/208:40:182
233599428,6cyclictest33-21ksoftirqd/212:15:242
233599428,5cyclictest33-21ksoftirqd/207:25:012
2335994230,9cyclictest33-21ksoftirqd/210:00:172
2335994217,7cyclictest33-21ksoftirqd/208:05:002
2335994215,7cyclictest33-21ksoftirqd/211:00:212
2335994213,6cyclictest33-21ksoftirqd/207:35:292
233399420,40cyclictest0-21swapper/107:41:391
233299420,8cyclictest10060-21vmstat08:30:290
233299420,14cyclictest24968-21cut10:10:200
137391423,2phc2sys121rcu_preempt09:40:153
137291420,0ptp4l401ktimersoftd/310:42:583
137291420,0ptp4l401ktimersoftd/307:27:483
233599418,6cyclictest33-21ksoftirqd/210:55:182
233599415,7cyclictest33-21ksoftirqd/209:45:202
233599415,5cyclictest33-21ksoftirqd/212:30:282
2335994134,4cyclictest27627-21dump-pmu-power11:25:002
2335994121,8cyclictest33-21ksoftirqd/209:00:162
233599411,7cyclictest33-21ksoftirqd/212:20:272
2335994116,7cyclictest33-21ksoftirqd/209:35:012
2335994116,6cyclictest33-21ksoftirqd/208:50:202
2335994112,4cyclictest33-21ksoftirqd/207:45:172
2335994111,4cyclictest33-21ksoftirqd/212:35:192
233299410,7cyclictest409-21python10:25:280
233599408,6cyclictest33-21ksoftirqd/211:10:292
233599408,4cyclictest33-21ksoftirqd/209:40:292
233599406,6cyclictest33-21ksoftirqd/212:00:012
2335994013,4cyclictest33-21ksoftirqd/208:18:492
2335994010,6cyclictest33-21ksoftirqd/210:35:012
2335994010,4cyclictest33-21ksoftirqd/208:20:192
233299400,9cyclictest3913-21threads07:10:280
233299400,8cyclictest22924-21ptp4l-jitter07:50:260
233299400,7cyclictest12786-21vmstat10:50:310
233299400,7cyclictest10926-21ps09:40:260
233299400,7cyclictest10046-21df_abs07:25:170
233299400,15cyclictest2494-21gnome-shell10:47:590
233599399,6cyclictest33-21ksoftirqd/207:20:182
233599398,6cyclictest33-21ksoftirqd/209:05:282
233599394,5cyclictest33-21ksoftirqd/211:15:192
233599393,6cyclictest33-21ksoftirqd/212:30:012
2335993933,3cyclictest33-21ksoftirqd/210:45:002
2335993915,6cyclictest33-21ksoftirqd/208:35:292
233299390,8cyclictest31227-21ls09:15:270
233299390,8cyclictest2494-21gnome-shell09:13:580
233299390,7cyclictest14364-21ptp4l08:40:280
233299390,7cyclictest10018-21apt-get11:55:130
2336993832,3cyclictest41-21ksoftirqd/307:47:523
233599386,6cyclictest33-21ksoftirqd/209:10:142
2335993827,9cyclictest33-21ksoftirqd/207:45:002
233399380,37cyclictest0-21swapper/107:15:391
2332993831,4cyclictest9-21ksoftirqd/007:32:320
233299380,7cyclictest6939-21iostat10:40:200
233299380,7cyclictest17497-21hddtemp_smartct09:55:180
233299380,7cyclictest1635-21sshd08:15:170
137291380,7ptp4l233699cyclictest11:58:093
2336993731,4cyclictest41-21ksoftirqd/311:41:343
233599379,6cyclictest33-21ksoftirqd/211:43:412
233599377,6cyclictest33-21ksoftirqd/207:35:012
233599376,6cyclictest33-21ksoftirqd/210:35:292
233599373,3cyclictest121rcu_preempt07:15:262
2333993730,4cyclictest25-21ksoftirqd/110:48:371
233399370,1cyclictest0-21swapper/108:51:381
2332993731,4cyclictest9-21ksoftirqd/012:05:070
233299370,8cyclictest5350-21iostat09:30:200
233299370,7cyclictest13681-21df10:55:130
2336993630,4cyclictest41-21ksoftirqd/312:14:523
2336993630,4cyclictest41-21ksoftirqd/312:14:523
2336993630,4cyclictest41-21ksoftirqd/311:35:543
2336993630,4cyclictest41-21ksoftirqd/311:13:233
2336993630,3cyclictest41-21ksoftirqd/312:04:593
2336993629,4cyclictest41-21ksoftirqd/312:38:453
2335993630,3cyclictest33-21ksoftirqd/210:09:592
233599361,6cyclictest33-21ksoftirqd/210:10:192
2333993631,3cyclictest25-21ksoftirqd/107:10:131
2333993611,24cyclictest0-21swapper/111:51:301
2332993630,4cyclictest9-21ksoftirqd/010:33:290
233299360,9cyclictest0-21swapper/007:59:330
233299360,7cyclictest4936-21meminfo10:35:230
233299360,7cyclictest28058-21ls10:15:280
233299360,6cyclictest12176-21ssh08:35:280
2336993529,4cyclictest41-21ksoftirqd/309:59:333
2336993529,4cyclictest41-21ksoftirqd/307:39:523
2336993528,4cyclictest41-21ksoftirqd/309:54:473
2336993528,4cyclictest41-21ksoftirqd/307:53:293
233599358,10cyclictest33-21ksoftirqd/208:25:162
233599356,6cyclictest33-21ksoftirqd/210:45:152
2335993529,3cyclictest33-21ksoftirqd/208:07:542
2335993517,10cyclictest33-21ksoftirqd/210:25:012
2335993517,10cyclictest33-21ksoftirqd/210:25:002
2333993529,4cyclictest25-21ksoftirqd/111:13:491
2333993529,4cyclictest25-21ksoftirqd/110:54:151
2333993529,4cyclictest25-21ksoftirqd/108:20:551
2333993527,5cyclictest25-21ksoftirqd/112:06:441
233399351,33cyclictest0-21swapper/112:15:301
233399350,33cyclictest0-21swapper/110:11:091
2332993529,4cyclictest9-21ksoftirqd/009:29:290
2332993528,4cyclictest9-21ksoftirqd/008:59:050
2336993429,3cyclictest41-21ksoftirqd/310:49:033
2336993428,4cyclictest41-21ksoftirqd/311:20:443
2336993428,4cyclictest41-21ksoftirqd/309:33:413
2336993428,4cyclictest41-21ksoftirqd/309:24:473
2336993428,4cyclictest41-21ksoftirqd/309:03:013
2336993427,4cyclictest41-21ksoftirqd/307:15:413
233599346,3cyclictest33-21ksoftirqd/208:30:292
2333993429,3cyclictest25-21ksoftirqd/110:15:141
2333993428,4cyclictest25-21ksoftirqd/110:36:251
2333993428,4cyclictest25-21ksoftirqd/110:29:151
2333993427,5cyclictest25-21ksoftirqd/110:25:011
2333993427,5cyclictest25-21ksoftirqd/110:25:001
2333993427,4cyclictest25-21ksoftirqd/107:24:371
2332993429,3cyclictest9-21ksoftirqd/012:28:480
2332993428,4cyclictest9-21ksoftirqd/009:20:450
2332993427,4cyclictest9-21ksoftirqd/010:21:490
2332993427,4cyclictest9-21ksoftirqd/010:21:480
2332993427,4cyclictest9-21ksoftirqd/008:24:030
233299340,7cyclictest30945-21hddtemp_smartct11:30:180
233299340,16cyclictest16441-21ls08:45:210
2336993327,4cyclictest41-21ksoftirqd/312:16:113
2336993327,4cyclictest41-21ksoftirqd/311:51:053
2336993327,4cyclictest41-21ksoftirqd/309:36:543
2336993327,4cyclictest41-21ksoftirqd/308:27:213
2336993327,3cyclictest41-21ksoftirqd/312:25:083
2336993326,5cyclictest41-21ksoftirqd/309:45:083
2336993325,5cyclictest41-21ksoftirqd/310:34:273
2336993325,5cyclictest41-21ksoftirqd/308:40:013
2333993328,3cyclictest25-21ksoftirqd/109:11:421
2333993327,4cyclictest25-21ksoftirqd/110:30:561
2333993327,4cyclictest25-21ksoftirqd/108:37:431
2333993326,5cyclictest25-21ksoftirqd/107:29:391
2333993326,4cyclictest25-21ksoftirqd/108:30:211
233399330,1cyclictest0-21swapper/112:12:391
233399330,1cyclictest0-21swapper/112:12:391
2336993226,4cyclictest41-21ksoftirqd/312:20:583
2336993226,4cyclictest41-21ksoftirqd/311:05:053
2336993226,3cyclictest41-21ksoftirqd/308:35:573
2336993225,5cyclictest41-21ksoftirqd/311:30:013
2333993226,4cyclictest25-21ksoftirqd/112:35:411
2333993226,4cyclictest25-21ksoftirqd/112:31:481
2333993226,4cyclictest25-21ksoftirqd/111:27:091
2333993226,4cyclictest25-21ksoftirqd/109:56:421
2333993226,4cyclictest25-21ksoftirqd/108:58:091
2333993226,4cyclictest25-21ksoftirqd/107:39:501
2333993225,4cyclictest25-21ksoftirqd/109:42:541
233399320,2cyclictest121rcu_preempt10:44:351
2332993225,4cyclictest9-21ksoftirqd/012:10:450
2332993225,4cyclictest9-21ksoftirqd/012:10:450
137291320,1ptp4l391rcuc/311:30:193
2336993126,3cyclictest41-21ksoftirqd/311:18:203
2336993126,3cyclictest41-21ksoftirqd/310:50:133
2336993126,3cyclictest41-21ksoftirqd/309:15:023
2336993126,3cyclictest41-21ksoftirqd/308:53:533
2336993126,3cyclictest41-21ksoftirqd/308:15:473
2336993125,4cyclictest41-21ksoftirqd/311:49:093
2336993125,4cyclictest41-21ksoftirqd/310:16:153
2336993125,4cyclictest41-21ksoftirqd/307:43:583
2336993124,4cyclictest41-21ksoftirqd/308:56:143
2333993125,4cyclictest25-21ksoftirqd/111:56:211
2333993125,4cyclictest25-21ksoftirqd/111:04:551
2333993125,4cyclictest25-21ksoftirqd/109:53:211
2333993125,4cyclictest25-21ksoftirqd/109:27:341
2333993125,4cyclictest25-21ksoftirqd/108:13:011
2333993125,3cyclictest25-21ksoftirqd/108:07:461
2333993124,4cyclictest25-21ksoftirqd/107:57:401
2332993124,4cyclictest9-21ksoftirqd/011:22:500
137291310,9ptp4l401ktimersoftd/310:12:533
137291310,8ptp4l401ktimersoftd/310:35:583
233699308,3cyclictest26631-21head09:05:303
2336993025,3cyclictest41-21ksoftirqd/311:02:453
2336993025,3cyclictest41-21ksoftirqd/310:59:313
2336993025,3cyclictest41-21ksoftirqd/308:12:383
2336993024,4cyclictest41-21ksoftirqd/308:04:563
2336993012,3cyclictest9087-21sed08:30:203
2333993025,3cyclictest25-21ksoftirqd/109:32:021
2333993025,3cyclictest25-21ksoftirqd/109:04:521
2333993024,4cyclictest25-21ksoftirqd/108:48:381
2333993024,3cyclictest25-21ksoftirqd/109:23:161
2333993024,3cyclictest25-21ksoftirqd/109:08:131
233299300,19cyclictest0-21swapper/012:15:140
233299300,17cyclictest0-21swapper/008:10:210
233699298,3cyclictest1314-21snmpd09:28:163
233699293,4cyclictest218-21systemd-journal07:55:013
2336992923,4cyclictest41-21ksoftirqd/310:08:563
2336992923,4cyclictest41-21ksoftirqd/309:11:523
2333992924,3cyclictest25-21ksoftirqd/108:42:591
2333992923,4cyclictest25-21ksoftirqd/112:25:171
2333992923,4cyclictest25-21ksoftirqd/111:36:061
2333992923,4cyclictest25-21ksoftirqd/111:23:231
2333992923,4cyclictest25-21ksoftirqd/110:05:531
2332992924,3cyclictest9-21ksoftirqd/008:29:360
2332992922,4cyclictest9-21ksoftirqd/012:00:340
2336992823,3cyclictest41-21ksoftirqd/312:06:583
2336992811,3cyclictest13309-21phc2sys07:30:233
2336992810,2cyclictest31007-21latency_hist10:25:003
2336992810,2cyclictest31007-21latency_hist10:25:003
2333992823,3cyclictest25-21ksoftirqd/111:05:421
2333992823,3cyclictest25-21ksoftirqd/108:00:531
2333992822,4cyclictest25-21ksoftirqd/108:15:361
233699277,3cyclictest28018-21latency12:30:253
233699272,5cyclictest15428-21apt-get08:45:123
2333992722,3cyclictest25-21ksoftirqd/109:35:311
2333992722,3cyclictest25-21ksoftirqd/107:48:171
2333992721,4cyclictest25-21ksoftirqd/110:04:091
2332992722,3cyclictest9-21ksoftirqd/011:00:110
2332992721,4cyclictest9-21ksoftirqd/007:39:180
2333992621,3cyclictest25-21ksoftirqd/112:22:321
2333992621,3cyclictest25-21ksoftirqd/109:47:211
2333992620,3cyclictest25-21ksoftirqd/110:56:491
233399260,2cyclictest0-21swapper/111:48:461
233399260,15cyclictest0-21swapper/112:00:061
233299260,17cyclictest0-21swapper/009:45:160
2333992520,3cyclictest25-21ksoftirqd/107:52:311
2333992519,3cyclictest25-21ksoftirqd/111:30:521
137291250,0ptp4l401ktimersoftd/310:00:153
2333992320,2cyclictest0-21swapper/109:18:501
233299220,8cyclictest0-21swapper/011:27:580
2333991813,3cyclictest12476-21sed07:30:151
2333991711,4cyclictest7954-21seq08:28:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional