You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 09:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Mon Dec 08, 2025 00:45:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
136691227159,23phc2sys0-21swapper/319:09:233
289012208174,22sleep20-21swapper/219:07:452
289052207173,23sleep10-21swapper/119:07:491
289862203170,22sleep00-21swapper/019:08:510
2376521530,6sleep12936799cyclictest21:10:281
136591650,0ptp4l401ktimersoftd/300:23:053
195072600,4sleep02936699cyclictest00:25:240
165212590,4sleep229365-21cyclictest20:55:282
29368995828,7cyclictest33-21ksoftirqd/222:45:282
94112560,2sleep00-21swapper/019:35:160
29368995426,7cyclictest33-21ksoftirqd/200:00:002
316102510,2sleep30-21swapper/319:15:003
29368995120,8cyclictest33-21ksoftirqd/220:40:282
29368995018,5cyclictest33-21ksoftirqd/200:07:362
2936899492,5cyclictest33-21ksoftirqd/221:05:172
29368994912,5cyclictest33-21ksoftirqd/223:25:242
133391490,7getstats2936999cyclictest23:25:103
29368994824,5cyclictest33-21ksoftirqd/219:50:252
29368994820,5cyclictest33-21ksoftirqd/221:10:262
136591480,2ptp4l401ktimersoftd/321:02:213
2936899473,6cyclictest33-21ksoftirqd/220:45:262
29368994717,5cyclictest33-21ksoftirqd/220:05:222
2936899466,7cyclictest33-21ksoftirqd/221:50:272
2936899463,6cyclictest33-21ksoftirqd/223:40:212
2936899456,7cyclictest33-21ksoftirqd/223:20:242
2936899455,7cyclictest33-21ksoftirqd/223:35:182
2936899455,5cyclictest33-21ksoftirqd/222:00:152
2936899454,8cyclictest33-21ksoftirqd/220:50:212
2936899453,6cyclictest33-21ksoftirqd/220:25:202
29368994517,6cyclictest33-21ksoftirqd/200:20:152
2936899446,8cyclictest33-21ksoftirqd/220:10:202
2936899446,5cyclictest33-21ksoftirqd/221:25:142
2936899445,7cyclictest33-21ksoftirqd/223:50:232
2936899445,7cyclictest33-21ksoftirqd/222:10:002
2936899445,7cyclictest33-21ksoftirqd/222:10:002
2936899445,5cyclictest33-21ksoftirqd/223:05:232
2936899444,6cyclictest33-21ksoftirqd/220:30:132
2936899444,5cyclictest33-21ksoftirqd/200:30:152
2936899444,5cyclictest33-21ksoftirqd/200:30:152
2936899444,5cyclictest31834-21cstates23:45:142
2936899443,6cyclictest33-21ksoftirqd/221:45:152
2936899442,8cyclictest33-21ksoftirqd/221:35:192
2936899442,6cyclictest33-21ksoftirqd/222:35:252
29368994417,5cyclictest33-21ksoftirqd/219:36:402
2936899436,8cyclictest33-21ksoftirqd/223:10:012
2936899436,7cyclictest33-21ksoftirqd/200:15:252
2936899436,5cyclictest33-21ksoftirqd/220:20:012
2936899435,7cyclictest33-21ksoftirqd/219:50:002
2936899435,5cyclictest33-21ksoftirqd/222:30:202
2936899435,5cyclictest33-21ksoftirqd/221:55:212
2936899435,5cyclictest33-21ksoftirqd/221:20:212
2936899435,5cyclictest33-21ksoftirqd/219:11:002
2936899434,6cyclictest33-21ksoftirqd/221:00:162
2936899434,3cyclictest33-21ksoftirqd/219:25:302
2936899433,9cyclictest33-21ksoftirqd/222:50:232
2936899433,6cyclictest33-21ksoftirqd/219:30:212
2936899433,5cyclictest33-21ksoftirqd/221:30:172
2936899433,5cyclictest33-21ksoftirqd/200:25:172
2936899433,4cyclictest33-21ksoftirqd/222:55:182
2936699430,4cyclictest27261-21proc_pri20:10:260
2936899426,8cyclictest33-21ksoftirqd/222:45:012
2936899426,6cyclictest33-21ksoftirqd/221:40:232
2936899425,7cyclictest33-21ksoftirqd/223:30:242
2936899425,7cyclictest33-21ksoftirqd/221:15:202
2936899425,5cyclictest33-21ksoftirqd/220:20:302
2936899425,5cyclictest33-21ksoftirqd/200:35:192
2936899425,5cyclictest33-21ksoftirqd/200:00:462
2936899425,5cyclictest33-21ksoftirqd/200:00:462
2936899425,5cyclictest32193-21fschecks_count19:15:182
2936899425,3cyclictest33-21ksoftirqd/222:25:212
2936899424,5cyclictest33-21ksoftirqd/219:40:262
2936899423,8cyclictest33-21ksoftirqd/200:10:172
2936899423,4cyclictest0-21swapper/223:15:192
29368994216,4cyclictest33-21ksoftirqd/220:00:152
2936899420,6cyclictest33-21ksoftirqd/223:00:162
2936899415,7cyclictest33-21ksoftirqd/222:20:162
2936899415,5cyclictest33-21ksoftirqd/222:15:152
2936899414,7cyclictest33-21ksoftirqd/220:35:202
2936899414,6cyclictest33-21ksoftirqd/222:10:152
2936699410,2cyclictest19554-21dpkg23:20:010
2936899402,7cyclictest33-21ksoftirqd/219:20:152
2936899402,7cyclictest33-21ksoftirqd/219:20:152
2936899390,11cyclictest33-21ksoftirqd/219:55:152
136591390,0ptp4l401ktimersoftd/323:00:213
2936799380,25cyclictest0-21swapper/119:10:161
136591380,0ptp4l401ktimersoftd/322:05:303
136591380,0ptp4l401ktimersoftd/322:05:303
29367993615,18cyclictest21169-21apache219:17:061
2936799360,4cyclictest5823-21swap19:25:291
2936699360,3cyclictest7201-21unixbench_multi20:35:290
29367993516,17cyclictest11921-21tr19:40:141
29367993512,20cyclictest7362-21cron22:55:011
2936799350,4cyclictest6239-21sed20:35:221
51698340,6rtkit-daemon0-21swapper/300:36:283
29367993428,4cyclictest25-21ksoftirqd/123:29:291
2936799340,3cyclictest16569-21grep20:55:291
29366993411,19cyclictest25882-21grep22:25:150
51698330,6rtkit-daemon0-21swapper/320:05:023
29369993327,4cyclictest41-21ksoftirqd/319:46:023
29369993327,3cyclictest41-21ksoftirqd/320:00:013
2936799339,21cyclictest14784-21cron20:54:591
2936799339,20cyclictest28212-21sendmail_mailqu23:35:251
29367993327,4cyclictest25-21ksoftirqd/121:50:491
29367993327,4cyclictest25-21ksoftirqd/121:15:121
29367993313,18cyclictest24567-21irqstats20:05:201
29367993312,18cyclictest23934-21awk22:20:181
29367993310,20cyclictest16452-21expr22:05:161
29367993310,20cyclictest16452-21expr22:05:151
29367993310,19cyclictest19536-21latency_hist23:20:011
2936799330,4cyclictest21474-21hddtemp_smartct22:15:171
2936799330,3cyclictest9293-21systemctl20:40:301
2936799330,3cyclictest30480-21/usr/sbin/munin22:35:131
2936799330,3cyclictest19767-21uname19:55:221
2936799330,3cyclictest0-21swapper/122:40:171
29366993328,3cyclictest9-21ksoftirqd/022:30:130
29366993327,4cyclictest9-21ksoftirqd/019:10:510
29366993326,5cyclictest9-21ksoftirqd/022:05:550
29366993326,5cyclictest9-21ksoftirqd/022:05:550
29366993326,4cyclictest9-21ksoftirqd/021:26:170
29366993313,17cyclictest30417-21seq20:18:560
2936699330,4cyclictest12946-21cat00:10:250
29369993226,4cyclictest41-21ksoftirqd/320:19:513
29367993226,4cyclictest25-21ksoftirqd/122:03:561
29367993226,4cyclictest25-21ksoftirqd/121:26:201
29367993226,4cyclictest25-21ksoftirqd/119:51:491
29367993226,4cyclictest25-21ksoftirqd/100:23:571
29367993225,4cyclictest25-21ksoftirqd/122:10:571
29367993225,4cyclictest25-21ksoftirqd/121:35:071
2936799320,4cyclictest12454-21memory21:55:221
2936799320,3cyclictest7616-21/usr/sbin/munin22:55:141
2936799320,3cyclictest252792sleep100:39:251
2936799320,3cyclictest19771-21/usr/sbin/munin21:05:211
2936799320,3cyclictest12058-21taskset23:04:091
2936699328,22cyclictest18350-21sshd19:52:080
2936699328,19cyclictest23142-21latency_hist00:35:010
2936699328,19cyclictest23142-21latency_hist00:35:010
29366993227,3cyclictest9-21ksoftirqd/019:56:050
29366993226,4cyclictest9-21ksoftirqd/000:24:520
29366993226,3cyclictest9-21ksoftirqd/022:42:010
29366993213,16cyclictest15455-21grep20:55:190
29366993211,18cyclictest21412-21cut20:00:140
29366993210,19cyclictest15376-21sed00:15:290
2936699320,4cyclictest7647-21cat21:45:210
2936699320,4cyclictest6632-21/usr/sbin/munin22:50:250
2936699320,3cyclictest22842-21irqrtprio23:25:210
2936699320,2cyclictest0-21swapper/021:05:180
136591320,1ptp4l391rcuc/320:45:213
136591320,0ptp4l401ktimersoftd/300:00:293
136591320,0ptp4l401ktimersoftd/300:00:293
29369993125,4cyclictest41-21ksoftirqd/321:57:163
29369993125,4cyclictest41-21ksoftirqd/319:29:203
29369993125,4cyclictest41-21ksoftirqd/319:22:343
29369993125,4cyclictest41-21ksoftirqd/319:22:343
29369993125,4cyclictest41-21ksoftirqd/300:17:493
2936799319,19cyclictest10687-21timerandwakeup19:35:271
2936799318,20cyclictest1401-21irqrtprio20:25:201
2936799315,23cyclictest19552-21grep21:05:001
29367993125,4cyclictest25-21ksoftirqd/123:42:001
29367993125,4cyclictest25-21ksoftirqd/123:14:201
29367993125,4cyclictest25-21ksoftirqd/121:42:411
29367993125,4cyclictest25-21ksoftirqd/100:28:381
29367993125,4cyclictest25-21ksoftirqd/100:16:021
2936799312,25cyclictest24636-21cpuspeed_turbos23:30:151
29367993110,18cyclictest3292-21cpuspeed_turbos20:30:131
29367993110,18cyclictest2943-21ptp4l23:50:251
2936799310,3cyclictest22737-21wc20:00:261
2936799310,3cyclictest10551-21sshd00:05:451
2936799310,2cyclictest0-21swapper/120:20:171
29366993126,3cyclictest9-21ksoftirqd/021:56:110
29366993125,4cyclictest9-21ksoftirqd/020:54:100
29366993125,4cyclictest9-21ksoftirqd/020:34:120
29366993125,4cyclictest9-21ksoftirqd/020:05:030
29366993124,5cyclictest9-21ksoftirqd/023:03:200
29366993124,5cyclictest9-21ksoftirqd/021:36:380
29366993111,17cyclictest22449-21expr21:10:160
29366993110,18cyclictest31303-21hwlatdetect20:20:200
2936699310,7cyclictest8594-21sed20:40:240
2936699310,4cyclictest32705-21cat23:45:210
136591310,0ptp4l401ktimersoftd/322:00:183
136591310,0ptp4l401ktimersoftd/321:49:013
29369993025,3cyclictest41-21ksoftirqd/322:57:333
29369993025,3cyclictest41-21ksoftirqd/321:19:173
29369993025,3cyclictest41-21ksoftirqd/320:50:203
29369993025,2cyclictest41-21ksoftirqd/321:05:223
29369993024,4cyclictest41-21ksoftirqd/322:26:563
29369993024,4cyclictest41-21ksoftirqd/321:52:353
29369993024,4cyclictest41-21ksoftirqd/321:21:133
29369993024,4cyclictest41-21ksoftirqd/300:32:013
29369993024,4cyclictest41-21ksoftirqd/300:32:013
29369993024,3cyclictest41-21ksoftirqd/323:40:213
2936799309,19cyclictest19938-21expr23:20:141
29367993024,4cyclictest25-21ksoftirqd/122:26:301
29367993024,4cyclictest25-21ksoftirqd/121:23:311
29367993023,5cyclictest25-21ksoftirqd/119:45:321
2936799302,24cyclictest0-21swapper/123:05:181
2936799300,3cyclictest7052-21df19:30:171
2936799300,3cyclictest3388-21hddtemp_smartct22:45:171
2936799300,2cyclictest1757-21apt-get19:20:131
2936799300,2cyclictest1757-21apt-get19:20:131
2936699307,21cyclictest8274-21irqrtprio22:55:200
2936699303,24cyclictest21422-21cut22:15:170
29366993025,3cyclictest9-21ksoftirqd/023:44:240
29366993025,3cyclictest9-21ksoftirqd/022:45:570
29366993025,3cyclictest9-21ksoftirqd/019:26:410
29366993025,3cyclictest9-21ksoftirqd/000:02:210
29366993025,3cyclictest9-21ksoftirqd/000:02:210
29366993024,4cyclictest9-21ksoftirqd/019:48:140
2936699302,26cyclictest26617-21cron23:35:000
29366993012,15cyclictest16863-21latency_hist23:15:010
29366993011,17cyclictest23591-21expr22:20:160
2936699300,4cyclictest11573-21dpkg19:40:010
2936699300,3cyclictest8120-21mailstats19:30:300
2936699300,3cyclictest18555-21cpu22:10:130
2936699300,2cyclictest26853-21/usr/sbin/munin23:35:210
2936699300,2cyclictest0-21swapper/019:20:220
2936699300,2cyclictest0-21swapper/019:20:220
136591300,0ptp4l401ktimersoftd/320:20:303
320142291,2sleep3121rcu_preempt23:45:163
29369992924,3cyclictest41-21ksoftirqd/323:10:113
29369992924,3cyclictest41-21ksoftirqd/322:45:003
29369992923,4cyclictest41-21ksoftirqd/322:23:033
29369992923,4cyclictest41-21ksoftirqd/322:16:253
29369992923,4cyclictest41-21ksoftirqd/319:41:193
2936799298,19cyclictest12306-21sed00:10:191
2936799296,20cyclictest6849-21fschecks_count00:00:191
2936799296,20cyclictest6849-21fschecks_count00:00:191
29367992923,4cyclictest25-21ksoftirqd/123:45:371
29367992923,4cyclictest25-21ksoftirqd/122:33:041
29367992923,4cyclictest25-21ksoftirqd/121:32:421
29367992923,4cyclictest25-21ksoftirqd/120:11:461
2936799290,2cyclictest0-21swapper/120:15:211
2936699299,17cyclictest25082-21df_abs21:15:160
2936699298,19cyclictest1960-21df23:50:160
2936699297,19cyclictest949-21switchtime21:30:270
2936699296,20cyclictest63362chrt21:43:400
2936699293,19cyclictest31766-21apt-get19:15:140
29366992924,3cyclictest9-21ksoftirqd/021:20:450
2936699292,21cyclictest17111-21apt-get23:15:120
29366992910,17cyclictest9286-21expr21:50:140
136591290,7ptp4l401ktimersoftd/321:42:353
136591290,0ptp4l401ktimersoftd/323:20:173
136591290,0ptp4l401ktimersoftd/322:12:573
29369992823,3cyclictest41-21ksoftirqd/300:05:463
29369992822,4cyclictest41-21ksoftirqd/323:58:593
29369992822,4cyclictest41-21ksoftirqd/323:30:593
29369992822,4cyclictest41-21ksoftirqd/319:38:223
2936799288,17cyclictest11093-21irqrtprio20:45:201
2936799285,20cyclictest22330-21threads00:30:241
2936799285,20cyclictest22330-21threads00:30:241
2936799281,22cyclictest6693-21apt-get21:45:151
2936699286,20cyclictest314242chrt22:35:200
29366992822,4cyclictest9-21ksoftirqd/020:27:470
29366992822,3cyclictest9-21ksoftirqd/023:59:450
2936699282,20cyclictest12385-21apt-get23:05:180
2936699282,19cyclictest13817-21apt-get22:00:140
2936699281,4cyclictest25326-21cron00:40:000
2936699280,3cyclictest10189-21python00:05:250
2936699280,2cyclictest1307-21runrttasks21:01:380
1366912821,3phc2sys41-21ksoftirqd/321:25:193
136591280,0ptp4l401ktimersoftd/322:39:173
136591280,0ptp4l401ktimersoftd/320:14:113
29369992723,3cyclictest41-21ksoftirqd/319:30:213
29369992722,3cyclictest41-21ksoftirqd/321:36:503
29369992722,3cyclictest41-21ksoftirqd/320:26:573
29369992721,4cyclictest41-21ksoftirqd/321:14:473
29369992721,4cyclictest41-21ksoftirqd/320:43:543
2936799271,20cyclictest0-21swapper/123:55:141
2936699279,16cyclictest11696-21grep20:45:260
136591270,1ptp4l401ktimersoftd/322:33:113
136591270,0ptp4l401ktimersoftd/319:19:043
29369992621,3cyclictest41-21ksoftirqd/323:07:033
29369992621,3cyclictest41-21ksoftirqd/321:31:223
29369992621,3cyclictest41-21ksoftirqd/320:36:323
29369992621,3cyclictest41-21ksoftirqd/319:50:043
136591260,1ptp4l401ktimersoftd/322:45:273
136591260,1ptp4l401ktimersoftd/320:55:143
136591260,1ptp4l401ktimersoftd/300:10:123
29369992520,3cyclictest41-21ksoftirqd/320:30:563
136591250,0ptp4l401ktimersoftd/323:35:143
136591250,0ptp4l401ktimersoftd/320:00:123
136591240,1ptp4l401ktimersoftd/323:15:253
136591240,0ptp4l401ktimersoftd/323:50:143
136591240,0ptp4l401ktimersoftd/300:25:143
136591230,0ptp4l401ktimersoftd/322:50:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional