You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 16:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot2.osadl.org (updated Mon Dec 15, 2025 12:45:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8062220187,22sleep20-21swapper/207:08:522
8592217182,24sleep30-21swapper/307:09:323
6762213179,23sleep10-21swapper/107:07:141
8092211174,25sleep00-21swapper/007:08:540
1316421300,6sleep3119399cyclictest12:05:013
1192997530,3cyclictest33-21ksoftirqd/211:45:002
1191997411,19cyclictest13748-21fschecks_time12:05:381
119099711,24cyclictest9069-21cut09:40:390
119099711,23cyclictest2861-21timerandwakeup07:10:420
119099710,25cyclictest21247-21sed09:00:010
1190997012,28cyclictest11802-21sed07:30:400
119099700,25cyclictest9373-21irqrtprio11:55:410
119099690,24cyclictest6171-21expr09:35:360
119099690,24cyclictest5185-21cat07:15:460
119099690,24cyclictest20807-21tune2fs07:50:360
119099690,24cyclictest16903-21grep07:40:440
119099682,24cyclictest30721-21latency_hist09:20:010
119099681,25cyclictest3326-21cat11:45:000
119099680,24cyclictest25084-21latency_hist12:30:000
119099680,24cyclictest15395-21latency_hist09:55:010
119099680,24cyclictest10145-21fschecks_count10:50:380
119199679,16cyclictest20036-21latency_hist07:50:001
119099670,25cyclictest4668-21missed_timers09:30:410
119099670,24cyclictest7359-21expr11:50:470
119099670,23cyclictest12431-21irqrtprio08:40:400
119099661,23cyclictest31684-21awk10:25:480
119099660,25cyclictest6799-21cat08:30:000
119099660,24cyclictest17371-21users12:10:500
119099660,24cyclictest15460-21sed07:39:590
119099651,24cyclictest11693-21sed08:40:000
119099650,26cyclictest4908-21date10:40:010
119099650,26cyclictest4908-21date10:40:010
119099650,25cyclictest32384-21ps11:35:460
119099650,24cyclictest2796-21sed09:25:450
119099641,31cyclictest5504-21perl10:40:370
119099640,26cyclictest12200-21sed09:45:450
119099640,25cyclictest31441-21expr08:10:450
119099640,23cyclictest27020-21irqrtprio09:10:390
119099631,23cyclictest3785-21expr11:45:350
119099621,23cyclictest1576-21users08:15:470
119099620,25cyclictest17106-21/usr/sbin/munin08:50:350
119099620,24cyclictest14977-21grep11:00:370
1192996128,11cyclictest33-21ksoftirqd/211:50:002
1191996111,12cyclictest23748-21latency_hist11:20:001
1191996110,11cyclictest2447-21dpkg10:35:011
1191996110,11cyclictest2447-21dpkg10:35:011
119099611,24cyclictest2521-21gnome-shell10:00:460
1191996011,12cyclictest9638-21sed10:50:011
139291590,1ptp4l401ktimersoftd/310:21:413
1192995922,8cyclictest33-21ksoftirqd/208:45:002
1192995920,4cyclictest33-21ksoftirqd/208:30:362
119199597,18cyclictest0-21swapper/111:55:011
1191995813,12cyclictest22007-21/usr/sbin/munin10:05:461
1191995810,16cyclictest29797-21meminfo11:30:411
1191995810,12cyclictest29077-21sed12:35:481
119099580,24cyclictest32573-21seq10:30:360
119099580,24cyclictest19737-21/usr/sbin/munin11:10:380
119099580,23cyclictest25425-21systemctl11:20:460
1192995719,4cyclictest33-21ksoftirqd/209:25:392
119099570,25cyclictest10196-21users07:25:480
69582560,2sleep16963-21missed_timers11:50:431
136662560,2sleep10-21swapper/109:50:361
1192995619,9cyclictest33-21ksoftirqd/209:10:392
1192995617,8cyclictest33-21ksoftirqd/209:55:002
119199569,10cyclictest17190-21df_abs11:05:351
1191995610,15cyclictest23383-21iostat_ios07:55:391
1191995610,15cyclictest23383-21iostat_ios07:55:391
1191995610,11cyclictest9870-21/usr/sbin/munin07:25:461
119099560,22cyclictest21105-21ps10:05:390
159612550,5sleep315963-21timerwakeupswit08:45:483
159612550,5sleep315963-21timerwakeupswit08:45:483
1192995518,9cyclictest33-21ksoftirqd/210:40:492
1192995518,9cyclictest33-21ksoftirqd/208:15:332
119199559,17cyclictest16137-21hddtemp_smartct12:10:381
1191995511,14cyclictest31526-21diskstats11:35:381
1191995510,12cyclictest14544-21expr08:45:351
1191995510,12cyclictest14544-21expr08:45:351
1192995420,10cyclictest33-21ksoftirqd/207:55:002
1192995416,8cyclictest33-21ksoftirqd/209:55:352
1192995416,11cyclictest33-21ksoftirqd/210:20:452
1192995415,8cyclictest33-21ksoftirqd/210:05:432
119199549,14cyclictest20773-21diskstats12:20:361
1191995411,12cyclictest16476-21sed07:40:391
1191995410,12cyclictest14021-21sed07:35:401
119099541,24cyclictest20513-21grep12:20:340
119099540,27cyclictest25327-21/usr/sbin/munin10:15:340
119099540,25cyclictest28087-21sort10:20:370
1192995320,4cyclictest33-21ksoftirqd/207:50:002
119199539,11cyclictest29858-21dpkg08:10:311
119099530,24cyclictest13725-21cat10:55:490
322482520,2sleep00-21swapper/009:20:420
1192995224,9cyclictest33-21ksoftirqd/207:55:472
1192995224,9cyclictest33-21ksoftirqd/207:55:472
1192995224,3cyclictest33-21ksoftirqd/208:50:472
1192995222,9cyclictest33-21ksoftirqd/208:25:472
1192995221,9cyclictest33-21ksoftirqd/211:20:492
1192995219,9cyclictest33-21ksoftirqd/209:40:332
119199529,12cyclictest7527-21sed09:35:471
1192995118,10cyclictest33-21ksoftirqd/208:50:002
1192995118,10cyclictest33-21ksoftirqd/208:50:002
1192995114,10cyclictest33-21ksoftirqd/211:50:392
1192995113,10cyclictest33-21ksoftirqd/209:30:372
1192995112,6cyclictest33-21ksoftirqd/210:55:362
119199519,14cyclictest10124-21diskstats10:50:391
1191995111,10cyclictest1873-21sed11:40:411
139291500,0ptp4l401ktimersoftd/308:26:433
119299505,3cyclictest121rcu_preempt09:15:332
1192995018,9cyclictest33-21ksoftirqd/208:35:382
1192995016,10cyclictest33-21ksoftirqd/207:30:332
1192995014,9cyclictest33-21ksoftirqd/212:10:452
1192995013,5cyclictest33-21ksoftirqd/210:10:402
1192994919,7cyclictest33-21ksoftirqd/207:40:442
1192994918,9cyclictest33-21ksoftirqd/208:05:452
1192994917,10cyclictest33-21ksoftirqd/207:15:462
1192994912,9cyclictest33-21ksoftirqd/207:20:332
119299489,6cyclictest33-21ksoftirqd/212:30:382
119299485,3cyclictest121rcu_preempt09:35:362
1192994820,8cyclictest33-21ksoftirqd/210:50:362
1192994810,4cyclictest33-21ksoftirqd/211:00:392
119199488,15cyclictest29007-21df_inode09:15:361
1191994810,9cyclictest9474-21cat09:40:411
119099480,25cyclictest23943-21phc2sys-jitter07:55:430
119099480,25cyclictest23943-21phc2sys-jitter07:55:430
119099480,21cyclictest0-21swapper/012:00:400
119299479,5cyclictest33-21ksoftirqd/210:30:492
1192994726,10cyclictest33-21ksoftirqd/210:20:002
1192994719,7cyclictest33-21ksoftirqd/212:35:492
1192994712,9cyclictest33-21ksoftirqd/211:25:442
119199479,7cyclictest4065-21hddtemp_smartct11:45:391
119199479,7cyclictest19460-21df_inode08:55:351
139291460,0ptp4l401ktimersoftd/311:40:063
119299463,2cyclictest121rcu_preempt12:20:472
1192994618,7cyclictest33-21ksoftirqd/210:35:412
1192994618,7cyclictest33-21ksoftirqd/210:35:402
1192994616,3cyclictest33-21ksoftirqd/207:10:342
1192994612,4cyclictest33-21ksoftirqd/209:00:452
1192994610,6cyclictest33-21ksoftirqd/211:35:372
1191994611,10cyclictest26379-21grep09:10:351
1191994610,11cyclictest12913-21latency_hist07:35:001
119299459,6cyclictest33-21ksoftirqd/210:28:312
1192994532,3cyclictest33-21ksoftirqd/212:00:352
1192994515,7cyclictest33-21ksoftirqd/209:45:472
1192994515,10cyclictest33-21ksoftirqd/212:15:372
1192994510,4cyclictest33-21ksoftirqd/211:55:012
139291440,1ptp4l31034-21open_inodes08:10:413
139291440,0ptp4l401ktimersoftd/307:55:283
139291440,0ptp4l401ktimersoftd/307:55:283
1193994438,3cyclictest41-21ksoftirqd/311:30:003
1192994414,6cyclictest33-21ksoftirqd/208:00:402
1192994412,4cyclictest33-21ksoftirqd/212:25:402
1191994411,10cyclictest1219-21grep08:15:441
119099440,20cyclictest0-21swapper/011:30:450
139291430,0ptp4l401ktimersoftd/307:40:413
1192994320,7cyclictest33-21ksoftirqd/208:55:352
1192994314,8cyclictest33-21ksoftirqd/210:45:352
1192994314,7cyclictest33-21ksoftirqd/211:10:352
1192994314,7cyclictest33-21ksoftirqd/207:35:482
1191994311,12cyclictest20039-21irqrtprio11:10:391
1193994238,3cyclictest41-21ksoftirqd/310:00:013
1193994237,3cyclictest41-21ksoftirqd/309:20:013
119299428,5cyclictest33-21ksoftirqd/211:05:422
1192994236,3cyclictest4373-21sh08:25:002
1192994231,3cyclictest33-21ksoftirqd/210:00:352
1192994217,6cyclictest33-21ksoftirqd/211:32:302
119299421,3cyclictest121rcu_preempt08:10:332
1191994210,12cyclictest25230-21/usr/sbin/munin10:15:461
119099420,27cyclictest26829-21unixbench_multi12:30:490
1193994137,2cyclictest41-21ksoftirqd/310:55:013
1193994136,3cyclictest41-21ksoftirqd/308:40:003
119299413,9cyclictest33-21ksoftirqd/211:15:422
1192994136,3cyclictest33-21ksoftirqd/209:10:002
119299410,2cyclictest121rcu_preempt09:25:002
1191994110,11cyclictest5176-21timerandwakeup09:30:461
1193994035,3cyclictest41-21ksoftirqd/312:20:003
1193994035,3cyclictest41-21ksoftirqd/310:29:593
119399402,3cyclictest121rcu_preempt09:35:463
119399401,6cyclictest0-21swapper/308:20:403
119399400,2cyclictest121rcu_preempt10:15:003
1192994035,3cyclictest33-21ksoftirqd/207:30:012
1191994010,11cyclictest11600-21irqrtprio12:00:401
119099400,29cyclictest26386-21/usr/sbin/munin11:25:440
1193993935,2cyclictest41-21ksoftirqd/309:25:013
1193993934,3cyclictest41-21ksoftirqd/312:25:003
1193993934,3cyclictest41-21ksoftirqd/310:34:593
1193993934,2cyclictest41-21ksoftirqd/310:05:013
119399391,3cyclictest121rcu_preempt10:40:003
119399391,3cyclictest121rcu_preempt10:40:003
1191993911,18cyclictest6943-21cut08:30:001
1191993910,11cyclictest31674-21irqrtprio09:20:391
119099390,25cyclictest20254-21latency_hist12:20:000
139291380,1ptp4l401ktimersoftd/310:55:513
1193993833,3cyclictest41-21ksoftirqd/311:40:003
1193993833,3cyclictest41-21ksoftirqd/311:10:013
1193993833,3cyclictest41-21ksoftirqd/309:05:413
1193993833,2cyclictest41-21ksoftirqd/310:10:013
1193993832,4cyclictest41-21ksoftirqd/312:28:293
1193993832,4cyclictest41-21ksoftirqd/310:40:343
1193993832,3cyclictest41-21ksoftirqd/308:30:393
119299381,2cyclictest33-21ksoftirqd/212:05:392
1191993810,18cyclictest31180-21open_inodes10:25:431
119099380,20cyclictest0-21swapper/008:00:420
1395913730,4phc2sys41-21ksoftirqd/308:08:453
1193993732,3cyclictest41-21ksoftirqd/311:50:003
1193993732,3cyclictest41-21ksoftirqd/309:29:593
1193993732,3cyclictest41-21ksoftirqd/308:45:013
1193993732,3cyclictest41-21ksoftirqd/307:27:563
1193993731,4cyclictest41-21ksoftirqd/308:03:483
1193993731,4cyclictest41-21ksoftirqd/307:45:373
1193993731,3cyclictest41-21ksoftirqd/308:50:393
1193993731,3cyclictest41-21ksoftirqd/308:15:403
119399372,5cyclictest1644-21df_abs07:10:303
119399371,4cyclictest41-21ksoftirqd/311:55:383
119199376,10cyclictest0-21swapper/108:20:471
1191993732,3cyclictest2077-21cron10:34:591
119399364,3cyclictest41-21ksoftirqd/309:00:003
119399363,3cyclictest41-21ksoftirqd/311:20:013
1193993632,3cyclictest41-21ksoftirqd/309:40:393
1193993631,3cyclictest41-21ksoftirqd/311:20:413
1193993631,3cyclictest41-21ksoftirqd/309:55:013
1193993630,4cyclictest41-21ksoftirqd/312:32:573
1193993630,4cyclictest41-21ksoftirqd/307:30:143
1193993622,3cyclictest41-21ksoftirqd/310:50:003
119199369,15cyclictest9381-21apt-get08:35:321
119199367,18cyclictest0-21swapper/107:10:351
1191993631,3cyclictest25-21ksoftirqd/110:10:331
1191993610,8cyclictest24685-21iostat_ios11:20:411
119099361,22cyclictest0-21swapper/008:30:380
119399355,3cyclictest41-21ksoftirqd/307:50:453
1193993531,2cyclictest41-21ksoftirqd/307:15:403
1193993530,4cyclictest41-21ksoftirqd/312:10:483
1193993530,3cyclictest41-21ksoftirqd/312:40:003
1193993530,3cyclictest41-21ksoftirqd/312:05:293
1193993530,3cyclictest41-21ksoftirqd/311:30:233
1193993530,3cyclictest41-21ksoftirqd/311:10:493
119399352,6cyclictest121rcu_preempt09:13:103
119399352,4cyclictest4713-21sed09:30:413
119399350,5cyclictest15708-21phc2sys-jitter11:00:443
119199359,15cyclictest18205-21ls08:50:451
119199358,15cyclictest24909-21ls09:05:421
119199357,18cyclictest0-21swapper/107:20:311
119199357,17cyclictest0-21swapper/109:45:321
119199356,17cyclictest0-21swapper/112:30:341
1191993529,4cyclictest25-21ksoftirqd/108:33:581
1191993511,7cyclictest13184-21python08:40:461
1191993510,7cyclictest29052-21python10:20:441
1191993510,7cyclictest22549-21meminfo09:00:421
1190993530,3cyclictest9-21ksoftirqd/010:10:280
1190993529,4cyclictest9-21ksoftirqd/012:35:140
119099351,22cyclictest0-21swapper/011:09:290
119099350,22cyclictest6866-21irqrtprio07:20:380
119099350,19cyclictest0-21swapper/008:05:400
119399344,3cyclictest41-21ksoftirqd/307:20:463
1193993430,2cyclictest41-21ksoftirqd/309:45:333
1193993429,3cyclictest41-21ksoftirqd/311:50:413
1193993429,3cyclictest41-21ksoftirqd/310:15:363
119399340,3cyclictest41-21ksoftirqd/309:00:483
119199348,16cyclictest0-21swapper/107:15:381
119199348,15cyclictest15699-21apt-get09:55:331
119199346,17cyclictest0-21swapper/112:15:351
119199346,17cyclictest0-21swapper/109:25:041
119199346,11cyclictest0-21swapper/110:40:381
119199345,18cyclictest0-21swapper/111:25:321
1191993428,4cyclictest25-21ksoftirqd/110:58:081
1191993410,7cyclictest18731-21iostat_ios10:00:401
1191993328,3cyclictest25-21ksoftirqd/111:01:531
1191993327,4cyclictest25-21ksoftirqd/112:25:291
1190993327,4cyclictest9-21ksoftirqd/009:55:030
1190993310,21cyclictest4548-21sed08:25:020
1193993227,3cyclictest41-21ksoftirqd/307:35:483
119199329,7cyclictest21337-21meminfo07:50:401
119199328,8cyclictest27489-21apt-get08:05:331
119199326,16cyclictest0-21swapper/108:00:461
1190993126,3cyclictest9-21ksoftirqd/007:47:280
1190993024,4cyclictest9-21ksoftirqd/010:45:500
1190993010,18cyclictest21731-21cstates09:00:360
119099299,18cyclictest21626-21sshd11:15:290
1190992923,4cyclictest9-21ksoftirqd/012:05:180
119099292,22cyclictest14352-21apt-get08:45:340
119099292,22cyclictest14352-21apt-get08:45:340
119099282,22cyclictest23893-21apt-get09:05:340
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional