You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 00:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Sun Dec 28, 2025 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
225162241226,11sleep10-21swapper/107:05:421
226202198181,12sleep30-21swapper/307:07:043
226022190173,12sleep00-21swapper/007:06:490
22686211386,22sleep20-21swapper/207:07:562
173982450,0sleep00-21swapper/009:05:150
321902110,0sleep232193-21cpuspeed11:45:112
21471290,0sleep20-21swapper/211:25:002
230549964,2cyclictest25604-21kworker/3:210:44:313
230549955,0cyclictest26447-21kworker/3:107:11:473
230549955,0cyclictest25604-21kworker/3:212:36:273
230549955,0cyclictest25604-21kworker/3:212:25:183
230549955,0cyclictest25604-21kworker/3:212:14:053
230549955,0cyclictest25604-21kworker/3:212:02:543
230549955,0cyclictest25604-21kworker/3:211:40:303
230549955,0cyclictest25604-21kworker/3:211:29:173
230549955,0cyclictest25604-21kworker/3:211:18:053
230549955,0cyclictest25604-21kworker/3:209:48:313
230549955,0cyclictest25604-21kworker/3:209:26:103
230549955,0cyclictest25604-21kworker/3:209:14:583
230549955,0cyclictest25604-21kworker/3:209:03:463
230549955,0cyclictest25604-21kworker/3:207:22:593
230549950,5cyclictest0-21swapper/311:45:173
230549950,4cyclictest22039-21diskstats10:20:153
230549950,4cyclictest16775-21apt-get09:05:023
230479953,1cyclictest16843-21dump-pmu-power12:20:002
230479950,5cyclictest25491-21irqstats12:35:182
230479950,5cyclictest0-21swapper/211:00:152
230479950,5cyclictest0-21swapper/210:22:342
230479950,4cyclictest4679-21apt-get11:55:012
230479950,4cyclictest21429-21apt-get11:25:022
230369950,5cyclictest0-21swapper/009:20:180
230369950,0cyclictest0-21swapper/011:20:130
230549944,0cyclictest25604-21kworker/3:210:55:433
230549944,0cyclictest25604-21kworker/3:209:59:443
230549944,0cyclictest25604-21kworker/3:208:07:453
230549944,0cyclictest25604-21kworker/3:207:45:223
230549944,0cyclictest0-21swapper/311:00:023
230549943,1cyclictest7362-21perl08:45:133
230549943,1cyclictest25109-21wc10:25:143
230549940,4cyclictest16110-21python12:15:213
230549940,4cyclictest0-21swapper/312:20:153
230549940,3cyclictest6029-21grep08:40:193
230549940,3cyclictest522-21ntp_states08:30:203
230549940,3cyclictest284032sleep310:31:293
230549940,3cyclictest2693-21sed07:30:173
230549940,3cyclictest24229-21sed08:15:013
230549940,3cyclictest11581-21grep08:55:003
230549940,3cyclictest0-21swapper/310:35:143
230549940,1cyclictest28989-21latency_hist08:25:003
230479944,0cyclictest0-21swapper/210:35:132
230479944,0cyclictest0-21swapper/208:30:042
230479944,0cyclictest0-21swapper/207:25:112
230479943,1cyclictest2946-21ntp_kernel_err09:40:182
230479940,4cyclictest27272-21df08:20:152
230479940,4cyclictest19558-21apt-get12:25:142
230479940,4cyclictest10284-21ntp_states07:45:172
230479940,4cyclictest0-21swapper/211:50:112
230479940,4cyclictest0-21swapper/211:42:532
230479940,4cyclictest0-21swapper/209:07:122
230479940,4cyclictest0-21swapper/208:35:362
230479940,1cyclictest8327-21cat09:50:202
230479940,1cyclictest26232-21munin-run10:30:012
230479940,0cyclictest0-21swapper/212:33:322
230409941,2cyclictest0-21swapper/110:55:121
230409941,2cyclictest0-21swapper/110:35:141
230409941,2cyclictest0-21swapper/110:05:121
230409941,2cyclictest0-21swapper/107:40:121
230409941,1cyclictest0-21swapper/110:20:121
230409941,1cyclictest0-21swapper/107:30:111
230369943,1cyclictest4177-21/usr/sbin/munin09:45:180
230369943,1cyclictest27633-21grep10:30:160
230369942,1cyclictest4134-21apt10:50:130
230369941,3cyclictest7543-21cat10:55:160
230369941,2cyclictest0-21swapper/008:05:120
230369940,4cyclictest517-21ntp_states08:30:200
230369940,4cyclictest28082-21ntp_states11:35:170
230369940,4cyclictest0-21swapper/009:10:120
230369940,4cyclictest0-21swapper/008:25:180
230369940,3cyclictest32346-21perl09:35:160
230369940,3cyclictest31022-21sensors_temp11:40:200
230369940,3cyclictest23940-21latency_hist10:25:010
230369940,3cyclictest21731-21/usr/sbin/munin09:15:200
230369940,3cyclictest17602-21df12:20:140
230369940,3cyclictest11680-21/usr/sbin/munin07:50:130
230369940,1cyclictest6649-21/usr/sbin/munin09:50:180
230369940,1cyclictest23981-21/usr/sbin/munin10:25:170
230369940,1cyclictest2241-21diskstats09:40:140
230369940,1cyclictest19554-21/usr/sbin/munin12:25:150
230369940,1cyclictest1728-21/usr/sbin/munin08:35:200
230369940,1cyclictest1577-21/usr/sbin/munin07:30:130
230369940,1cyclictest0-21swapper/007:45:130
230369940,0cyclictest7392-21perl08:45:130
230369940,0cyclictest5324-21ntp_states07:35:200
230369940,0cyclictest25612-21ntp_states11:30:170
230549932,1cyclictest774-21snmpd10:10:083
230549932,1cyclictest774-21snmpd07:25:053
230549932,1cyclictest15309-21grep07:55:173
230549931,2cyclictest25604-21kworker/3:208:55:013
230549931,1cyclictest774-21snmpd09:40:033
230549931,1cyclictest18770-21cron10:15:013
230549931,1cyclictest14134-21aten2_r5power_e10:05:123
230549930,3cyclictest7388-21sed07:40:133
230549930,3cyclictest7163-21expr09:50:133
230549930,3cyclictest31649-21apt-get09:35:133
230549930,3cyclictest29897-21perl09:30:153
230549930,3cyclictest27921-21sed11:35:153
230549930,3cyclictest2741-21perl11:50:123
230549930,3cyclictest25798-21hddtemp_smartct07:15:003
230549930,3cyclictest24998-21grep09:20:153
230549930,3cyclictest14218-21dpkg11:10:003
230549930,3cyclictest12315-21cut10:00:173
230549930,3cyclictest10178-21df12:05:143
230549930,3cyclictest0-21swapper/312:30:193
230549930,3cyclictest0-21swapper/311:55:113
230549930,3cyclictest0-21swapper/311:30:123
230549930,3cyclictest0-21swapper/311:20:123
230549930,3cyclictest0-21swapper/311:05:133
230549930,3cyclictest0-21swapper/310:50:013
230549930,3cyclictest0-21swapper/310:45:123
230549930,3cyclictest0-21swapper/308:35:133
230549930,3cyclictest0-21swapper/308:25:003
230549930,3cyclictest0-21swapper/308:15:133
230549930,3cyclictest0-21swapper/308:00:133
230549930,3cyclictest0-21swapper/307:51:303
230549930,3cyclictest0-21swapper/307:35:133
230479933,0cyclictest31-21ksoftirqd/210:55:112
230479933,0cyclictest0-21swapper/208:25:112
230479932,1cyclictest9815-21tune2fs09:55:142
230479932,1cyclictest8206-21expr07:40:192
230479932,1cyclictest774-21snmpd10:45:062
230479932,1cyclictest774-21snmpd10:32:232
230479932,1cyclictest774-21snmpd10:00:032
230479932,1cyclictest774-21snmpd08:46:222
230479932,1cyclictest31896-21expr10:40:132
230479932,1cyclictest30958-21grep09:30:212
230479932,1cyclictest27617-21cat09:25:162
230479932,1cyclictest24755-21expr11:30:142
230479932,1cyclictest23154-21aten2_r5power_c07:10:132
230479932,1cyclictest14439-21cpuspeed_turbos09:00:142
230479931,2cyclictest24675-21expr08:15:142
230479931,1cyclictest24847-21cut09:20:162
230479931,1cyclictest17496-21if_eth110:10:182
230479931,1cyclictest12695-21fschecks_count12:10:142
230479930,3cyclictest7717-21diskstats12:00:152
230479930,3cyclictest5313-21memory08:40:162
230479930,3cyclictest4821-21grep09:45:142
230479930,3cyclictest31730-21wget09:35:132
230479930,3cyclictest25628-21cstates07:15:002
230479930,3cyclictest20102-21hddtemp_smartct09:10:142
230479930,3cyclictest19832-21sed10:15:152
230479930,3cyclictest19462-21expr08:05:132
230479930,3cyclictest17605-21df12:20:132
230479930,3cyclictest16916-21grep11:15:122
230479930,3cyclictest15710-21grep11:10:192
230479930,3cyclictest12153-21df_inode08:55:112
230479930,3cyclictest12094-21diskstats11:05:162
230479930,3cyclictest10140-21expr12:05:142
230479930,3cyclictest0-21swapper/211:35:112
230479930,3cyclictest0-21swapper/210:50:012
230479930,3cyclictest0-21swapper/210:05:132
230479930,3cyclictest0-21swapper/209:15:112
230479930,3cyclictest0-21swapper/208:50:102
230479930,3cyclictest0-21swapper/208:00:162
230479930,3cyclictest0-21swapper/207:55:152
230479930,3cyclictest0-21swapper/207:35:152
230479930,3cyclictest0-21swapper/207:30:012
230479930,3cyclictest0-21swapper/207:20:002
230479930,2cyclictest0-21swapper/208:10:012
230479930,0cyclictest0-21swapper/207:50:112
230409932,1cyclictest0-21swapper/111:20:051
230409931,2cyclictest0-21swapper/109:15:111
230409931,1cyclictest0-21swapper/112:35:151
230409931,1cyclictest0-21swapper/112:15:141
230409931,1cyclictest0-21swapper/112:00:141
230409931,1cyclictest0-21swapper/111:55:121
230409931,1cyclictest0-21swapper/111:40:111
230409931,1cyclictest0-21swapper/111:35:121
230409931,1cyclictest0-21swapper/111:10:131
230409931,1cyclictest0-21swapper/110:45:131
230409931,1cyclictest0-21swapper/110:40:121
230409931,1cyclictest0-21swapper/110:30:121
230409931,1cyclictest0-21swapper/110:15:131
230409931,1cyclictest0-21swapper/110:00:141
230409931,1cyclictest0-21swapper/109:45:101
230409931,1cyclictest0-21swapper/109:25:131
230409931,1cyclictest0-21swapper/109:05:141
230409931,1cyclictest0-21swapper/109:00:131
230409931,1cyclictest0-21swapper/108:50:111
230409931,1cyclictest0-21swapper/108:35:121
230409931,1cyclictest0-21swapper/108:30:121
230409931,1cyclictest0-21swapper/108:00:141
230409931,1cyclictest0-21swapper/107:45:131
230409931,1cyclictest0-21swapper/107:35:131
230409931,1cyclictest0-21swapper/107:15:131
230409930,3cyclictest0-21swapper/108:44:141
230409930,3cyclictest0-21swapper/108:09:541
230409930,2cyclictest24348-21chrt11:30:131
230409930,2cyclictest0-21swapper/112:30:131
230409930,2cyclictest0-21swapper/108:10:101
230369933,0cyclictest0-21swapper/009:00:130
230369932,1cyclictest774-21snmpd08:50:080
230369932,1cyclictest774-21snmpd08:15:070
230369932,1cyclictest16748-21aten2_r5power_v08:00:140
230369931,1cyclictest774-21snmpd07:25:110
230369931,1cyclictest26385-21cut07:15:120
230369931,1cyclictest2018-21apt-get11:50:100
230369931,1cyclictest15293-21fschecks_time12:15:160
230369931,1cyclictest0-21swapper/012:30:120
230369931,1cyclictest0-21swapper/011:45:090
230369931,1cyclictest0-21swapper/010:05:120
230369931,1cyclictest0-21swapper/009:25:120
230369930,3cyclictest9852-21grep11:00:170
230369930,3cyclictest9518-21expr09:55:140
230369930,3cyclictest4260-21apt-get11:55:010
230369930,3cyclictest4221-21/usr/sbin/munin08:40:110
230369930,3cyclictest29253-21tr10:35:150
230369930,3cyclictest2202-21perl10:45:150
230369930,3cyclictest0-21swapper/012:35:190
230369930,3cyclictest0-21swapper/012:05:120
230369930,3cyclictest0-21swapper/011:15:120
230369930,3cyclictest0-21swapper/011:10:120
230369930,3cyclictest0-21swapper/011:05:130
230369930,3cyclictest0-21swapper/010:40:130
230369930,3cyclictest0-21swapper/010:00:150
230369930,3cyclictest0-21swapper/009:30:120
230369930,3cyclictest0-21swapper/008:55:010
230369930,3cyclictest0-21swapper/008:10:110
230369930,3cyclictest0-21swapper/007:20:020
230369930,3cyclictest0-21swapper/007:10:110
230369930,2cyclictest7264-21cut12:00:150
230369930,2cyclictest0-21swapper/011:25:010
230369930,1cyclictest26968-21cut08:20:130
230369930,1cyclictest0-21swapper/007:55:120
230369930,0cyclictest7240-21perl07:40:130
230369930,0cyclictest13388-21ntp_states12:10:170
230369930,0cyclictest0-21swapper/010:15:150
230369930,0cyclictest0-21swapper/010:10:210
230549920,2cyclictest0-21swapper/309:15:013
230409922,0cyclictest0-21swapper/107:58:151
230409921,1cyclictest242292chrt09:20:091
230409921,1cyclictest0-21swapper/112:10:111
230409921,1cyclictest0-21swapper/112:05:111
230409921,1cyclictest0-21swapper/111:50:111
230409921,1cyclictest0-21swapper/111:45:091
230409921,1cyclictest0-21swapper/110:50:141
230409921,1cyclictest0-21swapper/110:10:141
230409921,1cyclictest0-21swapper/109:55:121
230409921,1cyclictest0-21swapper/108:45:111
230409921,1cyclictest0-21swapper/107:50:111
230409921,0cyclictest0-21swapper/111:25:011
230409921,0cyclictest0-21swapper/109:40:121
230409920,1cyclictest29210-21chrt08:25:051
230409920,1cyclictest26013-21chrt10:27:311
230409920,1cyclictest25360-21chrt07:14:581
230409920,1cyclictest21521-21chrt12:26:441
230409920,1cyclictest18158-21chrt11:15:191
230409920,1cyclictest0-21swapper/112:20:111
230409920,1cyclictest0-21swapper/111:05:131
230409920,1cyclictest0-21swapper/111:00:131
230409920,1cyclictest0-21swapper/109:50:131
230409920,1cyclictest0-21swapper/109:35:121
230409920,1cyclictest0-21swapper/109:30:131
230409920,1cyclictest0-21swapper/109:10:121
230409920,1cyclictest0-21swapper/108:55:101
230409920,1cyclictest0-21swapper/108:20:121
230409920,1cyclictest0-21swapper/108:15:121
230409920,1cyclictest0-21swapper/107:25:101
230409920,1cyclictest0-21swapper/107:20:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional