You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 18:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Wed Nov 26, 2025 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245032198182,11sleep10-21swapper/107:09:411
243092194178,12sleep00-21swapper/007:07:100
243372187170,12sleep30-21swapper/307:07:333
241822158139,14sleep20-21swapper/207:05:322
6612510,0sleep20-21swapper/212:13:542
150532400,0sleep315050-21awk11:00:013
2470799110,7cyclictest10298-21apt-get10:25:123
212962100,0sleep20-21swapper/209:41:492
247079991,7cyclictest25714-21apt-get11:05:103
247079990,9cyclictest3287-21apt-get11:10:113
247079990,9cyclictest3287-21apt-get11:10:113
247079990,9cyclictest20599-21apt-get08:05:123
247079990,8cyclictest8672-21ssh09:20:013
247079990,8cyclictest8239-21apt-get11:45:123
247079990,8cyclictest8235-21apt-get07:40:113
247079990,8cyclictest23242-21apt-get11:20:123
247079990,8cyclictest21854-21irqstats11:35:153
247079990,8cyclictest19122-21apt-get09:25:153
247079990,8cyclictest10701-21/usr/sbin/munin07:45:163
247079990,8cyclictest10457-21apt-get08:50:113
247079981,6cyclictest844-21apt-get11:25:133
247079980,8cyclictest17809-21apt-get10:45:113
247079980,8cyclictest16375-21perf09:40:003
247079980,8cyclictest12615-21apt-get10:10:133
247079980,7cyclictest7767-21munin-run08:44:593
247079980,7cyclictest583-21apt-get07:25:103
247079980,7cyclictest4941-21sh09:18:013
247079980,7cyclictest4133-21grep09:50:013
247079980,7cyclictest32697-21apt-get10:20:123
247079980,7cyclictest32120-21latency_hist10:20:003
247079980,7cyclictest32082-21turbostat12:29:593
247079980,7cyclictest3027-21apt-get08:35:103
247079980,7cyclictest29903-21users07:15:233
247079980,7cyclictest29275-21interrupts11:55:163
247079980,7cyclictest28691-21hddtemp_smartct10:50:143
247079980,7cyclictest28043-21apt-get08:20:133
247079980,7cyclictest2797-21munin-run08:34:593
247079980,7cyclictest24938-21apt10:00:103
247079980,7cyclictest24822-21/usr/sbin/munin07:10:173
247079980,7cyclictest24266-21sh09:43:423
247079980,7cyclictest24240-21ssh11:52:513
247079980,7cyclictest24232-21seq10:00:003
247079980,7cyclictest225-21systemd-journal09:55:003
247079980,7cyclictest20923-21apt-get09:10:143
247079980,7cyclictest20363-21dump-pmu-power08:05:003
247079980,7cyclictest20345-21apt-get10:30:143
247079980,7cyclictest18133-21apt-get09:05:113
247079980,7cyclictest15750-21apt-get11:00:123
247079980,7cyclictest15600-21apt-get12:05:113
247079980,7cyclictest13263-21apt-get11:15:123
247079980,7cyclictest13172-21apt-get07:50:123
247079980,7cyclictest12847-21ssh12:20:113
247079980,7cyclictest11553-21sed12:35:183
247079980,7cyclictest0-21swapper/311:40:123
247079980,7cyclictest0-21swapper/308:15:123
247079980,5cyclictest3528-21diskstats12:15:153
247079970,7cyclictest9315-21ntpq10:40:193
247079970,7cyclictest61-21kswapd012:30:103
247079970,7cyclictest61-21kswapd010:05:123
247079970,7cyclictest0-21swapper/309:00:123
247079970,7cyclictest0-21swapper/308:25:123
247079970,6cyclictest7991-21apt-get08:45:123
247079970,6cyclictest3070-21apt07:30:123
247079970,6cyclictest30551-21apt07:20:143
247079970,6cyclictest30528-21cpuspeed10:35:133
247079970,6cyclictest29112-21apt09:30:113
247079970,6cyclictest24941-21cron12:10:013
247079970,6cyclictest23065-21apt-get08:10:103
247079970,6cyclictest10789-21ssh11:30:103
247019970,5cyclictest0-21swapper/207:10:132
246989971,3cyclictest30497-21awk09:15:010
246989970,3cyclictest4136-21munin-run09:50:010
247079960,6cyclictest12929-21apt-get08:55:113
247079960,6cyclictest0-21swapper/312:00:113
247079960,6cyclictest0-21swapper/307:55:123
247079960,6cyclictest0-21swapper/307:35:113
247019960,5cyclictest0-21swapper/208:30:122
246999961,3cyclictest0-21swapper/109:25:161
246989962,4cyclictest14168-21perf09:55:000
247019950,5cyclictest0-21swapper/211:47:032
247019950,4cyclictest0-21swapper/210:20:132
247019950,4cyclictest0-21swapper/209:25:162
247019950,0cyclictest0-21swapper/211:22:022
246999951,2cyclictest0-21swapper/110:40:131
246999951,2cyclictest0-21swapper/107:10:141
246999950,5cyclictest0-21swapper/112:32:281
246999950,5cyclictest0-21swapper/111:04:251
246999950,5cyclictest0-21swapper/109:35:151
246989950,5cyclictest0-21swapper/011:43:480
246989950,5cyclictest0-21swapper/011:13:240
246989950,5cyclictest0-21swapper/011:13:230
246989950,5cyclictest0-21swapper/010:09:400
246989950,5cyclictest0-21swapper/009:44:560
246989950,4cyclictest20375-21diskmemload10:40:130
6891240,3sleep07038-21sensors_temp08:40:230
247019944,0cyclictest31-21ksoftirqd/210:00:002
247019944,0cyclictest31-21ksoftirqd/209:15:142
247019944,0cyclictest0-21swapper/209:30:172
247019940,4cyclictest0-21swapper/212:20:282
247019940,4cyclictest0-21swapper/212:15:122
247019940,4cyclictest0-21swapper/212:00:192
247019940,4cyclictest0-21swapper/211:50:392
247019940,4cyclictest0-21swapper/211:35:112
247019940,4cyclictest0-21swapper/211:25:362
247019940,4cyclictest0-21swapper/211:15:302
247019940,4cyclictest0-21swapper/211:10:122
247019940,4cyclictest0-21swapper/211:10:112
247019940,4cyclictest0-21swapper/211:02:582
247019940,4cyclictest0-21swapper/210:45:122
247019940,4cyclictest0-21swapper/210:44:312
247019940,4cyclictest0-21swapper/210:35:122
247019940,4cyclictest0-21swapper/210:32:342
247019940,4cyclictest0-21swapper/210:15:132
247019940,4cyclictest0-21swapper/210:10:132
247019940,4cyclictest0-21swapper/209:53:342
247019940,4cyclictest0-21swapper/209:10:132
247019940,4cyclictest0-21swapper/208:40:132
247019940,3cyclictest5465-21wc08:39:592
247019940,3cyclictest4150-21ssh09:50:012
247019940,3cyclictest28990-21sh10:50:162
247019940,3cyclictest28842-21ssh11:55:152
247019940,3cyclictest27379-21threads07:15:002
247019940,3cyclictest21601-21ssh12:07:512
247019940,3cyclictest20560-21ls09:10:002
247019940,3cyclictest20265-21munin-run11:35:012
247019940,3cyclictest15984-21ssh10:27:382
247019940,3cyclictest11580-21ssh09:37:022
247019940,3cyclictest0-21swapper/212:25:122
247019940,3cyclictest0-21swapper/210:55:012
247019940,1cyclictest25494-21diskstats10:00:122
247019940,0cyclictest0-21swapper/209:22:092
246999944,0cyclictest0-21swapper/111:31:561
246999944,0cyclictest0-21swapper/111:25:491
246999944,0cyclictest0-21swapper/111:10:331
246999944,0cyclictest0-21swapper/111:10:331
246999944,0cyclictest0-21swapper/110:52:131
246999944,0cyclictest0-21swapper/109:20:231
246999942,1cyclictest0-21swapper/112:35:111
246999941,2cyclictest0-21swapper/112:10:121
246999941,2cyclictest0-21swapper/112:05:121
246999941,2cyclictest0-21swapper/111:40:131
246999941,2cyclictest0-21swapper/111:15:111
246999941,2cyclictest0-21swapper/110:25:131
246999941,2cyclictest0-21swapper/110:00:111
246999941,2cyclictest0-21swapper/109:50:121
246999941,2cyclictest0-21swapper/108:10:121
246999941,2cyclictest0-21swapper/107:40:121
246999941,2cyclictest0-21swapper/107:15:151
246999941,1cyclictest0-21swapper/111:55:121
246999941,1cyclictest0-21swapper/111:35:121
246999940,4cyclictest0-21swapper/110:35:281
246999940,4cyclictest0-21swapper/109:12:141
246999940,4cyclictest0-21swapper/108:36:011
246999940,2cyclictest0-21swapper/112:25:121
246999940,2cyclictest0-21swapper/109:40:121
246999940,0cyclictest0-21swapper/110:45:231
246989944,0cyclictest0-21swapper/012:28:190
246989944,0cyclictest0-21swapper/011:55:520
246989944,0cyclictest0-21swapper/010:23:110
246989944,0cyclictest0-21swapper/009:22:390
246989942,2cyclictest23296-21kworker/0:209:25:170
246989942,1cyclictest30281-21latency_hist08:25:010
246989941,2cyclictest24467-21dpkg10:00:020
246989941,2cyclictest0-21swapper/008:50:110
246989941,2cyclictest0-21swapper/008:45:120
246989941,1cyclictest0-21swapper/010:10:120
246989941,1cyclictest0-21swapper/009:05:120
246989940,4cyclictest7662-21sh12:17:020
246989940,4cyclictest31988-21ssh09:15:170
246989940,4cyclictest31418-21taskset10:19:340
246989940,4cyclictest0-21swapper/011:34:160
246989940,4cyclictest0-21swapper/011:23:070
246989940,4cyclictest0-21swapper/011:16:070
246989940,4cyclictest0-21swapper/010:45:110
246989940,4cyclictest0-21swapper/010:30:070
246989940,4cyclictest0-21swapper/008:55:430
246989940,4cyclictest0-21swapper/008:30:130
246989940,3cyclictest7501-21perl09:35:140
246989940,3cyclictest6591-21hddtemp_smartct12:00:130
246989940,3cyclictest6339-21perl10:55:150
246989940,3cyclictest31275-21users09:30:240
246989940,3cyclictest26291-21diskstats11:05:120
246989940,3cyclictest26242-21perl08:15:140
246989940,3cyclictest24061-21sh09:59:550
246989940,3cyclictest23063-21apt08:10:100
246989940,3cyclictest1757-21hddtemp_smartct11:25:150
246989940,3cyclictest15473-21sh11:00:020
246989940,3cyclictest1094-21hddtemp_smartct12:30:130
246989940,3cyclictest0-21swapper/011:35:110
246989940,1cyclictest32745-21latency_hist08:30:000
246989940,0cyclictest0-21swapper/010:50:330
246989940,0cyclictest0-21swapper/010:39:560
246989940,0cyclictest0-21swapper/008:05:050
24934230,2sleep020375-21diskmemload12:10:010
247019933,0cyclictest27272-21kworker/2:009:00:092
247019933,0cyclictest27272-21kworker/2:007:35:062
247019933,0cyclictest27272-21kworker/2:007:30:062
247019932,1cyclictest714-21snmpd07:45:122
247019931,1cyclictest0-21swapper/212:30:102
247019930,3cyclictest8097-21dpkg07:40:012
247019930,3cyclictest30545-21ssh11:40:042
247019930,3cyclictest0-21swapper/212:35:002
247019930,3cyclictest0-21swapper/211:05:112
247019930,3cyclictest0-21swapper/210:05:012
247019930,3cyclictest0-21swapper/208:55:052
247019930,3cyclictest0-21swapper/208:50:112
247019930,3cyclictest0-21swapper/208:25:122
247019930,3cyclictest0-21swapper/208:20:122
247019930,3cyclictest0-21swapper/208:15:002
247019930,3cyclictest0-21swapper/208:10:102
247019930,3cyclictest0-21swapper/208:05:132
247019930,3cyclictest0-21swapper/208:00:122
247019930,3cyclictest0-21swapper/207:55:032
247019930,3cyclictest0-21swapper/207:50:112
247019930,3cyclictest0-21swapper/207:20:102
247019930,1cyclictest0-21swapper/207:25:092
247019930,0cyclictest7991-21apt-get08:45:112
246999933,0cyclictest0-21swapper/111:50:061
246999933,0cyclictest0-21swapper/111:46:191
246999931,2cyclictest0-21swapper/107:55:111
246999931,1cyclictest0-21swapper/112:20:111
246999931,1cyclictest0-21swapper/112:15:121
246999931,1cyclictest0-21swapper/111:20:111
246999931,1cyclictest0-21swapper/111:05:101
246999931,1cyclictest0-21swapper/110:55:011
246999931,1cyclictest0-21swapper/110:15:111
246999931,1cyclictest0-21swapper/110:10:121
246999931,1cyclictest0-21swapper/110:05:111
246999931,1cyclictest0-21swapper/109:45:141
246999931,1cyclictest0-21swapper/109:00:101
246999931,1cyclictest0-21swapper/108:45:111
246999931,1cyclictest0-21swapper/108:40:151
246999931,1cyclictest0-21swapper/108:20:121
246999931,1cyclictest0-21swapper/107:50:111
246999931,1cyclictest0-21swapper/107:45:131
246999931,1cyclictest0-21swapper/107:25:091
246999930,2cyclictest2555-21chrt08:32:091
246999930,2cyclictest0-21swapper/110:30:131
246999930,2cyclictest0-21swapper/109:30:121
246999930,2cyclictest0-21swapper/109:15:131
246999930,2cyclictest0-21swapper/108:25:111
246999930,2cyclictest0-21swapper/108:05:151
246999930,2cyclictest0-21swapper/107:35:111
246999930,2cyclictest0-21swapper/107:20:131
246989933,0cyclictest0-21swapper/012:20:010
246989932,1cyclictest18763-21ls11:50:140
246989931,1cyclictest27795-21hddtemp_smartct07:15:030
246989931,1cyclictest0-21swapper/007:30:110
246989930,3cyclictest6987-21find07:35:180
246989930,3cyclictest15944-21diskstats09:00:110
246989930,3cyclictest10850-21df10:25:140
246989930,3cyclictest0-21swapper/012:35:110
246989930,3cyclictest0-21swapper/012:05:020
246989930,3cyclictest0-21swapper/011:45:120
246989930,3cyclictest0-21swapper/008:35:100
246989930,3cyclictest0-21swapper/008:00:050
246989930,3cyclictest0-21swapper/007:55:120
246989930,3cyclictest0-21swapper/007:50:120
246989930,3cyclictest0-21swapper/007:45:140
246989930,3cyclictest0-21swapper/007:40:010
246989930,3cyclictest0-21swapper/007:25:100
246989930,3cyclictest0-21swapper/007:20:140
246989930,3cyclictest0-21swapper/007:10:080
246999921,1cyclictest0-21swapper/110:20:131
246999921,1cyclictest0-21swapper/109:55:131
246999921,1cyclictest0-21swapper/108:15:111
246999921,1cyclictest0-21swapper/108:00:121
246999920,2cyclictest0-21swapper/108:50:101
246999920,1cyclictest18116-21chrt09:05:021
246999920,1cyclictest0-21swapper/112:00:111
246999920,1cyclictest0-21swapper/108:55:111
246999920,1cyclictest0-21swapper/107:30:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional