You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 17:54
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Fri Jul 04, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
76202203189,10sleep10-21swapper/107:07:121
75312187154,28sleep00-21swapper/007:06:030
75612181165,12sleep30-21swapper/307:06:263
77202177160,13sleep20-21swapper/207:08:292
260852470,0sleep20-21swapper/210:09:402
220882440,0sleep10-21swapper/112:01:451
107722410,0sleep10-21swapper/111:06:441
185772140,1sleep00-21swapper/009:16:420
80179982,3cyclictest3150-21munin-run11:20:000
80179981,6cyclictest0-21swapper/011:50:130
80179971,5cyclictest0-21swapper/008:00:000
80179970,4cyclictest8655-21sh10:50:000
80249960,5cyclictest0-21swapper/208:40:102
80249960,5cyclictest0-21swapper/207:40:132
80229962,2cyclictest0-21swapper/112:35:141
80179963,1cyclictest3-21ksoftirqd/008:25:000
80179963,1cyclictest3-21ksoftirqd/007:35:000
80179960,3cyclictest1006-21perf12:25:000
80319954,1cyclictest23490-21ssh11:30:013
80319950,5cyclictest0-21swapper/312:33:593
80319950,5cyclictest0-21swapper/311:26:373
80319950,5cyclictest0-21swapper/309:56:423
80319950,4cyclictest4592-21apt-get10:15:013
80319950,4cyclictest31907-21memory11:50:173
80319950,4cyclictest0-21swapper/307:35:003
80319950,3cyclictest0-21swapper/310:00:583
80249955,0cyclictest0-21swapper/210:53:032
80249950,5cyclictest0-21swapper/212:13:312
80249950,5cyclictest0-21swapper/211:59:432
80249950,5cyclictest0-21swapper/211:11:502
80249950,5cyclictest0-21swapper/210:18:122
80249950,5cyclictest0-21swapper/209:31:362
80249950,0cyclictest0-21swapper/211:22:372
80229951,2cyclictest0-21swapper/109:15:121
80229950,5cyclictest0-21swapper/110:27:141
80229950,5cyclictest0-21swapper/110:11:231
80229950,5cyclictest0-21swapper/109:00:221
80179955,0cyclictest3-21ksoftirqd/009:10:010
80179954,1cyclictest3-21ksoftirqd/010:35:000
80179953,0cyclictest3-21ksoftirqd/009:05:010
80179951,1cyclictest81rcu_preempt12:25:120
80179950,5cyclictest0-21swapper/011:48:100
80179950,5cyclictest0-21swapper/011:10:290
80179950,5cyclictest0-21swapper/009:59:380
80179950,1cyclictest1186-21apt-get08:00:000
80179950,0cyclictest0-21swapper/007:50:250
20888250,0sleep10-21swapper/111:28:371
9378240,0sleep10-21swapper/112:11:581
80319944,0cyclictest22541-21kworker/3:107:27:493
80319944,0cyclictest0-21swapper/311:48:093
80319943,1cyclictest14030-21/usr/sbin/munin08:25:183
80319942,2cyclictest0-21swapper/311:22:063
80319942,1cyclictest12090-21diskstats08:20:153
80319941,3cyclictest30997-21snmpd08:55:203
80319941,3cyclictest15054-21fschecks_count09:15:153
80319940,4cyclictest9048-21ssh10:50:073
80319940,4cyclictest4119-21apt-get08:05:163
80319940,4cyclictest28566-21/usr/sbin/munin10:10:193
80319940,4cyclictest1507-21/usr/sbin/munin12:25:213
80319940,4cyclictest0-21swapper/312:15:283
80319940,4cyclictest0-21swapper/312:10:123
80319940,4cyclictest0-21swapper/311:35:153
80319940,4cyclictest0-21swapper/310:47:023
80319940,4cyclictest0-21swapper/310:30:453
80319940,4cyclictest0-21swapper/310:20:123
80319940,4cyclictest0-21swapper/309:45:433
80319940,4cyclictest0-21swapper/309:36:143
80319940,4cyclictest0-21swapper/309:20:133
80319940,4cyclictest0-21swapper/309:00:123
80319940,3cyclictest7942-21sh09:11:193
80319940,3cyclictest6227-21sh11:05:013
80319940,3cyclictest28817-21ssh11:00:023
80319940,3cyclictest28157-21cstates12:05:163
80319940,3cyclictest25938-21latency_hist11:15:003
80319940,3cyclictest24769-21sh10:40:513
80319940,3cyclictest21587-21sh09:50:223
80319940,3cyclictest20595-21ssh12:35:013
80319940,3cyclictest19557-21/usr/sbin/munin12:00:203
80319940,3cyclictest17278-21sh10:05:143
80319940,3cyclictest1717-21sh09:25:013
80319940,3cyclictest16135-21rm11:10:013
80319940,3cyclictest0-21swapper/309:40:123
80319940,2cyclictest9719-21/usr/sbin/munin11:55:193
80319940,1cyclictest8127-21/usr/sbin/munin07:10:193
80319940,1cyclictest28901-21/usr/sbin/munin07:50:203
80319940,1cyclictest24263-21/usr/sbin/munin12:20:193
80319940,1cyclictest23964-21/usr/sbin/munin07:40:193
80319940,1cyclictest19259-21/usr/sbin/munin10:55:163
80319940,1cyclictest1601-21/usr/sbin/munin09:05:183
80319940,1cyclictest10595-21munin-node07:15:013
80249944,0cyclictest0-21swapper/211:50:062
80249944,0cyclictest0-21swapper/210:24:582
80249944,0cyclictest0-21swapper/209:35:222
80249944,0cyclictest0-21swapper/209:26:392
80249944,0cyclictest0-21swapper/208:46:202
80249943,1cyclictest0-21swapper/211:18:502
80249942,1cyclictest25870-21ssh10:58:172
80249942,1cyclictest17491-21ssh09:16:012
80249941,2cyclictest0-21swapper/208:30:122
80249941,2cyclictest0-21swapper/208:05:172
80249940,4cyclictest12957-21ssh10:02:392
80249940,4cyclictest0-21swapper/212:25:132
80249940,4cyclictest0-21swapper/212:15:132
80249940,4cyclictest0-21swapper/212:05:142
80249940,4cyclictest0-21swapper/212:00:142
80249940,4cyclictest0-21swapper/211:42:422
80249940,4cyclictest0-21swapper/211:25:132
80249940,4cyclictest0-21swapper/211:00:122
80249940,4cyclictest0-21swapper/210:42:072
80249940,4cyclictest0-21swapper/210:35:132
80249940,4cyclictest0-21swapper/210:30:132
80249940,4cyclictest0-21swapper/210:10:382
80249940,4cyclictest0-21swapper/209:52:412
80249940,4cyclictest0-21swapper/208:25:152
80249940,4cyclictest0-21swapper/207:50:142
80249940,4cyclictest0-21swapper/207:45:162
80249940,4cyclictest0-21swapper/207:25:122
80249940,3cyclictest8797-21perl07:10:142
80249940,3cyclictest4950-21ssh09:10:142
80249940,3cyclictest27399-21rm11:31:222
80249940,3cyclictest27100-21ssh12:38:042
80249940,3cyclictest25860-21ntp_states09:20:192
80249940,3cyclictest19204-21ls09:50:012
80249940,3cyclictest10598-21munin-node07:15:002
80249940,3cyclictest0-21swapper/211:35:162
80249940,3cyclictest0-21swapper/210:45:142
80249940,0cyclictest0-21swapper/209:44:242
80229944,0cyclictest0-21swapper/111:55:251
80229944,0cyclictest0-21swapper/110:59:591
80229944,0cyclictest0-21swapper/110:16:371
80229944,0cyclictest0-21swapper/109:59:181
80229944,0cyclictest0-21swapper/109:28:281
80229941,2cyclictest0-21swapper/111:20:141
80229941,2cyclictest0-21swapper/109:50:141
80229941,2cyclictest0-21swapper/109:10:131
80229941,2cyclictest0-21swapper/108:45:121
80229941,2cyclictest0-21swapper/108:20:131
80229941,2cyclictest0-21swapper/107:55:131
80229941,2cyclictest0-21swapper/107:50:141
80229941,1cyclictest0-21swapper/107:35:141
80229940,4cyclictest0-21swapper/111:12:381
80229940,4cyclictest0-21swapper/110:39:021
80229940,4cyclictest0-21swapper/110:21:511
80229940,4cyclictest0-21swapper/109:49:251
80229940,4cyclictest0-21swapper/109:36:361
80229940,4cyclictest0-21swapper/109:21:281
80229940,2cyclictest0-21swapper/112:25:131
80229940,0cyclictest0-21swapper/111:35:291
80179944,0cyclictest3-21ksoftirqd/009:25:120
80179944,0cyclictest3-21ksoftirqd/008:30:110
80179944,0cyclictest0-21swapper/010:24:220
80179944,0cyclictest0-21swapper/008:55:000
80179943,1cyclictest3-21ksoftirqd/011:05:170
80179943,1cyclictest3-21ksoftirqd/009:50:130
80179943,1cyclictest3-21ksoftirqd/008:40:120
80179943,1cyclictest21435-21munin-run08:40:000
80179942,1cyclictest30997-21snmpd10:05:090
80179941,3cyclictest24968-21expr09:20:140
80179941,2cyclictest0-21swapper/008:10:140
80179941,2cyclictest0-21swapper/007:15:130
80179941,2cyclictest0-21swapper/007:10:120
80179941,1cyclictest3-21ksoftirqd/011:55:130
80179941,1cyclictest3-21ksoftirqd/010:00:120
80179940,4cyclictest3839-21diskmemload11:40:120
80179940,4cyclictest18411-21ssh10:38:080
80179940,4cyclictest0-21swapper/012:15:130
80179940,4cyclictest0-21swapper/012:10:550
80179940,4cyclictest0-21swapper/011:30:240
80179940,4cyclictest0-21swapper/011:29:290
80179940,4cyclictest0-21swapper/011:20:140
80179940,4cyclictest0-21swapper/011:00:120
80179940,4cyclictest0-21swapper/010:57:560
80179940,4cyclictest0-21swapper/010:50:130
80179940,4cyclictest0-21swapper/010:25:140
80179940,4cyclictest0-21swapper/010:15:140
80179940,4cyclictest0-21swapper/009:41:190
80179940,4cyclictest0-21swapper/009:37:460
80179940,4cyclictest0-21swapper/008:15:110
80179940,4cyclictest0-21swapper/007:25:120
80179940,3cyclictest9478-21ssh09:45:110
80179940,3cyclictest4356-21ssh09:10:120
80179940,3cyclictest27200-21sh10:10:130
80179940,3cyclictest24497-21diskstats07:40:140
80179940,3cyclictest17796-21sh12:00:080
80179940,3cyclictest13210-21memory09:30:170
80179940,2cyclictest0-21swapper/008:25:150
80179940,1cyclictest26385-21awk08:50:000
80179940,0cyclictest0-21swapper/010:40:050
80179940,0cyclictest0-21swapper/007:38:190
30774240,0sleep10-21swapper/112:06:051
10575240,1sleep2802499cyclictest11:06:372
80319930,3cyclictest31828-21expr07:55:153
80319930,3cyclictest27117-21df08:50:143
80319930,3cyclictest24876-21cpu10:25:153
80319930,3cyclictest11906-21sh10:35:093
80319930,3cyclictest0-21swapper/308:40:153
80319930,3cyclictest0-21swapper/308:35:013
80319930,3cyclictest0-21swapper/308:15:113
80319930,3cyclictest0-21swapper/308:10:143
80319930,3cyclictest0-21swapper/308:00:013
80319930,3cyclictest0-21swapper/307:45:123
80319930,3cyclictest0-21swapper/307:30:113
80319930,3cyclictest0-21swapper/307:20:013
80319930,2cyclictest10791-21ssh11:40:013
80319930,2cyclictest0-21swapper/309:30:023
80319930,1cyclictest16497-21/usr/sbin/munin08:30:123
80319930,0cyclictest24189-21cut08:45:133
80249931,2cyclictest3888-21kworker/2:212:30:042
80249930,3cyclictest6918-21grep08:10:152
80249930,3cyclictest29345-21cat08:55:122
80249930,3cyclictest29336-21sed09:55:022
80249930,3cyclictest24697-21apt-get10:25:142
80249930,3cyclictest1501-21apt-config08:00:002
80249930,3cyclictest0-21swapper/212:20:032
80249930,3cyclictest0-21swapper/211:45:032
80249930,3cyclictest0-21swapper/209:05:122
80249930,3cyclictest0-21swapper/209:00:132
80249930,3cyclictest0-21swapper/208:50:132
80249930,3cyclictest0-21swapper/208:35:012
80249930,3cyclictest0-21swapper/208:20:022
80249930,3cyclictest0-21swapper/208:15:002
80249930,3cyclictest0-21swapper/207:55:142
80249930,3cyclictest0-21swapper/207:35:012
80249930,3cyclictest0-21swapper/207:30:112
80249930,3cyclictest0-21swapper/207:20:122
80229933,0cyclictest0-21swapper/111:46:321
80229933,0cyclictest0-21swapper/110:03:051
80229931,1cyclictest0-21swapper/111:50:121
80229931,1cyclictest0-21swapper/111:15:121
80229931,1cyclictest0-21swapper/111:00:121
80229931,1cyclictest0-21swapper/110:50:131
80229931,1cyclictest0-21swapper/110:45:121
80229931,1cyclictest0-21swapper/110:40:131
80229931,1cyclictest0-21swapper/110:30:121
80229931,1cyclictest0-21swapper/110:05:141
80229931,1cyclictest0-21swapper/109:40:121
80229931,1cyclictest0-21swapper/109:05:131
80229931,1cyclictest0-21swapper/108:55:111
80229931,1cyclictest0-21swapper/108:50:131
80229931,1cyclictest0-21swapper/108:35:131
80229931,1cyclictest0-21swapper/108:25:151
80229931,1cyclictest0-21swapper/108:00:111
80229931,1cyclictest0-21swapper/107:15:131
80229931,1cyclictest0-21swapper/107:10:121
80229930,3cyclictest0-21swapper/107:47:101
80229930,2cyclictest0-21swapper/112:20:131
80229930,2cyclictest0-21swapper/107:40:131
80229930,0cyclictest0-21swapper/108:06:401
80179933,0cyclictest41ktimersoftd/007:45:120
80179932,1cyclictest20662-21ssh12:35:040
80179930,3cyclictest11296-21ssh12:30:130
80179930,3cyclictest0-21swapper/011:35:030
80179930,3cyclictest0-21swapper/008:50:130
80179930,3cyclictest0-21swapper/008:05:160
80179930,3cyclictest0-21swapper/007:20:120
80179930,1cyclictest0-21swapper/012:05:140
80229921,1cyclictest0-21swapper/107:25:111
80229920,1cyclictest9045-21chrt08:15:011
80229920,1cyclictest6570-21chrt08:10:051
80229920,1cyclictest18798-21chrt07:30:101
80229920,1cyclictest15841-21chrt07:21:531
80229920,1cyclictest0-21swapper/112:30:131
80229920,1cyclictest0-21swapper/112:15:111
80229920,1cyclictest0-21swapper/111:40:121
80229920,1cyclictest0-21swapper/111:30:011
80229920,1cyclictest0-21swapper/109:30:131
80229920,1cyclictest0-21swapper/108:40:121
80229920,1cyclictest0-21swapper/108:30:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional