You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 09:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Wed Nov 12, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178372206191,10sleep10-21swapper/119:08:241
176362190173,12sleep30-21swapper/319:05:503
176832189172,12sleep00-21swapper/019:06:260
179152156136,14sleep20-21swapper/219:09:252
116502450,0sleep00-21swapper/022:29:390
228262440,0sleep3381rcuc/322:35:153
166332440,0sleep10-21swapper/122:48:101
119112440,0sleep10-21swapper/123:18:321
236312430,0sleep30-21swapper/321:14:483
49452410,0sleep30-21swapper/321:20:413
176562410,0sleep10-21swapper/121:27:231
40912330,0sleep00-21swapper/022:09:340
55332100,0sleep20-21swapper/223:47:492
181459982,3cyclictest24174-21munin-run23:25:000
181559970,6cyclictest0-21swapper/319:15:133
181459970,4cyclictest28908-21awk19:30:000
181559965,1cyclictest26525-21kworker/3:123:19:403
181559960,1cyclictest23274-21cron00:30:003
181519961,4cyclictest0-21swapper/223:35:112
181459960,5cyclictest0-21swapper/021:10:030
181559955,0cyclictest26525-21kworker/3:122:01:153
181559955,0cyclictest26525-21kworker/3:121:05:183
181559954,1cyclictest26525-21kworker/3:122:34:513
181559954,1cyclictest22047-21sed21:30:023
181559952,3cyclictest0-21swapper/323:37:473
181559950,5cyclictest0-21swapper/323:00:313
181559950,5cyclictest0-21swapper/321:48:473
181559950,3cyclictest0-21swapper/323:14:083
181559950,3cyclictest0-21swapper/300:14:443
181559950,0cyclictest0-21swapper/322:49:293
181519955,0cyclictest0-21swapper/222:07:192
181519955,0cyclictest0-21swapper/221:36:122
181519950,5cyclictest0-21swapper/223:58:302
181519950,5cyclictest0-21swapper/222:20:262
181519950,5cyclictest0-21swapper/222:10:142
181519950,5cyclictest0-21swapper/200:35:272
181519950,5cyclictest0-21swapper/200:14:212
181519950,4cyclictest5659-21diskmemload22:42:292
181519950,1cyclictest0-21swapper/223:07:252
181499952,2cyclictest0-21swapper/122:35:151
181499951,2cyclictest0-21swapper/123:30:131
181499951,2cyclictest0-21swapper/121:40:141
181499951,2cyclictest0-21swapper/100:10:121
181499950,5cyclictest0-21swapper/123:42:571
181499950,5cyclictest0-21swapper/122:25:331
181459955,0cyclictest25266-21apt-get22:20:130
181459955,0cyclictest0-21swapper/021:54:020
181459950,5cyclictest22014-21df_inode19:15:140
181459950,5cyclictest18875-21ssh22:49:370
181459950,5cyclictest0-21swapper/022:18:460
181459950,5cyclictest0-21swapper/021:56:550
181459950,5cyclictest0-21swapper/021:30:530
181459950,5cyclictest0-21swapper/000:25:380
9676240,0sleep10-21swapper/119:55:161
4234240,0sleep10-21swapper/122:09:401
22521240,0sleep30-21swapper/323:56:033
181559944,0cyclictest40-21ksoftirqd/323:25:133
181559944,0cyclictest40-21ksoftirqd/319:20:123
181559944,0cyclictest0-21swapper/319:44:403
181559944,0cyclictest0-21swapper/300:02:473
181559943,1cyclictest40-21ksoftirqd/322:55:113
181559943,1cyclictest20337-21grep20:15:193
181559942,1cyclictest25843-21diskstats22:20:153
181559941,3cyclictest55950irq/130-eth023:45:163
181559941,3cyclictest0-21swapper/321:42:363
181559940,4cyclictest31135-21sh00:33:443
181559940,4cyclictest31135-21sh00:33:443
181559940,4cyclictest26055-21ssh21:31:223
181559940,4cyclictest22167-21ssh23:40:113
181559940,4cyclictest18010-21df_inode23:05:143
181559940,4cyclictest15421-21diskstats23:20:173
181559940,4cyclictest0-21swapper/323:50:123
181559940,4cyclictest0-21swapper/322:10:123
181559940,4cyclictest0-21swapper/321:35:153
181559940,4cyclictest0-21swapper/321:15:133
181559940,4cyclictest0-21swapper/321:15:133
181559940,4cyclictest0-21swapper/319:35:163
181559940,4cyclictest0-21swapper/300:20:123
181559940,3cyclictest8758-21/usr/sbin/munin21:55:163
181559940,3cyclictest6442-21sh00:05:023
181559940,3cyclictest5829-21ssh00:36:593
181559940,3cyclictest4096-21apt20:50:153
181559940,3cyclictest2728-21df23:30:133
181559940,3cyclictest24140-21sh22:51:403
181559940,3cyclictest21364-21ssh22:17:553
181559940,3cyclictest10254-21sh22:28:463
181559940,3cyclictest0-21swapper/320:45:123
181559940,1cyclictest9740-21perl19:55:153
181559940,1cyclictest6599-21/usr/sbin/munin19:50:113
181559940,1cyclictest24137-21/usr/sbin/munin20:25:203
181559940,0cyclictest0-21swapper/321:50:593
181519944,0cyclictest0-21swapper/223:27:392
181519944,0cyclictest0-21swapper/223:17:532
181519944,0cyclictest0-21swapper/219:45:272
181519944,0cyclictest0-21swapper/219:26:412
181519943,1cyclictest29525-21tr00:00:122
181519940,4cyclictest1626-21apt-get20:45:132
181519940,4cyclictest0-21swapper/223:54:432
181519940,4cyclictest0-21swapper/223:20:152
181519940,4cyclictest0-21swapper/222:50:452
181519940,4cyclictest0-21swapper/222:35:052
181519940,4cyclictest0-21swapper/222:30:132
181519940,4cyclictest0-21swapper/221:55:132
181519940,4cyclictest0-21swapper/221:45:052
181519940,4cyclictest0-21swapper/221:16:082
181519940,4cyclictest0-21swapper/221:16:072
181519940,4cyclictest0-21swapper/221:10:132
181519940,4cyclictest0-21swapper/220:10:132
181519940,4cyclictest0-21swapper/200:22:392
181519940,4cyclictest0-21swapper/200:05:062
181519940,3cyclictest7196-21ssh23:00:012
181519940,3cyclictest5926-21taskset21:54:112
181519940,3cyclictest32040-21rm22:55:232
181519940,3cyclictest28347-21ssh23:42:592
181519940,3cyclictest24259-21rm22:19:432
181519940,3cyclictest18971-21/usr/sbin/munin20:15:142
181519940,3cyclictest1868-21ssh21:20:012
181519940,3cyclictest18105-21ssh21:27:412
181519940,3cyclictest16406-21ssh21:43:062
181519940,3cyclictest12358-21hddtemp_smartct21:05:202
181519940,3cyclictest0-21swapper/223:10:192
181519940,3cyclictest0-21swapper/222:26:432
181519940,3cyclictest0-21swapper/221:00:002
181519940,1cyclictest2980-21perl23:30:142
181519940,1cyclictest10785-21diskstats22:45:152
181499944,0cyclictest0-21swapper/123:20:241
181499944,0cyclictest0-21swapper/122:40:161
181499944,0cyclictest0-21swapper/121:33:541
181499944,0cyclictest0-21swapper/100:00:261
181499941,2cyclictest0-21swapper/122:10:131
181499941,2cyclictest0-21swapper/122:00:111
181499941,2cyclictest0-21swapper/121:55:121
181499941,2cyclictest0-21swapper/121:35:161
181499941,2cyclictest0-21swapper/121:15:141
181499941,2cyclictest0-21swapper/121:15:141
181499941,2cyclictest0-21swapper/120:45:131
181499941,2cyclictest0-21swapper/120:05:131
181499941,2cyclictest0-21swapper/119:40:131
181499941,2cyclictest0-21swapper/119:35:151
181499941,2cyclictest0-21swapper/119:30:121
181499941,2cyclictest0-21swapper/119:20:121
181499941,2cyclictest0-21swapper/100:20:131
181499941,2cyclictest0-21swapper/100:15:111
181499941,1cyclictest0-21swapper/121:50:121
181499940,3cyclictest0-21swapper/123:50:121
181499940,0cyclictest0-21swapper/123:06:531
181499940,0cyclictest0-21swapper/121:20:251
181499940,0cyclictest0-21swapper/120:03:411
181459944,0cyclictest0-21swapper/000:33:560
181459944,0cyclictest0-21swapper/000:33:560
181459941,3cyclictest714-21snmpd21:17:130
181459941,3cyclictest714-21snmpd21:17:120
181459941,3cyclictest0-21swapper/021:00:130
181459941,2cyclictest0-21swapper/020:05:140
181459941,1cyclictest0-21swapper/020:30:120
181459940,4cyclictest0-21swapper/023:56:090
181459940,4cyclictest0-21swapper/023:25:130
181459940,4cyclictest0-21swapper/023:10:130
181459940,4cyclictest0-21swapper/022:50:120
181459940,4cyclictest0-21swapper/022:40:120
181459940,4cyclictest0-21swapper/022:30:370
181459940,4cyclictest0-21swapper/022:12:560
181459940,4cyclictest0-21swapper/022:00:110
181459940,4cyclictest0-21swapper/021:40:130
181459940,4cyclictest0-21swapper/021:38:090
181459940,4cyclictest0-21swapper/021:20:030
181459940,4cyclictest0-21swapper/020:50:150
181459940,4cyclictest0-21swapper/020:15:110
181459940,4cyclictest0-21swapper/019:45:150
181459940,4cyclictest0-21swapper/000:35:120
181459940,4cyclictest0-21swapper/000:14:130
181459940,4cyclictest0-21swapper/000:05:480
181459940,3cyclictest8761-21ssh23:16:320
181459940,3cyclictest3672-21grep00:20:000
181459940,3cyclictest32298-21perl20:40:190
181459940,3cyclictest31590-21ssh00:00:310
181459940,3cyclictest24812-21hddtemp_smartct19:20:140
181459940,3cyclictest23670-21hddtemp_smartct22:35:170
181459940,3cyclictest21996-21date21:30:020
181459940,3cyclictest21934-21chrt23:40:020
181459940,3cyclictest20208-21ssh21:45:130
181459940,3cyclictest190912sleep023:38:300
181459940,3cyclictest18019-21diskstats23:05:150
181459940,3cyclictest11305-21latency_hist20:00:000
181459940,3cyclictest0-21swapper/019:50:160
181459940,2cyclictest12375-21gltestperf20:00:190
181459940,1cyclictest5285-21sed00:20:170
181559932,1cyclictest714-21snmpd22:40:043
181559932,1cyclictest714-21snmpd20:05:033
181559932,1cyclictest714-21snmpd19:10:033
181559932,1cyclictest40-21ksoftirqd/320:35:153
181559932,1cyclictest29520-21cat19:30:133
181559932,1cyclictest27463-21ssh22:05:003
181559932,1cyclictest26567-21gpg-connect-age19:25:003
181559930,3cyclictest27328-21tune2fs20:30:153
181559930,3cyclictest26215-21sh00:15:003
181559930,3cyclictest21605-21dpkg20:20:003
181559930,3cyclictest0-21swapper/321:00:113
181559930,3cyclictest0-21swapper/320:55:123
181559930,3cyclictest0-21swapper/320:40:183
181559930,3cyclictest0-21swapper/320:10:133
181559930,3cyclictest0-21swapper/320:00:093
181559930,3cyclictest0-21swapper/319:45:053
181519931,1cyclictest0-21swapper/220:35:142
181519931,1cyclictest0-21swapper/219:35:152
181519930,3cyclictest4838-21perl20:50:182
181519930,3cyclictest31577-21apt-get20:40:182
181519930,3cyclictest24790-21expr19:20:152
181519930,3cyclictest24141-21apt-get20:25:112
181519930,3cyclictest21190-21if_eth019:15:012
181519930,3cyclictest11843-21grep20:00:162
181519930,3cyclictest0-21swapper/222:00:122
181519930,3cyclictest0-21swapper/221:30:132
181519930,3cyclictest0-21swapper/220:30:112
181519930,3cyclictest0-21swapper/219:55:022
181519930,3cyclictest0-21swapper/219:50:112
181519930,3cyclictest0-21swapper/219:40:132
181519930,3cyclictest0-21swapper/219:30:012
181519930,3cyclictest0-21swapper/219:10:132
181519930,3cyclictest0-21swapper/200:30:122
181519930,3cyclictest0-21swapper/200:30:122
181519930,3cyclictest0-21swapper/200:25:012
181519930,2cyclictest21572-21apt-key20:20:012
181519930,2cyclictest14028-21/usr/sbin/munin20:05:142
181519930,2cyclictest0-21swapper/200:15:122
181519930,0cyclictest9053-21apt-get21:00:132
181499933,0cyclictest0-21swapper/119:46:211
181499933,0cyclictest0-21swapper/100:33:451
181499933,0cyclictest0-21swapper/100:33:441
181499931,2cyclictest0-21swapper/122:55:111
181499931,2cyclictest0-21swapper/120:15:111
181499931,1cyclictest0-21swapper/123:25:141
181499931,1cyclictest0-21swapper/123:10:111
181499931,1cyclictest0-21swapper/122:20:121
181499931,1cyclictest0-21swapper/122:15:121
181499931,1cyclictest0-21swapper/121:45:031
181499931,1cyclictest0-21swapper/121:10:041
181499931,1cyclictest0-21swapper/121:00:121
181499931,1cyclictest0-21swapper/120:55:131
181499931,1cyclictest0-21swapper/120:50:161
181499931,1cyclictest0-21swapper/120:35:141
181499931,1cyclictest0-21swapper/120:25:121
181499931,1cyclictest0-21swapper/120:20:141
181499931,1cyclictest0-21swapper/120:10:121
181499931,1cyclictest0-21swapper/119:10:131
181499931,1cyclictest0-21swapper/100:35:121
181499931,1cyclictest0-21swapper/100:25:141
181499931,1cyclictest0-21swapper/100:05:121
181499930,3cyclictest0-21swapper/120:31:191
181499930,2cyclictest0-21swapper/121:05:161
181499930,2cyclictest0-21swapper/120:40:171
181499930,2cyclictest0-21swapper/119:25:121
181499930,1cyclictest0-21swapper/122:30:131
181499930,1cyclictest0-21swapper/119:15:121
181459932,1cyclictest714-21snmpd20:45:050
181459931,1cyclictest21533-21dpkg20:20:000
181459931,1cyclictest0-21swapper/020:55:130
181459930,3cyclictest32626-21df23:45:150
181459930,3cyclictest32119-21df_inode19:35:180
181459930,3cyclictest29822-21ssh22:55:070
181459930,3cyclictest29083-21apt-get20:35:150
181459930,3cyclictest2164-21expr19:40:150
181459930,3cyclictest17026-21diskstats20:10:150
181459930,3cyclictest0-21swapper/023:50:050
181459930,3cyclictest0-21swapper/023:30:040
181459930,3cyclictest0-21swapper/023:00:120
181459930,3cyclictest0-21swapper/021:05:170
181459930,3cyclictest0-21swapper/020:25:110
181459930,3cyclictest0-21swapper/019:10:090
181459930,2cyclictest29311-21tr19:30:120
181499921,1cyclictest0-21swapper/123:45:121
181499920,2cyclictest0-21swapper/119:50:121
181499920,1cyclictest0-21swapper/123:55:111
181499920,1cyclictest0-21swapper/123:35:121
181499920,1cyclictest0-21swapper/123:00:131
181499920,1cyclictest0-21swapper/122:50:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional