You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-04 - 15:54
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Tue Nov 04, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11172199168,12sleep10-21swapper/107:07:371
10142195180,11sleep00-21swapper/007:06:140
12572193177,11sleep30-21swapper/307:09:223
12562184168,11sleep20-21swapper/207:09:222
100012710,0sleep20-21swapper/212:14:432
69302430,0sleep30-21swapper/310:50:483
272352420,0sleep30-21swapper/312:39:433
40962410,0sleep00-21swapper/010:17:110
105472110,0sleep30-21swapper/310:04:523
7747290,0sleep20-21swapper/210:51:202
16168290,0sleep20-21swapper/210:23:302
19656280,1sleep30-21swapper/307:45:003
15309971,6cyclictest0-21swapper/210:45:142
15089970,7cyclictest0-21swapper/007:15:120
15309965,1cyclictest0-21swapper/210:33:572
15089961,5cyclictest30404-21kworker/0:011:26:580
15089961,5cyclictest30404-21kworker/0:008:25:140
15089960,3cyclictest5134-21perf09:30:000
8542250,0sleep10-21swapper/112:30:111
7840250,0sleep20-21swapper/209:46:422
15369950,5cyclictest30406-1kworker/3:0H09:36:423
15369950,5cyclictest0-21swapper/310:44:403
15369950,5cyclictest0-21swapper/309:11:063
15369950,1cyclictest20354-21munin-run12:19:593
15309955,0cyclictest0-21swapper/212:05:122
15309954,1cyclictest14756-21ssh11:27:392
15309954,1cyclictest0-21swapper/210:41:232
15309951,2cyclictest0-21swapper/211:50:162
15309950,5cyclictest0-21swapper/211:41:552
15309950,5cyclictest0-21swapper/209:41:352
15309950,4cyclictest16340-21ssh11:45:132
15219951,2cyclictest0-21swapper/110:30:161
15219951,2cyclictest0-21swapper/110:15:111
15097250,4sleep215126-21ssh09:34:592
15089955,0cyclictest0-21swapper/012:26:150
15089954,1cyclictest0-21swapper/009:46:290
15089952,3cyclictest30404-21kworker/0:010:40:230
15089952,3cyclictest30404-21kworker/0:010:24:160
15089952,3cyclictest30404-21kworker/0:009:15:150
15089952,3cyclictest30404-21kworker/0:009:10:220
15089951,4cyclictest30404-21kworker/0:011:18:000
15089951,3cyclictest0-21swapper/007:50:150
15089950,5cyclictest30404-21kworker/0:012:00:200
15089950,5cyclictest29525-21diskmemload09:31:030
15089950,5cyclictest27060-21munin-run09:05:000
15089950,3cyclictest0-21swapper/008:14:550
15089950,1cyclictest28452-21munin-run11:35:010
26579240,0sleep00-21swapper/011:00:380
16117240,0sleep30-21swapper/311:11:563
15369944,0cyclictest0-21swapper/312:10:043
15369944,0cyclictest0-21swapper/311:28:453
15369944,0cyclictest0-21swapper/310:30:333
15369944,0cyclictest0-21swapper/310:22:253
15369943,1cyclictest7641-21tail10:35:183
15369943,1cyclictest5378-21cpuspeed_turbos07:15:133
15369943,1cyclictest19902-21perl10:25:163
15369942,1cyclictest5705-21ssh10:18:123
15369941,3cyclictest63932chrt09:45:483
15369941,2cyclictest0-21swapper/308:10:123
15369940,4cyclictest3162-21hddtemp_smartct08:15:173
15369940,4cyclictest0-21swapper/311:45:123
15369940,4cyclictest0-21swapper/311:40:133
15369940,4cyclictest0-21swapper/311:37:273
15369940,4cyclictest0-21swapper/310:57:043
15369940,4cyclictest0-21swapper/310:45:153
15369940,4cyclictest0-21swapper/309:50:163
15369940,4cyclictest0-21swapper/309:40:123
15369940,4cyclictest0-21swapper/308:35:133
15369940,4cyclictest0-21swapper/307:32:533
15369940,3cyclictest5345-21diskstats08:20:143
15369940,3cyclictest32084-21irqstats12:25:173
15369940,3cyclictest28540-21df09:25:143
15369940,3cyclictest24466-21perl09:55:173
15369940,3cyclictest19293-21ntp_states09:20:183
15369940,3cyclictest0-21swapper/312:10:003
15369940,1cyclictest31545-21/usr/sbin/munin11:20:183
15369940,1cyclictest26338-21/usr/sbin/munin11:50:163
15369940,1cyclictest22453-21tr07:50:153
15369940,1cyclictest22324-21/usr/sbin/munin08:55:203
15369940,1cyclictest21574-21/usr/sbin/munin11:15:193
15369940,1cyclictest21151-21/usr/sbin/munin12:20:203
15369940,1cyclictest19893-21/usr/sbin/munin07:45:163
15369940,1cyclictest1754-21tr11:05:133
15369940,1cyclictest15029-21ntp_states12:00:183
15369940,0cyclictest0-21swapper/311:33:153
15309944,0cyclictest0-21swapper/211:17:552
15309944,0cyclictest0-21swapper/207:10:332
15309943,1cyclictest8301-21ssh12:30:012
15309942,2cyclictest28731-21fschecks_time09:25:162
15309942,1cyclictest0-21swapper/209:15:122
15309942,1cyclictest0-21swapper/208:05:152
15309941,2cyclictest0-21swapper/211:00:122
15309941,2cyclictest0-21swapper/210:25:152
15309941,2cyclictest0-21swapper/210:05:142
15309941,2cyclictest0-21swapper/208:30:152
15309941,2cyclictest0-21swapper/208:30:152
15309940,4cyclictest4815-21apt-get08:20:122
15309940,4cyclictest22327-21apt-get08:55:132
15309940,4cyclictest0-21swapper/212:35:112
15309940,4cyclictest0-21swapper/212:25:132
15309940,4cyclictest0-21swapper/212:15:142
15309940,4cyclictest0-21swapper/212:00:522
15309940,4cyclictest0-21swapper/211:56:052
15309940,4cyclictest0-21swapper/211:35:142
15309940,4cyclictest0-21swapper/211:11:282
15309940,4cyclictest0-21swapper/211:05:122
15309940,4cyclictest0-21swapper/210:55:132
15309940,4cyclictest0-21swapper/210:20:002
15309940,4cyclictest0-21swapper/210:02:222
15309940,4cyclictest0-21swapper/209:55:152
15309940,4cyclictest0-21swapper/209:35:132
15309940,4cyclictest0-21swapper/209:20:092
15309940,4cyclictest0-21swapper/209:10:142
15309940,4cyclictest0-21swapper/208:40:012
15309940,4cyclictest0-21swapper/208:00:122
15309940,3cyclictest4813-21munin-node07:15:012
15309940,3cyclictest31555-21ssh11:20:132
15309940,3cyclictest21166-21ssh12:20:142
15309940,3cyclictest20268-21/usr/sbin/munin11:30:162
15309940,3cyclictest15175-21rm10:39:242
15309940,3cyclictest0-21swapper/207:45:112
15309940,3cyclictest0-21swapper/207:30:122
15219944,0cyclictest0-21swapper/111:36:211
15219944,0cyclictest0-21swapper/111:18:191
15219944,0cyclictest0-21swapper/111:09:181
15219944,0cyclictest0-21swapper/109:52:461
15219944,0cyclictest0-21swapper/109:25:331
15219942,1cyclictest0-21swapper/110:50:131
15219941,2cyclictest0-21swapper/110:20:161
15219941,1cyclictest0-21swapper/110:35:161
15219941,1cyclictest0-21swapper/107:40:131
15219940,4cyclictest0-21swapper/111:49:101
15219940,4cyclictest0-21swapper/109:24:491
15219940,2cyclictest0-21swapper/108:35:141
15219940,0cyclictest0-21swapper/111:04:481
15219940,0cyclictest0-21swapper/109:40:161
15089943,1cyclictest27077-21latency_hist08:00:010
15089942,1cyclictest29709-21grep09:10:000
15089942,1cyclictest0-21swapper/010:29:550
15089941,3cyclictest714-21snmpd08:31:470
15089941,3cyclictest714-21snmpd08:31:470
15089941,2cyclictest0-21swapper/008:45:140
15089941,2cyclictest0-21swapper/007:40:130
15089941,2cyclictest0-21swapper/007:30:130
15089940,4cyclictest30404-21kworker/0:009:36:420
15089940,4cyclictest0-21swapper/012:30:120
15089940,4cyclictest0-21swapper/012:16:560
15089940,4cyclictest0-21swapper/011:50:160
15089940,4cyclictest0-21swapper/011:20:120
15089940,4cyclictest0-21swapper/011:09:590
15089940,4cyclictest0-21swapper/010:55:330
15089940,4cyclictest0-21swapper/010:50:120
15089940,4cyclictest0-21swapper/010:35:150
15089940,4cyclictest0-21swapper/010:30:150
15089940,4cyclictest0-21swapper/010:05:140
15089940,4cyclictest0-21swapper/009:55:140
15089940,4cyclictest0-21swapper/009:20:120
15089940,4cyclictest0-21swapper/008:05:140
15089940,4cyclictest0-21swapper/007:25:130
15089940,3cyclictest4906-21ssh11:55:170
15089940,3cyclictest4144-21awk08:15:230
15089940,3cyclictest3328-21sed10:00:200
15089940,3cyclictest29139-21ssh11:35:150
15089940,3cyclictest28253-21ps10:45:200
15089940,3cyclictest27234-21ntp_states09:40:180
15089940,3cyclictest23558-21sh12:05:120
15089940,3cyclictest22071-21perl10:10:160
15089940,3cyclictest21719-21df12:20:160
15089940,3cyclictest21129-21ntp_states08:50:200
15089940,3cyclictest17285-21ssh09:51:200
15089940,3cyclictest16879-21df11:45:130
15089940,3cyclictest15924-21irqstats07:35:160
15089940,3cyclictest12307-21perl11:10:160
15089940,3cyclictest0-21swapper/012:36:150
15089940,3cyclictest0-21swapper/008:40:010
15369933,0cyclictest14935-21apt-get07:35:123
15369932,1cyclictest714-21snmpd09:30:153
15369932,1cyclictest714-21snmpd08:30:033
15369932,1cyclictest714-21snmpd08:30:023
15369932,1cyclictest714-21snmpd07:10:083
15369932,1cyclictest27771-21cat08:00:153
15369931,2cyclictest55950irq/130-eth011:55:033
15369931,1cyclictest0-21swapper/309:05:163
15369931,1cyclictest0-21swapper/307:20:143
15369930,3cyclictest8253-21apt-config12:30:023
15369930,3cyclictest21022-21sh10:10:023
15369930,3cyclictest15071-21cpu08:40:143
15369930,3cyclictest0-21swapper/311:00:083
15369930,3cyclictest0-21swapper/310:05:143
15369930,3cyclictest0-21swapper/309:15:033
15369930,3cyclictest0-21swapper/309:00:113
15369930,3cyclictest0-21swapper/308:50:153
15369930,3cyclictest0-21swapper/308:45:133
15369930,3cyclictest0-21swapper/308:25:133
15369930,3cyclictest0-21swapper/308:05:143
15369930,3cyclictest0-21swapper/307:55:103
15369930,3cyclictest0-21swapper/307:25:133
15309932,1cyclictest7532-21apt-get07:20:142
15309932,1cyclictest714-21snmpd09:00:062
15309932,1cyclictest714-21snmpd08:45:082
15309932,1cyclictest714-21snmpd08:35:112
15309932,1cyclictest714-21snmpd08:15:112
15309931,1cyclictest0-21swapper/209:05:162
15309930,3cyclictest21047-21ssh10:10:022
15309930,3cyclictest20425-21expr08:50:152
15309930,3cyclictest17427-21apt-get07:40:142
15309930,3cyclictest13376-21ssh09:50:022
15309930,3cyclictest11269-21ntp_states07:25:172
15309930,3cyclictest0-21swapper/208:25:182
15309930,3cyclictest0-21swapper/208:10:122
15309930,3cyclictest0-21swapper/207:55:152
15309930,3cyclictest0-21swapper/207:35:132
15309930,0cyclictest22370-21apt-get07:50:142
15219933,0cyclictest0-21swapper/111:31:511
15219933,0cyclictest0-21swapper/111:22:491
15219933,0cyclictest0-21swapper/110:40:121
15219933,0cyclictest0-21swapper/110:27:321
15219931,1cyclictest0-21swapper/112:15:141
15219931,1cyclictest0-21swapper/112:05:121
15219931,1cyclictest0-21swapper/112:00:111
15219931,1cyclictest0-21swapper/111:50:171
15219931,1cyclictest0-21swapper/111:25:141
15219931,1cyclictest0-21swapper/111:10:121
15219931,1cyclictest0-21swapper/110:10:121
15219931,1cyclictest0-21swapper/110:05:151
15219931,1cyclictest0-21swapper/110:00:121
15219931,1cyclictest0-21swapper/109:55:141
15219931,1cyclictest0-21swapper/109:35:141
15219931,1cyclictest0-21swapper/109:30:141
15219931,1cyclictest0-21swapper/109:15:121
15219931,1cyclictest0-21swapper/108:40:141
15219931,1cyclictest0-21swapper/108:20:121
15219931,1cyclictest0-21swapper/107:35:121
15219931,1cyclictest0-21swapper/107:30:121
15219931,1cyclictest0-21swapper/107:25:121
15219930,3cyclictest0-21swapper/108:53:401
15219930,3cyclictest0-21swapper/108:25:531
15219930,3cyclictest0-21swapper/107:19:071
15219930,2cyclictest3292sleep110:47:581
15219930,2cyclictest26729-21chrt09:01:041
15219930,2cyclictest0-21swapper/110:55:131
15219930,2cyclictest0-21swapper/109:45:011
15219930,2cyclictest0-21swapper/108:05:141
15219930,2cyclictest0-21swapper/108:00:121
15089931,1cyclictest0-21swapper/008:35:130
15089930,3cyclictest75212sleep007:20:070
15089930,3cyclictest6469-21apt11:40:130
15089930,3cyclictest22327-21apt-get08:55:120
15089930,3cyclictest1173-21hostname12:10:120
15089930,3cyclictest0-21swapper/008:20:040
15089930,3cyclictest0-21swapper/008:00:120
15089930,3cyclictest0-21swapper/007:45:110
15089930,3cyclictest0-21swapper/007:10:010
12023230,0sleep10-21swapper/111:59:221
15219921,1cyclictest0-21swapper/112:20:031
15219921,1cyclictest0-21swapper/112:10:111
15219921,1cyclictest0-21swapper/107:55:091
15219921,1cyclictest0-21swapper/107:10:021
15219921,0cyclictest0-21swapper/108:55:121
15219920,1cyclictest30710-21chrt12:25:021
15219920,1cyclictest1714-21chrt08:10:251
15219920,1cyclictest0-21swapper/112:35:021
15219920,1cyclictest0-21swapper/111:40:391
15219920,1cyclictest0-21swapper/109:10:151
15219920,1cyclictest0-21swapper/109:05:161
15219920,1cyclictest0-21swapper/108:45:131
15219920,1cyclictest0-21swapper/108:30:141
15219920,1cyclictest0-21swapper/108:30:141
15219920,1cyclictest0-21swapper/108:15:151
15219920,1cyclictest0-21swapper/107:50:151
15219920,1cyclictest0-21swapper/107:45:121
15219920,1cyclictest0-21swapper/107:20:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional