You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 12:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Thu Nov 13, 2025 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
53412205169,23sleep10-21swapper/119:06:491
53012203164,25sleep30-21swapper/319:06:203
52392200160,27sleep20-21swapper/219:05:352
54222197161,23sleep00-21swapper/019:07:540
197642690,0sleep10-21swapper/120:45:151
266072660,0sleep121-21ksoftirqd/123:15:341
7802590,0sleep30-21swapper/323:25:213
191392180,0sleep20-21swapper/223:50:202
208272160,0sleep30-21swapper/321:39:563
567899130,13cyclictest0-21swapper/122:10:211
5674991311,1cyclictest9-21ksoftirqd/022:05:000
567499130,12cyclictest0-21swapper/020:35:200
568399120,12cyclictest0-21swapper/222:50:182
5674991211,1cyclictest9-21ksoftirqd/021:05:000
5674991211,1cyclictest9-21ksoftirqd/000:40:000
5683991111,0cyclictest0-21swapper/222:42:072
5683991110,1cyclictest28-21ksoftirqd/220:20:002
568399110,9cyclictest31966-21munin-node00:10:122
568399110,5cyclictest19110-21snmpd23:38:052
567499119,1cyclictest9-21ksoftirqd/023:15:000
5674991110,1cyclictest9-21ksoftirqd/023:41:370
5674991110,1cyclictest9-21ksoftirqd/023:00:000
567499110,7cyclictest19110-21snmpd21:15:430
568399109,1cyclictest28-21ksoftirqd/221:40:012
5683991010,0cyclictest0-21swapper/221:59:122
567499109,1cyclictest9-21ksoftirqd/021:40:010
567499108,1cyclictest9351-21basename20:25:000
567499102,4cyclictest20954-21perf23:10:000
56919998,1cyclictest35-21ksoftirqd/321:00:003
56919998,1cyclictest20602-21sh23:53:523
56839997,1cyclictest28-21ksoftirqd/219:50:012
56839991,5cyclictest30231-21/usr/sbin/munin22:35:242
56839990,3cyclictest27811-21apt-get19:55:102
56839990,1cyclictest0-21swapper/221:30:422
56749999,0cyclictest9-21ksoftirqd/000:30:180
56749998,1cyclictest9-21ksoftirqd/022:30:170
56919987,1cyclictest35-21ksoftirqd/300:02:063
56919986,1cyclictest35-21ksoftirqd/321:05:003
56919983,4cyclictest19110-21snmpd00:14:443
56839988,0cyclictest0-21swapper/200:16:432
56839986,1cyclictest60950irq/123-eno123:15:122
56839986,1cyclictest28-21ksoftirqd/220:50:012
56839986,1cyclictest28-21ksoftirqd/200:34:222
56839986,1cyclictest28-21ksoftirqd/200:00:132
56839985,2cyclictest60950irq/123-eno122:30:152
56839984,2cyclictest60950irq/123-eno123:45:092
56839984,2cyclictest0-21swapper/200:37:142
56839982,5cyclictest28-21ksoftirqd/222:07:242
56839982,5cyclictest28-21ksoftirqd/222:07:242
56839981,5cyclictest304962sleep223:21:332
56839981,4cyclictest9589-21apt-get20:25:172
56839980,4cyclictest22704-21python23:55:222
56839980,3cyclictest0-21swapper/222:15:212
56789981,6cyclictest0-21swapper/122:21:001
56789980,4cyclictest0-21swapper/100:30:131
56749988,0cyclictest30591-21diskmemload00:00:060
56749987,1cyclictest9-21ksoftirqd/021:50:000
56749987,1cyclictest9-21ksoftirqd/020:20:000
56749986,1cyclictest9-21ksoftirqd/022:20:150
56749986,1cyclictest9-21ksoftirqd/021:25:000
56749986,1cyclictest9-21ksoftirqd/019:30:000
56749985,2cyclictest14706-21ssh23:45:150
56749982,5cyclictest30591-21diskmemload22:15:200
56749981,6cyclictest891-21ssh23:25:220
56749981,5cyclictest13536-21/usr/sbin/munin21:30:250
56749980,7cyclictest31215-21ssh23:23:450
56749980,7cyclictest21967-21sed22:25:190
56749980,7cyclictest12138-21ssh00:25:280
56749980,4cyclictest0-21swapper/023:02:080
56749980,1cyclictest0-21swapper/000:10:200
56919976,1cyclictest35-21ksoftirqd/320:45:013
56919976,1cyclictest35-21ksoftirqd/320:00:003
56919976,1cyclictest19110-21snmpd23:09:403
56919975,1cyclictest35-21ksoftirqd/320:35:153
56919975,1cyclictest19110-21snmpd21:28:283
56919975,1cyclictest19110-21snmpd21:28:283
56919971,5cyclictest20780-21kworker/3:022:53:083
56919971,5cyclictest19110-21snmpd22:19:153
56919971,3cyclictest0-21swapper/322:55:143
56919971,0cyclictest0-21swapper/322:32:173
56919970,6cyclictest26978-21ssh21:46:373
56919970,4cyclictest742-21runrttasks21:17:523
56919970,3cyclictest0-21swapper/321:13:253
56839976,1cyclictest60950irq/123-eno123:13:372
56839976,1cyclictest60950irq/123-eno122:57:242
56839976,1cyclictest60950irq/123-eno122:02:332
56839976,1cyclictest60950irq/123-eno121:51:092
56839976,1cyclictest60950irq/123-eno121:45:082
56839976,1cyclictest60950irq/123-eno121:15:312
56839976,1cyclictest19110-21snmpd23:27:522
56839976,1cyclictest19110-21snmpd19:26:102
56839976,0cyclictest60950irq/123-eno121:41:102
56839976,0cyclictest60950irq/123-eno100:23:212
56839975,1cyclictest60950irq/123-eno122:10:142
56839975,1cyclictest60950irq/123-eno100:25:132
56839973,1cyclictest0-21swapper/221:13:322
56839970,4cyclictest7095-21/usr/sbin/munin21:20:172
56839970,3cyclictest0-21swapper/222:46:302
56839970,3cyclictest0-21swapper/220:00:122
56839970,2cyclictest0-21swapper/221:25:192
56839970,2cyclictest0-21swapper/221:25:192
56839970,0cyclictest0-21swapper/220:23:412
56789971,4cyclictest0-21swapper/100:35:011
56789971,3cyclictest0-21swapper/121:52:431
56789971,3cyclictest0-21swapper/100:25:181
56789970,6cyclictest0-21swapper/123:51:051
56789970,6cyclictest0-21swapper/100:10:161
56789970,5cyclictest0-21swapper/121:55:201
56789970,5cyclictest0-21swapper/121:46:171
56789970,4cyclictest0-21swapper/122:25:141
56789970,4cyclictest0-21swapper/122:15:071
56789970,3cyclictest0-21swapper/121:10:551
56749976,1cyclictest9-21ksoftirqd/021:10:000
56749976,1cyclictest9-21ksoftirqd/020:45:130
56749976,1cyclictest9-21ksoftirqd/019:45:130
56749976,1cyclictest31033-1kworker/u9:200:20:410
56749976,1cyclictest29623-21pmu-power00:05:210
56749976,1cyclictest19110-21snmpd19:14:180
56749975,1cyclictest9-21ksoftirqd/023:50:130
56749975,1cyclictest9-21ksoftirqd/020:30:000
56749972,4cyclictest6497-21nscd21:55:290
56749972,1cyclictest16008-21/usr/sbin/munin19:30:170
56749971,6cyclictest6698-21ssh22:50:060
56749971,6cyclictest30889-21ssh22:37:260
56749970,6cyclictest7861-21ssh23:35:160
56749970,6cyclictest5926-21ssh22:48:260
56749970,6cyclictest27386-21ssh23:18:000
56749970,6cyclictest2140-21ssh22:42:430
56749970,6cyclictest0-21swapper/022:06:290
56749970,6cyclictest0-21swapper/022:06:280
56749970,6cyclictest0-21swapper/021:40:090
56749970,6cyclictest0-21swapper/019:20:130
56749970,4cyclictest0-21swapper/021:50:400
56749970,4cyclictest0-21swapper/019:50:200
56749970,3cyclictest0-21swapper/023:33:420
56749970,2cyclictest0-21swapper/020:00:120
56749970,1cyclictest30591-21diskmemload23:56:430
56749970,1cyclictest0-21swapper/022:10:420
56749970,1cyclictest0-21swapper/021:25:450
56749970,1cyclictest0-21swapper/021:25:450
56749970,1cyclictest0-21swapper/000:15:170
56919965,1cyclictest9970-21ssh00:24:503
56919965,1cyclictest9093-21ssh23:36:173
56919965,1cyclictest5954-21ssh23:32:353
56919965,1cyclictest31854-21ssh21:55:083
56919965,1cyclictest30149-21ssh21:50:283
56919965,1cyclictest27342-21ssh23:17:523
56919965,1cyclictest2416-21ssh22:43:363
56919965,1cyclictest22675-21sshd19:42:413
56919965,1cyclictest20450-21sshd19:39:413
56919965,1cyclictest19110-21snmpd22:01:413
56919965,1cyclictest19110-21snmpd20:33:333
56919965,1cyclictest19110-21snmpd20:18:373
56919965,1cyclictest19110-21snmpd19:32:283
56919965,1cyclictest17401-21ssh23:49:583
56919965,1cyclictest11580-21ssh22:10:193
56919965,1cyclictest10053-21ssh22:09:493
56919965,1cyclictest10053-21ssh22:09:493
56919964,2cyclictest19110-21snmpd21:24:193
56919964,1cyclictest3650-21diskstats00:15:123
56919964,1cyclictest23581-21ssh23:57:043
56919964,1cyclictest13131-21ssh23:42:453
56919964,0cyclictest30591-21diskmemload22:45:163
56919963,1cyclictest30591-21diskmemload21:30:533
56919962,2cyclictest21205-21ssh23:10:113
56919961,3cyclictest22696-21/usr/sbin/munin21:40:213
56919961,1cyclictest30591-21diskmemload23:20:373
56919961,1cyclictest0-21swapper/300:05:223
56919960,2cyclictest0-21swapper/322:35:083
56919960,2cyclictest0-21swapper/322:22:013
56839966,0cyclictest60950irq/123-eno123:40:272
56839966,0cyclictest60950irq/123-eno123:01:302
56839966,0cyclictest60950irq/123-eno121:02:222
56839966,0cyclictest60950irq/123-eno120:55:082
56839966,0cyclictest60950irq/123-eno120:50:202
56839966,0cyclictest60950irq/123-eno120:40:122
56839966,0cyclictest60950irq/123-eno120:30:132
56839966,0cyclictest60950irq/123-eno120:05:042
56839966,0cyclictest60950irq/123-eno119:40:222
56839966,0cyclictest60950irq/123-eno119:15:022
56839965,1cyclictest60950irq/123-eno122:20:062
56839965,1cyclictest60950irq/123-eno100:05:242
56839965,1cyclictest28-21ksoftirqd/220:10:122
56839964,1cyclictest60950irq/123-eno123:30:192
56839964,1cyclictest17670-21aten_r1power_cu23:05:132
56839964,1cyclictest14827-21cut20:35:172
56839960,6cyclictest0-21swapper/219:36:192
56839960,3cyclictest8001-21apt19:15:002
56839960,2cyclictest0-21swapper/222:25:202
56839960,2cyclictest0-21swapper/221:05:132
56789961,3cyclictest0-21swapper/122:50:161
56789961,3cyclictest0-21swapper/121:15:151
56789961,1cyclictest0-21swapper/123:25:221
56789960,6cyclictest0-21swapper/123:36:221
56789960,6cyclictest0-21swapper/123:34:131
56789960,6cyclictest0-21swapper/123:10:031
56789960,6cyclictest0-21swapper/122:04:261
56789960,6cyclictest0-21swapper/120:52:531
56789960,6cyclictest0-21swapper/119:31:071
56789960,5cyclictest0-21swapper/121:00:211
56789960,4cyclictest0-21swapper/100:00:121
56789960,2cyclictest0-21swapper/123:02:471
56789960,2cyclictest0-21swapper/121:30:161
56789960,1cyclictest0-21swapper/120:22:051
56789960,1cyclictest0-21swapper/120:12:081
56789960,1cyclictest0-21swapper/119:38:521
56789960,1cyclictest0-21swapper/119:29:481
56789960,0cyclictest0-21swapper/121:20:281
56789960,0cyclictest0-21swapper/121:05:121
56749965,1cyclictest9-21ksoftirqd/019:40:010
56749965,1cyclictest9-21ksoftirqd/019:15:000
56749965,1cyclictest19110-21snmpd19:56:080
56749961,1cyclictest0-21swapper/020:30:120
56749960,6cyclictest0-21swapper/020:10:210
56749960,5cyclictest0-21swapper/020:05:210
56749960,0cyclictest0-21swapper/021:05:190
32542260,0sleep30-21swapper/320:05:123
56919955,0cyclictest35-21ksoftirqd/322:27:023
56919954,1cyclictest19110-21snmpd19:47:073
56919954,1cyclictest12203-21ssh00:25:403
56919953,1cyclictest19538-21/usr/sbin/munin20:45:143
56919953,1cyclictest17735-21tr00:35:143
56919952,2cyclictest31098-21/usr/sbin/munin20:00:163
56919952,2cyclictest11301-21apt-get19:20:133
56919952,2cyclictest11125-21latency_hist19:20:003
56919952,1cyclictest254-21systemd-journal20:10:003
56919952,1cyclictest0-21swapper/319:50:143
56919950,5cyclictest0-21swapper/319:11:093
56919950,3cyclictest0-21swapper/323:00:123
56919950,2cyclictest0-21swapper/321:05:123
56919950,2cyclictest0-21swapper/320:50:203
56919950,0cyclictest0-21swapper/319:29:353
56839955,0cyclictest60950irq/123-eno119:20:152
56839950,3cyclictest0-21swapper/219:50:132
56839950,2cyclictest0-21swapper/219:30:102
56789955,0cyclictest0-21swapper/123:20:431
56789953,1cyclictest19901-21chrt23:06:491
56789953,1cyclictest0-21swapper/120:55:161
56789951,3cyclictest0-21swapper/121:35:171
56789951,2cyclictest0-21swapper/121:26:131
56789951,2cyclictest0-21swapper/121:26:131
56789950,5cyclictest0-21swapper/122:05:171
56789950,5cyclictest0-21swapper/122:05:171
56789950,5cyclictest0-21swapper/120:02:201
56789950,5cyclictest0-21swapper/119:40:221
56789950,4cyclictest0-21swapper/119:45:191
56789950,4cyclictest0-21swapper/100:15:201
56789950,3cyclictest0-21swapper/123:55:021
56789950,3cyclictest0-21swapper/122:45:161
56789950,3cyclictest0-21swapper/122:37:231
56789950,3cyclictest0-21swapper/121:43:081
56789950,2cyclictest0-21swapper/123:46:531
56789950,2cyclictest0-21swapper/122:55:171
56789950,2cyclictest0-21swapper/119:25:001
56789950,2cyclictest0-21swapper/100:08:391
56789950,0cyclictest0-21swapper/120:31:151
56749954,1cyclictest9-21ksoftirqd/019:40:160
56749953,1cyclictest16779-21cut20:40:130
56749950,5cyclictest0-21swapper/020:55:070
56749950,4cyclictest0-21swapper/020:50:170
13999250,0sleep30-21swapper/300:30:153
56919943,1cyclictest7536-21cstates20:20:143
56919943,1cyclictest10389-21hddtemp_smartct20:25:203
56789943,1cyclictest16244-21chrt20:37:581
56789941,1cyclictest0-21swapper/120:40:141
56789940,4cyclictest0-21swapper/119:15:451
56789940,3cyclictest0-21swapper/119:10:151
56789940,3cyclictest0-21swapper/100:20:011
56789940,2cyclictest0-21swapper/123:40:381
56789940,2cyclictest0-21swapper/122:40:281
56789940,2cyclictest0-21swapper/122:30:001
56789940,2cyclictest0-21swapper/120:25:171
56789940,2cyclictest0-21swapper/119:55:101
56789940,1cyclictest0-21swapper/119:50:161
56789940,0cyclictest0-21swapper/120:05:211
56789930,1cyclictest0-21swapper/120:20:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional