You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 07:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack1slot1.osadl.org (updated Mon Nov 17, 2025 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
66022212174,24sleep30-21swapper/319:09:463
63122203164,26sleep10-21swapper/119:06:001
63872200160,26sleep00-21swapper/019:06:570
63652199162,24sleep20-21swapper/219:06:422
125232690,0sleep10-21swapper/121:25:221
286902560,0sleep20-21swapper/222:33:402
249322160,0sleep10-21swapper/122:28:031
81592140,0sleep20-21swapper/221:20:192
669699143,5cyclictest11043-21perf20:24:590
670799130,13cyclictest0-21swapper/222:25:102
670799130,13cyclictest0-21swapper/221:51:322
6707991211,1cyclictest28-21ksoftirqd/221:50:012
670799120,0cyclictest0-21swapper/222:13:592
670799120,0cyclictest0-21swapper/221:17:572
6696991211,1cyclictest9-21ksoftirqd/000:35:000
6696991211,1cyclictest9-21ksoftirqd/000:35:000
151572120,0sleep10-21swapper/122:15:131
671399119,1cyclictest32251-21diskmemload21:15:033
670799119,1cyclictest28-21ksoftirqd/220:55:012
6707991110,1cyclictest28-21ksoftirqd/219:55:002
6707991110,1cyclictest15243-21ssh23:00:102
669699113,4cyclictest32515-21perf21:54:590
63982110,0sleep10-21swapper/123:31:401
671399108,1cyclictest35-21ksoftirqd/320:50:013
671399108,1cyclictest35-21ksoftirqd/319:16:453
670799109,1cyclictest28-21ksoftirqd/221:25:232
670799104,5cyclictest9074-21munin-node22:35:362
670799100,9cyclictest9074-21munin-node23:33:442
670799100,0cyclictest0-21swapper/222:00:262
669699109,1cyclictest9-21ksoftirqd/023:25:150
669699109,1cyclictest9-21ksoftirqd/022:15:000
669699108,1cyclictest9-21ksoftirqd/023:05:190
669699108,1cyclictest9-21ksoftirqd/000:25:140
669699100,2cyclictest0-21swapper/023:50:170
67139998,1cyclictest35-21ksoftirqd/322:15:003
67139998,1cyclictest35-21ksoftirqd/319:55:003
67079999,0cyclictest28-21ksoftirqd/200:10:242
67079997,1cyclictest28-21ksoftirqd/220:05:012
66969998,1cyclictest9-21ksoftirqd/023:15:000
66969998,1cyclictest9-21ksoftirqd/023:05:000
66969998,1cyclictest9-21ksoftirqd/022:20:130
66969998,1cyclictest9-21ksoftirqd/022:20:130
66969997,1cyclictest9-21ksoftirqd/022:25:140
66969994,4cyclictest566-21nscd21:46:510
66969992,4cyclictest25860-21apt-get22:30:150
66969990,4cyclictest11064-21/usr/sbin/munin00:25:010
66969990,4cyclictest11064-21/usr/sbin/munin00:25:000
67139987,1cyclictest19110-21snmpd23:09:233
67139986,1cyclictest8715-21cat00:20:203
67139986,1cyclictest8715-21cat00:20:203
67139986,1cyclictest35-21ksoftirqd/322:30:013
67139986,1cyclictest35-21ksoftirqd/320:15:003
67139986,1cyclictest35-21ksoftirqd/319:25:003
67139986,1cyclictest1449-21ntpd20:36:223
67079987,1cyclictest28-21ksoftirqd/219:20:002
67079986,1cyclictest60950irq/123-eno123:45:212
67079986,1cyclictest60950irq/123-eno122:45:162
67079984,3cyclictest1449-21ntpd22:53:582
67079980,8cyclictest0-21swapper/222:57:142
67079980,7cyclictest60950irq/123-eno123:16:162
67079980,4cyclictest742-21runrttasks22:17:302
67079980,4cyclictest1497-21sendmail-mta23:39:062
67079980,0cyclictest0-21swapper/221:33:022
67039980,5cyclictest0-21swapper/123:01:301
66969987,1cyclictest9-21ksoftirqd/023:20:000
66969987,1cyclictest9-21ksoftirqd/019:34:590
66969986,1cyclictest9-21ksoftirqd/022:15:140
66969986,1cyclictest9-21ksoftirqd/021:40:150
66969986,1cyclictest9-21ksoftirqd/019:20:100
66969986,1cyclictest19110-21snmpd20:59:100
66969985,1cyclictest278-21systemd-udevd00:37:090
66969982,4cyclictest7432-21apt-get21:20:150
66969980,7cyclictest20413-21ssh21:35:490
66969980,7cyclictest10089-21ssh23:37:020
66969980,7cyclictest0-21swapper/023:40:120
66969980,4cyclictest18251-21perf23:50:000
66969980,4cyclictest18122-21latency_hist20:40:010
66969980,4cyclictest0-21swapper/022:02:400
66969980,1cyclictest5551-21/usr/sbin/munin22:45:180
67139976,1cyclictest8588-21cat22:05:193
67139976,1cyclictest5356-21ntpq00:15:203
67139976,1cyclictest35-21ksoftirqd/319:50:013
67139976,1cyclictest32514-21cat20:00:193
67139976,1cyclictest19110-21snmpd19:26:393
67139975,1cyclictest8620-21awk20:20:013
67139975,1cyclictest31251-21ssh21:51:113
67139975,1cyclictest19110-21snmpd21:00:483
67139973,3cyclictest1-21systemd20:09:183
67139972,4cyclictest25468-21cat21:45:003
67139971,3cyclictest32251-21diskmemload22:24:303
67139971,3cyclictest32251-21diskmemload22:24:303
67139971,3cyclictest32251-21diskmemload22:15:193
67139971,3cyclictest11340-21apt-get00:25:123
67139970,6cyclictest32251-21diskmemload21:45:573
67139970,4cyclictest0-21swapper/321:40:003
67139970,3cyclictest0-21swapper/323:15:263
67139970,3cyclictest0-21swapper/321:31:233
67139970,3cyclictest0-21swapper/300:38:203
67139970,2cyclictest0-21swapper/323:22:293
67139970,1cyclictest254-21systemd-journal20:54:553
67139970,0cyclictest32251-21diskmemload23:13:363
67139970,0cyclictest0-21swapper/321:14:333
67079976,1cyclictest60950irq/123-eno122:40:522
67079976,1cyclictest60950irq/123-eno121:35:162
67079976,1cyclictest60950irq/123-eno100:32:082
67079976,1cyclictest60950irq/123-eno100:32:072
67079976,1cyclictest28-21ksoftirqd/219:25:012
67079976,1cyclictest28-21ksoftirqd/219:15:012
67079976,1cyclictest28-21ksoftirqd/219:15:002
67079976,1cyclictest28-21ksoftirqd/200:35:202
67079976,0cyclictest60950irq/123-eno123:10:172
67079975,1cyclictest60950irq/123-eno123:40:212
67079975,1cyclictest60950irq/123-eno121:55:272
67079974,2cyclictest8631-21systemctl20:20:002
67079973,1cyclictest254-21systemd-journal23:20:012
67079970,7cyclictest0-21swapper/200:16:222
67079970,6cyclictest0-21swapper/221:45:002
67079970,6cyclictest0-21swapper/219:48:402
67079970,5cyclictest1449-21ntpd23:07:182
67079970,4cyclictest32251-21diskmemload23:50:142
67079970,1cyclictest0-21swapper/219:35:132
67039970,6cyclictest0-21swapper/119:45:171
67039970,5cyclictest0-21swapper/122:00:121
67039970,1cyclictest0-21swapper/121:33:111
67039970,1cyclictest0-21swapper/120:50:181
66969976,1cyclictest9-21ksoftirqd/023:55:150
66969976,1cyclictest9-21ksoftirqd/019:15:010
66969976,1cyclictest6323-21ssh23:31:250
66969976,1cyclictest19110-21snmpd19:39:250
66969975,1cyclictest8619-21munin-run20:20:010
66969975,1cyclictest27133-21ntpq23:15:220
66969975,1cyclictest19110-21snmpd20:43:500
66969975,1cyclictest19110-21snmpd20:26:060
66969973,3cyclictest7944-21apt-get22:50:130
66969971,5cyclictest32511-21/usr/sbin/munin21:10:200
66969971,1cyclictest254-21systemd-journal19:55:000
66969970,7cyclictest4782-1kworker/u9:221:15:340
66969970,7cyclictest32077-21ssh00:07:470
66969970,6cyclictest3187-21ssh21:58:430
66969970,6cyclictest2415-21ssh22:40:290
66969970,6cyclictest23278-21cpuspeed_turbos20:50:140
66969970,6cyclictest15435-21cut21:30:150
66969970,6cyclictest13526-21ssh21:27:190
66969970,6cyclictest0-21swapper/022:57:010
66969970,6cyclictest0-21swapper/022:35:020
66969970,6cyclictest0-21swapper/022:05:240
66969970,6cyclictest0-21swapper/019:45:180
66969970,4cyclictest0-21swapper/020:45:170
66969970,1cyclictest29145-21ssh00:04:440
66969970,1cyclictest15718-21perf20:34:590
66969970,1cyclictest0-21swapper/020:14:350
66969970,1cyclictest0-21swapper/000:15:360
31967270,2sleep10-21swapper/122:37:451
67139965,1cyclictest9763-21cat20:20:173
67139965,1cyclictest6297-21ssh23:31:213
67139965,1cyclictest5922-21ssh22:00:263
67139965,1cyclictest31829-21ssh22:37:203
67139965,1cyclictest30225-21ntpq19:55:203
67139965,1cyclictest2671-21ssh22:41:173
67139965,1cyclictest2463-21ssh21:56:263
67139965,1cyclictest19110-21snmpd21:20:133
67139965,1cyclictest19110-21snmpd21:07:043
67139965,1cyclictest19110-21snmpd19:32:403
67139965,1cyclictest17073-21ssh00:31:333
67139965,1cyclictest17073-21ssh00:31:333
67139965,1cyclictest1449-21ntpd20:41:223
67139965,1cyclictest13593-21ssh21:27:303
67139965,0cyclictest32251-21diskmemload23:00:123
67139964,1cyclictest35-21ksoftirqd/320:35:003
67139964,1cyclictest32251-21diskmemload23:35:273
67139964,1cyclictest27174-21ntp_states00:00:193
67139964,1cyclictest19110-21snmpd19:14:303
67139964,1cyclictest19110-21snmpd19:14:303
67139963,1cyclictest32251-21diskmemload23:25:083
67139961,3cyclictest21746-21nscd00:10:023
67139961,3cyclictest0-21swapper/322:45:203
67139961,1cyclictest16201-21awk23:45:233
67139960,6cyclictest23593-21fgrep23:55:213
67139960,5cyclictest0-21swapper/322:55:013
67139960,4cyclictest0-21swapper/323:50:313
67139960,3cyclictest0-21swapper/322:53:103
67139960,2cyclictest0-21swapper/323:40:323
67139960,2cyclictest0-21swapper/322:30:573
67079966,0cyclictest60950irq/123-eno123:25:202
67079966,0cyclictest60950irq/123-eno122:06:012
67079966,0cyclictest60950irq/123-eno121:05:002
67079966,0cyclictest60950irq/123-eno120:56:542
67079966,0cyclictest60950irq/123-eno120:43:322
67079966,0cyclictest60950irq/123-eno120:26:032
67079966,0cyclictest60950irq/123-eno120:20:132
67079966,0cyclictest60950irq/123-eno120:07:052
67079966,0cyclictest60950irq/123-eno119:26:402
67079966,0cyclictest60950irq/123-eno100:21:072
67079966,0cyclictest60950irq/123-eno100:21:072
67079966,0cyclictest60950irq/123-eno100:01:022
67079965,1cyclictest60950irq/123-eno120:30:182
67079965,1cyclictest28-21ksoftirqd/219:45:002
67079965,1cyclictest19110-21snmpd21:10:302
67079964,1cyclictest60950irq/123-eno120:45:162
67079963,1cyclictest28-21ksoftirqd/219:55:092
67079962,3cyclictest6693-21cyclictest00:05:042
67079961,3cyclictest1497-21sendmail-mta23:55:562
67079961,2cyclictest0-21swapper/200:25:122
67079960,6cyclictest0-21swapper/220:10:202
67079960,5cyclictest19184-21perf19:35:002
67079960,5cyclictest18526-21/usr/sbin/munin22:20:142
67079960,5cyclictest18526-21/usr/sbin/munin22:20:132
67039966,0cyclictest0-21swapper/122:30:301
67039966,0cyclictest0-21swapper/121:35:281
67039966,0cyclictest0-21swapper/120:02:181
67039961,3cyclictest0-21swapper/123:42:371
67039960,6cyclictest0-21swapper/122:23:341
67039960,6cyclictest0-21swapper/122:23:331
67039960,6cyclictest0-21swapper/122:07:051
67039960,6cyclictest0-21swapper/121:51:421
67039960,6cyclictest0-21swapper/120:12:161
67039960,6cyclictest0-21swapper/100:25:161
67039960,6cyclictest0-21swapper/100:20:201
67039960,6cyclictest0-21swapper/100:20:201
67039960,5cyclictest0-21swapper/120:25:181
67039960,4cyclictest0-21swapper/121:57:001
67039960,3cyclictest0-21swapper/123:10:191
67039960,1cyclictest0-21swapper/123:57:341
67039960,1cyclictest0-21swapper/100:33:081
67039960,1cyclictest0-21swapper/100:33:071
67039960,0cyclictest0-21swapper/121:49:001
66969965,1cyclictest8808-21sshd19:12:160
66969965,1cyclictest8808-21sshd19:12:160
66969962,1cyclictest0-21swapper/019:40:240
66969960,6cyclictest0-21swapper/021:07:100
66969960,4cyclictest0-21swapper/000:10:130
2413260,0sleep10-21swapper/122:40:291
20808260,1sleep10-21swapper/100:37:011
20777260,0sleep10-21swapper/123:05:561
67139954,1cyclictest31348-21ssh00:05:313
67139952,0cyclictest11230-21apt-get20:25:133
67139950,4cyclictest25751-21expr20:55:133
67079955,0cyclictest60950irq/123-eno121:00:492
67079950,2cyclictest0-21swapper/220:35:212
67039951,1cyclictest0-21swapper/120:45:201
67039950,5cyclictest0-21swapper/121:11:381
67039950,5cyclictest0-21swapper/120:43:401
67039950,5cyclictest0-21swapper/120:37:591
67039950,5cyclictest0-21swapper/120:30:091
67039950,5cyclictest0-21swapper/119:53:401
67039950,5cyclictest0-21swapper/119:44:281
67039950,5cyclictest0-21swapper/119:15:071
67039950,4cyclictest0-21swapper/121:20:341
67039950,4cyclictest0-21swapper/121:06:211
67039950,4cyclictest0-21swapper/100:07:191
67039950,4cyclictest0-21swapper/100:03:261
67039950,3cyclictest0-21swapper/123:35:221
67039950,3cyclictest0-21swapper/122:45:081
67039950,3cyclictest0-21swapper/122:12:361
67039950,3cyclictest0-21swapper/100:15:131
67039950,2cyclictest0-21swapper/123:54:471
67039950,2cyclictest0-21swapper/123:25:111
67039950,2cyclictest0-21swapper/123:23:251
67039950,2cyclictest0-21swapper/121:15:151
67039950,1cyclictest0-21swapper/123:45:291
67039950,1cyclictest0-21swapper/121:41:421
67039950,1cyclictest0-21swapper/100:10:201
67039950,0cyclictest0-21swapper/123:19:031
67039950,0cyclictest0-21swapper/122:55:231
67039950,0cyclictest0-21swapper/120:55:151
67039950,0cyclictest0-21swapper/120:05:171
67039950,0cyclictest0-21swapper/119:36:541
66969954,1cyclictest9-21ksoftirqd/021:00:100
66969950,5cyclictest0-21swapper/020:05:020
66969950,4cyclictest0-21swapper/019:55:230
66969950,2cyclictest31637-21apt-get20:00:150
66969950,0cyclictest0-21swapper/019:25:110
67139942,1cyclictest19408-21/usr/sbin/munin19:35:143
67139940,1cyclictest0-21swapper/319:40:153
67039944,0cyclictest0-21swapper/119:13:131
67039944,0cyclictest0-21swapper/119:13:121
67039941,1cyclictest0-21swapper/120:00:001
67039940,3cyclictest0-21swapper/121:00:171
67039940,3cyclictest0-21swapper/119:30:131
67039940,2cyclictest0-21swapper/122:50:481
67039940,1cyclictest0-21swapper/120:15:131
67039940,0cyclictest0-21swapper/119:25:121
67039940,0cyclictest0-21swapper/119:20:181
67039931,1cyclictest0-21swapper/120:20:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional