You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 08:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Thu Dec 04, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
255802219157,25sleep10-21swapper/119:06:051
258762212156,24sleep00-21swapper/019:09:540
257822212158,42sleep30-21swapper/319:08:403
256852201161,27sleep20-21swapper/219:07:272
212652620,0sleep00-21swapper/020:05:160
183332590,0sleep10-21swapper/120:00:001
25959991918,1cyclictest28-21ksoftirqd/223:10:132
25959991815,1cyclictest28-21ksoftirqd/200:17:382
25959991815,1cyclictest28-21ksoftirqd/200:17:382
25959991716,1cyclictest28-21ksoftirqd/222:01:042
25959991612,4cyclictest12439-21df_inode22:30:142
25959991312,1cyclictest28-21ksoftirqd/223:30:172
25959991312,0cyclictest28-21ksoftirqd/223:40:192
25959991311,1cyclictest34751/usr/sbin/munin00:35:252
25959991311,1cyclictest28-21ksoftirqd/222:05:152
2595999128,4cyclictest273032chrt20:15:512
25959991211,1cyclictest28-21ksoftirqd/221:50:122
25959991210,1cyclictest28-21ksoftirqd/223:25:002
25959991210,1cyclictest28-21ksoftirqd/221:12:372
25959991210,1cyclictest28-21ksoftirqd/220:25:102
25959991210,1cyclictest28-21ksoftirqd/220:00:142
2595799120,1cyclictest254-21systemd-journal21:53:460
2596099119,1cyclictest4930-21apt-get20:40:133
2596099119,1cyclictest35-21ksoftirqd/321:35:003
2596099119,1cyclictest35-21ksoftirqd/300:00:153
25960991111,0cyclictest35-21ksoftirqd/300:15:203
25960991111,0cyclictest35-21ksoftirqd/300:15:203
2595999119,1cyclictest28-21ksoftirqd/222:40:122
2595999119,1cyclictest28-21ksoftirqd/222:20:162
2595999119,1cyclictest28-21ksoftirqd/222:10:232
2595999119,1cyclictest28-21ksoftirqd/200:06:402
25959991111,0cyclictest28-21ksoftirqd/221:45:222
25959991111,0cyclictest28-21ksoftirqd/200:25:122
25959991110,1cyclictest28-21ksoftirqd/223:58:022
25959991110,1cyclictest28-21ksoftirqd/223:09:332
25959991110,1cyclictest28-21ksoftirqd/222:55:322
25959991110,1cyclictest28-21ksoftirqd/222:25:122
25959991110,1cyclictest28-21ksoftirqd/219:25:002
25959991110,1cyclictest28-21ksoftirqd/200:20:162
25959991110,0cyclictest28-21ksoftirqd/223:50:132
2595999110,10cyclictest60950irq/123-eno121:26:002
2595799119,1cyclictest9-21ksoftirqd/019:15:000
25957991110,1cyclictest9-21ksoftirqd/023:45:000
25957991110,1cyclictest9-21ksoftirqd/022:40:140
25957991110,0cyclictest21745-21ssh00:15:150
25957991110,0cyclictest21745-21ssh00:15:150
2596099109,1cyclictest35-21ksoftirqd/322:25:143
2596099109,1cyclictest35-21ksoftirqd/322:25:013
2596099109,1cyclictest35-21ksoftirqd/322:20:003
2596099109,1cyclictest35-21ksoftirqd/322:20:003
2596099108,1cyclictest35-21ksoftirqd/319:25:003
25960991010,0cyclictest35-21ksoftirqd/323:45:163
2595999109,1cyclictest28-21ksoftirqd/223:49:302
2595999109,1cyclictest28-21ksoftirqd/223:35:182
2595999109,1cyclictest28-21ksoftirqd/223:25:162
2595999109,1cyclictest28-21ksoftirqd/223:15:292
2595999109,1cyclictest28-21ksoftirqd/222:35:162
2595999109,1cyclictest28-21ksoftirqd/222:15:162
2595999109,1cyclictest28-21ksoftirqd/222:15:162
2595999109,1cyclictest28-21ksoftirqd/221:36:442
2595999109,1cyclictest28-21ksoftirqd/221:32:282
2595999109,1cyclictest28-21ksoftirqd/221:15:162
2595999109,1cyclictest28-21ksoftirqd/220:50:152
2595999109,1cyclictest28-21ksoftirqd/200:01:332
2595999108,1cyclictest28-21ksoftirqd/222:50:182
2595999108,1cyclictest28-21ksoftirqd/220:45:002
2595999108,1cyclictest28-21ksoftirqd/220:35:002
25959991010,0cyclictest28-21ksoftirqd/221:56:132
25959991010,0cyclictest28-21ksoftirqd/221:40:122
25959991010,0cyclictest28-21ksoftirqd/221:21:012
2595799109,1cyclictest9-21ksoftirqd/022:55:000
2595799109,1cyclictest9-21ksoftirqd/022:40:000
2595799109,1cyclictest9-21ksoftirqd/022:25:140
2595799109,1cyclictest9-21ksoftirqd/000:10:000
2595799109,1cyclictest1449-21ntpd22:00:550
2595799108,1cyclictest9-21ksoftirqd/023:45:130
2595799105,2cyclictest570-21systemd-logind00:25:210
259609999,0cyclictest35-21ksoftirqd/323:15:013
259609998,1cyclictest35-21ksoftirqd/319:40:003
259609998,0cyclictest35-21ksoftirqd/322:35:143
259599999,0cyclictest323431/usr/sbin/munin00:30:182
259599998,1cyclictest28-21ksoftirqd/222:45:352
259599998,1cyclictest28-21ksoftirqd/221:10:002
259599998,1cyclictest28-21ksoftirqd/220:05:142
259599997,1cyclictest28-21ksoftirqd/219:50:002
259599997,1cyclictest28-21ksoftirqd/219:15:012
259599995,1cyclictest3767-21kworker/u8:019:42:182
259579998,1cyclictest9-21ksoftirqd/023:50:220
259579998,1cyclictest9-21ksoftirqd/023:05:140
259579998,1cyclictest9-21ksoftirqd/019:15:010
259579998,0cyclictest9-21ksoftirqd/022:20:300
259579997,1cyclictest9-21ksoftirqd/022:35:000
259579997,1cyclictest9-21ksoftirqd/020:25:010
259579993,0cyclictest19095-21diskmemload23:32:130
259579991,4cyclictest8632-21latency_hist21:40:000
259579990,5cyclictest254-21systemd-journal19:50:000
25335290,0sleep30-21swapper/321:17:043
259609988,0cyclictest35-21ksoftirqd/322:50:173
259609988,0cyclictest35-21ksoftirqd/322:05:253
259609988,0cyclictest35-21ksoftirqd/321:50:133
259609988,0cyclictest35-21ksoftirqd/321:10:153
259609987,1cyclictest35-21ksoftirqd/323:40:163
259609987,1cyclictest35-21ksoftirqd/322:40:123
259609987,1cyclictest35-21ksoftirqd/322:30:203
259609987,1cyclictest35-21ksoftirqd/322:10:163
259609984,1cyclictest19095-21diskmemload22:47:133
259609982,4cyclictest742-21runrttasks23:50:163
259609980,7cyclictest0-21swapper/321:56:213
259609980,4cyclictest0-21swapper/320:05:003
259599987,1cyclictest28-21ksoftirqd/223:00:162
259599987,1cyclictest28-21ksoftirqd/200:10:242
259599980,8cyclictest0-21swapper/220:35:202
259589981,5cyclictest0-21swapper/123:13:561
259589980,3cyclictest0-21swapper/120:30:171
259579988,0cyclictest9-21ksoftirqd/022:10:190
259579987,1cyclictest9-21ksoftirqd/023:30:000
259579987,1cyclictest9-21ksoftirqd/021:45:000
259579987,1cyclictest9-21ksoftirqd/021:20:150
259579987,1cyclictest9-21ksoftirqd/021:10:000
259579987,1cyclictest1449-21ntpd21:14:050
259579986,1cyclictest9-21ksoftirqd/020:50:000
259579984,3cyclictest12916-21awk00:00:240
259609977,0cyclictest35-21ksoftirqd/300:30:133
259609976,1cyclictest557-21nscd19:10:173
259609976,1cyclictest35-21ksoftirqd/321:40:143
259609976,1cyclictest35-21ksoftirqd/320:15:003
259609975,1cyclictest35-21ksoftirqd/320:23:383
259609975,1cyclictest20043-21ntpq23:25:173
259609972,2cyclictest15406-21gltestperf00:05:183
259609971,4cyclictest29737-21apt-get22:55:113
259609971,1cyclictest0-21swapper/321:37:313
259609970,4cyclictest0-21swapper/323:39:193
259599976,1cyclictest28-21ksoftirqd/221:00:002
259599976,1cyclictest28-21ksoftirqd/219:10:182
259599975,1cyclictest60950irq/123-eno119:50:132
259599975,1cyclictest28-21ksoftirqd/221:00:172
259599975,1cyclictest28-21ksoftirqd/220:20:142
259599975,1cyclictest1078-21kworker/2:019:25:122
259589971,3cyclictest0-21swapper/123:50:131
259589970,5cyclictest0-21swapper/100:00:131
259589970,4cyclictest0-21swapper/123:49:321
259589970,4cyclictest0-21swapper/123:20:131
259589970,4cyclictest0-21swapper/122:45:381
259589970,4cyclictest0-21swapper/122:35:121
259589970,4cyclictest0-21swapper/121:30:081
259589970,4cyclictest0-21swapper/100:05:141
259579976,1cyclictest9-21ksoftirqd/021:30:140
259579976,1cyclictest9-21ksoftirqd/020:55:000
259579976,1cyclictest24538-1kworker/u9:100:20:070
259579975,1cyclictest27381-21ssh23:35:230
259579975,1cyclictest12754-21cut21:45:150
259579974,1cyclictest254-21systemd-journal20:30:000
259579971,1cyclictest32606-21/usr/sbin/munin20:30:190
259579971,1cyclictest278-21systemd-udevd22:17:090
259579971,1cyclictest278-21systemd-udevd22:17:090
259579970,6cyclictest7630-21ssh23:55:120
259579970,6cyclictest684-21head00:30:150
259579970,6cyclictest30234-21expr22:55:140
259579970,6cyclictest29692-21ssh22:09:500
259579970,6cyclictest25049-21ssh22:47:130
259579970,6cyclictest23638-21ssh21:15:160
259579970,6cyclictest19095-21diskmemload23:13:580
259579970,6cyclictest1430-21apt-get19:25:010
259579970,6cyclictest0-21swapper/019:35:140
259579970,6cyclictest0-21swapper/019:30:140
259579970,2cyclictest570-21systemd-logind19:20:570
259579970,1cyclictest9326-21cstates19:40:130
259579970,1cyclictest847-21/usr/sbin/munin23:00:120
259579970,1cyclictest30337-21ssh21:25:080
259579970,1cyclictest0-21swapper/023:15:160
259609966,0cyclictest35-21ksoftirqd/320:55:213
259609966,0cyclictest35-21ksoftirqd/320:50:143
259609966,0cyclictest35-21ksoftirqd/300:20:163
259609965,1cyclictest35-21ksoftirqd/323:30:203
259609965,1cyclictest35-21ksoftirqd/321:25:163
259609965,1cyclictest35-21ksoftirqd/320:45:143
259609965,1cyclictest35-21ksoftirqd/319:45:003
259609965,1cyclictest2954-21ssh23:01:223
259609965,1cyclictest18883-21ssh00:10:183
259609963,1cyclictest254-21systemd-journal19:27:433
259609963,1cyclictest254-21systemd-journal19:16:133
259609963,1cyclictest254-21systemd-journal00:35:203
259609962,3cyclictest8525-21irqstats23:55:173
259609962,1cyclictest254-21systemd-journal23:21:033
259609962,1cyclictest19095-21diskmemload00:25:223
259609961,4cyclictest14661-1kworker/u9:221:47:433
259609960,3cyclictest0-21swapper/323:10:493
259609960,2cyclictest0-21swapper/323:07:133
259609960,2cyclictest0-21swapper/322:03:413
259599966,0cyclictest8728-21sensors20:45:182
259599966,0cyclictest60950irq/123-eno119:35:442
259599964,1cyclictest28-21ksoftirqd/220:15:002
259599964,1cyclictest28-21ksoftirqd/219:55:132
259589961,3cyclictest0-21swapper/123:00:041
259589961,3cyclictest0-21swapper/122:43:371
259589961,3cyclictest0-21swapper/120:25:101
259589960,6cyclictest0-21swapper/123:35:111
259589960,6cyclictest0-21swapper/122:16:501
259589960,6cyclictest0-21swapper/122:16:501
259589960,5cyclictest0-21swapper/121:40:161
259589960,4cyclictest0-21swapper/123:40:141
259589960,4cyclictest0-21swapper/121:11:411
259589960,4cyclictest0-21swapper/100:20:201
259589960,3cyclictest0-21swapper/123:15:331
259589960,3cyclictest0-21swapper/121:18:461
259589960,3cyclictest0-21swapper/100:35:081
259589960,2cyclictest0-21swapper/123:57:431
259589960,2cyclictest0-21swapper/100:34:531
259589960,1cyclictest0-21swapper/122:25:121
259589960,1cyclictest0-21swapper/122:10:141
259589960,0cyclictest0-21swapper/122:03:571
259579965,1cyclictest9-21ksoftirqd/023:20:150
259579965,1cyclictest9-21ksoftirqd/020:40:000
259579965,1cyclictest9-21ksoftirqd/020:15:000
259579965,1cyclictest3752-21kworker/0:120:15:510
259579965,1cyclictest101rcuc/021:00:130
259579964,1cyclictest9-21ksoftirqd/020:55:140
259579964,1cyclictest15829-21turbostat19:55:000
259579960,6cyclictest19095-21diskmemload21:55:120
259579960,6cyclictest0-21swapper/019:59:370
259579960,5cyclictest18203-21ssh00:10:140
259579960,0cyclictest19095-21diskmemload00:35:160
259579960,0cyclictest0-21swapper/020:40:130
259609955,0cyclictest35-21ksoftirqd/321:20:163
259609954,1cyclictest35-21ksoftirqd/321:05:013
259609954,1cyclictest35-21ksoftirqd/320:30:003
259609954,1cyclictest35-21ksoftirqd/319:30:193
259609953,1cyclictest35-21ksoftirqd/320:05:163
259609952,1cyclictest254-21systemd-journal21:00:453
259609951,3cyclictest23189-21kworker/3:020:30:203
259609951,2cyclictest3259-21/usr/sbin/munin20:35:153
259599953,1cyclictest28-21ksoftirqd/219:30:142
259589955,0cyclictest0-21swapper/119:52:381
259589955,0cyclictest0-21swapper/119:48:071
259589955,0cyclictest0-21swapper/100:26:131
259589951,3cyclictest0-21swapper/121:20:131
259589950,5cyclictest0-21swapper/122:30:061
259589950,5cyclictest0-21swapper/121:07:381
259589950,5cyclictest0-21swapper/119:32:311
259589950,5cyclictest0-21swapper/119:12:331
259589950,4cyclictest0-21swapper/122:07:561
259589950,4cyclictest0-21swapper/121:50:041
259589950,4cyclictest0-21swapper/121:27:591
259589950,4cyclictest0-21swapper/100:15:191
259589950,4cyclictest0-21swapper/100:15:191
259589950,3cyclictest0-21swapper/119:15:131
259589950,3cyclictest0-21swapper/100:12:081
259589950,2cyclictest0-21swapper/123:25:091
259589950,2cyclictest0-21swapper/122:52:171
259589950,2cyclictest0-21swapper/121:55:151
259589950,2cyclictest0-21swapper/121:47:401
259589950,2cyclictest0-21swapper/121:35:141
259589950,1cyclictest0-21swapper/123:05:121
259589950,1cyclictest0-21swapper/122:20:161
259579950,4cyclictest18440-21/usr/sbin/munin20:00:200
259609942,1cyclictest13711-21apt-get19:50:123
259609940,2cyclictest254-21systemd-journal20:17:143
259609940,1cyclictest254-21systemd-journal19:57:143
259609940,1cyclictest11355-21apt-get19:45:113
259589944,0cyclictest0-21swapper/120:43:551
259589944,0cyclictest0-21swapper/120:12:281
259589941,2cyclictest0-21swapper/120:45:101
259589941,2cyclictest0-21swapper/120:15:101
259589941,2cyclictest0-21swapper/120:05:141
259589941,2cyclictest0-21swapper/119:35:121
259589940,4cyclictest0-21swapper/119:40:191
259589940,3cyclictest0-21swapper/123:30:101
259589940,3cyclictest0-21swapper/119:20:121
259589940,1cyclictest0-21swapper/120:50:121
259589940,1cyclictest0-21swapper/120:37:381
259589940,0cyclictest0-21swapper/122:56:051
259589940,0cyclictest0-21swapper/120:00:181
259589931,1cyclictest13884-21chrt20:56:391
259589930,2cyclictest0-21swapper/120:20:191
259589930,2cyclictest0-21swapper/119:25:011
259589930,1cyclictest0-21swapper/121:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional