You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 09:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Thu Oct 30, 2025 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
312512226152,50sleep00-21swapper/019:06:370
311982202162,27sleep20-21swapper/219:05:552
312662200164,23sleep30-21swapper/319:06:483
313932196158,25sleep10-21swapper/119:08:271
74172680,1sleep37415-21cat22:15:003
291212620,1sleep328943-21latency_hist22:45:003
229982180,1sleep2271rcuc/200:05:222
257462170,1sleep228-21ksoftirqd/221:55:012
154892150,0sleep00-21swapper/021:40:120
31598991210,1cyclictest28-21ksoftirqd/221:35:122
31598991210,1cyclictest28-21ksoftirqd/221:20:192
31598991210,1cyclictest28-21ksoftirqd/221:15:012
261662120,0sleep10-21swapper/122:40:141
31598991110,1cyclictest28-21ksoftirqd/220:40:012
31598991110,1cyclictest28-21ksoftirqd/200:10:192
3159899110,11cyclictest0-21swapper/223:52:002
3158599113,4cyclictest10397-21cat20:40:010
3159899108,1cyclictest28-21ksoftirqd/223:17:112
3159899108,1cyclictest271rcuc/223:35:092
31598991010,0cyclictest0-21swapper/223:27:152
31598991010,0cyclictest0-21swapper/221:17:582
3159899100,10cyclictest0-21swapper/223:34:192
3159899100,10cyclictest0-21swapper/222:00:082
3158599109,1cyclictest9-21ksoftirqd/019:25:010
3158599104,4cyclictest742-21runrttasks00:01:430
3158599100,5cyclictest25671-21perf23:25:000
316059999,0cyclictest35-21ksoftirqd/300:34:253
316059990,7cyclictest22360-21apt-get23:20:133
316059990,5cyclictest0-21swapper/323:40:113
315989999,0cyclictest28-21ksoftirqd/223:45:152
315989998,1cyclictest28-21ksoftirqd/223:55:232
315989998,1cyclictest28-21ksoftirqd/223:55:222
315989998,1cyclictest28-21ksoftirqd/223:40:142
315989998,1cyclictest28-21ksoftirqd/222:45:012
315989998,1cyclictest28-21ksoftirqd/221:45:002
315989997,1cyclictest28-21ksoftirqd/222:40:122
315989997,1cyclictest28-21ksoftirqd/222:20:142
315989990,1cyclictest0-21swapper/222:15:252
315989990,1cyclictest0-21swapper/200:24:202
315899990,4cyclictest0-21swapper/100:25:091
315859992,4cyclictest12211-21netstat21:35:140
315859990,5cyclictest24515-21diskmemload23:10:590
315859990,4cyclictest29188-21cut22:00:010
316059987,1cyclictest35-21ksoftirqd/322:00:013
316059981,3cyclictest0-21swapper/322:47:113
316059981,3cyclictest0-21swapper/321:45:003
315989988,0cyclictest28-21ksoftirqd/200:25:202
315989988,0cyclictest28-21ksoftirqd/200:00:212
315989988,0cyclictest0-21swapper/223:24:192
315989988,0cyclictest0-21swapper/222:37:462
315989987,1cyclictest28-21ksoftirqd/222:30:002
315989987,1cyclictest28-21ksoftirqd/200:35:182
315989986,1cyclictest60950irq/123-eno122:06:352
315989986,1cyclictest28-21ksoftirqd/200:30:182
315899985,2cyclictest274992sleep121:13:181
315859988,0cyclictest9-21ksoftirqd/022:55:070
315859988,0cyclictest9-21ksoftirqd/021:13:390
315859986,1cyclictest9-21ksoftirqd/019:30:000
315859986,1cyclictest30854-21sensors22:00:210
315859982,5cyclictest24515-21diskmemload23:00:150
316059975,1cyclictest35-21ksoftirqd/320:02:393
316059973,2cyclictest24515-21diskmemload22:53:023
316059972,3cyclictest24515-21diskmemload23:55:223
316059972,3cyclictest24515-21diskmemload23:55:213
316059972,3cyclictest24515-21diskmemload22:57:593
316059972,2cyclictest20031-21apt-get21:00:113
316059971,2cyclictest14243-21apt-get19:40:123
316059971,1cyclictest0-21swapper/300:16:283
316059970,3cyclictest0-21swapper/321:16:273
316059970,1cyclictest0-21swapper/323:08:063
316059970,1cyclictest0-21swapper/322:00:133
315989977,0cyclictest28-21ksoftirqd/222:10:202
315989976,1cyclictest60950irq/123-eno123:01:592
315989976,1cyclictest60950irq/123-eno121:25:132
315989976,1cyclictest60950irq/123-eno120:30:152
315989976,1cyclictest28-21ksoftirqd/222:25:222
315989976,1cyclictest28-21ksoftirqd/219:35:012
315989976,1cyclictest28-21ksoftirqd/219:20:012
315989975,1cyclictest60950irq/123-eno120:20:132
315989975,1cyclictest28-21ksoftirqd/220:30:012
315989975,1cyclictest28-21ksoftirqd/220:30:002
315989975,1cyclictest15678-21kworker/2:223:10:002
315989974,1cyclictest24515-21diskmemload22:55:132
315989973,4cyclictest1530-21munin-node21:30:422
315989971,4cyclictest0-21swapper/219:50:132
315989970,6cyclictest26759-21ssh21:55:202
315899972,4cyclictest0-21swapper/121:19:271
315899970,4cyclictest0-21swapper/122:35:101
315899970,1cyclictest0-21swapper/120:35:131
315859976,1cyclictest9-21ksoftirqd/020:15:140
315859976,1cyclictest24515-21diskmemload21:20:210
315859976,1cyclictest1546-21sh00:20:210
315859975,1cyclictest9-21ksoftirqd/019:54:590
315859975,1cyclictest631-21basename22:05:160
315859975,1cyclictest396-21kworker/0:200:25:230
315859975,1cyclictest2924-21ssh22:52:050
315859975,1cyclictest27181-21ntpq20:05:190
315859973,3cyclictest24515-21diskmemload00:37:250
315859972,3cyclictest0-21swapper/023:05:120
315859972,0cyclictest0-21swapper/019:35:140
315859971,6cyclictest32177-21ssh00:19:070
315859971,5cyclictest9017-21ssh00:30:190
315859971,5cyclictest22020-21/usr/sbin/munin00:05:270
315859970,7cyclictest0-21swapper/021:25:050
315859970,6cyclictest4504-21expr22:10:130
315859970,6cyclictest30605-21ssh21:16:270
315859970,6cyclictest24515-21diskmemload23:40:120
315859970,6cyclictest20384-21ssh22:30:340
315859970,6cyclictest18787-21ssh23:15:100
315859970,6cyclictest14817-21apt-get23:55:130
315859970,6cyclictest14817-21apt-get23:55:130
315859970,6cyclictest11207-21ssh23:50:040
315859970,6cyclictest0-21swapper/023:30:020
315859970,6cyclictest0-21swapper/022:45:200
315859970,6cyclictest0-21swapper/022:15:170
315859970,6cyclictest0-21swapper/021:54:210
315859970,3cyclictest0-21swapper/023:35:140
315859970,1cyclictest21658-21ssh21:48:590
315859970,1cyclictest17355-21taskset20:54:190
315859970,1cyclictest0-21swapper/023:25:400
315859970,1cyclictest0-21swapper/022:40:130
315859970,1cyclictest0-21swapper/022:29:050
20862270,1sleep10-21swapper/100:02:301
316059965,1cyclictest35-21ksoftirqd/319:30:003
316059965,1cyclictest27448-21ssh00:10:313
316059965,1cyclictest27092-21ntpq23:25:183
316059965,1cyclictest24440-21ssh21:51:303
316059965,1cyclictest23825-21ssh00:05:303
316059965,1cyclictest2377-21ssh00:21:013
316059965,1cyclictest19453-21cat22:30:173
316059965,1cyclictest1632-21ssh21:20:303
316059965,1cyclictest13594-21ssh00:36:273
316059965,1cyclictest12861-21ssh21:35:213
316059965,1cyclictest11225-21ssh22:20:113
316059964,0cyclictest24515-21diskmemload23:00:283
316059963,1cyclictest24515-21diskmemload23:32:313
316059962,2cyclictest254-21systemd-journal20:54:093
316059962,2cyclictest24515-21diskmemload23:37:113
316059962,2cyclictest24515-21diskmemload22:15:183
316059962,1cyclictest10613-21aten_r1power_cu20:40:113
316059961,4cyclictest18791-21/usr/sbin/munin23:15:213
316059960,6cyclictest2776-21ssh22:06:443
316059960,4cyclictest0-21swapper/300:25:133
316059960,3cyclictest5651-21sh21:26:023
316059960,3cyclictest16599-21apt-get19:45:123
316059960,3cyclictest0-21swapper/323:51:183
316059960,3cyclictest0-21swapper/321:46:103
316059960,2cyclictest0-21swapper/323:46:523
316059960,2cyclictest0-21swapper/323:10:413
316059960,2cyclictest0-21swapper/321:30:183
316059960,2cyclictest0-21swapper/321:10:363
316059960,1cyclictest22614-21df_inode22:35:133
315989966,0cyclictest60950irq/123-eno123:05:132
315989966,0cyclictest60950irq/123-eno122:50:142
315989966,0cyclictest60950irq/123-eno121:00:252
315989966,0cyclictest60950irq/123-eno120:14:522
315989966,0cyclictest60950irq/123-eno120:05:142
315989966,0cyclictest60950irq/123-eno119:40:132
315989966,0cyclictest60950irq/123-eno119:39:572
315989966,0cyclictest60950irq/123-eno119:25:162
315989966,0cyclictest60950irq/123-eno100:15:122
315989965,1cyclictest60950irq/123-eno119:15:002
315989965,1cyclictest28-21ksoftirqd/221:40:112
315989965,1cyclictest28-21ksoftirqd/220:15:162
315989964,1cyclictest5971-21ntpq19:20:212
315989964,1cyclictest28-21ksoftirqd/221:05:112
315989964,1cyclictest28-21ksoftirqd/219:45:182
315989960,1cyclictest0-21swapper/219:55:112
315899961,3cyclictest0-21swapper/120:55:191
315899961,3cyclictest0-21swapper/100:10:161
315899961,1cyclictest0-21swapper/100:32:021
315899960,6cyclictest0-21swapper/122:12:081
315899960,6cyclictest0-21swapper/100:35:181
315899960,5cyclictest0-21swapper/119:10:171
315899960,4cyclictest0-21swapper/123:17:471
315899960,3cyclictest0-21swapper/122:57:231
315899960,2cyclictest0-21swapper/123:45:181
315899960,2cyclictest0-21swapper/123:10:231
315899960,2cyclictest0-21swapper/121:20:261
315899960,0cyclictest0-21swapper/121:34:371
315899960,0cyclictest0-21swapper/119:15:001
315899960,0cyclictest0-21swapper/100:18:171
315859965,1cyclictest9-21ksoftirqd/020:05:010
315859965,1cyclictest24515-21diskmemload22:20:360
315859962,2cyclictest254-21systemd-journal19:35:010
315859961,3cyclictest0-21swapper/019:47:520
315859961,1cyclictest31800-21/usr/sbin/munin19:10:200
315859960,6cyclictest0-21swapper/020:20:260
315859960,5cyclictest7333-21ssh21:30:080
315859960,5cyclictest12946-21/usr/sbin/munin20:45:150
315859960,5cyclictest0-21swapper/022:35:120
315859960,5cyclictest0-21swapper/021:00:150
315859960,3cyclictest12726-21perf20:44:590
315859960,3cyclictest0-21swapper/023:45:170
315859960,2cyclictest0-21swapper/021:05:120
315859960,2cyclictest0-21swapper/020:30:150
315859960,2cyclictest0-21swapper/019:40:120
315859960,1cyclictest4079-21cat20:25:170
315859960,1cyclictest4079-21cat20:25:160
315859960,0cyclictest0-21swapper/020:10:110
315859960,0cyclictest0-21swapper/000:10:040
316059955,0cyclictest19493-21ssh00:00:173
316059953,1cyclictest8161-21apt-config20:35:013
316059952,1cyclictest35-21ksoftirqd/320:32:093
316059952,1cyclictest28383-21/usr/sbin/munin20:10:183
316059952,1cyclictest254-21systemd-journal20:07:233
316059951,1cyclictest0-21swapper/319:35:113
316059950,5cyclictest0-21swapper/319:23:403
316059950,5cyclictest0-21swapper/319:17:133
316059950,2cyclictest254-21systemd-journal19:11:223
316059950,2cyclictest11631-21idleruntime-cro19:35:013
316059950,1cyclictest18662-21/usr/sbin/munin20:55:153
316059950,1cyclictest0-21swapper/322:25:223
316059950,1cyclictest0-21swapper/321:05:103
316059950,1cyclictest0-21swapper/320:20:153
316059950,1cyclictest0-21swapper/319:55:173
315989955,0cyclictest60950irq/123-eno120:55:112
315989955,0cyclictest60950irq/123-eno120:40:062
315989955,0cyclictest60950irq/123-eno120:00:122
315989953,1cyclictest15549-21tr20:50:162
315989950,4cyclictest12946-21/usr/sbin/munin20:45:202
315899955,0cyclictest0-21swapper/122:24:231
315899955,0cyclictest0-21swapper/122:15:511
315899953,1cyclictest22280-21chrt21:50:111
315899952,1cyclictest0-21swapper/100:05:261
315899951,1cyclictest0-21swapper/122:07:251
315899951,1cyclictest0-21swapper/121:57:531
315899950,5cyclictest0-21swapper/120:53:011
315899950,4cyclictest0-21swapper/123:00:171
315899950,3cyclictest0-21swapper/122:45:341
315899950,3cyclictest0-21swapper/122:27:221
315899950,3cyclictest0-21swapper/121:25:021
315899950,3cyclictest0-21swapper/100:20:241
315899950,2cyclictest0-21swapper/123:55:101
315899950,2cyclictest0-21swapper/123:55:091
315899950,2cyclictest0-21swapper/123:40:111
315899950,2cyclictest0-21swapper/123:20:321
315899950,2cyclictest0-21swapper/122:01:531
315899950,2cyclictest0-21swapper/121:49:201
315899950,0cyclictest0-21swapper/121:40:481
315859952,1cyclictest13650irq/124-ahci[0020:59:550
315859950,5cyclictest0-21swapper/019:15:050
315859950,4cyclictest21302-21/usr/sbin/munin19:55:130
316059943,1cyclictest3426-21wc20:25:143
316059943,1cyclictest3426-21wc20:25:143
316059941,2cyclictest14218-21proc_pri20:45:193
316059941,1cyclictest30742-21apt-get20:15:113
316059941,0cyclictest18954-21apt-get19:50:123
315899944,0cyclictest0-21swapper/120:17:181
315899944,0cyclictest0-21swapper/120:07:161
315899944,0cyclictest0-21swapper/119:57:121
315899944,0cyclictest0-21swapper/119:34:541
315899944,0cyclictest0-21swapper/119:27:031
315899943,1cyclictest0-21swapper/120:40:131
315899943,1cyclictest0-21swapper/119:20:191
315899942,1cyclictest246832chrt20:00:171
315899942,1cyclictest1828-21chrt22:50:231
315899941,0cyclictest0-21swapper/119:45:201
315899940,4cyclictest0-21swapper/120:28:241
315899940,4cyclictest0-21swapper/120:28:231
315899940,3cyclictest0-21swapper/123:50:281
315899940,3cyclictest0-21swapper/123:35:021
315899940,3cyclictest0-21swapper/121:35:151
315899940,2cyclictest0-21swapper/123:30:131
315899940,2cyclictest0-21swapper/123:26:031
315899940,2cyclictest0-21swapper/119:40:141
315899940,1cyclictest0-21swapper/123:05:131
315899940,1cyclictest0-21swapper/122:32:491
315899940,1cyclictest0-21swapper/121:05:151
315899940,1cyclictest0-21swapper/120:30:191
315899940,0cyclictest0-21swapper/120:45:201
315899940,0cyclictest0-21swapper/119:50:201
315899931,1cyclictest0-21swapper/121:00:111
315899930,3cyclictest0-21swapper/119:35:151
315899930,2cyclictest0-21swapper/120:20:161
315899930,2cyclictest0-21swapper/120:10:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional