You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 18:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Tue Aug 26, 2025 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
269442210159,25sleep00-21swapper/007:05:560
251512209160,24sleep30-21swapper/307:05:113
269312201160,27sleep20-21swapper/207:05:472
271532198161,24sleep10-21swapper/107:08:401
42912670,0sleep10-21swapper/112:24:501
35552590,0sleep20-21swapper/211:39:512
27341992019,1cyclictest28-21ksoftirqd/211:40:142
27341992019,1cyclictest28-21ksoftirqd/211:40:132
320922140,0sleep10-21swapper/111:34:121
27348991210,1cyclictest35-21ksoftirqd/308:49:593
27341991211,1cyclictest28-21ksoftirqd/211:05:162
27341991211,1cyclictest28-21ksoftirqd/211:05:152
27348991110,1cyclictest35-21ksoftirqd/312:15:193
27348991110,1cyclictest35-21ksoftirqd/312:00:203
2734899110,7cyclictest254-21systemd-journal10:50:013
2734199119,1cyclictest28-21ksoftirqd/212:31:102
2734199119,1cyclictest28-21ksoftirqd/211:50:202
2734199119,1cyclictest28-21ksoftirqd/211:10:182
2734199119,1cyclictest28-21ksoftirqd/210:40:232
2734199119,1cyclictest28-21ksoftirqd/209:49:282
2734199119,1cyclictest28-21ksoftirqd/209:30:232
2734199119,1cyclictest28-21ksoftirqd/209:20:172
27341991110,1cyclictest28-21ksoftirqd/212:25:202
27341991110,1cyclictest28-21ksoftirqd/210:35:222
27341991110,1cyclictest28-21ksoftirqd/207:40:012
2734199110,11cyclictest0-21swapper/212:15:152
2734199110,0cyclictest0-21swapper/210:32:162
2732899118,2cyclictest0-21swapper/008:55:120
2732899114,6cyclictest9-21ksoftirqd/010:14:270
2732899112,4cyclictest20060-21perf10:35:010
2732899112,4cyclictest16870-21date07:54:590
2734899109,1cyclictest35-21ksoftirqd/312:25:013
2734899109,1cyclictest35-21ksoftirqd/312:15:003
2734899109,1cyclictest35-21ksoftirqd/309:10:173
2734899109,1cyclictest35-21ksoftirqd/309:10:173
27348991010,0cyclictest35-21ksoftirqd/311:25:193
2734199109,1cyclictest28-21ksoftirqd/209:35:162
2734199109,1cyclictest28-21ksoftirqd/209:15:202
2734199108,1cyclictest28-21ksoftirqd/211:55:142
2734199108,1cyclictest28-21ksoftirqd/211:00:252
2734199108,1cyclictest28-21ksoftirqd/207:35:002
2734199100,9cyclictest29697-21munin-node10:48:342
2734199100,7cyclictest0-21swapper/211:48:592
2734199100,10cyclictest0-21swapper/212:36:272
2734199100,0cyclictest0-21swapper/209:27:432
2733699100,9cyclictest0-21swapper/112:15:151
273489998,1cyclictest35-21ksoftirqd/312:35:143
273489998,1cyclictest35-21ksoftirqd/307:40:013
273489998,1cyclictest20196-21diskmemload12:25:193
273489998,0cyclictest35-21ksoftirqd/308:20:013
273489990,7cyclictest0-21swapper/309:39:563
273489990,6cyclictest20196-21diskmemload11:40:223
273489990,6cyclictest20196-21diskmemload11:40:213
273489990,5cyclictest17741-21needreboot10:30:233
273419998,1cyclictest28-21ksoftirqd/212:05:142
273419997,1cyclictest28-21ksoftirqd/209:50:152
273419994,2cyclictest570-21systemd-logind11:32:082
273419993,2cyclictest556-21rs:main2
273289993,5cyclictest9-21ksoftirqd/011:42:330
273289993,5cyclictest9-21ksoftirqd/011:42:320
273289993,5cyclictest26280-21latency_hist08:15:000
273289990,4cyclictest29772-21cat07:15:000
273289990,1cyclictest254-21systemd-journal12:03:320
273489988,0cyclictest35-21ksoftirqd/311:25:003
273489988,0cyclictest35-21ksoftirqd/309:00:143
273489987,1cyclictest35-21ksoftirqd/309:50:003
273489987,1cyclictest35-21ksoftirqd/309:40:163
273489986,1cyclictest35-21ksoftirqd/309:55:173
273489982,5cyclictest30255-21apt-get10:05:143
273489980,5cyclictest0-21swapper/309:53:083
273419988,0cyclictest28-21ksoftirqd/210:13:122
273419988,0cyclictest28-21ksoftirqd/210:05:162
273419987,1cyclictest28-21ksoftirqd/212:04:502
273419987,1cyclictest28-21ksoftirqd/210:02:092
273419987,1cyclictest28-21ksoftirqd/207:50:132
273419986,1cyclictest60950irq/123-eno111:25:162
273419986,1cyclictest60950irq/123-eno110:20:142
273419986,1cyclictest60950irq/123-eno110:17:472
273419986,1cyclictest28-21ksoftirqd/211:20:122
273419986,1cyclictest28-21ksoftirqd/210:55:152
273419986,1cyclictest28-21ksoftirqd/208:35:162
273419980,7cyclictest1592-21/usr/sbin/munin08:30:122
273369980,4cyclictest0-21swapper/109:30:211
273289988,0cyclictest9-21ksoftirqd/011:35:590
273289987,1cyclictest9-21ksoftirqd/008:25:000
273289983,4cyclictest0-21swapper/011:55:130
273289981,6cyclictest11134-21ssh10:20:470
273289981,1cyclictest0-21swapper/009:15:120
273289980,5cyclictest0-21swapper/011:02:350
273289980,2cyclictest10665-21ssh11:49:060
273489977,0cyclictest35-21ksoftirqd/308:55:143
273489976,1cyclictest35-21ksoftirqd/311:15:243
273489976,1cyclictest35-21ksoftirqd/310:15:163
273489976,1cyclictest35-21ksoftirqd/309:10:003
273489975,1cyclictest35-21ksoftirqd/308:15:013
273489975,1cyclictest35-21ksoftirqd/307:54:593
273489975,1cyclictest26572-21ssh10:41:583
273489971,4cyclictest20196-21diskmemload12:33:593
273489970,6cyclictest31190-21apt-get08:25:013
273489970,3cyclictest0-21swapper/311:51:563
273489970,2cyclictest0-21swapper/311:35:133
273489970,1cyclictest0-21swapper/309:23:433
273419977,0cyclictest28-21ksoftirqd/208:00:152
273419976,1cyclictest60950irq/123-eno107:10:172
273419976,1cyclictest28-21ksoftirqd/208:45:192
273419976,1cyclictest28-21ksoftirqd/208:10:152
273419976,1cyclictest28-21ksoftirqd/207:55:162
273419975,1cyclictest60950irq/123-eno110:50:262
273419975,1cyclictest28-21ksoftirqd/212:20:022
273419975,1cyclictest28-21ksoftirqd/211:15:172
273419975,1cyclictest28-21ksoftirqd/209:05:192
273419975,1cyclictest28-21ksoftirqd/208:50:142
273419975,1cyclictest28-21ksoftirqd/207:40:142
273419973,2cyclictest254-21systemd-journal09:15:012
273419973,2cyclictest254-21systemd-journal09:15:002
273419973,2cyclictest254-21systemd-journal08:25:022
273419970,6cyclictest0-21swapper/212:10:072
273369976,1cyclictest21-21ksoftirqd/112:08:411
273369971,4cyclictest0-21swapper/109:40:371
273369971,2cyclictest0-21swapper/111:26:141
273369970,6cyclictest0-21swapper/107:55:161
273369970,4cyclictest0-21swapper/111:40:211
273369970,4cyclictest0-21swapper/111:40:201
273369970,3cyclictest0-21swapper/112:12:161
273369970,2cyclictest0-21swapper/110:45:471
273369970,1cyclictest0-21swapper/109:55:151
273289976,1cyclictest15756-1kworker/u9:010:29:340
273289976,1cyclictest15756-1kworker/u9:010:29:330
273289974,1cyclictest20196-21diskmemload11:07:130
273289974,1cyclictest20196-21diskmemload11:07:120
273289972,2cyclictest20196-21diskmemload10:55:030
273289971,5cyclictest8311-21/usr/sbin/munin12:30:250
273289971,5cyclictest3788-21/usr/sbin/munin09:30:210
273289971,2cyclictest20196-21diskmemload09:29:530
273289970,6cyclictest7800-21aten_r1power_vo09:35:130
273289970,6cyclictest6503-21ssh12:25:340
273289970,6cyclictest32029-21sort10:50:140
273289970,6cyclictest31961-21ssh11:33:520
273289970,6cyclictest28423-21ssh09:20:150
273289970,6cyclictest25441-21aten_r1power_vo11:25:130
273289970,6cyclictest24787-1kworker/u9:212:10:060
273289970,6cyclictest23683-21ssh09:14:400
273289970,6cyclictest23683-21ssh09:14:400
273289970,6cyclictest20300-21ssh10:35:110
273289970,6cyclictest17864-21ssh11:15:110
273289970,6cyclictest17675-21ssh09:46:400
273289970,6cyclictest1496-21ssh12:20:160
273289970,6cyclictest12941-21cat12:35:170
273289970,6cyclictest11592-21aten2.4-expect09:40:140
273289970,6cyclictest0-21swapper/012:15:020
273289970,6cyclictest0-21swapper/010:05:070
273289970,6cyclictest0-21swapper/009:55:150
273289970,6cyclictest0-21swapper/009:50:150
273289970,4cyclictest26552-21/usr/sbin/munin10:00:160
273289970,4cyclictest14694-21cat11:55:010
273289970,3cyclictest0-21swapper/010:40:160
273289970,2cyclictest21931-21cstates11:20:140
273289970,1cyclictest8095-21ssh10:18:210
273289970,1cyclictest5405-21aten2.4-expect07:30:120
273289970,1cyclictest254-21systemd-journal07:40:010
273289970,1cyclictest20196-21diskmemload11:10:200
273489966,0cyclictest35-21ksoftirqd/311:05:153
273489966,0cyclictest35-21ksoftirqd/311:05:143
273489966,0cyclictest35-21ksoftirqd/310:20:013
273489965,1cyclictest35-21ksoftirqd/310:55:153
273489965,1cyclictest35-21ksoftirqd/308:55:003
273489965,1cyclictest35-21ksoftirqd/307:15:013
273489965,1cyclictest35-21ksoftirqd/307:15:003
273489965,1cyclictest30851-21ssh10:48:403
273489965,1cyclictest24137-21ssh12:05:263
273489965,1cyclictest23585-21ssh10:39:413
273489965,1cyclictest14916-21ssh10:26:023
273489965,1cyclictest14916-21ssh10:26:013
273489964,1cyclictest35-21ksoftirqd/308:35:153
273489964,1cyclictest35-21ksoftirqd/308:00:163
273489964,1cyclictest28987-21aten_r1power_cu11:30:133
273489964,1cyclictest20196-21diskmemload09:16:133
273489964,0cyclictest20196-21diskmemload11:45:153
273489964,0cyclictest20196-21diskmemload09:26:243
273489962,3cyclictest2900-21apt-get07:25:123
273489961,4cyclictest26552-21/usr/sbin/munin10:00:163
273489961,2cyclictest0-21swapper/311:55:113
273489960,6cyclictest0-21swapper/311:10:553
273489960,5cyclictest0-21swapper/311:01:513
273489960,3cyclictest0-21swapper/307:40:163
273489960,1cyclictest0-21swapper/309:33:573
273489960,1cyclictest0-21swapper/307:55:133
273419966,0cyclictest60950irq/123-eno110:25:222
273419966,0cyclictest60950irq/123-eno110:25:212
273419966,0cyclictest60950irq/123-eno109:55:022
273419966,0cyclictest60950irq/123-eno108:59:372
273419966,0cyclictest60950irq/123-eno108:21:252
273419964,1cyclictest28-21ksoftirqd/209:00:212
273419962,3cyclictest12675-21netstat09:40:182
273419961,3cyclictest6287-21apt-get08:40:112
273419960,0cyclictest0-21swapper/208:05:142
273369966,0cyclictest0-21swapper/110:14:201
273369961,3cyclictest0-21swapper/109:25:111
273369960,6cyclictest0-21swapper/108:23:221
273369960,4cyclictest0-21swapper/110:05:111
273369960,3cyclictest0-21swapper/111:23:281
273369960,3cyclictest0-21swapper/111:02:431
273369960,3cyclictest0-21swapper/109:46:061
273369960,3cyclictest0-21swapper/108:55:131
273369960,2cyclictest0-21swapper/112:25:171
273369960,2cyclictest0-21swapper/112:04:241
273369960,2cyclictest0-21swapper/111:55:121
273369960,0cyclictest0-21swapper/111:38:201
273289965,1cyclictest20309-21cat08:00:160
273289964,1cyclictest18120-21grep07:55:180
273289963,1cyclictest254-21systemd-journal07:45:000
273289961,2cyclictest0-21swapper/008:25:110
273289960,6cyclictest0-21swapper/008:39:570
273289960,5cyclictest26253-21apt-get08:15:010
273289960,5cyclictest22453-21telnet12:05:120
273289960,5cyclictest0-21swapper/007:20:150
273289960,1cyclictest20196-21diskmemload10:45:180
273289960,1cyclictest0-21swapper/008:40:120
273489954,1cyclictest321-21ntpq08:25:173
273489954,1cyclictest1496-21ssh10:10:093
273489953,1cyclictest5246-21taskset07:30:083
273489953,1cyclictest35-21ksoftirqd/307:20:153
273489951,1cyclictest0-21swapper/307:45:133
273419955,0cyclictest60950irq/123-eno107:20:162
273419954,1cyclictest28-21ksoftirqd/207:45:152
273419953,1cyclictest2883-21date07:25:002
273419953,1cyclictest1989-21kworker/2:107:15:002
273369955,0cyclictest0-21swapper/111:17:581
273369955,0cyclictest0-21swapper/110:40:041
273369953,1cyclictest0-21swapper/112:35:171
273369951,3cyclictest0-21swapper/110:53:211
273369951,1cyclictest0-21swapper/107:15:181
273369951,0cyclictest0-21swapper/109:00:141
273369950,5cyclictest0-21swapper/111:11:591
273369950,5cyclictest0-21swapper/110:33:221
273369950,5cyclictest0-21swapper/107:52:101
273369950,4cyclictest0-21swapper/110:21:591
273369950,4cyclictest0-21swapper/109:35:331
273369950,4cyclictest0-21swapper/107:40:121
273369950,3cyclictest0-21swapper/111:45:511
273369950,3cyclictest0-21swapper/110:28:511
273369950,3cyclictest0-21swapper/110:28:501
273369950,3cyclictest0-21swapper/109:15:131
273369950,3cyclictest0-21swapper/109:10:051
273369950,3cyclictest0-21swapper/109:10:041
273369950,3cyclictest0-21swapper/107:48:261
273369950,2cyclictest0-21swapper/110:15:211
273369950,2cyclictest0-21swapper/110:00:481
273369950,2cyclictest0-21swapper/109:22:011
273369950,1cyclictest0-21swapper/111:05:161
273369950,1cyclictest0-21swapper/111:05:161
273289955,0cyclictest0-21swapper/007:45:220
273289954,1cyclictest16853-21ntpq09:00:180
273289953,1cyclictest9-21ksoftirqd/007:15:130
273289951,3cyclictest0-21swapper/008:30:120
273289950,5cyclictest10044-21python08:45:190
273289950,4cyclictest2897-21/usr/sbin/munin07:25:160
273289950,2cyclictest13102-21cron08:55:000
273289950,1cyclictest18078-21/usr/sbin/munin09:05:200
273289950,1cyclictest0-21swapper/008:05:150
273489941,2cyclictest21777-21apt-get08:05:133
273489941,1cyclictest0-21swapper/308:30:143
273489940,0cyclictest0-21swapper/308:40:123
273369942,1cyclictest21516-21chrt08:04:541
273369942,1cyclictest0-21swapper/112:30:421
273369941,1cyclictest0-21swapper/110:55:131
273369941,1cyclictest0-21swapper/108:45:101
273369941,0cyclictest0-21swapper/107:20:201
273369940,3cyclictest0-21swapper/111:50:571
273369940,3cyclictest0-21swapper/108:40:121
273369940,3cyclictest0-21swapper/107:25:151
273369940,2cyclictest0-21swapper/109:50:181
273369940,2cyclictest0-21swapper/108:30:131
273369940,2cyclictest0-21swapper/107:35:141
273369940,1cyclictest0-21swapper/110:35:011
273369940,1cyclictest0-21swapper/108:15:001
273369931,1cyclictest0-21swapper/108:35:161
273369931,1cyclictest0-21swapper/108:25:111
273369931,1cyclictest0-21swapper/108:14:261
273369931,1cyclictest0-21swapper/108:05:131
273369931,1cyclictest0-21swapper/107:30:111
273369931,1cyclictest0-21swapper/107:10:121
273369930,1cyclictest0-21swapper/109:05:181
273369930,0cyclictest0-21swapper/108:50:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional