You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 02:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Sat Nov 22, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
181402230158,26sleep00-21swapper/019:05:570
181952212150,20sleep30-21swapper/319:06:403
183922205163,29sleep20-21swapper/219:09:122
161022197157,27sleep10-21swapper/119:05:011
314062730,0sleep20-21swapper/223:06:102
120312690,0sleep20-21swapper/222:40:142
28042580,0sleep20-21swapper/223:57:522
18556991818,0cyclictest35-21ksoftirqd/322:14:293
20562170,1sleep121-21ksoftirqd/123:10:251
18556991716,0cyclictest35-21ksoftirqd/323:23:143
18551991717,0cyclictest28-21ksoftirqd/222:35:182
215952160,0sleep10-21swapper/122:53:431
18556991615,1cyclictest35-21ksoftirqd/300:19:283
18556991515,0cyclictest35-21ksoftirqd/322:15:123
18556991414,0cyclictest35-21ksoftirqd/323:52:353
18556991414,0cyclictest35-21ksoftirqd/323:01:233
18556991414,0cyclictest35-21ksoftirqd/322:58:363
18556991411,2cyclictest28887-21munin-run00:35:013
18556991313,0cyclictest35-21ksoftirqd/323:41:333
18556991313,0cyclictest35-21ksoftirqd/323:12:393
18556991313,0cyclictest35-21ksoftirqd/322:52:363
18556991313,0cyclictest35-21ksoftirqd/322:36:223
18556991313,0cyclictest35-21ksoftirqd/322:32:013
18556991313,0cyclictest35-21ksoftirqd/321:25:373
18556991313,0cyclictest35-21ksoftirqd/300:35:273
18556991310,1cyclictest35-21ksoftirqd/321:47:263
1855699128,2cyclictest254-21systemd-journal23:57:183
18556991212,0cyclictest35-21ksoftirqd/322:42:593
18556991212,0cyclictest35-21ksoftirqd/322:22:193
18556991212,0cyclictest35-21ksoftirqd/321:56:533
18556991212,0cyclictest35-21ksoftirqd/300:21:203
18556991212,0cyclictest35-21ksoftirqd/300:01:113
18556991210,1cyclictest5423-21fschecks_time20:55:143
18556991210,1cyclictest35-21ksoftirqd/321:32:243
18556991210,1cyclictest31050-21runrttasks19:33:083
18551991211,1cyclictest28-21ksoftirqd/222:15:002
18551991210,1cyclictest28-21ksoftirqd/222:50:242
1854299122,5cyclictest12030-21wc21:55:000
1855699119,1cyclictest7638-21aten_r1power_en00:05:133
1855699119,1cyclictest11685-21diskmemload23:39:383
1855699118,2cyclictest4698-21diskstats23:15:133
18556991111,0cyclictest35-21ksoftirqd/323:30:373
18556991111,0cyclictest35-21ksoftirqd/323:25:053
18556991111,0cyclictest35-21ksoftirqd/323:25:043
18556991111,0cyclictest35-21ksoftirqd/322:06:153
18556991111,0cyclictest35-21ksoftirqd/322:01:223
18556991111,0cyclictest35-21ksoftirqd/321:39:173
18556991111,0cyclictest35-21ksoftirqd/300:26:473
18556991111,0cyclictest35-21ksoftirqd/300:13:203
18556991110,1cyclictest35-21ksoftirqd/322:49:283
18556991110,1cyclictest35-21ksoftirqd/322:25:343
18556991110,1cyclictest35-21ksoftirqd/321:51:423
18556991110,1cyclictest35-21ksoftirqd/321:16:083
18556991110,1cyclictest11685-21diskmemload21:10:203
1855699111,9cyclictest31197-21ssh23:05:293
1855199119,1cyclictest28-21ksoftirqd/220:00:002
1855199118,1cyclictest28-21ksoftirqd/220:44:592
18551991110,1cyclictest28-21ksoftirqd/223:10:242
18551991110,1cyclictest28-21ksoftirqd/200:00:122
1855699109,1cyclictest0-21swapper/320:45:143
1855699109,0cyclictest35-21ksoftirqd/319:15:013
1855699105,4cyclictest341rcuc/321:40:023
18556991010,0cyclictest35-21ksoftirqd/323:45:093
18556991010,0cyclictest35-21ksoftirqd/321:20:093
18556991010,0cyclictest35-21ksoftirqd/319:38:203
1855699101,8cyclictest10607-21apt-get20:00:013
1855699101,1cyclictest18903-21cpu19:10:113
1855699100,9cyclictest19236-21irqstats20:15:153
1855699100,2cyclictest3452-21memory20:50:153
1855199109,1cyclictest28-21ksoftirqd/222:15:142
1855199109,1cyclictest28-21ksoftirqd/221:50:132
1855199109,1cyclictest28-21ksoftirqd/200:40:012
1855199108,1cyclictest28-21ksoftirqd/223:50:112
1855199108,1cyclictest28-21ksoftirqd/221:40:182
1855199108,1cyclictest28-21ksoftirqd/221:30:142
1855199108,1cyclictest28-21ksoftirqd/220:50:002
1854299108,1cyclictest9-21ksoftirqd/019:54:590
185569993,4cyclictest6116-21apt-get19:50:113
185569991,8cyclictest35-21ksoftirqd/319:44:203
185569991,1cyclictest35-21ksoftirqd/320:24:243
185569991,1cyclictest35-21ksoftirqd/319:29:193
185569991,0cyclictest35-21ksoftirqd/320:06:073
185569990,1cyclictest25829-21expr20:30:133
185569990,1cyclictest0-21swapper/319:20:193
185519999,0cyclictest28-21ksoftirqd/222:30:142
185519998,1cyclictest28-21ksoftirqd/223:45:212
185519998,1cyclictest28-21ksoftirqd/223:30:202
185519997,2cyclictest18538-21cyclictest00:18:442
185519990,8cyclictest0-21swapper/222:00:182
185519990,0cyclictest0-21swapper/222:57:222
185429998,1cyclictest9-21ksoftirqd/020:50:200
185429995,1cyclictest254-21systemd-journal21:15:000
185429994,4cyclictest11685-21diskmemload23:40:530
185429992,3cyclictest20516-21perf20:20:000
185429990,6cyclictest26928-21kworker/0:119:45:140
185429990,6cyclictest254-21systemd-journal00:40:000
185429990,5cyclictest0-21swapper/023:03:220
185429990,5cyclictest0-21swapper/000:01:370
8513280,0sleep10-21swapper/123:20:151
185569988,0cyclictest35-21ksoftirqd/320:40:123
185569988,0cyclictest35-21ksoftirqd/320:25:143
185569988,0cyclictest35-21ksoftirqd/320:10:183
185569988,0cyclictest35-21ksoftirqd/319:55:143
185569988,0cyclictest0-21swapper/321:00:413
185569988,0cyclictest0-21swapper/319:45:213
185569981,7cyclictest341rcuc/321:05:113
185569981,0cyclictest35-21ksoftirqd/320:35:113
185519988,0cyclictest28-21ksoftirqd/200:10:142
185519987,1cyclictest28-21ksoftirqd/222:45:212
185519987,1cyclictest28-21ksoftirqd/222:09:062
185519987,1cyclictest28-21ksoftirqd/221:16:392
185519987,1cyclictest28-21ksoftirqd/220:30:152
185519987,1cyclictest28-21ksoftirqd/219:25:222
185519987,1cyclictest28-21ksoftirqd/219:10:182
185519987,1cyclictest28-21ksoftirqd/200:30:132
185519986,1cyclictest8493-21gltestperf23:20:142
185519986,1cyclictest28-21ksoftirqd/221:45:192
185519986,1cyclictest28-21ksoftirqd/220:37:172
185519986,1cyclictest28-21ksoftirqd/219:40:002
185519986,1cyclictest28-21ksoftirqd/200:29:572
185519986,1cyclictest28-21ksoftirqd/200:05:212
185519981,6cyclictest819-21lldpd22:21:302
185519980,4cyclictest254-21systemd-journal21:06:592
185449987,0cyclictest24802sleep123:56:521
185449980,5cyclictest0-21swapper/120:05:121
185429984,0cyclictest0-21swapper/000:30:210
185429981,1cyclictest32511-21/usr/sbin/munin20:45:160
185429980,7cyclictest20634-21ssh00:21:360
185429980,7cyclictest19350-21apt-get22:05:110
185429980,7cyclictest11832-21ssh00:10:140
185429980,6cyclictest0-21swapper/020:05:120
185429980,4cyclictest8545-21/usr/sbin/munin23:20:150
185429980,4cyclictest11685-21diskmemload23:10:010
185519977,0cyclictest28-21ksoftirqd/223:15:142
185519977,0cyclictest28-21ksoftirqd/222:25:202
185519976,1cyclictest60950irq/123-eno123:40:172
185519976,1cyclictest60950irq/123-eno121:10:132
185519976,1cyclictest28-21ksoftirqd/221:55:162
185519975,1cyclictest60950irq/123-eno120:15:142
185519975,1cyclictest28-21ksoftirqd/223:25:202
185519975,1cyclictest28-21ksoftirqd/223:25:202
185519975,1cyclictest28-21ksoftirqd/221:35:212
185519975,1cyclictest28-21ksoftirqd/220:25:132
185519975,1cyclictest20824-21ssh23:37:002
185519972,2cyclictest254-21systemd-journal23:03:312
185449971,3cyclictest0-21swapper/122:05:101
185449970,6cyclictest0-21swapper/119:15:171
185449970,4cyclictest0-21swapper/123:49:471
185449970,4cyclictest0-21swapper/100:00:121
185429976,1cyclictest12096-1kworker/u9:221:13:230
185429975,1cyclictest26500-21dump-pmu-power19:25:000
185429975,1cyclictest1186-21latency_hist19:40:000
185429974,2cyclictest31240-21strings20:40:140
185429972,2cyclictest0-21swapper/019:55:180
185429971,6cyclictest29321-1kworker/u9:023:10:210
185429971,5cyclictest22888-21/usr/sbin/munin22:10:190
185429970,7cyclictest0-21swapper/021:30:210
185429970,6cyclictest8257-21ssh00:05:150
185429970,6cyclictest8094-21ssh22:35:030
185429970,6cyclictest6745-21ssh22:31:300
185429970,6cyclictest5652-21ssh21:45:130
185429970,6cyclictest3803-21ssh21:41:220
185429970,6cyclictest31343-21ssh23:51:440
185429970,6cyclictest31340-21ssh22:20:180
185429970,6cyclictest29301-21ssh22:18:510
185429970,6cyclictest25613-21cut23:45:120
185429970,6cyclictest2134-21ssh22:25:180
185429970,6cyclictest21048-21ssh23:37:420
185429970,6cyclictest18813-21sh22:50:150
185429970,6cyclictest17832-21ssh00:19:040
185429970,6cyclictest14487-21ssh21:56:430
185429970,6cyclictest13300-1kworker/u9:023:30:040
185429970,6cyclictest11685-21diskmemload23:55:140
185429970,6cyclictest11660-21latency_hist21:10:000
185429970,6cyclictest0-21swapper/023:15:120
185429970,6cyclictest0-21swapper/022:40:130
185429970,6cyclictest0-21swapper/021:25:060
185429970,4cyclictest1473-21perf21:40:000
185429970,3cyclictest0-21swapper/021:20:120
185429970,1cyclictest27903-21ntpq19:25:190
185429970,1cyclictest15783-21ssh22:00:040
185429970,1cyclictest13851-21ssh23:27:570
185429970,1cyclictest13851-21ssh23:27:570
185429970,1cyclictest11685-21diskmemload00:26:560
185519966,0cyclictest60950irq/123-eno121:25:192
185519966,0cyclictest60950irq/123-eno121:02:132
185519966,0cyclictest60950irq/123-eno120:56:312
185519966,0cyclictest60950irq/123-eno119:32:112
185519966,0cyclictest60950irq/123-eno100:20:132
185519966,0cyclictest28-21ksoftirqd/219:20:212
185519965,1cyclictest28-21ksoftirqd/221:20:152
185519965,1cyclictest28-21ksoftirqd/220:20:162
185519965,1cyclictest28-21ksoftirqd/220:05:152
185519965,1cyclictest28-21ksoftirqd/219:45:132
185519965,1cyclictest17096-21ntpq20:10:162
185519964,1cyclictest28-21ksoftirqd/219:15:012
185519963,2cyclictest0-21swapper/219:50:132
185519962,2cyclictest570-21systemd-logind19:40:232
185449966,0cyclictest0-21swapper/100:07:211
185449963,1cyclictest0-21swapper/122:17:591
185449962,2cyclictest25216-21taskset22:12:061
185449962,2cyclictest180772sleep122:01:531
185449960,6cyclictest0-21swapper/122:57:151
185449960,4cyclictest0-21swapper/123:33:231
185449960,4cyclictest0-21swapper/121:57:161
185449960,4cyclictest0-21swapper/121:50:091
185449960,4cyclictest0-21swapper/121:46:341
185449960,3cyclictest0-21swapper/123:53:011
185449960,3cyclictest0-21swapper/123:40:171
185449960,3cyclictest0-21swapper/120:40:101
185449960,2cyclictest0-21swapper/100:15:001
185449960,1cyclictest0-21swapper/120:00:141
185449960,0cyclictest0-21swapper/122:39:271
185429965,1cyclictest30220-21ntpq19:30:190
185429965,1cyclictest19892-21ntpq19:10:160
185429961,3cyclictest0-21swapper/022:55:110
185429961,2cyclictest0-21swapper/020:25:110
185429960,6cyclictest0-21swapper/020:34:230
185429960,6cyclictest0-21swapper/020:21:320
185429960,6cyclictest0-21swapper/020:11:290
185429960,5cyclictest22152-21cstates19:15:150
185429960,5cyclictest17347-21ssh22:46:280
185429960,5cyclictest0-21swapper/020:35:140
185429960,1cyclictest1396-21/usr/sbin/munin19:40:180
185519953,1cyclictest28-21ksoftirqd/220:00:172
185519950,5cyclictest0-21swapper/220:54:262
185449955,0cyclictest0-21swapper/121:08:211
185449951,3cyclictest0-21swapper/123:16:021
185449951,2cyclictest0-21swapper/122:45:151
185449950,5cyclictest0-21swapper/119:50:161
185449950,5cyclictest0-21swapper/119:22:491
185449950,4cyclictest0-21swapper/121:15:171
185449950,3cyclictest0-21swapper/121:36:001
185449950,3cyclictest0-21swapper/119:14:591
185449950,2cyclictest0-21swapper/123:36:271
185449950,2cyclictest0-21swapper/123:28:271
185449950,2cyclictest0-21swapper/123:28:271
185449950,2cyclictest0-21swapper/122:20:121
185449950,2cyclictest0-21swapper/120:25:111
185449950,1cyclictest0-21swapper/123:00:131
185449950,1cyclictest0-21swapper/122:40:121
185449950,1cyclictest0-21swapper/122:25:031
185449950,1cyclictest0-21swapper/121:00:121
185449950,1cyclictest0-21swapper/100:15:181
185449950,0cyclictest0-21swapper/121:11:501
185449950,0cyclictest0-21swapper/120:37:171
185429953,1cyclictest7564-21cat21:00:140
185429951,2cyclictest254-21systemd-journal20:05:010
185429950,2cyclictest0-21swapper/020:55:120
185449944,0cyclictest0-21swapper/120:53:401
185449944,0cyclictest0-21swapper/119:35:111
185449942,1cyclictest28714-21taskset19:28:211
185449941,2cyclictest0-21swapper/122:30:111
185449940,3cyclictest183932sleep100:20:021
185449940,3cyclictest0-21swapper/123:05:081
185449940,3cyclictest0-21swapper/121:25:131
185449940,3cyclictest0-21swapper/100:30:421
185449940,2cyclictest0-21swapper/121:30:231
185449940,2cyclictest0-21swapper/121:21:161
185449940,2cyclictest0-21swapper/100:35:081
185449940,2cyclictest0-21swapper/100:25:141
185449940,1cyclictest0-21swapper/120:30:121
185449940,1cyclictest0-21swapper/120:20:141
185449940,1cyclictest0-21swapper/120:00:011
185449940,1cyclictest0-21swapper/119:45:111
185449940,1cyclictest0-21swapper/119:30:171
185449940,0cyclictest0-21swapper/121:41:081
185449940,0cyclictest0-21swapper/120:15:191
185449931,1cyclictest0-21swapper/120:45:151
185449931,1cyclictest0-21swapper/119:40:151
185449930,2cyclictest0-21swapper/120:55:251
185449930,2cyclictest0-21swapper/120:10:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional