You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 04:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Tue May 13, 2025 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162662229157,25sleep30-21swapper/319:06:553
144002229156,24sleep00-21swapper/019:05:110
143922202163,26sleep20-21swapper/219:05:062
163762200161,26sleep10-21swapper/119:08:231
2596921360,0sleep225971-21ssh00:15:182
52472780,0sleep10-21swapper/100:30:201
215292620,0sleep00-21swapper/023:29:340
1659499150,14cyclictest9196-21diskmemload23:00:192
1659499150,14cyclictest0-21swapper/222:50:132
1659499150,14cyclictest0-21swapper/222:00:132
1659499150,1cyclictest0-21swapper/221:00:122
1659499140,8cyclictest254-21systemd-journal20:25:012
1659499140,13cyclictest0-21swapper/223:30:182
1659499140,13cyclictest0-21swapper/223:10:162
1659499140,13cyclictest0-21swapper/222:55:172
1659499140,13cyclictest0-21swapper/222:20:202
1659499140,13cyclictest0-21swapper/200:22:142
1659499140,1cyclictest0-21swapper/221:44:572
1659499132,10cyclictest22215-21apt-get19:20:162
1659499130,12cyclictest0-21swapper/200:10:192
1659499130,11cyclictest31556-21cat21:35:222
1659499130,1cyclictest0-21swapper/219:55:202
1659499130,0cyclictest0-21swapper/222:05:192
1659499130,0cyclictest0-21swapper/200:25:192
1657599132,5cyclictest5139-21latency_hist21:44:590
16575991313,0cyclictest0-21swapper/020:35:400
16575991311,1cyclictest9-21ksoftirqd/023:54:590
16575991311,1cyclictest9-21ksoftirqd/021:15:000
1659499123,8cyclictest6478-21ssh00:31:312
1659499122,9cyclictest4302-21stty22:25:152
1659499122,9cyclictest26666-21cron19:30:002
1659499120,12cyclictest0-21swapper/221:52:142
1659499120,1cyclictest16861-21apt-get00:05:132
1659499120,1cyclictest0-21swapper/221:59:112
1659499120,0cyclictest9196-21diskmemload23:35:162
1659499120,0cyclictest9196-21diskmemload21:15:232
1659499120,0cyclictest11839-21ssh22:35:062
1659499120,0cyclictest0-21swapper/221:20:132
1659499120,0cyclictest0-21swapper/220:50:212
1659499120,0cyclictest0-21swapper/200:38:042
16575991211,1cyclictest9-21ksoftirqd/023:50:010
1659499110,10cyclictest0-21swapper/223:43:062
1659499110,10cyclictest0-21swapper/223:18:502
1659499110,10cyclictest0-21swapper/222:45:182
1659499110,1cyclictest21750-21apt-get21:25:112
1659499110,1cyclictest0-21swapper/223:29:202
1659499110,1cyclictest0-21swapper/223:05:202
1659499110,1cyclictest0-21swapper/222:44:182
1659499110,0cyclictest0-21swapper/223:59:082
1659499110,0cyclictest0-21swapper/223:51:072
1659499110,0cyclictest0-21swapper/223:46:202
1659499110,0cyclictest0-21swapper/223:20:582
1659499110,0cyclictest0-21swapper/222:32:112
1659499110,0cyclictest0-21swapper/222:18:032
1659499110,0cyclictest0-21swapper/200:03:302
1657599119,1cyclictest9-21ksoftirqd/023:05:000
1660399109,1cyclictest557-21nscd00:21:433
1660399100,5cyclictest11890-21ssh22:35:133
1659499102,7cyclictest3687-21latency_hist19:50:002
1659499100,9cyclictest0-21swapper/221:30:242
1659499100,9cyclictest0-21swapper/221:10:562
1659499100,8cyclictest23998-21memory20:30:192
1659499100,1cyclictest7466-21ssh21:46:062
1659499100,0cyclictest0-21swapper/222:10:092
1659499100,0cyclictest0-21swapper/219:10:122
1658699100,9cyclictest0-21swapper/122:45:181
1657599108,1cyclictest9-21ksoftirqd/022:39:590
1657599108,1cyclictest9-21ksoftirqd/022:00:010
1657599103,3cyclictest17905-21perf23:25:000
166039999,0cyclictest35-21ksoftirqd/323:03:543
166039997,1cyclictest35-21ksoftirqd/320:43:313
166039992,7cyclictest1500-21ntp_states00:25:213
166039992,3cyclictest557-21nscd22:50:163
165949996,3cyclictest60950irq/123-eno121:05:192
165949994,4cyclictest29416-21aten2.4-expect19:35:142
165949994,4cyclictest29416-21aten2.4-expect19:35:132
165949990,9cyclictest0-21swapper/220:25:242
165949990,6cyclictest254-21systemd-journal19:20:002
165949990,1cyclictest3924-21apt-get19:50:142
165949990,1cyclictest27164-21cpuspeed_turbos19:30:152
165949990,0cyclictest15982-21/usr/sbin/munin20:15:112
165759998,1cyclictest9-21ksoftirqd/022:02:050
165759998,1cyclictest9-21ksoftirqd/019:35:000
165759998,1cyclictest12356-21ssh23:59:150
165759997,1cyclictest9-21ksoftirqd/019:15:010
165759997,1cyclictest9-21ksoftirqd/000:10:170
165759990,6cyclictest21998-21/usr/sbin/munin23:30:140
166039987,1cyclictest35-21ksoftirqd/319:50:153
166039986,1cyclictest35-21ksoftirqd/320:27:313
166039982,3cyclictest841-21nscd21:33:433
166039982,2cyclictest9196-21diskmemload23:20:373
165949981,6cyclictest13357-21apt-get20:10:142
165949980,8cyclictest0-21swapper/220:35:232
165949980,7cyclictest8649-21apt-get20:00:122
165949980,7cyclictest31738-21aten_r1power_po19:40:142
165949980,7cyclictest2966-21fschecks_time20:55:142
165949980,6cyclictest254-21systemd-journal20:50:002
165759987,1cyclictest9-21ksoftirqd/021:50:000
165759987,1cyclictest9-21ksoftirqd/000:31:020
165759986,1cyclictest9-21ksoftirqd/023:00:010
165759986,1cyclictest9-21ksoftirqd/021:15:180
165759986,1cyclictest9-21ksoftirqd/020:48:320
165759986,1cyclictest9-21ksoftirqd/020:35:000
165759986,1cyclictest9-21ksoftirqd/020:25:170
165759986,1cyclictest9-21ksoftirqd/020:00:000
165759984,2cyclictest254-21systemd-journal00:40:000
165759981,6cyclictest23464-21telnet22:50:140
165759981,4cyclictest32553-21latency_hist22:20:010
165759980,7cyclictest21170-21fgrep19:15:220
165759980,6cyclictest12693-21ssh23:16:360
165759980,1cyclictest254-21systemd-journal21:08:460
6351270,0sleep10-21swapper/123:10:001
166039976,1cyclictest8969-21cat22:30:183
166039976,1cyclictest35-21ksoftirqd/320:16:303
166039975,1cyclictest6928-21ssh21:45:233
166039975,1cyclictest31515-21ssh22:17:303
166039975,1cyclictest30470-21ssh22:59:133
166039975,1cyclictest27061-21ntp_states23:35:223
166039975,1cyclictest1449-21ntpd23:14:233
166039975,1cyclictest1449-21ntpd22:29:223
166039974,3cyclictest26979-21unixbench_multi20:35:243
166039974,1cyclictest32424-21apt-get20:50:153
166039971,4cyclictest9196-21diskmemload00:33:463
166039971,4cyclictest19569-21/usr/sbin/munin22:45:163
166039971,3cyclictest9196-21diskmemload22:40:133
166039970,2cyclictest0-21swapper/321:12:023
166039970,1cyclictest1342-21apt-get23:45:123
165949971,5cyclictest27875-21telnet20:40:142
165949970,6cyclictest11009-21cut20:05:132
165869970,6cyclictest0-21swapper/121:45:181
165869970,6cyclictest0-21swapper/100:05:241
165869970,5cyclictest0-21swapper/123:47:501
165869970,4cyclictest0-21swapper/123:55:191
165869970,4cyclictest0-21swapper/122:05:231
165869970,3cyclictest0-21swapper/120:00:151
165869970,2cyclictest0-21swapper/121:58:081
165869970,2cyclictest0-21swapper/119:50:151
165759977,0cyclictest17953-21ssh22:41:300
165759976,1cyclictest9-21ksoftirqd/022:10:160
165759976,1cyclictest9-21ksoftirqd/020:10:000
165759976,1cyclictest9-21ksoftirqd/019:20:010
165759975,1cyclictest9-21ksoftirqd/020:59:590
165759971,5cyclictest13504-21ssh00:00:160
165759970,7cyclictest27019-21ssh00:16:200
165759970,7cyclictest0-21swapper/021:35:100
165759970,6cyclictest9989-21ssh23:14:370
165759970,6cyclictest9795-21ntp_kernel_pll_20:00:190
165759970,6cyclictest9196-21diskmemload22:25:260
165759970,6cyclictest3837-21awk23:05:220
165759970,6cyclictest0-21swapper/022:45:140
165759970,6cyclictest0-21swapper/020:10:210
165759970,2cyclictest0-21swapper/000:22:360
165759970,1cyclictest28755-21ssh21:33:000
165759970,1cyclictest2064-21ssh22:20:210
165759970,1cyclictest20222-21ssh00:09:300
165759970,1cyclictest11066-21ssh21:50:240
165759970,1cyclictest0-21swapper/023:40:220
165759970,1cyclictest0-21swapper/023:36:340
165759970,1cyclictest0-21swapper/022:07:400
165759970,1cyclictest0-21swapper/021:25:150
166039965,1cyclictest8061-21ssh23:53:093
166039965,1cyclictest4358-21ssh21:43:023
166039965,1cyclictest32070-21ssh21:36:113
166039965,1cyclictest22841-21ssh00:11:123
166039965,1cyclictest21603-21ssh22:05:153
166039965,1cyclictest15305-21ssh00:01:463
166039965,1cyclictest11482-21ssh23:56:593
166039964,1cyclictest27434-21ssh22:11:523
166039964,0cyclictest9196-21diskmemload22:04:133
166039963,2cyclictest31001-21missed_timers23:40:173
166039963,1cyclictest254-21systemd-journal21:05:323
166039963,1cyclictest254-21systemd-journal20:14:553
166039963,1cyclictest254-21systemd-journal19:50:013
166039963,1cyclictest254-21systemd-journal19:32:283
166039963,1cyclictest254-21systemd-journal19:30:013
166039962,2cyclictest9196-21diskmemload00:17:013
166039961,4cyclictest3950-21ssh22:25:013
166039961,2cyclictest9196-21diskmemload23:16:273
166039960,6cyclictest0-21swapper/321:19:303
166039960,6cyclictest0-21swapper/300:36:403
166039960,5cyclictest18064-21ntp_states00:05:213
166039960,4cyclictest22781-21memory21:25:163
166039960,3cyclictest0-21swapper/323:33:353
166039960,3cyclictest0-21swapper/323:26:153
166039960,3cyclictest0-21swapper/321:50:013
166039960,0cyclictest13642-21ssh21:55:163
165869962,1cyclictest0-21swapper/100:17:271
165869961,3cyclictest0-21swapper/122:15:141
165869961,2cyclictest0-21swapper/121:37:081
165869960,6cyclictest0-21swapper/122:20:451
165869960,6cyclictest0-21swapper/122:00:161
165869960,5cyclictest0-21swapper/123:00:161
165869960,4cyclictest0-21swapper/122:31:551
165869960,4cyclictest0-21swapper/121:53:581
165869960,3cyclictest0-21swapper/123:25:191
165869960,3cyclictest0-21swapper/121:42:251
165869960,1cyclictest0-21swapper/121:25:161
165869960,1cyclictest0-21swapper/100:25:191
165869960,1cyclictest0-21swapper/100:00:141
165759965,1cyclictest9-21ksoftirqd/020:45:000
165759965,1cyclictest32673-21ssh00:25:060
165759964,1cyclictest6036-21sh19:54:590
165759960,6cyclictest0-21swapper/020:19:250
165759960,6cyclictest0-21swapper/019:45:150
165759960,5cyclictest17602-21ssh21:20:000
165759960,1cyclictest0-21swapper/020:50:140
165759960,0cyclictest0-21swapper/022:30:150
165759960,0cyclictest0-21swapper/021:00:120
165759960,0cyclictest0-21swapper/019:35:210
165759960,0cyclictest0-21swapper/019:35:200
166039954,1cyclictest6635-21sshd21:02:313
166039954,1cyclictest35-21ksoftirqd/320:30:423
166039954,1cyclictest31631-21/usr/sbin/munin19:40:153
166039954,1cyclictest19542-21ssh21:20:283
166039953,1cyclictest20347-21/usr/sbin/munin19:15:163
166039952,1cyclictest254-21systemd-journal20:03:003
166039951,1cyclictest0-21swapper/323:05:143
166039950,0cyclictest0-21swapper/319:13:593
165869955,0cyclictest0-21swapper/122:54:381
165869955,0cyclictest0-21swapper/120:22:571
165869951,1cyclictest0-21swapper/122:25:131
165869950,5cyclictest0-21swapper/123:20:151
165869950,5cyclictest0-21swapper/122:10:101
165869950,5cyclictest0-21swapper/119:29:231
165869950,5cyclictest0-21swapper/119:15:031
165869950,4cyclictest0-21swapper/123:35:561
165869950,4cyclictest0-21swapper/120:50:191
165869950,3cyclictest0-21swapper/123:17:421
165869950,3cyclictest0-21swapper/120:40:001
165869950,3cyclictest0-21swapper/120:35:011
165869950,3cyclictest0-21swapper/100:35:441
165869950,3cyclictest0-21swapper/100:13:081
165869950,2cyclictest0-21swapper/123:40:501
165869950,2cyclictest0-21swapper/123:30:161
165869950,2cyclictest0-21swapper/122:35:211
165869950,2cyclictest0-21swapper/120:40:191
165869950,1cyclictest0-21swapper/123:10:141
165869950,1cyclictest0-21swapper/122:40:121
165869950,1cyclictest0-21swapper/121:18:351
165869950,1cyclictest0-21swapper/100:21:241
165759952,1cyclictest25888-21grep19:25:180
165759951,3cyclictest0-21swapper/020:20:140
165759950,3cyclictest0-21swapper/019:40:160
166039943,1cyclictest7980-21unixbench_singl19:55:213
166039942,1cyclictest24139-21runrttasks19:22:473
166039941,2cyclictest2352-21apt-get20:55:103
166039941,1cyclictest29264-21apt-get19:35:133
166039941,1cyclictest29264-21apt-get19:35:133
166039940,1cyclictest30214-21aten_r1power_vo20:45:153
166039940,1cyclictest254-21systemd-journal20:06:593
166039940,1cyclictest0-21swapper/320:20:133
165869944,0cyclictest0-21swapper/119:37:211
165869944,0cyclictest0-21swapper/119:37:201
165869942,2cyclictest0-21swapper/121:00:141
165869942,1cyclictest0-21swapper/120:55:111
165869941,3cyclictest0-21swapper/121:10:111
165869941,2cyclictest0-21swapper/120:10:151
165869940,4cyclictest0-21swapper/120:09:021
165869940,3cyclictest0-21swapper/119:10:171
165869940,2cyclictest0-21swapper/122:55:141
165869940,2cyclictest0-21swapper/121:34:101
165869940,2cyclictest0-21swapper/120:25:151
165869940,2cyclictest0-21swapper/119:40:151
165869940,2cyclictest0-21swapper/119:30:201
165869940,1cyclictest0-21swapper/123:52:491
165869940,1cyclictest0-21swapper/120:15:161
165869940,1cyclictest0-21swapper/119:55:281
165869940,1cyclictest0-21swapper/119:45:181
165869940,0cyclictest0-21swapper/121:20:041
165869931,1cyclictest0-21swapper/121:05:141
165869930,1cyclictest0-21swapper/120:45:191
165869930,1cyclictest0-21swapper/119:20:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional