You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-19 - 14:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Sun Oct 19, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178812304270,21sleep20-21swapper/219:09:382
176802277215,21sleep30-21swapper/319:07:213
177272267218,37sleep00-21swapper/019:07:390
178282252219,22sleep10-21swapper/119:08:571
258412650,1sleep30-21swapper/321:22:003
245772640,0sleep10-21swapper/120:35:001
233902640,0sleep10-21swapper/123:59:071
304142630,0sleep10-21swapper/123:40:211
169222610,0sleep10-21swapper/123:09:341
154272600,0sleep30-21swapper/321:37:203
294352580,0sleep00-21swapper/000:26:290
101432570,0sleep30-21swapper/321:55:433
308192560,0sleep00-21swapper/021:48:000
324302540,0sleep30-21swapper/300:29:563
123412540,0sleep00-21swapper/022:20:160
187302190,0sleep00-21swapper/000:18:470
157372180,0sleep00-21swapper/021:37:380
20902170,0sleep20-21swapper/222:35:222
167722150,0sleep00-21swapper/022:00:210
68332140,0sleep20-21swapper/223:25:002
63692140,0sleep00-21swapper/021:30:300
290082120,0sleep20-21swapper/223:17:322
1802799129,2cyclictest34-21ksoftirqd/300:00:003
18027991210,2cyclictest34-21ksoftirqd/322:40:013
1802099120,3cyclictest19982-21systemd-journal20:20:001
1802799119,1cyclictest34-21ksoftirqd/300:40:003
1802099113,2cyclictest15822-21wc22:00:161
1802099111,3cyclictest19982-21systemd-journal23:00:011
1802099111,2cyclictest21658-21apt-get23:35:161
1802099110,3cyclictest632-21ntp_states00:05:161
1802099110,3cyclictest19982-21systemd-journal19:11:331
1802099110,3cyclictest10421-21sh00:37:431
1801699119,1cyclictest12-21ksoftirqd/020:55:010
1802799109,1cyclictest34-21ksoftirqd/319:35:003
1802799107,2cyclictest34-21ksoftirqd/321:10:013
1802799107,2cyclictest34-21ksoftirqd/321:10:013
1802799107,2cyclictest34-21ksoftirqd/320:10:013
1802799107,2cyclictest34-21ksoftirqd/319:30:013
1802799107,2cyclictest34-21ksoftirqd/319:20:023
1802799103,5cyclictest34-21ksoftirqd/321:00:203
1802599107,2cyclictest28-21ksoftirqd/220:50:002
1802099103,3cyclictest4964-21df_inode22:15:141
1802099103,2cyclictest508-21dbus-daemon19:26:101
1802099103,2cyclictest14625-21cat22:45:001
1802099103,2cyclictest12391-21grep20:05:201
1802099102,2cyclictest28992-21readlink19:30:221
1802099101,3cyclictest508-21dbus-daemon19:41:361
1802099101,3cyclictest19150-21find23:10:221
1802099101,2cyclictest14591-21ssh21:14:531
1802099101,2cyclictest10260-21idleruntime-cro23:05:011
1802099100,4cyclictest26431-21ssh23:15:201
1802099100,3cyclictest28787-21sshd20:42:381
1802099100,3cyclictest26751-21cron20:40:011
1802099100,3cyclictest19982-21systemd-journal00:15:001
1802099100,2cyclictest5763-21sh00:34:261
1802099100,2cyclictest19982-21systemd-journal22:20:331
1802099100,2cyclictest19982-21systemd-journal21:10:011
1802099100,2cyclictest19982-21systemd-journal21:10:001
1802099100,2cyclictest19982-21systemd-journal19:20:011
180279997,2cyclictest34-21ksoftirqd/320:25:013
180279996,2cyclictest34-21ksoftirqd/323:25:013
180279996,2cyclictest34-21ksoftirqd/323:00:003
180279993,4cyclictest508-21dbus-daemon23:41:043
180279993,4cyclictest34-21ksoftirqd/321:28:383
180279993,4cyclictest22154-21grep23:35:173
180279993,1cyclictest141rcu_preempt19:55:173
180279992,5cyclictest34-21ksoftirqd/321:45:213
180279992,5cyclictest34-21ksoftirqd/300:34:043
180279992,5cyclictest29741-11brltty20:43:203
180279991,4cyclictest34-21ksoftirqd/323:09:123
180279990,4cyclictest0-21swapper/321:31:123
180259995,3cyclictest19982-21systemd-journal22:25:002
180209993,2cyclictest30654-21snmpd23:25:411
180209993,1cyclictest4442-21cut21:00:201
180209993,1cyclictest19962-21pmu-power21:40:201
180209993,1cyclictest0-21swapper/122:05:161
180209992,2cyclictest5600-21open_inodes21:30:171
180209992,2cyclictest29555-21rm21:25:001
180209992,1cyclictest25438-21grep00:00:161
180209992,1cyclictest19140-21head20:20:191
180209991,7cyclictest0-21swapper/121:20:101
180209991,3cyclictest7899-21ssh21:55:061
180209991,3cyclictest30654-21snmpd22:35:191
180209991,3cyclictest14920-21/usr/sbin/munin22:45:141
180209991,2cyclictest508-21dbus-daemon20:01:371
180209991,2cyclictest4914-21expr19:50:161
180209990,4cyclictest0-21swapper/123:24:461
180209990,3cyclictest7134-21tail19:55:131
180209990,3cyclictest534-21idleruntime-cro21:50:011
180209990,3cyclictest4531-21ntp_states23:45:231
180209990,3cyclictest30203-21grep19:35:161
180209990,3cyclictest26932-21wc22:30:191
180209990,3cyclictest21351-21date23:35:001
180209990,3cyclictest19071-21cat22:25:131
180209990,3cyclictest14468-21rm21:36:281
180209990,3cyclictest13707-21expr20:10:141
180209990,3cyclictest12142-21unixbench_multi23:50:201
180209990,3cyclictest1033-21rm22:12:091
180209990,2cyclictest19982-21systemd-journal20:47:561
180209990,2cyclictest19982-21systemd-journal19:47:431
180209990,2cyclictest19982-21systemd-journal19:24:291
180209990,2cyclictest19982-21systemd-journal00:15:161
180169990,4cyclictest141rcu_preempt23:44:000
19195280,1sleep20-21swapper/221:17:302
180279987,1cyclictest34-21ksoftirqd/319:50:003
180279986,1cyclictest331rcuc/320:05:013
180279984,2cyclictest19982-21systemd-journal20:49:253
180279984,1cyclictest141rcu_preempt22:15:083
180279983,3cyclictest19982-21systemd-journal23:46:293
180279982,4cyclictest34-21ksoftirqd/323:15:153
180279982,4cyclictest34-21ksoftirqd/321:18:123
180279982,4cyclictest34-21ksoftirqd/320:30:183
180279982,4cyclictest34-21ksoftirqd/300:24:263
180279982,4cyclictest34-21ksoftirqd/300:15:223
180279982,4cyclictest331rcuc/322:40:053
180279982,4cyclictest15603-21uniq22:00:153
180279981,5cyclictest34-21ksoftirqd/323:28:303
180279981,5cyclictest34-21ksoftirqd/322:20:183
180279981,5cyclictest34-21ksoftirqd/322:05:213
180279981,5cyclictest34-21ksoftirqd/321:52:343
180279981,5cyclictest34-21ksoftirqd/321:14:133
180279981,5cyclictest34-21ksoftirqd/300:04:463
180279981,4cyclictest0-21swapper/322:34:333
180279980,7cyclictest0-21swapper/300:05:043
180279980,5cyclictest34-21ksoftirqd/322:27:533
180259987,1cyclictest28-21ksoftirqd/219:15:022
180259986,1cyclictest0-21swapper/221:22:592
180259984,3cyclictest19982-21systemd-journal23:04:372
180259981,1cyclictest28-21ksoftirqd/221:35:142
180209982,1cyclictest0-21swapper/121:50:081
180209981,1cyclictest508-21dbus-daemon20:50:011
180209980,3cyclictest26556-21rm00:25:031
180209980,3cyclictest24096-21ssh22:50:491
180209980,3cyclictest20418-21ssh00:20:141
180209980,3cyclictest15421-21expr21:15:151
180209980,3cyclictest1506-21ls20:55:141
180209980,2cyclictest20733-21fschecks_count20:25:141
180169985,2cyclictest4557-21apt22:15:120
180169984,3cyclictest19982-21systemd-journal23:58:350
180169984,3cyclictest19982-21systemd-journal22:07:370
180169981,1cyclictest12-21ksoftirqd/023:50:410
180169980,3cyclictest141rcu_preempt22:40:130
180169980,1cyclictest141rcu_preempt22:30:160
19103270,0sleep00-21swapper/022:47:430
180279976,1cyclictest34-21ksoftirqd/320:10:013
180279976,1cyclictest34-21ksoftirqd/319:15:023
180279973,1cyclictest34-21ksoftirqd/323:10:343
180279973,1cyclictest141rcu_preempt23:50:453
180279972,3cyclictest19982-21systemd-journal19:41:353
180279971,4cyclictest34-21ksoftirqd/322:50:313
180279971,4cyclictest34-21ksoftirqd/320:35:203
180279971,4cyclictest34-21ksoftirqd/320:15:183
180279971,4cyclictest30845-21ssh22:10:183
180279970,5cyclictest34-21ksoftirqd/323:31:013
180279970,5cyclictest34-21ksoftirqd/323:00:443
180279970,4cyclictest34-21ksoftirqd/322:45:143
180279970,4cyclictest0-21swapper/320:55:183
180279970,3cyclictest0-21swapper/321:43:333
180279970,2cyclictest21782-21unixbench_multi20:25:213
180279970,1cyclictest6910-21apt-get00:10:143
180259974,2cyclictest28-21ksoftirqd/221:15:012
180259974,2cyclictest27042-21ntpq22:30:192
180259974,2cyclictest23047-21sshd23:13:372
180259974,2cyclictest11736-21in:imuxsock22:50:002
180259974,2cyclictest11736-21in:imuxsock00:14:022
180259974,2cyclictest10081-21cron23:50:002
180259973,3cyclictest8164-21expr22:40:132
180259973,3cyclictest7123-21seq22:16:112
180259973,3cyclictest508-21dbus-daemon20:58:472
180259973,3cyclictest26915-21cut20:40:002
180259973,3cyclictest26086-21gltestperf19:25:172
180259973,3cyclictest21348-21expr19:15:162
180259973,3cyclictest1-21systemd22:50:102
180259973,3cyclictest11245-21fw_packets23:05:162
180259973,3cyclictest0-21swapper/223:35:482
180259972,3cyclictest19982-21systemd-journal19:59:582
180169976,1cyclictest12-21ksoftirqd/023:00:010
180169974,2cyclictest12770-21sshd20:07:320
180169973,3cyclictest29741-11brltty20:46:160
180169973,3cyclictest19982-21systemd-journal20:14:580
180169973,3cyclictest0-21swapper/000:36:070
180169973,2cyclictest19982-21systemd-journal19:49:550
180169972,1cyclictest141rcu_preempt21:15:000
180169971,3cyclictest3279-21idleruntime-cro21:00:000
180169971,3cyclictest24870-21rm21:21:060
180279963,2cyclictest30254-21cstates19:35:153
180279963,1cyclictest34-21ksoftirqd/319:20:153
180279960,4cyclictest5154-21missed_timers19:50:163
180279960,4cyclictest34-21ksoftirqd/320:50:213
180259964,1cyclictest7755-21diskmemload21:45:132
180259964,1cyclictest248402sleep200:00:132
180259963,2cyclictest9401-21sh20:00:152
180259963,2cyclictest7661-21lldpd23:51:172
180259963,2cyclictest6997-21expr00:35:142
180259963,2cyclictest508-21dbus-daemon21:05:272
180259963,2cyclictest508-21dbus-daemon21:05:272
180259963,2cyclictest3856-21sshd19:47:432
180259963,2cyclictest25100-21taskset00:23:492
180259963,2cyclictest23855-21timerwakeupswit22:05:232
180259962,3cyclictest7661-21lldpd20:31:142
180259962,3cyclictest508-21dbus-daemon20:26:392
180259962,3cyclictest32327-21expr00:05:122
180259962,3cyclictest30726-21interrupts19:35:172
180259962,3cyclictest30654-21snmpd23:40:302
180259962,3cyclictest30654-21snmpd20:10:362
180259962,3cyclictest30654-21snmpd19:43:052
180259962,3cyclictest30654-21snmpd19:21:392
180259962,3cyclictest30654-21snmpd00:27:432
180259962,3cyclictest30547-21cat21:25:162
180259962,3cyclictest29179-21cpuspeed_turbos22:55:152
180259962,3cyclictest28472-21cat19:30:172
180259962,3cyclictest27092-21cat20:40:122
180259962,3cyclictest20350-21cat22:25:202
180259962,3cyclictest18843-21apt21:40:132
180259962,3cyclictest18228-21grep20:20:142
180259962,3cyclictest1-21systemd19:52:272
180259962,3cyclictest0-21swapper/221:50:212
180259962,1cyclictest0-21swapper/223:56:302
180259962,1cyclictest0-21swapper/223:20:382
180259962,1cyclictest0-21swapper/222:00:562
180259962,1cyclictest0-21swapper/200:31:122
180259962,1cyclictest0-21swapper/200:15:032
180259961,3cyclictest19982-21systemd-journal20:07:482
180259960,6cyclictest0-21swapper/221:55:082
180259960,6cyclictest0-21swapper/221:30:482
180259960,2cyclictest15095-21fw_packets23:30:172
180259960,1cyclictest141rcu_preempt22:10:192
180169963,2cyclictest764-21processes00:05:160
180169963,2cyclictest6519-21taskset19:55:060
180169963,2cyclictest508-21dbus-daemon19:15:180
180169963,2cyclictest4332-21ssh23:45:220
180169963,2cyclictest27546-21gltestperf20:40:150
180169963,2cyclictest25019-21ssh23:37:000
180169963,2cyclictest19982-21systemd-journal22:35:100
180169963,2cyclictest19310-21grep19:10:200
180169963,2cyclictest14948-21df23:30:160
180169963,2cyclictest0-21swapper/023:20:150
180169962,3cyclictest7249-21awk21:05:220
180169962,3cyclictest7249-21awk21:05:220
180169962,3cyclictest32677-21expr19:40:150
180169962,3cyclictest30654-21snmpd20:01:520
180169962,3cyclictest28202-21diskstats19:30:170
180169962,3cyclictest27057-21grep19:25:230
180169962,3cyclictest25950-21expr23:15:160
180169962,3cyclictest24743-21cat19:20:230
180169962,3cyclictest20987-21cat20:25:160
180169962,3cyclictest2053-21ntpq21:50:190
180169962,3cyclictest20188-21sh00:20:060
180169962,3cyclictest19982-21systemd-journal20:30:270
180169962,3cyclictest18942-21cat20:20:190
180169962,3cyclictest17901-21apt-get23:10:150
180169962,3cyclictest16358-21expr20:15:170
180169962,3cyclictest11220-21fschecks_time23:05:140
180169962,3cyclictest0-21swapper/022:10:440
180169962,3cyclictest0-21swapper/000:30:120
180169962,1cyclictest0-21swapper/022:25:070
180169962,1cyclictest0-21swapper/021:55:150
180169961,3cyclictest19982-21systemd-journal21:41:210
180169961,3cyclictest19982-21systemd-journal21:01:160
180169960,6cyclictest0-21swapper/000:00:050
180169960,5cyclictest22335-21df22:50:150
180169960,3cyclictest8061-21grep23:25:190
180169960,3cyclictest6824-21rm00:10:090
180169960,3cyclictest14964-21rm21:15:000
180169960,2cyclictest4295-21/usr/sbin/munin19:50:140
180169960,1cyclictest29038-21apt-get21:25:020
180169960,1cyclictest141rcu_preempt19:35:150
180169960,0cyclictest0-21swapper/023:00:110
180259952,2cyclictest5101-21uptime21:00:242
180259952,2cyclictest16082-21expr20:15:162
180259950,3cyclictest19982-21systemd-journal20:50:012
180169952,2cyclictest25054-21expr20:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional