You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 04:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8.osadl.org (updated Fri Jul 04, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
119632309241,22sleep10-21swapper/119:08:391
118932270215,18sleep30-21swapper/319:07:453
117432268234,22sleep00-21swapper/019:06:050
117112264226,23sleep20-21swapper/219:05:422
140922690,1sleep114097-21unixbench_multi21:40:241
191242610,0sleep00-21swapper/022:07:290
78922590,0sleep20-21swapper/221:36:062
137112570,0sleep00-21swapper/000:23:550
153972560,0sleep20-21swapper/222:28:132
288132550,0sleep10-21swapper/100:09:501
1219199160,10cyclictest3418-21expr21:10:203
1219199150,11cyclictest24106-21ssh00:31:223
1219199150,11cyclictest17850-21ssh21:43:483
207032140,0sleep00-21swapper/023:40:060
1219199143,6cyclictest508-21dbus-daemon19:26:063
1219199142,9cyclictest6617-21rm21:58:523
1219199142,9cyclictest10350-21ssh22:00:503
1219199142,7cyclictest6138-21cat21:35:183
1219199141,9cyclictest3045-21ssh22:19:443
1219199140,8cyclictest22868-21ntpq19:30:193
1219199140,8cyclictest19982-21systemd-journal00:20:383
1219199140,7cyclictest27087-21grep19:40:183
1219199140,11cyclictest26490-21ssh22:35:353
1219199140,11cyclictest25444-21ssh00:06:163
1219199140,11cyclictest17558-21ntpq21:20:203
1219199140,11cyclictest12151-21grep22:25:193
1219199140,10cyclictest22614-21ssh23:19:073
1219199140,10cyclictest19811-21ssh22:30:433
1218299143,1cyclictest22-21ksoftirqd/121:15:141
12182991412,1cyclictest22-21ksoftirqd/123:05:011
1219199133,7cyclictest24202-21swap20:45:263
1219199132,8cyclictest0-21swapper/322:57:513
1219199130,9cyclictest4412-21sh23:50:423
1219199130,9cyclictest28992-21grep23:45:183
1219199130,9cyclictest24916-21sh21:48:433
1219199130,8cyclictest3573-21rm21:33:453
1219199130,8cyclictest32160-21/usr/sbin/munin23:25:143
1219199130,8cyclictest14664-21proc_pri23:35:183
1219199130,7cyclictest2608-21grep23:05:133
1219199130,6cyclictest29092-21awk00:35:213
1219199130,10cyclictest6622-21ssh22:45:093
1219199130,10cyclictest5126-21ssh00:16:133
1219199130,10cyclictest3144-21ssh22:42:263
1219199130,10cyclictest28625-21sh22:14:513
1219199130,10cyclictest27034-21rm21:26:553
1219199130,10cyclictest24810-21cut23:20:173
1219199130,10cyclictest21492-21ssh22:09:423
1219199130,10cyclictest14996-21awk20:25:183
1219199130,10cyclictest14267-21rm23:13:063
1219199130,10cyclictest11744-21ssh23:33:493
1218299131,1cyclictest141rcu_preempt20:00:011
82482120,0sleep20-21swapper/223:54:182
1219199121,8cyclictest27830-21wc20:55:143
1219199120,9cyclictest29268-21ssh21:51:063
1219199120,9cyclictest27619-21ssh23:00:073
1219199120,9cyclictest24772-21basename19:35:163
1219199120,9cyclictest19839-21grep20:35:223
1219199120,9cyclictest15842-21hwlatdetect19:15:153
1219199120,8cyclictest17445-21ntpq00:00:183
1219199120,8cyclictest15289-21grep22:50:223
1219199120,7cyclictest5277-21find22:20:223
1219199120,7cyclictest32736-21cat21:05:153
1219199120,7cyclictest31572-21sh00:11:273
1219199120,7cyclictest24846-21ssh23:42:343
1219199120,7cyclictest21046-21expr20:40:143
1219199120,7cyclictest2040-21diskmemload00:28:033
1219199120,7cyclictest12812-21cut20:20:193
1219199120,7cyclictest1-21systemd19:49:283
1219199120,6cyclictest11260-21ssh21:15:253
12182991211,1cyclictest22-21ksoftirqd/122:00:011
12182991211,1cyclictest22-21ksoftirqd/100:00:001
12182991210,1cyclictest22-21ksoftirqd/100:20:011
1219199110,9cyclictest25469-21apt-get20:50:013
1219199110,7cyclictest32210-21timerwakeupswit19:50:213
1219199110,7cyclictest18433-21ntpq19:20:203
1219199110,7cyclictest1262-21cat19:55:153
1219199110,6cyclictest3503-21basename20:00:193
1219199110,6cyclictest17392-21which20:30:193
1219199110,6cyclictest13509-21gunzip19:10:203
1219199110,6cyclictest11731-21ssh23:55:593
1218299119,1cyclictest22-21ksoftirqd/123:35:001
1218299119,1cyclictest22-21ksoftirqd/123:00:011
1218299118,2cyclictest22-21ksoftirqd/121:35:011
1218299118,2cyclictest22-21ksoftirqd/119:30:021
12182991110,1cyclictest22-21ksoftirqd/122:55:001
12182991110,1cyclictest22-21ksoftirqd/100:00:011
1219199100,6cyclictest8140-21expr20:10:203
1219199100,6cyclictest10182-21ls20:15:183
1219199100,5cyclictest30947-21ntp_states21:00:223
1218299109,1cyclictest22-21ksoftirqd/123:25:001
1218299109,1cyclictest22-21ksoftirqd/123:10:001
1218299109,1cyclictest22-21ksoftirqd/121:40:021
1218299108,1cyclictest22-21ksoftirqd/120:35:011
1218299107,2cyclictest22-21ksoftirqd/120:25:011
121919990,8cyclictest5250-21cpuspeed_turbos20:05:153
121829998,1cyclictest22-21ksoftirqd/122:40:001
121829998,1cyclictest22-21ksoftirqd/121:30:001
121829998,1cyclictest22-21ksoftirqd/120:45:011
121829997,2cyclictest22-21ksoftirqd/100:25:001
121829997,1cyclictest211rcuc/122:25:011
121829997,1cyclictest211rcuc/119:25:001
121829996,2cyclictest22-21ksoftirqd/123:40:001
121829994,3cyclictest19982-21systemd-journal22:01:181
121829993,4cyclictest22-21ksoftirqd/123:41:121
121829991,5cyclictest19116-21gdbus20:40:011
32756280,0sleep10-21swapper/121:05:141
121879986,1cyclictest0-21swapper/223:46:052
121879983,3cyclictest19982-21systemd-journal22:20:552
121829987,1cyclictest22-21ksoftirqd/120:30:011
121829987,1cyclictest22-21ksoftirqd/119:15:011
121829985,2cyclictest22-21ksoftirqd/123:50:001
121829984,3cyclictest19982-21systemd-journal00:28:091
121829984,1cyclictest22-21ksoftirqd/100:34:001
121829983,4cyclictest14123-21sshd19:12:251
121829982,3cyclictest9296-21idleruntime-cro20:15:011
121829982,3cyclictest24129-21ls22:35:001
121829981,4cyclictest22-21ksoftirqd/122:05:161
121789984,3cyclictest508-21dbus-daemon23:49:460
121789984,3cyclictest508-21dbus-daemon00:37:450
121789984,1cyclictest31123-21readlink22:40:010
121789980,7cyclictest0-21swapper/022:45:210
18777270,0sleep00-21swapper/019:20:230
12312270,0sleep20-21swapper/222:48:592
121879974,2cyclictest11736-21in:imuxsock22:35:002
121879974,2cyclictest11736-21in:imuxsock19:20:002
121879973,3cyclictest7661-21lldpd00:02:412
121879973,3cyclictest508-21dbus-daemon21:14:342
121879973,3cyclictest508-21dbus-daemon00:05:372
121879973,3cyclictest30830-21apt-get21:30:122
121879973,3cyclictest30654-21snmpd21:59:372
121879973,3cyclictest30654-21snmpd00:18:512
121879973,3cyclictest25516-21apt-get20:50:172
121879973,3cyclictest19982-21systemd-journal20:15:012
121879973,3cyclictest19982-21systemd-journal00:29:592
121879973,3cyclictest13907-21df23:35:122
121879973,2cyclictest7661-21lldpd23:12:212
121879973,1cyclictest0-21swapper/223:30:512
121879973,1cyclictest0-21swapper/222:14:462
121879973,1cyclictest0-21swapper/221:46:182
121879972,3cyclictest19982-21systemd-journal23:24:202
121879970,4cyclictest14720-21awk00:25:002
121829976,1cyclictest22-21ksoftirqd/123:15:001
121829976,1cyclictest22-21ksoftirqd/121:15:001
121829975,1cyclictest22-21ksoftirqd/123:30:001
121829975,1cyclictest22-21ksoftirqd/121:05:001
121829974,2cyclictest22-21ksoftirqd/122:25:111
121829974,1cyclictest22-21ksoftirqd/121:20:201
121829974,1cyclictest141rcu_preempt00:10:181
121829973,3cyclictest30654-21snmpd20:03:161
121829973,1cyclictest141rcu_preempt22:15:141
121829973,1cyclictest141rcu_preempt20:50:181
121829973,1cyclictest141rcu_preempt00:39:001
121829973,1cyclictest0-21swapper/122:47:121
121829973,0cyclictest141rcu_preempt22:13:241
121829972,1cyclictest141rcu_preempt21:52:081
121829971,4cyclictest22-21ksoftirqd/123:18:081
121829971,3cyclictest7074-21munin-run20:10:011
121829970,5cyclictest22-21ksoftirqd/121:46:441
121829970,4cyclictest22-21ksoftirqd/119:50:191
121789974,2cyclictest12-21ksoftirqd/000:00:010
121789974,1cyclictest19116-21gdbus22:30:010
121789973,3cyclictest5895-21cron00:17:000
121789973,3cyclictest539-21sshd00:13:180
121789973,3cyclictest508-21dbus-daemon22:59:420
121789973,3cyclictest508-21dbus-daemon21:55:210
121789973,3cyclictest508-21dbus-daemon21:45:010
121789973,3cyclictest508-21dbus-daemon19:20:000
121789973,3cyclictest19982-21systemd-journal23:54:550
121789973,3cyclictest19982-21systemd-journal22:34:160
121789973,3cyclictest19982-21systemd-journal20:05:010
121789973,3cyclictest16923-21cat20:30:170
121789973,3cyclictest0-21swapper/023:35:230
121789973,2cyclictest7661-21lldpd00:00:420
121789973,2cyclictest19982-21systemd-journal22:16:190
121789973,1cyclictest0-21swapper/023:05:070
121789973,1cyclictest0-21swapper/021:37:310
121789972,3cyclictest19982-21systemd-journal22:22:100
121789970,3cyclictest502-21systemd-logind23:25:340
15652260,0sleep20-21swapper/222:05:152
121879963,2cyclictest9901-21cut23:55:142
121879963,2cyclictest5486-21df_inode20:05:152
121879963,2cyclictest30654-21snmpd22:40:072
121879963,2cyclictest30654-21snmpd21:19:432
121879963,2cyclictest2892-21sh23:05:152
121879963,2cyclictest26983-21tail19:40:172
121879963,2cyclictest24827-21ssh22:57:382
121879963,2cyclictest20037-21ssh23:16:472
121879963,2cyclictest19982-21systemd-journal20:55:192
121879963,2cyclictest19267-21taskset19:22:552
121879963,2cyclictest13692-21taskset20:24:162
121879963,2cyclictest0-21swapper/222:36:262
121879962,3cyclictest508-21dbus-daemon22:18:252
121879962,3cyclictest31149-21/usr/sbin/munin23:25:172
121879962,3cyclictest30654-21snmpd21:02:492
121879962,3cyclictest30654-21snmpd20:40:292
121879962,3cyclictest30654-21snmpd20:35:022
121879962,3cyclictest30654-21snmpd19:59:222
121879962,3cyclictest30654-21snmpd19:45:172
121879962,3cyclictest30654-21snmpd19:13:182
121879962,3cyclictest30031-21cat00:10:162
121879962,3cyclictest29205-21cat00:35:222
121879962,3cyclictest21562-21cat23:40:162
121879962,3cyclictest19982-21systemd-journal23:02:562
121879962,3cyclictest19116-21gdbus21:05:002
121879962,3cyclictest18200-21grep21:20:232
121879962,3cyclictest17439-21tr20:30:202
121879962,3cyclictest1-21systemd20:27:012
121879962,2cyclictest21552-21chrt19:28:462
121879962,1cyclictest0-21swapper/222:00:212
121879962,1cyclictest0-21swapper/221:50:572
121879962,1cyclictest0-21swapper/221:40:012
121879961,4cyclictest15587-21grep22:50:232
121879961,3cyclictest19982-21systemd-journal20:45:232
121879960,4cyclictest19982-21systemd-journal20:00:192
121879960,0cyclictest0-21swapper/221:25:272
121829965,1cyclictest22-21ksoftirqd/119:45:021
121829964,2cyclictest22-21ksoftirqd/120:50:001
121829963,2cyclictest668-21runrttasks20:15:011
121829963,2cyclictest22-21ksoftirqd/122:40:001
121829963,2cyclictest1-21systemd19:46:081
121829962,4cyclictest31762-21avahi-daemon21:00:001
121829962,3cyclictest30654-21snmpd19:35:351
121829961,3cyclictest19982-21systemd-journal23:50:111
121829960,2cyclictest22-21ksoftirqd/119:30:151
121789963,2cyclictest30654-21snmpd19:33:160
121789963,2cyclictest27375-21ssh22:13:400
121789963,2cyclictest20303-21cat21:45:160
121789963,2cyclictest19982-21systemd-journal19:36:570
121789963,2cyclictest19262-21sh00:28:010
121789963,2cyclictest16941-21cpuspeed_turbos23:15:130
121789963,2cyclictest14136-21taskset19:12:290
121789963,2cyclictest14006-21cat22:50:150
121789963,2cyclictest12944-21wc20:20:200
121789962,3cyclictest9744-21sort23:10:140
121789962,3cyclictest765-21cron22:45:010
121789962,3cyclictest30654-21snmpd21:50:470
121789962,3cyclictest30654-21snmpd21:30:460
121789962,3cyclictest30654-21snmpd20:50:120
121789962,3cyclictest30654-21snmpd20:42:180
121789962,3cyclictest30654-21snmpd20:25:360
121789962,3cyclictest30173-21expr21:00:170
121789962,3cyclictest24034-21expr21:25:140
121789962,3cyclictest22711-21cron20:45:000
121789962,3cyclictest20377-21cat19:25:180
121789962,3cyclictest1304-21cat19:55:150
121789962,3cyclictest1101-21cut21:05:190
121789962,2cyclictest7661-21lldpd20:10:440
121789962,2cyclictest19982-21systemd-journal20:37:380
121789962,1cyclictest141rcu_preempt23:00:130
121789962,1cyclictest0-21swapper/023:30:100
121789962,1cyclictest0-21swapper/021:20:220
121789962,1cyclictest0-21swapper/021:13:160
121789960,5cyclictest29495-21sshd20:57:400
121789960,5cyclictest25470-21sshd00:06:190
121789960,3cyclictest26287-21rm00:33:460
121789960,3cyclictest13462-21rm21:17:350
121789960,2cyclictest19146-21kworker/u8:223:20:130
121879953,1cyclictest30654-21snmpd20:15:112
121879953,1cyclictest22380-21fschecks_time19:30:162
121879952,2cyclictest30817-21acpi19:50:142
121879951,2cyclictest19982-21systemd-journal19:35:012
121879950,4cyclictest21378-21chrt00:30:022
121789953,1cyclictest9735-21cpuspeed_turbos20:15:150
121789953,1cyclictest26696-21expr19:40:150
121789952,2cyclictest9554-21/usr/sbin/munin22:00:210
121789952,2cyclictest30654-21snmpd19:50:130
121789952,2cyclictest29036-21fschecks_count19:45:170
121789951,3cyclictest30654-21snmpd20:05:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional