You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-18 - 10:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Sat Jan 18, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
35722333299,21sleep20-21swapper/219:09:592
35472282247,23sleep10-21swapper/119:09:411
34072276245,20sleep00-21swapper/019:08:000
15952257224,21sleep30-21swapper/319:05:093
44802630,0sleep20-21swapper/219:10:132
136772630,0sleep00-21swapper/023:30:110
153062590,0sleep00-21swapper/021:51:010
222462550,0sleep30-21swapper/300:00:163
19362440,0sleep00-21swapper/022:30:450
63282200,0sleep10-21swapper/122:59:571
286322190,0sleep00-21swapper/021:10:280
17302150,0sleep10-21swapper/100:10:161
122562150,0sleep20-21swapper/223:04:022
369399142,7cyclictest4384-21cut20:20:143
369399132,8cyclictest31252-21chrt20:08:443
3680991310,2cyclictest12-21ksoftirqd/000:10:000
369399123,5cyclictest30654-21snmpd22:07:073
369399122,9cyclictest13066-21apt-get22:15:123
369399120,8cyclictest26371-21diskmemload21:17:403
369399120,8cyclictest0-21swapper/323:23:143
369399120,2cyclictest14491-21irqstats22:40:163
369399115,4cyclictest0-21swapper/321:42:223
369399112,5cyclictest1852-21sh22:30:403
369399111,9cyclictest11788-21grep00:20:123
369399111,9cyclictest0-21swapper/323:20:013
369399111,6cyclictest25866-21ssh00:33:463
369399110,3cyclictest24637-21ssh23:14:343
369399110,2cyclictest9740-21ssh22:12:083
369399110,2cyclictest9666-21ssh23:51:103
369399110,2cyclictest30656-21ssh00:08:183
369399110,2cyclictest30347-21ssh21:37:283
369399110,2cyclictest30154-21ssh21:11:543
369399110,2cyclictest29198-21ssh22:52:063
369399110,2cyclictest29172-21ssh00:35:393
369399110,2cyclictest29092-21sh22:01:043
369399110,2cyclictest20897-21cut23:35:153
369399110,2cyclictest17584-21ssh23:09:203
369399110,2cyclictest15309-21ssh23:30:553
369399110,2cyclictest14306-21missed_timers19:30:203
369399110,1cyclictest26782-21apt-get22:25:023
369399110,1cyclictest26371-21diskmemload21:49:243
369399110,1cyclictest2115-21ssh00:10:443
369399110,1cyclictest15843-21ssh21:51:323
369399110,1cyclictest13987-21apt-get23:55:013
369399110,1cyclictest0-21swapper/320:15:163
368099119,1cyclictest12-21ksoftirqd/020:55:000
368099119,1cyclictest12-21ksoftirqd/020:00:000
369399103,6cyclictest449-21grep22:55:103
369399102,7cyclictest508-21dbus-daemon19:10:163
369399102,7cyclictest31762-21avahi-daemon19:25:013
369399101,7cyclictest19982-21systemd-journal21:30:263
369399101,2cyclictest1-21systemd20:29:573
369399100,3cyclictest27867-21/usr/sbin/munin23:40:163
369399100,3cyclictest10322-21ssh23:02:053
369399100,2cyclictest19357-21apt20:55:003
369399100,1cyclictest9922-21users21:20:193
369399100,1cyclictest26371-21diskmemload21:58:153
369399100,1cyclictest2582-21ssh23:46:143
369399100,1cyclictest24393-21apt-get21:05:113
369399100,1cyclictest21583-21apt-get21:00:023
369399100,1cyclictest21215-21ssh22:20:203
369399100,1cyclictest19877-21ssh00:28:213
369399100,1cyclictest16352-21ssh21:26:513
369399100,1cyclictest16352-21ssh21:26:513
369399100,1cyclictest12704-21ssh22:39:283
369399100,1cyclictest0-21swapper/323:25:033
368099107,2cyclictest12-21ksoftirqd/023:00:010
368099107,2cyclictest12-21ksoftirqd/021:00:000
368099107,2cyclictest12-21ksoftirqd/000:00:000
36939992,6cyclictest30654-21snmpd20:40:463
36939992,6cyclictest21149-21chrt19:45:183
36939991,7cyclictest17005-21cat19:35:223
36939991,3cyclictest22825-21fschecks_count19:50:163
36939991,2cyclictest19982-21systemd-journal00:15:003
36939991,1cyclictest10386-21apt-get20:35:103
36939990,3cyclictest6460-21cut19:15:023
36939990,3cyclictest1190-21awk20:15:003
36939990,3cyclictest11661-21tune2fs19:25:133
36939990,3cyclictest11661-21tune2fs19:25:133
36939990,2cyclictest26604-21run-parts20:00:003
36939990,2cyclictest21688-21apt-key21:00:013
36939990,2cyclictest19982-21systemd-journal20:00:183
36939990,2cyclictest16811-21cron20:50:003
36939990,1cyclictest20591-21ssh22:45:073
36939990,1cyclictest19982-21systemd-journal20:32:523
36939990,1cyclictest0-21swapper/319:40:213
36849994,3cyclictest19982-21systemd-journal19:52:241
36809998,1cyclictest12-21ksoftirqd/023:45:000
36809998,1cyclictest12-21ksoftirqd/022:50:010
36809998,1cyclictest12-21ksoftirqd/022:25:010
36809998,1cyclictest12-21ksoftirqd/022:20:000
36809998,1cyclictest12-21ksoftirqd/019:55:000
36809997,2cyclictest12-21ksoftirqd/019:30:010
36809997,2cyclictest12-21ksoftirqd/019:30:010
36809996,2cyclictest12-21ksoftirqd/023:55:000
36809996,2cyclictest12-21ksoftirqd/023:40:010
36809996,2cyclictest12-21ksoftirqd/022:00:010
36809992,4cyclictest12-21ksoftirqd/020:05:140
36899984,3cyclictest508-21dbus-daemon21:15:252
36899984,3cyclictest508-21dbus-daemon00:25:402
36899984,3cyclictest19982-21systemd-journal23:57:312
36899983,4cyclictest0-21swapper/222:19:232
36899983,3cyclictest7661-21lldpd22:06:162
36899983,3cyclictest19982-21systemd-journal20:37:472
36899982,3cyclictest7908-21grep20:30:002
36849983,3cyclictest19982-21systemd-journal21:47:081
36849983,3cyclictest19982-21systemd-journal19:46:081
36849980,5cyclictest25421-21ls22:50:011
36809987,1cyclictest12-21ksoftirqd/020:05:000
36809987,1cyclictest12-21ksoftirqd/000:00:000
36809986,1cyclictest12-21ksoftirqd/022:40:010
36809985,2cyclictest12-21ksoftirqd/023:20:020
36809985,2cyclictest12-21ksoftirqd/021:50:020
36809985,2cyclictest12-21ksoftirqd/020:30:000
36809984,3cyclictest508-21dbus-daemon20:22:500
36809984,3cyclictest19982-21systemd-journal23:04:560
36809982,4cyclictest26541-21apt-get21:35:120
36809982,2cyclictest12-21ksoftirqd/023:10:080
18389280,0sleep00-21swapper/019:40:150
36899973,3cyclictest668-21runrttasks00:14:302
36899973,3cyclictest508-21dbus-daemon19:54:282
36899973,3cyclictest30654-21snmpd20:58:102
36899973,3cyclictest22018-21cat00:00:162
36899973,3cyclictest19982-21systemd-journal21:28:442
36899973,3cyclictest19982-21systemd-journal21:28:442
36899973,3cyclictest19982-21systemd-journal21:02:542
36899973,3cyclictest19982-21systemd-journal20:00:192
36899973,3cyclictest1-21systemd21:45:272
36899973,3cyclictest1-21systemd00:07:432
36899973,3cyclictest0-21swapper/223:31:382
36899973,3cyclictest0-21swapper/223:25:372
36899973,3cyclictest0-21swapper/200:38:112
36899973,2cyclictest11736-21in:imuxsock23:08:172
36899973,1cyclictest0-21swapper/221:21:272
36899973,1cyclictest0-21swapper/200:23:002
36899971,4cyclictest19982-21systemd-journal22:23:512
36849974,2cyclictest11736-21in:imuxsock22:50:081
36849974,2cyclictest11736-21in:imuxsock19:33:371
36849974,2cyclictest10136-21ls20:34:591
36849973,4cyclictest2212-21kworker/1:220:11:391
36849973,3cyclictest7661-21lldpd21:59:531
36849973,3cyclictest508-21dbus-daemon23:25:351
36849973,3cyclictest508-21dbus-daemon19:28:371
36849973,3cyclictest508-21dbus-daemon19:28:371
36849973,3cyclictest20697-21ls00:00:001
36849973,3cyclictest20269-21df22:20:131
36849973,3cyclictest19982-21systemd-journal23:51:521
36849973,3cyclictest19982-21systemd-journal00:26:301
36849973,3cyclictest11736-21in:imuxsock21:44:131
36849973,3cyclictest0-21swapper/100:00:491
36849973,2cyclictest19982-21systemd-journal00:24:001
36849973,1cyclictest0-21swapper/122:30:211
36849973,1cyclictest0-21swapper/121:37:201
36849972,4cyclictest0-21swapper/120:59:221
36849972,3cyclictest19982-21systemd-journal20:47:281
36849970,4cyclictest14316-21idleruntime-cro21:25:001
36809976,1cyclictest12-21ksoftirqd/023:10:010
36809976,1cyclictest12-21ksoftirqd/020:45:000
36809975,2cyclictest12-21ksoftirqd/020:50:000
36809975,2cyclictest12-21ksoftirqd/019:20:010
36809974,2cyclictest12-21ksoftirqd/022:10:010
36809973,3cyclictest30654-21snmpd19:38:200
36809973,3cyclictest30654-21snmpd00:26:420
36809973,2cyclictest19982-21systemd-journal00:16:290
36809973,1cyclictest12-21ksoftirqd/022:00:100
36809973,1cyclictest0-21swapper/021:40:540
36809973,1cyclictest0-21swapper/021:28:420
36809973,1cyclictest0-21swapper/021:28:420
36809972,4cyclictest131rcuc/022:50:470
36809972,3cyclictest141rcu_preempt21:20:400
36809972,1cyclictest141rcu_preempt20:35:220
36809971,4cyclictest141rcu_preempt22:26:560
36809971,4cyclictest12-21ksoftirqd/023:48:360
36809971,3cyclictest12-21ksoftirqd/022:10:150
36809971,3cyclictest12-21ksoftirqd/021:31:020
36809971,1cyclictest140361ssh00:21:110
36809970,5cyclictest508-21dbus-daemon21:02:540
36809970,3cyclictest26545-21ntp_states23:15:150
36809970,1cyclictest0-21swapper/020:30:120
30501270,0sleep10-21swapper/122:28:461
36899963,2cyclictest8005-21unixbench_multi00:15:202
36899963,2cyclictest668-21runrttasks22:32:212
36899963,2cyclictest668-21runrttasks22:10:112
36899963,2cyclictest508-21dbus-daemon23:50:002
36899963,2cyclictest3548-21cat20:20:002
36899963,2cyclictest30654-21snmpd20:21:092
36899963,2cyclictest26371-21diskmemload23:16:002
36899963,2cyclictest21772-21sshd19:45:452
36899963,2cyclictest19564-21grep23:10:122
36899963,2cyclictest15570-21sh21:51:162
36899963,2cyclictest15408-21grep20:45:132
36899963,2cyclictest13231-21hddtemp_smartct20:40:142
36899963,2cyclictest1-21systemd21:55:042
36899962,3cyclictest8815-21cat20:30:152
36899962,3cyclictest7661-21lldpd20:10:192
36899962,3cyclictest30654-21snmpd23:45:472
36899962,3cyclictest30654-21snmpd23:20:152
36899962,3cyclictest30654-21snmpd22:45:262
36899962,3cyclictest30654-21snmpd22:41:282
36899962,3cyclictest30654-21snmpd21:30:052
36899962,3cyclictest30654-21snmpd21:06:562
36899962,3cyclictest30654-21snmpd20:06:092
36899962,3cyclictest30654-21snmpd19:55:092
36899962,3cyclictest30654-21snmpd19:26:202
36899962,3cyclictest30654-21snmpd19:26:202
36899962,3cyclictest30654-21snmpd19:15:142
36899962,3cyclictest27583-21ntpq21:35:172
36899962,3cyclictest27173-21expr21:10:172
36899962,3cyclictest26924-21ls22:00:132
36899962,3cyclictest26510-21cat22:50:112
36899962,3cyclictest2269-21sed21:40:192
36899962,3cyclictest1-21systemd19:34:532
36899962,1cyclictest0-21swapper/223:36:442
36899962,1cyclictest0-21swapper/222:35:372
36899962,1cyclictest0-21swapper/222:25:212
36899962,1cyclictest0-21swapper/200:31:042
36899960,6cyclictest0-21swapper/223:42:532
36899960,6cyclictest0-21swapper/219:20:252
36899960,4cyclictest19982-21systemd-journal22:58:032
36849963,2cyclictest8591-21ssh22:35:251
36849963,2cyclictest30654-21snmpd19:15:081
36849963,2cyclictest29741-11brltty19:11:061
36849963,2cyclictest29678-21sh23:41:181
36849963,2cyclictest27838-21tr00:35:151
36849963,2cyclictest21760-21fschecks_count00:30:141
36849963,2cyclictest1623-21head23:20:171
36849963,2cyclictest15020-21sh22:15:391
36849963,2cyclictest14791-21ntp_states22:40:171
36849963,2cyclictest13600-21cat21:50:111
36849963,2cyclictest11736-21in:imuxsock20:03:151
36849963,2cyclictest0-21swapper/122:06:341
36849963,1cyclictest0-21swapper/121:16:081
36849962,3cyclictest6426-21fw_packets20:25:131
36849962,3cyclictest30654-21snmpd20:54:041
36849962,3cyclictest30654-21snmpd19:59:571
36849962,3cyclictest30654-21snmpd19:24:201
36849962,3cyclictest27179-21cat21:10:171
36849962,3cyclictest21225-21run-parts23:35:171
36849962,3cyclictest1-21systemd21:06:141
36849962,3cyclictest0-21swapper/121:30:061
36849962,1cyclictest0-21swapper/123:45:141
36849962,1cyclictest0-21swapper/123:30:131
36849962,1cyclictest0-21swapper/123:07:021
36849962,1cyclictest0-21swapper/123:04:301
36849962,1cyclictest0-21swapper/122:12:121
36849962,1cyclictest0-21swapper/122:00:391
36849962,1cyclictest0-21swapper/100:15:071
36849961,4cyclictest0-21swapper/120:40:181
36849961,3cyclictest19982-21systemd-journal19:42:231
36849960,6cyclictest0-21swapper/121:25:461
36849960,6cyclictest0-21swapper/121:25:461
36849960,3cyclictest24144-21grep21:05:001
36849960,3cyclictest20039-21ls23:10:151
36849960,3cyclictest17671-21idleruntime-cro19:40:011
36849960,3cyclictest10360-21/usr/sbin/munin20:35:111
36849960,2cyclictest2391-21ntp_states20:15:171
36809963,2cyclictest8388-21sh23:26:020
36809963,2cyclictest1323-21apt21:15:110
36809962,3cyclictest24814-21tune2fs21:05:130
36809962,3cyclictest1631-21cat20:15:130
36809962,1cyclictest141rcu_preempt19:20:170
36809962,1cyclictest141rcu_preempt00:30:140
36809962,1cyclictest12-21ksoftirqd/019:45:150
36809962,1cyclictest12-21ksoftirqd/019:30:160
36809962,1cyclictest12-21ksoftirqd/000:36:260
36809961,3cyclictest29741-11brltty19:12:460
36809961,2cyclictest17775-21ssh22:42:350
36809960,4cyclictest0-21swapper/020:10:170
36809960,1cyclictest554-21uniq00:10:100
31176260,1sleep122-21ksoftirqd/120:07:461
28612260,0sleep10-21swapper/100:05:521
36899952,2cyclictest18206-21expr19:40:142
36899952,2cyclictest16052-21expr19:35:152
36899951,3cyclictest5164-21ntpd20:50:212
36849952,2cyclictest25412-21ssh23:15:051
36849952,2cyclictest0-21swapper/120:20:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional