You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-11 - 20:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Sat May 11, 2024 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
322352940,1sleep332233-21lspci12:27:013
318122650,0sleep031813-21kworker/u8:012:38:040
92002630,12sleep375550irq/24-eno1-rx-10:24:493
76922620,1sleep37659-21sshd10:11:593
311712580,0sleep30-21swapper/309:10:473
317412560,0sleep231738-21lspci11:42:422
290052560,0sleep10-21swapper/111:34:131
263192530,0sleep20-21swapper/212:09:582
238742530,0sleep10-21swapper/109:38:541
68232510,0sleep20-21swapper/209:24:322
196022510,0sleep10-21swapper/110:25:431
100912510,0sleep20-21swapper/211:24:162
226042500,0sleep022602-21bash12:20:370
180712500,0sleep20-21swapper/212:11:582
94682490,0sleep20-21swapper/211:26:562
326862490,0sleep1231rcuc/112:38:091
257762490,2sleep1565699cyclictest11:11:411
93712480,1sleep10-21swapper/112:00:111
31712480,1sleep0111rcuc/009:52:430
180212480,0sleep00-21swapper/011:49:540
114782480,0sleep311472-21sshd10:28:063
300392470,0sleep10-21swapper/109:42:371
177612470,0sleep20-21swapper/210:41:272
511724536,4sleep00-21swapper/007:05:370
103622410,2sleep275650irq/25-eno1-tx-12:16:512
61762390,0sleep10-21swapper/110:24:311
77742380,1sleep1565699cyclictest12:13:541
54442319,7sleep20-21swapper/207:08:512
54302298,7sleep30-21swapper/307:08:413
54332287,7sleep10-21swapper/107:08:421
204502270,0sleep30-21swapper/311:19:373
135722220,0sleep30-21swapper/309:50:203
318482190,0sleep30-21swapper/310:17:323
99082180,0sleep30-21swapper/312:38:593
93982170,2sleep1565699cyclictest10:12:091
5656991714,1cyclictest4183-21kworker/1:212:15:071
5651991716,0cyclictest25063-21lspci10:32:370
323332170,0sleep00-21swapper/009:30:150
282162170,0sleep3391rcuc/308:01:003
124892170,0sleep20-21swapper/210:31:212
5656991615,0cyclictest31550-21kworker/1:210:56:471
5656991615,0cyclictest2814-21lspci09:24:061
5656991615,0cyclictest19085-21kworker/1:011:16:111
5656991615,0cyclictest18101-21lspci11:27:411
5656991614,1cyclictest4183-21kworker/1:212:07:511
282762160,0sleep10-21swapper/109:26:311
189642160,0sleep00-21swapper/009:44:450
5651991514,0cyclictest32454-21lspci11:06:460
300512150,0sleep3391rcuc/309:26:413
154012150,0sleep215344-21sshd09:47:272
149972150,0sleep20-21swapper/209:25:162
5656991414,0cyclictest21273-21kworker/1:311:52:351
5656991414,0cyclictest18584-21kworker/1:109:12:481
5656991414,0cyclictest11767-21kworker/1:110:32:311
5656991413,0cyclictest3824-21kworker/1:010:15:221
5656991413,0cyclictest30099-21lspci12:24:071
5656991413,0cyclictest18164-21lspci09:15:471
236862140,0sleep00-21swapper/012:18:010
566899130,1cyclictest25100-21lspci12:15:183
5656991312,0cyclictest8438-21kworker/1:312:30:231
5656991312,0cyclictest241ktimersoftd/111:37:221
5656991312,0cyclictest14514-21kworker/1:208:55:271
5651991312,0cyclictest19074-21lspci11:27:460
26862130,1sleep30-21swapper/310:49:563
187072130,0sleep20-21swapper/211:58:132
150602130,0sleep00-21swapper/009:47:250
95112120,0sleep10-21swapper/111:21:221
5668991210,1cyclictest10402-21lspci11:07:383
566899120,1cyclictest25038-21systemd-cgroups12:04:183
566199120,11cyclictest0-21swapper/209:30:272
565699125,1cyclictest25-21ksoftirqd/111:01:031
5656991211,0cyclictest25147-21kworker/1:409:45:431
5656991211,0cyclictest10446-21kworker/1:008:45:141
5651991211,0cyclictest17176-21lspci09:15:410
5651991210,1cyclictest3445-21sshd10:52:580
565199120,11cyclictest31199-21bash12:10:190
322822120,2sleep1565699cyclictest10:05:001
26302120,0sleep20-21swapper/210:20:512
186662120,0sleep0111rcuc/009:25:350
566899119,1cyclictest29069-21sshd11:25:483
566899110,10cyclictest648-21systemd-logind11:23:283
566899110,10cyclictest0-21swapper/311:04:383
566899110,10cyclictest0-21swapper/309:46:483
566899110,0cyclictest0-21swapper/311:11:483
5661991110,0cyclictest0-21swapper/209:59:482
5661991110,0cyclictest0-21swapper/209:50:082
5661991110,0cyclictest0-21swapper/208:25:172
566199110,10cyclictest0-21swapper/210:00:592
566199110,10cyclictest0-21swapper/209:44:082
566199110,0cyclictest0-21swapper/211:51:182
565699119,1cyclictest24499-21sshd11:06:041
565699112,1cyclictest2913-21sshd10:43:141
565699111,1cyclictest25-21ksoftirqd/112:26:551
565699110,10cyclictest0-21swapper/111:56:341
565199119,1cyclictest23292-21lspci10:38:480
565199119,1cyclictest0-21swapper/011:31:390
565199113,1cyclictest9-21ksoftirqd/011:05:000
5651991110,0cyclictest23784-21lspci10:00:500
5651991110,0cyclictest15885-21lspci09:12:290
5651991110,0cyclictest12437-21lspci10:56:380
565199110,10cyclictest0-21swapper/011:50:480
566899109,0cyclictest0-21swapper/312:14:083
566899100,0cyclictest0-21swapper/309:34:083
566199109,0cyclictest0-21swapper/212:04:282
566199109,0cyclictest0-21swapper/211:47:182
566199109,0cyclictest0-21swapper/211:07:082
566199109,0cyclictest0-21swapper/210:17:082
566199109,0cyclictest0-21swapper/210:06:082
566199109,0cyclictest0-21swapper/208:50:172
566199109,0cyclictest0-21swapper/208:05:182
566199109,0cyclictest0-21swapper/208:05:172
566199108,1cyclictest31671-21lspci12:26:582
566199100,9cyclictest0-21swapper/207:47:582
566199100,0cyclictest0-21swapper/207:25:272
565699109,0cyclictest7623-21kworker/1:007:20:281
565699109,0cyclictest27230-21kworker/1:111:46:531
565699109,0cyclictest22660-21kworker/1:108:05:211
565699109,0cyclictest22660-21kworker/1:108:05:211
565699109,0cyclictest0-21swapper/109:08:521
565699108,1cyclictest25-21ksoftirqd/110:38:531
565699101,1cyclictest25-21ksoftirqd/110:09:161
565699101,1cyclictest131rcu_sched10:49:521
565699101,1cyclictest131rcu_sched09:59:391
5656991010,0cyclictest18584-21kworker/1:109:00:291
565699100,0cyclictest0-21swapper/109:34:421
565699100,0cyclictest0-21swapper/107:58:521
565199109,0cyclictest25545-21sshd11:58:480
565199109,0cyclictest0-21swapper/011:35:080
565199109,0cyclictest0-21swapper/011:17:080
565199108,1cyclictest9-21ksoftirqd/011:20:140
565199108,1cyclictest16811-21sshd12:09:080
565199101,1cyclictest131rcu_sched09:55:170
565199100,1cyclictest131rcu_sched12:25:030
565199100,1cyclictest131rcu_sched12:02:540
56689998,0cyclictest0-21swapper/311:38:083
56689998,0cyclictest0-21swapper/310:34:483
56689990,0cyclictest0-21swapper/312:32:493
56619998,0cyclictest0-21swapper/211:02:582
56619998,0cyclictest0-21swapper/208:39:572
56619997,1cyclictest3899-21lspci09:11:182
56619996,2cyclictest684-21polkitd10:45:302
56569997,1cyclictest25-21ksoftirqd/109:50:211
56569997,1cyclictest14514-21kworker/1:208:50:151
56569991,1cyclictest684-21polkitd10:52:331
56519998,0cyclictest14092-21lspci10:06:210
56519998,0cyclictest0-21swapper/009:22:380
56519997,1cyclictest9-21ksoftirqd/011:12:090
56519992,1cyclictest9-21ksoftirqd/010:16:560
56519991,7cyclictest28452-21lspci09:39:200
56519991,2cyclictest131rcu_sched10:26:120
56519990,8cyclictest25606-21kworker/0:010:22:300
56519990,0cyclictest9-21ksoftirqd/012:31:130
9002280,1sleep19006-21gltestperf08:35:141
56689987,0cyclictest13430-21lspci10:44:183
56689987,0cyclictest0-21swapper/309:44:583
56689986,1cyclictest29632-21lspci09:23:263
56689986,1cyclictest21775-21grep10:00:403
56689986,1cyclictest17499-21kworker/u8:109:59:483
56689985,2cyclictest649-21avahi-daemon11:40:383
56689985,2cyclictest649-21avahi-daemon09:15:533
56619986,1cyclictest6540-21lspci10:37:082
56619986,1cyclictest3766-21lspci09:36:562
56619986,1cyclictest33-21ksoftirqd/209:15:162
56619986,1cyclictest3180-21bash10:14:482
56619986,1cyclictest31783-21lspci11:39:592
56619986,1cyclictest18648-21sshd11:33:192
56569987,0cyclictest0-21swapper/108:00:131
56569986,1cyclictest25-21ksoftirqd/111:40:181
56569986,1cyclictest14326-21kworker/1:007:30:211
56569986,1cyclictest14270-21sshd07:29:521
56519986,1cyclictest9-21ksoftirqd/011:42:090
56519984,3cyclictest649-21avahi-daemon10:41:320
56519981,1cyclictest9-21ksoftirqd/010:11:200
56519980,4cyclictest14305-21munin-run07:30:000
56519980,4cyclictest12285-21idleruntime-cro07:25:000
56519980,1cyclictest131rcu_sched10:45:160
4753280,1sleep0111rcuc/008:25:120
56689975,1cyclictest700-21gdbus11:55:053
56689975,1cyclictest30141-21lspci12:21:193
56689975,1cyclictest27501-21lspci11:45:093
56689975,1cyclictest20294-21sshd11:30:353
56689975,1cyclictest20155-21lspci11:50:033
56689975,1cyclictest0-21swapper/310:55:383
56689974,2cyclictest684-21polkitd10:51:193
56689974,2cyclictest649-21avahi-daemon10:37:043
56689974,2cyclictest1993-21sshd10:05:133
56689974,2cyclictest17299-21snmpd09:09:363
56619975,1cyclictest26452-21lspci10:29:342
56619975,1cyclictest22889-21sshd11:17:002
56619975,1cyclictest15118-21lspci12:31:042
56619974,2cyclictest700-21gdbus12:36:492
56619974,2cyclictest700-21gdbus11:11:352
56619974,2cyclictest649-21avahi-daemon12:20:582
56619972,2cyclictest19316-21sshd10:57:142
56619972,2cyclictest10980-21sshd10:50:362
56569977,0cyclictest7623-21kworker/1:007:16:591
56569977,0cyclictest7623-21kworker/1:007:14:551
56569977,0cyclictest395-21kworker/1:308:20:381
56569976,0cyclictest0-21swapper/107:46:521
4815270,5sleep35078-21cut12:05:203
23568270,1sleep323706-21sshd09:35:443
56689964,1cyclictest17299-21snmpd07:41:573
56689964,1cyclictest17299-21snmpd07:10:083
56689963,2cyclictest17299-21snmpd08:53:473
56689963,2cyclictest17299-21snmpd08:48:143
56689963,2cyclictest17299-21snmpd08:13:553
56689963,2cyclictest17299-21snmpd08:08:413
56689963,2cyclictest17299-21snmpd08:08:403
56619964,1cyclictest17299-21snmpd07:15:562
56619960,0cyclictest0-21swapper/208:45:082
56569966,0cyclictest395-21kworker/1:308:26:191
56569966,0cyclictest14326-21kworker/1:007:42:591
56569966,0cyclictest14326-21kworker/1:007:36:041
56569966,0cyclictest10446-21kworker/1:008:43:091
56519964,1cyclictest649-21avahi-daemon08:05:010
56519963,2cyclictest17299-21snmpd08:56:530
13443260,0sleep3391rcuc/307:25:213
56689953,1cyclictest8001-21ntp_offset07:15:003
56689953,1cyclictest649-21avahi-daemon08:25:003
56689953,1cyclictest25163-21cat07:55:143
56689953,1cyclictest21127-21cat07:45:163
56689953,1cyclictest17299-21snmpd09:03:543
56689953,1cyclictest17299-21snmpd08:55:213
56689953,1cyclictest17299-21snmpd08:41:503
56689953,1cyclictest17299-21snmpd08:36:563
56689953,1cyclictest17299-21snmpd08:30:263
56689953,1cyclictest17299-21snmpd08:21:013
56689953,1cyclictest17299-21snmpd08:16:553
56689953,1cyclictest17299-21snmpd07:51:123
56689953,1cyclictest17299-21snmpd07:30:123
56689953,1cyclictest10728-21df07:20:113
56689952,2cyclictest17299-21snmpd07:36:093
56619954,0cyclictest0-21swapper/207:32:272
56619953,1cyclictest6625-21/usr/sbin/munin08:30:132
56619953,1cyclictest6328-21cat07:10:152
56619953,1cyclictest35762sleep208:20:222
56619953,1cyclictest23164-21cat07:50:132
56619953,1cyclictest20150-21unixbench_multi09:00:282
56619953,1cyclictest18687-21cpuspeed_turbos07:40:132
56619953,1cyclictest18175-21grep08:55:272
56619953,1cyclictest17299-21snmpd09:05:122
56619953,1cyclictest17299-21snmpd08:40:142
56619953,1cyclictest17299-21snmpd08:17:492
56619953,1cyclictest17299-21snmpd08:00:122
56619953,1cyclictest17299-21snmpd07:55:062
56619953,1cyclictest17299-21snmpd07:22:092
56619953,1cyclictest1017-21runrttasks07:37:032
56619950,0cyclictest0-21swapper/208:11:272
56569955,0cyclictest22660-21kworker/1:108:10:331
56569953,1cyclictest17299-21snmpd08:30:461
56569953,1cyclictest17299-21snmpd08:16:191
56569953,1cyclictest17299-21snmpd07:51:031
56519955,0cyclictest14282-21kworker/0:007:51:040
56519953,1cyclictest649-21avahi-daemon07:44:080
56519953,1cyclictest26038-21unixbench_multi07:55:260
56519953,1cyclictest21372-21cat09:05:140
56519953,1cyclictest20400-21latency_hist07:45:000
56519953,1cyclictest17299-21snmpd09:01:250
56519953,1cyclictest17299-21snmpd08:45:090
56519953,1cyclictest17299-21snmpd08:30:010
56519953,1cyclictest17299-21snmpd08:20:260
56519953,1cyclictest17299-21snmpd08:15:340
56519953,1cyclictest17299-21snmpd08:05:130
56519953,1cyclictest17299-21snmpd08:05:130
56519953,1cyclictest17299-21snmpd07:16:500
56519953,1cyclictest15127-21grep08:50:120
56519953,1cyclictest14989-21cat07:30:130
56519953,1cyclictest11191-21cat08:40:140
56519952,2cyclictest646-21dbus-daemon07:10:010
56519952,2cyclictest17299-21snmpd07:38:520
56519952,2cyclictest10098-21taskset08:35:400
56519942,1cyclictest0-21swapper/008:10:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional