You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 09:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Wed Nov 19, 2025 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
216352900,0sleep021634-21bash00:18:560
77182800,0sleep10-21swapper/123:57:341
205062620,0sleep30-21swapper/322:15:313
262452610,0sleep00-21swapper/019:25:170
262452610,0sleep00-21swapper/019:25:160
34212600,0sleep30-21swapper/323:57:083
240702570,0sleep00-21swapper/000:08:590
89052560,1sleep30-21swapper/321:45:393
6572560,1sleep2652-21lspci21:28:562
49162540,0sleep10-21swapper/122:49:301
195012540,1sleep119504-21bash23:42:321
70022520,0sleep3391rcuc/321:07:543
47132520,0sleep20-21swapper/221:51:392
148202520,0sleep20-21swapper/200:25:192
169282510,1sleep216921-21sshd23:00:202
101072500,0sleep10-21swapper/122:17:451
276902490,0sleep00-21swapper/022:38:470
47402470,1sleep04737-21lspci23:34:380
308832470,1sleep230884-21grep22:07:012
1824424736,7sleep10-21swapper/119:05:451
201592460,0sleep20-21swapper/222:47:422
188372450,0sleep30-21swapper/322:34:423
280702430,0sleep30-21swapper/322:22:443
222302420,1sleep3401ktimersoftd/323:36:173
204082420,0sleep10-21swapper/121:18:021
16572410,0sleep20-21swapper/223:47:142
17822370,1sleep21785-21lspci22:20:112
21312340,1sleep32122-21bash21:16:113
1827323220,7sleep00-21swapper/019:06:070
182122329,7sleep20-21swapper/219:05:262
185632298,7sleep30-21swapper/319:09:443
322922180,1sleep232289-21sshd21:22:232
260342170,1sleep326032-21lspci00:33:333
236642170,0sleep30-21swapper/322:44:513
277322160,0sleep30-21swapper/300:12:463
25182160,0sleep10-21swapper/122:20:141
194262160,0sleep20-21swapper/223:10:212
184482160,0sleep1231rcuc/100:32:431
315732150,0sleep20-21swapper/223:34:042
31312150,0sleep2311rcuc/221:06:162
25412150,0sleep00-21swapper/023:18:200
47452140,1sleep30-21swapper/323:28:093
281372140,0sleep20-21swapper/222:51:472
251642140,0sleep00-21swapper/023:39:550
1871499140,1cyclictest31676-21sshd22:00:363
1871499140,1cyclictest1-21systemd21:26:563
18707991412,1cyclictest48-21kauditd21:11:042
41912130,0sleep30-21swapper/321:51:363
19802130,0sleep10-21swapper/122:32:591
1869799130,1cyclictest9354-21rm23:19:021
18693991312,0cyclictest6503-21kworker/0:122:10:270
18693991311,1cyclictest14371-21kworker/0:000:34:180
129812130,0sleep10-21swapper/121:33:201
18714991211,0cyclictest0-21swapper/323:12:053
1871499120,1cyclictest4136-21lspci21:32:263
1871499120,1cyclictest25065-21tty22:06:263
1871499120,11cyclictest0-21swapper/323:08:363
18707991210,1cyclictest13681-21sshd21:49:242
1870799120,11cyclictest28904-21sshd22:13:142
1870799120,11cyclictest17163-21sshd00:08:152
1870799120,11cyclictest0-21swapper/221:55:442
1870799120,11cyclictest0-21swapper/200:36:052
1870799120,0cyclictest0-21swapper/221:37:442
18697991210,1cyclictest23826-21lspci00:15:421
18693991210,1cyclictest1048-21sshd00:37:490
18693991210,1cyclictest0-21swapper/022:55:190
18693991210,0cyclictest7279-21kworker/0:023:56:300
1869399120,1cyclictest796-21sshd23:11:390
134172120,0sleep10-21swapper/121:39:491
18714991110,0cyclictest0-21swapper/321:58:153
18714991110,0cyclictest0-21swapper/300:38:163
18714991110,0cyclictest0-21swapper/300:04:153
1871499110,1cyclictest3204-21bash22:45:553
1871499110,10cyclictest0-21swapper/322:37:163
1871499110,10cyclictest0-21swapper/300:26:463
1870799119,1cyclictest0-21swapper/219:20:142
18707991110,0cyclictest31863-21sshd22:29:342
18707991110,0cyclictest0-21swapper/223:56:052
18707991110,0cyclictest0-21swapper/223:15:552
1870799110,10cyclictest0-21swapper/200:14:552
1870799110,0cyclictest0-21swapper/223:39:252
1869799119,1cyclictest0-21swapper/123:51:211
18697991110,0cyclictest0-21swapper/121:21:221
18697991110,0cyclictest0-21swapper/100:22:221
1869799110,10cyclictest0-21swapper/123:20:121
1869799110,10cyclictest0-21swapper/122:29:421
1869799110,0cyclictest0-21swapper/123:34:521
1869399119,1cyclictest0-21swapper/022:52:590
1869399119,1cyclictest0-21swapper/022:23:390
1869399119,1cyclictest0-21swapper/021:54:190
18693991110,0cyclictest0-21swapper/022:40:490
18693991110,0cyclictest0-21swapper/021:49:090
18693991110,0cyclictest0-21swapper/000:12:190
1869399110,10cyclictest15609-21bash22:18:190
1869399110,10cyclictest0-21swapper/023:29:490
109272110,4sleep110926-21grepconf.sh22:01:471
86632100,0sleep30-21swapper/322:27:133
1871499109,0cyclictest0-21swapper/323:41:263
1871499109,0cyclictest0-21swapper/323:03:563
1871499109,0cyclictest0-21swapper/319:57:363
1871499108,1cyclictest9142-21bash00:17:363
1871499108,1cyclictest8624-21lspci23:15:363
1871499108,1cyclictest20954-21sshd21:40:263
1871499100,0cyclictest0-21swapper/300:24:163
1870799109,0cyclictest0-21swapper/223:40:352
1870799109,0cyclictest0-21swapper/223:07:452
1870799108,1cyclictest29733-21bash22:19:442
1870799100,0cyclictest0-21swapper/221:42:042
1870799100,0cyclictest0-21swapper/219:45:242
1869799109,0cyclictest10460-21sshd21:09:221
1869799109,0cyclictest0-21swapper/121:54:421
1869799109,0cyclictest0-21swapper/121:28:021
1869799109,0cyclictest0-21swapper/119:45:011
1869799109,0cyclictest0-21swapper/100:07:121
1869799108,1cyclictest21760-21lspci00:26:021
1869799100,0cyclictest0-21swapper/121:42:121
1869399109,0cyclictest9777-21sshd22:30:290
1869399109,0cyclictest18305-21kworker/0:022:00:120
1869399109,0cyclictest0-21swapper/021:43:390
1869399109,0cyclictest0-21swapper/021:34:590
1869399109,0cyclictest0-21swapper/021:11:590
1869399109,0cyclictest0-21swapper/000:01:590
1869399108,1cyclictest4576-21sshd23:40:590
1869399107,2cyclictest3645-21kworker/0:223:24:460
1869399100,0cyclictest0-21swapper/023:01:090
1869399100,0cyclictest0-21swapper/000:26:190
187149998,0cyclictest75550irq/24-eno1-rx-23:50:163
187149998,0cyclictest0-21swapper/321:14:163
187149998,0cyclictest0-21swapper/321:01:363
187149998,0cyclictest0-21swapper/319:30:263
187149990,0cyclictest0-21swapper/320:42:363
187079998,0cyclictest0-21swapper/223:52:252
187079998,0cyclictest0-21swapper/220:35:242
187079997,1cyclictest620-13audispd00:19:452
187079997,1cyclictest14368-21sshd22:43:542
186979998,0cyclictest0-21swapper/123:36:021
186979998,0cyclictest0-21swapper/122:38:511
186979998,0cyclictest0-21swapper/120:52:021
186979998,0cyclictest0-21swapper/119:51:501
186979997,1cyclictest0-21swapper/122:14:311
186939998,0cyclictest21825-21lspci21:56:300
186939998,0cyclictest0-21swapper/020:00:190
186939996,2cyclictest4271-21kworker/0:023:06:060
186939996,2cyclictest3128-21kworker/0:122:47:370
186939992,3cyclictest15302-21munin-run21:40:000
187149986,1cyclictest32083-21lspci00:06:293
187149986,1cyclictest16494-21lspci21:24:043
187149985,2cyclictest649-21avahi-daemon23:34:573
187079987,0cyclictest0-21swapper/222:37:342
187079987,0cyclictest0-21swapper/222:32:242
187079985,2cyclictest649-21avahi-daemon21:17:242
187079985,1cyclictest30418-21sshd00:23:292
187079984,3cyclictest649-21avahi-daemon23:28:332
187079984,3cyclictest15423-21sshd21:30:182
186979986,1cyclictest25-21ksoftirqd/122:55:201
186979986,1cyclictest19924-21lspci21:59:331
186939987,0cyclictest15612-21sshd21:23:580
186939987,0cyclictest0-21swapper/020:40:290
186939986,1cyclictest9-21ksoftirqd/023:53:230
186939986,1cyclictest9-21ksoftirqd/023:48:270
186939986,1cyclictest9-21ksoftirqd/021:25:180
186939986,1cyclictest9-21ksoftirqd/021:15:220
186939986,1cyclictest14284-21lspci00:21:390
186939985,2cyclictest649-21avahi-daemon22:27:240
8651270,1sleep08692-21lspci21:08:370
29339270,5sleep129519-21sshd22:51:551
20659271,1sleep021392-21yum19:15:010
187149976,0cyclictest0-21swapper/321:36:063
187149975,1cyclictest649-21avahi-daemon23:45:433
187149974,2cyclictest28135-21sshd23:20:483
187149974,2cyclictest13346-21sshd22:56:453
187149972,2cyclictest616-13auditd22:15:013
187149970,6cyclictest19635-21sshd22:50:533
187079975,1cyclictest700-21gdbus22:00:232
187079975,1cyclictest25527-21lspci23:23:502
187079974,2cyclictest649-21avahi-daemon00:00:482
187079973,1cyclictest0-21swapper/222:56:342
187079971,2cyclictest11851-21sshd00:31:592
186979976,0cyclictest0-21swapper/123:46:021
186979976,0cyclictest0-21swapper/121:12:221
186979976,0cyclictest0-21swapper/120:40:211
186979975,1cyclictest4096-21lspci00:38:081
186979975,1cyclictest32713-21sshd00:00:081
186979975,1cyclictest26543-21lspci23:10:581
186979975,1cyclictest26004-21lspci23:01:141
186979975,1cyclictest10827-21lspci21:45:501
186979974,2cyclictest692-21NetworkManager22:05:291
186979974,2cyclictest649-21avahi-daemon23:26:531
186979974,2cyclictest649-21avahi-daemon23:06:061
186939974,2cyclictest10909-21sshd22:05:070
186939970,6cyclictest0-21swapper/019:30:390
187149964,1cyclictest17299-21snmpd20:45:133
187149964,1cyclictest17299-21snmpd20:33:273
187149962,3cyclictest11932-21if_eno220:10:183
187149960,5cyclictest0-21swapper/320:01:263
187079964,1cyclictest19953-21grep20:30:142
187079960,5cyclictest0-21swapper/220:03:342
187079960,5cyclictest0-21swapper/219:31:042
187079960,5cyclictest0-21swapper/219:11:542
186979965,0cyclictest0-21swapper/121:00:521
186979964,1cyclictest649-21avahi-daemon00:10:221
186979964,1cyclictest646-21dbus-daemon22:40:041
186979963,2cyclictest17299-21snmpd20:00:001
186979963,2cyclictest17299-21snmpd19:44:111
186939964,1cyclictest7146-21kworker/0:020:35:160
186939964,1cyclictest646-21dbus-daemon19:50:000
186939964,1cyclictest20886-21latency_hist19:15:000
186939962,3cyclictest13288-21perf20:15:000
187149953,1cyclictest29900-21grep20:55:133
187149953,1cyclictest23787-21gltestperf19:20:153
187149953,1cyclictest18058-21cat20:25:133
187149953,1cyclictest17299-21snmpd20:36:013
187149953,1cyclictest17299-21snmpd20:15:573
187149953,1cyclictest17299-21snmpd19:54:093
187149953,1cyclictest17299-21snmpd19:46:113
187149953,1cyclictest17299-21snmpd19:37:593
187149953,1cyclictest17299-21snmpd19:28:203
187149953,1cyclictest17299-21snmpd19:28:203
187149953,1cyclictest17299-21snmpd19:16:143
187149953,1cyclictest17299-21snmpd19:10:103
187149953,1cyclictest15843-21gltestperf20:20:153
187149950,4cyclictest28005-21df_inode20:50:143
187149950,1cyclictest10304-21ntp_states20:05:183
187079954,0cyclictest0-21swapper/220:09:442
187079954,0cyclictest0-21swapper/219:54:342
187079953,1cyclictest682-21lsmd20:25:422
187079953,1cyclictest28227-21cat20:50:172
187079953,1cyclictest25703-21/usr/sbin/munin20:45:222
187079953,1cyclictest20934-21sort19:14:592
187079953,1cyclictest17299-21snmpd21:02:342
187079953,1cyclictest17299-21snmpd20:56:102
187079953,1cyclictest17299-21snmpd20:41:462
187079953,1cyclictest17299-21snmpd20:10:182
187079953,1cyclictest17299-21snmpd19:55:212
187079953,1cyclictest17299-21snmpd19:37:292
187079953,1cyclictest17299-21snmpd19:25:262
187079953,1cyclictest17299-21snmpd19:25:262
187079953,1cyclictest15537-21/usr/sbin/munin20:20:172
187079953,1cyclictest14372-21ntp_states20:15:212
186979953,1cyclictest9929-21cat20:05:151
186979953,1cyclictest649-21avahi-daemon20:35:001
186979953,1cyclictest26155-21cat20:45:161
186979953,1cyclictest17299-21snmpd20:57:131
186979953,1cyclictest17299-21snmpd20:35:091
186979953,1cyclictest17299-21snmpd20:28:571
186979953,1cyclictest17299-21snmpd20:20:061
186979953,1cyclictest17299-21snmpd20:11:061
186979953,1cyclictest17299-21snmpd20:02:541
186979953,1cyclictest17299-21snmpd19:36:591
186979953,1cyclictest17299-21snmpd19:30:321
186979953,1cyclictest17299-21snmpd19:25:051
186979953,1cyclictest17299-21snmpd19:25:051
186979953,1cyclictest17299-21snmpd19:21:291
186979953,1cyclictest17299-21snmpd19:17:321
186979953,1cyclictest17299-21snmpd19:13:501
186979950,0cyclictest0-21swapper/120:15:311
186939953,1cyclictest32108-21cat19:40:140
186939953,1cyclictest31560-21sadc21:00:000
186939953,1cyclictest17299-21snmpd20:52:580
186939953,1cyclictest17299-21snmpd20:31:480
186939953,1cyclictest17299-21snmpd20:26:510
186939953,1cyclictest17299-21snmpd20:20:420
186939953,1cyclictest17299-21snmpd20:18:450
186939953,1cyclictest17299-21snmpd20:05:300
186939953,1cyclictest17299-21snmpd19:58:360
186939953,1cyclictest17299-21snmpd19:24:230
186939953,1cyclictest1049-21rpcbind20:47:290
186939950,0cyclictest0-21swapper/019:38:090
187149942,1cyclictest17299-21snmpd19:40:053
187079942,1cyclictest0-21swapper/219:40:122
186939942,1cyclictest3572-21expr19:50:140
186939942,1cyclictest31799-21/usr/sbin/munin21:00:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional