You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 02:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Tue May 13, 2025 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48232990,0sleep10-21swapper/123:48:331
220922970,0sleep322091-21bash23:13:103
20462970,0sleep00-21swapper/021:09:470
107952840,0sleep010794-21sshd00:18:400
13292820,1sleep01324-21sshd22:52:360
298152710,0sleep0111rcuc/023:50:520
174152580,0sleep10-21swapper/123:55:541
252102560,1sleep325188-21sshd00:34:173
280132550,0sleep30-21swapper/323:07:333
236992530,0sleep10-21swapper/121:49:451
25032520,1sleep0111rcuc/000:38:390
131062520,0sleep00-21swapper/000:08:240
236112510,0sleep00-21swapper/000:26:530
177892510,1sleep09-21ksoftirqd/020:40:000
176992510,1sleep017671-21sshd21:24:170
127452510,0sleep10-21swapper/122:56:461
327252500,1sleep232724-21sshd23:01:482
205002500,1sleep220496-21lspci21:43:132
10082500,0sleep10-21swapper/100:31:351
126372480,0sleep30-21swapper/300:05:053
125402480,0sleep10-21swapper/122:28:581
25962440,0sleep11304499cyclictest22:43:281
25402420,2sleep01303899cyclictest00:31:470
61892360,1sleep26170-21sshd22:25:152
231862360,0sleep30-21swapper/300:16:153
1286623525,6sleep30-21swapper/319:09:153
1273022926,1sleep00-21swapper/019:07:310
111412286,7sleep20-21swapper/219:05:062
144402220,0sleep10-21swapper/119:10:201
13044992221,0cyclictest13295-21lspci21:33:101
13044992221,0cyclictest13295-21lspci21:33:101
1286422211,7sleep10-21swapper/119:09:131
8112200,0sleep2311rcuc/222:49:312
51322200,0sleep30-21swapper/322:31:173
194962190,2sleep21304999cyclictest21:18:142
28502180,0sleep10-21swapper/122:00:061
214722170,2sleep11304499cyclictest22:17:211
76612160,0sleep0101ktimersoftd/022:06:420
6722160,0sleep20-21swapper/222:18:302
79462150,10sleep375550irq/24-eno1-rx-23:48:513
240142150,1sleep30-21swapper/322:51:393
13044991514,0cyclictest0-21swapper/122:07:131
117042150,1sleep2311rcuc/223:36:522
211842140,0sleep321182-21bash21:37:013
1305799140,13cyclictest692-21NetworkManager21:13:553
1305799140,12cyclictest2564-21sshd21:16:353
13038991412,1cyclictest11767-21sshd21:11:020
269192130,0sleep10-21swapper/100:23:501
241922130,0sleep20-21swapper/200:16:232
13057991312,0cyclictest0-21swapper/323:50:263
1304999130,12cyclictest0-21swapper/222:14:592
13044991311,1cyclictest1-21systemd00:04:041
1303899130,12cyclictest1011-21sshd23:56:230
13057991210,1cyclictest0-21swapper/321:45:053
13057991210,1cyclictest0-21swapper/321:42:553
1305799120,1cyclictest12129-21sshd21:26:453
1305799120,1cyclictest0-21swapper/321:55:153
1305799120,11cyclictest28249-21sshd22:36:363
1305799120,11cyclictest0-21swapper/323:26:263
1305799120,11cyclictest0-21swapper/322:57:063
13049991211,0cyclictest321ktimersoftd/223:21:012
13049991210,1cyclictest646-21dbus-daemon23:31:192
13049991210,1cyclictest17015-21sshd22:35:292
1304999120,1cyclictest2185-21sshd00:28:092
1304999120,11cyclictest19727-21bash22:41:592
1304999120,11cyclictest0-21swapper/223:08:592
13044991211,0cyclictest2071-21lspci21:35:131
13044991210,1cyclictest19578-21lspci00:09:041
1304499120,1cyclictest646-21dbus-daemon22:36:541
1304499120,1cyclictest29120-21lspci21:22:131
13038991210,1cyclictest0-21swapper/000:03:030
1303899120,11cyclictest24400-21lspci23:07:130
1303899120,11cyclictest0-21swapper/022:58:330
1303899120,11cyclictest0-21swapper/021:51:520
13057991110,0cyclictest0-21swapper/322:45:463
1305799110,10cyclictest0-21swapper/323:42:163
1305799110,10cyclictest0-21swapper/321:21:563
1305799110,10cyclictest0-21swapper/300:37:063
1304999119,1cyclictest0-21swapper/222:22:002
13049991110,0cyclictest3650-21sshd00:24:492
13049991110,0cyclictest0-21swapper/221:56:492
1304999110,10cyclictest0-21swapper/223:11:092
1304999110,10cyclictest0-21swapper/221:32:402
1304999110,10cyclictest0-21swapper/221:32:392
1304999110,0cyclictest131rcu_sched00:33:472
1304499119,1cyclictest24205-21entropy19:35:131
13044991110,0cyclictest0-21swapper/122:46:041
1304499110,10cyclictest0-21swapper/123:22:541
1304499110,10cyclictest0-21swapper/121:52:531
1304499110,10cyclictest0-21swapper/100:14:041
1303899113,4cyclictest21808-21munin-run20:50:000
13038991110,0cyclictest0-21swapper/023:25:430
13038991110,0cyclictest0-21swapper/023:11:330
13038991110,0cyclictest0-21swapper/021:33:320
13038991110,0cyclictest0-21swapper/021:33:320
13038991110,0cyclictest0-21swapper/019:10:120
1303899110,0cyclictest0-21swapper/023:00:440
1303899110,0cyclictest0-21swapper/022:47:130
1303899110,0cyclictest0-21swapper/000:11:230
120782110,2sleep01303899cyclictest21:26:450
1305799109,0cyclictest0-21swapper/322:20:563
1305799108,1cyclictest11190-21bash00:29:163
1305799100,0cyclictest0-21swapper/322:09:053
1305799100,0cyclictest0-21swapper/321:08:353
1304999109,0cyclictest0-21swapper/222:55:592
1304999109,0cyclictest0-21swapper/222:05:192
1304999109,0cyclictest0-21swapper/219:53:382
1304999109,0cyclictest0-21swapper/200:39:192
1304999108,1cyclictest33-21ksoftirqd/223:55:162
1304999108,1cyclictest1-21systemd21:48:192
1304999108,1cyclictest0-21swapper/221:50:592
1304999100,0cyclictest0-21swapper/223:47:092
1304499109,0cyclictest0-21swapper/123:38:441
1304499109,0cyclictest0-21swapper/122:30:241
1304499108,1cyclictest646-21dbus-daemon23:28:551
1304499108,1cyclictest25-21ksoftirqd/123:15:221
1304499108,1cyclictest25-21ksoftirqd/121:17:571
1304499108,1cyclictest1-21systemd21:56:031
1304499100,0cyclictest0-21swapper/100:15:441
1303899109,0cyclictest26736-21sshd23:47:330
1303899109,0cyclictest0-21swapper/023:41:330
1303899109,0cyclictest0-21swapper/022:11:040
1303899109,0cyclictest0-21swapper/022:03:330
1303899109,0cyclictest0-21swapper/021:00:120
1303899100,0cyclictest0-21swapper/023:35:330
1303899100,0cyclictest0-21swapper/022:31:330
130579998,0cyclictest0-21swapper/323:32:263
130499998,0cyclictest0-21swapper/222:33:192
130499998,0cyclictest0-21swapper/221:25:382
130499998,0cyclictest0-21swapper/221:10:082
130499997,1cyclictest33-21ksoftirqd/223:43:162
130499997,1cyclictest33-21ksoftirqd/223:18:002
130499997,1cyclictest33-21ksoftirqd/221:23:472
130499996,2cyclictest33-21ksoftirqd/200:05:142
130499990,0cyclictest0-21swapper/220:18:382
130449998,0cyclictest0-21swapper/123:44:441
130449997,1cyclictest988-21lspci21:09:231
130449990,2cyclictest646-21dbus-daemon23:54:531
130449990,1cyclictest131rcu_sched23:32:221
130449990,1cyclictest131rcu_sched23:12:181
130449990,1cyclictest0-21swapper/100:35:341
130389998,0cyclictest0-21swapper/023:15:530
130389997,2cyclictest20794-21kworker/0:423:21:340
130389997,2cyclictest15849-21kworker/0:322:26:040
130389997,2cyclictest10180-21kworker/0:300:20:500
130389997,1cyclictest0-21swapper/021:16:220
130389996,2cyclictest21456-21kworker/0:022:41:220
130579987,0cyclictest0-21swapper/323:38:273
130579987,0cyclictest0-21swapper/323:22:363
130579987,0cyclictest0-21swapper/321:52:353
130579987,0cyclictest0-21swapper/319:10:353
130579986,1cyclictest20399-21lspci22:04:553
130579984,3cyclictest646-21dbus-daemon22:11:573
130499987,0cyclictest0-21swapper/220:27:382
130499986,1cyclictest33-21ksoftirqd/223:25:262
130499986,1cyclictest33-21ksoftirqd/222:52:542
130499986,1cyclictest33-21ksoftirqd/200:10:172
130499986,1cyclictest29832-21lspci21:37:532
130499984,3cyclictest1095-21nfsd23:50:052
130499980,1cyclictest131rcu_sched00:01:252
130499980,1cyclictest0-21swapper/222:00:322
130449987,0cyclictest0-21swapper/121:29:431
130449986,1cyclictest6625-21lspci22:50:021
130449986,1cyclictest25-21ksoftirqd/121:40:391
130449985,2cyclictest684-21polkitd23:01:361
130449985,2cyclictest649-21avahi-daemon22:23:351
130449981,1cyclictest131rcu_sched21:14:151
130449980,2cyclictest131rcu_sched00:25:231
130389987,0cyclictest0-21swapper/021:55:020
130389986,2cyclictest593-21kworker/0:122:35:170
130389986,2cyclictest27701-21kworker/0:223:30:100
130389986,2cyclictest19422-21kworker/0:022:15:040
130389986,2cyclictest13361-21kworker/0:122:20:120
130389986,2cyclictest13309-21kworker/0:321:45:020
130389986,2cyclictest13309-21kworker/0:321:40:100
130389986,2cyclictest13309-21kworker/0:321:35:260
19097270,5sleep319269-21bash21:30:303
19097270,5sleep319269-21bash21:30:303
130579975,1cyclictest649-21avahi-daemon23:15:343
130579975,1cyclictest649-21avahi-daemon23:00:113
130579975,1cyclictest649-21avahi-daemon22:25:373
130579975,1cyclictest26178-21lspci00:03:143
130579974,2cyclictest649-21avahi-daemon23:55:163
130579974,2cyclictest649-21avahi-daemon00:10:283
130579974,2cyclictest646-21dbus-daemon22:15:023
130579974,2cyclictest646-21dbus-daemon00:23:113
130579972,3cyclictest616-13auditd22:43:093
130579970,6cyclictest0-21swapper/319:20:453
130499976,0cyclictest0-21swapper/220:20:182
130499976,0cyclictest0-21swapper/219:43:382
130449976,0cyclictest0-21swapper/120:10:231
130449976,0cyclictest0-21swapper/119:30:231
130449975,1cyclictest10233-21lspci23:05:481
130449974,2cyclictest14890-21sshd22:10:241
130389977,0cyclictest9601-21kworker/0:120:20:420
130389977,0cyclictest5584-21kworker/0:020:18:380
130389977,0cyclictest21881-21kworker/0:319:40:000
130389977,0cyclictest14998-21kworker/0:219:15:240
130579963,2cyclictest17299-21snmpd21:00:543
130579963,2cyclictest17299-21snmpd20:06:373
130579960,5cyclictest0-21swapper/320:40:353
130579960,5cyclictest0-21swapper/319:31:353
130499964,1cyclictest33-21ksoftirqd/220:55:002
130499964,1cyclictest29833-21sshd21:08:002
130499960,5cyclictest0-21swapper/220:10:482
130449964,1cyclictest25-21ksoftirqd/119:50:001
130449964,1cyclictest17299-21snmpd19:22:271
130389966,0cyclictest9601-21kworker/0:120:25:000
130389966,0cyclictest5584-21kworker/0:020:10:160
130389966,0cyclictest23817-21kworker/0:120:55:140
130389966,0cyclictest21881-21kworker/0:320:05:160
130389966,0cyclictest21881-21kworker/0:320:00:160
130389966,0cyclictest21881-21kworker/0:319:55:140
130389966,0cyclictest21881-21kworker/0:319:50:220
130389966,0cyclictest21881-21kworker/0:319:35:160
130389966,0cyclictest19615-21kworker/0:119:30:240
130389966,0cyclictest19615-21kworker/0:119:25:160
130389966,0cyclictest17725-21kworker/0:020:50:560
130389966,0cyclictest17725-21kworker/0:020:40:000
130389966,0cyclictest14998-21kworker/0:219:21:090
130389966,0cyclictest13665-21kworker/0:220:30:160
130389964,1cyclictest279722sleep019:45:030
130579953,1cyclictest17299-21snmpd20:55:403
130579953,1cyclictest17299-21snmpd20:52:473
130579953,1cyclictest17299-21snmpd20:47:083
130579953,1cyclictest17299-21snmpd20:35:143
130579953,1cyclictest17299-21snmpd20:25:233
130579953,1cyclictest17299-21snmpd20:21:413
130579953,1cyclictest17299-21snmpd20:15:533
130579953,1cyclictest17299-21snmpd20:11:143
130579953,1cyclictest17299-21snmpd20:00:103
130579953,1cyclictest17299-21snmpd19:57:013
130579953,1cyclictest17299-21snmpd19:54:433
130579953,1cyclictest17299-21snmpd19:49:133
130579953,1cyclictest17299-21snmpd19:43:163
130579953,1cyclictest17299-21snmpd19:35:103
130579953,1cyclictest17299-21snmpd19:15:543
130579953,1cyclictest14267-21gltestperf20:30:163
130499953,1cyclictest697-21gdbus20:49:552
130499953,1cyclictest32534-21cat19:55:132
130499953,1cyclictest24488-21idleruntime20:55:122
130499953,1cyclictest22386-21cat19:30:152
130499953,1cyclictest18478-21cat20:40:162
130499953,1cyclictest18296-21cat19:20:182
130499953,1cyclictest17299-21snmpd21:00:242
130499953,1cyclictest17299-21snmpd20:38:502
130499953,1cyclictest17299-21snmpd20:05:282
130499953,1cyclictest17299-21snmpd19:45:522
130499953,1cyclictest17299-21snmpd19:25:422
130499953,1cyclictest17299-21snmpd19:17:212
130499953,1cyclictest17299-21snmpd19:10:182
130499950,4cyclictest0-21swapper/220:34:282
130499950,4cyclictest0-21swapper/220:00:182
130449954,0cyclictest25-21ksoftirqd/121:00:001
130449954,0cyclictest0-21swapper/119:53:431
130449953,1cyclictest682-21lsmd20:45:381
130449953,1cyclictest19339-21grep20:40:221
130449953,1cyclictest17299-21snmpd21:03:121
130449953,1cyclictest17299-21snmpd20:50:261
130449953,1cyclictest17299-21snmpd20:35:501
130449953,1cyclictest17299-21snmpd20:05:071
130449953,1cyclictest17299-21snmpd20:01:011
130449953,1cyclictest17299-21snmpd19:55:591
130449953,1cyclictest17299-21snmpd19:42:131
130449953,1cyclictest17299-21snmpd19:26:091
130449953,1cyclictest12913-21find20:25:201
130449953,1cyclictest10349-21cat20:20:161
130449953,1cyclictest1017-21runrttasks19:17:141
130449952,2cyclictest17299-21snmpd20:17:261
130449952,2cyclictest14795-21ntpq20:30:191
130579940,1cyclictest20253-21smartctl19:25:133
130499941,2cyclictest23925-21/usr/sbin/munin19:35:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional