You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 06:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Mon Dec 01, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2637621080,0sleep32573499cyclictest00:08:383
189002920,0sleep00-21swapper/021:13:590
136432730,1sleep1241ktimersoftd/122:17:451
124012720,0sleep00-21swapper/023:22:010
243532650,1sleep324356-21bash23:58:313
254362630,0sleep10-21swapper/100:18:091
45932590,0sleep14595-21grepconf.sh00:06:231
38292580,0sleep20-21swapper/221:12:102
253662560,0sleep10-21swapper/122:54:221
130182540,0sleep00-21swapper/022:46:350
216622530,0sleep30-21swapper/322:21:443
216622530,0sleep30-21swapper/322:21:433
69202520,0sleep10-21swapper/122:39:361
60322510,1sleep26027-21bash22:33:032
88262500,0sleep38824-21sshd21:41:513
48292490,0sleep30-21swapper/321:18:593
264612490,0sleep30-21swapper/322:28:413
193182490,0sleep119315-21lspci21:46:041
89452480,0sleep1241ktimersoftd/100:03:301
195272400,2sleep275650irq/25-eno1-tx-21:39:452
231502390,0sleep10-21swapper/122:31:301
4122380,2sleep32573499cyclictest22:00:213
99572370,0sleep30-21swapper/323:08:533
177652350,0sleep20-21swapper/223:38:372
162452340,0sleep20-21swapper/200:04:152
103292340,0sleep30-21swapper/321:35:303
2555423110,6sleep20-21swapper/219:09:092
255352319,7sleep30-21swapper/319:08:553
139382270,1sleep0648-21systemd-logind00:04:010
254642267,7sleep10-21swapper/119:08:001
2558122210,7sleep00-21swapper/019:09:280
132442220,1sleep0111rcuc/022:24:090
132442220,1sleep0111rcuc/022:24:080
249672180,0sleep10-21swapper/122:47:481
99032170,1sleep39899-21bash00:03:373
56782170,0sleep00-21swapper/022:26:330
180062170,0sleep1231rcuc/121:33:071
170822170,0sleep20-21swapper/223:12:482
76492150,0sleep10-21swapper/123:34:231
60992150,0sleep20-21swapper/223:08:292
45302150,0sleep20-21swapper/200:39:442
325012150,0sleep30-21swapper/322:10:023
303262150,1sleep30-21swapper/322:54:523
217452150,1sleep121743-21sshd21:17:261
19942150,0sleep20-21swapper/222:48:462
7372140,0sleep30-21swapper/321:57:123
72682140,0sleep00-21swapper/019:40:190
295042140,0sleep0111rcuc/000:12:070
25730991412,0cyclictest0-21swapper/222:13:362
2573099140,13cyclictest25430-21sshd23:20:062
2571799140,13cyclictest0-21swapper/023:02:570
103872140,0sleep00-21swapper/022:14:150
90532130,1sleep19056-21grep00:33:151
2571799130,12cyclictest1011-21sshd21:35:460
130962130,0sleep00-21swapper/021:51:570
295682120,0sleep30-21swapper/323:39:503
25730991210,1cyclictest616-17auditd21:57:462
25730991210,1cyclictest27744-21lspci23:42:472
25730991210,1cyclictest16907-21id22:59:562
25730991210,1cyclictest0-21swapper/223:25:372
2573099120,1cyclictest4907-21sshd21:41:272
2573099120,11cyclictest20205-21sshd00:20:472
2573099120,11cyclictest0-21swapper/200:31:472
2572399120,11cyclictest0-21swapper/123:56:201
2571799120,11cyclictest0-21swapper/023:19:280
2571799120,11cyclictest0-21swapper/022:07:270
2571799120,11cyclictest0-21swapper/021:48:360
2573499119,1cyclictest27594-21bash23:23:323
25734991110,0cyclictest0-21swapper/322:47:333
25734991110,0cyclictest0-21swapper/322:09:433
25734991110,0cyclictest0-21swapper/300:26:033
2573499110,10cyclictest4329-21bash22:36:033
2573499110,10cyclictest0-21swapper/323:18:033
2573499110,10cyclictest0-21swapper/323:12:333
2573499110,10cyclictest0-21swapper/322:40:523
2573499110,10cyclictest0-21swapper/321:49:033
2573499110,10cyclictest0-21swapper/300:16:533
25730991110,0cyclictest0-21swapper/221:25:572
2573099110,10cyclictest4887-21sshd21:51:072
2573099110,10cyclictest0-21swapper/200:28:072
2572399119,1cyclictest29816-21grep22:03:201
25723991110,0cyclictest0-21swapper/100:13:291
2571799119,1cyclictest1922-21grepconf.sh22:03:460
2571799119,1cyclictest0-21swapper/023:43:580
25717991110,0cyclictest23120-21lspci00:08:170
2571799110,10cyclictest0-21swapper/022:31:570
2571799110,10cyclictest0-21swapper/000:18:180
2571799110,0cyclictest0-21swapper/021:55:170
46422100,0sleep20-21swapper/221:18:592
327662100,0sleep10-21swapper/123:04:431
2573499109,0cyclictest3586-21bash00:22:223
2573499109,0cyclictest0-21swapper/323:27:133
2573499109,0cyclictest0-21swapper/323:01:333
2573499109,0cyclictest0-21swapper/322:30:323
2573499109,0cyclictest0-21swapper/322:15:133
2573499109,0cyclictest0-21swapper/321:31:523
2573499109,0cyclictest0-21swapper/321:27:333
2573499109,0cyclictest0-21swapper/321:07:523
2573499108,1cyclictest616-17auditd21:21:233
2573499108,1cyclictest0-21swapper/321:11:333
2573499100,9cyclictest0-21swapper/320:54:223
2573499100,0cyclictest0-21swapper/323:40:433
2573099107,2cyclictest33-21ksoftirqd/221:31:152
2573099104,5cyclictest646-21dbus-daemon21:23:412
2573099100,0cyclictest0-21swapper/223:03:372
2572399109,0cyclictest0-21swapper/122:43:001
2572399109,0cyclictest0-21swapper/122:21:401
2572399109,0cyclictest0-21swapper/122:21:391
2572399109,0cyclictest0-21swapper/121:28:301
2572399109,0cyclictest0-21swapper/121:24:091
2572399108,1cyclictest9991-21sshd00:26:191
2572399100,9cyclictest0-21swapper/120:36:401
2572399100,0cyclictest0-21swapper/123:35:291
2571799109,0cyclictest0-21swapper/022:18:270
2571799108,1cyclictest6738-21bash22:55:380
2571799108,1cyclictest437-21kworker/0:023:58:410
2571799100,0cyclictest0-21swapper/023:12:070
2571799100,0cyclictest0-21swapper/023:06:380
238702100,0sleep10-21swapper/123:16:411
257349998,0cyclictest0-21swapper/322:59:423
257349990,0cyclictest0-21swapper/319:43:123
257309998,0cyclictest0-21swapper/222:39:172
257309998,0cyclictest0-21swapper/221:06:152
257309998,0cyclictest0-21swapper/220:09:562
257309998,0cyclictest0-21swapper/200:07:172
257309997,1cyclictest8840-21bash22:17:172
257239998,0cyclictest0-21swapper/121:53:001
257239997,1cyclictest6347-21sshd00:22:401
257239997,1cyclictest31378-21sshd00:35:301
257179998,0cyclictest28732-21lspci23:33:150
257179997,1cyclictest29629-21grep22:38:380
257179997,1cyclictest28250-21sshd23:26:480
257179997,1cyclictest19864-21grepconf.sh22:53:480
257179997,1cyclictest12555-21lspci00:20:070
257349986,1cyclictest27951-21lspci00:35:143
257349986,1cyclictest26098-21lspci00:31:333
257309986,1cyclictest6312-21lspci22:29:552
257309986,1cyclictest10365-21lspci23:15:192
257309985,2cyclictest649-21avahi-daemon22:42:312
257309983,4cyclictest33-21ksoftirqd/221:47:152
257239987,0cyclictest0-21swapper/123:41:001
257239986,1cyclictest5527-21sshd21:41:301
257239986,1cyclictest26499-21sshd21:37:101
257239985,2cyclictest649-21avahi-daemon23:13:111
257179987,0cyclictest0-21swapper/021:16:370
257179986,1cyclictest9-21ksoftirqd/022:40:480
257179986,1cyclictest9-21ksoftirqd/021:40:280
257179986,1cyclictest9-21ksoftirqd/000:25:470
257179986,1cyclictest23676-21lspci23:45:300
257179986,1cyclictest22766-21lspci00:38:120
257179985,2cyclictest649-21avahi-daemon21:32:230
257349975,1cyclictest684-21polkitd21:51:423
257349975,1cyclictest13584-21lspci00:13:483
257349974,2cyclictest649-21avahi-daemon23:46:443
257349974,2cyclictest11510-21sshd23:50:443
257349974,1cyclictest1861-21sshd23:33:473
257309976,0cyclictest0-21swapper/223:31:072
257309976,0cyclictest0-21swapper/219:26:262
257309975,1cyclictest649-21avahi-daemon00:12:432
257309975,1cyclictest48-21kauditd23:45:552
257309975,1cyclictest4107-21lspci22:00:432
257309974,2cyclictest649-21avahi-daemon00:16:132
257309974,1cyclictest700-21gdbus23:59:052
257309974,1cyclictest10345-21sshd22:23:512
257309974,1cyclictest10345-21sshd22:23:502
257309973,3cyclictest700-21gdbus23:51:112
257309973,1cyclictest0-21swapper/222:51:032
257309971,5cyclictest1095-21nfsd22:05:112
257239975,1cyclictest9843-21grep22:10:531
257239975,1cyclictest9114-21lspci22:07:401
257239975,1cyclictest649-21avahi-daemon23:05:421
257239975,1cyclictest32054-21sshd21:57:061
257239975,1cyclictest23673-21lspci23:45:291
257239975,1cyclictest23538-21sshd22:57:191
257239975,1cyclictest20163-21lspci21:14:081
257239975,1cyclictest0-21swapper/120:44:591
257239974,2cyclictest649-21avahi-daemon22:25:451
257239974,2cyclictest6278-21sshd23:50:161
257179976,0cyclictest9774-21kworker/0:219:55:160
257179976,0cyclictest9-21ksoftirqd/023:35:130
257179976,0cyclictest9-21ksoftirqd/000:31:550
257179976,0cyclictest0-21swapper/021:00:170
257179976,0cyclictest0-21swapper/020:20:170
257179975,1cyclictest9-21ksoftirqd/023:51:000
257179975,1cyclictest9-21ksoftirqd/021:27:000
257179975,1cyclictest9-21ksoftirqd/021:20:140
257179975,1cyclictest11660-21lspci21:06:470
257349965,0cyclictest0-21swapper/319:36:133
257309965,0cyclictest0-21swapper/220:49:262
257309963,2cyclictest17299-21snmpd21:00:242
257309960,0cyclictest0-21swapper/220:10:362
257309960,0cyclictest0-21swapper/220:10:362
257239965,0cyclictest0-21swapper/123:20:091
257239965,0cyclictest0-21swapper/120:50:101
257239964,1cyclictest13432-21sshd23:25:181
257239963,2cyclictest616-17auditd21:06:261
257179963,2cyclictest17299-21snmpd20:37:530
257179963,2cyclictest17299-21snmpd20:15:320
257179960,5cyclictest0-21swapper/019:33:260
257349954,0cyclictest0-21swapper/320:25:123
257349953,1cyclictest851-21cat20:45:153
257349953,1cyclictest31027-21cat19:20:173
257349953,1cyclictest27117-21cat20:30:153
257349953,1cyclictest225932sleep320:20:153
257349953,1cyclictest17299-21snmpd21:00:213
257349953,1cyclictest17299-21snmpd20:40:253
257349953,1cyclictest17299-21snmpd20:11:013
257349953,1cyclictest17299-21snmpd20:11:013
257349953,1cyclictest17299-21snmpd20:08:093
257349953,1cyclictest17299-21snmpd20:00:283
257349953,1cyclictest17299-21snmpd19:55:363
257349953,1cyclictest17299-21snmpd19:50:373
257349953,1cyclictest17299-21snmpd19:47:193
257349953,1cyclictest17299-21snmpd19:32:113
257349953,1cyclictest17299-21snmpd19:27:423
257349953,1cyclictest17299-21snmpd19:15:083
257349953,1cyclictest17299-21snmpd19:12:273
257349953,1cyclictest1049-21rpcbind20:36:223
257349953,1cyclictest0-21swapper/320:55:143
257349952,2cyclictest20810-21expr20:15:123
257309953,1cyclictest682-21lsmd20:35:532
257309953,1cyclictest4770-21cat19:35:152
257309953,1cyclictest31207-21mii-tool20:40:152
257309953,1cyclictest17299-21snmpd20:57:302
257309953,1cyclictest17299-21snmpd20:50:442
257309953,1cyclictest17299-21snmpd20:30:232
257309953,1cyclictest17299-21snmpd20:25:082
257309953,1cyclictest17299-21snmpd20:16:192
257309953,1cyclictest17299-21snmpd19:52:222
257309953,1cyclictest17299-21snmpd19:47:492
257309953,1cyclictest17299-21snmpd19:42:042
257309953,1cyclictest17299-21snmpd19:33:232
257309953,1cyclictest17299-21snmpd19:11:062
257309953,1cyclictest12902-21cat19:55:132
257309953,1cyclictest1053-21lldpd20:20:162
257309951,3cyclictest30300-21sadc19:19:592
257309950,4cyclictest30536-21/usr/sbin/munin19:20:152
257309950,4cyclictest14447-21/usr/sbin/munin20:00:202
257239954,0cyclictest0-21swapper/120:27:301
257239954,0cyclictest0-21swapper/120:15:191
257239954,0cyclictest0-21swapper/119:41:101
257239953,1cyclictest817-21cat20:45:151
257239953,1cyclictest7001-21cat21:00:141
257239953,1cyclictest32560-21/usr/sbin/munin19:25:151
257239953,1cyclictest28495-21sshd19:15:131
257239953,1cyclictest27118-21cat20:30:141
257239953,1cyclictest25912-21/usr/sbin/munin19:10:191
257239953,1cyclictest17299-21snmpd20:55:051
257239953,1cyclictest17299-21snmpd20:10:551
257239953,1cyclictest17299-21snmpd20:10:541
257239953,1cyclictest17299-21snmpd20:05:161
257239953,1cyclictest17299-21snmpd19:51:131
257239953,1cyclictest17299-21snmpd19:30:561
257239953,1cyclictest1109-21ntpd20:20:181
257239951,3cyclictest616-13auditd19:22:211
257239950,0cyclictest0-21swapper/119:47:291
257179953,1cyclictest9774-21kworker/0:219:52:330
257179953,1cyclictest629-21tail20:45:130
257179953,1cyclictest4648-21grep19:35:140
257179953,1cyclictest31074-21gltestperf20:40:150
257179953,1cyclictest2903-21cat20:50:160
257179953,1cyclictest27152-21cat20:30:140
257179953,1cyclictest17299-21snmpd20:55:230
257179953,1cyclictest17299-21snmpd20:13:440
257179953,1cyclictest17299-21snmpd20:13:430
257179953,1cyclictest17299-21snmpd19:47:150
257179953,1cyclictest17299-21snmpd19:26:060
257179953,1cyclictest17299-21snmpd19:21:140
257179953,1cyclictest1053-21lldpd20:25:150
257179952,2cyclictest28568-21cpuspeed_turbos19:15:140
257179952,2cyclictest26055-21cut19:10:140
257179952,2cyclictest14859-21sed20:00:140
257239942,1cyclictest4445-21cstates19:35:131
257239942,1cyclictest17299-21snmpd20:00:101
257239942,1cyclictest12674-21sh19:55:111
257179942,1cyclictest17299-21snmpd20:05:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional