You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-14 - 03:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack2slot0.osadl.org (updated Mon Oct 14, 2024 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114062850,0sleep311407-21sshd00:16:333
123142760,2sleep21215599cyclictest23:09:322
196362740,1sleep0101ktimersoftd/021:59:370
50232550,0sleep30-21swapper/300:35:123
130052550,0sleep00-21swapper/023:27:360
10362550,0sleep30-21swapper/321:48:183
240202540,1sleep324017-21sshd23:59:033
268932520,0sleep30-21swapper/322:28:113
237552510,0sleep20-21swapper/200:05:022
213942500,0sleep30-21swapper/322:24:383
150402500,0sleep00-21swapper/022:02:120
258822490,0sleep025877-21lspci21:53:480
273782480,0sleep00-21swapper/023:31:560
244592480,0sleep10-21swapper/119:35:201
280252470,0sleep00-21swapper/021:22:390
237602470,0sleep20-21swapper/221:34:582
191362440,0sleep00-21swapper/022:11:570
1167623927,7sleep20-21swapper/219:05:322
1183023210,7sleep00-21swapper/019:07:270
1170923119,7sleep30-21swapper/319:05:523
116662278,7sleep10-21swapper/119:05:271
296872240,1sleep2311rcuc/223:47:172
279412230,2sleep11214899cyclictest21:29:051
214962210,0sleep30-21swapper/322:39:483
259452180,1sleep025946-21grepconf.sh22:09:340
199272180,1sleep0111rcuc/022:15:110
53692170,0sleep10-21swapper/122:32:141
32742160,0sleep10-21swapper/121:20:151
122732160,1sleep212260-21rm23:42:352
12148991615,0cyclictest2267-21lspci23:56:581
12148991614,1cyclictest30139-21kworker/1:300:13:401
12148991614,1cyclictest16085-21lspci00:20:141
18192150,0sleep30-21swapper/323:20:303
147092150,0sleep10-21swapper/123:06:381
12148991514,0cyclictest9384-21kworker/1:021:37:101
12148991514,0cyclictest6104-21kworker/1:200:25:181
280462140,0sleep09-21ksoftirqd/022:49:350
236232140,0sleep30-21swapper/323:25:303
137992140,0sleep20-21swapper/200:01:032
136192140,0sleep3391rcuc/321:55:423
1216199140,2cyclictest646-21dbus-daemon23:33:023
12148991412,1cyclictest28963-21kworker/1:222:03:101
12148991412,1cyclictest28963-21kworker/1:221:56:331
118772140,0sleep30-21swapper/300:00:523
210122130,1sleep321010-21bash22:09:053
1216199130,12cyclictest0-21swapper/323:11:013
1215599130,1cyclictest0-21swapper/221:12:262
1215599130,12cyclictest0-21swapper/223:54:462
1215599130,12cyclictest0-21swapper/221:43:162
1215599130,11cyclictest17668-21sshd22:51:362
12148991312,0cyclictest7957-21kworker/1:123:26:511
12148991312,0cyclictest33050irq/27-ahci[00022:07:321
12148991311,1cyclictest2098-21kworker/1:223:37:161
12148991311,1cyclictest20662-21kworker/1:022:17:321
12148991311,1cyclictest0-21swapper/122:55:091
1214399130,12cyclictest24764-21sshd00:11:210
293102120,0sleep10-21swapper/123:14:071
12161991211,0cyclictest0-21swapper/322:30:123
12161991210,1cyclictest22340-21sshd22:42:523
1216199120,1cyclictest31777-21grepconf.sh21:51:123
1216199120,1cyclictest18420-21users00:20:223
1216199120,11cyclictest0-21swapper/322:48:413
1216199120,11cyclictest0-21swapper/322:03:523
1215599120,1cyclictest0-21swapper/223:15:362
12148991210,1cyclictest13912-21kworker/1:200:39:401
12148991210,1cyclictest0-21swapper/122:37:381
1214899120,11cyclictest648-21systemd-logind21:46:181
12143991210,1cyclictest648-21systemd-logind00:25:410
12143991210,1cyclictest12132-21lspci22:35:510
1214399120,1cyclictest8073-21taskset20:15:200
1214399120,1cyclictest6861-21lspci21:11:100
1216199119,1cyclictest0-21swapper/323:36:323
1216199119,1cyclictest0-21swapper/323:01:213
1216199110,10cyclictest75550irq/24-eno1-rx-23:16:123
1216199110,10cyclictest0-21swapper/323:46:323
1216199110,0cyclictest0-21swapper/322:18:523
1215599119,1cyclictest0-21swapper/223:00:452
12155991110,0cyclictest0-21swapper/222:19:562
12155991110,0cyclictest0-21swapper/221:59:052
12155991110,0cyclictest0-21swapper/221:46:152
12155991110,0cyclictest0-21swapper/221:23:052
1215599110,10cyclictest0-21swapper/223:10:262
1215599110,10cyclictest0-21swapper/222:08:462
1215599110,10cyclictest0-21swapper/200:35:572
1215599110,0cyclictest0-21swapper/222:41:062
1215599110,0cyclictest0-21swapper/222:10:252
1214899119,1cyclictest681-21gdbus00:31:081
1214899119,1cyclictest20662-21kworker/1:022:14:031
1214899119,1cyclictest0-21swapper/123:20:491
12143991110,0cyclictest0-21swapper/000:15:310
1214399110,10cyclictest0-21swapper/000:22:400
1214399110,0cyclictest0-21swapper/022:26:400
1214399110,0cyclictest0-21swapper/022:21:010
1216199109,0cyclictest0-21swapper/323:54:113
1216199109,0cyclictest0-21swapper/322:12:413
1216199109,0cyclictest0-21swapper/321:38:323
1216199100,0cyclictest0-21swapper/321:32:013
1216199100,0cyclictest0-21swapper/300:13:123
1215599109,0cyclictest0-21swapper/219:30:162
1215599109,0cyclictest0-21swapper/200:17:562
1215599108,1cyclictest23963-21lspci23:55:572
1215599108,1cyclictest1-21systemd21:36:062
1215599100,0cyclictest0-21swapper/223:37:072
1215599100,0cyclictest0-21swapper/222:03:162
1214899109,0cyclictest30139-21kworker/1:300:16:581
1214899109,0cyclictest30121-21kworker/1:321:13:041
1214899108,1cyclictest9783-21kworker/1:423:43:511
1214399109,0cyclictest0-21swapper/023:44:400
1214399109,0cyclictest0-21swapper/023:16:400
1214399109,0cyclictest0-21swapper/023:11:310
1214399109,0cyclictest0-21swapper/019:25:200
1214399109,0cyclictest0-21swapper/000:39:310
1214399108,1cyclictest0-21swapper/022:44:310
1214399102,4cyclictest23927-21munin-run21:35:000
12143991010,0cyclictest0-21swapper/000:03:010
121619998,0cyclictest0-21swapper/322:51:023
121559998,0cyclictest29334-21sshd22:28:262
121489999,0cyclictest420-21kworker/1:221:30:331
121489999,0cyclictest22927-21kworker/1:122:22:341
121489998,0cyclictest2098-21kworker/1:223:31:441
121489998,0cyclictest0-21swapper/121:15:591
121489997,1cyclictest9783-21kworker/1:423:52:511
121489997,1cyclictest9384-21kworker/1:021:40:381
121489997,1cyclictest3322-21kworker/1:422:46:401
121489997,1cyclictest25-21ksoftirqd/123:00:101
121489996,1cyclictest25-21ksoftirqd/123:49:531
121489991,1cyclictest131rcu_sched21:50:361
121489990,1cyclictest131rcu_sched00:06:321
121439998,0cyclictest21875-21sshd23:58:500
121439998,0cyclictest0-21swapper/023:35:510
121439998,0cyclictest0-21swapper/020:03:290
121439996,2cyclictest19071-21kworker/0:221:40:590
8631280,0sleep20-21swapper/222:56:542
6711280,1sleep26856-21sshd23:33:042
26763280,1sleep11-21systemd22:40:151
121619987,0cyclictest0-21swapper/323:07:023
121619987,0cyclictest0-21swapper/320:20:203
121619986,1cyclictest29033-21lspci22:58:513
121619986,1cyclictest15159-21lspci21:18:203
121619985,2cyclictest649-21avahi-daemon00:34:183
121619985,2cyclictest649-21avahi-daemon00:07:083
121559987,0cyclictest0-21swapper/223:21:562
121559987,0cyclictest0-21swapper/221:26:062
121559986,1cyclictest3988-21lspci22:32:062
121559986,1cyclictest33-21ksoftirqd/223:26:152
121559986,1cyclictest33-21ksoftirqd/222:46:152
121559986,1cyclictest33-21ksoftirqd/200:25:282
121559986,1cyclictest17915-21bash00:30:032
121559986,1cyclictest11093-21systemd-cgroups00:13:182
121559985,2cyclictest700-21gdbus22:20:242
121559985,2cyclictest649-21avahi-daemon21:15:092
121559985,2cyclictest649-21avahi-daemon00:21:382
121489988,0cyclictest3322-21kworker/1:422:50:171
121489988,0cyclictest16701-21kworker/1:119:23:511
121489987,0cyclictest0-21swapper/121:07:281
121489987,0cyclictest0-21swapper/120:22:291
121489987,0cyclictest0-21swapper/120:05:591
121489986,1cyclictest25-21ksoftirqd/123:18:591
121489986,1cyclictest25-21ksoftirqd/122:25:351
121489986,1cyclictest25-21ksoftirqd/100:00:291
121439987,0cyclictest0-21swapper/023:01:510
121439987,0cyclictest0-21swapper/021:18:510
121439987,0cyclictest0-21swapper/000:31:400
121439986,1cyclictest9-21ksoftirqd/023:06:410
121439986,1cyclictest11065-21lspci21:36:480
121439985,2cyclictest649-21avahi-daemon21:48:290
121439985,2cyclictest646-21dbus-daemon22:55:280
121439984,3cyclictest8259-21kworker/0:422:54:380
121439984,3cyclictest1321-21sshd23:47:400
121439980,4cyclictest28938-21munin-run19:50:000
121619975,1cyclictest7343-21lspci21:11:143
121619975,1cyclictest648-21systemd-logind21:26:003
121619974,2cyclictest649-21avahi-daemon21:21:183
121619974,2cyclictest649-21avahi-daemon00:25:203
121619974,2cyclictest29907-21sshd21:41:423
121619974,2cyclictest25935-21sshd23:40:493
121559975,1cyclictest33-21ksoftirqd/222:37:462
121559975,1cyclictest33-21ksoftirqd/221:50:212
121489976,0cyclictest0-21swapper/119:32:291
121439976,0cyclictest0-21swapper/019:56:300
121439975,1cyclictest684-21polkitd22:31:260
121439975,1cyclictest684-21polkitd00:05:170
121439975,1cyclictest649-21avahi-daemon23:23:550
121439975,1cyclictest616-17auditd23:50:020
121439975,1cyclictest15219-21sshd21:27:500
121619964,1cyclictest649-21avahi-daemon19:19:593
121619964,1cyclictest646-21dbus-daemon20:55:013
121619964,1cyclictest646-21dbus-daemon19:20:003
121619964,1cyclictest1245-21grep20:00:143
121619962,2cyclictest26863-21sshd21:01:323
121619960,5cyclictest0-21swapper/320:57:423
121619960,5cyclictest0-21swapper/320:46:023
121619960,5cyclictest0-21swapper/320:46:013
121559965,0cyclictest0-21swapper/220:04:362
121559960,5cyclictest0-21swapper/219:56:562
121559960,5cyclictest0-21swapper/219:39:052
121559960,5cyclictest0-21swapper/219:12:162
121489966,0cyclictest23045-21kworker/1:019:55:141
121489965,0cyclictest0-21swapper/120:10:081
121489964,1cyclictest25-21ksoftirqd/121:00:001
121439964,1cyclictest9524-21mii-tool20:20:150
6017250,3sleep20-21swapper/220:10:182
121619954,0cyclictest0-21swapper/321:05:213
121619953,1cyclictest649-21avahi-daemon19:45:133
121619953,1cyclictest29656-21cat19:50:143
121619953,1cyclictest17299-21snmpd20:40:353
121619953,1cyclictest17299-21snmpd20:05:163
121619953,1cyclictest17299-21snmpd19:29:593
121619953,1cyclictest17299-21snmpd19:25:483
121619953,1cyclictest11536-21cat20:25:153
121619952,2cyclictest17299-21snmpd20:15:503
121619952,2cyclictest17299-21snmpd19:59:513
121619952,2cyclictest17299-21snmpd19:37:073
121619952,2cyclictest14171-21which20:30:213
121619951,3cyclictest6269-21seq20:10:193
121619950,1cyclictest0-21swapper/320:35:173
121559953,1cyclictest649-21avahi-daemon19:26:142
121559953,1cyclictest646-21dbus-daemon20:40:012
121559953,1cyclictest3764-21nfsd420:05:162
121559953,1cyclictest17299-21snmpd21:05:412
121559953,1cyclictest17299-21snmpd20:54:332
121559953,1cyclictest17299-21snmpd20:45:332
121559953,1cyclictest17299-21snmpd20:45:322
121559953,1cyclictest17299-21snmpd20:37:132
121559953,1cyclictest17299-21snmpd20:30:442
121559953,1cyclictest17299-21snmpd20:26:292
121559953,1cyclictest17299-21snmpd20:22:552
121559953,1cyclictest17299-21snmpd20:15:322
121559953,1cyclictest17299-21snmpd19:42:462
121559953,1cyclictest1058-21sendmail20:59:352
121559953,1cyclictest0-21swapper/219:45:122
121559953,1cyclictest0-21swapper/219:15:002
121559952,2cyclictest17299-21snmpd19:51:522
121559950,1cyclictest17824-21ntp_states19:20:162
121489954,0cyclictest0-21swapper/120:01:081
121489954,0cyclictest0-21swapper/119:10:481
121489953,1cyclictest21722-21cat20:50:141
121489953,1cyclictest18983-21kworker/1:020:45:201
121489953,1cyclictest18983-21kworker/1:020:45:201
121489953,1cyclictest17299-21snmpd21:03:211
121489953,1cyclictest17299-21snmpd20:41:381
121489953,1cyclictest17299-21snmpd20:36:311
121489953,1cyclictest17299-21snmpd20:32:441
121489953,1cyclictest17299-21snmpd20:15:021
121489953,1cyclictest17299-21snmpd19:26:001
121489953,1cyclictest14900-21mii-tool19:15:011
121489953,1cyclictest11573-21cat20:25:141
121489953,1cyclictest1058-21sendmail19:50:011
121489953,1cyclictest1049-21rpcbind19:49:531
121489952,2cyclictest25376-21entropy19:40:121
121439953,1cyclictest23514-21cat19:35:130
121439953,1cyclictest17299-21snmpd20:46:270
121439953,1cyclictest17299-21snmpd20:46:260
121439953,1cyclictest17299-21snmpd20:42:020
121439953,1cyclictest17299-21snmpd20:35:100
121439953,1cyclictest17299-21snmpd20:30:260
121439953,1cyclictest17299-21snmpd20:25:040
121439953,1cyclictest17299-21snmpd20:10:210
121439953,1cyclictest17299-21snmpd20:06:130
121439953,1cyclictest17299-21snmpd19:50:520
121439953,1cyclictest17299-21snmpd19:32:030
121439953,1cyclictest17299-21snmpd19:11:110
121439953,1cyclictest15370-21cat19:15:150
121439953,1cyclictest1109-21ntpd21:05:170
121439952,2cyclictest700-21gdbus19:40:000
121439952,2cyclictest21658-21hwlatdetect20:50:140
121439952,2cyclictest17299-21snmpd21:01:090
121439952,2cyclictest17299-21snmpd20:55:140
121439950,0cyclictest0-21swapper/019:23:100
121619942,1cyclictest25359-21sh19:40:123
121619942,1cyclictest0-21swapper/319:10:123
121559942,1cyclictest0-21swapper/221:00:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional