You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 04:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Fri Apr 26, 2024 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
959821490,0sleep09596-21bash23:55:370
2585321450,3sleep3672099cyclictest22:37:053
1475821330,2sleep31877-21nfsd21:34:543
2729521260,4sleep2671999cyclictest21:21:052
3266321170,1sleep2671999cyclictest21:48:232
1118321150,4sleep1671899cyclictest00:32:131
360221020,2sleep13603-21nfsd400:35:211
637829827,41sleep30-21swapper/319:08:593
78702960,2sleep00-21swapper/022:35:020
110292960,4sleep2671999cyclictest22:27:432
621929348,41sleep10-21swapper/119:07:071
277522920,2sleep127735-21sshd23:23:161
43972910,0sleep00-21swapper/023:01:180
199082910,4sleep2671999cyclictest23:45:172
62552900,2sleep20-21swapper/222:34:522
260982880,1sleep30-21swapper/300:34:113
214622880,1sleep221457-21sshd23:41:362
92942870,2sleep10-21swapper/100:11:331
83052870,2sleep30-21swapper/323:47:473
307202870,1sleep20-21swapper/223:00:332
206522870,2sleep120647-21sshd00:21:031
296932860,2sleep30-21swapper/321:29:053
269012850,0sleep00-21swapper/022:21:560
248852830,1sleep10-21swapper/123:34:261
71232820,2sleep17120-21sshd21:45:221
60372820,2sleep30-21swapper/300:19:203
53352820,2sleep10-21swapper/123:39:471
91802810,3sleep30-21swapper/321:41:473
284872810,2sleep30-21swapper/323:11:523
280342810,2sleep10-21swapper/123:08:011
266232810,2sleep20-21swapper/200:38:192
222562800,1sleep022252-21sshd22:32:540
47352790,2sleep10-21swapper/122:38:281
213482790,4sleep10-21swapper/100:05:001
249042760,1sleep30-21swapper/300:21:373
255912720,2sleep10-21swapper/123:11:301
55282710,2sleep35527-21bash21:18:383
29212710,1sleep12121-21runrttasks22:45:491
299132660,2sleep10-21swapper/122:22:161
632326348,10sleep20-21swapper/219:08:172
162432630,3sleep0671799cyclictest00:16:290
634126136,13sleep00-21swapper/019:08:320
35582590,3sleep2671999cyclictest21:41:062
79462550,53sleep328750irq/26-ahci[00021:11:183
264652310,0sleep226466-21sshd21:51:242
218592290,2sleep20-21swapper/200:25:212
278792260,1sleep20-21swapper/223:38:362
21362260,2sleep10-21swapper/122:11:231
213092260,4sleep2671999cyclictest22:51:512
163942260,3sleep30-21swapper/321:54:043
145702260,1sleep00-21swapper/021:57:390
37762250,2sleep13778-21id22:07:491
283272250,22sleep328750irq/26-ahci[00021:47:533
6719992421,2cyclictest23349-21sshd00:13:232
280572240,22sleep3138550irq/27-eth022:52:413
106112230,2sleep310610-21sshd22:31:283
671999225,9cyclictest12686-21bash22:39:252
28692220,2sleep32865-21sshd00:06:383
17222220,2sleep10-21swapper/123:01:001
106092220,20sleep1138550irq/27-eth023:40:201
81302211,18sleep2138550irq/27-eth023:32:242
8062200,1sleep10-21swapper/122:15:071
671999205,8cyclictest6807-21sshd22:11:572
671999205,3cyclictest31044-21chrt20:02:332
671999204,8cyclictest1876-21nfsd22:01:572
671999204,8cyclictest0-21swapper/222:57:332
671999204,3cyclictest9088-21bash23:51:432
671999202,8cyclictest0-21swapper/223:07:062
6719992012,6cyclictest2582-21id22:15:142
671999201,14cyclictest0-21swapper/222:47:132
308772200,18sleep1138550irq/27-eth021:21:341
259942200,18sleep3138550irq/27-eth023:42:113
157362200,18sleep3138550irq/27-eth022:09:173
671999195,3cyclictest26274-21bash23:11:352
671999194,3cyclictest30781-21sshd23:19:512
671999194,3cyclictest17433-21bash22:20:452
671999193,8cyclictest10365-21sshd00:15:432
671999191,11cyclictest0-21swapper/220:30:242
671999190,9cyclictest23971-21id22:40:372
671999190,8cyclictest3941-21sshd23:20:232
671899199,5cyclictest10950irq/23-ehci_hcd00:09:421
671799190,18cyclictest29875-21sshd22:18:300
49852190,17sleep1138550irq/27-eth000:27:211
671999184,8cyclictest0-21swapper/220:46:562
671999184,2cyclictest0-21swapper/221:35:372
671999184,2cyclictest0-21swapper/221:35:362
671999181,9cyclictest6209-21sshd21:33:532
18062180,16sleep128750irq/26-ahci[00022:57:121
6720991710,2cyclictest0-21swapper/321:22:573
672099170,4cyclictest0-21swapper/319:47:503
672099170,15cyclictest0-21swapper/322:19:423
671999179,6cyclictest3958-21sshd00:31:162
671999172,7cyclictest24007-21sshd00:05:152
671999171,4cyclictest4565-21sshd21:10:532
6719991710,3cyclictest11331-21sshd23:25:102
6719991710,3cyclictest11331-21sshd23:25:102
671999170,15cyclictest0-21swapper/222:05:172
671899172,14cyclictest0-21swapper/119:50:511
671899171,4cyclictest15174-21sshd21:38:441
671899171,4cyclictest15174-21sshd21:38:441
671899170,15cyclictest0-21swapper/121:13:181
671799178,4cyclictest0-21swapper/021:39:160
671799178,4cyclictest0-21swapper/021:39:150
671799170,2cyclictest0-21swapper/023:39:460
671799170,15cyclictest31921-21sshd22:45:250
671799170,12cyclictest0-21swapper/000:00:420
285422170,1sleep128537-21sshd22:25:531
6720991611,4cyclictest1987-21sshd21:39:353
6720991611,4cyclictest1987-21sshd21:39:343
6720991611,2cyclictest0-21swapper/322:25:573
672099160,14cyclictest1992-21sshd23:16:223
672099160,14cyclictest12603-21sshd23:06:083
672099160,11cyclictest4884-21sshd21:56:283
671999162,3cyclictest0-21swapper/221:18:372
671999162,2cyclictest0-21swapper/220:26:212
671999162,12cyclictest30583-21sshd00:02:062
671999162,0cyclictest0-21swapper/221:56:482
671999160,5cyclictest7419-21sshd21:26:232
671899165,4cyclictest25151-21sshd23:19:101
6718991611,3cyclictest0-21swapper/122:41:241
671899160,4cyclictest25957-21sshd23:45:571
671899160,14cyclictest1955-21snmpd22:50:591
671899160,14cyclictest0-21swapper/121:41:221
671799167,5cyclictest3-21ksoftirqd/022:06:540
671799160,2cyclictest9621-21bash00:11:360
671799160,12cyclictest15458-21sshd00:20:240
671799160,12cyclictest1301-21sshd00:30:550
56492160,14sleep3138550irq/27-eth023:24:313
672099158,3cyclictest0-21swapper/300:36:433
672099157,3cyclictest7696-21sshd23:51:323
672099152,4cyclictest32247-21bash22:14:593
672099150,3cyclictest8223-21bash23:55:293
672099150,14cyclictest1987-21sshd23:02:313
672099150,13cyclictest1955-21snmpd22:56:033
672099150,13cyclictest0-21swapper/322:41:063
672099150,13cyclictest0-21swapper/319:12:503
671999154,10cyclictest147450irq/28-eth1-rx-19:38:242
671999153,7cyclictest138550irq/27-eth019:30:462
671999153,10cyclictest11146-21sshd23:55:492
671999152,8cyclictest1877-21nfsd00:20:072
671999151,7cyclictest0-21swapper/220:21:292
671999151,13cyclictest147450irq/28-eth1-rx-20:15:452
671999150,3cyclictest147450irq/28-eth1-rx-19:58:492
671999150,3cyclictest147450irq/28-eth1-rx-19:58:492
671899159,5cyclictest26997-21sshd21:55:171
671899153,11cyclictest14455-21sshd23:56:141
6718991511,3cyclictest0-21swapper/119:31:021
671899150,9cyclictest1042-21munin-run20:10:011
671899150,3cyclictest0-21swapper/123:28:031
671899150,3cyclictest0-21swapper/123:28:031
671899150,13cyclictest7221-21sshd21:26:231
671899150,13cyclictest19425-21sshd21:31:341
671899150,13cyclictest0-21swapper/120:12:271
671899150,13cyclictest0-21swapper/100:19:581
671899150,10cyclictest0-21swapper/122:32:161
671799159,4cyclictest0-21swapper/021:00:180
671799158,5cyclictest3-21ksoftirqd/023:12:430
671799154,8cyclictest30258-21sshd22:10:500
671799154,10cyclictest0-21swapper/022:54:260
671799151,12cyclictest0-21swapper/021:53:340
6717991510,2cyclictest29449-21sshd21:21:230
671799150,9cyclictest22167-21sshd21:31:550
671799150,5cyclictest0-21swapper/023:43:540
671799150,1cyclictest0-21swapper/023:16:000
671799150,14cyclictest0-21swapper/020:46:440
671799150,13cyclictest0-21swapper/020:25:180
671799150,12cyclictest0-21swapper/019:20:410
671799150,11cyclictest0-21swapper/021:18:450
671799150,11cyclictest0-21swapper/021:13:380
671799150,11cyclictest0-21swapper/000:26:480
203622150,1sleep00-21swapper/021:27:570
13892150,1sleep01377-21sshd22:26:310
9572140,2sleep11933-21nscd23:50:411
672099148,4cyclictest8533-21bash22:04:383
672099143,3cyclictest0-21swapper/320:37:143
672099140,2cyclictest19935-21sshd00:00:443
672099140,13cyclictest24863-21sshd22:21:403
672099140,13cyclictest1875-21nfsd23:31:243
672099140,13cyclictest16216-21sshd23:25:373
672099140,13cyclictest16216-21sshd23:25:373
672099140,13cyclictest0-21swapper/300:28:223
672099140,12cyclictest30480-21idleruntime22:45:183
672099140,12cyclictest30423-21sshd23:35:063
672099140,12cyclictest29692-21sshd00:10:033
671999145,3cyclictest0-21swapper/220:05:292
671999143,10cyclictest10950irq/23-ehci_hcd19:50:472
671999142,7cyclictest0-21swapper/220:11:482
671899145,4cyclictest26124-21sshd21:51:211
671899143,3cyclictest0-21swapper/120:22:081
671899140,12cyclictest1877-21nfsd21:15:191
671799145,5cyclictest23730-21sshd22:59:510
671799142,1cyclictest0-21swapper/021:45:390
6717991412,1cyclictest0-21swapper/022:43:350
671799141,12cyclictest0-21swapper/023:50:420
6717991411,1cyclictest0-21swapper/019:45:290
671799140,13cyclictest21290-21bash21:43:150
671799140,13cyclictest20498-21sshd23:33:540
671799140,13cyclictest0-21swapper/023:22:450
671799140,13cyclictest0-21swapper/023:07:260
671799140,13cyclictest0-21swapper/020:44:430
671799140,12cyclictest0-21swapper/000:06:460
671799140,10cyclictest0-21swapper/022:02:210
672099139,3cyclictest1877-21nfsd20:50:123
672099132,3cyclictest138550irq/27-eth020:05:563
672099132,3cyclictest138550irq/27-eth019:50:103
6720991311,1cyclictest0-21swapper/320:56:333
672099130,12cyclictest0-21swapper/321:00:013
672099130,12cyclictest0-21swapper/320:30:363
672099130,12cyclictest0-21swapper/320:29:093
672099130,12cyclictest0-21swapper/320:11:083
672099130,11cyclictest29545-21diskstats20:00:153
672099130,11cyclictest26966-21munin-node19:55:173
672099130,11cyclictest26966-21munin-node19:55:173
672099130,11cyclictest17840-21munin-node20:45:153
672099130,11cyclictest15591-21iostat_ios20:40:133
671999138,3cyclictest9720-21iostat19:15:162
671999138,3cyclictest20556-21diskstats19:40:112
671999134,8cyclictest147450irq/28-eth1-rx-21:07:322
671999131,11cyclictest138550irq/27-eth019:12:102
671999130,5cyclictest138550irq/27-eth020:50:582
671999130,5cyclictest0-21swapper/220:40:312
671999130,12cyclictest138550irq/27-eth020:35:102
671999130,12cyclictest0-21swapper/221:00:522
671999130,12cyclictest0-21swapper/219:24:242
671899139,3cyclictest1877-21nfsd20:48:551
671899138,4cyclictest138550irq/27-eth019:20:111
671899136,6cyclictest1877-21nfsd20:30:331
671899135,3cyclictest0-21swapper/122:01:101
671899133,1cyclictest0-21swapper/120:37:031
671899133,1cyclictest0-21swapper/120:25:131
6718991311,1cyclictest0-21swapper/121:02:221
671899130,5cyclictest0-21swapper/120:00:461
671899130,12cyclictest22181-21runrttasks19:44:171
671899130,12cyclictest0-21swapper/121:09:231
671899130,12cyclictest0-21swapper/120:15:221
671899130,12cyclictest0-21swapper/119:55:591
671899130,12cyclictest0-21swapper/119:55:581
671899130,12cyclictest0-21swapper/119:47:261
671899130,12cyclictest0-21swapper/119:25:501
671899130,12cyclictest0-21swapper/119:17:211
671899130,12cyclictest0-21swapper/119:13:321
671899130,11cyclictest22572-21munin-node20:55:201
671899130,11cyclictest2121-21runrttasks20:40:021
671799136,0cyclictest1805-40kipmi019:10:200
671799133,9cyclictest0-21swapper/019:35:120
671799130,8cyclictest533-21id23:27:410
671799130,8cyclictest533-21id23:27:410
671799130,1cyclictest0-21swapper/020:15:380
671799130,12cyclictest0-21swapper/023:49:280
671799130,12cyclictest0-21swapper/021:06:300
671799130,12cyclictest0-21swapper/020:55:220
671799130,12cyclictest0-21swapper/020:37:530
671799130,12cyclictest0-21swapper/020:04:010
671799130,12cyclictest0-21swapper/019:30:210
671799130,12cyclictest0-21swapper/019:26:020
671799130,12cyclictest0-21swapper/000:39:540
671799130,11cyclictest0-21swapper/020:30:180
672099127,3cyclictest0-21swapper/319:20:283
6720991210,1cyclictest0-21swapper/319:36:493
672099120,3cyclictest26428-21diskstats21:05:133
672099120,3cyclictest21842-21munin-node19:40:263
672099120,3cyclictest2121-21runrttasks20:15:203
672099120,11cyclictest16437-21if_err_eth019:30:133
672099120,11cyclictest0-21swapper/320:21:063
672099120,11cyclictest0-21swapper/319:15:033
672099120,10cyclictest0-21swapper/319:25:143
671999128,3cyclictest22247-21hddtemp_smartct20:55:152
671999128,3cyclictest1877-21nfsd19:45:422
671999120,3cyclictest14235-21irqstats19:25:192
671899120,11cyclictest19350-21munin-node20:50:191
671899120,11cyclictest0-21swapper/119:35:091
671799120,8cyclictest0-21swapper/019:52:570
671799120,2cyclictest0-21swapper/020:50:540
671799120,11cyclictest0-21swapper/020:20:030
671799120,11cyclictest0-21swapper/020:12:550
671799120,11cyclictest0-21swapper/020:06:050
671799120,11cyclictest0-21swapper/019:15:490
671799118,2cyclictest0-21swapper/019:42:390
671799110,3cyclictest0-21swapper/019:57:140
671799110,3cyclictest0-21swapper/019:57:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional