You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 12:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Fri May 02, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245421570,1sleep30-21swapper/300:25:503
1753221430,0sleep317512-21sshd23:23:253
2296321370,1sleep11848-21nfsd22:49:191
1776521330,0sleep017751-21sshd23:10:520
1199521330,2sleep3155299cyclictest22:04:443
3240621310,1sleep132385-21sshd23:43:431
719421260,3sleep2154699cyclictest21:20:472
2645021250,1sleep30-21swapper/322:18:283
3119521200,1sleep30-21swapper/322:09:413
3119521200,1sleep30-21swapper/322:09:413
1941521200,0sleep00-21swapper/000:34:440
1367421170,3sleep0153799cyclictest22:17:120
1107721170,2sleep20-21swapper/222:01:262
3099321160,1sleep30-21swapper/300:18:453
2506121160,1sleep225059-21sshd00:21:242
2506621060,1sleep225103-21kworker/2:123:14:452
1595021060,1sleep10-21swapper/122:58:051
1090221040,2sleep30-21swapper/322:57:353
1041210434,40sleep30-21swapper/319:06:573
3050021030,1sleep2154699cyclictest21:50:532
2276121010,3sleep3155299cyclictest23:30:073
2017121010,2sleep220167-21rm22:52:102
2950521000,3sleep3155299cyclictest23:37:003
196042990,4sleep1154299cyclictest21:28:141
93142980,2sleep30-21swapper/321:48:573
70402980,2sleep30-21swapper/323:50:393
70402980,2sleep30-21swapper/323:50:393
254772970,1sleep3155299cyclictest22:37:023
200992970,2sleep30-21swapper/323:26:423
125392970,1sleep30-21swapper/322:42:013
170292960,2sleep00-21swapper/022:39:240
265742950,2sleep30-21swapper/321:10:173
152422950,1sleep30-21swapper/300:23:543
311042930,0sleep00-21swapper/000:05:460
314982910,2sleep12405-21runrttasks23:56:171
146822910,22sleep2135650irq/27-eth021:15:172
293972900,1sleep20-21swapper/221:57:022
252362900,0sleep00-21swapper/021:44:180
143762890,2sleep10-21swapper/100:27:111
61772870,0sleep26179-21kernelversion20:30:172
61772870,0sleep26179-21kernelversion20:30:172
58762860,2sleep10-21swapper/121:33:081
3220828634,43sleep10-21swapper/119:05:151
94412850,1sleep09431-21sshd21:36:320
133842850,2sleep213385-21bash22:35:492
158642840,3sleep0153799cyclictest21:58:520
78372830,2sleep10-21swapper/122:13:361
279092830,0sleep20-21swapper/221:47:362
258852820,3sleep10-21swapper/100:05:201
119942820,0sleep00-21swapper/022:41:580
136742810,2sleep213670-21sshd00:30:342
101922810,2sleep210188-21cp23:28:522
270192800,3sleep227013-21sshd22:55:582
161782800,3sleep20-21swapper/222:33:022
140552800,2sleep114054-21sshd23:04:191
103222800,2sleep10-21swapper/100:16:301
89342790,1sleep10-21swapper/121:45:421
291032790,1sleep30-21swapper/323:11:583
286312790,3sleep1154299cyclictest00:35:341
6232780,1sleep20-21swapper/200:02:522
289352780,4sleep3155299cyclictest23:02:323
215852780,1sleep31847-21nfsd21:15:553
75972770,6sleep1135650irq/27-eth022:01:061
144932770,2sleep30-21swapper/323:16:473
264082750,2sleep20-21swapper/222:24:442
18852750,3sleep0153799cyclictest23:50:160
18852750,3sleep0153799cyclictest23:50:160
213942730,2sleep30-21swapper/322:30:213
127562720,2sleep212755-21sshd21:30:362
86122700,2sleep10-21swapper/123:19:261
49422680,0sleep14943-21bash23:09:441
216522670,0sleep00-21swapper/021:22:120
52822650,1sleep00-21swapper/021:14:270
287512640,0sleep01847-21nfsd23:49:430
221312630,1sleep20-21swapper/222:11:482
27662620,2sleep30-21swapper/300:03:043
273062620,2sleep1154299cyclictest23:24:221
207282600,3sleep2154699cyclictest00:14:222
119725934,13sleep00-21swapper/019:08:470
72572580,0sleep00-21swapper/021:51:460
50552580,2sleep1154299cyclictest21:23:451
229692580,1sleep2154699cyclictest00:35:072
45742570,3sleep34568-21sshd22:50:343
126425439,9sleep20-21swapper/219:09:372
289412500,1sleep10-21swapper/100:02:271
69952490,3sleep0153799cyclictest23:00:200
111802460,1sleep011168-21sshd22:54:310
16152420,2sleep11614-21sshd22:16:001
87192380,2sleep18716-21sshd23:31:481
25952380,4sleep0153799cyclictest00:39:490
153799374,26cyclictest19155-21bash23:39:120
155299360,5cyclictest0-21swapper/300:06:153
1537993514,9cyclictest3-21ksoftirqd/023:17:450
1537993312,20cyclictest3-21ksoftirqd/000:25:440
1537993311,3cyclictest17762-21bash00:04:360
155299327,22cyclictest1844-21nfsd22:20:233
155299320,5cyclictest0-21swapper/322:45:203
153799325,26cyclictest18251-21sshd23:29:410
153799321,28cyclictest3-21ksoftirqd/000:24:430
1537993211,5cyclictest3-21ksoftirqd/022:13:000
166742312,27sleep226950irq/26-ahci[00021:12:252
155299311,5cyclictest0-21swapper/323:05:503
155299311,25cyclictest0-21swapper/300:35:333
1552993110,13cyclictest19107-21bash23:55:113
155299310,3cyclictest0-21swapper/322:26:593
155299310,3cyclictest0-21swapper/321:41:343
153799316,22cyclictest16151-21sshd22:05:080
153799316,22cyclictest16151-21sshd22:05:070
153799312,27cyclictest12271-21sshd00:10:190
1537993114,5cyclictest3-21ksoftirqd/022:00:060
87452300,3sleep3155299cyclictest21:24:073
314272300,2sleep231421-21cp22:28:172
155299301,4cyclictest0-21swapper/323:40:133
155299300,5cyclictest0-21swapper/321:32:103
153799300,3cyclictest3-21ksoftirqd/023:20:490
153799300,1cyclictest1847-21nfsd21:29:030
155299291,6cyclictest0-21swapper/321:25:203
155299291,4cyclictest0-21swapper/323:45:573
155299291,3cyclictest0-21swapper/321:39:143
155299290,24cyclictest15917-21sshd22:14:243
1542992913,5cyclictest25011-21bash23:52:291
1542992913,5cyclictest25011-21bash23:52:281
153799297,20cyclictest20412-21sshd23:55:150
153799293,4cyclictest3-21ksoftirqd/023:05:090
1537992914,2cyclictest3-21ksoftirqd/022:26:180
153799291,21cyclictest4209-21sshd21:17:250
153799291,18cyclictest0-21swapper/022:55:150
199292280,1sleep018015-21kworker/0:322:24:050
155299281,3cyclictest0-21swapper/321:50:333
155299281,3cyclictest0-21swapper/300:10:203
155299280,3cyclictest0-21swapper/300:30:573
153799284,6cyclictest20190-21sshd21:46:500
153799282,4cyclictest23133-21bash23:42:460
1537992812,14cyclictest3-21ksoftirqd/021:33:480
1537992810,17cyclictest24834-21bash23:30:160
155299270,5cyclictest0-21swapper/321:55:153
153799273,16cyclictest1959-21sshd00:19:180
1537992712,9cyclictest3-21ksoftirqd/022:45:290
178362260,4sleep10-21swapper/123:45:241
154699267,5cyclictest15376-21bash23:38:502
154699260,5cyclictest16953-21sshd22:48:432
1542992611,4cyclictest0-21swapper/122:30:581
153799263,12cyclictest0-21swapper/022:31:010
45622250,2sleep24540-21sshd23:47:162
155299257,8cyclictest10950irq/23-ehci_hcd19:50:163
154699258,16cyclictest33-21ksoftirqd/223:08:012
154699253,4cyclictest33-21ksoftirqd/200:08:082
1542992511,11cyclictest22-21ksoftirqd/122:05:221
1542992511,11cyclictest22-21ksoftirqd/122:05:221
154699248,15cyclictest33-21ksoftirqd/221:26:092
1546992417,5cyclictest20563-21sshd23:55:162
1546992417,1cyclictest135650irq/27-eth022:40:242
154299249,5cyclictest5268-21id21:54:481
154299241,3cyclictest21384-21bash23:11:131
129042240,4sleep20-21swapper/223:04:122
155299239,8cyclictest26950irq/26-ahci[00020:30:043
155299239,8cyclictest26950irq/26-ahci[00020:30:043
155299230,3cyclictest0-21swapper/320:35:213
1546992312,3cyclictest33-21ksoftirqd/223:16:572
154299238,4cyclictest16480-21sshd00:30:521
153799233,1cyclictest17708-21cut20:55:220
219912220,0sleep10-21swapper/100:14:301
155299220,3cyclictest0-21swapper/320:25:223
155299220,3cyclictest0-21swapper/319:45:173
155299220,3cyclictest0-21swapper/319:10:163
155299220,2cyclictest0-21swapper/321:00:183
155299220,2cyclictest0-21swapper/319:22:253
154699226,12cyclictest33-21ksoftirqd/200:27:572
1546992212,8cyclictest33-21ksoftirqd/221:42:542
154299228,3cyclictest0-21swapper/100:20:061
1542992214,5cyclictest22-21ksoftirqd/121:36:031
1542992213,6cyclictest3341-21sshd22:50:261
1542992213,6cyclictest24020-21bash21:59:401
1542992210,3cyclictest0-21swapper/123:27:191
155299216,13cyclictest13180-21grep20:45:243
155299214,10cyclictest10950irq/23-ehci_hcd20:15:163
155299210,5cyclictest0-21swapper/321:05:193
155299210,3cyclictest0-21swapper/319:35:183
155299210,10cyclictest0-21swapper/319:30:153
154699219,7cyclictest33-21ksoftirqd/222:07:292
154699219,7cyclictest33-21ksoftirqd/222:07:292
1546992116,3cyclictest135650irq/27-eth023:52:062
1546992116,3cyclictest135650irq/27-eth023:52:062
1546992114,5cyclictest16336-21bash23:20:112
1546992111,4cyclictest0-21swapper/223:44:122
154299219,3cyclictest2791-21sshd23:37:331
154299219,11cyclictest0-21swapper/121:00:281
154299212,1cyclictest22517-21sshd22:42:591
154299212,16cyclictest22-21ksoftirqd/121:10:221
1542992116,3cyclictest30295-21sshd22:37:321
1542992113,4cyclictest9545-21sshd22:26:061
1542992111,3cyclictest24098-21sshd21:40:591
155299201,2cyclictest0-21swapper/320:00:143
155299200,3cyclictest0-21swapper/320:55:223
155299200,2cyclictest0-21swapper/319:29:223
154699202,4cyclictest0-21swapper/221:36:042
1546992016,3cyclictest14848-21id22:17:182
1546992015,3cyclictest17588-21sshd23:32:422
1546992014,4cyclictest1893-21nscd00:18:032
1542992013,4cyclictest0-21swapper/121:17:121
1542992012,7cyclictest0-21swapper/122:20:231
155299192,15cyclictest0-21swapper/320:05:143
155299190,3cyclictest0-21swapper/320:40:273
155299190,3cyclictest0-21swapper/319:45:003
155299190,2cyclictest0-21swapper/320:10:223
153799191,9cyclictest0-21swapper/019:15:170
155299180,8cyclictest0-21swapper/319:55:243
155299180,3cyclictest0-21swapper/320:20:153
155299180,3cyclictest0-21swapper/319:15:123
155299180,1cyclictest0-21swapper/320:53:433
1537991813,4cyclictest0-21swapper/019:25:140
153799181,14cyclictest3-21ksoftirqd/019:40:280
154299179,5cyclictest23538-21bash21:10:011
153799172,5cyclictest8-21rcu_preempt19:20:150
154699161,14cyclictest0-21swapper/219:15:142
1546991611,3cyclictest3134-21basename20:25:142
154299169,3cyclictest10950irq/23-ehci_hcd20:45:271
154299157,6cyclictest0-21swapper/119:15:191
154299154,3cyclictest10950irq/23-ehci_hcd20:21:251
154299150,13cyclictest0-21swapper/119:51:521
154299150,13cyclictest0-21swapper/119:45:241
153799150,5cyclictest0-21swapper/019:56:210
154699149,4cyclictest10950irq/23-ehci_hcd19:11:112
154699143,10cyclictest10950irq/23-ehci_hcd20:58:202
154699141,6cyclictest0-21swapper/220:10:162
1546991411,2cyclictest23486-21sshd21:10:012
154299143,10cyclictest10950irq/23-ehci_hcd20:36:421
154299143,10cyclictest10950irq/23-ehci_hcd19:43:131
154299142,11cyclictest0-21swapper/119:38:251
154299141,11cyclictest17409-21iostat20:55:201
154299140,5cyclictest0-21swapper/119:25:151
154299140,11cyclictest135650irq/27-eth020:51:401
153799147,5cyclictest0-21swapper/020:27:430
153799143,1cyclictest0-21swapper/020:54:330
1537991411,2cyclictest3-21ksoftirqd/020:10:260
1537991410,2cyclictest3-21ksoftirqd/019:10:200
153799140,13cyclictest0-21swapper/021:05:500
153799140,13cyclictest0-21swapper/019:30:080
154699138,4cyclictest135650irq/27-eth019:30:022
154699138,3cyclictest8675-21df19:25:132
154699137,5cyclictest773-21munin-node20:20:272
154699132,10cyclictest10950irq/23-ehci_hcd19:56:082
154699131,4cyclictest0-21swapper/220:50:232
154699131,11cyclictest135650irq/27-eth020:15:512
1546991311,1cyclictest0-21swapper/220:35:012
154699131,10cyclictest0-21swapper/219:35:182
1546991310,2cyclictest0-21swapper/220:00:232
154699130,12cyclictest0-21swapper/220:45:092
154699130,12cyclictest0-21swapper/219:52:322
154699130,12cyclictest0-21swapper/219:47:242
154699130,12cyclictest0-21swapper/219:41:502
154699130,12cyclictest0-21swapper/219:21:512
154299139,3cyclictest10950irq/23-ehci_hcd20:12:591
1542991311,1cyclictest0-21swapper/119:30:571
154299130,12cyclictest0-21swapper/120:44:281
154299130,12cyclictest0-21swapper/120:05:361
154299130,12cyclictest0-21swapper/120:00:231
154299130,12cyclictest0-21swapper/119:21:081
154299130,11cyclictest3390-21df20:25:141
154299130,11cyclictest1728-21cpu19:10:131
153799137,5cyclictest1847-21nfsd20:15:320
1537991311,1cyclictest0-21swapper/019:45:180
153799130,1cyclictest0-21swapper/020:32:240
153799130,1cyclictest0-21swapper/020:32:240
153799130,1cyclictest0-21swapper/020:02:190
153799130,12cyclictest0-21swapper/021:02:420
153799130,12cyclictest0-21swapper/020:09:270
153799130,12cyclictest0-21swapper/019:51:290
153799130,12cyclictest0-21swapper/019:36:350
154699128,3cyclictest27221-21iostat_ios20:05:192
154699121,10cyclictest0-21swapper/221:00:192
154699120,11cyclictest1848-21nfsd20:40:562
154299120,6cyclictest6391-21nfsd420:30:201
154299120,6cyclictest6391-21nfsd420:30:201
154299120,3cyclictest23226-21munin-node19:55:241
154299120,1cyclictest1848-21nfsd20:15:421
153799126,2cyclictest3-21ksoftirqd/020:40:190
153799123,1cyclictest0-21swapper/020:20:140
153799120,1cyclictest0-21swapper/020:46:100
153799120,10cyclictest0-21swapper/020:35:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional