You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-29 - 08:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Thu Feb 29, 2024 00:46:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
675621650,7sleep22824999cyclictest23:00:002
1235911320,103ptp4l401ktimersoftd/319:05:553
576981150,15rtkit-daemon0-21swapper/219:06:362
28248999688,5cyclictest25-21ksoftirqd/122:25:211
28248999587,4cyclictest25-21ksoftirqd/123:30:241
28248999386,4cyclictest25-21ksoftirqd/120:40:221
28248999284,5cyclictest25-21ksoftirqd/123:15:201
28248998879,5cyclictest25-21ksoftirqd/123:40:241
28248998477,4cyclictest25-21ksoftirqd/120:00:241
28248998376,4cyclictest25-21ksoftirqd/121:05:231
276422835,18sleep10-21swapper/119:05:501
28248998145,5cyclictest25-21ksoftirqd/121:30:421
28248997941,5cyclictest25-21ksoftirqd/120:30:001
28248997664,8cyclictest25-21ksoftirqd/121:15:211
28249997565,6cyclictest33-21ksoftirqd/219:40:212
123591750,2ptp4l391rcuc/322:05:343
123591750,1ptp4l401ktimersoftd/319:51:263
123591750,1ptp4l401ktimersoftd/319:23:373
123591750,1ptp4l401ktimersoftd/319:23:363
28248997461,9cyclictest25-21ksoftirqd/122:50:251
28248997439,5cyclictest25-21ksoftirqd/122:40:411
28248997438,4cyclictest25-21ksoftirqd/100:40:011
28248997365,4cyclictest25-21ksoftirqd/123:00:051
28248997365,4cyclictest25-21ksoftirqd/121:20:051
28248997335,5cyclictest25-21ksoftirqd/100:00:281
28248997333,5cyclictest25-21ksoftirqd/123:55:011
28248997333,5cyclictest25-21ksoftirqd/123:55:001
28249997261,7cyclictest33-21ksoftirqd/221:40:202
28248997264,4cyclictest25-21ksoftirqd/119:40:061
11802720,2sleep10-21swapper/119:20:291
11802720,2sleep10-21swapper/119:20:281
28248997163,4cyclictest25-21ksoftirqd/120:15:041
28248997137,4cyclictest25-21ksoftirqd/120:35:361
28248997136,4cyclictest25-21ksoftirqd/121:35:351
28248997035,4cyclictest25-21ksoftirqd/100:05:341
28248997028,5cyclictest25-21ksoftirqd/100:25:391
28248996927,8cyclictest25-21ksoftirqd/121:10:291
28249996851,10cyclictest33-21ksoftirqd/221:25:202
28248996833,4cyclictest25-21ksoftirqd/100:30:311
28248996832,5cyclictest25-21ksoftirqd/122:55:431
28248996733,4cyclictest25-21ksoftirqd/123:45:251
28248996732,5cyclictest25-21ksoftirqd/119:55:401
28248996731,4cyclictest25-21ksoftirqd/121:00:261
28248996731,4cyclictest25-21ksoftirqd/100:18:591
28248996632,4cyclictest25-21ksoftirqd/123:20:261
28248996631,4cyclictest25-21ksoftirqd/121:00:001
28248996627,5cyclictest25-21ksoftirqd/122:50:011
28248996627,5cyclictest25-21ksoftirqd/120:45:301
28248996530,5cyclictest25-21ksoftirqd/121:55:251
28248996529,5cyclictest25-21ksoftirqd/122:10:371
28248996529,4cyclictest25-21ksoftirqd/122:00:381
28248996526,7cyclictest25-21ksoftirqd/123:55:261
28248996525,5cyclictest25-21ksoftirqd/123:30:001
28248996432,8cyclictest25-21ksoftirqd/122:30:251
28248996429,5cyclictest25-21ksoftirqd/122:15:381
28248996427,9cyclictest25-21ksoftirqd/119:45:301
28248996427,5cyclictest25-21ksoftirqd/122:35:271
28248996425,10cyclictest25-21ksoftirqd/121:25:361
28248996424,5cyclictest25-21ksoftirqd/119:30:281
28248996423,8cyclictest25-21ksoftirqd/121:45:291
123591640,1ptp4l401ktimersoftd/322:02:393
28248996329,4cyclictest25-21ksoftirqd/119:10:281
28248996328,4cyclictest25-21ksoftirqd/119:15:391
28248996328,4cyclictest25-21ksoftirqd/119:15:391
28249996245,9cyclictest33-21ksoftirqd/220:50:222
28249996245,9cyclictest33-21ksoftirqd/220:50:212
28248996248,4cyclictest25-21ksoftirqd/120:25:001
28248996229,4cyclictest25-21ksoftirqd/123:35:221
28248996228,5cyclictest25-21ksoftirqd/122:05:251
28248996227,4cyclictest25-21ksoftirqd/119:50:221
28248996224,8cyclictest25-21ksoftirqd/119:25:281
2824899616,4cyclictest25-21ksoftirqd/122:20:231
2824899616,4cyclictest25-21ksoftirqd/122:20:221
28248996126,5cyclictest25-21ksoftirqd/100:20:301
28248996123,5cyclictest25-21ksoftirqd/120:50:311
28248996123,5cyclictest25-21ksoftirqd/120:50:301
28249996053,4cyclictest33-21ksoftirqd/221:20:052
28249996044,10cyclictest33-21ksoftirqd/220:25:242
28249996043,9cyclictest33-21ksoftirqd/219:20:222
28249996043,9cyclictest33-21ksoftirqd/219:20:222
28248996045,4cyclictest25-21ksoftirqd/100:10:291
28248996037,11cyclictest25-21ksoftirqd/119:40:001
28248996036,3cyclictest25-21ksoftirqd/123:10:311
28248996022,8cyclictest25-21ksoftirqd/120:30:241
28248996021,8cyclictest25-21ksoftirqd/120:05:421
2824799605,3cyclictest121rcu_preempt21:20:050
246862600,6sleep02824799cyclictest23:35:360
28249995945,8cyclictest33-21ksoftirqd/222:30:222
28249995942,9cyclictest33-21ksoftirqd/200:25:222
28248995953,4cyclictest25-21ksoftirqd/123:05:311
28248995922,5cyclictest25-21ksoftirqd/121:40:191
28248995852,4cyclictest25-21ksoftirqd/120:15:001
28249995741,8cyclictest33-21ksoftirqd/222:20:212
28249995741,8cyclictest33-21ksoftirqd/222:20:202
28249995740,10cyclictest33-21ksoftirqd/222:00:222
28248995751,4cyclictest25-21ksoftirqd/121:50:231
28249995539,9cyclictest33-21ksoftirqd/222:05:222
123591550,1ptp4l391rcuc/300:19:453
28249995447,4cyclictest33-21ksoftirqd/221:00:012
28249995347,4cyclictest33-21ksoftirqd/219:50:002
28249995340,8cyclictest33-21ksoftirqd/200:05:192
28249995246,4cyclictest33-21ksoftirqd/221:40:012
28249995044,3cyclictest33-21ksoftirqd/223:20:002
28249995037,7cyclictest33-21ksoftirqd/200:35:222
28249994943,3cyclictest33-21ksoftirqd/220:19:592
28249994942,4cyclictest33-21ksoftirqd/223:05:232
28247994943,4cyclictest9-21ksoftirqd/019:20:000
28247994943,4cyclictest9-21ksoftirqd/019:19:590
28250994831,14cyclictest41-21ksoftirqd/322:14:343
28249994841,4cyclictest33-21ksoftirqd/220:09:592
28249994841,4cyclictest33-21ksoftirqd/220:00:002
28249994740,4cyclictest33-21ksoftirqd/222:50:012
28249994729,11cyclictest33-21ksoftirqd/223:55:202
28249994639,3cyclictest33-21ksoftirqd/221:10:002
263572465,25sleep026372-21/usr/sbin/munin19:05:250
123591460,1ptp4l401ktimersoftd/323:15:323
28250994537,5cyclictest41-21ksoftirqd/323:10:253
28249994539,3cyclictest33-21ksoftirqd/220:50:012
28249994539,3cyclictest33-21ksoftirqd/219:40:002
28249994538,4cyclictest33-21ksoftirqd/221:20:012
28249994531,11cyclictest33-21ksoftirqd/219:55:012
28247994539,3cyclictest9-21ksoftirqd/000:21:580
123591450,1ptp4l401ktimersoftd/322:45:263
123591450,1ptp4l401ktimersoftd/319:28:433
123591440,1ptp4l401ktimersoftd/323:57:193
123591440,1ptp4l401ktimersoftd/323:05:233
123591440,1ptp4l401ktimersoftd/321:28:413
28250994336,4cyclictest41-21ksoftirqd/322:58:583
28250994327,12cyclictest41-21ksoftirqd/320:53:053
28250994327,12cyclictest41-21ksoftirqd/320:53:053
28247994338,3cyclictest9-21ksoftirqd/022:50:000
28247994337,3cyclictest9-21ksoftirqd/020:13:010
28247994336,4cyclictest9-21ksoftirqd/021:10:420
123591430,3ptp4l121rcu_preempt21:47:303
123591430,1ptp4l401ktimersoftd/321:10:453
28249994236,3cyclictest33-21ksoftirqd/220:40:002
28249994227,10cyclictest33-21ksoftirqd/222:55:212
28247994235,4cyclictest9-21ksoftirqd/023:25:000
28247994235,4cyclictest9-21ksoftirqd/022:05:000
123591420,2ptp4l401ktimersoftd/320:32:063
28250994135,3cyclictest41-21ksoftirqd/322:36:273
28250994135,3cyclictest41-21ksoftirqd/321:36:493
28250994134,4cyclictest41-21ksoftirqd/319:55:203
28250994134,4cyclictest41-21ksoftirqd/319:38:343
28249994136,3cyclictest33-21ksoftirqd/219:10:302
28249994135,4cyclictest33-21ksoftirqd/223:39:592
28249994135,4cyclictest33-21ksoftirqd/222:25:452
28249994134,4cyclictest33-21ksoftirqd/222:51:592
28249994134,4cyclictest33-21ksoftirqd/222:10:242
28249994133,4cyclictest33-21ksoftirqd/219:30:222
28249994126,8cyclictest33-21ksoftirqd/200:00:262
28247994135,3cyclictest9-21ksoftirqd/021:20:010
28250994035,3cyclictest41-21ksoftirqd/319:13:463
28250994034,4cyclictest41-21ksoftirqd/321:50:293
28250994034,4cyclictest41-21ksoftirqd/300:39:473
28250994033,4cyclictest41-21ksoftirqd/323:42:063
28250994033,4cyclictest41-21ksoftirqd/323:36:113
28249994034,4cyclictest33-21ksoftirqd/221:45:242
28249994034,4cyclictest33-21ksoftirqd/221:30:342
28249994034,3cyclictest33-21ksoftirqd/220:40:242
28249994034,3cyclictest33-21ksoftirqd/219:30:002
28249994033,4cyclictest33-21ksoftirqd/221:50:282
28247994034,3cyclictest9-21ksoftirqd/019:30:010
28247994033,4cyclictest9-21ksoftirqd/023:17:010
28247994033,4cyclictest9-21ksoftirqd/000:35:540
28247994033,4cyclictest9-21ksoftirqd/000:16:470
28250993933,4cyclictest41-21ksoftirqd/321:18:343
28250993932,4cyclictest41-21ksoftirqd/321:44:143
28250993932,4cyclictest41-21ksoftirqd/300:26:173
28250993931,5cyclictest41-21ksoftirqd/320:08:153
28250993931,5cyclictest41-21ksoftirqd/319:18:463
28250993931,5cyclictest41-21ksoftirqd/319:18:453
28250993930,5cyclictest41-21ksoftirqd/320:37:593
28250993913,4cyclictest17726-21wc23:20:403
28249993933,4cyclictest33-21ksoftirqd/223:50:012
28249993933,4cyclictest33-21ksoftirqd/222:15:422
28249993933,4cyclictest33-21ksoftirqd/220:15:002
28249993933,3cyclictest33-21ksoftirqd/221:59:592
28249993930,5cyclictest31190-21missed_timers23:50:322
28249993930,5cyclictest31190-21missed_timers23:50:322
28247993934,3cyclictest9-21ksoftirqd/022:30:320
28247993934,3cyclictest9-21ksoftirqd/000:30:340
28247993933,4cyclictest9-21ksoftirqd/023:14:360
28247993933,4cyclictest9-21ksoftirqd/000:02:210
28247993933,3cyclictest9-21ksoftirqd/000:00:000
28247993932,4cyclictest9-21ksoftirqd/023:49:530
28247993932,4cyclictest9-21ksoftirqd/020:56:580
28247993932,4cyclictest9-21ksoftirqd/019:47:410
6512380,8sleep02824799cyclictest19:20:240
6512380,8sleep02824799cyclictest19:20:230
28250993832,4cyclictest41-21ksoftirqd/320:41:053
28250993832,3cyclictest41-21ksoftirqd/321:20:403
28250993832,3cyclictest41-21ksoftirqd/320:17:563
28250993831,4cyclictest41-21ksoftirqd/323:31:533
28250993831,4cyclictest41-21ksoftirqd/323:28:383
28250993831,4cyclictest41-21ksoftirqd/320:29:113
28250993814,5cyclictest29935-21nfs4_client20:20:333
28249993833,3cyclictest33-21ksoftirqd/221:00:332
28249993832,4cyclictest33-21ksoftirqd/220:21:212
28249993832,3cyclictest33-21ksoftirqd/221:15:002
28249993831,4cyclictest33-21ksoftirqd/223:25:162
28249993830,5cyclictest33-21ksoftirqd/200:30:052
28247993832,4cyclictest9-21ksoftirqd/022:42:210
28247993832,4cyclictest9-21ksoftirqd/021:49:540
28247993832,3cyclictest9-21ksoftirqd/023:31:490
28247993832,3cyclictest9-21ksoftirqd/023:29:170
28247993831,4cyclictest9-21ksoftirqd/021:55:200
28247993831,4cyclictest9-21ksoftirqd/021:52:380
28247993831,4cyclictest9-21ksoftirqd/020:41:170
28247993831,4cyclictest9-21ksoftirqd/020:38:440
28247993831,4cyclictest9-21ksoftirqd/020:23:580
28247993830,5cyclictest9-21ksoftirqd/000:29:110
28250993731,4cyclictest41-21ksoftirqd/321:01:073
28250993731,4cyclictest41-21ksoftirqd/300:02:123
28250993731,3cyclictest41-21ksoftirqd/323:03:293
28250993731,3cyclictest41-21ksoftirqd/320:46:353
28250993730,4cyclictest41-21ksoftirqd/323:53:103
28250993730,4cyclictest41-21ksoftirqd/323:53:093
28250993730,4cyclictest41-21ksoftirqd/322:26:093
28250993730,4cyclictest41-21ksoftirqd/321:56:593
28250993730,4cyclictest41-21ksoftirqd/320:55:243
28250993730,4cyclictest41-21ksoftirqd/319:32:393
28250993718,4cyclictest41-21ksoftirqd/300:10:193
28249993731,4cyclictest33-21ksoftirqd/223:20:342
28249993731,3cyclictest33-21ksoftirqd/223:45:002
28247993730,4cyclictest9-21ksoftirqd/023:01:200
28247993730,4cyclictest9-21ksoftirqd/022:59:180
28247993730,4cyclictest9-21ksoftirqd/022:36:030
28247993730,4cyclictest9-21ksoftirqd/022:15:320
28247993730,4cyclictest9-21ksoftirqd/020:49:390
28247993730,4cyclictest9-21ksoftirqd/019:51:550
28247993730,4cyclictest9-21ksoftirqd/019:44:520
28247993730,4cyclictest9-21ksoftirqd/019:12:290
28247993730,4cyclictest9-21ksoftirqd/000:12:570
28250993630,4cyclictest41-21ksoftirqd/320:10:393
28250993629,4cyclictest41-21ksoftirqd/322:50:233
28250993629,4cyclictest41-21ksoftirqd/322:23:283
28250993629,4cyclictest41-21ksoftirqd/322:23:283
28249993629,4cyclictest33-21ksoftirqd/220:03:262
28247993631,3cyclictest9-21ksoftirqd/022:06:450
28247993631,3cyclictest9-21ksoftirqd/021:31:100
28247993630,3cyclictest9-21ksoftirqd/022:26:460
28247993630,3cyclictest9-21ksoftirqd/021:38:010
28247993630,3cyclictest9-21ksoftirqd/020:33:370
28247993630,3cyclictest9-21ksoftirqd/000:07:490
28247993629,4cyclictest9-21ksoftirqd/023:52:140
28247993629,4cyclictest9-21ksoftirqd/023:52:130
28247993629,4cyclictest9-21ksoftirqd/020:09:090
28247993629,4cyclictest9-21ksoftirqd/020:03:070
28250993530,3cyclictest41-21ksoftirqd/320:01:283
28250993529,3cyclictest41-21ksoftirqd/321:07:263
28250993528,4cyclictest41-21ksoftirqd/300:30:433
28250993528,4cyclictest41-21ksoftirqd/300:21:513
28250993522,10cyclictest41-21ksoftirqd/322:16:083
28249993529,4cyclictest33-21ksoftirqd/200:10:292
28249993528,4cyclictest33-21ksoftirqd/222:35:212
28249993528,4cyclictest33-21ksoftirqd/220:32:202
28249993528,4cyclictest33-21ksoftirqd/219:15:062
28249993528,4cyclictest33-21ksoftirqd/219:15:052
28249993528,4cyclictest33-21ksoftirqd/200:15:322
28249993528,3cyclictest33-21ksoftirqd/223:12:002
28249993527,5cyclictest33-21ksoftirqd/200:21:232
28247993529,4cyclictest9-21ksoftirqd/021:00:400
28247993528,4cyclictest9-21ksoftirqd/019:37:500
28250993427,4cyclictest41-21ksoftirqd/321:33:113
28247993428,3cyclictest9-21ksoftirqd/020:27:090
28247993427,4cyclictest9-21ksoftirqd/022:54:010
28247993427,4cyclictest9-21ksoftirqd/022:24:300
28247993427,4cyclictest9-21ksoftirqd/022:24:300
28247993427,4cyclictest9-21ksoftirqd/022:10:290
28250993327,3cyclictest41-21ksoftirqd/319:46:253
28250993326,5cyclictest41-21ksoftirqd/319:44:113
28249993326,4cyclictest33-21ksoftirqd/222:43:272
28247993327,4cyclictest9-21ksoftirqd/021:44:190
28247993327,3cyclictest9-21ksoftirqd/021:26:030
28247993327,3cyclictest9-21ksoftirqd/020:53:220
28247993327,3cyclictest9-21ksoftirqd/020:53:220
28247993326,4cyclictest9-21ksoftirqd/021:08:350
28247993326,4cyclictest9-21ksoftirqd/020:19:450
28250993225,4cyclictest41-21ksoftirqd/323:49:273
28250993225,4cyclictest41-21ksoftirqd/322:43:483
28250993225,4cyclictest41-21ksoftirqd/300:08:393
28247993226,3cyclictest9-21ksoftirqd/019:59:040
28250993125,4cyclictest41-21ksoftirqd/322:31:153
28249993125,3cyclictest33-21ksoftirqd/223:35:002
28247993125,4cyclictest9-21ksoftirqd/023:05:410
28247993023,4cyclictest9-21ksoftirqd/023:41:220
28247992720,4cyclictest9-21ksoftirqd/019:31:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional