You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-13 - 07:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot2.osadl.org (updated Fri Sep 13, 2024 00:46:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
87921740,8sleep21386899cyclictest22:10:152
977021590,8sleep11386799cyclictest23:35:291
1235911450,118ptp4l401ktimersoftd/319:07:283
1335229012,18sleep20-21swapper/219:07:112
1353328511,18sleep00-21swapper/019:09:230
1386999714,62cyclictest18186-21cut21:35:263
324372690,6sleep31386999cyclictest00:25:173
123591620,1ptp4l391rcuc/321:02:103
13868996148,7cyclictest33-21ksoftirqd/221:30:122
303252590,43sleep235392irq/88-enp1s0-T23:10:252
1386899571,53cyclictest17389-21turbostat23:55:012
1386899571,53cyclictest17389-21turbostat23:55:002
13868995650,4cyclictest33-21ksoftirqd/221:00:002
1386699552,49cyclictest17383-21ls23:55:010
1386699552,49cyclictest17383-21ls23:55:000
57698540,22rtkit-daemon13432-21seq19:08:091
239302530,4sleep023929-21ps19:30:180
239302530,4sleep023929-21ps19:30:180
322732520,2sleep10-21swapper/100:25:151
57698510,2rtkit-daemon0-21swapper/023:01:310
13867995133,15cyclictest25-21ksoftirqd/100:18:531
13868995043,4cyclictest33-21ksoftirqd/222:50:012
13868995043,4cyclictest33-21ksoftirqd/200:20:002
13867995033,14cyclictest25-21ksoftirqd/100:33:331
13868994942,4cyclictest33-21ksoftirqd/221:20:002
13866994939,6cyclictest9-21ksoftirqd/000:35:010
57698480,2rtkit-daemon0-21swapper/019:14:450
1386999483,34cyclictest0-21swapper/323:20:123
13868994842,4cyclictest33-21ksoftirqd/223:00:002
13869994740,4cyclictest41-21ksoftirqd/321:20:003
13869994727,17cyclictest41-21ksoftirqd/320:33:083
13868994740,4cyclictest33-21ksoftirqd/222:25:012
13868994740,4cyclictest33-21ksoftirqd/222:25:002
13867994629,14cyclictest25-21ksoftirqd/119:19:191
13866994640,4cyclictest9-21ksoftirqd/019:37:300
1386999452,35cyclictest24419-21apt-get23:00:123
1386999450,7cyclictest10330-21df21:20:153
13868994538,4cyclictest33-21ksoftirqd/223:15:112
13868994538,4cyclictest33-21ksoftirqd/220:33:122
123591450,1ptp4l401ktimersoftd/300:12:083
1386999442,36cyclictest875-21apt-get22:10:143
1386999442,35cyclictest20951-21apt-get19:25:113
1386999441,37cyclictest4330-21apt-get00:35:133
13868994437,4cyclictest33-21ksoftirqd/222:40:002
13868994437,4cyclictest33-21ksoftirqd/220:10:132
13868994436,4cyclictest33-21ksoftirqd/219:34:592
13868994436,4cyclictest33-21ksoftirqd/219:34:582
123591440,1ptp4l401ktimersoftd/319:35:323
123591440,1ptp4l401ktimersoftd/300:20:163
13869994336,4cyclictest41-21ksoftirqd/320:36:103
1386999431,35cyclictest11863-21apt-get20:15:103
13868994336,4cyclictest33-21ksoftirqd/221:02:092
13868994335,4cyclictest33-21ksoftirqd/200:05:152
13866994337,3cyclictest9-21ksoftirqd/020:19:590
13866994336,4cyclictest9-21ksoftirqd/019:54:590
123591430,1ptp4l401ktimersoftd/321:30:313
13869994235,4cyclictest41-21ksoftirqd/322:55:013
13868994236,4cyclictest33-21ksoftirqd/223:30:152
13868994236,4cyclictest33-21ksoftirqd/222:05:002
13868994236,3cyclictest33-21ksoftirqd/200:30:212
13868994235,4cyclictest33-21ksoftirqd/222:30:002
13868994235,4cyclictest33-21ksoftirqd/222:09:592
13868994234,4cyclictest33-21ksoftirqd/220:51:362
13868994234,4cyclictest33-21ksoftirqd/220:51:352
13867994236,4cyclictest25-21ksoftirqd/122:58:101
13867994236,4cyclictest25-21ksoftirqd/122:33:191
13867994225,14cyclictest25-21ksoftirqd/121:00:451
13866994237,3cyclictest9-21ksoftirqd/021:46:300
13866994235,4cyclictest9-21ksoftirqd/019:25:110
13866994235,4cyclictest24109-21latency_hist23:00:000
123591420,19ptp4l1386999cyclictest22:02:013
13869994134,4cyclictest41-21ksoftirqd/322:25:453
1386999412,33cyclictest22080-21apt-get22:55:123
1386999411,33cyclictest6068-21apt-get23:30:133
1386999410,4cyclictest27566-21grep21:55:273
13868994135,4cyclictest33-21ksoftirqd/220:40:002
13868994135,3cyclictest33-21ksoftirqd/200:40:002
13867994135,4cyclictest25-21ksoftirqd/100:09:251
13866994134,4cyclictest9-21ksoftirqd/020:36:100
13866994134,4cyclictest9-21ksoftirqd/000:18:400
13869994034,4cyclictest41-21ksoftirqd/323:55:203
13869994033,4cyclictest41-21ksoftirqd/319:49:163
13869994033,4cyclictest41-21ksoftirqd/319:30:413
13869994033,4cyclictest41-21ksoftirqd/319:30:413
13869994033,4cyclictest41-21ksoftirqd/300:04:093
13869994032,5cyclictest41-21ksoftirqd/323:07:453
13869994032,5cyclictest41-21ksoftirqd/321:11:063
1386999401,35cyclictest13276-21sort21:25:223
1386999401,32cyclictest16547-21apt-get20:25:123
13869994012,24cyclictest20792-21ls19:25:003
13869994011,25cyclictest28369-21acpi20:50:123
13869994011,25cyclictest28369-21acpi20:50:113
1386999400,4cyclictest31836-21fschecks_count23:15:153
1386999400,4cyclictest14226-21expr19:10:153
1386999400,3cyclictest4209-21sshd19:55:313
13868994035,3cyclictest33-21ksoftirqd/200:00:002
13868994034,3cyclictest33-21ksoftirqd/223:45:582
13868994034,3cyclictest33-21ksoftirqd/220:40:142
13868994033,4cyclictest33-21ksoftirqd/223:03:222
13868994033,4cyclictest33-21ksoftirqd/222:40:282
13868994033,4cyclictest33-21ksoftirqd/222:32:582
13868994033,4cyclictest33-21ksoftirqd/219:21:402
13868994033,4cyclictest33-21ksoftirqd/200:24:122
13868994033,4cyclictest33-21ksoftirqd/200:10:112
13868994033,4cyclictest33-21ksoftirqd/200:05:002
13868994032,5cyclictest33-21ksoftirqd/219:37:022
13868994032,4cyclictest33-21ksoftirqd/219:12:272
1386799404,3cyclictest121rcu_preempt21:50:291
13867994034,4cyclictest25-21ksoftirqd/119:23:581
13867994033,4cyclictest25-21ksoftirqd/123:26:381
13867994033,4cyclictest25-21ksoftirqd/122:48:281
13867994033,4cyclictest25-21ksoftirqd/121:58:581
13867994024,13cyclictest25-21ksoftirqd/120:00:091
13866994035,3cyclictest9-21ksoftirqd/021:00:010
13866994035,3cyclictest9-21ksoftirqd/019:15:310
13866994034,3cyclictest9-21ksoftirqd/022:14:240
13866994034,3cyclictest9-21ksoftirqd/000:08:190
13866994033,4cyclictest9-21ksoftirqd/023:58:560
13866994033,4cyclictest9-21ksoftirqd/022:49:460
123591400,1ptp4l401ktimersoftd/319:50:063
13869993933,4cyclictest41-21ksoftirqd/323:28:133
13869993933,4cyclictest41-21ksoftirqd/320:44:323
13869993933,4cyclictest41-21ksoftirqd/319:16:073
13869993932,4cyclictest41-21ksoftirqd/323:45:023
13869993932,4cyclictest41-21ksoftirqd/320:21:103
1386999392,30cyclictest7230-21apt-get20:05:103
1386999392,30cyclictest15363-21apt-get23:50:133
1386999392,30cyclictest15363-21apt-get23:50:123
1386999392,29cyclictest8381-21apt-get23:35:133
1386999391,31cyclictest22331-21apt-get00:05:133
13869993912,23cyclictest2964-21/usr/sbin/munin21:05:263
1386999390,4cyclictest15056-21acpi22:40:113
1386999390,4cyclictest12798-21grep22:35:143
1386999390,2cyclictest0-21swapper/319:40:303
13868993933,4cyclictest33-21ksoftirqd/221:12:512
13868993933,4cyclictest33-21ksoftirqd/219:50:212
13868993933,3cyclictest33-21ksoftirqd/222:54:172
13868993933,3cyclictest33-21ksoftirqd/221:49:382
13868993932,5cyclictest33-21ksoftirqd/219:25:222
13868993932,4cyclictest33-21ksoftirqd/223:20:122
13868993932,4cyclictest33-21ksoftirqd/220:01:442
13867993933,4cyclictest25-21ksoftirqd/120:45:001
13867993933,3cyclictest25-21ksoftirqd/121:18:551
13867993932,4cyclictest25-21ksoftirqd/123:44:351
13867993932,4cyclictest25-21ksoftirqd/120:50:321
13867993932,4cyclictest25-21ksoftirqd/120:50:321
13867993932,4cyclictest25-21ksoftirqd/119:54:551
13867993931,5cyclictest25-21ksoftirqd/123:17:541
13867993913,23cyclictest8335-21cat22:25:171
13867993912,24cyclictest29354-21expr23:10:121
13866993933,3cyclictest9-21ksoftirqd/023:39:120
13866993932,4cyclictest9-21ksoftirqd/023:13:390
13866993932,4cyclictest9-21ksoftirqd/022:26:140
13866993932,4cyclictest9-21ksoftirqd/021:19:340
13866993932,4cyclictest9-21ksoftirqd/020:51:200
13866993932,4cyclictest9-21ksoftirqd/020:51:190
1386999388,26cyclictest0-21swapper/320:00:173
13869993832,3cyclictest41-21ksoftirqd/320:10:243
1386999382,30cyclictest21640-21apt-get21:45:113
1386999381,30cyclictest29108-21apt-get23:10:113
13869993812,22cyclictest3402-21timedrift00:30:283
1386999380,5cyclictest4767-21ls22:15:263
1386999380,2cyclictest0-21swapper/300:15:193
13868993832,4cyclictest33-21ksoftirqd/223:10:002
13868993832,4cyclictest33-21ksoftirqd/220:30:002
13868993832,3cyclictest33-21ksoftirqd/222:00:002
13868993832,3cyclictest33-21ksoftirqd/220:50:002
13868993832,3cyclictest33-21ksoftirqd/219:47:202
13868993831,4cyclictest33-21ksoftirqd/223:35:412
13868993831,4cyclictest33-21ksoftirqd/221:40:062
13867993833,3cyclictest25-21ksoftirqd/119:49:111
13867993831,4cyclictest25-21ksoftirqd/123:56:161
13867993831,4cyclictest25-21ksoftirqd/123:45:571
13867993831,4cyclictest25-21ksoftirqd/121:06:061
13867993831,4cyclictest25-21ksoftirqd/119:32:211
13867993831,4cyclictest25-21ksoftirqd/119:32:201
13867993831,4cyclictest25-21ksoftirqd/119:25:061
13867993831,4cyclictest25-21ksoftirqd/119:12:051
13867993831,4cyclictest25-21ksoftirqd/100:13:551
13867993830,5cyclictest25-21ksoftirqd/121:38:391
13867993830,5cyclictest25-21ksoftirqd/119:43:521
13867993811,23cyclictest31256-21expr22:05:131
13866993832,3cyclictest9-21ksoftirqd/021:39:590
13866993832,3cyclictest9-21ksoftirqd/021:32:290
13866993831,4cyclictest9-21ksoftirqd/023:28:090
13866993831,4cyclictest9-21ksoftirqd/022:16:100
13866993831,4cyclictest9-21ksoftirqd/020:44:440
13866993831,4cyclictest9-21ksoftirqd/020:11:580
13866993831,4cyclictest9-21ksoftirqd/020:08:080
13866993831,4cyclictest9-21ksoftirqd/000:37:220
13866993830,5cyclictest9-21ksoftirqd/022:53:090
13869993732,3cyclictest41-21ksoftirqd/323:43:503
1386999371,4cyclictest32329-21/usr/sbin/munin22:05:263
13869993712,22cyclictest5822-21cstates22:20:143
13869993712,22cyclictest5822-21cstates22:20:133
1386999370,2cyclictest0-21swapper/321:50:193
13868993730,5cyclictest33-21ksoftirqd/220:20:232
13868993730,4cyclictest33-21ksoftirqd/222:19:022
1386799379,4cyclictest12092-21date21:25:011
13867993731,3cyclictest25-21ksoftirqd/119:36:361
13867993730,4cyclictest25-21ksoftirqd/122:21:021
13867993730,4cyclictest25-21ksoftirqd/122:21:011
13867993730,4cyclictest25-21ksoftirqd/121:14:051
13867993730,4cyclictest25-21ksoftirqd/120:58:071
13867993730,4cyclictest25-21ksoftirqd/120:47:231
13867993729,5cyclictest25-21ksoftirqd/122:19:211
13867993729,5cyclictest25-21ksoftirqd/120:09:141
13867993715,19cyclictest13010-21irqrtprio21:25:191
13866993731,4cyclictest9-21ksoftirqd/023:33:500
13866993731,4cyclictest9-21ksoftirqd/023:19:060
13866993731,4cyclictest9-21ksoftirqd/021:51:390
13866993731,4cyclictest9-21ksoftirqd/020:45:540
13866993731,3cyclictest9-21ksoftirqd/021:13:550
13866993731,3cyclictest9-21ksoftirqd/020:26:220
13866993730,4cyclictest9-21ksoftirqd/000:29:070
1386999369,24cyclictest25262-21snmpd22:34:553
13868993630,4cyclictest33-21ksoftirqd/221:39:592
13868993630,4cyclictest33-21ksoftirqd/221:05:282
13868993629,4cyclictest33-21ksoftirqd/221:52:522
13867993614,18cyclictest18914-21cpu20:30:131
13867993611,22cyclictest4716-21diskstats00:35:181
13867993611,22cyclictest25884-21sed23:00:281
13867993611,22cyclictest21921-21expr21:45:141
13866993631,3cyclictest9-21ksoftirqd/022:08:480
13866993630,3cyclictest9-21ksoftirqd/022:44:130
13866993630,3cyclictest9-21ksoftirqd/022:23:280
13866993630,3cyclictest9-21ksoftirqd/022:23:270
13866993630,3cyclictest9-21ksoftirqd/020:21:080
13866993630,3cyclictest9-21ksoftirqd/019:20:020
13866993629,4cyclictest9-21ksoftirqd/023:24:080
13866993629,4cyclictest9-21ksoftirqd/019:45:520
13866993628,5cyclictest9-21ksoftirqd/023:07:540
13866993628,5cyclictest9-21ksoftirqd/022:38:230
1386999358,24cyclictest26851-21sed20:45:213
1386999354,28cyclictest20333-21cut21:40:243
13869993528,4cyclictest41-21ksoftirqd/322:45:083
13868993529,4cyclictest33-21ksoftirqd/220:05:272
13868993529,4cyclictest33-21ksoftirqd/219:40:312
13868993528,4cyclictest33-21ksoftirqd/221:23:262
13868993528,4cyclictest33-21ksoftirqd/219:57:062
1386799358,4cyclictest20581-21cat00:00:181
13867993529,3cyclictest25-21ksoftirqd/119:59:131
13867993528,4cyclictest25-21ksoftirqd/120:11:581
13867993513,19cyclictest6729-21irqrtprio23:30:221
13866993530,3cyclictest9-21ksoftirqd/023:40:020
13866993529,3cyclictest9-21ksoftirqd/023:45:540
13866993528,4cyclictest9-21ksoftirqd/020:30:110
13866993528,4cyclictest9-21ksoftirqd/020:02:300
1386999349,22cyclictest30817-21cat20:55:133
13868993427,4cyclictest33-21ksoftirqd/221:26:312
13868993427,4cyclictest33-21ksoftirqd/200:25:142
13868993426,5cyclictest33-21ksoftirqd/219:17:562
1386799348,4cyclictest21439-21expr20:35:141
13867993413,18cyclictest12504-21interrupts20:15:171
13866993429,3cyclictest9-21ksoftirqd/021:59:550
13866993428,4cyclictest9-21ksoftirqd/019:44:110
13866993428,4cyclictest9-21ksoftirqd/000:20:280
13866993427,4cyclictest9-21ksoftirqd/022:00:330
13866993427,4cyclictest9-21ksoftirqd/021:43:470
13866993427,4cyclictest9-21ksoftirqd/021:27:450
13866993427,4cyclictest9-21ksoftirqd/021:05:300
13868993327,3cyclictest33-21ksoftirqd/223:41:472
13867993328,3cyclictest25-21ksoftirqd/122:38:351
13867993327,3cyclictest25-21ksoftirqd/123:23:511
13867993326,5cyclictest25-21ksoftirqd/123:05:251
13867993326,4cyclictest25-21ksoftirqd/121:43:231
13867993312,18cyclictest17345-21sed22:45:011
13867993310,3cyclictest21021-21tr22:50:271
13866993327,4cyclictest9-21ksoftirqd/000:14:020
13866993327,3cyclictest9-21ksoftirqd/021:20:540
13866993327,3cyclictest9-21ksoftirqd/021:04:300
13866993327,3cyclictest9-21ksoftirqd/019:59:480
13868993226,4cyclictest33-21ksoftirqd/223:25:152
13868993225,4cyclictest33-21ksoftirqd/220:17:412
1386799329,4cyclictest1126-21runrttasks22:13:331
1386799328,5cyclictest25262-21snmpd23:51:011
1386799328,5cyclictest25262-21snmpd23:51:011
13867993124,4cyclictest25-21ksoftirqd/122:03:151
13867993124,4cyclictest25-21ksoftirqd/121:30:111
13867993124,4cyclictest25-21ksoftirqd/100:20:441
1386799304,4cyclictest15304-21phc2sys20:20:271
13867993023,4cyclictest25-21ksoftirqd/120:29:371
13866993024,4cyclictest9-21ksoftirqd/022:34:410
13866992923,4cyclictest9-21ksoftirqd/000:03:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional