You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-26 - 21:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot3.osadl.org (updated Tue Sep 26, 2023 12:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283032159137,10sleep30-21swapper/307:09:293
576981060,20rtkit-daemon0-21swapper/207:08:102
157772920,2sleep20-21swapper/209:00:182
28633997637,4cyclictest25-21ksoftirqd/112:40:001
2863299736,14cyclictest3236-21sed08:30:410
28633997237,4cyclictest25-21ksoftirqd/110:35:001
28633997135,4cyclictest25-21ksoftirqd/109:55:001
28633997135,4cyclictest25-21ksoftirqd/109:55:001
28632997110,13cyclictest1224-21irqrtprio10:45:300
280532712,57sleep00-21swapper/007:06:180
114732710,3sleep30-21swapper/312:15:253
28633997033,8cyclictest25-21ksoftirqd/109:34:591
2863299698,12cyclictest21958-21sed09:10:380
2863299697,13cyclictest11153-21open_files11:05:330
2863299696,14cyclictest19263-21sort11:25:000
28633996834,4cyclictest25-21ksoftirqd/109:15:261
28633996833,5cyclictest25-21ksoftirqd/112:10:361
2863299684,14cyclictest5180-21sed10:55:000
2863299684,14cyclictest16743-21latency_hist11:20:000
28633996732,8cyclictest25-21ksoftirqd/107:15:241
28633996658,4cyclictest25-21ksoftirqd/107:50:041
28633996656,6cyclictest25-21ksoftirqd/110:10:191
28633996632,5cyclictest25-21ksoftirqd/110:35:221
28633996631,5cyclictest25-21ksoftirqd/108:10:361
2863299666,13cyclictest9144-21sed08:45:280
2863299662,13cyclictest6274-21unixbench_singl09:45:380
28633996558,4cyclictest25-21ksoftirqd/108:25:171
28633996558,4cyclictest25-21ksoftirqd/108:25:171
28633996531,4cyclictest25-21ksoftirqd/111:45:211
28633996530,4cyclictest25-21ksoftirqd/110:40:361
28633996529,5cyclictest25-21ksoftirqd/110:00:251
28633996529,5cyclictest25-21ksoftirqd/108:45:011
2863299654,14cyclictest22523-21sed10:20:370
825591640,1ptp4l401ktimersoftd/308:14:513
28633996430,5cyclictest25-21ksoftirqd/112:05:221
28633996429,5cyclictest25-21ksoftirqd/111:20:261
28633996429,5cyclictest25-21ksoftirqd/109:00:311
2863299646,12cyclictest4289-21df_abs08:35:230
2863299645,13cyclictest19961-21latency_hist12:35:010
28633996327,9cyclictest25-21ksoftirqd/110:20:231
2863299634,14cyclictest13731-21if_enp1s012:20:230
2863299633,14cyclictest16428-21missed_timers12:25:310
28633996228,5cyclictest25-21ksoftirqd/107:10:261
28633996227,5cyclictest25-21ksoftirqd/111:55:341
2863299624,14cyclictest13153-21systemctl10:00:350
2863299623,13cyclictest29677-21/usr/sbin/munin09:30:360
2863299623,13cyclictest1749-21latency_hist09:40:000
28633996128,8cyclictest25-21ksoftirqd/110:50:191
28633996128,4cyclictest25-21ksoftirqd/107:55:341
28633996121,5cyclictest25-21ksoftirqd/109:00:011
2863299614,20cyclictest16353-21idleruntime09:00:260
2863299614,13cyclictest6134-21cat12:05:190
2863299613,14cyclictest1282-21df_abs11:55:220
2863299613,14cyclictest11484-21irqrtprio12:15:260
2863299613,13cyclictest31896-21df_abs08:25:210
2863299613,13cyclictest31896-21df_abs08:25:200
2863299612,13cyclictest13398-21acpi08:55:180
28633996050,6cyclictest25-21ksoftirqd/109:40:191
28633996028,4cyclictest25-21ksoftirqd/111:35:311
28633996026,6cyclictest25-21ksoftirqd/110:25:221
28633996026,6cyclictest25-21ksoftirqd/108:20:281
28633996024,4cyclictest25-21ksoftirqd/111:00:231
28633996021,5cyclictest25-21ksoftirqd/107:20:341
2863399601,7cyclictest25-21ksoftirqd/109:55:271
2863399601,7cyclictest25-21ksoftirqd/109:55:271
28633995951,4cyclictest25-21ksoftirqd/110:45:351
28633995924,7cyclictest25-21ksoftirqd/110:10:011
2863299593,13cyclictest18888-21mailstats07:55:410
28633995845,3cyclictest25-21ksoftirqd/109:25:281
2863399584,3cyclictest121rcu_preempt08:40:011
28633995828,7cyclictest25-21ksoftirqd/111:15:001
28633995823,4cyclictest25-21ksoftirqd/111:40:281
28633995817,8cyclictest25-21ksoftirqd/108:00:201
28633995725,8cyclictest25-21ksoftirqd/109:35:391
28633995724,8cyclictest25-21ksoftirqd/110:55:231
28633995724,4cyclictest25-21ksoftirqd/109:45:371
28633995723,4cyclictest25-21ksoftirqd/112:30:201
28633995717,8cyclictest25-21ksoftirqd/108:45:201
28633995715,7cyclictest25-21ksoftirqd/111:05:211
2863299572,13cyclictest23037-21sensors_temp11:30:410
2863299571,10cyclictest7145-21cat07:30:390
28633995620,5cyclictest25-21ksoftirqd/107:35:251
28633995616,7cyclictest25-21ksoftirqd/107:30:001
2863299560,14cyclictest31729-21find10:40:320
281172565,40sleep10-21swapper/107:07:081
28633995518,8cyclictest25-21ksoftirqd/108:35:001
2863299554,10cyclictest3036-21sshd07:24:100
194272550,2sleep00-21swapper/009:05:340
28635995438,8cyclictest41-21ksoftirqd/310:35:173
28633995446,5cyclictest25-21ksoftirqd/112:00:071
28633995422,4cyclictest25-21ksoftirqd/111:50:171
28633995420,4cyclictest25-21ksoftirqd/112:20:241
28633995420,4cyclictest25-21ksoftirqd/111:30:401
28633995416,7cyclictest25-21ksoftirqd/108:05:241
28635995346,4cyclictest41-21ksoftirqd/309:45:013
28633995346,4cyclictest25-21ksoftirqd/112:19:591
28633995340,10cyclictest25-21ksoftirqd/107:40:331
28635995245,4cyclictest41-21ksoftirqd/309:40:003
28633995243,6cyclictest25-21ksoftirqd/111:20:001
28633995242,6cyclictest25-21ksoftirqd/109:05:191
28633995240,3cyclictest25-21ksoftirqd/109:10:291
28633995217,4cyclictest25-21ksoftirqd/112:30:011
28633995215,4cyclictest25-21ksoftirqd/109:25:001
28633995213,8cyclictest25-21ksoftirqd/108:15:231
825591510,1ptp4l401ktimersoftd/309:07:323
28633995121,4cyclictest25-21ksoftirqd/110:15:181
28633995114,10cyclictest25-21ksoftirqd/107:45:251
270592510,3sleep20-21swapper/208:15:222
825591500,1ptp4l401ktimersoftd/309:15:173
825591500,1ptp4l401ktimersoftd/307:17:513
2863399502,4cyclictest121rcu_preempt07:30:261
28635994943,3cyclictest41-21ksoftirqd/311:00:003
28634994929,4cyclictest33-21ksoftirqd/211:17:362
28634994927,4cyclictest33-21ksoftirqd/211:41:322
2863299496,11cyclictest22949-21irqrtprio08:05:270
28635994842,4cyclictest41-21ksoftirqd/310:30:003
28633994840,4cyclictest25-21ksoftirqd/108:55:001
28633994814,4cyclictest25-21ksoftirqd/111:25:271
28633994814,4cyclictest25-21ksoftirqd/111:25:271
28635994731,9cyclictest41-21ksoftirqd/308:00:193
2863299475,7cyclictest0-21swapper/009:15:330
28635994641,3cyclictest41-21ksoftirqd/308:39:593
28635994639,4cyclictest41-21ksoftirqd/312:35:273
28635994639,4cyclictest41-21ksoftirqd/307:45:213
28635994539,3cyclictest41-21ksoftirqd/309:15:003
28635994538,4cyclictest41-21ksoftirqd/311:40:283
28635994538,4cyclictest41-21ksoftirqd/307:40:003
28635994532,6cyclictest41-21ksoftirqd/307:55:243
2863299457,6cyclictest0-21swapper/008:10:240
2863299457,11cyclictest13495-21phc2sys11:10:290
825591440,1ptp4l401ktimersoftd/311:25:353
825591440,1ptp4l401ktimersoftd/311:25:343
825591440,1ptp4l401ktimersoftd/310:20:273
825591440,1ptp4l401ktimersoftd/308:26:293
825591440,1ptp4l401ktimersoftd/308:26:293
825591440,1ptp4l401ktimersoftd/307:40:133
28635994438,3cyclictest41-21ksoftirqd/309:55:003
28635994438,3cyclictest41-21ksoftirqd/309:55:003
28635994337,4cyclictest41-21ksoftirqd/309:00:293
28635994337,3cyclictest41-21ksoftirqd/310:04:593
28635994337,3cyclictest41-21ksoftirqd/308:55:003
28635994337,3cyclictest41-21ksoftirqd/307:10:313
28635994336,4cyclictest41-21ksoftirqd/309:20:193
28635994330,7cyclictest41-21ksoftirqd/310:30:203
28635994330,7cyclictest41-21ksoftirqd/310:15:173
28635994236,3cyclictest41-21ksoftirqd/312:30:003
28635994235,4cyclictest41-21ksoftirqd/311:55:003
28635994136,3cyclictest41-21ksoftirqd/311:35:243
28635994136,3cyclictest41-21ksoftirqd/308:55:283
28635994135,3cyclictest41-21ksoftirqd/310:50:203
28635994135,3cyclictest41-21ksoftirqd/310:10:013
2863299413,11cyclictest29079-21latency_hist08:20:000
28635994033,4cyclictest41-21ksoftirqd/310:00:003
28635994033,4cyclictest41-21ksoftirqd/310:00:003
28634994035,3cyclictest33-21ksoftirqd/210:45:342
28632994033,4cyclictest9-21ksoftirqd/011:50:370
28635993934,3cyclictest41-21ksoftirqd/311:45:393
28635993934,3cyclictest41-21ksoftirqd/311:05:203
28635993933,4cyclictest41-21ksoftirqd/308:45:243
28635993933,3cyclictest41-21ksoftirqd/311:25:003
28635993932,4cyclictest41-21ksoftirqd/310:10:283
28634993933,4cyclictest33-21ksoftirqd/207:37:282
2863499392,34cyclictest25215-21sshd10:26:282
2863299393,10cyclictest24899-21ssh10:25:380
825591380,14ptp4l0-21swapper/308:23:533
28635993832,4cyclictest41-21ksoftirqd/312:05:013
28635993832,3cyclictest41-21ksoftirqd/308:40:273
28635993824,7cyclictest41-21ksoftirqd/307:25:223
28634993832,3cyclictest33-21ksoftirqd/210:07:332
2863299389,12cyclictest8777-21grep11:00:340
2863299385,20cyclictest30800-21sed07:15:010
28632993832,4cyclictest9-21ksoftirqd/009:45:010
825591370,1ptp4l401ktimersoftd/312:05:363
825591370,12ptp4l0-21swapper/312:13:263
28635993732,3cyclictest41-21ksoftirqd/311:15:333
28635993731,4cyclictest41-21ksoftirqd/312:30:303
28635993731,4cyclictest41-21ksoftirqd/311:55:313
28635993731,4cyclictest41-21ksoftirqd/310:49:293
28635993731,4cyclictest41-21ksoftirqd/310:45:003
28635993731,4cyclictest41-21ksoftirqd/309:25:293
28635993731,3cyclictest41-21ksoftirqd/311:30:243
28635993730,4cyclictest41-21ksoftirqd/308:08:553
28634993731,4cyclictest33-21ksoftirqd/207:16:422
28634993730,4cyclictest33-21ksoftirqd/209:45:472
28634993730,4cyclictest33-21ksoftirqd/207:45:002
28634993729,5cyclictest33-21ksoftirqd/211:02:152
2863299374,12cyclictest0-21swapper/010:35:260
825591360,13ptp4l0-21swapper/308:33:183
2863599360,4cyclictest121rcu_preempt09:45:213
28634993630,4cyclictest33-21ksoftirqd/210:03:482
28634993630,4cyclictest33-21ksoftirqd/209:40:052
28634993630,3cyclictest33-21ksoftirqd/208:37:592
28634993629,4cyclictest33-21ksoftirqd/212:26:382
28634993629,4cyclictest33-21ksoftirqd/210:51:412
28634993629,4cyclictest33-21ksoftirqd/209:34:392
28634993629,4cyclictest33-21ksoftirqd/208:34:212
28634993629,4cyclictest33-21ksoftirqd/207:57:342
28634993629,4cyclictest33-21ksoftirqd/207:50:502
28634993628,5cyclictest33-21ksoftirqd/210:18:552
2863299367,12cyclictest27349-21/usr/sbin/munin09:25:320
28632993631,3cyclictest9-21ksoftirqd/012:13:290
28632993629,4cyclictest9-21ksoftirqd/011:46:040
28635993529,4cyclictest41-21ksoftirqd/307:50:383
28635993529,3cyclictest41-21ksoftirqd/307:35:003
28635993529,3cyclictest41-21ksoftirqd/307:20:293
28635993528,4cyclictest41-21ksoftirqd/308:20:003
28634993529,3cyclictest33-21ksoftirqd/211:29:182
28634993529,3cyclictest33-21ksoftirqd/211:29:172
28634993528,4cyclictest33-21ksoftirqd/211:38:032
28634993528,4cyclictest33-21ksoftirqd/211:08:122
28634993528,4cyclictest33-21ksoftirqd/210:22:312
28634993528,4cyclictest33-21ksoftirqd/210:13:052
28634993528,4cyclictest33-21ksoftirqd/209:54:192
28634993528,4cyclictest33-21ksoftirqd/209:54:192
28634993528,4cyclictest33-21ksoftirqd/209:16:162
28634993528,4cyclictest33-21ksoftirqd/208:41:512
28634993528,4cyclictest33-21ksoftirqd/207:33:282
28634993527,5cyclictest33-21ksoftirqd/210:40:182
28632993528,4cyclictest9-21ksoftirqd/011:40:450
28632993528,4cyclictest9-21ksoftirqd/011:29:340
28632993528,4cyclictest9-21ksoftirqd/011:29:330
825591340,1ptp4l401ktimersoftd/309:30:453
28635993428,4cyclictest41-21ksoftirqd/312:20:253
28634993429,3cyclictest33-21ksoftirqd/208:20:332
28634993428,4cyclictest33-21ksoftirqd/210:34:282
28634993428,4cyclictest33-21ksoftirqd/207:14:252
28634993427,4cyclictest33-21ksoftirqd/212:39:282
28634993427,4cyclictest33-21ksoftirqd/212:04:592
28634993427,4cyclictest33-21ksoftirqd/209:56:022
28634993427,4cyclictest33-21ksoftirqd/209:56:022
28634993427,4cyclictest33-21ksoftirqd/208:26:582
28634993427,4cyclictest33-21ksoftirqd/208:26:582
28634993427,4cyclictest33-21ksoftirqd/208:14:572
28634993427,4cyclictest33-21ksoftirqd/207:47:442
28634993427,4cyclictest33-21ksoftirqd/207:29:382
28632993428,4cyclictest12429-21cron07:45:000
28632993427,4cyclictest9-21ksoftirqd/007:45:420
28635993326,4cyclictest41-21ksoftirqd/311:13:443
28635993326,4cyclictest41-21ksoftirqd/311:03:323
28634993327,3cyclictest33-21ksoftirqd/209:13:452
28634993326,4cyclictest33-21ksoftirqd/209:09:052
28634993326,4cyclictest33-21ksoftirqd/208:49:402
28634993326,4cyclictest33-21ksoftirqd/207:20:342
2863299337,11cyclictest6863-21if_enp1s008:40:250
28632993327,4cyclictest9-21ksoftirqd/009:21:120
28632993327,3cyclictest9-21ksoftirqd/010:32:130
28632993326,4cyclictest9-21ksoftirqd/007:54:090
28632993326,4cyclictest9-21ksoftirqd/007:25:110
28634993227,3cyclictest33-21ksoftirqd/212:06:192
28634993226,3cyclictest33-21ksoftirqd/212:19:572
2863299326,6cyclictest0-21swapper/008:20:230
2863299325,13cyclictest0-21swapper/007:16:140
28632993226,3cyclictest9-21ksoftirqd/012:02:350
28634993125,4cyclictest33-21ksoftirqd/208:55:312
28634993125,4cyclictest33-21ksoftirqd/208:05:002
28634993124,4cyclictest33-21ksoftirqd/211:55:132
28634993124,4cyclictest33-21ksoftirqd/211:47:022
28634993124,4cyclictest33-21ksoftirqd/208:08:522
28634993023,4cyclictest33-21ksoftirqd/211:12:212
28634993023,4cyclictest33-21ksoftirqd/209:24:092
28632993025,3cyclictest9-21ksoftirqd/009:54:280
28632993025,3cyclictest9-21ksoftirqd/009:54:280
28632993024,4cyclictest9-21ksoftirqd/011:36:440
28632993023,4cyclictest9-21ksoftirqd/012:38:500
28632993023,4cyclictest9-21ksoftirqd/010:10:180
28632993023,4cyclictest9-21ksoftirqd/008:55:000
2863299302,11cyclictest0-21swapper/010:20:000
2863299301,12cyclictest0-21swapper/007:35:200
28634992924,3cyclictest33-21ksoftirqd/211:35:002
28634992923,4cyclictest33-21ksoftirqd/210:35:402
28634992923,4cyclictest33-21ksoftirqd/208:54:472
28634992923,3cyclictest33-21ksoftirqd/212:22:052
28634992923,3cyclictest33-21ksoftirqd/210:57:192
28634992922,4cyclictest33-21ksoftirqd/209:37:312
2863299291,11cyclictest0-21swapper/008:00:370
28634992822,4cyclictest33-21ksoftirqd/209:29:532
28634992822,3cyclictest33-21ksoftirqd/211:21:382
28632992821,4cyclictest9-21ksoftirqd/010:09:380
2863299280,13cyclictest0-21swapper/010:55:310
28632992721,4cyclictest9-21ksoftirqd/009:58:250
28632992721,4cyclictest9-21ksoftirqd/009:58:250
28634992620,3cyclictest33-21ksoftirqd/212:10:262
28634992620,3cyclictest33-21ksoftirqd/211:51:212
28634992519,4cyclictest33-21ksoftirqd/212:34:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional