You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 08:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3.osadl.org (updated Sat Jul 27, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292565699671,3cyclictest3079041-21fschecks_time23:35:121
292565699651,1cyclictest0-21swapper/121:11:411
2925656996333,28cyclictest3009487-21kworker/1:322:25:171
2925656996333,28cyclictest2927112-21kworker/1:219:25:141
292565699630,61cyclictest0-21swapper/100:27:111
2925656996235,26cyclictest3055534-21kworker/1:123:10:131
2925656996230,30cyclictest3117226-21kworker/1:300:35:121
2925656996143,17cyclictest3009487-21kworker/1:322:22:451
2925656996143,17cyclictest2982696-21kworker/1:321:35:141
2925656996142,18cyclictest3055534-21kworker/1:123:15:131
2925656996132,28cyclictest3117226-21kworker/1:300:30:011
2925656996041,17cyclictest3097228-21kworker/1:100:10:161
2925656996035,23cyclictest2965444-21kworker/1:121:25:201
2925656995956,2cyclictest0-21swapper/122:14:551
2925656995950,7cyclictest0-21swapper/120:30:211
2925656995942,16cyclictest2944777-21kworker/1:320:10:131
2925656995940,18cyclictest3060047-21kworker/1:223:30:311
2925656995843,14cyclictest3060047-21kworker/1:223:20:171
2925656995843,14cyclictest2927112-21kworker/1:219:20:121
2925656995742,14cyclictest3051520-21kworker/1:323:02:111
2925656995742,14cyclictest2955709-21kworker/1:020:42:401
2925656995742,14cyclictest2936657-21kworker/1:120:00:141
2925656995742,14cyclictest2936657-21kworker/1:119:55:401
2925656995741,15cyclictest2982696-21kworker/1:321:33:131
2925656995740,16cyclictest3039044-21kworker/1:122:45:141
2925656995739,17cyclictest1366723-1kworker/1:1H22:30:131
2925656995737,18cyclictest1366723-1kworker/1:1H22:00:141
2925656995733,23cyclictest3051169-21kworker/1:023:00:001
2925656995721,25cyclictest0-21swapper/122:40:101
292565699570,55cyclictest0-21swapper/123:43:111
2925656995641,14cyclictest3097228-21kworker/1:100:00:151
2925656995641,14cyclictest3097228-21kworker/1:100:00:151
2925656995641,14cyclictest3009487-21kworker/1:322:38:401
2925656995641,14cyclictest3009487-21kworker/1:322:38:401
2925656995641,14cyclictest2922724-21kworker/1:019:10:131
2925656995640,15cyclictest3060047-21kworker/1:223:25:201
2925656995640,15cyclictest2995499-21kworker/1:021:55:141
2925656995640,15cyclictest2944777-21kworker/1:320:15:151
2925656995640,15cyclictest2936657-21kworker/1:119:53:401
2925656995639,16cyclictest3097228-21kworker/1:100:00:011
2925656995631,24cyclictest2958005-21kworker/1:321:05:151
2925656995625,30cyclictest2955709-21kworker/1:020:45:131
2925656995540,14cyclictest3081055-21kworker/1:023:50:121
2925656995540,14cyclictest2965444-21kworker/1:121:20:111
2925656995539,15cyclictest1366723-1kworker/1:1H19:17:331
2925656995538,16cyclictest2982696-21kworker/1:321:40:221
2925656995534,20cyclictest3097228-21kworker/1:100:15:131
2925659995449,3cyclictest1632082-1kworker/2:1H22:20:142
2925656995439,14cyclictest3081055-21kworker/1:023:45:051
2925656995439,14cyclictest3009487-21kworker/1:322:15:131
2925656995439,14cyclictest2944777-21kworker/1:320:25:131
2925656995439,14cyclictest2936657-21kworker/1:119:40:151
2925656995439,14cyclictest2936657-21kworker/1:119:40:151
2925656995439,14cyclictest1366723-1kworker/1:1H23:05:131
2925656995436,16cyclictest2944777-21kworker/1:320:20:141
292565699543,49cyclictest2933450-21kworker/1:319:35:001
292565699541,2cyclictest378-21systemd-journal19:37:331
2925659995348,3cyclictest1632082-1kworker/2:1H22:57:392
2925656995338,14cyclictest3111235-21kworker/1:000:20:141
2925656995338,14cyclictest2953841-21kworker/1:120:35:261
2925656995338,14cyclictest2944125-21kworker/1:020:05:141
2925656995337,15cyclictest2958005-21kworker/1:320:55:141
2925656995337,15cyclictest2936657-21kworker/1:119:45:261
2925656995337,15cyclictest2936657-21kworker/1:119:45:251
2925656995335,17cyclictest2995499-21kworker/1:022:05:171
2925660995247,3cyclictest3394600-1kworker/3:2H23:55:203
2925659995247,3cyclictest1632082-1kworker/2:1H00:00:152
2925659995247,3cyclictest1632082-1kworker/2:1H00:00:152
2925656995237,14cyclictest2958005-21kworker/1:321:00:281
2925656995236,15cyclictest2958005-21kworker/1:320:50:001
292565699521,50cyclictest0-21swapper/122:50:091
292565699521,49cyclictest0-21swapper/121:18:001
292565699520,2cyclictest3001201-21ssh21:55:011
292565699520,2cyclictest0-21swapper/100:05:211
292565699520,2cyclictest0-21swapper/100:05:211
2925659995147,2cyclictest2948982-21kworker/2:221:29:392
2925659995146,3cyclictest1632082-1kworker/2:1H23:21:402
2925654995148,2cyclictest2988744-21kworker/0:121:44:460
2925654995146,3cyclictest1308596-21kworker/0:019:25:410
2925654995145,3cyclictest278-1kworker/0:2H23:10:130
2925659995045,3cyclictest2981438-21kworker/2:123:13:392
2925654995047,2cyclictest3109551-21kworker/0:100:22:560
2925654995047,2cyclictest3031686-21kworker/0:022:37:400
2925654995047,2cyclictest3031686-21kworker/0:022:37:400
2925654995047,2cyclictest3022630-21kworker/0:222:30:460
2925654995047,2cyclictest2934646-21kworker/0:119:37:040
2925654995046,2cyclictest2980274-21kworker/0:221:33:390
2925654995046,2cyclictest278-1kworker/0:2H22:25:170
2925654995045,3cyclictest278-1kworker/0:2H20:39:550
2925660994946,2cyclictest3394600-1kworker/3:2H22:30:263
2925660994945,3cyclictest3394600-1kworker/3:2H23:25:203
2925660994945,3cyclictest3394600-1kworker/3:2H21:20:533
2925660994943,4cyclictest3394600-1kworker/3:2H00:17:063
2925659994946,2cyclictest2981438-21kworker/2:121:45:402
2925659994946,2cyclictest2934399-21kworker/2:119:39:392
2925659994945,2cyclictest1632082-1kworker/2:1H00:07:412
2925659994945,2cyclictest1632082-1kworker/2:1H00:07:412
2925654994946,2cyclictest3039778-21kworker/0:222:47:100
2925654994946,2cyclictest2934646-21kworker/0:120:53:090
2925660994844,3cyclictest3394600-1kworker/3:2H21:45:063
2925660994844,2cyclictest3022832-21kworker/3:122:25:053
2925660994842,3cyclictest3394600-1kworker/3:2H00:40:013
2925659994845,2cyclictest3070702-21kworker/2:023:36:112
2925659994845,2cyclictest2981438-21kworker/2:122:45:562
2925659994844,3cyclictest1632082-1kworker/2:1H20:28:112
2925659994844,2cyclictest1632082-1kworker/2:1H22:35:182
2925659994844,2cyclictest1632082-1kworker/2:1H22:35:182
2925654994845,2cyclictest278-1kworker/0:2H19:45:370
2925654994845,2cyclictest278-1kworker/0:2H19:45:360
2925654994844,3cyclictest2953838-21kworker/0:020:34:020
2925660994735,1cyclictest2967491-21kworker/3:121:15:203
2925659994743,3cyclictest1632082-1kworker/2:1H22:10:412
2925659994742,3cyclictest2981438-21kworker/2:122:17:062
292565699471,45cyclictest125550irq/16-snd_hda_21:48:101
2925660994642,2cyclictest3394600-1kworker/3:2H22:53:343
2925659994643,2cyclictest2981438-21kworker/2:123:05:262
2925659994643,2cyclictest2981438-21kworker/2:122:06:552
2925659994642,3cyclictest1632082-1kworker/2:1H21:57:402
2925659994642,2cyclictest1632082-1kworker/2:1H20:15:152
2925659994641,3cyclictest1632082-1kworker/2:1H00:10:172
2925659994640,3cyclictest1632082-1kworker/2:1H21:50:152
2925654994643,2cyclictest3101292-21kworker/0:000:14:050
2925654994642,3cyclictest278-1kworker/0:2H00:30:010
2925654994641,3cyclictest3035702-21kworker/0:122:41:050
2925654994641,3cyclictest278-1kworker/0:2H22:50:120
2925660994542,2cyclictest3394600-1kworker/3:2H22:05:473
2925660994542,2cyclictest3394600-1kworker/3:2H21:34:493
2925660994542,2cyclictest3081151-21kworker/3:023:39:403
2925660994541,3cyclictest3394600-1kworker/3:2H19:48:063
2925660994541,3cyclictest3394600-1kworker/3:2H19:48:063
2925660994541,2cyclictest3394600-1kworker/3:2H23:43:503
2925660994541,2cyclictest3394600-1kworker/3:2H19:55:063
2925659994542,2cyclictest3093715-21kworker/2:200:21:022
2925659994542,2cyclictest2981438-21kworker/2:121:33:462
2925659994542,2cyclictest1632082-1kworker/2:1H20:02:422
2925659994541,2cyclictest3070702-21kworker/2:023:55:402
2925659994540,3cyclictest1632082-1kworker/2:1H21:00:162
2925654994542,2cyclictest3008558-21kworker/0:122:20:310
2925654994542,2cyclictest2980274-21kworker/0:221:49:100
2925654994542,2cyclictest2934646-21kworker/0:120:58:100
2925654994542,2cyclictest2934646-21kworker/0:120:00:330
2925654994542,2cyclictest278-1kworker/0:2H23:43:450
2925660994440,3cyclictest3394600-1kworker/3:2H00:10:423
2925660994439,3cyclictest3394600-1kworker/3:2H21:43:403
2925660994439,3cyclictest3093344-21kworker/3:000:05:063
2925660994439,3cyclictest3093344-21kworker/3:000:05:063
2925659994441,2cyclictest3070702-21kworker/2:023:40:442
2925659994441,2cyclictest2981438-21kworker/2:123:15:232
2925659994440,3cyclictest1632082-1kworker/2:1H19:48:112
2925659994440,3cyclictest1632082-1kworker/2:1H19:48:112
2925654994441,2cyclictest278-1kworker/0:2H00:00:360
2925654994441,2cyclictest278-1kworker/0:2H00:00:360
2925654994439,3cyclictest3008558-21kworker/0:122:15:000
2925654994438,4cyclictest278-1kworker/0:2H21:35:060
2925654994437,5cyclictest278-1kworker/0:2H19:10:100
2925654994437,4cyclictest278-1kworker/0:2H19:20:140
2925660994340,2cyclictest3394600-1kworker/3:2H22:10:253
2925660994340,2cyclictest2967491-21kworker/3:121:14:053
2925660994339,3cyclictest3394600-1kworker/3:2H23:24:403
2925660994339,2cyclictest3394600-1kworker/3:2H21:25:203
2925659994340,2cyclictest1632082-1kworker/2:1H00:34:052
2925659994339,3cyclictest2924070-21kworker/2:019:11:402
2925654994340,2cyclictest278-1kworker/0:2H20:49:400
2925654994339,2cyclictest278-1kworker/0:2H23:30:150
2925654994337,3cyclictest278-1kworker/0:2H20:30:010
2925660994239,2cyclictest3394600-1kworker/3:2H20:00:063
2925660994239,2cyclictest3394600-1kworker/3:2H19:23:393
2925660994238,3cyclictest3394600-1kworker/3:2H22:45:273
2925660994238,3cyclictest3394600-1kworker/3:2H21:08:403
2925660994238,3cyclictest3394600-1kworker/3:2H20:45:183
2925660994238,3cyclictest3394600-1kworker/3:2H19:25:403
2925660994238,2cyclictest2947366-21kworker/3:120:19:563
2925659994239,2cyclictest2981438-21kworker/2:123:01:082
2925659994239,2cyclictest2948982-21kworker/2:221:16:402
2925659994239,2cyclictest2948982-21kworker/2:220:35:132
2925659994239,2cyclictest2934399-21kworker/2:120:00:002
2925659994239,2cyclictest2928007-21kworker/2:219:25:212
2925659994238,2cyclictest2928007-21kworker/2:219:20:382
2925659994236,4cyclictest1632082-1kworker/2:1H19:15:412
2925654994240,1cyclictest3045681-21kworker/0:023:06:520
2925654994239,2cyclictest3057287-21kworker/0:123:45:390
2925654994239,2cyclictest278-1kworker/0:2H21:05:160
2925654994237,3cyclictest278-1kworker/0:2H19:15:010
2925660994138,2cyclictest3394600-1kworker/3:2H21:00:043
2925660994138,2cyclictest3394600-1kworker/3:2H20:33:053
2925660994138,2cyclictest3394600-1kworker/3:2H19:11:073
2925660994136,3cyclictest3394600-1kworker/3:2H21:40:013
2925660994135,4cyclictest3394600-1kworker/3:2H23:50:103
2925659994139,1cyclictest2948982-21kworker/2:221:09:112
2925659994138,2cyclictest2981438-21kworker/2:122:33:402
2925659994138,2cyclictest2981438-21kworker/2:122:27:342
2925659994137,2cyclictest1632082-1kworker/2:1H20:55:142
2925659994136,3cyclictest1632082-1kworker/2:1H21:12:072
2925654994138,2cyclictest2960831-21kworker/0:021:10:340
2925654994138,2cyclictest278-1kworker/0:2H23:53:400
2925654994137,3cyclictest278-1kworker/0:2H00:10:000
2925654994137,3cyclictest278-1kworker/0:2H00:10:000
2925654994136,3cyclictest3057287-21kworker/0:123:22:150
2925654994136,3cyclictest278-1kworker/0:2H20:40:200
2925660994037,2cyclictest3394600-1kworker/3:2H22:19:053
2925660994037,2cyclictest3394600-1kworker/3:2H20:51:033
2925660994037,2cyclictest3394600-1kworker/3:2H00:20:373
2925660994036,3cyclictest3394600-1kworker/3:2H20:41:093
2925660994035,3cyclictest3394600-1kworker/3:2H20:10:363
2925660994031,6cyclictest3394600-1kworker/3:2H22:40:003
2925660994031,6cyclictest3394600-1kworker/3:2H22:40:003
2925659994037,2cyclictest3093715-21kworker/2:200:25:112
2925659994036,2cyclictest2934399-21kworker/2:119:51:402
2925659994035,3cyclictest2981438-21kworker/2:121:40:052
292565699400,37cyclictest2925683-21latency_hist19:10:001
2925654994037,2cyclictest2980274-21kworker/0:221:55:320
2925654994037,2cyclictest278-1kworker/0:2H20:20:270
2925654994036,3cyclictest278-1kworker/0:2H00:35:210
2925654994036,2cyclictest278-1kworker/0:2H22:55:150
2925660993936,2cyclictest3394600-1kworker/3:2H20:06:063
2925660993936,2cyclictest3394600-1kworker/3:2H19:40:053
2925660993936,2cyclictest3394600-1kworker/3:2H19:40:043
2925660993936,2cyclictest3394600-1kworker/3:2H00:32:463
2925660993936,2cyclictest3051258-21kworker/3:223:05:173
2925660993935,3cyclictest2996490-21kworker/3:021:55:443
2925659993936,2cyclictest2981438-21kworker/2:122:50:182
2925659993936,2cyclictest2934399-21kworker/2:119:42:052
2925659993936,2cyclictest2934399-21kworker/2:119:42:052
2925659993936,2cyclictest1632082-1kworker/2:1H20:32:052
2925654993936,2cyclictest3090131-21kworker/0:223:55:160
2925654993936,2cyclictest3016645-21kworker/0:022:15:400
2925660993835,2cyclictest3394600-1kworker/3:2H23:49:153
2925660993835,2cyclictest3394600-1kworker/3:2H22:42:403
2925659993835,2cyclictest2948982-21kworker/2:220:53:392
2925659993834,3cyclictest2981438-21kworker/2:122:00:412
2925659993833,3cyclictest2981438-21kworker/2:122:40:402
2925659993833,3cyclictest2948982-21kworker/2:220:46:072
2925654993835,2cyclictest2947357-21kworker/0:220:16:110
2925654993835,2cyclictest2934646-21kworker/0:120:08:080
2925654993835,2cyclictest278-1kworker/0:2H23:29:100
2925654993834,3cyclictest278-1kworker/0:2H22:02:400
2925654993834,3cyclictest278-1kworker/0:2H21:52:050
2925654993834,2cyclictest2980274-21kworker/0:221:30:000
2925660993734,2cyclictest3051258-21kworker/3:222:59:553
292566099370,6cyclictest151rcu_preempt19:35:213
2925659993734,2cyclictest2948982-21kworker/2:220:40:272
2925659993733,3cyclictest3093715-21kworker/2:200:15:572
2925654993733,2cyclictest278-1kworker/0:2H23:35:120
2925660993634,1cyclictest3394600-1kworker/3:2H22:00:463
2925660993633,2cyclictest3394600-1kworker/3:2H19:19:553
2925660993633,2cyclictest3064903-21kworker/3:023:19:553
2925660993632,2cyclictest3394600-1kworker/3:2H20:25:053
2925659993631,3cyclictest2948982-21kworker/2:220:21:062
2925659993631,3cyclictest2928007-21kworker/2:219:30:252
2925654993632,3cyclictest278-1kworker/0:2H22:08:150
2925654993632,2cyclictest278-1kworker/0:2H20:10:140
2925660993532,2cyclictest3394600-1kworker/3:2H19:33:053
2925660993532,2cyclictest3073056-21kworker/3:223:30:303
2925660993532,2cyclictest2960582-21kworker/3:020:59:553
2925659993532,2cyclictest3070702-21kworker/2:023:29:402
2925659993530,3cyclictest1632082-1kworker/2:1H23:50:122
2925659993530,3cyclictest1632082-1kworker/2:1H23:35:002
2925660993432,1cyclictest2996490-21kworker/3:021:50:143
2925660993430,3cyclictest3394600-1kworker/3:2H22:20:263
2925659993431,2cyclictest2948982-21kworker/2:221:24:042
2925654993430,2cyclictest278-1kworker/0:2H19:30:130
2925654993430,2cyclictest278-1kworker/0:2H00:15:130
2925654993425,7cyclictest278-1kworker/0:2H21:20:050
2925660993331,1cyclictest3107555-21kworker/3:100:29:553
2925659993328,3cyclictest3070702-21kworker/2:023:45:252
2925659993328,3cyclictest1632082-1kworker/2:1H21:38:052
2925654993128,2cyclictest278-1kworker/0:2H19:45:000
2925654993128,2cyclictest278-1kworker/0:2H19:45:000
2925654993117,8cyclictest2973633-21turbostat21:20:000
2925654993027,2cyclictest3057287-21kworker/0:123:19:390
2925654993026,3cyclictest278-1kworker/0:2H23:02:100
2925660992926,2cyclictest3051258-21kworker/3:223:04:053
2925660992925,3cyclictest3394600-1kworker/3:2H19:53:013
292566099283,6cyclictest151rcu_preempt00:00:123
292566099283,6cyclictest151rcu_preempt00:00:123
2925660992825,2cyclictest2947366-21kworker/3:120:25:013
2925659992823,3cyclictest2945804-21kworker/2:020:13:102
2925659992623,2cyclictest3093715-21kworker/2:200:37:062
2925659992622,3cyclictest1632082-1kworker/2:1H20:05:102
2925654992623,2cyclictest278-1kworker/0:2H19:59:400
2925660992417,5cyclictest40-21ksoftirqd/323:15:013
292565499220,20cyclictest3123359-21ssh00:32:160
292566099214,5cyclictest2955192-21/usr/sbin/munin20:35:193
292565499203,15cyclictest2963844-21cron21:05:000
292565499201,15cyclictest2939535-21apt-get19:50:100
292565999160,7cyclictest0-21swapper/219:09:592
292565499161,12cyclictest2925692-21date19:10:000
292566099131,2cyclictest151rcu_preempt19:09:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional