You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 16:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3.osadl.org (updated Sat Apr 27, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2998927997236,34cyclictest114228-1kworker/1:1H23:05:011
2998927997138,31cyclictest3129441-21kworker/1:223:25:151
2998927996937,30cyclictest114228-1kworker/1:1H19:35:121
2998927996836,31cyclictest3027930-21kworker/1:020:44:591
2998927996836,30cyclictest114228-1kworker/1:1H00:25:001
2998927996640,24cyclictest114228-1kworker/1:1H20:55:001
299892799660,65cyclictest0-21swapper/121:12:511
2998927996541,22cyclictest114228-1kworker/1:1H20:59:591
2998927996532,31cyclictest3043155-21kworker/1:121:30:041
2998927996531,33cyclictest114228-1kworker/1:1H22:20:131
2998927996531,32cyclictest3184931-21kworker/1:100:30:131
2998927996433,29cyclictest2996506-21kworker/1:119:25:131
2998927996359,2cyclictest0-21swapper/123:20:271
2998927996337,24cyclictest2996506-21kworker/1:119:40:151
2998927996333,28cyclictest3066581-21kworker/1:221:55:141
2998927996259,2cyclictest0-21swapper/122:33:001
2998927996235,26cyclictest3184931-21kworker/1:100:40:011
2998927996151,8cyclictest0-21swapper/119:20:141
2998927996151,8cyclictest0-21swapper/119:20:141
2998927996139,21cyclictest3028921-21kworker/1:321:05:151
2998927996137,23cyclictest3010352-21kworker/1:220:05:011
2998927996128,31cyclictest3085180-21kworker/1:222:15:131
2998927996057,2cyclictest0-21swapper/122:35:161
2998927996044,15cyclictest3066581-21kworker/1:221:49:551
2998927996041,18cyclictest3102229-21kworker/1:122:45:151
2998927996033,26cyclictest114228-1kworker/1:1H20:05:111
2998927996030,29cyclictest3018658-21kworker/1:120:30:221
2998927996026,33cyclictest3143755-21kworker/1:023:35:011
2998927995944,14cyclictest3085180-21kworker/1:222:25:121
2998927995942,16cyclictest3180092-21kworker/1:200:15:131
2998927995937,21cyclictest3184931-21kworker/1:100:25:151
2998928995853,3cyclictest1135395-1kworker/2:1H22:00:102
2998927995843,14cyclictest3143755-21kworker/1:023:50:131
2998927995843,14cyclictest114228-1kworker/1:1H23:30:151
2998927995840,17cyclictest3143755-21kworker/1:000:00:161
2998927995826,30cyclictest3143755-21kworker/1:023:55:141
2998927995826,30cyclictest3143755-21kworker/1:023:55:131
2998927995824,33cyclictest3043155-21kworker/1:121:40:151
2998927995742,14cyclictest3102229-21kworker/1:123:05:151
2998927995742,14cyclictest3075473-21kworker/1:122:02:591
2998927995742,14cyclictest3066581-21kworker/1:221:50:171
2998927995742,14cyclictest114228-1kworker/1:1H00:10:151
2998927995741,15cyclictest3143755-21kworker/1:023:45:051
2998927995740,15cyclictest3018658-21kworker/1:120:20:131
2998927995730,25cyclictest114228-1kworker/1:1H22:05:171
2998927995641,14cyclictest3143755-21kworker/1:000:05:131
2998927995641,14cyclictest3018658-21kworker/1:120:35:191
2998927995641,14cyclictest2996506-21kworker/1:119:30:131
2998927995641,14cyclictest114228-1kworker/1:1H21:15:131
2998927995640,15cyclictest3129441-21kworker/1:223:15:161
2998927995640,15cyclictest3043155-21kworker/1:121:24:551
2998927995626,28cyclictest3028921-21kworker/1:321:00:121
299892799562,52cyclictest1-21systemd20:15:011
2998927995539,15cyclictest2996506-21kworker/1:119:16:561
2998927995537,16cyclictest3043155-21kworker/1:121:25:141
2998927995535,18cyclictest3075473-21kworker/1:122:10:141
2998927995527,27cyclictest114228-1kworker/1:1H23:40:141
2998927995523,31cyclictest114228-1kworker/1:1H19:55:161
2998927995439,14cyclictest3018658-21kworker/1:120:29:551
2998927995439,14cyclictest3018658-21kworker/1:120:15:121
2998927995438,15cyclictest3010352-21kworker/1:219:46:551
2998927995437,16cyclictest3043155-21kworker/1:121:39:561
2998927995414,37cyclictest3131798-21diskmemload23:11:431
2998930995348,3cyclictest2062265-1kworker/3:1H19:38:533
2998927995337,15cyclictest2996506-21kworker/1:119:10:061
2998926995349,2cyclictest3022701-21kworker/0:020:57:450
2998928995248,2cyclictest2997433-21kworker/2:219:27:192
2998927995237,14cyclictest3102229-21kworker/1:122:50:141
2998927995237,14cyclictest3028921-21kworker/1:320:45:551
2998927995237,14cyclictest3010352-21kworker/1:219:51:571
2998927995235,16cyclictest3102229-21kworker/1:122:45:011
2998926995248,2cyclictest3035036-21kworker/0:321:45:550
2998930995148,2cyclictest3030975-21kworker/3:221:05:343
2998930995148,2cyclictest2940607-21kworker/3:119:41:223
2998930995147,3cyclictest2940607-21kworker/3:119:51:273
2998930995146,3cyclictest2062265-1kworker/3:1H21:19:593
2998928995147,2cyclictest3071708-21kworker/2:022:50:412
2998926995148,2cyclictest3002370-21kworker/0:221:03:250
2998926995148,2cyclictest2997356-21kworker/0:119:12:540
2998926995147,2cyclictest3022701-21kworker/0:020:25:350
2998930995045,3cyclictest3188151-21kworker/3:000:31:083
2998930995045,3cyclictest3030975-21kworker/3:221:30:013
2998928995046,2cyclictest3014723-21kworker/2:120:24:252
2998928995046,2cyclictest1135395-1kworker/2:1H00:30:132
2998928995044,3cyclictest1135395-1kworker/2:1H22:50:012
2998927995035,14cyclictest3102229-21kworker/1:122:55:161
2998926995047,2cyclictest3002370-21kworker/0:219:45:350
2998926995047,2cyclictest2977383-21kworker/0:019:17:020
2998926995046,2cyclictest3136606-21kworker/0:223:45:410
2998926995045,3cyclictest3081806-21kworker/0:222:18:210
2998930994946,2cyclictest3030975-21kworker/3:223:31:283
2998930994946,2cyclictest3030975-21kworker/3:222:35:543
2998928994945,3cyclictest3152954-21kworker/2:000:17:542
2998928994945,2cyclictest3071708-21kworker/2:022:29:182
2998926994946,2cyclictest3081806-21kworker/0:222:30:350
2998926994946,2cyclictest3043121-21kworker/0:121:21:260
2998926994946,2cyclictest3008534-21kworker/0:319:43:190
2998926994943,3cyclictest1705941-1kworker/0:2H19:20:130
2998926994943,3cyclictest1705941-1kworker/0:2H19:20:120
2998930994846,1cyclictest2062265-1kworker/3:1H22:00:153
2998930994844,2cyclictest2062265-1kworker/3:1H20:50:143
2998928994844,3cyclictest1135395-1kworker/2:1H19:51:542
2998928994843,3cyclictest2994642-21kworker/2:019:11:072
2998928994840,5cyclictest1135395-1kworker/2:1H20:07:052
2998926994845,2cyclictest3167134-21kworker/0:300:03:240
2998926994845,2cyclictest3112088-21kworker/0:023:15:460
2998926994842,3cyclictest1705941-1kworker/0:2H00:10:140
2998930994742,3cyclictest2062265-1kworker/3:1H23:49:223
2998928994744,2cyclictest1135395-1kworker/2:1H23:23:552
2998928994743,3cyclictest3071708-21kworker/2:021:59:542
2998928994742,3cyclictest3071708-21kworker/2:022:07:302
2998928994742,3cyclictest1135395-1kworker/2:1H19:45:112
2998926994744,2cyclictest3120231-21kworker/0:323:21:540
2998926994744,2cyclictest3014714-21kworker/0:120:17:340
2998926994744,2cyclictest1705941-1kworker/0:2H23:51:100
2998926994744,2cyclictest1705941-1kworker/0:2H21:09:550
2998926994743,3cyclictest1705941-1kworker/0:2H23:03:050
2998930994642,3cyclictest2062265-1kworker/3:1H23:17:063
2998930994641,3cyclictest2062265-1kworker/3:1H23:40:323
2998928994643,2cyclictest3027395-21kworker/2:121:02:452
2998928994642,2cyclictest2997433-21kworker/2:219:41:172
2998928994641,3cyclictest1135395-1kworker/2:1H19:40:002
2998926994643,2cyclictest3120231-21kworker/0:323:10:400
2998926994643,2cyclictest3081806-21kworker/0:222:36:030
2998926994643,2cyclictest1705941-1kworker/0:2H22:55:010
2998926994641,3cyclictest1705941-1kworker/0:2H23:35:130
2998930994542,2cyclictest2940607-21kworker/3:119:28:553
2998930994541,3cyclictest2062265-1kworker/3:1H23:20:273
2998930994541,3cyclictest2062265-1kworker/3:1H22:33:003
2998930994540,3cyclictest2062265-1kworker/3:1H22:17:303
2998930994536,5cyclictest2062265-1kworker/3:1H22:22:253
2998928994542,2cyclictest3071708-21kworker/2:023:19:182
2998928994542,2cyclictest3027395-21kworker/2:120:50:262
2998928994542,2cyclictest3021223-21kworker/2:220:28:192
2998928994542,2cyclictest1135395-1kworker/2:1H00:38:242
2998928994541,3cyclictest3152954-21kworker/2:000:01:192
2998928994541,3cyclictest1135395-1kworker/2:1H23:32:532
2998928994540,3cyclictest1135395-1kworker/2:1H23:46:542
2998928994540,3cyclictest1135395-1kworker/2:1H19:17:282
2998926994542,2cyclictest3043121-21kworker/0:121:52:550
2998926994542,2cyclictest3043121-21kworker/0:121:34:000
2998926994542,2cyclictest3035036-21kworker/0:321:38:540
2998926994542,2cyclictest1705941-1kworker/0:2H19:54:530
2998926994542,2cyclictest1705941-1kworker/0:2H19:30:470
2998926994540,3cyclictest1705941-1kworker/0:2H21:17:530
2998930994441,2cyclictest2062265-1kworker/3:1H19:14:543
2998930994440,3cyclictest2062265-1kworker/3:1H19:22:193
2998930994440,3cyclictest2062265-1kworker/3:1H19:22:183
2998930994438,4cyclictest2062265-1kworker/3:1H21:40:353
2998928994441,2cyclictest3071708-21kworker/2:023:05:532
2998928994441,2cyclictest3027395-21kworker/2:121:05:592
2998928994441,2cyclictest1135395-1kworker/2:1H20:43:542
2998928994439,3cyclictest2997433-21kworker/2:219:25:002
2998928994439,3cyclictest2997433-21kworker/2:219:24:592
2998926994442,1cyclictest3167134-21kworker/0:300:20:020
2998926994440,2cyclictest3175406-21kworker/0:200:19:250
2998930994340,2cyclictest2062265-1kworker/3:1H21:02:543
2998930994338,4cyclictest2062265-1kworker/3:1H00:35:443
2998930994338,3cyclictest2062265-1kworker/3:1H23:50:393
2998930994338,3cyclictest2062265-1kworker/3:1H00:04:543
2998930994338,2cyclictest2062265-1kworker/3:1H19:55:163
2998928994340,2cyclictest2997433-21kworker/2:219:33:202
2998928994340,2cyclictest1135395-1kworker/2:1H21:20:012
2998928994339,3cyclictest1135395-1kworker/2:1H20:35:292
2998928994339,2cyclictest3027395-21kworker/2:120:47:192
2998928994338,3cyclictest1135395-1kworker/2:1H20:30:062
2998926994340,2cyclictest3043121-21kworker/0:122:01:270
2998926994340,2cyclictest1705941-1kworker/0:2H19:35:140
2998930994238,3cyclictest2062265-1kworker/3:1H23:10:193
2998930994238,2cyclictest2062265-1kworker/3:1H21:35:003
2998930994236,3cyclictest2062265-1kworker/3:1H20:12:053
2998928994239,2cyclictest3064036-21kworker/2:121:51:202
2998928994239,2cyclictest1135395-1kworker/2:1H21:14:542
2998928994236,3cyclictest1135395-1kworker/2:1H22:32:252
2998926994239,2cyclictest3136606-21kworker/0:223:44:540
2998930994138,2cyclictest2062265-1kworker/3:1H23:25:223
2998930994138,2cyclictest2062265-1kworker/3:1H21:45:503
2998930994138,2cyclictest2062265-1kworker/3:1H20:30:283
2998930994138,2cyclictest2062265-1kworker/3:1H20:15:263
2998930994136,4cyclictest2062265-1kworker/3:1H21:53:553
2998930994136,3cyclictest2062265-1kworker/3:1H23:00:143
2998930994136,3cyclictest2062265-1kworker/3:1H22:14:543
2998930994136,3cyclictest2062265-1kworker/3:1H20:40:133
2998928994138,2cyclictest3014723-21kworker/2:120:10:122
2998928994138,2cyclictest1135395-1kworker/2:1H20:00:262
2998928994137,3cyclictest1135395-1kworker/2:1H22:37:532
2998928994137,3cyclictest1135395-1kworker/2:1H21:32:242
2998928994137,3cyclictest1135395-1kworker/2:1H00:20:102
2998928994137,2cyclictest3027395-21kworker/2:120:58:192
2998928994136,3cyclictest2997433-21kworker/2:219:57:342
2998928994136,3cyclictest1135395-1kworker/2:1H22:57:352
2998928994136,3cyclictest1135395-1kworker/2:1H22:11:052
2998928994136,3cyclictest1135395-1kworker/2:1H00:10:062
2998926994139,1cyclictest3014714-21kworker/0:120:22:410
2998926994138,2cyclictest3158574-21kworker/0:100:07:000
2998926994138,2cyclictest3022701-21kworker/0:020:45:530
2998926994138,2cyclictest3014714-21kworker/0:120:00:540
2998930994036,3cyclictest3030975-21kworker/3:221:11:173
2998930994035,3cyclictest2062265-1kworker/3:1H22:44:243
2998928994036,3cyclictest1135395-1kworker/2:1H23:50:052
2998928994035,3cyclictest3048330-21kworker/2:221:40:012
2998928994035,3cyclictest1135395-1kworker/2:1H23:10:152
2998926994037,2cyclictest3014714-21kworker/0:120:09:240
2998926994036,3cyclictest1705941-1kworker/0:2H23:59:100
2998926994036,3cyclictest1705941-1kworker/0:2H23:59:090
2998926994036,3cyclictest1705941-1kworker/0:2H22:10:460
2998926994036,2cyclictest3112088-21kworker/0:022:55:540
2998926994035,3cyclictest1705941-1kworker/0:2H22:22:250
2998926994035,3cyclictest1705941-1kworker/0:2H21:25:130
2998926994035,3cyclictest1705941-1kworker/0:2H19:55:150
2998926994034,3cyclictest1705941-1kworker/0:2H22:45:140
2998930993936,2cyclictest3168608-21kworker/3:100:13:543
2998930993936,2cyclictest2062265-1kworker/3:1H20:39:193
2998930993936,2cyclictest2062265-1kworker/3:1H20:25:543
2998930993936,2cyclictest2062265-1kworker/3:1H20:04:193
2998930993935,3cyclictest2062265-1kworker/3:1H22:26:213
2998930993934,3cyclictest2062265-1kworker/3:1H21:25:013
2998930993934,3cyclictest2062265-1kworker/3:1H00:25:023
2998930993933,3cyclictest2062265-1kworker/3:1H23:05:133
2998928993936,2cyclictest3027395-21kworker/2:121:20:332
2998928993936,2cyclictest1135395-1kworker/2:1H22:42:392
2998928993936,2cyclictest1135395-1kworker/2:1H00:05:262
2998926993936,2cyclictest3002370-21kworker/0:220:33:540
2998926993936,2cyclictest1705941-1kworker/0:2H20:10:200
2998926993934,3cyclictest1705941-1kworker/0:2H20:40:000
2998930993835,2cyclictest2062265-1kworker/3:1H20:20:543
2998930993835,2cyclictest2062265-1kworker/3:1H20:05:303
2998930993835,2cyclictest2062265-1kworker/3:1H19:49:193
2998930993834,3cyclictest2062265-1kworker/3:1H21:55:133
2998928993834,3cyclictest1135395-1kworker/2:1H23:58:012
2998928993834,3cyclictest1135395-1kworker/2:1H23:58:012
2998928993834,2cyclictest1135395-1kworker/2:1H00:29:482
299892799381,21cyclictest2998942-21latency_hist19:10:001
2998926993835,2cyclictest3136606-21kworker/0:223:34:540
2998926993833,3cyclictest3081806-21kworker/0:222:10:000
2998930993734,2cyclictest3168608-21kworker/3:100:20:413
2998928993734,2cyclictest3071708-21kworker/2:022:15:412
2998928993734,2cyclictest1135395-1kworker/2:1H20:17:532
2998928993733,3cyclictest1135395-1kworker/2:1H21:49:232
2998926993734,2cyclictest3175406-21kworker/0:200:32:430
2998926993734,2cyclictest3043121-21kworker/0:122:00:000
2998926993734,2cyclictest1705941-1kworker/0:2H21:14:240
2998930993632,3cyclictest2062265-1kworker/3:1H23:39:543
2998928993633,2cyclictest1135395-1kworker/2:1H21:40:222
2998930993532,2cyclictest3030975-21kworker/3:222:05:293
2998928993530,3cyclictest1135395-1kworker/2:1H21:28:542
2998926993532,2cyclictest1705941-1kworker/0:2H23:07:530
2998926993532,2cyclictest1705941-1kworker/0:2H22:43:520
2998926993531,3cyclictest1705941-1kworker/0:2H23:28:370
2998930993431,2cyclictest3030975-21kworker/3:222:55:353
2998926993430,3cyclictest1705941-1kworker/0:2H00:28:250
2998926993430,2cyclictest1705941-1kworker/0:2H00:35:120
2998930993330,2cyclictest2062265-1kworker/3:1H20:49:553
2998928993330,2cyclictest1135395-1kworker/2:1H23:40:212
2998928993330,2cyclictest1135395-1kworker/2:1H23:27:242
2998930993229,2cyclictest2062265-1kworker/3:1H00:09:553
2998930993227,3cyclictest2062265-1kworker/3:1H23:55:143
2998930993227,3cyclictest2062265-1kworker/3:1H23:55:143
2998926993229,2cyclictest1705941-1kworker/0:2H21:41:590
2998930993028,1cyclictest3168608-21kworker/3:100:18:293
2998930993028,1cyclictest2062265-1kworker/3:1H19:19:543
2998928993027,2cyclictest1135395-1kworker/2:1H23:35:072
2998928993027,2cyclictest1135395-1kworker/2:1H22:21:252
299893099290,2cyclictest0-21swapper/322:45:243
299893099271,23cyclictest3006160-21hwlatdetect19:30:133
2998926992718,6cyclictest3031894-21grep20:50:230
2998928992622,3cyclictest1135395-1kworker/2:1H23:04:032
299893099250,23cyclictest3117928-21munin-run22:55:003
299892699251,21cyclictest0-21swapper/022:25:120
299893099241,21cyclictest3032510-21/usr/sbin/munin20:55:143
299893099240,22cyclictest3057617-21/usr/sbin/munin21:35:233
2998926992419,3cyclictest0-21swapper/020:40:170
299892699241,21cyclictest0-21swapper/019:25:100
299892899160,13cyclictest2999020-21latency_hist19:10:012
2998926991511,2cyclictest0-21swapper/019:10:000
299893099110,2cyclictest151rcu_preempt19:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional