You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-21 - 07:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3.osadl.org (updated Sun Apr 21, 2024 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4111800997059,1cyclictest0-21swapper/122:04:551
4111800997042,16cyclictest4175728-21kworker/1:022:05:071
4111800996638,27cyclictest106334-21kworker/1:100:30:001
4111800996461,2cyclictest0-21swapper/119:19:231
4111800996446,17cyclictest96382-21kworker/1:200:10:381
411180099640,63cyclictest0-21swapper/122:42:511
411180099640,62cyclictest0-21swapper/121:56:231
4111800996361,1cyclictest0-21swapper/120:45:431
4111800996361,1cyclictest0-21swapper/120:00:381
4111800996343,18cyclictest4129367-21kworker/1:020:59:591
4111800996247,14cyclictest13436-21kworker/1:222:25:011
4111800996159,1cyclictest0-21swapper/100:00:291
4111800996141,19cyclictest42160-21kworker/1:123:15:121
411180099611,58cyclictest76457-21unin-run23:45:001
4111800996057,2cyclictest0-21swapper/119:48:591
411180099600,1cyclictest610-21rs:main1
4111800995943,15cyclictest61708-21kworker/1:223:47:221
4111800995943,15cyclictest4175728-21kworker/1:022:14:281
4111800995943,15cyclictest4169703-21kworker/1:121:40:221
4111800995943,15cyclictest4129367-21kworker/1:021:11:561
4111800995943,15cyclictest4129367-21kworker/1:021:11:561
4111800995943,15cyclictest13436-21kworker/1:222:24:561
4111800995942,16cyclictest4160477-21kworker/1:221:25:021
4111800995941,17cyclictest26110-21kworker/1:222:47:461
4111800995941,17cyclictest17525-21kworker/1:122:30:181
4111800995940,18cyclictest86752-21kworker/1:100:05:161
4111800995856,1cyclictest0-21swapper/123:23:571
4111800995842,15cyclictest61708-21kworker/1:223:52:221
4111800995842,15cyclictest4175728-21kworker/1:022:17:021
4111800995842,15cyclictest4165852-21kworker/1:321:34:281
4111800995842,15cyclictest4129367-21kworker/1:020:42:211
4111800995842,15cyclictest4129367-21kworker/1:020:35:281
4111800995842,15cyclictest4129367-21kworker/1:020:17:161
4111800995842,15cyclictest4129367-21kworker/1:020:10:001
4111800995842,15cyclictest4111595-21kworker/1:219:35:001
4111800995842,15cyclictest4111595-21kworker/1:219:27:071
4111800995841,16cyclictest61708-21kworker/1:223:30:281
4111800995841,16cyclictest4129367-21kworker/1:020:50:301
4111800995841,16cyclictest1211310-1kworker/1:2H23:00:001
4111800995840,17cyclictest4129367-21kworker/1:021:07:581
4111800995838,19cyclictest1211310-1kworker/1:2H20:25:231
4111800995741,15cyclictest42160-21kworker/1:123:10:511
4111800995740,16cyclictest42160-21kworker/1:123:03:571
4111800995740,16cyclictest4122953-21kworker/1:119:54:591
4111800995740,16cyclictest4111595-21kworker/1:219:40:391
4111800995740,15cyclictest1211310-1kworker/1:2H21:15:151
4111800995739,17cyclictest96382-21kworker/1:200:20:131
4111800995739,17cyclictest96382-21kworker/1:200:20:131
4111804995642,11cyclictest3392978-1kworker/3:1H19:20:003
4111800995640,15cyclictest86752-21kworker/1:123:59:591
4111800995640,15cyclictest4175728-21kworker/1:021:50:011
4111800995640,15cyclictest4111595-21kworker/1:219:39:571
4111800995640,15cyclictest17525-21kworker/1:122:38:201
4111800995640,15cyclictest1211310-1kworker/1:2H00:25:231
4111800995640,15cyclictest1211310-1kworker/1:2H00:25:221
4111800995639,16cyclictest4111595-21kworker/1:219:25:001
4111800995638,17cyclictest4175728-21kworker/1:021:49:281
4111800995538,16cyclictest4129367-21kworker/1:020:10:011
4111800995538,16cyclictest4111595-21kworker/1:219:12:581
4111800995535,18cyclictest4169703-21kworker/1:121:35:141
411180099551,52cyclictest4134515-21/usr/sbin/munin20:20:221
4111800995438,15cyclictest96382-21kworker/1:200:15:141
4111800995438,15cyclictest61708-21kworker/1:223:35:161
4111800995438,15cyclictest42160-21kworker/1:123:05:011
4111800995438,15cyclictest42160-21kworker/1:123:05:011
4111800995438,15cyclictest4155022-21kworker/1:121:20:171
4111800995438,15cyclictest4129367-21kworker/1:020:32:221
4111798995449,3cyclictest1093705-1kworker/0:1H21:35:370
4111800995349,2cyclictest4150021-21diskmemload00:36:021
4111800995337,15cyclictest4126327-21kworker/1:219:55:161
4111800995336,16cyclictest61708-21kworker/1:223:25:221
4111800995336,16cyclictest4129367-21kworker/1:021:01:571
4111804995248,2cyclictest4116844-21kworker/3:020:55:393
4111803995249,2cyclictest4118328-21kworker/2:121:50:572
4111803995247,3cyclictest56642-21kworker/2:223:35:332
4111804995147,2cyclictest3392978-1kworker/3:1H00:10:273
4111803995147,2cyclictest4118328-21kworker/2:121:00:402
4111803995146,3cyclictest56642-21kworker/2:223:55:232
4111803995146,3cyclictest56642-21kworker/2:200:17:272
4111804995040,7cyclictest3392978-1kworker/3:1H00:10:003
4111804995038,9cyclictest3392978-1kworker/3:1H22:45:123
4111803995045,3cyclictest1074229-1kworker/2:2H22:00:162
4111803995045,2cyclictest1074229-1kworker/2:2H22:25:152
4111798995045,3cyclictest1093705-1kworker/0:1H00:26:500
4111798995045,3cyclictest1093705-1kworker/0:1H00:26:500
4111804994938,9cyclictest3392978-1kworker/3:1H19:57:143
4111803994946,2cyclictest4118328-21kworker/2:122:40:482
4111798994944,3cyclictest1093705-1kworker/0:1H21:17:340
4111798994944,3cyclictest1093705-1kworker/0:1H20:49:280
4111804994844,2cyclictest41442-21kworker/3:223:40:113
4111804994837,8cyclictest3392978-1kworker/3:1H00:25:153
4111804994837,8cyclictest3392978-1kworker/3:1H00:25:153
4111803994845,2cyclictest56642-21kworker/2:200:34:152
4111803994844,2cyclictest4118328-21kworker/2:120:56:572
4111803994843,3cyclictest1074229-1kworker/2:2H23:45:272
4111798994844,3cyclictest4119858-21kworker/0:119:35:320
4111798994844,3cyclictest1093705-1kworker/0:1H22:52:570
4111798994843,4cyclictest1093705-1kworker/0:1H00:04:320
4111798994842,4cyclictest1093705-1kworker/0:1H22:57:280
4111804994744,2cyclictest4193097-21kworker/3:222:20:573
4111804994744,2cyclictest4116844-21kworker/3:021:50:443
4111804994744,2cyclictest3392978-1kworker/3:1H19:40:043
4111804994743,3cyclictest4116844-21kworker/3:021:30:223
4111804994743,2cyclictest4116844-21kworker/3:021:39:553
4111804994737,9cyclictest3392978-1kworker/3:1H00:24:563
4111804994737,9cyclictest3392978-1kworker/3:1H00:24:553
4111803994744,2cyclictest4118328-21kworker/2:123:00:042
4111803994744,2cyclictest4118328-21kworker/2:121:45:152
4111803994743,3cyclictest4118328-21kworker/2:123:17:572
4111798994742,3cyclictest75619-21kworker/0:323:46:570
4111798994742,3cyclictest1093705-1kworker/0:1H21:20:170
4111804994643,2cyclictest4116844-21kworker/3:021:55:323
4111804994643,2cyclictest4116844-21kworker/3:020:15:373
4111804994643,2cyclictest3392978-1kworker/3:1H19:28:273
4111804994643,2cyclictest2069626-21kworker/3:219:13:253
4111804994642,3cyclictest3392978-1kworker/3:1H23:39:553
4111804994640,4cyclictest3392978-1kworker/3:1H23:55:173
4111803994644,1cyclictest4118328-21kworker/2:121:35:232
4111803994642,3cyclictest1074229-1kworker/2:2H19:50:242
4111803994642,2cyclictest4118328-21kworker/2:122:35:212
4111798994643,2cyclictest75619-21kworker/0:300:11:100
4111798994642,3cyclictest4111678-21kworker/0:019:34:220
4111798994641,3cyclictest4146894-21kworker/0:121:00:120
4111798994641,3cyclictest1093705-1kworker/0:1H22:02:480
4111804994542,2cyclictest41442-21kworker/3:200:01:573
4111804994540,3cyclictest3392978-1kworker/3:1H23:14:553
4111803994542,2cyclictest56642-21kworker/2:200:07:392
4111803994542,2cyclictest4108887-21kworker/2:119:10:302
4111803994541,2cyclictest56642-21kworker/2:223:43:272
4111803994540,3cyclictest4118328-21kworker/2:123:05:212
4111803994540,3cyclictest4118328-21kworker/2:123:05:212
4111803994540,3cyclictest4118328-21kworker/2:122:45:202
4111803994540,2cyclictest1074229-1kworker/2:2H19:37:072
4111803994538,3cyclictest1074229-1kworker/2:2H00:13:022
4111798994542,2cyclictest83574-21kworker/0:123:59:320
4111798994542,2cyclictest75619-21kworker/0:323:52:570
4111798994542,2cyclictest21850-21kworker/0:322:44:220
4111798994541,3cyclictest1093705-1kworker/0:1H19:40:390
4111798994541,2cyclictest1093705-1kworker/0:1H19:57:200
4111804994441,2cyclictest4116844-21kworker/3:021:20:043
4111804994441,2cyclictest3392978-1kworker/3:1H21:06:563
4111803994441,2cyclictest4118328-21kworker/2:122:24:222
4111803994441,2cyclictest4118328-21kworker/2:122:14:272
4111803994440,3cyclictest1074229-1kworker/2:2H00:25:272
4111803994440,3cyclictest1074229-1kworker/2:2H00:25:272
4111803994439,3cyclictest1074229-1kworker/2:2H23:50:272
4111803994438,3cyclictest1074229-1kworker/2:2H20:50:132
4111803994438,3cyclictest1074229-1kworker/2:2H20:12:152
4111798994441,2cyclictest1093705-1kworker/0:1H23:26:390
4111798994440,3cyclictest1093705-1kworker/0:1H23:10:000
4111798994440,3cyclictest1093705-1kworker/0:1H23:10:000
4111798994440,3cyclictest1093705-1kworker/0:1H22:30:180
4111798994440,2cyclictest4146894-21kworker/0:121:06:080
4111798994438,4cyclictest1093705-1kworker/0:1H22:15:220
4111804994340,2cyclictest4193097-21kworker/3:222:11:033
4111804994340,2cyclictest41442-21kworker/3:223:26:233
4111804994339,2cyclictest3392978-1kworker/3:1H23:50:323
4111804994339,2cyclictest3392978-1kworker/3:1H20:45:283
4111804994338,3cyclictest3392978-1kworker/3:1H20:05:153
4111803994340,2cyclictest4118328-21kworker/2:123:11:562
4111803994340,2cyclictest4118328-21kworker/2:121:43:572
4111803994340,2cyclictest4113182-21kworker/2:019:28:572
4111803994339,3cyclictest4118328-21kworker/2:121:55:272
4111803994339,2cyclictest4118328-21kworker/2:120:20:052
4111803994339,2cyclictest4118328-21kworker/2:119:59:282
4111803994337,3cyclictest1074229-1kworker/2:2H21:17:282
4111798994340,2cyclictest1093705-1kworker/0:1H20:07:570
4111798994338,3cyclictest1093705-1kworker/0:1H20:55:160
4111804994239,2cyclictest4193097-21kworker/3:222:05:063
4111804994239,2cyclictest4116844-21kworker/3:021:44:553
4111804994239,2cyclictest4116844-21kworker/3:019:54:223
4111804994239,2cyclictest3392978-1kworker/3:1H20:39:283
4111804994238,2cyclictest41442-21kworker/3:223:15:573
4111804994238,2cyclictest3392978-1kworker/3:1H19:37:283
4111804994237,3cyclictest3392978-1kworker/3:1H20:20:153
4111804994237,3cyclictest3392978-1kworker/3:1H19:45:293
4111803994238,3cyclictest1074229-1kworker/2:2H00:35:162
4111803994238,2cyclictest1074229-1kworker/2:2H20:32:202
4111798994237,3cyclictest4139130-21kworker/0:020:55:000
4111798994237,3cyclictest1093705-1kworker/0:1H21:52:460
4111798994237,3cyclictest1093705-1kworker/0:1H20:37:260
4111804994138,2cyclictest3392978-1kworker/3:1H21:11:563
4111804994138,2cyclictest3392978-1kworker/3:1H21:11:563
4111804994138,2cyclictest3392978-1kworker/3:1H00:33:223
4111804994136,3cyclictest3392978-1kworker/3:1H20:32:183
4111803994138,2cyclictest4118328-21kworker/2:120:25:132
4111803994138,2cyclictest4113182-21kworker/2:019:21:222
4111803994138,2cyclictest4113182-21kworker/2:019:15:162
4111803994137,3cyclictest1074229-1kworker/2:2H22:50:212
4111803994135,3cyclictest1074229-1kworker/2:2H20:07:142
4111798994137,2cyclictest1093705-1kworker/0:1H21:59:130
4111798994136,3cyclictest1093705-1kworker/0:1H00:37:270
4111798994135,4cyclictest1093705-1kworker/0:1H22:15:010
4111804994038,1cyclictest4116844-21kworker/3:020:02:023
4111804994037,2cyclictest3392978-1kworker/3:1H19:22:113
4111804994036,2cyclictest4116844-21kworker/3:021:27:353
4111804994035,3cyclictest41442-21kworker/3:223:45:123
4111804994035,3cyclictest3392978-1kworker/3:1H22:01:033
4111803994036,2cyclictest4118328-21kworker/2:121:30:382
4111803994036,2cyclictest1074229-1kworker/2:2H20:15:132
4111798994037,2cyclictest1093705-1kworker/0:1H22:25:000
4111798994037,2cyclictest1093705-1kworker/0:1H21:14:270
4111798994037,2cyclictest1093705-1kworker/0:1H21:14:270
4111798994037,2cyclictest1093705-1kworker/0:1H20:29:590
4111798994036,2cyclictest1093705-1kworker/0:1H20:15:140
4111798994035,3cyclictest1093705-1kworker/0:1H22:36:320
4111804993936,2cyclictest41442-21kworker/3:223:24:223
4111804993936,2cyclictest4116844-21kworker/3:021:01:573
4111804993936,2cyclictest3392978-1kworker/3:1H22:50:153
4111804993936,2cyclictest3392978-1kworker/3:1H00:35:233
4111804993935,2cyclictest3392978-1kworker/3:1H20:52:583
4111804993934,3cyclictest3392978-1kworker/3:1H21:45:153
4111803993936,2cyclictest1074229-1kworker/2:2H20:02:562
4111798993936,2cyclictest1093705-1kworker/0:1H00:30:430
4111804993835,2cyclictest3392978-1kworker/3:1H23:09:013
4111804993835,2cyclictest3392978-1kworker/3:1H23:09:013
4111804993835,2cyclictest3392978-1kworker/3:1H20:10:403
4111804993835,2cyclictest18398-21kworker/3:122:55:053
4111804993834,2cyclictest4116844-21kworker/3:020:25:203
4111804993833,3cyclictest41442-21kworker/3:223:00:133
4111803993835,2cyclictest4118328-21kworker/2:122:58:572
4111803993835,2cyclictest4118328-21kworker/2:121:22:282
4111803993834,2cyclictest1074229-1kworker/2:2H20:50:002
4111803993834,2cyclictest1074229-1kworker/2:2H20:35:142
4111798993835,2cyclictest4159917-21kworker/0:221:48:050
4111798993835,2cyclictest1093705-1kworker/0:1H20:04:280
4111798993834,3cyclictest1093705-1kworker/0:1H23:20:120
4111798993833,4cyclictest83574-21kworker/0:100:20:000
4111804993733,3cyclictest3392978-1kworker/3:1H00:17:063
4111803993734,2cyclictest56642-21kworker/2:200:23:562
4111803993734,2cyclictest56642-21kworker/2:200:23:562
4111803993734,2cyclictest4118328-21kworker/2:122:17:392
4111803993734,2cyclictest4118328-21kworker/2:122:07:432
4111803993734,2cyclictest4118328-21kworker/2:119:44:282
4111803993732,4cyclictest56642-21kworker/2:223:30:272
4111803993732,3cyclictest1074229-1kworker/2:2H21:26:562
4111798993734,2cyclictest4134233-21kworker/0:320:30:180
4111798993734,2cyclictest1093705-1kworker/0:1H19:25:040
4111804993633,2cyclictest3392978-1kworker/3:1H22:44:553
4111804993633,2cyclictest3392978-1kworker/3:1H22:18:573
4111804993633,2cyclictest3392978-1kworker/3:1H19:30:433
4111804993633,2cyclictest18398-21kworker/3:122:35:573
4111798993630,4cyclictest1093705-1kworker/0:1H23:00:140
4111804993532,2cyclictest4193097-21kworker/3:222:29:563
4111804993532,2cyclictest3392978-1kworker/3:1H20:42:273
4111803993532,2cyclictest4118328-21kworker/2:121:07:572
4111803993532,2cyclictest1074229-1kworker/2:2H20:40:412
4111798993531,2cyclictest1093705-1kworker/0:1H21:27:300
4111804993418,4cyclictest4155442-21ls21:15:223
411180499340,2cyclictest65624-21ssh23:30:143
4111803993431,2cyclictest56642-21kworker/2:200:00:272
4111803993429,3cyclictest4118328-21kworker/2:121:12:272
4111803993429,3cyclictest4118328-21kworker/2:121:12:272
4111798993429,3cyclictest4140668-21kworker/0:120:40:210
4111798993429,3cyclictest1093705-1kworker/0:1H19:47:010
4111803993329,3cyclictest4118328-21kworker/2:119:33:572
4111798993330,2cyclictest83574-21kworker/0:100:05:220
4111798993330,2cyclictest1093705-1kworker/0:1H23:15:120
4111798993330,2cyclictest1093705-1kworker/0:1H19:11:270
4111798993329,3cyclictest4155886-21kworker/0:021:43:050
4111804993229,2cyclictest4193097-21kworker/3:222:31:063
411180399320,22cyclictest0-21swapper/222:30:152
4111798993128,2cyclictest4121398-21kworker/0:319:51:560
4111798993127,3cyclictest1093705-1kworker/0:1H22:48:530
4111798993025,3cyclictest75619-21kworker/0:300:21:010
4111798993025,3cyclictest75619-21kworker/0:300:21:010
4111798993024,3cyclictest1093705-1kworker/0:1H19:22:050
4111798992924,4cyclictest52570-21kworker/0:223:30:220
4111798992824,3cyclictest4166147-21cat21:30:100
4111803992723,3cyclictest1074229-1kworker/2:2H23:27:562
411180399270,24cyclictest61051-21systemd23:25:002
4111798992724,2cyclictest1093705-1kworker/0:1H20:24:280
411179899261,4cyclictest16028-21/usr/sbin/munin22:25:230
411179899261,22cyclictest4132710-21cron20:15:000
4111803992520,3cyclictest4118328-21kworker/2:119:48:232
4111798992522,2cyclictest40060-21kworker/0:323:13:560
411179899251,22cyclictest73462-21/usr/sbin/munin23:40:130
411179899241,21cyclictest0-21swapper/023:35:140
411179899240,22cyclictest309-21/usr/sbin/munin22:05:200
411179899240,21cyclictest4115085-21grep19:15:230
411180099231,2cyclictest0-21swapper/119:10:011
41118039991,4cyclictest4111846-21latency_hist19:10:012
41118049981,4cyclictest4111880-21cut19:10:013
41117989981,5cyclictest4111858-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional