You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 11:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3.osadl.org (updated Tue Jul 15, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4141147996663,2cyclictest0-21swapper/100:17:091
4141147996562,2cyclictest0-21swapper/120:21:141
4141147996541,22cyclictest1967474-1kworker/1:0H23:00:001
414114799650,63cyclictest0-21swapper/121:11:421
4141147996242,18cyclictest19681-21kworker/1:222:45:151
4141147995943,15cyclictest4287-21kworker/1:322:03:091
4141147995943,15cyclictest4189323-21kworker/1:221:35:141
4141147995941,17cyclictest4141242-21kworker/1:119:30:171
4141147995843,14cyclictest6453-21kworker/1:022:25:001
4141147995843,14cyclictest6453-21kworker/1:022:08:441
4141147995843,14cyclictest49638-21kworker/1:100:30:261
4141147995843,14cyclictest4171037-21kworker/1:220:45:181
4141147995843,14cyclictest39552-21kworker/1:223:50:161
4141147995843,14cyclictest39552-21kworker/1:200:10:141
4141147995843,14cyclictest31791-21kworker/1:023:42:381
4141147995843,14cyclictest19681-21kworker/1:223:09:131
4141147995842,15cyclictest4187702-21kworker/1:021:30:131
4141147995842,15cyclictest4174464-21kworker/1:021:02:151
4141147995842,15cyclictest19681-21kworker/1:223:01:041
4141147995842,15cyclictest1967474-1kworker/1:0H20:10:141
4141147995842,15cyclictest1967474-1kworker/1:0H19:35:141
4141147995842,15cyclictest12848-21kworker/1:122:35:171
4141147995841,16cyclictest56034-21kworker/1:200:35:131
4141147995841,16cyclictest29807-21kworker/1:123:15:161
4141147995841,16cyclictest19681-21kworker/1:223:20:161
4141147995840,17cyclictest49638-21kworker/1:100:20:151
4141147995840,17cyclictest31791-21kworker/1:023:35:151
4141147995742,14cyclictest6453-21kworker/1:022:15:291
4141147995742,14cyclictest4151327-21kworker/1:019:46:491
4141147995742,14cyclictest4141242-21kworker/1:119:25:161
4141147995742,14cyclictest1967474-1kworker/1:0H23:59:421
4141147995742,14cyclictest1967474-1kworker/1:0H21:17:131
4141147995742,14cyclictest1967474-1kworker/1:0H21:17:121
4141147995742,14cyclictest17513-21kworker/1:022:44:481
4141147995741,15cyclictest4159428-21kworker/1:320:05:141
4141147995741,15cyclictest4154496-21kworker/1:119:50:181
4141147995741,15cyclictest39552-21kworker/1:223:45:291
4141147995741,15cyclictest1967474-1kworker/1:0H00:07:491
4141147995741,14cyclictest1967474-1kworker/1:0H22:50:151
4141147995740,16cyclictest4164473-21kworker/1:120:25:151
4141147995740,16cyclictest4137074-21kworker/1:219:10:021
4141147995740,14cyclictest4141242-21kworker/1:119:15:071
414114799570,56cyclictest0-21swapper/121:57:241
4141147995641,14cyclictest49638-21kworker/1:100:25:131
4141147995641,14cyclictest4191308-21kworker/1:121:54:421
4141147995641,14cyclictest4191308-21kworker/1:121:45:051
4141147995641,14cyclictest4179846-21kworker/1:121:25:191
4141147995641,14cyclictest4179846-21kworker/1:121:21:431
4141147995641,14cyclictest4179846-21kworker/1:121:05:091
4141147995641,14cyclictest4174464-21kworker/1:020:57:151
4141147995641,14cyclictest4164473-21kworker/1:120:32:191
4141147995641,14cyclictest31791-21kworker/1:023:25:021
4141147995641,14cyclictest19681-21kworker/1:223:10:251
4141147995641,14cyclictest1967474-1kworker/1:0H20:00:051
4141147995640,15cyclictest6453-21kworker/1:022:13:001
4141147995640,15cyclictest4174464-21kworker/1:020:50:421
4141147995640,15cyclictest4164473-21kworker/1:120:37:201
4141147995640,15cyclictest4154496-21kworker/1:119:55:141
4141147995540,14cyclictest6453-21kworker/1:022:20:161
4141147995540,14cyclictest4159428-21kworker/1:320:15:021
4141147995540,14cyclictest4151327-21kworker/1:019:40:161
4141147995540,14cyclictest12848-21kworker/1:122:30:151
4141147995439,14cyclictest4191308-21kworker/1:121:40:301
4141147995439,14cyclictest4171037-21kworker/1:220:40:181
4141147995439,14cyclictest31791-21kworker/1:023:30:131
4141146995449,3cyclictest3945724-1kworker/0:2H21:04:080
4141148995348,3cyclictest1501592-1kworker/2:2H00:31:122
4141147995336,16cyclictest4141242-21kworker/1:119:20:161
4141152995247,3cyclictest2672433-1kworker/3:2H22:36:423
4141148995248,2cyclictest1501592-1kworker/2:2H21:40:062
4141148995247,3cyclictest4179644-21kworker/2:022:57:132
4141148995247,3cyclictest4179644-21kworker/2:021:36:432
4141148995247,3cyclictest1501592-1kworker/2:2H00:27:082
4141147995237,14cyclictest39552-21kworker/1:200:00:381
4141148995148,2cyclictest4179644-21kworker/2:022:32:332
4141148995147,3cyclictest4140950-21kworker/2:119:18:442
4141148995147,2cyclictest4176081-21kworker/2:121:05:482
4141148995147,2cyclictest40909-21kworker/2:223:54:432
4141148995147,2cyclictest40909-21kworker/2:200:18:132
4141148995147,2cyclictest1501592-1kworker/2:2H19:20:072
4141146995147,3cyclictest3945724-1kworker/0:2H21:47:280
4141152995045,3cyclictest2672433-1kworker/3:2H20:44:183
4141152995044,4cyclictest2672433-1kworker/3:2H20:49:133
4141152995042,4cyclictest2672433-1kworker/3:2H19:47:073
4141148995046,2cyclictest4179644-21kworker/2:023:44:132
4141148995046,2cyclictest4179644-21kworker/2:023:26:132
4141148995046,2cyclictest4167728-21kworker/2:020:52:422
4141148995046,2cyclictest4149691-21kworker/2:220:05:562
4141148995046,2cyclictest4149691-21kworker/2:219:55:222
4141148995045,4cyclictest1501592-1kworker/2:2H22:35:262
4141148995045,3cyclictest1501592-1kworker/2:2H20:55:282
4141146995047,2cyclictest3945724-1kworker/0:2H23:35:280
4141146995045,3cyclictest3945724-1kworker/0:2H23:59:070
4141148994946,2cyclictest4167728-21kworker/2:020:50:002
4141148994945,3cyclictest4149691-21kworker/2:220:24:132
4141152994842,4cyclictest5859-21kworker/3:022:10:133
4141152994841,4cyclictest2672433-1kworker/3:2H21:05:003
4141148994845,2cyclictest4167728-21kworker/2:020:35:252
4141148994845,2cyclictest1501592-1kworker/2:2H23:35:272
4141148994845,2cyclictest1501592-1kworker/2:2H23:00:162
4141148994844,3cyclictest1501592-1kworker/2:2H23:33:022
4141148994844,3cyclictest1501592-1kworker/2:2H22:51:412
4141148994844,3cyclictest1501592-1kworker/2:2H21:11:422
4141148994843,3cyclictest1501592-1kworker/2:2H22:44:432
4141148994843,3cyclictest1501592-1kworker/2:2H19:40:272
4141148994837,7cyclictest1501592-1kworker/2:2H20:10:252
4141146994844,3cyclictest3945724-1kworker/0:2H19:26:260
4141152994742,3cyclictest2672433-1kworker/3:2H20:55:333
4141152994741,3cyclictest2672433-1kworker/3:2H19:20:173
4141152994739,4cyclictest2672433-1kworker/3:2H23:15:163
4141148994744,2cyclictest1501592-1kworker/2:2H21:47:422
4141148994743,3cyclictest4179644-21kworker/2:023:08:422
4141146994744,2cyclictest3945724-1kworker/0:2H19:15:420
4141146994744,2cyclictest37668-21kworker/0:023:50:140
4141146994743,2cyclictest45852-21kworker/0:300:26:420
4141152994641,3cyclictest2672433-1kworker/3:2H23:30:133
4141152994639,4cyclictest2672433-1kworker/3:2H00:30:153
4141148994642,3cyclictest1501592-1kworker/2:2H19:31:092
4141148994642,2cyclictest1501592-1kworker/2:2H22:10:142
4141148994636,8cyclictest1501592-1kworker/2:2H21:27:432
4141152994541,3cyclictest2672433-1kworker/3:2H19:14:133
4141152994535,7cyclictest2672433-1kworker/3:2H20:30:153
4141148994541,3cyclictest1501592-1kworker/2:2H00:07:442
4141148994541,2cyclictest1501592-1kworker/2:2H21:20:482
4141148994538,5cyclictest1501592-1kworker/2:2H00:13:482
4141148994536,2cyclictest1501592-1kworker/2:2H20:05:012
4141146994540,3cyclictest3945724-1kworker/0:2H23:47:370
4141152994438,4cyclictest2672433-1kworker/3:2H23:43:483
4141152994438,3cyclictest2672433-1kworker/3:2H22:57:273
4141152994435,5cyclictest2672433-1kworker/3:2H20:50:183
4141148994441,2cyclictest1501592-1kworker/2:2H22:45:112
4141148994440,3cyclictest4179644-21kworker/2:022:17:422
4141146994441,2cyclictest3945724-1kworker/0:2H00:13:080
4141146994438,4cyclictest3945724-1kworker/0:2H00:07:420
414115299436,9cyclictest18029-21diskstats22:40:143
4141152994338,3cyclictest5859-21kworker/3:022:47:383
4141152994338,3cyclictest2672433-1kworker/3:2H21:50:143
4141152994338,3cyclictest2672433-1kworker/3:2H21:39:423
4141152994338,3cyclictest2672433-1kworker/3:2H19:41:423
4141152994338,3cyclictest2672433-1kworker/3:2H19:25:423
4141152994336,5cyclictest2672433-1kworker/3:2H23:50:003
4141148994338,3cyclictest1501592-1kworker/2:2H19:35:402
4141146994339,3cyclictest4140953-21kworker/0:119:14:080
4141146994338,3cyclictest3945724-1kworker/0:2H23:30:150
4141146994338,3cyclictest3945724-1kworker/0:2H23:20:160
4141152994236,3cyclictest2672433-1kworker/3:2H19:50:183
4141152994235,4cyclictest2672433-1kworker/3:2H19:37:063
4141148994238,3cyclictest1501592-1kworker/2:2H00:20:052
4141148994238,2cyclictest1501592-1kworker/2:2H23:59:002
4141146994239,2cyclictest4160988-21kworker/0:120:16:450
4141146994238,2cyclictest4169344-21kworker/0:120:40:130
4141146994237,3cyclictest3945724-1kworker/0:2H22:05:060
4141146994237,3cyclictest3945724-1kworker/0:2H19:45:000
4141146994237,3cyclictest3945724-1kworker/0:2H00:22:440
4141146994236,4cyclictest4159368-21kworker/0:020:30:000
4141152994137,3cyclictest2672433-1kworker/3:2H00:35:083
4141152994136,3cyclictest2672433-1kworker/3:2H22:17:133
4141152994136,3cyclictest2672433-1kworker/3:2H21:08:433
4141152994136,2cyclictest2672433-1kworker/3:2H00:05:143
4141152994133,4cyclictest2672433-1kworker/3:2H21:12:173
4141148994138,2cyclictest4179644-21kworker/2:021:57:232
4141148994138,2cyclictest1501592-1kworker/2:2H19:11:122
4141148994137,3cyclictest1501592-1kworker/2:2H22:21:072
4141148994137,2cyclictest1501592-1kworker/2:2H23:45:142
4141148994137,2cyclictest1501592-1kworker/2:2H22:10:012
4141148994136,4cyclictest1501592-1kworker/2:2H21:03:112
4141148994136,3cyclictest1501592-1kworker/2:2H23:10:142
4141146994136,3cyclictest3945724-1kworker/0:2H22:37:580
4141146994136,3cyclictest3945724-1kworker/0:2H22:33:420
4141146994136,3cyclictest3945724-1kworker/0:2H22:22:260
4141146994136,3cyclictest23072-21kworker/0:123:30:000
4141146994135,4cyclictest3945724-1kworker/0:2H21:26:010
4141152994036,2cyclictest4184455-21kworker/3:121:55:063
4141148994037,2cyclictest1501592-1kworker/2:2H22:26:422
4141148994037,2cyclictest1501592-1kworker/2:2H20:30:202
4141148994037,2cyclictest1501592-1kworker/2:2H00:35:282
4141148994037,2cyclictest1501592-1kworker/2:2H00:00:092
4141148994035,3cyclictest1501592-1kworker/2:2H21:30:332
4141148994035,3cyclictest1501592-1kworker/2:2H21:15:342
4141148994035,3cyclictest1501592-1kworker/2:2H21:15:342
4141146994037,2cyclictest4186093-21kworker/0:021:40:130
4141146994037,2cyclictest3945724-1kworker/0:2H21:05:230
4141146994037,2cyclictest3945724-1kworker/0:2H20:00:330
4141146994037,2cyclictest3945724-1kworker/0:2H19:58:080
4141146994036,2cyclictest4140953-21kworker/0:119:31:480
4141152993934,3cyclictest2672433-1kworker/3:2H23:26:313
4141152993934,3cyclictest2672433-1kworker/3:2H22:05:433
4141148993934,3cyclictest4179644-21kworker/2:023:20:422
4141148993934,3cyclictest1501592-1kworker/2:2H19:26:072
4141146993936,2cyclictest3945724-1kworker/0:2H20:30:330
4141146993936,2cyclictest3945724-1kworker/0:2H19:45:260
4141146993935,3cyclictest4152936-21kworker/0:319:51:540
4141152993835,2cyclictest2672433-1kworker/3:2H21:20:183
4141152993834,2cyclictest2672433-1kworker/3:2H21:27:183
4141152993834,2cyclictest2672433-1kworker/3:2H20:05:143
4141148993835,2cyclictest1501592-1kworker/2:2H22:02:102
4141146993835,2cyclictest4152936-21kworker/0:320:05:140
4141146993835,2cyclictest3945724-1kworker/0:2H22:55:100
4141146993833,3cyclictest4159368-21kworker/0:020:10:430
4141146993833,3cyclictest3945724-1kworker/0:2H22:15:080
4141146993832,4cyclictest4190931-21kworker/0:221:40:000
4141146993831,4cyclictest3945724-1kworker/0:2H23:07:270
4141152993735,1cyclictest4156115-21kworker/3:119:59:423
4141152993733,2cyclictest2672433-1kworker/3:2H23:09:423
4141152993732,3cyclictest5859-21kworker/3:023:10:433
4141152993732,3cyclictest2672433-1kworker/3:2H00:03:433
4141148993734,2cyclictest1501592-1kworker/2:2H19:50:182
4141146993734,2cyclictest4164551-21kworker/0:220:20:430
4141152993633,2cyclictest2672433-1kworker/3:2H22:34:563
4141152993631,3cyclictest2672433-1kworker/3:2H00:12:473
4141148993633,2cyclictest1501592-1kworker/2:2H23:15:282
4141148993631,3cyclictest1501592-1kworker/2:2H20:17:082
414114899362,2cyclictest151rcu_preempt21:50:152
4141146993633,2cyclictest10950-21kworker/0:022:45:000
4141146993632,3cyclictest3945724-1kworker/0:2H20:47:450
4141146993631,3cyclictest3945724-1kworker/0:2H21:10:080
4141146993631,3cyclictest3945724-1kworker/0:2H00:33:190
4141146993631,3cyclictest23072-21kworker/0:123:15:140
4141152993532,2cyclictest4184455-21kworker/3:121:33:423
4141152993532,2cyclictest2672433-1kworker/3:2H23:04:553
4141152993532,2cyclictest2672433-1kworker/3:2H20:03:433
4141152993529,3cyclictest4164509-21sh20:20:003
4141152993527,5cyclictest4191460-21/usr/sbin/munin21:40:143
4141146993532,2cyclictest3945724-1kworker/0:2H22:02:340
4141146993531,3cyclictest3945724-1kworker/0:2H19:20:420
4141146993531,3cyclictest3945724-1kworker/0:2H00:37:520
4141148993432,1cyclictest1501592-1kworker/2:2H20:30:002
4141146993431,2cyclictest3945724-1kworker/0:2H20:50:070
4141146993430,2cyclictest3945724-1kworker/0:2H19:36:420
4141146993429,3cyclictest3945724-1kworker/0:2H00:00:130
4141146993426,5cyclictest3945724-1kworker/0:2H22:25:180
4141152993330,2cyclictest2672433-1kworker/3:2H22:54:553
4141152993325,5cyclictest2672433-1kworker/3:2H20:25:163
4141147993330,1cyclictest0-21swapper/119:10:011
4141146993328,3cyclictest3945724-1kworker/0:2H23:10:160
4141146993327,4cyclictest3945724-1kworker/0:2H22:12:240
414115299325,13cyclictest12257-21sensors22:20:203
4141152993228,2cyclictest2672433-1kworker/3:2H23:50:163
4141152993227,3cyclictest2672433-1kworker/3:2H22:30:003
4141146993224,5cyclictest3945724-1kworker/0:2H21:15:130
4141146993224,5cyclictest3945724-1kworker/0:2H21:15:120
414115299307,9cyclictest32962-21date23:25:003
4141152993024,3cyclictest654-21snmpd23:57:133
4141152993021,6cyclictest0-21swapper/321:50:003
4141148993025,3cyclictest1501592-1kworker/2:2H19:47:122
4141146993025,3cyclictest3945724-1kworker/0:2H00:15:140
414115299296,11cyclictest4183340-21cstates21:15:113
414115299296,11cyclictest4183340-21cstates21:15:113
414115299296,11cyclictest4143616-21cstates19:15:163
414115299295,9cyclictest5352-21ntpq22:00:183
4141152992925,2cyclictest0-21swapper/300:25:183
4141152992924,3cyclictest50665-21sshd00:18:373
4141152992923,3cyclictest0-21swapper/319:35:003
4141152992921,5cyclictest4165674-21grep20:20:193
4141146992927,1cyclictest37668-21kworker/0:023:42:360
4141146992926,2cyclictest3945724-1kworker/0:2H20:55:270
414115299287,9cyclictest37317-21grep23:35:213
414115299286,10cyclictest52050-21sed00:20:203
4141152992823,3cyclictest4169758-21cpuspeed_turbos20:35:163
4141152992810,4cyclictest4162863-21sshd20:12:373
4141146992624,1cyclictest10950-21kworker/0:022:46:430
4141146992623,2cyclictest53250irq/160-enp0s2921:55:160
4141146992522,2cyclictest4192818-21kworker/0:121:54:180
414114699250,15cyclictest4184747-21apt-get21:20:130
4141148992417,2cyclictest33-21ksoftirqd/220:43:182
414114699242,14cyclictest21229-21apt-get22:50:120
4141146992412,10cyclictest4164551-21kworker/0:220:35:150
4141152992316,4cyclictest4141205-21latency_hist19:10:013
4141146992320,2cyclictest3945724-1kworker/0:2H21:30:090
414114699220,20cyclictest25961-21sendmail_mailtr23:00:190
414114899140,3cyclictest151rcu_preempt19:10:012
414114699104,3cyclictest171rcuc/019:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional