You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 14:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3.osadl.org (updated Mon Nov 17, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
969218996936,31cyclictest1012346-21kworker/1:022:25:141
96921899690,2cyclictest0-21swapper/120:45:141
969218996460,2cyclictest0-21swapper/100:36:521
96921899630,62cyclictest0-21swapper/122:00:161
969218996259,2cyclictest0-21swapper/100:14:221
969218996246,15cyclictest1054357-21kworker/1:223:29:551
969218996244,16cyclictest218583-1kworker/1:0H20:00:171
969218996244,16cyclictest1012346-21kworker/1:021:25:161
969218996158,2cyclictest0-21swapper/123:17:061
969218996157,2cyclictest0-21swapper/123:41:541
969223996046,11cyclictest1712070-1kworker/3:2H00:15:153
969218996057,2cyclictest0-21swapper/122:52:191
969218996044,15cyclictest218583-1kworker/1:0H22:10:241
969218996044,14cyclictest218583-1kworker/1:0H00:25:151
969218996043,16cyclictest999984-21kworker/1:021:05:011
969218996043,16cyclictest1012346-21kworker/1:022:15:161
969218996041,17cyclictest954520-21kworker/1:019:40:181
969218996038,20cyclictest981873-21kworker/1:219:56:541
969218995956,2cyclictest0-21swapper/122:32:251
969218995956,2cyclictest0-21swapper/121:18:551
969218995944,14cyclictest1012346-21kworker/1:022:45:161
969218995943,15cyclictest1070240-21kworker/1:100:15:241
969218995943,15cyclictest1054357-21kworker/1:223:33:151
969218995943,15cyclictest1054357-21kworker/1:223:33:151
969218995943,15cyclictest1012346-21kworker/1:022:41:541
969218995942,16cyclictest954520-21kworker/1:019:25:151
969218995942,16cyclictest954520-21kworker/1:019:15:151
969218995942,16cyclictest1075369-21kworker/1:100:35:001
969218995942,15cyclictest218583-1kworker/1:0H19:20:121
969218995938,19cyclictest218583-1kworker/1:0H20:25:161
969218995856,1cyclictest0-21swapper/121:11:541
969218995843,14cyclictest218583-1kworker/1:0H20:19:551
969218995843,14cyclictest1012346-21kworker/1:022:07:241
969218995842,14cyclictest1012346-21kworker/1:022:00:001
969218995841,16cyclictest1056987-21kworker/1:000:20:061
969218995841,16cyclictest1056987-21kworker/1:000:05:001
969218995840,17cyclictest999984-21kworker/1:020:55:191
969218995840,17cyclictest1044558-21kworker/1:123:00:151
969218995754,2cyclictest0-21swapper/123:38:151
969218995754,2cyclictest0-21swapper/122:37:191
969218995754,2cyclictest0-21swapper/120:08:191
969218995742,14cyclictest990231-21kworker/1:120:14:171
969218995742,14cyclictest1012346-21kworker/1:021:23:301
969218995741,15cyclictest999984-21kworker/1:020:53:541
969218995741,15cyclictest218583-1kworker/1:0H00:05:071
969218995741,15cyclictest1044558-21kworker/1:123:05:261
969218995741,14cyclictest218583-1kworker/1:0H21:35:181
969218995740,16cyclictest1044558-21kworker/1:123:10:171
969220995646,8cyclictest130910-1kworker/2:1H21:20:182
969218995652,2cyclictest0-21swapper/121:49:551
969218995641,14cyclictest981873-21kworker/1:219:50:161
969218995640,15cyclictest990231-21kworker/1:120:35:171
969218995640,15cyclictest218583-1kworker/1:0H21:42:231
969218995640,15cyclictest1012346-21kworker/1:021:52:251
969218995640,15cyclictest1012346-21kworker/1:021:30:191
969218995638,17cyclictest990231-21kworker/1:120:30:151
96921899560,54cyclictest0-21swapper/121:02:541
969220995550,3cyclictest130910-1kworker/2:1H23:58:102
969218995540,14cyclictest954520-21kworker/1:019:34:241
969218995540,14cyclictest1056987-21kworker/1:023:58:571
969218995540,14cyclictest1056987-21kworker/1:023:45:201
969218995539,15cyclictest1044558-21kworker/1:123:20:151
969223995436,4cyclictest1712070-1kworker/3:2H23:50:153
969223995435,4cyclictest1712070-1kworker/3:2H23:35:153
969218995439,14cyclictest997759-21kworker/1:220:40:001
969218995439,14cyclictest1012346-21kworker/1:022:20:011
969218995438,15cyclictest954520-21kworker/1:019:11:511
969216995450,2cyclictest981279-21kworker/0:019:46:190
969223995348,3cyclictest1712070-1kworker/3:2H19:46:523
969223995348,3cyclictest1712070-1kworker/3:2H00:35:173
969220995348,3cyclictest130910-1kworker/2:1H00:40:012
969220995347,3cyclictest130910-1kworker/2:1H00:25:152
969220995346,4cyclictest130910-1kworker/2:1H00:08:212
969218995338,14cyclictest1044558-21kworker/1:122:56:001
969218995337,15cyclictest954520-21kworker/1:019:35:161
969218995336,16cyclictest990231-21kworker/1:120:20:151
969218995335,17cyclictest1056987-21kworker/1:023:51:181
969216995348,3cyclictest983143-21kworker/0:120:02:340
969223995239,10cyclictest1712070-1kworker/3:2H20:37:033
969223995238,11cyclictest1712070-1kworker/3:2H21:52:183
969218995236,15cyclictest981873-21kworker/1:219:49:541
969223995148,2cyclictest1005852-21kworker/3:221:57:243
969223995146,3cyclictest1712070-1kworker/3:2H22:01:203
969220995143,6cyclictest130910-1kworker/2:1H21:05:162
969216995137,6cyclictest3964510-1kworker/0:0H00:17:360
969223995047,2cyclictest1712070-1kworker/3:2H23:56:033
969223995044,4cyclictest1712070-1kworker/3:2H22:15:163
969216995045,3cyclictest3964510-1kworker/0:0H22:52:350
969216995045,3cyclictest1020496-21kworker/0:021:55:150
969223994945,2cyclictest1005852-21kworker/3:221:35:543
969223994944,3cyclictest1712070-1kworker/3:2H21:31:533
969220994944,3cyclictest130910-1kworker/2:1H23:20:152
969216994945,3cyclictest1030726-21kworker/0:022:25:200
969223994845,2cyclictest1061865-21kworker/3:000:00:353
969223994845,2cyclictest1035856-21kworker/3:222:30:243
969220994842,4cyclictest130910-1kworker/2:1H22:12:232
969216994845,2cyclictest1043952-21kworker/0:122:57:540
969216994843,3cyclictest3964510-1kworker/0:0H19:25:220
969216994840,4cyclictest3964510-1kworker/0:0H20:27:020
969223994743,3cyclictest1712070-1kworker/3:2H20:10:593
969223994743,2cyclictest1712070-1kworker/3:2H19:19:523
969223994742,3cyclictest1712070-1kworker/3:2H21:13:193
969223994741,4cyclictest1712070-1kworker/3:2H23:22:193
969220994743,3cyclictest1027342-21kworker/2:022:30:532
969220994729,11cyclictest1027342-21kworker/2:022:50:002
969216994743,2cyclictest988008-21kworker/0:020:30:260
969216994743,2cyclictest1066744-21kworker/0:000:28:230
969216994742,3cyclictest3964510-1kworker/0:0H21:27:540
969223994643,2cyclictest1712070-1kworker/3:2H23:28:593
969223994640,3cyclictest1712070-1kworker/3:2H21:45:163
969216994644,1cyclictest988008-21kworker/0:020:20:540
969216994644,1cyclictest1055381-21kworker/0:023:42:450
969216994643,2cyclictest3964510-1kworker/0:0H00:08:530
969216994642,3cyclictest1032641-21kworker/0:222:21:200
969216994641,3cyclictest3964510-1kworker/0:0H23:08:540
969223994541,3cyclictest997757-21kworker/3:120:45:143
969223994541,3cyclictest1712070-1kworker/3:2H21:00:293
969223994540,3cyclictest1712070-1kworker/3:2H20:51:003
969223994540,3cyclictest1712070-1kworker/3:2H19:56:173
969223994540,3cyclictest1055628-21kworker/3:123:45:183
969220994542,2cyclictest130910-1kworker/2:1H19:42:232
969216994543,1cyclictest1063764-21kworker/0:223:59:530
969216994543,1cyclictest1017236-21kworker/0:221:45:230
969216994542,2cyclictest1030726-21kworker/0:022:42:350
969216994541,2cyclictest1027240-21kworker/0:122:05:540
969216994540,3cyclictest3964510-1kworker/0:0H22:32:540
969216994539,3cyclictest3964510-1kworker/0:0H21:40:160
969223994440,3cyclictest1712070-1kworker/3:2H20:56:533
969223994440,3cyclictest1005852-21kworker/3:221:17:103
969223994439,3cyclictest1712070-1kworker/3:2H20:00:263
969220994438,3cyclictest130910-1kworker/2:1H23:00:012
969216994442,1cyclictest1047209-21kworker/0:223:20:150
969216994441,2cyclictest1007487-21kworker/0:121:16:230
969216994441,2cyclictest1004250-21kworker/0:020:55:060
969216994440,3cyclictest997750-21kworker/0:220:50:550
969216994440,2cyclictest1047209-21kworker/0:223:36:240
969216994440,2cyclictest1030726-21kworker/0:022:38:540
969216994438,4cyclictest3964510-1kworker/0:0H00:30:200
969223994340,2cyclictest1005852-21kworker/3:221:40:223
969223994339,2cyclictest1712070-1kworker/3:2H23:05:533
969223994339,2cyclictest1005852-21kworker/3:221:20:183
969223994338,3cyclictest986373-21kworker/3:120:05:183
969223994338,3cyclictest1712070-1kworker/3:2H23:17:343
969223994338,3cyclictest1712070-1kworker/3:2H22:41:533
969223994336,5cyclictest1035856-21kworker/3:222:39:193
969220994340,2cyclictest130910-1kworker/2:1H20:45:172
969220994339,2cyclictest988070-21kworker/2:221:18:542
969220994338,3cyclictest130910-1kworker/2:1H20:25:012
969220994338,3cyclictest130910-1kworker/2:1H20:00:532
969216994340,2cyclictest966211-21kworker/0:219:11:190
969216994340,2cyclictest3964510-1kworker/0:0H19:34:240
969216994340,2cyclictest1047209-21kworker/0:223:13:210
969216994340,2cyclictest1013997-21kworker/0:221:23:540
969216994339,3cyclictest1050459-21kworker/0:123:18:320
969216994339,2cyclictest991247-21kworker/0:120:13:270
969216994339,2cyclictest1023979-21kworker/0:222:04:240
969216994337,3cyclictest3964510-1kworker/0:0H21:35:170
969223994238,2cyclictest1035856-21kworker/3:223:13:093
969223994237,3cyclictest1712070-1kworker/3:2H22:20:063
969223994237,3cyclictest1712070-1kworker/3:2H00:24:213
969223994236,3cyclictest1712070-1kworker/3:2H19:25:143
969220994238,3cyclictest130910-1kworker/2:1H22:18:532
969220994238,2cyclictest130910-1kworker/2:1H22:43:542
969220994237,3cyclictest130910-1kworker/2:1H22:00:172
969220994237,2cyclictest130910-1kworker/2:1H00:23:302
969216994239,2cyclictest988008-21kworker/0:020:19:000
969216994239,2cyclictest1060257-21kworker/0:123:50:470
969216994238,3cyclictest1020496-21kworker/0:021:54:550
969216994237,3cyclictest966211-21kworker/0:219:43:180
969216994237,3cyclictest3964510-1kworker/0:0H00:10:180
969223994136,3cyclictest1712070-1kworker/3:2H21:27:453
969220994137,2cyclictest130910-1kworker/2:1H19:15:152
969220994131,7cyclictest130910-1kworker/2:1H21:10:172
96922099411,37cyclictest1756130-21systemd-journal19:49:592
969216994139,1cyclictest966211-21kworker/0:219:15:390
969216994138,2cyclictest1063764-21kworker/0:200:00:240
969216994138,2cyclictest1043952-21kworker/0:123:00:070
969216994137,2cyclictest3964510-1kworker/0:0H19:35:230
969223994036,3cyclictest1712070-1kworker/3:2H22:10:193
969223994035,3cyclictest1712070-1kworker/3:2H21:09:553
969223994035,3cyclictest1712070-1kworker/3:2H19:10:263
969223994034,4cyclictest1712070-1kworker/3:2H22:25:043
969223994033,4cyclictest1712070-1kworker/3:2H20:25:163
969220994037,2cyclictest130910-1kworker/2:1H21:29:242
969220994036,3cyclictest130910-1kworker/2:1H22:50:532
969220994033,5cyclictest130910-1kworker/2:1H23:15:162
969220994028,11cyclictest130910-1kworker/2:1H19:35:182
969216994037,2cyclictest997750-21kworker/0:221:02:530
969216994036,2cyclictest988008-21kworker/0:020:06:540
969216994036,2cyclictest983143-21kworker/0:119:52:240
969216994036,2cyclictest3964510-1kworker/0:0H21:15:000
969216994036,2cyclictest1047209-21kworker/0:223:27:550
969223993935,3cyclictest967605-21kworker/3:219:36:483
969223993934,3cyclictest1712070-1kworker/3:2H23:02:363
969223993934,3cyclictest1061865-21kworker/3:000:10:173
969220993936,2cyclictest1055435-21kworker/2:223:45:002
969220993934,3cyclictest130910-1kworker/2:1H20:35:172
96922099391,34cyclictest996144-21cron20:30:002
969223993834,2cyclictest1061865-21kworker/3:000:26:233
969223993833,3cyclictest967605-21kworker/3:219:40:183
969223993833,3cyclictest1712070-1kworker/3:2H22:45:313
969220993835,2cyclictest130910-1kworker/2:1H20:42:542
969216993835,2cyclictest997750-21kworker/0:220:44:240
969216993835,2cyclictest3964510-1kworker/0:0H21:30:220
969216993835,2cyclictest1030726-21kworker/0:022:46:530
969216993832,3cyclictest3964510-1kworker/0:0H22:10:150
969223993733,2cyclictest989637-21kworker/3:220:23:543
969223993733,2cyclictest1712070-1kworker/3:2H22:57:363
969220993734,2cyclictest130910-1kworker/2:1H21:40:182
969220993734,2cyclictest1071766-21kworker/2:000:34:212
969220993732,3cyclictest130910-1kworker/2:1H19:20:122
969216993734,2cyclictest997750-21kworker/0:220:35:250
969216993734,2cyclictest1066744-21kworker/0:000:20:270
969216993733,2cyclictest1027240-21kworker/0:122:17:340
969223993632,2cyclictest1712070-1kworker/3:2H19:31:303
969220993633,2cyclictest130910-1kworker/2:1H23:37:242
969220993632,2cyclictest130910-1kworker/2:1H20:52:232
96922099360,34cyclictest0-21swapper/222:20:132
969216993633,2cyclictest983143-21kworker/0:119:56:070
969223993531,2cyclictest1712070-1kworker/3:2H00:09:493
969220993531,2cyclictest988070-21kworker/2:221:50:552
969220993528,5cyclictest130910-1kworker/2:1H20:55:142
96922099351,31cyclictest1071643-21readlink00:19:592
969216993531,2cyclictest1047209-21kworker/0:223:30:060
969216993531,2cyclictest1047209-21kworker/0:223:30:060
969223993431,2cyclictest1712070-1kworker/3:2H23:34:563
969223993431,2cyclictest1712070-1kworker/3:2H23:34:553
969223993430,2cyclictest1712070-1kworker/3:2H20:34:193
969220993431,2cyclictest1027342-21kworker/2:022:25:542
969220993430,2cyclictest952947-21kworker/2:019:28:532
96922099340,3cyclictest1039111-21munin-plugin-st22:40:002
96922099340,32cyclictest0-21swapper/223:00:142
969216993431,2cyclictest1007487-21kworker/0:121:06:520
969220993330,2cyclictest952947-21kworker/2:019:10:182
969220993330,2cyclictest130910-1kworker/2:1H21:55:532
969220993322,9cyclictest130910-1kworker/2:1H23:30:002
96922099330,30cyclictest997765-21cron20:35:002
96922099330,30cyclictest0-21swapper/200:00:142
96922099330,29cyclictest0-21swapper/220:11:272
969223993228,3cyclictest1712070-1kworker/3:2H19:50:333
969223993227,3cyclictest1712070-1kworker/3:2H22:50:193
96922099320,24cyclictest1049841-21missed_timers23:10:202
969216993230,1cyclictest1072757-21kworker/0:200:38:430
969223993128,2cyclictest997757-21kworker/3:120:42:053
969223993127,2cyclictest989637-21kworker/3:220:15:543
969220993128,2cyclictest130910-1kworker/2:1H19:30:132
969220993127,2cyclictest130910-1kworker/2:1H20:15:152
96922099311,22cyclictest0-21swapper/221:45:142
96922099310,7cyclictest1028983-21sed22:05:232
96922099310,27cyclictest1006942-21head21:00:192
96922399303,9cyclictest974700-21latency_hist19:25:003
96922099304,23cyclictest1062048-21grep23:50:012
96922099301,26cyclictest986404-21munin-run20:00:002
96922099301,26cyclictest984536-21munin-run19:55:002
96922099300,2cyclictest1062121-21/usr/sbin/munin23:50:152
96922099300,23cyclictest0-21swapper/200:10:172
969218993016,13cyclictest0-21swapper/119:10:001
969216993028,1cyclictest1060257-21kworker/0:123:47:540
969216993027,2cyclictest3964510-1kworker/0:0H20:45:230
969216993026,2cyclictest3964510-1kworker/0:0H19:21:460
96922099291,25cyclictest988752-21head20:05:182
96922099291,25cyclictest0-21swapper/223:30:232
96922099291,25cyclictest0-21swapper/223:30:232
96922099290,22cyclictest0-21swapper/223:07:502
96922099280,26cyclictest1017950-21gltestperf21:35:182
96922099280,26cyclictest1016174-21sh21:30:142
969223992724,2cyclictest1712070-1kworker/3:2H00:33:213
969223992421,2cyclictest1712070-1kworker/3:2H22:08:533
96922099231,20cyclictest0-21swapper/219:09:592
969223992116,3cyclictest654-21snmpd23:44:493
969216991511,2cyclictest0-21swapper/019:10:000
96922399137,4cyclictest1712070-1kworker/3:2H19:09:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional