You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 20:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Nov 17, 2025 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
90421579,21sleep00-21swapper/007:09:210
10869913852,56cyclictest0-21swapper/011:14:410
32652212234,47sleep20-21swapper/207:05:242
10869911646,48cyclictest7174-21mailstats07:25:160
726121140,7sleep10-21swapper/107:25:171
1086991116,96cyclictest0-21swapper/009:49:530
1086991116,96cyclictest0-21swapper/009:49:530
11029910664,35cyclictest207-21systemd-journal08:48:162
11029910664,35cyclictest207-21systemd-journal08:48:162
10869910641,54cyclictest0-21swapper/012:05:050
1086991022,9cyclictest0-21swapper/009:34:550
11029910157,36cyclictest0-21swapper/212:30:212
1102991013,45cyclictest26467-21ssh09:39:552
10869910146,46cyclictest0-21swapper/010:59:440
10869910140,51cyclictest0-21swapper/011:20:180
1086991003,86cyclictest22582-21apt-get10:29:550
1086991003,6cyclictest0-21swapper/010:39:540
1086991003,6cyclictest0-21swapper/010:39:540
1086991002,8cyclictest0-21swapper/010:10:000
52829827,42sleep30-21swapper/307:05:293
110299983,27cyclictest355-21runrttasks07:47:522
110299983,27cyclictest355-21runrttasks07:47:522
108699984,86cyclictest19566-21apt-get10:24:540
108699984,86cyclictest19566-21apt-get10:24:540
1086999847,44cyclictest0-21swapper/009:55:050
108699982,81cyclictest20881-21apt-get11:24:540
110299974,7cyclictest0-21swapper/212:09:552
110299973,42cyclictest2684-21ssh11:49:552
323432969,60sleep132379-21cat07:05:101
1086999646,43cyclictest17316-21sh10:20:140
1086999637,52cyclictest0-21swapper/012:19:550
108699963,6cyclictest0-21swapper/008:24:540
108699962,7cyclictest0-21swapper/007:49:580
110299952,31cyclictest30286-21seq10:44:132
110299952,31cyclictest30286-21seq10:44:132
1097999544,44cyclictest12372-21grep07:40:211
108699954,62cyclictest6980-21kworker/u17:1+rpciod10:45:170
108699953,6cyclictest0-21swapper/011:39:570
108699953,6cyclictest0-21swapper/011:39:570
108699952,23cyclictest355-21runrttasks09:22:560
108699952,22cyclictest12445-21seq07:41:040
1097999426,37cyclictest0-21swapper/108:04:491
1086999443,43cyclictest0-21swapper/008:10:100
1086999441,47cyclictest6698-21sh10:00:260
108699943,82cyclictest25443-21apt-get10:34:570
108699943,82cyclictest25443-21apt-get10:34:570
108699943,7cyclictest0-21swapper/009:00:220
108699943,7cyclictest0-21swapper/008:29:550
108699943,6cyclictest0-21swapper/011:54:530
108699943,64cyclictest32589-21apt-get08:44:550
108699943,64cyclictest32589-21apt-get08:44:550
110299933,54cyclictest2788-21apt-get09:54:562
1097999348,40cyclictest4343-21sh09:56:511
108699933,8cyclictest0-21swapper/011:04:580
1086999332,51cyclictest0-21swapper/008:35:080
1086999332,51cyclictest0-21swapper/008:35:080
108699932,6cyclictest0-21swapper/007:59:550
108699932,66cyclictest19312-21seq12:17:160
108699932,63cyclictest27024-21/usr/sbin/munin12:30:120
108699932,19cyclictest28637-21runrttasks11:38:430
1102999249,37cyclictest13898-21/usr/sbin/munin10:15:182
1102999249,37cyclictest13898-21/usr/sbin/munin10:15:182
1102999246,39cyclictest24495-21sh12:25:182
1097999257,30cyclictest1073-21cyclictest11:40:251
1097999257,30cyclictest1073-21cyclictest11:40:251
1097999249,37cyclictest14599-21mailstats10:15:181
1097999249,37cyclictest14599-21mailstats10:15:181
109799922,81cyclictest32144-21ssh09:49:531
109799922,81cyclictest32144-21ssh09:49:531
108699924,80cyclictest17822-21apt-get09:24:580
108699924,71cyclictest13916-21apt-get10:15:000
108699924,71cyclictest13916-21apt-get10:15:000
108699923,83cyclictest30722-21wc12:35:210
108699923,83cyclictest30722-21wc12:35:210
108699923,80cyclictest2666-21apt-get11:49:530
108699922,26cyclictest10416-21taskset09:11:110
110299912,29cyclictest355-21runrttasks08:11:542
110299912,27cyclictest355-21runrttasks08:51:362
110299912,27cyclictest355-21runrttasks08:51:362
110299912,27cyclictest355-21runrttasks08:16:402
110299912,27cyclictest355-21runrttasks08:16:402
1097999148,36cyclictest24519-21switchtime11:30:181
109799912,73cyclictest29855-21ssh12:35:011
109799912,73cyclictest29855-21ssh12:35:011
1086999141,41cyclictest0-21swapper/007:45:110
1086999141,41cyclictest0-21swapper/007:45:110
1086999137,47cyclictest0-21swapper/007:59:410
1086999130,56cyclictest0-21swapper/012:00:040
108699912,63cyclictest23578-21ssh12:24:540
108699912,23cyclictest8941-21runrttasks11:04:040
108699912,23cyclictest8941-21runrttasks11:04:040
110299903,7cyclictest0-21swapper/210:10:002
110299903,7cyclictest0-21swapper/207:29:572
110299903,38cyclictest23391-21grep11:29:412
110299903,27cyclictest355-21runrttasks09:06:472
1097999046,33cyclictest0-21swapper/111:55:031
109799903,81cyclictest0-21swapper/112:30:061
108699903,7cyclictest0-21swapper/010:05:190
108699902,23cyclictest15578-21/usr/sbin/munin12:10:200
110299893,9cyclictest0-21swapper/212:05:002
110299893,38cyclictest30941-21fschecks_time10:45:052
110299892,26cyclictest355-21runrttasks07:17:022
110299892,25cyclictest355-21runrttasks09:26:122
1097998942,40cyclictest0-21swapper/110:31:041
108699894,77cyclictest0-21swapper/008:49:570
108699894,77cyclictest0-21swapper/008:49:570
108699894,74cyclictest32144-21apt-get11:44:550
108699893,79cyclictest21751-21head09:30:190
108699893,79cyclictest21751-21head09:30:190
110299882,28cyclictest355-21runrttasks07:11:072
110299882,25cyclictest355-21runrttasks08:23:492
110299882,25cyclictest355-21runrttasks08:23:492
1097998867,15cyclictest314-21in:imuxsock10:59:421
1097998845,36cyclictest18383-21if_eth011:20:041
1097998845,35cyclictest9337-21mailstats12:00:161
108699884,70cyclictest9763-21apt-get07:34:540
1086998833,48cyclictest0-21swapper/009:09:410
108699882,7cyclictest0-21swapper/007:19:540
108699882,7cyclictest0-21swapper/007:19:540
108699882,24cyclictest27967-21ssh09:41:360
1102998746,33cyclictest0-21swapper/209:45:102
110299873,39cyclictest9744-21/usr/sbin/munin07:34:592
108699874,75cyclictest3866-21apt-get08:54:580
108699873,74cyclictest24555-21apt-get08:20:000
108699873,74cyclictest24555-21apt-get08:20:000
1102998643,35cyclictest0-21swapper/211:00:202
1102998643,35cyclictest0-21swapper/211:00:202
110299863,8cyclictest0-21swapper/211:39:572
110299863,8cyclictest0-21swapper/211:39:572
110299863,8cyclictest0-21swapper/211:09:522
110299863,7cyclictest0-21swapper/207:59:552
110299863,40cyclictest6189-21latency09:00:332
1097998646,32cyclictest0-21swapper/108:29:441
108699865,73cyclictest19701-21apt-get08:04:590
108699864,77cyclictest0-21swapper/008:44:450
108699862,76cyclictest30712-21sh09:46:180
1086998627,51cyclictest0-21swapper/007:29:590
108699862,58cyclictest2669-21ssh10:52:010
1102998564,11cyclictest0-21swapper/208:35:052
1102998564,11cyclictest0-21swapper/208:35:052
1102998561,12cyclictest0-21swapper/211:24:442
1102998551,28cyclictest11892-21ntp_kernel_pll_07:40:132
110299854,35cyclictest4927-21/usr/sbin/munin07:20:162
110299854,35cyclictest4927-21/usr/sbin/munin07:20:162
110299854,10cyclictest0-21swapper/211:54:542
110299853,25cyclictest355-21runrttasks08:41:432
110299853,25cyclictest355-21runrttasks08:41:432
110299852,39cyclictest5218-21sendmail08:59:402
110299852,37cyclictest23700-21chrt11:29:532
110299852,27cyclictest355-21runrttasks08:28:312
1097998545,33cyclictest0-21swapper/108:44:431
1097998545,33cyclictest0-21swapper/108:44:431
109799853,73cyclictest0-21swapper/109:04:581
108699853,7cyclictest0-21swapper/009:15:280
108699853,7cyclictest0-21swapper/009:15:280
108699852,26cyclictest4194-21unixbench_multi07:15:210
108699852,24cyclictest17919-21latency_hist11:19:440
110299843,38cyclictest9312-21grep09:10:052
110299842,7cyclictest0-21swapper/209:34:552
110299842,42cyclictest368-21snmpd10:34:552
110299842,42cyclictest368-21snmpd10:34:552
110299842,36cyclictest22056-21apt-get10:29:562
1097998448,29cyclictest18950-21sh10:23:501
1097998437,39cyclictest0-21swapper/111:00:161
1097998437,39cyclictest0-21swapper/111:00:161
108699843,26cyclictest24331-21open_files11:30:140
108699842,26cyclictest31951-21mailstats08:40:520
1102998341,34cyclictest0-21swapper/210:50:192
110299833,7cyclictest0-21swapper/208:29:552
110299833,37cyclictest20885-21/usr/sbin/munin12:20:062
110299833,35cyclictest355-21runrttasks12:35:272
110299833,35cyclictest355-21runrttasks12:35:272
110299833,35cyclictest32693-21/usr/sbin/munin11:45:102
110299833,35cyclictest12546-21ssh09:15:362
110299833,35cyclictest12546-21ssh09:15:362
110299832,38cyclictest9495-21ssh11:04:572
110299832,38cyclictest15182-21ssh11:14:552
110299832,35cyclictest18321-21switchtime12:15:172
110299832,23cyclictest355-21runrttasks07:28:122
1097998361,14cyclictest1073-21cyclictest08:45:191
1097998361,14cyclictest1073-21cyclictest08:45:191
109799834,59cyclictest23548-21apt-get09:34:551
109799833,66cyclictest0-21swapper/107:49:581
108699832,26cyclictest2086-21timerwakeupswit07:10:180
1102998265,8cyclictest0-21swapper/210:05:132
110299823,38cyclictest355-21runrttasks10:03:332
110299823,36cyclictest15246-21ssh09:20:062
110299822,35cyclictest26434-21ssh11:34:532
1097998255,19cyclictest17042-21uname07:55:171
108699823,7cyclictest0-21swapper/008:14:590
108699823,7cyclictest0-21swapper/008:14:590
110299814,33cyclictest3969-21grep10:55:052
110299813,8cyclictest0-21swapper/208:04:562
110299813,35cyclictest16657-21sort07:55:062
110299813,34cyclictest9188-21ssh12:00:112
110299813,24cyclictest14610-21apt-get07:49:592
110299812,9cyclictest0-21swapper/209:29:572
110299812,9cyclictest0-21swapper/209:29:572
110299812,34cyclictest32262-21cat09:49:592
110299812,34cyclictest32262-21cat09:49:592
1097998142,32cyclictest6437-21mailstats09:00:411
1097998141,32cyclictest28681-21diskmemload09:43:431
110299803,36cyclictest17676-21/usr/sbin/munin10:20:192
110299802,38cyclictest19593-21ssh10:24:572
110299802,38cyclictest19593-21ssh10:24:572
1097998046,27cyclictest10664-21diskmemload09:11:561
1097998043,33cyclictest1073-21cyclictest10:52:431
109799803,68cyclictest14483-21apt-get12:09:551
109799802,65cyclictest0-21swapper/108:09:581
1097997961,12cyclictest3809-21ntp_kernel_pll_07:15:131
1097997957,14cyclictest207-21systemd-journal12:22:151
1097997937,34cyclictest9496-21latency_hist07:34:441
1097997852,14cyclictest0-21swapper/110:09:451
1097997841,32cyclictest0-21swapper/108:35:171
1097997841,32cyclictest0-21swapper/108:35:171
1097997838,33cyclictest0-21swapper/110:40:061
1097997838,33cyclictest0-21swapper/110:40:061
109799783,66cyclictest0-21swapper/108:14:571
109799783,66cyclictest0-21swapper/108:14:571
109799783,62cyclictest0-21swapper/112:04:551
1097997751,17cyclictest0-21swapper/109:20:191
1097997751,15cyclictest0-21swapper/111:09:571
109799772,66cyclictest0-21swapper/108:54:581
1097997651,14cyclictest0-21swapper/112:25:051
1097997639,30cyclictest0-21swapper/109:33:291
1097997639,30cyclictest0-21swapper/109:33:291
1097997638,31cyclictest13102-21sh09:15:491
1097997638,31cyclictest13102-21sh09:15:491
1097997637,33cyclictest18096-21sh12:15:121
109799763,63cyclictest0-21swapper/107:44:581
109799763,63cyclictest0-21swapper/107:44:581
109799763,62cyclictest0-21swapper/107:19:541
109799763,62cyclictest0-21swapper/107:19:541
109799763,61cyclictest0-21swapper/107:34:541
1097997635,34cyclictest0-21swapper/111:54:411
1097997626,41cyclictest0-21swapper/108:49:581
1097997626,41cyclictest0-21swapper/108:49:581
109799759,55cyclictest32172-21ssh11:44:571
1097997556,13cyclictest29948-21sh09:45:161
1097997553,13cyclictest0-21swapper/110:00:211
109799753,56cyclictest18087-21apt-get07:59:531
26122740,6sleep30-21swapper/309:54:443
26122740,6sleep30-21swapper/309:54:443
1097997458,10cyclictest11848-21ssh11:09:151
1097997444,22cyclictest369-21ntpd08:20:181
1097997444,22cyclictest369-21ntpd08:20:181
1097997435,32cyclictest0-21swapper/110:26:581
1097997435,32cyclictest0-21swapper/110:26:581
1097997429,39cyclictest0-21swapper/111:35:131
109799742,64cyclictest0-21swapper/111:24:541
110999733,37cyclictest14319-21taskset09:18:373
110999733,37cyclictest14319-21taskset09:18:373
1097997341,26cyclictest28063-21kworker/u17:0-xprtiod10:46:311
109799733,61cyclictest0-21swapper/110:34:561
109799733,61cyclictest0-21swapper/110:34:561
1097997225,41cyclictest0-21swapper/109:25:181
109799722,53cyclictest27769-21apt-get08:29:591
110999713,27cyclictest21977-21chrt11:25:213
1097997129,36cyclictest0-21swapper/110:10:031
110999705,51cyclictest0-21swapper/312:09:553
110999703,36cyclictest9208-21taskset12:00:133
110999703,36cyclictest28749-21chrt08:30:203
1097997057,9cyclictest1073-21cyclictest07:10:061
1097997029,34cyclictest0-21swapper/111:15:191
110999693,36cyclictest29522-21taskset08:35:023
110999693,36cyclictest29522-21taskset08:35:023
110999693,35cyclictest5979-21chrt10:58:513
110999693,32cyclictest3584-21chrt07:15:073
110999686,30cyclictest29715-21taskset09:45:083
110999684,31cyclictest1770-21chrt10:50:163
110999683,33cyclictest28623-21taskset12:32:143
110999683,31cyclictest28823-21chrt09:44:113
110999682,34cyclictest6311-21chrt09:00:383
110999673,33cyclictest30227-21chrt12:35:103
110999673,33cyclictest30227-21chrt12:35:103
110999673,33cyclictest25664-21taskset09:38:193
110999673,32cyclictest31245-21chrt10:45:143
110999673,31cyclictest31850-21chrt11:44:413
110999673,31cyclictest31850-21chrt11:44:413
110999672,34cyclictest13959-21chrt07:45:203
110999672,34cyclictest13959-21chrt07:45:203
110999664,27cyclictest23389-21chrt08:15:133
110999664,27cyclictest23389-21chrt08:15:133
110999663,32cyclictest23253-21chrt12:24:403
110999663,31cyclictest19706-21chrt08:04:563
110999662,35cyclictest8398-21chrt09:06:013
110999662,34cyclictest13781-21chrt12:08:403
110999662,32cyclictest17852-21chrt09:24:573
110999653,33cyclictest23899-21taskset11:30:033
110999653,32cyclictest32513-21chrt11:45:063
110999653,32cyclictest25582-21chrt08:20:483
110999653,32cyclictest25582-21chrt08:20:483
110999653,30cyclictest22993-21taskset09:34:013
110999653,30cyclictest22993-21taskset09:34:013
110999652,33cyclictest12898-21chrt10:12:323
110999652,27cyclictest7704-21chrt11:00:213
110999652,27cyclictest7704-21chrt11:00:213
110999644,29cyclictest11155-21taskset11:07:093
110999644,24cyclictest24819-21chrt10:33:473
110999643,32cyclictest4781-21chrt11:53:163
110999643,31cyclictest7182-21chrt11:56:413
110999643,30cyclictest27089-21chrt08:25:303
110999642,33cyclictest19538-21chrt11:21:403
110999642,31cyclictest27590-21chrt11:35:303
110999642,25cyclictest27301-21chrt10:38:173
110999642,25cyclictest27301-21chrt10:38:173
110999642,23cyclictest14100-21chrt11:12:283
110999633,34cyclictest2533-21taskset07:13:573
110999633,31cyclictest20171-21taskset10:25:123
110999633,31cyclictest20171-21taskset10:25:123
110999633,31cyclictest15705-21chrt10:17:383
110999633,31cyclictest15705-21chrt10:17:383
110999633,30cyclictest17461-21taskset11:19:053
110999633,27cyclictest6198-21chrt10:00:163
110999633,26cyclictest3034-21chrt08:50:173
110999633,26cyclictest3034-21chrt08:50:173
110999633,26cyclictest28386-21chrt10:40:033
110999633,26cyclictest28386-21chrt10:40:033
110999633,25cyclictest9627-21chrt10:05:553
110999633,25cyclictest24194-21chrt12:25:133
110999632,27cyclictest18245-21chrt08:00:023
110999632,27cyclictest16614-21taskset07:55:043
110999624,28cyclictest6101-21chrt07:22:283
110999624,28cyclictest6101-21chrt07:22:283
110999623,27cyclictest10977-21chrt07:38:153
110999623,22cyclictest14612-21chrt07:49:543
110999622,29cyclictest2753-21chrt09:54:533
110999622,26cyclictest19050-21chrt12:16:343
110999615,24cyclictest7639-21chrt07:26:373
110999614,28cyclictest21322-21taskset08:09:593
110999613,23cyclictest16356-21chrt09:21:003
110999613,21cyclictest31254-21chrt08:40:003
110999613,21cyclictest31254-21chrt08:40:003
110999612,24cyclictest12418-21chrt07:40:423
110999612,24cyclictest10346-21chrt09:10:593
110999603,23cyclictest9187-21chrt07:30:503
110999603,23cyclictest3870-21chrt08:54:553
110999593,23cyclictest18799-21chrt10:23:223
110999593,21cyclictest653-21chrt08:45:023
110999593,21cyclictest653-21chrt08:45:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional