You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 16:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Fri Jan 02, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30327991203,94cyclictest3840-21apt-get12:30:102
30327991203,94cyclictest3840-21apt-get12:30:102
30306991133,43cyclictest30791-21ssh10:40:090
30306991075,26cyclictest14550-21/usr/sbin/munin11:05:070
30306991075,26cyclictest14550-21/usr/sbin/munin11:05:070
303069910548,51cyclictest0-21swapper/008:20:350
303069910535,64cyclictest0-21swapper/010:00:170
30306991044,26cyclictest18412-21uname10:20:210
303199910336,35cyclictest8096-21sh09:14:051
303199910336,35cyclictest8096-21sh09:14:051
303069910347,50cyclictest30500-21kworker/u17:3+rpciod11:45:260
30306991023,31cyclictest4388-21apt-get12:30:080
30306991023,31cyclictest4388-21apt-get12:30:080
30306991023,31cyclictest352-21runrttasks09:15:410
30306991014,31cyclictest24332-21find12:10:300
30306991014,31cyclictest24332-21find12:10:300
303069910139,55cyclictest3989-21cstates09:05:120
30306991013,37cyclictest7586-21ssh12:35:070
303069910114,14cyclictest77582latency_hist10:55:000
303069910055,41cyclictest30296-21cyclictest11:32:260
30306991004,33cyclictest352-21runrttasks10:25:230
3030699994,36cyclictest17579-21idleruntime12:00:200
3030699993,27cyclictest27377-21latency_hist10:34:590
3030699993,27cyclictest27377-21latency_hist10:34:590
30306999918,29cyclictest14549-21rm10:14:590
3030699984,31cyclictest24630-21/usr/sbin/munin11:20:220
3030699984,31cyclictest24630-21/usr/sbin/munin11:20:220
30306999835,55cyclictest0-21swapper/010:05:220
3030699983,32cyclictest18099-21unin-run09:29:570
30306999827,62cyclictest0-21swapper/011:00:300
3030699982,26cyclictest1333-21taskset12:25:400
30306999814,28cyclictest7545-21wc11:44:590
30306999814,28cyclictest7545-21wc11:44:590
298592986,18sleep20-21swapper/207:06:052
3031999973,83cyclictest0-21swapper/107:50:101
3030699974,32cyclictest7873-21/usr/sbin/munin10:55:240
3030699974,30cyclictest21544-21/usr/sbin/munin09:35:130
3030699974,30cyclictest21544-21/usr/sbin/munin09:35:130
30306999739,51cyclictest0-21swapper/009:52:480
3030699973,36cyclictest31147-21mailstats07:10:310
3030699973,23cyclictest30216-21/usr/sbin/munin12:20:170
3030699972,21cyclictest28994-21/usr/sbin/munin09:45:350
3030699964,25cyclictest30296-21cyclictest07:10:000
3030699963,30cyclictest15028-21ssh09:25:000
3030699963,30cyclictest15028-21ssh09:25:000
30306999615,23cyclictest15671-21sed10:15:340
3030699954,30cyclictest12712-21modprobe07:54:570
3030699953,32cyclictest19681-21/usr/sbin/munin08:15:350
3030699953,30cyclictest3322-21grep09:01:390
3030699952,26cyclictest1955-21latency_hist08:59:590
3030699952,26cyclictest1955-21latency_hist08:59:590
30306999514,24cyclictest29307-21rm12:19:050
30306999514,24cyclictest11580-21rm11:50:300
30306999512,27cyclictest17861-21sort11:10:140
3031999943,77cyclictest0-21swapper/108:55:111
3031999943,77cyclictest0-21swapper/108:55:111
3030699945,35cyclictest28753-21switchtime08:40:310
3030699944,30cyclictest31666-21cat08:50:230
3030699944,26cyclictest4149-21sendmail11:39:560
3030699943,34cyclictest18741-21idleruntime09:30:200
3030699943,32cyclictest32080-21modprobe07:15:010
3030699943,31cyclictest352-21runrttasks10:35:480
3030699943,30cyclictest352-21runrttasks11:18:560
3030699943,30cyclictest352-21runrttasks11:18:560
3030699943,29cyclictest14603-21/usr/sbin/munin08:00:220
30306999430,57cyclictest0-21swapper/009:40:220
3030699942,23cyclictest9779-21apt-get07:45:110
30306999416,21cyclictest1-21systemd08:34:590
30306999415,27cyclictest19607-21runrttasks08:14:590
30306999415,27cyclictest19607-21runrttasks08:14:590
303172930,9sleep30-21swapper/307:10:003
3030699934,34cyclictest352-21runrttasks11:28:570
3030699934,30cyclictest1786-21fschecks_count10:45:180
30306999337,49cyclictest0-21swapper/009:55:190
30306999337,49cyclictest0-21swapper/009:55:190
30306999317,22cyclictest14525-21cat08:00:000
30306999316,23cyclictest14111-21cat11:55:140
30306999316,23cyclictest14111-21cat11:55:140
3030699924,31cyclictest3901-21grep07:25:290
3030699923,34cyclictest5660-21ls09:10:130
3030699923,34cyclictest5660-21ls09:10:130
30306999223,61cyclictest0-21swapper/012:05:200
3030699922,31cyclictest17244-21switchtime08:06:200
3030699922,31cyclictest17244-21switchtime08:06:200
3031999912,77cyclictest0-21swapper/107:20:121
3031999912,77cyclictest0-21swapper/107:20:121
3030699915,29cyclictest8489-21if_err_eth007:40:200
30327999067,16cyclictest1-21systemd12:20:182
3030699904,29cyclictest30499-21grep08:45:330
3030699903,31cyclictest7527-21awk07:35:350
3030699903,30cyclictest6357-21latency_hist07:34:590
3031999893,72cyclictest0-21swapper/108:25:121
3031999892,75cyclictest30809-21ssh10:40:121
3030699893,31cyclictest2674-21chrt07:20:350
3030699893,31cyclictest2674-21chrt07:20:350
3030699893,31cyclictest23523-21ps08:25:290
3030699893,24cyclictest26427-21taskset08:35:270
30319998870,13cyclictest352-21runrttasks12:15:101
30319998746,33cyclictest0-21swapper/110:29:561
3031999873,72cyclictest0-21swapper/107:30:121
3033499863,70cyclictest4494-21chrt12:30:233
3033499863,70cyclictest4494-21chrt12:30:233
3031999864,77cyclictest0-21swapper/108:06:011
3031999864,77cyclictest0-21swapper/108:06:011
30319998643,36cyclictest0-21swapper/109:30:351
3031999863,76cyclictest0-21swapper/109:50:131
3032799853,49cyclictest32005-21latency_hist07:15:002
30319998569,10cyclictest365-21ntpd09:01:341
3031999853,72cyclictest0-21swapper/107:40:121
3032799842,48cyclictest1326-21df12:25:402
30319998447,32cyclictest23222-21kworker/u17:0-xprtiod12:30:141
30319998447,32cyclictest23222-21kworker/u17:0-xprtiod12:30:141
30319998442,35cyclictest803-21sendmail-mta12:35:231
30319998441,36cyclictest0-21swapper/112:04:581
30319998365,10cyclictest1849-21fschecks_time10:45:191
30319998341,35cyclictest3421-21cstates07:25:141
30319998340,36cyclictest0-21swapper/112:05:301
30327998255,18cyclictest10771-21cron10:59:562
30327998252,20cyclictest2173-21latency_hist09:00:002
30327998252,20cyclictest2173-21latency_hist09:00:002
30319998263,12cyclictest978-21latency_hist11:34:581
30319998249,26cyclictest2315-21timerwakeupswit12:26:271
30319998240,34cyclictest0-21swapper/110:50:341
30319998239,37cyclictest0-21swapper/111:06:061
30319998239,37cyclictest0-21swapper/111:06:061
30319998239,36cyclictest0-21swapper/111:50:221
3031999823,73cyclictest0-21swapper/110:35:121
3031999823,73cyclictest0-21swapper/110:10:091
30327998163,12cyclictest300-21rs:main2
30319998166,10cyclictest25408-21ntp_offset09:40:261
30319998158,15cyclictest200-21systemd-journal11:45:481
30319998137,35cyclictest11493-21sh10:10:001
30319998058,14cyclictest200-21systemd-journal12:24:481
30319998042,32cyclictest14639-21kworker/u17:1-nfsiod11:58:381
30319998042,32cyclictest14639-21kworker/u17:1-nfsiod11:58:381
30319998039,33cyclictest0-21swapper/111:25:331
30319998038,33cyclictest0-21swapper/107:45:271
3031999803,70cyclictest0-21swapper/110:00:071
3031999802,73cyclictest0-21swapper/109:25:111
3031999802,72cyclictest0-21swapper/111:40:071
3031999802,72cyclictest0-21swapper/111:40:071
3031999802,67cyclictest0-21swapper/108:40:101
30319997937,33cyclictest0-21swapper/108:54:591
3031999793,71cyclictest0-21swapper/108:15:171
3031999783,69cyclictest0-21swapper/111:15:131
3031999783,69cyclictest0-21swapper/111:15:131
30319997835,35cyclictest0-21swapper/111:10:201
30319997834,37cyclictest0-21swapper/112:10:291
30319997834,37cyclictest0-21swapper/112:10:291
3031999773,65cyclictest24411-21ssh10:30:131
3031999773,65cyclictest24411-21ssh10:30:131
3031999773,64cyclictest0-21swapper/109:05:081
30319997733,37cyclictest0-21swapper/111:39:561
30319997731,39cyclictest0-21swapper/109:20:331
30319997731,39cyclictest0-21swapper/109:20:331
30319997731,37cyclictest22845-21latency_hist08:25:001
30327997652,15cyclictest1551-21cpu10:45:132
3032799764,40cyclictest19432-21munin-plugin-st08:14:562
3032799764,40cyclictest19432-21munin-plugin-st08:14:562
30319997655,15cyclictest13177-21fschecks_count07:55:171
3031999763,68cyclictest0-21swapper/111:20:121
3031999763,68cyclictest0-21swapper/111:20:121
30319997633,34cyclictest0-21swapper/111:05:001
3031999762,67cyclictest0-21swapper/108:30:111
3031999762,63cyclictest0-21swapper/108:10:081
3031999762,63cyclictest0-21swapper/108:10:081
3032799754,25cyclictest0-21swapper/209:05:162
3031999753,65cyclictest0-21swapper/107:35:091
30319997536,32cyclictest0-21swapper/110:24:561
3031999752,68cyclictest0-21swapper/108:45:101
3033499746,34cyclictest26638-21chrt10:33:283
3033499746,34cyclictest26638-21chrt10:33:283
30327997456,11cyclictest58-21kcompactd009:10:542
30327997456,11cyclictest58-21kcompactd009:10:542
30327997454,13cyclictest30296-21cyclictest10:39:592
30327997453,12cyclictest0-21swapper/210:00:172
3031999743,63cyclictest14801-21ssh10:15:131
3031999743,61cyclictest1980-21ssh09:55:101
3031999743,61cyclictest1980-21ssh09:55:101
30319997434,33cyclictest0-21swapper/109:35:311
30319997434,33cyclictest0-21swapper/109:35:311
3031999742,67cyclictest0-21swapper/108:00:111
3033499732,32cyclictest8569-21chrt10:55:283
30327997355,12cyclictest352-21runrttasks08:06:142
30327997355,12cyclictest352-21runrttasks08:06:142
30327997352,15cyclictest20961-21netstat12:05:232
30327997351,10cyclictest0-21swapper/211:44:562
30327997351,10cyclictest0-21swapper/211:44:562
30327997346,20cyclictest30296-21cyclictest08:50:002
3031999733,60cyclictest27865-21sed08:39:591
30319997326,41cyclictest0-21swapper/110:59:581
3031999732,60cyclictest0-21swapper/107:10:091
30327997251,10cyclictest0-21swapper/211:05:002
30327997249,17cyclictest1-21systemd09:52:102
30319997257,9cyclictest28519-21sh09:45:251
3031999722,65cyclictest0-21swapper/107:15:101
3033499712,33cyclictest21592-21chrt10:25:223
30327997143,18cyclictest25222-21mailstats08:30:302
3032799713,39cyclictest8304-21find12:35:282
3032799712,43cyclictest31996-21sed08:50:322
3031999713,63cyclictest0-21swapper/109:15:091
2899627111,38sleep128998-21/usr/sbin/munin07:05:231
3032799704,48cyclictest15561-21grep08:00:332
3032799702,44cyclictest27414-21latency_hist10:35:002
3032799702,44cyclictest27414-21latency_hist10:35:002
3032799702,40cyclictest11249-21idleruntime-cro10:09:562
3033499693,30cyclictest17578-21chrt12:00:203
3033499693,27cyclictest31534-21chrt08:50:203
30327996950,12cyclictest0-21swapper/211:20:312
30327996950,12cyclictest0-21swapper/211:20:312
3032799692,44cyclictest29023-21tail09:45:352
3033499683,27cyclictest3861-21chrt07:25:283
3033499682,31cyclictest30230-21chrt11:29:573
3033499682,28cyclictest15115-21chrt10:15:213
30327996849,7cyclictest181rcu_preempt11:39:412
30327996847,15cyclictest21185-21sh11:15:212
30327996847,15cyclictest21185-21sh11:15:212
30327996847,14cyclictest3059-21idleruntime-cro07:24:562
30327996847,14cyclictest3059-21idleruntime-cro07:24:562
30327996845,14cyclictest30741-21fschecks_count11:30:182
30327996844,16cyclictest14055-21cpu11:55:142
30327996844,16cyclictest14055-21cpu11:55:142
30327996831,8cyclictest181rcu_preempt07:50:002
30327996828,31cyclictest11932-21basename10:10:202
3032799682,46cyclictest11795-21latency_hist09:19:592
3033499672,30cyclictest25127-21taskset11:20:333
3033499672,30cyclictest25127-21taskset11:20:333
30334996721,35cyclictest135992chrt11:54:453
30327996743,16cyclictest200-21systemd-journal12:01:482
3032799672,26cyclictest39-21ksoftirqd/210:27:012
3033499663,27cyclictest10784-21chrt07:45:493
3033499662,26cyclictest28238-21taskset12:16:253
3033499662,26cyclictest20192-21taskset11:13:563
3033499662,25cyclictest4546-21chrt09:59:163
3033499662,25cyclictest4546-21chrt09:59:163
30327996647,7cyclictest181rcu_preempt10:43:102
30327996646,13cyclictest0-21swapper/210:20:332
3032799664,40cyclictest1936-21/usr/sbin/munin09:55:192
3032799664,40cyclictest1936-21/usr/sbin/munin09:55:192
30327996637,20cyclictest5240-21diskmemload09:35:132
30327996637,20cyclictest5240-21diskmemload09:35:132
3032799663,39cyclictest39-21ksoftirqd/209:44:362
3032799663,34cyclictest18643-21fschecks_count09:30:192
3032799663,30cyclictest27652-21switchtime12:15:322
30327996618,33cyclictest39-21ksoftirqd/211:47:172
3033499653,27cyclictest26175-21chrt09:41:133
3033499653,25cyclictest22044-21chrt09:35:223
3033499653,25cyclictest22044-21chrt09:35:223
3033499652,27cyclictest4952-21taskset09:06:283
3032799654,27cyclictest15681-21latency09:25:242
3032799653,44cyclictest28220-21fschecks_count08:40:172
3032799653,33cyclictest26598-21ssh12:14:592
3032799653,33cyclictest26598-21ssh12:14:592
3033499643,24cyclictest32437-21chrt10:41:553
30334996425,30cyclictest0-21swapper/312:25:433
3033499642,28cyclictest28708-21taskset10:35:393
3033499642,26cyclictest5167-21chrt10:00:053
3033499642,25cyclictest13715-21taskset09:22:043
3033499642,25cyclictest13715-21taskset09:22:043
30327996442,16cyclictest27753-21idleruntime-cro08:39:572
30327996442,15cyclictest3028-21mailstats09:01:362
30327996439,17cyclictest12316-21fschecks_time09:20:192
30327996439,17cyclictest12316-21fschecks_time09:20:192
3032799643,39cyclictest352-21runrttasks08:25:142
3032799643,32cyclictest14551-21apt-get11:05:152
3032799643,32cyclictest14551-21apt-get11:05:152
3033499633,25cyclictest32445-21chrt11:32:363
3033499633,23cyclictest8152-21chrt09:14:143
3033499633,23cyclictest8152-21chrt09:14:143
30334996312,40cyclictest0-21swapper/311:59:583
30334996312,40cyclictest0-21swapper/311:59:583
30334996311,43cyclictest0-21swapper/310:09:593
3032799634,38cyclictest27303-21/usr/sbin/munin11:25:222
3032799633,45cyclictest21235-21latency_hist08:20:002
3032799632,41cyclictest5449-21find10:50:302
3033499624,12cyclictest0-21swapper/310:50:313
3033499623,23cyclictest25458-21chrt12:12:273
3033499623,23cyclictest25458-21chrt12:12:273
3033499623,23cyclictest2290-21chrt09:00:123
3033499623,22cyclictest20434-21chrt12:05:103
3033499623,20cyclictest20570-21chrt10:24:163
3033499623,20cyclictest14826-21chrt11:05:293
3033499623,20cyclictest14826-21chrt11:05:293
3033499622,30cyclictest7618-21taskset07:36:453
3033499622,26cyclictest8789-21chrt11:45:373
3033499622,26cyclictest17198-21taskset09:27:483
3033499622,25cyclictest21345-21chrt08:20:113
3033499622,24cyclictest6779-21chrt11:43:343
3033499622,24cyclictest6779-21chrt11:43:343
3033499622,24cyclictest24867-21chrt08:30:193
3033499622,24cyclictest21596-21chrt11:15:303
3033499622,24cyclictest21596-21chrt11:15:303
3033499622,24cyclictest13042-21chrt07:55:123
3033499622,23cyclictest29508-21chrt09:46:363
3033499622,20cyclictest5682-21chrt07:30:343
30327996247,10cyclictest566-21ntp_kernel_pll_07:15:272
3032799623,41cyclictest11638-21fschecks_time07:50:202
3032799622,43cyclictest22668-21modprobe08:24:562
30327996222,28cyclictest8042-21latency_hist07:39:592
3033499614,21cyclictest19451-21chrt08:14:563
3033499614,21cyclictest19451-21chrt08:14:563
3033499612,23cyclictest11543-21taskset07:50:183
3033499612,19cyclictest11713-21chrt09:19:573
3032799612,37cyclictest5163-21fschecks_time07:30:212
3032799612,34cyclictest39-21ksoftirqd/210:15:222
3033499603,23cyclictest17577-21chrt08:06:553
3033499603,23cyclictest17577-21chrt08:06:553
3033499603,19cyclictest15717-21chrt08:01:483
3033499602,28cyclictest9158-21taskset12:36:433
3033499602,23cyclictest32107-21chrt12:23:013
3033499602,22cyclictest564-21chrt09:52:153
3033499602,21cyclictest3806-21chrt10:48:313
30327996041,13cyclictest8861-21swap07:40:312
3033499594,37cyclictest23473-21taskset08:25:253
3033499593,25cyclictest11937-21chrt11:00:313
3033499593,21cyclictest8175-21chrt07:40:103
3033499593,21cyclictest26390-21chrt08:35:053
3033499592,24cyclictest20053-21taskset09:31:573
3033499592,23cyclictest3517-21chrt11:38:283
3033499592,23cyclictest30891-21chrt08:48:543
3033499592,20cyclictest2851-21chrt07:22:283
3033499592,20cyclictest2851-21chrt07:22:283
3032799593,36cyclictest3707-21cat07:25:222
3032799592,24cyclictest39-21ksoftirqd/211:50:212
3033499588,8cyclictest0-21swapper/307:15:323
3033499582,24cyclictest31035-21chrt07:10:293
3033499582,23cyclictest1465-21chrt08:56:283
3033499582,23cyclictest1465-21chrt08:56:283
3033499582,22cyclictest12941-21chrt10:11:123
3033499573,16cyclictest0-21swapper/308:15:233
3033499572,20cyclictest29023-21chrt08:40:363
3032799573,40cyclictest13240-21fschecks_time07:55:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional