You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 18:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Wed Jun 03, 2026 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
126639912758,60cyclictest0-21swapper/011:54:530
126639912758,60cyclictest0-21swapper/011:54:530
12519211714,67sleep112554-21munin-run06:59:511
12663991123,29cyclictest12537-21latency_hist06:59:540
126639911145,61cyclictest0-21swapper/010:05:120
126639911038,10cyclictest26507-21ntp_kernel_pll_09:10:180
12663991094,27cyclictest15053-21find09:45:240
126639910829,7cyclictest0-21swapper/009:50:150
12663991063,29cyclictest8326-21taskset12:16:330
126809910552,46cyclictest544-21cron11:39:502
12670991032,23cyclictest16097-21seq08:46:461
126639910335,8cyclictest0-21swapper/008:40:140
126639910332,8cyclictest0-21swapper/007:10:180
12663991032,51cyclictest15727-21ssh11:35:070
12663991022,42cyclictest6443-21latency_hist11:19:520
126639910135,8cyclictest0-21swapper/010:25:210
126639910135,8cyclictest0-21swapper/010:25:210
12663991013,41cyclictest31172-21mailstats12:00:250
12663991013,33cyclictest24042-21ssh10:54:500
12663991013,33cyclictest24042-21ssh10:54:500
126639910119,26cyclictest4539-21open_inodes12:10:210
126639910039,54cyclictest0-21swapper/007:40:070
126639910035,58cyclictest30659-21kernelversion11:05:150
126639910035,58cyclictest30659-21kernelversion11:05:150
12663991002,30cyclictest25140-21switchtime10:55:260
1266399994,32cyclictest5962-21latency_hist10:24:530
12663999938,55cyclictest23931-21sh10:00:210
12663999937,56cyclictest21847-21ssh09:02:460
12663999934,10cyclictest0-21swapper/011:59:190
1266399993,30cyclictest19388-21unixbench_multi10:45:270
12663999932,10cyclictest0-21swapper/011:00:280
12663999932,10cyclictest0-21swapper/011:00:280
12663999931,8cyclictest0-21swapper/008:39:510
12663999931,8cyclictest0-21swapper/008:39:510
1266399992,25cyclictest10544-21ssh10:30:550
12663999919,24cyclictest31981-21cat09:20:060
12663999919,24cyclictest31981-21cat09:20:060
1266399984,28cyclictest27857-21cpuspeed07:45:070
1266399983,37cyclictest25507-21grep07:35:270
1266399983,32cyclictest12626-21/usr/sbin/munin11:30:240
1266399983,28cyclictest9159-21mailstats09:35:230
1266399982,31cyclictest32411-21apt-key10:14:540
12663999813,29cyclictest10733-21wc12:20:260
1267099973,27cyclictest16018-21seq07:07:141
1267099973,27cyclictest16018-21seq07:07:141
1267099972,30cyclictest25716-21ssh10:04:091
1266399974,36cyclictest6676-21/usr/sbin/munin11:20:130
1266399973,31cyclictest12733-21/usr/sbin/munin12:25:220
1266399972,42cyclictest24065-21runrttasks11:49:380
1266399972,31cyclictest32514-21/usr/sbin/munin10:16:320
1266399972,31cyclictest32514-21/usr/sbin/munin10:16:320
1266399972,30cyclictest867-21cat08:00:240
12663999711,27cyclictest3002-21grep09:25:180
12680999656,35cyclictest12653-21cyclictest11:10:272
1266399963,33cyclictest21365-21unixbench_multi09:55:270
1266399963,31cyclictest10354-21grep08:30:110
1266399963,31cyclictest10354-21grep08:30:110
1266399963,29cyclictest352-21runrttasks09:34:370
1266399963,29cyclictest352-21runrttasks09:34:370
12663999632,8cyclictest0-21swapper/010:40:050
12663999629,9cyclictest0-21swapper/010:38:240
12680999576,9cyclictest0-21swapper/208:09:532
12663999535,7cyclictest0-21swapper/008:20:540
1266399953,35cyclictest21288-21latency_hist11:44:520
1266399953,32cyclictest21926-21proc_pri07:25:230
1266399953,32cyclictest21926-21proc_pri07:25:230
1266399953,32cyclictest14444-21munin-node11:09:590
1266399953,31cyclictest2020-21fschecks_count08:05:120
1266399953,30cyclictest9632-21/usr/sbin/munin11:25:150
1266399953,30cyclictest9632-21/usr/sbin/munin11:25:150
12663999529,7cyclictest0-21swapper/009:09:520
1266399952,34cyclictest31762-21switchtime07:55:250
1266399952,34cyclictest31762-21switchtime07:55:250
12663999518,22cyclictest23073-21cstates07:30:080
12663999518,22cyclictest23073-21cstates07:30:080
12663999511,27cyclictest17610-21grep08:50:270
12680999471,13cyclictest0-21swapper/207:14:552
12670999447,39cyclictest462-21fschecks_time08:00:121
1267099942,73cyclictest11943-21apt-get10:35:041
1266399944,31cyclictest13352-21ntp_kernel_pll_07:00:200
1266399943,31cyclictest18926-21cat07:15:260
1266399943,30cyclictest29408-21/usr/sbin/munin07:50:180
1266399943,30cyclictest29408-21/usr/sbin/munin07:50:180
12663999431,7cyclictest0-21swapper/009:15:080
12663999424,12cyclictest12416-21grep09:40:270
12663999424,12cyclictest12416-21grep09:40:270
12680999343,44cyclictest19606-21diskmemload09:39:052
12680999338,48cyclictest7698-21sh11:20:262
12670999348,38cyclictest1007-21latency10:16:351
12670999348,38cyclictest1007-21latency10:16:351
1266399934,34cyclictest15853-21grep07:05:300
1266399934,34cyclictest15853-21grep07:05:300
1266399933,34cyclictest15969-21grep08:45:270
1266399933,30cyclictest6587-21apt-key08:19:530
1266399932,31cyclictest879-21/usr/sbin/munin12:05:170
12680999274,11cyclictest0-21swapper/211:19:532
12680999273,10cyclictest0-21swapper/208:04:532
1266399924,30cyclictest3476-21/usr/sbin/munin08:10:140
1266399923,33cyclictest10052-21latency_hist08:29:530
1266399923,31cyclictest352-21runrttasks08:55:210
12680999172,9cyclictest0-21swapper/207:44:522
12680999145,39cyclictest0-21swapper/211:45:172
1266399913,29cyclictest19724-21/usr/sbin/munin07:20:230
12680999072,11cyclictest0-21swapper/210:25:262
12680999072,11cyclictest0-21swapper/210:25:262
12680999049,33cyclictest0-21swapper/207:34:522
12680999049,33cyclictest0-21swapper/207:34:522
12680999033,49cyclictest0-21swapper/211:30:222
12670999046,36cyclictest0-21swapper/111:55:261
12670999039,25cyclictest0-21swapper/109:30:071
12670999039,25cyclictest0-21swapper/109:30:071
12680998971,8cyclictest0-21swapper/211:29:532
12680998971,8cyclictest0-21swapper/211:29:532
12680998938,42cyclictest0-21swapper/209:44:532
12680998938,42cyclictest0-21swapper/209:44:532
12680998928,56cyclictest0-21swapper/208:20:492
12680998872,8cyclictest0-21swapper/207:29:542
12680998867,13cyclictest12653-21cyclictest10:09:532
12680998841,39cyclictest0-21swapper/212:00:092
1267099882,37cyclictest18601-21/usr/sbin/munin08:55:151
12680998770,8cyclictest0-21swapper/207:55:142
12680998770,8cyclictest0-21swapper/207:55:142
12680998739,40cyclictest0-21swapper/209:29:522
12680998734,45cyclictest0-21swapper/212:25:112
12670998737,26cyclictest23805-21kworker/u17:2-xprtiod11:29:161
12670998737,26cyclictest23805-21kworker/u17:2-xprtiod11:29:161
1267099872,6cyclictest0-21swapper/109:00:021
12680998661,13cyclictest0-21swapper/207:24:542
12680998648,34cyclictest12653-21cyclictest10:30:202
12680998641,39cyclictest0-21swapper/210:54:532
12680998641,39cyclictest0-21swapper/210:54:532
12680998640,39cyclictest0-21swapper/209:24:542
12680998640,39cyclictest0-21swapper/209:24:542
1268099863,37cyclictest378-21ssh10:15:532
1268099863,37cyclictest378-21ssh10:15:532
12680998630,47cyclictest0-21swapper/207:50:172
12680998630,47cyclictest0-21swapper/207:50:172
12680998627,9cyclictest0-21swapper/209:59:222
1267099865,58cyclictest0-21swapper/111:46:201
12680998536,42cyclictest0-21swapper/212:13:362
12680998530,49cyclictest0-21swapper/209:09:522
12680998527,8cyclictest0-21swapper/212:05:002
12680998527,8cyclictest0-21swapper/210:58:452
12670998540,39cyclictest0-21swapper/108:19:541
1267099853,7cyclictest0-21swapper/112:25:051
1267099852,24cyclictest19676-21snmpd11:01:171
1267099852,24cyclictest19676-21snmpd11:01:171
12670998519,47cyclictest12410-21seq12:24:471
12680998435,42cyclictest0-21swapper/211:02:382
12680998435,42cyclictest0-21swapper/211:02:382
1268099843,28cyclictest8543-21sort08:25:022
12680998429,44cyclictest0-21swapper/209:52:412
12680998425,11cyclictest0-21swapper/211:50:062
12680998425,11cyclictest0-21swapper/211:50:062
1268099838,26cyclictest14753-21switchtime07:05:022
1268099838,26cyclictest14753-21switchtime07:05:022
1268099833,33cyclictest30773-21netstat11:05:172
1268099833,33cyclictest30773-21netstat11:05:172
1268099833,28cyclictest19735-21latency_hist08:59:532
1268099833,28cyclictest12438-21ls12:24:492
12680998321,8cyclictest0-21swapper/209:46:122
12670998346,30cyclictest0-21swapper/109:35:181
1267099833,66cyclictest24513-21ssh11:50:041
1267099833,66cyclictest24513-21ssh11:50:041
1267099833,21cyclictest32312-21runrttasks07:58:301
1267099833,21cyclictest32312-21runrttasks07:58:301
1267099832,37cyclictest16518-21ssh11:35:251
1268499823,35cyclictest17324-21chrt09:50:003
1268099824,26cyclictest29457-21ssh09:15:202
12680998234,43cyclictest0-21swapper/207:45:262
1268099823,28cyclictest3035-21/usr/sbin/munin10:20:172
1268099823,28cyclictest26913-21unixbench_multi09:10:262
12680998232,41cyclictest0-21swapper/210:37:432
1268099823,23cyclictest12855-21apt-get07:00:072
1267099825,60cyclictest18135-21apt-get07:15:041
12670998242,34cyclictest19331-21sh11:40:231
1267099823,36cyclictest13331-21/usr/sbin/munin11:30:231
1267099823,33cyclictest21797-21/usr/sbin/munin10:50:161
1267099823,33cyclictest21797-21/usr/sbin/munin10:50:161
1268099814,27cyclictest10809-21taskset08:30:232
1268099814,27cyclictest10809-21taskset08:30:232
1268099813,27cyclictest5717-21fschecks_time09:30:122
1268099813,27cyclictest5717-21fschecks_time09:30:122
12680998132,43cyclictest0-21swapper/211:40:062
12680998125,9cyclictest0-21swapper/208:35:242
12680998125,9cyclictest0-21swapper/208:35:242
1268099812,27cyclictest24176-21awk10:00:262
1268099812,22cyclictest32344-21latency_hist10:14:542
12680998120,8cyclictest0-21swapper/211:59:512
12670998145,31cyclictest12653-21cyclictest11:11:441
1267099813,7cyclictest0-21swapper/112:10:011
1267099812,6cyclictest0-21swapper/108:20:411
1267099812,36cyclictest8933-21taskset11:23:411
1267099812,34cyclictest3678-21modprobe11:14:531
1267099812,33cyclictest18734-21sh09:51:151
1268499804,37cyclictest6962-21chrt08:20:123
1268499804,35cyclictest759-21taskset12:04:523
1268499803,39cyclictest15596-21chrt07:05:273
1268499803,39cyclictest15596-21chrt07:05:273
1268499803,38cyclictest30835-21chrt11:05:193
1268499803,38cyclictest30835-21chrt11:05:193
1268099803,34cyclictest15393-21grep08:45:162
1268099803,29cyclictest19671-21latency_hist07:19:542
1268099803,28cyclictest6819-21latency_hist08:19:542
1268099803,27cyclictest20220-21tune2fs09:00:122
1268099803,25cyclictest13415-21/usr/sbin/munin08:40:082
12680998027,7cyclictest0-21swapper/210:41:282
1267099805,66cyclictest900-21apt-get12:05:021
1267099803,33cyclictest3183-21cstates10:20:071
12670998033,37cyclictest0-21swapper/108:25:081
1267099802,36cyclictest24347-21runrttasks10:55:021
1267099802,34cyclictest5728-21apt-get10:24:551
1267099802,34cyclictest5728-21apt-get10:24:551
12680997963,9cyclictest0-21swapper/208:54:522
1268099793,28cyclictest22705-21sendmail07:29:502
12680997923,8cyclictest0-21swapper/210:45:052
1268099792,24cyclictest7881-21grep12:15:282
1267099795,65cyclictest11792-21apt-get08:35:021
1267099795,65cyclictest11792-21apt-get08:35:021
12670997937,33cyclictest0-21swapper/109:20:061
12670997937,33cyclictest0-21swapper/109:20:061
12670997935,36cyclictest5074-21latency_hist09:29:531
1267099793,32cyclictest21403-21unixbench_singl09:55:281
1267099792,36cyclictest10741-21/usr/sbin/munin08:30:221
1267099792,36cyclictest10741-21/usr/sbin/munin08:30:221
1267099792,35cyclictest14115-21latency_hist09:44:531
1267099792,35cyclictest14115-21latency_hist09:44:531
1267099792,33cyclictest16528-21apt-get07:10:021
1267099792,32cyclictest32520-21ssh12:03:221
1268099782,24cyclictest24825-21tune2fs07:35:132
1267099783,6cyclictest0-21swapper/110:10:041
1267099783,31cyclictest9854-21ntpq10:30:181
1267099783,29cyclictest27949-21/usr/sbin/munin07:45:101
1267099783,26cyclictest30744-21sh11:05:171
1267099783,26cyclictest30744-21sh11:05:171
1267099782,36cyclictest26862-21ntpq10:05:191
1267099782,34cyclictest7063-21cat12:15:121
1267099782,34cyclictest26871-21/usr/sbin/munin09:10:251
1268099773,25cyclictest3476-21/usr/sbin/munin08:10:182
1267099773,6cyclictest0-21swapper/107:25:071
1267099773,6cyclictest0-21swapper/107:25:071
1267099773,33cyclictest25145-21/usr/sbin/munin07:35:231
1267099773,32cyclictest20528-21/usr/sbin/munin07:20:261
1267099772,7cyclictest0-21swapper/109:45:021
1267099772,33cyclictest17798-21runrttasks10:44:041
1267099772,29cyclictest18496-21awk10:45:061
1267099763,6cyclictest0-21swapper/107:30:031
1267099763,6cyclictest0-21swapper/107:30:031
12670997636,32cyclictest0-21swapper/108:14:501
12670997633,36cyclictest0-21swapper/109:16:291
1267099763,28cyclictest23065-21cstates09:05:091
1267099762,33cyclictest13847-21/usr/sbin/munin08:40:171
1267099762,31cyclictest29403-21taskset07:49:561
1267099762,31cyclictest29403-21taskset07:49:561
1267099762,29cyclictest13661-21sed07:00:271
1267099762,28cyclictest2614-21wc08:05:261
1267099752,29cyclictest18008-21awk08:54:501
1267099752,26cyclictest26691-21tr07:40:191
117882756,17sleep2181rcu_preempt06:55:322
144722580,8sleep30-21swapper/312:27:143
1268499573,40cyclictest15245-21taskset09:45:263
1268499563,41cyclictest6339-21chrt10:25:113
1268499563,41cyclictest6339-21chrt10:25:113
1268499563,41cyclictest28751-21chrt09:14:533
1268499563,40cyclictest29196-21chrt11:02:473
1268499563,40cyclictest29196-21chrt11:02:473
1268499563,39cyclictest17513-21taskset08:50:263
1268499563,38cyclictest13351-21chrt10:36:043
1268499555,37cyclictest6532-21chrt12:14:493
1268499554,38cyclictest5068-21taskset09:29:493
1268499554,38cyclictest21409-21taskset09:55:283
1268499554,37cyclictest9846-21taskset10:30:183
1268499545,37cyclictest19101-21chrt07:15:283
1268499544,38cyclictest27967-21taskset11:55:163
1268499544,37cyclictest26032-21chrt10:04:523
1268499544,36cyclictest16354-21chrt10:40:253
1268499543,40cyclictest10058-21taskset11:25:133
1268499543,40cyclictest10058-21taskset11:25:133
1268499543,38cyclictest32754-21taskset10:15:383
1268499543,38cyclictest32754-21taskset10:15:383
1268499543,38cyclictest11928-21chrt12:23:243
1268499542,40cyclictest13543-21taskset08:40:093
1268499542,39cyclictest27618-21chrt10:06:063
1268499534,36cyclictest3846-21taskset10:20:253
1268499534,35cyclictest19170-21taskset10:45:243
1268499533,38cyclictest18561-21chrt11:40:003
1268499533,38cyclictest13329-21taskset09:42:553
1268499533,38cyclictest13329-21taskset09:42:553
1268499533,37cyclictest29471-21chrt10:10:073
1268499533,37cyclictest23018-21chrt09:05:083
1268499533,37cyclictest13834-21taskset07:00:293
1268499532,38cyclictest25037-21chrt11:50:183
1268499532,38cyclictest25037-21chrt11:50:183
1268499524,35cyclictest24142-21chrt11:49:493
1268499523,35cyclictest2021-21taskset11:10:543
1268499523,35cyclictest10462-21chrt08:30:133
1268499523,35cyclictest10462-21chrt08:30:133
1268499522,37cyclictest27938-21chrt07:45:093
1268499514,34cyclictest30691-21chrt09:17:253
1268499514,34cyclictest12299-21taskset08:35:193
1268499514,34cyclictest12299-21taskset08:35:193
1268499514,33cyclictest6115-21chrt11:19:053
1268499513,35cyclictest29531-21chrt07:50:073
1268499513,35cyclictest29531-21chrt07:50:073
1268499513,33cyclictest22969-21chrt07:30:023
1268499513,33cyclictest22969-21chrt07:30:023
1268499504,32cyclictest14453-21taskset11:32:273
1268499504,31cyclictest8704-21taskset08:25:103
1268499503,35cyclictest20019-21chrt07:20:143
1268499503,35cyclictest1931-21chrt09:24:263
1268499503,35cyclictest1931-21chrt09:24:263
1268499503,34cyclictest8389-21chrt09:35:003
1268499502,36cyclictest21942-21chrt10:50:193
1268499502,36cyclictest21942-21chrt10:50:193
1268499502,35cyclictest16766-21taskset07:10:133
1268499502,34cyclictest8590-21chrt11:22:423
1268499494,32cyclictest7374-21taskset12:15:183
1268499494,29cyclictest917-21kworker/3:1+events09:00:113
1268499493,33cyclictest26509-21chrt07:40:143
1268499493,33cyclictest15618-21taskset11:35:023
1268499492,34cyclictest6514-21chrt08:19:523
1268499483,30cyclictest25946-21chrt07:39:503
1268499473,32cyclictest22872-21chrt07:29:543
1268499473,32cyclictest22872-21chrt07:29:543
1268499472,35cyclictest2406-21chrt12:06:353
120962479,22sleep30-21swapper/306:55:373
1268499464,30cyclictest15364-21chrt08:45:153
1268499463,30cyclictest5424-21taskset09:30:023
1268499463,30cyclictest5424-21taskset09:30:023
1268499463,30cyclictest24853-21chrt10:55:173
1268499454,29cyclictest19550-21chrt08:59:293
1268499453,29cyclictest3498-21chrt08:10:043
1268499453,29cyclictest32703-21taskset08:00:033
1268499444,28cyclictest31049-21taskset07:55:053
1268499444,28cyclictest31049-21taskset07:55:053
1268499443,28cyclictest3031-21taskset08:07:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional