You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 22:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Dec 22, 2025 12:44:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1089021248,91sleep210909-21apt-get07:05:102
1089021248,91sleep210909-21apt-get07:05:102
12398991222,88cyclictest5828-21apt-get12:20:171
12398991193,102cyclictest14923-21apt-get09:55:131
10920210810,67sleep110921-21cpu07:05:171
10920210810,67sleep110921-21cpu07:05:171
123889910332,33cyclictest0-21swapper/009:05:300
1239899993,59cyclictest27611-21apt-get10:15:121
12398999559,13cyclictest181rcu_preempt10:45:281
12398999470,15cyclictest19271-21diskmemload12:14:411
12398999261,9cyclictest181rcu_preempt07:50:011
12398999260,14cyclictest181rcu_preempt08:20:171
1239899908,71cyclictest21282-21apt-get10:05:401
1239899908,71cyclictest21282-21apt-get10:05:401
1239899903,41cyclictest19271-21diskmemload10:55:221
12398998829,17cyclictest0-21swapper/110:35:161
12398998766,15cyclictest20849-21apt-key10:05:001
12398998766,15cyclictest20849-21apt-key10:05:001
12398998759,9cyclictest181rcu_preempt08:40:171
12398998759,9cyclictest181rcu_preempt08:40:171
12398998752,14cyclictest181rcu_preempt08:30:341
12398998752,14cyclictest181rcu_preempt08:30:341
12398998744,11cyclictest181rcu_preempt09:33:271
1239899872,61cyclictest5061-21runrttasks08:25:501
1239899872,61cyclictest5061-21runrttasks08:25:501
12398998665,14cyclictest12381-21cyclictest11:45:171
12398998648,13cyclictest181rcu_preempt11:05:471
12398998643,13cyclictest181rcu_preempt10:40:101
12398998643,13cyclictest181rcu_preempt10:40:101
12398998627,20cyclictest0-21swapper/109:00:331
12398998624,30cyclictest0-21swapper/111:10:171
12398998560,9cyclictest181rcu_preempt08:55:001
12398998560,9cyclictest181rcu_preempt08:55:001
12398998549,13cyclictest181rcu_preempt11:24:581
12398998547,14cyclictest181rcu_preempt09:41:171
12398998547,14cyclictest181rcu_preempt09:41:171
12398998541,15cyclictest181rcu_preempt08:35:251
12398998541,15cyclictest181rcu_preempt08:35:251
12398998538,16cyclictest181rcu_preempt12:27:321
1239899852,41cyclictest31-21ksoftirqd/107:10:411
1239899852,41cyclictest31-21ksoftirqd/107:10:411
12398998523,16cyclictest0-21swapper/111:00:271
1238899852,56cyclictest25334-21kworker/u19:2+flush-179:809:55:130
12398998450,9cyclictest181rcu_preempt09:13:571
12398998450,9cyclictest181rcu_preempt09:13:571
12398998447,12cyclictest181rcu_preempt08:19:141
12398998447,12cyclictest181rcu_preempt08:19:141
12398998427,11cyclictest0-21swapper/110:32:231
12388998461,15cyclictest20595-21ssh09:10:350
12388998461,15cyclictest20595-21ssh09:10:350
113482846,47sleep30-21swapper/307:05:313
113482846,47sleep30-21swapper/307:05:313
12404998365,12cyclictest201-21systemd-journal10:11:062
12404998365,12cyclictest201-21systemd-journal10:11:062
12398998354,10cyclictest181rcu_preempt09:35:121
12398998346,14cyclictest181rcu_preempt10:10:141
12398998346,14cyclictest181rcu_preempt10:10:141
12398998345,17cyclictest181rcu_preempt11:20:001
12398998343,11cyclictest181rcu_preempt07:20:321
12398998341,12cyclictest181rcu_preempt11:36:251
12398998333,8cyclictest181rcu_preempt09:52:061
12398998329,12cyclictest0-21swapper/112:15:091
12398998329,12cyclictest0-21swapper/112:15:091
12398998239,12cyclictest181rcu_preempt08:50:001
12398998239,12cyclictest181rcu_preempt08:50:001
12388998264,13cyclictest21027-21apt-key10:05:020
12388998264,13cyclictest21027-21apt-key10:05:020
12398998150,8cyclictest181rcu_preempt10:51:091
12398998141,10cyclictest181rcu_preempt08:00:361
12398998115,28cyclictest301ktimers/107:55:211
12388998111,40cyclictest15-21ksoftirqd/008:32:470
12388998111,40cyclictest15-21ksoftirqd/008:32:470
12388998111,31cyclictest15-21ksoftirqd/007:46:340
12404998064,10cyclictest4543-21ssh11:23:182
12398998025,19cyclictest0-21swapper/109:00:011
12388998011,27cyclictest15-21ksoftirqd/007:35:590
12398997958,7cyclictest181rcu_preempt09:45:371
12398997947,9cyclictest181rcu_preempt12:30:271
12398997947,12cyclictest181rcu_preempt09:20:151
12398997942,7cyclictest181rcu_preempt07:16:441
12398997929,18cyclictest0-21swapper/111:25:261
12398997928,14cyclictest0-21swapper/111:57:401
12388997945,28cyclictest346-21runrttasks10:14:580
12388997945,28cyclictest346-21runrttasks10:14:580
12388997921,30cyclictest15-21ksoftirqd/012:37:550
12398997844,13cyclictest181rcu_preempt09:15:351
12398997844,12cyclictest181rcu_preempt08:10:321
12398997835,12cyclictest181rcu_preempt10:24:331
12398997830,10cyclictest0-21swapper/108:10:011
12398997815,18cyclictest0-21swapper/107:41:131
12388997859,14cyclictest12381-21cyclictest11:40:200
12388997856,14cyclictest13668-21sshd07:12:060
12388997856,14cyclictest13668-21sshd07:12:060
12388997810,30cyclictest15-21ksoftirqd/008:22:020
12404997762,11cyclictest12381-21cyclictest09:43:552
12404997762,11cyclictest12381-21cyclictest09:43:552
12404997762,11cyclictest12381-21cyclictest09:14:232
12404997762,11cyclictest12381-21cyclictest09:14:232
12404997761,11cyclictest29655-21latency_hist12:05:012
12404997761,11cyclictest29655-21latency_hist12:05:012
12404997761,10cyclictest17000-21rm09:58:152
12404997754,16cyclictest14414-21sh08:54:582
12404997754,16cyclictest14414-21sh08:54:582
12404997743,24cyclictest30480-21modprobe08:04:582
12398997743,13cyclictest181rcu_preempt07:55:011
12398997737,9cyclictest181rcu_preempt09:26:281
12398997737,9cyclictest181rcu_preempt09:26:281
1238899779,43cyclictest0-21swapper/010:05:400
1238899779,43cyclictest0-21swapper/010:05:400
12388997761,10cyclictest14406-21systemd-run09:54:590
1150327711,41sleep011565-21cut07:05:410
1150327711,41sleep011565-21cut07:05:410
12404997662,10cyclictest12381-21cyclictest11:25:422
12404997661,10cyclictest12381-21cyclictest10:50:342
12404997659,10cyclictest11220-21systemd-run08:44:592
12404997659,10cyclictest11220-21systemd-run08:44:592
12404997646,21cyclictest13889-21cron07:14:582
12404997646,21cyclictest13889-21cron07:14:582
12404997610,29cyclictest24012-21cron07:44:582
12398997648,12cyclictest181rcu_preempt11:34:091
12398997648,12cyclictest181rcu_preempt11:34:091
12398997641,11cyclictest181rcu_preempt11:50:091
12398997637,13cyclictest181rcu_preempt07:29:391
12398997637,12cyclictest181rcu_preempt12:07:331
12398997636,13cyclictest181rcu_preempt07:30:231
12398997627,10cyclictest0-21swapper/111:40:091
12388997655,15cyclictest3360-21timerandwakeup09:35:320
12388997635,16cyclictest181rcu_preempt11:38:000
12388997635,10cyclictest3048-21ssh10:27:200
12388997635,10cyclictest3048-21ssh10:27:200
12388997617,30cyclictest15-21ksoftirqd/010:20:560
12404997559,10cyclictest4983-21/usr/sbin/munin08:26:072
12404997559,10cyclictest4983-21/usr/sbin/munin08:26:072
12404997557,12cyclictest1-21systemd08:20:072
12404997553,15cyclictest8855-21/usr/sbin/munin12:25:182
12398997550,17cyclictest0-21swapper/112:04:581
12398997550,17cyclictest0-21swapper/112:04:581
12398997547,8cyclictest181rcu_preempt12:35:351
12398997539,13cyclictest181rcu_preempt09:10:021
12398997523,23cyclictest0-21swapper/110:28:031
12398997523,23cyclictest0-21swapper/110:28:031
12388997548,19cyclictest1450-21latency_hist08:15:020
1238899754,27cyclictest15-21ksoftirqd/012:14:480
12388997515,29cyclictest15-21ksoftirqd/012:08:270
12388997513,32cyclictest15-21ksoftirqd/010:37:470
12404997456,11cyclictest15662-21unixbench_multi11:40:352
12404997455,11cyclictest10405-21kworker/u17:4-xprtiod10:40:152
12404997455,11cyclictest10405-21kworker/u17:4-xprtiod10:40:152
1238899748,33cyclictest15-21ksoftirqd/008:02:350
12388997451,16cyclictest0-21swapper/010:40:320
12388997451,16cyclictest0-21swapper/010:40:320
12388997444,8cyclictest181rcu_preempt07:17:040
12388997418,26cyclictest6211-21mailstats11:25:310
12388997414,36cyclictest15-21ksoftirqd/011:02:580
12388997410,41cyclictest15-21ksoftirqd/008:18:540
12388997410,41cyclictest15-21ksoftirqd/008:18:540
1240499733,52cyclictest25600-21latency_hist11:05:012
12404997326,8cyclictest0-21swapper/211:15:152
12404997326,10cyclictest27066-21apt-get10:15:062
12404997322,21cyclictest31962-21seq08:08:302
12404997322,13cyclictest2436-21apt-get09:35:092
12398997325,24cyclictest0-21swapper/107:35:181
1238899739,37cyclictest15-21ksoftirqd/009:41:460
1238899739,37cyclictest15-21ksoftirqd/009:41:460
12388997356,12cyclictest16683-21cstates10:50:150
1238899735,26cyclictest15-21ksoftirqd/011:15:310
12388997346,19cyclictest19271-21diskmemload10:55:320
12388997312,32cyclictest15-21ksoftirqd/008:50:380
12388997312,32cyclictest15-21ksoftirqd/008:50:380
12404997247,18cyclictest21228-21fschecks_count07:35:182
12388997253,13cyclictest2806-21fschecks_time12:15:210
12388997253,13cyclictest2806-21fschecks_time12:15:210
12388997253,12cyclictest28211-21fschecks_count10:16:170
12388997248,8cyclictest181rcu_preempt11:50:320
12388997234,19cyclictest181rcu_preempt11:57:550
12388997233,10cyclictest181rcu_preempt12:33:330
1238899722,51cyclictest30339-21ssh09:26:460
1238899722,51cyclictest30339-21ssh09:26:460
1238899722,38cyclictest15-21ksoftirqd/007:20:110
12388997215,34cyclictest15-21ksoftirqd/012:23:180
12388997213,27cyclictest15-21ksoftirqd/011:08:490
12388997211,35cyclictest15-21ksoftirqd/007:29:370
12388997210,36cyclictest15-21ksoftirqd/009:16:500
1240499718,27cyclictest12632-21ssh11:35:382
12404997153,11cyclictest14645-21ssh09:55:022
1240499714,52cyclictest971-21modprobe10:24:582
12404997125,8cyclictest0-21swapper/207:24:582
12404997115,21cyclictest19271-21diskmemload09:21:032
12404997113,19cyclictest0-21swapper/212:38:122
12404997110,24cyclictest0-21swapper/210:30:002
12404997110,24cyclictest0-21swapper/210:30:002
1238899719,30cyclictest15-21ksoftirqd/012:25:310
1238899715,30cyclictest15-21ksoftirqd/007:57:570
12388997114,23cyclictest2092-21seq09:34:550
12404997050,14cyclictest20948-21forks11:50:202
12404997026,9cyclictest0-21swapper/212:18:032
12404997026,9cyclictest0-21swapper/212:18:032
12404997013,15cyclictest10190-21ssh09:46:582
1238899703,39cyclictest15-21ksoftirqd/007:50:280
12388997025,29cyclictest15-21ksoftirqd/007:43:310
12388997014,31cyclictest15-21ksoftirqd/011:33:550
12388997014,31cyclictest15-21ksoftirqd/011:33:550
12404996951,12cyclictest7276-21sshd12:21:302
12404996920,17cyclictest19271-21diskmemload12:31:392
12404996917,13cyclictest30877-21ssh12:07:582
12404996910,24cyclictest0-21swapper/211:48:102
12404996910,21cyclictest0-21swapper/209:27:542
12404996910,21cyclictest0-21swapper/209:27:542
1238899698,30cyclictest15-21ksoftirqd/008:41:440
1238899698,30cyclictest15-21ksoftirqd/008:41:440
12388996953,10cyclictest5215-21ssh10:30:340
1238899694,29cyclictest15-21ksoftirqd/008:26:030
1238899694,29cyclictest15-21ksoftirqd/008:26:030
1238899693,29cyclictest15-21ksoftirqd/008:59:580
1238899692,26cyclictest9098-21memory09:45:260
1240499688,27cyclictest320-21dbus-daemon08:34:592
1240499688,27cyclictest320-21dbus-daemon08:34:592
12404996850,12cyclictest20395-21unixbench_singl07:30:362
12404996848,14cyclictest25871-21kworker/u18:0+events_unbound08:39:582
12404996848,14cyclictest25871-21kworker/u18:0+events_unbound08:39:582
12404996822,9cyclictest0-21swapper/210:30:212
12404996820,11cyclictest10067-21sh10:39:522
12404996818,39cyclictest0-21swapper/210:49:592
12404996818,12cyclictest14294-21df_inode07:15:062
12404996817,12cyclictest16315-21apt-get09:00:102
12404996814,17cyclictest19271-21diskmemload11:30:492
12404996814,17cyclictest19271-21diskmemload11:30:492
1238899686,31cyclictest15-21ksoftirqd/008:05:140
1238899685,25cyclictest15-21ksoftirqd/008:35:220
1238899685,25cyclictest15-21ksoftirqd/008:35:220
12388996847,7cyclictest181rcu_preempt09:21:050
1238899684,26cyclictest15-21ksoftirqd/011:21:590
1238899683,27cyclictest15-21ksoftirqd/011:46:410
12388996811,25cyclictest15-21ksoftirqd/008:47:120
12388996811,25cyclictest15-21ksoftirqd/008:47:120
1240499676,26cyclictest0-21swapper/210:02:562
1240499676,26cyclictest0-21swapper/210:02:562
1240499674,46cyclictest26176-21fschecks_time07:50:212
1238899673,30cyclictest15-21ksoftirqd/009:01:290
12388996713,32cyclictest15-21ksoftirqd/010:46:490
1240499666,27cyclictest21282-21apt-get10:05:162
1240499666,27cyclictest21282-21apt-get10:05:162
12404996651,10cyclictest12381-21cyclictest11:11:102
12404996631,25cyclictest27919-21grep07:55:242
1240499662,46cyclictest32273-21fschecks_time12:10:212
12404996618,11cyclictest26038-21perl11:05:222
12404996614,12cyclictest0-21swapper/209:34:582
12404996610,42cyclictest24995-21ssh11:55:552
12388996648,12cyclictest27841-21unixbench_multi12:00:360
12388996648,12cyclictest27841-21unixbench_multi12:00:360
12388996612,29cyclictest15-21ksoftirqd/011:10:090
12404996535,20cyclictest22318-21modprobe10:59:582
12404996517,11cyclictest2927-21systemd-run08:19:592
12404996517,11cyclictest2927-21systemd-run08:19:592
12388996544,14cyclictest0-21swapper/007:34:580
12409996420,32cyclictest0-21swapper/309:55:223
1240499647,27cyclictest23276-21runrttasks09:15:282
1240499646,23cyclictest18981-21seq09:06:272
1240499645,25cyclictest346-21runrttasks08:57:242
12404996448,9cyclictest0-21swapper/208:45:342
12404996448,9cyclictest0-21swapper/208:45:342
1240499637,19cyclictest0-21swapper/207:25:452
1240499633,44cyclictest32367-21cpu08:10:162
1240499632,43cyclictest25815-21latency_hist07:50:012
12409996217,30cyclictest451rcuc/308:25:463
12409996217,30cyclictest451rcuc/308:25:463
1240999522,39cyclictest13528-21taskset10:45:123
12409995138,7cyclictest47092sleep308:25:003
1240999513,36cyclictest22414-21taskset09:15:013
1240999513,36cyclictest22414-21taskset09:15:013
1240999502,35cyclictest19760-21chrt10:55:173
1240999492,34cyclictest28824-21chrt09:25:143
1240999492,34cyclictest28824-21chrt09:25:143
1240999492,33cyclictest23687-21taskset11:55:013
1240999492,33cyclictest14137-21taskset09:54:233
1240999483,30cyclictest11606-21taskset12:29:583
1240999482,34cyclictest6039-21taskset12:20:423
1240999482,34cyclictest14465-21taskset07:15:093
1240999482,33cyclictest15016-21taskset11:40:223
1240999473,31cyclictest29301-21chrt10:17:083
1240999473,29cyclictest7561-21chrt08:30:353
1240999473,29cyclictest7561-21chrt08:30:353
1240999473,29cyclictest5173-21chrt10:30:333
12409994732,10cyclictest451rcuc/310:11:093
12409994732,10cyclictest451rcuc/310:11:093
1240999472,33cyclictest788-21chrt10:24:303
235152463,7sleep30-21swapper/309:15:353
1240999463,30cyclictest14874-21chrt08:55:213
1240999463,29cyclictest2118-21chrt10:25:353
1240999463,29cyclictest2118-21chrt10:25:353
1240999463,29cyclictest0-21swapper/310:05:383
1240999463,29cyclictest0-21swapper/310:05:383
1240999463,28cyclictest26156-21taskset09:20:243
12409994630,10cyclictest451rcuc/308:14:173
1240999462,31cyclictest32458-21taskset12:10:253
1240999462,30cyclictest4435-21chrt11:23:003
1240999462,30cyclictest17908-21chrt09:05:093
1240999453,30cyclictest30332-21chrt12:06:283
1240999453,29cyclictest14386-21taskset12:34:193
1240999453,27cyclictest22440-21chrt07:39:593
1240999452,32cyclictest18093-21chrt07:25:213
1240999452,31cyclictest19832-21chrt10:02:333
1240999452,31cyclictest19832-21chrt10:02:333
1240999452,30cyclictest2931-21taskset12:15:233
1240999452,30cyclictest2931-21taskset12:15:233
1240999452,27cyclictest7921-21taskset09:44:063
1240999452,27cyclictest7921-21taskset09:44:063
1240999444,27cyclictest9695-21chrt10:38:523
12409994428,11cyclictest451rcuc/311:00:373
1240999442,30cyclictest19152-21taskset11:46:343
1240999442,29cyclictest10744-21chrt11:33:383
1240999442,29cyclictest10744-21chrt11:33:383
1240999442,28cyclictest24099-21chrt07:45:013
1240999442,28cyclictest1369-21chrt11:18:023
1240999442,26cyclictest29778-21taskset12:05:023
1240999442,26cyclictest29778-21taskset12:05:023
1240999433,30cyclictest24301-21chrt07:45:133
1240999433,29cyclictest13394-21taskset07:10:353
1240999433,29cyclictest13394-21taskset07:10:353
1240999432,29cyclictest16246-21chrt12:36:063
1240999432,28cyclictest11735-21chrt08:45:213
1240999432,28cyclictest11735-21chrt08:45:213
1240999423,29cyclictest32091-21taskset08:09:583
1240999423,26cyclictest12686-21taskset10:43:383
1240999423,26cyclictest12686-21taskset10:43:383
1240999422,27cyclictest12939-21chrt11:36:173
1240999414,27cyclictest28888-21taskset07:59:583
1240999413,29cyclictest26106-21taskset11:58:583
1240999413,28cyclictest29476-21taskset08:00:233
1240999413,27cyclictest31793-21taskset09:30:043
1240999413,27cyclictest30481-21taskset11:12:153
1240999413,23cyclictest18139-21chrt10:51:533
1240999412,27cyclictest9936-21chrt09:46:173
12409994120,14cyclictest273162chrt07:55:013
1240999404,27cyclictest16817-21taskset09:00:273
1240999403,27cyclictest2952-21taskset09:35:233
1240999403,26cyclictest1638-21chrt08:15:173
1240999403,26cyclictest1638-21chrt08:15:173
1240999403,26cyclictest13054-21chrt08:50:073
1240999403,26cyclictest13054-21chrt08:50:073
12409994026,9cyclictest451rcuc/311:05:063
1240999402,29cyclictest20579-21taskset07:32:263
1240999402,26cyclictest5611-21chrt11:25:153
1240999393,27cyclictest16396-21chrt07:20:193
1240999392,28cyclictest10348-21taskset08:40:273
1240999392,28cyclictest10348-21taskset08:40:273
1240999383,27cyclictest8717-21taskset08:35:293
1240999383,27cyclictest8717-21taskset08:35:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional