You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 19:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Wed Sep 23, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12141211313,74sleep012144-21cron07:08:370
12141211313,74sleep012144-21cron07:08:370
12301991123,82cyclictest13056-21apt-get12:33:522
123019910380,15cyclictest1-21systemd11:59:472
1230199982,84cyclictest2312-21apt-get09:33:562
1228199979,76cyclictest13136-21ssh12:34:000
12281999346,8cyclictest181rcu_preempt10:53:580
12281999346,8cyclictest181rcu_preempt10:53:580
12281999125,57cyclictest707-21ssh10:24:010
1230199893,67cyclictest343-21runrttasks11:49:032
12286998971,12cyclictest13917-21ssh12:34:241
12281998759,8cyclictest181rcu_preempt11:20:090
12281998730,12cyclictest181rcu_preempt10:43:590
12281998730,12cyclictest181rcu_preempt10:43:590
12301998663,15cyclictest200-21systemd-journal09:51:472
12286998666,14cyclictest21843-21sh11:53:591
12286998666,14cyclictest21843-21sh11:53:591
1228199864,61cyclictest18710-21apt-get11:48:520
12281998565,14cyclictest25213-21rm09:18:370
12281998565,14cyclictest25213-21rm09:18:370
12281998556,21cyclictest7069-21kernelversion10:34:040
1230799845,59cyclictest0-21swapper/310:23:523
12301998456,22cyclictest29617-21unixbench_singl09:24:152
12301998452,19cyclictest181rcu_preempt11:44:132
12281998465,13cyclictest358-21ntpd11:09:180
12281998443,11cyclictest181rcu_preempt09:43:380
12301998361,12cyclictest0-21swapper/211:38:392
12301998336,13cyclictest181rcu_preempt12:33:412
12286998367,10cyclictest15976-21ssh09:55:481
12286998367,10cyclictest15976-21ssh09:55:481
12281998348,10cyclictest181rcu_preempt12:20:460
12301998259,16cyclictest22177-21sh11:54:052
12301998259,16cyclictest22177-21sh11:54:052
12301998247,10cyclictest181rcu_preempt09:53:502
12301998247,10cyclictest181rcu_preempt09:53:502
12301998233,9cyclictest181rcu_preempt08:39:112
12286998212,37cyclictest15912-21fschecks_count10:49:021
12281998246,8cyclictest181rcu_preempt08:34:010
12281998240,8cyclictest181rcu_preempt10:22:020
12301998136,14cyclictest181rcu_preempt12:14:002
12286998164,11cyclictest8410-21ssh09:43:541
12286998164,11cyclictest8410-21ssh09:43:541
12281998158,18cyclictest12271-21cyclictest07:08:540
12281998142,8cyclictest181rcu_preempt11:46:070
12281998140,9cyclictest0-21swapper/007:23:390
12301998051,20cyclictest26585-21latency_hist10:13:402
12301998050,18cyclictest27107-21latency_hist11:08:402
12301998050,14cyclictest6441-21ssh12:23:072
1230199804,63cyclictest630-21apt-get10:24:012
12301998044,11cyclictest181rcu_preempt10:58:522
12301998041,8cyclictest181rcu_preempt10:01:322
12301998011,37cyclictest9324-21latency_hist11:33:392
12301998011,37cyclictest9324-21latency_hist11:33:392
12281998047,8cyclictest181rcu_preempt10:49:170
12281998038,22cyclictest0-21swapper/008:25:500
12281998038,11cyclictest0-21swapper/007:49:160
12281998036,8cyclictest181rcu_preempt11:18:410
12301997918,36cyclictest0-21swapper/212:24:122
12286997964,10cyclictest12463-21kworker/u17:1-nfsiod10:09:071
12286997960,12cyclictest200-21systemd-journal10:15:461
12286997960,12cyclictest200-21systemd-journal10:15:461
12286997947,21cyclictest13243-21grep07:09:121
12281997953,19cyclictest32103-21kworker/u18:1+events_unbound11:58:370
12281997953,19cyclictest32103-21kworker/u18:1+events_unbound11:58:370
12301997855,15cyclictest13262-21ntp_offset10:44:062
12301997855,15cyclictest13262-21ntp_offset10:44:062
12301997815,24cyclictest0-21swapper/212:05:502
12286997861,10cyclictest19200-21diskmemload10:43:161
1228199788,40cyclictest15-21ksoftirqd/009:04:020
12281997855,15cyclictest343-21runrttasks09:56:150
12281997855,15cyclictest343-21runrttasks09:56:150
12281997844,11cyclictest181rcu_preempt09:19:180
12281997843,11cyclictest181rcu_preempt08:18:540
12281997837,12cyclictest181rcu_preempt11:39:120
12301997748,9cyclictest181rcu_preempt07:08:532
12301997738,13cyclictest181rcu_preempt10:28:512
12301997734,17cyclictest181rcu_preempt11:28:042
12286997761,10cyclictest14823-21fschecks_time08:54:021
12286997761,10cyclictest14823-21fschecks_time08:54:021
12281997755,14cyclictest19200-21diskmemload09:59:160
12281997743,11cyclictest181rcu_preempt10:31:520
12301997653,9cyclictest39-21ksoftirqd/208:28:582
12301997653,9cyclictest39-21ksoftirqd/208:28:582
12301997652,8cyclictest181rcu_preempt09:40:412
12301997644,25cyclictest16711-21unixbench_singl10:49:182
12301997643,14cyclictest181rcu_preempt09:13:402
12301997620,11cyclictest0-21swapper/208:58:522
12281997642,8cyclictest181rcu_preempt12:32:020
12281997631,18cyclictest181rcu_preempt11:27:090
12307997518,40cyclictest0-21swapper/310:03:583
12301997551,15cyclictest17686-21latency_hist07:23:412
12301997540,12cyclictest181rcu_preempt07:33:402
12301997536,8cyclictest181rcu_preempt09:03:502
12286997558,11cyclictest200-21systemd-journal12:23:231
12286997517,19cyclictest21786-21ps07:34:101
12286997517,19cyclictest21786-21ps07:34:101
12281997542,8cyclictest181rcu_preempt08:58:560
12281997540,10cyclictest181rcu_preempt08:30:480
12281997540,10cyclictest181rcu_preempt08:30:480
12281997537,13cyclictest12271-21cyclictest09:08:460
12281997535,8cyclictest181rcu_preempt09:45:280
12281997535,8cyclictest181rcu_preempt09:45:280
12281997534,8cyclictest181rcu_preempt11:02:470
12301997445,9cyclictest1385-21ssh09:32:362
12301997445,9cyclictest1385-21ssh09:32:362
12301997439,14cyclictest181rcu_preempt11:14:042
12301997436,16cyclictest181rcu_preempt11:09:002
12301997427,16cyclictest181rcu_preempt11:19:422
12286997452,16cyclictest9722-21latency_hist12:28:391
12286997418,23cyclictest31620-21unixbench_singl12:09:181
12286997417,16cyclictest0-21swapper/109:28:501
12286997417,16cyclictest0-21swapper/109:28:501
12281997444,8cyclictest181rcu_preempt07:39:000
12281997444,8cyclictest181rcu_preempt07:39:000
12281997441,8cyclictest181rcu_preempt08:13:560
12281997437,11cyclictest181rcu_preempt09:24:060
1230799736,41cyclictest0-21swapper/310:09:033
12301997357,10cyclictest19608-21ssh10:54:382
12301997357,10cyclictest19608-21ssh10:54:382
12301997350,12cyclictest0-21swapper/207:58:412
12301997332,13cyclictest181rcu_preempt08:54:152
12301997332,13cyclictest181rcu_preempt08:54:152
12301997328,34cyclictest9545-21cat08:38:392
12301997311,30cyclictest6955-21/usr/sbin/munin10:34:012
12286997354,12cyclictest10979-21kworker/u17:1-nfsiod12:30:321
12286997353,13cyclictest343-21runrttasks11:39:011
12286997327,34cyclictest31613-21grep08:04:141
12286997327,34cyclictest31613-21grep08:04:141
12286997316,19cyclictest20809-21sh10:04:251
12286997310,24cyclictest17050-21timerwakeupswit08:59:141
12281997342,8cyclictest181rcu_preempt10:05:220
12281997337,11cyclictest181rcu_preempt10:39:140
12281997336,8cyclictest181rcu_preempt12:04:050
12281997332,8cyclictest181rcu_preempt07:44:120
12281997325,39cyclictest15112-21sed08:54:080
12281997325,39cyclictest15112-21sed08:54:080
118952734,13sleep30-21swapper/307:05:223
118952734,13sleep30-21swapper/307:05:223
1230199726,28cyclictest39-21ksoftirqd/210:08:412
12301997250,16cyclictest25229-21sendmail09:18:382
12301997250,16cyclictest25229-21sendmail09:18:382
1230199722,35cyclictest25802-21fschecks_count09:19:002
12301997215,29cyclictest12271-21cyclictest07:49:232
12286997255,11cyclictest200-21systemd-journal09:25:471
12286997251,15cyclictest1280-21fschecks_count12:14:001
12281997251,12cyclictest24587-21diskmemload10:09:270
12281997245,8cyclictest181rcu_preempt09:29:410
12281997245,8cyclictest181rcu_preempt09:29:410
12281997241,8cyclictest181rcu_preempt10:15:540
12281997241,8cyclictest181rcu_preempt10:15:540
12281997238,10cyclictest12271-21cyclictest07:30:090
12281997232,8cyclictest181rcu_preempt07:34:120
12281997232,8cyclictest181rcu_preempt07:34:120
12281997229,11cyclictest181rcu_preempt08:49:300
1160327210,25sleep20-21swapper/207:04:242
1160327210,25sleep20-21swapper/207:04:242
12301997148,16cyclictest0-21swapper/208:09:142
12301997148,11cyclictest181rcu_preempt12:09:032
12301997147,18cyclictest27582-21mailstats10:14:112
12301997147,18cyclictest27582-21mailstats10:14:112
12301997123,36cyclictest15143-21mailstats07:14:132
12281997136,8cyclictest181rcu_preempt12:14:090
12281997136,7cyclictest181rcu_preempt11:03:510
12281997133,7cyclictest181rcu_preempt11:38:110
1228199713,35cyclictest17936-21fschecks_count07:23:590
1230199709,33cyclictest39-21ksoftirqd/207:34:052
1230199709,33cyclictest39-21ksoftirqd/207:34:052
12301997057,9cyclictest9192-21kworker/u18:2-rpciod10:20:592
12286997050,13cyclictest10657-21unixbench_multi08:39:141
12286997049,15cyclictest18445-21apt-key11:48:421
12286997038,23cyclictest6256-21modprobe08:28:371
12286997017,16cyclictest26445-21sh11:07:151
12286997017,12cyclictest13334-21/usr/sbin/munin10:44:101
12286997017,12cyclictest13334-21/usr/sbin/munin10:44:101
12286997012,18cyclictest19157-21ssh10:54:121
12286997012,18cyclictest19157-21ssh10:54:121
12281997040,9cyclictest181rcu_preempt12:10:060
12281997028,9cyclictest0-21swapper/008:43:570
12281997028,9cyclictest0-21swapper/008:43:570
12281997018,11cyclictest161ktimers/008:09:090
1230199697,27cyclictest381ktimers/209:43:482
1230199697,27cyclictest381ktimers/209:43:482
12301996940,8cyclictest181rcu_preempt11:43:052
12301996936,24cyclictest526-21rs:main2
12301996934,8cyclictest0-21swapper/207:43:592
12301996933,13cyclictest181rcu_preempt08:14:002
12286996955,10cyclictest12271-21cyclictest11:28:231
12286996950,12cyclictest6200-21timerandwakeup09:39:131
12286996947,15cyclictest0-21swapper/108:23:401
12286996913,24cyclictest0-21swapper/111:13:561
12281996950,12cyclictest2639-21fschecks_count09:34:150
12281996928,22cyclictest181rcu_preempt12:00:240
1228199692,36cyclictest15-21ksoftirqd/007:13:560
12286996854,8cyclictest0-21swapper/109:19:151
12286996852,10cyclictest12271-21cyclictest11:59:011
1228699684,46cyclictest14501-21/usr/sbin/munin07:13:511
12286996837,21cyclictest14325-21modprobe08:53:371
12286996829,25cyclictest24017-21systemctl11:03:381
12286996818,15cyclictest4492-21switchtime10:29:171
12286996817,22cyclictest30065-21sh10:19:001
12286996811,28cyclictest6836-21cat08:29:021
12286996811,28cyclictest6836-21cat08:29:021
12286996811,17cyclictest19544-21ssh10:02:181
1228199689,40cyclictest15-21ksoftirqd/008:39:000
12281996831,12cyclictest181rcu_preempt08:03:410
12281996829,8cyclictest181rcu_preempt11:30:410
12281996829,8cyclictest181rcu_preempt11:30:410
12301996747,8cyclictest181rcu_preempt08:26:352
12301996729,8cyclictest181rcu_preempt08:44:032
12301996729,8cyclictest181rcu_preempt08:44:032
12286996750,9cyclictest0-21swapper/111:49:221
12286996747,13cyclictest9043-21unixbench_multi08:34:141
12286996733,13cyclictest181rcu_preempt08:44:081
12286996733,13cyclictest181rcu_preempt08:44:081
12286996717,17cyclictest1311-21taskset08:12:321
12286996716,11cyclictest0-21swapper/111:34:151
12281996741,11cyclictest181rcu_preempt09:48:560
107842670,6sleep30-21swapper/310:39:173
1230799663,41cyclictest0-21swapper/312:33:523
12301996644,8cyclictest181rcu_preempt08:52:392
12301996636,21cyclictest29479-21fschecks_count07:58:572
12301996617,24cyclictest0-21swapper/207:28:412
12286996649,11cyclictest28924-21ssh11:10:551
1228699664,37cyclictest27648-21/usr/sbin/munin07:54:041
1228699663,44cyclictest3546-21latency_hist11:23:411
1228699663,25cyclictest4260-21ssh09:36:371
1228699663,25cyclictest170592grep07:19:141
12286996618,12cyclictest23832-21/usr/sbin/munin07:39:151
12286996618,12cyclictest23832-21/usr/sbin/munin07:39:151
12281996628,8cyclictest0-21swapper/012:24:140
1088726612,25sleep10-21swapper/107:03:581
1088726612,25sleep10-21swapper/107:03:581
1230199653,45cyclictest31144-21fschecks_time08:04:022
1230199653,45cyclictest31144-21fschecks_time08:04:022
1230199652,33cyclictest4648-21cron08:23:372
12301996520,29cyclictest0-21swapper/207:38:522
12301996520,29cyclictest0-21swapper/207:38:522
1228699656,20cyclictest0-21swapper/109:14:131
1228699656,20cyclictest0-21swapper/109:14:131
12286996546,9cyclictest0-21swapper/109:48:471
12286996545,15cyclictest28994-21kworker/u19:0+events_unbound08:18:371
12286996545,12cyclictest0-21swapper/109:09:071
12286996528,14cyclictest0-21swapper/107:28:381
12281996538,8cyclictest181rcu_preempt08:04:110
12281996538,8cyclictest181rcu_preempt08:04:110
1228699644,38cyclictest8479-21sh11:31:241
1228699644,38cyclictest8479-21sh11:31:241
1228699643,43cyclictest18070-21fschecks_time09:04:001
12286996416,11cyclictest29477-21rm12:06:341
1228699634,42cyclictest630-21apt-get10:23:581
1228699633,41cyclictest7541-21timerandwakeup10:34:141
12286996320,32cyclictest24939-21ntp_kernel_err07:44:061
1228199627,38cyclictest28088-21cat07:54:040
1228699617,41cyclictest26324-21fschecks_time07:48:591
1230799609,40cyclictest0-21swapper/309:33:563
1228699603,40cyclictest21136-21latency_hist07:33:411
1228699602,32cyclictest30148-21cat07:59:111
1230799592,37cyclictest901-21kworker/3:1+events09:11:203
1230799582,33cyclictest901-21kworker/3:1+events11:41:553
1230799573,36cyclictest12833-21taskset12:33:403
1230799564,38cyclictest3071-21taskset08:18:383
1230799562,37cyclictest901-21kworker/3:1+events10:20:013
1230799562,34cyclictest901-21kworker/3:1+events08:33:583
1230799554,37cyclictest31415-21taskset08:04:083
1230799554,37cyclictest31415-21taskset08:04:083
1230799553,39cyclictest14307-21taskset10:45:463
1230799553,39cyclictest14307-21taskset10:45:463
1230799553,37cyclictest1007-21chrt08:09:143
1230799542,31cyclictest901-21kworker/3:1+events09:51:243
1230799533,33cyclictest5768-21taskset09:39:033
1230799533,32cyclictest6865-21chrt08:29:023
1230799533,32cyclictest6865-21chrt08:29:023
1230799532,35cyclictest2393-21taskset12:15:003
1230799532,35cyclictest17859-21taskset07:23:543
12307995311,30cyclictest19676-21chrt07:29:413
1230799523,34cyclictest7066-21taskset11:29:053
1230799523,34cyclictest7066-21taskset11:29:053
1230799523,34cyclictest31211-21chrt09:28:373
1230799523,34cyclictest23467-21chrt11:02:113
1230799513,36cyclictest30542-21chrt11:14:023
1230799513,33cyclictest25288-21chrt11:04:133
1230799513,33cyclictest16154-21chrt11:44:093
1230799513,31cyclictest10338-21chrt11:34:123
12307995126,13cyclictest0-21swapper/307:48:413
1230799512,37cyclictest14494-21taskset08:53:413
12307995110,23cyclictest0-21swapper/311:48:523
1230799503,32cyclictest5472-21taskset08:24:083
1230799503,27cyclictest17594-21taskset09:59:003
1230799502,34cyclictest28919-21taskset11:10:543
1230799502,30cyclictest11290-21chrt08:43:403
1230799492,34cyclictest19795-21chrt10:55:103
1230799492,34cyclictest19795-21chrt10:55:103
1230799492,33cyclictest26434-21chrt12:01:193
1230799483,33cyclictest22667-21chrt11:54:143
1230799483,33cyclictest22667-21chrt11:54:143
1230799483,32cyclictest9577-21taskset10:38:403
1230799483,32cyclictest30063-21taskset12:08:163
1230799482,34cyclictest26675-21taskset07:49:093
1230799482,31cyclictest7122-21taskset12:23:563
1230799482,30cyclictest9500-21chrt09:44:263
1230799482,30cyclictest9500-21chrt09:44:263
182002472,29sleep30-21swapper/310:53:413
1230799473,32cyclictest382-21chrt12:12:453
1230799473,29cyclictest901-21kworker/3:1+events11:25:013
1230799473,28cyclictest29538-21chrt07:58:573
1230799472,34cyclictest924-21chrt11:18:563
1230799472,32cyclictest1178-21chrt09:32:013
1230799472,32cyclictest1178-21chrt09:32:013
1230799472,29cyclictest3439-21chrt08:18:543
1230799462,31cyclictest11872-21chrt08:44:053
1230799462,31cyclictest11872-21chrt08:44:053
1230799462,28cyclictest16416-21chrt08:58:593
1230799453,31cyclictest4825-21taskset10:29:293
1230799453,28cyclictest23950-21chrt07:40:473
1230799453,28cyclictest23950-21chrt07:40:473
1230799452,30cyclictest23731-21chrt09:14:353
1230799452,30cyclictest23731-21chrt09:14:353
1230799444,26cyclictest16939-21chrt07:19:133
1230799443,31cyclictest27788-21chrt07:53:553
1230799443,29cyclictest6253-21taskset12:22:343
1230799443,28cyclictest15538-21chrt09:54:353
1230799443,28cyclictest15538-21chrt09:54:353
1230799433,32cyclictest27361-21chrt09:21:163
12307994229,8cyclictest124612sleep307:08:493
1230799413,28cyclictest29571-21taskset10:18:383
1230799413,28cyclictest29571-21taskset10:18:383
1230799404,28cyclictest15999-21chrt08:58:393
1230799404,28cyclictest15999-21chrt08:58:393
1230799402,26cyclictest15210-21chrt07:14:143
1230799382,28cyclictest17998-21taskset09:03:573
1230799382,21cyclictest22678-21chrt07:38:413
1230799382,21cyclictest22678-21chrt07:38:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional