You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 08:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Jun 23, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27597216110,110sleep127668-21fschecks_time18:53:581
275762968,59sleep227582-21diskstats18:53:552
2890099922,32cyclictest13997-21apt-get23:58:530
28900999166,8cyclictest181rcu_preempt00:18:570
28900999138,30cyclictest22682-21munin-run23:18:300
28900998957,12cyclictest181rcu_preempt21:39:520
28900998947,11cyclictest181rcu_preempt22:44:370
28900998947,11cyclictest181rcu_preempt22:44:370
2890099878,46cyclictest15-21ksoftirqd/020:26:270
59472860,5sleep20-21swapper/222:49:072
2891999865,43cyclictest39-21ksoftirqd/219:18:482
2890099867,48cyclictest15-21ksoftirqd/019:46:170
28900998657,20cyclictest6274-21diskmemload23:44:190
28900998648,7cyclictest181rcu_preempt22:11:010
28900998648,7cyclictest181rcu_preempt22:11:010
28900998645,11cyclictest181rcu_preempt21:33:510
28919998568,8cyclictest0-21swapper/220:03:472
28900998558,9cyclictest181rcu_preempt23:53:510
28912998464,14cyclictest24366-21sendmail_mailqu00:14:041
28912998464,14cyclictest24366-21sendmail_mailqu00:14:041
28912998451,25cyclictest352-21runrttasks20:58:441
28919998354,7cyclictest181rcu_preempt22:14:002
28919998354,7cyclictest181rcu_preempt22:14:002
28919998353,8cyclictest181rcu_preempt22:28:162
28919998342,10cyclictest181rcu_preempt00:22:322
28919998316,35cyclictest39-21ksoftirqd/222:10:342
28919998316,35cyclictest39-21ksoftirqd/222:10:342
28912998362,15cyclictest11379-21cpu23:53:471
28900998365,7cyclictest181rcu_preempt22:24:050
28900998356,7cyclictest181rcu_preempt00:14:340
28900998356,7cyclictest181rcu_preempt00:14:340
28900998348,12cyclictest29422-21ssh22:34:060
28900998348,12cyclictest29422-21ssh22:34:060
28900998347,9cyclictest181rcu_preempt22:17:480
28900998347,9cyclictest181rcu_preempt22:17:480
28900998345,13cyclictest181rcu_preempt21:02:080
28900998314,51cyclictest10693-21ssh22:03:450
28900998314,51cyclictest10693-21ssh22:03:450
105002830,3sleep0211rcuc/022:03:360
28919998268,8cyclictest0-21swapper/220:14:042
28919998268,8cyclictest0-21swapper/220:14:042
28919998259,8cyclictest181rcu_preempt21:54:102
28919998212,40cyclictest39-21ksoftirqd/200:09:232
28912998266,10cyclictest19348-21kworker/u17:0-rpciod23:44:211
28900998262,11cyclictest0-21swapper/020:19:060
28919998135,12cyclictest181rcu_preempt19:49:542
28900998156,16cyclictest14385-21unixbench_singl19:49:100
28919998053,8cyclictest181rcu_preempt21:26:102
28919998050,15cyclictest181rcu_preempt20:23:332
28919998035,22cyclictest8243-21ssh21:04:432
28919998034,17cyclictest181rcu_preempt23:16:572
28912998026,23cyclictest0-21swapper/120:08:491
28912998026,23cyclictest0-21swapper/120:08:491
28900998046,12cyclictest181rcu_preempt23:02:330
28919997938,12cyclictest181rcu_preempt21:12:232
28919997938,12cyclictest181rcu_preempt21:12:232
28919997936,17cyclictest181rcu_preempt23:38:232
28919997931,10cyclictest0-21swapper/219:48:302
28912997964,8cyclictest0-21swapper/123:26:461
28912997961,10cyclictest24601-21kworker/u17:1-rpciod21:44:061
28912997949,20cyclictest11030-21head21:09:091
28912997949,20cyclictest11030-21head21:09:091
28900997956,11cyclictest0-21swapper/019:38:470
28900997956,11cyclictest0-21swapper/019:38:470
28900997951,10cyclictest181rcu_preempt20:34:070
28900997945,28cyclictest14105-21cpu23:03:470
2891999788,33cyclictest39-21ksoftirqd/223:12:362
2891999788,33cyclictest39-21ksoftirqd/223:12:362
28919997861,10cyclictest0-21swapper/222:33:552
28919997861,10cyclictest0-21swapper/222:33:552
28919997842,8cyclictest181rcu_preempt21:40:402
28919997839,10cyclictest181rcu_preempt21:01:242
28919997837,10cyclictest32529-21taskset20:45:332
28919997832,11cyclictest181rcu_preempt23:19:122
28919997832,11cyclictest181rcu_preempt23:19:122
28919997812,39cyclictest39-21ksoftirqd/220:24:252
28912997861,10cyclictest28383-21sshd20:33:531
28912997859,12cyclictest352-21runrttasks22:38:461
28912997857,15cyclictest13198-21systemd-run19:48:311
28912997826,25cyclictest0-21swapper/121:54:011
28912997822,23cyclictest352-21runrttasks22:47:341
28912997822,23cyclictest352-21runrttasks22:47:341
28900997862,11cyclictest0-21swapper/021:13:560
28900997853,18cyclictest418-21latency_hist19:08:350
28900997853,18cyclictest418-21latency_hist19:08:350
28900997847,24cyclictest25726-21cpu22:28:470
28900997839,12cyclictest181rcu_preempt22:49:060
28900997838,17cyclictest181rcu_preempt23:36:190
28919997756,15cyclictest13413-21sh22:08:332
28919997756,15cyclictest13413-21sh22:08:332
28919997742,8cyclictest181rcu_preempt23:49:452
28919997742,8cyclictest181rcu_preempt23:49:452
28919997738,9cyclictest181rcu_preempt23:42:212
28919997733,15cyclictest181rcu_preempt19:41:482
28919997733,15cyclictest181rcu_preempt19:41:482
28919997726,22cyclictest39-21ksoftirqd/222:45:562
28919997726,22cyclictest39-21ksoftirqd/222:45:562
28919997721,31cyclictest39-21ksoftirqd/219:10:402
28919997721,31cyclictest39-21ksoftirqd/219:10:402
28919997715,34cyclictest39-21ksoftirqd/219:57:022
28912997758,12cyclictest3581-21diskmemload22:48:511
28912997756,15cyclictest22022-21sh23:16:411
28912997744,26cyclictest0-21swapper/100:24:551
2891299772,36cyclictest0-21swapper/121:13:471
2890099776,35cyclictest24613-21grep21:33:310
28900997757,14cyclictest1850-21cron22:43:300
28900997753,7cyclictest181rcu_preempt23:24:190
28900997751,18cyclictest30601-21kernelversion00:24:560
28900997751,18cyclictest16510-21apt-key19:58:340
28919997636,10cyclictest181rcu_preempt21:49:022
28919997635,12cyclictest181rcu_preempt23:45:022
28919997634,11cyclictest16896-21apt-get19:58:492
28919997610,40cyclictest39-21ksoftirqd/223:05:192
28912997659,10cyclictest20940-21fschecks_count00:08:521
28912997649,19cyclictest3677-21latency_hist20:58:341
28912997649,19cyclictest3677-21latency_hist20:58:341
2891299764,62cyclictest10129-21apt-get22:03:451
2891299764,62cyclictest10129-21apt-get22:03:451
28900997644,8cyclictest181rcu_preempt23:09:220
28900997644,8cyclictest181rcu_preempt23:09:220
28900997623,17cyclictest0-21swapper/022:58:340
28919997555,14cyclictest26829-21sh22:29:202
28919997541,8cyclictest181rcu_preempt23:29:252
28919997539,11cyclictest181rcu_preempt21:20:202
28919997536,10cyclictest181rcu_preempt20:54:042
28919997536,10cyclictest181rcu_preempt20:54:042
28919997528,9cyclictest181rcu_preempt21:18:002
28919997526,11cyclictest181rcu_preempt19:06:092
28919997526,11cyclictest181rcu_preempt19:06:092
28919997518,34cyclictest39-21ksoftirqd/200:17:422
28919997518,34cyclictest39-21ksoftirqd/200:17:422
28919997515,33cyclictest39-21ksoftirqd/223:01:332
28912997556,12cyclictest2374-21systemd-journal19:03:311
28912997556,12cyclictest2374-21systemd-journal19:03:311
28912997545,21cyclictest24691-21modprobe20:23:301
2890099758,41cyclictest15-21ksoftirqd/000:04:350
2890099758,41cyclictest15-21ksoftirqd/000:04:350
28900997558,11cyclictest297-21systemd-logind20:13:320
28900997558,11cyclictest297-21systemd-logind20:13:320
28900997552,13cyclictest0-21swapper/021:53:340
28900997549,20cyclictest7208-21fschecks_time19:28:530
28900997549,18cyclictest30757-21latency_hist19:03:350
28900997549,18cyclictest30757-21latency_hist19:03:350
28900997543,9cyclictest181rcu_preempt21:24:510
28900997538,9cyclictest181rcu_preempt19:18:330
2891999749,40cyclictest39-21ksoftirqd/221:30:592
28919997438,12cyclictest181rcu_preempt21:47:212
28919997412,38cyclictest39-21ksoftirqd/219:29:202
28912997455,12cyclictest298-21in:imuxsock21:38:311
28912997453,14cyclictest31587-21sh23:33:311
28912997448,19cyclictest0-21swapper/121:19:071
28912997446,21cyclictest0-21swapper/122:38:331
28912997446,21cyclictest0-21swapper/122:38:331
28912997431,17cyclictest13344-21kworker/u17:1-rpciod23:10:241
28912997431,17cyclictest13344-21kworker/u17:1-rpciod23:10:241
28900997455,10cyclictest0-21swapper/020:04:060
28900997453,9cyclictest181rcu_preempt23:50:500
28900997453,9cyclictest181rcu_preempt23:50:500
28900997441,23cyclictest30000-21fschecks_count20:38:520
28900997441,23cyclictest30000-21fschecks_count20:38:520
2890099743,37cyclictest15-21ksoftirqd/019:37:260
28900997427,27cyclictest0-21swapper/023:28:540
2891999737,38cyclictest39-21ksoftirqd/220:34:342
28919997355,13cyclictest0-21swapper/200:00:002
28919997352,8cyclictest181rcu_preempt21:35:412
28919997337,9cyclictest181rcu_preempt23:53:452
28919997333,11cyclictest181rcu_preempt22:21:432
28919997331,19cyclictest0-21swapper/223:28:122
28912997348,18cyclictest3581-21diskmemload23:34:051
2890099733,35cyclictest15-21ksoftirqd/019:19:060
28919997230,13cyclictest0-21swapper/200:28:192
28919997217,30cyclictest39-21ksoftirqd/220:29:352
28919997214,29cyclictest381ktimers/219:28:312
28919997214,23cyclictest39-21ksoftirqd/220:43:342
28919997214,23cyclictest39-21ksoftirqd/220:43:342
28919997211,36cyclictest39-21ksoftirqd/219:00:562
28919997211,36cyclictest39-21ksoftirqd/219:00:562
28912997255,11cyclictest26453-21switchtime22:29:051
2890099729,34cyclictest15-21ksoftirqd/020:56:170
2890099729,34cyclictest15-21ksoftirqd/020:56:170
2890099725,38cyclictest15-21ksoftirqd/021:18:520
28900997253,13cyclictest31284-21ssh21:43:550
28900997238,12cyclictest766-21diskstats20:48:510
28900997221,20cyclictest0-21swapper/021:08:480
28900997221,20cyclictest0-21swapper/021:08:480
2879627210,37sleep028809-21munin-run18:58:340
28919997134,9cyclictest181rcu_preempt22:01:282
28919997119,27cyclictest39-21ksoftirqd/220:12:262
28919997119,27cyclictest39-21ksoftirqd/220:12:262
28912997124,22cyclictest0-21swapper/119:04:401
28912997124,22cyclictest0-21swapper/119:04:401
2890099713,42cyclictest15-21ksoftirqd/022:19:000
28900997134,12cyclictest181rcu_preempt21:56:020
28919997047,8cyclictest181rcu_preempt22:40:032
28919997035,7cyclictest181rcu_preempt19:34:092
28912997053,11cyclictest298-21in:imuxsock19:41:021
28912997053,11cyclictest298-21in:imuxsock19:41:021
28912997053,11cyclictest1985-21systemd-run20:53:321
28912997051,13cyclictest8425-21timerwakeupswit21:59:091
28912997048,16cyclictest3581-21diskmemload23:38:521
28912997024,31cyclictest0-21swapper/121:49:341
28912997020,25cyclictest18823-21sed00:04:501
28912997020,25cyclictest18823-21sed00:04:501
2891299701,28cyclictest17111-21seq00:03:211
2890099708,32cyclictest15-21ksoftirqd/023:39:250
28900997048,8cyclictest181rcu_preempt00:08:440
2890099704,40cyclictest15-21ksoftirqd/019:28:020
28900997040,11cyclictest181rcu_preempt20:30:550
28900997037,8cyclictest181rcu_preempt20:00:120
2890099702,36cyclictest1891-21awk19:13:310
2890099702,36cyclictest1891-21awk19:13:310
2890099701,42cyclictest15-21ksoftirqd/023:21:090
2890099701,42cyclictest15-21ksoftirqd/023:21:090
28919996925,14cyclictest0-21swapper/222:53:522
28912996949,14cyclictest2374-21systemd-journal20:48:311
2891299694,48cyclictest30401-21grep20:39:031
2891299694,48cyclictest30401-21grep20:39:031
28912996944,18cyclictest15845-21timerandwakeup19:54:071
2891299693,31cyclictest29473-21find00:23:331
28912996913,27cyclictest0-21swapper/121:05:221
2890099694,40cyclictest15-21ksoftirqd/021:03:580
28919996844,10cyclictest181rcu_preempt19:14:232
28919996836,9cyclictest181rcu_preempt00:05:162
28919996836,9cyclictest181rcu_preempt00:05:162
2891299685,53cyclictest19683-21ssh22:18:461
28912996852,11cyclictest28888-21cyclictest21:26:341
28912996846,15cyclictest23760-21vmstat22:24:111
28912996846,15cyclictest22979-21/usr/sbin/munin23:19:011
28912996846,15cyclictest22979-21/usr/sbin/munin23:19:011
28912996825,30cyclictest0-21swapper/119:28:411
28912996819,39cyclictest277-21dbus-daemon19:53:321
28900996833,11cyclictest181rcu_preempt20:44:000
28919996737,8cyclictest181rcu_preempt20:49:032
28912996753,10cyclictest31-21ksoftirqd/123:52:011
28912996753,10cyclictest31-21ksoftirqd/123:52:011
2891299675,27cyclictest0-21swapper/119:26:161
28912996747,13cyclictest24615-21idleruntime-cro21:33:311
28912996746,15cyclictest16357-21latency_hist22:13:331
28912996746,15cyclictest16357-21latency_hist22:13:331
28912996739,20cyclictest8185-21sh22:53:481
2890099672,32cyclictest22238-21ntp_kernel_err20:14:000
2890099672,32cyclictest22238-21ntp_kernel_err20:14:000
28912996646,14cyclictest4318-21proc_pri19:19:031
28912996646,13cyclictest2374-21systemd-journal23:05:021
28912996547,11cyclictest0-21swapper/120:00:091
28912996545,14cyclictest3581-21diskmemload21:38:531
28912996544,15cyclictest11091-21sh22:58:421
2891299653,35cyclictest759-21fschecks_time19:08:521
2891299653,35cyclictest759-21fschecks_time19:08:521
2891299644,42cyclictest352-21runrttasks19:36:311
2891299641,24cyclictest16810-21expr22:13:461
2891299641,24cyclictest16810-21expr22:13:461
2891299633,44cyclictest22026-21fschecks_time20:13:541
2891299633,44cyclictest22026-21fschecks_time20:13:541
2891299623,44cyclictest19264-21timerandwakeup20:04:051
2891299623,38cyclictest352-21runrttasks20:32:441
28912996219,33cyclictest2517-21kernelversion19:13:551
28912996141,12cyclictest0-21swapper/120:23:561
2892499604,40cyclictest20673-21chrt00:08:413
28924995835,13cyclictest135972sleep322:08:343
28924995835,13cyclictest135972sleep322:08:343
2892499574,39cyclictest27697-21taskset22:31:463
2892499573,34cyclictest29258-21taskset23:28:533
2892499563,40cyclictest32154-21taskset23:33:523
2892499553,37cyclictest7982-21chrt22:53:343
284372555,36sleep30-21swapper/318:54:333
142482550,4sleep30-21swapper/323:58:333
2892499543,42cyclictest2479-21chrt23:38:473
2892499543,37cyclictest8757-21chrt21:06:093
2892499542,34cyclictest21648-21chrt20:13:343
2892499542,34cyclictest21648-21chrt20:13:343
2892499533,38cyclictest21357-21taskset21:27:543
2892499523,32cyclictest16964-21chrt23:08:343
2892499514,33cyclictest30533-21taskset22:36:203
2892499514,33cyclictest30533-21taskset22:36:203
2892499513,35cyclictest4861-21chrt22:48:313
2892499513,35cyclictest4861-21chrt22:48:313
2892499513,35cyclictest2824-21chrt19:14:043
2892499513,35cyclictest16922-21taskset19:58:563
2892499503,38cyclictest16861-21chrt22:13:493
2892499503,38cyclictest16861-21chrt22:13:493
2892499503,37cyclictest32008-21taskset22:39:393
2892499503,36cyclictest25172-21chrt21:33:503
2892499503,36cyclictest15277-21chrt19:53:533
2892499502,36cyclictest30693-21chrt19:03:343
2892499502,36cyclictest30693-21chrt19:03:343
2892499492,35cyclictest30385-21chrt20:39:023
2892499492,35cyclictest30385-21chrt20:39:023
2892499492,34cyclictest11047-21taskset19:39:073
2892499492,34cyclictest11047-21taskset19:39:073
2892499492,33cyclictest29565-21chrt21:40:393
2892499484,33cyclictest5872-21taskset19:24:003
2892499484,29cyclictest22265-21chrt22:23:223
2892499483,34cyclictest29183-21taskset20:34:083
2892499483,31cyclictest20449-21taskset00:08:333
2892499483,31cyclictest20449-21taskset00:08:333
2892499483,31cyclictest10693-21chrt21:09:053
2892499483,31cyclictest10693-21chrt21:09:053
2892499482,33cyclictest26021-21chrt00:17:453
2892499482,33cyclictest26021-21chrt00:17:453
2892499474,33cyclictest18926-21chrt23:11:183
2892499474,33cyclictest18926-21chrt23:11:183
2892499473,34cyclictest16261-21taskset21:18:543
2892499473,33cyclictest6653-21chrt23:44:353
2892499473,33cyclictest30853-21taskset00:25:043
2892499473,31cyclictest31949-21chrt21:44:083
2892499473,29cyclictest28129-21chrt00:20:173
2892499464,31cyclictest8710-21chrt19:33:483
2892499463,33cyclictest7833-21chrt19:29:063
2892499463,31cyclictest31953-21taskset19:04:443
2892499463,31cyclictest31953-21taskset19:04:443
2892499462,35cyclictest1075-21chrt20:48:593
2892499462,34cyclictest10069-21chrt22:56:303
2892499454,31cyclictest14823-21chrt19:53:333
28924994534,7cyclictest267572sleep320:28:503
2892499453,33cyclictest12529-21chrt19:44:053
2892499453,33cyclictest10360-21chrt22:03:343
2892499453,32cyclictest3638-21chrt21:52:073
2892499453,31cyclictest6729-21chrt21:03:333
2892499453,31cyclictest24375-21taskset20:19:263
2892499453,28cyclictest19439-21chrt20:04:073
2892499453,26cyclictest13005-21chrt23:01:243
2892499443,30cyclictest6161-21chrt21:55:483
2892499443,30cyclictest20053-21taskset23:13:453
2892499434,29cyclictest25122-21chrt22:28:003
2892499433,30cyclictest25997-21chrt23:23:463
2892499432,31cyclictest16047-21chrt22:12:403
2892499432,31cyclictest16047-21chrt22:12:403
2892499432,27cyclictest25299-21chrt23:22:333
2892499432,27cyclictest25299-21chrt23:22:333
2892499413,28cyclictest14539-21taskset23:58:423
2892499413,25cyclictest14984-21chrt21:16:473
2892499412,29cyclictest23883-21chrt21:31:313
2892499403,28cyclictest1773-21taskset19:12:253
2892499403,28cyclictest1773-21taskset19:12:253
2892499403,27cyclictest4289-21taskset19:19:023
2892499403,27cyclictest2796-21taskset20:54:053
2892499403,27cyclictest2796-21taskset20:54:053
2892499403,26cyclictest0-21swapper/323:48:543
2892499403,26cyclictest0-21swapper/323:48:543
2892499403,25cyclictest547-21taskset20:48:343
2892499393,27cyclictest21762-21chrt20:13:473
2892499393,27cyclictest21762-21chrt20:13:473
2892499383,25cyclictest25315-21taskset20:23:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional