You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 07:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Apr 04, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23140211411,70sleep023196-21sort18:59:390
238669911057,46cyclictest14598-21threads21:29:270
23866991092,22cyclictest107472grep00:04:200
23866991053,47cyclictest19261-21mailstats23:24:270
23866991053,47cyclictest19261-21mailstats23:24:270
23866991044,28cyclictest2618-21taskset23:50:270
23866991044,28cyclictest2618-21taskset23:50:270
23866991043,43cyclictest19767-21/usr/sbin/munin00:19:250
23866991043,27cyclictest9460-21/usr/sbin/munin23:09:270
23866991043,27cyclictest9460-21/usr/sbin/munin23:09:270
23866991043,27cyclictest10764-21grep19:59:310
23866991033,28cyclictest30829-21diskmemload22:39:210
23866991033,28cyclictest29727-21latency_hist22:48:570
23866991033,28cyclictest29727-21latency_hist22:48:570
23866991033,26cyclictest2036-21rm21:09:170
238669910322,24cyclictest6552-21sort19:48:570
23290210310,63sleep223378-21unixbench_singl18:59:412
238669910244,52cyclictest363-21fschecks_time19:29:150
23866991022,31cyclictest19765-21/usr/sbin/munin21:39:170
238669910218,77cyclictest0-21swapper/019:04:170
23866991014,29cyclictest352-21runrttasks22:53:090
23866991013,30cyclictest10437-21latency_hist21:23:560
238669910029,50cyclictest0-21swapper/023:48:140
2386699999,32cyclictest26589-21processes20:49:250
2386699999,32cyclictest26589-21processes20:49:250
2386699993,31cyclictest21241-21/usr/sbin/munin22:34:180
2386699993,25cyclictest27887-21latency_hist00:33:560
2386699993,23cyclictest2952-21systemd-journal20:48:540
2386699992,34cyclictest87-21kswapd022:19:470
2386699992,24cyclictest3412-21runrttasks22:58:570
2386699992,24cyclictest3412-21runrttasks22:58:570
2386699984,38cyclictest22116-21/usr/sbin/munin23:29:210
2386699984,29cyclictest17208-21grep00:14:320
2386699984,29cyclictest17208-21grep00:14:320
2386699984,28cyclictest7430-21modprobe21:18:540
23866999842,50cyclictest7314-21ssh23:59:110
2386699982,32cyclictest13184-21proc_pri23:14:250
2386699973,83cyclictest6929-21ssh23:58:560
2386699973,37cyclictest352-21runrttasks22:04:590
2386699973,22cyclictest15516-21/usr/sbin/munin22:24:260
2386699972,21cyclictest14713-21apt-get20:14:060
23866999718,24cyclictest30137-21seq23:43:380
23873999656,20cyclictest181rcu_preempt22:56:012
23873999656,20cyclictest181rcu_preempt22:56:012
23866999673,18cyclictest0-21swapper/022:14:180
23866999673,18cyclictest0-21swapper/022:14:180
2386699964,33cyclictest8301-21ls22:13:570
2386699963,38cyclictest30048-21switchtime19:19:260
2386699963,34cyclictest17464-21uname21:34:270
2386699962,26cyclictest13149-21runrttasks21:28:080
2386699953,33cyclictest9595-21latency_hist19:58:570
2386699953,33cyclictest352-21runrttasks00:25:540
2386699953,33cyclictest352-21runrttasks00:25:540
2386699953,32cyclictest28241-21sendmail_mailqu20:54:260
2386699953,29cyclictest13110-21/usr/sbin/munin00:09:290
2386699953,28cyclictest12802-21latency_hist20:08:540
2386699953,21cyclictest26161-21ssh23:35:550
2386699953,21cyclictest26161-21ssh23:35:550
2386699953,19cyclictest16341-21/usr/sbin/munin23:19:280
2386699952,29cyclictest28895-21/usr/sbin/munin21:54:100
2386699952,29cyclictest28895-21/usr/sbin/munin21:54:100
23866999514,26cyclictest31172-21taskset21:04:080
23866999514,26cyclictest31172-21taskset21:04:080
2386699944,32cyclictest26539-21grep21:49:270
2386699943,31cyclictest6653-21expr23:04:100
2386699943,18cyclictest13067-21apt-get20:09:040
2386699942,28cyclictest6793-21fschecks_count19:49:140
2386699942,27cyclictest31884-21cpu21:59:100
23873999347,15cyclictest181rcu_preempt22:29:152
2386699937,11cyclictest0-21swapper/022:59:310
2386699934,31cyclictest4867-21latency_hist19:43:570
2386699933,35cyclictest17260-21unixbench_singl20:19:300
2386699933,32cyclictest21175-21cpu20:34:090
2386699933,32cyclictest21175-21cpu20:34:090
2386699933,31cyclictest3259-21latency_hist19:38:570
2386699932,32cyclictest19627-21cstates20:29:110
2386699932,32cyclictest19627-21cstates20:29:110
2386699932,26cyclictest18449-21awk22:29:230
23873999260,16cyclictest16836-21ssh22:26:572
23866999237,48cyclictest0-21swapper/021:44:100
2386699923,31cyclictest31855-21grep19:24:280
2386699923,31cyclictest22751-21/usr/sbin/munin20:39:290
2386699923,27cyclictest18330-21latency20:24:190
23870999117,37cyclictest670-21ssh22:00:051
2386699913,32cyclictest28018-21fschecks_time19:14:170
2386699913,30cyclictest352-21runrttasks20:59:090
2386699912,31cyclictest26385-21fschecks_time19:09:180
23873999026,45cyclictest30829-21diskmemload22:19:102
23873998862,17cyclictest10160-21ssh23:09:252
23873998862,17cyclictest10160-21ssh23:09:252
23873998848,9cyclictest181rcu_preempt22:53:572
23873998839,17cyclictest181rcu_preempt19:19:092
23873998761,10cyclictest181rcu_preempt22:44:292
23873998761,10cyclictest181rcu_preempt22:44:292
23873998758,9cyclictest181rcu_preempt21:33:142
23873998756,12cyclictest181rcu_preempt21:11:222
23873998747,11cyclictest181rcu_preempt21:15:092
225782877,55sleep122579-21diskstats18:59:191
23873998655,8cyclictest181rcu_preempt21:38:002
23873998639,14cyclictest0-21swapper/221:46:142
23873998550,10cyclictest181rcu_preempt19:53:182
23873998549,8cyclictest181rcu_preempt00:10:442
23873998548,9cyclictest181rcu_preempt21:49:262
23873998546,14cyclictest181rcu_preempt21:42:462
23873998533,11cyclictest0-21swapper/223:22:342
23873998459,10cyclictest181rcu_preempt22:04:552
23873998454,14cyclictest181rcu_preempt19:48:572
23873998451,14cyclictest181rcu_preempt19:24:142
23873998445,13cyclictest181rcu_preempt20:58:172
23873998441,14cyclictest181rcu_preempt00:28:402
23873998441,14cyclictest181rcu_preempt00:28:402
2387099847,62cyclictest11861-21apt-get22:19:081
23870998455,19cyclictest16450-21latency_hist21:33:571
23873998356,10cyclictest22150-21ssh22:34:582
23873998349,11cyclictest181rcu_preempt21:54:302
23873998349,11cyclictest181rcu_preempt21:54:302
23873998343,8cyclictest181rcu_preempt20:39:152
23870998359,16cyclictest19173-21diskmemload23:24:261
23870998359,16cyclictest19173-21diskmemload23:24:261
23870998311,59cyclictest24257-21kworker/u18:0+flush-179:022:14:321
23870998311,59cyclictest24257-21kworker/u18:0+flush-179:022:14:321
23873998257,17cyclictest21923-21kernelversion23:29:182
23873998244,10cyclictest181rcu_preempt23:53:472
23873998244,10cyclictest181rcu_preempt23:53:472
23873998243,12cyclictest181rcu_preempt00:33:082
23873998238,25cyclictest0-21swapper/223:08:332
23873998222,25cyclictest0-21swapper/221:04:142
23873998222,25cyclictest0-21swapper/221:04:142
2388099814,28cyclictest8318-21chrt21:19:233
23873998149,8cyclictest181rcu_preempt00:04:142
23873998149,13cyclictest181rcu_preempt20:34:102
23873998149,13cyclictest181rcu_preempt20:34:102
23873998144,13cyclictest181rcu_preempt19:15:322
23873998144,11cyclictest181rcu_preempt22:00:012
23873998141,13cyclictest181rcu_preempt20:48:562
23873998129,25cyclictest0-21swapper/219:38:572
23873998056,8cyclictest181rcu_preempt22:09:282
23873998055,9cyclictest181rcu_preempt21:19:172
23873998055,19cyclictest19341-21fschecks_count00:19:152
23873998050,8cyclictest181rcu_preempt22:39:112
23873998047,10cyclictest181rcu_preempt00:18:572
23873998047,10cyclictest181rcu_preempt00:18:572
23873998040,13cyclictest181rcu_preempt23:39:182
23873998039,13cyclictest181rcu_preempt20:19:472
23873998035,13cyclictest181rcu_preempt19:54:102
23873997950,11cyclictest181rcu_preempt23:27:492
23873997950,11cyclictest181rcu_preempt23:27:492
23873997940,12cyclictest181rcu_preempt20:24:042
23873997936,12cyclictest181rcu_preempt21:24:112
23873997935,10cyclictest181rcu_preempt21:00:112
23873997912,16cyclictest32671-21kworker/2:2-events_power_efficient23:55:012
23870997964,9cyclictest32597-21munin-run22:53:531
23870997956,16cyclictest7963-21sendmail19:53:541
23870997954,18cyclictest30829-21diskmemload22:39:161
23870997947,22cyclictest16519-21fschecks_count20:19:151
23870997915,23cyclictest14292-21seq20:12:251
23870997913,26cyclictest19110-21seq21:37:561
2388099783,32cyclictest15836-21chrt00:13:533
23873997849,9cyclictest181rcu_preempt20:49:252
23873997849,9cyclictest181rcu_preempt20:49:252
23873997844,11cyclictest181rcu_preempt20:06:382
23873997842,12cyclictest181rcu_preempt22:59:242
23873997832,13cyclictest181rcu_preempt20:29:162
23873997832,13cyclictest181rcu_preempt20:29:162
23870997859,13cyclictest5760-21sh23:55:481
23870997822,26cyclictest3156-21sh22:58:521
23870997822,26cyclictest3156-21sh22:58:521
2388099774,29cyclictest30868-21chrt21:03:553
2388099773,33cyclictest32010-21chrt23:45:273
23873997753,9cyclictest181rcu_preempt00:00:142
23873997751,11cyclictest181rcu_preempt22:14:562
23873997751,11cyclictest181rcu_preempt22:14:562
23873997749,12cyclictest181rcu_preempt23:48:562
23873997744,10cyclictest181rcu_preempt19:13:332
2388099763,35cyclictest5983-21chrt23:56:263
23873997650,9cyclictest181rcu_preempt23:34:292
23873997650,9cyclictest181rcu_preempt23:34:292
23873997649,11cyclictest181rcu_preempt19:04:112
23870997661,11cyclictest23857-21cyclictest21:26:021
23870997657,13cyclictest20019-21timerandwakeup00:19:291
23870997614,24cyclictest14578-21ssh00:10:301
2388099752,32cyclictest15995-21taskset22:24:363
23873997537,18cyclictest181rcu_preempt19:59:302
2387099758,35cyclictest24198-21latency_hist20:43:551
23870997551,15cyclictest352-21runrttasks22:44:331
23870997551,15cyclictest352-21runrttasks22:44:331
23873997449,8cyclictest181rcu_preempt19:29:272
2387399744,44cyclictest15665-21unixbench_singl20:14:302
23873997442,9cyclictest181rcu_preempt20:09:182
2387099749,29cyclictest0-21swapper/119:19:161
23870997455,13cyclictest352-21runrttasks21:04:331
23870997455,13cyclictest352-21runrttasks21:04:331
23870997415,23cyclictest22240-21seq20:35:381
23870997415,23cyclictest22240-21seq20:35:381
23870997415,23cyclictest1914-21seq19:34:151
23870997412,23cyclictest26550-21ssh00:30:321
2296227415,34sleep30-21swapper/318:59:333
23870997354,13cyclictest20622-21latency_hist22:33:561
23870997335,10cyclictest0-21swapper/120:49:201
23870997335,10cyclictest0-21swapper/120:49:201
23870997316,20cyclictest1177-21seq19:30:301
23870997314,25cyclictest12384-21latency_hist23:13:571
23870997314,25cyclictest12384-21latency_hist23:13:571
23870997311,27cyclictest18807-21munin-run00:18:531
23870997311,27cyclictest18807-21munin-run00:18:531
2388099727,43cyclictest0-21swapper/322:14:183
2388099727,43cyclictest0-21swapper/322:14:183
23873997251,14cyclictest12646-21cstates23:14:102
23873997235,12cyclictest181rcu_preempt19:43:572
2387099727,31cyclictest8144-21unin-run22:13:551
23870997254,12cyclictest15193-21fschecks_time22:24:161
23870997216,24cyclictest24541-21seq00:28:171
23870997216,24cyclictest24541-21seq00:28:171
23870997211,27cyclictest2211-21unixbench_multi23:49:301
23870997211,27cyclictest2211-21unixbench_multi23:49:301
23870997211,27cyclictest0-21swapper/119:54:481
2388099713,29cyclictest24857-21chrt21:47:113
2387099718,30cyclictest971-21latency_hist23:48:561
23870997144,18cyclictest18260-21latency_hist23:23:551
23870997121,22cyclictest4745-21sh22:59:561
23870997115,24cyclictest23381-21seq23:31:381
23870997114,21cyclictest0-21swapper/123:08:471
23870997113,23cyclictest0-21swapper/121:40:031
2387099709,26cyclictest0-21swapper/100:04:231
2387099708,24cyclictest352-21runrttasks20:27:591
23870997015,23cyclictest15845-21seq20:16:221
2388099693,32cyclictest17345-21chrt23:21:313
23870996953,10cyclictest3919-21ntp_offset19:39:211
23870996916,18cyclictest0-21swapper/121:50:201
23870996914,23cyclictest9490-21seq00:02:551
23870996912,24cyclictest22373-21seq22:35:351
23870996910,26cyclictest15293-21latency_hist23:18:551
23870996910,23cyclictest0-21swapper/120:56:421
2387099687,25cyclictest27865-21perl23:39:161
23870996848,13cyclictest352-21runrttasks23:38:081
23870996848,13cyclictest352-21runrttasks23:38:081
2387099684,26cyclictest0-21swapper/121:44:191
23870996818,18cyclictest30829-21diskmemload21:56:431
23870996818,18cyclictest30829-21diskmemload21:56:431
23870996811,42cyclictest0-21swapper/120:04:361
68352670,6sleep30-21swapper/321:17:163
2387099679,28cyclictest0-21swapper/119:44:171
2387099677,20cyclictest0-21swapper/119:13:251
23870996746,15cyclictest19720-21fschecks_count20:29:151
23870996746,15cyclictest19720-21fschecks_count20:29:151
23870996727,34cyclictest0-21swapper/121:14:061
23870996717,31cyclictest25608-21kworker/u17:3-rpciod22:03:581
23870996712,25cyclictest25425-21seq20:45:141
2387099669,43cyclictest29084-21latency_hist19:18:541
2387099665,49cyclictest29710-21entropy21:00:111
2387099663,43cyclictest8545-21ssh21:19:301
2387099662,46cyclictest352-21runrttasks21:12:381
23870996613,22cyclictest32016-21seq19:25:241
23870996514,22cyclictest25204-21seq19:06:251
2387099633,24cyclictest9844-21/usr/sbin/munin19:59:301
2388099614,39cyclictest28006-21taskset23:39:193
2388099603,37cyclictest18737-21chrt23:24:153
2388099603,37cyclictest18737-21chrt23:24:153
2388099603,36cyclictest24378-21taskset22:39:193
2388099594,38cyclictest25955-21chrt00:29:303
2388099593,39cyclictest25458-21chrt23:34:293
2388099593,39cyclictest25458-21chrt23:34:293
2388099584,36cyclictest2699-21taskset21:09:313
2388099583,37cyclictest22565-21taskset00:24:193
2388099583,37cyclictest22565-21taskset00:24:193
2388099572,39cyclictest18398-21taskset22:29:223
2388099563,35cyclictest31747-21chrt22:51:363
2388099554,35cyclictest26585-21chrt19:09:233
2388099554,34cyclictest31943-21taskset21:04:283
2388099554,34cyclictest31943-21taskset21:04:283
2388099553,38cyclictest2366-21chrt19:34:263
2388099553,36cyclictest2887-21chrt22:58:083
2388099553,36cyclictest2887-21chrt22:58:083
2388099553,36cyclictest22417-21taskset21:43:473
2388099552,38cyclictest21437-21chrt20:34:153
2388099552,38cyclictest21437-21chrt20:34:153
2388099544,35cyclictest21661-21chrt23:29:093
2388099544,32cyclictest5348-21taskset19:44:183
2388099543,36cyclictest1482-21chrt22:02:183
2388099543,35cyclictest28389-21chrt20:54:283
2388099543,34cyclictest24638-21taskset20:44:153
2388099543,34cyclictest16236-21chrt20:18:563
2388099542,39cyclictest1543-21chrt23:49:163
2388099542,39cyclictest1543-21chrt23:49:163
2388099533,37cyclictest16824-21taskset21:34:103
2388099533,36cyclictest6527-21taskset23:04:083
2388099533,36cyclictest14293-21chrt21:29:203
2388099533,36cyclictest12174-21taskset23:13:533
2388099533,36cyclictest12174-21taskset23:13:533
2388099533,35cyclictest8661-21chrt00:00:353
2388099532,33cyclictest19479-21taskset20:28:573
2388099524,36cyclictest26648-21taskset20:49:263
2388099524,36cyclictest26648-21taskset20:49:263
2388099524,32cyclictest15355-21taskset23:18:563
2388099523,36cyclictest6959-21taskset22:10:533
2388099522,39cyclictest31132-21taskset19:24:133
2388099522,37cyclictest8902-21chrt19:54:263
2388099522,37cyclictest10744-21taskset00:04:203
2388099522,33cyclictest11950-21taskset22:19:173
2388099514,31cyclictest11709-21chrt20:04:143
2388099513,34cyclictest26823-21chrt21:49:303
2388099513,34cyclictest11680-21chrt21:24:303
2388099513,32cyclictest30890-21chrt21:57:123
2388099513,32cyclictest30890-21chrt21:57:123
2388099512,34cyclictest13547-21chrt20:09:213
2388099503,32cyclictest5254-21chrt23:01:213
2388099494,35cyclictest3953-21taskset19:39:243
2388099483,31cyclictest21699-21chrt22:34:293
2388099483,31cyclictest19636-21taskset00:19:203
23880994823,15cyclictest0-21swapper/319:29:293
2388099474,32cyclictest28923-21chrt22:47:063
2388099474,32cyclictest28923-21chrt22:47:063
2388099474,30cyclictest7943-21taskset19:53:543
2388099473,32cyclictest28560-21chrt19:14:303
2388099462,30cyclictest4541-21chrt22:07:293
2388099462,26cyclictest30874-21taskset19:23:573
2388099454,27cyclictest22828-21taskset20:39:093
2388099443,32cyclictest20732-21chrt20:32:173
2388099443,32cyclictest20732-21chrt20:32:173
2388099442,34cyclictest16635-21chrt20:19:173
2388099433,27cyclictest18879-21chrt00:18:563
2388099433,27cyclictest18879-21chrt00:18:563
2388099403,27cyclictest25358-21taskset19:08:263
2388099393,26cyclictest11045-21taskset20:02:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional