You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 08:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue May 12, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2038521410,12sleep30-21swapper/319:01:253
204019910617,43cyclictest0-21swapper/122:01:361
19910210311,64sleep019920-21vmstat18:57:110
20406999742,48cyclictest27278-21diskmemload23:56:252
20406999742,48cyclictest27278-21diskmemload23:56:252
20406999714,33cyclictest1-21systemd21:31:222
2039699963,81cyclictest22380-21apt-get23:36:360
158172960,7sleep30-21swapper/323:26:033
2039699953,83cyclictest16412-21rm21:36:470
196982958,59sleep119791-21sed18:57:081
20406999434,53cyclictest0-21swapper/221:06:592
20406999434,53cyclictest0-21swapper/221:06:592
2040699943,34cyclictest28159-21ssh21:56:362
20406999268,18cyclictest29419-21kernelversion19:26:462
2040699923,31cyclictest31059-21/usr/sbin/munin22:01:322
20396999264,21cyclictest29307-21cut19:26:430
2039699914,57cyclictest15-21ksoftirqd/019:41:240
20406999018,24cyclictest29132-21sh21:03:112
20396999052,31cyclictest22308-21sendmail00:31:210
20396999049,32cyclictest0-21swapper/023:06:240
20406998962,18cyclictest16361-21latency_hist00:21:242
20406998962,18cyclictest16361-21latency_hist00:21:242
2040699893,44cyclictest21944-21apt-key23:36:222
2040699884,42cyclictest26013-21switchtime21:51:582
2040699884,42cyclictest26013-21switchtime21:51:582
2040699884,41cyclictest352-21runrttasks20:26:512
20406998843,38cyclictest0-21swapper/223:21:332
2040699884,24cyclictest22492-21fschecks_count19:06:322
20406998835,46cyclictest16639-21sh23:26:452
20406998835,46cyclictest16639-21sh23:26:452
20406998830,50cyclictest0-21swapper/222:41:402
2040699882,32cyclictest10215-21latency_hist22:21:232
20396998811,46cyclictest15-21ksoftirqd/023:15:390
2040699873,32cyclictest24938-21cut21:51:232
20406998710,29cyclictest7269-21grep19:56:572
20401998710,60cyclictest16494-21chrt21:37:021
2039699878,45cyclictest15-21ksoftirqd/019:13:040
20396998748,18cyclictest181rcu_preempt00:22:120
1908328713,38sleep20-21swapper/218:56:482
2040699868,27cyclictest26109-21fschecks_count20:56:432
2040699864,43cyclictest1905-21fschecks_count23:01:432
2040699864,34cyclictest9770-21ssh21:26:202
2040699864,32cyclictest14383-21apt-key20:21:252
2040699864,28cyclictest16332-21/usr/sbin/munin22:31:482
2040699863,33cyclictest16983-21fschecks_time21:37:532
2040699863,31cyclictest2596-21unixbench_multi22:06:582
2040699863,31cyclictest22359-21/usr/sbin/munin23:36:322
2040699863,30cyclictest24895-21wc20:51:532
2040699862,26cyclictest19472-21/usr/sbin/munin22:36:382
2040699862,26cyclictest19472-21/usr/sbin/munin22:36:382
20396998653,10cyclictest181rcu_preempt22:16:550
20396998631,33cyclictest15-21ksoftirqd/023:27:350
20396998631,33cyclictest15-21ksoftirqd/023:27:350
2040699854,33cyclictest1495-21idleruntime-cro23:56:212
2040699854,24cyclictest30846-21/usr/sbin/munin19:31:392
20406998534,43cyclictest0-21swapper/223:46:572
2040699853,35cyclictest3680-21wc19:46:542
2040699853,35cyclictest3680-21wc19:46:542
2040699853,33cyclictest8154-21ps23:11:522
2040699853,32cyclictest31159-21/usr/sbin/munin22:56:542
2040699853,30cyclictest6251-21latency_hist19:56:242
2040699853,26cyclictest19356-21/usr/sbin/munin21:41:362
2040699853,26cyclictest19356-21/usr/sbin/munin21:41:362
2040699853,25cyclictest352-21runrttasks20:06:382
2040699852,35cyclictest2947-21latency_hist19:46:232
2040699852,35cyclictest2947-21latency_hist19:46:232
2040699852,32cyclictest25304-21/usr/sbin/munin22:46:452
2040699852,32cyclictest22666-21/usr/sbin/munin20:46:542
2040699852,32cyclictest22666-21/usr/sbin/munin20:46:542
2039699853,45cyclictest13544-21latency_hist00:16:240
2039699852,46cyclictest15-21ksoftirqd/023:51:590
2040699844,36cyclictest17130-21grep00:21:472
2040699844,32cyclictest7895-21/usr/sbin/munin20:01:432
2040699843,36cyclictest27664-21fschecks_count19:21:422
2040699843,34cyclictest30868-21idleruntime-cro22:56:212
2040699843,34cyclictest30868-21idleruntime-cro22:56:212
2040699843,33cyclictest4530-21/usr/sbin/munin23:06:572
2040699843,30cyclictest7509-21ssh00:06:232
2040699843,30cyclictest24232-21/usr/sbin/munin19:11:442
2040699843,29cyclictest14894-21tune2fs20:22:132
2040699842,31cyclictest7959-21tune2fs00:06:422
2040699842,30cyclictest13542-21grep00:16:242
20401998452,22cyclictest14294-21sh23:21:591
20401998418,25cyclictest5697-21runrttasks21:18:161
20401998418,25cyclictest5697-21runrttasks21:18:161
20406998359,15cyclictest2374-21systemd-journal21:34:462
2040699834,41cyclictest19951-21fschecks_time00:26:432
2040699834,32cyclictest11429-21grep23:16:592
2040699834,32cyclictest11429-21grep23:16:592
2040699833,34cyclictest1193-21ssh21:11:372
2040699833,32cyclictest21659-21grep20:41:542
2040699833,30cyclictest819-21grep19:36:582
2040699833,29cyclictest13311-21latency_hist22:26:252
2040699832,31cyclictest19545-21ls20:36:382
2040699832,29cyclictest4881-21basename22:11:432
20406998320,56cyclictest0-21swapper/223:41:342
20406998320,56cyclictest0-21swapper/223:41:342
20401998361,15cyclictest11414-21cut20:11:431
20396998362,16cyclictest4885-21mailstats21:16:560
20396998362,16cyclictest4885-21mailstats21:16:560
20396998361,8cyclictest181rcu_preempt22:11:010
2039699834,34cyclictest15-21ksoftirqd/020:01:240
2039699833,54cyclictest15-21ksoftirqd/022:26:240
2039699833,35cyclictest28060-21cut21:01:490
20406998250,23cyclictest16002-21cron22:31:212
2040699824,29cyclictest18442-21processes20:31:532
2040699824,28cyclictest26088-21fschecks_time19:16:432
2040699823,45cyclictest6804-21apt21:21:212
2040699823,45cyclictest6804-21apt21:21:212
2040699823,38cyclictest21519-21grep19:01:572
20401998212,34cyclictest22313-21latency_hist23:36:251
20396998259,14cyclictest4867-21fschecks_time23:06:440
20396998246,24cyclictest3323-21grep19:46:440
20396998246,24cyclictest3323-21grep19:46:440
2039699823,65cyclictest10394-21idleruntime-cro00:11:210
2039699823,38cyclictest15-21ksoftirqd/021:14:310
20396998213,36cyclictest15-21ksoftirqd/019:33:250
20401998155,17cyclictest28151-21latency_hist23:46:241
20401998155,17cyclictest28151-21latency_hist23:46:241
20401998114,24cyclictest9536-21seq22:19:421
20396998149,11cyclictest181rcu_preempt23:44:270
20396998149,11cyclictest181rcu_preempt23:44:270
303852800,8sleep30-21swapper/321:06:233
20401998060,14cyclictest27278-21diskmemload22:09:321
20401998010,37cyclictest2314-21ps23:01:541
20396998058,14cyclictest20387-21cyclictest22:04:550
2040699794,33cyclictest11734-21proc_pri20:11:532
20401997917,22cyclictest22155-21seq21:46:251
20401997916,24cyclictest18239-21seq23:29:371
20401997916,24cyclictest18239-21seq23:29:371
2039699795,36cyclictest15-21ksoftirqd/019:53:500
20396997931,26cyclictest15-21ksoftirqd/022:11:520
20396997856,14cyclictest23496-21unixbench_multi19:06:590
2039699785,35cyclictest15-21ksoftirqd/020:41:540
20396997841,13cyclictest181rcu_preempt23:57:030
20396997841,13cyclictest181rcu_preempt23:57:030
20396997838,11cyclictest0-21swapper/022:26:580
20396997837,12cyclictest181rcu_preempt22:53:300
20396997837,12cyclictest181rcu_preempt22:53:300
2039699783,45cyclictest15-21ksoftirqd/021:55:060
2039699783,45cyclictest15-21ksoftirqd/021:55:060
20401997724,17cyclictest381-21sh21:10:101
20401997724,17cyclictest381-21sh21:10:101
20401997722,18cyclictest0-21swapper/122:44:101
20396997751,18cyclictest31407-21fschecks_count22:56:420
20396997746,12cyclictest181rcu_preempt21:10:070
20396997746,12cyclictest181rcu_preempt21:10:070
2039699774,48cyclictest15-21ksoftirqd/021:41:560
2039699774,48cyclictest15-21ksoftirqd/021:41:560
2039699774,41cyclictest15-21ksoftirqd/020:11:550
20396997738,18cyclictest181rcu_preempt19:25:290
2039699773,43cyclictest15-21ksoftirqd/019:41:580
2039699773,43cyclictest15-21ksoftirqd/019:41:580
20396997733,9cyclictest0-21swapper/021:32:170
20411997642,24cyclictest7930-21chrt22:16:493
2040199763,50cyclictest27278-21diskmemload21:16:241
20401997615,25cyclictest18975-21seq20:33:391
20401997615,24cyclictest32630-21seq23:52:401
20401997614,23cyclictest15937-21seq20:26:181
20401997610,27cyclictest2996-21cat19:46:241
20401997610,27cyclictest2996-21cat19:46:241
2039699766,39cyclictest15-21ksoftirqd/020:26:430
20396997650,7cyclictest181rcu_preempt19:17:520
20396997632,17cyclictest181rcu_preempt23:20:130
20396997632,17cyclictest181rcu_preempt23:20:130
20401997522,20cyclictest27587-21kworker/u17:2-xprtiod23:01:241
20401997517,23cyclictest10753-21seq20:10:021
20401997517,19cyclictest26783-21seq22:48:021
20401997516,22cyclictest22248-21seq00:31:141
20401997514,23cyclictest22191-21seq20:43:361
2039699756,50cyclictest1-21systemd23:51:220
2039699753,44cyclictest15-21ksoftirqd/020:41:250
2039699752,22cyclictest9023-21sshd20:03:340
20401997453,15cyclictest27278-21diskmemload22:36:491
20401997453,15cyclictest27278-21diskmemload22:36:491
20401997452,14cyclictest29635-21sh21:04:361
2040199743,39cyclictest0-21swapper/119:17:261
20401997419,20cyclictest0-21swapper/120:53:521
20401997417,25cyclictest27150-21seq21:00:001
20401997413,23cyclictest5735-21seq19:52:331
20401997411,30cyclictest10721-21fschecks_time22:21:431
20401997410,33cyclictest352-21runrttasks20:03:221
20401997410,26cyclictest0-21swapper/123:46:551
2039699746,34cyclictest15-21ksoftirqd/020:31:460
20396997459,10cyclictest25543-21ssh22:46:430
20396997447,12cyclictest181rcu_preempt22:01:210
2039699744,48cyclictest15-21ksoftirqd/021:00:410
20396997439,16cyclictest181rcu_preempt23:35:440
2039699743,39cyclictest15-21ksoftirqd/021:49:350
20396997428,9cyclictest181rcu_preempt23:21:590
20396997428,35cyclictest15512-21ls20:22:270
2039699742,52cyclictest15-21ksoftirqd/020:51:240
2039699742,52cyclictest15-21ksoftirqd/020:51:240
2039699742,37cyclictest19582-21fschecks_time22:36:430
2039699742,37cyclictest19582-21fschecks_time22:36:430
2039699741,39cyclictest15-21ksoftirqd/020:06:460
20401997353,14cyclictest6933-21sh22:15:401
20401997353,14cyclictest29267-21fschecks_count19:26:431
20401997344,19cyclictest0-21swapper/100:21:211
20401997344,19cyclictest0-21swapper/100:21:211
20401997331,30cyclictest16472-21cat22:31:381
20401997316,23cyclictest4329-21seq19:50:341
20401997316,23cyclictest4329-21seq19:50:341
20401997313,23cyclictest913-21seq19:37:591
2039699735,39cyclictest15-21ksoftirqd/019:01:470
20396997336,16cyclictest181rcu_preempt22:36:170
2040199729,26cyclictest10386-21munin-run00:11:211
20401997215,20cyclictest21787-21seq19:04:541
2039699725,38cyclictest15-21ksoftirqd/022:42:380
20396997251,15cyclictest14144-21sh20:21:210
2039699723,35cyclictest7099-21/usr/sbin/munin21:21:580
20401997150,13cyclictest1-21systemd19:11:221
20401997116,24cyclictest4307-21seq00:00:591
20401997116,24cyclictest4307-21seq00:00:591
20401997112,24cyclictest15033-21runrttasks22:28:331
20401997111,24cyclictest0-21swapper/123:06:521
20401997110,23cyclictest0-21swapper/120:46:471
20401997110,23cyclictest0-21swapper/120:46:471
20396997148,9cyclictest181rcu_preempt21:30:380
2039699713,36cyclictest352-21runrttasks00:02:270
20401997020,20cyclictest6034-21sh00:02:281
20401997015,28cyclictest17527-21seq20:31:041
20401997011,23cyclictest0-21swapper/119:14:181
2039699703,57cyclictest13812-21cstates00:16:390
2039699703,57cyclictest13812-21cstates00:16:390
20401996916,23cyclictest28773-21seq19:26:121
20401996915,21cyclictest18458-21seq00:23:531
20401996912,28cyclictest352-21runrttasks20:38:291
20401996911,24cyclictest0-21swapper/121:57:161
20401996815,22cyclictest7477-21seq19:59:211
20401996814,23cyclictest13876-21seq20:18:331
20401996813,23cyclictest29698-21seq22:52:561
20401996813,23cyclictest29698-21seq22:52:561
20401996718,17cyclictest0-21swapper/121:25:511
20401996713,22cyclictest13298-21seq00:16:111
20396996744,15cyclictest25665-21munin-plugin-st20:56:230
20401996645,14cyclictest8057-21sh23:11:491
20401996618,22cyclictest27278-21diskmemload23:19:321
20401996618,22cyclictest27278-21diskmemload23:19:321
20401996614,20cyclictest27010-21seq21:54:041
20401996614,20cyclictest27010-21seq21:54:041
20401996549,11cyclictest20387-21cyclictest21:32:011
20401996515,21cyclictest21456-21ssh21:45:081
20401996515,21cyclictest21456-21ssh21:45:081
20401996510,22cyclictest0-21swapper/119:34:471
2040199642,45cyclictest21824-21apt-get23:36:351
20401996315,26cyclictest11327-21seq21:27:201
2041199618,32cyclictest0-21swapper/323:36:463
2041199593,38cyclictest28883-21chrt19:26:243
2041199592,30cyclictest0-21swapper/320:21:563
20411995911,33cyclictest0-21swapper/321:36:393
2041199574,40cyclictest1881-21chrt22:06:433
2041199573,40cyclictest8024-21taskset20:01:393
2041199564,34cyclictest17854-21chrt20:31:323
2041199554,35cyclictest29209-21chrt21:56:583
2041199543,36cyclictest24610-21taskset19:11:463
2041199543,34cyclictest4753-21chrt00:01:313
2041199542,41cyclictest30768-21taskset21:06:403
2041199542,41cyclictest30768-21taskset21:06:403
2041199533,37cyclictest31903-21chrt22:01:573
2041199523,36cyclictest29053-21chrt19:26:323
2041199523,35cyclictest24963-21chrt21:51:233
2041199523,34cyclictest3013-21taskset19:46:243
2041199523,34cyclictest3013-21taskset19:46:243
2041199523,32cyclictest6249-21taskset19:56:243
2041199522,38cyclictest8133-21taskset00:06:463
2041199522,37cyclictest5990-21taskset23:07:593
2041199522,37cyclictest12936-21taskset21:31:243
2041199522,33cyclictest20582-21chrt19:01:373
2041199514,36cyclictest30875-21chrt19:31:423
2041199513,36cyclictest19949-21chrt23:31:553
2041199512,38cyclictest23534-21chrt22:42:543
2041199512,36cyclictest1225-21chrt21:11:383
2041199503,35cyclictest610-21chrt19:36:573
2041199503,35cyclictest22078-21chrt21:46:243
2041199503,35cyclictest22078-21chrt21:46:243
2041199503,34cyclictest5654-21taskset22:11:593
2041199503,33cyclictest5571-21chrt21:17:543
2041199503,33cyclictest5571-21chrt21:17:543
2041199503,30cyclictest22364-21chrt22:41:263
2041199503,30cyclictest22364-21chrt22:41:263
2041199502,35cyclictest24179-21chrt20:51:243
2041199502,35cyclictest24179-21chrt20:51:243
2041199502,34cyclictest8553-21chrt23:11:573
2041199493,35cyclictest14315-21chrt21:32:183
2041199492,34cyclictest20932-21taskset20:41:243
2041199492,34cyclictest11415-21chrt20:11:433
2041199492,33cyclictest1042-21chrt23:00:333
2041199484,33cyclictest21166-21chrt00:28:093
2041199483,34cyclictest11135-21chrt23:16:553
2041199483,34cyclictest11135-21chrt23:16:553
2041199483,33cyclictest32278-21chrt23:51:583
2041199483,32cyclictest7968-21chrt21:21:563
2041199483,32cyclictest28234-21chrt22:51:293
2041199483,32cyclictest28234-21chrt22:51:293
2041199483,31cyclictest26688-21chrt22:47:453
2041199483,31cyclictest16242-21chrt20:26:383
2041199482,35cyclictest19052-21chrt22:36:233
2041199482,32cyclictest27005-21chrt23:43:393
2041199482,32cyclictest27005-21chrt23:43:393
2041199473,36cyclictest17801-21chrt00:22:023
2041199473,35cyclictest27261-21chrt21:54:473
2041199473,35cyclictest27261-21chrt21:54:473
2041199473,34cyclictest3271-21chrt23:58:023
2041199473,34cyclictest3271-21chrt23:58:023
2041199473,34cyclictest28496-21taskset23:46:433
2041199472,34cyclictest13342-21chrt20:16:543
2041199472,31cyclictest15449-21chrt22:29:463
2041199463,30cyclictest13746-21chrt00:16:393
2041199463,30cyclictest13746-21chrt00:16:393
2041199454,32cyclictest2085-21chrt23:01:473
2041199453,29cyclictest12411-21chrt22:24:313
2041199447,20cyclictest0-21swapper/300:16:243
2041199423,27cyclictest23999-21chrt19:11:213
2041199413,30cyclictest26095-21taskset20:56:433
2041199412,29cyclictest16637-21taskset23:26:443
2041199412,29cyclictest16637-21taskset23:26:443
2041199402,27cyclictest25864-21taskset19:16:363
2041199402,25cyclictest7598-21taskset20:00:543
2041199393,26cyclictest24474-21taskset20:51:413
2041199392,26cyclictest4147-21taskset19:48:143
2041199392,26cyclictest4147-21taskset19:48:143
2041199372,26cyclictest9969-21taskset20:06:493
2041199372,25cyclictest21086-21taskset20:41:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional