You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-04 - 11:13
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sun Jan 04, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
461499116100,10cyclictest4927-21kworker/u19:1-nfsiod00:10:110
3367210710,62sleep13422-21grep19:05:201
46149910681,17cyclictest24786-21sh21:29:530
436421067,17sleep20-21swapper/219:08:352
46239910152,42cyclictest7896-21latency_hist00:29:531
46149910157,7cyclictest181rcu_preempt22:37:550
4614999736,8cyclictest181rcu_preempt19:20:270
4614999736,8cyclictest181rcu_preempt19:20:270
4631999674,11cyclictest0-21swapper/219:29:532
4623999531,35cyclictest0-21swapper/121:48:441
4623999531,35cyclictest0-21swapper/121:48:441
446029510,49sleep04468-21modprobe19:09:510
4623999448,36cyclictest0-21swapper/122:09:501
4623999439,42cyclictest0-21swapper/123:25:031
4623999439,42cyclictest0-21swapper/123:25:031
4623999432,32cyclictest0-21swapper/119:30:021
4614999473,14cyclictest4601-21cyclictest23:00:140
4614999442,45cyclictest0-21swapper/000:25:150
461499943,69cyclictest8645-21/usr/sbin/munin00:30:190
4623999327,44cyclictest0-21swapper/120:04:361
4623999327,36cyclictest0-21swapper/121:55:131
4614999356,27cyclictest0-21swapper/022:55:270
463199922,66cyclictest20212-21apt-get19:55:052
463199922,66cyclictest20212-21apt-get19:55:052
4623999129,34cyclictest0-21swapper/123:35:101
461499914,49cyclictest20013-21apt-get23:05:090
461499913,67cyclictest19715-21apt-get19:55:050
461499913,67cyclictest19715-21apt-get19:55:050
4631999069,11cyclictest0-21swapper/223:00:202
463199903,65cyclictest11973-21diskmemload23:05:102
462399903,53cyclictest32670-21seq20:32:271
4623999021,39cyclictest341-21diskmemload21:41:361
4614999063,18cyclictest4889-21latency_hist00:24:520
4614999060,12cyclictest181rcu_preempt21:39:510
4614999060,12cyclictest181rcu_preempt21:39:510
4623998918,38cyclictest0-21swapper/120:11:451
4631998840,20cyclictest4601-21cyclictest19:52:342
4631998840,12cyclictest181rcu_preempt21:44:392
4623998821,33cyclictest352-21runrttasks20:56:521
4614998836,26cyclictest15-21ksoftirqd/020:55:280
463199877,68cyclictest28231-21apt-get00:10:092
4631998739,18cyclictest181rcu_preempt22:49:472
4623998747,35cyclictest4601-21cyclictest22:53:461
4623998661,18cyclictest1-21systemd22:46:441
462399864,71cyclictest0-21swapper/120:15:051
4623998638,40cyclictest0-21swapper/100:30:451
4623998638,39cyclictest0-21swapper/119:50:281
4614998659,18cyclictest11969-21latency_hist21:09:530
4614998657,20cyclictest31655-21fschecks_time00:15:130
4614998657,20cyclictest31655-21fschecks_time00:15:130
4614998654,15cyclictest181rcu_preempt19:40:150
4614998640,22cyclictest181rcu_preempt23:59:390
4614998625,25cyclictest0-21swapper/020:46:170
4631998546,10cyclictest181rcu_preempt21:32:272
4631998536,16cyclictest181rcu_preempt23:17:002
4623998540,40cyclictest0-21swapper/123:55:261
462399853,72cyclictest0-21swapper/121:30:061
4623998535,42cyclictest0-21swapper/119:40:321
4614998565,11cyclictest0-21swapper/022:10:170
4631998466,10cyclictest12169-21unin-run23:44:522
4631998465,9cyclictest0-21swapper/221:05:122
4631998465,14cyclictest4601-21cyclictest21:59:532
4631998459,11cyclictest181rcu_preempt23:22:002
4623998435,41cyclictest0-21swapper/100:10:321
4614998463,11cyclictest0-21swapper/019:49:530
4631998341,8cyclictest181rcu_preempt23:46:502
4631998334,17cyclictest181rcu_preempt22:32:242
4631998331,17cyclictest181rcu_preempt21:28:212
462399833,58cyclictest352-21runrttasks20:51:141
462399833,54cyclictest0-21swapper/119:45:141
462399833,46cyclictest8054-21cat21:54:541
4623998326,36cyclictest0-21swapper/122:15:071
4623998325,43cyclictest277-21dbus-daemon20:24:521
4614998365,12cyclictest12908-21ntp_kernel_pll_21:10:210
4614998361,15cyclictest4601-21cyclictest21:33:490
4614998323,33cyclictest15-21ksoftirqd/020:30:230
4614998321,35cyclictest15-21ksoftirqd/020:12:380
4631998260,12cyclictest13796-21grep22:55:132
463199824,59cyclictest8456-21apt-get00:30:182
4631998243,13cyclictest181rcu_preempt23:26:332
4631998243,13cyclictest181rcu_preempt23:26:332
4631998237,9cyclictest181rcu_preempt22:43:112
4631998237,9cyclictest181rcu_preempt22:43:112
4631998235,12cyclictest181rcu_preempt21:46:012
4631998235,12cyclictest181rcu_preempt21:46:012
462399823,51cyclictest352-21runrttasks20:38:181
462399823,51cyclictest352-21runrttasks20:38:181
4623998231,43cyclictest0-21swapper/121:19:541
4614998265,8cyclictest0-21swapper/019:10:170
4631998160,13cyclictest4601-21cyclictest21:04:532
4631998160,13cyclictest4601-21cyclictest21:04:532
4631998146,13cyclictest181rcu_preempt22:51:512
4631998136,13cyclictest181rcu_preempt21:39:352
4631998136,13cyclictest181rcu_preempt21:39:352
4631998136,10cyclictest181rcu_preempt22:14:172
4631998131,13cyclictest181rcu_preempt21:54:122
462399814,71cyclictest0-21swapper/122:00:461
462399814,66cyclictest0-21swapper/119:35:041
461499818,39cyclictest15-21ksoftirqd/022:20:140
461499818,39cyclictest15-21ksoftirqd/022:20:140
4614998160,11cyclictest0-21swapper/019:34:540
4614998156,8cyclictest181rcu_preempt23:44:180
461499814,46cyclictest15-21ksoftirqd/000:35:560
461499814,46cyclictest15-21ksoftirqd/000:35:560
4614998125,31cyclictest15-21ksoftirqd/023:14:350
4614998125,31cyclictest15-21ksoftirqd/023:14:350
4631998056,13cyclictest0-21swapper/222:19:542
4631998054,15cyclictest0-21swapper/220:59:542
4631998042,13cyclictest181rcu_preempt19:10:222
4631998035,12cyclictest181rcu_preempt22:38:072
4631998035,10cyclictest181rcu_preempt23:14:062
4631998035,10cyclictest181rcu_preempt23:14:062
4631998034,16cyclictest181rcu_preempt20:49:012
4631998032,12cyclictest181rcu_preempt21:10:202
4623998062,13cyclictest4601-21cyclictest00:37:481
4623998062,13cyclictest4601-21cyclictest00:37:481
4623998012,39cyclictest0-21swapper/123:13:351
4623998012,39cyclictest0-21swapper/123:13:351
4623998010,35cyclictest352-21runrttasks21:09:001
461499805,41cyclictest18846-21apt-get21:20:180
4614998037,16cyclictest181rcu_preempt23:18:280
4614998030,16cyclictest181rcu_preempt21:55:410
288532803,7sleep00-21swapper/022:27:170
4631997962,11cyclictest16168-21ntp_offset23:50:212
4631997934,16cyclictest181rcu_preempt21:16:292
4631997928,12cyclictest181rcu_preempt00:18:432
4631997928,12cyclictest181rcu_preempt00:18:432
4623997960,11cyclictest6387-21munin-node22:40:021
4623997960,11cyclictest6387-21munin-node22:40:021
462399794,64cyclictest0-21swapper/119:15:081
462399794,64cyclictest0-21swapper/119:15:081
4623997910,49cyclictest0-21swapper/121:25:281
4614997926,25cyclictest15-21ksoftirqd/022:00:590
4631997844,10cyclictest181rcu_preempt00:00:162
4631997844,10cyclictest181rcu_preempt00:00:162
4631997837,16cyclictest181rcu_preempt20:40:252
4631997836,16cyclictest181rcu_preempt22:06:002
4631997833,16cyclictest181rcu_preempt00:36:032
4631997833,16cyclictest181rcu_preempt00:36:032
4631997824,27cyclictest39-21ksoftirqd/221:22:062
4623997858,13cyclictest200-21systemd-journal22:33:441
462399785,62cyclictest0-21swapper/120:40:071
462399784,61cyclictest12378-21ssh23:44:541
462399783,63cyclictest0-21swapper/122:35:041
4623997830,37cyclictest0-21swapper/121:03:341
4623997830,37cyclictest0-21swapper/121:03:341
461499782,44cyclictest15-21ksoftirqd/021:40:300
4631997752,8cyclictest181rcu_preempt20:24:512
4631997751,10cyclictest181rcu_preempt20:00:142
4631997741,8cyclictest181rcu_preempt19:19:542
4631997741,8cyclictest181rcu_preempt19:19:542
4631997737,11cyclictest181rcu_preempt20:15:192
4631997735,15cyclictest181rcu_preempt23:58:232
4631997729,18cyclictest181rcu_preempt19:20:102
4631997729,18cyclictest181rcu_preempt19:20:102
4623997753,18cyclictest0-21swapper/123:00:291
4623997735,35cyclictest0-21swapper/123:30:111
4623997732,25cyclictest17671-21kworker/u17:0-rpciod00:05:351
462399772,64cyclictest0-21swapper/122:25:041
4623997725,41cyclictest0-21swapper/122:10:021
4614997755,7cyclictest181rcu_preempt20:03:410
4614997755,13cyclictest0-21swapper/019:29:530
4614997750,18cyclictest18426-21apt-key21:19:540
4614997749,19cyclictest11675-21timerandwakeup22:51:220
4614997721,31cyclictest15-21ksoftirqd/021:00:450
4614997721,31cyclictest15-21ksoftirqd/021:00:450
4631997644,7cyclictest181rcu_preempt23:34:412
4631997634,7cyclictest181rcu_preempt00:07:392
4631997634,10cyclictest181rcu_preempt22:03:232
4631997633,10cyclictest181rcu_preempt22:25:492
462399763,68cyclictest0-21swapper/121:10:061
4623997623,29cyclictest0-21swapper/122:56:591
461499767,41cyclictest15-21ksoftirqd/023:20:280
4614997651,19cyclictest24960-21latency_hist20:09:540
4614997646,11cyclictest181rcu_preempt22:40:130
4614997646,11cyclictest181rcu_preempt22:40:130
461499763,41cyclictest15-21ksoftirqd/020:44:530
461499763,35cyclictest13489-21grep23:45:280
4614997624,25cyclictest15-21ksoftirqd/019:15:160
4614997624,25cyclictest15-21ksoftirqd/019:15:160
4631997556,10cyclictest0-21swapper/220:05:252
4631997550,17cyclictest0-21swapper/219:30:142
4631997534,8cyclictest181rcu_preempt22:22:542
4631997534,8cyclictest181rcu_preempt22:22:542
4631997534,7cyclictest181rcu_preempt00:26:272
4631997532,7cyclictest181rcu_preempt00:21:522
4631997523,10cyclictest0-21swapper/219:42:012
4631997519,30cyclictest39-21ksoftirqd/223:39:242
462399758,45cyclictest0-21swapper/119:14:241
462399754,66cyclictest0-21swapper/123:20:061
4623997532,35cyclictest0-21swapper/119:20:271
4623997532,35cyclictest0-21swapper/119:20:271
4623997525,43cyclictest0-21swapper/123:09:111
4614997528,30cyclictest15-21ksoftirqd/020:25:150
4631997437,13cyclictest181rcu_preempt20:10:092
4623997453,15cyclictest4601-21cyclictest23:19:531
4623997453,15cyclictest4601-21cyclictest21:38:341
4623997453,15cyclictest4601-21cyclictest21:38:341
462399744,62cyclictest352-21runrttasks20:46:061
462399743,66cyclictest0-21swapper/120:05:061
462399742,65cyclictest19142-21sh21:20:521
4623997426,42cyclictest0-21swapper/100:03:391
4623997426,42cyclictest0-21swapper/100:03:391
4614997458,8cyclictest0-21swapper/022:17:240
4614997453,14cyclictest4601-21cyclictest21:49:510
4614997453,14cyclictest4601-21cyclictest21:49:510
4614997443,24cyclictest0-21swapper/022:45:120
4614997427,21cyclictest15-21ksoftirqd/019:54:240
345327415,33sleep30-21swapper/319:05:223
4631997339,12cyclictest181rcu_preempt19:37:082
4631997334,12cyclictest181rcu_preempt20:27:312
4623997350,14cyclictest0-21swapper/119:56:031
4623997350,14cyclictest0-21swapper/119:56:031
4623997335,31cyclictest0-21swapper/100:20:181
4614997356,10cyclictest0-21swapper/020:20:270
4614997354,13cyclictest370-21snmpd23:37:440
4614997352,8cyclictest181rcu_preempt23:50:440
4614997343,7cyclictest181rcu_preempt23:26:150
4614997343,7cyclictest181rcu_preempt23:26:150
4614997340,8cyclictest181rcu_preempt00:00:190
4614997340,8cyclictest181rcu_preempt00:00:190
4614997338,18cyclictest181rcu_preempt21:52:160
461499733,35cyclictest30214-21cpu22:30:090
4614997332,19cyclictest0-21swapper/019:35:140
4614997320,9cyclictest181rcu_preempt23:30:180
4614997311,32cyclictest15-21ksoftirqd/020:15:420
463199723,28cyclictest39-21ksoftirqd/219:45:212
4623997251,15cyclictest328-21sh00:16:231
4623997251,15cyclictest328-21sh00:16:231
4623997230,30cyclictest0-21swapper/122:22:291
4623997230,30cyclictest0-21swapper/122:22:291
4614997225,25cyclictest0-21swapper/000:05:150
4631997134,7cyclictest181rcu_preempt20:54:512
462399713,60cyclictest31352-21latency_hist20:29:531
4614997114,30cyclictest15-21ksoftirqd/022:08:450
4631997043,8cyclictest181rcu_preempt20:30:202
4631997035,7cyclictest181rcu_preempt20:35:432
4631997035,7cyclictest181rcu_preempt20:35:432
462399703,58cyclictest12707-21/usr/sbin/munin23:45:131
4614997021,27cyclictest15-21ksoftirqd/020:50:190
462399693,60cyclictest0-21swapper/123:50:031
462399684,34cyclictest11737-21apt-config19:29:541
4614996811,26cyclictest15-21ksoftirqd/020:37:080
4614996811,26cyclictest15-21ksoftirqd/020:37:080
4639996725,30cyclictest0-21swapper/300:10:093
4639996123,23cyclictest0-21swapper/322:50:403
463999602,36cyclictest20079-21chrt23:05:123
463999582,38cyclictest29799-21taskset22:29:463
4639995743,9cyclictest451rcuc/323:55:543
4639995714,27cyclictest0-21swapper/300:30:183
4639995541,9cyclictest451rcuc/320:57:503
4639995541,9cyclictest451rcuc/319:31:363
4639995541,8cyclictest451rcuc/300:29:033
463999553,40cyclictest9074-21chrt23:39:523
463999553,39cyclictest25998-21chrt21:30:283
463999552,39cyclictest28173-21chrt00:09:553
463999552,38cyclictest16640-21chrt22:07:533
4639995441,8cyclictest451rcuc/320:38:343
4639995441,8cyclictest451rcuc/320:38:343
4639995440,9cyclictest451rcuc/321:20:563
463999543,36cyclictest24709-21chrt22:20:263
463999543,36cyclictest24709-21chrt22:20:263
463999542,37cyclictest21715-21chrt22:15:273
4639995411,32cyclictest4170-21taskset20:45:053
4639995339,9cyclictest451rcuc/320:20:563
463999533,36cyclictest15479-21chrt21:15:073
463999533,34cyclictest31737-21chrt23:23:483
463999533,34cyclictest11273-21chrt21:59:543
463999532,35cyclictest19238-21taskset22:11:343
4639995239,9cyclictest451rcuc/320:30:003
4639995238,9cyclictest451rcuc/319:17:203
4639995238,9cyclictest451rcuc/319:17:203
463999523,40cyclictest19195-21chrt23:04:103
463999523,38cyclictest17105-21chrt19:45:163
463999523,35cyclictest6961-21taskset22:44:533
463999523,35cyclictest6961-21taskset22:44:533
463999523,31cyclictest28228-21chrt20:19:543
463999522,39cyclictest13364-21taskset23:45:273
463999522,36cyclictest3839-21chrt20:44:533
4639995212,27cyclictest0-21swapper/319:55:323
4639995212,27cyclictest0-21swapper/319:55:323
463999514,33cyclictest3614-21chrt22:39:513
4639995134,12cyclictest451rcuc/321:53:513
463999513,36cyclictest12016-21taskset21:09:523
463999512,38cyclictest24968-21chrt23:12:253
463999512,38cyclictest24968-21chrt23:12:253
463999512,38cyclictest11601-21taskset23:43:293
463999512,36cyclictest31470-21chrt20:29:543
463999512,34cyclictest22698-21chrt21:25:253
4639995036,9cyclictest451rcuc/319:13:593
4639995035,10cyclictest451rcuc/320:01:453
463999503,37cyclictest4165-21chrt23:30:283
463999503,35cyclictest13928-21chrt22:55:153
463999502,38cyclictest25819-21taskset20:10:263
463999502,37cyclictest12381-21taskset21:10:083
4639994935,9cyclictest451rcuc/323:17:083
4639994935,9cyclictest451rcuc/319:39:013
463999493,34cyclictest22716-21chrt00:00:273
463999493,34cyclictest22716-21chrt00:00:273
463999493,30cyclictest1851-21taskset21:44:553
463999492,37cyclictest11980-21chrt22:00:523
463999492,32cyclictest1961-21chrt21:45:073
463999492,32cyclictest1961-21chrt21:45:073
4639994833,10cyclictest451rcuc/320:50:583
4639994734,9cyclictest451rcuc/300:38:083
4639994734,9cyclictest451rcuc/300:38:083
463999473,32cyclictest16178-21taskset23:50:223
463999473,32cyclictest11749-21taskset19:29:543
463999473,27cyclictest10576-21chrt21:04:543
463999473,27cyclictest10576-21chrt21:04:543
4639994729,12cyclictest451rcuc/320:04:573
463999464,32cyclictest31862-21taskset00:15:183
463999464,32cyclictest31862-21taskset00:15:183
463999462,35cyclictest30198-21chrt22:30:063
463999452,34cyclictest7106-21chrt22:45:013
4639994431,8cyclictest451rcuc/319:52:303
4639994431,8cyclictest451rcuc/319:23:113
4639994431,8cyclictest451rcuc/319:23:113
463999442,33cyclictest1246-21chrt23:25:463
463999442,33cyclictest1246-21chrt23:25:463
4639994422,13cyclictest451rcuc/300:20:293
463999423,28cyclictest16438-21taskset19:44:193
4639994227,10cyclictest451rcuc/321:38:153
4639994227,10cyclictest451rcuc/321:38:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional