You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 20:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Mar 23, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17197215712,101sleep016315-21/usr/sbin/munin07:00:270
17197215712,101sleep016315-21/usr/sbin/munin07:00:270
178409911242,39cyclictest0-21swapper/211:14:562
178409910890,11cyclictest11356-21if_err_eth009:35:032
17840991053,30cyclictest16246-21seq08:38:132
17840991042,51cyclictest2569-21ssh10:14:532
17840991032,28cyclictest32546-21seq07:48:162
17840991032,28cyclictest32546-21seq07:48:162
1784099954,31cyclictest9605-21runrttasks08:15:422
1784099954,31cyclictest9605-21runrttasks08:15:422
229622940,6sleep10-21swapper/110:49:271
17832999354,34cyclictest0-21swapper/108:19:391
17832999354,34cyclictest0-21swapper/108:19:391
17840999241,41cyclictest0-21swapper/212:20:112
17840999150,36cyclictest0-21swapper/212:29:572
17840999149,36cyclictest6239-21sh12:09:552
17840999149,36cyclictest6239-21sh12:09:552
17840999148,35cyclictest0-21swapper/208:30:162
1784099913,26cyclictest25694-21runrttasks11:48:102
1784099913,26cyclictest25694-21runrttasks11:48:102
1783299913,31cyclictest5292-21taskset10:19:411
1783299912,53cyclictest13967-21apt-get09:39:511
1783299912,53cyclictest13967-21apt-get09:39:511
17832999112,67cyclictest26767-21apt-get11:50:111
17840999070,14cyclictest32450-21systemd-run11:59:402
1784099903,31cyclictest352-21runrttasks07:21:332
17840998968,14cyclictest17810-21cyclictest10:00:042
17840998963,18cyclictest1-21systemd09:08:062
17840998942,36cyclictest0-21swapper/210:48:472
17832998942,38cyclictest0-21swapper/107:04:441
17832998942,38cyclictest0-21swapper/107:04:441
17832998866,11cyclictest0-21swapper/109:20:021
1784099872,26cyclictest352-21runrttasks08:46:152
17832998746,33cyclictest0-21swapper/109:19:391
1783299872,25cyclictest352-21runrttasks10:53:521
17840998661,14cyclictest0-21swapper/207:35:012
1784099862,38cyclictest3522-21fschecks_time12:05:002
1783299863,36cyclictest352-21runrttasks10:12:411
1783299863,36cyclictest16381-21idleruntime-cro08:39:391
1783299863,36cyclictest10998-21ssh12:17:141
1783299862,34cyclictest352-21runrttasks10:38:581
17820998656,24cyclictest19706-21apt-key09:49:420
17820998656,24cyclictest19706-21apt-key09:49:420
17840998542,35cyclictest0-21swapper/208:20:142
17840998542,35cyclictest0-21swapper/208:20:142
17840998539,39cyclictest0-21swapper/211:24:412
1784099852,7cyclictest0-21swapper/208:24:542
1784099852,7cyclictest0-21swapper/208:24:542
1784099852,43cyclictest352-21runrttasks09:24:402
17832998538,40cyclictest0-21swapper/109:10:031
17832998538,40cyclictest0-21swapper/109:10:031
1783299853,30cyclictest18470-21/usr/sbin/munin08:44:511
17832998532,45cyclictest0-21swapper/107:20:031
1783299852,27cyclictest5722-21latency08:05:031
17840998466,12cyclictest550-21unixbench_multi10:10:172
17840998439,37cyclictest0-21swapper/212:15:112
17840998436,42cyclictest0-21swapper/209:40:152
17840998436,42cyclictest0-21swapper/209:40:152
1784099843,40cyclictest3471-21awk07:59:392
1784099842,38cyclictest31278-21apt-get09:14:502
17832998459,16cyclictest25174-21latency_hist09:04:451
17832998459,16cyclictest25174-21latency_hist09:04:451
1783299843,37cyclictest576-21fschecks_time12:00:001
1783299843,37cyclictest576-21fschecks_time12:00:001
1783299843,36cyclictest7102-21cat08:10:001
1783299843,36cyclictest7102-21cat08:10:001
1783299843,34cyclictest10971-21/usr/sbin/munin09:34:551
17820998469,9cyclictest200-21systemd-journal12:14:400
17820998469,9cyclictest200-21systemd-journal12:14:400
1784099833,68cyclictest22622-21/usr/sbin/munin08:55:542
1784099833,38cyclictest25822-21munin-run09:59:412
1784099833,38cyclictest25822-21munin-run09:59:412
1784099832,33cyclictest4204-21/usr/sbin/munin08:00:082
1784099832,33cyclictest25762-21/usr/sbin/munin07:25:162
1784099832,32cyclictest22548-21/usr/sbin/munin07:15:152
17832998365,11cyclictest0-21swapper/110:40:141
1783299835,35cyclictest23485-21latency_hist08:59:421
17832998345,31cyclictest9417-21timerwakeupswit10:25:141
17832998345,31cyclictest9417-21timerwakeupswit10:25:141
17832998334,11cyclictest58-21kcompactd009:59:211
17832998334,11cyclictest58-21kcompactd009:59:211
1783299833,35cyclictest27432-21grep07:30:181
17832998313,26cyclictest12308-21sh11:25:101
1784099827,33cyclictest15222-21cpuspeed12:24:552
17840998265,10cyclictest25593-21kworker/u17:3-rpciod10:35:032
1784099825,31cyclictest20190-21/usr/sbin/munin08:49:572
17840998247,12cyclictest181rcu_preempt09:54:422
1784099823,7cyclictest0-21swapper/207:39:552
1784099823,36cyclictest27825-21modprobe07:34:402
1784099823,33cyclictest18415-21grep10:40:142
1784099822,39cyclictest2451-21latency_hist11:09:422
1784099822,38cyclictest352-21runrttasks10:21:552
1784099822,32cyclictest7138-21sort08:10:012
1784099822,32cyclictest7138-21sort08:10:012
17832998244,33cyclictest25266-21kworker/u17:2-rpciod09:24:511
17832998244,33cyclictest25266-21kworker/u17:2-rpciod09:24:511
1783299824,33cyclictest5192-21ssh12:07:411
1783299823,34cyclictest6191-21/usr/sbin/munin12:10:131
1783299823,34cyclictest6191-21/usr/sbin/munin12:10:131
1783299823,32cyclictest11440-21/usr/sbin/munin10:30:051
1783299823,21cyclictest11790-21apt-get08:24:501
1783299823,21cyclictest11790-21apt-get08:24:501
17832998229,47cyclictest0-21swapper/108:30:021
1783299822,35cyclictest26901-21ssh10:00:171
17820998265,9cyclictest25266-21kworker/u17:2+rpciod09:20:010
17820998260,16cyclictest21929-21latency_hist08:54:430
17820998253,24cyclictest8214-21kworker/u19:3-rpciod10:31:020
17820998251,25cyclictest20034-21ssh10:44:340
1784099814,67cyclictest23571-21/usr/sbin/munin08:59:512
1784099814,67cyclictest23571-21/usr/sbin/munin08:59:512
17840998141,33cyclictest0-21swapper/210:05:052
17840998139,35cyclictest0-21swapper/211:50:312
1784099813,7cyclictest0-21swapper/210:54:542
1784099813,36cyclictest21481-21timerandwakeup11:40:152
1784099813,33cyclictest19966-21latency_hist09:49:432
1784099813,33cyclictest19966-21latency_hist09:49:432
1784099813,31cyclictest5499-21/usr/sbin/munin08:04:592
1784099813,23cyclictest21296-21runrttasks07:13:492
1784099812,34cyclictest23388-21ssh10:49:542
17832998146,25cyclictest27630-21kworker/u18:2+nfsiod11:16:471
1783299814,33cyclictest32475-21latency_hist11:59:411
1783299813,38cyclictest29037-21ssh10:04:501
1783299813,37cyclictest18409-21apt-key08:44:431
1783299813,37cyclictest18409-21apt-key08:44:431
1783299813,35cyclictest352-21runrttasks09:49:461
1783299813,34cyclictest18292-21diskmemload12:30:001
1783299813,32cyclictest352-21runrttasks11:06:331
1783299812,34cyclictest25232-21/usr/sbin/munin09:05:161
1784099803,6cyclictest0-21swapper/210:29:552
1784099803,36cyclictest31963-21munin-run11:04:402
1784099803,32cyclictest5520-21/usr/sbin/munin09:25:562
1784099803,32cyclictest5520-21/usr/sbin/munin09:25:562
1784099803,31cyclictest9096-21/usr/sbin/munin10:25:092
1784099803,31cyclictest9096-21/usr/sbin/munin10:25:092
1784099803,30cyclictest28778-21/usr/sbin/munin09:10:082
1784099803,30cyclictest28778-21/usr/sbin/munin09:10:082
1783299805,33cyclictest23564-21/usr/sbin/munin11:45:161
1783299805,33cyclictest23564-21/usr/sbin/munin11:45:161
17832998036,37cyclictest0-21swapper/111:00:171
1783299803,36cyclictest20595-21/usr/sbin/munin11:40:031
1783299803,34cyclictest1825-21idleruntime-cro07:54:391
1783299803,33cyclictest1-21systemd10:59:401
1783299802,37cyclictest17892-21ssh11:35:021
17820998060,14cyclictest27878-21systemd-run07:34:410
17840997938,34cyclictest0-21swapper/207:50:172
1784099793,35cyclictest2752-21/usr/sbin/munin11:09:552
1784099793,35cyclictest2752-21/usr/sbin/munin11:09:552
1784099793,30cyclictest8671-21ssh09:30:112
1784099793,30cyclictest19493-21wc07:09:422
1784099793,30cyclictest19493-21wc07:09:422
1784099792,38cyclictest12262-21ntp_kernel_pll_11:25:082
1784099792,35cyclictest15621-21ls11:30:162
1784099792,33cyclictest18659-21grep11:35:162
17832997965,10cyclictest17810-21cyclictest07:05:011
17832997965,10cyclictest17810-21cyclictest07:05:011
1783299794,32cyclictest29703-21/usr/sbin/munin07:39:571
1783299793,33cyclictest5275-21cat08:04:431
1783299793,32cyclictest15207-21/usr/sbin/munin11:30:071
1783299793,29cyclictest11707-21latency_hist08:24:421
1783299793,29cyclictest11707-21latency_hist08:24:421
1783299792,37cyclictest5539-21ssh10:19:541
1783299792,35cyclictest9748-21runrttasks11:20:161
17832997911,29cyclictest21838-21date08:54:431
1784099783,6cyclictest0-21swapper/211:59:502
1784099783,6cyclictest0-21swapper/211:59:502
1784099783,33cyclictest17905-21latency_hist07:04:442
1784099783,33cyclictest17905-21latency_hist07:04:442
1784099782,32cyclictest16690-21/usr/sbin/munin08:39:562
1784099782,32cyclictest16690-21/usr/sbin/munin08:39:562
1783299783,31cyclictest12141-21/usr/sbin/munin12:20:011
1783299782,37cyclictest25871-21vmstat07:25:191
1783299782,36cyclictest32283-21unixbench_singl07:45:171
1783299782,36cyclictest32283-21unixbench_singl07:45:171
1783299782,36cyclictest3170-21irqstats11:10:031
1783299782,36cyclictest3170-21irqstats11:10:031
1783299782,36cyclictest15409-21fschecks_time12:25:011
17820997865,9cyclictest20129-21kworker/u17:1+rpciod10:56:060
17820997859,14cyclictest20830-21kworker/u19:2+events_unbound11:19:380
17820997859,13cyclictest26224-21latency_hist11:49:420
17820997859,13cyclictest26224-21latency_hist11:49:420
1783299777,8cyclictest0-21swapper/107:55:151
1783299773,24cyclictest8168-21cstates09:29:561
17832997729,42cyclictest0-21swapper/109:45:161
17832997729,42cyclictest0-21swapper/109:45:161
17832997723,44cyclictest0-21swapper/107:34:501
17820997756,15cyclictest20376-21latency_hist11:39:410
17820997750,11cyclictest14757-21ssh11:29:570
1783299763,36cyclictest352-21runrttasks07:09:571
1783299762,34cyclictest22188-21ntp_offset07:15:071
17820997662,9cyclictest9684-21kworker/u17:2+rpciod10:09:260
17820997661,9cyclictest29583-21rm09:10:550
17820997661,9cyclictest29583-21rm09:10:550
17820997553,16cyclictest10770-21processes08:20:110
17820997553,16cyclictest10770-21processes08:20:110
1782099753,42cyclictest11827-21ls09:35:140
17820997459,9cyclictest24940-21diskmemload11:03:130
17820997447,11cyclictest4409-21ssh10:17:260
17820997417,28cyclictest22793-21munin-run09:54:390
17820997359,9cyclictest17810-21cyclictest07:37:060
17820997357,12cyclictest17810-21cyclictest07:44:240
17820997355,12cyclictest3729-21timerwakeupswit11:10:150
17820997355,12cyclictest3729-21timerwakeupswit11:10:150
17820997353,13cyclictest365-21ntpd08:44:400
17820997353,13cyclictest365-21ntpd08:44:400
17820997352,15cyclictest27493-21sh10:01:120
17820997258,10cyclictest17810-21cyclictest11:50:490
17820997257,9cyclictest16199-21ssh09:43:290
17820997257,9cyclictest16199-21ssh09:43:290
17820997257,10cyclictest58-21kcompactd007:14:540
17820997253,13cyclictest9301-21sh11:20:080
17820997236,10cyclictest0-21swapper/010:49:500
63002710,5sleep30-21swapper/311:15:063
17820997156,8cyclictest0-21swapper/010:25:150
17820997156,8cyclictest0-21swapper/010:25:150
17820997153,12cyclictest7701-21ssh10:23:220
17820997149,9cyclictest181rcu_preempt10:36:480
17820997146,20cyclictest17810-21cyclictest09:30:070
1782099713,54cyclictest11786-21cpuspeed11:24:540
1782099713,48cyclictest352-21runrttasks10:49:100
17820997053,11cyclictest17810-21cyclictest11:44:220
1782099705,29cyclictest12161-21apt-get12:19:490
17820997034,27cyclictest11962-21grep08:25:000
17820997034,27cyclictest11962-21grep08:25:000
17820996950,13cyclictest25932-21latency_hist09:59:430
17820996950,13cyclictest25932-21latency_hist09:59:430
1782099693,57cyclictest9298-21ssh12:14:540
1782099693,33cyclictest32114-21switchtime09:15:140
17820996849,14cyclictest19928-21ssh12:32:220
17820996849,12cyclictest277-21dbus-daemon12:29:400
17820996846,15cyclictest23804-21fschecks_count09:00:020
17820996846,15cyclictest23804-21fschecks_count09:00:020
1782099684,38cyclictest488-21fschecks_count11:59:590
1782099684,38cyclictest488-21fschecks_count11:59:590
17820996839,20cyclictest23317-21modprobe08:59:390
17820996746,15cyclictest20523-21ntp_kernel_pll_07:10:100
1782099673,36cyclictest2594-21latency_hist11:09:420
17820996719,31cyclictest5377-21cpu08:04:550
17820996648,12cyclictest0-21swapper/008:34:420
17820996645,15cyclictest3658-21grep12:05:040
1782099663,51cyclictest15222-21fschecks_count08:35:010
1782099663,34cyclictest28128-21date09:09:420
1782099663,34cyclictest19581-21munin-node07:54:510
1782099663,33cyclictest352-21runrttasks07:45:080
1782099663,33cyclictest352-21runrttasks07:45:080
17820996625,32cyclictest663-21vmstat10:10:190
17820996611,44cyclictest26339-21latency_hist07:29:420
1784499656,36cyclictest0-21swapper/309:24:583
1784499656,36cyclictest0-21swapper/309:24:583
1782099653,35cyclictest352-21runrttasks11:57:300
1782099644,32cyclictest3909-21fschecks_count08:00:020
1782099643,47cyclictest18240-21fschecks_time07:05:010
1782099643,47cyclictest18240-21fschecks_time07:05:010
1782099643,46cyclictest18809-21fschecks_time08:45:310
1782099643,34cyclictest4505-21apt-get09:24:530
1782099643,34cyclictest4505-21apt-get09:24:530
186752630,7sleep30-21swapper/311:35:163
17820996346,12cyclictest0-21swapper/008:15:050
17820996346,12cyclictest0-21swapper/008:15:050
1782099633,34cyclictest7665-21switchtime08:10:130
1782099633,34cyclictest7665-21switchtime08:10:130
17844996210,39cyclictest0-21swapper/311:50:113
1782099623,25cyclictest1815-21modprobe07:54:390
163112624,13sleep30-21swapper/306:59:493
163112624,13sleep30-21swapper/306:59:493
1782099603,37cyclictest23663-21latency07:20:060
1784499583,39cyclictest23106-21taskset09:54:503
1784499583,39cyclictest23106-21taskset09:54:503
1784499574,35cyclictest17975-21chrt12:29:423
1784499573,43cyclictest29309-21taskset10:59:513
1784499563,40cyclictest19421-21chrt09:49:153
1784499563,40cyclictest19421-21chrt09:49:153
1784499563,39cyclictest27706-21taskset10:01:483
1784499562,44cyclictest782-21taskset10:10:403
1784499554,38cyclictest14070-21chrt11:28:563
1784499553,39cyclictest18376-21chrt10:40:143
1784499553,36cyclictest25173-21chrt09:04:453
1784499553,36cyclictest25173-21chrt09:04:453
1784499552,40cyclictest16432-21chrt11:32:123
1784499544,37cyclictest30433-21taskset09:13:163
1784499544,37cyclictest30433-21taskset09:13:163
1784499542,33cyclictest10941-21taskset09:34:443
1784499534,37cyclictest13935-21chrt08:30:103
1784499534,35cyclictest9468-21taskset12:14:593
1784499533,38cyclictest2680-21taskset12:03:573
1784499533,38cyclictest2680-21taskset12:03:573
1784499533,37cyclictest24692-21taskset07:24:423
1784499533,37cyclictest18542-21taskset08:45:253
1784499525,34cyclictest14694-21taskset12:24:193
1784499524,35cyclictest26491-21chrt10:54:563
1784499523,36cyclictest21003-21chrt10:45:083
1784499523,33cyclictest3314-21chrt12:04:543
1784499523,32cyclictest2679-21chrt10:14:573
1784499522,34cyclictest22524-21chrt07:15:143
1784499514,38cyclictest24725-21taskset11:45:253
1784499514,38cyclictest24725-21taskset11:45:253
1784499514,29cyclictest29808-21chrt07:39:573
1784499513,37cyclictest29617-21chrt10:05:083
1784499513,36cyclictest18018-21chrt08:44:403
1784499513,36cyclictest18018-21chrt08:44:403
1784499513,32cyclictest15230-21chrt09:40:443
1784499513,32cyclictest15230-21chrt09:40:443
1784499512,39cyclictest850-21chrt07:50:053
1784499512,36cyclictest22973-21chrt08:55:583
1784499512,35cyclictest3045-21chrt09:20:383
1784499504,36cyclictest25516-21chrt09:05:013
1784499504,35cyclictest11602-21taskset11:24:433
1784499504,32cyclictest21457-21chrt07:14:423
1784499503,37cyclictest27306-21chrt07:30:173
1784499503,37cyclictest15090-21chrt10:35:113
1784499503,35cyclictest2229-21taskset07:54:593
1784499503,33cyclictest20880-21chrt09:51:013
1784499502,35cyclictest13744-21taskset10:33:413
1784499502,35cyclictest11054-21chrt10:29:283
1784499502,35cyclictest11054-21chrt10:29:283
1784499502,32cyclictest15111-21chrt08:34:563
1784499494,35cyclictest25593-21chrt07:25:153
1784499494,32cyclictest5518-21taskset10:19:513
1784499494,31cyclictest25618-21chrt10:53:353
1784499493,37cyclictest1474-21chrt11:06:523
1784499493,33cyclictest2977-21taskset11:10:003
1784499493,33cyclictest2977-21taskset11:10:003
1784499493,31cyclictest20860-21taskset08:50:153
1784499493,29cyclictest5274-21chrt08:04:433
1784499492,37cyclictest30550-21taskset11:55:143
1784499484,35cyclictest22489-21chrt11:42:293
1784499483,33cyclictest20051-21chrt12:32:423
1784499483,32cyclictest7310-21chrt12:10:183
1784499483,32cyclictest7310-21chrt12:10:183
1784499474,30cyclictest8041-21chrt09:29:503
1784499473,31cyclictest5719-21chrt08:05:033
1784499473,30cyclictest31470-21chrt09:14:583
1784499472,31cyclictest28269-21chrt07:34:593
1784499463,31cyclictest6914-21chrt08:09:453
1784499463,31cyclictest6914-21chrt08:09:453
1784499463,28cyclictest10014-21taskset08:19:413
1784499463,28cyclictest10014-21taskset08:19:413
1784499452,30cyclictest19348-21taskset07:09:393
1784499452,30cyclictest19348-21taskset07:09:393
1784499423,26cyclictest13247-21taskset08:29:423
1784499423,26cyclictest13247-21taskset08:29:423
1784499422,27cyclictest32087-21chrt07:45:153
1784499422,27cyclictest32087-21chrt07:45:153
1784499413,26cyclictest10169-21chrt08:19:503
1784499413,26cyclictest10169-21chrt08:19:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional