You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 23:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Aug 12, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30012213114,63sleep029991-21apt-get19:05:230
314839912646,31cyclictest13067-21apt-get19:50:090
30250211711,75sleep230251-21/usr/sbin/munin19:06:312
314839911437,41cyclictest2557-21/usr/sbin/munin23:35:200
314839910738,20cyclictest20486-21ssh21:30:460
314839910644,25cyclictest10963-21apt-get19:45:130
314839910638,34cyclictest0-21swapper/023:45:140
314839910638,29cyclictest18476-21ssh22:19:190
314839910631,52cyclictest0-21swapper/023:50:150
314839910631,52cyclictest0-21swapper/023:50:150
314839910579,17cyclictest28188-21latency_hist23:25:010
314839910541,23cyclictest29978-21sh22:36:050
314839910535,33cyclictest31033-21fschecks_count20:45:220
314839910440,27cyclictest11555-21ssh21:17:250
314839910432,44cyclictest0-21swapper/020:30:050
314839910354,43cyclictest0-21swapper/022:55:280
314839910354,43cyclictest0-21swapper/022:55:280
314839910341,23cyclictest203-21systemd-journal21:55:000
314839910341,23cyclictest203-21systemd-journal21:55:000
314839910340,25cyclictest8182-21/usr/sbin/munin00:35:250
314839910339,28cyclictest24764-21seq23:19:430
314839910339,26cyclictest20807-21ssh23:12:490
314839910339,26cyclictest20807-21ssh23:12:490
314839910338,30cyclictest25310-21ssh21:39:520
314839910338,26cyclictest3711-21sh21:55:300
314839910338,24cyclictest1046-21ssh23:32:330
314839910338,24cyclictest1046-21ssh23:32:330
314839910337,33cyclictest20285-21ssh00:02:420
314839910337,23cyclictest1082-21seq20:51:520
314839910242,23cyclictest10382-21/usr/sbin/munin19:40:350
314899910159,36cyclictest22550-21latency_hist20:20:011
314839910159,29cyclictest21135-21apt-get20:15:120
314839910143,24cyclictest28994-21ssh23:25:280
314839910139,26cyclictest3215-21ssh00:26:160
314839910139,25cyclictest15643-21/usr/sbin/munin23:05:120
314839910138,28cyclictest6696-21/usr/sbin/munin22:50:350
314839910138,28cyclictest6696-21/usr/sbin/munin22:50:350
314839910137,31cyclictest5945-21/usr/sbin/munin23:40:240
314839910041,24cyclictest13791-21/usr/sbin/munin22:10:360
314839910041,24cyclictest13791-21/usr/sbin/munin22:10:360
314839910041,21cyclictest23465-21taskset00:07:420
314839910040,24cyclictest1386-21taskset00:24:410
314839910039,28cyclictest30559-21ssh00:19:320
314839910039,28cyclictest30559-21ssh00:19:320
314839910039,28cyclictest30559-21ssh00:19:320
314839910036,29cyclictest6579-21/usr/sbin/munin19:30:250
314839910035,29cyclictest6747-21/usr/sbin/munin21:10:210
31483999939,26cyclictest353-21runrttasks21:44:400
31483999937,25cyclictest20227-21/usr/sbin/munin22:20:370
31483999936,25cyclictest32510-21/usr/sbin/munin22:40:270
31483999936,22cyclictest27919-21ssh22:33:570
31483999936,22cyclictest27919-21ssh22:33:570
31483999935,32cyclictest8145-21/usr/sbin/munin19:35:300
31483999935,24cyclictest29303-21/usr/sbin/munin21:45:230
31483999934,35cyclictest0-21swapper/023:02:500
31483999842,25cyclictest22481-21/usr/sbin/munin22:25:180
31483999836,26cyclictest17254-21/usr/sbin/munin21:25:400
31483999836,26cyclictest17254-21/usr/sbin/munin21:25:400
31483999836,25cyclictest5702-21ps00:30:310
31483999835,27cyclictest10034-21/usr/sbin/munin22:05:270
31483999834,32cyclictest18535-21/usr/sbin/munin20:05:400
31483999740,23cyclictest17139-21ssh23:57:550
31483999740,23cyclictest17139-21ssh23:57:550
31483999738,22cyclictest14936-21ssh21:22:500
31483999735,23cyclictest29201-21seq20:40:050
31483999731,24cyclictest0-21swapper/019:12:430
311092976,68sleep10-21swapper/119:07:111
3148999963,47cyclictest15886-21apt-get22:15:151
31483999635,29cyclictest0-21swapper/022:00:120
31483999635,28cyclictest353-21runrttasks22:48:210
31483999635,28cyclictest353-21runrttasks22:48:210
31483999630,24cyclictest0-21swapper/019:18:130
31483999551,33cyclictest0-21swapper/000:10:130
31483999551,33cyclictest0-21swapper/000:10:130
31483999535,28cyclictest5231-21/usr/sbin/munin19:25:340
31483999535,28cyclictest5231-21/usr/sbin/munin19:25:340
31483999533,30cyclictest353-21runrttasks20:04:440
31483999533,24cyclictest2911-21/usr/sbin/munin19:20:230
31483999531,30cyclictest14967-21idleruntime19:55:240
31489999444,42cyclictest14852-21latency_hist23:55:011
31489999444,42cyclictest14852-21latency_hist23:55:011
31489999440,46cyclictest0-21swapper/123:44:591
31483999438,23cyclictest27854-21runrttasks20:35:240
31483999338,20cyclictest4292-21seq21:01:230
31483999335,22cyclictest2612-21taskset20:55:510
31483999335,22cyclictest2612-21taskset20:55:510
31483999328,29cyclictest24443-21cpuspeed20:25:170
3149299924,78cyclictest11460-21apt-get19:45:182
3149299923,61cyclictest7983-21apt-get19:35:132
31489999256,31cyclictest31476-21cyclictest00:36:321
3148999924,35cyclictest23222-21kworker/u19:1+ext4-rsv-conversion21:51:171
3148999924,35cyclictest23222-21kworker/u19:1+ext4-rsv-conversion21:51:171
31492999152,13cyclictest181rcu_preempt19:34:492
31483999134,18cyclictest20848-21taskset20:14:560
31483999134,18cyclictest20848-21taskset20:14:560
31489999040,42cyclictest0-21swapper/100:35:011
31492998945,9cyclictest181rcu_preempt20:50:172
31492998941,13cyclictest181rcu_preempt22:24:132
31492998912,46cyclictest39-21ksoftirqd/221:57:152
31489998970,12cyclictest19020-21runrttasks23:10:191
31489998970,12cyclictest19020-21runrttasks23:10:191
3148999893,32cyclictest31581-21/usr/sbin/munin23:30:281
3148999893,32cyclictest31581-21/usr/sbin/munin23:30:281
3148999893,28cyclictest353-21runrttasks00:06:451
3148999892,36cyclictest8724-21ssh22:55:001
3148999892,36cyclictest8724-21ssh22:55:001
3148999892,31cyclictest353-21runrttasks22:37:181
31483998936,19cyclictest5936-21taskset21:06:530
31492998856,23cyclictest17643-21sh23:59:112
31492998856,23cyclictest17643-21sh23:59:112
31489998868,15cyclictest31476-21cyclictest20:05:451
31489998842,39cyclictest6204-21diskmemload21:20:331
3148999882,69cyclictest353-21runrttasks19:35:231
3148999882,30cyclictest22806-21cpu20:20:161
31483998831,27cyclictest23442-21mailstats20:20:330
3149299878,67cyclictest32336-21apt-get21:50:152
3149299878,67cyclictest32336-21apt-get21:50:152
3149299877,57cyclictest9260-21ssh22:55:152
3149299877,57cyclictest9260-21ssh22:55:152
31492998763,16cyclictest31476-21cyclictest22:40:022
3148999873,36cyclictest16893-21mailstats21:25:351
3148999873,36cyclictest16893-21mailstats21:25:351
3148999873,34cyclictest32305-21irqstats22:40:231
3148999865,31cyclictest18311-21/usr/sbin/munin00:00:291
3148999864,33cyclictest28752-21fschecks_time23:25:221
3148999864,26cyclictest25166-21idleruntime-cro22:29:591
3148999863,44cyclictest20079-21cat22:20:351
3148999863,32cyclictest3061-21/usr/sbin/munin21:55:101
3148999863,30cyclictest5461-21grep19:25:361
3148999863,30cyclictest5461-21grep19:25:361
31489998623,53cyclictest18142-21latency_hist00:00:021
31489998623,53cyclictest18142-21latency_hist00:00:021
3148999862,30cyclictest15626-21/usr/sbin/munin23:05:361
31489998620,24cyclictest9976-21cat22:05:251
31492998564,14cyclictest203-21systemd-journal23:51:552
31492998564,14cyclictest203-21systemd-journal23:51:552
3148999854,38cyclictest353-21runrttasks23:24:291
3148999854,29cyclictest6343-21taskset22:00:161
3148999854,27cyclictest6119-21/usr/sbin/munin19:30:241
3148999853,34cyclictest31923-21apt-key21:50:021
3148999853,33cyclictest4314-21latency_hist19:25:021
3148999853,32cyclictest30088-21timerwakeupswit20:40:331
3148999853,32cyclictest13598-21cat22:10:341
3148999853,32cyclictest13598-21cat22:10:341
3148999853,27cyclictest27578-21/usr/sbin/munin20:35:391
3148999852,35cyclictest26740-21unixbench_multi21:40:351
3148999852,34cyclictest32039-21mailstats00:20:541
3148999852,31cyclictest14741-21cpuspeed19:55:171
3149299846,37cyclictest39-21ksoftirqd/223:25:262
31492998441,13cyclictest181rcu_preempt00:05:382
31492998414,41cyclictest39-21ksoftirqd/220:03:492
31489998462,16cyclictest10371-21open_files21:15:321
3148999844,32cyclictest19296-21/usr/sbin/munin21:30:381
3148999844,29cyclictest28187-21tune2fs00:15:221
3148999844,29cyclictest28187-21tune2fs00:15:221
3148999844,29cyclictest28187-21tune2fs00:15:221
3148999844,29cyclictest2604-21/usr/sbin/munin22:45:321
3148999844,29cyclictest2604-21/usr/sbin/munin22:45:321
3148999844,28cyclictest6752-21ssh21:10:221
3148999843,31cyclictest13647-21ps19:50:301
3148999842,30cyclictest1333-21modprobe20:55:001
31489998421,25cyclictest22623-21cstates21:35:171
31489998420,29cyclictest9668-21cpu19:40:161
3148999833,31cyclictest5215-21munin-run23:39:591
3148999833,31cyclictest25473-21mailstats00:10:321
3148999833,31cyclictest25473-21mailstats00:10:321
3148999833,31cyclictest2072-21fschecks_count00:25:211
3148999833,28cyclictest353-21runrttasks20:48:411
31489998327,47cyclictest0-21swapper/121:05:371
3148999832,31cyclictest2641-21latency_hist19:20:021
3149299829,36cyclictest39-21ksoftirqd/220:45:342
31492998241,10cyclictest5350-21fschecks_time00:30:222
31492998219,35cyclictest39-21ksoftirqd/200:25:252
3148999825,38cyclictest1584-21/usr/sbin/munin20:55:271
3148999825,38cyclictest1584-21/usr/sbin/munin20:55:271
3148999824,32cyclictest8728-21/usr/sbin/munin23:45:301
3148999824,29cyclictest28341-21latency_hist22:35:011
3148999824,29cyclictest28341-21latency_hist22:35:011
3148999823,35cyclictest353-21runrttasks19:11:011
3148999823,33cyclictest24771-21ls20:25:261
3149299812,50cyclictest39-21ksoftirqd/223:06:122
3148999813,39cyclictest21987-21/usr/sbin/munin23:15:271
3148999813,30cyclictest17276-21unixbench_singl20:00:401
31489998123,47cyclictest0-21swapper/123:01:271
3148999812,30cyclictest3623-21latency21:00:261
31492998019,33cyclictest39-21ksoftirqd/219:15:052
3148999803,32cyclictest19883-21kernelversion20:10:261
3148999803,32cyclictest19883-21kernelversion20:10:261
3148999803,32cyclictest11506-21cpu19:45:371
3148999802,32cyclictest9513-21cat22:55:341
3148999802,32cyclictest9513-21cat22:55:341
31492997957,8cyclictest181rcu_preempt00:19:282
31492997957,8cyclictest181rcu_preempt00:19:282
31492997957,8cyclictest181rcu_preempt00:19:282
3149299794,36cyclictest39-21ksoftirqd/200:00:232
31492997942,17cyclictest181rcu_preempt20:09:482
31492997937,16cyclictest181rcu_preempt22:41:202
31492997926,31cyclictest39-21ksoftirqd/200:36:212
3149299792,43cyclictest19316-21apt-get21:30:162
31492997924,21cyclictest39-21ksoftirqd/222:29:192
3149299792,38cyclictest39-21ksoftirqd/220:21:512
3148999792,30cyclictest26605-21grep20:30:331
31492997857,17cyclictest31476-21cyclictest23:45:362
31492997828,12cyclictest181rcu_preempt23:13:092
31492997828,12cyclictest181rcu_preempt23:13:092
31492997824,27cyclictest39-21ksoftirqd/221:26:232
31492997824,27cyclictest39-21ksoftirqd/221:26:232
31492997814,42cyclictest39-21ksoftirqd/222:45:162
31492997814,42cyclictest39-21ksoftirqd/222:45:162
31492997811,35cyclictest39-21ksoftirqd/220:59:492
31492997811,35cyclictest39-21ksoftirqd/220:59:492
31492997746,12cyclictest181rcu_preempt21:49:312
31492997745,13cyclictest181rcu_preempt21:45:022
31492997738,13cyclictest181rcu_preempt22:11:042
31492997738,13cyclictest181rcu_preempt22:11:042
31492997736,14cyclictest181rcu_preempt23:15:242
31492997736,11cyclictest181rcu_preempt21:24:182
31492997730,25cyclictest39-21ksoftirqd/222:10:032
31492997718,34cyclictest39-21ksoftirqd/219:40:572
31492997716,27cyclictest39-21ksoftirqd/223:35:032
31492997657,13cyclictest10382-21sh21:15:322
31492997652,16cyclictest2028-21/usr/sbin/munin23:35:002
31492997644,10cyclictest181rcu_preempt23:01:072
31492997642,13cyclictest181rcu_preempt00:21:292
31492997629,15cyclictest181rcu_preempt22:00:062
31492997619,30cyclictest39-21ksoftirqd/220:00:022
31492997618,37cyclictest39-21ksoftirqd/220:41:232
3149299754,35cyclictest39-21ksoftirqd/220:25:402
31492997542,12cyclictest181rcu_preempt21:39:302
3149299752,52cyclictest39-21ksoftirqd/221:13:512
3149299749,27cyclictest39-21ksoftirqd/221:00:212
31492997444,21cyclictest27325-21sh20:34:592
31492997444,11cyclictest181rcu_preempt19:20:342
31492997436,16cyclictest181rcu_preempt22:15:232
31492997434,11cyclictest181rcu_preempt20:10:242
31492997434,11cyclictest181rcu_preempt20:10:242
3149299742,47cyclictest39-21ksoftirqd/223:40:582
31492997336,13cyclictest181rcu_preempt23:37:472
31492997329,12cyclictest181rcu_preempt23:22:592
31492997244,11cyclictest181rcu_preempt20:19:362
31492997237,12cyclictest181rcu_preempt19:50:232
3149299722,39cyclictest25410-21ps00:10:322
3149299722,39cyclictest25410-21ps00:10:322
31492997145,18cyclictest0-21swapper/220:40:022
31492997143,12cyclictest181rcu_preempt21:09:392
3149299712,41cyclictest39-21ksoftirqd/222:33:032
3149299712,41cyclictest39-21ksoftirqd/222:33:032
31492997123,25cyclictest39-21ksoftirqd/219:26:562
31492997123,25cyclictest39-21ksoftirqd/219:26:562
3149299712,27cyclictest5884-21cpu22:50:162
3149299712,27cyclictest5884-21cpu22:50:162
313902716,44sleep30-21swapper/319:10:033
31492996942,12cyclictest181rcu_preempt19:11:062
3150099683,28cyclictest31909-21chrt22:40:123
3150099683,28cyclictest23771-21chrt22:26:313
3150099673,29cyclictest25927-21chrt21:40:203
3150099672,27cyclictest8116-21chrt19:35:293
3150099672,26cyclictest30678-21chrt00:19:513
3150099672,26cyclictest30678-21chrt00:19:513
3150099672,26cyclictest30678-21chrt00:19:513
3150099663,26cyclictest19138-21chrt23:10:223
3150099663,26cyclictest19138-21chrt23:10:223
3150099663,24cyclictest27914-21taskset22:33:563
3150099663,24cyclictest27914-21taskset22:33:563
3150099662,25cyclictest7591-21chrt21:10:383
3150099662,24cyclictest974-21chrt20:50:373
3150099653,25cyclictest22443-21chrt20:19:433
3150099653,25cyclictest18815-21chrt00:00:263
3150099653,24cyclictest2472-21taskset00:25:303
3150099653,24cyclictest19536-21taskset20:10:163
3150099653,24cyclictest19536-21taskset20:10:163
3150099652,26cyclictest6525-21chrt00:31:343
3150099652,26cyclictest13359-21taskset21:20:263
3150099652,25cyclictest13912-21chrt23:01:273
3150099652,23cyclictest4541-21chrt19:25:123
3150099652,23cyclictest4541-21chrt19:25:123
3150099652,23cyclictest20112-21chrt22:20:353
3150099643,27cyclictest354-21chrt00:22:063
3150099642,26cyclictest2660-21taskset22:45:143
3150099642,26cyclictest2660-21taskset22:45:143
3150099642,25cyclictest31652-21chrt19:10:103
3150099642,24cyclictest9516-21chrt22:05:123
3150099642,24cyclictest16557-21chrt22:15:333
3150099633,27cyclictest30717-21chrt22:37:573
3150099633,26cyclictest7994-21chrt23:43:573
3150099633,24cyclictest717-21chrt21:51:263
3150099633,24cyclictest717-21chrt21:51:263
3150099633,23cyclictest3377-21chrt19:20:273
3150099632,28cyclictest2343-21taskset23:35:103
3150099632,26cyclictest17720-21chrt23:08:043
3150099632,24cyclictest9065-21chrt22:55:033
3150099632,24cyclictest9065-21chrt22:55:033
3150099632,24cyclictest21176-21taskset21:32:303
3150099632,23cyclictest17603-21chrt20:04:333
3150099624,42cyclictest13356-21taskset23:51:233
3150099624,42cyclictest13356-21taskset23:51:233
3150099622,25cyclictest10241-21chrt00:37:543
3150099622,24cyclictest26209-21taskset00:11:363
3150099622,24cyclictest26209-21taskset00:11:363
3150099622,24cyclictest25067-21chrt21:39:153
3150099622,23cyclictest19173-21chrt20:09:023
3150099622,23cyclictest15761-21chrt19:56:513
3150099622,22cyclictest11754-21taskset21:17:553
3150099613,27cyclictest1429-21taskset23:33:323
3150099613,27cyclictest1429-21taskset23:33:323
3150099613,23cyclictest5996-21chrt21:07:423
3150099613,23cyclictest25363-21chrt20:25:393
3150099612,23cyclictest10739-21chrt19:42:173
31500996110,42cyclictest0-21swapper/319:50:263
3150099602,24cyclictest24195-21chrt00:09:343
3150099602,24cyclictest11467-21chrt19:45:133
3150099593,23cyclictest31347-21chrt21:48:553
3150099593,23cyclictest27664-21chrt23:23:563
3150099593,22cyclictest990-21chrt19:15:083
3150099593,21cyclictest23812-21chrt23:17:193
3150099592,43cyclictest13633-21chrt22:10:343
3150099592,43cyclictest13633-21chrt22:10:343
3150099592,25cyclictest17729-21chrt21:26:513
3150099592,25cyclictest17729-21chrt21:26:513
3150099592,23cyclictest27179-21taskset20:33:163
3150099592,22cyclictest8008-21chrt22:02:043
3150099592,22cyclictest2628-21chrt20:56:043
3150099592,22cyclictest2628-21chrt20:56:043
3150099592,21cyclictest6990-21chrt19:30:353
3150099589,40cyclictest0-21swapper/323:55:283
3150099589,40cyclictest0-21swapper/323:55:283
3150099583,21cyclictest4074-21chrt21:55:363
3150099583,19cyclictest10591-21chrt22:56:133
3150099583,19cyclictest10591-21chrt22:56:133
3150099582,26cyclictest29706-21taskset23:26:023
3150099582,23cyclictest28847-21chrt20:39:033
3150099582,22cyclictest4502-21chrt21:04:103
3150099582,20cyclictest31930-21chrt20:46:523
3150099582,20cyclictest10769-21chrt23:47:543
3150099573,12cyclictest0-21swapper/320:20:333
3150099572,21cyclictest30295-21chrt20:41:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional