You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 01:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Apr 15, 2026 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304729911195,10cyclictest31780-21xargs10:40:231
30469991097,92cyclictest27938-21ssh12:23:250
304699910474,23cyclictest18461-21latency_hist08:03:110
304729910351,32cyclictest0-21swapper/110:48:081
304729910351,32cyclictest0-21swapper/110:48:081
304699910365,14cyclictest6940-21diskmemload09:58:300
3030929915,57sleep130343-21idleruntime-cro07:03:081
3047299943,54cyclictest18104-21apt-get12:08:211
3047299943,54cyclictest18104-21apt-get12:08:211
30469999374,13cyclictest2952-21systemd-journal09:05:340
30469999374,13cyclictest2952-21systemd-journal09:05:340
30475999250,35cyclictest0-21swapper/209:43:252
30475999250,35cyclictest0-21swapper/209:43:252
3047299928,64cyclictest21278-21apt-get12:13:221
30469999257,14cyclictest181rcu_preempt09:23:210
30469999155,27cyclictest10156-21latency_hist07:38:110
3047599903,39cyclictest22226-21grep09:28:442
30475999032,52cyclictest26873-21modprobe11:28:072
3047599902,36cyclictest13783-21ssh11:59:042
3047599902,36cyclictest13783-21ssh11:59:042
30475998946,36cyclictest0-21swapper/208:28:332
3047599892,37cyclictest32124-21grep08:43:272
3047599884,36cyclictest18085-21latency_hist11:13:102
30475998839,35cyclictest5016-21cat08:59:132
3046999882,76cyclictest20992-21modprobe10:23:070
3046999882,76cyclictest20992-21modprobe10:23:070
3048299877,44cyclictest0-21swapper/312:08:413
3048299877,44cyclictest0-21swapper/312:08:413
3047599875,35cyclictest1028-21grep09:48:332
30475998746,36cyclictest30454-21cyclictest12:13:102
30475998746,36cyclictest30454-21cyclictest12:13:102
3047599873,35cyclictest18400-21/usr/sbin/munin11:13:432
3047599873,35cyclictest18400-21/usr/sbin/munin11:13:432
3047599863,32cyclictest6986-21fschecks_time10:53:312
30475998623,55cyclictest14256-21kworker/u18:1+flush-179:012:13:272
30475998613,61cyclictest3763-21apt-get11:43:222
3046999863,31cyclictest26966-21latency_hist09:38:100
3047599854,35cyclictest352-21runrttasks07:03:242
3047599853,38cyclictest32746-21latency_hist11:38:102
3047599853,38cyclictest32746-21latency_hist11:38:102
3047599853,35cyclictest16569-21apt-config07:58:102
3047599852,37cyclictest4024-21apt-get07:18:362
3047599852,37cyclictest4024-21apt-get07:18:362
30472998563,15cyclictest5937-21diskmemload09:43:351
30472998563,15cyclictest5937-21diskmemload09:43:351
3047299853,21cyclictest3812-21ssh11:43:261
30469998563,16cyclictest25863-21vmstat12:18:480
30469998563,16cyclictest25863-21vmstat12:18:480
3047599845,36cyclictest31088-21timerandwakeup08:38:422
3047599845,36cyclictest31088-21timerandwakeup08:38:422
3047599843,40cyclictest352-21runrttasks09:54:062
3047599843,37cyclictest27400-21/usr/sbin/munin11:28:282
3047599843,35cyclictest27863-21/usr/sbin/munin12:23:162
3047599843,34cyclictest15509-21ssh12:03:202
3047599843,34cyclictest15509-21ssh12:03:202
3047599842,39cyclictest2451-21modprobe08:53:072
30469998445,29cyclictest0-21swapper/011:13:290
30469998445,29cyclictest0-21swapper/011:13:290
30475998344,33cyclictest6493-21fschecks_count09:03:292
30475998344,33cyclictest6493-21fschecks_count09:03:292
3047599834,37cyclictest509-21idleruntime07:08:332
3047599834,35cyclictest20197-21cpuspeed08:08:252
3047599834,35cyclictest16426-21unixbench_multi10:13:422
3047599833,39cyclictest19370-21timerwakeupswit08:03:442
3047599833,38cyclictest30242-21latency_hist10:38:122
3047599833,36cyclictest7330-21/usr/sbin/munin09:58:402
3047599833,36cyclictest5176-21cpu07:23:222
3047599833,36cyclictest5176-21cpu07:23:222
3047599833,35cyclictest30329-21/usr/sbin/munin10:40:162
3047599833,31cyclictest24090-21latency_hist11:23:092
30469998365,12cyclictest1-21systemd07:58:090
30469998360,16cyclictest21265-21latency_hist12:13:120
30469998360,16cyclictest21265-21latency_hist12:13:120
30469998359,18cyclictest5937-21diskmemload11:08:100
30469998359,18cyclictest5937-21diskmemload11:08:100
3047599824,38cyclictest21277-21ssh10:23:162
3047599824,35cyclictest8509-21cmp07:33:122
3047599824,35cyclictest8509-21cmp07:33:122
30475998236,37cyclictest0-21swapper/211:48:342
3047599823,38cyclictest26476-21/usr/sbin/munin08:24:002
3047599823,37cyclictest786-21/usr/sbin/munin10:43:272
3047599823,37cyclictest786-21/usr/sbin/munin10:43:272
3047599823,34cyclictest4019-21fschecks_time10:48:292
3047599823,32cyclictest352-21runrttasks12:30:002
3047599823,32cyclictest14031-21chrt11:04:522
3047599823,32cyclictest14031-21chrt11:04:522
3047599823,31cyclictest15179-21/usr/sbin/munin09:18:312
3047599823,25cyclictest9758-21ssh10:03:292
3047599822,40cyclictest17528-21runrttasks07:59:032
3047599822,35cyclictest2286-21sort07:13:322
3047599822,35cyclictest2286-21sort07:13:322
3047599822,24cyclictest27101-21apt-get09:38:182
30472998252,21cyclictest396-21unixbench_multi08:43:421
30472998230,26cyclictest32233-21munin-node07:17:321
30472998230,26cyclictest32233-21munin-node07:17:321
30469998252,22cyclictest544-21cron08:08:070
3047599819,44cyclictest0-21swapper/209:36:272
3047599814,33cyclictest3421-21latency_hist11:43:102
3047599813,35cyclictest14395-21unixbench_singl07:48:462
3047599813,34cyclictest23563-21latency_hist08:18:122
3047599813,33cyclictest3445-21switchtime08:53:412
3047599813,33cyclictest10328-21cstates07:38:262
3047599813,25cyclictest12214-21/usr/sbin/munin09:13:342
3047599812,37cyclictest29056-21/usr/sbin/munin08:33:322
3047599812,37cyclictest29056-21/usr/sbin/munin08:33:322
3047599812,34cyclictest25108-21apt-config08:23:102
3047599812,31cyclictest9916-21fschecks_time10:58:302
30469998136,28cyclictest21716-21ssh09:28:350
3046999812,44cyclictest4409-21ssh11:43:500
3047599804,32cyclictest24943-21find10:28:422
3047599803,37cyclictest10248-21ssh11:53:372
3047599803,35cyclictest13552-21vmstat10:08:452
3047599803,35cyclictest11339-21taskset09:11:352
3047599803,34cyclictest20771-21taskset09:27:592
3047599803,33cyclictest24883-21cstates12:18:252
3047599803,33cyclictest24883-21cstates12:18:252
3047599802,25cyclictest12354-21/usr/sbin/munin07:43:362
30472998051,12cyclictest181rcu_preempt08:28:351
30472998020,25cyclictest29502-21seq09:42:271
30472998020,25cyclictest27220-21ssh12:22:471
30472998020,25cyclictest27220-21ssh12:22:471
30469998060,13cyclictest30454-21cyclictest11:09:090
30469998046,28cyclictest0-21swapper/011:51:160
30469998041,13cyclictest181rcu_preempt07:18:290
30469998041,13cyclictest181rcu_preempt07:18:290
30469998037,19cyclictest352-21runrttasks08:19:400
3046999802,24cyclictest257482ssh10:29:570
3047599794,33cyclictest352-21runrttasks07:33:392
3047599794,33cyclictest18450-21ssh10:18:232
3047599794,33cyclictest18450-21ssh10:18:232
30472997950,20cyclictest13167-21idleruntime-cro07:48:071
30469997961,10cyclictest25572-21kworker/u17:2+rpciod11:28:390
30469997959,12cyclictest30454-21cyclictest10:39:460
30469997958,14cyclictest24588-21fschecks_time11:23:290
3046999795,28cyclictest15-21ksoftirqd/011:37:310
3046999795,28cyclictest15-21ksoftirqd/011:37:310
30469997950,10cyclictest181rcu_preempt07:28:210
30469997950,10cyclictest181rcu_preempt07:28:210
3047599784,40cyclictest30454-21cyclictest07:03:122
30472997856,16cyclictest15114-21idleruntime-cro10:13:081
30472997812,42cyclictest18982-21cat09:23:411
30469997858,14cyclictest5937-21diskmemload09:48:420
30469997852,11cyclictest181rcu_preempt09:53:190
3046999783,44cyclictest18279-21latency_hist12:08:090
3046999783,44cyclictest18279-21latency_hist12:08:090
30469997829,10cyclictest29981-21date09:43:110
30472997722,22cyclictest8135-21seq09:05:561
30472997722,22cyclictest8135-21seq09:05:561
3046999777,40cyclictest9743-21fschecks_count10:03:290
30469997752,19cyclictest11904-21munin-plugin-st09:13:090
3046999773,40cyclictest30905-21switchtime09:43:420
3046999773,40cyclictest30905-21switchtime09:43:420
30472997656,14cyclictest25084-21apt-key08:23:101
30472997652,17cyclictest15764-21sh11:08:291
30472997617,28cyclictest22312-21seq08:13:371
30472997614,22cyclictest0-21swapper/111:50:071
3046999767,59cyclictest21834-21apt-get12:13:220
30469997661,10cyclictest16287-21find10:13:400
30469997658,13cyclictest30454-21cyclictest12:32:250
3046999762,50cyclictest15336-21ssh10:13:110
30472997558,11cyclictest297-21systemd-logind07:28:081
30472997558,11cyclictest297-21systemd-logind07:28:081
30472997556,13cyclictest1208-21sshd11:38:351
30472997553,14cyclictest0-21swapper/109:13:431
30472997531,29cyclictest20136-21apt-get08:08:211
30472997521,20cyclictest14527-21ssh12:01:141
30472997521,20cyclictest14527-21ssh12:01:141
30472997519,25cyclictest14133-21seq11:05:101
30472997519,25cyclictest14133-21seq11:05:101
30469997557,12cyclictest1593-21ssh10:43:410
30469997557,12cyclictest1593-21ssh10:43:410
3046999753,44cyclictest15-21ksoftirqd/009:18:250
3046999752,34cyclictest1583-21modprobe07:13:080
3048299743,32cyclictest31949-21chrt11:35:493
3048299743,32cyclictest31949-21chrt11:35:493
30472997458,10cyclictest19428-21unixbench_multi08:03:441
30472997420,23cyclictest456-21seq12:31:381
3046999745,49cyclictest12214-21/usr/sbin/munin09:13:450
3046999744,40cyclictest352-21runrttasks11:54:390
3046999743,35cyclictest21619-21fschecks_time10:23:300
3047299733,22cyclictest29335-21grep08:33:411
3047299733,22cyclictest29335-21grep08:33:411
30469997352,14cyclictest30454-21cyclictest10:37:390
3046999733,56cyclictest13248-21uname11:58:390
3046999733,56cyclictest13248-21uname11:58:390
2899727317,35sleep30-21swapper/306:58:253
3048299723,33cyclictest6931-21chrt09:03:403
3048299723,33cyclictest6931-21chrt09:03:403
30472997249,15cyclictest12341-21latency_hist10:08:111
30472997231,36cyclictest0-21swapper/109:58:171
30469997247,9cyclictest181rcu_preempt10:48:180
30469997246,10cyclictest181rcu_preempt08:44:040
30469997243,12cyclictest181rcu_preempt07:23:420
30469997243,12cyclictest181rcu_preempt07:23:420
30469997240,8cyclictest181rcu_preempt08:53:110
2898327211,28sleep00-21swapper/006:58:180
30472997152,12cyclictest0-21swapper/107:03:431
30472997149,15cyclictest4189-21apt-key08:58:101
30472997147,14cyclictest0-21swapper/112:24:311
30472997118,23cyclictest16532-21apt-key07:58:101
30472997116,22cyclictest0-21swapper/111:15:361
30472997116,22cyclictest0-21swapper/111:15:361
3046999713,32cyclictest7691-21unixbench_multi10:53:450
30469997132,17cyclictest181rcu_preempt07:43:420
30469997128,34cyclictest3479-21cat11:43:100
3048299703,32cyclictest0-21swapper/312:13:563
30472997048,15cyclictest5937-21diskmemload11:28:331
30472997046,16cyclictest21950-21ntp_offset11:18:361
3047299703,41cyclictest18144-21apt-config12:08:071
3047299703,41cyclictest18144-21apt-config12:08:071
3047299703,33cyclictest2449-21systemctl08:53:071
3047299702,34cyclictest5937-21diskmemload09:53:451
3046999704,34cyclictest21507-21awk08:13:080
30469997019,12cyclictest181rcu_preempt08:53:430
30472996949,15cyclictest30454-21cyclictest09:48:291
30472996943,18cyclictest27882-21sh10:33:411
30472996941,21cyclictest544-21cron07:53:081
30472996927,36cyclictest0-21swapper/107:38:241
30469996950,13cyclictest2952-21systemd-journal08:28:080
3046999693,39cyclictest11017-21timerandwakeup07:38:440
30469996930,14cyclictest181rcu_preempt11:18:230
3046999692,44cyclictest28711-21kworker/u18:0+ext4-rsv-conversion08:58:340
30469996920,15cyclictest181rcu_preempt11:02:140
30472996846,16cyclictest1-21systemd10:59:351
30472996834,23cyclictest8610-21apt-get07:33:361
3047299682,42cyclictest1795-21latency_hist07:13:111
30472996745,12cyclictest0-21swapper/107:59:061
3047299674,31cyclictest7522-21switchtime10:53:441
30472996743,13cyclictest0-21swapper/110:53:071
3047299673,9cyclictest0-21swapper/109:33:111
3047299673,43cyclictest3698-21fschecks_time07:18:281
3047299673,43cyclictest3698-21fschecks_time07:18:281
3047299672,24cyclictest4632-21apt-get08:58:561
3046999674,31cyclictest352-21runrttasks08:13:270
30472996651,11cyclictest17016-21kworker/u17:0-rpciod11:57:561
30472996647,12cyclictest0-21swapper/110:20:341
30472996647,12cyclictest0-21swapper/110:20:341
3046999663,50cyclictest31666-21apt08:43:070
3046999663,50cyclictest31666-21apt08:43:070
3046999663,42cyclictest14777-21latency_hist07:53:110
30469996632,12cyclictest181rcu_preempt08:33:110
30472996547,13cyclictest8085-21idleruntime-cro07:33:071
30472996547,13cyclictest8085-21idleruntime-cro07:33:071
3047299653,42cyclictest18084-21ls10:18:071
30469996550,8cyclictest0-21swapper/007:03:450
3046999653,41cyclictest2618-21switchtime07:13:410
3046999653,41cyclictest2618-21switchtime07:13:410
3048299643,38cyclictest917-21kworker/3:1+events08:58:223
30472996420,33cyclictest9381-21expr09:08:251
30469996428,16cyclictest181rcu_preempt08:33:430
30469996428,16cyclictest181rcu_preempt08:33:430
30472996342,14cyclictest32758-21idleruntime-cro11:38:061
30472996342,14cyclictest32758-21idleruntime-cro11:38:061
30472996333,22cyclictest15199-21apt-get09:18:221
30472996325,25cyclictest21814-21load10:23:341
3048299624,43cyclictest32135-21taskset10:40:453
3048299624,39cyclictest28173-21chrt10:33:443
30472996239,14cyclictest0-21swapper/110:33:091
30472996230,24cyclictest25658-21cat08:23:431
3047299622,46cyclictest24952-21uname11:23:391
313652610,7sleep30-21swapper/307:03:423
3048299613,34cyclictest0-21swapper/311:43:263
3047299613,32cyclictest26792-21munin-run09:38:071
3048299603,40cyclictest25120-21taskset08:23:103
3048299602,36cyclictest12381-21chrt11:58:093
30472996029,21cyclictest30320-21apt-get08:38:231
30472996029,21cyclictest30320-21apt-get08:38:231
3048299592,38cyclictest3681-21taskset07:18:283
3048299592,38cyclictest3681-21taskset07:18:283
3048299583,39cyclictest434-21chrt07:08:323
35102570,3sleep3451rcuc/311:43:103
3048299573,38cyclictest6817-21taskset10:53:263
3048299572,39cyclictest29004-21taskset09:41:003
3048299563,40cyclictest7503-21chrt11:48:413
3048299562,41cyclictest16506-21taskset10:13:423
3048299553,38cyclictest5118-21chrt09:54:583
3048299553,38cyclictest11741-21chrt07:43:113
3048299553,37cyclictest24730-21chrt09:33:353
3048299553,37cyclictest24564-21chrt11:23:293
3048299553,37cyclictest23790-21chrt10:27:483
3048299553,34cyclictest1497-21chrt08:48:333
3048299552,38cyclictest15080-21chrt09:18:113
3048299552,38cyclictest10339-21chrt10:58:413
3048299544,36cyclictest20290-21chrt11:15:593
3048299544,36cyclictest20290-21chrt11:15:593
3048299544,35cyclictest13074-21taskset11:03:353
3048299544,35cyclictest13074-21taskset11:03:353
3048299543,37cyclictest27524-21chrt08:28:333
3048299543,36cyclictest5288-21chrt07:23:263
3048299543,36cyclictest5288-21chrt07:23:263
3048299543,33cyclictest8318-21chrt07:33:103
3048299543,33cyclictest8318-21chrt07:33:103
3048299543,33cyclictest1427-21chrt10:43:373
3048299543,33cyclictest1427-21chrt10:43:373
3048299542,39cyclictest21407-21taskset09:28:283
3048299542,38cyclictest32051-21taskset12:29:063
3048299542,34cyclictest25527-21chrt08:23:393
3048299533,37cyclictest19788-21chrt09:25:123
3048299533,36cyclictest2341-21chrt09:50:373
3048299533,36cyclictest21871-21taskset11:18:353
3048299533,35cyclictest24956-21chrt12:18:283
3048299533,35cyclictest24956-21chrt12:18:283
3048299533,35cyclictest18527-21chrt10:18:243
3048299533,35cyclictest18527-21chrt10:18:243
3048299533,34cyclictest30128-21chrt12:27:033
3048299533,32cyclictest20350-21taskset08:08:303
3048299523,35cyclictest0-21swapper/307:58:233
3048299514,34cyclictest6752-21taskset09:58:273
3048299513,35cyclictest32296-21chrt08:43:313
3048299513,35cyclictest15850-21chrt11:08:303
3048299513,34cyclictest2766-21chrt08:53:233
3048299513,34cyclictest10778-21chrt09:10:013
3048299513,32cyclictest25406-21taskset10:28:583
3048299503,34cyclictest14964-21chrt12:02:313
3048299503,34cyclictest14964-21chrt12:02:313
3048299502,35cyclictest12704-21taskset10:08:283
3048299502,33cyclictest22300-21chrt08:13:363
3048299493,34cyclictest12098-21chrt10:08:033
3048299492,34cyclictest27769-21chrt11:28:353
3048299484,33cyclictest8818-21chrt07:33:433
3048299482,29cyclictest19429-21chrt08:03:443
3048299473,32cyclictest28999-21taskset08:33:313
3048299473,32cyclictest28999-21taskset08:33:313
3048299473,28cyclictest16476-21chrt07:58:093
3048299463,30cyclictest3819-21chrt10:48:233
3048299462,26cyclictest14994-21chrt07:53:113
3048299453,31cyclictest18454-21chrt12:08:103
3048299453,31cyclictest18454-21chrt12:08:103
3048299443,28cyclictest31198-21chrt09:43:483
3048299443,28cyclictest31198-21chrt09:43:483
3048299443,28cyclictest1875-21taskset07:13:223
3048299443,28cyclictest1875-21taskset07:13:223
3048299442,29cyclictest31063-21taskset08:38:413
3048299442,29cyclictest31063-21taskset08:38:413
3048299432,28cyclictest17945-21taskset09:23:103
3048299423,29cyclictest12431-21taskset07:43:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional