You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 00:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Jul 14, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4042215911,109sleep03757-21/usr/sbin/munin06:52:290
5272991153,90cyclictest27990-21apt-get09:22:241
394221149,69sleep23972-21fschecks_count06:52:272
52619911246,15cyclictest2374-21systemd-journal10:57:360
5272991083,77cyclictest2382-21apt-get10:27:311
5272991083,77cyclictest2382-21apt-get10:27:311
52619910252,10cyclictest2366-21ssh09:32:360
5261999754,34cyclictest16272-21grep07:27:240
528099929,64cyclictest20683-21apt-get10:57:232
527299922,61cyclictest12802-21apt-get07:17:221
527299922,61cyclictest12802-21apt-get07:17:221
509929211,53sleep15128-21idleruntime-cro06:57:031
5280999112,62cyclictest12802-21apt-get07:17:172
5280999112,62cyclictest12802-21apt-get07:17:172
426129124,40sleep30-21swapper/306:52:373
5280998965,15cyclictest10911-21ssh09:47:252
5280998965,15cyclictest10911-21ssh09:47:252
528799883,52cyclictest0-21swapper/309:22:243
5280998863,16cyclictest25418-21latency_hist10:12:052
527299884,34cyclictest19805-21fschecks_count10:02:251
527299883,37cyclictest15589-21idleruntime11:42:261
527299883,24cyclictest9126-21processes10:37:361
527299883,24cyclictest28796-21sort08:07:051
5261998865,14cyclictest12405-21diskmemload11:56:550
5280998758,20cyclictest12405-21diskmemload10:29:532
5280998758,20cyclictest12405-21diskmemload10:29:532
528099873,37cyclictest39-21ksoftirqd/209:53:472
5272998746,34cyclictest12405-21diskmemload11:22:341
527299873,38cyclictest352-21runrttasks10:42:421
5261998727,42cyclictest19726-21ssh09:07:390
5261998727,42cyclictest19726-21ssh09:07:390
528099867,57cyclictest39-21ksoftirqd/210:13:532
5272998663,16cyclictest12033-21ssh11:37:051
527299863,34cyclictest32202-21cpu08:17:201
527299863,34cyclictest32202-21cpu08:17:201
527299863,26cyclictest17893-21rm09:05:361
5272998632,46cyclictest0-21swapper/109:52:251
528799859,61cyclictest12810-21chrt07:17:173
528799859,61cyclictest12810-21chrt07:17:173
5280998551,11cyclictest181rcu_preempt12:00:082
527299853,37cyclictest27537-21ssh10:15:001
527299852,27cyclictest15946-21ls07:27:051
528099846,32cyclictest39-21ksoftirqd/211:27:022
528099844,45cyclictest39-21ksoftirqd/207:27:382
528099842,54cyclictest39-21ksoftirqd/211:12:192
5280998413,54cyclictest27990-21apt-get09:22:232
527299844,37cyclictest6749-21/usr/sbin/munin11:27:291
527299844,36cyclictest769-21/usr/sbin/munin12:12:131
527299844,36cyclictest769-21/usr/sbin/munin12:12:131
527299843,34cyclictest352-21runrttasks08:54:111
527299843,30cyclictest25455-21processes09:17:361
527299843,29cyclictest9702-21latency_hist12:27:051
527299843,29cyclictest9702-21latency_hist12:27:051
527299843,29cyclictest352-21runrttasks10:19:421
5272998420,53cyclictest0-21swapper/111:02:291
5261998460,14cyclictest18331-21cpuspeed11:47:190
527299834,38cyclictest9774-21idleruntime07:09:101
527299834,12cyclictest0-21swapper/110:22:141
5272998335,40cyclictest0-21swapper/110:37:061
527299833,41cyclictest6454-21fschecks_time08:37:241
527299833,35cyclictest1790-21uniq09:32:231
5261998368,9cyclictest5107-21ntp_offset08:32:320
5261998362,15cyclictest30397-21latency_hist08:12:050
5261998355,19cyclictest12050-21awk10:42:340
5261998351,12cyclictest181rcu_preempt11:37:180
5272998258,18cyclictest28977-21sh11:10:571
5272998258,18cyclictest28977-21sh11:10:571
527299824,23cyclictest29147-21fschecks_count08:07:221
5272998238,37cyclictest0-21swapper/107:32:211
527299823,36cyclictest29674-21/usr/sbin/munin11:12:281
527299823,36cyclictest11583-21sendmail_mailqu07:12:371
527299823,32cyclictest352-21runrttasks10:09:171
5272998231,42cyclictest0-21swapper/111:17:211
527299822,35cyclictest12705-21/usr/sbin/munin08:57:341
527299822,35cyclictest12705-21/usr/sbin/munin08:57:341
527299822,29cyclictest5544-21grep08:32:401
527299822,28cyclictest9325-21latency_hist08:47:051
5280998162,12cyclictest15937-21sshd11:42:362
528099815,51cyclictest39-21ksoftirqd/207:16:272
5280998138,9cyclictest0-21swapper/208:43:462
527299814,35cyclictest352-21runrttasks07:01:121
527299814,35cyclictest10062-21find08:47:351
527299814,33cyclictest21144-21fschecks_time07:42:241
527299813,38cyclictest7452-21taskset09:42:061
527299813,38cyclictest7452-21taskset09:42:061
527299813,38cyclictest12263-21ssh11:37:171
527299813,34cyclictest2374-21systemd-journal12:19:261
527299813,33cyclictest17318-21latency_hist10:52:051
527299813,32cyclictest25705-21ssh11:58:411
527299813,29cyclictest4319-21latency_hist08:32:021
5261998166,9cyclictest15-21ksoftirqd/010:37:270
5261998150,9cyclictest181rcu_preempt11:27:180
5280998037,8cyclictest181rcu_preempt09:59:192
5280998034,14cyclictest181rcu_preempt12:04:302
527299804,34cyclictest13259-21apt09:52:021
527299804,34cyclictest13259-21apt09:52:021
527299804,30cyclictest352-21runrttasks07:50:581
527299803,38cyclictest30242-21/usr/sbin/munin12:07:191
527299803,37cyclictest21077-21fschecks_count10:57:471
527299803,36cyclictest16539-21/usr/sbin/munin09:57:281
527299803,33cyclictest352-21runrttasks08:22:241
527299803,29cyclictest21744-21/usr/sbin/munin09:12:151
5272998026,46cyclictest0-21swapper/109:07:171
5272998026,46cyclictest0-21swapper/109:07:171
527299802,27cyclictest19285-21grep11:47:391
5261998052,17cyclictest1812-21apt-get10:27:020
528099793,42cyclictest39-21ksoftirqd/211:47:132
5280997920,32cyclictest13177-21sshd10:44:262
527299794,26cyclictest27715-21latency12:02:291
527299792,27cyclictest19498-21fschecks_time07:37:261
527299792,22cyclictest7824-21/usr/sbin/munin07:02:301
5261997950,19cyclictest7301-21latency_hist09:42:050
5261997950,19cyclictest7301-21latency_hist09:42:050
5280997862,10cyclictest8121-21sh10:37:022
5280997857,15cyclictest12467-21fschecks_count11:37:242
5280997855,13cyclictest181rcu_preempt08:54:402
5280997848,15cyclictest181rcu_preempt09:38:322
5280997848,15cyclictest181rcu_preempt09:38:322
5280997843,8cyclictest181rcu_preempt08:12:322
5280997835,10cyclictest181rcu_preempt08:26:422
527299783,35cyclictest25034-21unixbench_multi07:52:391
527299782,36cyclictest21513-21fschecks_time11:52:241
527299782,25cyclictest16571-21ntp_offset07:27:311
527299782,22cyclictest26287-21/usr/sbin/munin07:57:341
5261997842,30cyclictest16815-21ssh09:57:240
5280997744,12cyclictest181rcu_preempt07:57:002
528099774,34cyclictest39-21ksoftirqd/207:02:042
5280997734,12cyclictest181rcu_preempt10:49:352
5280997718,21cyclictest0-21swapper/207:42:292
527299773,28cyclictest352-21runrttasks08:12:071
527299773,26cyclictest18076-21ntpq10:52:321
5261997759,12cyclictest12405-21diskmemload11:32:270
5261997757,13cyclictest9335-21latency_hist08:47:050
5261997756,13cyclictest12405-21diskmemload09:17:400
5261997740,8cyclictest0-21swapper/007:12:060
5261997718,48cyclictest1114-21fschecks_time12:12:250
5261997718,48cyclictest1114-21fschecks_time12:12:250
5280997656,13cyclictest1-21systemd09:07:022
5280997653,16cyclictest12405-21diskmemload10:02:142
528099762,36cyclictest39-21ksoftirqd/207:22:142
528099762,29cyclictest39-21ksoftirqd/211:27:382
527299763,34cyclictest7819-21fschecks_count09:42:251
527299763,34cyclictest30977-21/usr/sbin/munin09:27:121
5261997654,14cyclictest2374-21systemd-journal10:56:270
528099758,37cyclictest39-21ksoftirqd/211:52:062
5280997548,10cyclictest181rcu_preempt07:33:392
5280997543,24cyclictest31263-21fschecks_count09:27:242
5280997541,10cyclictest181rcu_preempt09:09:322
5280997541,10cyclictest181rcu_preempt09:09:322
5280997540,15cyclictest181rcu_preempt10:55:472
5280997539,16cyclictest181rcu_preempt08:31:522
528099752,29cyclictest39-21ksoftirqd/208:37:242
5261997562,8cyclictest15-21ksoftirqd/009:02:040
5261997562,8cyclictest15-21ksoftirqd/009:02:040
5261997545,22cyclictest0-21swapper/010:47:240
5261997541,12cyclictest181rcu_preempt08:37:140
5261997536,10cyclictest181rcu_preempt07:37:180
528099746,27cyclictest39-21ksoftirqd/207:57:382
5280997440,16cyclictest181rcu_preempt10:21:042
5280997438,16cyclictest181rcu_preempt09:33:242
5280997437,8cyclictest181rcu_preempt09:20:072
528099743,27cyclictest39-21ksoftirqd/207:02:252
528099742,34cyclictest39-21ksoftirqd/212:16:262
528099742,34cyclictest39-21ksoftirqd/212:16:262
5280997410,22cyclictest381ktimers/209:12:342
5261997460,8cyclictest15-21ksoftirqd/012:22:360
5261997460,8cyclictest15-21ksoftirqd/012:22:360
5261997449,17cyclictest0-21swapper/012:02:320
5261997448,17cyclictest20626-21unixbench_singl10:02:420
5261997439,10cyclictest58-21kcompactd011:02:460
526199743,62cyclictest31586-21awk09:27:310
526199743,42cyclictest5217-21/usr/sbin/munin10:32:420
526199743,41cyclictest352-21runrttasks09:52:460
5287997312,37cyclictest0-21swapper/310:57:233
5280997347,11cyclictest181rcu_preempt07:49:522
5280997346,12cyclictest181rcu_preempt08:17:202
5280997346,12cyclictest181rcu_preempt08:17:202
5280997344,8cyclictest181rcu_preempt12:10:442
528099732,30cyclictest9389-21ssh09:44:422
5280997321,23cyclictest39-21ksoftirqd/209:01:162
5280997321,23cyclictest39-21ksoftirqd/209:01:162
5261997348,19cyclictest29923-21latency_hist12:07:060
528799724,37cyclictest0-21swapper/310:27:313
528799724,37cyclictest0-21swapper/310:27:313
528099729,28cyclictest39-21ksoftirqd/210:37:192
5280997257,11cyclictest5251-21cyclictest11:32:392
5280997250,7cyclictest181rcu_preempt08:07:052
5280997244,10cyclictest181rcu_preempt10:24:482
5280997235,12cyclictest181rcu_preempt11:07:102
5280997235,12cyclictest181rcu_preempt11:07:102
5280997234,8cyclictest181rcu_preempt07:37:282
5261997258,8cyclictest27769-21rm10:15:380
5261997258,10cyclictest5251-21cyclictest08:12:260
526199722,55cyclictest25371-21latency_hist10:12:040
5280997137,11cyclictest181rcu_preempt08:07:282
5261997152,12cyclictest12405-21diskmemload09:23:240
5261997142,10cyclictest0-21swapper/011:07:250
5261997142,10cyclictest0-21swapper/011:07:250
528099706,46cyclictest9353-21apt-get07:07:182
5280997039,9cyclictest181rcu_preempt12:17:442
528099702,28cyclictest39-21ksoftirqd/212:25:182
528099702,28cyclictest39-21ksoftirqd/212:25:182
5261997051,13cyclictest16606-21timerandwakeup09:02:390
5261997049,15cyclictest28704-21idleruntime-cro08:07:020
5280996935,16cyclictest181rcu_preempt08:36:022
5261996954,9cyclictest24796-21ssh11:57:310
5261996951,12cyclictest4349-21ssh12:17:310
5261996951,11cyclictest31967-21ssh11:15:580
5261996946,13cyclictest0-21swapper/009:47:250
5261996946,13cyclictest0-21swapper/009:47:250
5261996934,9cyclictest181rcu_preempt10:31:170
5261996934,9cyclictest181rcu_preempt10:31:170
5280996837,9cyclictest181rcu_preempt11:03:432
528099683,28cyclictest39-21ksoftirqd/211:17:332
526199684,36cyclictest9023-21seq09:43:400
526199683,22cyclictest15563-21ssh11:42:250
318182680,7sleep30-21swapper/312:08:553
5280996752,8cyclictest181rcu_preempt08:47:582
5261996753,9cyclictest12303-21apt-get07:17:170
5261996753,9cyclictest12303-21apt-get07:17:170
526199674,43cyclictest28765-21ssh10:17:200
5261996744,13cyclictest0-21swapper/009:12:260
5261996726,13cyclictest181rcu_preempt07:02:090
5261996649,11cyclictest6803-21systemd-run07:02:030
526199663,28cyclictest1256-21tail11:17:390
5261996621,31cyclictest352-21runrttasks11:26:150
526199652,36cyclictest15243-21timerwakeupswit07:22:390
5261996441,8cyclictest0-21swapper/007:57:220
5261996418,37cyclictest21442-21grep07:42:340
526199635,46cyclictest10944-21/usr/sbin/munin07:12:270
526199633,39cyclictest11158-21cstates08:52:200
5287996226,25cyclictest3421-21chrt11:22:233
5287996223,28cyclictest10098-21taskset07:09:193
5261996249,7cyclictest0-21swapper/007:52:300
5261996246,8cyclictest0-21swapper/008:22:040
5261996246,8cyclictest0-21swapper/008:22:040
526199624,35cyclictest58-21kcompactd008:47:230
526199623,22cyclictest1538-21fschecks_count08:22:220
526199613,43cyclictest3194-21fschecks_count08:27:240
526199613,40cyclictest18490-21timerwakeupswit07:32:400
5261996131,8cyclictest24052-21latency_hist07:52:050
528799603,44cyclictest20445-21taskset10:02:403
5287996020,27cyclictest31011-21taskset09:27:143
5287995918,29cyclictest20904-21taskset09:10:353
5287995918,29cyclictest20904-21taskset09:10:353
5287995820,27cyclictest27116-21taskset11:07:283
5287995820,27cyclictest27116-21taskset11:07:283
5287995819,27cyclictest26113-21taskset11:06:223
5287995819,26cyclictest19802-21taskset10:56:093
5287995819,26cyclictest11882-21taskset09:48:133
5287995819,26cyclictest11882-21taskset09:48:133
5287995818,26cyclictest18329-21taskset09:06:503
528799573,39cyclictest29824-21chrt12:06:493
5287995724,22cyclictest19112-21chrt10:01:423
528799572,38cyclictest21656-21taskset07:42:373
5287995722,24cyclictest22989-21taskset10:07:263
5287995719,27cyclictest24932-21taskset09:17:213
5287995719,26cyclictest9961-21taskset10:38:513
5287995719,25cyclictest9087-21taskset09:43:513
5287995718,25cyclictest9210-21taskset12:26:223
5287995718,25cyclictest9210-21taskset12:26:223
528799567,37cyclictest2414-21taskset08:23:443
528799564,35cyclictest18453-21chrt11:47:243
5287995624,20cyclictest26556-21chrt12:01:053
5287995623,22cyclictest6614-21chrt12:22:043
5287995623,21cyclictest31613-21chrt10:22:173
5287995622,22cyclictest23483-21chrt07:47:393
5287995621,24cyclictest2843-21chrt11:21:543
5287995620,23cyclictest24216-21taskset09:16:273
5287995619,24cyclictest13009-21taskset10:43:573
5287995617,28cyclictest926-21chrt12:12:203
5287995617,28cyclictest926-21chrt12:12:203
5287995523,20cyclictest16350-21taskset11:42:423
5287995523,20cyclictest14765-21chrt07:22:283
5287995522,22cyclictest18002-21chrt07:32:283
5287995520,23cyclictest13609-21chrt09:52:183
5287995424,19cyclictest8151-21chrt08:42:273
5287995424,19cyclictest30045-21chrt11:12:263
5287995423,20cyclictest27780-21chrt10:15:403
5287995422,21cyclictest7608-21chrt11:28:233
5287995422,21cyclictest5334-21chrt09:37:353
5287995422,21cyclictest5334-21chrt09:37:353
5287995421,21cyclictest9593-21chrt08:47:213
5287995421,21cyclictest7476-21chrt08:41:263
5287995421,21cyclictest5437-21chrt06:57:123
5287995421,21cyclictest27339-21chrt08:02:173
5287995421,21cyclictest10624-21chrt11:33:253
5287995420,21cyclictest7015-21chrt07:02:063
5287995419,24cyclictest16166-21taskset10:49:203
5287995419,23cyclictest3630-21chrt09:35:243
5287995419,22cyclictest12162-21chrt08:54:103
528799532,33cyclictest12887-21chrt11:37:353
5287995322,19cyclictest25116-21chrt07:52:463
5287995321,22cyclictest6964-21chrt10:34:003
5287995321,20cyclictest31834-21chrt08:16:063
5287995321,20cyclictest17242-21chrt07:30:223
5287995320,22cyclictest21244-21chrt11:52:143
528799524,36cyclictest26095-21taskset07:57:273
5287995220,19cyclictest32264-21chrt08:17:203
5287995220,19cyclictest32264-21chrt08:17:203
5287995219,21cyclictest5476-21taskset08:32:393
5287995120,21cyclictest10939-21chrt07:12:113
5287995120,20cyclictest14291-21chrt08:58:563
5287995120,20cyclictest14291-21chrt08:58:563
5287995120,19cyclictest20384-21chrt07:39:253
5287995119,20cyclictest29723-21chrt10:17:433
5287995019,20cyclictest4023-21chrt08:28:223
5287995018,20cyclictest30097-21chrt08:09:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional