You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 17:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Mar 14, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2645323340,19sleep20-21swapper/207:00:362
27779991142,34cyclictest18142-21latency_hist10:25:190
27779991142,34cyclictest18142-21latency_hist10:25:190
27779991135,53cyclictest28623-21latency_hist12:30:180
27779991135,53cyclictest28623-21latency_hist12:30:180
27779991093,33cyclictest1491-21unixbench_singl09:55:530
277799910932,10cyclictest0-21swapper/011:30:330
27779991083,33cyclictest7591-21grep11:00:380
27779991074,40cyclictest28831-21fschecks_time11:35:380
27779991074,40cyclictest28831-21fschecks_time11:35:380
277799910718,19cyclictest0-21swapper/008:55:360
27779991063,34cyclictest31076-21cat10:45:390
27779991063,27cyclictest4020-21/usr/sbin/munin10:00:440
277799910552,46cyclictest30581-21ssh10:45:190
277799910552,46cyclictest30581-21ssh10:45:190
277799910540,58cyclictest0-21swapper/008:40:160
27779991053,36cyclictest25720-21grep10:35:500
3150021040,7sleep20-21swapper/210:45:502
277859910413,47cyclictest2267-21diskmemload10:25:371
27779991043,32cyclictest27639-21latency_hist07:05:200
27779991043,32cyclictest12171-21idleruntime-cro10:15:160
27779991043,32cyclictest12171-21idleruntime-cro10:15:160
27779991043,29cyclictest8544-21ntpq11:55:450
277799910431,23cyclictest26969-21cstates09:45:310
277799910431,23cyclictest26969-21cstates09:45:310
27779991033,36cyclictest31904-21fschecks_time07:15:380
27779991024,31cyclictest5622-21/usr/sbin/munin07:35:340
277799910231,8cyclictest0-21swapper/012:07:480
27779991013,36cyclictest8727-21cpuspeed09:15:330
27779991013,36cyclictest8727-21cpuspeed09:15:330
27779991013,32cyclictest15349-21latency_hist10:20:190
277799910130,63cyclictest0-21swapper/007:25:190
277799910130,63cyclictest0-21swapper/007:25:190
27779991012,40cyclictest898-21/usr/sbin/munin09:00:420
27779991004,36cyclictest352-21runrttasks11:14:330
277799910039,53cyclictest0-21swapper/012:10:360
277799910039,53cyclictest0-21swapper/012:10:360
27779991003,35cyclictest27679-21grep08:40:500
27779991003,33cyclictest17471-21rm11:16:010
27779991003,32cyclictest19892-21grep11:21:330
27779991003,28cyclictest6658-21sshd11:52:340
277799910032,60cyclictest30699-21wc08:50:470
277799910030,8cyclictest0-21swapper/012:00:450
64102990,7sleep10-21swapper/110:05:241
64102990,7sleep10-21swapper/110:05:241
2777999994,22cyclictest4641-21ssh10:55:400
2777999993,37cyclictest22941-21fschecks_time11:25:540
2777999993,37cyclictest22941-21fschecks_time11:25:540
2777999993,29cyclictest5491-21latency_hist09:10:190
2777999993,29cyclictest20597-21sendmail_mailst12:15:500
27779999929,9cyclictest0-21swapper/009:31:130
2777999992,38cyclictest25767-21kworker/u17:2+xprtiod10:30:290
2777999992,34cyclictest7169-21ssh10:05:470
2777999992,34cyclictest7169-21ssh10:05:470
2777999984,39cyclictest11272-21grep07:50:490
2777999984,39cyclictest11272-21grep07:50:490
2777999984,34cyclictest20177-21latency_hist08:20:180
27779999835,55cyclictest0-21swapper/009:10:430
2777999983,33cyclictest24580-21/usr/sbin/munin09:40:480
2777999983,31cyclictest10263-21/usr/sbin/munin11:05:420
27779999832,8cyclictest0-21swapper/009:20:370
2777999983,24cyclictest12334-21chrt07:55:290
2777999982,40cyclictest21267-21chrt10:29:590
27779999823,13cyclictest0-21swapper/012:31:220
27779999814,29cyclictest32549-21latency_hist09:55:180
2777999973,36cyclictest17543-21ls08:10:420
2777999973,34cyclictest1252-21/usr/sbin/munin10:50:400
2777999973,34cyclictest1252-21/usr/sbin/munin10:50:400
27779999729,9cyclictest0-21swapper/008:05:500
27779999727,9cyclictest0-21swapper/011:40:320
2777999972,34cyclictest23077-21fschecks_count12:20:370
27785999630,40cyclictest0-21swapper/111:25:161
2777999964,31cyclictest9065-21/usr/sbin/munin07:46:450
2777999963,32cyclictest30919-21grep07:10:550
2777999963,31cyclictest4690-21unin-run11:50:160
2777999963,31cyclictest4690-21unin-run11:50:160
2777999963,31cyclictest3877-21latency_hist07:30:180
2777999962,38cyclictest21309-21grep08:20:520
2777999962,31cyclictest24556-21grep08:30:520
2777999954,34cyclictest22964-21unixbench_singl08:25:530
2777999954,34cyclictest22964-21unixbench_singl08:25:530
2777999954,29cyclictest8587-21latency_hist07:45:190
2777999953,35cyclictest21659-21/usr/sbin/munin09:36:470
2777999953,32cyclictest17406-21sed09:30:180
2777999952,32cyclictest29943-21sort08:50:180
27785999466,19cyclictest24684-21/usr/sbin/munin10:35:471
2777999944,33cyclictest3989-21/usr/sbin/munin07:30:520
2777999943,36cyclictest28023-21sort07:05:320
2777999942,28cyclictest14881-21grep08:00:550
2687929218,41sleep10-21swapper/107:00:571
2778599892,82cyclictest0-21swapper/112:31:041
27789998858,10cyclictest181rcu_preempt12:05:512
2778599872,72cyclictest0-21swapper/109:15:311
2778599872,72cyclictest0-21swapper/109:15:311
27785998723,32cyclictest352-21runrttasks09:01:091
27789998651,7cyclictest181rcu_preempt11:45:482
27789998651,7cyclictest181rcu_preempt11:45:482
27785998648,31cyclictest803-21sendmail-mta09:20:271
2778599863,78cyclictest0-21swapper/111:10:291
27789998549,18cyclictest181rcu_preempt09:22:092
27785998546,31cyclictest0-21swapper/110:35:151
2778599853,73cyclictest0-21swapper/108:05:281
27785998535,37cyclictest0-21swapper/110:10:261
27785998535,37cyclictest0-21swapper/110:10:261
2778599852,69cyclictest0-21swapper/107:40:271
27789998430,31cyclictest39-21ksoftirqd/211:30:432
2778599843,75cyclictest0-21swapper/111:35:311
2778599843,75cyclictest0-21swapper/111:35:311
2778599843,67cyclictest0-21swapper/108:55:261
27789998362,15cyclictest13760-21sh12:05:192
27789998357,12cyclictest181rcu_preempt09:51:052
27789998354,10cyclictest181rcu_preempt11:07:292
27789998354,10cyclictest181rcu_preempt10:59:142
27789998353,13cyclictest181rcu_preempt11:44:152
27789998348,16cyclictest181rcu_preempt08:52:512
2778999834,53cyclictest39-21ksoftirqd/210:40:042
2778999833,68cyclictest19541-21apt-get11:20:322
27785998363,14cyclictest2486-21latency11:45:401
27785998363,14cyclictest2486-21latency11:45:401
2778599834,58cyclictest24175-21fschecks_time09:40:371
2778599834,45cyclictest22516-21apt-get11:25:421
2778599834,45cyclictest22516-21apt-get11:25:421
27789998255,9cyclictest28647-21ssh10:40:512
27789998255,9cyclictest28647-21ssh10:40:512
27789998250,12cyclictest181rcu_preempt10:07:312
27789998250,12cyclictest181rcu_preempt10:07:312
2778999822,53cyclictest29270-21threads08:45:492
27785998265,10cyclictest5483-21latency_hist09:10:191
27785998257,17cyclictest25437-21fschecks_count08:35:371
2778599822,74cyclictest0-21swapper/107:55:271
27789998153,18cyclictest530-21ssh09:55:332
27789998125,12cyclictest0-21swapper/207:45:192
2778599813,69cyclictest0-21swapper/108:50:311
2778599813,68cyclictest0-21swapper/108:10:291
2778599813,63cyclictest0-21swapper/108:25:291
2778599813,63cyclictest0-21swapper/108:25:291
27785998136,39cyclictest297-21systemd-logind08:45:161
27789998059,8cyclictest181rcu_preempt07:25:492
27789998056,8cyclictest181rcu_preempt09:15:342
27789998056,8cyclictest181rcu_preempt09:15:342
27789998047,9cyclictest181rcu_preempt11:17:052
27789998036,10cyclictest0-21swapper/210:10:352
27789998036,10cyclictest0-21swapper/210:10:352
27785998061,12cyclictest18940-21kworker/u17:2-rpciod12:20:471
27785998035,24cyclictest25767-21kworker/u17:2-nfsiod11:07:401
2779599793,36cyclictest15045-21taskset11:12:413
27789997960,13cyclictest29618-21ssh12:31:172
27789997949,11cyclictest181rcu_preempt09:25:422
27789997937,13cyclictest181rcu_preempt09:44:402
27789997914,38cyclictest39-21ksoftirqd/208:30:512
27785997955,15cyclictest20898-21ssh09:35:321
27785997937,35cyclictest0-21swapper/107:30:361
2778599793,71cyclictest0-21swapper/111:40:281
27785997935,37cyclictest0-21swapper/109:45:361
27785997935,37cyclictest0-21swapper/109:45:361
2778999782,64cyclictest22578-21modprobe11:25:352
2778999782,64cyclictest22578-21modprobe11:25:352
27789997817,35cyclictest39-21ksoftirqd/207:20:532
27789997817,35cyclictest39-21ksoftirqd/207:20:532
2778599784,69cyclictest0-21swapper/111:15:301
27785997837,34cyclictest0-21swapper/108:45:451
2778599783,70cyclictest0-21swapper/111:50:281
2778599782,66cyclictest0-21swapper/107:35:321
2778599782,38cyclictest2493-21sshd09:58:341
27789997754,9cyclictest181rcu_preempt12:18:192
27789997752,8cyclictest181rcu_preempt07:35:162
27789997744,8cyclictest181rcu_preempt07:20:182
27789997743,8cyclictest181rcu_preempt10:29:192
27789997735,13cyclictest181rcu_preempt11:14:372
27789997729,29cyclictest39-21ksoftirqd/207:10:252
2778999772,43cyclictest39-21ksoftirqd/210:32:382
27785997759,11cyclictest200-21systemd-journal09:50:341
27785997756,15cyclictest8145-21sh11:55:351
2778599773,67cyclictest25833-21diskmemload12:25:291
2778599773,67cyclictest25833-21diskmemload12:25:291
2778599773,62cyclictest0-21swapper/110:45:281
2778599772,68cyclictest0-21swapper/108:00:291
27785997722,34cyclictest0-21swapper/110:18:371
2778999769,23cyclictest381ktimers/208:10:442
27789997656,8cyclictest181rcu_preempt09:09:462
27789997651,10cyclictest181rcu_preempt09:33:422
27789997647,8cyclictest181rcu_preempt08:28:252
27789997647,8cyclictest181rcu_preempt08:28:252
27789997642,7cyclictest181rcu_preempt07:50:342
27789997642,7cyclictest181rcu_preempt07:50:342
2778999763,50cyclictest10094-21latency_hist11:05:182
27789997633,12cyclictest181rcu_preempt12:25:472
27789997633,12cyclictest181rcu_preempt12:25:472
27785997657,13cyclictest297-21systemd-logind10:55:171
27785997657,13cyclictest297-21systemd-logind10:55:171
2778599763,64cyclictest0-21swapper/107:50:281
2778599763,64cyclictest0-21swapper/107:50:281
27785997636,32cyclictest0-21swapper/109:30:451
27785997634,35cyclictest0-21swapper/108:15:381
27785997634,34cyclictest0-21swapper/110:04:051
2778599762,69cyclictest0-21swapper/108:20:291
27789997557,11cyclictest2267-21diskmemload09:10:502
27789997546,13cyclictest181rcu_preempt08:05:392
27789997540,13cyclictest181rcu_preempt08:35:482
27789997539,12cyclictest181rcu_preempt08:01:572
27789997534,15cyclictest181rcu_preempt12:21:222
27785997559,10cyclictest373-21lldpd12:07:251
2778599754,67cyclictest301ktimers/107:10:341
2778599753,67cyclictest0-21swapper/112:10:291
2778599753,67cyclictest0-21swapper/112:10:291
2778599753,62cyclictest15462-21apt-get10:20:261
2778599753,62cyclictest15462-21apt-get10:20:261
27789997455,10cyclictest5227-21ssh10:02:492
27789997449,10cyclictest181rcu_preempt10:15:592
27789997447,18cyclictest26825-21sh09:45:252
27789997447,18cyclictest26825-21sh09:45:252
27789997440,10cyclictest181rcu_preempt08:55:512
27789997440,10cyclictest181rcu_preempt07:37:592
27785997436,30cyclictest0-21swapper/109:15:181
2778599743,53cyclictest0-21swapper/111:00:481
27785997431,32cyclictest0-21swapper/110:42:001
27785997431,32cyclictest0-21swapper/110:42:001
27785997428,30cyclictest0-21swapper/112:15:211
27785997426,38cyclictest0-21swapper/112:01:131
27789997351,8cyclictest181rcu_preempt09:36:022
27789997347,8cyclictest8457-21kworker/u19:0+rpciod10:23:392
27789997347,8cyclictest8457-21kworker/u19:0+rpciod10:23:392
27789997343,10cyclictest27768-21cyclictest11:55:502
27789997342,9cyclictest181rcu_preempt08:15:382
27789997339,11cyclictest181rcu_preempt11:53:402
2778999732,45cyclictest39-21ksoftirqd/210:54:492
2778999732,45cyclictest39-21ksoftirqd/210:54:492
2778599737,47cyclictest9267-21cstates07:46:321
2778599733,33cyclictest14890-21fschecks_time09:25:371
27785997329,33cyclictest0-21swapper/111:35:181
2778599732,63cyclictest0-21swapper/107:05:261
27789997242,14cyclictest181rcu_preempt11:35:472
27789997242,14cyclictest181rcu_preempt11:35:472
27789997210,44cyclictest39-21ksoftirqd/212:10:362
27789997210,44cyclictest39-21ksoftirqd/212:10:362
27789997210,22cyclictest39-21ksoftirqd/208:25:192
27785997256,10cyclictest21178-21kworker/u17:3-rpciod11:00:161
27785997231,33cyclictest0-21swapper/107:15:481
2778599722,65cyclictest0-21swapper/107:20:291
2778599722,65cyclictest0-21swapper/107:20:291
2779599713,56cyclictest5092-21chrt07:32:073
2779599702,35cyclictest3175-21taskset11:46:023
2779599702,35cyclictest3175-21taskset11:46:023
2779599702,33cyclictest31100-21taskset08:50:523
27789997040,21cyclictest12777-21needreboot07:55:412
2778999703,45cyclictest28447-21latency_hist08:45:192
2778599702,59cyclictest0-21swapper/107:25:301
2779599693,53cyclictest1358-21chrt10:50:333
2779599693,53cyclictest1358-21chrt10:50:333
2779599693,35cyclictest23879-21taskset09:40:293
27789996915,26cyclictest381ktimers/207:05:432
2778599696,56cyclictest0-21swapper/108:30:521
2779599683,28cyclictest19595-21chrt12:15:173
2779599683,28cyclictest19595-21chrt12:15:173
2779599682,32cyclictest8036-21chrt10:07:173
2779599682,32cyclictest8036-21chrt10:07:173
2778999689,13cyclictest39-21ksoftirqd/207:46:332
27789996834,13cyclictest181rcu_preempt09:02:522
2779599672,34cyclictest31452-21taskset12:34:393
2779599672,32cyclictest15308-21taskset08:05:163
2779599672,24cyclictest25868-21chrt12:25:313
2779599672,24cyclictest25868-21chrt12:25:313
2779599663,31cyclictest13619-21chrt12:05:163
2779599663,28cyclictest21981-21chrt11:25:153
2779599662,33cyclictest32426-21taskset07:15:513
2779599662,33cyclictest11100-21chrt11:05:493
2779599662,32cyclictest919-21chrt10:50:123
2779599662,32cyclictest5786-21chrt07:35:353
2779599662,32cyclictest22528-21chrt11:25:273
2779599662,32cyclictest22528-21chrt11:25:273
2779599662,31cyclictest28086-21taskset09:46:133
2779599662,31cyclictest28086-21taskset09:46:133
2779599662,31cyclictest24463-21chrt12:22:183
2779599654,27cyclictest18643-21chrt11:19:183
2779599653,29cyclictest26124-21chrt11:30:463
2779599652,32cyclictest9168-21taskset09:15:443
2779599652,32cyclictest9168-21taskset09:15:443
2779599652,32cyclictest7250-21chrt07:40:293
2779599652,32cyclictest32438-21chrt09:55:003
2779599652,32cyclictest29957-21chrt07:10:283
2779599652,31cyclictest6739-21chrt09:11:023
2779599652,31cyclictest3852-21taskset09:06:193
2779599652,31cyclictest21128-21taskset12:16:183
2779599652,30cyclictest27361-21chrt10:40:113
2779599652,27cyclictest32230-21chrt08:55:423
2779599643,30cyclictest23929-21chrt08:30:393
2779599643,29cyclictest585-21chrt11:42:093
2779599642,33cyclictest20966-21taskset09:35:423
2779599642,32cyclictest23371-21taskset10:32:293
2779599642,31cyclictest9491-21chrt11:57:053
2779599642,31cyclictest16161-21chrt12:09:093
2779599642,31cyclictest14199-21chrt10:17:383
2779599642,30cyclictest11564-21chrt09:20:223
2779599642,29cyclictest23009-21taskset08:26:013
2779599642,29cyclictest23009-21taskset08:26:013
2779599642,27cyclictest28918-21chrt11:35:413
2779599642,27cyclictest28918-21chrt11:35:413
2779599642,23cyclictest5471-21chrt11:50:403
2779599633,29cyclictest16807-21chrt08:09:253
2779599633,26cyclictest17160-21chrt08:10:303
2779599633,25cyclictest18110-21chrt09:30:443
2779599632,31cyclictest10377-21chrt07:49:453
2779599632,30cyclictest8827-21taskset11:02:043
2779599632,30cyclictest20699-21chrt08:20:393
2779599632,30cyclictest15907-21taskset09:26:263
2779599632,29cyclictest763-21chrt07:20:203
2779599632,29cyclictest763-21chrt07:20:203
2779599632,28cyclictest436-21chrt09:55:293
2779599624,26cyclictest11832-21chrt07:52:443
2779599624,26cyclictest11832-21chrt07:52:443
2779599623,29cyclictest27964-21chrt08:42:223
2779599623,27cyclictest6063-21taskset10:05:103
2779599623,24cyclictest29864-21chrt10:43:483
2779599623,24cyclictest29864-21chrt10:43:483
2779599622,30cyclictest2911-21taskset07:25:433
2779599622,30cyclictest28453-21chrt07:05:433
2779599622,30cyclictest17310-21chrt10:22:573
2779599622,30cyclictest17310-21chrt10:22:573
2779599622,27cyclictest10851-21chrt10:11:463
2779599622,27cyclictest10851-21chrt10:11:463
2779599613,26cyclictest19893-21chrt08:17:153
2779599613,25cyclictest13529-21chrt07:58:323
2779599612,29cyclictest20875-21chrt10:28:533
2779599612,28cyclictest4638-21chrt10:55:393
2779599612,28cyclictest26385-21taskset08:37:483
2779599612,27cyclictest29603-21chrt08:47:073
2779599592,25cyclictest1991-21chrt09:01:583
2769525711,28sleep30-21swapper/307:05:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional