You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 18:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Wed Dec 10, 2025 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2477421090,8sleep30-21swapper/309:43:483
398121020,7sleep00-21swapper/011:10:040
3028321006,15sleep20-21swapper/207:06:342
30561999971,15cyclictest0-21swapper/011:03:400
30581999465,21cyclictest8869-21ntp_kernel_err09:13:402
30581999465,21cyclictest8869-21ntp_kernel_err09:13:402
3056199917,72cyclictest7113-21apt-get10:13:280
30571998862,19cyclictest18588-21latency_hist09:33:431
30581998753,11cyclictest181rcu_preempt07:28:152
3058199873,23cyclictest39-21ksoftirqd/209:58:302
30581998715,45cyclictest39-21ksoftirqd/212:10:492
30561998751,11cyclictest181rcu_preempt12:33:320
30581998568,12cyclictest26417-21latency_hist11:53:112
3056199856,66cyclictest15120-21apt-get07:58:270
30561998554,10cyclictest181rcu_preempt10:33:260
30561998552,13cyclictest181rcu_preempt09:28:250
30581998463,14cyclictest6263-21unixbench_singl11:13:502
30581998414,44cyclictest39-21ksoftirqd/207:37:262
30581998360,15cyclictest29285-21ssh11:58:132
30581998351,7cyclictest181rcu_preempt12:23:472
30581998315,45cyclictest39-21ksoftirqd/210:56:512
30581998313,44cyclictest39-21ksoftirqd/209:24:042
30581998312,45cyclictest39-21ksoftirqd/207:45:402
30581998242,23cyclictest17261-21ssh10:33:132
30581998237,12cyclictest18790-21ssh11:38:272
30581998237,12cyclictest18790-21ssh11:38:272
3057199822,66cyclictest2024-21apt-get08:58:241
30561998257,17cyclictest14626-21ssh12:30:360
30561998257,17cyclictest14626-21ssh12:30:360
30561998256,17cyclictest5518-21diskmemload09:53:340
2943728211,41sleep30-21swapper/307:03:423
30581998159,8cyclictest181rcu_preempt11:01:032
30581998157,11cyclictest181rcu_preempt10:38:362
30571998137,35cyclictest27080-21modprobe10:53:091
3057199813,61cyclictest24547-21apt-get08:28:251
3057199812,61cyclictest31998-21apt-get11:03:241
30561998156,16cyclictest7777-21fschecks_count12:18:310
30561998156,16cyclictest5518-21diskmemload10:48:410
3042628112,43sleep030434-21munin-run07:08:110
30581998050,9cyclictest181rcu_preempt12:18:562
30581998048,9cyclictest181rcu_preempt08:38:492
30581998014,35cyclictest39-21ksoftirqd/209:10:552
30561998054,9cyclictest0-21swapper/010:08:320
30561998054,9cyclictest0-21swapper/010:08:320
3056199802,38cyclictest742-21latency07:13:380
30581997937,20cyclictest32600-21timerwakeupswit07:13:202
30581997914,38cyclictest39-21ksoftirqd/208:04:562
30581997914,38cyclictest39-21ksoftirqd/208:04:562
30561997923,28cyclictest4857-21sort12:13:280
30581997827,13cyclictest0-21swapper/210:33:162
30581997822,20cyclictest39-21ksoftirqd/208:23:262
30581997810,43cyclictest39-21ksoftirqd/207:53:182
30561997857,15cyclictest29545-21latency_hist10:58:120
30561997849,9cyclictest181rcu_preempt09:48:270
3058199778,33cyclictest39-21ksoftirqd/211:45:162
30581997761,10cyclictest25760-21diskmemload10:48:442
30581997758,12cyclictest207-21systemd-journal07:23:402
30581997754,17cyclictest9293-21timerwakeupswit07:38:472
30581997739,22cyclictest32519-21ntp_kernel_err12:03:392
30581997739,21cyclictest10642-21awk11:23:272
30581997739,21cyclictest10642-21awk11:23:272
30581997739,11cyclictest1915-21cstates10:03:302
3058199772,45cyclictest39-21ksoftirqd/207:22:112
30571997758,13cyclictest9558-21latency_hist10:18:121
30571997757,14cyclictest19379-21sh10:36:581
30571997738,15cyclictest207-21systemd-journal11:01:021
3056199773,40cyclictest29249-21latency_hist11:58:130
30581997651,10cyclictest181rcu_preempt09:48:432
30581997649,10cyclictest181rcu_preempt11:03:512
30581997646,7cyclictest181rcu_preempt08:08:212
30581997646,7cyclictest181rcu_preempt08:08:212
30581997641,12cyclictest181rcu_preempt07:08:222
30581997641,12cyclictest181rcu_preempt07:08:222
3058199762,64cyclictest7113-21apt-get10:13:312
30571997659,12cyclictest1-21systemd11:24:411
30571997659,12cyclictest1-21systemd11:24:411
30571997654,15cyclictest314-21in:imuxsock12:03:101
30571997645,10cyclictest26842-21ssh11:53:291
30571997644,7cyclictest181rcu_preempt11:15:171
30571997642,24cyclictest6564-21sh10:13:091
30571997642,24cyclictest6564-21sh10:13:091
3057199763,54cyclictest14590-21apt-get07:58:241
30561997644,13cyclictest181rcu_preempt08:41:260
3058199757,32cyclictest39-21ksoftirqd/210:23:422
3058199757,32cyclictest39-21ksoftirqd/210:23:422
30581997550,8cyclictest181rcu_preempt11:37:012
30581997547,8cyclictest181rcu_preempt09:38:452
30581997538,11cyclictest6318-21ssh10:12:062
30581997538,11cyclictest6318-21ssh10:12:062
30581997531,9cyclictest181rcu_preempt08:47:012
30581997518,21cyclictest39-21ksoftirqd/207:48:462
30581997518,21cyclictest39-21ksoftirqd/207:48:462
3057199756,40cyclictest31-21ksoftirqd/108:14:031
30571997537,11cyclictest12328-21/usr/sbin/munin07:48:461
30571997537,11cyclictest12328-21/usr/sbin/munin07:48:461
30561997554,15cyclictest10520-21/usr/sbin/munin11:23:380
30561997554,15cyclictest10520-21/usr/sbin/munin11:23:380
30561997551,16cyclictest6661-21latency_hist07:33:130
30561997548,20cyclictest31853-21latency_hist12:03:120
30561997545,10cyclictest181rcu_preempt10:43:250
3056199752,41cyclictest3929-21sendmail-mta12:11:180
30581997450,9cyclictest181rcu_preempt12:33:132
30581997450,9cyclictest181rcu_preempt12:33:132
30581997446,8cyclictest181rcu_preempt09:46:472
30581997418,34cyclictest39-21ksoftirqd/208:13:362
30571997453,13cyclictest0-21swapper/108:58:101
30571997444,8cyclictest181rcu_preempt09:46:421
30571997430,14cyclictest301ktimers/110:18:211
30561997459,11cyclictest30548-21cyclictest09:03:460
30561997459,11cyclictest30548-21cyclictest09:03:460
30561997455,14cyclictest10456-21latency_hist11:23:130
30561997452,15cyclictest30548-21cyclictest09:43:180
30561997449,17cyclictest11575-21latency_hist07:48:140
3056199742,22cyclictest12025-21munin-run10:23:090
30581997360,7cyclictest0-21swapper/209:20:412
30581997349,8cyclictest181rcu_preempt09:31:452
30581997339,8cyclictest181rcu_preempt12:17:302
30581997328,8cyclictest181rcu_preempt11:12:112
30571997353,14cyclictest30427-21ntp_offset08:43:411
30571997352,7cyclictest181rcu_preempt11:39:551
30571997352,7cyclictest181rcu_preempt11:39:551
30571997352,12cyclictest0-21swapper/108:28:111
30571997336,12cyclictest207-21systemd-journal12:16:341
3057199732,40cyclictest31-21ksoftirqd/109:16:241
3057199732,40cyclictest31-21ksoftirqd/109:16:241
30561997342,13cyclictest10807-21kworker/u17:2+xprtiod09:20:210
30561997332,24cyclictest24828-21apt-key08:28:140
30561997328,25cyclictest8614-21fschecks_time09:13:340
30561997328,25cyclictest8614-21fschecks_time09:13:340
3058199729,41cyclictest39-21ksoftirqd/208:51:262
30581997259,9cyclictest30548-21cyclictest11:18:412
3058199725,29cyclictest39-21ksoftirqd/208:28:392
3058199725,22cyclictest39-21ksoftirqd/210:21:162
30581997248,18cyclictest14954-21kworker/u17:1-xprtiod09:36:182
30581997243,8cyclictest181rcu_preempt09:53:292
30581997242,8cyclictest181rcu_preempt09:03:462
30581997242,8cyclictest181rcu_preempt09:03:462
30581997231,17cyclictest39-21ksoftirqd/211:31:022
30581997221,20cyclictest39-21ksoftirqd/207:59:142
3057199727,39cyclictest4083-21ssh10:08:121
30571997251,14cyclictest5518-21diskmemload11:44:401
30571997240,12cyclictest20322-21ssh09:36:261
30571997236,12cyclictest207-21systemd-journal11:13:081
30571997223,8cyclictest301ktimers/109:38:331
30561997236,10cyclictest181rcu_preempt08:13:340
30561997235,15cyclictest0-21swapper/010:42:590
30561997229,34cyclictest30826-21unixbench_multi08:43:490
30581997149,16cyclictest21828-21fschecks_count08:19:142
30581997146,13cyclictest181rcu_preempt12:33:352
3058199711,44cyclictest29922-21netstat11:58:392
30571997157,8cyclictest0-21swapper/110:38:461
30571997155,10cyclictest314-21in:imuxsock09:58:101
3057199715,45cyclictest355-21runrttasks07:21:591
30571997153,10cyclictest32721-21kworker/u17:2-rpciod09:08:591
3057199714,47cyclictest28499-21fschecks_count08:38:351
3057199713,45cyclictest23695-21unixbench_singl10:43:511
30571997121,11cyclictest301ktimers/112:03:351
3056199713,56cyclictest2092-21apt-config08:58:110
3056199713,56cyclictest11775-21ssh12:25:160
3056199713,40cyclictest14310-21/usr/sbin/munin11:28:480
3056199713,38cyclictest355-21runrttasks09:25:570
30561997113,19cyclictest181rcu_preempt09:13:120
30581997043,7cyclictest181rcu_preempt08:58:532
30571997052,8cyclictest0-21swapper/109:08:121
30571997052,8cyclictest0-21swapper/109:08:121
30571997046,16cyclictest0-21swapper/110:03:121
30571997045,8cyclictest181rcu_preempt12:37:141
3057199704,27cyclictest31-21ksoftirqd/111:34:511
30571997042,10cyclictest14704-21ssh11:30:011
30571997036,11cyclictest355-21runrttasks12:20:591
30571997021,10cyclictest301ktimers/108:33:421
30561997056,10cyclictest30548-21cyclictest11:17:300
30561997041,21cyclictest24339-21fschecks_time11:48:350
30561997038,10cyclictest181rcu_preempt08:08:220
30561997038,10cyclictest181rcu_preempt08:08:220
3056199703,25cyclictest21453-21/usr/sbin/munin08:19:190
3056199702,26cyclictest24547-21apt-get08:28:330
30561997016,23cyclictest18996-21ssh11:38:320
30561997016,23cyclictest18996-21ssh11:38:320
30581996917,21cyclictest0-21swapper/210:43:482
30571996954,10cyclictest22414-21timerandwakeup08:19:281
30571996952,10cyclictest17696-21kworker/u17:2-rpciod09:48:491
30571996949,14cyclictest30548-21cyclictest07:53:331
30571996943,7cyclictest181rcu_preempt10:55:171
30571996943,18cyclictest32090-21proc_pri08:48:431
30571996923,9cyclictest301ktimers/107:08:481
30571996923,9cyclictest301ktimers/107:08:481
30571996922,9cyclictest301ktimers/107:13:171
30561996928,9cyclictest32363-21latency_hist11:03:130
3056199692,34cyclictest16299-21ssh10:30:110
30581996829,9cyclictest181rcu_preempt08:33:312
30571996851,12cyclictest8801-21kernelversion07:38:371
3057199684,40cyclictest12818-21latency_hist12:28:121
30571996839,20cyclictest8642-21cut11:18:451
30571996821,8cyclictest301ktimers/111:48:581
30571996819,15cyclictest31-21ksoftirqd/109:18:311
30561996850,11cyclictest0-21swapper/007:38:330
30561996840,19cyclictest1704-21latency_hist10:03:130
3056199683,30cyclictest21094-21/usr/sbin/munin09:38:320
30561996825,32cyclictest31505-21timerandwakeup07:08:470
30561996825,32cyclictest31505-21timerandwakeup07:08:470
3056199682,31cyclictest26882-21grep08:33:330
30581996729,7cyclictest181rcu_preempt08:53:242
30571996746,7cyclictest181rcu_preempt07:33:551
30571996741,14cyclictest0-21swapper/112:28:461
30571996741,14cyclictest0-21swapper/112:28:461
3057199672,46cyclictest10416-21latency07:43:381
30561996746,8cyclictest181rcu_preempt08:03:380
30561996746,8cyclictest181rcu_preempt08:03:380
3056199674,47cyclictest2615-21cpu08:58:490
30561996744,14cyclictest7708-21unixbench_multi07:33:480
30561996740,10cyclictest181rcu_preempt07:18:250
3056199672,30cyclictest12548-21fschecks_count10:23:320
3056199672,30cyclictest12548-21fschecks_count10:23:320
3057199663,36cyclictest15289-21latency10:28:371
3057199663,30cyclictest31-21ksoftirqd/107:23:451
30571996618,8cyclictest301ktimers/108:08:381
30571996618,8cyclictest301ktimers/108:08:381
30571996618,11cyclictest18077-21latency_hist08:08:131
30571996618,11cyclictest18077-21latency_hist08:08:131
30561996642,16cyclictest207-21systemd-journal11:36:480
3056199662,48cyclictest31499-21/usr/sbin/munin08:48:310
30561996623,12cyclictest181rcu_preempt10:06:510
3057199653,30cyclictest31-21ksoftirqd/110:26:531
3057199653,30cyclictest31-21ksoftirqd/110:26:531
3057199652,30cyclictest31-21ksoftirqd/109:23:291
30571996515,13cyclictest291rcuc/112:08:371
30561996522,12cyclictest181rcu_preempt07:53:330
30561996516,14cyclictest0-21swapper/012:08:120
3057199643,26cyclictest31-21ksoftirqd/107:28:341
30561996439,16cyclictest20795-21cron09:38:100
30561996422,11cyclictest181rcu_preempt11:43:460
290942648,33sleep129086-21apt-get07:03:271
3056199632,34cyclictest4258-21/usr/sbin/munin07:23:450
30561996312,12cyclictest181rcu_preempt07:48:280
30561996312,12cyclictest181rcu_preempt07:48:280
3058699616,39cyclictest2786-21chrt08:59:023
30586996037,15cyclictest451rcuc/310:56:513
30586995945,9cyclictest451rcuc/312:15:313
30586995836,15cyclictest451rcuc/308:00:353
30586995743,10cyclictest451rcuc/309:58:113
30586995738,14cyclictest451rcuc/312:10:493
30586995642,10cyclictest451rcuc/310:13:123
30586995642,10cyclictest451rcuc/310:13:123
3058699552,40cyclictest16963-21taskset10:32:473
30586995439,11cyclictest451rcuc/309:03:253
30586995439,11cyclictest451rcuc/309:03:253
3058699543,36cyclictest30015-21chrt10:58:323
3058699533,40cyclictest17817-21taskset10:33:363
3058699533,38cyclictest10325-21chrt09:17:393
3058699533,38cyclictest10325-21chrt09:17:393
3058699533,35cyclictest8193-21taskset09:13:133
30586995333,13cyclictest451rcuc/308:57:253
3058699524,34cyclictest20926-21taskset09:38:123
30586995237,9cyclictest451rcuc/310:48:493
3058699523,39cyclictest20679-21chrt11:42:033
3058699523,39cyclictest20679-21chrt11:42:033
3058699523,39cyclictest12363-21taskset10:23:263
3058699523,39cyclictest12363-21taskset10:23:263
3058699523,36cyclictest27603-21chrt08:33:473
3058699523,35cyclictest25274-21chrt08:28:493
30586995231,14cyclictest451rcuc/308:51:263
3058699522,38cyclictest1843-21taskset11:06:503
3058699522,37cyclictest14018-21chrt12:28:473
3058699522,37cyclictest14018-21chrt12:28:473
3058699522,37cyclictest12093-21chrt11:25:183
3058699522,37cyclictest12093-21chrt11:25:183
3058699522,37cyclictest11868-21chrt12:25:353
3058699522,36cyclictest3632-21chrt07:23:283
3058699514,34cyclictest1138-21taskset12:05:413
3058699513,37cyclictest8450-21taskset12:18:463
3058699513,37cyclictest12402-21taskset07:48:473
3058699513,37cyclictest12402-21taskset07:48:473
3058699513,34cyclictest22922-21chrt08:23:103
3058699513,32cyclictest11314-21taskset10:20:473
3058699512,34cyclictest14313-21chrt09:23:463
3058699512,34cyclictest1183-21chrt07:13:483
30586995037,9cyclictest451rcuc/311:44:033
30586995034,9cyclictest451rcuc/308:46:473
3058699503,35cyclictest17582-21chrt12:36:253
3058699503,34cyclictest7137-21chrt07:33:363
3058699503,34cyclictest24739-21chrt10:47:583
30586995033,10cyclictest451rcuc/307:57:303
3058699502,39cyclictest4066-21chrt10:08:103
3058699502,37cyclictest17453-21chrt09:30:273
3058699502,34cyclictest6024-21chrt11:13:473
3058699502,34cyclictest4086-21chrt11:10:233
3058699502,34cyclictest23333-21chrt09:42:463
30586994936,9cyclictest451rcuc/311:51:003
30586994936,8cyclictest451rcuc/308:23:353
3058699493,35cyclictest15223-21taskset11:31:463
3058699493,33cyclictest11937-21chrt09:19:093
3058699493,31cyclictest21882-21chrt10:41:383
3058699492,30cyclictest30081-21chrt11:58:453
3058699484,34cyclictest31460-21taskset07:08:463
3058699484,34cyclictest31460-21taskset07:08:463
30586994836,8cyclictest451rcuc/308:04:073
30586994836,8cyclictest451rcuc/308:04:073
30586994835,8cyclictest451rcuc/310:17:483
30586994835,8cyclictest451rcuc/307:45:533
30586994834,9cyclictest451rcuc/311:18:263
3058699483,34cyclictest28402-21taskset09:51:543
179072480,2sleep3451rcuc/311:36:533
3058699474,31cyclictest28187-21chrt11:55:203
3058699472,30cyclictest28794-21chrt08:38:413
3058699462,32cyclictest1195-21chrt10:02:153
30586994533,8cyclictest451rcuc/307:20:583
3058699443,29cyclictest18319-21taskset08:08:153
3058699443,29cyclictest18319-21taskset08:08:153
3058699443,27cyclictest20630-21chrt08:13:463
3058699442,31cyclictest5841-21taskset07:28:473
30586994330,9cyclictest451rcuc/307:42:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional