You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 01:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Sun Dec 28, 2025 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1367991203,82cyclictest11343-21/usr/sbin/munin11:55:130
1367991183,72cyclictest0-21swapper/007:19:530
13679911350,54cyclictest0-21swapper/010:20:120
1367991133,97cyclictest0-21swapper/009:49:460
1367991113,99cyclictest3410-21idleruntime-cro10:49:350
13679910715,65cyclictest11296-21kworker/u17:2+xprtiod11:24:470
32656210612,68sleep032346-21/usr/sbin/munin07:05:050
1367991068,60cyclictest0-21swapper/012:14:360
1367991068,60cyclictest0-21swapper/012:14:360
13679910639,62cyclictest1359-21cyclictest10:16:100
1374991053,75cyclictest9626-21apt-get10:05:242
1367991054,67cyclictest9755-21ntp_states09:10:060
13679910530,47cyclictest0-21swapper/011:40:050
13679910449,48cyclictest8822-21diskmemload09:09:380
1374991033,85cyclictest22280-21apt-get12:14:482
13749910310,72cyclictest15976-21apt-get11:09:502
13679910343,54cyclictest3938-21/usr/sbin/munin10:51:010
13679910321,52cyclictest22009-21cstates11:19:520
13679910312,68cyclictest11766-21kworker/u17:3+rpciod09:22:200
1367991012,65cyclictest12598-21kworker/u17:4+xprtiod12:04:500
13679910034,55cyclictest0-21swapper/011:31:050
1367991002,75cyclictest0-21swapper/007:10:050
13679910016,75cyclictest22278-21apt-get12:14:480
1367999946,47cyclictest30969-21ssh11:34:560
1367999946,47cyclictest30969-21ssh11:34:560
136799994,61cyclictest489-21true10:44:350
1367999845,47cyclictest1359-21cyclictest10:24:430
136799983,85cyclictest0-21swapper/010:54:460
136799983,85cyclictest0-21swapper/010:54:460
136799983,62cyclictest7707-21ssh10:01:180
136799983,58cyclictest0-21swapper/011:09:370
136799974,77cyclictest0-21swapper/011:59:480
1367999736,55cyclictest1359-21cyclictest09:39:580
136799966,81cyclictest9854-21apt-get10:59:490
136799963,74cyclictest4223-21apt-get08:54:450
136799963,61cyclictest7711-21taskset11:49:580
136799963,61cyclictest7711-21taskset11:49:580
123529617,45sleep21260-21munin-run07:09:352
1374999572,16cyclictest1359-21cyclictest07:20:042
1374999570,19cyclictest21715-21latency_hist12:14:382
1374999570,19cyclictest21715-21latency_hist12:14:382
136799953,80cyclictest4370-21apt-get11:44:480
136799953,79cyclictest0-21swapper/007:54:520
136799953,57cyclictest12037-21apt-get10:09:460
136799952,80cyclictest0-21swapper/008:14:470
136799952,61cyclictest0-21swapper/011:17:150
136799952,61cyclictest0-21swapper/011:17:150
1374999478,10cyclictest16002-21kworker/u17:1+ext4-rsv-conversion07:59:502
1372999418,60cyclictest858-21apt-get11:39:491
136799943,57cyclictest21144-21taskset09:29:550
136799934,53cyclictest0-21swapper/007:24:450
136799933,61cyclictest10052-21/usr/sbin/munin07:35:160
136799932,80cyclictest0-21swapper/008:19:490
136799932,62cyclictest0-21swapper/009:39:310
136799932,58cyclictest0-21swapper/008:10:510
136799932,58cyclictest0-21swapper/008:10:510
137499926,76cyclictest12586-21apt-get10:09:572
1367999217,46cyclictest11296-21kworker/u17:2+xprtiod11:11:170
1374999160,11cyclictest181rcu_preempt12:04:512
136799913,61cyclictest7219-21modprobe09:04:340
136799913,61cyclictest7219-21modprobe09:04:340
1367999133,47cyclictest0-21swapper/010:06:000
136799912,59cyclictest4282-21true12:39:350
136799912,59cyclictest4282-21true12:39:350
136799904,81cyclictest0-21swapper/010:34:510
136799903,80cyclictest0-21swapper/012:19:500
136799903,56cyclictest380-21in:imuxsock12:29:340
136799902,75cyclictest3253-21apt-get09:54:460
136799902,75cyclictest3253-21apt-get09:54:460
3253028911,56sleep132557-21fschecks_count07:05:021
136799893,78cyclictest18659-21grep08:00:390
1367998934,47cyclictest0-21swapper/007:19:390
1367998925,56cyclictest0-21swapper/009:49:380
1374998863,16cyclictest6914-21ssh10:54:512
1374998863,16cyclictest6914-21ssh10:54:512
137499873,73cyclictest21853-21apt-get08:09:492
137499873,73cyclictest21853-21apt-get08:09:492
136799874,77cyclictest0-21swapper/009:14:490
136799874,77cyclictest0-21swapper/009:14:490
1367998728,49cyclictest0-21swapper/008:04:470
1374998668,13cyclictest1359-21cyclictest11:09:372
136799862,53cyclictest0-21swapper/010:30:550
136799862,53cyclictest0-21swapper/010:30:550
136799854,72cyclictest15468-21awk07:50:060
136799852,52cyclictest0-21swapper/008:40:080
136799843,52cyclictest0-21swapper/008:24:540
1374998363,12cyclictest28965-21kworker/u17:0-rpciod10:45:052
1374998348,15cyclictest181rcu_preempt11:24:502
137299833,55cyclictest1084-21date11:39:381
137299833,55cyclictest1084-21date11:39:381
136799834,72cyclictest2601-21/usr/sbin/munin08:50:020
136799833,73cyclictest13309-21apt-get07:44:470
136799833,70cyclictest11701-21apt-get07:39:500
1374998264,12cyclictest6386-21ssh09:59:542
137299828,48cyclictest21352-21apt-get08:09:491
137299828,48cyclictest21352-21apt-get08:09:491
1372998257,17cyclictest201-21systemd-journal10:08:051
136799823,72cyclictest31366-21apt-get08:37:480
136799822,73cyclictest1393-21latency_hist12:34:370
136799822,57cyclictest8559-21/usr/sbin/munin07:29:520
9922815,13sleep30-21swapper/307:06:233
1374998165,10cyclictest21548-21ntp_offset09:30:042
137499812,41cyclictest21542-21/usr/sbin/munin10:24:502
1372998159,16cyclictest29563-21latency_hist09:44:371
1374998063,11cyclictest39-21ksoftirqd/207:39:562
1372998064,10cyclictest19139-21fschecks_count11:14:561
1372998064,10cyclictest19139-21fschecks_count11:14:561
1372998057,16cyclictest6134-21latency_hist09:59:381
137299804,66cyclictest17861-21apt-get07:59:491
137299803,67cyclictest29656-21apt-get08:34:481
137299802,45cyclictest31-21ksoftirqd/110:33:481
137299802,45cyclictest31-21ksoftirqd/110:33:481
1367998028,46cyclictest0-21swapper/009:29:370
136799802,71cyclictest984-21/usr/sbin/munin08:44:530
1367998026,47cyclictest0-21swapper/008:30:080
1372997964,10cyclictest12598-21kworker/u17:4-rpciod11:19:571
1372997958,15cyclictest9927-21kworker/u17:0+events_unbound12:24:341
1374997841,30cyclictest32082-21sshd11:36:052
1374997841,30cyclictest32082-21sshd11:36:052
1372997863,10cyclictest16819-21proc_pri12:05:071
1372997861,12cyclictest1359-21cyclictest12:15:311
137299783,46cyclictest31-21ksoftirqd/109:49:201
1374997756,15cyclictest7148-21latency_hist11:49:372
137499775,33cyclictest181rcu_preempt11:39:492
1374997749,19cyclictest24828-21sendmail08:19:352
1374997734,16cyclictest11781-21munin-plugin-st09:14:342
1374997726,39cyclictest1382-21kernelversion08:45:012
1372997745,21cyclictest15731-21apt-key11:09:391
137499762,40cyclictest25535-21tune2fs12:19:572
1372997659,11cyclictest26279-21kworker/u17:2-rpciod09:13:271
137299763,63cyclictest12586-21apt-get10:09:561
137299762,37cyclictest31-21ksoftirqd/110:22:191
1374997547,10cyclictest6650-21timerandwakeup09:00:112
1374997547,10cyclictest6650-21timerandwakeup09:00:112
1374997537,13cyclictest181rcu_preempt12:24:482
1372997549,19cyclictest12115-21cpu09:14:511
1372997549,19cyclictest12115-21cpu09:14:511
1372997537,25cyclictest3817-21awk09:55:031
1374997441,10cyclictest181rcu_preempt12:36:272
1374997441,10cyclictest181rcu_preempt12:36:272
1372997428,17cyclictest181rcu_preempt09:08:051
137299742,43cyclictest31-21ksoftirqd/110:57:141
137299742,43cyclictest31-21ksoftirqd/110:57:141
1372997424,28cyclictest31-21ksoftirqd/110:00:061
1374997357,11cyclictest39-21ksoftirqd/210:39:362
1374997356,11cyclictest10390-21ssh11:54:522
1374997341,24cyclictest21680-21latency_hist08:09:382
1374997341,10cyclictest181rcu_preempt08:42:302
1372997354,13cyclictest12598-21kworker/u17:4-xprtiod12:29:341
137299735,38cyclictest31-21ksoftirqd/109:25:511
1372997352,15cyclictest20169-21unixbench_singl12:10:121
1372997352,15cyclictest20169-21unixbench_singl12:10:121
1372997351,16cyclictest2971-21sh12:35:591
1372997351,16cyclictest2971-21sh12:35:591
137299732,41cyclictest31-21ksoftirqd/111:02:591
1374997249,15cyclictest8822-21diskmemload09:26:062
1374997245,19cyclictest5860-21modprobe09:59:342
1374997245,19cyclictest5860-21modprobe09:59:342
137499724,30cyclictest24361-21/usr/sbin/munin10:29:452
137499724,30cyclictest24361-21/usr/sbin/munin10:29:452
1374997230,11cyclictest181rcu_preempt08:35:432
1374997228,9cyclictest181rcu_preempt07:44:472
1372997255,11cyclictest5783-21ntp_offset07:20:091
1372997249,15cyclictest358-21ntpd08:05:041
137299724,21cyclictest6613-21latency_hist10:54:361
137299723,42cyclictest31-21ksoftirqd/112:03:551
137299722,30cyclictest3115-21ssh09:54:381
1374997157,9cyclictest1359-21cyclictest11:01:232
1374997155,10cyclictest22260-21if_err_eth011:19:582
1374997150,15cyclictest0-21swapper/208:24:372
137499714,43cyclictest29870-21date08:34:372
1372997155,9cyclictest18354-21sh10:19:371
137299715,26cyclictest31-21ksoftirqd/110:43:271
1372997151,13cyclictest16902-21latency07:55:021
1372997124,9cyclictest301ktimers/108:44:501
137299711,37cyclictest31-21ksoftirqd/111:45:071
137499703,46cyclictest346-21runrttasks09:08:122
1374997017,13cyclictest0-21swapper/207:09:502
137299708,43cyclictest15435-21apt-get11:09:501
1372997051,12cyclictest0-21swapper/110:25:091
1372997048,17cyclictest2964-21kernelversion08:49:591
1372997011,20cyclictest4593-21vmstat07:15:141
137499695,43cyclictest346-21runrttasks11:54:162
137499695,43cyclictest346-21runrttasks11:54:162
1374996952,10cyclictest13264-21ssh11:59:532
1374996946,14cyclictest39-21ksoftirqd/208:59:382
137299697,34cyclictest31-21ksoftirqd/109:23:541
1372996953,10cyclictest3584-21apt-config10:49:371
1372996953,10cyclictest24821-21timerandwakeup09:35:121
1372996951,11cyclictest0-21swapper/108:29:581
137499685,44cyclictest24208-21ssh09:34:582
1374996847,12cyclictest0-21swapper/210:40:082
1374996840,20cyclictest3801-21tune2fs07:14:582
1374996838,8cyclictest181rcu_preempt10:49:392
137499682,46cyclictest346-21runrttasks11:32:392
137499682,30cyclictest29493-21ls09:44:342
137299684,24cyclictest31-21ksoftirqd/111:31:461
1372996842,18cyclictest14707-21latency_hist07:49:371
137299683,36cyclictest10173-21cstates07:34:551
137299682,46cyclictest11323-21grep11:55:121
1372996810,34cyclictest31-21ksoftirqd/111:54:151
1372996810,34cyclictest31-21ksoftirqd/111:54:151
137499676,46cyclictest15503-21/usr/sbin/munin10:15:032
1374996747,13cyclictest620-21perl09:49:572
1374996737,8cyclictest0-21swapper/208:25:092
137499673,47cyclictest17780-21latency_hist09:24:372
1374996724,11cyclictest181rcu_preempt07:24:472
137299674,43cyclictest346-21runrttasks09:03:481
137299674,43cyclictest346-21runrttasks09:03:481
1372996743,18cyclictest7325-21netstat07:25:011
137299673,43cyclictest4485-21uniq08:54:561
1372996729,12cyclictest0-21swapper/110:36:051
1372996715,24cyclictest31-21ksoftirqd/112:31:351
137499663,42cyclictest18751-21cut10:19:572
1374996628,13cyclictest21676-21ls11:19:372
1374996628,13cyclictest21676-21ls11:19:372
137499662,39cyclictest18194-21sed07:59:382
1374996622,34cyclictest16255-21latency_hist07:54:382
1372996645,14cyclictest201-21systemd-journal08:22:061
137299664,41cyclictest1630-21cpuspeed07:09:521
1372996644,16cyclictest392-21cron08:19:351
1372996644,15cyclictest0-21swapper/109:29:531
1381996520,31cyclictest0-21swapper/310:50:093
1374996543,9cyclictest0-21swapper/207:34:372
137499653,43cyclictest12636-21runrttasks09:15:022
137499653,43cyclictest12636-21runrttasks09:15:022
137499651,39cyclictest10174-21cstates07:34:552
138199643,48cyclictest22023-21taskset09:30:133
138199643,47cyclictest9433-21taskset11:53:103
138199643,47cyclictest9433-21taskset11:53:103
138199643,46cyclictest31286-21chrt12:29:543
138199643,45cyclictest6515-21taskset07:23:483
1374996442,13cyclictest0-21swapper/212:33:502
1372996447,11cyclictest11956-21fschecks_time07:39:571
137299642,44cyclictest25048-21fschecks_count11:24:571
138199632,46cyclictest24405-21chrt10:29:483
138199632,46cyclictest24405-21chrt10:29:483
137499633,18cyclictest30125-21fschecks_time09:44:562
137299638,26cyclictest31-21ksoftirqd/107:50:071
1372996348,10cyclictest9313-21timerwakeupswit07:30:101
1372996341,13cyclictest1359-21cyclictest08:24:561
1372996323,29cyclictest794-21latency_hist08:44:371
138199624,45cyclictest1429-21chrt11:39:453
138199623,46cyclictest12894-21taskset07:42:513
138199623,45cyclictest27004-21taskset12:22:083
138199623,45cyclictest18373-21chrt07:59:473
138199622,47cyclictest5865-21taskset08:59:513
138199622,47cyclictest5865-21taskset08:59:513
138199622,46cyclictest14067-21taskset07:45:103
1374996219,32cyclictest3343-21cat08:50:082
138199613,46cyclictest21355-21chrt08:09:353
138199613,45cyclictest17871-21chrt07:59:353
138199613,44cyclictest28801-21chrt12:25:083
138199613,43cyclictest6129-21chrt09:59:383
138199613,43cyclictest2838-21taskset10:47:593
138199613,43cyclictest2247-21chrt09:52:373
138199612,46cyclictest24853-21taskset11:24:503
138199612,45cyclictest25366-21taskset09:36:163
138199612,44cyclictest23572-21taskset12:15:473
138199603,45cyclictest21892-21taskset08:10:083
138199603,45cyclictest21892-21taskset08:10:083
138199603,45cyclictest10044-21taskset07:34:413
138199603,44cyclictest3723-21chrt08:51:413
138199603,44cyclictest1995-21taskset12:35:013
138199603,44cyclictest1995-21taskset12:35:013
138199603,44cyclictest1651-21chrt07:09:523
138199603,44cyclictest11895-21chrt11:56:473
138199602,45cyclictest32344-21taskset11:36:503
138199602,45cyclictest32344-21taskset11:36:503
138199602,45cyclictest21906-21chrt11:19:483
138199594,41cyclictest9334-21chrt09:09:553
138199593,44cyclictest8739-21taskset10:57:233
138199593,44cyclictest8739-21taskset10:57:233
138199593,44cyclictest29306-21chrt11:31:313
138199593,44cyclictest18180-21taskset12:07:533
138199593,43cyclictest9586-21chrt07:31:583
138199593,43cyclictest25665-21chrt08:20:083
138199593,43cyclictest17545-21chrt11:11:343
138199593,43cyclictest14753-21chrt09:19:353
138199593,43cyclictest14753-21chrt09:19:353
138199593,42cyclictest8696-21chrt09:08:173
138199593,42cyclictest16707-21taskset10:15:263
138199592,44cyclictest27648-21chrt10:34:593
138199592,43cyclictest20465-21taskset10:22:433
138199592,42cyclictest29568-21taskset08:33:403
138199592,42cyclictest19697-21chrt12:10:043
138199592,42cyclictest19697-21chrt12:10:043
1381995924,25cyclictest112412chrt10:07:203
138199584,41cyclictest19561-21chrt11:15:063
138199584,41cyclictest19561-21chrt11:15:063
138199583,43cyclictest4517-21chrt09:55:413
138199583,43cyclictest32729-21chrt10:44:063
138199583,43cyclictest30608-21taskset08:36:193
138199583,43cyclictest15987-21chrt07:51:173
138199583,42cyclictest29221-21chrt09:43:503
138199583,42cyclictest2136-21chrt08:47:003
138199583,41cyclictest14970-21chrt12:02:063
138199583,40cyclictest32222-21chrt08:39:593
138199582,45cyclictest8141-21taskset10:02:333
138199582,44cyclictest15221-21chrt11:09:023
138199582,44cyclictest12612-21chrt10:09:513
138199582,42cyclictest3545-21chrt07:14:463
138199582,41cyclictest4377-21chrt11:44:443
138199573,42cyclictest27525-21chrt08:25:103
138199572,42cyclictest20577-21taskset09:29:273
138199572,42cyclictest0-21swapper/310:24:453
138199572,41cyclictest4225-21chrt08:54:453
138199563,41cyclictest16350-21chrt09:20:443
138199563,40cyclictest24486-21chrt08:15:383
138199562,42cyclictest29936-21chrt09:44:513
138199562,41cyclictest11394-21taskset11:01:293
138199543,37cyclictest6858-21chrt07:24:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional