You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 21:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Dec 06, 2025 12:44:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25254991314,86cyclictest18824-21apt-get11:38:420
25254991314,86cyclictest18824-21apt-get11:38:420
25254991214,68cyclictest19525-21apt-get09:43:420
252549912125,65cyclictest0-21swapper/009:13:580
25254991162,70cyclictest0-21swapper/009:48:420
25254991162,70cyclictest0-21swapper/009:48:420
252549911424,56cyclictest14943-21/usr/sbin/munin09:34:050
25254991132,73cyclictest19331-21rm10:40:480
25254991113,64cyclictest5773-21seq07:44:030
25254991113,64cyclictest5773-21seq07:44:030
25254991104,64cyclictest11307-21df_inode09:28:500
25254991103,69cyclictest2929-21latency_hist09:13:280
25254991103,69cyclictest2929-21latency_hist09:13:280
252549910918,61cyclictest0-21swapper/012:20:480
25254991082,69cyclictest16348-21rm11:33:520
252549910822,51cyclictest13060-21latency_hist11:28:280
252549910818,57cyclictest11032-21latency_hist09:28:300
25254991074,67cyclictest28406-21munin-run10:58:270
25254991074,67cyclictest28406-21munin-run10:58:270
25254991073,68cyclictest29207-21kworker/u17:1+rpciod09:54:280
25254991073,64cyclictest30721-21/usr/sbin/munin07:23:500
25254991064,64cyclictest2508-21grep07:34:050
25254991063,65cyclictest22817-21users11:44:070
25254991058,58cyclictest0-21swapper/010:18:280
25254991053,65cyclictest14447-21/usr/sbin/munin08:14:000
252549910519,60cyclictest355-21runrttasks12:06:000
25254991044,64cyclictest31182-21munin-run11:03:260
252549910444,53cyclictest0-21swapper/008:58:260
25254991043,60cyclictest15923-21find12:28:390
25254991042,66cyclictest32316-21ssh11:04:280
252549910423,48cyclictest0-21swapper/012:34:590
25254991033,72cyclictest355-21runrttasks10:08:400
25254991025,58cyclictest0-21swapper/008:59:030
25254991025,58cyclictest0-21swapper/008:59:030
25254991023,64cyclictest0-21swapper/011:14:500
25254991023,63cyclictest5190-21cat12:09:260
25254991023,57cyclictest7253-21awk10:19:010
252549910222,61cyclictest32561-21sh12:02:040
252549910219,55cyclictest0-21swapper/008:03:370
25254991017,56cyclictest0-21swapper/011:53:280
25254991013,64cyclictest30061-21cron11:58:260
25254991013,62cyclictest355-21runrttasks08:43:470
25254991013,61cyclictest6213-21kworker/u17:3+xprtiod11:18:440
25254991013,54cyclictest0-21swapper/008:08:420
25254991012,64cyclictest23689-21ssh10:48:590
25254991012,59cyclictest16583-21/usr/sbin/munin08:18:540
252549910121,60cyclictest0-21swapper/012:25:410
25254991004,49cyclictest0-21swapper/009:40:450
252549910026,45cyclictest0-21swapper/007:09:020
252549910026,45cyclictest0-21swapper/007:09:020
2525499992,57cyclictest17674-21latency_hist10:38:300
25254999918,48cyclictest11631-21sh10:27:180
2525499986,54cyclictest0-21swapper/011:09:010
2525499983,60cyclictest3746-21/usr/sbin/munin07:38:580
2525499983,60cyclictest3746-21/usr/sbin/munin07:38:580
2525499983,60cyclictest14279-21unixbench_singl11:29:040
2525499983,56cyclictest28996-21head09:59:040
2525499983,56cyclictest28996-21head09:59:040
2525499983,56cyclictest22165-21head08:34:370
2525499983,56cyclictest22165-21head08:34:370
2525499982,65cyclictest22194-21sh10:46:210
2525499982,64cyclictest8887-21ssh12:16:110
2525499982,56cyclictest27615-21expr07:13:460
25254999812,66cyclictest32477-21diskmemload10:03:530
2525499974,56cyclictest0-21swapper/007:29:040
2525499972,57cyclictest6702-21cat07:48:500
25254999714,73cyclictest0-21swapper/008:23:490
25258999656,11cyclictest31308-21kworker/u19:0-ext4-rsv-conversion10:23:401
2525499964,54cyclictest0-21swapper/007:53:490
2525499963,56cyclictest6749-21/usr/sbin/munin09:19:020
2525499953,52cyclictest9919-21runrttasks07:58:490
2525499952,57cyclictest12645-21fschecks_time10:28:510
2525499952,56cyclictest31126-21taskset09:03:390
249342956,17sleep10-21swapper/107:05:571
2525499942,53cyclictest19468-21seq08:28:460
2525499942,53cyclictest19468-21seq08:28:460
2525499937,56cyclictest29630-21grep07:18:540
2525499933,84cyclictest0-21swapper/007:08:290
2525499933,52cyclictest22510-21apt-get08:38:430
25258999270,8cyclictest0-21swapper/111:04:011
25262999137,10cyclictest27913-21latency11:53:522
25258999120,39cyclictest31-21ksoftirqd/107:48:491
244152918,54sleep224441-21awk07:04:102
2525899903,64cyclictest4086-21apt-get12:08:461
2525499902,55cyclictest27621-21modprobe08:53:260
25262998933,9cyclictest0-21swapper/210:33:412
25258998976,9cyclictest22008-21kworker/u17:0-rpciod10:03:261
25258998976,9cyclictest22008-21kworker/u17:0-rpciod10:03:261
25258998923,29cyclictest31-21ksoftirqd/108:08:501
25262998840,10cyclictest181rcu_preempt07:53:282
25258998860,8cyclictest181rcu_preempt10:03:471
25262998758,23cyclictest369-21ntpd08:09:052
25258998762,16cyclictest29207-21kworker/u17:1+flush-179:009:18:291
25258998666,14cyclictest32477-21diskmemload09:34:061
2526299856,60cyclictest9695-21ssh10:24:122
2526299853,57cyclictest23035-21apt-get08:39:092
25262998463,14cyclictest1685-21kworker/u17:2-writeback07:38:472
25262998463,14cyclictest1685-21kworker/u17:2-writeback07:38:472
25262998443,12cyclictest181rcu_preempt11:28:482
25258998461,8cyclictest181rcu_preempt08:04:051
25262998335,10cyclictest181rcu_preempt09:14:482
2525899834,53cyclictest23082-21taskset08:39:101
25258998313,27cyclictest31-21ksoftirqd/107:26:101
2428328327,38sleep30-21swapper/307:04:033
25262998262,13cyclictest19441-21ssh11:38:582
25262998262,13cyclictest19441-21ssh11:38:582
25262998244,18cyclictest22061-21latency_hist09:48:272
25262998239,18cyclictest16715-21ssh09:38:382
25262998217,11cyclictest39-21ksoftirqd/209:48:422
25262998217,11cyclictest39-21ksoftirqd/209:48:422
25258998266,10cyclictest16198-21fschecks_count11:33:491
25258998254,12cyclictest181rcu_preempt11:42:431
25258998254,12cyclictest181rcu_preempt11:42:431
25258998244,31cyclictest30718-21kworker/u18:3+nfsiod11:11:061
25258998211,41cyclictest31-21ksoftirqd/110:52:351
25262998163,10cyclictest16389-21modprobe09:38:262
2525899812,45cyclictest31-21ksoftirqd/109:56:221
2525899812,39cyclictest31-21ksoftirqd/108:18:221
25258998117,46cyclictest31-21ksoftirqd/109:48:291
2526299809,22cyclictest39-21ksoftirqd/211:36:502
25262998060,12cyclictest176-21jbd2/mmcblk2p2-812:16:582
25262998056,18cyclictest27231-21unixbench_singl08:49:072
25262998046,9cyclictest181rcu_preempt11:04:412
2525899802,49cyclictest31-21ksoftirqd/110:36:101
25258998020,31cyclictest31-21ksoftirqd/109:28:291
25262997956,9cyclictest181rcu_preempt09:58:492
25262997956,9cyclictest181rcu_preempt09:58:492
25262997929,12cyclictest181rcu_preempt08:58:302
25262997929,12cyclictest181rcu_preempt08:58:302
25258997916,31cyclictest31-21ksoftirqd/108:44:221
25258997911,28cyclictest31-21ksoftirqd/107:44:451
25258997911,28cyclictest31-21ksoftirqd/107:44:451
25258997910,38cyclictest31-21ksoftirqd/107:31:321
2526299782,33cyclictest6844-21sed12:13:282
25258997857,13cyclictest0-21swapper/108:58:261
25258997850,19cyclictest30593-21latency_hist07:23:291
2525899783,55cyclictest355-21runrttasks08:33:501
2525899783,55cyclictest355-21runrttasks08:33:501
2525899783,42cyclictest31-21ksoftirqd/112:14:231
2525899781,40cyclictest31-21ksoftirqd/107:58:451
25262997762,10cyclictest0-21swapper/210:43:262
25262997740,9cyclictest181rcu_preempt09:53:422
25262997726,7cyclictest181rcu_preempt07:19:022
25262997711,28cyclictest39-21ksoftirqd/208:54:002
25258997737,11cyclictest1-21systemd11:49:001
25262997637,12cyclictest181rcu_preempt09:10:432
25262997637,12cyclictest181rcu_preempt09:10:432
25262997635,9cyclictest181rcu_preempt07:08:372
25262997635,9cyclictest181rcu_preempt07:08:372
2526299762,42cyclictest25611-21latency_hist10:53:282
2525899762,44cyclictest31-21ksoftirqd/109:21:331
25258997611,34cyclictest31-21ksoftirqd/110:53:411
25258997611,34cyclictest31-21ksoftirqd/110:53:411
25262997552,15cyclictest355-21runrttasks08:03:262
25262997546,21cyclictest8621-21kworker/u18:0+events_unbound12:23:292
25262997536,16cyclictest181rcu_preempt10:14:122
2526299753,41cyclictest1801-21ssh12:04:032
25262997534,19cyclictest181rcu_preempt10:59:522
25258997558,10cyclictest20709-21systemctl08:33:271
25258997558,10cyclictest20709-21systemctl08:33:271
25258997541,8cyclictest181rcu_preempt10:41:381
25258997532,8cyclictest181rcu_preempt12:19:311
2525899752,44cyclictest31-21ksoftirqd/107:54:491
2525899752,28cyclictest31-21ksoftirqd/111:48:281
25258997511,39cyclictest31-21ksoftirqd/107:39:261
25258997511,39cyclictest31-21ksoftirqd/107:39:261
25262997437,12cyclictest181rcu_preempt11:08:432
2526299743,48cyclictest5615-21mailstats11:14:002
2525899749,37cyclictest31-21ksoftirqd/109:51:591
2525899749,37cyclictest31-21ksoftirqd/109:51:591
25258997454,13cyclictest30588-21sh11:58:471
25258997442,8cyclictest181rcu_preempt10:46:401
2525899742,43cyclictest31-21ksoftirqd/109:30:571
2525899742,43cyclictest31-21ksoftirqd/107:37:351
2525899742,41cyclictest31-21ksoftirqd/108:50:581
25258997420,24cyclictest31-21ksoftirqd/107:13:331
25258997418,31cyclictest31-21ksoftirqd/108:27:211
25258997416,32cyclictest31-21ksoftirqd/109:03:461
25258997415,30cyclictest31-21ksoftirqd/110:18:511
25262997347,17cyclictest32477-21diskmemload12:29:002
25262997335,13cyclictest181rcu_preempt09:29:512
25262997327,8cyclictest181rcu_preempt09:06:112
2526299732,24cyclictest20014-21/usr/sbin/munin08:29:022
2526299732,24cyclictest20014-21/usr/sbin/munin08:29:022
25258997354,14cyclictest26209-21threads07:09:021
25258997354,14cyclictest26209-21threads07:09:021
25258997351,10cyclictest181rcu_preempt10:13:291
25258997335,19cyclictest4788-21users10:14:151
25258997328,32cyclictest1600-21processes12:04:001
25258997324,27cyclictest31-21ksoftirqd/112:23:521
25258997318,34cyclictest31-21ksoftirqd/111:54:241
25258997314,27cyclictest31-21ksoftirqd/111:30:551
25258997312,26cyclictest31-21ksoftirqd/111:24:071
25258997310,37cyclictest31-21ksoftirqd/109:02:401
25258997310,37cyclictest31-21ksoftirqd/109:02:401
2526299727,28cyclictest24920-21ssh11:48:482
25262997248,11cyclictest181rcu_preempt10:23:292
25262997246,20cyclictest32477-21diskmemload09:18:432
25262997231,9cyclictest181rcu_preempt07:43:592
25262997231,9cyclictest181rcu_preempt07:43:592
25262997229,15cyclictest181rcu_preempt08:14:022
2526299722,46cyclictest31437-21cat07:24:002
25258997252,7cyclictest181rcu_preempt11:03:161
25258997251,7cyclictest181rcu_preempt12:29:011
25262997129,10cyclictest181rcu_preempt12:33:402
25262997125,37cyclictest31493-21rm11:59:052
25258997135,8cyclictest181rcu_preempt09:39:171
25258997132,9cyclictest58-21kcompactd012:34:521
25258997121,25cyclictest31-21ksoftirqd/111:17:061
25258997120,22cyclictest31-21ksoftirqd/109:10:431
25258997120,22cyclictest31-21ksoftirqd/109:10:431
25262997054,10cyclictest28344-21sendmail10:58:262
25262997054,10cyclictest28344-21sendmail10:58:262
25262997051,8cyclictest39-21ksoftirqd/209:27:202
25262997051,12cyclictest309-21systemd-logind08:28:262
25262997050,14cyclictest30738-21apt-get10:03:402
25262997045,9cyclictest181rcu_preempt11:20:282
2526299704,45cyclictest10781-21mailstats12:18:592
25262997043,19cyclictest21825-21grep08:34:332
25262997043,19cyclictest21825-21grep08:34:332
25262997039,11cyclictest181rcu_preempt07:33:292
25258997012,32cyclictest31-21ksoftirqd/111:18:491
25262996951,12cyclictest11212-21mailstats11:24:152
25262996942,10cyclictest181rcu_preempt10:48:272
25262996930,13cyclictest181rcu_preempt07:33:452
2526299692,46cyclictest21734-21apt-get11:43:432
2526299683,34cyclictest11235-21/usr/sbin/munin08:03:372
2526299682,41cyclictest15345-21irqstats10:33:552
25262996824,14cyclictest181rcu_preempt10:10:062
2526299674,39cyclictest8227-21fschecks_time07:53:502
2526299672,29cyclictest16109-21apt-get08:18:422
184542670,6sleep30-21swapper/312:33:113
2526699662,38cyclictest0-21swapper/308:39:243
25258996613,31cyclictest31-21ksoftirqd/108:22:181
25262996535,16cyclictest181rcu_preempt08:44:042
25266996413,29cyclictest0-21swapper/312:08:463
2525899642,28cyclictest31-21ksoftirqd/110:28:551
2526699622,46cyclictest31863-21chrt12:00:043
25262996229,10cyclictest181rcu_preempt07:14:032
25266995744,8cyclictest451rcuc/308:03:523
2526699572,38cyclictest15599-21taskset11:33:223
2526699564,38cyclictest22958-21chrt10:48:293
2526699553,38cyclictest14214-21taskset10:32:113
2526699552,43cyclictest15117-21taskset12:27:033
2526699542,36cyclictest31512-21taskset09:03:533
25266995339,9cyclictest451rcuc/309:33:333
25266995338,9cyclictest451rcuc/311:21:103
2526699533,37cyclictest19166-21chrt09:43:203
2526699532,40cyclictest19747-21chrt08:28:543
2526699532,40cyclictest19747-21chrt08:28:543
2526699532,38cyclictest9418-21taskset12:17:433
2526699532,38cyclictest11008-21chrt08:03:263
25266995238,10cyclictest451rcuc/309:58:583
25266995238,10cyclictest451rcuc/309:58:583
2526699523,39cyclictest24497-21chrt10:50:073
2526699513,38cyclictest10382-21taskset12:18:483
2526699513,36cyclictest6260-21taskset11:14:593
2526699513,35cyclictest19277-21chrt10:40:383
2526699513,31cyclictest28567-21chrt10:58:293
2526699513,31cyclictest28567-21chrt10:58:293
2526699512,37cyclictest13165-21chrt09:32:063
2526699512,36cyclictest27243-21chrt09:57:193
2526699512,35cyclictest894-21chrt10:08:293
2526699512,34cyclictest6074-21chrt09:18:483
2526699509,32cyclictest31845-21taskset11:04:003
2526699504,32cyclictest4322-21chrt10:14:083
2526699503,32cyclictest3015-21chrt09:13:293
2526699503,32cyclictest3015-21chrt09:13:293
2526699502,34cyclictest22703-21chrt08:38:293
2526699502,34cyclictest22703-21chrt08:38:293
2526699494,36cyclictest8263-21taskset07:53:503
25266994936,8cyclictest451rcuc/308:58:523
25266994936,8cyclictest451rcuc/308:58:523
2526699493,36cyclictest1913-21chrt07:33:543
2526699493,36cyclictest10356-21chrt09:27:073
2526699493,33cyclictest30231-21chrt11:00:323
2526699493,33cyclictest26850-21taskset07:13:283
2526699493,33cyclictest26850-21taskset07:13:283
2526699492,38cyclictest19995-21taskset11:39:183
2526699492,38cyclictest19995-21taskset11:39:183
2526699492,37cyclictest8828-21chrt10:22:553
2526699486,31cyclictest14169-21chrt08:13:143
25266994833,8cyclictest451rcuc/312:34:573
2526699483,34cyclictest21683-21taskset11:43:353
2526699483,32cyclictest10545-21chrt11:23:583
2526699482,36cyclictest5355-21chrt09:18:113
2526699482,36cyclictest3074-21chrt10:12:303
2526699482,35cyclictest11324-21taskset10:26:203
2526699473,35cyclictest26224-21chrt11:50:443
2526699473,35cyclictest16208-21chrt10:34:393
2526699473,32cyclictest2034-21chrt11:08:343
2526699473,29cyclictest22476-21chrt09:48:443
2526699473,29cyclictest22476-21chrt09:48:443
2526699463,34cyclictest29021-21taskset11:55:303
2526699463,34cyclictest1040-21chrt12:03:453
2526699463,32cyclictest18656-21chrt11:38:283
2526699462,32cyclictest30617-21chrt07:23:293
2526699454,31cyclictest6762-21chrt07:48:523
2526699453,29cyclictest5301-21chrt07:43:543
2526699453,29cyclictest5301-21chrt07:43:543
2526699443,29cyclictest4540-21chrt07:43:253
2526699443,29cyclictest4540-21chrt07:43:253
2526699443,29cyclictest19480-21chrt09:43:333
25266994432,8cyclictest451rcuc/307:15:573
2526699424,28cyclictest15772-21taskset08:18:043
2526699423,29cyclictest28222-21chrt08:53:533
2526699423,28cyclictest388-21chrt07:28:563
2526699422,28cyclictest25156-21chrt08:43:563
2526699403,28cyclictest18058-21chrt08:23:513
2526699403,26cyclictest17419-21chrt08:23:153
2526699393,27cyclictest31293-21taskset07:23:573
2526699393,27cyclictest26611-21taskset08:48:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional