You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 12:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Wed Jan 07, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1765821090,5sleep017660-21grep22:05:150
7520999971,20cyclictest19080-21unixbench_multi23:00:170
7520999969,11cyclictest181rcu_preempt20:29:420
7520999969,11cyclictest181rcu_preempt20:29:420
753499983,71cyclictest7459-21runrttasks21:49:581
7534999550,8cyclictest181rcu_preempt20:34:411
7534999550,8cyclictest181rcu_preempt20:34:411
70792947,60sleep27080-21sshd19:05:582
670129227,36sleep30-21swapper/319:05:223
7538999175,10cyclictest29734-21timerandwakeup00:10:142
7534999044,21cyclictest4231-21ssh22:36:311
7520998951,9cyclictest181rcu_preempt21:45:060
7520998946,8cyclictest181rcu_preempt21:09:410
7534998753,10cyclictest58-21kcompactd021:28:441
7534998753,10cyclictest58-21kcompactd021:28:441
7534998746,10cyclictest181rcu_preempt23:40:151
7520998711,59cyclictest7496-21ssh21:50:040
72132875,61sleep00-21swapper/019:07:430
7538998669,11cyclictest297-21systemd-logind21:59:412
7538998669,11cyclictest297-21systemd-logind21:59:412
7534998647,15cyclictest181rcu_preempt22:50:041
7534998647,15cyclictest181rcu_preempt22:50:041
7520998650,12cyclictest300-21rs:main0
7520998650,12cyclictest300-21rs:main0
7520998638,29cyclictest181rcu_preempt20:40:310
7534998558,14cyclictest0-21swapper/120:44:421
7534998556,12cyclictest181rcu_preempt22:02:311
7520998553,8cyclictest181rcu_preempt20:50:140
7534998448,12cyclictest181rcu_preempt23:59:421
7538998366,11cyclictest17367-21ssh23:50:382
7520998354,9cyclictest181rcu_preempt00:06:390
7520998335,8cyclictest181rcu_preempt00:25:140
7538998266,11cyclictest14681-21diskmemload21:48:332
7538998264,12cyclictest200-21systemd-journal21:33:582
7534998226,10cyclictest0-21swapper/100:20:051
7520998235,14cyclictest181rcu_preempt23:46:470
7520998235,14cyclictest181rcu_preempt23:46:470
7538998165,10cyclictest3095-21kernelversion22:35:032
7534998147,8cyclictest8787-21ssh00:29:421
7520998141,10cyclictest181rcu_preempt00:19:420
7520998141,10cyclictest181rcu_preempt00:19:420
7520998130,12cyclictest181rcu_preempt22:15:400
7534998059,15cyclictest29105-21fschecks_count00:10:011
7534998059,13cyclictest6705-21ssh21:49:181
7534998051,10cyclictest181rcu_preempt23:20:361
7534998047,14cyclictest31205-21ssh22:28:181
7534998047,14cyclictest29733-21ssh21:33:491
7534998045,14cyclictest22270-21ssh23:05:221
7534998029,24cyclictest0-21swapper/119:24:411
7534998029,24cyclictest0-21swapper/119:24:411
7520998065,8cyclictest0-21swapper/020:24:390
7520998064,8cyclictest0-21swapper/022:49:410
7534997953,20cyclictest13101-21latency_hist21:04:411
7534997950,9cyclictest181rcu_preempt23:11:271
7534997944,14cyclictest28155-21ssh00:08:401
7534997932,11cyclictest181rcu_preempt22:48:141
7534997922,12cyclictest0-21swapper/119:54:541
7534997922,12cyclictest0-21swapper/119:54:541
7520997954,10cyclictest181rcu_preempt21:39:500
7520997943,7cyclictest181rcu_preempt21:11:440
7520997941,7cyclictest181rcu_preempt21:27:190
7520997941,7cyclictest181rcu_preempt21:27:190
7520997941,12cyclictest181rcu_preempt22:49:590
7520997941,12cyclictest181rcu_preempt22:49:590
7538997861,11cyclictest14681-21diskmemload00:25:162
7534997856,14cyclictest12997-21kworker/u17:0-xprtiod22:09:571
7534997856,14cyclictest12997-21kworker/u17:0-xprtiod22:09:571
7534997848,10cyclictest181rcu_preempt22:56:181
7534997845,14cyclictest181rcu_preempt23:34:571
7534997842,10cyclictest181rcu_preempt00:04:281
7534997842,10cyclictest181rcu_preempt00:04:281
7534997828,9cyclictest0-21swapper/122:59:541
7520997859,13cyclictest22350-21latency_hist23:59:430
7520997846,9cyclictest181rcu_preempt00:09:500
7520997845,25cyclictest21017-21latency_hist19:49:430
7520997844,11cyclictest352-21runrttasks00:21:560
7520997843,15cyclictest181rcu_preempt21:55:340
7520997843,15cyclictest181rcu_preempt21:55:340
7520997837,8cyclictest181rcu_preempt23:15:020
7520997837,8cyclictest181rcu_preempt23:15:020
7520997836,12cyclictest181rcu_preempt23:24:070
7520997827,9cyclictest181rcu_preempt23:39:330
7538997760,10cyclictest19777-21diskmemload22:09:562
7538997760,10cyclictest19777-21diskmemload22:09:562
753899774,44cyclictest39-21ksoftirqd/219:15:132
753899774,44cyclictest39-21ksoftirqd/219:15:132
753899772,41cyclictest39-21ksoftirqd/220:52:132
7534997756,10cyclictest0-21swapper/119:25:081
7534997756,10cyclictest0-21swapper/119:25:081
7520997757,13cyclictest23926-21latency_hist23:09:400
7520997749,7cyclictest181rcu_preempt23:31:160
7520997749,7cyclictest181rcu_preempt23:31:160
7520997747,16cyclictest4296-21kworker/u17:3+xprtiod22:42:270
7520997747,16cyclictest4296-21kworker/u17:3+xprtiod22:42:270
7538997653,16cyclictest17753-21idleruntime-cro21:14:392
7534997661,11cyclictest7512-21cyclictest21:35:031
7534997651,11cyclictest181rcu_preempt21:23:061
7534997643,14cyclictest1781-21ssh22:33:111
7534997628,9cyclictest181rcu_preempt20:10:071
7520997654,13cyclictest161ktimers/023:24:500
7520997643,10cyclictest181rcu_preempt21:30:010
7520997639,13cyclictest58-21kcompactd022:34:110
7520997634,11cyclictest181rcu_preempt22:35:250
7534997548,8cyclictest181rcu_preempt22:05:151
753499753,52cyclictest16693-21ssh23:50:131
7534997535,11cyclictest170-21jbd2/mmcblk2p2-820:35:101
7534997535,11cyclictest170-21jbd2/mmcblk2p2-820:35:101
753499752,45cyclictest30118-21cat23:19:411
753499752,45cyclictest30118-21cat23:19:411
752099758,51cyclictest16656-21ssh23:50:080
7520997552,17cyclictest10221-21fschecks_count19:15:550
7520997552,17cyclictest10221-21fschecks_count19:15:550
7520997539,9cyclictest181rcu_preempt20:49:390
7538997455,12cyclictest277-21dbus-daemon00:34:402
7538997453,15cyclictest16180-21cut19:34:412
753899744,35cyclictest39-21ksoftirqd/220:30:472
753899744,35cyclictest39-21ksoftirqd/220:30:472
753899742,39cyclictest352-21runrttasks23:25:052
753899742,36cyclictest39-21ksoftirqd/223:24:392
7534997453,9cyclictest181rcu_preempt21:40:181
7534997439,10cyclictest181rcu_preempt21:09:461
7534997433,32cyclictest10303-21cut20:55:021
7520997442,12cyclictest181rcu_preempt19:29:570
7520997436,11cyclictest181rcu_preempt19:53:110
7520997435,8cyclictest181rcu_preempt21:59:470
7520997433,7cyclictest181rcu_preempt23:43:530
7520997428,9cyclictest181rcu_preempt21:35:000
7520997427,9cyclictest0-21swapper/019:35:170
7538997357,8cyclictest0-21swapper/222:54:422
7538997357,8cyclictest0-21swapper/222:54:422
7538997355,11cyclictest297-21systemd-logind23:49:412
7538997355,11cyclictest297-21systemd-logind23:49:412
753899734,34cyclictest39-21ksoftirqd/223:14:112
7534997354,7cyclictest181rcu_preempt20:03:291
7534997346,18cyclictest764-21kernelversion20:25:071
7534997346,18cyclictest764-21kernelversion20:25:071
7534997313,25cyclictest31-21ksoftirqd/100:34:571
7520997338,17cyclictest58-21kcompactd019:55:050
7520997338,17cyclictest58-21kcompactd019:55:050
7520997338,11cyclictest7512-21cyclictest00:35:340
7520997332,8cyclictest181rcu_preempt21:24:130
7520997332,8cyclictest181rcu_preempt19:39:570
7520997331,8cyclictest181rcu_preempt19:20:000
7520997331,8cyclictest181rcu_preempt19:20:000
753899723,38cyclictest39-21ksoftirqd/221:24:012
7538997225,8cyclictest381ktimers/223:15:042
7538997225,8cyclictest381ktimers/223:15:042
7538997212,18cyclictest181rcu_preempt21:09:392
7534997250,10cyclictest0-21swapper/119:09:431
7534997249,16cyclictest7512-21cyclictest21:09:411
7534997248,7cyclictest181rcu_preempt21:16:031
7534997240,11cyclictest181rcu_preempt23:33:441
7534997240,11cyclictest181rcu_preempt23:33:441
753499723,26cyclictest7771-21grep20:45:151
7534997225,9cyclictest0-21swapper/123:49:411
7534997225,9cyclictest0-21swapper/123:49:411
7534997221,11cyclictest181rcu_preempt21:59:391
7534997221,11cyclictest181rcu_preempt21:59:391
7520997259,7cyclictest0-21swapper/000:30:160
7520997248,8cyclictest181rcu_preempt20:30:120
7520997248,8cyclictest181rcu_preempt20:30:120
7520997244,7cyclictest181rcu_preempt22:25:150
7520997237,7cyclictest181rcu_preempt19:25:060
7520997237,7cyclictest181rcu_preempt19:25:060
7520997233,7cyclictest181rcu_preempt22:55:580
752099722,46cyclictest11034-21grep20:55:160
7520997217,19cyclictest0-21swapper/023:10:120
7538997150,13cyclictest298-21in:imuxsock21:50:132
753899714,52cyclictest39-21ksoftirqd/219:37:162
7538997126,9cyclictest381ktimers/200:15:112
7538997126,9cyclictest381ktimers/200:15:112
7538997122,10cyclictest381ktimers/200:20:222
7538997118,41cyclictest27032-21rm21:29:402
7538997118,41cyclictest27032-21rm21:29:402
7534997141,14cyclictest181rcu_preempt19:50:381
7534997140,12cyclictest181rcu_preempt23:25:161
7534997132,17cyclictest1625-21ssh00:18:071
7534997132,17cyclictest1625-21ssh00:18:071
7534997129,14cyclictest181rcu_preempt22:19:121
7534997127,19cyclictest0-21swapper/119:16:011
7534997127,19cyclictest0-21swapper/119:16:011
7520997143,7cyclictest181rcu_preempt21:15:140
7520997136,18cyclictest0-21swapper/000:03:120
7520997136,18cyclictest0-21swapper/000:03:120
7520997134,7cyclictest181rcu_preempt19:14:400
7520997132,8cyclictest181rcu_preempt22:10:160
7520997132,8cyclictest181rcu_preempt22:10:160
7520997132,8cyclictest181rcu_preempt20:59:510
7538997054,11cyclictest7512-21cyclictest21:15:022
7538997053,11cyclictest16597-21kworker/u17:3-rpciod23:55:212
7538997053,10cyclictest0-21swapper/223:40:022
7538997051,12cyclictest30931-21cron20:19:392
7538997011,34cyclictest39-21ksoftirqd/223:33:392
7538997011,34cyclictest39-21ksoftirqd/223:33:392
7534997055,9cyclictest10330-21diskmemload00:31:061
7534997030,13cyclictest0-21swapper/119:10:111
7534997017,28cyclictest30005-21latency20:15:061
7520997031,7cyclictest181rcu_preempt20:15:130
7520997030,7cyclictest181rcu_preempt19:59:560
7538996952,11cyclictest9423-21kworker/u17:2-rpciod00:35:022
7538996952,10cyclictest0-21swapper/221:44:412
7538996951,12cyclictest741-21systemd-run21:39:402
7538996950,13cyclictest18053-21/usr/sbin/munin19:40:172
7538996949,14cyclictest16422-21latency_hist22:04:412
7538996949,13cyclictest0-21swapper/223:35:162
7538996948,15cyclictest2469-21latency_hist22:34:412
753899694,34cyclictest39-21ksoftirqd/220:56:552
7538996922,8cyclictest381ktimers/219:49:502
7538996919,10cyclictest381ktimers/220:09:562
7534996941,8cyclictest181rcu_preempt20:04:481
36832690,6sleep00-21swapper/020:34:560
36832690,6sleep00-21swapper/020:34:560
7538996836,23cyclictest1361-21head20:25:182
7538996836,23cyclictest1361-21head20:25:182
753899683,27cyclictest39-21ksoftirqd/223:08:462
7538996822,10cyclictest381ktimers/220:45:062
7534996853,9cyclictest0-21swapper/122:39:561
7534996853,9cyclictest0-21swapper/122:39:561
7520996843,8cyclictest181rcu_preempt20:09:410
7520996830,8cyclictest181rcu_preempt20:10:140
753899679,19cyclictest39-21ksoftirqd/200:09:402
7538996752,8cyclictest0-21swapper/222:05:022
7538996745,16cyclictest26133-21fschecks_count22:20:012
7538996745,16cyclictest26133-21fschecks_count22:20:012
753899674,26cyclictest39-21ksoftirqd/222:17:122
753899673,34cyclictest39-21ksoftirqd/219:21:142
753899673,34cyclictest39-21ksoftirqd/219:21:142
7538996731,13cyclictest39-21ksoftirqd/222:46:022
753899672,26cyclictest39-21ksoftirqd/220:07:582
7538996722,35cyclictest21031-21latency_hist23:04:432
7534996734,13cyclictest181rcu_preempt22:19:541
7534996734,13cyclictest181rcu_preempt22:19:541
7545996652,9cyclictest451rcuc/300:21:403
753899664,45cyclictest13747-21ntp_offset19:25:112
753899664,45cyclictest13747-21ntp_offset19:25:112
7538996626,23cyclictest0-21swapper/220:20:112
7538996622,8cyclictest381ktimers/222:55:012
7538996620,8cyclictest381ktimers/221:04:422
7538996616,12cyclictest371rcuc/220:40:262
7538996612,14cyclictest23225-21ssh00:00:062
7538996612,14cyclictest23225-21ssh00:00:062
7534996644,14cyclictest16713-21fschecks_time19:35:011
7534996633,16cyclictest58-21kcompactd020:20:071
753899654,45cyclictest9174-21latency_hist19:14:422
7538996525,12cyclictest181rcu_preempt19:49:172
753899652,25cyclictest25305-21ntp_states20:00:102
7534996534,22cyclictest21015-21sh19:49:391
7534996516,23cyclictest0-21swapper/119:30:521
753899644,43cyclictest30059-21unixbench_singl22:25:172
753499642,27cyclictest19051-21/usr/sbin/munin19:40:161
753899635,44cyclictest5224-21sed20:40:152
753899635,44cyclictest5224-21sed20:40:152
7538996314,20cyclictest39-21ksoftirqd/222:41:012
7538996314,20cyclictest39-21ksoftirqd/222:41:012
753899631,27cyclictest39-21ksoftirqd/219:55:172
753899631,27cyclictest39-21ksoftirqd/219:55:172
7534996143,13cyclictest8449-21/usr/sbin/munin20:50:131
7545996046,9cyclictest451rcuc/321:45:093
7545995843,9cyclictest451rcuc/323:13:233
7545995843,10cyclictest451rcuc/319:39:563
754599583,45cyclictest28861-21taskset00:09:513
7545995742,10cyclictest451rcuc/321:27:023
7545995742,10cyclictest451rcuc/321:27:023
7545995741,9cyclictest451rcuc/322:23:593
7545995741,9cyclictest451rcuc/322:23:593
7545995741,9cyclictest451rcuc/322:11:473
7545995741,9cyclictest451rcuc/322:11:473
7545995642,9cyclictest0-21swapper/323:50:133
121232560,7sleep30-21swapper/321:00:063
7545995541,9cyclictest451rcuc/320:35:043
7545995541,9cyclictest451rcuc/320:35:043
7545995538,10cyclictest451rcuc/300:06:273
7545995442,8cyclictest451rcuc/323:32:243
7545995442,8cyclictest451rcuc/323:32:243
7545995440,9cyclictest451rcuc/320:44:523
7545995440,9cyclictest451rcuc/300:34:493
7545995438,9cyclictest451rcuc/321:31:353
7545995437,10cyclictest451rcuc/319:10:343
754599543,42cyclictest30843-21taskset22:27:193
754599543,39cyclictest758-21chrt23:24:413
754599534,40cyclictest16543-21taskset22:56:333
7545995339,9cyclictest451rcuc/322:30:323
7545995339,9cyclictest451rcuc/319:24:533
7545995339,9cyclictest451rcuc/319:24:533
7545995338,9cyclictest451rcuc/321:18:393
7545995337,9cyclictest451rcuc/320:53:213
7545995337,9cyclictest451rcuc/319:47:323
754599524,38cyclictest3026-21taskset22:35:013
7545995239,8cyclictest451rcuc/321:41:243
754599523,38cyclictest11164-21taskset00:33:223
754599523,37cyclictest14842-21chrt22:54:423
754599523,37cyclictest14842-21chrt22:54:423
7545995137,9cyclictest451rcuc/323:45:583
7545995137,9cyclictest451rcuc/323:45:583
7545995137,9cyclictest451rcuc/320:05:153
754599513,38cyclictest2293-21chrt00:19:403
754599513,38cyclictest2293-21chrt00:19:403
754599513,37cyclictest16486-21taskset22:04:423
754599513,37cyclictest12887-21taskset21:58:463
754599513,37cyclictest12887-21taskset21:58:463
754599513,37cyclictest11287-21chrt22:48:553
754599512,39cyclictest30950-21taskset21:35:073
754599512,37cyclictest447-21taskset20:24:583
754599512,37cyclictest447-21taskset20:24:583
7545995037,8cyclictest451rcuc/319:49:523
7545995036,9cyclictest451rcuc/319:34:533
7545995035,9cyclictest451rcuc/323:25:353
754599503,37cyclictest20198-21taskset23:55:093
754599502,37cyclictest8682-21taskset22:44:413
754599502,37cyclictest8682-21taskset22:44:413
7545994934,10cyclictest451rcuc/320:16:453
754599493,37cyclictest25662-21taskset22:19:413
754599493,36cyclictest5943-21taskset00:24:553
754599493,36cyclictest5529-21chrt20:40:263
754599493,36cyclictest28030-21chrt23:15:133
754599493,36cyclictest28030-21chrt23:15:133
754599493,36cyclictest10940-21chrt23:40:133
754599493,35cyclictest11530-21chrt20:59:413
754599493,34cyclictest26203-21chrt20:04:413
7545994931,13cyclictest451rcuc/320:31:403
7545994931,13cyclictest451rcuc/320:31:403
754599492,35cyclictest21998-21chrt21:20:163
754599492,35cyclictest18186-21chrt22:59:573
7545994910,29cyclictest10031-21chrt19:15:323
7545994910,29cyclictest10031-21chrt19:15:323
7545994834,9cyclictest451rcuc/319:54:463
7545994834,9cyclictest451rcuc/319:54:463
754599483,34cyclictest7397-21taskset21:49:503
754599483,32cyclictest24070-21chrt23:09:423
754599482,36cyclictest13916-21chrt21:05:123
754599482,35cyclictest31911-21chrt20:20:133
754599482,33cyclictest28334-21chrt20:10:043
754599473,34cyclictest23673-21chrt00:00:153
754599473,34cyclictest23673-21chrt00:00:153
754599473,34cyclictest16934-21chrt21:12:293
7545994731,9cyclictest451rcuc/322:07:583
754599463,33cyclictest12029-21chrt19:20:063
754599463,33cyclictest12029-21chrt19:20:063
754599463,32cyclictest14982-21taskset19:29:583
754599462,34cyclictest7436-21chrt23:35:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional