You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 16:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Thu May 14, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
128109911645,10cyclictest0-21swapper/009:01:430
128109911333,13cyclictest0-21swapper/007:51:320
12810991123,39cyclictest10036-21latency_hist10:31:140
128109911148,10cyclictest9853-21rm09:36:400
128109910938,11cyclictest0-21swapper/012:26:220
128109910938,11cyclictest0-21swapper/012:26:220
128109910937,12cyclictest0-21swapper/007:21:140
128109910834,11cyclictest0-21swapper/008:26:370
12810991074,25cyclictest5259-21expr11:16:270
12810991074,25cyclictest5259-21expr11:16:270
128109910722,14cyclictest0-21swapper/011:37:360
12810991063,39cyclictest6065-21grep08:16:450
128109910629,12cyclictest0-21swapper/010:56:140
128109910541,11cyclictest19676-21snmpd09:26:310
128109910536,7cyclictest0-21swapper/009:16:500
128109910536,7cyclictest0-21swapper/009:16:500
12810991053,45cyclictest32225-21/usr/sbin/munin12:01:340
12810991053,28cyclictest30277-21grep11:56:460
128109910437,8cyclictest0-21swapper/011:31:320
12810991043,38cyclictest352-21runrttasks10:05:390
12810991043,38cyclictest352-21runrttasks10:05:390
128109910377,19cyclictest960-21latency_hist10:16:130
12810991034,36cyclictest19847-21latency10:46:370
12810991033,37cyclictest24836-21latency_hist07:36:140
12810991032,38cyclictest11596-21cut11:26:320
12810991032,38cyclictest11596-21cut11:26:320
12810991017,39cyclictest8330-21apt-get11:21:280
128109910134,9cyclictest181rcu_preempt09:16:110
12810991013,36cyclictest15256-21grep07:06:350
12810991013,36cyclictest15256-21grep07:06:350
12810991013,29cyclictest16884-21latency_hist08:51:130
12810991012,35cyclictest13669-21latency_hist08:41:140
12810991003,31cyclictest26433-21sendmail09:11:110
12810991003,31cyclictest26433-21sendmail09:11:110
128109910030,12cyclictest0-21swapper/007:26:140
1281099998,82cyclictest16414-21apt-get10:41:330
12810999941,12cyclictest352-21runrttasks10:59:580
12810999941,12cyclictest352-21runrttasks10:59:580
1281099993,36cyclictest24124-21latency_hist10:01:140
1281099983,36cyclictest6793-21apt-get08:21:230
1281099982,28cyclictest12197-21/usr/sbin/munin09:41:380
1281099982,28cyclictest12197-21/usr/sbin/munin09:41:380
1281099973,40cyclictest15946-21apt-key10:41:120
12830999669,7cyclictest181rcu_preempt09:54:332
1281099955,40cyclictest27247-21timerwakeupswit11:51:470
1281099953,24cyclictest18429-21/usr/sbin/munin09:52:230
119562959,54sleep212001-21awk06:56:532
12830999475,13cyclictest26989-21sendmail_mailqu11:51:442
12830999473,15cyclictest18041-21apt-key09:51:132
1281099933,36cyclictest2775-21grep08:06:420
1281099933,36cyclictest2775-21grep08:06:420
12830999270,16cyclictest11958-21fschecks_count12:21:302
1282099923,55cyclictest31-21ksoftirqd/109:35:061
12810999257,24cyclictest5382-21latency_hist08:16:140
125782926,16sleep10-21swapper/107:00:101
12820999170,14cyclictest30591-21sh11:57:261
12830999073,10cyclictest0-21swapper/211:11:112
12830999056,11cyclictest181rcu_preempt10:57:242
12830999056,11cyclictest181rcu_preempt10:57:242
1131329024,38sleep30-21swapper/306:56:213
12830998975,10cyclictest12797-21cyclictest07:21:132
12830998967,15cyclictest4958-21latency_hist11:16:142
12830998965,17cyclictest2209-21ssh10:16:482
12830998957,9cyclictest181rcu_preempt10:46:572
12830998953,10cyclictest181rcu_preempt08:31:222
1283099872,47cyclictest39-21ksoftirqd/211:43:082
1283099872,47cyclictest39-21ksoftirqd/211:43:082
12810998755,20cyclictest18788-21latency_hist08:56:140
12810998755,20cyclictest18788-21latency_hist08:56:140
12830998639,12cyclictest181rcu_preempt12:10:002
1283099862,37cyclictest39-21ksoftirqd/211:33:022
1282099863,74cyclictest16414-21apt-get10:41:261
12830998552,7cyclictest181rcu_preempt09:24:392
12830998440,12cyclictest181rcu_preempt09:31:042
12820998464,14cyclictest7293-21apt-get08:21:441
12820998463,15cyclictest23734-21acpi09:06:211
12820998463,15cyclictest23734-21acpi09:06:211
12820998456,20cyclictest20188-21modprobe09:01:101
12820998456,20cyclictest20188-21modprobe09:01:101
12820998444,29cyclictest14898-21modprobe09:46:111
12820998444,29cyclictest14898-21modprobe09:46:111
12830998340,15cyclictest181rcu_preempt10:55:202
12830998334,14cyclictest0-21swapper/209:56:282
1282099835,48cyclictest31-21ksoftirqd/108:41:071
1282099832,49cyclictest31-21ksoftirqd/110:54:571
12820998318,40cyclictest31-21ksoftirqd/107:14:031
1283099829,38cyclictest39-21ksoftirqd/211:21:282
12830998238,13cyclictest181rcu_preempt10:27:512
12830998233,12cyclictest181rcu_preempt09:44:062
12830998233,12cyclictest181rcu_preempt09:44:062
12830998230,13cyclictest181rcu_preempt12:28:482
12830998230,13cyclictest181rcu_preempt12:28:482
12820998266,10cyclictest17950-21apt-key09:51:111
1281099824,35cyclictest24241-21timerwakeupswit11:46:470
12830998159,8cyclictest181rcu_preempt10:38:402
12830998148,11cyclictest181rcu_preempt07:56:292
12830998145,15cyclictest181rcu_preempt11:20:362
12830998145,15cyclictest181rcu_preempt11:20:362
12820998166,11cyclictest12797-21cyclictest12:01:481
12810998155,17cyclictest18148-21latency_hist09:51:140
12830998063,10cyclictest7082-21apt-key08:21:152
12830998047,7cyclictest181rcu_preempt11:49:312
12830998040,12cyclictest181rcu_preempt10:22:122
12830998038,13cyclictest181rcu_preempt09:34:502
12830998037,12cyclictest181rcu_preempt09:03:542
12830998036,12cyclictest181rcu_preempt09:08:422
12830998036,12cyclictest181rcu_preempt09:08:422
1283099802,40cyclictest39-21ksoftirqd/208:10:332
1283099802,40cyclictest39-21ksoftirqd/208:10:332
12820998014,37cyclictest31-21ksoftirqd/111:01:491
12810998012,51cyclictest12797-21cyclictest07:01:150
12830997963,8cyclictest0-21swapper/210:41:302
12830997962,12cyclictest9805-21ntp_kernel_err09:36:392
12830997954,7cyclictest181rcu_preempt08:11:382
12830997947,9cyclictest181rcu_preempt10:09:252
1283099793,55cyclictest24899-21ssh10:02:002
1283099793,55cyclictest24899-21ssh10:02:002
12830997923,25cyclictest39-21ksoftirqd/210:35:012
1282099797,41cyclictest31-21ksoftirqd/110:50:431
12820997963,11cyclictest12797-21cyclictest11:47:381
12820997960,13cyclictest298-21in:imuxsock10:40:011
12820997957,15cyclictest16897-21latency_hist08:51:141
12820997918,35cyclictest31-21ksoftirqd/109:02:011
12820997916,34cyclictest31-21ksoftirqd/107:58:131
12810997927,10cyclictest181rcu_preempt12:23:130
1283099785,35cyclictest39-21ksoftirqd/210:13:342
12830997847,11cyclictest181rcu_preempt08:01:322
12830997846,8cyclictest181rcu_preempt11:04:012
12830997843,9cyclictest181rcu_preempt07:36:142
12830997833,10cyclictest181rcu_preempt08:37:482
12820997858,14cyclictest20211-21diskmemload12:06:461
12820997858,13cyclictest352-21runrttasks08:16:131
12820997837,9cyclictest181rcu_preempt08:52:541
12820997837,9cyclictest181rcu_preempt08:52:541
12810997858,12cyclictest6653-21kworker/u17:4+rpciod09:35:540
12830997755,16cyclictest0-21swapper/208:41:452
12830997755,15cyclictest17458-21netstat07:12:342
12830997751,7cyclictest181rcu_preempt08:57:062
12830997751,7cyclictest181rcu_preempt08:57:062
12830997749,7cyclictest181rcu_preempt12:00:482
12830997737,12cyclictest181rcu_preempt11:26:352
12830997737,12cyclictest181rcu_preempt11:26:352
12830997734,10cyclictest181rcu_preempt07:01:342
12830997734,10cyclictest181rcu_preempt07:01:342
12820997755,15cyclictest20192-21latency_hist11:41:141
1282099773,44cyclictest31-21ksoftirqd/112:22:591
12810997749,19cyclictest3038-21sh09:26:130
12830997652,8cyclictest181rcu_preempt12:11:302
1283099762,43cyclictest39-21ksoftirqd/207:55:052
12820997650,19cyclictest31297-21sh09:18:051
12820997650,19cyclictest31297-21sh09:18:051
1282099764,35cyclictest31-21ksoftirqd/111:18:261
1282099764,35cyclictest31-21ksoftirqd/111:18:261
1282099764,28cyclictest31-21ksoftirqd/110:21:151
12820997621,9cyclictest301ktimers/112:16:201
12820997612,44cyclictest13179-21fschecks_count07:01:331
12820997612,44cyclictest13179-21fschecks_count07:01:331
12810997628,16cyclictest12797-21cyclictest07:01:370
12810997628,16cyclictest12797-21cyclictest07:01:370
12830997549,7cyclictest181rcu_preempt12:18:322
12830997522,28cyclictest0-21swapper/209:18:112
12830997522,28cyclictest0-21swapper/209:18:112
1282099757,43cyclictest31-21ksoftirqd/110:58:421
1282099757,43cyclictest31-21ksoftirqd/110:58:421
12820997537,10cyclictest181rcu_preempt09:51:551
12820997534,9cyclictest181rcu_preempt09:11:361
1282099753,42cyclictest31-21ksoftirqd/111:09:231
12820997529,39cyclictest301ktimers/109:37:391
1281099758,39cyclictest20221-21awk09:01:110
1281099758,39cyclictest20221-21awk09:01:110
12810997545,11cyclictest181rcu_preempt11:06:360
12830997455,13cyclictest352-21runrttasks11:38:302
12830997452,16cyclictest297-21systemd-logind07:26:122
12830997441,10cyclictest181rcu_preempt12:03:372
1282099749,37cyclictest31-21ksoftirqd/110:23:361
1282099743,62cyclictest24674-21apt-get10:01:281
1282099743,62cyclictest24674-21apt-get10:01:281
1282099743,36cyclictest31-21ksoftirqd/110:31:331
12810997451,13cyclictest0-21swapper/010:31:320
1281099744,48cyclictest15245-21latency_hist08:46:140
12830997348,16cyclictest16382-21apt-get07:11:122
12830997348,16cyclictest16382-21apt-get07:11:122
12830997344,10cyclictest181rcu_preempt07:46:412
12830997344,10cyclictest181rcu_preempt07:46:412
12820997348,18cyclictest0-21swapper/109:56:451
12820997334,9cyclictest181rcu_preempt10:06:221
1282099732,31cyclictest31-21ksoftirqd/107:46:481
1282099732,31cyclictest31-21ksoftirqd/107:46:481
1282099732,27cyclictest31-21ksoftirqd/111:54:061
12820997313,34cyclictest31-21ksoftirqd/110:30:211
12830997247,7cyclictest181rcu_preempt09:11:502
12830997242,8cyclictest181rcu_preempt07:41:152
12830997213,22cyclictest39-21ksoftirqd/208:21:422
12820997245,7cyclictest181rcu_preempt12:12:231
12820997244,8cyclictest181rcu_preempt09:28:001
12820997242,8cyclictest181rcu_preempt10:13:101
12820997241,7cyclictest181rcu_preempt11:22:021
12810997254,10cyclictest21810-21ssh11:42:580
12810997254,10cyclictest21810-21ssh11:42:580
1281099723,48cyclictest16909-21apt-get07:11:280
12830997140,12cyclictest181rcu_preempt08:46:452
12820997147,8cyclictest181rcu_preempt09:23:021
1282099714,46cyclictest808-21fschecks_time08:01:311
12820997134,10cyclictest181rcu_preempt11:31:231
12820997133,10cyclictest181rcu_preempt12:30:191
12820997133,10cyclictest181rcu_preempt12:30:191
12820997132,12cyclictest181rcu_preempt11:43:191
12820997132,12cyclictest181rcu_preempt11:43:191
12820997118,15cyclictest301ktimers/107:36:211
1281099715,18cyclictest29612-21ssh10:09:090
1282099707,37cyclictest31-21ksoftirqd/111:27:491
1282099707,37cyclictest31-21ksoftirqd/111:27:491
12820997048,13cyclictest0-21swapper/108:41:321
1282099703,26cyclictest31-21ksoftirqd/108:35:551
12820997031,8cyclictest181rcu_preempt07:23:501
1282099701,30cyclictest31-21ksoftirqd/107:26:231
1281099703,27cyclictest31369-21latency_hist11:06:130
1281099703,26cyclictest352-21runrttasks10:23:590
12810997031,14cyclictest181rcu_preempt07:41:330
12830996945,13cyclictest0-21swapper/207:36:322
1282099697,37cyclictest31-21ksoftirqd/107:43:181
1282099694,45cyclictest19902-21latency_hist07:21:131
12820996930,9cyclictest0-21swapper/108:16:441
1282099692,25cyclictest31-21ksoftirqd/111:11:441
12820996912,26cyclictest31-21ksoftirqd/107:32:581
12810996950,13cyclictest15-21ksoftirqd/007:26:270
12810996929,7cyclictest0-21swapper/008:31:430
12830996843,10cyclictest181rcu_preempt08:51:582
12830996843,10cyclictest181rcu_preempt08:51:582
12830996840,8cyclictest181rcu_preempt07:26:452
12830996833,7cyclictest181rcu_preempt08:26:392
12820996745,6cyclictest181rcu_preempt08:06:341
12820996745,6cyclictest181rcu_preempt08:06:341
12810996746,16cyclictest14056-21kworker/u17:0+rpciod12:16:130
12810996746,14cyclictest1710-21rm10:16:370
12810996746,11cyclictest0-21swapper/012:06:330
1281099674,25cyclictest8754-21apt-get12:16:250
12810996738,11cyclictest181rcu_preempt08:01:270
1283599663,46cyclictest21863-21chrt10:50:403
12820996639,7cyclictest181rcu_preempt08:26:231
1282099663,45cyclictest30096-21fschecks_time07:51:331
12820996616,22cyclictest31-21ksoftirqd/107:06:391
12820996616,22cyclictest31-21ksoftirqd/107:06:391
1281099663,43cyclictest24955-21/usr/sbin/munin07:36:370
1281099663,23cyclictest31451-21cpu07:56:270
12810996529,15cyclictest181rcu_preempt11:14:370
1281099633,47cyclictest28391-21fschecks_count07:46:330
1281099633,47cyclictest28391-21fschecks_count07:46:330
1283599614,40cyclictest24424-21taskset10:01:143
1283599594,36cyclictest0-21swapper/310:01:563
1283599594,36cyclictest0-21swapper/310:01:563
1283599573,45cyclictest2175-21taskset10:16:483
12835995642,9cyclictest0-21swapper/310:41:303
1283599563,39cyclictest30903-21taskset10:11:333
1283599553,41cyclictest26316-21taskset11:51:273
1283599553,40cyclictest17387-21chrt12:31:103
1283599553,40cyclictest17387-21chrt12:31:103
12835995440,9cyclictest451rcuc/309:01:143
12835995440,9cyclictest451rcuc/309:01:143
1283599543,40cyclictest5863-21taskset09:31:023
1283599543,40cyclictest20754-21chrt09:01:353
1283599543,38cyclictest7860-21taskset12:14:333
1283599543,38cyclictest15904-21taskset10:41:113
1283599533,39cyclictest25328-21taskset11:49:283
1283599533,38cyclictest11400-21taskset11:26:263
1283599533,38cyclictest11400-21taskset11:26:263
1283599533,37cyclictest2184-21chrt09:24:053
1283599533,36cyclictest12528-21taskset09:41:323
1283599533,36cyclictest12528-21taskset09:41:323
151962523,8sleep30-21swapper/311:31:463
1283599523,35cyclictest5175-21chrt08:16:103
1283599523,35cyclictest26936-21chrt09:11:283
1283599523,35cyclictest26781-21chrt10:56:563
1283599523,35cyclictest26781-21chrt10:56:563
1283599523,34cyclictest7936-21chrt08:22:593
1283599522,39cyclictest15372-21chrt08:46:243
1283599522,35cyclictest29747-21taskset09:16:233
1283599522,35cyclictest29747-21taskset09:16:233
1283599522,34cyclictest2660-21taskset12:06:143
1283599514,36cyclictest4660-21chrt11:15:483
1283599514,32cyclictest17338-21chrt07:12:323
1283599513,36cyclictest5274-21chrt12:10:443
1283599513,33cyclictest21112-21taskset09:56:073
12835995131,12cyclictest451rcuc/307:56:523
1283599512,38cyclictest15224-21taskset09:46:223
1283599512,35cyclictest9400-21taskset08:26:403
1283599512,33cyclictest8240-21chrt11:21:153
1283599512,33cyclictest8240-21chrt11:21:153
1283599503,37cyclictest23980-21taskset07:31:433
1283599503,35cyclictest10061-21chrt09:36:463
1283599503,34cyclictest2012-21chrt11:11:143
1283599503,34cyclictest11527-21chrt12:21:133
1283599502,38cyclictest17993-21taskset11:36:443
1283599493,31cyclictest19742-21chrt07:19:493
1283599492,34cyclictest9216-21taskset10:29:243
1283599492,33cyclictest20152-21chrt07:21:273
1283599483,35cyclictest28713-21chrt11:01:263
1283599483,34cyclictest12242-21chrt08:36:283
1283599483,32cyclictest10216-21chrt11:23:593
12835994832,9cyclictest451rcuc/307:43:593
12835994830,10cyclictest451rcuc/307:47:353
12835994830,10cyclictest451rcuc/307:47:353
1283599482,33cyclictest23166-21chrt10:52:263
1283599482,31cyclictest13714-21chrt07:01:463
1283599482,31cyclictest13714-21chrt07:01:463
1283599473,35cyclictest11528-21chrt08:31:503
1283599473,33cyclictest25575-21chrt09:08:493
1283599473,33cyclictest25575-21chrt09:08:493
12835994725,15cyclictest451rcuc/308:53:343
12835994725,15cyclictest451rcuc/308:53:343
1283599472,35cyclictest1362-21chrt08:01:453
1283599472,33cyclictest29566-21chrt10:09:023
1283599472,33cyclictest2282-21chrt08:06:263
1283599472,33cyclictest2282-21chrt08:06:263
1283599472,32cyclictest13451-21taskset12:23:283
1283599472,30cyclictest8618-21chrt09:35:203
12835994633,8cyclictest451rcuc/307:11:143
12835994633,8cyclictest451rcuc/307:11:143
12835994632,9cyclictest451rcuc/307:26:213
1283599462,30cyclictest5605-21chrt10:22:393
1283599453,32cyclictest29819-21taskset11:56:383
1283599453,31cyclictest10598-21chrt10:31:353
12835994529,10cyclictest451rcuc/307:38:033
1283599452,30cyclictest30028-21chrt07:51:323
12835994430,9cyclictest451rcuc/311:42:123
12835994430,9cyclictest451rcuc/311:42:123
12835994128,8cyclictest451rcuc/308:16:203
1283599412,25cyclictest13761-21chrt08:41:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional