You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 12:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Jun 29, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21850216010,98sleep021916-21timerwakeupswit18:53:520
21850216010,98sleep021916-21timerwakeupswit18:53:520
225429911331,53cyclictest0-21swapper/023:28:200
225469910465,14cyclictest1618-40find23:48:061
2255899993,70cyclictest3621-21apt-get21:08:262
2255899993,70cyclictest3621-21apt-get21:08:262
22558999578,11cyclictest10481-21/usr/sbin/munin23:08:382
270632930,3sleep2371rcuc/220:48:412
270632930,3sleep2371rcuc/220:48:412
22542999365,21cyclictest0-21swapper/022:13:090
22542999365,21cyclictest0-21swapper/022:13:090
163952930,6sleep30-21swapper/323:18:093
22558999249,12cyclictest31154-21idleruntime22:48:312
22542999165,16cyclictest18079-21latency_hist20:23:080
22558999027,18cyclictest0-21swapper/221:00:352
22558999018,15cyclictest381ktimers/220:23:182
22558999018,15cyclictest381ktimers/220:23:182
22546999053,8cyclictest181rcu_preempt20:05:061
22546999048,22cyclictest22534-21cyclictest22:13:571
22558998851,12cyclictest181rcu_preempt22:28:562
22558998851,12cyclictest181rcu_preempt22:28:562
22546998828,13cyclictest31-21ksoftirqd/120:58:191
22546998828,13cyclictest31-21ksoftirqd/120:58:191
22558998749,9cyclictest181rcu_preempt22:38:252
22546998738,26cyclictest0-21swapper/121:41:201
22546998738,26cyclictest0-21swapper/121:41:201
29102860,7sleep30-21swapper/319:33:423
22558998642,8cyclictest181rcu_preempt23:26:092
22546998646,8cyclictest181rcu_preempt21:53:031
22546998646,8cyclictest181rcu_preempt21:53:031
22558998563,7cyclictest181rcu_preempt20:13:342
22558998463,8cyclictest181rcu_preempt23:49:112
22558998450,11cyclictest181rcu_preempt22:02:242
22558998450,11cyclictest181rcu_preempt22:02:242
22546998442,13cyclictest22534-21cyclictest23:52:491
22546998441,21cyclictest0-21swapper/100:07:411
22546998410,38cyclictest31-21ksoftirqd/120:43:421
22542998468,10cyclictest4912-21timerwakeupswit22:03:430
22558998356,13cyclictest181rcu_preempt20:43:242
22558998344,7cyclictest181rcu_preempt23:57:502
22546998362,10cyclictest0-21swapper/100:08:281
22546998360,12cyclictest0-21swapper/120:23:081
22558998259,8cyclictest9249-21ssh22:12:092
22558998259,8cyclictest9249-21ssh22:12:092
22558998247,7cyclictest181rcu_preempt22:16:402
22558998239,10cyclictest22534-21cyclictest23:03:312
22546998263,10cyclictest0-21swapper/120:43:051
22546998263,10cyclictest0-21swapper/120:43:051
22546998252,7cyclictest181rcu_preempt23:29:181
22546998251,8cyclictest181rcu_preempt22:45:271
22542998257,19cyclictest14548-21ssh22:20:190
22542998257,19cyclictest14548-21ssh22:20:190
22558998164,12cyclictest22534-21cyclictest00:18:222
22558998164,12cyclictest22534-21cyclictest00:18:222
22558998145,7cyclictest181rcu_preempt19:32:212
22558998141,16cyclictest17921-21ssh22:26:262
22558998058,8cyclictest181rcu_preempt00:10:272
22558998054,8cyclictest181rcu_preempt23:18:302
22558998050,7cyclictest181rcu_preempt20:38:382
22558998050,7cyclictest181rcu_preempt20:38:382
22558998048,11cyclictest181rcu_preempt21:42:162
22558998048,11cyclictest181rcu_preempt21:42:162
22558998028,21cyclictest0-21swapper/222:38:072
22546998064,9cyclictest0-21swapper/123:58:441
22546998060,11cyclictest0-21swapper/119:43:091
22546998060,11cyclictest0-21swapper/119:43:091
22546998057,12cyclictest0-21swapper/119:28:091
22546998047,12cyclictest181rcu_preempt23:13:381
22546998043,9cyclictest181rcu_preempt21:12:131
22546998043,9cyclictest181rcu_preempt21:12:131
22546998034,20cyclictest0-21swapper/121:19:561
22546998034,20cyclictest0-21swapper/121:19:561
22546998024,32cyclictest31-21ksoftirqd/119:53:321
22546998024,20cyclictest31-21ksoftirqd/119:08:081
22546998020,32cyclictest31-21ksoftirqd/119:08:291
22558997939,11cyclictest27041-21mailstats19:08:402
22558997927,29cyclictest0-21swapper/220:28:212
22558997927,29cyclictest0-21swapper/220:28:212
22558997926,31cyclictest0-21swapper/223:43:282
22558997924,27cyclictest0-21swapper/219:33:422
22558997920,23cyclictest0-21swapper/221:13:352
22546997960,8cyclictest0-21swapper/120:28:301
22546997960,8cyclictest0-21swapper/120:28:301
22546997957,12cyclictest0-21swapper/120:18:081
22546997953,19cyclictest24832-21kworker/u18:1+nfsiod23:03:341
22546997952,8cyclictest181rcu_preempt22:18:231
22546997952,8cyclictest181rcu_preempt22:18:231
22546997940,11cyclictest24849-21kworker/u19:1-rpciod23:36:281
2254299794,38cyclictest15-21ksoftirqd/023:08:140
22558997847,8cyclictest181rcu_preempt21:25:202
22558997841,9cyclictest181rcu_preempt00:13:342
22558997839,8cyclictest181rcu_preempt22:18:242
22558997839,8cyclictest181rcu_preempt22:18:242
22558997838,11cyclictest29515-21latency_hist20:58:092
22558997838,11cyclictest29515-21latency_hist20:58:092
2255899783,65cyclictest1671-21ssh22:53:372
22558997829,22cyclictest0-21swapper/220:38:092
22558997829,22cyclictest0-21swapper/220:38:092
22546997858,11cyclictest0-21swapper/120:38:071
22546997858,11cyclictest0-21swapper/120:38:071
22546997842,8cyclictest181rcu_preempt19:48:311
22546997840,8cyclictest181rcu_preempt19:23:091
22546997838,11cyclictest5626-21kworker/u19:0-rpciod21:25:011
22546997836,17cyclictest22534-21cyclictest22:33:031
22546997836,17cyclictest22534-21cyclictest22:33:031
22546997828,14cyclictest0-21swapper/121:33:281
22542997849,19cyclictest23035-21grep00:23:290
22558997756,8cyclictest181rcu_preempt21:52:552
22558997756,8cyclictest181rcu_preempt21:52:552
22558997739,22cyclictest30427-21/usr/sbin/munin21:53:412
22558997729,28cyclictest0-21swapper/219:13:432
22558997725,32cyclictest0-21swapper/221:28:292
22558997722,25cyclictest0-21swapper/200:04:302
22546997760,8cyclictest0-21swapper/122:48:421
22546997758,10cyclictest0-21swapper/120:13:091
22546997758,10cyclictest0-21swapper/119:58:301
22546997757,12cyclictest568-21diskstats19:28:271
22546997754,15cyclictest24481-21cpu21:43:221
22546997745,7cyclictest181rcu_preempt21:16:041
22546997744,8cyclictest181rcu_preempt23:53:321
22546997744,7cyclictest181rcu_preempt21:00:131
22546997742,7cyclictest181rcu_preempt22:35:281
2254699771,39cyclictest31-21ksoftirqd/123:20:581
22542997721,24cyclictest0-21swapper/023:22:440
22542997712,34cyclictest15-21ksoftirqd/021:38:290
22542997712,34cyclictest15-21ksoftirqd/021:38:290
22542997710,33cyclictest15-21ksoftirqd/022:16:060
22558997652,11cyclictest181rcu_preempt23:15:102
22558997631,21cyclictest0-21swapper/222:43:332
22558997624,28cyclictest0-21swapper/223:34:522
22546997647,8cyclictest0-21swapper/120:52:161
22546997647,8cyclictest0-21swapper/120:52:161
22546997645,8cyclictest181rcu_preempt19:33:401
22546997641,20cyclictest11244-21kworker/u19:0-rpciod22:39:181
22546997640,21cyclictest1-21systemd23:13:061
22546997637,10cyclictest19484-21ssh23:23:241
22546997634,13cyclictest181rcu_preempt00:16:551
22542997657,14cyclictest12458-21threads20:03:420
22542997655,13cyclictest19504-21sh00:18:050
2254299762,46cyclictest15-21ksoftirqd/019:41:250
2254299762,46cyclictest15-21ksoftirqd/019:41:250
22558997541,7cyclictest181rcu_preempt21:03:472
22558997537,8cyclictest181rcu_preempt18:58:352
22558997537,8cyclictest181rcu_preempt18:58:352
22558997527,13cyclictest0-21swapper/219:58:422
22558997523,30cyclictest0-21swapper/221:45:142
22558997521,30cyclictest0-21swapper/221:18:332
22558997521,30cyclictest0-21swapper/221:18:332
22546997549,8cyclictest181rcu_preempt22:58:351
22546997549,8cyclictest181rcu_preempt22:58:351
2254299759,37cyclictest15-21ksoftirqd/021:37:420
2254299759,26cyclictest8633-21ssh23:03:470
2254299758,37cyclictest15-21ksoftirqd/018:59:310
2254299758,37cyclictest15-21ksoftirqd/018:59:310
2254299754,27cyclictest15-21ksoftirqd/023:15:400
2254299753,42cyclictest15-21ksoftirqd/019:27:520
2254299752,27cyclictest15-21ksoftirqd/022:38:550
2239927513,39sleep222416-21modprobe18:58:062
2239927513,39sleep222416-21modprobe18:58:062
22558997455,13cyclictest8925-21unixbench_multi23:58:442
22558997451,10cyclictest181rcu_preempt00:27:462
22558997437,10cyclictest181rcu_preempt19:42:542
22558997437,10cyclictest181rcu_preempt19:42:542
22558997434,12cyclictest0-21swapper/223:40:392
22558997427,23cyclictest0-21swapper/219:08:092
22558997414,24cyclictest381ktimers/222:03:222
22546997454,12cyclictest20655-21switchtime00:18:411
22546997454,12cyclictest20655-21switchtime00:18:411
22546997451,18cyclictest22534-21cyclictest19:13:231
22546997450,9cyclictest181rcu_preempt22:54:511
22546997450,9cyclictest181rcu_preempt21:56:181
22546997445,7cyclictest181rcu_preempt22:06:321
22546997443,7cyclictest181rcu_preempt18:58:411
22546997443,7cyclictest181rcu_preempt18:58:411
22546997442,8cyclictest181rcu_preempt23:38:171
22542997454,14cyclictest3046-21latency_hist21:08:090
2254299744,45cyclictest0-21swapper/022:55:400
2254299743,44cyclictest15-21ksoftirqd/019:51:250
22542997433,32cyclictest31034-21cron23:43:050
2254299743,27cyclictest1068-21ssh22:53:070
2254299742,38cyclictest15-21ksoftirqd/022:01:380
2254299742,38cyclictest15-21ksoftirqd/022:01:380
2254299741,28cyclictest15-21ksoftirqd/023:00:200
2254299741,28cyclictest15-21ksoftirqd/023:00:200
22542997412,33cyclictest15-21ksoftirqd/019:47:160
22542997412,33cyclictest15-21ksoftirqd/019:47:160
22558997352,15cyclictest4850-21fschecks_time22:58:292
22558997352,15cyclictest4850-21fschecks_time22:58:292
22558997351,9cyclictest181rcu_preempt20:03:302
22558997348,8cyclictest181rcu_preempt21:34:272
22558997342,11cyclictest181rcu_preempt19:48:232
22558997331,8cyclictest0-21swapper/219:48:082
22558997331,8cyclictest0-21swapper/219:48:082
22546997350,15cyclictest2374-21systemd-journal22:12:201
22546997350,15cyclictest2374-21systemd-journal22:12:201
22546997345,7cyclictest181rcu_preempt21:03:511
22546997343,7cyclictest181rcu_preempt00:24:261
22542997357,9cyclictest19676-21snmpd23:24:480
2254299735,30cyclictest15-21ksoftirqd/019:18:370
2254299735,21cyclictest15-21ksoftirqd/021:11:230
2254299735,21cyclictest15-21ksoftirqd/021:11:230
22542997352,15cyclictest21583-21latency_hist22:33:080
22542997352,15cyclictest21583-21latency_hist22:33:080
2254299734,35cyclictest0-21swapper/000:09:050
2254299732,38cyclictest15-21ksoftirqd/021:50:220
2254299732,38cyclictest15-21ksoftirqd/021:50:220
2254299732,36cyclictest15-21ksoftirqd/021:55:200
2254299732,29cyclictest15-21ksoftirqd/000:03:310
22542997313,33cyclictest15-21ksoftirqd/022:24:320
22542997310,35cyclictest15-21ksoftirqd/019:13:380
219202737,41sleep122012-21unixbench_multi18:53:531
219202737,41sleep122012-21unixbench_multi18:53:531
22558997256,9cyclictest0-21swapper/223:28:262
22558997246,7cyclictest181rcu_preempt20:18:232
22558997236,8cyclictest181rcu_preempt19:18:242
22558997236,7cyclictest181rcu_preempt19:23:182
22546997253,14cyclictest0-21swapper/122:03:081
22546997253,14cyclictest0-21swapper/122:03:081
22546997238,9cyclictest58-21kcompactd022:24:481
22546997230,7cyclictest181rcu_preempt19:48:081
22546997230,7cyclictest181rcu_preempt19:48:081
22542997252,14cyclictest27667-21apt-get20:53:100
22542997252,14cyclictest27667-21apt-get20:53:100
2254299723,45cyclictest15-21ksoftirqd/020:33:170
2254299723,45cyclictest15-21ksoftirqd/020:33:170
2254299722,29cyclictest15-21ksoftirqd/021:28:440
2254299722,28cyclictest15-21ksoftirqd/023:47:140
22542997214,34cyclictest31526-21rm21:00:210
22558997120,32cyclictest0-21swapper/219:53:322
22546997151,14cyclictest19676-21snmpd21:29:001
22546997145,7cyclictest181rcu_preempt20:23:281
22546997145,7cyclictest181rcu_preempt20:23:281
22542997155,10cyclictest12206-21munin-run21:23:060
22542997155,10cyclictest12206-21munin-run21:23:060
22542997150,15cyclictest544-21cron20:28:060
22542997150,15cyclictest544-21cron20:28:060
2254299713,35cyclictest15-21ksoftirqd/021:23:140
2254299712,40cyclictest15-21ksoftirqd/022:36:220
2254299712,36cyclictest15-21ksoftirqd/022:48:020
2254299712,26cyclictest15-21ksoftirqd/000:22:080
2254299712,26cyclictest15-21ksoftirqd/000:22:080
22558997042,8cyclictest181rcu_preempt20:08:172
22542997044,20cyclictest17583-21kworker/u17:3+nfsiod23:34:080
22542997028,21cyclictest15-21ksoftirqd/020:13:340
2254299702,36cyclictest15-21ksoftirqd/023:55:000
2254299693,26cyclictest15-21ksoftirqd/021:47:420
22542996853,9cyclictest10604-21ntp_kernel_pll_19:58:360
22542996839,21cyclictest9822-21modprobe19:58:060
2254299682,27cyclictest15-21ksoftirqd/021:15:300
22542996748,13cyclictest2376-21latency23:48:320
2254299673,26cyclictest15-21ksoftirqd/019:28:260
2254299672,28cyclictest15-21ksoftirqd/019:33:200
22542996721,29cyclictest26033-21idleruntime-cro20:48:060
2254299671,34cyclictest15-21ksoftirqd/020:10:380
2254299665,21cyclictest15-21ksoftirqd/020:38:340
2254299665,21cyclictest15-21ksoftirqd/020:38:340
2254299663,26cyclictest15-21ksoftirqd/023:59:240
2254299662,34cyclictest15-21ksoftirqd/020:57:590
2254299662,34cyclictest15-21ksoftirqd/020:57:590
2254299657,33cyclictest15-21ksoftirqd/020:29:440
2254299657,33cyclictest15-21ksoftirqd/020:29:440
22542996534,10cyclictest181rcu_preempt19:08:270
2254299642,35cyclictest15-21ksoftirqd/019:03:120
171022600,5sleep30-21swapper/322:24:073
22565995236,11cyclictest451rcuc/321:14:293
22565995138,9cyclictest451rcuc/323:49:363
22565995138,9cyclictest451rcuc/322:40:293
22565995037,9cyclictest451rcuc/320:47:043
2256599483,34cyclictest11014-21taskset22:13:453
2256599483,32cyclictest6875-21taskset19:48:203
22565994734,9cyclictest451rcuc/323:58:073
2256599463,32cyclictest6642-21chrt22:08:063
2256599463,32cyclictest13892-21taskset21:24:293
2256599463,32cyclictest1146-21taskset23:46:553
2256599462,31cyclictest30740-21chrt22:48:103
217772469,21sleep30-21swapper/318:53:513
217772469,21sleep30-21swapper/318:53:513
2256599454,30cyclictest3539-21taskset22:02:553
2256599454,30cyclictest3539-21taskset22:02:553
2256599453,32cyclictest3953-21taskset21:08:443
2256599453,32cyclictest3953-21taskset21:08:443
2256599453,32cyclictest12910-21taskset22:18:193
2256599453,32cyclictest12910-21taskset22:18:193
2256599453,31cyclictest28563-21taskset23:38:243
2256599453,31cyclictest1673-21chrt22:53:373
2256599453,30cyclictest2809-21taskset21:07:303
22565994532,9cyclictest451rcuc/320:02:593
22565994531,9cyclictest451rcuc/321:50:283
22565994531,9cyclictest451rcuc/321:50:283
22565994531,9cyclictest451rcuc/319:56:393
2256599452,31cyclictest23540-21taskset21:41:273
2256599452,31cyclictest23540-21taskset21:41:273
2256599452,28cyclictest11792-21taskset23:09:143
2256599444,30cyclictest31522-21chrt21:00:203
2256599443,31cyclictest20620-21chrt00:18:413
2256599443,31cyclictest20620-21chrt00:18:413
2256599443,30cyclictest9830-21taskset21:18:293
2256599443,30cyclictest9830-21taskset21:18:293
2256599443,30cyclictest15716-21chrt21:28:263
2256599443,29cyclictest29978-21taskset20:58:263
2256599443,29cyclictest29978-21taskset20:58:263
2256599443,29cyclictest20324-21taskset20:28:333
2256599443,29cyclictest20324-21taskset20:28:333
2256599443,29cyclictest19652-21taskset23:23:283
2256599443,28cyclictest831-21taskset21:58:093
22565994432,8cyclictest451rcuc/320:22:313
2256599443,28cyclictest24394-21taskset19:03:123
22565994423,12cyclictest0-21swapper/300:28:053
2256599434,29cyclictest27722-21chrt20:53:073
2256599434,29cyclictest27722-21chrt20:53:073
2256599433,30cyclictest18730-21chrt21:33:293
2256599433,30cyclictest18522-21chrt20:23:313
2256599433,30cyclictest18522-21chrt20:23:313
2256599433,30cyclictest11035-21taskset00:03:243
2256599433,29cyclictest4160-21chrt19:38:363
2256599433,29cyclictest4160-21chrt19:38:363
2256599433,29cyclictest29635-21chrt19:18:193
2256599433,29cyclictest29537-21taskset22:45:313
2256599433,29cyclictest14479-21chrt00:08:353
22565994330,8cyclictest451rcuc/320:39:563
22565994330,8cyclictest451rcuc/320:39:563
22565994329,9cyclictest451rcuc/320:10:243
22565994329,9cyclictest451rcuc/319:24:283
2256599432,31cyclictest631-21taskset19:28:283
2256599432,31cyclictest23957-21taskset23:29:533
2256599432,29cyclictest6874-21taskset22:08:103
2256599432,29cyclictest6874-21taskset22:08:103
2256599432,28cyclictest7512-21chrt23:03:183
2256599428,25cyclictest9820-21chrt00:01:063
2256599423,30cyclictest24491-21chrt22:38:053
2256599423,30cyclictest19044-21taskset22:28:243
2256599423,30cyclictest19044-21taskset22:28:243
2256599423,30cyclictest19018-21taskset00:16:463
2256599423,30cyclictest15300-21chrt20:13:313
22565994228,9cyclictest451rcuc/319:13:413
2256599422,30cyclictest26426-21chrt19:08:233
2256599422,29cyclictest12044-21taskset20:03:313
2256599413,28cyclictest5145-21taskset22:58:363
2256599413,28cyclictest5145-21taskset22:58:363
2256599413,28cyclictest25453-21chrt23:33:213
2256599413,27cyclictest5238-21taskset19:43:183
2256599413,27cyclictest5238-21taskset19:43:183
2256599412,30cyclictest17682-21taskset23:19:023
2256599412,29cyclictest22687-21taskset20:36:393
2256599412,29cyclictest22687-21taskset20:36:393
2256599412,28cyclictest22713-21taskset18:58:143
2256599412,28cyclictest22713-21taskset18:58:143
2256599402,28cyclictest24478-21taskset21:43:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional