You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 06:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Tue Mar 17, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
104969911948,41cyclictest10478-21cyclictest19:58:531
104969911941,35cyclictest0-21swapper/121:19:541
2213221050,7sleep10-21swapper/122:05:221
10496991054,78cyclictest30516-21ssh21:25:311
104969910538,36cyclictest0-21swapper/120:00:191
10496991053,70cyclictest8100-21ssh22:35:271
861121030,8sleep1301ktimers/100:25:031
10496991033,83cyclictest20641-21apt-get21:10:211
10496991002,29cyclictest19689-21sh21:07:271
10496999925,43cyclictest0-21swapper/123:55:161
10496999925,43cyclictest0-21swapper/123:55:161
1049699992,33cyclictest352-21runrttasks22:16:351
10487999971,21cyclictest0-21swapper/000:30:330
10496999645,40cyclictest0-21swapper/123:50:091
10496999645,40cyclictest0-21swapper/123:50:091
1049699943,7cyclictest0-21swapper/121:40:191
1049699943,7cyclictest0-21swapper/121:40:191
10487999465,19cyclictest0-21swapper/021:00:070
10487999254,7cyclictest181rcu_preempt21:40:170
10487999254,7cyclictest181rcu_preempt21:40:170
10508999175,10cyclictest16139-21kworker/u17:2-rpciod22:50:282
1049699913,8cyclictest0-21swapper/123:40:211
1049699912,32cyclictest13530-21diskmemload23:38:291
10508999069,12cyclictest0-21swapper/219:40:082
1049699902,28cyclictest352-21runrttasks19:22:231
1049699902,28cyclictest352-21runrttasks19:22:231
10487999066,9cyclictest181rcu_preempt20:35:280
10508998969,10cyclictest0-21swapper/221:45:312
1049699898,66cyclictest4780-21apt-get22:30:211
1049699898,66cyclictest4780-21apt-get22:30:211
1049699893,27cyclictest352-21runrttasks19:30:491
10487998951,29cyclictest7919-21modprobe20:35:050
10508998866,16cyclictest544-21cron21:15:062
10508998866,16cyclictest450-21sh22:22:542
10496998841,40cyclictest0-21swapper/122:00:211
1049699882,39cyclictest32616-21ssh23:15:341
1049699882,39cyclictest32616-21ssh23:15:341
10508998770,10cyclictest0-21swapper/223:56:382
10508998770,10cyclictest0-21swapper/223:56:382
10508998769,11cyclictest0-21swapper/222:31:252
10508998769,11cyclictest0-21swapper/222:31:252
1049699873,7cyclictest0-21swapper/120:35:191
1049699863,39cyclictest4181-21ssh21:35:241
1049699863,33cyclictest11168-21latency_hist20:45:081
1048799863,36cyclictest4241-21apt-get22:30:520
1048799863,36cyclictest4241-21apt-get22:30:520
10508998569,9cyclictest0-21swapper/219:25:072
10508998569,9cyclictest0-21swapper/219:25:072
10508998567,10cyclictest0-21swapper/219:15:082
10508998567,10cyclictest0-21swapper/219:15:082
10508998565,11cyclictest0-21swapper/220:40:082
1049699852,32cyclictest28120-21sed21:20:411
1049699852,32cyclictest28120-21sed21:20:411
1049699852,30cyclictest30512-21/usr/sbin/munin00:05:411
10487998570,8cyclictest0-21swapper/022:00:390
10487998561,15cyclictest2767-21fschecks_count23:20:270
10487998557,15cyclictest181rcu_preempt23:42:240
10487998555,12cyclictest181rcu_preempt19:45:410
10508998469,8cyclictest0-21swapper/220:05:082
10508998466,9cyclictest0-21swapper/222:15:322
10496998465,13cyclictest26510-21/usr/sbin/munin00:00:201
1049699845,71cyclictest0-21swapper/120:25:171
1049699842,9cyclictest0-21swapper/123:00:211
1049699842,39cyclictest21061-21sshd19:35:211
1049699842,38cyclictest16869-21latency_hist22:50:081
10487998465,13cyclictest1-21systemd23:55:070
10487998464,10cyclictest0-21swapper/019:15:220
10487998463,8cyclictest181rcu_preempt23:47:240
10487998463,8cyclictest181rcu_preempt23:47:240
10508998364,10cyclictest0-21swapper/219:30:282
10508998363,13cyclictest0-21swapper/219:55:272
10508998362,9cyclictest181rcu_preempt00:00:392
10508998362,15cyclictest3595-21fschecks_time20:20:272
1049699834,36cyclictest26895-21grep23:05:411
1049699833,34cyclictest18261-21taskset22:51:081
1049699833,32cyclictest13010-21ssh22:43:111
1049699833,28cyclictest6768-21fschecks_count20:30:271
1049699832,34cyclictest3234-21/usr/sbin/munin00:15:261
10487998343,8cyclictest0-21swapper/023:06:080
1049699824,33cyclictest16762-21/usr/sbin/munin21:55:391
1049699824,33cyclictest16762-21/usr/sbin/munin21:55:391
10496998243,32cyclictest0-21swapper/122:10:271
1049699824,14cyclictest32575-21apt-get20:10:191
1049699823,6cyclictest0-21swapper/120:05:211
1049699823,6cyclictest0-21swapper/120:05:211
1049699823,69cyclictest0-21swapper/100:10:231
1049699823,33cyclictest12034-21modprobe19:10:061
1049699823,31cyclictest26024-21/usr/sbin/munin19:50:251
1049699823,31cyclictest26024-21/usr/sbin/munin19:50:251
1049699822,36cyclictest2170-21ntpq22:25:341
1049699822,36cyclictest2170-21ntpq22:25:341
1049699822,33cyclictest13781-21/usr/sbin/munin21:50:391
1049699822,31cyclictest12665-21munin-node19:10:181
1049699822,31cyclictest12665-21munin-node19:10:181
1049699822,25cyclictest2221-21netstat20:15:331
10487998266,9cyclictest0-21swapper/000:10:310
10487998264,8cyclictest0-21swapper/019:35:280
10487998263,9cyclictest0-21swapper/020:00:080
10487998262,10cyclictest0-21swapper/020:40:280
10487998257,11cyclictest181rcu_preempt22:26:050
10487998257,11cyclictest181rcu_preempt22:26:050
10487998254,20cyclictest0-21swapper/021:15:100
10487998252,11cyclictest58-21kcompactd021:26:070
10487998249,28cyclictest10478-21cyclictest22:13:360
10487998244,22cyclictest181rcu_preempt23:35:380
10508998163,13cyclictest996-21timerwakeupswit00:10:432
10508998162,13cyclictest11751-21ssh00:30:082
1050899813,68cyclictest29886-21apt-get21:25:212
10508998129,10cyclictest181rcu_preempt20:30:402
1049699814,8cyclictest0-21swapper/123:30:171
1049699814,7cyclictest0-21swapper/122:20:201
1049699814,34cyclictest20814-21/usr/sbin/munin22:55:401
10496998139,35cyclictest0-21swapper/121:30:291
1049699813,6cyclictest0-21swapper/123:20:201
1049699813,36cyclictest9140-21ssh00:25:241
1049699813,35cyclictest352-21runrttasks20:50:201
1049699813,35cyclictest352-21runrttasks20:50:201
1049699813,31cyclictest3534-21/usr/sbin/munin20:20:261
1049699813,30cyclictest8154-21awk23:30:051
1049699813,30cyclictest8154-21awk23:30:051
1049699812,36cyclictest14676-21apt-get20:55:161
1049699812,35cyclictest10500-21latency_hist19:05:101
1049699812,31cyclictest15350-21/usr/sbin/munin19:15:441
10487998163,12cyclictest17794-21rm21:05:080
10487998143,15cyclictest58-21kcompactd020:07:420
10487998143,15cyclictest58-21kcompactd020:07:420
10487998138,12cyclictest0-21swapper/022:35:410
10487998136,16cyclictest10478-21cyclictest19:10:140
10487998136,16cyclictest10478-21cyclictest19:10:140
10487998133,14cyclictest11662-21latency_hist00:30:070
10508998062,12cyclictest16139-21kworker/u17:2-xprtiod23:08:022
10508998055,19cyclictest11002-21/usr/sbin/munin22:40:152
10508998035,37cyclictest4624-21readlink22:30:092
10508998035,37cyclictest4624-21readlink22:30:092
1049699803,37cyclictest29634-21rm23:10:361
1049699803,37cyclictest20359-21taskset23:50:181
1049699803,35cyclictest9993-21apt-get21:45:161
10487998064,8cyclictest0-21swapper/020:30:090
10487998061,10cyclictest0-21swapper/020:20:080
10487998060,8cyclictest181rcu_preempt21:15:400
10487998055,10cyclictest181rcu_preempt20:01:350
10487998054,12cyclictest181rcu_preempt23:00:210
10508997964,10cyclictest646-21ntp_offset20:10:362
10508997958,14cyclictest24037-21unixbench_multi23:00:422
10508997941,12cyclictest181rcu_preempt00:05:162
10508997932,8cyclictest181rcu_preempt19:45:182
10508997917,11cyclictest381ktimers/221:05:162
1049699792,32cyclictest12057-21find20:45:381
1049699792,31cyclictest18014-21/usr/sbin/munin19:25:281
1049699792,30cyclictest12536-21rm00:30:341
1049699792,29cyclictest25026-21/usr/sbin/munin19:45:411
10487997954,11cyclictest181rcu_preempt22:53:000
10487997939,11cyclictest181rcu_preempt21:08:560
10508997857,15cyclictest352-21runrttasks23:33:262
10508997845,24cyclictest12329-21grep20:45:422
1050899784,46cyclictest4387-21if_eth021:35:302
10508997834,12cyclictest181rcu_preempt21:35:092
1049699783,6cyclictest0-21swapper/119:40:171
1049699783,31cyclictest17268-21/usr/sbin/munin21:00:421
10487997864,10cyclictest10478-21cyclictest23:10:410
10487997858,7cyclictest181rcu_preempt22:45:390
10487997854,11cyclictest181rcu_preempt00:00:140
10487997847,19cyclictest13691-21cron22:45:050
10487997842,10cyclictest0-21swapper/021:34:100
1034927813,42sleep010382-21munin-run19:05:060
10508997760,11cyclictest0-21swapper/221:50:292
10508997757,14cyclictest7544-21netstat21:40:332
10508997757,14cyclictest7544-21netstat21:40:332
10508997757,13cyclictest21993-21kworker/u17:0-ext4-rsv-conversion23:55:052
10508997756,15cyclictest30060-21unixbench_multi23:10:422
10508997731,13cyclictest0-21swapper/200:33:092
10487997764,7cyclictest0-21swapper/023:25:380
10487997764,7cyclictest0-21swapper/023:25:380
10487997757,14cyclictest17635-21diskmemload21:58:570
10487997757,14cyclictest17635-21diskmemload21:58:570
10487997747,15cyclictest181rcu_preempt19:25:070
10487997747,15cyclictest181rcu_preempt19:25:070
1048799773,64cyclictest23571-21apt-get23:55:510
1048799773,64cyclictest23571-21apt-get23:55:510
10487997726,12cyclictest0-21swapper/021:52:500
10508997658,10cyclictest0-21swapper/223:45:062
10508997656,15cyclictest10478-21cyclictest22:14:572
10508997650,18cyclictest8134-21modprobe23:30:052
10508997650,18cyclictest8134-21modprobe23:30:052
10487997656,7cyclictest181rcu_preempt22:55:280
10487997654,9cyclictest181rcu_preempt00:10:090
10487997654,16cyclictest318-21ssh22:22:320
10487997627,13cyclictest0-21swapper/019:45:090
10508997553,13cyclictest0-21swapper/220:45:092
10508997551,13cyclictest0-21swapper/220:55:092
10508997551,13cyclictest0-21swapper/220:55:092
10508997537,8cyclictest181rcu_preempt23:15:192
10508997537,8cyclictest181rcu_preempt23:15:192
10487997558,10cyclictest0-21swapper/021:45:370
10487997537,11cyclictest0-21swapper/000:23:000
10487997517,33cyclictest30367-21sed21:25:110
10508997456,11cyclictest0-21swapper/220:10:052
10508997456,11cyclictest0-21swapper/220:10:052
10508997454,13cyclictest0-21swapper/219:10:092
10508997438,9cyclictest181rcu_preempt20:15:272
10508997424,17cyclictest181rcu_preempt19:45:082
1050899742,40cyclictest3061-21cpu00:15:212
10508997415,27cyclictest16544-21tune2fs21:00:282
10487997460,10cyclictest2588-21kworker/u17:0+rpciod21:35:400
10515997340,23cyclictest25734-21chrt23:05:063
10508997352,11cyclictest0-21swapper/221:20:412
10508997352,11cyclictest0-21swapper/221:20:412
10508997349,17cyclictest0-21swapper/223:50:052
10508997349,17cyclictest0-21swapper/223:50:052
10508997337,10cyclictest0-21swapper/222:50:072
10487997354,7cyclictest181rcu_preempt23:30:230
10487997345,20cyclictest30631-21ssh22:19:540
10487997337,16cyclictest58-21kcompactd000:17:560
10487997323,12cyclictest181rcu_preempt22:10:060
10508997256,9cyclictest0-21swapper/222:00:262
10508997253,9cyclictest0-21swapper/219:15:412
10508997253,12cyclictest371rcuc/220:55:192
10487997253,7cyclictest181rcu_preempt23:15:270
10487997253,7cyclictest181rcu_preempt23:15:270
10487997250,12cyclictest0-21swapper/020:15:080
10487997230,12cyclictest0-21swapper/019:30:230
969727111,37sleep29731-21/usr/sbin/munin19:00:492
10508997151,14cyclictest297-21systemd-logind21:20:062
10508997151,10cyclictest0-21swapper/223:40:082
10508997143,12cyclictest181rcu_preempt22:38:072
10508997054,9cyclictest0-21swapper/221:55:392
10508997054,9cyclictest0-21swapper/221:55:392
10508997051,9cyclictest0-21swapper/222:55:372
10508997051,9cyclictest0-21swapper/222:08:212
10508997031,8cyclictest181rcu_preempt00:20:272
10487997048,14cyclictest0-21swapper/020:50:390
10487997048,14cyclictest0-21swapper/020:50:390
10487997044,20cyclictest12106-21latency_hist19:10:080
10487997040,6cyclictest181rcu_preempt19:25:290
10508996950,9cyclictest0-21swapper/219:30:082
10487996937,12cyclictest181rcu_preempt21:21:450
10487996851,12cyclictest10478-21cyclictest19:50:330
10487996851,12cyclictest10478-21cyclictest19:50:330
10487996722,16cyclictest0-21swapper/020:45:410
1050899663,45cyclictest26449-21grep19:50:382
1050899663,45cyclictest26449-21grep19:50:382
10508996549,7cyclictest0-21swapper/223:20:412
1048799654,40cyclictest4177-21timerandwakeup20:20:400
1050899633,39cyclictest5178-21fschecks_count20:25:262
102252636,13sleep30-21swapper/319:03:263
10515996123,29cyclictest0-21swapper/323:55:443
10515996123,29cyclictest0-21swapper/323:55:443
1051599574,36cyclictest23449-21chrt19:40:413
1051599563,38cyclictest18909-21chrt22:00:093
1051599563,38cyclictest18909-21chrt22:00:093
1051599562,42cyclictest9020-21taskset22:36:083
1051599557,24cyclictest0-21swapper/322:30:533
1051599557,24cyclictest0-21swapper/322:30:533
1051599554,38cyclictest20978-21chrt22:55:423
1051599553,42cyclictest3117-21taskset20:20:053
1051599553,34cyclictest1771-21taskset22:25:263
1051599553,34cyclictest1771-21taskset22:25:263
1051599552,36cyclictest10009-21chrt21:45:183
1051599544,40cyclictest28620-21taskset22:15:333
1051599544,37cyclictest4041-21taskset21:35:213
1051599544,37cyclictest19141-21chrt22:00:223
1051599544,35cyclictest1538-21chrt20:15:073
1051599543,38cyclictest15283-21chrt19:15:423
1051599543,36cyclictest32334-21chrt22:21:173
1051599542,37cyclictest15740-21chrt22:47:273
1051599542,33cyclictest2354-21chrt21:31:283
1051599534,35cyclictest29335-21taskset00:05:073
1051599534,32cyclictest15198-21taskset20:55:333
1051599533,38cyclictest6326-21chrt23:25:403
1051599533,38cyclictest6326-21chrt23:25:403
1051599533,38cyclictest18723-21taskset21:05:383
1051599533,37cyclictest26732-21chrt23:05:393
1051599533,37cyclictest20987-21chrt23:50:373
1051599533,35cyclictest16975-21chrt21:00:393
1051599533,32cyclictest8474-21chrt20:35:283
1051599532,37cyclictest16590-21chrt19:20:523
1051599532,37cyclictest16590-21chrt19:20:523
1051599523,39cyclictest27260-21chrt22:13:553
1051599523,39cyclictest13027-21chrt21:50:213
1051599523,36cyclictest26697-21chrt21:19:363
1051599523,35cyclictest11236-21chrt19:05:373
1051599523,32cyclictest7538-21chrt20:30:433
1051599522,35cyclictest5327-21taskset23:25:083
1051599515,30cyclictest0-21swapper/320:00:223
1051599513,36cyclictest5861-21chrt20:25:403
1051599513,36cyclictest31672-21chrt20:05:403
1051599513,36cyclictest31672-21chrt20:05:403
1051599513,30cyclictest21678-21chrt19:35:393
1051599512,36cyclictest7782-21chrt21:40:403
1051599512,36cyclictest7782-21chrt21:40:403
1051599512,35cyclictest10011-21chrt00:25:423
1051599512,34cyclictest3540-21chrt20:20:263
1051599503,34cyclictest19081-21taskset23:47:243
1051599503,34cyclictest19081-21taskset23:47:243
1051599503,33cyclictest24776-21chrt19:45:353
1051599503,32cyclictest930-21chrt00:10:433
1051599503,32cyclictest17117-21chrt22:50:233
1051599503,31cyclictest32204-21chrt00:10:063
10515995018,26cyclictest0-21swapper/321:25:213
1051599493,32cyclictest7999-21chrt00:23:193
1051599493,31cyclictest12261-21chrt00:30:283
1051599492,33cyclictest1883-21chrt23:19:123
1051599492,33cyclictest1883-21chrt23:19:123
1051599492,33cyclictest12695-21chrt23:36:063
1051599486,20cyclictest0-21swapper/321:10:243
1051599484,30cyclictest9040-21taskset23:30:353
1051599483,36cyclictest12470-21taskset19:10:153
1051599483,36cyclictest12470-21taskset19:10:153
1051599482,34cyclictest18111-21chrt19:25:303
1051599482,33cyclictest9875-21chrt20:40:223
1051599473,34cyclictest23213-21chrt22:06:013
1051599473,33cyclictest29159-21chrt20:00:093
1051599473,33cyclictest27282-21chrt21:20:213
1051599473,33cyclictest27282-21chrt21:20:213
1051599473,31cyclictest30133-21chrt23:10:453
1051599472,29cyclictest13170-21taskset22:43:383
1051599463,32cyclictest15652-21chrt23:41:043
1051599453,28cyclictest5409-21chrt00:19:223
1051599443,29cyclictest11445-21taskset20:45:213
1051599433,29cyclictest25880-21taskset19:50:213
1051599433,29cyclictest25880-21taskset19:50:213
1051599423,30cyclictest19958-21taskset19:30:363
1051599423,29cyclictest13734-21taskset20:50:393
1051599423,29cyclictest13734-21taskset20:50:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional