You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 08:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Apr 07, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
163949910251,43cyclictest0-21swapper/219:14:032
16394991023,73cyclictest3460-21apt-get22:18:572
16394991023,73cyclictest3460-21apt-get22:18:572
163769910084,10cyclictest21841-21ssh21:54:250
1639499973,80cyclictest21642-21apt-get21:54:022
16394999727,42cyclictest17606-21sh22:40:582
16394999642,47cyclictest11138-21open_files20:24:122
16386999637,7cyclictest0-21swapper/121:23:591
16386999637,7cyclictest0-21swapper/121:23:591
1578629610,57sleep015870-21sed18:59:300
1638699954,37cyclictest16865-21latency_hist20:43:441
16386999540,48cyclictest22814-21sh22:49:181
16386999442,45cyclictest0-21swapper/100:23:451
16386999442,45cyclictest0-21swapper/100:23:451
16386999424,23cyclictest17229-21fschecks_count20:44:021
1638699942,36cyclictest27594-21fschecks_time22:04:051
16394999348,38cyclictest23247-21diskmemload23:44:492
1638699932,44cyclictest4006-21idleruntime-cro23:13:421
16386999229,12cyclictest0-21swapper/100:13:451
1638699922,38cyclictest4541-21uniq23:14:011
1639499914,45cyclictest14592-21ssh23:30:022
1638699913,40cyclictest17389-21timerwakeupswit19:04:171
1638699913,34cyclictest25376-21sort23:49:031
1638699913,34cyclictest25376-21sort23:49:031
16386999131,12cyclictest0-21swapper/120:24:021
16386999130,52cyclictest0-21swapper/122:18:451
16386999130,52cyclictest0-21swapper/122:18:451
16386999125,9cyclictest0-21swapper/123:59:041
16386999125,9cyclictest0-21swapper/123:59:041
16376999161,9cyclictest181rcu_preempt19:29:170
16394999030,52cyclictest0-21swapper/220:14:162
1638699904,37cyclictest2672-21grep00:04:191
16386999039,44cyclictest0-21swapper/123:38:541
1638699903,35cyclictest19553-21unixbench_multi20:49:171
1638699903,34cyclictest23901-21seq21:04:051
1638699903,34cyclictest22726-21timerwakeupswit20:59:181
1638699903,34cyclictest22499-21grep19:19:151
1638699903,34cyclictest12616-21cut21:39:021
1638699903,33cyclictest698-21cpuspeed21:19:371
1638699903,33cyclictest23930-21seq19:24:101
1638699903,33cyclictest23930-21seq19:24:101
1638699903,33cyclictest19842-21timerwakeupswit22:44:191
1638699903,33cyclictest19842-21timerwakeupswit22:44:191
1638699903,33cyclictest13789-21/usr/sbin/munin23:29:101
1638699903,32cyclictest15368-21latency_hist20:38:451
1638699903,32cyclictest13073-21latency_hist23:28:441
16386999027,9cyclictest0-21swapper/122:29:161
16386999027,9cyclictest0-21swapper/122:29:161
1638699902,36cyclictest26931-21grep19:34:031
1638699902,35cyclictest25529-21ps22:54:141
1638699894,37cyclictest26576-21latency_hist19:33:451
1638699894,33cyclictest24910-21latency_hist23:48:431
16386998937,45cyclictest0-21swapper/122:18:541
16386998937,45cyclictest0-21swapper/122:18:541
1638699893,37cyclictest9822-21cut22:28:441
1638699893,35cyclictest16360-21latency22:39:071
1638699893,33cyclictest12773-21grep20:29:131
1638699893,33cyclictest12773-21grep20:29:131
1638699893,32cyclictest31430-21awk23:04:121
16386998933,13cyclictest0-21swapper/119:09:121
1638699893,30cyclictest31203-21sh21:15:441
16386998926,12cyclictest0-21swapper/121:09:151
1638699892,32cyclictest7600-21/usr/sbin/munin00:14:051
1638699883,35cyclictest30126-21fschecks_time19:44:041
1638699883,33cyclictest28774-21timerwakeupswit22:59:181
1638699883,31cyclictest30811-21modprobe23:58:411
1638699883,31cyclictest15946-21ntpq21:44:101
16386998827,11cyclictest0-21swapper/120:04:091
16386998825,8cyclictest0-21swapper/122:37:461
1638699882,39cyclictest32379-21grep19:49:171
1638699882,37cyclictest6087-21kernelversion20:09:061
16386998811,27cyclictest436-21cron22:13:411
1638699874,32cyclictest8323-21tail23:19:191
1638699873,37cyclictest7595-21fschecks_count20:14:041
1638699873,34cyclictest24866-21netstat21:59:121
1638699873,33cyclictest9774-21wc20:19:161
1638699873,33cyclictest352-21runrttasks23:36:391
1638699872,35cyclictest2712-21fschecks_count19:59:031
1638699872,35cyclictest2712-21fschecks_count19:59:031
1638699872,31cyclictest29778-21latency_hist19:43:451
1638699872,31cyclictest21267-21apt-config21:53:451
1638699872,31cyclictest21267-21apt-config21:53:451
1638699872,30cyclictest12167-21cat21:38:441
1638699872,30cyclictest12167-21cat21:38:441
16386998710,56cyclictest13883-21runrttasks00:24:091
16376998716,61cyclictest13827-21ssh00:23:590
160202878,53sleep116023-21sshd19:01:101
1638699863,34cyclictest22015-21ssh21:55:301
1638699863,34cyclictest20855-21mailstats19:14:141
1638699862,32cyclictest1252-21fschecks_time19:55:221
1638699862,32cyclictest1252-21fschecks_time19:55:221
1638699853,32cyclictest20540-21fschecks_time20:54:051
1638699853,32cyclictest17308-21ntpq00:29:081
16376998539,12cyclictest181rcu_preempt21:10:360
1637699853,71cyclictest4384-21cstates22:19:350
1637699853,71cyclictest4384-21cstates22:19:350
16376998533,12cyclictest181rcu_preempt22:25:520
16376998516,50cyclictest15-21ksoftirqd/022:44:110
16376998516,50cyclictest15-21ksoftirqd/022:44:110
16394998443,36cyclictest0-21swapper/200:23:422
16394998443,36cyclictest0-21swapper/200:23:422
16394998443,33cyclictest0-21swapper/221:29:022
16394998441,34cyclictest0-21swapper/200:28:452
1638699843,31cyclictest6666-21sort21:29:021
16376998464,15cyclictest19676-21snmpd22:49:140
16376998451,17cyclictest181rcu_preempt20:59:140
16376998444,14cyclictest181rcu_preempt23:38:010
16394998339,36cyclictest0-21swapper/221:18:452
16394998334,40cyclictest0-21swapper/221:59:102
16376998362,14cyclictest10675-21kernelversion23:24:070
16394998260,14cyclictest16364-21cyclictest22:18:462
16394998260,14cyclictest16364-21cyclictest22:18:462
1639499823,67cyclictest31117-21apt-get23:58:532
1639499823,67cyclictest31117-21apt-get23:58:532
1639499822,50cyclictest27266-21ssh23:52:152
1639499822,50cyclictest27266-21ssh23:52:152
1639499814,66cyclictest898-21/usr/sbin/munin19:53:532
1639499814,66cyclictest898-21/usr/sbin/munin19:53:532
1639499812,40cyclictest25866-21wc19:29:172
16376998169,8cyclictest0-21swapper/020:08:370
16394998036,36cyclictest0-21swapper/221:03:442
1639499802,73cyclictest0-21swapper/220:48:572
1639499802,63cyclictest0-21swapper/220:18:562
1639499794,61cyclictest0-21swapper/220:38:542
16394997937,35cyclictest0-21swapper/200:09:072
16394997934,38cyclictest0-21swapper/221:24:092
16394997934,38cyclictest0-21swapper/221:24:092
16394997927,45cyclictest0-21swapper/223:14:222
16376997955,8cyclictest181rcu_preempt20:19:010
16376997928,25cyclictest181rcu_preempt20:44:180
16394997828,45cyclictest0-21swapper/222:04:082
1639499782,69cyclictest0-21swapper/221:43:542
1639499782,62cyclictest0-21swapper/220:33:532
1639499782,62cyclictest0-21swapper/220:28:572
1639499782,62cyclictest0-21swapper/220:28:572
1637699785,52cyclictest30799-21latency_hist23:58:440
16376997847,14cyclictest298-21in:imuxsock22:31:150
16376997847,14cyclictest298-21in:imuxsock22:31:150
16376997825,14cyclictest181rcu_preempt21:59:210
1640199772,51cyclictest0-21swapper/322:18:593
1640199772,51cyclictest0-21swapper/322:18:593
1639499778,61cyclictest13817-21ssh00:23:572
16394997751,17cyclictest30739-21sh23:03:452
1639499773,47cyclictest21231-21sh22:47:512
1639499773,47cyclictest21231-21sh22:47:512
16394997732,36cyclictest0-21swapper/222:24:162
16376997758,13cyclictest21695-21latency_hist20:58:440
16376997754,8cyclictest181rcu_preempt23:58:590
16376997754,8cyclictest181rcu_preempt23:58:590
16376997747,14cyclictest181rcu_preempt22:54:240
16376997747,12cyclictest181rcu_preempt23:06:000
1637699772,48cyclictest13079-21/usr/sbin/munin20:29:170
1637699772,48cyclictest13079-21/usr/sbin/munin20:29:170
16376997721,14cyclictest181rcu_preempt23:39:050
16376997710,53cyclictest363-21apt-get21:19:000
16376997710,12cyclictest161ktimers/023:29:150
1639499763,59cyclictest18393-21apt-get21:48:542
1639499763,59cyclictest18393-21apt-get21:48:542
16394997635,34cyclictest0-21swapper/223:23:532
16394997629,41cyclictest0-21swapper/222:58:432
1639499762,66cyclictest0-21swapper/221:38:532
1639499762,62cyclictest0-21swapper/223:38:532
16376997650,12cyclictest181rcu_preempt19:51:120
16376997648,20cyclictest19296-21timerandwakeup21:49:190
16376997648,20cyclictest19296-21timerandwakeup21:49:190
16376997647,10cyclictest181rcu_preempt21:39:160
16376997645,9cyclictest181rcu_preempt23:47:420
16376997641,29cyclictest0-21swapper/023:09:190
16376997638,12cyclictest181rcu_preempt21:34:010
16376997638,12cyclictest181rcu_preempt21:34:010
1640199759,43cyclictest21901-21chrt21:54:373
16394997527,40cyclictest3295-21latency_hist21:23:452
1639499752,67cyclictest0-21swapper/222:28:542
1639499752,67cyclictest0-21swapper/222:28:542
1639499752,63cyclictest0-21swapper/220:03:522
16376997547,9cyclictest181rcu_preempt19:24:170
16376997547,9cyclictest181rcu_preempt19:24:170
16376997546,11cyclictest181rcu_preempt22:34:380
16376997544,16cyclictest181rcu_preempt22:12:530
1637699754,34cyclictest10807-21grep20:24:020
1637699754,32cyclictest29582-21/usr/sbin/munin21:14:110
16376997543,16cyclictest181rcu_preempt19:48:440
16376997542,24cyclictest0-21swapper/000:23:450
16376997542,24cyclictest0-21swapper/000:23:450
16376997542,10cyclictest181rcu_preempt00:14:130
16376997534,12cyclictest181rcu_preempt19:13:450
16376997533,33cyclictest5909-21grep20:09:020
16376997531,33cyclictest2345-21cat19:58:440
16376997531,33cyclictest2345-21cat19:58:440
16376997519,24cyclictest15-21ksoftirqd/021:24:180
16376997519,24cyclictest15-21ksoftirqd/021:24:180
1639499743,62cyclictest13005-21ssh22:33:572
1639499743,61cyclictest16957-21/usr/sbin/munin19:04:062
1639499743,56cyclictest1644-21apt-get00:03:532
16394997427,39cyclictest0-21swapper/219:28:422
16394997427,39cyclictest0-21swapper/219:28:422
16394997426,41cyclictest30825-21ntp_kernel_pll_22:09:092
1639499742,63cyclictest7332-21ssh23:18:562
1637699745,38cyclictest4318-21apt-get23:13:510
16376997453,15cyclictest17371-21sh22:40:180
16376997452,8cyclictest181rcu_preempt19:14:070
16376997451,9cyclictest181rcu_preempt22:17:120
16376997451,9cyclictest181rcu_preempt22:17:120
16376997450,19cyclictest24172-21sh21:04:110
16376997449,10cyclictest181rcu_preempt22:04:080
1637699743,59cyclictest4933-21fschecks_time00:09:030
1640199733,49cyclictest5728-21chrt20:08:523
1639499733,63cyclictest0-21swapper/219:58:552
1639499733,63cyclictest0-21swapper/219:58:552
16394997328,39cyclictest0-21swapper/223:54:022
16376997350,10cyclictest181rcu_preempt23:51:130
16376997350,10cyclictest181rcu_preempt23:51:130
16376997348,8cyclictest181rcu_preempt20:34:020
16376997345,8cyclictest181rcu_preempt23:23:300
16376997337,15cyclictest181rcu_preempt19:36:480
1640199722,56cyclictest4976-21chrt23:14:123
16394997245,17cyclictest30676-21latency_hist23:03:452
16394997244,20cyclictest16505-21fschecks_time23:34:032
1639499723,63cyclictest0-21swapper/220:53:582
1639499723,53cyclictest17646-21/usr/sbin/munin20:44:132
1639499722,63cyclictest0-21swapper/219:38:522
16376997249,17cyclictest23329-21latency_hist19:23:450
16376997242,13cyclictest181rcu_preempt21:44:180
16376997240,18cyclictest181rcu_preempt00:28:450
16376997234,16cyclictest181rcu_preempt20:39:430
1640199713,53cyclictest12866-21taskset23:28:353
1640199712,55cyclictest16956-21taskset21:45:353
1639499713,61cyclictest9396-21apt-get21:33:552
1639499713,61cyclictest9396-21apt-get21:33:552
1639499713,60cyclictest0-21swapper/219:43:532
1639499713,59cyclictest25512-21rm21:06:412
16394997132,32cyclictest0-21swapper/221:09:152
16394997131,33cyclictest0-21swapper/200:14:092
1639499712,62cyclictest0-21swapper/219:18:532
16376997148,10cyclictest181rcu_preempt21:31:110
16376997142,11cyclictest181rcu_preempt00:07:250
16376997139,13cyclictest181rcu_preempt19:59:230
16376997139,13cyclictest181rcu_preempt19:59:230
1640199703,52cyclictest31334-21chrt22:09:183
1639499704,56cyclictest18139-21munin-node19:08:492
16394997026,36cyclictest0-21swapper/222:51:122
1639499702,61cyclictest388-21ls19:53:422
16376997043,10cyclictest181rcu_preempt19:39:040
1637699702,40cyclictest15-21ksoftirqd/023:00:440
16394996932,31cyclictest1410-21ssh23:08:592
1639499692,62cyclictest0-21swapper/220:08:572
1640199682,52cyclictest8699-21taskset20:18:403
1640199682,52cyclictest16109-21chrt22:39:013
1640199682,51cyclictest368-21chrt00:01:023
1640199682,51cyclictest368-21chrt00:01:023
1640199682,50cyclictest3018-21taskset23:11:003
148942689,35sleep214897-21apt-get18:58:542
1640199672,52cyclictest600-21chrt21:19:323
1640199672,50cyclictest9299-21taskset22:27:463
1639499672,59cyclictest0-21swapper/219:33:542
1640199664,49cyclictest29421-21chrt23:00:423
1640199663,50cyclictest3170-21chrt22:17:533
1640199663,50cyclictest3170-21chrt22:17:533
1640199662,51cyclictest813-21chrt23:08:123
1640199662,50cyclictest20720-21taskset22:46:233
1640199662,50cyclictest20720-21taskset22:46:233
16376996652,9cyclictest31392-21kworker/u18:3-events_unbound19:04:080
1640199653,50cyclictest24037-21chrt23:46:313
1640199653,49cyclictest10789-21chrt21:35:093
1640199653,49cyclictest10789-21chrt21:35:093
1640199653,47cyclictest30211-21chrt21:14:133
1640199653,45cyclictest13832-21taskset20:33:543
1640199652,50cyclictest3679-21taskset20:01:273
1640199652,50cyclictest3679-21taskset20:01:273
1640199652,49cyclictest18407-21chrt21:48:553
1640199652,49cyclictest18407-21chrt21:48:553
16376996534,13cyclictest181rcu_preempt20:49:020
1640199643,48cyclictest2838-21chrt00:04:353
1640199643,48cyclictest28283-21chrt19:38:553
1640199643,48cyclictest25120-21taskset19:28:593
1640199643,47cyclictest4810-21chrt00:09:013
1640199643,46cyclictest27686-21chrt22:04:073
1640199643,46cyclictest21378-21taskset20:55:373
1640199642,49cyclictest27930-21chrt23:53:453
1640199642,49cyclictest27930-21chrt23:53:453
1640199642,48cyclictest8993-21chrt00:15:053
1640199642,48cyclictest32238-21chrt19:49:163
1640199642,48cyclictest19085-21taskset00:32:493
1640199642,47cyclictest10228-21chrt20:22:413
1637699643,39cyclictest7606-21fschecks_time20:14:050
1640199634,47cyclictest18228-21chrt20:46:503
1640199633,46cyclictest4035-21chrt21:24:143
1640199633,46cyclictest4035-21chrt21:24:143
1640199633,46cyclictest25497-21chrt22:54:143
1640199633,46cyclictest21521-21chrt19:18:213
1640199633,46cyclictest13481-21chrt21:39:183
1640199633,45cyclictest22887-21taskset20:59:223
1640199632,49cyclictest29033-21taskset21:13:073
1640199632,48cyclictest28141-21taskset23:53:533
1640199632,47cyclictest17597-21chrt23:35:043
1640199632,46cyclictest25538-21taskset21:59:413
1640199623,46cyclictest21677-21chrt23:43:153
1640199623,46cyclictest10750-21chrt22:29:173
1640199623,46cyclictest10750-21chrt22:29:173
1640199623,45cyclictest7601-21taskset21:29:413
1640199622,47cyclictest11930-21chrt00:19:593
1640199622,47cyclictest11930-21chrt00:19:593
1640199622,45cyclictest5125-21taskset20:04:223
1640199613,46cyclictest22029-21chrt22:48:593
1640199613,46cyclictest15519-21chrt23:32:413
1640199613,45cyclictest25161-21chrt21:05:423
1640199613,45cyclictest19955-21chrt20:53:263
1640199613,44cyclictest1876-21taskset19:55:353
1640199613,44cyclictest1876-21taskset19:55:353
1640199612,46cyclictest9717-21chrt23:23:043
1640199612,46cyclictest11030-21chrt20:24:063
1640199612,45cyclictest30924-21taskset19:44:583
1640199612,45cyclictest27853-21chrt19:36:483
1640199612,45cyclictest14242-21chrt22:34:523
1640199604,43cyclictest22858-21chrt19:19:513
1640199603,45cyclictest13792-21chrt00:23:533
1640199603,45cyclictest13478-21chrt20:33:103
1640199603,45cyclictest13478-21chrt20:33:103
1640199602,45cyclictest19786-21chrt19:11:583
1640199602,45cyclictest16558-21chrt19:03:563
1640199592,46cyclictest16545-21chrt20:40:513
1640199592,44cyclictest24444-21chrt19:24:373
1640199592,44cyclictest24444-21chrt19:24:373
156902479,23sleep30-21swapper/318:59:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional