You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 15:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon Jun 29, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11089991183,42cyclictest24419-21ssh11:49:060
10194211411,69sleep110261-21awk06:53:491
11089991054,29cyclictest29313-21nfs4_client10:08:330
111119910446,50cyclictest0-21swapper/211:28:262
111119910446,50cyclictest0-21swapper/211:28:262
110899910446,51cyclictest0-21swapper/008:03:060
110899910446,49cyclictest0-21swapper/009:33:210
11089991043,30cyclictest20149-21latency_hist10:48:070
110899910351,45cyclictest0-21swapper/010:13:410
110899910350,45cyclictest0-21swapper/010:08:050
110899910350,45cyclictest0-21swapper/010:08:050
110899910342,53cyclictest23417-21apt-key07:33:080
110899910342,53cyclictest23417-21apt-key07:33:080
11089991033,41cyclictest20012-21fschecks_count09:53:270
11089991033,41cyclictest20012-21fschecks_count09:53:270
11089991032,26cyclictest5957-21cpuspeed11:18:210
11089991032,24cyclictest6644-21taskset12:13:150
11089991032,24cyclictest6644-21taskset12:13:150
110899910249,46cyclictest18602-21diskmemload08:58:050
110899910249,46cyclictest18602-21diskmemload08:58:050
11089991023,7cyclictest0-21swapper/009:23:180
11089991023,7cyclictest0-21swapper/009:23:180
11089991023,33cyclictest30455-21taskset11:03:450
111119910152,40cyclictest29323-21acpi11:03:152
110899910135,60cyclictest0-21swapper/012:03:260
110899910135,60cyclictest0-21swapper/012:03:260
11089991013,30cyclictest2762-21grep10:18:300
11089991013,30cyclictest2762-21grep10:18:300
11089991013,30cyclictest18957-21cpu08:58:210
11089991013,27cyclictest11098-21latency_hist06:58:080
11089991013,22cyclictest20413-21ls10:48:190
11101999910,76cyclictest13948-21sort09:43:391
11101999910,76cyclictest13948-21sort09:43:391
11089999942,51cyclictest10925-21sh08:33:230
1108999993,31cyclictest26431-21cpuspeed10:58:210
1108999992,27cyclictest29039-21taskset11:55:580
11089999843,49cyclictest4081-21ssh11:13:560
11089999843,48cyclictest0-21swapper/011:38:260
11089999836,56cyclictest0-21swapper/007:03:060
11089999826,65cyclictest0-21swapper/012:08:350
11089999740,50cyclictest18747-21fschecks_time07:18:280
11089999740,50cyclictest18747-21fschecks_time07:18:280
11089999736,51cyclictest0-21swapper/007:08:220
1108999972,27cyclictest12963-21seq10:35:170
1108999972,27cyclictest12963-21seq10:35:170
1108999972,24cyclictest23826-21latency10:53:300
11089999642,47cyclictest0-21swapper/008:03:370
11089999642,47cyclictest0-21swapper/008:03:370
11089999631,58cyclictest0-21swapper/010:43:040
11089999629,61cyclictest0-21swapper/011:43:290
11111999573,17cyclictest10622-21cstates09:38:212
11111999573,17cyclictest10622-21cstates09:38:212
1108999954,82cyclictest12606-21/usr/sbin/munin12:23:380
11089999540,49cyclictest26049-21open_inodes07:39:010
11089999540,49cyclictest26049-21open_inodes07:39:010
1108999953,42cyclictest31245-21fschecks_time09:18:280
1108999953,32cyclictest7635-21cstates08:23:220
1108999953,31cyclictest32472-21ls11:08:200
1108999953,31cyclictest29201-21ssh09:14:070
1108999953,31cyclictest29201-21ssh09:14:070
1108999953,31cyclictest23739-21/usr/sbin/munin09:58:400
1108999953,30cyclictest7949-21modprobe10:28:030
1108999952,34cyclictest12544-21rm09:41:190
1108999952,34cyclictest12544-21rm09:41:190
1108999952,30cyclictest14627-21grep09:43:580
1108999952,30cyclictest14627-21grep09:43:580
11089999515,25cyclictest4728-21fschecks_count09:28:230
1111199947,40cyclictest24949-21ssh09:08:152
1111199944,39cyclictest352-21runrttasks08:38:322
11111999431,54cyclictest0-21swapper/209:08:072
1108999944,30cyclictest25468-21needreboot09:08:320
11089999433,56cyclictest0-21swapper/011:28:060
1108999943,32cyclictest30293-21/usr/sbin/munin07:53:290
1108999943,25cyclictest9056-21/usr/sbin/munin10:28:390
11111999339,47cyclictest6536-21sendmail_mailqu08:18:352
1111199933,35cyclictest15564-21fschecks_time07:08:272
1108999933,35cyclictest13994-21fschecks_time07:04:110
1108999933,34cyclictest30756-21/usr/sbin/munin11:58:350
1108999933,34cyclictest30756-21/usr/sbin/munin11:58:350
1108999933,31cyclictest15851-21fschecks_time08:48:240
1108999933,31cyclictest13717-21modprobe08:43:040
1108999933,29cyclictest5903-21/usr/sbin/munin08:18:360
1108999932,35cyclictest10198-21awk12:18:300
1108999932,27cyclictest22272-21fschecks_time09:03:300
11111999237,48cyclictest0-21swapper/210:43:392
1111199923,39cyclictest798-21latency_hist08:03:062
1108999924,33cyclictest23581-21/usr/sbin/munin07:34:230
1108999923,34cyclictest27235-21fschecks_count07:43:260
1108999923,33cyclictest12127-21fschecks_count11:28:250
1108999923,33cyclictest12127-21fschecks_count11:28:250
1108999923,30cyclictest15604-21awk11:33:380
1108999923,29cyclictest9354-21tune2fs08:28:230
1108999923,28cyclictest2902-21fschecks_count08:08:250
1108999923,27cyclictest4299-21/usr/sbin/munin08:13:370
1108999923,24cyclictest17405-21open_inodes09:48:370
1111199914,35cyclictest23232-21latency_hist07:33:062
1111199914,35cyclictest23232-21latency_hist07:33:062
1111199914,35cyclictest17054-21latency_hist10:43:062
1111199914,34cyclictest3633-21/usr/sbin/munin12:08:382
11111999138,46cyclictest0-21swapper/207:03:142
1111199913,38cyclictest25096-21memory11:49:472
1111199913,38cyclictest12122-21latency_hist08:38:072
11111999132,10cyclictest0-21swapper/209:18:432
1110199913,56cyclictest23542-21apt-get11:48:191
1108999913,24cyclictest13977-21/usr/sbin/munin08:43:340
1108999913,24cyclictest13977-21/usr/sbin/munin08:43:340
1108999912,27cyclictest20361-21fschecks_time07:23:270
97812909,51sleep29824-21if_eth006:53:332
1111199904,39cyclictest25551-21latency_hist10:03:062
1111199904,37cyclictest3358-21cut08:08:362
1111199904,37cyclictest27046-21processes10:58:382
1111199903,26cyclictest25552-21cpuspeed07:38:442
1111199903,26cyclictest25552-21cpuspeed07:38:442
1111199902,38cyclictest352-21runrttasks08:48:212
1111199902,27cyclictest14416-21ssh12:25:372
1108999903,31cyclictest16890-21/usr/sbin/munin07:13:310
1108999903,30cyclictest28676-21/usr/sbin/munin07:48:220
1111199894,35cyclictest12313-21idleruntime-cro12:23:032
1111199894,34cyclictest21944-21grep11:43:412
11111998938,42cyclictest0-21swapper/210:28:312
11111998934,48cyclictest11251-21sh10:33:112
11111998934,48cyclictest11251-21sh10:33:112
1111199893,37cyclictest28013-21grep07:43:412
1111199893,35cyclictest2299-21/usr/sbin/munin09:23:382
1111199893,35cyclictest2299-21/usr/sbin/munin09:23:382
11111998868,11cyclictest0-21swapper/210:08:352
1111199886,36cyclictest352-21runrttasks10:13:242
11111998835,46cyclictest0-21swapper/208:03:282
11111998835,46cyclictest0-21swapper/208:03:282
1111199883,39cyclictest7266-21ssh12:13:352
1111199883,39cyclictest7266-21ssh12:13:352
11111998826,10cyclictest0-21swapper/207:53:352
11101998872,10cyclictest18691-21switchtime11:38:381
1111199874,38cyclictest8118-21grep10:28:062
1111199874,36cyclictest17025-21fschecks_time09:48:272
1111199874,35cyclictest352-21runrttasks11:58:212
1111199874,35cyclictest352-21runrttasks11:58:212
1111199874,32cyclictest305-21kernelversion11:08:292
11111998733,8cyclictest0-21swapper/211:23:522
1111199873,33cyclictest18902-21/usr/sbin/munin08:58:372
1111199873,32cyclictest14567-21/usr/sbin/munin09:43:572
1111199873,32cyclictest14567-21/usr/sbin/munin09:43:572
1111199873,25cyclictest24578-21unixbench_multi07:34:312
1111199872,31cyclictest28164-21grep11:53:412
1111199872,30cyclictest28681-21acpi07:48:142
1111199864,37cyclictest8425-21timerandwakeup09:33:412
1111199864,37cyclictest352-21runrttasks09:30:002
1111199864,32cyclictest25717-21/usr/sbin/munin10:03:152
1111199864,32cyclictest25717-21/usr/sbin/munin10:03:152
1111199864,26cyclictest11367-21/usr/sbin/munin06:58:162
1111199863,37cyclictest5954-21cpuspeed11:18:212
1111199863,35cyclictest9854-21switchtime08:28:352
1111199863,35cyclictest18183-21grep08:53:422
1111199863,35cyclictest18183-21grep08:53:422
1111199863,35cyclictest18177-21grep11:38:262
1111199863,27cyclictest14857-21/usr/sbin/munin11:33:282
11111998630,7cyclictest0-21swapper/212:03:312
11111998630,7cyclictest0-21swapper/212:03:312
11111998625,8cyclictest0-21swapper/207:13:372
1111199862,36cyclictest20339-21fschecks_count07:23:272
1111199855,31cyclictest27917-21/usr/sbin/munin09:13:392
1111199855,31cyclictest27917-21/usr/sbin/munin09:13:392
1111199854,31cyclictest20509-21fschecks_count10:48:232
1111199853,33cyclictest20368-21ssh09:53:352
1111199853,33cyclictest20368-21ssh09:53:352
11111998528,7cyclictest0-21swapper/208:43:222
11111998528,7cyclictest0-21swapper/208:43:222
11101998556,20cyclictest23218-21apt-key07:33:071
11101998556,20cyclictest23218-21apt-key07:33:071
1110199854,37cyclictest12656-21fschecks_time08:38:251
1111199843,34cyclictest352-21runrttasks07:18:412
1111199843,34cyclictest352-21runrttasks07:18:412
1111199843,27cyclictest4299-21/usr/sbin/munin08:13:282
1111199834,36cyclictest352-21runrttasks10:57:412
1111199833,33cyclictest3191-21fschecks_time11:13:242
1111199832,37cyclictest7517-21/usr/sbin/munin08:23:162
1111199832,31cyclictest352-21runrttasks10:18:312
1111199832,31cyclictest352-21runrttasks10:18:312
1110199833,55cyclictest13661-21apt-get07:03:191
11101998257,19cyclictest19676-21snmpd07:33:191
11101998219,26cyclictest27217-21seq07:43:241
11101998210,28cyclictest29021-21fschecks_count10:08:271
11101998158,15cyclictest11073-21cyclictest07:13:391
11101998153,21cyclictest23023-21kworker/u19:3+events_unbound12:23:031
11101998017,25cyclictest15593-21seq07:08:281
1032428011,40sleep30-21swapper/306:53:513
11101997953,17cyclictest2374-21systemd-journal10:32:431
11101997857,14cyclictest13074-21latency_hist07:03:071
11101997856,16cyclictest19243-21fschecks_time08:58:281
1112999773,31cyclictest28894-21chrt07:48:253
11101997754,16cyclictest2731-21latency10:18:291
11101997754,16cyclictest2731-21latency10:18:291
1110199773,65cyclictest24949-21apt-get07:38:181
1110199773,65cyclictest24949-21apt-get07:38:181
11101997718,20cyclictest27348-21cut11:53:261
11101997655,15cyclictest3658-21unixbench_singl08:08:401
11101997648,20cyclictest21084-21fschecks_count11:43:251
11101997559,10cyclictest18602-21diskmemload09:48:361
11101997555,12cyclictest18602-21diskmemload11:28:131
11101997555,12cyclictest18602-21diskmemload11:28:131
1112999743,30cyclictest19507-21chrt07:18:443
1112999743,30cyclictest19507-21chrt07:18:443
11101997451,15cyclictest18602-21diskmemload09:03:461
11101997413,26cyclictest24399-21unixbench_singl10:53:411
1112999733,24cyclictest23628-21taskset07:33:303
11101997313,24cyclictest26642-21tune2fs10:58:271
1112999723,25cyclictest15313-21chrt12:28:033
1112999712,35cyclictest22115-21taskset10:50:323
1112999712,28cyclictest19400-21chrt09:53:013
11101997156,10cyclictest11073-21cyclictest08:53:061
11101997128,31cyclictest28696-21apt-get07:48:151
1110199712,47cyclictest31776-21latency_hist07:58:071
11101997043,19cyclictest17257-21cpu08:53:211
11101997043,19cyclictest17257-21cpu08:53:211
1110199703,47cyclictest12325-21tail10:33:411
1110199703,47cyclictest12325-21tail10:33:411
1110199702,51cyclictest2374-21systemd-journal10:39:091
1112999693,28cyclictest15654-21taskset09:46:003
1112999693,28cyclictest15654-21taskset09:46:003
1112999693,24cyclictest21944-21chrt09:03:193
1112999693,22cyclictest17301-21chrt10:43:193
11101996942,18cyclictest26125-21sh09:08:551
1110199693,47cyclictest352-21runrttasks10:00:111
1112999683,25cyclictest13764-21taskset08:43:053
1112999683,22cyclictest5417-21chrt11:17:433
1112999682,26cyclictest16916-21chrt07:13:173
11101996846,16cyclictest7163-21ssh09:33:031
11101996832,12cyclictest201492latency_hist10:48:071
1112999673,26cyclictest9384-21chrt08:28:233
1112999673,24cyclictest2798-21chrt10:18:303
1112999673,24cyclictest2798-21chrt10:18:303
1112999673,24cyclictest20130-21chrt07:23:173
1112999673,23cyclictest10779-21chrt08:33:173
1112999673,22cyclictest12691-21chrt11:28:383
1112999673,22cyclictest12691-21chrt11:28:383
1112999672,26cyclictest32607-21chrt12:02:383
1112999672,26cyclictest32607-21chrt12:02:383
1112999672,25cyclictest6961-21chrt10:25:193
1112999672,25cyclictest1618-21chrt11:10:263
1112999672,22cyclictest13950-21chrt10:38:043
1112999672,22cyclictest13950-21chrt10:38:043
1110199677,24cyclictest0-21swapper/111:18:221
1110199675,23cyclictest0-21swapper/111:23:401
11101996750,11cyclictest7468-21/usr/sbin/munin09:33:411
11101996747,13cyclictest1-21systemd09:58:031
11101996747,13cyclictest1-21systemd09:58:031
11101996744,13cyclictest0-21swapper/112:13:251
11101996744,13cyclictest0-21swapper/112:13:251
11101996732,26cyclictest5272-21apt-get10:23:171
1112999664,23cyclictest22753-21chrt09:58:163
1112999663,26cyclictest22697-21chrt11:45:443
1112999663,24cyclictest26348-21taskset10:03:353
1112999663,24cyclictest26348-21taskset10:03:353
1112999663,23cyclictest29169-21chrt09:14:023
1112999663,23cyclictest29169-21chrt09:14:023
1112999663,22cyclictest16192-21chrt11:34:173
1112999662,23cyclictest30782-21taskset07:53:313
1112999662,23cyclictest28463-21chrt07:48:053
1112999662,23cyclictest28401-21chrt11:01:223
1112999662,22cyclictest21768-21chrt07:28:203
1112999662,22cyclictest21768-21chrt07:28:203
1112999662,22cyclictest1771-21chrt12:03:473
1112999662,22cyclictest1771-21chrt12:03:473
1112999662,22cyclictest1076-21chrt08:03:203
1112999662,22cyclictest1076-21chrt08:03:203
1112999662,21cyclictest23729-21chrt10:53:283
1110199667,43cyclictest0-21swapper/110:18:061
1110199665,31cyclictest9337-21fschecks_count08:28:221
11101996649,11cyclictest13699-21sshd12:23:421
11101996641,13cyclictest0-21swapper/108:23:411
1110199663,46cyclictest31881-21timerwakeupswit09:18:421
1112999654,22cyclictest16424-21chrt07:10:213
1112999653,26cyclictest10827-21taskset11:26:063
1112999653,24cyclictest9063-21chrt12:17:253
1112999653,24cyclictest9063-21chrt12:17:253
1112999653,24cyclictest12534-21chrt09:41:173
1112999653,24cyclictest12534-21chrt09:41:173
1112999653,21cyclictest8761-21chrt09:34:083
1112999652,27cyclictest8642-21chrt11:23:063
1112999652,25cyclictest19613-21chrt11:40:293
1112999652,24cyclictest585-21chrt10:14:133
1112999652,23cyclictest31555-21chrt09:18:363
1112999652,23cyclictest29536-21chrt11:03:253
1112999652,22cyclictest15184-21chrt10:38:413
1110199655,27cyclictest0-21swapper/112:03:211
1110199655,27cyclictest0-21swapper/112:03:211
11101996541,16cyclictest30433-21cut11:58:251
11101996541,16cyclictest30433-21cut11:58:251
1110199653,43cyclictest12300-21latency_hist08:38:071
1110199652,39cyclictest5786-21latency_hist08:18:061
1112999649,12cyclictest0-21swapper/308:53:423
1112999649,12cyclictest0-21swapper/308:53:423
1112999643,25cyclictest27111-21taskset11:53:143
1112999642,25cyclictest5295-21taskset08:13:413
1112999642,23cyclictest27587-21chrt09:13:003
1112999642,22cyclictest14154-21chrt08:43:263
1112999642,22cyclictest14154-21chrt08:43:263
1112999642,22cyclictest11711-21chrt12:21:263
11101996437,17cyclictest0-21swapper/110:03:401
11101996437,17cyclictest0-21swapper/110:03:401
1110199643,46cyclictest20503-21ssh10:48:231
1110199643,42cyclictest2558-21cron11:13:041
1112999633,25cyclictest20556-21taskset09:00:113
1112999633,23cyclictest438-21chrt07:58:423
1112999633,23cyclictest26488-21taskset11:52:123
1112999632,25cyclictest13886-21taskset07:04:103
1112999632,24cyclictest5756-21taskset12:11:293
1112999632,24cyclictest16164-21chrt08:48:323
1112999632,24cyclictest11593-21taskset06:59:033
1112999632,23cyclictest21220-21chrt09:54:433
1112999632,23cyclictest21220-21chrt09:54:433
1112999632,22cyclictest7133-21chrt08:21:303
1112999632,22cyclictest6896-21chrt09:32:173
11101996347,10cyclictest8734-21kworker/u17:2-xprtiod09:13:231
11101996347,10cyclictest8734-21kworker/u17:2-xprtiod09:13:231
1112999628,14cyclictest0-21swapper/309:23:383
1112999628,14cyclictest0-21swapper/309:23:383
1112999624,20cyclictest8602-21chrt08:24:423
1112999623,21cyclictest30473-21chrt10:10:243
1112999622,22cyclictest25461-21chrt07:38:203
1112999622,22cyclictest25461-21chrt07:38:203
1112999622,22cyclictest10285-21chrt10:31:113
11101996246,11cyclictest11073-21cyclictest07:58:371
11101996244,14cyclictest11073-21cyclictest11:33:261
1110199623,40cyclictest4258-21latency_hist09:28:061
1110199623,40cyclictest4258-21latency_hist09:28:061
1110199623,34cyclictest3630-21mailstats11:13:361
1110199623,21cyclictest13615-21apt-key09:43:091
1110199623,21cyclictest13615-21apt-key09:43:091
11101996224,33cyclictest0-21swapper/112:08:191
1110199622,38cyclictest14204-21fschecks_count08:43:271
1110199622,38cyclictest14204-21fschecks_count08:43:271
1112999613,20cyclictest3999-21chrt08:12:353
11101996146,10cyclictest29983-21open_inodes11:03:371
1110199613,42cyclictest19925-21latency_hist07:23:071
1110199613,42cyclictest19925-21latency_hist07:23:071
11101996119,29cyclictest7075-21sshd08:20:441
1110199583,37cyclictest21654-21latency_hist07:28:071
11101995640,9cyclictest0-21swapper/108:03:151
11101995640,9cyclictest0-21swapper/108:03:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional