You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 04:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Fri Jan 09, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
84639912598,20cyclictest13814-21latency_hist19:24:351
2778021190,3sleep30-21swapper/321:29:313
2778021190,3sleep30-21swapper/321:29:313
84639911288,16cyclictest8449-21cyclictest20:09:551
84669910658,40cyclictest0-21swapper/200:24:572
84669910429,10cyclictest0-21swapper/222:22:492
2367621030,6sleep20-21swapper/221:21:392
7627210211,62sleep17672-21sensors_temp19:05:131
8463999876,16cyclictest8449-21cyclictest22:14:541
8463999876,16cyclictest8449-21cyclictest22:14:541
780929712,53sleep07892-21grep19:05:160
8466999455,32cyclictest15574-21diskmemload22:49:472
8466999443,44cyclictest297-21systemd-logind22:59:312
846699942,39cyclictest3573-21ssh22:34:432
8463999447,40cyclictest0-21swapper/122:45:091
846699933,39cyclictest16378-21sort21:09:562
846699933,39cyclictest16378-21sort21:09:562
8463999353,9cyclictest181rcu_preempt19:49:491
846399932,27cyclictest15526-21runrttasks19:29:421
846399932,27cyclictest15526-21runrttasks19:29:421
8466999246,41cyclictest8449-21cyclictest23:32:222
846699923,36cyclictest21276-21grep22:10:162
846699923,36cyclictest21276-21grep22:10:162
8466999230,53cyclictest0-21swapper/223:14:342
846399924,37cyclictest352-21runrttasks19:34:421
846399923,32cyclictest7760-21taskset20:44:451
846699914,34cyclictest1597-21idleruntime20:24:532
846699914,34cyclictest1597-21idleruntime20:24:532
8466999128,13cyclictest0-21swapper/221:44:582
846699912,36cyclictest19192-21fschecks_count21:14:532
8466999119,23cyclictest19113-21fschecks_count22:59:522
8466999119,23cyclictest19113-21fschecks_count22:59:522
8463999151,32cyclictest0-21swapper/122:39:581
8463999146,39cyclictest0-21swapper/121:55:081
8460999161,14cyclictest7537-21latency_hist23:34:330
8466999051,31cyclictest0-21swapper/222:44:532
8466999047,38cyclictest8449-21cyclictest22:19:292
8466999047,38cyclictest8449-21cyclictest22:19:292
846699903,39cyclictest11892-21cat00:34:342
846699903,39cyclictest11892-21cat00:34:342
846699903,37cyclictest27894-21ssh21:29:342
846699903,37cyclictest27894-21ssh21:29:342
846699903,36cyclictest15546-21cron21:09:312
8466999032,51cyclictest0-21swapper/222:00:072
8466999028,11cyclictest0-21swapper/223:34:582
8466999027,8cyclictest0-21swapper/221:56:222
846699902,41cyclictest19724-21latency_hist23:54:342
846699902,41cyclictest18539-21latency_hist19:39:332
8463999067,12cyclictest0-21swapper/120:09:341
8463999059,25cyclictest30880-21kworker/u17:2+events_unbound22:29:301
846399903,33cyclictest1551-21latency_hist23:24:341
8463999023,13cyclictest0-21swapper/120:29:331
8463999023,13cyclictest0-21swapper/120:29:331
8460999068,16cyclictest25909-21latency_hist00:04:330
846699894,29cyclictest352-21runrttasks20:24:082
846699894,29cyclictest352-21runrttasks20:24:082
8466998936,45cyclictest0-21swapper/219:44:422
846699892,37cyclictest15387-21latency_hist19:29:342
846699892,37cyclictest15387-21latency_hist19:29:342
846699892,34cyclictest20305-21cat22:09:342
8463998936,46cyclictest0-21swapper/120:40:071
846399893,34cyclictest3426-21latency_hist22:34:341
846399892,27cyclictest19988-21apt-get22:09:451
846399892,27cyclictest19988-21apt-get22:09:451
8466998849,32cyclictest0-21swapper/220:30:062
8466998844,34cyclictest30084-21kworker/u19:0+nfsiod00:16:052
8466998843,38cyclictest0-21swapper/221:54:322
8466998843,38cyclictest0-21swapper/221:54:322
8466998842,39cyclictest30126-21apt-key20:14:332
8466998834,48cyclictest0-21swapper/220:49:522
846699883,37cyclictest11363-21ntp_kernel_pll_23:40:002
846699883,34cyclictest31191-21find20:15:542
8466998833,47cyclictest0-21swapper/200:04:302
8466998831,8cyclictest0-21swapper/219:30:052
8466998831,8cyclictest0-21swapper/219:30:052
846699882,37cyclictest28437-21fschecks_time21:29:542
846699882,33cyclictest32007-21cmp00:14:332
846399884,33cyclictest352-21runrttasks21:05:071
846399883,37cyclictest5517-21uname21:45:051
846399883,36cyclictest31537-21head20:15:591
846399883,36cyclictest25271-21latency_hist19:59:341
846699874,36cyclictest4672-21cat21:44:352
846699874,36cyclictest4672-21cat21:44:352
846699874,34cyclictest25270-21latency_hist19:59:342
846699874,33cyclictest20304-21latency_hist19:44:352
846699873,35cyclictest23315-21latency_hist19:54:352
846699873,30cyclictest3415-21wc00:19:562
846699873,30cyclictest3415-21wc00:19:562
846699872,36cyclictest21875-21/usr/sbin/munin23:04:482
846699872,36cyclictest21875-21/usr/sbin/munin23:04:482
846699872,33cyclictest13884-21latency_hist19:24:352
8463998767,10cyclictest0-21swapper/120:19:411
8463998767,10cyclictest0-21swapper/120:19:411
8463998765,16cyclictest24625-21latency_hist23:09:331
8463998765,16cyclictest24625-21latency_hist23:09:331
846399874,35cyclictest13554-21sed22:50:061
8463998740,40cyclictest0-21swapper/123:49:531
846399873,35cyclictest4507-21latency_hist23:29:331
846399873,33cyclictest12696-21cstates20:59:471
846399873,31cyclictest14107-21fschecks_count19:24:521
846399873,31cyclictest14107-21fschecks_count19:24:521
846399872,29cyclictest19444-21grep19:40:591
8463998713,31cyclictest10210-21timerwakeupswit20:50:061
8466998628,7cyclictest0-21swapper/221:35:102
846699862,36cyclictest11796-21grep20:55:062
846699862,31cyclictest31584-21kernelversion23:19:542
8463998672,10cyclictest8449-21cyclictest21:34:421
8463998639,40cyclictest11820-21awk23:40:081
846399863,35cyclictest27835-21modprobe23:14:311
846399863,34cyclictest28770-21sh21:30:031
8466998567,12cyclictest20575-21awk23:55:052
846699854,35cyclictest10622-21fschecks_count19:14:442
846699854,33cyclictest26087-21/usr/sbin/munin00:05:082
846699853,35cyclictest6337-21fschecks_count20:39:522
846699853,35cyclictest13575-21unixbench_singl21:00:072
846699853,32cyclictest1276-21grep22:30:062
846699853,31cyclictest8672-21grep20:45:102
846399853,35cyclictest22872-21switchtime21:20:071
846399853,35cyclictest20462-21sh23:55:031
846399853,33cyclictest27791-21latency_hist21:29:341
846399853,33cyclictest27791-21latency_hist21:29:341
846399853,32cyclictest4500-21find20:34:411
8463998530,8cyclictest0-21swapper/121:19:321
8463998524,14cyclictest0-21swapper/123:39:351
846399852,33cyclictest24075-21awk00:00:401
846399852,30cyclictest3436-21ntp_offset20:29:591
846699843,34cyclictest4505-21latency_hist23:29:332
846699843,32cyclictest26166-21switchtime20:00:032
8466998426,11cyclictest0-21swapper/220:34:532
846699842,32cyclictest9464-21latency_hist22:44:332
8463998458,17cyclictest26335-21fschecks_count00:04:511
846399844,30cyclictest4029-21latency22:34:561
8463998442,35cyclictest0-21swapper/121:49:451
8463998442,35cyclictest0-21swapper/121:49:451
8463998436,41cyclictest0-21swapper/121:39:511
8463998436,41cyclictest0-21swapper/121:39:511
846399843,35cyclictest19656-21switchtime23:00:051
846399843,35cyclictest19656-21switchtime23:00:051
846399843,27cyclictest10664-21sort19:14:451
8463998431,7cyclictest0-21swapper/100:20:021
8463998431,7cyclictest0-21swapper/100:20:021
846399842,23cyclictest15747-21/usr/sbin/munin22:55:041
8460998467,10cyclictest20530-21apt-get22:09:540
8460998467,10cyclictest20530-21apt-get22:09:540
8466998370,8cyclictest0-21swapper/222:24:592
846699833,34cyclictest28240-21cstates23:14:482
846699833,34cyclictest28240-21cstates23:14:482
846699833,29cyclictest27333-21fschecks_time20:04:532
846699833,25cyclictest9277-21cut19:10:042
8463998357,19cyclictest352-21runrttasks19:49:241
846399833,32cyclictest29154-21unixbench_multi23:15:081
846399833,32cyclictest29154-21unixbench_multi23:15:081
846399833,31cyclictest4815-21ls23:29:471
8463998329,8cyclictest0-21swapper/122:04:571
8463998324,11cyclictest0-21swapper/122:03:291
846399832,33cyclictest9978-21/usr/sbin/munin19:14:321
8463998314,20cyclictest12527-21sh00:34:561
8460998362,14cyclictest5664-21munin-plugin-st00:24:330
8460998362,14cyclictest5664-21munin-plugin-st00:24:330
8460998360,17cyclictest28732-21sh21:30:010
846699823,34cyclictest12943-21timerwakeupswit00:35:062
846399824,34cyclictest16647-21ssh21:10:041
846399824,34cyclictest16647-21ssh21:10:041
846399823,33cyclictest352-21runrttasks00:30:391
846399823,33cyclictest352-21runrttasks00:30:391
846399823,29cyclictest14592-21grep23:45:051
8463998218,13cyclictest0-21swapper/100:25:081
846699813,33cyclictest14094-21fschecks_time23:44:522
846399813,30cyclictest645-21ssh00:15:201
8463998128,10cyclictest0-21swapper/100:09:561
8463998127,6cyclictest0-21swapper/122:23:411
846399812,34cyclictest11478-21ntp_kernel_pll_20:54:591
8460998164,11cyclictest1418-21ssh21:39:310
8460998163,12cyclictest13776-21unixbench_multi22:50:080
8460998155,21cyclictest8449-21cyclictest23:22:240
8463998065,10cyclictest8449-21cyclictest19:59:501
8460998034,31cyclictest181rcu_preempt22:59:340
8471997932,22cyclictest16809-21chrt22:04:133
846099799,36cyclictest15-21ksoftirqd/023:03:020
846099799,36cyclictest15-21ksoftirqd/023:03:020
8460997966,9cyclictest58-21kcompactd022:39:080
8460997964,10cyclictest27039-21kworker/u19:0-writeback20:09:520
8460997963,11cyclictest58-21kcompactd023:26:340
846699783,24cyclictest8530-21latency_hist19:09:362
8460997855,15cyclictest30287-21idleruntime22:24:550
8460997842,8cyclictest181rcu_preempt00:09:430
8460997752,18cyclictest544-21cron00:39:310
8460997743,23cyclictest22589-40mandb23:59:360
847199764,49cyclictest0-21swapper/322:09:463
847199764,49cyclictest0-21swapper/322:09:463
847199763,60cyclictest31820-21taskset00:14:303
8460997662,9cyclictest0-21swapper/020:56:470
8460997656,13cyclictest17519-21ssh23:50:040
846099763,58cyclictest7683-21unixbench_singl22:40:080
8460997560,10cyclictest13679-21kworker/u19:2-events_unbound21:44:390
8460997554,13cyclictest58-21kcompactd023:37:040
8460997553,16cyclictest9137-21sh21:51:200
8460997553,16cyclictest9137-21sh21:51:200
8460997541,26cyclictest0-21swapper/022:15:050
8460997541,26cyclictest0-21swapper/022:15:050
8460997534,11cyclictest181rcu_preempt23:04:580
8460997534,11cyclictest181rcu_preempt23:04:580
846099753,40cyclictest25895-21ntpq23:10:360
8460997528,14cyclictest181rcu_preempt21:21:100
8460997459,8cyclictest181rcu_preempt20:44:540
8460997459,10cyclictest58-21kcompactd000:30:270
8460997459,10cyclictest58-21kcompactd000:30:270
8460997453,15cyclictest1814-21sh00:17:200
846099744,38cyclictest497-21/usr/sbin/munin22:29:550
8460997443,14cyclictest181rcu_preempt20:09:340
8460997441,17cyclictest29679-21diskmemload23:16:240
8460997441,17cyclictest29679-21diskmemload23:16:240
846099743,40cyclictest17609-21ls22:04:540
8460997420,38cyclictest181rcu_preempt21:40:130
8460997420,38cyclictest181rcu_preempt21:40:130
847199733,43cyclictest0-21swapper/321:50:313
847199733,43cyclictest0-21swapper/321:50:313
8460997355,12cyclictest26831-21systemd-run20:04:320
8460997354,11cyclictest0-21swapper/020:28:170
8460997354,11cyclictest0-21swapper/020:28:170
8460997333,13cyclictest181rcu_preempt22:45:090
8471997232,28cyclictest17440-21taskset22:04:493
8460997257,9cyclictest15574-21diskmemload22:19:410
8460997255,10cyclictest0-21swapper/021:27:520
8460997255,10cyclictest0-21swapper/021:27:520
846099723,47cyclictest15170-21timerwakeupswit22:00:070
8460997145,12cyclictest181rcu_preempt19:14:510
8460997143,10cyclictest181rcu_preempt21:19:100
8460997050,14cyclictest9385-21threads19:10:060
846099704,34cyclictest18455-21cron19:39:310
8460997042,12cyclictest181rcu_preempt19:19:570
846099703,55cyclictest15762-21latency_hist21:09:340
8460997035,12cyclictest181rcu_preempt23:41:010
846099703,35cyclictest5057-21open_files20:35:020
8471996930,28cyclictest15492-21taskset22:54:323
8471996929,27cyclictest27405-21taskset00:05:383
8460996954,10cyclictest11284-21kworker/u19:0-rpciod21:55:560
846099693,39cyclictest30010-21apt-get20:14:370
846099692,36cyclictest16128-21chrt23:48:320
8460996923,35cyclictest5973-21sh00:24:410
8471996831,26cyclictest16801-21taskset23:49:413
8460996848,8cyclictest181rcu_preempt19:29:330
8460996848,8cyclictest181rcu_preempt19:29:330
8460996846,8cyclictest181rcu_preempt20:19:410
8460996846,8cyclictest181rcu_preempt20:19:410
8460996843,9cyclictest181rcu_preempt19:44:470
8460996843,8cyclictest181rcu_preempt20:31:400
846099684,37cyclictest12587-21/usr/sbin/munin20:59:470
8460996835,17cyclictest181rcu_preempt20:50:070
8460996829,18cyclictest181rcu_preempt19:32:550
8460996829,18cyclictest181rcu_preempt19:32:550
81702685,14sleep30-21swapper/319:07:483
8471996732,25cyclictest15798-21chrt22:54:443
8471996730,21cyclictest9040-21taskset23:35:393
8460996751,9cyclictest0-21swapper/020:40:070
8460996745,9cyclictest181rcu_preempt19:55:110
846099673,32cyclictest28836-21awk00:09:310
846099673,31cyclictest22195-21fschecks_time19:49:540
8471996632,23cyclictest3973-21chrt21:43:093
8471996632,23cyclictest3973-21chrt21:43:093
8471996632,23cyclictest19836-21chrt23:54:433
8471996632,21cyclictest20372-21chrt19:44:463
8471996632,21cyclictest11827-21chrt23:40:083
8471996631,23cyclictest23144-21chrt23:05:303
8471996631,23cyclictest23144-21chrt23:05:303
8471996629,25cyclictest10227-21taskset19:14:363
8471996629,24cyclictest13537-21taskset19:23:153
8471996627,26cyclictest4930-21taskset23:29:513
8471996627,26cyclictest15269-21taskset19:29:313
8471996627,26cyclictest15269-21taskset19:29:313
8471996623,31cyclictest0-21swapper/300:14:583
8460996634,12cyclictest181rcu_preempt21:13:380
8460996634,12cyclictest181rcu_preempt21:13:380
8471996531,22cyclictest13804-21chrt23:44:423
8471996531,21cyclictest30487-21chrt20:14:393
8471996531,21cyclictest11612-21chrt22:47:223
8471996530,21cyclictest8091-21chrt22:41:063
8471996529,22cyclictest31461-21taskset21:34:543
8471996528,25cyclictest29740-21taskset21:31:333
8471996527,24cyclictest30355-21taskset23:18:153
8471996527,24cyclictest30355-21taskset23:18:153
8460996532,17cyclictest18743-21/usr/sbin/munin19:40:500
8471996431,21cyclictest18537-21chrt21:14:183
8471996431,21cyclictest18537-21chrt21:14:183
8471996431,19cyclictest2144-21chrt22:31:333
8471996429,22cyclictest9683-21chrt19:10:563
8471996429,21cyclictest22831-21chrt21:20:063
8471996428,24cyclictest19883-21chrt19:41:543
8471996428,23cyclictest26955-21chrt22:19:543
8471996428,22cyclictest20279-21chrt23:01:003
8471996428,22cyclictest20279-21chrt23:01:003
8471996331,21cyclictest19059-21chrt21:14:473
8471996331,20cyclictest5333-21chrt20:35:063
8471996330,22cyclictest23789-21taskset19:54:383
8471996330,21cyclictest17135-21chrt19:34:453
8471996330,20cyclictest9481-21chrt20:49:483
8471996330,20cyclictest9009-21chrt00:29:423
8471996330,20cyclictest9009-21chrt00:29:423
8471996329,22cyclictest5992-21chrt00:24:443
8471996329,21cyclictest31358-21chrt23:19:503
8471996328,22cyclictest3322-21chrt23:26:303
8471996328,22cyclictest29688-21chrt20:10:303
8471996327,22cyclictest12812-21chrt21:56:433
8471996239,14cyclictest109752sleep320:54:433
8471996231,21cyclictest24692-21chrt22:15:113
8471996231,21cyclictest24692-21chrt22:15:113
8471996231,20cyclictest32494-21chrt20:19:553
8471996231,20cyclictest32494-21chrt20:19:553
8471996230,21cyclictest3588-21chrt22:34:463
8471996230,20cyclictest22315-21chrt19:49:563
8471996230,20cyclictest1845-21chrt20:25:013
8471996230,20cyclictest1845-21chrt20:25:013
8471996229,22cyclictest26977-21chrt20:04:353
8471996228,22cyclictest2963-21chrt20:29:473
8471996228,21cyclictest16738-21chrt19:33:073
8471996228,21cyclictest16738-21chrt19:33:073
8471996227,22cyclictest7008-21chrt21:48:073
8471996227,22cyclictest4594-21chrt00:21:363
8471996227,22cyclictest4594-21chrt00:21:363
8471996130,20cyclictest11973-21chrt00:34:373
8471996130,19cyclictest31144-21chrt22:25:253
8471996128,21cyclictest15456-21chrt21:08:233
8471996026,23cyclictest23059-21taskset23:59:383
8471996024,24cyclictest7219-21taskset20:41:133
8471996024,23cyclictest13611-21taskset21:00:213
8471995928,20cyclictest9050-21chrt20:49:033
8471995928,20cyclictest25184-21chrt23:09:443
8471995925,24cyclictest28124-21taskset20:06:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional