You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-12 - 12:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Thu Mar 12, 2026 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
834621309,89sleep08159-21/usr/sbin/munin19:00:510
9659991143,81cyclictest2022-21apt-get00:15:401
9659991143,81cyclictest2022-21apt-get00:15:401
9659991072,74cyclictest15111-21ssh21:55:491
9659991013,38cyclictest21614-21fschecks_count22:06:101
96599910051,42cyclictest0-21swapper/122:15:481
9659999933,13cyclictest0-21swapper/122:10:501
9659999933,13cyclictest0-21swapper/122:10:501
9654999982,10cyclictest5581-21apt-get21:40:420
95702999,63sleep19599-21sed19:05:291
965999983,37cyclictest12366-21latency_hist22:45:281
965999983,37cyclictest12366-21latency_hist22:45:281
9659999744,10cyclictest0-21swapper/100:26:011
965999968,48cyclictest18892-21apt-get22:55:581
965999963,39cyclictest25402-21latency19:50:501
9659999538,10cyclictest0-21swapper/122:36:021
965999944,40cyclictest13991-21rm23:40:561
965999944,40cyclictest13991-21rm23:40:561
9659999442,15cyclictest9642-21cyclictest21:26:291
9659999412,40cyclictest2523-21fschecks_time20:20:471
9659999336,13cyclictest0-21swapper/123:35:401
9667999265,19cyclictest28011-21idleruntime-cro20:00:262
965999923,36cyclictest10181-21latency_hist20:45:281
9659999214,33cyclictest4848-21cat23:25:501
9654999236,11cyclictest181rcu_preempt21:35:410
965999913,43cyclictest16547-21diskmemload21:41:441
9659999127,24cyclictest31672-21uname00:10:511
965499902,63cyclictest15032-21apt-get21:55:460
966799893,73cyclictest20196-21apt-get19:35:422
965999892,79cyclictest900-21kworker/u17:1+ext4-rsv-conversion21:45:551
965999892,79cyclictest900-21kworker/u17:1+ext4-rsv-conversion21:45:551
9654998955,11cyclictest181rcu_preempt20:15:550
82602899,56sleep28310-21cstates19:00:452
9667998866,16cyclictest16547-21diskmemload21:51:002
965999883,42cyclictest24997-21munin-run00:00:251
9654998859,21cyclictest22055-21ssh23:00:430
9667998665,15cyclictest9347-21sh21:46:442
9667998665,15cyclictest9347-21sh21:46:442
9654998558,7cyclictest181rcu_preempt23:45:070
9654998558,7cyclictest181rcu_preempt23:45:070
9667998414,41cyclictest0-21swapper/223:45:412
9659998465,14cyclictest9642-21cyclictest20:46:011
9667998354,19cyclictest19550-21fschecks_count23:50:452
965999833,34cyclictest4598-21switchtime20:25:581
9654998357,10cyclictest181rcu_preempt00:09:440
9654998351,7cyclictest181rcu_preempt00:30:590
9654998260,14cyclictest298-21in:imuxsock22:51:070
9654998253,9cyclictest181rcu_preempt19:20:410
9654998245,17cyclictest181rcu_preempt23:05:470
967199812,60cyclictest8897-21taskset21:45:553
967199812,60cyclictest8897-21taskset21:45:553
9659998155,17cyclictest19815-21apt-key19:35:281
9654998160,15cyclictest25188-21fschecks_count19:50:460
966799803,68cyclictest15032-21apt-get21:55:492
9667998030,32cyclictest30511-21kworker/u17:3-rpciod23:35:582
9659998055,17cyclictest31425-21latency_hist20:10:291
9654998053,17cyclictest11164-21timerwakeupswit20:46:000
9654998051,19cyclictest12795-21sh22:45:420
9654998025,43cyclictest27859-21ssh23:10:410
9667997958,15cyclictest5786-21mailstats00:21:042
966799793,67cyclictest1470-21apt-get00:15:402
966799793,67cyclictest1470-21apt-get00:15:402
9659997959,12cyclictest0-21swapper/123:11:031
9659997943,11cyclictest181rcu_preempt22:30:371
965499798,49cyclictest18325-21apt-get22:55:460
9654997954,7cyclictest181rcu_preempt00:17:530
9654997954,7cyclictest181rcu_preempt00:17:530
9654997927,22cyclictest0-21swapper/021:47:290
9654997927,22cyclictest0-21swapper/021:47:290
965499792,40cyclictest7385-21ssh23:30:380
967199783,41cyclictest0-21swapper/322:05:483
9667997857,15cyclictest16547-21diskmemload23:33:192
966799785,56cyclictest4392-21ssh23:25:352
966799784,38cyclictest0-21swapper/222:40:532
966799784,38cyclictest0-21swapper/222:40:532
9659997841,13cyclictest181rcu_preempt21:15:251
965499783,42cyclictest352-21runrttasks22:44:400
965499783,42cyclictest352-21runrttasks22:44:400
967199773,40cyclictest14092-21chrt23:41:003
967199773,40cyclictest14092-21chrt23:41:003
9667997761,10cyclictest6330-21kernelversion21:41:472
9667997747,21cyclictest14331-21threads20:55:592
9667997747,21cyclictest14331-21threads20:55:592
9667997724,40cyclictest1940-21sh21:35:132
9659997761,9cyclictest6332-21kworker/u17:1-rpciod23:09:101
9659997757,11cyclictest745-21diskmemload22:25:391
9659997757,11cyclictest745-21diskmemload22:25:391
9659997754,14cyclictest32060-21ssh21:30:461
9659997654,16cyclictest16091-21kernelversion22:50:551
9659997644,8cyclictest0-21swapper/119:36:181
9654997660,10cyclictest31740-21ssh22:22:160
9654997656,14cyclictest19973-21apt-key19:35:300
9654997647,11cyclictest181rcu_preempt20:56:060
9654997647,11cyclictest181rcu_preempt20:56:060
9654997632,13cyclictest181rcu_preempt21:20:480
65272760,5sleep09642-21cyclictest22:35:280
966799758,57cyclictest20688-21apt-get22:05:402
9667997559,10cyclictest14823-21kworker/u18:1+flush-179:021:15:432
9667997558,12cyclictest9642-21cyclictest22:29:332
9667997558,12cyclictest9642-21cyclictest22:29:332
9667997555,15cyclictest22592-21sh21:15:272
9667997549,20cyclictest25625-21fschecks_count00:01:032
9667997521,13cyclictest0-21swapper/200:10:482
9667997514,26cyclictest0-21swapper/200:05:452
9659997557,12cyclictest2956-21ntp_kernel_pll_21:35:551
9659997547,11cyclictest0-21swapper/100:35:291
9654997541,12cyclictest181rcu_preempt23:24:020
9654997539,17cyclictest181rcu_preempt22:11:030
9654997539,17cyclictest181rcu_preempt22:11:030
966799747,27cyclictest0-21swapper/219:25:282
9667997454,13cyclictest0-21swapper/223:00:462
9667997445,19cyclictest19756-21latency_hist21:10:292
9667997426,28cyclictest25059-21sh23:05:452
965999742,35cyclictest28259-21/usr/sbin/munin00:05:531
9654997451,8cyclictest181rcu_preempt23:50:500
9654997443,9cyclictest181rcu_preempt23:40:010
9654997442,23cyclictest19531-21modprobe21:10:250
92032740,7sleep09642-21cyclictest00:26:230
9667997350,16cyclictest365-21ntpd19:30:462
9667997324,10cyclictest0-21swapper/223:55:402
9659997351,16cyclictest15138-21latency_hist21:00:281
9659997351,16cyclictest15138-21latency_hist21:00:281
9659997345,20cyclictest515-21latency_hist20:15:281
9659997338,13cyclictest181rcu_preempt23:50:571
9654997351,13cyclictest16547-21diskmemload23:48:000
9654997351,13cyclictest1465-21latency_hist00:15:240
9654997345,9cyclictest181rcu_preempt22:19:430
9654997339,17cyclictest181rcu_preempt21:29:170
9654997335,8cyclictest181rcu_preempt21:15:470
9654997332,8cyclictest181rcu_preempt19:55:500
967199724,28cyclictest19749-21chrt22:56:183
967199723,37cyclictest12589-21chrt23:39:093
966799728,28cyclictest0-21swapper/223:40:432
966799728,28cyclictest0-21swapper/223:40:432
9667997252,14cyclictest21333-21kworker/u17:4-xprtiod22:20:482
9667997249,17cyclictest5043-21grep21:40:282
9667997247,15cyclictest0-21swapper/222:46:012
9667997246,20cyclictest19713-21processes22:56:172
9667997243,17cyclictest0-21swapper/223:25:292
9659997254,13cyclictest58-21kcompactd020:30:591
9659997244,19cyclictest26830-21fschecks_count19:55:481
9659997243,21cyclictest10050-21latency_hist23:35:281
9654997254,11cyclictest28176-21ssh00:05:290
9654997253,13cyclictest352-21runrttasks20:25:590
9654997251,15cyclictest11342-21latency_hist19:10:280
9654997251,15cyclictest11342-21latency_hist19:10:280
9654997232,11cyclictest0-21swapper/019:40:280
965999712,32cyclictest31266-21unixbench_singl22:21:041
965499717,33cyclictest12213-21uname21:50:530
9654997152,12cyclictest864-21kworker/u19:1-writeback23:20:280
9654997151,7cyclictest181rcu_preempt20:10:570
965499714,31cyclictest32516-21ssh21:30:570
9654997139,24cyclictest0-21swapper/022:05:290
965499713,34cyclictest5489-21ls00:20:560
967199703,45cyclictest0-21swapper/300:16:533
967199703,45cyclictest0-21swapper/300:16:533
9667997052,13cyclictest352-21runrttasks22:11:202
9667997052,13cyclictest352-21runrttasks22:11:202
9659997048,12cyclictest0-21swapper/123:45:351
9659997043,18cyclictest28515-21latency_hist21:25:281
965999703,33cyclictest12954-21fschecks_time22:45:471
9654997043,9cyclictest181rcu_preempt20:33:300
56622702,9sleep05668-21ssh23:26:310
9667996947,11cyclictest0-21swapper/222:30:482
9667996946,12cyclictest0-21swapper/223:15:452
966799692,49cyclictest28441-21fschecks_count20:00:462
9659996950,12cyclictest0-21swapper/100:05:251
9659996946,10cyclictest58-21kcompactd019:20:531
9659996945,15cyclictest0-21swapper/122:05:281
9659996934,13cyclictest181rcu_preempt20:05:261
9659996927,17cyclictest181rcu_preempt00:23:211
965999692,26cyclictest1119-21apt-get23:20:291
9654996954,9cyclictest20429-21ntp_kernel_pll_21:10:540
9654996945,9cyclictest181rcu_preempt22:25:460
9654996945,9cyclictest181rcu_preempt22:25:460
965499692,31cyclictest16552-21modprobe21:05:250
967199683,25cyclictest29969-21chrt00:07:423
967199682,30cyclictest5600-21taskset21:40:403
9667996847,12cyclictest0-21swapper/200:25:502
9667996846,12cyclictest0-21swapper/223:10:522
966799684,42cyclictest18096-21cpuspeed22:00:412
966799684,39cyclictest4144-21fschecks_time20:25:462
9667996843,17cyclictest0-21swapper/219:20:282
966799683,42cyclictest23706-21idleruntime19:45:492
9659996846,13cyclictest0-21swapper/123:05:281
965999683,33cyclictest1545-21cstates23:20:421
9659996824,32cyclictest13928-21latency19:15:491
9659996821,34cyclictest15931-21cat21:01:001
9654996842,11cyclictest181rcu_preempt19:14:210
9654996837,12cyclictest181rcu_preempt19:20:000
9654996832,9cyclictest181rcu_preempt19:45:400
967199673,37cyclictest16290-21chrt23:45:293
967199673,32cyclictest2284-21chrt21:35:293
966799678,22cyclictest16390-21find22:51:032
9667996749,13cyclictest22560-21threads19:41:022
966799673,34cyclictest13359-21cat20:55:282
9667996722,11cyclictest5636-21df_inode20:30:432
9654996740,10cyclictest181rcu_preempt22:06:110
965499673,42cyclictest17079-21kernelversion19:25:500
965499672,55cyclictest22594-21fschecks_time23:55:460
954826711,34sleep30-21swapper/319:05:273
967199664,26cyclictest23070-21chrt23:01:163
967199662,29cyclictest26999-21chrt23:08:583
9667996649,10cyclictest277-21dbus-daemon21:25:262
966799663,43cyclictest30116-21fschecks_time20:05:482
9667996629,24cyclictest18057-21modprobe19:30:252
966799662,47cyclictest7327-21fschecks_count20:35:452
9667996622,8cyclictest0-21swapper/219:10:322
9667996618,12cyclictest29034-21diskstats21:25:462
965499663,50cyclictest3695-21latency_hist20:25:280
967199654,27cyclictest7457-21chrt00:24:523
967199653,29cyclictest21533-21taskset19:40:253
967199652,24cyclictest11290-21chrt00:30:483
967199652,23cyclictest26758-21chrt00:02:033
966799654,44cyclictest12064-21vmstat00:31:032
966799653,51cyclictest25032-21chrt19:50:412
966799653,42cyclictest10582-21fschecks_time20:45:472
966799653,41cyclictest1060-21sort20:15:512
9667996521,12cyclictest0-21swapper/222:15:472
9659996534,11cyclictest181rcu_preempt19:05:471
9659996534,11cyclictest181rcu_preempt19:05:471
965999653,31cyclictest23133-21cron19:45:251
965999652,45cyclictest12853-21unixbench_multi20:51:021
9659996517,32cyclictest14699-21cat21:55:281
9654996540,17cyclictest9642-21cyclictest20:00:460
9654996537,8cyclictest181rcu_preempt22:36:290
967199648,46cyclictest0-21swapper/322:36:053
967199643,27cyclictest22366-21chrt19:40:583
967199643,24cyclictest10288-21chrt20:45:293
967199643,23cyclictest5586-21chrt22:33:163
967199643,23cyclictest29459-21chrt21:25:573
967199642,28cyclictest23159-21chrt21:15:473
967199642,27cyclictest15419-21taskset21:56:043
967199642,22cyclictest17476-21chrt21:05:543
9667996441,12cyclictest0-21swapper/220:40:362
966799642,40cyclictest352-21runrttasks21:01:182
966799642,40cyclictest10237-21latency19:05:502
966799642,40cyclictest10237-21latency19:05:502
9667996420,32cyclictest8093-21diskmemload22:36:512
9667996420,12cyclictest31640-21diskstats20:10:482
9659996449,9cyclictest16231-21kworker/u17:1-rpciod21:05:461
9659996438,16cyclictest0-21swapper/120:35:581
965999642,42cyclictest12250-21latency19:10:531
9654996425,10cyclictest0-21swapper/020:50:500
9654996412,34cyclictest31275-21latency_hist20:10:280
967199635,22cyclictest29505-21chrt20:04:143
967199634,23cyclictest2920-21chrt21:35:543
967199633,25cyclictest11111-21chrt22:41:553
967199633,25cyclictest11111-21chrt22:41:553
967199633,23cyclictest2770-21chrt22:28:423
967199633,23cyclictest2770-21chrt22:28:423
967199633,22cyclictest6682-21chrt20:33:433
967199633,22cyclictest4056-21taskset20:25:453
967199632,23cyclictest22837-21chrt23:55:513
967199632,23cyclictest14240-21chrt21:54:503
9667996318,11cyclictest0-21swapper/220:25:282
9659996343,15cyclictest23250-21if_eth021:15:491
9659996342,12cyclictest0-21swapper/120:15:421
965999633,37cyclictest23665-21fschecks_time19:45:471
967199624,27cyclictest27605-21chrt22:15:503
967199624,21cyclictest1940-21chrt20:19:413
967199623,28cyclictest12370-21taskset20:50:523
967199623,24cyclictest32534-21chrt23:17:413
967199623,23cyclictest2781-21chrt23:21:473
967199623,22cyclictest30135-21chrt23:14:253
967199623,22cyclictest21084-21chrt23:52:553
967199623,22cyclictest13308-21chrt22:45:563
967199623,20cyclictest3400-21chrt20:22:453
967199622,23cyclictest9169-21taskset00:26:183
967199622,23cyclictest26746-21chrt22:15:053
967199622,23cyclictest26746-21chrt22:15:053
967199622,22cyclictest4752-21chrt23:25:483
967199622,22cyclictest26011-21chrt19:51:063
967199622,22cyclictest20969-21chrt21:11:053
9654996213,14cyclictest7397-21runrttasks20:35:460
967199614,20cyclictest27980-21chrt20:00:223
967199613,23cyclictest9636-21chrt23:34:193
967199613,23cyclictest14673-21chrt19:18:183
967199612,25cyclictest1419-21chrt00:15:193
967199612,23cyclictest16729-21chrt19:25:393
965999612,35cyclictest17136-21grep19:25:501
9654996138,8cyclictest181rcu_preempt20:40:350
967199603,24cyclictest11964-21chrt19:10:453
967199603,22cyclictest31482-21chrt20:10:383
967199603,20cyclictest16440-21chrt21:04:203
967199603,19cyclictest8228-21chrt20:37:353
967199602,25cyclictest27773-21taskset21:23:213
967199602,23cyclictest14629-21chrt20:56:433
967199602,23cyclictest14629-21chrt20:56:433
967199602,22cyclictest11371-21chrt20:46:343
9671996010,41cyclictest0-21swapper/319:35:293
965499603,31cyclictest22067-21grep19:40:490
967199592,20cyclictest16289-21chrt19:23:243
967199582,26cyclictest18031-21chrt22:00:383
967199582,21cyclictest24497-21chrt19:47:183
9671995812,39cyclictest0-21swapper/319:05:583
9671995812,39cyclictest0-21swapper/319:05:583
967199563,20cyclictest30513-21chrt22:20:483
967199562,22cyclictest30892-21taskset20:06:253
967199562,21cyclictest17414-21chrt22:52:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional