You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 21:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Tue Apr 14, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2621121348,17sleep00-21swapper/006:59:400
26647991172,47cyclictest20098-21apt-get11:23:292
26647991153,52cyclictest5858-21ssh10:59:182
266389911137,32cyclictest6045-21apt-get11:53:210
266389911137,32cyclictest6045-21apt-get11:53:210
266389910935,31cyclictest3067-21apt-get11:48:260
266389910435,30cyclictest12617-21ssh12:03:410
266389910234,30cyclictest14499-21apt-get11:13:290
26638991022,88cyclictest20098-21apt-get11:23:260
266389910215,71cyclictest22913-21apt-get10:33:290
266389910132,22cyclictest32708-21find07:18:450
2664299993,36cyclictest22468-21/usr/sbin/munin09:38:331
2664799987,43cyclictest24232-21ssh12:23:362
26638999825,48cyclictest31910-21latency_hist08:58:150
26647999734,51cyclictest0-21swapper/211:33:152
26647999734,51cyclictest0-21swapper/211:33:152
26647999734,10cyclictest0-21swapper/211:58:152
26647999734,10cyclictest0-21swapper/211:58:152
2663899973,67cyclictest5526-21apt-get10:58:290
26638999731,29cyclictest1078-21diskmemload09:23:290
26638999723,32cyclictest31440-21unixbench_singl07:13:490
26642999644,16cyclictest26089-21fschecks_count10:38:331
26642999644,16cyclictest26089-21fschecks_count10:38:331
26638999622,34cyclictest7254-21latency_hist09:13:150
26638999621,38cyclictest26472-21cat11:33:150
26638999621,38cyclictest26472-21cat11:33:150
26647999560,29cyclictest7994-21ntp_offset09:13:422
26638999532,28cyclictest29004-21ssh10:43:290
26638999532,22cyclictest18235-21/usr/sbin/munin11:18:470
26638999532,22cyclictest18235-21/usr/sbin/munin11:18:470
2664799943,27cyclictest9257-21/usr/sbin/munin11:58:332
26638999466,18cyclictest8436-21sh09:13:500
26638999465,23cyclictest12608-21latency_hist07:58:140
26638999457,30cyclictest19529-21apt-get09:33:430
26647999373,13cyclictest298-21in:imuxsock10:38:112
2664799932,33cyclictest5567-21wc07:33:542
2664799932,33cyclictest5567-21wc07:33:542
2663899937,73cyclictest3093-21apt-get07:28:270
26638999330,31cyclictest1078-21diskmemload09:53:330
26638999330,31cyclictest1078-21diskmemload09:53:330
26638999321,24cyclictest29453-21latency_hist11:38:140
26647999242,40cyclictest0-21swapper/207:28:242
26647999223,26cyclictest22732-21sshd09:38:352
26638999255,25cyclictest9547-21apt-get07:48:240
26647999168,16cyclictest26751-21latency_hist07:03:172
26647999140,42cyclictest0-21swapper/209:08:152
26647999140,42cyclictest0-21swapper/209:08:152
26647999136,8cyclictest0-21swapper/209:34:132
26647999134,9cyclictest0-21swapper/209:13:122
26647999134,51cyclictest0-21swapper/208:28:482
2664799913,38cyclictest8153-21fschecks_count07:43:332
2664799913,38cyclictest8153-21fschecks_count07:43:332
2664799913,36cyclictest3011-21latency_hist07:28:162
2664799913,32cyclictest30521-21ls11:38:472
2664799913,27cyclictest1750350ssh09:29:072
26647999128,57cyclictest0-21swapper/211:18:482
26647999128,57cyclictest0-21swapper/211:18:482
26647999124,11cyclictest0-21swapper/210:03:342
2664799912,35cyclictest11679-21sh11:08:342
2664799912,34cyclictest1995-21latency_hist10:53:112
26638999161,21cyclictest298-21in:imuxsock08:33:340
26638999124,25cyclictest21151-21ssh12:18:330
26638999124,25cyclictest21151-21ssh12:18:330
26638999122,30cyclictest1467-21ssh09:03:280
26638999122,30cyclictest1467-21ssh09:03:280
26638999122,26cyclictest26151-21uname09:43:450
26638999122,26cyclictest26151-21uname09:43:450
2647829116,49sleep126503-21munin-plugin-st07:03:121
2664799903,36cyclictest13397-21processes07:58:442
2664799902,33cyclictest1078-21diskmemload11:48:282
26638999035,41cyclictest16508-21sshd09:28:340
26638999032,24cyclictest19056-21rm12:13:490
26638999028,21cyclictest0-21swapper/011:08:490
26638999024,27cyclictest0-21swapper/008:28:340
26638999023,30cyclictest29207-21fschecks_time08:48:350
26638999023,30cyclictest29207-21fschecks_time08:48:350
26638999022,27cyclictest6654-21rm10:05:470
26638999015,27cyclictest0-21swapper/012:13:130
2665199893,43cyclictest17377-21chrt08:13:113
2665199893,43cyclictest17377-21chrt08:13:113
2664799893,36cyclictest13046-21grep12:03:472
2664799893,35cyclictest2188-21sh09:58:332
2664799893,32cyclictest28403-21/usr/sbin/munin09:48:402
2664799893,28cyclictest17148-21/usr/sbin/munin10:23:382
26647998928,8cyclictest0-21swapper/208:03:492
2664799892,34cyclictest25437-21modprobe08:38:112
2664799892,33cyclictest14389-21ssh10:18:422
26638998958,20cyclictest16007-21apt-get08:08:290
26638998958,20cyclictest16007-21apt-get08:08:290
26638998927,27cyclictest11781-21ssh09:19:430
26638998927,27cyclictest11781-21ssh09:19:430
26638998926,22cyclictest8040-21cstates07:43:290
26638998926,22cyclictest8040-21cstates07:43:290
26638998918,37cyclictest15603-21ssh10:20:530
26647998834,7cyclictest0-21swapper/210:53:422
2664799883,35cyclictest22774-21cmp10:33:162
2664799883,35cyclictest21173-21fschecks_count12:18:332
2664799883,35cyclictest21173-21fschecks_count12:18:332
2664799883,33cyclictest32520-21/usr/sbin/munin11:43:322
2664799883,33cyclictest32520-21/usr/sbin/munin11:43:322
2664799883,33cyclictest26463-21threads08:38:462
2664799883,32cyclictest32086-21uname09:53:452
2664799883,32cyclictest32086-21uname09:53:452
2664799883,31cyclictest32672-21runrttasks08:58:402
2664799883,31cyclictest16673-21mailstats08:08:462
2664799883,31cyclictest16673-21mailstats08:08:462
2664799883,28cyclictest29116-21fschecks_count08:48:342
2664799883,28cyclictest29116-21fschecks_count08:48:342
2664799883,28cyclictest27277-21/usr/sbin/munin11:33:462
26647998828,8cyclictest0-21swapper/211:03:292
26647998826,10cyclictest0-21swapper/210:38:492
26647998826,10cyclictest0-21swapper/210:38:492
2664799882,35cyclictest10372-21timerwakeupswit07:48:472
26638998824,28cyclictest28409-21munin-node07:08:170
26638998823,29cyclictest28654-21modprobe08:48:120
26638998821,23cyclictest32399-21latency_hist11:43:140
2664799874,37cyclictest352-21runrttasks07:16:472
2664799874,30cyclictest8161-21/usr/sbin/munin10:08:352
2664799873,34cyclictest29740-21grep07:08:502
2664799873,31cyclictest31179-21sendmail_mailqu08:53:432
2664799873,31cyclictest13300-21/usr/sbin/munin09:23:342
2664799873,30cyclictest19029-21latency_hist08:18:132
2664799873,29cyclictest10341-21/usr/sbin/munin09:18:382
2664799873,29cyclictest10341-21/usr/sbin/munin09:18:382
26647998726,8cyclictest0-21swapper/207:03:432
2664799872,33cyclictest7742-21latency_hist07:43:132
2664799872,33cyclictest7742-21latency_hist07:43:132
2664799872,32cyclictest19494-21fschecks_time08:18:322
26647998719,27cyclictest15323-21switchtime11:13:482
26647998718,17cyclictest0-21swapper/209:43:482
26647998718,17cyclictest0-21swapper/209:43:482
26642998765,16cyclictest20692-21kworker/u17:3+events_unbound11:33:111
26642998765,16cyclictest20692-21kworker/u17:3+events_unbound11:33:111
26638998730,19cyclictest1106-21grep11:43:480
26638998730,19cyclictest1106-21grep11:43:480
26638998729,38cyclictest31108-21seq09:53:120
26638998728,27cyclictest16797-21grep10:23:290
26638998726,20cyclictest0-21swapper/008:53:390
26638998722,29cyclictest19236-21/usr/sbin/munin08:18:420
26638998717,40cyclictest0-21swapper/008:13:470
26638998717,34cyclictest0-21swapper/010:48:470
2664799863,34cyclictest27784-21timerandwakeup12:28:492
2664799863,29cyclictest32311-21fschecks_count07:18:342
2664799863,27cyclictest11389-21ntp_kernel_pll_10:13:392
26647998629,8cyclictest0-21swapper/210:43:282
26638998628,23cyclictest23864-21rm09:39:370
26638998625,35cyclictest5456-21latency_hist10:58:150
26638998623,36cyclictest0-21swapper/008:38:350
26638998623,28cyclictest6554-21fschecks_time07:38:330
26638998623,28cyclictest6554-21fschecks_time07:38:330
2664799853,32cyclictest14942-21/usr/sbin/munin12:08:222
2664799853,30cyclictest20665-21latency_hist12:18:132
2664799852,31cyclictest27696-21kernelversion08:43:372
26638998533,24cyclictest11165-21sh11:08:120
26638998527,20cyclictest0-21swapper/007:03:390
26638998523,29cyclictest22259-21date08:28:130
26638998522,24cyclictest11034-21sh10:13:320
26638998520,32cyclictest8303-21latency10:08:370
2664799843,34cyclictest11383-21fschecks_count07:53:332
2664799843,32cyclictest20956-21cstates08:23:302
26638998455,14cyclictest12793-21apt-get07:58:240
26638998442,36cyclictest0-21swapper/012:23:250
26638998426,23cyclictest26880-21apt-get12:28:260
26638998420,32cyclictest9019-21/usr/sbin/munin11:58:520
26638998325,25cyclictest6152-21grep07:38:130
26638998325,25cyclictest6152-21grep07:38:130
26638998324,23cyclictest0-21swapper/010:40:350
26638998324,23cyclictest0-21swapper/010:40:350
26638998323,24cyclictest14488-21awk08:03:290
26642998262,13cyclictest26624-21cyclictest09:31:591
26638998227,18cyclictest20025-21ssh10:28:350
26638998131,20cyclictest3520-21taskset10:00:180
26642998058,15cyclictest26624-21cyclictest09:27:151
26642998039,13cyclictest0-21swapper/109:33:421
26638998043,30cyclictest2906-21apt-config07:28:150
26642997963,10cyclictest24224-21fschecks_time12:23:351
26642997962,11cyclictest21956-21sh11:25:101
26642997956,15cyclictest26624-21cyclictest08:53:461
26642997858,13cyclictest23228-21awk08:28:471
26642997857,15cyclictest26624-21cyclictest12:13:451
26642997842,25cyclictest27294-21kworker/u18:0+nfsiod11:21:241
26642997842,25cyclictest27294-21kworker/u18:0+nfsiod11:21:241
2665199773,36cyclictest1973-21chrt07:23:463
26642997559,9cyclictest21157-21kworker/u19:1+events_unbound07:48:111
26642997559,9cyclictest21157-21kworker/u19:1+events_unbound07:48:111
26642997554,13cyclictest26624-21cyclictest07:08:421
2664299753,30cyclictest22382-21ssh10:33:121
26642997355,12cyclictest297-21systemd-logind07:28:141
26642997350,12cyclictest0-21swapper/111:38:431
26642997314,38cyclictest18548-21grep08:13:491
26642997258,10cyclictest26624-21cyclictest10:53:001
26642997254,12cyclictest0-21swapper/109:03:281
26642997254,12cyclictest0-21swapper/109:03:281
26642997252,14cyclictest3400-21cut11:48:331
26642997157,10cyclictest25459-21kworker/u19:2-rpciod09:13:381
26642997155,10cyclictest8779-21latency_hist11:58:131
26642997155,10cyclictest8779-21latency_hist11:58:131
2664299714,49cyclictest22413-21latency_hist08:28:151
26642997140,9cyclictest31-21ksoftirqd/112:03:061
2664299713,32cyclictest25427-21ssh09:43:221
2664299713,32cyclictest25427-21ssh09:43:221
26642997121,19cyclictest15636-21grep12:08:441
26642997052,12cyclictest2371-21ssh10:53:271
26642997049,14cyclictest16480-21kworker/u17:2-rpciod10:10:361
26642997048,13cyclictest32424-21unixbench_multi09:53:491
26642997048,13cyclictest32424-21unixbench_multi09:53:491
2664299704,45cyclictest344-21fschecks_time11:43:331
2664299704,45cyclictest344-21fschecks_time11:43:331
2664299703,48cyclictest4351-21sh09:08:201
2664299703,45cyclictest352-21runrttasks09:01:031
2664299702,41cyclictest352-21runrttasks08:03:241
26642996950,12cyclictest28508-21sort09:48:291
2664299694,53cyclictest5864-21ssh10:59:191
26642996937,12cyclictest181rcu_preempt07:48:331
2664299693,55cyclictest28285-21grep08:43:491
26642996916,28cyclictest22913-21apt-get10:33:261
26642996849,10cyclictest0-21swapper/107:13:341
26642996839,23cyclictest12797-21kworker/u18:2+nfsiod10:28:091
26642996837,22cyclictest9168-21grep11:03:451
2664299675,32cyclictest27056-21cut07:03:331
26642996749,12cyclictest16355-21kernelversion08:08:381
26642996749,12cyclictest16355-21kernelversion08:08:381
26642996748,12cyclictest0-21swapper/111:08:341
2664299674,46cyclictest352-21runrttasks07:29:571
2664299674,41cyclictest13756-21/usr/sbin/munin10:18:471
2664299674,37cyclictest352-21runrttasks10:00:261
26642996739,20cyclictest6064-21modprobe07:38:111
26642996739,20cyclictest6064-21modprobe07:38:111
2664299664,36cyclictest10791-21/usr/sbin/munin10:13:431
2664299664,35cyclictest20891-21/usr/sbin/munin12:18:441
2664299664,35cyclictest20891-21/usr/sbin/munin12:18:441
26642996634,23cyclictest30258-21modprobe08:53:121
26642996634,23cyclictest30258-21modprobe08:53:121
2664299663,41cyclictest25057-21unixbench_singl08:33:481
26642996614,13cyclictest29279-21latency10:43:371
26642996546,13cyclictest5491-21sh10:03:431
26642996543,12cyclictest0-21swapper/111:33:431
2664299653,42cyclictest32343-21fschecks_time07:18:341
26642996527,27cyclictest26905-21sh12:28:281
26642996450,9cyclictest15064-21ntp_kernel_err11:13:421
2664299643,34cyclictest10801-21ssh09:18:381
2664299643,34cyclictest10801-21ssh09:18:381
2664299642,45cyclictest7201-21timerwakeupswit07:38:471
2664299642,45cyclictest7201-21timerwakeupswit07:38:471
2665199633,39cyclictest23890-21chrt09:39:413
2664299632,43cyclictest19621-21kernelversion08:18:351
2664299623,32cyclictest14199-21latency_hist08:03:141
2664299623,25cyclictest27155-21latency_hist08:43:141
26642996217,32cyclictest11275-21cat07:53:291
2574525917,26sleep30-21swapper/306:58:543
2665199584,38cyclictest16509-21taskset09:28:343
2665199582,41cyclictest14940-21chrt10:19:003
2665199573,38cyclictest29284-21taskset07:08:403
26642995720,10cyclictest11969-21/usr/sbin/munin12:03:341
2665199567,33cyclictest7360-21chrt07:39:173
2665199567,33cyclictest7360-21chrt07:39:173
2665199563,36cyclictest27331-21chrt08:43:283
2665199563,34cyclictest28516-21chrt12:30:213
2665199556,28cyclictest0-21swapper/311:24:293
2665199555,34cyclictest19900-21chrt09:33:573
2665199554,35cyclictest9102-21chrt10:09:193
2665199553,40cyclictest25815-21chrt11:31:583
2665199553,40cyclictest25815-21chrt11:31:583
2665199553,37cyclictest27251-21chrt11:33:443
2665199553,36cyclictest23855-21chrt08:33:123
2665199553,34cyclictest24182-21taskset08:33:283
2665199552,39cyclictest525-21taskset08:58:473
2665199552,38cyclictest28133-21chrt09:48:123
2665199552,38cyclictest28133-21chrt09:48:123
2665199543,38cyclictest20682-21chrt12:18:133
2665199543,35cyclictest4538-21taskset07:33:143
2665199543,35cyclictest10903-21chrt11:07:353
2665199542,39cyclictest12085-21taskset12:03:263
2665199533,38cyclictest14590-21taskset09:24:173
2665199533,35cyclictest15169-21taskset08:03:483
2665199533,33cyclictest9597-21chrt11:58:403
2665199533,33cyclictest20935-21taskset12:18:263
2665199533,33cyclictest20935-21taskset12:18:263
2665199533,32cyclictest1879-21chrt09:58:213
2665199532,38cyclictest13235-21chrt09:23:153
2665199532,38cyclictest13235-21chrt09:23:153
2665199532,35cyclictest31472-21chrt07:13:503
2665199532,34cyclictest29581-21taskset11:38:223
2665199532,34cyclictest1900-21chrt11:45:483
2665199532,34cyclictest1900-21chrt11:45:483
2665199523,35cyclictest6998-21chrt09:13:063
2665199523,35cyclictest11640-21chrt11:08:323
2665199523,34cyclictest7772-21chrt11:55:323
2665199523,34cyclictest7772-21chrt11:55:323
2665199522,36cyclictest8416-21chrt07:43:393
2665199522,36cyclictest8416-21chrt07:43:393
2665199514,32cyclictest3734-21chrt10:54:383
2665199513,33cyclictest6913-21taskset10:59:453
2665199513,31cyclictest28743-21taskset10:43:153
2665199513,31cyclictest28743-21taskset10:43:153
2665199512,37cyclictest1840-21taskset09:03:373
2665199512,37cyclictest1840-21taskset09:03:373
2665199512,36cyclictest31776-21chrt10:48:223
2665199512,36cyclictest19051-21taskset08:18:133
2665199512,34cyclictest12796-21chrt10:16:203
2665199512,33cyclictest6063-21taskset10:04:053
2665199512,32cyclictest20339-21chrt10:28:443
2665199504,33cyclictest8172-21chrt09:13:473
2665199503,36cyclictest29474-21chrt09:48:493
2665199503,33cyclictest24084-21taskset12:23:323
2665199503,33cyclictest14597-21chrt11:13:303
2665199503,30cyclictest9636-21chrt07:48:293
2665199493,33cyclictest684-21chrt09:55:383
2665199493,33cyclictest684-21chrt09:55:383
2665199493,33cyclictest27073-21chrt07:03:333
2665199483,30cyclictest23462-21chrt10:34:243
2665199482,35cyclictest17842-21chrt11:18:373
2665199482,35cyclictest17842-21chrt11:18:373
2665199473,35cyclictest3660-21chrt11:48:393
2665199472,35cyclictest30978-21chrt10:46:483
2665199452,30cyclictest29375-21chrt08:48:393
2665199452,30cyclictest29375-21chrt08:48:393
2665199452,30cyclictest20620-21chrt08:23:113
2665199447,26cyclictest18433-21chrt10:25:323
2665199444,28cyclictest15709-21taskset12:08:453
2665199443,30cyclictest32101-21chrt08:58:143
2665199423,27cyclictest32578-21taskset07:18:403
2665199413,25cyclictest26266-21chrt08:38:433
2665199402,28cyclictest12798-21chrt07:58:253
2665199402,26cyclictest20874-21taskset08:23:283
2665199393,27cyclictest12222-21taskset07:54:243
2665199392,27cyclictest5873-21taskset07:35:493
2665199392,27cyclictest5873-21taskset07:35:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional