You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 09:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat Sep 19, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19947212415,80sleep120005-21grep19:05:191
19947212415,80sleep120005-21grep19:05:191
21099991083,75cyclictest343-21runrttasks23:39:200
211119910379,16cyclictest200-21systemd-journal23:13:551
2048321037,55sleep220518-21timerandwakeup19:05:372
2048321037,55sleep220518-21timerandwakeup19:05:372
2062429914,50sleep0359-21snmpd19:05:390
2062429914,50sleep0359-21snmpd19:05:390
21111999638,50cyclictest0-21swapper/121:49:221
21111999638,50cyclictest0-21swapper/121:49:221
2111199952,11cyclictest3007550ssh21:11:161
21099999267,15cyclictest28784-21kworker/u19:2-rpciod22:09:130
290372912,22sleep029098-21fschecks_count21:10:390
2111199913,67cyclictest16126-21apt-get23:29:201
2111199912,6cyclictest0-21swapper/120:54:121
21116999065,15cyclictest15412-21kworker/u19:0+events_unbound19:23:582
21111999048,36cyclictest24549-21interrupts22:49:191
21111999048,36cyclictest24549-21interrupts22:49:191
2111199894,77cyclictest0-21swapper/121:14:061
21111998941,40cyclictest0-21swapper/123:43:571
2111199893,34cyclictest3853-21ssh00:01:251
21111998844,37cyclictest0-21swapper/122:33:571
21111998840,42cyclictest1720-21sh23:58:551
21111998840,42cyclictest1720-21sh23:58:551
21099998857,9cyclictest181rcu_preempt21:49:100
21099998857,9cyclictest181rcu_preempt21:49:100
21116998767,14cyclictest4670-21idleruntime-cro21:23:552
2111199874,37cyclictest13190-21perl23:24:171
21111998736,42cyclictest0-21swapper/123:44:301
2111199873,41cyclictest6854-21sh23:14:101
2111699864,63cyclictest22508-21sh23:39:212
2111199863,37cyclictest1974-21taskset21:19:081
21099998664,16cyclictest31607-21fschecks_count19:39:160
21099998664,16cyclictest31607-21fschecks_count19:39:160
2109999862,42cyclictest31478-21ssh23:54:110
2109999862,42cyclictest31478-21ssh23:54:110
2111199852,6cyclictest0-21swapper/122:09:091
2111199844,63cyclictest14089-21ssh21:39:101
21111998439,39cyclictest0-21swapper/100:24:211
2111199843,70cyclictest23207-21ssh00:34:101
21111998435,40cyclictest30292-21cpuspeed22:59:121
21111998435,40cyclictest30292-21cpuspeed22:59:121
2111199843,38cyclictest9884-21switchtime22:24:301
2111199842,6cyclictest0-21swapper/120:29:101
2111199842,35cyclictest14857-21latency_hist20:28:581
21099998459,19cyclictest24154-21unixbench_singl20:54:330
21099998441,32cyclictest0-21swapper/023:13:120
21111998338,36cyclictest0-21swapper/119:58:591
2111199833,6cyclictest0-21swapper/119:24:091
2111199833,6cyclictest0-21swapper/100:09:091
2111199833,40cyclictest8386-21cron20:08:561
2111199833,40cyclictest6239-21diskstats22:19:151
2111199833,37cyclictest15210-21cstates22:34:131
2111199833,37cyclictest15210-21cstates22:34:131
2111199832,6cyclictest0-21swapper/123:49:071
2109999833,46cyclictest26534-21/usr/sbin/munin21:59:200
2109999833,46cyclictest26534-21/usr/sbin/munin21:59:200
21116998259,17cyclictest17346-21diskstats00:24:162
2111199823,7cyclictest0-21swapper/121:34:101
2111199823,7cyclictest0-21swapper/100:29:061
2111199823,15cyclictest9894-21ssh23:19:091
2111199822,66cyclictest21319-21wc19:09:121
21099998260,16cyclictest27789-21diskmemload23:33:570
2109999824,34cyclictest343-21runrttasks00:07:170
2109999824,34cyclictest343-21runrttasks00:07:170
21116998158,17cyclictest17387-21fschecks_time21:44:182
2111199813,7cyclictest0-21swapper/121:24:081
21111998137,35cyclictest0-21swapper/122:41:421
2111199813,6cyclictest0-21swapper/120:59:101
2111199813,6cyclictest0-21swapper/120:59:101
21111998135,35cyclictest0-21swapper/120:14:171
21111998135,35cyclictest0-21swapper/120:14:171
2111199813,35cyclictest22273-21switchtime20:49:291
2111199813,32cyclictest21493-21latency_hist20:48:591
2111199813,31cyclictest26764-21ntp_states21:59:261
2111199813,31cyclictest26764-21ntp_states21:59:261
21111998132,42cyclictest0-21swapper/120:19:181
21111998132,42cyclictest0-21swapper/120:19:181
2111199812,39cyclictest8596-21taskset21:29:241
2111199812,37cyclictest30936-21runrttasks22:06:211
21099998159,14cyclictest23582-21rm00:34:220
21099998157,17cyclictest24013-21latency_hist22:48:580
21099998152,9cyclictest181rcu_preempt19:44:110
21099998152,9cyclictest181rcu_preempt19:44:110
2109999814,15cyclictest27789-21diskmemload23:49:080
21099998137,22cyclictest27651-21ssh23:47:400
21099998123,13cyclictest181rcu_preempt21:23:590
2111199803,6cyclictest0-21swapper/119:44:111
2111199803,6cyclictest0-21swapper/119:44:111
2111199803,36cyclictest23229-21/usr/sbin/munin21:54:111
2111199803,30cyclictest1467-21grep23:04:281
2111199803,29cyclictest343-21runrttasks19:52:231
2111199802,35cyclictest30113-21/usr/sbin/munin19:34:211
2111199802,35cyclictest30113-21/usr/sbin/munin19:34:211
2111199802,34cyclictest5115-21diskmemload00:04:111
2111199802,34cyclictest5115-21diskmemload00:04:111
2111199802,33cyclictest19666-21/usr/sbin/munin23:34:231
2111199802,32cyclictest5890-21/usr/sbin/munin19:59:221
2111199802,30cyclictest27470-21tune2fs22:54:171
21099998060,14cyclictest343-21runrttasks23:04:120
2053828029,32sleep30-21swapper/319:05:383
2053828029,32sleep30-21swapper/319:05:383
2112599793,25cyclictest16733-21chrt20:34:083
21116997960,12cyclictest16158-21unixbench_singl22:34:322
21116997960,12cyclictest16158-21unixbench_singl22:34:322
21116997956,15cyclictest27789-21diskmemload23:26:342
2111199793,37cyclictest14162-21/usr/sbin/munin00:19:111
2111199793,37cyclictest14162-21/usr/sbin/munin00:19:111
2111199793,32cyclictest28300-21expr19:29:121
2111199793,32cyclictest21711-21/usr/sbin/munin22:44:221
2111199793,31cyclictest27322-21grep21:04:321
2111199792,36cyclictest24736-21lockfile-remove19:18:561
2111199792,32cyclictest25233-21tune2fs19:19:171
21099997961,11cyclictest14779-21munin-run22:33:550
21099997955,17cyclictest7042-21/usr/sbin/munin20:04:310
21116997849,20cyclictest22804-21idleruntime-cro21:53:562
21116997849,20cyclictest22804-21idleruntime-cro21:53:562
21116997848,24cyclictest4002-21sed22:15:352
21116997829,9cyclictest381ktimers/200:14:102
2111199784,30cyclictest17257-21/usr/sbin/munin20:34:251
2111199783,6cyclictest0-21swapper/121:44:071
2111199783,36cyclictest343-21runrttasks00:14:231
2111199783,32cyclictest31921-21/usr/sbin/munin19:39:231
2111199783,32cyclictest31921-21/usr/sbin/munin19:39:231
2111199782,7cyclictest0-21swapper/120:39:081
2111199782,35cyclictest9996-21kworker/u18:120:13:561
2111199782,31cyclictest3005-21apt-get22:14:081
21099997845,12cyclictest181rcu_preempt20:24:320
21116997759,11cyclictest15341-21fschecks_time20:29:172
21116997757,14cyclictest3078-21sh23:07:482
21116997754,17cyclictest2577-21apt-key22:13:582
2111699774,31cyclictest24833-21latency_hist19:18:592
21099997749,9cyclictest181rcu_preempt19:49:070
21099997737,12cyclictest0-21swapper/019:33:560
21116997655,15cyclictest26064-21latency_hist00:38:592
21116997653,17cyclictest5242-21fschecks_time21:24:162
21099997652,17cyclictest17090-21kernelversion20:34:210
21116997558,10cyclictest4973-21latency_hist00:03:582
21116997550,19cyclictest5634-21fschecks_count19:59:172
21116997531,33cyclictest4739-21timerandwakeup23:09:312
2109999756,14cyclictest4569-21timerandwakeup19:54:300
21099997554,15cyclictest8842-21fschecks_count20:09:170
21099997545,21cyclictest20482-21cut00:29:180
2109999753,55cyclictest13932-21grep23:24:320
2109999753,42cyclictest25832-21idleruntime-cro21:58:550
2109999753,36cyclictest30357-21runrttasks22:59:120
2109999753,36cyclictest30357-21runrttasks22:59:120
21116997458,10cyclictest28135-21modprobe23:48:552
21116997453,15cyclictest15563-21cron23:29:202
2111699744,54cyclictest5705-21ntp_states00:04:262
2111699744,54cyclictest5705-21ntp_states00:04:262
21116997358,9cyclictest3159-21kworker/u17:2-events_unbound00:33:552
21116997352,15cyclictest21730-21ntp_kernel_pll_22:44:232
21099997357,11cyclictest21090-21cyclictest22:19:130
21099997353,14cyclictest27951-21latency_hist21:08:580
21099997351,14cyclictest1-21systemd19:17:390
2109999734,38cyclictest26337-21modprobe19:23:560
2109999734,37cyclictest22027-21sed23:38:580
21116997255,11cyclictest10011-21ssh23:19:142
21116997245,9cyclictest184512sort22:39:202
21099997246,17cyclictest0-21swapper/019:28:580
21099997245,17cyclictest0-21swapper/022:04:130
21099997242,23cyclictest0-21swapper/000:27:470
21099997240,24cyclictest0-21swapper/023:19:160
21099997229,25cyclictest181rcu_preempt22:38:550
21099997229,25cyclictest181rcu_preempt22:38:550
2109999722,42cyclictest9958-21cat22:24:310
21116997151,13cyclictest10462-21cut20:14:162
21116997151,13cyclictest10462-21cut20:14:162
21099997149,16cyclictest9607-21ssh23:18:580
2109999714,33cyclictest8377-21fschecks_time00:09:170
2109999713,32cyclictest31655-21fschecks_time21:14:170
21099997127,12cyclictest181rcu_preempt22:53:570
21099997127,12cyclictest181rcu_preempt22:53:570
21099997052,11cyclictest22895-21munin-node21:19:330
2109999704,56cyclictest8014-21ssh21:29:100
2109999702,51cyclictest17872-21switchtime21:44:300
21116996951,12cyclictest27789-21diskmemload23:34:182
21099996946,15cyclictest161ktimers/020:19:060
21099996946,15cyclictest161ktimers/020:19:060
2109999693,55cyclictest14115-21ssh00:19:050
2109999693,55cyclictest14115-21ssh00:19:050
2109999693,31cyclictest5636-21fschecks_count19:59:170
21099996921,8cyclictest0-21swapper/021:39:330
21099996913,8cyclictest0-21swapper/020:59:100
21099996913,8cyclictest0-21swapper/020:59:100
21099996913,39cyclictest4348-21timerandwakeup22:15:430
253842680,7sleep30-21swapper/323:44:023
2112599684,32cyclictest13306-21chrt23:24:203
21116996847,15cyclictest26866-21idleruntime-cro22:53:562
21116996847,15cyclictest26866-21idleruntime-cro22:53:562
21116996840,22cyclictest0-21swapper/220:09:182
2111699683,45cyclictest343-21runrttasks22:01:172
2111699683,45cyclictest343-21runrttasks22:01:172
2111699683,44cyclictest465-21latency_hist23:03:592
2111699683,44cyclictest465-21latency_hist23:03:592
2111699682,34cyclictest39-21ksoftirqd/223:55:422
2111699682,34cyclictest39-21ksoftirqd/223:55:422
21116996812,13cyclictest39-21ksoftirqd/223:53:582
2109999684,33cyclictest2191-21cstates23:59:120
2109999683,49cyclictest11632-21cron20:18:560
2109999683,49cyclictest11632-21cron20:18:560
2109999683,32cyclictest343-21runrttasks21:38:400
21099996826,32cyclictest19139-21timerandwakeup20:39:300
2109999682,25cyclictest18475-21/usr/sbin/munin22:39:210
2112599673,27cyclictest23282-21chrt21:54:133
21116996751,9cyclictest23194-21cpuspeed21:54:112
21116996747,13cyclictest21090-21cyclictest20:34:142
21116996745,12cyclictest0-21swapper/222:24:132
2111699674,39cyclictest4057-21fschecks_time19:54:172
2111699673,44cyclictest29420-21/usr/sbin/munin21:10:452
2111699673,40cyclictest690-21cstates19:44:132
2111699673,40cyclictest690-21cstates19:44:132
2111699673,39cyclictest14350-21ssh22:32:472
2111699672,44cyclictest2472-21fschecks_time19:49:162
21116996710,43cyclictest28237-21grep22:54:322
21099996751,10cyclictest29532-21ssh22:58:020
2109999673,26cyclictest22771-21latency_hist19:13:580
21099996726,10cyclictest181rcu_preempt20:29:120
2112599663,23cyclictest31869-21chrt19:39:203
2112599663,23cyclictest31869-21chrt19:39:203
21116996642,15cyclictest27789-21diskmemload00:23:582
21116996642,15cyclictest27789-21diskmemload00:23:582
21116996640,17cyclictest343-21runrttasks00:13:562
21116996623,8cyclictest371rcuc/222:04:292
21099996649,11cyclictest21090-21cyclictest00:17:270
2112599654,28cyclictest29153-21chrt22:04:053
2112599653,28cyclictest5288-21chrt00:04:163
2112599653,28cyclictest5288-21chrt00:04:163
2112599653,25cyclictest24132-21taskset23:41:143
2112599653,24cyclictest29649-21chrt23:49:493
2112599652,29cyclictest6205-21chrt19:59:373
2112599652,25cyclictest4260-21chrt23:09:213
2112599652,25cyclictest11200-21chrt21:34:153
21125996511,45cyclictest0-21swapper/323:18:553
21116996545,13cyclictest9554-21modprobe23:18:552
2111699654,46cyclictest14473-21grep21:39:212
21116996542,14cyclictest0-21swapper/222:19:172
2111699653,41cyclictest26168-21latency_hist21:03:552
2111699653,41cyclictest26168-21latency_hist21:03:552
2111699652,45cyclictest12321-21latency20:19:212
2111699652,45cyclictest12321-21latency20:19:212
2111699652,40cyclictest14846-21modprobe20:28:552
2112599643,29cyclictest19102-21chrt22:39:353
2112599643,23cyclictest29872-21chrt19:34:143
2112599643,23cyclictest29872-21chrt19:34:143
2112599643,23cyclictest15451-21chrt00:20:083
2112599643,23cyclictest15451-21chrt00:20:083
2112599643,20cyclictest28630-21chrt19:29:203
2112599642,25cyclictest21673-21taskset22:44:213
2112599642,24cyclictest9644-21chrt20:09:343
2112599642,24cyclictest10645-21chrt23:19:303
21116996444,15cyclictest1636-21sh21:18:562
2111699643,46cyclictest18550-21fschecks_count20:39:162
2111699643,40cyclictest21549-21fschecks_time19:09:172
2111699642,45cyclictest26730-21sort21:04:192
2111699642,41cyclictest23623-21latency20:54:222
21099996439,16cyclictest21790-21fschecks_count20:49:170
21099996423,12cyclictest181rcu_preempt19:34:300
21099996423,12cyclictest181rcu_preempt19:34:300
2112599633,26cyclictest7069-21chrt20:04:113
2112599633,24cyclictest3246-21chrt22:14:473
2112599633,24cyclictest15403-21chrt20:29:183
2112599633,23cyclictest32417-21chrt23:54:323
2112599633,23cyclictest32417-21chrt23:54:323
2112599633,22cyclictest30403-21chrt21:12:113
2112599633,21cyclictest4068-21chrt21:22:203
2112599633,21cyclictest25510-21chrt19:19:243
2112599632,25cyclictest7249-21chrt21:27:473
2112599632,24cyclictest23304-21chrt00:34:143
2112599632,23cyclictest31669-21chrt21:14:173
21125996310,44cyclictest0-21swapper/319:48:553
21125996310,44cyclictest0-21swapper/319:48:553
2111699634,23cyclictest8208-21cut21:29:162
21099996342,11cyclictest0-21swapper/020:48:580
2112599629,44cyclictest0-21swapper/300:24:293
2112599629,44cyclictest0-21swapper/300:13:573
2112599623,26cyclictest1595-21taskset22:11:353
2112599623,24cyclictest8288-21chrt22:22:423
2112599623,24cyclictest2088-21chrt23:59:093
2112599623,24cyclictest16223-21chrt22:34:373
2112599623,24cyclictest16223-21chrt22:34:373
2112599622,28cyclictest25759-21taskset20:59:313
2112599622,28cyclictest25759-21taskset20:59:313
2112599622,28cyclictest22070-21taskset00:31:473
2112599622,28cyclictest19453-21taskset21:48:053
2112599622,26cyclictest2459-21taskset23:06:053
2112599622,24cyclictest7939-21chrt21:29:013
2112599622,24cyclictest31910-21chrt23:01:113
2112599622,24cyclictest31910-21chrt23:01:113
2112599622,24cyclictest19970-21chrt20:44:083
2112599622,24cyclictest16182-21chrt21:42:223
2112599622,23cyclictest9259-21chrt22:24:163
2112599622,23cyclictest21413-21taskset23:37:463
2112599622,23cyclictest18202-21taskset23:32:153
2112599622,22cyclictest27413-21chrt21:04:333
2112599622,22cyclictest22602-21taskset20:49:593
21125996210,42cyclictest0-21swapper/322:54:153
21116996240,11cyclictest0-21swapper/220:48:582
2111699623,45cyclictest11272-21cut21:34:192
2111699623,39cyclictest343-21runrttasks19:26:512
2111699623,31cyclictest21752-21fschecks_count20:49:162
21116996228,14cyclictest181rcu_preempt19:29:162
2112599613,25cyclictest11455-21chrt20:16:593
2112599613,25cyclictest11455-21chrt20:16:593
2112599613,24cyclictest25433-21chrt22:50:013
2112599613,24cyclictest25433-21chrt22:50:013
2112599613,22cyclictest23684-21chrt19:14:223
2112599613,21cyclictest11890-21chrt20:19:053
2112599613,21cyclictest11890-21chrt20:19:053
2112599612,25cyclictest4978-21chrt19:56:503
2112599612,24cyclictest21973-21chrt21:51:423
2112599612,24cyclictest21973-21chrt21:51:423
2112599612,23cyclictest21461-21chrt19:09:163
2112599612,23cyclictest12070-21chrt22:29:063
2112599612,22cyclictest19629-21chrt20:43:113
2112599612,22cyclictest13092-21chrt00:17:003
2111699613,22cyclictest8401-21idleruntime-cro20:08:562
2111699612,30cyclictest29784-21/usr/sbin/munin19:34:152
2111699612,30cyclictest29784-21/usr/sbin/munin19:34:152
2112599603,20cyclictest3304-21chrt19:50:593
2112599602,22cyclictest27634-21chrt19:25:193
2112599602,22cyclictest24289-21taskset20:56:033
2111699603,45cyclictest32285-21timerwakeupswit19:39:302
2111699603,45cyclictest32285-21timerwakeupswit19:39:302
2112599593,21cyclictest27346-21chrt21:59:483
2112599593,21cyclictest27346-21chrt21:59:483
2112599592,23cyclictest14794-21chrt20:28:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional