You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 21:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Mon Jun 08, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25441991173,78cyclictest30736-21apt-get07:14:561
25415991163,58cyclictest16089-21cstates09:25:250
25415991163,58cyclictest16089-21cstates09:25:250
25415991156,34cyclictest4012-21runrttasks10:52:520
254159911231,14cyclictest0-21swapper/011:14:530
254159911158,48cyclictest25393-21cyclictest12:15:460
254159911032,15cyclictest0-21swapper/010:44:320
25415991102,53cyclictest9065-21latency11:54:570
25415991102,53cyclictest9065-21latency11:54:570
254159910934,10cyclictest0-21swapper/009:10:050
254159910934,10cyclictest0-21swapper/009:10:050
25415991084,29cyclictest1796-21seq11:43:080
254159910831,14cyclictest0-21swapper/012:24:550
25415991073,40cyclictest18974-21grep09:29:500
254159910728,17cyclictest0-21swapper/009:50:030
254159910728,17cyclictest0-21swapper/009:50:030
254419910636,40cyclictest0-21swapper/110:30:051
254159910644,55cyclictest24533-21ssh11:26:360
25415991064,26cyclictest0-21swapper/006:59:470
25415991064,26cyclictest0-21swapper/006:59:470
254159910629,22cyclictest30732-21modprobe07:14:300
254159910629,22cyclictest30732-21modprobe07:14:300
254159910612,34cyclictest22132-21if_eth009:34:540
254159910533,12cyclictest0-21swapper/011:49:510
25415991052,43cyclictest20411-21cat08:19:540
254159910436,10cyclictest0-21swapper/011:31:040
25415991042,73cyclictest352-21runrttasks07:14:590
25415991042,37cyclictest16791-21acpi08:09:400
25415991042,26cyclictest5483-21cat11:49:340
25415991042,25cyclictest28867-21sshd07:06:370
24648210414,60sleep124731-21grep06:55:161
25415991033,42cyclictest27609-21cut09:44:330
25415991033,42cyclictest27609-21cut09:44:330
25415991033,36cyclictest16241-21/usr/sbin/munin10:19:510
25415991033,36cyclictest16241-21/usr/sbin/munin10:19:510
25415991023,37cyclictest2225-21/usr/sbin/munin07:24:580
25415991013,37cyclictest29679-21/usr/sbin/munin11:35:000
25415991013,37cyclictest29679-21/usr/sbin/munin11:35:000
25415991013,36cyclictest28820-21grep09:45:080
254159910133,13cyclictest0-21swapper/012:09:330
25441991004,80cyclictest25739-21runrttasks06:59:551
25441991004,80cyclictest25739-21runrttasks06:59:551
254159910039,54cyclictest0-21swapper/008:49:460
25415991003,34cyclictest25540-21latency_hist06:59:350
25415991003,22cyclictest10400-21/usr/sbin/munin10:09:450
254159910010,34cyclictest12797-21fschecks_time09:19:510
25453999981,12cyclictest17221-21ssh10:20:062
25453999981,12cyclictest17221-21ssh10:20:062
2541599995,42cyclictest32585-21apt-get07:19:450
2541599994,38cyclictest28156-21latency_hist10:39:320
2541599994,38cyclictest13575-21fschecks_time10:14:520
25415999934,14cyclictest3529-21rm09:04:400
2541599993,37cyclictest4890-21/usr/sbin/munin10:55:030
2541599993,37cyclictest28356-21fschecks_time08:44:530
2541599993,36cyclictest9644-21sed07:45:080
2541599993,35cyclictest18997-21ps08:15:020
25415999933,15cyclictest10407-21ssh09:15:050
25415999932,11cyclictest0-21swapper/011:59:580
2541599992,35cyclictest8430-21latency_hist07:44:330
25441999870,22cyclictest3576-21ssh09:04:461
2541599983,39cyclictest1-21systemd08:09:310
2541599983,37cyclictest5932-21needreboot07:34:560
2541599983,37cyclictest27970-21latency_hist08:44:330
2541599983,35cyclictest352-21runrttasks08:00:380
2541599983,35cyclictest19911-21/usr/sbin/munin11:19:590
25415999815,31cyclictest25723-21timerwakeupswit08:35:040
2541599974,35cyclictest3862-21/usr/sbin/munin07:30:010
2541599973,39cyclictest4908-21ntpq10:00:010
2541599973,36cyclictest21614-21/usr/sbin/munin08:25:080
2541599973,36cyclictest21614-21/usr/sbin/munin08:25:080
25453999674,15cyclictest28873-21ssh10:40:092
25441999634,38cyclictest0-21swapper/110:09:571
2541599963,35cyclictest16713-21latency_hist11:14:310
2541599963,32cyclictest10508-21fschecks_count07:49:500
2541599963,32cyclictest10508-21fschecks_count07:49:500
25453999566,21cyclictest19837-21latency_hist11:19:332
2541599954,32cyclictest31351-21cpu08:54:460
2541599944,35cyclictest352-21runrttasks10:09:200
2541599943,38cyclictest21881-21cron10:29:290
2541599942,23cyclictest32013-21ntp_states10:45:000
25415999412,28cyclictest12712-21timerandwakeup07:55:060
25415999412,28cyclictest12712-21timerandwakeup07:55:060
25453999364,21cyclictest11283-21unixbench_multi10:10:052
25453999264,20cyclictest26812-21sh11:31:042
25453999262,20cyclictest22171-21sh09:34:552
2544199927,76cyclictest87-21kswapd010:39:501
2541599923,35cyclictest24906-21munin-run10:34:300
2541599913,38cyclictest1711-21grep09:54:560
25441999064,18cyclictest32690-21diskmemload11:15:041
2544199903,63cyclictest16153-21ssh09:25:271
2544199903,63cyclictest16153-21ssh09:25:271
2541599903,33cyclictest13823-21latency_hist11:09:320
2541599903,33cyclictest13823-21latency_hist11:09:320
25415999017,15cyclictest0-21swapper/008:30:050
2407029024,39sleep30-21swapper/306:54:583
25441998945,39cyclictest25393-21cyclictest11:00:021
25453998865,16cyclictest32059-21threads07:15:512
25441998845,38cyclictest0-21swapper/112:04:291
25453998760,18cyclictest18467-21timerwakeupswit12:10:082
2527528715,45sleep225241-21munin-run06:59:332
25453998665,13cyclictest15238-21systemd-run09:24:312
25441998661,17cyclictest30740-21latency_hist07:14:331
25441998661,17cyclictest30740-21latency_hist07:14:331
25441998623,35cyclictest0-21swapper/110:50:101
25453998556,11cyclictest181rcu_preempt09:30:462
25453998547,13cyclictest181rcu_preempt11:40:072
25441998538,37cyclictest0-21swapper/108:39:311
25453998463,15cyclictest10971-21sh11:04:382
25453998463,15cyclictest10971-21sh11:04:382
25441998444,34cyclictest2374-21systemd-journal09:52:361
25441998444,34cyclictest2374-21systemd-journal09:52:361
25441998353,21cyclictest29466-21idleruntime-cro08:49:301
25441998335,40cyclictest0-21swapper/111:54:471
25441998335,40cyclictest0-21swapper/111:54:471
2544199833,51cyclictest27030-21ssh10:37:061
25453998265,10cyclictest14342-21unixbench_singl10:15:062
25453998245,12cyclictest181rcu_preempt09:55:052
2544199823,70cyclictest0-21swapper/109:29:411
25441998233,41cyclictest0-21swapper/110:05:001
2544199823,29cyclictest14953-21timerwakeupswit11:10:071
25415998266,9cyclictest18475-21rm12:10:080
2545399812,33cyclictest39-21ksoftirqd/207:49:402
2545399812,33cyclictest39-21ksoftirqd/207:49:402
2544199813,63cyclictest0-21swapper/108:29:431
25453998057,12cyclictest0-21swapper/210:55:052
25453998053,10cyclictest6890-21ssh10:04:202
25453998043,14cyclictest181rcu_preempt07:39:502
25453998030,23cyclictest0-21swapper/212:04:312
2544199805,62cyclictest1284-21apt-get09:54:461
2544199804,70cyclictest0-21swapper/108:14:421
25441998033,35cyclictest0-21swapper/108:19:511
25441998026,48cyclictest0-21swapper/109:44:541
25453997938,15cyclictest181rcu_preempt09:14:332
25453997938,15cyclictest181rcu_preempt09:14:332
25453997937,10cyclictest0-21swapper/209:25:452
25453997937,10cyclictest0-21swapper/209:25:452
25441997960,11cyclictest25627-21gpgv11:29:321
25441997956,17cyclictest300-21rs:main1
25441997938,34cyclictest0-21swapper/109:03:291
25441997938,34cyclictest0-21swapper/109:03:291
2544199793,69cyclictest0-21swapper/111:34:431
2544199793,69cyclictest0-21swapper/111:34:431
2544199793,67cyclictest23577-21awk12:19:471
2544199792,61cyclictest0-21swapper/107:44:441
25441997925,49cyclictest11277-21fschecks_time11:04:511
25441997925,49cyclictest11277-21fschecks_time11:04:511
25453997853,9cyclictest181rcu_preempt08:25:072
25453997853,9cyclictest181rcu_preempt08:25:072
2544199783,62cyclictest0-21swapper/108:04:411
25441997833,37cyclictest0-21swapper/107:20:191
25441997821,32cyclictest0-21swapper/112:28:541
25415997862,10cyclictest19676-21snmpd10:59:550
2545999773,61cyclictest1058-21chrt09:54:323
2545999773,61cyclictest1058-21chrt09:54:323
25453997749,17cyclictest0-21swapper/211:00:232
25453997745,22cyclictest20577-21ls12:14:462
25441997750,18cyclictest18923-21latency_hist10:24:321
25441997750,18cyclictest18923-21latency_hist10:24:321
2544199773,68cyclictest0-21swapper/110:14:441
25453997653,8cyclictest181rcu_preempt08:10:382
25453997651,9cyclictest5759-21ssh11:49:482
25441997650,20cyclictest27646-21fschecks_count07:04:421
25441997634,35cyclictest0-21swapper/109:14:441
2544199762,68cyclictest0-21swapper/108:54:431
25453997552,15cyclictest28011-21apt-key10:39:322
25453997545,11cyclictest181rcu_preempt08:44:412
25453997543,10cyclictest181rcu_preempt11:56:042
25453997543,10cyclictest181rcu_preempt11:56:042
25441997561,10cyclictest25393-21cyclictest08:44:301
2544199753,66cyclictest0-21swapper/111:49:421
2544199753,66cyclictest0-21swapper/111:39:431
25441997536,31cyclictest0-21swapper/111:29:411
25441997535,32cyclictest0-21swapper/109:39:311
25441997534,33cyclictest0-21swapper/112:04:561
25441997513,53cyclictest13572-21awk09:20:061
25453997451,8cyclictest181rcu_preempt08:04:442
25453997450,9cyclictest373-21lldpd10:05:082
2544199743,65cyclictest0-21swapper/107:39:421
2544199743,57cyclictest3880-21apt-get07:29:451
2544199743,47cyclictest22364-21ssh11:23:531
2545999734,56cyclictest27230-21chrt08:40:053
2545399733,38cyclictest27549-21latency_hist09:44:332
2545399733,38cyclictest27549-21latency_hist09:44:332
2545399731,36cyclictest20621-21ssh10:25:542
2544199733,57cyclictest10333-21apt-get07:49:401
2544199733,57cyclictest10333-21apt-get07:49:401
25441997323,41cyclictest18320-21cut08:14:331
25441997315,50cyclictest5461-21seq07:34:381
2545999724,54cyclictest31454-21taskset08:54:473
2545999723,37cyclictest0-21swapper/309:25:253
2545999723,37cyclictest0-21swapper/309:25:253
2545999722,57cyclictest1148-21chrt11:41:173
25453997256,12cyclictest16103-21kworker/u17:2+rpciod11:14:322
25453997250,12cyclictest0-21swapper/212:04:562
25453997249,8cyclictest181rcu_preempt09:04:002
25453997249,8cyclictest181rcu_preempt09:04:002
25453997245,7cyclictest181rcu_preempt11:44:502
25453997236,17cyclictest181rcu_preempt08:34:302
25453997233,12cyclictest181rcu_preempt10:46:202
2544199724,62cyclictest0-21swapper/108:24:431
2544199724,62cyclictest0-21swapper/108:24:431
2544199724,58cyclictest4273-21apt-get09:59:451
25441997227,37cyclictest0-21swapper/110:25:061
2545999713,33cyclictest31262-21chrt07:14:573
25453997145,10cyclictest181rcu_preempt09:04:422
25453997144,9cyclictest181rcu_preempt08:14:462
25453997115,23cyclictest6858-21munin-run07:39:322
25441997128,38cyclictest0-21swapper/112:09:581
25441997126,40cyclictest14586-21sshd08:00:371
2545999703,55cyclictest22930-21chrt09:35:203
2545399708,51cyclictest630-21apt-get07:19:462
25453997053,11cyclictest16103-21kworker/u17:2-flush-0:3211:38:412
25453997053,11cyclictest16103-21kworker/u17:2-flush-0:3211:38:412
25453997039,13cyclictest181rcu_preempt10:30:432
25453997035,16cyclictest181rcu_preempt07:24:582
25453997014,11cyclictest181rcu_preempt11:24:542
2544199703,57cyclictest6553-21apt-get09:09:411
2544199703,57cyclictest6553-21apt-get09:09:411
25441997031,19cyclictest181rcu_preempt09:40:041
25441997031,19cyclictest181rcu_preempt09:40:041
2544199702,55cyclictest29726-21apt-get08:49:401
25453996948,14cyclictest381ktimers/208:54:452
25453996942,15cyclictest0-21swapper/209:54:322
25453996942,15cyclictest0-21swapper/209:54:322
25453996936,11cyclictest20821-21ssh11:20:072
2545399693,37cyclictest26544-21sort12:24:462
2544199692,57cyclictest11952-21apt-get07:54:471
2544199692,57cyclictest11952-21apt-get07:54:471
25441996918,12cyclictest181rcu_preempt10:54:541
2545999683,34cyclictest14888-21taskset10:16:333
2545999683,33cyclictest4923-21chrt10:54:433
2545999682,34cyclictest19403-21chrt11:18:553
25453996851,11cyclictest803-21sendmail-mta09:46:562
2545399684,44cyclictest21358-21modprobe08:24:302
25453996843,11cyclictest181rcu_preempt08:40:042
25453996837,8cyclictest181rcu_preempt08:50:052
25453996828,12cyclictest181rcu_preempt07:59:572
25441996843,15cyclictest0-21swapper/107:24:521
175042680,6sleep30-21swapper/308:10:043
2545999673,31cyclictest28064-21taskset12:26:373
2545999673,31cyclictest12536-21chrt11:06:223
2545999673,31cyclictest12536-21chrt11:06:223
2545999673,30cyclictest4368-21chrt11:47:003
2545399674,45cyclictest25831-21unixbench_singl08:35:052
25453996736,12cyclictest181rcu_preempt07:29:422
25453996720,10cyclictest181rcu_preempt07:11:312
25453996720,10cyclictest181rcu_preempt07:11:312
25415996751,11cyclictest25393-21cyclictest09:02:450
25415996751,11cyclictest25393-21cyclictest09:02:450
2545999662,32cyclictest6531-21chrt07:35:503
2545999662,32cyclictest30863-21taskset07:14:323
2545999662,32cyclictest30863-21taskset07:14:323
2545999662,32cyclictest21044-21taskset10:27:063
2545999662,31cyclictest7429-21chrt11:52:113
2545999662,31cyclictest2691-21chrt09:03:023
2545999662,31cyclictest2691-21chrt09:03:023
2545999662,31cyclictest18963-21chrt10:24:313
2545999662,31cyclictest18963-21chrt10:24:313
2545999662,30cyclictest13704-21chrt07:59:483
25453996636,12cyclictest181rcu_preempt10:50:442
2545399663,34cyclictest26159-21latency07:00:202
2545399663,34cyclictest26159-21latency07:00:202
2544199662,52cyclictest31367-21ssh10:44:381
2545999653,32cyclictest7099-21chrt07:39:433
2545999653,32cyclictest10976-21chrt09:16:033
2545999653,31cyclictest644-21chrt07:19:463
2545999653,31cyclictest2469-21chrt09:55:223
2545999653,31cyclictest22900-21chrt12:18:503
2545999653,30cyclictest30371-21chrt10:42:303
2545999652,32cyclictest9634-21chrt10:08:383
2545999652,32cyclictest8244-21chrt09:11:453
2545999652,32cyclictest8244-21chrt09:11:453
2545999652,32cyclictest3329-21chrt07:26:233
2545999652,31cyclictest30759-21chrt11:36:383
2545999652,31cyclictest30759-21chrt11:36:383
2545999652,31cyclictest16391-21chrt11:13:483
25453996549,11cyclictest11933-21/usr/sbin/munin07:55:082
25453996549,11cyclictest11933-21/usr/sbin/munin07:55:082
25453996540,13cyclictest0-21swapper/207:45:062
25453996540,12cyclictest0-21swapper/212:19:472
2544199653,53cyclictest20513-21ssh12:14:451
2545999645,48cyclictest0-21swapper/309:39:403
2545999645,48cyclictest0-21swapper/309:39:403
2545999643,31cyclictest14839-21chrt09:23:323
2545999642,32cyclictest29884-21taskset08:49:493
2545999642,32cyclictest17005-21taskset12:09:013
2545999642,30cyclictest4223-21chrt07:29:553
2545999642,30cyclictest3916-21chrt10:52:363
25453996442,8cyclictest181rcu_preempt07:04:412
25453996415,35cyclictest39-21ksoftirqd/209:16:452
25415996438,10cyclictest181rcu_preempt12:23:210
2545999633,30cyclictest24679-21taskset10:33:553
2545999633,29cyclictest25994-21chrt08:36:453
2545999633,22cyclictest9438-21chrt07:45:063
2545999632,31cyclictest13870-21taskset12:03:343
2545999632,30cyclictest4776-21chrt09:05:293
2545999632,30cyclictest29057-21chrt07:09:053
2545999623,33cyclictest19536-21chrt09:30:253
2545999623,27cyclictest24567-21chrt11:26:413
2545999623,22cyclictest6820-21chrt10:04:093
2545999622,33cyclictest16221-21taskset08:05:593
2545999622,29cyclictest32515-21chrt10:45:103
2545999613,26cyclictest29832-21chrt09:47:563
2545999612,33cyclictest24325-21taskset08:31:023
2545999612,29cyclictest12942-21chrt07:55:103
2545999612,29cyclictest12942-21chrt07:55:103
2545999612,27cyclictest21174-21chrt08:22:113
2545999612,22cyclictest26107-21chrt11:29:373
2545999603,31cyclictest21264-21chrt11:20:453
2545999603,26cyclictest8180-21chrt10:59:583
2545999602,30cyclictest10186-21taskset11:56:333
2545999602,30cyclictest10186-21taskset11:56:333
2545999602,28cyclictest27424-21chrt10:38:123
2545999593,41cyclictest10625-21chrt10:09:523
2545999592,31cyclictest23635-21taskset12:19:483
2545999592,31cyclictest11642-21taskset07:54:093
2545999592,31cyclictest11642-21taskset07:54:093
2545999592,28cyclictest19521-21chrt08:16:363
2545999584,23cyclictest21791-21chrt08:24:503
2545999584,23cyclictest21791-21chrt08:24:503
2545999583,23cyclictest26961-21taskset07:03:263
2545999583,23cyclictest26961-21taskset07:03:263
2545999582,30cyclictest29359-21taskset08:48:443
2545999552,21cyclictest17880-21chrt12:09:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional