You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 22:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Tue May 26, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15607991053,37cyclictest31536-21apt-get11:55:390
15621991014,83cyclictest6827-21apt-get10:20:382
156219910056,39cyclictest15595-21cyclictest10:55:512
15621999475,13cyclictest2374-21systemd-journal11:44:402
244672930,10sleep3451rcuc/311:43:583
1560799933,75cyclictest5651-21ssh09:25:420
1560799913,55cyclictest624-21apt-get10:10:390
1562199903,7cyclictest0-21swapper/211:50:382
1562199902,31cyclictest352-21runrttasks08:40:032
1562199894,6cyclictest0-21swapper/210:35:332
15613998965,15cyclictest14841-21mailstats12:20:561
1475428912,45sleep014798-21mailstats06:56:070
15621998745,36cyclictest30328-21latency10:05:472
15621998744,38cyclictest0-21swapper/208:15:252
15621998744,38cyclictest0-21swapper/208:15:252
15621998740,40cyclictest0-21swapper/212:15:422
15607998762,11cyclictest181rcu_preempt10:40:240
15607998753,13cyclictest181rcu_preempt08:35:440
1562199864,7cyclictest0-21swapper/207:25:342
1562199863,41cyclictest13297-21/usr/sbin/munin11:25:322
1562199863,30cyclictest352-21runrttasks07:00:582
15613998664,14cyclictest300-21rs:main1
15621998544,33cyclictest22519-21diskmemload12:25:252
1562199854,39cyclictest25858-21/usr/sbin/munin10:51:002
1562199854,39cyclictest25858-21/usr/sbin/munin10:51:002
1562199854,36cyclictest22842-21grep10:45:582
15621998540,37cyclictest0-21swapper/211:56:432
1562199853,9cyclictest0-21swapper/210:15:342
1562199853,9cyclictest0-21swapper/210:15:342
1562199853,74cyclictest755-21sh10:10:542
1562199853,44cyclictest8284-21apt-get09:30:382
15621998532,48cyclictest0-21swapper/209:16:002
15621998461,17cyclictest300-21rs:main2
1562199843,8cyclictest0-21swapper/209:45:352
15621998434,43cyclictest0-21swapper/211:05:252
1562199843,33cyclictest31460-21sed07:45:502
1562199842,8cyclictest0-21swapper/208:55:332
15607998458,20cyclictest17853-21latency_hist08:45:260
15607998456,12cyclictest181rcu_preempt11:50:430
1560799843,51cyclictest8284-21apt-get09:30:400
141202845,53sleep214131-21apt-get06:55:362
15621998341,35cyclictest0-21swapper/209:30:222
1562199833,8cyclictest0-21swapper/209:40:342
15621998338,33cyclictest0-21swapper/211:25:252
1562199833,41cyclictest16329-21/usr/sbin/munin11:30:442
1562199833,37cyclictest16944-21taskset08:40:562
1562199833,32cyclictest28829-21/usr/sbin/munin09:11:022
1562199832,38cyclictest2080-21apt-get12:00:342
1562199832,34cyclictest17414-21/usr/sbin/munin12:25:452
1562199832,34cyclictest17414-21/usr/sbin/munin12:25:452
1560799839,30cyclictest22919-21apt-get07:20:380
15607998334,26cyclictest181rcu_preempt12:25:440
15607998334,26cyclictest181rcu_preempt12:25:440
1562199823,37cyclictest19676-21snmpd09:50:452
1562199823,36cyclictest12808-21/usr/sbin/munin10:30:512
1562199823,34cyclictest2667-21fschecks_time09:20:452
1562199823,34cyclictest2667-21fschecks_time09:20:452
1562199823,34cyclictest2461-21/usr/sbin/munin07:55:592
1562199823,33cyclictest1-21systemd11:20:242
1562199823,10cyclictest0-21swapper/208:45:332
1562199822,35cyclictest11196-21basename08:25:222
1562199822,33cyclictest20889-21cron08:55:222
1562199822,33cyclictest20889-21cron08:55:222
1561399823,62cyclictest5597-21apt-get09:25:401
1562199813,7cyclictest0-21swapper/212:05:362
1562199813,40cyclictest22525-21cp07:20:242
1562199813,40cyclictest21999-21apt11:40:222
1562199813,38cyclictest30787-21sshd07:45:252
1562199813,38cyclictest30787-21sshd07:45:252
1562199813,36cyclictest8180-21cpuspeed12:10:392
1562199813,35cyclictest22422-21apt-get07:20:382
1562199813,34cyclictest5298-21/usr/sbin/munin11:10:582
1562199813,33cyclictest24184-21ssh09:01:402
1562199813,33cyclictest24184-21ssh09:01:402
1562199813,32cyclictest26768-21processes07:30:542
1562199813,31cyclictest3826-21/usr/sbin/munin08:00:502
1562199812,36cyclictest25855-21ssh09:05:372
1562199812,35cyclictest21497-21idleruntime-cro10:45:222
1562199812,35cyclictest21497-21idleruntime-cro10:45:222
1562199812,30cyclictest18149-21/usr/sbin/munin07:05:482
1562199812,30cyclictest18149-21/usr/sbin/munin07:05:482
1562199812,30cyclictest12127-21fschecks_time09:35:442
1562199812,30cyclictest12127-21fschecks_time09:35:442
15613998158,16cyclictest1-21systemd10:17:401
15613998158,16cyclictest1-21systemd10:17:401
15613998152,19cyclictest11386-21fschecks_time12:15:421
15607998139,12cyclictest181rcu_preempt11:40:250
15607998130,28cyclictest181rcu_preempt11:40:460
15621998059,15cyclictest10716-21timerandwakeup10:25:592
15621998059,15cyclictest10716-21timerandwakeup10:25:592
1562199804,8cyclictest0-21swapper/207:10:392
1562199803,39cyclictest27518-21ps10:00:552
1562199803,36cyclictest1613-21fschecks_time11:05:452
1562199803,32cyclictest5916-21/usr/sbin/munin08:05:582
1562199803,32cyclictest5916-21/usr/sbin/munin08:05:582
1562199803,32cyclictest24792-21/usr/sbin/munin09:55:592
1562199803,31cyclictest28014-21modprobe11:50:232
15607998060,15cyclictest25739-21latency_hist09:05:260
15607998060,15cyclictest25739-21latency_hist09:05:260
1562199794,33cyclictest27748-21/usr/sbin/munin07:35:562
1562199793,30cyclictest13564-21netstat08:30:502
1562199793,30cyclictest11517-21sort08:25:402
1562199792,31cyclictest472-21/usr/sbin/munin07:50:462
15613997961,13cyclictest22519-21diskmemload11:01:101
15613997956,15cyclictest2374-21systemd-journal09:36:401
15613997956,15cyclictest2374-21systemd-journal09:36:401
1561399794,41cyclictest29128-21fschecks_time09:10:441
1561399793,38cyclictest5251-21fschecks_time08:05:441
1561399793,38cyclictest5251-21fschecks_time08:05:441
15613997930,21cyclictest277-21dbus-daemon10:30:241
15613997930,21cyclictest277-21dbus-daemon10:30:241
15613997926,43cyclictest23077-21cat07:21:371
15607997947,12cyclictest181rcu_preempt11:16:240
15607997944,13cyclictest181rcu_preempt10:26:320
15607997944,13cyclictest181rcu_preempt10:26:320
15607997942,20cyclictest3124-21modprobe08:00:230
15607997940,16cyclictest181rcu_preempt08:30:500
15607997912,37cyclictest15-21ksoftirqd/011:02:500
1549227922,34sleep10-21swapper/107:00:241
15607997857,14cyclictest2374-21systemd-journal09:14:400
15607997851,12cyclictest181rcu_preempt11:45:300
15607997835,13cyclictest181rcu_preempt10:17:440
15607997835,13cyclictest181rcu_preempt10:17:440
15607997835,12cyclictest181rcu_preempt10:45:250
15607997835,12cyclictest181rcu_preempt10:45:250
253152770,13sleep3451rcuc/309:56:593
15607997736,10cyclictest181rcu_preempt09:42:240
15607997735,13cyclictest181rcu_preempt09:09:410
1560799773,37cyclictest15-21ksoftirqd/012:05:030
15613997653,15cyclictest15595-21cyclictest09:35:261
15607997650,12cyclictest181rcu_preempt09:51:110
15607997645,10cyclictest181rcu_preempt08:00:520
15607997644,24cyclictest25977-21latency_hist07:30:250
15607997644,10cyclictest181rcu_preempt09:56:470
15613997534,28cyclictest27830-21sort07:35:401
15613997529,31cyclictest300-21rs:main1
15607997555,14cyclictest5058-21acpi12:05:330
15607997543,7cyclictest181rcu_preempt10:05:430
15607997542,8cyclictest181rcu_preempt11:20:580
15607997539,22cyclictest181rcu_preempt07:05:360
15607997539,22cyclictest181rcu_preempt07:05:360
15607997536,16cyclictest181rcu_preempt08:46:250
15607997535,9cyclictest181rcu_preempt10:47:400
15607997534,9cyclictest181rcu_preempt09:17:010
15607997533,13cyclictest181rcu_preempt12:23:170
1560799753,27cyclictest15-21ksoftirqd/008:55:390
34352740,7sleep10-21swapper/110:15:251
15613997416,27cyclictest21529-21kernelversion08:55:461
15607997449,9cyclictest181rcu_preempt11:26:380
15607997447,11cyclictest181rcu_preempt08:29:040
15607997440,12cyclictest181rcu_preempt10:53:590
15607997440,12cyclictest181rcu_preempt10:53:590
15607997439,13cyclictest181rcu_preempt07:04:060
15607997438,17cyclictest181rcu_preempt10:02:500
15613997353,14cyclictest544-21cron07:15:221
15613997352,15cyclictest22519-21diskmemload12:10:481
15607997359,10cyclictest58-21kcompactd012:17:100
15607997352,15cyclictest1-21systemd11:10:220
15607997344,11cyclictest181rcu_preempt09:21:080
15607997344,11cyclictest181rcu_preempt09:21:080
15607997339,12cyclictest181rcu_preempt07:47:500
15613997255,12cyclictest15595-21cyclictest08:15:401
15613997239,22cyclictest3485-21cstates08:00:411
15613997226,11cyclictest301ktimers/112:27:021
15613997226,11cyclictest301ktimers/112:27:021
15613997212,29cyclictest22519-21diskmemload11:40:511
15607997253,7cyclictest181rcu_preempt10:21:050
15607997250,15cyclictest14553-21kworker/u17:0-rpciod09:50:040
15607997249,9cyclictest181rcu_preempt11:32:050
15607997249,15cyclictest0-21swapper/007:50:500
15607997248,16cyclictest27700-21latency_hist07:35:260
15607997245,13cyclictest181rcu_preempt08:55:160
15607997245,13cyclictest181rcu_preempt08:55:160
15607997241,10cyclictest181rcu_preempt07:40:530
15607997241,10cyclictest181rcu_preempt07:40:530
15607997235,9cyclictest181rcu_preempt09:38:320
15607997235,9cyclictest181rcu_preempt09:38:320
15607997229,12cyclictest181rcu_preempt08:15:460
15613997151,15cyclictest25869-21sh09:05:401
15607997140,12cyclictest181rcu_preempt11:12:440
15607997136,12cyclictest181rcu_preempt07:35:590
1560799713,31cyclictest28370-21ntp_kernel_err10:55:510
15607997129,12cyclictest181rcu_preempt12:13:520
15613997054,10cyclictest20862-21kworker/u17:2-xprtiod10:00:501
15607997050,7cyclictest181rcu_preempt10:34:560
15613996945,15cyclictest0-21swapper/107:40:581
15613996945,15cyclictest0-21swapper/107:40:581
15613996927,8cyclictest301ktimers/107:15:341
15613996854,9cyclictest30391-21ntp_kernel_pll_10:05:491
15613996842,16cyclictest0-21swapper/111:25:441
15613996839,20cyclictest4749-21modprobe12:05:221
1561399683,49cyclictest7641-21fschecks_time11:15:441
15607996840,11cyclictest181rcu_preempt08:08:340
15607996840,11cyclictest181rcu_preempt08:08:340
15607996832,13cyclictest181rcu_preempt07:14:370
15613996752,8cyclictest0-21swapper/107:30:231
15613996751,9cyclictest5088-21apt-key09:25:231
15613996751,9cyclictest5088-21apt-key09:25:231
15613996747,8cyclictest181rcu_preempt08:13:481
15613996747,8cyclictest181rcu_preempt08:13:481
15613996746,15cyclictest19928-21ntp_offset11:35:511
15613996744,15cyclictest2374-21systemd-journal10:36:401
15607996746,13cyclictest22808-21apt-key07:20:260
15607996745,8cyclictest181rcu_preempt08:20:480
15613996644,16cyclictest23189-21idleruntime09:00:461
15613996644,16cyclictest23189-21idleruntime09:00:461
15613996644,13cyclictest0-21swapper/111:45:451
15613996642,16cyclictest2374-21systemd-journal10:58:401
1561399663,44cyclictest352-21runrttasks11:10:121
1561399663,42cyclictest16511-21fschecks_count08:40:441
15613996633,9cyclictest181rcu_preempt07:55:421
15613996615,28cyclictest22519-21diskmemload11:54:131
15613996544,15cyclictest15965-21fschecks_count07:00:441
15613996543,12cyclictest0-21swapper/112:05:591
15613996535,22cyclictest22250-21sed10:45:471
1561399653,40cyclictest7061-21cstates10:20:581
15607996519,22cyclictest0-21swapper/008:13:460
15607996519,22cyclictest0-21swapper/008:13:460
15613996450,8cyclictest1952-21kworker/u19:0+flush-179:009:55:471
15613996444,14cyclictest0-21swapper/107:55:251
15613996443,13cyclictest19035-21fschecks_count10:40:441
15613996443,13cyclictest19035-21fschecks_count10:40:441
15613996437,8cyclictest181rcu_preempt10:30:431
15613996420,11cyclictest301ktimers/110:53:571
15613996420,11cyclictest301ktimers/110:53:571
1561399633,43cyclictest4342-21/usr/sbin/munin11:10:431
15613996330,9cyclictest181rcu_preempt08:30:261
1561399632,35cyclictest31-21ksoftirqd/109:50:281
15613996318,25cyclictest14509-21cat08:35:251
15613996238,17cyclictest18380-21netstat08:45:481
1561399623,40cyclictest18405-21ssh09:45:511
1561399622,22cyclictest20888-21cron08:55:221
1561399622,22cyclictest20888-21cron08:55:221
15613996141,14cyclictest0-21swapper/111:30:401
15613996141,10cyclictest0-21swapper/109:40:401
15613996139,12cyclictest0-21swapper/108:20:571
15613996132,8cyclictest0-21swapper/107:46:001
1561399612,43cyclictest11191-21find11:20:571
15613996120,8cyclictest301ktimers/109:15:481
15613996043,12cyclictest26785-21mailstats07:30:551
1561399603,42cyclictest18075-21fschecks_time07:05:471
1561399603,42cyclictest18075-21fschecks_time07:05:471
1562899592,28cyclictest31550-21chrt11:55:373
1562899583,34cyclictest0-21swapper/307:20:503
55352540,7sleep3451rcuc/310:18:193
55352540,7sleep3451rcuc/310:18:193
15628995326,19cyclictest21874-21chrt08:55:553
1562899523,36cyclictest19439-21chrt11:35:393
1562899523,36cyclictest16173-21chrt12:23:453
1562899522,38cyclictest1558-21chrt07:55:253
1562899522,37cyclictest15351-21taskset10:35:013
1562899522,35cyclictest19695-21chrt08:50:413
1562899522,35cyclictest19695-21chrt08:50:413
1562899513,39cyclictest13856-21taskset11:25:503
1562899512,37cyclictest27923-21chrt11:50:183
1562899512,36cyclictest10098-21chrt08:20:443
15628995035,10cyclictest451rcuc/310:59:003
1562899503,37cyclictest5856-21taskset11:12:043
1562899503,35cyclictest16553-21taskset08:40:453
1562899502,37cyclictest4063-21taskset09:22:383
1562899502,37cyclictest4063-21taskset09:22:383
1562899494,32cyclictest11119-21chrt10:26:343
1562899494,32cyclictest11119-21chrt10:26:343
1562899493,37cyclictest21871-21taskset09:50:593
1562899493,36cyclictest21168-21chrt07:15:403
1562899493,34cyclictest32171-21taskset07:48:403
1562899492,35cyclictest754-21chrt10:10:533
1562899492,34cyclictest30070-21taskset10:05:423
1562899492,33cyclictest17575-21taskset09:45:243
1562899492,31cyclictest24727-21chrt07:25:443
15628994835,9cyclictest451rcuc/310:38:333
1562899483,35cyclictest9841-21chrt09:31:263
1562899483,35cyclictest31695-21taskset11:00:583
1562899483,33cyclictest1560-21chrt11:05:443
1562899483,32cyclictest5446-21chrt12:05:463
1562899482,34cyclictest18766-21chrt11:34:423
147042488,24sleep30-21swapper/306:56:063
1562899474,30cyclictest14655-21chrt09:40:263
1562899474,30cyclictest14655-21chrt09:40:263
1562899474,29cyclictest18629-21chrt08:45:553
1562899473,35cyclictest3987-21taskset12:03:153
1562899473,35cyclictest25843-21chrt09:05:363
1562899473,34cyclictest29531-21chrt10:05:223
1562899473,33cyclictest12992-21chrt11:25:213
1562899473,32cyclictest29286-21chrt07:40:263
1562899473,31cyclictest17334-21taskset12:25:443
1562899473,31cyclictest17334-21taskset12:25:443
1562899473,30cyclictest5875-21chrt08:05:583
1562899473,30cyclictest5875-21chrt08:05:583
1562899473,29cyclictest29427-21chrt11:51:033
1562899472,37cyclictest5728-21chrt09:25:553
1562899472,34cyclictest4192-21chrt08:01:003
1562899472,33cyclictest9162-21chrt11:18:003
1562899472,31cyclictest9198-21chrt12:11:013
1562899463,33cyclictest19411-21chrt09:47:193
1562899462,35cyclictest30556-21taskset09:12:413
1562899462,34cyclictest15631-21taskset08:35:593
1562899462,32cyclictest22026-21taskset10:45:433
1562899462,32cyclictest14621-21taskset08:35:263
1562899462,29cyclictest21347-21chrt10:45:023
1562899462,29cyclictest21347-21chrt10:45:023
1562899453,33cyclictest12311-21chrt12:16:203
1562899453,32cyclictest346-21chrt09:16:023
1562899452,30cyclictest23363-21taskset09:00:493
1562899452,30cyclictest23363-21taskset09:00:493
1562899443,32cyclictest9054-21chrt08:15:583
1562899443,29cyclictest29438-21chrt07:40:393
1562899443,29cyclictest29438-21chrt07:40:393
1562899442,29cyclictest9574-21chrt10:25:243
1562899433,31cyclictest3130-21chrt08:00:233
1562899433,30cyclictest27412-21chrt10:55:113
1562899433,30cyclictest27412-21chrt10:55:113
1562899433,29cyclictest19337-21chrt07:10:253
1562899433,29cyclictest19337-21chrt07:10:253
15628994327,8cyclictest451rcuc/308:15:253
15628994327,8cyclictest451rcuc/308:15:253
1562899432,32cyclictest26805-21chrt07:30:553
1562899423,29cyclictest17133-21chrt07:05:233
1562899417,23cyclictest12569-21chrt08:27:403
1562899393,27cyclictest20676-21taskset07:13:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional