You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 04:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun Apr 26, 2026 00:44:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
881991303,110cyclictest16562-21apt-get23:02:431
8769912258,57cyclictest863-21cyclictest22:51:300
8769911660,47cyclictest31271-21ssh23:27:280
876991163,46cyclictest875-21sort19:02:300
8769911357,51cyclictest0-21swapper/021:17:290
8769911357,51cyclictest0-21swapper/021:17:290
8769911157,47cyclictest0-21swapper/022:38:020
8769911156,48cyclictest0-21swapper/000:12:500
876991094,10cyclictest19537-21apt-get23:07:380
876991094,10cyclictest19537-21apt-get23:07:380
881991073,94cyclictest13602-21munin-run23:52:261
881991073,94cyclictest13602-21munin-run23:52:261
876991073,22cyclictest339-21apt-get21:42:380
876991063,28cyclictest19649-21latency_hist00:02:280
876991053,31cyclictest6369-21apt-get21:52:380
876991043,31cyclictest7934-21sh23:42:390
876991043,31cyclictest7934-21sh23:42:390
876991043,29cyclictest1066-21apt22:37:250
8769910243,52cyclictest25641-21cstates23:17:430
876991012,22cyclictest2249-21/usr/sbin/munin00:27:370
8769910047,45cyclictest0-21swapper/021:07:250
8769910019,21cyclictest19205-21sh21:19:290
87699994,28cyclictest16740-21sed23:57:280
87699993,34cyclictest352-21runrttasks23:03:320
87699993,11cyclictest31738-21apt-get00:22:370
876999921,23cyclictest2138-21sh23:32:430
876999921,23cyclictest2138-21sh23:32:430
87699984,32cyclictest25440-21modprobe20:17:250
87699983,32cyclictest10662-21runrttasks19:29:330
87699983,26cyclictest3356-21sh21:47:370
87699983,24cyclictest11737-21ntpq21:07:580
87699983,22cyclictest10476-21/usr/sbin/munin21:58:150
87699982,39cyclictest26713-21cstates21:32:420
87699974,31cyclictest19135-21tune2fs22:12:480
87699973,23cyclictest29593-21switchtime00:17:590
87699973,21cyclictest11465-21/usr/sbin/munin23:47:530
87699973,21cyclictest11465-21/usr/sbin/munin23:47:530
87699965,33cyclictest24182-21taskset21:27:550
87699963,30cyclictest25759-21ls22:23:020
876999616,12cyclictest0-21swapper/020:52:270
876999616,12cyclictest0-21swapper/020:52:270
87699954,36cyclictest13924-21awk22:57:540
87699954,36cyclictest13924-21awk22:57:540
876999536,51cyclictest0-21swapper/000:12:280
876999536,51cyclictest0-21swapper/000:12:280
87699953,38cyclictest7301-21rm23:41:370
87699953,35cyclictest28662-21latency_hist20:27:260
87699953,30cyclictest27265-21latency_hist20:22:290
87699953,29cyclictest9290-21latency_hist19:27:280
87699953,29cyclictest9290-21latency_hist19:27:280
876999531,58cyclictest0-21swapper/023:12:460
87699952,40cyclictest1899-21latency_hist23:32:290
87699952,30cyclictest32563-21latency_hist21:42:270
87699952,30cyclictest32563-21latency_hist21:42:270
87699952,30cyclictest32563-21latency_hist21:42:270
88199943,64cyclictest12822-21apt-get22:02:431
87699944,36cyclictest19676-21snmpd22:44:280
87699944,36cyclictest19676-21snmpd22:44:280
87699944,32cyclictest27903-21cpuspeed22:27:420
87699944,27cyclictest5082-21fschecks_time20:53:100
87699944,27cyclictest5082-21fschecks_time20:53:100
87699943,33cyclictest1204-21sh19:02:440
87699943,32cyclictest1264-21latency_hist20:42:280
87699943,32cyclictest1264-21latency_hist20:42:280
87699943,27cyclictest21542-21taskset00:04:480
87699942,32cyclictest16112-21fschecks_time22:07:470
87699934,37cyclictest4748-21cpu19:12:420
87699934,37cyclictest4748-21cpu19:12:420
87699934,29cyclictest22385-21ntpq22:17:540
87699934,29cyclictest22385-21ntpq22:17:540
87699934,27cyclictest6720-21/usr/sbin/munin20:57:500
87699933,33cyclictest20637-21cpu21:22:430
87699933,33cyclictest20637-21cpu21:22:430
87699933,27cyclictest11438-21tune2fs19:32:480
87699933,24cyclictest14409-21chrt19:42:370
876999317,21cyclictest16325-21cat19:47:480
13892930,3sleep10-21swapper/119:02:491
88399923,45cyclictest1060-21taskset19:02:372
87699924,25cyclictest20589-21modprobe20:02:250
87699924,25cyclictest20589-21modprobe20:02:250
87699924,25cyclictest12781-21/usr/sbin/munin19:38:040
87699923,33cyclictest15503-21latency_hist22:07:280
87699923,31cyclictest20846-21/usr/sbin/munin20:02:510
87699923,28cyclictest10451-21/usr/sbin/munin22:52:430
87699923,28cyclictest10451-21/usr/sbin/munin22:52:430
87699923,25cyclictest32103-21grep20:37:280
87699923,25cyclictest32103-21grep20:37:280
5772926,65sleep20-21swapper/219:00:352
883999170,15cyclictest23068-21fschecks_time00:07:472
883999170,15cyclictest23068-21fschecks_time00:07:472
883999152,32cyclictest0-21swapper/223:27:252
87699913,31cyclictest1308-21/usr/sbin/munin20:42:500
87699913,31cyclictest1308-21/usr/sbin/munin20:42:500
876999117,64cyclictest0-21swapper/020:12:290
883999048,35cyclictest863-21cyclictest21:17:202
883999048,35cyclictest863-21cyclictest21:17:202
883999048,35cyclictest0-21swapper/221:52:372
881999068,16cyclictest7276-21unixbench_singl19:18:031
881999068,16cyclictest7276-21unixbench_singl19:18:031
87699903,28cyclictest18570-21grep19:53:040
87699903,28cyclictest18570-21grep19:53:040
883998952,30cyclictest0-21swapper/222:03:282
883998917,27cyclictest19741-21sed22:13:022
87699893,30cyclictest28907-21/usr/sbin/munin20:27:420
87699893,30cyclictest28907-21/usr/sbin/munin20:27:420
87699892,32cyclictest3080-21/usr/sbin/munin19:07:530
87699892,31cyclictest6589-21fschecks_time19:17:490
87699892,31cyclictest6589-21fschecks_time19:17:490
88199883,53cyclictest29824-21apt-get21:37:421
88199883,53cyclictest29824-21apt-get21:37:421
88199883,53cyclictest29824-21apt-get21:37:421
883998744,36cyclictest0-21swapper/223:37:552
88399873,38cyclictest16534-21ssh23:02:342
88399873,37cyclictest29774-21unixbench_multi00:18:012
883998721,61cyclictest9958-21sort21:58:022
883998541,37cyclictest14460-21cpu19:42:422
88399853,32cyclictest25441-21modprobe20:17:252
881998561,15cyclictest17541-21latency_hist19:52:291
88399849,10cyclictest0-21swapper/222:29:232
88399845,35cyclictest8440-21netstat23:42:522
88399845,35cyclictest8440-21netstat23:42:522
883998447,30cyclictest0-21swapper/219:37:422
883998443,32cyclictest0-21swapper/223:48:022
883998443,32cyclictest0-21swapper/223:48:022
88399844,31cyclictest30497-21latency_hist20:32:292
88399844,31cyclictest30497-21latency_hist20:32:292
88399843,40cyclictest25786-21/usr/sbin/munin00:12:512
88399843,27cyclictest22882-21grep22:18:032
88399843,27cyclictest22882-21grep22:18:032
88199843,32cyclictest4702-21latency_hist19:12:301
883998360,10cyclictest0-21swapper/219:22:282
883998360,10cyclictest0-21swapper/219:22:282
88399834,34cyclictest32151-21/usr/sbin/munin20:37:592
88399834,34cyclictest32151-21/usr/sbin/munin20:37:592
88399833,36cyclictest1962-21/usr/sbin/munin23:32:422
88399833,36cyclictest1962-21/usr/sbin/munin23:32:422
88399833,31cyclictest26490-21grep21:32:292
88399833,29cyclictest4477-21taskset20:52:282
88399833,29cyclictest4477-21taskset20:52:282
88399833,25cyclictest4060-21modprobe22:42:252
88399833,23cyclictest5479-21/usr/sbin/munin22:43:032
88399833,23cyclictest5479-21/usr/sbin/munin22:43:032
88399832,33cyclictest352-21runrttasks23:53:042
883998317,22cyclictest10378-21timerwakeupswit19:28:022
88399827,8cyclictest0-21swapper/222:07:392
883998262,10cyclictest0-21swapper/219:52:282
883998262,10cyclictest0-21swapper/219:17:282
883998262,10cyclictest0-21swapper/219:17:282
88399825,38cyclictest3601-21fschecks_count21:47:462
88399824,39cyclictest5635-21grep20:53:242
88399824,39cyclictest5635-21grep20:53:242
88399824,32cyclictest352-21runrttasks21:01:472
88399823,36cyclictest30875-21timerandwakeup21:38:242
88399823,36cyclictest30875-21timerandwakeup21:38:242
88399823,36cyclictest30875-21timerandwakeup21:38:242
88399823,36cyclictest19515-21/usr/sbin/munin23:08:032
88399823,36cyclictest19515-21/usr/sbin/munin23:08:032
88399823,35cyclictest25976-21latency23:17:512
88399823,30cyclictest3284-21latency_hist21:47:292
88399823,29cyclictest4460-21idleruntime-cro19:12:262
88399822,33cyclictest7678-21seq22:47:482
881998261,16cyclictest863-21cyclictest23:37:471
890998114,57cyclictest0-21swapper/323:02:423
88399818,9cyclictest0-21swapper/200:29:542
88399814,32cyclictest23061-21sh23:12:522
88399814,27cyclictest17903-21fschecks_time19:52:502
88399814,27cyclictest17903-21fschecks_time19:52:502
883998134,39cyclictest0-21swapper/220:12:282
88399813,38cyclictest27647-21head21:33:022
88399813,29cyclictest7934-21/usr/sbin/munin19:22:522
88399813,29cyclictest7934-21/usr/sbin/munin19:22:522
88399813,27cyclictest2659-21modprobe20:47:252
88399813,27cyclictest2659-21modprobe20:47:252
88399813,26cyclictest12205-21/usr/sbin/munin21:08:042
883998117,24cyclictest31945-21sh23:27:522
883998113,29cyclictest17830-21cat21:17:582
881998156,17cyclictest29493-21apt-key21:37:281
883998062,8cyclictest0-21swapper/219:57:522
883998062,8cyclictest0-21swapper/219:57:522
88399803,36cyclictest20575-21/usr/sbin/munin21:22:562
88399803,36cyclictest20575-21/usr/sbin/munin21:22:562
88399803,29cyclictest10500-21taskset22:52:442
88399803,29cyclictest10500-21taskset22:52:442
88399803,28cyclictest12533-21munin-plugin-st19:37:252
88399802,29cyclictest17021-21cstates23:57:432
88399802,28cyclictest13419-21cpu22:57:432
88399802,28cyclictest13419-21cpu22:57:432
881998056,15cyclictest2374-21systemd-journal19:25:071
881998056,15cyclictest2374-21systemd-journal19:25:071
881998016,26cyclictest6778-21seq21:52:521
883997960,9cyclictest0-21swapper/220:22:282
883997933,38cyclictest0-21swapper/221:02:502
88399793,29cyclictest21242-21kernelversion20:02:512
88399793,28cyclictest31135-21proc_pri20:32:572
88399793,28cyclictest31135-21proc_pri20:32:572
88399793,28cyclictest25868-21grep22:23:032
88399793,28cyclictest20614-21sensors_temp00:03:002
88399793,27cyclictest32404-21/usr/sbin/munin00:22:582
88399792,29cyclictest28862-21latency_hist20:27:292
88399782,24cyclictest31172-21fschecks_time22:32:492
881997811,54cyclictest9605-21apt-get21:57:491
881997755,14cyclictest300-21rs:main1
881997753,16cyclictest28790-21fschecks_time23:22:491
881997749,20cyclictest5455-21switchtime19:13:001
881997749,20cyclictest5455-21switchtime19:13:001
88199773,64cyclictest4362-21/usr/sbin/munin22:42:491
88199773,64cyclictest4362-21/usr/sbin/munin22:42:491
881997659,11cyclictest17083-21systemd-run21:17:271
881997659,11cyclictest17083-21systemd-run21:17:271
881997619,23cyclictest12566-21seq19:37:271
881997617,21cyclictest31452-21ssh23:27:391
881997555,14cyclictest2992-21sh21:47:211
881997532,32cyclictest7436-21cpu22:47:421
189732740,6sleep30-21swapper/321:18:503
881997354,13cyclictest26307-21switchtime23:18:001
881997350,10cyclictest0-21swapper/121:17:421
881997330,36cyclictest0-21swapper/119:02:301
881997251,12cyclictest0-21swapper/122:57:251
881997251,12cyclictest0-21swapper/122:57:251
881997249,17cyclictest544-21cron21:52:261
881997214,26cyclictest13437-21seq21:11:281
881997150,15cyclictest1-21systemd20:21:081
881997150,14cyclictest863-21cyclictest20:42:481
881997150,14cyclictest863-21cyclictest20:42:481
881997149,16cyclictest31866-21unixbench_multi22:33:031
881997148,15cyclictest863-21cyclictest21:05:051
881997147,16cyclictest31312-21threads20:32:591
881997147,16cyclictest31312-21threads20:32:591
88199713,47cyclictest4884-21latency_hist23:37:291
88199713,47cyclictest4884-21latency_hist23:37:291
881997044,16cyclictest18247-21ssh22:11:451
881997010,34cyclictest20086-21/usr/sbin/munin23:07:531
881997010,34cyclictest20086-21/usr/sbin/munin23:07:531
88199694,35cyclictest281162fschecks_time22:27:481
881996941,20cyclictest18338-21switchtime19:53:021
881996941,20cyclictest18338-21switchtime19:53:021
88199688,28cyclictest4816-21modprobe20:52:421
88199688,28cyclictest4816-21modprobe20:52:421
881996844,13cyclictest0-21swapper/122:21:171
881996844,13cyclictest0-21swapper/122:21:171
881996821,10cyclictest0-21swapper/100:17:411
881996750,11cyclictest20634-21switchtime00:03:011
881996747,13cyclictest2374-21systemd-journal21:02:251
881996745,16cyclictest10911-21idleruntime-cro19:32:251
881996742,14cyclictest0-21swapper/119:47:261
88199673,36cyclictest13356-21/usr/sbin/munin22:57:481
88199673,36cyclictest13356-21/usr/sbin/munin22:57:481
881996649,11cyclictest300-21rs:main1
881996649,11cyclictest300-21rs:main1
881996640,16cyclictest0-21swapper/122:13:031
88199663,47cyclictest32031-21fschecks_time00:22:481
881996550,11cyclictest863-21cyclictest23:53:441
881996527,28cyclictest0-21swapper/120:07:371
88199652,46cyclictest24852-21ssh22:22:421
881996449,11cyclictest863-21cyclictest00:08:481
881996449,11cyclictest863-21cyclictest00:08:481
88199643,40cyclictest4913-21munin-run00:32:241
881996425,32cyclictest0-21swapper/120:02:411
88199640,30cyclictest0-21swapper/123:14:381
881996347,9cyclictest0-21swapper/119:38:021
881996344,9cyclictest0-21swapper/123:43:481
881996344,9cyclictest0-21swapper/123:43:481
881996339,12cyclictest0-21swapper/123:59:511
88199632,44cyclictest23642-21cpu21:27:421
89099623,40cyclictest0-21swapper/321:57:433
881996243,13cyclictest352-21runrttasks21:22:581
881996243,13cyclictest352-21runrttasks21:22:581
881996241,14cyclictest2435-21unixbench_singl22:38:031
88199622,44cyclictest29566-21mailstats20:27:571
88199622,44cyclictest29566-21mailstats20:27:571
88199613,42cyclictest19241-21/usr/sbin/munin19:57:371
88199613,42cyclictest19241-21/usr/sbin/munin19:57:371
881996121,28cyclictest0-21swapper/120:12:371
89099604,31cyclictest0-21swapper/321:13:503
89099604,31cyclictest0-21swapper/321:13:503
881996039,15cyclictest3505-21proc_pri20:47:581
881996039,15cyclictest3505-21proc_pri20:47:581
88199603,36cyclictest352-21runrttasks20:23:151
89099588,31cyclictest0-21swapper/322:02:543
89099562,36cyclictest14254-21taskset23:52:503
89099553,35cyclictest0-21swapper/321:37:553
89099553,35cyclictest0-21swapper/321:37:553
89099553,35cyclictest0-21swapper/321:37:553
890995441,8cyclictest451rcuc/322:22:373
890995438,9cyclictest451rcuc/323:46:383
890995438,9cyclictest451rcuc/323:46:383
89099533,36cyclictest28169-21taskset21:34:223
89099532,41cyclictest4537-21taskset21:48:173
89099524,34cyclictest22621-21taskset22:18:003
89099524,34cyclictest22621-21taskset22:18:003
89099524,33cyclictest14258-21chrt19:42:283
89099523,38cyclictest30318-21chrt20:32:263
89099523,38cyclictest30318-21chrt20:32:263
89099523,34cyclictest23465-21chrt21:27:293
89099523,34cyclictest23465-21chrt21:27:293
89099523,32cyclictest9411-21chrt21:57:293
89099523,32cyclictest2579-21taskset19:07:293
890995136,9cyclictest451rcuc/320:01:403
890995136,9cyclictest451rcuc/320:01:403
89099513,39cyclictest32257-21chrt00:22:533
89099513,38cyclictest10524-21taskset21:06:453
89099513,34cyclictest31004-21chrt20:32:513
89099513,34cyclictest31004-21chrt20:32:513
89099513,33cyclictest3476-21chrt19:07:533
89099512,36cyclictest14015-21chrt21:12:293
89099512,34cyclictest24138-21chrt21:27:543
890995113,22cyclictest0-21swapper/323:52:293
890995113,22cyclictest0-21swapper/323:52:293
200382515,13sleep30-21swapper/322:13:193
890995036,10cyclictest451rcuc/322:56:513
890995036,10cyclictest451rcuc/322:56:513
89099503,37cyclictest28616-21taskset23:22:443
89099503,36cyclictest32504-21taskset22:34:383
89099503,35cyclictest27075-21chrt20:22:263
89099503,33cyclictest29561-21chrt22:29:523
89099502,35cyclictest23985-21chrt00:08:183
89099502,35cyclictest23985-21chrt00:08:183
89099502,34cyclictest17748-21chrt19:52:453
89099502,34cyclictest17748-21chrt19:52:453
89099502,33cyclictest3819-21chrt20:48:013
89099502,33cyclictest3819-21chrt20:48:013
89099493,37cyclictest18411-21chrt23:59:193
89099493,34cyclictest21359-21chrt20:02:543
89099493,33cyclictest27084-21taskset23:19:233
89099493,33cyclictest10048-21chrt22:52:243
89099492,36cyclictest3487-21chrt22:40:503
89099492,35cyclictest5164-21taskset23:37:473
76924910,24sleep30-21swapper/319:02:283
890994834,9cyclictest451rcuc/319:22:413
890994834,9cyclictest451rcuc/319:22:413
89099483,34cyclictest28746-21chrt00:17:343
89099483,33cyclictest32657-21chrt23:28:233
89099483,33cyclictest16251-21chrt19:47:473
89099483,32cyclictest2674-21chrt00:27:523
89099483,32cyclictest13500-21chrt22:57:433
89099483,32cyclictest13500-21chrt22:57:433
89099473,34cyclictest23219-21taskset20:08:003
89099473,32cyclictest2820-21taskset21:46:513
89099473,32cyclictest28038-21taskset00:16:173
89099473,32cyclictest19961-21taskset00:02:433
89099473,31cyclictest6890-21chrt19:17:583
89099473,31cyclictest6890-21chrt19:17:583
89099473,30cyclictest11337-21chrt19:32:443
89099473,29cyclictest2893-21chrt20:47:293
890994731,9cyclictest451rcuc/322:45:073
890994731,9cyclictest451rcuc/322:45:073
89099472,37cyclictest16121-21chrt22:07:473
89099472,31cyclictest22171-21chrt23:12:223
89099472,31cyclictest22171-21chrt23:12:223
89099462,33cyclictest0-21swapper/320:52:423
89099462,33cyclictest0-21swapper/320:52:423
89099453,31cyclictest3772-21taskset23:34:593
89099453,31cyclictest3772-21taskset23:34:593
89099453,30cyclictest24992-21taskset23:16:523
89099453,29cyclictest32444-21chrt20:37:493
89099453,29cyclictest32444-21chrt20:37:493
89099452,31cyclictest9563-21chrt19:27:383
89099444,31cyclictest6719-21chrt20:57:503
89099443,27cyclictest1684-21chrt20:42:513
89099442,30cyclictest25077-21chrt20:13:023
89099433,32cyclictest27375-21chrt20:22:423
89099383,25cyclictest15605-21taskset19:45:483
89099373,25cyclictest5727-21taskset19:13:043
89099373,25cyclictest5727-21taskset19:13:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional