You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 04:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Tue May 26, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
221339124300,3sleep12213393-21kthreadcore19:05:501
2215251997358,10cyclictest1704890-21systemd-journal23:10:471
2215249996748,16cyclictest2375553-21ntpq21:45:530
2215249996640,23cyclictest2232722-21diskstats19:25:450
2215251996457,4cyclictest2366826-21kworker/u8:3+events_unbound23:26:011
2215251996450,10cyclictest1704890-21systemd-journal22:30:471
2215251996450,10cyclictest1704890-21systemd-journal22:30:461
2215249996452,9cyclictest2251982-21apt-get19:45:000
2215249996339,21cyclictest2476333-21diskstats23:30:450
2215251996152,7cyclictest2297155-21kworker/u8:1+events_unbound20:31:131
2215251995850,6cyclictest2262175-21base6419:55:011
2215249995746,9cyclictest2454664-21ntpq23:05:540
2215251995650,4cyclictest2366826-21kworker/u8:3+events_unbound23:31:001
2215251995551,3cyclictest2158197-21kworker/u8:0+flush-8:019:45:451
2215251995447,5cyclictest2480548-21kworker/u8:2+events_unbound00:20:571
2215249995346,5cyclictest2540141-21apt-get00:31:360
2215251995242,8cyclictest2499735-21ntpq23:50:571
2215251995242,7cyclictest1704890-21systemd-journal22:20:471
2215249995235,15cyclictest2511112-21sshd00:05:470
2215249995235,15cyclictest2511112-21sshd00:05:470
2215249995227,22cyclictest2524366-21ntpq00:15:520
2215251995138,9cyclictest1704890-21systemd-journal20:55:471
2215251995046,3cyclictest2297155-21kworker/u8:1+flush-8:020:40:471
2215254994930,16cyclictest1704890-21systemd-journal21:25:012
2215251994944,4cyclictest2366826-21kworker/u8:3+flush-8:023:16:011
2215251994940,6cyclictest1704890-21systemd-journal21:00:471
2215249994930,16cyclictest2326769-21latency_hist21:00:010
24019112480,4sleep10-21swapper/122:15:471
2215251994843,4cyclictest2232193-21kworker/u8:3+flush-8:020:45:451
2215251994737,8cyclictest2286336-21ntpq20:15:531
2215249994723,9cyclictest2541230-21diskstats00:35:450
2215251994640,5cyclictest2158197-21kworker/u8:0+events_unbound21:50:491
2215251994637,6cyclictest2445719-21latency_hist23:00:011
2215249994626,17cyclictest2272797-21fschecks_time20:05:470
2215251994541,3cyclictest2183655-21kworker/u8:2+flush-8:020:06:001
2215251994539,4cyclictest2435863-21kworker/u8:0+events_unbound22:50:581
2215249994536,7cyclictest2465688-21latency_hist23:20:000
2215251994439,4cyclictest2480548-21kworker/u8:2+flush-8:023:56:001
2215251994438,4cyclictest2158197-21kworker/u8:0+events_unbound21:05:451
2215251994434,7cyclictest2391313-21latency_hist22:05:011
2215251994338,4cyclictest2232193-21kworker/u8:3+flush-8:020:35:461
2215251994337,4cyclictest2183655-21kworker/u8:2+events_unbound20:00:441
2215251994336,5cyclictest2366826-21kworker/u8:3+flush-8:023:40:561
2215249994332,8cyclictest2281162-21ntpq20:10:540
2215249994325,5cyclictest2470239-21unixbench_multi23:21:020
2215249994322,6cyclictest2471436-21fschecks_count23:25:460
2215251994236,5cyclictest2158197-21kworker/u8:0+flush-8:019:15:551
2215251994232,8cyclictest2251986-21latency_hist19:45:011
2215249994223,2cyclictest2220155-21apt-key19:15:000
2215251994134,5cyclictest2509623-21ntpq00:00:561
2215249994031,7cyclictest2331715-21latency_hist21:05:000
2215251993933,4cyclictest2391334-21kworker/u8:1+events_unbound22:35:551
2215251993931,5cyclictest2158197-21kworker/u8:0+flush-8:019:26:001
2215254993829,7cyclictest2460621-21gpgv23:15:012
2215254993824,12cyclictest2386689-21aten2_r5power_c22:00:452
2215254993817,8cyclictest2540513-21base6400:35:012
2215251993834,3cyclictest2158197-21kworker/u8:0+flush-8:019:36:011
2215251993832,4cyclictest2183655-21kworker/u8:2+flush-8:020:25:501
2215251993830,6cyclictest2220132-21latency_hist19:15:011
2215249993826,9cyclictest2340940-21ntpq21:10:550
2215249993826,9cyclictest2340940-21ntpq21:10:540
2215251993733,3cyclictest2366826-21kworker/u8:3+events_unbound23:21:001
2215251993732,4cyclictest2158197-21kworker/u8:0+events_unbound21:45:561
2215251993732,3cyclictest2391334-21kworker/u8:1+flush-8:023:35:501
2215251993729,6cyclictest2183655-21kworker/u8:2+flush-8:019:20:481
2215249993726,8cyclictest2336651-21latency_hist21:10:010
2215249993726,8cyclictest2311200-21ntpq20:40:590
2215249993719,3cyclictest0-21swapper/019:45:490
2215254993624,10cyclictest2529900-21unixbench_singl00:21:022
2215254993616,8cyclictest2252242-21cat19:45:012
2215251993632,3cyclictest2183655-21kworker/u8:2+events_unbound19:30:521
2215251993632,3cyclictest2183655-21kworker/u8:2+events_unbound19:30:511
2215251993632,3cyclictest2158197-21kworker/u8:0+events_unbound21:55:441
2215251993630,4cyclictest2505539-21kworker/u8:3+events_unbound00:10:581
2215251993625,7cyclictest1704890-21systemd-journal21:40:471
2215249993627,6cyclictest2491222-21grep23:45:440
2215249993623,11cyclictest2530636-21apt-key00:25:020
221524999361,33cyclictest2440800-21kworker/0:1+events23:35:570
221524999361,32cyclictest1-21systemd22:35:000
221524999361,32cyclictest1-21systemd22:34:590
2215254993522,2cyclictest2506135-21fschecks_time00:00:472
2215254993521,2cyclictest2302661-21fschecks_count20:35:452
2215251993531,3cyclictest2232193-21kworker/u8:3+events_unbound19:56:011
2215251993429,3cyclictest2366222-21unixbench_singl21:36:011
2215251993427,5cyclictest2158197-21kworker/u8:0+flush-8:021:30:551
2215251993328,3cyclictest2158197-21kworker/u8:0+events_unbound22:10:551
2215251993326,5cyclictest2435863-21kworker/u8:0+events_unbound23:00:471
2215251993326,5cyclictest2415723-21kworker/u8:2+flush-8:022:40:551
2215249993328,3cyclictest2397108-21irqcore22:10:490
2215249993326,5cyclictest1704890-21systemd-journal21:40:010
2215249993323,8cyclictest2449745-21ntpq23:00:530
2215254993226,4cyclictest2311581-21unixbench_multi20:41:022
2215254993210,2cyclictest22724042cat20:05:002
2215251993227,4cyclictest2480548-21kworker/u8:2+flush-8:023:45:571
2215251993227,3cyclictest2535439-21kworker/u8:1+events_unbound00:35:431
2215251993227,3cyclictest2232193-21kworker/u8:3+events_unbound21:11:021
2215251993227,3cyclictest2232193-21kworker/u8:3+events_unbound21:11:021
2215249993226,5cyclictest2232193-21kworker/u8:3+flush-8:020:35:590
22591732310,14sleep20-21swapper/219:50:502
2215251993127,3cyclictest2435863-21kworker/u8:0+events_unbound23:05:491
2215251993126,3cyclictest2292224-21systemd-run20:25:011
2215251993124,5cyclictest2415723-21kworker/u8:2+events_unbound22:45:431
2215249993125,4cyclictest1704890-21systemd-journal22:00:000
2215249993123,6cyclictest2226350-21ntpq19:15:540
2215249993122,6cyclictest2361624-21latency_hist21:35:010
25285012300,9sleep375-21kworker/3:1+events00:20:523
2215251993021,6cyclictest2282076-21apt-get20:15:011
2215249993025,3cyclictest2500551-21systemd-run23:55:000
2215254992924,3cyclictest2381837-21aten2_r5power_v21:55:442
2215251992925,3cyclictest2232193-21kworker/u8:3+events_unbound20:51:011
2215251992924,4cyclictest2297155-21kworker/u8:1+flush-8:021:25:471
2215251992924,3cyclictest2232193-21kworker/u8:3+flush-8:021:20:521
2215251992922,5cyclictest2366826-21kworker/u8:3+flush-8:022:25:521
221524999294,24cyclictest2237152-21kworker/0:0+events21:24:410
2215249992925,2cyclictest2505499-21latency_hist00:00:000
2215249992924,3cyclictest2267776-21cpuspeed_turbos20:00:440
2215249992919,7cyclictest1704890-21systemd-journal00:30:010
221524999291,26cyclictest2237152-21kworker/0:0+events20:51:010
22147812293,8sleep20-21swapper/219:06:042
2215251992824,3cyclictest2480548-21kworker/u8:2+events_unbound00:25:591
2215249992824,2cyclictest2371553-21sh21:45:010
2215249992818,8cyclictest2519466-21ntpq00:10:550
221524999279,16cyclictest2237152-21kworker/0:0+events22:45:530
221524999278,17cyclictest2237152-21kworker/0:0+events21:50:430
2215249992723,2cyclictest2489680-21needreboot23:40:520
2215249992712,13cyclictest2237152-21kworker/0:0+events22:35:550
221524999270,25cyclictest2386909-21sshd22:00:460
22108782274,19sleep00-21swapper/019:05:460
2215254992621,3cyclictest2317282-21aten2_r5power_p20:50:432
2215251992622,3cyclictest2480548-21kworker/u8:2+flush-8:000:30:471
2215251992621,3cyclictest2524886-21unixbench_multi00:15:591
221524999269,15cyclictest2440800-21kworker/0:1+events22:55:450
221524999269,15cyclictest2237152-21kworker/0:0+events22:25:470
2215249992621,3cyclictest1-21systemd22:45:010
2215249992610,14cyclictest2237152-21kworker/0:0+events19:55:540
2215254992521,2cyclictest2491158-21cpuspeed_turbos23:45:442
2215254992521,2cyclictest2406731-21fschecks_count22:20:442
2215254992520,3cyclictest2273411-21kthreadcore20:05:512
2215254992519,4cyclictest2391334-21kworker/u8:1+flush-8:022:30:532
2215254992519,4cyclictest2391334-21kworker/u8:1+flush-8:022:30:522
2215254992517,6cyclictest0-21swapper/223:45:002
2215249992520,3cyclictest2316537-21unixbench_singl20:45:590
2215249992519,4cyclictest2439678-21wc22:50:530
2215249992518,5cyclictest2183655-21kworker/u8:2+flush-8:019:20:560
2215249992517,5cyclictest2396278-21latency_hist22:10:010
2215254992420,3cyclictest2362208-21fschecks_count21:35:452
2215254992420,2cyclictest2277503-21aten2_r5power_p20:10:452
2215254992416,6cyclictest2345166-21cut21:15:512
2215251992420,3cyclictest2232193-21kworker/u8:3+events_unbound21:15:451
2215251992418,4cyclictest2499714-21kworker/u8:1+flush-8:000:06:001
2215251992418,4cyclictest2499714-21kworker/u8:1+flush-8:000:05:591
221525499239,12cyclictest413-21systemd-logind22:05:592
221525499231,9cyclictest24019132basename22:15:472
2215254992319,2cyclictest2332310-21fschecks_count21:05:452
2215254992314,6cyclictest2287270-21awk20:20:002
221524999237,15cyclictest2440800-21kworker/0:1+events23:10:490
221524999235,17cyclictest2237152-21kworker/0:0+events21:18:230
221524999235,16cyclictest2237152-21kworker/0:0+events20:15:570
221524999232,5cyclictest2261660-21unixbench_multi19:51:010
221524999232,20cyclictest0-21swapper/020:26:000
2215249992319,2cyclictest0-21swapper/022:19:590
2215249992319,2cyclictest0-21swapper/019:40:000
2215249992318,4cyclictest2355838-21kworker/u8:2+flush-8:021:26:180
2215254992219,2cyclictest2501179-21fschecks_time23:55:462
2215254992218,2cyclictest0-21swapper/222:40:502
2215254992218,2cyclictest0-21swapper/200:20:012
2215251992218,2cyclictest2391782-21cpuspeed22:05:441
2215254992117,3cyclictest2465676-21cron23:20:002
2215254992117,2cyclictest0-21swapper/222:45:522
221524999212,17cyclictest2237152-21kworker/0:0+events20:30:520
2215249992119,1cyclictest0-21swapper/022:20:590
2215249992116,3cyclictest2241746-21unixbench_singl19:31:160
2215249992116,3cyclictest2241746-21unixbench_singl19:31:150
2215254992017,2cyclictest2381456-21latency_hist21:55:002
2215254992016,2cyclictest0-21swapper/223:24:592
2215254992016,2cyclictest0-21swapper/222:30:002
2215254992016,2cyclictest0-21swapper/219:30:552
2215254992016,2cyclictest0-21swapper/219:30:542
2215254992016,2cyclictest0-21swapper/219:15:022
221525499201,5cyclictest0-21swapper/219:10:422
2215254992012,6cyclictest2236531-21sed19:26:002
2215249992016,2cyclictest0-21swapper/000:00:420
2215249992010,8cyclictest2287565-21apt-get20:20:430
221525499192,3cyclictest23161812rudp20:45:572
2215254991916,2cyclictest2519960-21unixbench_multi00:11:022
2215254991916,2cyclictest2331321-21unixbench_multi21:01:022
2215254991915,3cyclictest0-21swapper/223:31:012
2215254991915,2cyclictest2451162-21cpuspeed_turbos23:05:432
2215254991915,2cyclictest0-21swapper/223:25:432
2215254991915,2cyclictest0-21swapper/221:40:452
2215254991914,3cyclictest0-21swapper/221:10:542
2215254991914,3cyclictest0-21swapper/221:10:542
2215254991910,7cyclictest2482051-21tr23:35:482
22148992193,9sleep30-21swapper/319:07:373
24482462180,4sleep2371rcuc/223:00:502
221525499189,7cyclictest2302061-21cron20:35:002
221525499183,13cyclictest2440227-21xargs22:51:022
2215254991814,3cyclictest2297131-21cron20:30:002
2215254991814,2cyclictest0-21swapper/219:35:432
2215254991814,2cyclictest0-21swapper/200:05:472
2215254991814,2cyclictest0-21swapper/200:05:472
221525499181,15cyclictest2231718-21grep19:21:032
221525499181,12cyclictest2256363-21/usr/sbin/munin19:45:562
2215254991810,6cyclictest2533530-21tr00:25:522
2215254991810,6cyclictest2500644-21sort23:55:002
2215254991810,6cyclictest2267265-21true20:00:012
221525499173,2cyclictest421rcuog/222:55:542
2215254991713,2cyclictest0-21swapper/221:48:012
2215254991713,2cyclictest0-21swapper/220:55:452
2215254991713,2cyclictest0-21swapper/220:20:542
23225022160,2sleep3481ktimers/320:55:473
2215254991613,2cyclictest2356240-21unixbench_multi21:26:032
221525499161,11cyclictest2396571-21apt-get22:10:422
221525499152,4cyclictest0-21swapper/221:30:352
2215254991510,3cyclictest0-21swapper/222:35:522
2215254991510,3cyclictest0-21swapper/200:35:492
24659032132,6sleep30-21swapper/323:20:023
24596412132,7sleep30-21swapper/323:10:553
23764912132,6sleep30-21swapper/321:50:023
23677872132,7sleep30-21swapper/321:40:513
22936292130,3sleep375-21kworker/3:1+events20:25:503
22771442132,6sleep30-21swapper/320:10:023
22661482132,7sleep30-21swapper/319:55:543
22620702132,7sleep30-21swapper/319:55:013
22475872132,7sleep30-21swapper/319:40:453
22158542132,6sleep30-21swapper/319:10:473
25372842122,6sleep30-21swapper/300:30:533
25206402122,6sleep30-21swapper/300:15:403
25005532122,6sleep30-21swapper/323:55:003
24958952122,6sleep30-21swapper/323:50:443
24757992122,5sleep30-21swapper/323:30:013
24710962122,6sleep30-21swapper/323:25:043
24506312122,5sleep30-21swapper/323:05:003
24493922122,6sleep30-21swapper/323:00:523
24414232122,6sleep30-21swapper/322:55:473
24322652122,6sleep30-21swapper/322:45:513
24265792122,6sleep30-21swapper/322:40:463
24256252122,6sleep30-21swapper/322:36:033
24172222122,6sleep30-21swapper/322:30:513
24172222122,6sleep30-21swapper/322:30:513
24012332122,5sleep30-21swapper/322:15:023
24003422123,6sleep30-21swapper/322:10:563
23883952123,6sleep30-21swapper/322:00:533
23757942122,6sleep30-21swapper/321:46:003
23517372122,5sleep30-21swapper/321:25:023
23470892122,6sleep30-21swapper/321:20:433
23329492122,6sleep30-21swapper/321:05:523
23276122122,6sleep30-21swapper/321:00:503
23119722122,5sleep30-21swapper/320:45:013
23076652122,6sleep30-21swapper/320:40:483
23021852122,6sleep30-21swapper/320:35:023
22976882122,6sleep30-21swapper/320:30:493
22826102122,6sleep30-21swapper/320:15:443
22528872122,6sleep30-21swapper/319:45:463
22470122121,6sleep30-21swapper/319:40:013
22329312122,6sleep30-21swapper/319:25:483
22266192122,6sleep30-21swapper/319:16:013
25409102112,5sleep30-21swapper/300:35:063
25345122112,5sleep30-21swapper/300:25:553
25157082112,6sleep30-21swapper/300:10:423
25107432112,6sleep30-21swapper/300:05:173
25107432112,6sleep30-21swapper/300:05:163
25103152112,6sleep30-21swapper/300:02:483
24908592112,5sleep30-21swapper/323:45:023
24859452112,6sleep30-21swapper/323:40:213
24814412112,5sleep30-21swapper/323:35:473
24646452112,5sleep30-21swapper/323:15:533
24361762112,5sleep30-21swapper/322:50:443
24114182112,5sleep30-21swapper/322:25:443
24064682112,6sleep30-21swapper/322:20:433
23919612112,5sleep30-21swapper/322:05:473
23829772112,5sleep30-21swapper/321:55:523
23618902112,5sleep30-21swapper/321:35:403
23569322112,5sleep30-21swapper/321:30:443
23421302112,5sleep30-21swapper/321:15:433
23366842112,5sleep30-21swapper/321:10:013
23366842112,5sleep30-21swapper/321:10:003
23180842112,5sleep30-21swapper/320:50:513
22875822112,6sleep30-21swapper/320:20:443
22746372112,5sleep30-21swapper/320:05:523
22675492112,5sleep30-21swapper/320:00:433
22374392112,5sleep30-21swapper/319:30:463
22374392112,5sleep30-21swapper/319:30:463
22274832112,5sleep30-21swapper/319:20:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional