You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 08:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sun Jun 14, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
255890021130,2chrt75-21kworker/3:1-events21:50:313
24869302880,3sleep375-21kworker/3:1+events20:35:363
2397529997166,4cyclictest2543074-21kworker/u8:1+events_unbound21:45:421
2397529997063,5cyclictest2275183-21kworker/u8:2+flush-8:020:00:421
2397524996858,7cyclictest2459245-21diskstats20:10:310
2397524996850,15cyclictest2583017-21apt-get22:15:240
2397524996849,17cyclictest2543533-21apt-get21:35:250
2397529996660,4cyclictest2533076-21kworker/u8:2+events_unbound23:05:281
2397529996659,6cyclictest2533076-21kworker/u8:2+events_unbound22:15:261
2397529996659,6cyclictest2275183-21kworker/u8:2+events_unbound19:10:431
2397529996659,5cyclictest2603064-21kworker/u8:0+flush-8:023:20:401
2397529996659,5cyclictest2478155-21kworker/u8:1+events_unbound20:45:301
2397529996559,4cyclictest2355418-21kworker/u8:0+flush-8:021:50:451
2397529996558,4cyclictest2687141-21kworker/u8:2+flush-8:000:00:431
2397529996458,5cyclictest2275183-21kworker/u8:2+flush-8:019:15:291
2397529996458,4cyclictest2543074-21kworker/u8:1+events_unbound22:05:291
2397529996458,4cyclictest2543074-21kworker/u8:1+events_unbound22:05:291
2397529996357,4cyclictest2580625-21kworker/u8:3+flush-8:022:50:461
2397529996258,3cyclictest933800-21kworker/u8:3+events_unbound20:20:361
2397529996257,4cyclictest2603064-21kworker/u8:0+events_unbound23:50:431
2397524996253,6cyclictest2482756-21ntpq20:30:390
2397531996137,20cyclictest1704890-21systemd-journal00:25:002
2397529996155,4cyclictest2275183-21kworker/u8:2+flush-8:020:30:471
2397529996151,8cyclictest2652600-21/usr/sbin/munin23:25:381
2397529996053,5cyclictest933800-21kworker/u8:3+flush-8:019:45:391
2397529996053,5cyclictest933800-21kworker/u8:3+flush-8:019:45:391
2397529996053,5cyclictest2603064-21kworker/u8:0+flush-8:022:35:411
2397529996053,5cyclictest2603064-21kworker/u8:0+events_unbound23:55:291
2397529995952,5cyclictest2275183-21kworker/u8:2+events_unbound19:30:411
2397529995854,3cyclictest2687141-21kworker/u8:2+flush-8:000:10:461
2397529995853,4cyclictest2533076-21kworker/u8:2+flush-8:022:20:471
2397529995853,4cyclictest2355418-21kworker/u8:0+events_unbound21:05:311
2397529995853,4cyclictest2355418-21kworker/u8:0+events_unbound20:55:291
2397524995849,7cyclictest2721424-21ntpq00:30:390
2397529995753,3cyclictest2275183-21kworker/u8:2+events_unbound20:25:471
2397529995751,4cyclictest2687141-21kworker/u8:2+events_unbound00:25:381
2397529995751,4cyclictest2687141-21kworker/u8:2+events_unbound00:25:371
26647702560,2chrt0-21swapper/023:35:340
2397529995648,6cyclictest2622285-21kworker/u8:1+events_unbound00:30:411
2397529995551,3cyclictest2355418-21kworker/u8:0+flush-8:019:50:441
2397524995548,5cyclictest2707768-21apt-get00:20:260
2397524995542,11cyclictest1704890-21systemd-journal20:50:000
2397524995529,9cyclictest2447972-21ntpq19:55:430
2397529995450,3cyclictest933800-21kworker/u8:3+flush-8:019:40:451
2397529995447,5cyclictest2478155-21kworker/u8:1+events_unbound21:10:431
2397524995446,6cyclictest2487743-21ntpq20:35:390
2397529995347,5cyclictest2355418-21kworker/u8:0+events_unbound21:55:331
2397529995347,4cyclictest2533076-21kworker/u8:2+events_unbound22:30:331
2397524995346,4cyclictest2593167-21apt-get22:25:250
25197202520,1sleep02519726-21cut21:10:360
2397529995248,3cyclictest2543074-21kworker/u8:1+events_unbound22:00:451
2397529995248,3cyclictest2275183-21kworker/u8:2+flush-8:019:20:451
2397529995246,4cyclictest2603064-21kworker/u8:0+events_unbound23:00:381
2397529995245,6cyclictest2543074-21kworker/u8:1+flush-8:022:25:411
2397529995241,9cyclictest1704890-21systemd-journal21:25:011
2397529995241,8cyclictest2527544-21ntpq21:15:421
2397524995241,8cyclictest2631613-21ntpq23:00:370
2397529995144,5cyclictest2533076-21kworker/u8:2+flush-8:021:40:401
2397529995139,8cyclictest2478155-21kworker/u8:1+flush-8:020:40:421
2397529995044,4cyclictest2533076-21kworker/u8:2+events_unbound21:35:411
2397524995044,4cyclictest2603061-21apt-get22:35:260
2397524995041,6cyclictest2443108-21ntpq19:50:430
2397529994945,3cyclictest2603064-21kworker/u8:0+flush-8:022:55:471
2397529994944,3cyclictest2533076-21kworker/u8:2+events_unbound23:15:331
2397529994939,8cyclictest2671464-21ntpq23:40:421
2397524994939,8cyclictest2571978-21ntpq22:00:400
2397524994925,8cyclictest2562071-21ntpq21:50:380
27076512480,2sleep3481ktimers/300:20:003
2397529994844,3cyclictest2498195-21kworker/u8:3+flush-8:021:30:301
2397529994841,5cyclictest2275183-21kworker/u8:2+events_unbound19:25:401
2397524994838,8cyclictest2428218-21ntpq19:35:400
2397524994821,9cyclictest2457850-21ntpq20:05:420
2397531994740,4cyclictest2687131-40logrotate23:59:592
2397529994743,3cyclictest2622285-21kworker/u8:1+events_unbound23:45:271
2397529994742,4cyclictest2580625-21kworker/u8:3+flush-8:022:45:301
2397529994740,5cyclictest2661907-21kworker/u8:3+events_unbound00:15:461
2397524994736,8cyclictest2611855-21ntpq22:40:400
2397524994735,9cyclictest2413378-21ntpq19:20:380
2397524994728,3cyclictest2434053-21awk19:45:000
2397531994637,6cyclictest2647372-21cat23:20:002
2397524994634,9cyclictest2452911-21ntpq20:00:390
2397524994626,3cyclictest2618174-21cpuspeed_turbos22:50:290
2397529994541,3cyclictest2721877-21kworker/u8:3+events_unbound00:35:471
2397529994541,3cyclictest2687141-21kworker/u8:2+events_unbound00:24:491
2397529994540,3cyclictest2355418-21kworker/u8:0+events_unbound21:00:361
2397529994538,5cyclictest2687141-21kworker/u8:2+events_unbound00:05:421
2397524994535,8cyclictest2661279-21ntpq23:30:380
2397529994440,3cyclictest2533076-21kworker/u8:2+events_unbound21:25:291
2397531994336,5cyclictest2667456-21store23:40:012
2397531994334,6cyclictest2439267-21apt-get19:50:252
2397524994332,8cyclictest2537407-21ntpq21:25:380
2397531994226,14cyclictest2618035-21aten2_r5power_p22:50:272
2397529994238,3cyclictest2355418-21kworker/u8:0+events_unbound22:10:301
2397524994220,9cyclictest2401526-21ntpq19:10:390
26749562410,1sleep02674960-21kthreadcore23:45:350
24126132410,6sleep3451irq_work/319:20:363
2397531994125,14cyclictest2637863-21aten2_r5power_v23:10:282
2397529994137,3cyclictest2603064-21kworker/u8:0+events_unbound22:40:381
2397529994137,3cyclictest2478155-21kworker/u8:1+events_unbound20:35:341
2397524994132,7cyclictest2632816-21/usr/sbin/munin23:05:440
2397529994036,3cyclictest2533076-21kworker/u8:2+events_unbound23:30:281
2397531993824,12cyclictest2524034-21fschecks_count21:15:302
2397529993834,3cyclictest2275183-21kworker/u8:2+events_unbound20:10:461
2397529993833,4cyclictest2661907-21kworker/u8:3+events_unbound23:35:281
2397524993830,6cyclictest2488901-21/usr/sbin/munin20:40:380
2397524993830,6cyclictest1704890-21systemd-journal22:00:000
2397524993827,8cyclictest2626638-21ntpq22:55:400
2397524993822,4cyclictest2617295-21unixbench_singl22:45:460
2397529993731,5cyclictest2275183-21kworker/u8:2+events_unbound20:05:411
2397524993730,5cyclictest2576948-21ntpq22:05:420
2397524993730,5cyclictest2576948-21ntpq22:05:420
239753199362,26cyclictest2528706-21apt-get21:20:292
2397531993520,13cyclictest2697887-21fschecks_count00:10:302
2397524993529,4cyclictest2591987-21ntpq22:20:420
239753199349,10cyclictest0-21swapper/222:10:472
239753199343,24cyclictest2563290-21apt-get21:55:282
239753199342,25cyclictest2513898-21apt-get21:05:272
2397531993413,18cyclictest2438014-21grep19:45:382
2397531993413,18cyclictest2438014-21grep19:45:382
2397529993429,4cyclictest2603064-21kworker/u8:0+flush-8:023:10:461
239753199334,23cyclictest2657537-21apt-get23:30:272
2397531993320,11cyclictest2459215-21cpuspeed_turbos20:10:292
2397531993319,12cyclictest2409703-21aten2_r5power_p19:20:282
2397529993329,3cyclictest2275183-21kworker/u8:2+events_unbound19:57:271
2397524993328,4cyclictest2721877-21kworker/u8:3+flush-8:000:38:160
2397524993321,9cyclictest2507471-21ntpq20:55:400
23970662333,9sleep10-21swapper/119:06:091
239753199322,24cyclictest2507565-21ntpq20:55:412
239753199322,23cyclictest2632841-21apt-get23:05:262
2397529993228,3cyclictest2355418-21kworker/u8:0+events_unbound20:53:361
2397529993227,4cyclictest933800-21kworker/u8:3+flush-8:019:35:501
2397524993226,4cyclictest2647931-21fschecks_count23:20:300
23972172323,7sleep00-21swapper/019:08:050
23970922324,8sleep20-21swapper/219:06:312
2397531993116,13cyclictest2583102-21aten2_r5power_p22:15:252
2397531993115,14cyclictest0-21swapper/221:00:312
2397531993112,16cyclictest2474890-21tr20:25:342
239752499318,21cyclictest2508463-21kworker/0:2+events21:45:300
2397524993123,6cyclictest2355418-21kworker/u8:0+flush-8:021:00:500
239752499311,28cyclictest2458754-21kworker/0:1+events20:22:040
239752499311,27cyclictest1-21systemd19:30:010
2397524993111,14cyclictest2657294-21kworker/0:2+events23:40:290
239753199302,24cyclictest2681534-21python323:50:452
239753199302,22cyclictest2429401-21apt-get19:40:272
2397531993012,15cyclictest2493871-21find20:45:272
239753199301,10cyclictest2627837-21/usr/sbin/munin23:00:242
239752499304,18cyclictest2672361-21kworker/0:0+events00:00:110
2397524993023,3cyclictest0-21swapper/000:15:480
2397524993020,8cyclictest2502610-21ntpq20:50:380
239753199293,22cyclictest2537195-21wc21:25:362
239753199291,22cyclictest2692889-21diskstats00:05:292
2397524992923,4cyclictest2603064-21kworker/u8:0+flush-8:023:11:170
239752499292,26cyclictest2397406-21kworker/0:0+events19:31:330
239752499292,25cyclictest2508463-21kworker/0:2+events21:40:340
239752499292,22cyclictest2627604-21kworker/0:1+events23:15:060
239753199286,18cyclictest2447872-21/usr/sbin/munin19:55:402
239753199281,9cyclictest2677007-21sshd23:46:472
239753199281,9cyclictest2603060-21/usr/sbin/munin22:35:262
2397531992814,12cyclictest0-21swapper/219:30:432
2397531992812,14cyclictest2671910-21unixbench_multi23:40:462
2397531992811,15cyclictest2654226-21tr23:25:342
2397531992811,14cyclictest2502286-21tr20:50:362
2397531992811,13cyclictest0-21swapper/222:05:322
2397531992811,13cyclictest0-21swapper/222:05:312
2397529992824,2cyclictest2468177-21unixbench_multi20:15:461
239753199278,16cyclictest2546848-21tr21:35:362
239753199276,17cyclictest2538701-21cut21:30:272
239753199272,20cyclictest2408396-21ntpq19:15:392
239753199271,8cyclictest2479247-21/usr/sbin/munin20:30:312
239753199271,8cyclictest2452896-21ntpq20:00:392
2397531992712,12cyclictest0-21swapper/221:50:292
239753199271,10cyclictest2458721-21/usr/sbin/munin20:10:012
239753199270,9cyclictest2557157-21/usr/sbin/munin21:45:382
239753199270,9cyclictest2487768-21ntpq20:35:412
239752499276,15cyclictest2572930-21kworker/0:1+events22:10:020
2397524992722,3cyclictest1-21systemd00:00:000
27134482260,3sleep375-21kworker/3:1+mm_percpu_wq00:25:353
27134482260,3sleep375-21kworker/3:1+mm_percpu_wq00:25:343
239753199265,17cyclictest2398892-21cut19:10:342
239753199262,19cyclictest2472637-21ntpq20:20:392
2397531992613,12cyclictest0-21swapper/220:45:002
239753199261,21cyclictest2616739-21ntpq22:45:392
2397531992612,12cyclictest0-21swapper/200:35:472
239753199261,19cyclictest2688248-21ps00:00:342
239753199261,11cyclictest25197262cut21:10:372
2397531992611,12cyclictest0-21swapper/221:40:352
239753199260,9cyclictest2647703-21cut23:20:272
239753199260,8cyclictest2623271-21/usr/sbin/munin22:55:332
239753199260,8cyclictest2588238-21/usr/sbin/munin22:20:292
239752499268,16cyclictest2397406-21kworker/0:0+events19:48:010
239752499268,16cyclictest2397406-21kworker/0:0+events19:48:000
2397524992621,3cyclictest2539521-21kthreadcore21:30:340
2397524992613,10cyclictest1704890-21systemd-journal23:50:460
239753199258,15cyclictest2593488-21cut22:25:312
239753199252,19cyclictest2428287-21ntpq19:35:432
2397531992520,3cyclictest2418831-21unixbench_singl19:25:452
2397531992511,12cyclictest0-21swapper/222:00:392
239753199251,10cyclictest2722368-21munin-run00:35:002
239753199250,8cyclictest2608299-21/usr/sbin/munin22:40:302
239752499253,16cyclictest2458754-21kworker/0:1+events20:25:290
239753199241,8cyclictest2464335-21/usr/sbin/munin20:15:312
239753199240,8cyclictest2707636-21wc00:20:012
239753199240,8cyclictest2600892-21cut22:30:352
239753199240,10cyclictest2713168-21/usr/sbin/munin00:25:332
239753199240,10cyclictest2713168-21/usr/sbin/munin00:25:322
239752499242,17cyclictest2672361-21kworker/0:0+events00:30:010
239752499242,17cyclictest2672361-21kworker/0:0+events00:30:000
2397524992420,2cyclictest0-21swapper/021:05:420
2397524992416,6cyclictest2687141-21kworker/u8:2+flush-8:000:05:450
2397524992414,3cyclictest0-21swapper/020:15:270
239752499237,15cyclictest2508463-21kworker/0:2+events21:20:330
239752499232,20cyclictest0-21swapper/000:10:460
2397524992220,1cyclictest0-21swapper/022:30:440
2397524992117,2cyclictest0-21swapper/023:25:350
2397524992117,2cyclictest0-21swapper/021:15:480
2397524992116,3cyclictest2408935-21unixbench_singl19:15:460
246775722012,5sleep30-21swapper/320:15:433
26077252186,6sleep30-21swapper/322:40:003
23930922183,8sleep30-21swapper/319:05:323
24195192170,2sleep375-21kworker/3:1+mm_percpu_wq19:30:263
25422842167,5sleep30-21swapper/321:30:393
24392412142,8sleep30-21swapper/319:50:273
27222692132,6sleep30-21swapper/300:34:023
26485532132,7sleep30-21swapper/323:20:353
25928722132,6sleep30-21swapper/322:25:003
25818132132,7sleep30-21swapper/322:10:403
25322832132,7sleep30-21swapper/321:20:393
25235172132,6sleep30-21swapper/321:15:013
25178922132,7sleep30-21swapper/321:05:473
24985402132,6sleep30-21swapper/320:50:023
24926372134,5sleep30-21swapper/320:40:393
23977822132,7sleep30-21swapper/319:10:293
27226422123,6sleep30-21swapper/300:35:273
27076512122,6sleep30-21swapper/300:20:023
26946862123,5sleep30-21swapper/300:05:373
26885442122,6sleep30-21swapper/300:00:353
26864682122,6sleep30-21swapper/323:55:463
26692462122,6sleep30-21swapper/323:40:363
26572812122,5sleep30-21swapper/323:30:023
26529702122,6sleep30-21swapper/323:25:313
26325792122,5sleep30-21swapper/323:05:023
26318602122,6sleep30-21swapper/323:00:443
26227312122,6sleep30-21swapper/322:55:023
26170972122,6sleep30-21swapper/322:45:473
26085332122,6sleep30-21swapper/322:40:343
25978462122,6sleep30-21swapper/322:30:023
25731492122,6sleep30-21swapper/322:05:263
25731492122,6sleep30-21swapper/322:05:253
25487682122,6sleep30-21swapper/321:40:303
25338192122,6sleep30-21swapper/321:25:293
25193452120,2sleep3481ktimers/321:10:363
24950352120,8sleep30-21swapper/320:45:353
24742592123,5sleep30-21swapper/320:25:333
24590002122,6sleep30-21swapper/320:10:283
24581432122,6sleep30-21swapper/320:05:483
24496232122,6sleep30-21swapper/320:00:343
24441772122,6sleep30-21swapper/319:55:273
24329872122,6sleep30-21swapper/319:40:393
24241692122,5sleep30-21swapper/319:35:023
24027422122,6sleep30-21swapper/319:15:033
26975952112,6sleep30-21swapper/300:10:273
26775532112,6sleep30-21swapper/323:50:283
26725622112,5sleep30-21swapper/323:45:033
26628042112,5sleep30-21swapper/323:35:313
26465472112,5sleep30-21swapper/323:15:413
26381202112,5sleep30-21swapper/323:10:303
26179172112,6sleep30-21swapper/322:50:203
25931712112,6sleep30-21swapper/322:25:293
25842702112,5sleep30-21swapper/322:15:343
25682172112,5sleep30-21swapper/322:00:293
25672962112,5sleep30-21swapper/321:55:473
25571862111,6sleep30-21swapper/321:45:403
25091302112,5sleep30-21swapper/321:00:323
25080912112,6sleep30-21swapper/320:56:083
24789662112,5sleep30-21swapper/320:30:283
24688892112,6sleep30-21swapper/320:20:283
24343672112,5sleep30-21swapper/319:45:293
24343672112,5sleep30-21swapper/319:45:283
24144972112,5sleep30-21swapper/319:25:023
25434802102,5sleep30-21swapper/321:35:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional