You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 15:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Thu Apr 02, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
250012521710,3sleep375-21kworker/3:1-events08:41:313
256759521090,3sleep1161rcu_preempt09:51:231
26734712820,5chrt0-21swapper/211:36:312
2432146997062,5cyclictest2401445-21kworker/u8:0+events_unbound07:20:010
2432146997062,5cyclictest2401445-21kworker/u8:0+events_unbound07:20:010
26925912630,5sleep20-21swapper/211:56:282
2432149996253,6cyclictest280-21systemd-journal07:15:451
2432149996253,6cyclictest280-21systemd-journal07:15:451
26977952490,4sleep375-21kworker/3:1+mm_percpu_wq12:01:323
2432149994943,4cyclictest2522363-21kworker/u8:1+events_unbound09:20:001
2432149994943,4cyclictest2444678-21kworker/u8:2+events_unbound10:01:371
2432146994639,5cyclictest2665972-21base6411:30:000
2432146994536,7cyclictest280-21systemd-journal08:51:380
2432149994339,3cyclictest2522363-21kworker/u8:1+events_unbound09:11:431
2432146994336,5cyclictest2641188-21latency_hist11:05:010
2432149994238,3cyclictest2621642-21kworker/u8:3+events_unbound12:05:011
2432149994237,3cyclictest2670640-21kworker/u8:1+events_unbound11:50:001
2432149994237,3cyclictest2670640-21kworker/u8:1+events_unbound11:50:001
2432149994235,5cyclictest2551994-21kworker/u8:0+flush-8:010:41:361
2432149994235,5cyclictest2468187-21kworker/u8:0+events_unbound08:16:391
2432149994235,4cyclictest2444678-21kworker/u8:2+flush-8:008:41:351
2432149994136,4cyclictest2551994-21kworker/u8:0+flush-8:010:06:401
2432149994134,5cyclictest2512170-21kworker/u8:3+events_unbound09:01:251
2432149994131,8cyclictest280-21systemd-journal09:20:101
2432149994035,4cyclictest2621642-21kworker/u8:3+events_unbound12:11:271
2432149994033,5cyclictest2412797-21kworker/u8:1+events_unbound07:31:181
2432149993934,4cyclictest2596948-21kworker/u8:2+events_unbound10:26:231
2432149993933,5cyclictest2444678-21kworker/u8:2+flush-8:008:56:361
2432149993933,3cyclictest2456211-21kworker/u8:1+flush-8:008:51:391
25237062380,4sleep20-21swapper/209:06:302
2432151993833,4cyclictest2670640-21kworker/u8:1+flush-8:011:41:292
2432149993834,3cyclictest2444678-21kworker/u8:2+events_unbound08:30:011
2432149993831,5cyclictest280-21systemd-journal07:40:011
2432146993832,4cyclictest2512170-21kworker/u8:3+flush-8:009:01:260
2432149993732,4cyclictest2719826-21kworker/u8:2+flush-8:012:27:151
2432149993732,3cyclictest2665988-21apt-key11:30:001
2432149993731,4cyclictest2551994-21kworker/u8:0+flush-8:011:06:271
2432149993730,6cyclictest2439209-21kworker/u8:3+flush-8:008:01:351
2432149993729,6cyclictest2719826-21kworker/u8:2+events_unbound12:31:331
2432146993730,5cyclictest280-21systemd-journal11:10:000
2432146993729,6cyclictest280-21systemd-journal11:25:000
2432149993631,3cyclictest2670640-21kworker/u8:1+events_unbound12:21:331
2432149993631,3cyclictest2652126-21irqrtprio11:16:281
2432149993630,4cyclictest2512170-21kworker/u8:3+flush-8:009:31:371
2432149993630,3cyclictest2670640-21kworker/u8:1+events_unbound12:16:361
2432149993629,5cyclictest2621642-21kworker/u8:3+flush-8:011:21:341
2432149993629,5cyclictest2596948-21kworker/u8:2+flush-8:010:21:371
2432149993628,6cyclictest280-21systemd-journal10:35:001
2432149993531,3cyclictest2444678-21kworker/u8:2+flush-8:009:36:391
2432149993531,3cyclictest2431648-21kworker/u8:2+events_unbound07:11:361
2432146993529,4cyclictest280-21systemd-journal10:43:100
2432146993528,5cyclictest280-21systemd-journal07:56:220
24318382354,9sleep10-21swapper/107:08:111
24318382354,9sleep10-21swapper/107:08:111
2432149993429,4cyclictest2450344-21kworker/u8:0+events_unbound07:41:191
2432149993429,3cyclictest2547038-21cron09:30:001
2432149993428,5cyclictest2591385-21kworker/u8:1+flush-8:010:46:251
2432149993428,4cyclictest2591385-21kworker/u8:1+flush-8:011:11:361
2432149993427,5cyclictest2621642-21kworker/u8:3+flush-8:011:31:371
2432149993424,8cyclictest2695021-21ntpq11:56:331
2432149993423,8cyclictest2481982-21ntpq08:21:331
2432146993429,3cyclictest2592789-21irqrtprio10:16:290
2432146993429,3cyclictest2439738-21awk07:20:000
2432146993429,3cyclictest2439738-21awk07:20:000
2432146993423,8cyclictest280-21systemd-journal11:35:000
2432149993329,3cyclictest2551994-21kworker/u8:0+events_unbound09:41:391
2432149993328,4cyclictest2621642-21kworker/u8:3+flush-8:010:56:261
2432149993328,4cyclictest2591385-21kworker/u8:1+events_unbound11:01:381
2432149993328,4cyclictest2551994-21kworker/u8:0+events_unbound11:36:251
2432149993325,6cyclictest280-21systemd-journal11:41:111
2432149993325,6cyclictest2439209-21kworker/u8:3+flush-8:007:21:321
2432149993325,6cyclictest2439209-21kworker/u8:3+flush-8:007:21:321
2432149993324,6cyclictest280-21systemd-journal07:50:001
2432146993326,5cyclictest280-21systemd-journal10:48:100
2432146993319,12cyclictest2562843-21fschecks_time09:46:250
2432151993227,3cyclictest2437501-21latency07:15:022
2432151993227,3cyclictest2437501-21latency07:15:022
2432149993228,3cyclictest2412797-21kworker/u8:1+events_unbound07:25:031
2432149993227,4cyclictest2444678-21kworker/u8:2+events_unbound09:06:341
2432149993225,5cyclictest2670640-21kworker/u8:1+flush-8:011:51:371
2432149993225,5cyclictest2572396-21tr09:56:231
2432151993127,3cyclictest2522363-21kworker/u8:1+events_unbound09:26:382
2432151993126,3cyclictest2670640-21kworker/u8:1+flush-8:012:06:262
2432149993126,4cyclictest2596948-21kworker/u8:2+flush-8:010:36:331
2432149993126,3cyclictest2497634-21sendmail08:40:001
2432149993126,3cyclictest2444678-21kworker/u8:2+events_unbound08:35:011
2432149993125,4cyclictest2631299-21latency_hist10:55:011
2432149993125,4cyclictest2592070-21apt-get10:16:221
2432149993125,4cyclictest2444678-21kworker/u8:2+flush-8:010:11:351
2432149993122,7cyclictest2704925-21ntpq12:06:341
2432146993127,3cyclictest2680728-21kworker/u8:2+flush-8:011:46:450
2432146993127,3cyclictest2680728-21kworker/u8:2+flush-8:011:46:450
2432146993127,3cyclictest2581848-21latency_hist10:05:000
2432146993124,5cyclictest280-21systemd-journal07:27:410
26457542300,7sleep3451irq_work/311:06:423
2432149993024,4cyclictest2444678-21kworker/u8:2+flush-8:009:46:301
2432149993024,4cyclictest2444678-21kworker/u8:2+flush-8:008:11:301
2432149993023,5cyclictest2502917-21apt-get08:46:211
2432151992925,2cyclictest280-21systemd-journal12:25:002
2432151992924,3cyclictest2665512-21unixbench_multi11:26:412
2432151992923,4cyclictest2482450-21unixbench_multi08:21:402
2432146992923,4cyclictest2456211-21kworker/u8:1+flush-8:007:51:230
2432146992922,5cyclictest2719826-21kworker/u8:2+events_unbound12:31:370
2432146992921,6cyclictest2571983-21latency_hist09:55:010
2432146992911,16cyclictest2401448-21kworker/0:0+events08:31:360
2432151992824,3cyclictest2591385-21kworker/u8:1+events_unbound10:16:442
2432151992824,2cyclictest2591396-21unixbench_singl10:11:412
2432149992823,4cyclictest2461393-21unixbench_multi07:51:371
2432149992821,5cyclictest2730744-21kworker/u8:1+flush-8:012:36:291
2432146992825,2cyclictest2631324-21systemd-run10:55:000
2432146992823,3cyclictest0-21swapper/009:30:000
2432151992723,3cyclictest2730744-21kworker/u8:1+flush-8:012:36:362
2432151992722,4cyclictest2444678-21kworker/u8:2+events_unbound09:46:392
2432151992722,3cyclictest2446074-21aten2_r5power_v07:31:202
2432146992723,3cyclictest2552251-21systemd-run09:35:000
2432146992722,3cyclictest2714766-21needreboot12:16:310
2432146992719,5cyclictest2710705-21latency_hist12:15:010
24316802272,5sleep00-21swapper/007:06:090
24316802272,5sleep00-21swapper/007:06:090
243215199262,22cyclictest2434557-21grep07:11:272
2432151992621,4cyclictest2591385-21kworker/u8:1+flush-8:010:46:392
2432146992621,3cyclictest2523702-21kthreadcore09:06:300
2432146992621,3cyclictest0-21swapper/008:50:000
2432146992620,4cyclictest2676933-21irqcore11:41:290
2432146992620,4cyclictest2647374-21kthreadcore11:11:300
2432146992619,5cyclictest2444678-21kworker/u8:2+events_unbound09:36:390
25826542250,5sleep3451irq_work/310:06:263
2432151992521,2cyclictest0-21swapper/209:00:002
2432151992519,4cyclictest2685644-21unixbench_singl11:46:422
2432151992519,4cyclictest2685644-21unixbench_singl11:46:422
2432151992518,4cyclictest0-21swapper/208:15:012
2432146992521,2cyclictest2597996-21kthreadcore10:21:300
2432146992521,2cyclictest0-21swapper/009:16:300
2432146992520,4cyclictest2621642-21kworker/u8:3+flush-8:012:03:470
2432146992520,3cyclictest2616136-21unixbench_singl10:36:400
2432146992518,5cyclictest2537791-21fschecks_count09:21:250
2432146992517,6cyclictest2551994-21kworker/u8:0+flush-8:011:16:340
243214699251,22cyclictest2497534-21sshd08:39:110
25615542240,5sleep02561634-21sed09:41:390
2432151992420,2cyclictest2606948-21aten2_r5power_c10:31:232
2432151992420,2cyclictest0-21swapper/207:30:002
2432151992419,4cyclictest2705791-21kworker/u8:0+flush-8:012:30:312
2432151992418,4cyclictest2457872-21irqrtprio07:51:252
2432151992415,7cyclictest280-21systemd-journal12:20:002
2432151992415,7cyclictest2521443-21needreboot09:01:332
243214699242,10cyclictest26925952kthreadcore11:56:290
2432146992420,3cyclictest2528166-21sort09:11:290
2432146992420,3cyclictest2436659-21unixbench_multi07:11:370
2432146992420,2cyclictest2502589-21systemd-run08:45:010
2432146992419,4cyclictest2621642-21kworker/u8:3+flush-8:012:08:590
2432146992419,3cyclictest2467326-21cron08:05:000
2432146992419,3cyclictest0-21swapper/008:56:330
2432146992412,11cyclictest2551994-21kworker/u8:0+flush-8:011:36:260
2432151992320,2cyclictest2721006-21aten2_r5power_e12:26:232
2432151992319,2cyclictest2532884-21fschecks_time09:16:272
2432151992318,3cyclictest2690567-21unixbench_multi11:51:412
2432151992318,3cyclictest2612068-21cpuspeed_turbos10:36:232
2432151992317,4cyclictest2551994-21kworker/u8:0+events_unbound09:40:102
2432151992315,6cyclictest280-21systemd-journal09:40:012
2432151992314,7cyclictest2475415-21kthreadcore08:16:312
2432149992318,3cyclictest2462377-21cron08:00:001
243214699236,16cyclictest2401448-21kworker/0:0+events08:16:370
2432146992319,3cyclictest2487737-21sh08:30:000
2432146992319,2cyclictest2451039-21runrttasks07:38:160
2432146992319,2cyclictest0-21swapper/011:51:410
2432146992318,4cyclictest2596948-21kworker/u8:2+flush-8:010:26:410
2432146992317,4cyclictest2591404-21unixbench_singl10:11:410
2432151992216,4cyclictest2650689-21unixbench_multi11:11:412
2432146992218,3cyclictest2455821-21unixbench_multi07:41:370
2432146992217,4cyclictest2586465-21unixbench_singl10:06:400
2432146992217,4cyclictest2444678-21kworker/u8:2+flush-8:008:23:390
2432146992217,3cyclictest2716348-21fschecks_time12:21:240
243215199212,18cyclictest0-21swapper/208:01:352
2432151992117,2cyclictest0-21swapper/210:51:412
2432151992115,4cyclictest2635889-21unixbench_singl10:56:412
243214699212,14cyclictest589-21runrttasks08:11:180
2432146992114,5cyclictest2412797-21kworker/u8:1+events_unbound07:31:330
2432151992016,3cyclictest2696233-21/usr/sbin/munin12:01:422
2432151992014,4cyclictest2512076-21unixbench_multi08:51:412
2432151992014,4cyclictest2506598-21needreboot08:46:332
2432151992013,5cyclictest280-21systemd-journal12:15:002
2432151992010,8cyclictest2498796-21sed08:41:292
2432149992016,3cyclictest2444678-21kworker/u8:2+flush-8:008:05:311
2432146992016,2cyclictest0-21swapper/009:56:240
2432146992014,4cyclictest2607232-21fschecks_time10:31:250
24383072190,7sleep30-21swapper/307:15:033
24383072190,7sleep30-21swapper/307:15:033
243215199195,2cyclictest24427391kthreadcore07:21:272
243215199195,2cyclictest24427391kthreadcore07:21:272
243215199194,3cyclictest26431402grep11:06:312
2432151991916,2cyclictest2602117-21aten2_r5power_v10:26:232
2432151991915,2cyclictest2640876-21unixbench_singl11:01:422
2432151991915,2cyclictest2567667-21fschecks_count09:51:252
2432151991915,2cyclictest0-21swapper/210:05:012
2432151991914,3cyclictest2586414-21unixbench_multi10:06:402
2432151991914,3cyclictest0-21swapper/210:00:002
2432151991910,7cyclictest2666690-21cpuspeed_turbos11:31:252
2432146991913,4cyclictest2705791-21kworker/u8:0+flush-8:012:26:250
2432146991913,4cyclictest0-21swapper/010:56:320
243215199189,7cyclictest2550174-21tr09:31:312
2432151991814,2cyclictest0-21swapper/207:10:002
2432151991814,2cyclictest0-21swapper/207:10:002
2432151991813,3cyclictest2617030-21cpuspeed_turbos10:41:242
243215199181,12cyclictest2537527-21apt-get09:21:232
243215199180,7cyclictest25971011aten2_r5power_c10:21:242
2432146991814,2cyclictest0-21swapper/012:36:250
2432146991810,6cyclictest280-21systemd-journal07:50:000
24315892183,8sleep30-21swapper/307:05:243
24315892183,8sleep30-21swapper/307:05:243
243215199172,11cyclictest2493030-21apt-get08:36:232
243215199171,7cyclictest2454639-21grep07:41:272
2432151991713,2cyclictest2656558-21cpuspeed_turbos11:21:232
2432151991712,3cyclictest0-21swapper/211:16:412
2432151991712,3cyclictest0-21swapper/207:40:012
243215199171,12cyclictest2527637-21apt-get09:11:232
2432146991710,5cyclictest2467955-21ls08:10:000
243215199168,6cyclictest2490039-21kthreadcore08:31:312
243215199162,13cyclictest0-21swapper/207:59:592
243215199162,13cyclictest0-21swapper/207:48:212
2432151991613,2cyclictest0-21swapper/208:05:142
24627552152,8sleep30-21swapper/308:01:193
243215199152,1cyclictest0-21swapper/208:29:562
27242012130,4sleep3481ktimers/312:26:323
26851272133,6sleep30-21swapper/311:46:363
26851272133,6sleep30-21swapper/311:46:363
26552232132,7sleep30-21swapper/311:16:383
26305582133,7sleep30-21swapper/310:51:383
25918732132,6sleep30-21swapper/310:15:023
25761092132,7sleep30-21swapper/309:56:363
25670752132,6sleep30-21swapper/309:50:023
25633532134,5sleep30-21swapper/309:46:293
25594602133,7sleep30-21swapper/309:41:313
25175922132,6sleep30-21swapper/309:00:023
24927032132,6sleep30-21swapper/308:35:023
27262882122,6sleep30-21swapper/312:31:263
27063672122,6sleep30-21swapper/312:11:283
27008412122,6sleep30-21swapper/312:05:023
26766462122,6sleep30-21swapper/311:41:263
26603672122,6sleep30-21swapper/311:21:403
26489032123,6sleep30-21swapper/311:11:323
26370792122,6sleep30-21swapper/311:01:293
26223532122,6sleep30-21swapper/310:46:283
26171522122,6sleep30-21swapper/310:41:273
26155702122,6sleep30-21swapper/310:36:333
26022872122,6sleep30-21swapper/310:26:263
25970772122,6sleep30-21swapper/310:21:243
25480082122,6sleep30-21swapper/309:31:283
25434012122,6sleep30-21swapper/309:26:303
25227012122,6sleep30-21swapper/309:06:243
25082012122,6sleep30-21swapper/308:51:273
25034692122,6sleep30-21swapper/308:46:293
24885642120,7sleep30-21swapper/308:31:283
24679742122,5sleep30-21swapper/308:10:013
24505742122,6sleep30-21swapper/307:35:023
24464322122,6sleep30-21swapper/307:31:253
24438162122,6sleep30-21swapper/307:21:303
24438162122,6sleep30-21swapper/307:21:303
24322812122,6sleep30-21swapper/307:10:063
27313352112,5sleep30-21swapper/312:36:283
27160072112,5sleep30-21swapper/312:21:193
27110592112,6sleep30-21swapper/312:16:213
26912842112,5sleep30-21swapper/311:56:233
26859942112,5sleep30-21swapper/311:50:013
26713092112,6sleep30-21swapper/311:35:283
26701222112,6sleep30-21swapper/311:31:353
26609612112,5sleep30-21swapper/311:25:013
26106842112,6sleep30-21swapper/310:31:353
25772592112,6sleep30-21swapper/310:01:183
25528352112,5sleep30-21swapper/309:36:263
25374362112,5sleep30-21swapper/309:20:323
25328182112,5sleep30-21swapper/309:16:273
25272992112,5sleep30-21swapper/309:10:013
25127052112,5sleep30-21swapper/308:55:403
24830312112,5sleep30-21swapper/308:25:023
24825272112,5sleep30-21swapper/308:21:413
24745162110,2sleep3481ktimers/308:16:313
24676812112,6sleep30-21swapper/308:06:323
24622552112,6sleep30-21swapper/307:58:303
24575022112,5sleep30-21swapper/307:51:233
24566452112,5sleep30-21swapper/307:48:063
24515942111,6sleep30-21swapper/307:41:203
24455952110,6sleep30-21swapper/307:30:003
26315712102,5sleep30-21swapper/310:55:393
25870192102,5sleep30-21swapper/310:10:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional