You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 14:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed Apr 01, 2026 12:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28766142800,3sleep375-21kworker/3:1-events09:21:323
2743248996140,18cyclictest3013080-21latency_hist11:40:000
2743248996036,21cyclictest2864688-21diskstats09:11:260
2743250995851,5cyclictest2898918-21kworker/u8:2+flush-8:010:46:411
2743250995751,5cyclictest2765114-21kworker/u8:2+flush-8:007:31:411
2743250995649,5cyclictest2789673-21kworker/u8:1+events_unbound08:46:331
2743250995646,8cyclictest280-21systemd-journal09:00:011
2743250995542,9cyclictest280-21systemd-journal07:16:131
2743248995533,20cyclictest2963680-21/usr/sbin/munin10:51:360
2743250995448,4cyclictest2849437-21kworker/u8:0+events_unbound10:15:001
2743248995417,28cyclictest2854789-21diskstats09:01:270
2743250995349,3cyclictest2849437-21kworker/u8:0+events_unbound09:01:411
2743250995348,4cyclictest2733539-21kworker/u8:0+flush-8:007:21:401
2743250995348,3cyclictest2988612-21kworker/u8:0+events_unbound11:31:411
2743250995348,3cyclictest2988612-21kworker/u8:0+events_unbound11:31:411
2743250995346,6cyclictest2898918-21kworker/u8:2+flush-8:010:16:381
2743254995247,4cyclictest2789673-21kworker/u8:1+flush-8:008:01:382
2743254995247,4cyclictest2789673-21kworker/u8:1+flush-8:008:01:382
2743250995245,5cyclictest3027953-21kworker/u8:1+events_unbound11:56:371
2743248995242,7cyclictest2993322-21latency_hist11:20:010
2743248995231,18cyclictest2778919-21ntpq07:41:370
2743248995228,21cyclictest2844298-21latency_hist08:50:010
2743254995142,7cyclictest280-21systemd-journal07:59:162
2743250995145,5cyclictest2953660-21kworker/u8:1+flush-8:011:06:411
2743250995145,5cyclictest2898918-21kworker/u8:2+events_unbound11:41:271
2743250995145,4cyclictest2888567-21kworker/u8:1+flush-8:009:36:311
2743250995143,5cyclictest2824538-21latency_hist08:30:011
2743250995042,6cyclictest2765114-21kworker/u8:2+flush-8:008:21:381
2743248995042,5cyclictest2918500-21latency_hist10:05:010
2743248995039,8cyclictest2894354-21diskstats09:41:260
30370552490,3sleep375-21kworker/3:1+events12:01:403
274324899491,43cyclictest2799805-21sshd08:05:180
28777562480,1sleep02877758-21kthreadcore09:21:330
2743250994841,5cyclictest2765114-21kworker/u8:2+flush-8:008:31:381
2743250994839,7cyclictest2765114-21kworker/u8:2+flush-8:008:06:381
2743250994740,5cyclictest2898918-21kworker/u8:2+events_unbound12:16:371
2743250994739,6cyclictest3022734-21kworker/u8:0+flush-8:012:31:411
2743254994642,3cyclictest2898918-21kworker/u8:2+flush-8:011:11:372
2743250994641,4cyclictest2864340-21kworker/u8:3+flush-8:009:46:421
2743250994640,4cyclictest3022734-21kworker/u8:0+flush-8:011:51:291
2743250994640,4cyclictest2953660-21kworker/u8:1+events_unbound10:51:371
2743250994640,4cyclictest2765114-21kworker/u8:2+events_unbound07:41:411
2743250994639,5cyclictest2789673-21kworker/u8:1+flush-8:008:01:411
2743250994639,5cyclictest2789673-21kworker/u8:1+flush-8:008:01:411
2743250994637,6cyclictest2973199-21latency_hist11:00:011
2743248994636,8cyclictest2879228-21/usr/sbin/munin09:26:350
274324899461,30cyclictest2903073-21python309:46:420
2743250994539,4cyclictest2938157-21kworker/u8:1+flush-8:010:26:311
2743250994539,4cyclictest2898918-21kworker/u8:2+events_unbound11:16:251
2743250994539,4cyclictest2864340-21kworker/u8:3+events_unbound09:15:011
2743250994539,4cyclictest2733539-21kworker/u8:0+events_unbound07:11:391
2743250994538,5cyclictest3027953-21kworker/u8:1+flush-8:012:11:411
2743250994538,5cyclictest2898918-21kworker/u8:2+flush-8:012:21:371
274324899452,42cyclictest2923332-21sshd10:08:590
2743248994524,18cyclictest2913520-21latency_hist10:00:010
2743248994515,20cyclictest0-21swapper/012:40:010
2743250994439,3cyclictest2765114-21kworker/u8:2+events_unbound08:51:421
2743250994439,3cyclictest1892797-21kworker/u8:4+events_unbound07:30:001
2743250994437,6cyclictest2858867-21kworker/u8:2+flush-8:009:21:351
2743250994437,5cyclictest2849437-21kworker/u8:0+flush-8:010:06:281
2743254994339,3cyclictest2953660-21kworker/u8:1+events_unbound10:51:432
2743254994338,4cyclictest2898918-21kworker/u8:2+flush-8:011:01:312
2743254994338,4cyclictest2849437-21kworker/u8:0+events_unbound10:16:312
2743250994337,5cyclictest2789673-21kworker/u8:1+flush-8:007:56:351
2743250994337,4cyclictest2849437-21kworker/u8:0+events_unbound10:45:011
2743248994310,24cyclictest2907846-21ntpq09:51:400
2743250994236,5cyclictest2943791-21kworker/u8:3+flush-8:010:31:381
2743250994235,5cyclictest2733539-21kworker/u8:0+flush-8:007:46:301
2743250994234,6cyclictest3022734-21kworker/u8:0+flush-8:012:26:381
2743250994234,5cyclictest2888777-21latency_hist09:35:011
274324899421,38cyclictest1-21systemd10:23:030
2743248994213,20cyclictest0-21swapper/010:11:340
2743250994135,4cyclictest3022734-21kworker/u8:0+events_unbound12:01:421
2743250994135,4cyclictest2819805-21kworker/u8:3+flush-8:008:36:291
2743250994134,5cyclictest2864340-21kworker/u8:3+flush-8:009:51:371
2743250994134,5cyclictest1892797-21kworker/u8:4+flush-8:008:11:441
2743248994124,15cyclictest3023014-21systemd-run11:50:010
2743254994035,4cyclictest2938157-21kworker/u8:1+events_unbound10:21:422
2743250994035,4cyclictest2898918-21kworker/u8:2+flush-8:011:21:401
2743250994034,4cyclictest2975874-21kworker/u8:0+flush-8:011:01:421
2743250994033,6cyclictest2864340-21kworker/u8:3+events_unbound09:16:371
2743248994033,5cyclictest2873080-21ntpq09:16:350
2743248994017,20cyclictest2952617-21ntpq10:36:360
2743254993934,4cyclictest3008114-21kworker/u8:3+flush-8:011:36:372
2743250993934,4cyclictest2733539-21kworker/u8:0+events_unbound07:40:001
2743250993933,5cyclictest2858867-21kworker/u8:2+events_unbound09:26:391
2743250993933,4cyclictest2938157-21kworker/u8:1+flush-8:010:25:011
2743250993932,5cyclictest2765114-21kworker/u8:2+flush-8:007:51:421
2743250993931,6cyclictest2898918-21kworker/u8:2+flush-8:011:26:381
2743254993833,3cyclictest2953660-21kworker/u8:1+flush-8:010:45:212
2743250993830,6cyclictest3042777-21latency_hist12:10:011
274324899385,23cyclictest0-21swapper/011:25:080
274324899385,23cyclictest0-21swapper/007:20:410
274324899382,24cyclictest0-21swapper/010:55:100
274324899380,27cyclictest589-21runrttasks12:00:430
274324899380,21cyclictest2795180-21tr08:01:250
274324899380,21cyclictest2795180-21tr08:01:250
274324899380,20cyclictest2833793-21/usr/sbin/munin08:36:410
2743254993731,4cyclictest280-21systemd-journal08:21:292
2743250993733,3cyclictest2849437-21kworker/u8:0+events_unbound10:01:301
274324899378,22cyclictest589-21runrttasks08:33:160
274324899378,22cyclictest589-21runrttasks08:29:390
274324899378,21cyclictest589-21runrttasks12:21:460
274324899377,22cyclictest589-21runrttasks11:05:050
274324899377,22cyclictest589-21runrttasks08:58:240
274324899372,23cyclictest0-21swapper/010:48:280
274324899371,25cyclictest2764050-21ntpq07:26:360
274324899370,28cyclictest2781518-21kthreadcore07:46:300
274324899370,20cyclictest2839523-21cut08:45:010
2743254993630,4cyclictest3037569-21kworker/u8:3+flush-8:012:05:262
2743254993630,4cyclictest3022734-21kworker/u8:0+events_unbound11:50:032
274324899363,23cyclictest0-21swapper/010:16:310
2743248993628,6cyclictest2947596-21ntpq10:31:370
274324899362,31cyclictest2892870-21ntpq09:36:350
274324899362,22cyclictest0-21swapper/010:41:140
274324899362,22cyclictest0-21swapper/009:09:480
274324899362,21cyclictest0-21swapper/007:36:500
2743248993619,15cyclictest2987837-21unixbench_singl11:11:440
2743248993618,15cyclictest0-21swapper/011:21:350
274324899361,25cyclictest2888197-21/usr/sbin/munin09:31:400
274324899361,21cyclictest3023401-21aten2_r5power_e11:51:260
274324899361,20cyclictest3043007-21runrttasks12:10:100
274324899361,20cyclictest2813919-21grep08:16:390
274324899360,27cyclictest589-21runrttasks08:53:030
274324899360,22cyclictest589-21runrttasks07:33:050
2743250993527,6cyclictest3047976-21kworker/u8:3+flush-8:012:36:411
274324899358,19cyclictest589-21runrttasks11:04:250
2743248993532,2cyclictest45250irq/128-enp3s0-TxRx-111:56:260
274324899352,24cyclictest0-21swapper/012:16:220
274324899352,22cyclictest2818731-21ntpq08:21:360
274324899352,21cyclictest0-21swapper/007:59:210
274324899352,21cyclictest0-21swapper/007:50:590
274324899352,20cyclictest0-21swapper/010:27:170
2743248993517,16cyclictest3058371-21idleruntime12:26:290
274324899350,19cyclictest2746042-21cut07:11:340
2743254993430,3cyclictest3022734-21kworker/u8:0+events_unbound11:50:012
2743250993429,4cyclictest2858867-21kworker/u8:2+events_unbound09:05:411
2743250993429,3cyclictest401-21dbus-daemon11:40:001
274324899342,23cyclictest0-21swapper/012:32:500
274324899342,16cyclictest0-21swapper/011:44:540
274324899342,16cyclictest0-21swapper/008:10:290
274324899340,20cyclictest3003405-21runrttasks11:30:010
274324899340,20cyclictest3003405-21runrttasks11:30:010
274324899340,19cyclictest2751894-21grep07:16:320
2743254993329,3cyclictest2789673-21kworker/u8:1+events_unbound09:06:422
2743254993325,5cyclictest2983352-21latency_hist11:10:012
2743250993328,4cyclictest2898918-21kworker/u8:2+events_unbound09:55:071
2743250993328,4cyclictest2789673-21kworker/u8:1+events_unbound08:16:251
274324899330,24cyclictest3041719-21latency12:06:320
29872002320,2sleep30-21swapper/311:11:353
2743250993227,3cyclictest2835094-21fschecks_time08:41:271
2743254993127,3cyclictest2938157-21kworker/u8:1+events_unbound10:30:012
274324899312,25cyclictest0-21swapper/007:10:010
2743254993027,2cyclictest2805393-21fschecks_count08:11:282
2743254993025,4cyclictest2898918-21kworker/u8:2+events_unbound10:55:032
2743254993025,4cyclictest2849437-21kworker/u8:0+flush-8:010:06:412
2743254993025,3cyclictest2849437-21kworker/u8:0+events_unbound09:41:342
2743254993024,5cyclictest2765114-21kworker/u8:2+flush-8:007:30:102
2743254993023,5cyclictest2928709-21cat10:15:012
2743254992925,2cyclictest2908292-21unixbench_singl09:51:442
2743254992924,4cyclictest2938157-21kworker/u8:1+events_unbound10:30:072
2743254992924,3cyclictest3053178-21aten2_r5power_v12:21:252
2743250992924,4cyclictest2898918-21kworker/u8:2+events_unbound11:14:491
2743254992823,4cyclictest2898918-21kworker/u8:2+flush-8:012:05:012
2743254992822,4cyclictest2888567-21kworker/u8:1+flush-8:009:34:382
274303822716,8sleep20-21swapper/207:09:532
30153452260,2sleep20-21swapper/211:41:322
2743254992622,3cyclictest2899393-21fschecks_time09:46:282
2743254992615,8cyclictest2849214-21turbostat.cron08:55:002
2743254992615,5cyclictest2989631-21grep11:16:292
2743250992622,3cyclictest2953660-21kworker/u8:1+events_unbound11:45:021
2743250992621,4cyclictest2849437-21kworker/u8:0+flush-8:009:41:471
28559722250,5sleep2371rcuc/209:01:322
2743254992519,4cyclictest2789649-21turbostat.cron07:55:002
2743254992510,10cyclictest2750411-21apt-get07:16:252
2743250992519,4cyclictest2943791-21kworker/u8:3+events_unbound10:36:271
2743254992421,2cyclictest2879473-21cpuspeed_turbos09:26:262
2743254992421,2cyclictest2825240-21fschecks_time08:31:282
2743254992420,2cyclictest3003623-21aten2_r5power_p11:31:262
2743254992420,2cyclictest3003623-21aten2_r5power_p11:31:262
2743254992420,2cyclictest2849645-21aten2_r5power_p08:56:242
274325499237,8cyclictest2909669-21irqrtprio09:56:302
2743254992318,3cyclictest3062970-21aten2_r5power_e12:31:252
27385052234,12sleep10-21swapper/107:06:151
2743254992217,4cyclictest2819805-21kworker/u8:3+events_unbound08:35:252
2743254992217,3cyclictest0-21swapper/207:40:002
2743254992117,3cyclictest2888567-21kworker/u8:1+events_unbound09:35:022
2743254992117,2cyclictest3068175-21fschecks_time12:36:272
2743254992117,2cyclictest2840026-21fschecks_time08:46:252
2743254992116,4cyclictest2789673-21kworker/u8:1+events_unbound08:05:042
2743254992116,4cyclictest2750299-21kworker/u8:1+flush-8:007:25:102
29570992200,5chrt0-21swapper/310:41:343
2743254992017,2cyclictest2755591-21tr07:21:262
2743254992015,4cyclictest2953410-21sendmail10:40:002
2743254992015,3cyclictest0-21swapper/211:26:432
2743254992014,4cyclictest2898918-21kworker/u8:2+flush-8:011:55:102
305303821910,5sleep30-21swapper/312:21:233
2743254991915,2cyclictest0-21swapper/211:21:402
274325499190,4cyclictest2836806-21cut08:41:322
27384352193,9sleep30-21swapper/307:05:203
274325499186,2cyclictest521rcuop/309:21:412
2743254991814,2cyclictest0-21swapper/208:16:292
2743254991813,3cyclictest2776075-21kthreadcore07:41:322
2743254991813,3cyclictest0-21swapper/209:11:402
274325499177,8cyclictest3059543-21tr12:26:322
2743254991714,2cyclictest3052728-21systemd-run12:20:012
2743254991713,2cyclictest0-21swapper/207:46:322
2743254991712,3cyclictest2871444-21kthreadcore09:16:322
274325499170,4cyclictest1265303-21kworker/2:0+events07:11:302
27991022160,3sleep3481ktimers/308:01:423
27991022160,3sleep3481ktimers/308:01:423
274325499161,2cyclictest161rcu_preempt10:01:332
2743254991612,2cyclictest0-21swapper/212:10:502
2743254991612,2cyclictest0-21swapper/210:41:432
30070352154,7sleep30-21swapper/311:31:353
30070352154,7sleep30-21swapper/311:31:353
27741832154,7sleep30-21swapper/307:36:433
274325499154,1cyclictest0-21swapper/208:25:552
28701162145,5sleep30-21swapper/309:16:313
30430802136,5sleep30-21swapper/312:11:083
28979362132,7sleep30-21swapper/309:41:393
28679862132,7sleep30-21swapper/309:11:343
28294982132,6sleep30-21swapper/308:35:023
27866012130,5sleep3451irq_work/307:51:313
30134842122,6sleep30-21swapper/311:41:263
30087032122,6sleep30-21swapper/311:36:283
29940632122,6sleep30-21swapper/311:21:283
29886822122,6sleep30-21swapper/311:16:223
29792302122,6sleep30-21swapper/311:06:313
29722092122,6sleep30-21swapper/310:56:363
29640802122,6sleep30-21swapper/310:51:303
29482882122,6sleep30-21swapper/310:33:043
29388202122,6sleep30-21swapper/310:26:263
29339102122,6sleep30-21swapper/310:21:243
29333352126,4sleep30-21swapper/310:17:393
29135502122,5sleep30-21swapper/310:00:013
29040042122,6sleep30-21swapper/309:51:263
28881092122,6sleep30-21swapper/309:31:403
28542112122,6sleep30-21swapper/309:00:013
28494792122,6sleep30-21swapper/308:55:403
28484452122,6sleep30-21swapper/308:51:393
28435482122,6sleep30-21swapper/308:46:383
28104132122,6sleep30-21swapper/308:16:283
27904702122,6sleep30-21swapper/307:56:293
27837992122,6sleep30-21swapper/307:46:343
27751262122,6sleep30-21swapper/307:41:243
27605502120,8sleep30-21swapper/307:26:283
27550632122,5sleep30-21swapper/307:20:023
27541252122,6sleep30-21swapper/307:16:363
30716892112,6sleep30-21swapper/312:36:403
30625822112,5sleep30-21swapper/312:30:023
30588962112,5sleep30-21swapper/312:26:333
30505492112,5sleep30-21swapper/312:16:333
30378342112,5sleep30-21swapper/312:05:023
30281282112,5sleep30-21swapper/311:55:023
30276352112,5sleep30-21swapper/311:51:453
30187842112,5sleep30-21swapper/311:46:283
29982492112,5sleep30-21swapper/311:25:013
29739242112,5sleep30-21swapper/311:01:293
29590952112,5sleep30-21swapper/310:46:283
29488682112,5sleep30-21swapper/310:36:253
29237832110,7sleep30-21swapper/310:11:253
29188202112,5sleep30-21swapper/310:06:233
29127732112,5sleep30-21swapper/309:56:403
28993882112,5sleep30-21swapper/309:46:293
28891332112,5sleep30-21swapper/309:36:263
28789692112,5sleep30-21swapper/309:25:023
28594392113,5sleep30-21swapper/309:06:163
28350302112,6sleep30-21swapper/308:41:283
28248142114,5sleep30-21swapper/308:30:443
28199032112,5sleep30-21swapper/308:26:233
28146652112,5sleep30-21swapper/308:20:013
28051062112,6sleep30-21swapper/308:11:263
28036742112,5sleep30-21swapper/308:06:373
27652452112,5sleep30-21swapper/307:31:253
27433512112,5sleep30-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional