You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 05:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Thu Jul 02, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
298261624390,3sleep375-21kworker/3:1-events00:10:253
27811802800,2chrt2781178-21sshd20:50:080
2679543996442,19cyclictest1704890-21systemd-journal23:20:000
2679545996257,3cyclictest2686989-21kworker/u8:1+events_unbound20:10:291
2679545996054,4cyclictest2966468-21kworker/u8:2+events_unbound00:15:301
2679545995850,6cyclictest1305440-21kworker/u8:2+flush-8:019:50:271
2679543995638,6cyclictest2780950-21latency_hist20:50:010
2679545995549,4cyclictest2765837-21kworker/u8:0+events_unbound21:20:301
2679543995545,6cyclictest1704890-21systemd-journal00:30:180
2679545995451,2cyclictest1305440-21kworker/u8:2+events_unbound20:50:301
2679545995348,4cyclictest1305440-21kworker/u8:2+events_unbound19:20:271
2679543995342,8cyclictest2720181-21ntpq19:45:230
2679545995145,4cyclictest2868278-21kworker/u8:1+events_unbound23:40:221
2679545995144,5cyclictest2910162-21kworker/u8:3+flush-8:023:35:251
2679545995144,5cyclictest2868278-21kworker/u8:1+flush-8:023:05:321
2679545995044,5cyclictest1305440-21kworker/u8:2+events_unbound20:30:131
2679545995043,5cyclictest1305440-21kworker/u8:2+events_unbound21:00:231
2679545995042,5cyclictest2910162-21kworker/u8:3+flush-8:023:25:251
2679545994944,3cyclictest1305440-21kworker/u8:2+flush-8:019:15:291
2679545994843,4cyclictest2836382-21kworker/u8:0+flush-8:022:55:261
2679545994843,3cyclictest2815709-21kworker/u8:3+flush-8:022:15:261
2679545994841,5cyclictest2765837-21kworker/u8:0+events_unbound21:15:261
2679543994839,7cyclictest2706497-21/usr/sbin/munin19:35:270
2679545994742,3cyclictest2765837-21kworker/u8:0+events_unbound21:40:001
2679545994741,4cyclictest2686989-21kworker/u8:1+events_unbound19:25:251
2679545994739,5cyclictest2815709-21kworker/u8:3+flush-8:022:20:251
2679545994738,7cyclictest2859816-21ntpq22:05:211
2679543994741,4cyclictest2836382-21kworker/u8:0+events_unbound22:25:300
2679543994736,8cyclictest1704890-21systemd-journal20:10:000
2679545994641,3cyclictest2910162-21kworker/u8:3+events_unbound23:45:181
2679545994640,4cyclictest2686989-21kworker/u8:1+flush-8:020:55:291
2679545994639,5cyclictest2686989-21kworker/u8:1+flush-8:019:30:261
2679545994539,4cyclictest2990615-21kworker/u8:3+events_unbound00:30:241
2679545994539,4cyclictest2868278-21kworker/u8:1+events_unbound23:00:251
2679545994539,4cyclictest2868278-21kworker/u8:1+events_unbound23:00:241
2679545994539,4cyclictest2836382-21kworker/u8:0+flush-8:022:10:251
2679545994539,4cyclictest2815709-21kworker/u8:3+flush-8:021:25:231
2679545994534,8cyclictest2889829-21ntpq22:35:241
2679545994532,4cyclictest2836382-21kworker/u8:0+flush-8:022:30:261
2679543994533,10cyclictest1704890-21systemd-journal21:30:010
2679545994439,3cyclictest1305440-21kworker/u8:2+events_unbound20:15:121
2679545994438,4cyclictest2910162-21kworker/u8:3+flush-8:023:30:141
2679545994438,4cyclictest1305440-21kworker/u8:2+events_unbound20:40:261
2679545994339,3cyclictest2974618-21kworker/u8:4+flush-8:000:00:311
2679545994338,4cyclictest2686989-21kworker/u8:1+events_unbound19:35:251
2679545994337,4cyclictest2686989-21kworker/u8:1+flush-8:020:20:241
2679545994337,4cyclictest2686989-21kworker/u8:1+flush-8:020:20:241
2679545994238,3cyclictest2868278-21kworker/u8:1+flush-8:022:40:191
2679545994236,4cyclictest2765837-21kworker/u8:0+flush-8:021:05:241
2679545994236,4cyclictest2765837-21kworker/u8:0+events_unbound20:35:231
2679545994235,5cyclictest2686989-21kworker/u8:1+flush-8:020:00:241
2679545994234,6cyclictest2920928-21kworker/u8:0+flush-8:023:55:211
2679543994232,8cyclictest2924729-21ntpq23:10:230
2679549994133,5cyclictest1305440-21kworker/u8:2+flush-8:019:40:312
2679545994134,5cyclictest2930639-21apt-get23:20:031
2679545994134,5cyclictest2930639-21apt-get23:20:021
2679545994134,5cyclictest2686989-21kworker/u8:1+events_unbound19:55:241
2679545994134,5cyclictest1305440-21kworker/u8:2+events_unbound19:45:281
2679543994130,8cyclictest2829769-21ntpq21:35:230
2679545994036,3cyclictest2885369-21kworker/u8:2+events_unbound22:45:181
2679543994032,6cyclictest2756372-21/usr/sbin/munin20:25:200
2679543994031,7cyclictest2939855-21ntpq23:25:250
26833692390,1sleep12683372-21missed_timers19:10:201
2679549993929,8cyclictest1704890-21systemd-journal22:20:002
2679545993930,6cyclictest2854867-21ntpq22:00:241
2679543993924,11cyclictest1704890-21systemd-journal00:25:010
2679543993833,4cyclictest2986101-21sshd00:15:180
267511323823,9sleep20-21swapper/219:05:162
2679545993733,3cyclictest2868278-21kworker/u8:1+events_unbound22:50:141
2679545993733,3cyclictest2830945-21kworker/u8:1+flush-8:021:50:251
2679545993732,4cyclictest1305440-21kworker/u8:2+flush-8:020:45:251
2679545993731,4cyclictest2990615-21kworker/u8:3+flush-8:000:25:211
267954399372,28cyclictest2940788-21kworker/0:0+events23:40:130
2679545993632,3cyclictest2836382-21kworker/u8:0+events_unbound22:25:191
2679545993630,4cyclictest2830945-21kworker/u8:1+flush-8:021:40:261
2679545993627,6cyclictest2964586-21ntpq23:50:231
2679543993629,5cyclictest2980910-21tr00:10:180
2679543993629,5cyclictest2855332-21grep22:00:310
2679543993628,6cyclictest2995547-21/usr/sbin/munin00:25:220
267954999353,24cyclictest2691602-21apt-get19:20:122
2679549993529,4cyclictest2970458-40logrotate00:00:012
267954999351,31cyclictest2725899-21unixbench_multi19:50:312
2679545993530,4cyclictest2984474-21kworker/u8:1+events_unbound00:35:131
2679543993511,18cyclictest2679397-21kworker/0:2+events19:15:030
2679549993421,11cyclictest2995631-21aten2_r5power_e00:25:132
2679545993429,4cyclictest2815709-21kworker/u8:3+flush-8:021:55:161
2679545993429,3cyclictest2686989-21kworker/u8:1+events_unbound20:05:281
2679545993425,6cyclictest1704890-21systemd-journal23:14:591
2679545993425,3cyclictest1305440-21kworker/u8:2+flush-8:020:25:101
267954399341,31cyclictest2940788-21kworker/0:0+events23:45:110
267954999333,23cyclictest2931154-21apt-get23:20:102
267954999333,23cyclictest2931154-21apt-get23:20:102
2679549993328,3cyclictest2706252-21cron19:34:592
2679549993327,4cyclictest2868278-21kworker/u8:1+flush-8:022:45:132
2679549993320,12cyclictest2775535-21unixbench_multi20:40:312
2679549993320,11cyclictest1-21systemd22:30:002
2679545993329,3cyclictest2966468-21kworker/u8:2+flush-8:000:20:311
2679545993329,3cyclictest2830945-21kworker/u8:1+flush-8:021:45:161
2679543993328,3cyclictest2751123-21cron20:19:590
2679543993310,21cyclictest2815818-21kworker/0:0+events21:50:260
267954999323,22cyclictest2716482-21apt-get19:45:112
2679545993227,4cyclictest1305440-21kworker/u8:2+events_unbound19:45:001
2679545993226,5cyclictest2920928-21kworker/u8:0+events_unbound00:05:131
267954399329,22cyclictest2870714-21kworker/0:2+events22:40:260
267954399328,22cyclictest2679397-21kworker/0:2+events19:25:220
2679543993227,3cyclictest2727311-21irqrtprio19:55:170
267954399322,29cyclictest2995351-21kworker/0:2+events00:35:110
2679543993222,8cyclictest2765163-21ntpq20:30:240
267954999313,22cyclictest2954763-21ntpq23:40:252
2679549993117,12cyclictest2941378-21fschecks_time23:30:152
2679549993117,12cyclictest2736764-21cpuspeed_turbos20:05:132
2679543993127,2cyclictest2861232-21cpuspeed_turbos22:10:130
2679543993123,5cyclictest2679347-21latency_hist19:10:000
267954399312,27cyclictest2940788-21kworker/0:0+events23:55:100
2679543993122,7cyclictest2701514-21/usr/sbin/munin19:30:290
267954399312,27cyclictest2679397-21kworker/0:2+events20:35:250
2679543993122,6cyclictest1704890-21systemd-journal22:45:530
2679549993021,7cyclictest1704890-21systemd-journal22:55:002
267954999302,10cyclictest2829870-21ntpq21:35:262
267954399302,27cyclictest2785881-21kworker/0:1+events20:55:250
267954399302,27cyclictest2679397-21kworker/0:2+events20:10:180
2679543993022,6cyclictest2934913-21ntpq23:20:220
2679543993022,6cyclictest2934913-21ntpq23:20:210
267954399302,26cyclictest2815818-21kworker/0:0+events22:15:190
2679543993022,6cyclictest2695888-21grep19:20:300
2679543993021,6cyclictest2799745-21ntpq21:05:230
2679543993010,17cyclictest2679397-21kworker/0:2+events20:25:010
267954999298,6cyclictest2804285-21tr21:10:202
267954999291,5cyclictest2860997-21apt-get22:10:102
2679549992914,13cyclictest0-21swapper/220:20:172
2679549992914,13cyclictest0-21swapper/220:20:162
2679545992923,4cyclictest2686989-21kworker/u8:1+events_unbound21:10:221
267954399296,19cyclictest2870714-21kworker/0:2+events22:55:010
267954399296,17cyclictest2870714-21kworker/0:2+events23:00:000
2679543992924,3cyclictest2836680-21irqrtprio21:45:170
267954399292,26cyclictest2679397-21kworker/0:2+events21:15:140
2679543992919,8cyclictest2871187-21diskstats22:20:120
267954399291,27cyclictest2679397-21kworker/0:2+events19:50:210
267954399291,26cyclictest2870714-21kworker/0:2+events23:05:260
267954399291,26cyclictest2815818-21kworker/0:0+events21:40:210
267954399291,26cyclictest2679397-21kworker/0:2+events21:15:010
267954999282,7cyclictest2809837-21ntpq21:15:222
267954999282,20cyclictest2683483-21ntpq19:10:212
2679549992820,6cyclictest2915926-21cp23:05:012
267954999281,8cyclictest2799746-21awk21:05:222
2679549992811,15cyclictest2990962-21fschecks_time00:20:142
267954999280,9cyclictest2859799-21awk22:05:212
267954999280,9cyclictest2710293-21ntpq19:35:222
267954399289,17cyclictest2940788-21kworker/0:0+events23:50:560
267954399288,19cyclictest2870714-21kworker/0:2+events22:31:190
267954399287,17cyclictest2815818-21kworker/0:0+events22:05:080
267954399282,25cyclictest2679397-21kworker/0:2+events19:11:250
2679543992821,5cyclictest2850568-21apt-get22:00:000
2679543992815,3cyclictest0-21swapper/000:00:110
267954399281,26cyclictest2815818-21kworker/0:0+events21:30:220
267954399281,25cyclictest2679397-21kworker/0:2+events20:20:210
2679543992811,15cyclictest2679397-21kworker/0:2+events21:20:130
2679543992810,16cyclictest2870714-21kworker/0:2+events23:00:190
2679543992810,16cyclictest2870714-21kworker/0:2+events23:00:190
26756282282,8sleep12676441-21kthreadcore19:05:191
267954999279,15cyclictest0-21swapper/223:15:012
2679549992723,2cyclictest2701271-21cron19:30:002
267954999271,9cyclictest2761843-21/usr/sbin/munin20:30:152
267954999271,8cyclictest2949775-21ntpq23:35:242
2679549992712,13cyclictest0-21swapper/220:00:282
2679549992712,12cyclictest0-21swapper/222:00:182
2679545992722,4cyclictest2984474-21kworker/u8:1+flush-8:000:10:351
2679545992722,4cyclictest2868278-21kworker/u8:1+events_unbound23:15:271
267954399279,16cyclictest2679397-21kworker/0:2+events20:00:150
267954399278,18cyclictest2940788-21kworker/0:0+events23:40:000
267954999269,14cyclictest0-21swapper/223:10:172
267954999268,7cyclictest2886679-21sed22:35:172
267954999267,2cyclictest0-21swapper/222:44:042
267954999266,7cyclictest2793074-21tr21:00:202
267954999266,4cyclictest2822411-21cut21:30:192
267954999261,9cyclictest2727113-21ps19:55:182
267954999261,8cyclictest2990340-21munin-run00:20:002
267954999261,8cyclictest2770148-21ntpq20:35:242
267954999261,8cyclictest2750735-21grep20:15:302
267954999261,5cyclictest2970732-21apt-get00:00:112
2679549992613,11cyclictest0-21swapper/223:30:002
2679549992612,12cyclictest0-21swapper/221:55:242
267954999260,8cyclictest3004732-21grep00:30:302
267954999260,8cyclictest2910539-21idleruntime-cro23:00:002
267954999260,8cyclictest2839769-21awk21:45:242
267954399269,15cyclictest2940788-21kworker/0:0+events00:05:110
267954999257,16cyclictest2910852-21cut23:00:102
267954999256,3cyclictest0-21swapper/221:55:012
267954999255,5cyclictest3008272-21cut00:35:192
2679549992521,2cyclictest2741531-21aten2_r5power_c20:10:112
267954999251,8cyclictest2884863-21ntpq22:30:252
267954999251,8cyclictest2834772-21ntpq21:40:252
267954999251,7cyclictest2684908-21munin-node19:15:032
2679549992510,13cyclictest0-21swapper/223:05:232
267954999250,9cyclictest2871661-21ps22:20:172
267954999250,9cyclictest2815389-21vmstat21:20:322
267954999250,8cyclictest2784959-21sed20:50:242
267954999250,7cyclictest2760264-21grep20:25:242
2679543992520,3cyclictest2712096-21sort19:40:170
267954399250,23cyclictest2775314-21sshd20:40:280
267954999242,8cyclictest2976242-21ps00:05:172
2679549992421,2cyclictest2960865-21aten2_r5power_c23:50:122
267954999241,9cyclictest2819966-21ntpq21:25:252
267954999240,9cyclictest2960644-21grep23:50:002
267954999240,8cyclictest2985228-21chrt00:13:372
267954999240,8cyclictest2776469-21sort20:45:142
267954999240,7cyclictest2786576-21/usr/sbin/munin20:55:162
267954399246,17cyclictest2679397-21kworker/0:2+events21:00:300
2679543992416,5cyclictest2944772-21ntpq23:30:230
267954399237,15cyclictest2870714-21kworker/0:2+events22:35:110
2679545992218,3cyclictest2815709-21kworker/u8:3+events_unbound21:30:131
27666502180,3chrt75-21kworker/3:1-events20:35:133
26790722183,9sleep30-21swapper/319:06:343
29780042170,2sleep3471rcuc/300:05:193
28698942156,5sleep30-21swapper/322:15:273
30054072132,6sleep30-21swapper/300:35:013
29556552132,6sleep30-21swapper/323:45:013
28207872132,5sleep30-21swapper/321:30:003
27997842132,7sleep30-21swapper/321:05:243
27230082132,7sleep30-21swapper/319:50:203
27013332132,6sleep30-21swapper/319:30:013
29951622123,6sleep30-21swapper/300:23:173
29894692122,6sleep30-21swapper/300:15:283
29713632122,6sleep30-21swapper/300:00:183
29645792122,6sleep30-21swapper/323:50:243
29553752122,6sleep30-21swapper/323:41:453
29458202122,6sleep30-21swapper/323:35:033
29366422122,6sleep30-21swapper/323:25:183
29353512122,6sleep30-21swapper/323:20:303
29353512122,6sleep30-21swapper/323:20:303
29263262122,6sleep30-21swapper/323:15:153
29214482120,8sleep30-21swapper/323:10:183
29111632122,6sleep30-21swapper/323:00:153
29111632122,6sleep30-21swapper/323:00:143
29045712122,6sleep30-21swapper/322:50:233
28999752122,6sleep30-21swapper/322:45:293
28801782122,6sleep30-21swapper/322:25:323
28707582122,5sleep30-21swapper/322:20:023
28513332122,6sleep30-21swapper/322:00:143
28456852122,5sleep30-21swapper/321:55:013
28413122122,6sleep30-21swapper/321:50:163
28359332122,6sleep30-21swapper/321:45:103
28356402122,6sleep30-21swapper/321:44:253
28108512122,6sleep30-21swapper/321:20:023
28060872122,6sleep30-21swapper/321:15:093
28012942122,6sleep30-21swapper/321:10:143
27919612122,6sleep30-21swapper/321:00:203
27858782122,6sleep30-21swapper/320:55:023
27815192122,6sleep30-21swapper/320:50:153
27761682122,6sleep30-21swapper/320:45:033
27612832123,5sleep30-21swapper/320:30:023
27518432122,6sleep30-21swapper/320:20:183
27518432122,6sleep30-21swapper/320:20:183
27464632122,6sleep30-21swapper/320:15:133
27460382122,6sleep30-21swapper/320:13:083
27363062122,5sleep30-21swapper/320:05:003
27330952123,6sleep30-21swapper/320:00:203
27265582122,6sleep30-21swapper/319:55:103
27112052122,5sleep30-21swapper/319:40:013
27086052122,6sleep30-21swapper/319:35:203
26917642122,6sleep30-21swapper/319:20:153
30009252112,6sleep30-21swapper/300:30:173
29961792112,5sleep30-21swapper/300:25:193
29657382112,5sleep30-21swapper/323:55:093
29449962112,5sleep30-21swapper/323:30:283
29159592112,6sleep30-21swapper/323:05:133
29057832112,5sleep30-21swapper/322:55:093
28910072112,5sleep30-21swapper/322:40:133
28859652112,5sleep30-21swapper/322:35:033
28810702112,5sleep30-21swapper/322:30:123
28609602112,5sleep30-21swapper/322:10:033
28262432112,6sleep30-21swapper/321:35:153
28210682112,5sleep30-21swapper/321:30:143
27713562112,5sleep30-21swapper/320:40:133
27589362112,6sleep30-21swapper/320:25:203
27167252112,5sleep30-21swapper/319:45:153
26966142113,5sleep30-21swapper/319:25:123
26892742112,5sleep30-21swapper/319:15:203
26796832112,5sleep30-21swapper/319:10:113
28567512102,5sleep30-21swapper/322:05:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional