You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 09:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sat Jul 04, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23802512720,2sleep30-21swapper/321:25:183
2240871997267,3cyclictest2402534-21kworker/u8:0+events_unbound22:05:271
2240871997263,7cyclictest2402534-21kworker/u8:0+flush-8:021:55:221
2240871996760,5cyclictest2291764-21kworker/u8:1+flush-8:022:35:271
2240871996758,6cyclictest2272966-21kworker/u8:0+events_unbound19:40:201
2240875996658,6cyclictest1704890-21systemd-journal20:00:002
2240871996660,4cyclictest2512162-21kworker/u8:1+events_unbound00:20:281
2240871996658,6cyclictest2402534-21kworker/u8:0+flush-8:023:35:231
2240869996662,3cyclictest2251650-21kworker/u8:4+events_unbound19:30:260
22995412630,6sleep00-21swapper/020:05:180
2240871996154,5cyclictest2251650-21kworker/u8:4+flush-8:019:30:211
2240871996150,8cyclictest2392385-21kworker/u8:3+flush-8:022:30:231
2240871996054,4cyclictest2392385-21kworker/u8:3+events_unbound22:20:091
2240871995954,4cyclictest2512162-21kworker/u8:1+events_unbound00:15:271
2240871995952,5cyclictest2392385-21kworker/u8:3+flush-8:022:00:231
2240871995952,5cyclictest2392385-21kworker/u8:3+flush-8:022:00:231
2240871995952,5cyclictest2240746-21kworker/u8:2+flush-8:020:45:291
2240869995949,7cyclictest2258444-21diskstats19:25:120
2240871995853,3cyclictest2240746-21kworker/u8:2+flush-8:021:15:241
2240871995852,5cyclictest2491118-21kworker/u8:3+flush-8:023:55:241
2240871995851,5cyclictest2240746-21kworker/u8:2+events_unbound20:15:231
2240871995851,5cyclictest2240746-21kworker/u8:2+events_unbound20:15:231
2240871995751,4cyclictest2240746-21kworker/u8:2+flush-8:020:40:281
2240871995652,3cyclictest2392385-21kworker/u8:3+events_unbound22:10:271
2240871995651,4cyclictest2491118-21kworker/u8:3+events_unbound23:40:211
2240871995548,5cyclictest2476612-21kworker/u8:1+flush-8:023:15:101
2240871995548,5cyclictest2251649-21kworker/u8:3+events_unbound19:35:201
2240871995547,6cyclictest2491118-21kworker/u8:3+flush-8:023:50:231
2240871995449,3cyclictest2452392-21kworker/u8:2+events_unbound23:10:231
2240871995448,5cyclictest2251650-21kworker/u8:4+events_unbound20:00:231
2240871995448,4cyclictest2291764-21kworker/u8:1+flush-8:020:30:251
2240871995448,4cyclictest2291764-21kworker/u8:1+events_unbound20:20:241
2240871995448,4cyclictest2291764-21kworker/u8:1+events_unbound20:20:231
2240871995445,7cyclictest2452392-21kworker/u8:2+flush-8:022:55:221
2240869995342,8cyclictest1704890-21systemd-journal20:15:000
2240869995342,8cyclictest1704890-21systemd-journal20:14:590
2240869995337,5cyclictest2209316-21munin-node19:15:010
2240871995246,4cyclictest2251650-21kworker/u8:4+events_unbound19:25:241
2240871995243,7cyclictest1704890-21systemd-journal19:10:181
2240875995144,5cyclictest2377100-21cp21:25:002
2240871995146,4cyclictest2512162-21kworker/u8:1+flush-8:000:10:291
2240871995144,5cyclictest2556589-21kworker/u8:2+events_unbound00:25:221
24429412500,2chrt0-21swapper/222:30:162
2240871995046,3cyclictest2240746-21kworker/u8:2+flush-8:021:40:271
2240871995045,4cyclictest2291764-21kworker/u8:1+flush-8:021:45:241
2240871995043,5cyclictest2240746-21kworker/u8:2+flush-8:021:35:241
2240871995042,7cyclictest2392385-21kworker/u8:3+flush-8:022:45:251
2240871995042,6cyclictest2240746-21kworker/u8:2+flush-8:020:10:201
2240871995042,6cyclictest2240746-21kworker/u8:2+flush-8:020:10:201
2240871994944,3cyclictest2371476-21kworker/u8:3+flush-8:021:25:231
2240871994943,5cyclictest2291764-21kworker/u8:1+flush-8:021:10:301
2240871994842,4cyclictest2392385-21kworker/u8:3+events_unbound22:15:251
2240871994840,5cyclictest2481012-21ntpq23:05:201
2240869994838,8cyclictest1704890-21systemd-journal00:20:000
2240871994741,5cyclictest2531788-21kworker/u8:2+flush-8:000:00:231
2240871994740,5cyclictest2251650-21kworker/u8:4+flush-8:019:55:221
2240871994739,6cyclictest2291764-21kworker/u8:1+flush-8:021:30:251
2240869994739,6cyclictest2441058-21ntpq22:25:210
2240871994640,5cyclictest2392385-21kworker/u8:3+events_unbound22:25:121
2240871994640,4cyclictest2491118-21kworker/u8:3+events_unbound23:50:011
2240871994540,3cyclictest2291764-21kworker/u8:1+events_unbound20:05:221
2240871994538,5cyclictest2512162-21kworker/u8:1+flush-8:000:05:231
2240871994538,5cyclictest2512162-21kworker/u8:1+flush-8:000:05:221
2240869994538,4cyclictest1-21systemd21:59:590
2240871994436,6cyclictest2240746-21kworker/u8:2+flush-8:020:25:211
2240871994433,9cyclictest2476013-21ntpq23:00:191
2240869994437,5cyclictest2277954-21apt-get19:45:090
2240869994433,9cyclictest2541074-21ntpq00:05:220
2240869994433,9cyclictest2541074-21ntpq00:05:210
2240869994432,9cyclictest2377743-21diskstats21:25:130
2240875994337,4cyclictest2251650-21kworker/u8:4+flush-8:019:45:132
2240875994331,9cyclictest2347362-21cat20:55:012
2240871994338,4cyclictest2491118-21kworker/u8:3+flush-8:023:30:261
2240871994336,5cyclictest2556589-21kworker/u8:2+flush-8:000:35:201
2240869994333,7cyclictest2462308-21/usr/sbin/munin22:50:190
2240871994236,5cyclictest2291764-21kworker/u8:1+flush-8:021:00:141
2240871994236,4cyclictest2311532-21kworker/u8:3+flush-8:020:35:181
2240871994236,4cyclictest2251650-21kworker/u8:4+events_unbound19:15:261
2240875994132,7cyclictest1704890-21systemd-journal20:30:002
2240871994137,3cyclictest2491118-21kworker/u8:3+flush-8:023:20:191
2240871994135,5cyclictest2291764-21kworker/u8:1+flush-8:020:50:221
2240871994135,4cyclictest2452392-21kworker/u8:2+events_unbound22:40:151
2240871994134,5cyclictest2251649-21kworker/u8:3+flush-8:019:20:111
2240869994132,7cyclictest2346088-21ntpq20:50:230
2240869994127,11cyclictest1704890-21systemd-journal20:25:000
2240869994127,11cyclictest1704890-21systemd-journal20:25:000
224063224127,9sleep10-21swapper/119:09:001
2240871993934,3cyclictest2240746-21kworker/u8:2+flush-8:019:50:261
2240871993933,5cyclictest2452392-21kworker/u8:2+flush-8:023:25:231
2240871993933,4cyclictest2291764-21kworker/u8:1+events_unbound20:55:191
2240875993731,4cyclictest2387252-21cp21:35:002
224086999378,12cyclictest589-21runrttasks21:45:100
2240869993726,9cyclictest2422665-21diskstats22:10:120
2240871993624,8cyclictest1704890-21systemd-journal00:30:381
2240869993629,5cyclictest2502021-21latency_hist23:30:020
2240869993626,7cyclictest2342063-21latency_hist20:50:010
2240875993529,4cyclictest2402534-21kworker/u8:0+events_unbound22:20:012
2240871993529,5cyclictest2402534-21kworker/u8:0+flush-8:021:50:111
2240869993528,4cyclictest2506958-21latency_hist23:35:010
2240869993526,7cyclictest2307620-21/usr/sbin/munin20:15:210
2240869993526,7cyclictest2307620-21/usr/sbin/munin20:15:200
2240869993526,6cyclictest2481954-21apt-get23:10:010
224086999352,18cyclictest0-21swapper/021:20:090
2240875993428,4cyclictest2402534-21kworker/u8:0+flush-8:022:50:112
224086999344,17cyclictest2391680-21/usr/sbin/munin21:35:260
2240869993430,2cyclictest2525749-21needreboot23:50:200
2240869993419,5cyclictest2492063-21latency_hist23:20:010
2240875993326,5cyclictest2561886-21idleruntime-cro00:30:012
224086999339,13cyclictest589-21runrttasks20:30:080
224086999338,14cyclictest2496395-21python323:20:290
224086999338,14cyclictest2406435-21ntpq21:50:230
224086999338,13cyclictest2336246-21ntpq20:40:230
224086999336,13cyclictest2361339-21sendmail_mailst21:05:260
224086999334,18cyclictest2472542-21/usr/sbin/munin23:00:110
2240869993325,6cyclictest2382701-21diskstats21:30:120
2240875993227,4cyclictest2291764-21kworker/u8:1+events_unbound20:55:292
224086999329,11cyclictest2461219-21ntpq22:45:260
224086999328,6cyclictest2557164-21/usr/sbin/munin00:25:070
224086999327,14cyclictest2356252-21ntpq21:00:230
224086999327,13cyclictest2516039-21ntpq23:40:250
224086999327,12cyclictest2526971-21/usr/sbin/munin23:55:080
224086999326,15cyclictest2351234-21ntpq20:55:230
224086999326,14cyclictest2268438-21hddtemp_smartct19:35:120
224086999326,12cyclictest25426741if_enp4s000:10:150
224086999326,11cyclictest2563476-21kthreadcore00:30:170
2240869993222,7cyclictest2536775-21apt-get00:05:000
224086999321,22cyclictest2432728-21if_enp3s022:20:110
2240871993127,3cyclictest2357159-21kworker/u8:0+flush-8:021:05:301
224086999317,14cyclictest2362860-21/usr/sbin/munin21:10:210
224086999317,10cyclictest2417618-21/usr/sbin/munin22:05:090
224086999316,15cyclictest2467695-21hddtemp_smartct22:55:120
224086999316,14cyclictest2456292-21ntpq22:40:240
224086999316,13cyclictest2327473-21apt-get20:35:100
224086999315,15cyclictest2396448-21ntpq21:40:250
224086999315,15cyclictest2296580-21ntpq20:00:210
224086999315,11cyclictest2251757-21ntpq19:15:220
224086999315,10cyclictest2554189-21kthreadcore00:20:180
224086999313,15cyclictest2447209-21apt-get22:35:070
2240869993110,11cyclictest2517334-21/usr/sbin/munin23:45:110
22406332314,9sleep20-21swapper/219:09:022
2240875993025,4cyclictest2452392-21kworker/u8:2+flush-8:023:10:272
2240875993025,4cyclictest2291764-21kworker/u8:1+flush-8:021:05:242
2240871993026,3cyclictest2251650-21kworker/u8:4+flush-8:019:45:291
224086999308,12cyclictest2428237-21/usr/sbin/munin22:15:160
224086999307,11cyclictest2570861-21ntpq00:35:200
224086999307,11cyclictest2446281-21python322:30:270
224086999305,15cyclictest2371145-21ntp_states21:15:260
224086999305,14cyclictest2415239-21kthreadcore22:00:200
224086999305,14cyclictest2415239-21kthreadcore22:00:200
224086999304,13cyclictest2257336-21vmstat19:20:280
224086999304,12cyclictest2478148-21kthreadcore23:05:150
224086999304,12cyclictest2277279-21unixbench_singl19:40:280
23927902290,7sleep22392795-21cpuspeed21:40:102
2240875992925,2cyclictest2263421-21fschecks_time19:30:132
2240875992924,4cyclictest2556589-21kworker/u8:2+flush-8:000:30:242
2240875992921,6cyclictest1704890-21systemd-journal22:00:012
2240871992917,8cyclictest1704890-21systemd-journal21:21:171
224086999296,13cyclictest2511069-21ntpq23:35:240
224086999295,14cyclictest2288110-21df_inode19:55:110
224086999293,16cyclictest2321288-21uname20:25:190
2240869992910,8cyclictest2283631-21ps19:50:150
2240875992824,2cyclictest2417749-21fschecks_count22:05:122
2240875992824,2cyclictest2362372-21aten2_r5power_c21:10:112
2240875992819,7cyclictest1704890-21systemd-journal00:00:012
2240875992722,3cyclictest2241103-21aten2_r5power_e19:10:102
2240875992721,4cyclictest2536796-21gpgv00:05:012
2240875992719,6cyclictest1704890-21systemd-journal21:48:182
2240871992722,3cyclictest2463054-21irqrtprio22:50:151
2240875992621,4cyclictest2392385-21kworker/u8:3+flush-8:021:50:382
22407082262,16sleep02240759-21munin-run19:10:000
2240875992521,3cyclictest2512162-21kworker/u8:1+flush-8:000:10:272
2240875992521,3cyclictest2452392-21kworker/u8:2+events_unbound23:30:232
2240875992521,2cyclictest2512455-21fschecks_count23:40:122
2240875992521,2cyclictest2507271-21aten2_r5power_e23:35:092
2240875992521,2cyclictest2487410-21aten2_r5power_c23:15:102
2240875992521,2cyclictest2272336-21grep19:35:282
2240875992520,3cyclictest2451464-21unixbench_multi22:35:282
2240875992519,4cyclictest413-21systemd-logind20:05:002
2240875992417,5cyclictest2571815-21latency_hist00:40:012
2240875992414,7cyclictest1704890-21systemd-journal23:10:012
2240875992319,2cyclictest2547316-21aten2_r5power_p00:15:102
2240875992319,2cyclictest2387879-21tr21:35:112
2240875992319,2cyclictest2287597-21cron19:55:002
2240875992318,4cyclictest2452392-21kworker/u8:2+flush-8:022:40:112
2240875992318,3cyclictest2467545-21tr22:55:112
2240875992218,3cyclictest2520944-21kworker/u8:0+events_unbound23:45:242
2240875992218,2cyclictest2556528-21grep00:20:282
2240875992218,2cyclictest2522292-21tr23:50:112
2240875992217,4cyclictest2392385-21kworker/u8:3+flush-8:022:20:212
2240875992217,3cyclictest2257879-21base6419:25:012
224087599219,4cyclictest0-21swapper/220:10:152
224087599219,4cyclictest0-21swapper/220:10:142
2240875992117,2cyclictest2382188-21cron21:29:592
2240875992017,2cyclictest2316937-21unixbench_singl20:20:282
2240875992017,2cyclictest2316937-21unixbench_singl20:20:282
2240875992015,3cyclictest0-21swapper/220:50:012
2240875992012,6cyclictest1704890-21systemd-journal22:30:002
2240875992010,7cyclictest1704890-21systemd-journal19:20:002
2240875991915,2cyclictest2497404-21aten2_r5power_p23:25:102
2240875991915,2cyclictest0-21swapper/220:10:002
2240875991910,7cyclictest2356616-21grep21:00:292
22358052192,8sleep30-21swapper/319:05:023
2240875991814,2cyclictest2422730-21fschecks_time22:10:112
2240875991814,2cyclictest0-21swapper/220:40:162
2240875991814,2cyclictest0-21swapper/219:40:152
2240875991814,2cyclictest0-21swapper/200:05:252
2240875991814,2cyclictest0-21swapper/200:05:252
2240875991813,3cyclictest0-21swapper/219:25:172
23390022170,2sleep3481ktimers/320:45:183
224087599179,6cyclictest2413672-21tr22:00:182
224087599179,6cyclictest2413672-21tr22:00:172
2240875991711,4cyclictest0-21swapper/221:15:132
2240875991711,4cyclictest0-21swapper/220:30:162
22767412167,6sleep30-21swapper/319:40:223
224087599166,8cyclictest2492972-21sed23:20:162
2240875991613,2cyclictest2476587-21grep23:00:262
2240875991612,2cyclictest0-21swapper/222:45:112
2240875991612,2cyclictest0-21swapper/220:35:222
2240875991612,2cyclictest0-21swapper/220:15:182
2240875991612,2cyclictest0-21swapper/220:15:182
23610992150,2sleep3471rcuc/321:05:213
22584342157,5sleep30-21swapper/319:25:133
23147292143,7sleep30-21swapper/320:20:193
23147292143,7sleep30-21swapper/320:20:183
25711362132,7sleep30-21swapper/300:35:273
25260642132,7sleep30-21swapper/323:50:283
25073112132,7sleep30-21swapper/323:35:103
25060232132,7sleep30-21swapper/323:30:223
25013522133,7sleep30-21swapper/323:25:283
24960412132,7sleep30-21swapper/323:20:213
24843062132,7sleep30-21swapper/323:10:193
24229102132,7sleep30-21swapper/322:10:143
24113122132,7sleep30-21swapper/321:55:223
23184652132,7sleep30-21swapper/320:25:173
22576912132,7sleep30-21swapper/319:25:013
25626602122,6sleep30-21swapper/300:30:173
25519702122,6sleep30-21swapper/300:20:013
25356252123,5sleep30-21swapper/300:00:193
25267432122,5sleep30-21swapper/323:55:013
25169502122,6sleep30-21swapper/323:45:023
24670752122,6sleep30-21swapper/322:55:023
24372862122,6sleep30-21swapper/322:25:083
24359822122,6sleep30-21swapper/322:20:183
24163942122,6sleep30-21swapper/322:00:283
24163942122,6sleep30-21swapper/322:00:273
23994342122,6sleep30-21swapper/321:45:183
23738262122,6sleep30-21swapper/321:20:193
23719312122,6sleep30-21swapper/321:19:183
23511422122,6sleep30-21swapper/320:55:213
23262532122,7sleep30-21swapper/320:30:223
23117502122,6sleep30-21swapper/320:15:293
23117502122,6sleep30-21swapper/320:15:283
23031922122,6sleep30-21swapper/320:10:163
23031922122,6sleep30-21swapper/320:10:163
22976882122,6sleep30-21swapper/320:05:013
22878322122,6sleep30-21swapper/319:55:033
22831512122,6sleep30-21swapper/319:50:133
22826382120,7sleep30-21swapper/319:50:003
22674372122,6sleep30-21swapper/319:30:303
22450672122,6sleep30-21swapper/319:10:273
25571622112,5sleep30-21swapper/300:25:073
25516292112,6sleep30-21swapper/300:16:013
25426182112,6sleep30-21swapper/300:10:153
25419382112,6sleep30-21swapper/300:09:303
25419382112,6sleep30-21swapper/300:09:293
25121732112,5sleep30-21swapper/323:40:083
24879512112,6sleep30-21swapper/323:15:173
24772332112,5sleep30-21swapper/323:05:083
24766532112,6sleep30-21swapper/323:01:043
24623092112,5sleep30-21swapper/322:50:073
24572422112,5sleep30-21swapper/322:45:023
24566172112,5sleep30-21swapper/322:40:293
24471812112,6sleep30-21swapper/322:35:073
24461112112,5sleep30-21swapper/322:30:253
24273252112,6sleep30-21swapper/322:15:093
24172262112,5sleep30-21swapper/322:05:013
24069412112,5sleep30-21swapper/321:51:013
23923802112,5sleep30-21swapper/321:40:013
23876192112,5sleep30-21swapper/321:35:083
23824542112,5sleep30-21swapper/321:30:093
23662112112,5sleep30-21swapper/321:10:303
23528942112,5sleep30-21swapper/321:00:163
23422882111,6sleep30-21swapper/320:50:093
23361622112,5sleep30-21swapper/320:40:213
23279832112,5sleep30-21swapper/320:35:153
22927662112,6sleep30-21swapper/320:00:033
22680382112,6sleep30-21swapper/319:35:083
22477172112,5sleep30-21swapper/319:15:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional