You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 03:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Mon May 04, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23807802770,4sleep0201rcuc/021:00:010
24463072750,4sleep0201rcuc/022:06:060
2269486996251,8cyclictest2464800-21apt-get22:25:001
24583592560,3sleep375-21kworker/3:1+events22:16:093
2269489995446,5cyclictest280-21systemd-journal20:50:012
2269486995443,8cyclictest2434124-21ntpq21:51:111
2269486995346,4cyclictest2479888-21latency_hist22:40:011
2269482995246,4cyclictest2425145-21latency_hist21:45:010
2269489995145,4cyclictest860673-21kworker/u8:1+flush-8:020:01:162
2269486995139,9cyclictest2419307-21ntpq21:36:121
2269486995040,8cyclictest2354571-21ntpq20:31:111
2269486995039,9cyclictest280-21systemd-journal20:30:001
2269486995039,9cyclictest280-21systemd-journal20:30:001
2269486994739,6cyclictest2548517-21ntpq23:46:121
2269482994638,6cyclictest2310993-21latency_hist19:50:010
2269489994540,3cyclictest860673-21kworker/u8:1+flush-8:019:11:172
2269486994534,8cyclictest2305208-21ntpq19:41:131
2269486994437,5cyclictest2320020-21ntpq19:56:111
2269486994437,5cyclictest2286567-21/usr/sbin/munin19:26:101
25826182430,6sleep30-21swapper/300:21:093
2269486994333,8cyclictest2469103-21ntpq22:26:101
2269486994136,3cyclictest2325818-21basename20:05:001
2269486994132,6cyclictest280-21systemd-journal21:30:001
2269482994132,6cyclictest2385567-21latency_hist21:05:010
2269486994033,5cyclictest2529923-21apt-get23:30:581
2269486994033,5cyclictest2524986-21apt-get23:25:581
2269486994029,8cyclictest2593084-21ntpq00:31:101
2269486993929,8cyclictest2379753-21ntpq20:56:111
2269486993928,8cyclictest2458909-21ntpq22:16:111
2269482993931,6cyclictest2276686-21apt-get19:15:570
2269482993929,7cyclictest2315948-21latency_hist19:55:010
226903623925,8sleep30-21swapper/319:06:253
2269486993832,5cyclictest860673-21kworker/u8:1+flush-8:020:16:181
2269486993829,7cyclictest2564533-21kworker/u8:2+flush-8:000:21:171
2269486993731,4cyclictest2559634-21apt-get00:00:591
2269486993726,9cyclictest2359560-21ntpq20:36:131
2269486993726,8cyclictest280-21systemd-journal19:50:571
2269482993731,4cyclictest2350515-21apt20:30:000
2269482993731,4cyclictest2350515-21apt20:29:590
2269486993632,2cyclictest2277037-21fschecks_time19:16:021
2269482993631,4cyclictest2459838-21kworker/u8:3+flush-8:022:26:170
22687082363,13sleep0161rcu_preempt19:06:170
2269486993530,3cyclictest2492291-21kthreadcore22:51:081
2269486993529,5cyclictest2222757-21kworker/u8:2+flush-8:019:46:041
2269486993528,5cyclictest2498858-21ntpq22:56:131
2269486993523,9cyclictest280-21systemd-journal00:10:001
2269482993529,4cyclictest2409880-21unixbench_multi21:26:170
2269482993528,5cyclictest280-21systemd-journal00:00:000
2269489993428,4cyclictest2222757-21kworker/u8:2+flush-8:022:21:172
2269486993429,3cyclictest2410939-21fschecks_time21:31:021
2269486993428,4cyclictest2584398-21apt-get00:25:591
2269486993426,6cyclictest2573308-21ntpq00:11:131
2269486993425,6cyclictest280-21systemd-journal22:34:571
2269489993328,4cyclictest2368359-21kworker/u8:0+flush-8:021:21:182
2269489993328,3cyclictest2222757-21kworker/u8:2+flush-8:022:11:172
2269486993328,4cyclictest2222757-21kworker/u8:2+flush-8:020:11:131
2269486993327,4cyclictest2385883-21apt-get21:05:571
2269486993325,6cyclictest2574482-21apt-get00:16:001
2269486993325,6cyclictest2291507-21apt-get19:30:591
2269482993327,4cyclictest2454828-21cron22:15:000
2269482993324,6cyclictest2345585-21latency_hist20:25:010
2269482993323,8cyclictest2488931-21ntpq22:46:120
2269482993323,8cyclictest2488931-21ntpq22:46:120
2269489993226,4cyclictest280-21systemd-journal21:39:572
2269486993226,4cyclictest280-21systemd-journal20:45:571
2269486993221,8cyclictest280-21systemd-journal23:43:571
2269482993229,1cyclictest2368359-21kworker/u8:0+flush-8:021:16:170
2269482993227,3cyclictest2360406-21sendmail20:40:010
2269482993222,7cyclictest2355450-21latency_hist20:35:010
2269489993126,4cyclictest2222757-21kworker/u8:2+events_unbound21:26:172
2269486993127,2cyclictest2439943-21basename22:00:001
2269486993127,2cyclictest0-21swapper/121:45:581
2269486993126,4cyclictest2222757-21kworker/u8:2+events_unbound21:20:011
2269486993125,4cyclictest2285937-21grep19:21:191
2269486993124,5cyclictest2539183-21grep23:36:181
2269482993127,3cyclictest2564533-21kworker/u8:2+flush-8:000:26:090
2269482993125,4cyclictest2222757-21kworker/u8:2+flush-8:019:31:160
2269482993124,5cyclictest280-21systemd-journal19:55:570
226948299311,27cyclictest1-21systemd22:00:570
2269486993027,2cyclictest2444948-21systemd-run22:05:001
2269486993026,2cyclictest0-21swapper/121:45:001
2269486993025,4cyclictest2474946-21kworker/u8:1+flush-8:023:06:131
2269486993024,4cyclictest2222757-21kworker/u8:2+events_unbound23:21:171
2269486993020,8cyclictest2329867-21ntpq20:06:121
2269482993025,3cyclictest2368359-21kworker/u8:0+flush-8:022:33:270
2269482993025,3cyclictest2368359-21kworker/u8:0+events_unbound22:36:080
2269482993021,7cyclictest2559303-21kworker/u8:0+flush-8:000:36:120
2269486992925,2cyclictest0-21swapper/122:11:011
2269486992925,2cyclictest0-21swapper/122:06:031
2269486992924,4cyclictest2474946-21kworker/u8:1+events_unbound22:41:171
2269486992924,4cyclictest2259575-21kworker/u8:3+events_unbound21:01:101
2269486992924,3cyclictest2474946-21kworker/u8:1+flush-8:023:10:021
2269486992922,5cyclictest2222757-21kworker/u8:2+flush-8:022:46:131
2269486992922,5cyclictest2222757-21kworker/u8:2+flush-8:022:46:121
2269486992920,7cyclictest2371105-21diskstats20:51:031
2269482992923,5cyclictest2503049-21kworker/u8:0+events_unbound23:11:170
25973702280,9sleep375-21kworker/3:1+mm_percpu_wq00:36:103
23017352280,4sleep00-21swapper/019:41:040
2269489992824,3cyclictest2368359-21kworker/u8:0+events_unbound22:01:112
2269489992824,2cyclictest2505233-21aten2_r5power_c23:05:592
2269489992824,2cyclictest2465439-21aten2_r5power_p22:25:592
2269489992823,3cyclictest2222757-21kworker/u8:2+flush-8:022:06:082
2269489992820,6cyclictest2375562-21cp20:55:012
2269482992824,3cyclictest2222757-21kworker/u8:2+flush-8:022:51:180
2269482992824,2cyclictest2465178-21awk22:25:010
2269482992823,4cyclictest2222757-21kworker/u8:2+events_unbound21:11:130
2269482992822,4cyclictest2340223-21unixbench_multi20:16:190
2269482992820,6cyclictest280-21systemd-journal21:55:000
2269482992820,6cyclictest280-21systemd-journal19:30:000
2269489992723,3cyclictest2503049-21kworker/u8:0+events_unbound23:01:102
2269489992723,2cyclictest2588692-21unixbench_singl00:26:182
2269489992723,2cyclictest2574596-21aten2_r5power_v00:16:022
2269489992722,4cyclictest2222757-21kworker/u8:2+flush-8:019:31:022
2269489992722,3cyclictest2281749-21aten2_r5power_v19:21:012
226948999270,11cyclictest23017342fschecks_time19:41:042
2269486992723,3cyclictest2534557-21kworker/u8:2+flush-8:023:51:161
2269486992720,5cyclictest2594289-21apt-get00:35:581
2269486992718,6cyclictest280-21systemd-journal19:35:571
2269482992724,2cyclictest2499698-21latency_hist23:00:010
2269482992723,3cyclictest588-21ntpd00:31:140
2269482992722,4cyclictest2559303-21kworker/u8:0+flush-8:000:11:170
2269482992722,4cyclictest2529197-21kworker/u8:3+events_unbound23:55:010
2269482992722,3cyclictest2474946-21kworker/u8:1+events_unbound23:36:150
2269482992722,3cyclictest2375723-21apt-key20:55:010
22685322271,8sleep12268554-21ntp_states19:06:151
2269489992622,2cyclictest2594508-21cpuspeed_turbos00:36:012
2269489992622,2cyclictest2564643-21aten2_r5power_c00:06:012
2269489992621,3cyclictest2340241-21unixbench_multi20:16:182
2269486992622,2cyclictest0-21swapper/119:15:001
2269486992622,2cyclictest0-21swapper/100:00:001
2269486992616,7cyclictest280-21systemd-journal20:23:571
2269482992622,2cyclictest0-21swapper/023:01:090
2269482992621,4cyclictest2503049-21kworker/u8:0+events_unbound23:26:050
2269482992621,3cyclictest2361449-21tr20:41:070
2269482992620,4cyclictest2582875-21kthreadcore00:21:080
2269482992620,4cyclictest2333084-21kthreadcore20:11:060
2269489992521,3cyclictest2559303-21kworker/u8:0+flush-8:000:11:182
2269489992520,4cyclictest2368359-21kworker/u8:0+flush-8:021:31:022
2269489992518,5cyclictest860673-21kworker/u8:1+flush-8:020:40:022
226948699259,8cyclictest0-21swapper/123:00:431
2269486992520,4cyclictest2222757-21kworker/u8:2+flush-8:021:11:201
2269486992520,4cyclictest20977-21cron20:45:001
2269486992519,4cyclictest2474946-21kworker/u8:1+events_unbound23:16:141
2269482992520,4cyclictest2414383-21kworker/u8:3+flush-8:021:46:170
2269482992519,5cyclictest2296781-21fschecks_time19:36:030
2269489992421,2cyclictest2425494-21aten2_r5power_c21:46:012
2269489992420,3cyclictest2222757-21kworker/u8:2+events_unbound19:36:162
2269489992419,3cyclictest2311629-21fschecks_time19:51:032
226948999240,8cyclictest25124791kthreadcore23:11:092
2269486992418,5cyclictest2222757-21kworker/u8:2+flush-8:021:21:061
2269482992420,2cyclictest0-21swapper/020:50:010
2269482992420,2cyclictest0-21swapper/020:01:110
2269482992415,6cyclictest2326382-21df_inode20:06:030
2269482992412,3cyclictest0-21swapper/022:41:200
2269489992320,2cyclictest2524276-21unixbench_multi23:21:182
2269489992319,2cyclictest2539977-21aten2_r5power_v23:41:022
2269489992318,3cyclictest2539180-21unixbench_singl23:36:172
2269489992318,3cyclictest2529197-21kworker/u8:3+events_unbound23:51:222
2269482992319,3cyclictest2549077-21unixbench_singl23:46:200
2269482992319,3cyclictest2509469-21unixbench_singl23:06:190
2269482992319,2cyclictest401-21dbus-daemon21:40:010
2269482992318,3cyclictest0-21swapper/021:30:590
2269482992314,7cyclictest2285420-21ntpq19:21:110
2269482992313,8cyclictest280-21systemd-journal23:45:000
2269489992219,2cyclictest0-21swapper/200:25:002
2269489992218,2cyclictest2534192-21unixbench_singl23:31:192
2269489992218,2cyclictest2485383-21cpuspeed_turbos22:46:022
2269489992218,2cyclictest2485383-21cpuspeed_turbos22:46:022
2269489992218,2cyclictest0-21swapper/200:05:002
226948999220,9cyclictest22760902grep19:15:042
2269482992218,3cyclictest2368359-21kworker/u8:0+flush-8:022:16:190
2269482992218,2cyclictest2578477-21mailstats00:16:180
2269482992218,2cyclictest2405353-21systemd-run21:25:010
2269482992217,4cyclictest2368359-21kworker/u8:0+events_unbound21:56:170
2269482992217,3cyclictest2566784-21kthreadcore00:06:090
2269482992211,8cyclictest280-21systemd-journal23:20:000
2269489992118,2cyclictest2356002-21cpuspeed_turbos20:36:022
2269489992117,2cyclictest2395818-21aten2_r5power_e21:16:002
2269489992117,2cyclictest0-21swapper/220:00:002
2269482992118,2cyclictest2533610-21needreboot23:31:090
2269482992116,3cyclictest2390693-21latency_hist21:10:010
2269482992116,3cyclictest2259575-21kworker/u8:3+events_unbound19:15:010
2269489992018,1cyclictest371rcuc/200:31:072
2269489992017,2cyclictest2390106-21unixbench_multi21:06:172
2269489992016,3cyclictest2439948-21sendmail22:00:002
2269489992016,3cyclictest2350130-21unixbench_multi20:26:192
2269489992016,3cyclictest2350130-21unixbench_multi20:26:182
2269489992016,2cyclictest0-21swapper/200:00:002
2269482992015,4cyclictest2560531-21kernelversion00:01:070
2269482992015,3cyclictest0-21swapper/023:23:570
22691282204,10sleep20-21swapper/219:07:362
2269489991915,3cyclictest2529197-21kworker/u8:3+flush-8:023:46:142
2269489991915,3cyclictest2525301-21fschecks_time23:26:032
2269489991915,2cyclictest2331362-21fschecks_count20:11:012
2269489991911,6cyclictest2431073-21sed21:51:062
2269489991910,7cyclictest2343314-21tr20:21:092
2269489991814,2cyclictest0-21swapper/220:56:082
2269489991814,2cyclictest0-21swapper/220:31:182
2269489991713,2cyclictest2516689-21kthreadcore23:16:072
2269489991713,2cyclictest0-21swapper/222:31:092
2269489991713,2cyclictest0-21swapper/219:45:482
2269489991712,3cyclictest371rcuc/223:00:022
2269489991711,4cyclictest2484116-21grep22:41:172
226948999171,12cyclictest2455165-21taskset22:16:002
226948999168,6cyclictest2475254-21seq22:35:592
2269489991613,2cyclictest0-21swapper/221:00:152
2269489991613,2cyclictest0-21swapper/220:05:282
2269489991612,2cyclictest0-21swapper/221:41:042
2269489991612,2cyclictest0-21swapper/221:11:212
24171532150,2sleep3481ktimers/321:36:073
2269489991510,3cyclictest0-21swapper/222:51:082
25700642132,7sleep30-21swapper/300:11:073
25198222132,6sleep30-21swapper/323:20:023
25025602133,6sleep30-21swapper/323:01:083
24752252134,5sleep30-21swapper/322:36:003
24361862133,7sleep30-21swapper/321:56:073
23319692133,6sleep30-21swapper/320:11:063
22856532133,6sleep30-21swapper/319:21:183
226948999134,7cyclictest2286942-21fschecks_time19:26:022
25898322122,6sleep30-21swapper/300:31:053
25691932123,6sleep30-21swapper/300:09:303
25635112123,5sleep30-21swapper/300:01:153
25543582122,5sleep30-21swapper/323:55:003
25487242123,6sleep30-21swapper/323:46:193
25387752122,6sleep30-21swapper/323:36:153
25295872122,5sleep30-21swapper/323:30:013
25253612122,6sleep30-21swapper/323:26:043
25079022122,6sleep30-21swapper/323:06:103
24950292122,6sleep30-21swapper/322:55:583
24918102123,5sleep30-21swapper/322:51:083
24856982122,6sleep30-21swapper/322:46:073
24856982122,6sleep30-21swapper/322:46:073
24705622122,6sleep30-21swapper/322:31:023
24640262122,6sleep30-21swapper/322:21:153
24454882122,6sleep30-21swapper/322:06:013
24300882122,5sleep30-21swapper/321:50:003
24296672122,6sleep30-21swapper/321:46:203
24219442120,8sleep30-21swapper/321:41:103
24058932122,6sleep30-21swapper/321:26:023
24045872122,6sleep30-21swapper/321:21:163
23955192122,6sleep30-21swapper/321:15:023
23873252122,6sleep30-21swapper/321:06:073
23759312122,6sleep30-21swapper/320:55:123
23705572122,6sleep30-21swapper/320:50:023
23585712122,6sleep30-21swapper/320:36:103
23545952122,6sleep30-21swapper/320:31:123
23448072123,6sleep30-21swapper/320:21:173
23299712122,6sleep30-21swapper/320:06:163
23115652122,6sleep30-21swapper/319:51:033
23060732122,5sleep30-21swapper/319:45:003
23013972122,6sleep30-21swapper/319:41:003
22963382122,6sleep30-21swapper/319:35:023
22961252122,6sleep30-21swapper/319:34:583
22911682122,6sleep30-21swapper/319:30:003
22750632122,6sleep30-21swapper/319:15:043
22699622123,6sleep30-21swapper/319:11:023
25843722112,5sleep30-21swapper/300:25:593
25744112112,6sleep30-21swapper/300:15:243
25496772112,6sleep30-21swapper/323:50:403
25437342112,5sleep30-21swapper/323:41:173
25150272112,6sleep30-21swapper/323:15:093
25101112112,5sleep30-21swapper/323:10:593
24802332112,5sleep30-21swapper/322:41:023
24693852112,5sleep30-21swapper/322:26:173
24504902112,5sleep30-21swapper/322:11:043
24447592112,5sleep30-21swapper/322:03:063
24115502112,5sleep30-21swapper/321:31:073
23908002112,5sleep30-21swapper/321:11:013
23811292112,6sleep30-21swapper/321:01:023
23685422112,6sleep30-21swapper/320:46:083
23612202112,6sleep30-21swapper/320:41:073
23460642112,5sleep30-21swapper/320:26:023
23460642112,5sleep30-21swapper/320:26:023
23365572112,5sleep30-21swapper/320:16:073
23211042112,5sleep30-21swapper/320:00:013
23162762112,5sleep30-21swapper/319:56:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional