You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 04:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Mon Dec 08, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2304494996864,3cyclictest2304650-21kworker/u8:3+events_unbound19:15:032
2304491996048,9cyclictest282-21systemd-journal20:31:071
2304488995848,7cyclictest2574354-21latency_hist23:40:010
25019702570,2sleep3586-21kworker/3:2-events22:26:113
2304494995651,4cyclictest2247489-21kworker/u8:0+flush-8:019:11:262
2304494995651,3cyclictest2594672-21kworker/u8:4+flush-8:000:31:202
2304488995544,8cyclictest282-21systemd-journal23:24:590
2304494995448,4cyclictest2474960-21kworker/u8:3+flush-8:000:11:202
2304494995348,4cyclictest2366230-21kworker/u8:3+events_unbound20:40:012
2304491995343,8cyclictest2494392-21ntpq22:16:161
2304491995245,5cyclictest2489432-21ntpq22:11:151
2304491995141,7cyclictest2543801-21ntpq23:06:201
2304488995039,9cyclictest282-21systemd-journal00:10:000
2304491994938,9cyclictest2326200-21apt-get19:30:001
2304488994941,6cyclictest282-21systemd-journal23:30:000
2304494994742,4cyclictest2331409-21kworker/u8:1+events_unbound20:20:002
2304491994737,7cyclictest282-21systemd-journal20:37:081
2304488994737,7cyclictest282-21systemd-journal23:15:000
2304491994636,8cyclictest2633376-21ntpq00:36:161
2304488994638,5cyclictest2554570-21latency_hist23:20:010
2304494994540,4cyclictest2366230-21kworker/u8:3+events_unbound20:41:202
2304494994540,4cyclictest1461127-21kworker/u8:2+flush-8:020:31:202
2304494994438,4cyclictest2331409-21kworker/u8:1+events_unbound20:20:042
2304491994438,5cyclictest2331409-21kworker/u8:1+flush-8:019:46:191
2304491994338,3cyclictest2366230-21kworker/u8:3+events_unbound20:45:011
2304491994335,6cyclictest2406045-21/usr/sbin/munin20:51:151
2304491994333,8cyclictest2424646-21ntpq21:06:181
2304491994333,8cyclictest2424646-21ntpq21:06:181
2304488994335,5cyclictest282-21systemd-journal20:18:080
2304488994332,9cyclictest2394963-21ntpq20:36:170
26099082420,4sleep2371rcuc/200:16:062
2304488994237,4cyclictest2474960-21kworker/u8:3+events_unbound22:16:120
2304491994136,4cyclictest1461127-21kworker/u8:2+flush-8:019:36:071
2304491994133,6cyclictest282-21systemd-journal22:35:011
2304488994036,3cyclictest2594672-21kworker/u8:4+events_unbound00:16:200
2304488994032,5cyclictest2524951-21latency_hist22:50:010
2304488994030,8cyclictest282-21systemd-journal19:15:010
2304494993933,4cyclictest2534515-21kworker/u8:0+events_unbound23:10:022
2304494993833,4cyclictest2331409-21kworker/u8:1+flush-8:021:56:242
2304491993831,5cyclictest282-21systemd-journal20:49:171
2304491993830,6cyclictest2326765-21apt-get19:31:051
2304488993828,8cyclictest2456007-21diskstats21:41:050
230448899382,31cyclictest2346471-21kworker/0:1+events20:05:240
2304491993729,5cyclictest2375254-21ntpq20:16:181
2304491993728,6cyclictest282-21systemd-journal23:22:241
2304488993731,4cyclictest2445281-21kworker/u8:0+events_unbound21:46:110
230448899372,31cyclictest2346471-21kworker/0:1+events23:00:520
230448899372,30cyclictest2346471-21kworker/0:1+events22:20:360
2304491993631,4cyclictest2331409-21kworker/u8:1+events_unbound22:01:211
2304491993631,3cyclictest2569413-21systemd-run23:35:001
2304491993629,5cyclictest2331409-21kworker/u8:1+flush-8:021:46:151
2304491993628,6cyclictest2609843-21apt-get00:16:021
230448899360,30cyclictest2336604-21kworker/0:0+events19:40:390
2304491993526,6cyclictest282-21systemd-journal21:21:221
2304491993525,8cyclictest282-21systemd-journal20:06:051
2304491993524,8cyclictest282-21systemd-journal22:00:011
2304488993529,4cyclictest2247489-21kworker/u8:0+events_unbound19:31:070
2304488993527,6cyclictest282-21systemd-journal19:55:000
2304488993527,6cyclictest2510539-21aten2_r5power_v22:36:040
2304491993429,4cyclictest2474960-21kworker/u8:3+flush-8:023:56:231
2304491993424,8cyclictest2548779-21ntpq23:11:151
2304488993430,3cyclictest2445281-21kworker/u8:0+flush-8:022:11:110
2304488993426,6cyclictest2435972-21diskstats21:21:080
230448899342,31cyclictest2346471-21kworker/0:1+events20:03:280
230448899341,32cyclictest2346471-21kworker/0:1+events21:58:520
230448899341,28cyclictest2594421-21kworker/0:2+events00:10:520
230448899340,32cyclictest2346471-21kworker/0:1+events20:54:360
2304494993329,3cyclictest2534515-21kworker/u8:0+events_unbound23:10:012
2304491993328,4cyclictest2509938-21kworker/u8:2+events_unbound22:46:191
2304491993328,3cyclictest2420552-21cron21:05:001
2304491993328,3cyclictest2420552-21cron21:05:001
2304491993327,4cyclictest2385629-21grep20:26:231
2304488993326,5cyclictest2624328-21cron00:30:000
2304488993326,5cyclictest2430844-21aten2_r5power_e21:16:050
2304488993326,5cyclictest2420180-21grep21:01:230
2304488993326,5cyclictest2420180-21grep21:01:220
230448899332,30cyclictest919000-21kworker/0:3+events19:17:120
230448899332,30cyclictest2614491-21kworker/0:1+events00:33:350
230448899332,30cyclictest2346471-21kworker/0:1+events22:55:040
2304488993322,9cyclictest2584494-21turbostat.cron23:50:000
230448899332,26cyclictest2346471-21kworker/0:1+events20:55:240
230448899331,31cyclictest2346471-21kworker/0:1+events20:42:040
2304488993310,19cyclictest2594421-21kworker/0:2+events00:20:040
230448899330,29cyclictest2346471-21kworker/0:1+events20:30:160
2304494993228,3cyclictest2247489-21kworker/u8:0+events_unbound20:04:592
2304491993226,5cyclictest1461127-21kworker/u8:2+events_unbound19:51:181
2304491993222,8cyclictest2484486-21ntpq22:06:161
230448899322,29cyclictest2346471-21kworker/0:1+events22:04:320
230448899321,30cyclictest2346471-21kworker/0:1+events22:44:480
230448899321,30cyclictest2346471-21kworker/0:1+events22:08:280
230448899321,30cyclictest2346471-21kworker/0:1+events19:53:360
230448899321,26cyclictest2346471-21kworker/0:1+events20:45:350
230448899320,31cyclictest2346471-21kworker/0:1+events23:33:480
230448899320,31cyclictest2346471-21kworker/0:1+events21:06:560
230448899320,31cyclictest2346471-21kworker/0:1+events21:06:560
25253182310,4sleep0201rcuc/022:51:050
2304491993127,3cyclictest2376655-21cpuspeed_turbos20:21:071
2304488993125,4cyclictest2505164-21latency_hist22:30:010
2304488993124,6cyclictest2346083-21unixbench_multi19:46:210
2304488993123,6cyclictest2334101-21kthreadcore19:36:120
230448899312,28cyclictest2346471-21kworker/0:1+events21:31:160
230448899311,29cyclictest2346471-21kworker/0:1+events23:57:350
230448899311,25cyclictest2346471-21kworker/0:1+events21:50:120
230448899310,29cyclictest919000-21kworker/0:3+events19:29:240
230448899310,26cyclictest2614491-21kworker/0:1+events00:35:040
230448899310,26cyclictest2346471-21kworker/0:1+events21:25:310
2304494993025,4cyclictest2331409-21kworker/u8:1+flush-8:020:25:102
2304491993026,3cyclictest2509938-21kworker/u8:2+events_unbound23:46:201
2304491993024,4cyclictest2331409-21kworker/u8:1+events_unbound19:56:071
2304491993022,6cyclictest2599404-21cron00:05:001
2304488993023,6cyclictest2584848-21aten2_r5power_c23:51:050
230448899302,24cyclictest2346471-21kworker/0:1+events21:35:040
230448899302,23cyclictest2346471-21kworker/0:1+events22:30:550
230448899301,26cyclictest2346471-21kworker/0:1+events21:10:080
230448899300,29cyclictest2346471-21kworker/0:1+events23:41:520
2304494992920,7cyclictest2326200-21apt-get19:30:002
2304491992925,3cyclictest2444897-21unixbench_multi21:26:221
2304491992925,2cyclictest2426181-21cut21:11:071
2304491992922,6cyclictest1461127-21kworker/u8:2+events_unbound19:16:181
2304491992922,5cyclictest2608461-21ntpq00:11:161
230448899292,26cyclictest2346471-21kworker/0:1+events20:10:400
2304488992911,17cyclictest2346471-21kworker/0:1+events23:09:040
2304494992824,3cyclictest2247489-21kworker/u8:0+events_unbound20:05:032
2304494992823,4cyclictest2509938-21kworker/u8:2+events_unbound23:26:262
2304494992819,6cyclictest2579440-21cat23:45:002
2304491992824,2cyclictest0-21swapper/120:56:241
2304491992823,4cyclictest2331409-21kworker/u8:1+flush-8:021:53:511
2304491992822,4cyclictest2336907-21apt-get19:41:021
2304491992822,4cyclictest2331409-21kworker/u8:1+events_unbound21:41:221
2304491992821,5cyclictest2515082-21latency_hist22:40:011
2304491992820,6cyclictest2455168-21cron21:40:001
230448899289,14cyclictest2594421-21kworker/0:2+events00:00:120
230448899282,25cyclictest919000-21kworker/0:3+events19:21:520
230448899281,26cyclictest2346471-21kworker/0:1+events20:23:120
2304494992720,5cyclictest2524948-21awk22:50:002
2304491992720,5cyclictest2361270-21latency_hist20:05:011
2304491992717,8cyclictest2588620-21ntpq23:51:181
24211542260,3sleep3586-21kworker/3:2-events21:06:093
24211542260,3sleep3586-21kworker/3:2-events21:06:083
2304494992622,3cyclictest2247489-21kworker/u8:0+flush-8:019:45:102
2304494992622,3cyclictest1461127-21kworker/u8:2+flush-8:021:26:072
2304494992622,2cyclictest2440016-21unixbench_singl21:21:242
2304494992618,6cyclictest2455176-21apt-get21:40:032
2304494992617,7cyclictest2351017-21unixbench_multi19:51:242
2304491992621,3cyclictest2600093-21cut00:06:091
2304491992621,3cyclictest2594672-21kworker/u8:4+flush-8:000:30:171
2304491992619,5cyclictest2430786-21apt-get21:16:021
2304491992618,6cyclictest282-21systemd-journal22:24:251
2304491992615,8cyclictest282-21systemd-journal19:14:041
230448899266,19cyclictest2346471-21kworker/0:1+events20:28:440
2304494992521,2cyclictest2490728-21aten2_r5power_p22:16:052
2304494992521,2cyclictest2411117-21aten2_r5power_v20:56:062
2304494992520,4cyclictest2331409-21kworker/u8:1+events_unbound22:33:242
2304494992520,3cyclictest2331409-21kworker/u8:1+flush-8:021:45:062
230449199256,5cyclictest0-21swapper/123:17:361
2304491992521,2cyclictest0-21swapper/123:41:091
2304491992518,6cyclictest1461127-21kworker/u8:2+flush-8:020:11:201
2304494992421,2cyclictest2629705-21aten2_r5power_p00:36:032
2304494992421,2cyclictest2480840-21aten2_r5power_v22:06:062
230449499241,9cyclictest24513572cut21:36:092
2304494992419,4cyclictest2474960-21kworker/u8:3+flush-8:000:21:542
2304494992419,4cyclictest2331409-21kworker/u8:1+flush-8:022:25:212
2304494992419,3cyclictest2529544-21unixbench_singl22:51:232
2304491992421,2cyclictest2505152-21cron22:30:001
2304491992420,3cyclictest2539445-21unixbench_singl23:01:231
2304491992413,9cyclictest282-21systemd-journal22:45:011
2304494992319,2cyclictest2445677-21aten2_r5power_p21:31:052
2304491992319,2cyclictest0-21swapper/100:26:251
2304491992318,4cyclictest2509938-21kworker/u8:2+flush-8:023:25:521
2304494992219,2cyclictest2514687-21unixbench_multi22:36:222
2304494992217,4cyclictest2474960-21kworker/u8:3+events_unbound22:10:022
2304494992217,3cyclictest2316880-21cpuspeed_turbos19:21:062
2304494992213,7cyclictest2601341-21tr00:06:132
2304494992211,9cyclictest2470238-21awk21:55:012
230449499220,7cyclictest25682581kthreadcore23:31:152
2304491992218,3cyclictest2474960-21kworker/u8:3+flush-8:000:21:101
2304491992218,2cyclictest0-21swapper/122:51:051
2304491992217,3cyclictest0-21swapper/119:10:011
2304491992216,5cyclictest2331409-21kworker/u8:1+flush-8:021:31:171
2304491992215,5cyclictest2570432-21grep23:36:101
2304494992116,4cyclictest1461127-21kworker/u8:2+events_unbound20:14:182
2304494992115,4cyclictest2519676-21unixbench_singl22:41:222
2304491992118,2cyclictest2534121-21mailstats22:56:221
2304494992011,7cyclictest2351733-21apt-get19:56:032
2304491992013,6cyclictest1461127-21kworker/u8:2+events_unbound19:21:251
22997802204,9sleep20-21swapper/219:06:022
23959502190,3sleep3586-21kworker/3:2-events20:40:013
2304494991916,2cyclictest2416307-21fschecks_time21:01:072
2304494991916,2cyclictest2416307-21fschecks_time21:01:072
2304494991913,4cyclictest2478008-21tr22:01:132
230449499187,3cyclictest371rcuc/223:01:232
230449499182,4cyclictest0-21swapper/219:36:102
2304494991814,3cyclictest2304483-21cyclictest23:36:132
2304494991814,2cyclictest0-21swapper/219:35:002
2304494991813,4cyclictest2474960-21kworker/u8:3+flush-8:022:55:212
2304494991811,5cyclictest2553125-21tr23:16:122
230449499181,12cyclictest2336907-21apt-get19:41:042
2304494991810,6cyclictest2623312-21seq00:26:162
2304494991810,6cyclictest2599403-21cron00:04:592
2304488991811,5cyclictest2304338-21latency_hist19:10:000
23040422183,8sleep30-21swapper/319:06:273
230449499177,8cyclictest2585036-21cpuspeed_turbos23:51:062
230449499173,6cyclictest0-21swapper/221:16:502
2304494991714,2cyclictest2496152-21cut22:21:092
230449499162,6cyclictest0-21swapper/223:21:172
2304494991613,2cyclictest0-21swapper/221:05:322
2304494991613,2cyclictest0-21swapper/221:05:322
230449499161,2cyclictest0-21swapper/221:11:172
2304494991612,2cyclictest0-21swapper/220:51:072
2304494991612,2cyclictest0-21swapper/220:46:232
230449499152,12cyclictest0-21swapper/223:45:042
2304494991510,3cyclictest0-21swapper/223:56:112
24803872142,6sleep30-21swapper/322:05:013
25894692132,6sleep30-21swapper/323:55:013
24613122132,7sleep30-21swapper/321:46:113
24345302132,7sleep30-21swapper/321:16:183
23899782132,7sleep30-21swapper/320:31:153
23212532132,6sleep30-21swapper/319:25:013
26338062122,6sleep30-21swapper/300:36:233
26153082123,5sleep30-21swapper/300:21:093
26089542122,6sleep30-21swapper/300:11:233
25951222122,6sleep30-21swapper/300:01:113
25750632122,6sleep30-21swapper/323:41:093
25644782122,6sleep30-21swapper/323:30:003
25578522122,6sleep30-21swapper/323:21:143
25499592123,5sleep30-21swapper/323:16:053
25487652123,6sleep30-21swapper/323:11:153
25397512122,6sleep30-21swapper/323:05:013
25158982122,6sleep30-21swapper/322:41:093
25120682122,6sleep30-21swapper/322:36:123
24870402122,6sleep30-21swapper/322:11:113
24832552122,6sleep30-21swapper/322:06:143
24748602122,6sleep30-21swapper/321:56:233
24509602122,6sleep30-21swapper/321:36:083
24406812122,6sleep30-21swapper/321:26:053
24397452122,6sleep30-21swapper/321:21:233
24198032122,6sleep30-21swapper/321:01:213
24198032122,6sleep30-21swapper/321:01:213
24114612122,6sleep30-21swapper/320:56:113
24057572122,6sleep30-21swapper/320:50:013
23621242122,6sleep30-21swapper/320:06:113
23595742123,6sleep30-21swapper/320:01:133
23555142123,6sleep30-21swapper/319:56:193
23358352122,6sleep30-21swapper/319:36:213
23267712122,6sleep30-21swapper/319:31:073
23256042122,6sleep30-21swapper/319:26:263
23145072122,6sleep30-21swapper/319:16:133
23085972122,6sleep30-21swapper/319:11:253
26251872112,6sleep30-21swapper/300:31:093
26197932112,5sleep30-21swapper/300:26:083
26136262112,5sleep30-21swapper/300:16:203
25999932112,5sleep30-21swapper/300:06:083
25850672112,5sleep30-21swapper/323:51:083
25835762113,5sleep30-21swapper/323:46:143
25740722112,6sleep30-21swapper/323:36:393
25353842112,5sleep30-21swapper/323:01:083
25302442112,5sleep30-21swapper/322:56:053
25256442112,6sleep30-21swapper/322:51:093
25203112112,6sleep30-21swapper/322:46:033
25054962112,5sleep30-21swapper/322:31:063
24955952112,5sleep30-21swapper/322:21:033
24906462112,5sleep30-21swapper/322:16:053
24656292112,5sleep30-21swapper/321:51:043
24557302112,5sleep30-21swapper/321:41:023
24455972112,5sleep30-21swapper/321:31:063
24260782112,5sleep30-21swapper/321:11:083
24000062112,5sleep30-21swapper/320:41:223
23813592112,5sleep30-21swapper/320:26:053
23763302113,5sleep30-21swapper/320:20:103
23711612112,5sleep30-21swapper/320:15:013
23705302112,5sleep30-21swapper/320:11:243
23464632112,5sleep30-21swapper/319:50:013
23418452112,5sleep30-21swapper/319:46:043
23371482112,5sleep30-21swapper/319:41:063
25636452102,5sleep30-21swapper/323:26:203
24010342102,5sleep30-21swapper/320:45:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional