You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 22:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Thu Nov 27, 2025 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26012042660,2sleep3586-21kworker/3:2+events08:01:143
2547507996256,5cyclictest2772710-21kworker/u8:0+flush-8:011:01:281
2547507995751,4cyclictest2842948-21kworker/u8:1+events_unbound12:31:231
2547507995649,5cyclictest2594638-21kworker/u8:0+flush-8:008:16:231
2547507995649,5cyclictest2594638-21kworker/u8:0+flush-8:008:16:231
2547507995645,8cyclictest2638411-21ntpq08:36:181
2547507995550,4cyclictest2649436-21kworker/u8:1+events_unbound09:06:171
2547507995449,3cyclictest2634051-21kworker/u8:3+flush-8:010:56:221
2547507995446,6cyclictest2634051-21kworker/u8:3+events_unbound10:41:231
2547505995438,13cyclictest2599694-21apt-get08:01:050
2547510995224,9cyclictest282-21systemd-journal11:40:012
2547507995245,5cyclictest2649436-21kworker/u8:1+flush-8:010:21:231
2547505995235,14cyclictest2808474-21apt-get11:31:070
2547507995143,5cyclictest2803446-21kworker/u8:3+flush-8:012:01:201
2547507995141,8cyclictest2817211-21ntpq11:36:251
2547507995042,5cyclictest2852653-21latency_hist12:15:011
2547507995039,8cyclictest2628494-21ntpq08:26:201
28052742490,4chrt371rcuc/211:26:192
2547507994944,4cyclictest2649436-21kworker/u8:1+events_unbound11:26:251
2547507994943,4cyclictest2649436-21kworker/u8:1+events_unbound10:46:241
27699422480,2sleep2371rcuc/210:51:162
2547507994843,4cyclictest2733162-21kworker/u8:2+flush-8:010:31:121
2547507994841,5cyclictest2793320-21latency_hist11:15:011
2547505994840,5cyclictest2668997-21latency_hist09:10:010
2547507994743,3cyclictest1719994-21kworker/u8:4+flush-8:007:16:271
2547507994742,4cyclictest2733162-21kworker/u8:2+events_unbound10:26:121
2547507994741,4cyclictest2817719-21kworker/u8:0+events_unbound12:36:211
2547507994741,4cyclictest2649436-21kworker/u8:1+flush-8:009:01:261
2547507994740,5cyclictest2713748-21kworker/u8:0+events_unbound09:56:251
2547507994740,5cyclictest2649436-21kworker/u8:1+flush-8:009:41:221
2547507994738,7cyclictest2673184-21ntpq09:11:231
2547505994738,7cyclictest2702872-21ntpq09:41:220
2547505994737,7cyclictest282-21systemd-journal09:30:000
2547510994626,7cyclictest282-21systemd-journal07:36:212
2547507994641,3cyclictest2634051-21kworker/u8:3+events_unbound10:16:161
2547507994640,4cyclictest1110744-21kworker/u8:2+flush-8:007:36:251
2547505994637,7cyclictest2833140-21/usr/sbin/munin11:56:220
2547505994634,9cyclictest2866545-21ntpq12:26:190
2547507994541,3cyclictest2778202-21kworker/u8:2+flush-8:011:51:271
2547507994541,3cyclictest2644021-21kworker/u8:0+flush-8:008:51:181
2547507994540,4cyclictest2634051-21kworker/u8:3+flush-8:010:01:221
2547507994539,4cyclictest2547220-21kworker/u8:0+flush-8:007:41:261
2547507994535,7cyclictest2707815-21ntpq09:46:231
2547505994537,6cyclictest2623500-21ntpq08:21:190
2547507994438,4cyclictest2634051-21kworker/u8:3+flush-8:009:16:221
2547505994423,18cyclictest2838091-21/usr/sbin/munin12:01:170
2547507994339,3cyclictest2649436-21kworker/u8:1+events_unbound09:55:011
2547507994338,4cyclictest2569639-21kworker/u8:1+events_unbound07:51:241
2547507994338,4cyclictest1110744-21kworker/u8:2+flush-8:007:21:231
2547507994338,3cyclictest2634051-21kworker/u8:3+flush-8:010:11:141
2547507994336,5cyclictest2634051-21kworker/u8:3+events_unbound08:35:011
2547505994335,6cyclictest1110744-21kworker/u8:2+flush-8:008:31:270
2547510994233,7cyclictest282-21systemd-journal12:40:002
2547507994238,3cyclictest2594638-21kworker/u8:0+events_unbound08:11:111
2547507994238,3cyclictest2594638-21kworker/u8:0+events_unbound08:11:111
2547507994237,4cyclictest1110744-21kworker/u8:2+events_unbound07:46:251
2547507994234,6cyclictest2842948-21kworker/u8:1+flush-8:012:26:211
2547505994233,7cyclictest2728340-21apt-get10:10:000
2547505994224,15cyclictest2712648-21ntpq09:51:220
2547507994137,3cyclictest2817719-21kworker/u8:0+flush-8:012:16:291
2547507994135,4cyclictest2649436-21kworker/u8:1+events_unbound11:06:211
2547507994133,6cyclictest1110744-21kworker/u8:2+flush-8:008:06:211
2547507994132,7cyclictest2772697-21ntpq10:51:211
2547505994123,16cyclictest2832471-21unixbench_multi11:51:280
2547507994034,5cyclictest2649436-21kworker/u8:1+flush-8:009:31:231
2547507994034,4cyclictest1110744-21kworker/u8:2+flush-8:007:26:161
2547507993934,4cyclictest2778202-21kworker/u8:2+flush-8:011:46:191
2547507993934,3cyclictest2733162-21kworker/u8:2+events_unbound10:36:181
2547507993932,5cyclictest2648321-21ntpq08:46:241
2547507993929,8cyclictest2802317-21ntpq11:21:221
2547505993931,6cyclictest2818591-21diskstats11:41:120
2547505993921,16cyclictest2599341-21cron08:00:000
2547507993832,4cyclictest2842949-21kworker/u8:4+flush-8:012:06:201
2547507993831,5cyclictest2634051-21kworker/u8:3+flush-8:009:36:251
25470802384,9sleep10-21swapper/107:08:521
2547507993732,4cyclictest2634518-21kworker/u8:4+events_unbound08:41:231
2547505993721,14cyclictest2654714-21tr08:56:100
2547505993720,15cyclictest2744736-21irqrtprio10:26:140
2547505993715,19cyclictest282-21systemd-journal10:20:030
2547507993630,4cyclictest1110744-21kworker/u8:2+flush-8:007:31:201
2547505993627,7cyclictest2797370-21ntpq11:16:220
2547505993621,13cyclictest2678922-21sendmail09:20:000
2547505993618,16cyclictest2868294-21sort12:31:150
2547505993618,16cyclictest2555544-21irqrtprio07:16:150
2547505993615,19cyclictest2792475-21ntpq11:11:210
2547505993612,7cyclictest2803171-21cron11:25:000
2547510993526,7cyclictest282-21systemd-journal07:35:002
2547507993529,4cyclictest1719994-21kworker/u8:4+events_unbound07:11:241
2547507993529,4cyclictest1110744-21kworker/u8:2+events_unbound08:01:211
2547507993527,6cyclictest2634051-21kworker/u8:3+flush-8:009:26:221
2547505993521,12cyclictest2683859-21munin-plugin-st09:25:000
2547505993519,14cyclictest0-21swapper/007:45:000
2547505993519,14cyclictest0-21swapper/007:30:010
2547505993518,15cyclictest2625004-21fschecks_count08:26:110
2547505993517,16cyclictest2650098-21sort08:51:160
2547505993514,18cyclictest2784311-21sort11:06:150
2547505993513,20cyclictest0-21swapper/010:42:010
27390542340,6sleep3451irq_work/310:21:073
2547507993429,4cyclictest2817719-21kworker/u8:0+flush-8:012:21:151
2547507993428,4cyclictest2803446-21kworker/u8:3+flush-8:011:31:221
2547507993424,8cyclictest282-21systemd-journal09:00:001
2547505993430,2cyclictest2644212-21idleruntime-cro08:45:000
2547505993429,3cyclictest2552521-21latency_hist07:15:010
2547505993418,14cyclictest0-21swapper/010:20:010
2547505993414,18cyclictest2669930-21irqcore09:11:160
2547510993323,8cyclictest282-21systemd-journal12:04:592
2547507993327,5cyclictest2803446-21kworker/u8:3+flush-8:011:56:241
2547507993326,5cyclictest2619758-21apt-get08:21:061
2547505993317,14cyclictest2694625-21sort09:36:120
2547505993317,14cyclictest2570668-21kthreadcore07:31:150
2547505993316,15cyclictest2614092-21unixbench_multi08:11:270
2547505993316,15cyclictest2614092-21unixbench_multi08:11:260
2547505993315,15cyclictest2589813-21apt-get07:51:110
2547505993314,17cyclictest2850339-21kthreadcore12:11:170
2547505993314,17cyclictest2758519-21systemd-run10:40:000
2547505993314,17cyclictest2638892-21unixbench_singl08:36:250
2547505993314,17cyclictest18912-21cron07:50:000
254750599331,16cyclictest2872702-21taskset12:36:080
254750599331,12cyclictest2722637-21ntpq10:01:230
2547510993226,4cyclictest2863200-21fschecks_time12:26:112
2547507993226,4cyclictest2727521-21ntpq10:06:241
2547507993225,5cyclictest2822991-21latency_hist11:45:001
2547505993225,5cyclictest2778881-21wget11:01:100
2547505993217,13cyclictest0-21swapper/011:40:000
25471382324,10sleep20-21swapper/207:09:372
2547510993123,6cyclictest2738610-21cp10:20:012
254750599319,7cyclictest2560804-21cut07:21:160
254750599319,4cyclictest0-21swapper/008:20:000
254750599319,4cyclictest0-21swapper/008:19:590
254750599312,4cyclictest0-21swapper/011:50:010
2547505993124,5cyclictest2728340-21apt-get10:10:030
2547505993124,5cyclictest2713852-21apt-get09:56:070
2547505993116,13cyclictest2662903-21kthreadcore09:01:160
2547505993114,2cyclictest0-21swapper/009:46:160
2547505993112,17cyclictest2858003-21tr12:21:090
2547505993110,7cyclictest2846780-21head12:06:190
2547510993022,6cyclictest282-21systemd-journal09:34:592
254750599303,5cyclictest0-21swapper/008:06:110
254750599303,10cyclictest0-21swapper/010:31:160
254750599301,3cyclictest2645437-21irqstats08:46:190
2547505993013,15cyclictest2803864-21fschecks_time11:26:150
2547510992924,3cyclictest2603954-21unixbench_singl08:01:252
2547510992923,4cyclictest2683840-21cron09:25:002
2547507992925,3cyclictest2594638-21kworker/u8:0+events_unbound07:56:511
254750599299,6cyclictest2764884-21cut10:46:170
254750599298,6cyclictest2775336-21cut10:56:160
254750599298,4cyclictest2692881-21/usr/sbin/munin09:31:220
254750599291,3cyclictest2773520-21munin-run10:55:000
2547505992913,14cyclictest2576857-21kthreadcore07:36:150
2547505992911,2cyclictest0-21swapper/012:16:190
2547510992824,2cyclictest2664629-21cpuspeed_turbos09:06:132
2547507992723,2cyclictest2679842-21irqcore09:21:161
2547507992721,4cyclictest2778202-21kworker/u8:2+flush-8:011:16:201
254750599272,7cyclictest2547184-21apt-get07:10:010
2547510992622,2cyclictest2858029-21aten2_r5power_v12:21:092
2547510992622,2cyclictest2808532-21aten2_r5power_e11:31:102
2547510992619,4cyclictest2648853-21unixbench_singl08:46:312
2547510992521,2cyclictest2708200-21unixbench_multi09:46:272
2547510992520,3cyclictest2663675-21unixbench_multi09:01:262
2547510992520,3cyclictest2649727-21cpuspeed_turbos08:51:122
254751099251,10cyclictest25941282vmstat07:51:322
26309372243,7sleep30-21swapper/308:31:173
2547510992421,2cyclictest2759070-21cpuspeed_turbos10:41:112
2547510992421,2cyclictest2565156-21fschecks_time07:26:112
2547510992420,2cyclictest2723021-21unixbench_multi10:01:282
2547510992418,4cyclictest2639237-21idleruntime-cro08:40:012
2547510992418,4cyclictest2628929-21unixbench_multi08:26:262
2547510992418,4cyclictest2552133-21unixbench_multi07:11:282
25845772230,3chrt586-21kworker/3:2-events07:45:013
2547510992320,2cyclictest2728354-21idleruntime-cro10:10:002
2547510992316,5cyclictest0-21swapper/207:45:032
2547510992315,6cyclictest2742757-21grep10:21:192
28463992222,6sleep30-21swapper/312:06:183
2547510992219,2cyclictest2853020-21aten2_r5power_c12:16:102
2547510992219,2cyclictest2843404-21fschecks_time12:06:122
2547510992219,2cyclictest2828544-21fschecks_time11:51:122
2547510992218,3cyclictest2678897-21turbostat.cron09:20:002
2547510992218,2cyclictest2713964-21aten2_r5power_v09:56:102
254751099212,13cyclictest2744009-21apt-get10:26:072
2547510992117,3cyclictest0-21swapper/207:20:002
2547510992117,2cyclictest2579917-21cpuspeed_turbos07:41:122
26794562200,2sleep3481ktimers/309:21:123
254751099204,3cyclictest27556152grep10:36:182
2547510992016,3cyclictest2818608-21fschecks_count11:41:122
2547510992016,2cyclictest2684405-21cpuspeed_turbos09:26:112
2547510992015,3cyclictest2802118-21latency11:21:202
2547510992014,4cyclictest2643880-21unixbench_singl08:41:272
254751099198,4cyclictest0-21swapper/209:41:102
2547510991915,2cyclictest0-21swapper/209:40:002
2547510991913,4cyclictest2783108-21unixbench_singl11:01:282
254751099191,12cyclictest2783782-21apt-get11:06:092
2547510991910,6cyclictest2753536-21cron10:35:002
25430402193,8sleep30-21swapper/307:06:173
254751099188,8cyclictest2711652-21tr09:51:202
254751099182,12cyclictest2827138-21ntpq11:46:252
2547510991813,3cyclictest2792975-21unixbench_singl11:11:262
2547510991812,4cyclictest0-21swapper/212:11:172
2547510991810,6cyclictest2619405-21cron08:20:002
2547510991810,6cyclictest2619405-21cron08:20:002
2547510991714,2cyclictest2654571-21aten2_r5power_p08:56:082
2547510991714,2cyclictest0-21swapper/210:46:282
2547510991713,2cyclictest0-21swapper/211:16:132
2547510991713,2cyclictest0-21swapper/210:56:202
2547510991713,2cyclictest0-21swapper/210:11:172
2547510991713,2cyclictest0-21swapper/208:35:012
2547510991712,3cyclictest0-21swapper/207:21:172
254751099171,12cyclictest2833165-21apt-get11:56:082
254751099171,11cyclictest2623526-21ntpq08:21:192
254751099168,6cyclictest2868429-21cut12:31:152
254751099162,2cyclictest0-21swapper/209:12:382
254751099162,13cyclictest0-21swapper/207:56:022
2547510991613,2cyclictest2604895-21aten2_r5power_c08:06:092
26594352152,8sleep30-21swapper/309:01:063
25502422150,2sleep3481ktimers/307:11:173
254751099152,12cyclictest0-21swapper/208:10:342
254751099152,12cyclictest0-21swapper/208:10:332
28414252142,8sleep30-21swapper/312:01:173
27709612140,3sleep30-21swapper/310:51:173
27924342132,7sleep30-21swapper/311:11:213
27376112132,7sleep30-21swapper/310:16:233
27184992132,6sleep30-21swapper/310:00:013
26510072132,7sleep30-21swapper/308:51:193
26404742133,7sleep30-21swapper/308:41:163
26194442132,6sleep30-21swapper/308:20:013
25953972133,6sleep30-21swapper/307:56:173
28677682122,6sleep30-21swapper/312:31:103
28627742122,6sleep30-21swapper/312:25:293
28581642122,6sleep30-21swapper/312:21:113
28278852121,6sleep30-21swapper/311:50:003
28178552122,6sleep30-21swapper/311:38:293
28035632122,6sleep30-21swapper/311:26:133
27981202122,6sleep30-21swapper/311:18:473
27870962122,6sleep30-21swapper/311:06:193
27785062122,6sleep30-21swapper/311:00:013
27540762122,6sleep30-21swapper/310:36:133
27491942122,6sleep30-21swapper/310:31:113
27444342123,5sleep30-21swapper/310:26:143
27329042122,6sleep30-21swapper/310:11:293
27145642122,6sleep30-21swapper/309:56:163
27015012122,6sleep30-21swapper/309:41:193
26937842122,6sleep30-21swapper/309:35:013
26894912122,6sleep30-21swapper/309:31:163
26880732123,5sleep30-21swapper/309:26:263
26788842122,6sleep30-21swapper/309:20:003
26702342122,6sleep30-21swapper/309:11:183
26647732122,6sleep30-21swapper/309:06:163
26557672123,5sleep30-21swapper/308:56:173
26289182122,6sleep30-21swapper/308:26:263
26193502122,6sleep30-21swapper/308:19:293
26193502122,6sleep30-21swapper/308:19:293
25906962122,6sleep30-21swapper/307:51:203
25845772122,5sleep30-21swapper/307:45:023
25740742122,6sleep30-21swapper/307:31:293
25656932122,6sleep30-21swapper/307:26:153
25595192122,5sleep30-21swapper/307:20:013
25550662123,6sleep30-21swapper/307:16:123
28771672112,6sleep30-21swapper/312:38:283
28526712112,5sleep30-21swapper/312:15:013
28485502112,5sleep30-21swapper/312:11:163
28371782112,6sleep30-21swapper/311:56:313
28282612112,6sleep30-21swapper/311:51:093
28183572112,5sleep30-21swapper/311:41:103
28084412112,5sleep30-21swapper/311:31:033
27984982112,5sleep30-21swapper/311:20:233
27737952112,5sleep30-21swapper/310:55:473
27679282112,5sleep30-21swapper/310:46:283
27587732112,6sleep30-21swapper/310:40:383
27236792112,6sleep30-21swapper/310:05:393
27098082112,5sleep30-21swapper/309:51:183
27043712112,6sleep30-21swapper/309:46:133
26444972112,5sleep30-21swapper/308:45:493
26348342112,5sleep30-21swapper/308:36:103
26102772112,5sleep30-21swapper/308:11:143
26102772112,5sleep30-21swapper/308:11:143
26052342112,5sleep30-21swapper/308:06:123
25749382112,5sleep30-21swapper/307:36:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional