You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 09:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Thu Nov 27, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6996332920,5chrt0-21swapper/321:21:193
8895142600,3sleep00-21swapper/000:36:150
573235995746,8cyclictest873144-21ntpq00:16:210
573240995647,6cyclictest649580-21latency_hist20:25:011
573235995342,8cyclictest838578-21ntpq23:41:190
573240995147,3cyclictest563296-21kworker/u8:3+flush-8:021:31:241
7975802500,2sleep1291ktimers/123:01:171
573240995046,3cyclictest685875-21kworker/u8:4+flush-8:021:11:261
573240995045,4cyclictest720549-21kworker/u8:1+flush-8:021:51:251
573240994843,4cyclictest531737-21kworker/u8:2+flush-8:019:46:241
573240994841,5cyclictest765253-21kworker/u8:2+flush-8:023:21:201
573242994722,8cyclictest735539-21cat22:00:012
573240994743,3cyclictest563296-21kworker/u8:3+flush-8:021:06:131
573240994742,4cyclictest661155-21kworker/u8:1+flush-8:020:51:231
573240994740,5cyclictest809758-21kworker/u8:0+events_unbound00:36:241
573235994741,4cyclictest563296-21kworker/u8:3+flush-8:020:51:120
573240994639,5cyclictest854510-21kworker/u8:1+flush-8:000:11:221
573240994638,6cyclictest720632-21apt-get21:46:071
573240994540,4cyclictest715569-21kworker/u8:0+events_unbound22:01:161
573240994540,4cyclictest531737-21kworker/u8:2+flush-8:019:56:271
573240994539,4cyclictest715569-21kworker/u8:0+flush-8:022:21:211
573235994540,4cyclictest720549-21kworker/u8:1+flush-8:021:56:240
573235994538,5cyclictest282-21systemd-journal23:56:150
573235994537,6cyclictest573073-21kworker/u8:1+flush-8:020:11:140
573235994436,6cyclictest282-21systemd-journal21:10:000
573235994435,7cyclictest705452-21latency_hist21:30:010
573235994435,6cyclictest755463-21/usr/sbin/munin22:21:200
573240994338,3cyclictest765253-21kworker/u8:2+events_unbound23:46:201
573240994338,3cyclictest715569-21kworker/u8:0+flush-8:022:11:241
573240994337,5cyclictest720549-21kworker/u8:1+flush-8:022:31:271
573240994337,4cyclictest765253-21kworker/u8:2+events_unbound23:31:271
573240994337,4cyclictest765253-21kworker/u8:2+events_unbound23:31:271
573240994336,5cyclictest720549-21kworker/u8:1+flush-8:022:46:241
573235994332,8cyclictest805420-21diskstats23:11:110
573240994237,3cyclictest715569-21kworker/u8:0+events_unbound22:06:231
573240994234,3cyclictest715569-21kworker/u8:0+events_unbound22:26:151
573240994228,5cyclictest809758-21kworker/u8:0+events_unbound23:51:291
573235994233,7cyclictest879591-21diskstats00:26:150
573242994127,3cyclictest815082-21aten2_r5power_c23:21:092
573242994118,8cyclictest609240-21ntpq19:41:212
573240994136,3cyclictest790032-21kworker/u8:3+flush-8:023:11:171
573240994135,4cyclictest573073-21kworker/u8:1+events_unbound20:20:001
573235994132,7cyclictest690956-21/usr/sbin/munin21:16:180
573235994130,8cyclictest584178-21ntpq19:16:210
573242994031,7cyclictest282-21systemd-journal21:30:012
573240994035,3cyclictest720549-21kworker/u8:1+events_unbound21:56:271
573235994036,3cyclictest765253-21kworker/u8:2+flush-8:023:46:150
573235994035,4cyclictest715569-21kworker/u8:0+flush-8:022:11:260
573235994032,5cyclictest615006-21latency_hist19:50:010
573240993934,4cyclictest573073-21kworker/u8:1+flush-8:020:11:291
573240993934,3cyclictest809758-21kworker/u8:0+events_unbound23:20:011
573240993927,8cyclictest282-21systemd-journal23:36:141
573235993932,5cyclictest282-21systemd-journal22:42:540
573240993834,3cyclictest563296-21kworker/u8:3+flush-8:019:31:161
573240993833,4cyclictest715569-21kworker/u8:0+flush-8:022:16:231
573235993833,3cyclictest720549-21kworker/u8:1+flush-8:021:46:230
573240993733,3cyclictest563296-21kworker/u8:3+flush-8:020:56:241
573240993733,3cyclictest531737-21kworker/u8:2+events_unbound19:16:261
573240993732,3cyclictest765253-21kworker/u8:2+events_unbound22:36:231
573240993729,6cyclictest700519-21latency_hist21:25:011
573240993729,5cyclictest794749-21latency_hist23:00:011
573235993733,3cyclictest685874-21kworker/u8:0+flush-8:021:21:240
573235993729,6cyclictest282-21systemd-journal20:05:000
573240993631,4cyclictest531737-21kworker/u8:2+events_unbound19:41:201
573240993631,3cyclictest765253-21kworker/u8:2+flush-8:023:41:161
573240993630,5cyclictest661155-21kworker/u8:1+flush-8:021:01:221
573235993632,3cyclictest531737-21kworker/u8:2+flush-8:020:06:250
573235993629,5cyclictest563296-21kworker/u8:3+events_unbound20:56:200
573235993627,6cyclictest282-21systemd-journal23:21:140
573240993531,2cyclictest602398-21kthreadcore19:36:181
573240993530,4cyclictest685874-21kworker/u8:0+events_unbound21:16:251
573240993530,3cyclictest563296-21kworker/u8:3+flush-8:019:21:261
573240993529,4cyclictest819967-21apt-get23:26:101
573240993529,4cyclictest765253-21kworker/u8:2+flush-8:000:20:311
573240993528,5cyclictest790032-21kworker/u8:3+flush-8:023:06:201
573240993527,6cyclictest765253-21kworker/u8:2+flush-8:000:06:201
573235993527,6cyclictest858385-21ntpq00:01:220
573240993430,3cyclictest854510-21kworker/u8:1+events_unbound00:01:261
573240993430,3cyclictest809758-21kworker/u8:0+flush-8:000:31:271
573240993430,2cyclictest712866-21kthreadcore21:36:161
573240993429,3cyclictest563296-21kworker/u8:3+events_unbound21:41:091
573240993429,3cyclictest531737-21kworker/u8:2+events_unbound19:11:151
573240993428,4cyclictest685875-21kworker/u8:4+events_unbound21:26:201
573240993426,6cyclictest282-21systemd-journal00:15:251
573240993425,6cyclictest619938-21latency_hist19:55:011
573235993428,4cyclictest720549-21kworker/u8:1+flush-8:022:06:180
573235993428,4cyclictest655744-21latency_hist20:40:000
573235993428,4cyclictest655744-21latency_hist20:40:000
573235993427,5cyclictest719417-21ntpq21:41:190
573235993424,8cyclictest282-21systemd-journal19:55:000
573235993422,10cyclictest715332-21systemctl21:40:000
573242993326,5cyclictest805029-21cp23:10:012
573242993323,8cyclictest844344-21cat23:50:012
573240993329,3cyclictest874276-21kworker/u8:3+events_unbound00:26:201
573240993327,4cyclictest531737-21kworker/u8:2+flush-8:019:26:211
573235993327,4cyclictest854510-21kworker/u8:1+flush-8:000:11:160
573235993323,8cyclictest282-21systemd-journal22:05:000
573240993228,2cyclictest563296-21kworker/u8:3+events_unbound20:31:101
573240993227,4cyclictest765253-21kworker/u8:2+flush-8:023:56:251
573240993226,5cyclictest563296-21kworker/u8:3+flush-8:020:46:211
573240993226,5cyclictest563296-21kworker/u8:3+flush-8:020:46:211
573235993227,4cyclictest531737-21kworker/u8:2+flush-8:019:26:120
573240993124,5cyclictest775271-21apt-get22:41:061
573235993127,3cyclictest649577-21cron20:25:000
573235993124,5cyclictest650506-21/usr/sbin/munin20:31:130
573242993023,5cyclictest282-21systemd-journal21:17:382
573240993023,6cyclictest563296-21kworker/u8:3+events_unbound20:41:201
573240993023,6cyclictest563296-21kworker/u8:3+events_unbound20:41:201
573235993025,3cyclictest720549-21kworker/u8:1+events_unbound22:31:230
573235993024,4cyclictest765253-21kworker/u8:2+events_unbound00:10:000
573235993024,4cyclictest600804-21sort19:36:160
573240992922,6cyclictest531737-21kworker/u8:2+events_unbound20:06:211
573235992926,2cyclictest784868-21systemd-run22:50:000
573235992926,2cyclictest589972-21turbostat.cron19:25:000
573235992923,4cyclictest720549-21kworker/u8:1+flush-8:023:01:260
573242992823,3cyclictest849266-21cron23:55:012
573240992823,4cyclictest531737-21kworker/u8:2+events_unbound20:01:111
573235992825,2cyclictest804608-21cat23:10:000
573235992824,3cyclictest639895-21kworker/u8:0+flush-8:020:16:270
573235992823,4cyclictest809758-21kworker/u8:0+flush-8:023:15:460
573235992823,4cyclictest563296-21kworker/u8:3+flush-8:020:44:460
573235992823,4cyclictest563296-21kworker/u8:3+flush-8:020:44:450
57323599282,25cyclictest0-21swapper/021:01:260
573235992822,4cyclictest720549-21kworker/u8:1+events_unbound22:20:000
573235992822,4cyclictest563296-21kworker/u8:3+flush-8:019:41:240
5720382283,6sleep20-21swapper/219:06:192
573242992724,2cyclictest854248-21awk00:00:002
573242992718,7cyclictest282-21systemd-journal19:30:002
573235992723,3cyclictest531737-21kworker/u8:2+events_unbound19:56:290
573235992722,4cyclictest685874-21kworker/u8:0+flush-8:021:13:460
573235992722,3cyclictest809758-21kworker/u8:0+flush-8:000:31:280
573235992721,4cyclictest729811-21unixbench_multi21:51:280
573235992716,9cyclictest282-21systemd-journal21:35:000
573242992621,4cyclictest765253-21kworker/u8:2-ext4-rsv-conversion23:20:002
573242992619,5cyclictest824286-21grep23:26:302
573240992621,2cyclictest0-21swapper/122:51:141
573235992623,2cyclictest715569-21kworker/u8:0+flush-8:022:26:280
573235992622,2cyclictest774911-21cron22:40:000
573235992620,4cyclictest832-21mta-sts-daemon20:29:150
57323599260,25cyclictest542016-21kworker/0:0+events23:38:300
74012422517,5sleep30-21swapper/322:02:133
573242992521,2cyclictest864754-21fschecks_count00:11:112
573242992515,8cyclictest745270-21awk22:10:002
57323599250,24cyclictest542016-21kworker/0:0+events23:25:100
573242992421,2cyclictest805460-21fschecks_count23:11:112
573242992421,2cyclictest750748-21cpuspeed_turbos22:16:102
573242992421,2cyclictest587523-21tr19:21:192
573242992420,3cyclictest681350-21fschecks_count21:06:122
573242992420,3cyclictest675764-21munin-run21:00:012
573242992420,2cyclictest0-21swapper/200:06:162
573235992420,2cyclictest828610-21needreboot23:31:200
573235992420,2cyclictest828610-21needreboot23:31:200
573235992420,2cyclictest0-21swapper/022:56:280
573235992418,4cyclictest874276-21kworker/u8:3+flush-8:000:21:180
573235992418,4cyclictest765253-21kworker/u8:2+events_unbound22:51:230
573235992418,4cyclictest531737-21kworker/u8:2+flush-8:019:31:310
573242992319,2cyclictest854659-21aten2_r5power_e00:01:102
573242992319,2cyclictest775620-21fschecks_time22:41:112
573242992319,2cyclictest649194-21unixbench_multi20:21:262
573242992319,2cyclictest640089-21aten2_r5power_p20:16:122
573242992318,3cyclictest690282-21grep21:11:272
57324299230,10cyclictest7835742kthreadcore22:46:202
57323599232,20cyclictest542016-21kworker/0:0+events23:51:060
573235992319,3cyclictest563296-21kworker/u8:3+flush-8:020:46:250
573235992319,3cyclictest563296-21kworker/u8:3+flush-8:020:46:250
573235992317,4cyclictest574036-21irqcore19:11:140
5728562234,13sleep10-21swapper/119:07:421
573242992219,2cyclictest755602-21aten2_r5power_v22:21:112
573242992219,2cyclictest604482-21unixbench_multi19:36:292
573242992218,2cyclictest615650-21fschecks_time19:51:132
573242992210,4cyclictest0-21swapper/200:31:252
57324299219,9cyclictest665916-21awk20:50:002
57324299219,9cyclictest665916-21awk20:50:002
57324299212,14cyclictest765363-21apt-get22:31:122
573242992118,2cyclictest838521-21needreboot23:41:172
573242992117,2cyclictest774549-21unixbench_multi22:36:282
573242992117,2cyclictest635600-21irqcore20:11:162
573242992117,2cyclictest0-21swapper/223:36:152
573242992117,2cyclictest0-21swapper/221:21:152
573242992116,3cyclictest878334-21/usr/sbin/munin00:21:262
573240992116,3cyclictest832-21mta-sts-daemon20:40:001
573240992116,3cyclictest832-21mta-sts-daemon20:40:001
5728502212,13sleep00-21swapper/019:07:370
573242992016,2cyclictest760536-21aten2_r5power_v22:26:112
573242992016,2cyclictest0-21swapper/220:45:002
573242992016,2cyclictest0-21swapper/220:45:002
573242992015,4cyclictest577784-21unixbench_singl19:11:282
573242992014,5cyclictest0-21swapper/221:01:262
573242992012,6cyclictest740356-21turbostat.cron22:04:592
5684772203,9sleep30-21swapper/319:05:473
573242991915,2cyclictest0-21swapper/221:46:132
57324299189,7cyclictest612325-21tr19:46:172
57324299186,4cyclictest0-21swapper/220:06:272
573242991814,2cyclictest0-21swapper/200:40:012
573242991813,3cyclictest726491-21sed21:51:162
57324299172,4cyclictest0-21swapper/219:59:012
573242991713,2cyclictest0-21swapper/221:36:152
573242991713,2cyclictest0-21swapper/221:31:122
573242991713,2cyclictest0-21swapper/220:01:122
573242991713,2cyclictest0-21swapper/200:16:132
573242991711,4cyclictest791022-21kthreadcore22:56:162
57324299171,11cyclictest599091-21ntpq19:31:222
57324299162,2cyclictest0-21swapper/222:13:362
57324299162,13cyclictest0-21swapper/220:50:022
57324299162,13cyclictest0-21swapper/200:26:142
57324299162,12cyclictest655727-21cron20:40:002
57324299162,12cyclictest655727-21cron20:39:592
573242991613,2cyclictest650821-21fschecks_count20:31:072
573242991613,2cyclictest0-21swapper/222:52:492
573242991612,2cyclictest0-21swapper/219:16:272
573240991613,2cyclictest831-21mta-sts-daemon20:29:121
6141342150,2sleep3586-21kworker/3:2-events19:46:203
57324299152,12cyclictest0-21swapper/220:25:372
573242991511,2cyclictest715679-21aten2_r5power_c21:41:092
573242991510,3cyclictest0-21swapper/223:31:172
573242991510,3cyclictest0-21swapper/223:31:172
57324299142,9cyclictest798846-21ntpq23:01:222
8839232132,6sleep30-21swapper/300:30:003
8747282132,7sleep30-21swapper/300:21:143
8550892132,7sleep30-21swapper/300:01:163
8463532130,3sleep3586-21kworker/3:2+events23:51:203
8345012132,6sleep30-21swapper/323:40:003
7848392132,6sleep30-21swapper/322:50:013
7749642132,6sleep30-21swapper/322:40:013
6606772132,7sleep30-21swapper/320:45:003
6606772132,7sleep30-21swapper/320:45:003
8802602122,6sleep30-21swapper/300:26:203
8699492122,6sleep30-21swapper/300:16:173
8643902122,6sleep30-21swapper/300:10:113
8595602122,6sleep30-21swapper/300:06:093
8496522123,6sleep30-21swapper/323:56:103
8433362122,6sleep30-21swapper/323:46:193
8216112122,6sleep30-21swapper/323:26:213
8186212122,6sleep30-21swapper/323:21:193
8005032122,6sleep30-21swapper/323:06:143
7959792123,5sleep30-21swapper/323:01:163
7687702122,6sleep30-21swapper/322:31:233
7606722122,6sleep30-21swapper/322:26:133
7502162122,6sleep30-21swapper/322:15:013
7311232121,7sleep30-21swapper/321:56:153
7252432122,6sleep30-21swapper/321:50:013
7208712122,6sleep30-21swapper/321:46:123
7161512122,7sleep30-21swapper/321:41:143
7106992122,6sleep30-21swapper/321:36:093
6953652122,6sleep30-21swapper/321:17:393
6845662122,6sleep30-21swapper/321:06:193
6799702122,6sleep30-21swapper/321:01:253
6443302122,6sleep30-21swapper/320:16:313
6347202122,6sleep30-21swapper/320:10:003
6249152122,5sleep30-21swapper/320:00:013
6202762122,6sleep30-21swapper/319:56:123
6153402122,6sleep30-21swapper/319:51:113
6059672122,6sleep30-21swapper/319:41:153
5998982122,5sleep30-21swapper/319:35:013
5993562122,6sleep30-21swapper/319:31:283
5907952122,7sleep30-21swapper/319:26:143
5754172122,6sleep30-21swapper/319:11:183
8932892112,5sleep30-21swapper/300:36:313
8297792113,5sleep30-21swapper/323:35:273
8292252112,5sleep30-21swapper/323:31:303
8292252112,5sleep30-21swapper/323:31:303
8097492112,5sleep30-21swapper/323:15:013
8050622112,5sleep30-21swapper/323:10:033
7928062112,5sleep30-21swapper/322:56:183
7851092112,5sleep30-21swapper/322:50:293
7707912112,5sleep30-21swapper/322:36:163
7555492112,5sleep30-21swapper/322:21:113
7457132112,5sleep30-21swapper/322:11:123
7444292112,5sleep30-21swapper/322:06:233
7056892112,6sleep30-21swapper/321:30:143
7012692112,5sleep30-21swapper/321:26:133
6862282112,5sleep30-21swapper/321:11:133
6720572112,5sleep30-21swapper/320:56:153
6667222111,6sleep30-21swapper/320:51:143
6615002112,5sleep30-21swapper/320:46:123
6615002112,5sleep30-21swapper/320:46:113
6551292112,5sleep30-21swapper/320:35:013
6551292112,5sleep30-21swapper/320:35:013
6504432112,5sleep30-21swapper/320:30:153
6500852112,6sleep30-21swapper/320:28:393
6453492112,5sleep30-21swapper/320:21:123
5891442112,5sleep30-21swapper/319:21:253
5783632112,5sleep30-21swapper/319:15:033
6349592102,5sleep30-21swapper/320:10:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional