You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 13:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Dec 05, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4076579994441,2cyclictest211481-21kworker/u8:1+events_unbound00:35:180
4076579994138,2cyclictest211rcuc/000:10:000
4076585994034,4cyclictest132934-21latency_hist23:10:002
4076579993935,3cyclictest91048-21kworker/u8:1+events_unbound22:45:230
4076579993834,3cyclictest4134102-21kworker/u8:1+events_unbound21:40:150
4076579993834,3cyclictest4124007-21kworker/u8:4+flush-259:020:30:320
4076579993734,2cyclictest4077031-21kworker/u8:4+flush-259:019:45:250
4076579993730,5cyclictest2698871-21systemd-journal22:15:010
4076579993632,3cyclictest4124007-21kworker/u8:4+flush-259:020:25:250
4076590993528,5cyclictest2698871-21systemd-journal21:10:003
4076585993529,4cyclictest138663-21base6423:15:012
4076590993329,3cyclictest49496-21kworker/u8:3+flush-259:022:05:003
4076585993326,6cyclictest91048-21kworker/u8:1+flush-259:022:50:012
4076579993330,2cyclictest4134102-21kworker/u8:1+events_unbound21:10:140
4076579993329,3cyclictest196290-21kworker/u8:0+flush-259:000:25:290
4076590993229,2cyclictest211481-21kworker/u8:1+flush-259:000:40:003
4076581993228,3cyclictest143667-21latency_hist23:19:591
4076579993229,2cyclictest4124007-21kworker/u8:4+events_unbound19:55:260
4076579993228,3cyclictest80258-21kworker/u8:0+events_unbound22:20:160
4076590993128,2cyclictest4113159-21gpgv19:45:003
4076579993128,2cyclictest4118134-21kworker/u8:2+events_unbound20:05:260
4076590993027,2cyclictest24986-21kworker/u8:3+flush-259:021:34:593
4076585993026,3cyclictest4186323-21kworker/u8:2+flush-259:021:20:002
4076581992926,2cyclictest24986-21kworker/u8:3+flush-259:021:30:321
4076581992925,3cyclictest185194-21latency_hist00:00:011
4076581992924,4cyclictest4143475-21ntpq20:10:241
4076581992922,5cyclictest226890-21latency_hist00:40:011
4076579992927,2cyclictest38539-21kworker/u8:0+flush-259:021:50:150
4076579992926,2cyclictest4134102-21kworker/u8:1+events_unbound21:30:310
4076579992926,2cyclictest24986-21kworker/u8:3+events_unbound21:35:120
4076579992926,2cyclictest106959-21kworker/u8:2+flush-259:023:25:190
4076579992826,2cyclictest91048-21kworker/u8:1+events_unbound23:00:230
4076579992825,2cyclictest80747-21kworker/u8:3+events_unbound22:40:240
4076579992825,2cyclictest70404-21kworker/u8:2+flush-259:022:15:230
4076579992825,2cyclictest4134102-21kworker/u8:1+flush-259:020:55:200
4076579992825,2cyclictest4134102-21kworker/u8:1+events_unbound20:50:180
4076579992824,3cyclictest4134102-21kworker/u8:1+events_unbound20:10:180
4076590992724,2cyclictest106959-21kworker/u8:2+events_unbound22:50:253
4076579992724,2cyclictest91048-21kworker/u8:1+events_unbound22:35:230
4076579992724,2cyclictest4124007-21kworker/u8:4+flush-259:020:40:290
4076579992723,3cyclictest4124007-21kworker/u8:4+events_unbound20:15:200
4076579992720,5cyclictest4081986-21latency_hist19:15:010
4076590992621,3cyclictest4124007-21kworker/u8:4+flush-259:020:35:173
4076581992623,2cyclictest4124007-21kworker/u8:4+flush-259:019:55:261
4076581992622,3cyclictest4160393-21/usr/sbin/munin20:30:211
4076581992619,5cyclictest127796-21latency_hist23:04:591
4076579992623,2cyclictest4134102-21kworker/u8:1+events_unbound20:20:250
4076579992623,2cyclictest221064-21kworker/u8:3+events_unbound00:31:280
4076579992622,3cyclictest180271-21kworker/u8:2+events_unbound00:15:210
4076579992621,4cyclictest163308-21ntpq23:35:220
4076585992520,4cyclictest180271-21kworker/u8:2+flush-259:000:00:192
4076579992522,2cyclictest91048-21kworker/u8:1+flush-259:022:55:220
4076579992522,2cyclictest4118134-21kworker/u8:2+events_unbound20:00:230
4076579992522,2cyclictest163836-21kworker/u8:1+events_unbound00:00:220
4076579992522,2cyclictest132949-21kworker/u8:0+events_unbound23:40:160
4076579992522,2cyclictest106959-21kworker/u8:2+events_unbound22:50:290
4076579992521,3cyclictest132949-21kworker/u8:0+events_unbound23:50:190
4076579992521,3cyclictest132949-21kworker/u8:0+events_unbound23:50:190
4076581992420,3cyclictest117661-21kworker/u8:4+events_unbound00:10:001
4076581992419,4cyclictest4144866-21base6420:15:011
4076581992417,5cyclictest4112199-21ntpq19:40:221
4076579992422,2cyclictest4175615-21kworker/u8:0+events_unbound21:00:280
4076579992422,2cyclictest117661-21kworker/u8:4+events_unbound23:20:230
4076579992421,2cyclictest4134102-21kworker/u8:1+events_unbound21:45:240
4076579992421,2cyclictest132949-21kworker/u8:0+events_unbound23:12:270
4076579992420,3cyclictest4134102-21kworker/u8:1+flush-259:021:20:200
4076590992320,2cyclictest4097569-21kworker/u8:1+events_unbound19:50:143
4076590992320,2cyclictest106959-21kworker/u8:2+events_unbound23:00:013
4076590992319,3cyclictest138239-21gpgv23:15:003
4076585992319,3cyclictest2698871-21systemd-journal19:50:012
4076585992319,3cyclictest174483-21idleruntime-cro23:50:002
4076581992320,2cyclictest132949-21kworker/u8:0+events_unbound23:50:231
4076581992320,2cyclictest132949-21kworker/u8:0+events_unbound23:50:231
4076581992319,3cyclictest106959-21kworker/u8:2+events_unbound22:51:291
4076579992317,4cyclictest4102701-21latency_hist19:35:000
4076585992219,2cyclictest180271-21kworker/u8:2+flush-259:000:20:002
4076581992219,2cyclictest80747-21kworker/u8:3+events_unbound22:25:191
4076581992218,3cyclictest49496-21kworker/u8:3+events_unbound22:00:171
4076581992218,3cyclictest4077031-21kworker/u8:4+events_unbound19:10:191
4076581992218,3cyclictest180271-21kworker/u8:2+events_unbound00:15:251
4076579992219,2cyclictest4077031-21kworker/u8:4+events_unbound19:20:120
4076590992118,2cyclictest38539-21kworker/u8:0+events_unbound22:05:013
4076581992117,3cyclictest91048-21kworker/u8:1+events_unbound23:05:261
4076581992117,3cyclictest49496-21kworker/u8:3+events_unbound22:05:221
4076581992117,3cyclictest4165550-21apt-get20:35:131
4076581992117,2cyclictest4134102-21kworker/u8:1+flush-259:021:20:181
4076581992115,4cyclictest74520-21ntpq22:10:241
4076581992115,4cyclictest59828-21latency_hist22:00:011
4076579992117,3cyclictest4134102-21kworker/u8:1+flush-259:020:48:370
4076590992017,2cyclictest91048-21kworker/u8:1+flush-259:022:35:183
4076585992017,2cyclictest4175615-21kworker/u8:0+events_unbound21:05:232
4076585992015,4cyclictest196290-21kworker/u8:0+flush-259:000:25:312
4076585992015,3cyclictest174496-21apt-get23:50:022
4076585992015,3cyclictest174496-21apt-get23:50:022
4076581992017,2cyclictest196290-21kworker/u8:0+flush-259:000:25:211
4076581992016,3cyclictest4134102-21kworker/u8:1+events_unbound20:55:171
4076581992015,4cyclictest4134102-21kworker/u8:1+events_unbound21:04:281
4076581992015,3cyclictest4175581-21apt-get20:45:021
4076579992017,2cyclictest117661-21kworker/u8:4+events_unbound23:15:280
4076579992016,3cyclictest4077031-21kworker/u8:4+events_unbound19:40:140
4076579992015,4cyclictest54908-21/usr/sbin/munin21:55:180
4076590991916,2cyclictest117661-21kworker/u8:4+flush-259:023:45:003
4076590991915,3cyclictest4175615-21kworker/u8:0+flush-259:021:15:243
4076590991915,3cyclictest185467-21gpgv00:00:003
4076590991915,3cyclictest106959-21kworker/u8:2+flush-259:023:25:003
4076590991915,2cyclictest0-21swapper/322:50:013
4076590991914,3cyclictest2698871-21systemd-journal22:32:273
4076585991916,2cyclictest185205-21munin-plugin-st00:00:002
4076585991915,2cyclictest0-21swapper/219:45:002
4076585991914,3cyclictest80757-21/usr/sbin/munin22:20:242
4076585991913,4cyclictest38539-21kworker/u8:0+flush-259:021:45:212
4076581991916,2cyclictest4124007-21kworker/u8:4+flush-259:020:40:151
4076581991915,3cyclictest3000061-21in:imuxsock21:54:591
4076581991913,4cyclictest138127-21latency_hist23:15:001
4076579991916,2cyclictest80747-21kworker/u8:3+flush-259:022:25:220
4076579991916,2cyclictest196290-21kworker/u8:0+flush-259:000:20:260
4076579991916,2cyclictest132949-21kworker/u8:0+flush-259:023:45:250
4076590991815,2cyclictest117661-21kworker/u8:4+flush-259:023:50:243
4076590991815,2cyclictest117661-21kworker/u8:4+flush-259:023:50:243
4076590991814,3cyclictest106959-21kworker/u8:2+flush-259:023:25:193
4076581991815,2cyclictest4118134-21kworker/u8:2+flush-259:020:05:151
4076581991814,3cyclictest4175615-21kworker/u8:0+events_unbound21:06:271
4076581991814,3cyclictest2698871-21systemd-journal20:22:111
4076579991815,2cyclictest4186323-21kworker/u8:2+events_unbound21:25:200
4076579991815,2cyclictest4134102-21kworker/u8:1+events_unbound20:35:350
4076579991815,2cyclictest4086181-21needreboot19:15:180
4076590991714,2cyclictest4097569-21kworker/u8:1+flush-259:019:45:173
4076590991713,3cyclictest4186323-21kworker/u8:2+flush-259:021:37:273
4076590991713,3cyclictest117661-21kworker/u8:4+events_unbound23:30:173
4076585991714,2cyclictest60511-21irqcore22:00:162
4076585991714,2cyclictest3002-21cron21:29:592
4076585991713,2cyclictest0-21swapper/220:35:002
4076581991714,3cyclictest4077031-21kworker/u8:4+events_unbound19:35:251
4076581991714,2cyclictest4077031-21kworker/u8:4+events_unbound19:45:221
4076581991714,2cyclictest4065762-21kworker/u8:0+flush-259:019:30:211
4076581991713,3cyclictest91049-21kworker/u8:2+events_unbound22:30:251
4076581991712,4cyclictest180271-21kworker/u8:2+events_unbound00:10:141
4076590991614,1cyclictest0-21swapper/300:10:003
4076590991613,2cyclictest4134102-21kworker/u8:1+events_unbound21:20:243
4076590991613,2cyclictest164215-21sh23:40:003
4076590991611,4cyclictest4129181-21/usr/sbin/munin20:00:173
407658599169,5cyclictest2698871-21systemd-journal21:25:002
407658599169,1cyclictest0-21swapper/221:35:002
4076585991612,3cyclictest70173-21munin-plugin-st22:09:592
4076585991611,4cyclictest106959-21kworker/u8:2+flush-259:023:25:172
4076581991613,2cyclictest4134102-21kworker/u8:1+events_unbound20:25:161
4076581991613,2cyclictest4065762-21kworker/u8:0+flush-259:019:50:241
4076581991613,2cyclictest117661-21kworker/u8:4+flush-259:023:30:261
4076581991613,1cyclictest0-21swapper/121:14:591
4076581991612,3cyclictest4134102-21kworker/u8:1+flush-259:021:45:271
4076581991612,3cyclictest4134102-21kworker/u8:1+events_unbound20:50:241
4076581991612,3cyclictest4087388-21kworker/u8:2+flush-259:019:25:001
4076581991612,3cyclictest24986-21kworker/u8:3+events_unbound21:35:181
4076579991614,2cyclictest117661-21kworker/u8:4+events_unbound23:08:270
4076579991613,2cyclictest1-21systemd22:35:000
4076579991613,2cyclictest0-21swapper/000:00:000
4076579991612,3cyclictest106959-21kworker/u8:2+events_unbound23:33:010
4076590991513,1cyclictest0-21swapper/300:20:003
4076590991512,2cyclictest4134102-21kworker/u8:1+flush-259:020:10:163
4076590991512,2cyclictest196290-21kworker/u8:0+flush-259:000:30:263
4076590991512,1cyclictest0-21swapper/320:10:003
4076590991511,3cyclictest4134102-21kworker/u8:1+events_unbound20:25:223
407658599158,5cyclictest2698871-21systemd-journal21:54:592
4076585991513,1cyclictest0-21swapper/219:40:002
4076585991512,2cyclictest96038-21sh22:35:012
4076585991511,3cyclictest163318-21ntp_kernel_pll_23:35:222
4076585991511,2cyclictest0-21swapper/223:20:172
4076585991511,2cyclictest0-21swapper/222:10:182
4076585991510,4cyclictest54896-21kworker/u8:1+flush-259:021:55:262
4076585991510,4cyclictest4124007-21kworker/u8:4+events_unbound20:15:182
4076581991512,2cyclictest4097543-21idleruntime-cro19:30:011
4076581991511,3cyclictest91048-21kworker/u8:1+events_unbound22:35:151
4076581991511,3cyclictest70404-21kworker/u8:2+events_unbound22:15:201
4076581991511,3cyclictest4124007-21kworker/u8:4+flush-259:020:05:001
4076579991513,1cyclictest60650irq/133-enp0s29f1:rx-022:00:230
4076579991513,1cyclictest60650irq/133-enp0s29f1:rx-019:25:170
4076579991512,2cyclictest60650irq/133-enp0s29f1:rx-019:35:150
4076579991512,2cyclictest4175615-21kworker/u8:0+events_unbound21:05:180
407659099149,4cyclictest2698871-21systemd-journal21:48:283
4076590991411,2cyclictest2002190-21kworker/u8:3+flush-259:019:15:233
4076590991411,2cyclictest2002190-21kworker/u8:3+events_unbound19:10:213
4076590991411,2cyclictest117661-21kworker/u8:4+events_unbound23:15:163
4076590991410,3cyclictest91048-21kworker/u8:1+events_unbound23:05:233
4076590991410,3cyclictest4124007-21kworker/u8:4+flush-259:020:45:243
407658599148,4cyclictest4102681-21cron19:35:012
4076585991411,2cyclictest90817-21sh22:30:012
4076585991411,2cyclictest0-21swapper/200:40:002
4076585991410,3cyclictest4077031-21kworker/u8:4+flush-259:019:10:162
4076585991410,2cyclictest0-21swapper/219:10:012
4076585991410,2cyclictest0-21swapper/219:10:002
4076581991411,2cyclictest0-21swapper/100:25:001
4076579991412,1cyclictest60650irq/133-enp0s29f1:rx-000:10:160
4076579991411,2cyclictest4097569-21kworker/u8:1+flush-259:019:50:110
4076579991411,2cyclictest18134-21latency_hist21:20:010
4076579991410,3cyclictest49496-21kworker/u8:3+events_unbound22:05:590
407659099139,3cyclictest4124007-21kworker/u8:4+flush-259:019:55:183
407659099139,3cyclictest132949-21kworker/u8:0+flush-259:023:45:263
407659099137,4cyclictest4087576-21diskstats19:20:133
4076590991310,2cyclictest4175142-21unixbench_singl20:40:293
4076590991310,2cyclictest196290-21kworker/u8:0+events_unbound00:20:253
4076590991310,2cyclictest127783-21cron23:05:003
407658599139,3cyclictest4077031-21kworker/u8:4+flush-259:019:25:242
4076585991310,2cyclictest97104-21sed22:35:172
4076585991310,2cyclictest200166-21needreboot00:10:212
407658199139,3cyclictest91048-21kworker/u8:1+events_unbound22:57:361
4076581991310,2cyclictest196290-21kworker/u8:0+flush-259:000:30:211
407659099129,2cyclictest81186-21irqcore22:20:173
407659099129,2cyclictest4185814-21unixbench_multi20:50:293
407659099129,2cyclictest4164814-21unixbench_multi20:30:313
407659099129,2cyclictest196290-21kworker/u8:0+flush-259:000:25:213
407659099121,9cyclictest4097530-21cron19:30:003
407658599129,2cyclictest4190972-21unixbench_multi20:55:272
407658599129,2cyclictest4124465-21irqcore19:55:192
407658599129,2cyclictest4119089-21fschecks_count19:50:142
407658599129,2cyclictest122945-21cpuspeed_turbos23:00:132
407658599129,1cyclictest0-21swapper/200:20:192
407658599128,2cyclictest0-21swapper/222:55:252
407658599127,3cyclictest158714-21sshd23:30:282
4076585991210,1cyclictest4144786-21apt-config20:15:012
4076585991210,1cyclictest0-21swapper/220:45:022
4076585991210,1cyclictest0-21swapper/220:10:002
407658199129,2cyclictest91048-21kworker/u8:1+flush-259:022:40:271
407658199128,3cyclictest60057-21kworker/u8:4+flush-259:022:22:471
407658199128,3cyclictest2002190-21kworker/u8:3+flush-259:019:15:191
407659099119,1cyclictest4134102-21kworker/u8:1+events_unbound21:40:003
407659099118,2cyclictest4124007-21kworker/u8:4+events_unbound20:15:233
407659099118,2cyclictest27964-21unixbench_multi21:25:283
407658599119,1cyclictest0-21swapper/222:40:162
407658599119,1cyclictest0-21swapper/200:05:262
407658599118,2cyclictest4150225-21cpuspeed_turbos20:20:132
407658599118,2cyclictest168928-21unixbench_singl23:40:252
407658599117,3cyclictest79542-21needreboot22:15:182
407658599117,3cyclictest4129428-21fschecks_time20:00:172
407658199118,2cyclictest132949-21kworker/u8:0+events_unbound23:22:421
407658199117,3cyclictest4186323-21kworker/u8:2+events_unbound21:25:171
407659099107,2cyclictest186025-21cpuspeed_turbos00:00:163
407659099106,3cyclictest51603-21tr21:50:173
407659099106,3cyclictest4186323-21kworker/u8:2+flush-259:021:00:003
407659099105,3cyclictest4102801-21cat19:35:013
407658599107,2cyclictest8492-21irqcore21:10:162
407658599107,2cyclictest62050irq/146-enp0s29f1:tx-621:43:082
407658599103,5cyclictest221735-21latency_hist00:35:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional