You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 09:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Jun 30, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3882770994845,2cyclictest4053811-21kworker/u8:1+flush-259:022:00:003
3882760994642,2cyclictest211rcuc/022:40:000
3882770994542,2cyclictest4099776-21kworker/u8:3+flush-259:022:45:003
3882765994340,2cyclictest3903384-21kworker/u8:0+flush-259:020:00:012
3882770994035,3cyclictest335557-21systemd-journal22:25:413
3882760993832,4cyclictest335557-21systemd-journal00:25:010
3882760993733,3cyclictest4183415-21latency_hist00:00:010
3882770993632,3cyclictest4147380-21latency_hist23:25:013
3882770993632,3cyclictest4137081-21kworker/u8:3+flush-259:023:40:153
3882760993630,4cyclictest335557-21systemd-journal22:34:590
3882760993629,5cyclictest4052877-21ntpq21:50:220
3882770993529,4cyclictest335557-21systemd-journal21:00:013
3882770993529,4cyclictest335557-21systemd-journal21:00:003
388276299350,34cyclictest1-21systemd22:34:591
3882770993429,4cyclictest3965855-21kworker/u8:0+flush-259:021:55:003
3882765993428,4cyclictest4032932-21cp21:35:012
3882770993330,2cyclictest4147407-21kworker/u8:1+flush-259:023:25:173
3882765993326,5cyclictest4069-21ntpq00:10:232
3882770993229,2cyclictest3913920-21kworker/u8:3+flush-259:020:15:113
3882770993228,3cyclictest3960643-21kworker/u8:1+events_unbound20:40:003
3882765993229,2cyclictest3965855-21kworker/u8:0+flush-259:021:45:162
3882762993127,3cyclictest3913920-21kworker/u8:3+flush-259:019:45:151
3882762993126,3cyclictest3966069-21/usr/sbin/munin20:30:141
3882762993123,5cyclictest4043507-21latency_hist21:45:021
3882770993026,3cyclictest3893267-21kworker/u8:2+events_unbound19:25:193
3882770993025,4cyclictest3960680-21latency_hist20:25:003
3882765993027,2cyclictest827180-21kworker/u8:1+flush-259:019:20:172
3882765993026,3cyclictest3913920-21kworker/u8:3+events_unbound19:55:012
3882765993025,4cyclictest4147407-21kworker/u8:1+events_unbound23:35:242
3882765993025,3cyclictest4183650-21/usr/sbin/munin00:00:132
3882765993024,4cyclictest4089835-21latency_hist22:30:012
3882762993023,4cyclictest335557-21systemd-journal19:40:001
3882765992926,2cyclictest3872134-21kworker/u8:0+flush-259:019:10:172
3882765992925,3cyclictest4147407-21kworker/u8:1+flush-259:023:30:152
3882762992924,3cyclictest3924240-21kworker/u8:1+flush-259:020:05:011
3882760992922,5cyclictest3939109-21ntpq20:00:230
3882770992824,3cyclictest3913920-21kworker/u8:3+events_unbound19:55:153
3882765992824,3cyclictest4048533-21kworker/u8:2+events_unbound22:36:422
3882765992824,3cyclictest3986886-21kworker/u8:2+flush-259:020:50:422
3882765992824,3cyclictest3913920-21kworker/u8:3+flush-259:019:40:152
3882762992824,3cyclictest3960643-21kworker/u8:1+events_unbound21:20:171
3882762992823,4cyclictest4099776-21kworker/u8:3+flush-259:022:49:591
3882760992821,5cyclictest4048655-21latency_hist21:50:010
3882765992724,2cyclictest4048533-21kworker/u8:2+flush-259:023:20:252
3882765992723,3cyclictest3950301-21kworker/u8:2+events_unbound20:15:002
3882762992720,5cyclictest4047780-21ntpq21:45:211
3882760992723,3cyclictest4100403-21apt-get22:40:120
3882760992719,5cyclictest4178276-21latency_hist23:55:000
3882770992623,2cyclictest4173133-21kworker/u8:4+events_unbound00:10:403
3882765992622,3cyclictest3913920-21kworker/u8:3+events_unbound20:25:192
3882762992619,4cyclictest4162822-21latency_hist23:40:001
3882760992620,5cyclictest3918869-21latency_hist19:45:010
3882760992620,4cyclictest4167118-21ntpq23:40:210
3882770992522,3cyclictest4012160-21kworker/u8:4+flush-259:021:25:413
3882770992521,3cyclictest4147407-21kworker/u8:1+events_unbound23:33:403
3882770992521,3cyclictest4012160-21kworker/u8:4+flush-259:021:22:403
3882770992520,4cyclictest3882691-21latency_hist19:10:013
3882762992521,3cyclictest4048533-21kworker/u8:2+flush-259:022:59:591
3882760992520,3cyclictest3960643-21kworker/u8:1+flush-259:021:05:000
3882770992419,4cyclictest335557-21systemd-journal00:30:003
3882762992419,3cyclictest3902500-21ntpq19:25:221
3882760992421,2cyclictest3965855-21kworker/u8:0+flush-259:021:55:250
3882760992417,5cyclictest4083851-21ntpq22:20:200
3882770992321,2cyclictest4173132-21kworker/u8:2+events_unbound00:15:413
3882770992320,2cyclictest4053811-21kworker/u8:1+flush-259:022:35:243
3882770992320,2cyclictest4048533-21kworker/u8:2+flush-259:023:00:183
3882765992320,2cyclictest4173132-21kworker/u8:2+flush-259:000:30:252
3882765992320,2cyclictest4053811-21kworker/u8:1+events_unbound22:20:112
3882765992319,3cyclictest4115652-21kworker/u8:0+events_unbound23:12:422
3882765992319,3cyclictest3960643-21kworker/u8:1+events_unbound20:56:422
3882765992319,3cyclictest3960643-21kworker/u8:1+events_unbound20:56:422
3882765992318,4cyclictest335557-21systemd-journal21:30:002
3882765992317,4cyclictest4007030-21latency_hist21:10:002
3882762992319,3cyclictest3882985-21apt-get19:10:121
3882760992317,4cyclictest4142252-21latency_hist23:20:010
3882770992219,2cyclictest4137081-21kworker/u8:3+events_unbound23:45:243
3882770992218,3cyclictest4053811-21kworker/u8:1+events_unbound22:15:183
3882770992218,3cyclictest3883270-21kworker/u8:3+flush-259:019:10:413
3882770992216,4cyclictest4115624-21latency_hist22:55:003
3882765992218,3cyclictest4173132-21kworker/u8:2+flush-259:000:30:012
3882765992218,3cyclictest4147407-21kworker/u8:1+events_unbound23:25:122
3882765992218,3cyclictest4137081-21kworker/u8:3+events_unbound23:42:142
3882765992218,3cyclictest3965855-21kworker/u8:0+events_unbound21:19:422
3882762992217,3cyclictest509-21rs:main1
3882762992217,3cyclictest4032749-21latency_hist21:35:001
3882762992216,4cyclictest4054027-21/usr/sbin/munin21:55:161
3882762992216,4cyclictest15485-21/usr/sbin/munin00:25:141
3882760992219,2cyclictest3913920-21kworker/u8:3+flush-259:019:50:000
3882760992218,3cyclictest4053811-21kworker/u8:1+events_unbound22:25:220
3882770992118,2cyclictest827180-21kworker/u8:1+flush-259:019:35:243
3882770992118,2cyclictest3913920-21kworker/u8:3+flush-259:020:00:203
3882770992117,3cyclictest4163050-21kworker/u8:0+events_unbound23:50:233
3882765992119,1cyclictest3913920-21kworker/u8:3+events_unbound20:00:172
3882765992118,2cyclictest4099776-21kworker/u8:3+flush-259:022:45:102
3882765992118,2cyclictest4048533-21kworker/u8:2+events_unbound23:00:222
3882765992118,2cyclictest4048533-21kworker/u8:2+events_unbound22:55:222
3882765992117,3cyclictest4188789-21kworker/u8:3+events_unbound00:05:392
3882765992117,3cyclictest3960643-21kworker/u8:1+flush-259:020:40:202
3882765992116,4cyclictest4110727-21apt-get22:50:142
3882762992114,4cyclictest4002103-21/usr/sbin/munin21:05:141
388276299211,11cyclictest4955-21munin-run00:15:011
3882760992118,2cyclictest4063725-21kworker/u8:3+events_unbound22:05:000
3882760992118,2cyclictest4053811-21kworker/u8:1+flush-259:022:15:000
3882760992117,3cyclictest4188789-21kworker/u8:3+flush-259:000:10:280
3882760992117,3cyclictest4173133-21kworker/u8:4+events_unbound00:05:190
3882760992117,3cyclictest4173132-21kworker/u8:2+flush-259:000:30:190
3882770992017,2cyclictest4115652-21kworker/u8:0+events_unbound23:17:413
3882770992017,2cyclictest4048533-21kworker/u8:2+flush-259:022:30:203
3882770992016,3cyclictest4173132-21kworker/u8:2+flush-259:000:35:163
3882770992016,3cyclictest4162449-21sshd23:35:403
3882770992016,3cyclictest4012160-21kworker/u8:4+flush-259:021:30:243
3882770992016,3cyclictest4012160-21kworker/u8:4+events_unbound21:35:123
3882770992015,4cyclictest3960919-21/usr/sbin/munin20:25:173
3882770992015,4cyclictest335557-21systemd-journal00:00:003
3882765992017,3cyclictest3960643-21kworker/u8:1+events_unbound20:48:402
3882765992017,3cyclictest3913920-21kworker/u8:3+events_unbound20:33:412
3882765992017,2cyclictest4173133-21kworker/u8:4+events_unbound23:55:142
3882765992017,2cyclictest3965855-21kworker/u8:0+events_unbound20:36:502
3882765992016,3cyclictest4048533-21kworker/u8:2+events_unbound22:02:352
3882760992016,3cyclictest3903384-21kworker/u8:0+flush-259:020:10:210
3882760992015,4cyclictest3898188-21latency_hist19:25:010
3882770991916,2cyclictest827180-21kworker/u8:1+flush-259:019:15:253
3882770991916,2cyclictest4048533-21kworker/u8:2+events_unbound22:55:253
3882770991916,2cyclictest3893267-21kworker/u8:2+flush-259:019:30:243
3882770991914,3cyclictest335557-21systemd-journal23:07:413
3882770991914,3cyclictest335557-21systemd-journal20:06:403
3882765991916,3cyclictest4173132-21kworker/u8:2+events_unbound00:22:522
3882765991916,2cyclictest3960643-21kworker/u8:1+events_unbound21:00:152
3882765991915,3cyclictest4069066-21kworker/u8:0+flush-259:022:10:212
3882762991916,2cyclictest3960643-21kworker/u8:1+flush-259:020:25:011
3882762991915,3cyclictest827180-21kworker/u8:1+flush-259:019:17:451
3882762991915,3cyclictest4173133-21kworker/u8:4+flush-259:023:55:151
3882762991914,3cyclictest4115652-21kworker/u8:0+flush-259:023:20:011
3882762991913,4cyclictest335557-21systemd-journal23:25:001
3882760991916,2cyclictest3960643-21kworker/u8:1+events_unbound21:05:180
3882760991915,3cyclictest4163050-21kworker/u8:0+events_unbound23:45:150
3882760991915,3cyclictest4053811-21kworker/u8:1+flush-259:022:15:150
3882760991915,3cyclictest3965855-21kworker/u8:0+flush-259:021:00:010
3882760991915,3cyclictest3965855-21kworker/u8:0+flush-259:021:00:000
3882760991914,4cyclictest4130616-21ntpq23:05:210
3882760991913,4cyclictest335557-21systemd-journal20:26:400
3882760991913,4cyclictest335557-21systemd-journal00:27:400
3882760991912,5cyclictest4157710-21latency_hist23:35:010
3882770991815,2cyclictest4173132-21kworker/u8:2+flush-259:000:30:253
3882765991815,2cyclictest4048533-21kworker/u8:2+flush-259:022:30:242
3882765991815,2cyclictest3965855-21kworker/u8:0+flush-259:021:50:242
3882765991815,2cyclictest3960643-21kworker/u8:1+flush-259:021:20:242
3882765991815,2cyclictest3913920-21kworker/u8:3+flush-259:019:45:232
3882765991814,3cyclictest827180-21kworker/u8:1+flush-259:019:30:262
3882765991814,3cyclictest4173132-21kworker/u8:2+events_unbound00:17:422
3882765991814,3cyclictest4069066-21kworker/u8:0+flush-259:022:40:202
3882765991814,3cyclictest3903384-21kworker/u8:0+flush-259:020:05:252
3882762991815,2cyclictest4173132-21kworker/u8:2+flush-259:000:30:171
3882762991814,3cyclictest4069066-21kworker/u8:0+flush-259:022:10:251
3882762991813,3cyclictest509-21rs:main1
3882762991813,3cyclictest3976405-21apt-get20:40:121
3882760991814,3cyclictest4038579-21kworker/u8:3+events_unbound21:40:150
3882760991814,3cyclictest3960643-21kworker/u8:1+flush-259:021:15:260
3882760991814,3cyclictest3960643-21kworker/u8:1+flush-259:020:35:000
3882760991813,4cyclictest335557-21systemd-journal20:10:000
3882760991811,5cyclictest4152568-21latency_hist23:30:000
3882770991714,2cyclictest3965855-21kworker/u8:0+flush-259:021:11:453
3882770991713,3cyclictest4059213-21apt-get22:00:133
3882770991711,4cyclictest3919031-21base6419:45:013
3882765991714,2cyclictest827180-21kworker/u8:1+events_unbound19:17:152
388276299172,7cyclictest3898187-21apt19:25:001
3882762991714,2cyclictest4192873-21ntp_kernel_pll_00:05:261
3882762991713,3cyclictest4173133-21kworker/u8:4+events_unbound00:20:011
3882762991713,3cyclictest3960643-21kworker/u8:1+flush-259:021:15:101
3882762991712,4cyclictest4168233-21apt-get23:45:121
3882762991712,3cyclictest3996990-21apt-get21:00:121
3882760991713,3cyclictest4115652-21kworker/u8:0+flush-259:023:15:000
3882760991713,3cyclictest3965855-21kworker/u8:0+events_unbound20:40:140
3882760991713,3cyclictest3893267-21kworker/u8:2+flush-259:019:25:160
3882760991713,2cyclictest60350irq/132-enp0s29f1:rx-020:50:140
3882760991712,4cyclictest4032742-21apt-get21:35:020
3882770991614,1cyclictest0-21swapper/319:50:003
3882770991613,2cyclictest4099776-21kworker/u8:3+events_unbound22:45:113
3882770991613,2cyclictest4053811-21kworker/u8:1+events_unbound22:20:203
3882770991613,2cyclictest3960643-21kworker/u8:1+flush-259:021:00:203
3882770991613,2cyclictest3960643-21kworker/u8:1+events_unbound20:40:193
3882770991612,3cyclictest4115652-21kworker/u8:0+events_unbound23:10:243
3882770991612,3cyclictest3913920-21kworker/u8:3+events_unbound20:35:003
3882765991614,1cyclictest4163050-21kworker/u8:0+events_unbound23:50:192
3882765991612,3cyclictest4137081-21kworker/u8:3+events_unbound23:15:202
3882765991612,3cyclictest4038579-21kworker/u8:3+events_unbound21:43:402
3882762991613,2cyclictest3963236-21kthreadcore20:25:191
3882762991612,2cyclictest0-21swapper/123:15:001
3882760991613,2cyclictest60350irq/132-enp0s29f1:rx-020:35:110
3882760991612,3cyclictest4173132-21kworker/u8:2+flush-259:000:15:210
3882760991612,3cyclictest4099776-21kworker/u8:3+flush-259:022:45:230
3882760991610,4cyclictest3954610-21ntpq20:15:190
3882770991512,2cyclictest3960643-21kworker/u8:1+events_unbound21:15:243
3882765991513,1cyclictest3909820-21kthreadcore19:35:172
3882765991512,2cyclictest4137081-21kworker/u8:3+flush-259:023:45:162
3882765991512,2cyclictest4053811-21kworker/u8:1+flush-259:022:20:002
3882765991512,2cyclictest4048533-21kworker/u8:2+flush-259:023:05:202
3882765991511,3cyclictest3903384-21kworker/u8:0+flush-259:020:12:172
3882762991513,1cyclictest0-21swapper/119:45:021
3882762991512,2cyclictest4163050-21kworker/u8:0+flush-259:000:02:541
3882762991512,2cyclictest3992269-21irqcore20:55:171
3882762991512,2cyclictest3992269-21irqcore20:55:171
3882762991512,2cyclictest2971-21cron22:30:001
3882762991511,3cyclictest4048533-21kworker/u8:2+flush-259:022:00:251
388276099159,4cyclictest3933906-21ntpq19:55:200
3882760991513,1cyclictest60350irq/132-enp0s29f1:rx-023:20:240
3882760991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:110
3882760991513,1cyclictest60350irq/132-enp0s29f1:rx-020:45:180
3882760991512,2cyclictest60350irq/132-enp0s29f1:rx-022:05:210
3882760991511,3cyclictest4048533-21kworker/u8:2+events_unbound22:55:160
3882760991511,3cyclictest3960643-21kworker/u8:1+events_unbound21:10:190
3882770991411,2cyclictest4173132-21kworker/u8:2+events_unbound00:07:493
3882770991410,3cyclictest4183053-21kworker/u8:1+events_unbound00:00:183
388276599148,4cyclictest335557-21systemd-journal00:39:592
3882765991411,2cyclictest4011282-21ntp_kernel_pll_21:10:222
3882765991410,3cyclictest4069066-21kworker/u8:0+events_unbound22:09:402
3882762991410,3cyclictest4053811-21kworker/u8:1+flush-259:022:35:251
3882762991410,3cyclictest3924240-21kworker/u8:1+flush-259:019:50:311
3882760991413,1cyclictest60350irq/132-enp0s29f1:rx-000:39:580
3882760991412,1cyclictest60350irq/132-enp0s29f1:rx-020:20:270
3882760991412,1cyclictest60350irq/132-enp0s29f1:rx-019:35:200
3882760991411,2cyclictest60350irq/132-enp0s29f1:rx-023:02:430
3882760991411,2cyclictest60350irq/132-enp0s29f1:rx-019:50:140
3882760991411,2cyclictest60350irq/132-enp0s29f1:rx-019:30:120
3882760991411,2cyclictest3872134-21kworker/u8:0+events_unbound19:15:220
3882760991411,2cyclictest3872134-21kworker/u8:0+events_unbound19:10:150
388277099139,3cyclictest4038578-21kworker/u8:1+events_unbound21:41:413
388277099139,3cyclictest3981563-21apt-get20:45:103
388277099139,3cyclictest3965855-21kworker/u8:0+events_unbound21:45:263
3882770991311,1cyclictest827180-21kworker/u8:1+events_unbound19:20:283
3882770991310,2cyclictest4069066-21kworker/u8:0+events_unbound22:10:253
3882770991310,2cyclictest3929144-21unixbench_multi19:50:253
3882770991310,2cyclictest3903384-21kworker/u8:0+events_unbound20:10:143
3882762991310,2cyclictest3913920-21kworker/u8:3+flush-259:019:55:241
3882760991311,1cyclictest60350irq/132-enp0s29f1:rx-021:20:230
3882760991310,2cyclictest60350irq/132-enp0s29f1:rx-022:50:240
3882760991310,2cyclictest60350irq/132-enp0s29f1:rx-021:25:140
3882760991310,2cyclictest4147407-21kworker/u8:1+flush-259:023:35:150
388277099129,2cyclictest4063725-21kworker/u8:3+events_unbound22:05:183
388276599128,3cyclictest4032736-21kworker/u8:2+events_unbound21:35:042
388276299129,2cyclictest3949844-21unixbench_multi20:10:241
388276299126,4cyclictest4115772-21cat22:55:011
388276299124,6cyclictest335557-21systemd-journal22:44:591
388277099118,2cyclictest3960643-21kworker/u8:1+events_unbound21:05:243
388277099118,2cyclictest10442-21cpuspeed_turbos00:20:153
388276599119,1cyclictest0-21swapper/220:20:242
388276599118,2cyclictest3965855-21kworker/u8:0+events_unbound21:55:232
388276599118,2cyclictest3893267-21kworker/u8:2+flush-259:019:26:112
388276099119,2cyclictest60350irq/132-enp0s29f1:rx-000:03:590
388276299105,3cyclictest15304-21sed00:25:001
388276299100,8cyclictest4012160-21kworker/u8:4+flush-259:021:30:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional