You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 03:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Mar 17, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3209413994640,4cyclictest3230125-21gpgv19:29:593
3209403994439,4cyclictest3458724-21kworker/u8:2+flush-259:000:00:160
3209403994137,3cyclictest3261284-21kworker/u8:3+flush-259:022:20:000
3209403994134,5cyclictest3261684-21diskstats20:00:140
3209403994032,5cyclictest3396526-21latency_hist22:10:010
3209403993935,3cyclictest3489553-21kworker/u8:3+flush-259:000:15:180
3209403993935,3cyclictest3469234-21kworker/u8:1+events_unbound23:20:170
3209403993935,3cyclictest3261284-21kworker/u8:3+events_unbound20:55:140
3209403993734,2cyclictest3458724-21kworker/u8:2+events_unbound23:40:250
3209403993734,2cyclictest3261284-21kworker/u8:3+events_unbound22:25:190
3209403993732,3cyclictest3489553-21kworker/u8:3+flush-259:023:55:150
3209413993630,4cyclictest3261284-21kworker/u8:3+flush-259:020:10:173
3209405993628,5cyclictest3525783-21gpgv00:15:001
3209403993632,3cyclictest3469234-21kworker/u8:1+flush-259:023:35:210
3209403993631,3cyclictest3150895-21kworker/u8:1+events_unbound20:15:190
3209413993529,4cyclictest3150895-21kworker/u8:1+flush-259:019:55:173
3209403993532,2cyclictest3338694-21kworker/u8:2+events_unbound21:45:220
3209403993532,2cyclictest3261284-21kworker/u8:3+events_unbound21:10:210
3209403993431,2cyclictest3396564-21kworker/u8:0+events_unbound22:55:190
3209403993431,2cyclictest3261284-21kworker/u8:3+flush-259:021:15:230
3209403993329,3cyclictest3150895-21kworker/u8:1+flush-259:019:35:270
3209403993328,3cyclictest3150895-21kworker/u8:1+events_unbound20:20:180
3209403993325,6cyclictest3478426-21ntpq23:25:210
3209403993228,3cyclictest3261284-21kworker/u8:3+flush-259:021:40:200
3209403993228,3cyclictest3214603-21kworker/u8:2+flush-259:019:40:250
3209403993128,2cyclictest3338694-21kworker/u8:2+events_unbound21:50:180
3209403993127,3cyclictest3261284-21kworker/u8:3+flush-259:022:30:250
3209413993027,2cyclictest3417587-21kworker/u8:2+flush-259:022:55:003
3209408993026,3cyclictest3386296-21kworker/u8:1+flush-259:022:10:122
3209408993026,3cyclictest3386296-21kworker/u8:1+flush-259:022:10:122
3209403993027,2cyclictest3261284-21kworker/u8:3+events_unbound20:40:210
3209413992925,3cyclictest3261284-21kworker/u8:3+flush-259:023:10:213
3209413992924,4cyclictest3396564-21kworker/u8:0+flush-259:022:55:173
3209403992724,2cyclictest3261284-21kworker/u8:3+flush-259:021:05:200
3209403992723,3cyclictest3396564-21kworker/u8:0+flush-259:022:35:160
3209403992723,3cyclictest3261284-21kworker/u8:3+events_unbound23:05:200
3209403992723,3cyclictest3261284-21kworker/u8:3+events_unbound22:20:150
3209413992623,2cyclictest3338694-21kworker/u8:2+flush-259:021:15:013
3209408992620,4cyclictest3313680-21latency_hist20:50:012
3209405992622,3cyclictest3214603-21kworker/u8:2+flush-259:020:00:261
3209403992622,3cyclictest3458724-21kworker/u8:2+events_unbound23:45:180
3209403992622,3cyclictest3199119-21kworker/u8:0+flush-259:019:20:110
3209408992520,4cyclictest252-21systemd-journal21:25:002
3209403992522,2cyclictest3458724-21kworker/u8:2+flush-259:023:30:200
3209403992522,2cyclictest3214603-21kworker/u8:2+flush-259:019:30:260
3209403992522,2cyclictest3214603-21kworker/u8:2+events_unbound19:50:170
3209403992522,2cyclictest3214603-21kworker/u8:2+events_unbound19:50:170
3209403992522,2cyclictest3150895-21kworker/u8:1+flush-259:019:10:200
3209403992521,3cyclictest3261284-21kworker/u8:3+events_unbound21:25:160
3209403992519,4cyclictest3214795-21/usr/sbin/munin19:15:230
3209408992421,2cyclictest3458724-21kworker/u8:2+flush-259:000:05:162
3209405992420,3cyclictest3199119-21kworker/u8:0+flush-259:019:26:421
3209405992418,4cyclictest3150895-21kworker/u8:1+flush-259:019:10:151
3209403992420,3cyclictest3458724-21kworker/u8:2+flush-259:023:50:240
3209403992420,3cyclictest3338694-21kworker/u8:2+flush-259:022:00:140
3209403992420,3cyclictest3150895-21kworker/u8:1+flush-259:020:05:180
3209408992319,3cyclictest3297962-21/usr/sbin/munin20:35:212
3209408992318,3cyclictest3277125-21latency_hist20:15:012
3209405992317,4cyclictest3308578-21cat20:45:011
3209403992321,2cyclictest3150895-21kworker/u8:1+events_unbound19:25:140
3209403992320,2cyclictest3520674-21kworker/u8:0+flush-259:000:35:120
3209403992320,2cyclictest3520674-21kworker/u8:0+events_unbound00:25:220
3209403992320,2cyclictest3520674-21kworker/u8:0+events_unbound00:25:220
3209403992320,2cyclictest3292836-21apt-get20:30:110
3209405992217,3cyclictest3266660-21apt-get20:05:101
3209405992217,3cyclictest3235835-21apt-get19:35:121
3209405992216,4cyclictest3520674-21kworker/u8:0+flush-259:000:20:171
3209403992219,2cyclictest3520674-21kworker/u8:0+events_unbound00:13:320
3209403992219,2cyclictest3308703-21kworker/u8:4+events_unbound20:45:220
3209403992218,3cyclictest3150895-21kworker/u8:1+events_unbound20:28:550
3209403992217,4cyclictest3433276-21apt-get22:45:110
3209413992118,2cyclictest3329403-21apt-get21:05:123
3209413992118,2cyclictest3214603-21kworker/u8:2+flush-259:019:55:003
3209413992118,2cyclictest3214603-21kworker/u8:2+flush-259:019:54:593
3209413992117,3cyclictest3199119-21kworker/u8:0+flush-259:019:14:593
3209413992115,5cyclictest3551788-21apt-get00:40:003
3209408992118,2cyclictest3338694-21kworker/u8:2+flush-259:021:45:252
3209408992115,4cyclictest3531407-21/usr/sbin/munin00:20:192
3209405992117,3cyclictest3412654-21apt-get22:25:091
3209405992115,4cyclictest3397043-21cat22:10:011
3209405992115,4cyclictest3397043-21cat22:10:011
3209405992114,5cyclictest3473246-21ntpq23:20:201
3209405992114,5cyclictest3380808-21gpgv21:54:591
3209403992118,2cyclictest3150895-21kworker/u8:1+events_unbound19:59:590
3209408992014,5cyclictest3359116-21ntpq21:30:192
3209405992017,2cyclictest3150895-21kworker/u8:1+events_unbound20:20:161
3209405992016,2cyclictest0-21swapper/119:55:001
3209405992016,2cyclictest0-21swapper/119:54:591
3209405992015,4cyclictest3261284-21kworker/u8:3+flush-259:022:15:281
3209405992014,4cyclictest3261284-21kworker/u8:3+flush-259:020:55:231
3209403992016,3cyclictest3318886-21kworker/u8:0+flush-259:021:00:110
3209403992013,5cyclictest3401187-21ntpq22:10:200
3209403992013,5cyclictest3401187-21ntpq22:10:190
3209408991915,3cyclictest3150895-21kworker/u8:1+flush-259:020:30:012
3209405991916,2cyclictest3344388-21kworker/u8:1+flush-259:021:20:211
3209405991915,3cyclictest3338694-21kworker/u8:2+events_unbound21:20:001
3209405991915,3cyclictest3297675-21kworker/u8:0+flush-259:020:40:011
3209405991915,3cyclictest3214820-21apt-get19:15:121
3209405991913,4cyclictest252-21systemd-journal19:35:001
3209403991916,2cyclictest3261284-21kworker/u8:3+events_unbound21:55:170
3209403991915,3cyclictest3261284-21kworker/u8:3+events_unbound22:44:320
3209403991915,3cyclictest3150895-21kworker/u8:1+events_unbound19:45:200
3209403991915,3cyclictest252-21systemd-journal21:40:000
3209403991914,3cyclictest3438393-21apt-get22:50:110
3209408991816,1cyclictest3396552-21systemd-run22:09:592
3209405991814,3cyclictest3551988-21apt-key00:40:001
3209405991814,3cyclictest3489553-21kworker/u8:3+events_unbound00:15:251
3209405991814,3cyclictest3458724-21kworker/u8:2+events_unbound23:50:211
3209405991814,3cyclictest3261284-21kworker/u8:3+flush-259:020:10:211
3209405991814,2cyclictest0-21swapper/122:20:151
3209405991813,3cyclictest3338694-21kworker/u8:2+flush-259:021:45:191
3209405991813,3cyclictest252-21systemd-journal22:09:311
3209405991812,4cyclictest3150895-21kworker/u8:1+flush-259:020:25:151
3209405991811,5cyclictest3308054-21latency_hist20:45:001
3209403991815,2cyclictest3261284-21kworker/u8:3+events_unbound20:38:460
3209408991712,4cyclictest3510445-21apt-get00:00:122
320940599179,5cyclictest3509257-21ntpq23:55:191
3209405991715,1cyclictest0-21swapper/121:10:011
3209405991713,3cyclictest3261284-21kworker/u8:3+events_unbound21:55:151
3209405991712,3cyclictest3396564-21kworker/u8:0+flush-259:022:50:211
3209405991711,4cyclictest3261259-21latency_hist20:00:001
3209403991714,2cyclictest3520674-21kworker/u8:0+flush-259:000:30:190
3209413991613,2cyclictest3261284-21kworker/u8:3+flush-259:022:20:243
3209413991613,1cyclictest0-21swapper/321:55:003
3209408991614,1cyclictest0-21swapper/220:35:012
3209408991614,1cyclictest0-21swapper/200:40:002
3209408991612,3cyclictest3261284-21kworker/u8:3+flush-259:022:45:162
3209405991613,2cyclictest3325262-21kthreadcore21:00:171
3209405991612,3cyclictest3520674-21kworker/u8:0+events_unbound00:30:141
3209405991611,3cyclictest3464123-21/usr/sbin/munin23:15:151
3209403991613,2cyclictest3344388-21kworker/u8:1+flush-259:021:20:180
3209403991613,1cyclictest60350irq/132-enp0s29f1:rx-023:17:580
320941399159,4cyclictest252-21systemd-journal23:25:003
3209413991513,1cyclictest3219713-21turbostat19:20:003
3209413991513,1cyclictest0-21swapper/322:10:003
3209413991512,2cyclictest3214603-21kworker/u8:2+events_unbound19:35:183
3209413991511,3cyclictest3458724-21kworker/u8:2+flush-259:000:05:223
3209413991511,3cyclictest3214603-21kworker/u8:2+flush-259:019:30:263
3209408991512,2cyclictest3261284-21kworker/u8:3+flush-259:021:05:262
3209408991512,2cyclictest3225537-21irqcore19:25:152
3209408991512,1cyclictest0-21swapper/223:00:002
3209408991511,3cyclictest3261284-21kworker/u8:3+flush-259:020:40:232
3209408991510,4cyclictest3505016-21latency_hist23:55:012
3209405991512,2cyclictest3199119-21kworker/u8:0+flush-259:019:20:201
3209405991512,2cyclictest0-21swapper/121:35:001
3209405991511,3cyclictest3261284-21kworker/u8:3+flush-259:023:10:241
320940599150,12cyclictest3370374-21systemd21:45:001
3209403991513,1cyclictest60350irq/132-enp0s29f1:rx-023:13:010
3209403991513,1cyclictest60350irq/132-enp0s29f1:rx-023:00:250
3209403991512,2cyclictest60350irq/132-enp0s29f1:rx-000:05:150
3209403991512,2cyclictest3261284-21kworker/u8:3+flush-259:021:30:180
3209403991511,3cyclictest3308703-21kworker/u8:4+events_unbound20:50:170
3209413991410,3cyclictest3423238-21fschecks_time22:35:143
3209413991410,2cyclictest0-21swapper/320:25:003
320940899148,4cyclictest3365443-21/usr/sbin/munin21:40:152
320940899147,5cyclictest3417587-21kworker/u8:2+flush-259:022:53:362
3209408991411,2cyclictest3214137-21unixbench_singl19:10:252
3209408991410,3cyclictest509-21rs:main2
3209408991410,3cyclictest3261284-21kworker/u8:3+flush-259:022:15:252
3209408991410,3cyclictest3261284-21kworker/u8:3+flush-259:020:55:212
3209408991410,3cyclictest3150895-21kworker/u8:1+flush-259:020:20:172
320940599142,8cyclictest3428143-21apt-get22:40:141
3209403991412,1cyclictest60350irq/132-enp0s29f1:rx-020:10:150
320941399139,3cyclictest3402102-21cron22:15:003
320941399139,3cyclictest3402102-21cron22:15:003
320941399139,3cyclictest3261284-21kworker/u8:3+flush-259:022:30:003
320941399139,3cyclictest3261284-21kworker/u8:3+flush-259:021:40:213
320941399138,4cyclictest3338694-21kworker/u8:2+flush-259:022:00:253
320941399138,3cyclictest34949542sleep323:45:133
3209413991310,2cyclictest3453098-21unixbench_multi23:00:253
3209413991310,2cyclictest3428209-21cpuspeed_turbos22:40:153
3209413991310,2cyclictest3308803-21cpuspeed_turbos20:45:133
320940899139,3cyclictest3261284-21kworker/u8:3+flush-259:022:20:262
320940899137,5cyclictest3452739-21ntpq23:00:222
320940899137,4cyclictest252-21systemd-journal19:44:592
3209408991310,2cyclictest3261284-21kworker/u8:3+flush-259:022:25:152
3209408991310,2cyclictest3218867-21ntp_offset19:15:222
3209408991310,2cyclictest3150895-21kworker/u8:1+flush-259:019:30:172
320940599139,3cyclictest3458724-21kworker/u8:2+flush-259:023:25:171
320940599139,3cyclictest3458724-21kworker/u8:2+events_unbound23:45:151
320940599139,3cyclictest3458724-21kworker/u8:2+events_unbound23:40:141
320940599139,2cyclictest3422713-21sh22:35:001
320940599138,3cyclictest3261284-21kworker/u8:3+flush-259:021:25:201
320940599137,4cyclictest3297727-21latency_hist20:35:011
320940599134,7cyclictest252-21systemd-journal23:00:001
3209405991310,2cyclictest3536969-21irqcore00:25:171
3209405991310,2cyclictest3536969-21irqcore00:25:161
3209405991310,2cyclictest3423999-21kthreadcore22:35:171
320941399129,2cyclictest3514350-21needreboot00:00:193
320941399129,2cyclictest3433343-21cpuspeed_turbos22:45:123
320941399129,2cyclictest3418218-21tr22:30:153
320941399129,2cyclictest0-21swapper/319:44:593
320941399128,3cyclictest509-21rs:main3
3209413991210,1cyclictest0-21swapper/323:55:003
3209413991210,1cyclictest0-21swapper/320:04:593
320940899129,2cyclictest3505688-21irqcore23:55:152
320940899128,3cyclictest3495152-21fschecks_count23:45:152
320940899127,4cyclictest3338694-21kworker/u8:2+events_unbound22:00:172
320940899121,9cyclictest3469007-21awk23:20:002
320940899121,5cyclictest0-21swapper/221:20:002
3209408991210,1cyclictest0-21swapper/220:04:592
320940599127,4cyclictest3246086-21apt-get19:45:121
320940599126,4cyclictest0-21swapper/100:10:001
320940599123,7cyclictest3458724-21kworker/u8:2-flush-259:000:05:001
320941399119,1cyclictest0-21swapper/321:55:153
320941399119,1cyclictest0-21swapper/320:40:203
320941399119,1cyclictest0-21swapper/320:10:003
320941399119,1cyclictest0-21swapper/300:20:223
320941399118,2cyclictest3536615-21cpuspeed_turbos00:25:133
320941399118,2cyclictest3536615-21cpuspeed_turbos00:25:133
320941399118,2cyclictest3464347-21fschecks_count23:15:143
320941399118,2cyclictest3287803-21cpuspeed_turbos20:25:123
320941399117,2cyclictest0-21swapper/321:00:183
320940899119,1cyclictest0-21swapper/222:35:242
320940899118,2cyclictest3526696-21cut00:15:182
320940899118,2cyclictest3236247-21irqcore19:35:172
320940899117,3cyclictest3469234-21kworker/u8:1+flush-259:023:35:262
320940899117,2cyclictest3375930-21cut21:50:122
320940899116,3cyclictest0-21swapper/200:15:002
320940899110,8cyclictest3318839-21latency_hist20:55:012
320940599118,2cyclictest3449102-21irqcore23:00:171
320940599118,2cyclictest3364306-21ntp_kernel_pll_21:35:201
320940599118,1cyclictest0-21swapper/122:45:151
320940599117,3cyclictest3261284-21kworker/u8:3+flush-259:023:05:161
320940599116,3cyclictest3308703-21kworker/u8:4+events_unbound20:50:241
320941399108,1cyclictest0-21swapper/300:30:213
320941399107,2cyclictest3299008-21kthreadcore20:35:183
320940899107,2cyclictest3277453-21cpuspeed_turbos20:15:142
320940899107,2cyclictest0-21swapper/223:20:182
320940899106,3cyclictest3199119-21kworker/u8:0+flush-259:019:25:002
320940899106,2cyclictest3260382-21ntpq19:55:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional