You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 04:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Aug 28, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3232742995953,4cyclictest3233035-21cat19:10:001
3232742995650,4cyclictest3389499-21kworker/u8:3+events_unbound21:45:011
3232742995246,4cyclictest3514692-21base6423:40:011
3232737994339,3cyclictest3551903-21apt-get00:15:110
3232749994237,3cyclictest3514597-21base6423:40:003
3232737994137,3cyclictest3472665-21kworker/u8:3+flush-259:023:25:110
3232737994136,3cyclictest3269752-21gpgv19:45:010
3232737994136,3cyclictest3269752-21gpgv19:45:000
3232737994037,2cyclictest3362879-21kworker/u8:0+flush-259:021:55:250
3232737994033,5cyclictest3509332-21diskstats23:35:140
3232747993932,4cyclictest3379078-21latency_hist21:30:002
3232742993835,2cyclictest3520312-21kworker/u8:4+events_unbound00:05:111
3232737993633,2cyclictest3347605-21kworker/u8:1+flush-259:021:00:160
3232737993633,2cyclictest3295573-21kworker/u8:2+flush-259:021:10:110
3232749993529,4cyclictest3327240-21apt-get20:40:123
3232747993529,4cyclictest3368818-21cat21:20:012
3232742993532,2cyclictest3232531-21kworker/u8:2+flush-259:019:20:201
323274299350,34cyclictest1-21systemd23:25:011
3232737993532,2cyclictest3452210-21kworker/u8:2+flush-259:000:25:250
3232742993429,3cyclictest3557395-21base6400:20:011
3232742993429,3cyclictest3557395-21base6400:20:001
3232742993429,3cyclictest3514254-21base6423:40:001
3232749993325,5cyclictest3556979-21cp00:20:003
3232742993327,4cyclictest3248766-21/usr/sbin/munin19:25:131
3232737993228,3cyclictest3405191-21kworker/u8:3+flush-259:021:55:000
3232742993128,2cyclictest3472665-21kworker/u8:3+flush-259:023:15:261
3232742993128,2cyclictest3274897-21kworker/u8:0+flush-259:020:25:231
3232742993126,3cyclictest3368321-21https21:20:001
3232737993128,2cyclictest3321813-21kworker/u8:4+flush-259:020:55:150
3232737993127,3cyclictest3243349-21kworker/u8:5+events_unbound19:55:200
3232749993024,4cyclictest3546479-21latency_hist00:10:003
3232747993025,3cyclictest3290639-21apt-get20:05:112
3232742993026,3cyclictest3503729-21latency_hist23:30:001
3232737993027,2cyclictest3415624-21kworker/u8:4+events_unbound22:10:250
3232737993027,2cyclictest3410536-21kworker/u8:1+events_unbound22:30:110
3232737993026,3cyclictest3452210-21kworker/u8:2+flush-259:022:50:250
3232737993026,3cyclictest3321813-21kworker/u8:4+flush-259:021:20:250
3232737993025,4cyclictest335557-21systemd-journal23:20:010
3232747992924,3cyclictest3399804-21kworker/u8:2+flush-259:021:50:182
3232747992924,3cyclictest3321813-21kworker/u8:4+flush-259:021:00:152
3232742992923,4cyclictest3269497-21cat19:45:011
3232742992923,4cyclictest3269497-21cat19:45:001
3232742992823,2cyclictest3253723-21kworker/u8:4+flush-259:019:45:201
3232742992822,5cyclictest3400452-21apt-get21:50:111
3232737992824,3cyclictest3513761-21kworker/u8:0+flush-259:023:40:220
3232737992824,3cyclictest335557-21systemd-journal23:05:010
3232737992824,2cyclictest3536097-21kworker/u8:0+flush-259:000:00:150
3232737992823,3cyclictest3367405-21ntpq21:15:230
3232742992724,2cyclictest3452210-21kworker/u8:2+flush-259:023:55:251
3232742992724,2cyclictest3295573-21kworker/u8:2+flush-259:020:20:161
3232749992619,5cyclictest335557-21systemd-journal00:40:003
3232742992622,2cyclictest3472665-21kworker/u8:3+flush-259:023:15:011
3232737992622,3cyclictest3520312-21kworker/u8:4+flush-259:023:45:150
3232749992522,1cyclictest3290370-21latency_hist20:05:003
3232749992519,4cyclictest3415962-21apt-get22:05:143
3232742992521,3cyclictest3415624-21kworker/u8:4+flush-259:022:10:211
3232742992519,4cyclictest3320938-21ntpq20:30:211
3232737992522,2cyclictest3452210-21kworker/u8:2+flush-259:023:00:000
3232737992522,2cyclictest3321813-21kworker/u8:4+events_unbound20:40:150
3232737992521,3cyclictest3270015-21apt-get19:45:120
3232749992417,5cyclictest3243566-21/usr/sbin/munin19:20:143
323274799244,12cyclictest3467485-21munin-run22:55:002
3232747992418,4cyclictest335557-21systemd-journal22:59:102
3232747992418,4cyclictest335557-21systemd-journal00:32:102
3232747992415,6cyclictest3332152-21latency_hist20:45:002
3232742992420,3cyclictest3452210-21kworker/u8:2+events_unbound23:30:121
3232737992421,2cyclictest3520312-21kworker/u8:4+flush-259:000:33:120
3232737992420,3cyclictest3531148-21apt-get23:55:130
3232747992311,6cyclictest3347593-21munin-run21:00:002
3232742992320,2cyclictest3415624-21kworker/u8:4+events_unbound22:15:171
3232742992320,2cyclictest177193-21kworker/u8:3+flush-259:019:35:011
3232742992319,3cyclictest3295573-21kworker/u8:2+events_unbound21:30:241
3232737992320,2cyclictest3305539-21kworker/u8:3+events_unbound20:25:120
3232749992216,4cyclictest3342885-21diskstats20:55:143
3232742992219,2cyclictest3520312-21kworker/u8:4+flush-259:000:10:171
3232742992219,2cyclictest3410536-21kworker/u8:1+flush-259:022:30:151
3232742992219,2cyclictest3295573-21kworker/u8:2+flush-259:021:00:241
3232742992219,2cyclictest3243349-21kworker/u8:5+flush-259:019:50:171
3232742992219,1cyclictest0-21swapper/120:00:011
3232742992218,3cyclictest3232531-21kworker/u8:2+flush-259:019:17:061
3232737992219,2cyclictest3551317-21kworker/u8:1+events_unbound00:39:110
3232737992219,2cyclictest3321813-21kworker/u8:4+events_unbound21:05:190
3232737992219,2cyclictest3295573-21kworker/u8:2+events_unbound20:45:240
3232737992218,3cyclictest3253969-21apt-get19:30:110
3232749992115,4cyclictest3514076-21latency_hist23:40:003
323274999211,11cyclictest1-21systemd22:54:113
3232747992117,3cyclictest3362879-21kworker/u8:0+flush-259:021:35:172
3232747992116,3cyclictest3520312-21kworker/u8:4+events_unbound00:35:192
3232742992118,2cyclictest3551317-21kworker/u8:1+events_unbound00:15:241
3232742992118,2cyclictest3452210-21kworker/u8:2+events_unbound22:55:241
3232742992118,2cyclictest3285458-21kworker/u8:1+flush-259:020:15:201
3232742992117,3cyclictest3321813-21kworker/u8:4+events_unbound21:20:121
3232742992117,3cyclictest3321813-21kworker/u8:4+events_unbound20:58:091
323274299210,19cyclictest1-21systemd23:05:011
3232737992118,2cyclictest3362879-21kworker/u8:0+events_unbound21:39:590
3232747992016,3cyclictest3321813-21kworker/u8:4+flush-259:020:50:162
3232747992014,4cyclictest3420894-21latency_hist22:10:002
3232747992014,4cyclictest3332391-21/usr/sbin/munin20:45:142
3232747992012,6cyclictest3556789-21latency_hist00:20:002
323274799201,10cyclictest3546491-21munin-run00:10:002
3232742992017,2cyclictest3520312-21kworker/u8:4+flush-259:023:45:191
3232742992017,2cyclictest3452210-21kworker/u8:2+events_unbound00:00:231
3232742992017,2cyclictest3410536-21kworker/u8:1+events_unbound22:35:161
3232742992017,2cyclictest3405191-21kworker/u8:3+events_unbound22:00:141
3232737992017,2cyclictest3410536-21kworker/u8:1+events_unbound22:25:280
3232737992017,2cyclictest3199746-21kworker/u8:1+flush-259:019:10:150
3232737992016,3cyclictest3457390-21apt-get22:45:120
3232737992016,3cyclictest3452236-21apt-get22:40:110
3232737992016,3cyclictest3331827-21kworker/u8:0+events_unbound20:53:090
3232737992015,3cyclictest3395323-21apt-get21:45:120
3232737992014,4cyclictest3451106-21ntpq22:35:240
3232749991911,7cyclictest3572325-21unixbench_singl00:30:253
3232747991916,2cyclictest0-21swapper/220:30:012
3232747991915,3cyclictest3536097-21kworker/u8:0+flush-259:000:00:172
3232742991917,2cyclictest3420916-21kworker/u8:0+events_unbound22:50:151
3232742991916,2cyclictest3452210-21kworker/u8:2+events_unbound23:05:151
3232742991916,2cyclictest3410536-21kworker/u8:1+events_unbound22:20:151
3232742991916,2cyclictest3362879-21kworker/u8:0+events_unbound21:55:191
3232742991916,1cyclictest0-21swapper/122:49:591
3232737991916,2cyclictest3452210-21kworker/u8:2+events_unbound23:50:170
3232737991916,2cyclictest3415624-21kworker/u8:4+events_unbound22:05:180
323274999185,11cyclictest3380115-21sed21:30:173
3232749991811,4cyclictest3373879-21latency_hist21:25:013
3232747991815,2cyclictest3532168-21kthreadcore23:55:172
3232747991811,5cyclictest3525282-21latency_hist23:50:002
3232737991816,2cyclictest3520312-21kworker/u8:4+events_unbound00:25:000
3232737991816,2cyclictest3520312-21kworker/u8:4+events_unbound00:25:000
323273799180,17cyclictest1-21systemd22:25:010
323274999179,7cyclictest3564921-21tr00:25:183
323274999177,9cyclictest0-21swapper/319:40:133
323274999177,9cyclictest0-21swapper/319:40:133
323274999172,3cyclictest3311436-21apt-key20:25:003
323274999172,2cyclictest3409637-21ntpq21:55:213
323274999171,5cyclictest3462301-21latency_hist22:50:003
323274999171,5cyclictest3368300-21latency_hist21:20:003
3232749991713,3cyclictest509-21rs:main3
323274999171,2cyclictest3507991-21ntpq23:30:213
3232749991711,4cyclictest3322224-21diskstats20:35:143
323274999170,5cyclictest3285222-21idleruntime-cro20:00:013
3232747991713,2cyclictest3305539-21kworker/u8:3-ext4-rsv-conversion20:25:012
3232747991711,4cyclictest3493360-21latency_hist23:20:012
3232742991714,2cyclictest3321813-21kworker/u8:4+events_unbound20:47:191
3232742991714,2cyclictest3285458-21kworker/u8:1+flush-259:020:10:181
3232742991713,3cyclictest3321813-21kworker/u8:4+events_unbound21:10:251
3232742991713,3cyclictest3274897-21kworker/u8:0+events_unbound20:09:111
3232742991712,4cyclictest335557-21systemd-journal20:52:181
3232737991715,1cyclictest3390145-21irqcore21:40:170
3232737991714,3cyclictest3285458-21kworker/u8:1+events_unbound20:02:130
3232737991714,2cyclictest3467340-21kworker/u8:1+flush-259:023:05:200
3232737991714,2cyclictest3285458-21kworker/u8:1+events_unbound20:15:130
3232737991714,2cyclictest3243349-21kworker/u8:5+flush-259:019:50:220
3232737991714,2cyclictest3232531-21kworker/u8:2+flush-259:019:25:200
3232737991713,3cyclictest60350irq/132-enp0s29f1:rx-020:35:110
323274999166,8cyclictest0-21swapper/321:05:183
323274999165,9cyclictest3336594-21switchtime20:45:243
323274999165,9cyclictest0-21swapper/323:50:153
323274999164,11cyclictest3473131-21fschecks_time23:00:153
323274999164,11cyclictest2971-21cron20:10:003
323274999164,10cyclictest3232488-21latency_hist19:10:013
323274999162,2cyclictest3440688-21ntpq22:25:253
323274999162,2cyclictest3233209-21apt-get19:10:123
323274999161,4cyclictest3562539-21sed00:25:003
323274999161,4cyclictest3562539-21sed00:25:003
323274999161,4cyclictest3457124-21munin-plugin-st22:45:003
323274999161,4cyclictest3431519-21runrttasks22:20:133
323274999161,4cyclictest3275553-21sed19:50:143
3232749991614,1cyclictest0-21swapper/319:30:003
323274999161,3cyclictest335557-21systemd-journal22:35:003
323274999161,3cyclictest3321797-21apt20:35:003
323274999161,3cyclictest3263650-21grep19:35:253
323274999161,2cyclictest3520766-21ps23:45:163
323274999161,2cyclictest3451058-21ntpq22:35:223
3232749991612,3cyclictest3253723-21kworker/u8:4+flush-259:019:45:173
3232749991611,3cyclictest3395227-21base6421:45:013
323274999160,5cyclictest3487182-21grep23:10:193
323274999160,4cyclictest3482972-21turbostat.cron23:10:013
323274999160,3cyclictest3238808-21ps19:15:163
323274999160,14cyclictest3389457-21latency_hist21:40:013
323274799169,5cyclictest335557-21systemd-journal21:35:002
3232747991613,2cyclictest3359232-21kthreadcore21:10:162
3232747991613,2cyclictest3199746-21kworker/u8:1+flush-259:019:30:142
3232747991610,4cyclictest335557-21systemd-journal23:09:102
3232742991613,2cyclictest3374145-21apt-get21:25:121
3232737991613,2cyclictest3520312-21kworker/u8:4+flush-259:000:10:000
3232737991613,2cyclictest3362879-21kworker/u8:0+events_unbound22:00:210
323274999154,10cyclictest3414156-21tr22:00:193
323274999153,3cyclictest1-21systemd22:15:013
323274999153,1cyclictest3404910-21grep21:50:263
323274999151,3cyclictest3301279-21hddtemp_smartct20:15:143
323274999151,2cyclictest3550751-21ntpq00:10:213
323274999151,2cyclictest3536772-21ps00:00:183
323274999151,2cyclictest3502833-21ntpq23:25:213
323274999151,2cyclictest3472637-21awk23:00:003
323274999151,2cyclictest3389853-21tr21:40:123
323274999151,2cyclictest3378222-21ntpq21:25:223
323274999151,2cyclictest3358619-21grep21:10:153
323274999151,2cyclictest3351916-21ntpq21:00:213
323274999151,2cyclictest3295823-21apt-get20:10:123
323274999151,2cyclictest0-21swapper/320:50:223
323274999150,4cyclictest3496114-21grep23:20:183
323274999150,4cyclictest3488501-21tr23:15:133
323274999150,4cyclictest3430443-21smartctl22:15:223
323274999150,4cyclictest3316636-21idleruntime-cro20:30:013
323274999150,13cyclictest0-21swapper/319:30:113
323274799159,4cyclictest3268504-21ntpq19:40:232
323274799159,4cyclictest3268504-21ntpq19:40:222
3232747991512,2cyclictest3457812-21irqcore22:45:162
3232747991511,2cyclictest0-21swapper/220:40:012
3232742991512,2cyclictest3232729-21apt-key19:10:001
3232742991511,3cyclictest3243349-21kworker/u8:5+events_unbound20:04:101
3232742991510,4cyclictest335557-21systemd-journal22:44:101
3232737991513,2cyclictest3415624-21kworker/u8:4+events_unbound22:15:200
3232737991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:130
3232737991513,1cyclictest60350irq/132-enp0s29f1:rx-019:20:170
3232737991512,2cyclictest3472665-21kworker/u8:3+flush-259:023:20:160
3232737991512,2cyclictest3305539-21kworker/u8:3+flush-259:020:32:150
323274799148,4cyclictest3264126-21latency_hist19:40:002
3232747991411,2cyclictest500-21systemd-logind00:25:002
3232747991411,2cyclictest500-21systemd-logind00:25:002
3232747991411,2cyclictest3406471-21irqcore21:55:172
3232747991410,3cyclictest3452210-21kworker/u8:2+flush-259:023:30:222
3232742991411,2cyclictest3520312-21kworker/u8:4+flush-259:000:25:101
3232742991411,2cyclictest3199746-21kworker/u8:1+events_unbound19:37:091
3232737991413,1cyclictest60350irq/132-enp0s29f1:rx-021:33:590
3232737991413,1cyclictest60350irq/132-enp0s29f1:rx-019:39:280
3232737991412,1cyclictest0-21swapper/000:10:170
3232737991411,2cyclictest60350irq/132-enp0s29f1:rx-019:15:150
3232737991411,2cyclictest3452210-21kworker/u8:2+events_unbound23:30:290
3232737991411,2cyclictest3362879-21kworker/u8:0+events_unbound21:27:140
3232737991410,3cyclictest3285458-21kworker/u8:1+events_unbound20:10:170
323274799139,3cyclictest3253723-21kworker/u8:4+flush-259:019:45:242
323274799138,4cyclictest509-21rs:main2
323274799138,3cyclictest335557-21systemd-journal21:18:102
323274799137,4cyclictest3399939-21journal-offline21:47:112
3232747991311,1cyclictest0-21swapper/222:45:002
3232747991310,2cyclictest3509593-21irqcore23:35:172
323274799130,11cyclictest3248538-21awk19:25:002
323274799130,11cyclictest1-21systemd19:53:082
323274299139,3cyclictest3525972-21apt-get23:50:121
3232742991310,2cyclictest3551317-21kworker/u8:1+events_unbound00:35:271
3232742991310,2cyclictest3321813-21kworker/u8:4+events_unbound20:42:181
323273799130,12cyclictest3294682-21ntpq20:05:230
323274799129,2cyclictest0-21swapper/222:10:162
323274799127,3cyclictest3452210-21kworker/u8:2+flush-259:023:10:162
323274799121,9cyclictest3290377-21awk20:05:002
323274799121,2cyclictest3567535-21awk00:30:002
323274799121,2cyclictest3477807-21awk23:04:592
323274299129,2cyclictest3362879-21kworker/u8:0+flush-259:021:50:001
3232737991210,1cyclictest60350irq/132-enp0s29f1:rx-020:20:340
323274799119,1cyclictest3441115-21tail22:25:292
323274799119,1cyclictest0-21swapper/220:15:012
323274799118,2cyclictest3248857-21cpuspeed_turbos19:25:122
323274799118,2cyclictest0-21swapper/223:44:592
323274799116,3cyclictest3445808-21ntpq22:30:212
323274799111,2cyclictest33196751kthreadcore20:30:172
323274799110,9cyclictest3451966-21awk22:40:002
323274299119,1cyclictest3436757-21cpuspeed_turbos22:25:171
323274799106,2cyclictest3354005-21irqcore21:05:182
323274299107,2cyclictest0-21swapper/120:35:181
32327479996,2cyclictest3498144-21unixbench_singl23:20:252
32327479995,3cyclictest3347605-21kworker/u8:1+events_unbound21:40:242
32327429996,2cyclictest3410536-21kworker/u8:1+flush-259:022:07:221
32327429995,3cyclictest3387235-21cut21:35:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional