You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 16:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Apr 19, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3533616994740,5cyclictest3709881-21latency_hist10:00:000
3533627994638,5cyclictest3787908-21latency_hist11:15:003
3533616994035,3cyclictest3736312-21apt-get10:25:130
3533627993933,4cyclictest3538532-21kworker/u8:2+flush-259:007:35:013
3533627993832,4cyclictest3570251-21kworker/u8:0+flush-259:008:10:233
3533623993731,4cyclictest3580297-21latency_hist07:55:012
3533627993629,5cyclictest3657955-21latency_hist09:10:013
3533623993630,5cyclictest3745407-21ntpq10:30:222
3533623993427,5cyclictest3574335-21ntpq07:45:232
3533620993227,4cyclictest252-21systemd-journal07:45:001
3533616993228,3cyclictest3544115-21apt-get07:20:110
3533623993124,5cyclictest252-21systemd-journal12:10:002
3533620993125,4cyclictest3580535-21/usr/sbin/munin07:55:191
3533616993024,4cyclictest252-21systemd-journal08:45:540
3533623992824,3cyclictest3777484-21apt-get11:05:112
3533616992823,4cyclictest3833448-21ntpq11:55:210
3533616992823,4cyclictest3663383-21apt-get09:15:110
3533627992723,3cyclictest3720607-21kworker/u8:0+flush-259:010:20:253
3533627992723,3cyclictest3616732-21kworker/u8:2+flush-259:010:25:173
3533623992721,4cyclictest3730883-21latency_hist10:20:012
353361699270,25cyclictest1-21systemd11:31:530
3533620992622,3cyclictest3788160-21/usr/sbin/munin11:15:241
3533616992620,4cyclictest3860796-21apt-get12:25:120
3533616992620,4cyclictest3860796-21apt-get12:25:120
3533616992620,4cyclictest3627296-21/usr/sbin/munin08:40:120
3533616992613,11cyclictest500-21systemd-logind09:45:000
3533627992521,3cyclictest3829366-21kworker/u8:4+flush-259:011:55:183
3533627992518,5cyclictest3610729-21ntpq08:20:223
3533623992521,3cyclictest3850449-21/usr/sbin/munin12:15:212
3533620992520,4cyclictest3709061-21ntpq09:55:221
353361699251,2cyclictest3687965-21ntpq09:35:210
3533627992419,4cyclictest3601558-21apt-get08:15:103
3533627992417,5cyclictest3828284-21ntpq11:50:223
3533623992419,4cyclictest3751698-21/usr/sbin/munin10:40:142
3533620992419,3cyclictest509-21rs:main1
3533620992419,3cyclictest3740426-21kworker/u8:3+flush-259:010:35:171
3533620992419,3cyclictest3668522-21apt-get09:20:101
3533620992419,3cyclictest3596129-21latency_hist08:10:021
3533627992316,5cyclictest3589703-21ntpq08:00:213
3533627992315,5cyclictest3812772-21ntpq11:35:223
3533623992319,3cyclictest3730910-21kworker/u8:1+flush-259:011:15:052
3533623992318,3cyclictest3740426-21kworker/u8:3+flush-259:010:30:002
3533623992317,5cyclictest3875101-21ntpq12:35:232
3533620992318,4cyclictest3616732-21kworker/u8:2+flush-259:009:10:001
3533620992318,3cyclictest3767205-21apt-get10:55:101
3533616992319,3cyclictest3782919-21gpgv11:10:020
3533616992317,4cyclictest3756594-21latency_hist10:45:010
3533623992216,4cyclictest252-21systemd-journal11:40:002
3533620992216,4cyclictest3544082-21kworker/u8:1+flush-259:007:25:151
3533616992218,3cyclictest3632243-21kworker/u8:3+flush-259:009:05:000
3533616992216,4cyclictest252-21systemd-journal09:27:540
3533616992216,4cyclictest252-21systemd-journal09:27:530
3533616992211,10cyclictest3813922-21apt-get11:40:110
3533627992117,3cyclictest3632243-21kworker/u8:3+flush-259:009:40:003
3533627992116,4cyclictest3839527-21kworker/u8:0+flush-259:012:15:203
3533627992116,4cyclictest3538532-21kworker/u8:2+events_unbound07:50:143
3533627992116,3cyclictest3538532-21kworker/u8:2+events_unbound08:05:193
3533627992115,4cyclictest3715239-21gpgv10:05:003
3533627992112,6cyclictest3662295-21ntpq09:10:203
3533623992117,3cyclictest3616732-21kworker/u8:2+flush-259:009:25:172
3533623992117,3cyclictest3616732-21kworker/u8:2+flush-259:009:25:162
353361699212,16cyclictest3745398-21ntpq10:30:220
3533616992117,2cyclictest3616732-21kworker/u8:2+flush-259:009:05:160
3533616992112,8cyclictest60350irq/132-enp0s29f1:rx-008:35:110
353361699211,18cyclictest3611631-21wc08:25:000
353361699210,19cyclictest3850202-21turbostat12:14:590
3533627992016,3cyclictest3570251-21kworker/u8:0+flush-259:008:25:243
3533627992016,2cyclictest0-21swapper/310:40:013
3533627992015,4cyclictest3730910-21kworker/u8:1+flush-259:011:45:263
3533627992015,3cyclictest3570259-21/usr/sbin/munin07:45:153
3533623992018,1cyclictest0-21swapper/210:05:002
3533623992014,4cyclictest252-21systemd-journal08:19:592
3533620992017,2cyclictest3818774-21cron11:45:001
3533620992016,3cyclictest3829366-21kworker/u8:4+flush-259:012:00:001
353361699207,12cyclictest513-21gdbus08:30:000
3533616992017,1cyclictest0-21swapper/007:45:000
353361699201,4cyclictest513-21gdbus08:35:010
353361699200,18cyclictest1-21systemd07:15:520
3533620991916,2cyclictest3616732-21kworker/u8:2+flush-259:009:50:531
3533620991915,3cyclictest3740426-21kworker/u8:3+flush-259:011:00:261
3533620991915,3cyclictest3720607-21kworker/u8:0+flush-259:010:20:231
3533620991915,3cyclictest3616732-21kworker/u8:2+flush-259:010:30:241
3533620991914,4cyclictest3859647-21ntpq12:20:211
3533620991914,4cyclictest3859647-21ntpq12:20:201
3533620991913,4cyclictest3797284-21ntpq11:20:221
3533616991914,3cyclictest3834592-21apt-get12:00:120
353361699191,3cyclictest3662192-21ntpq09:10:190
353361699191,2cyclictest3646729-21ntpq08:55:210
353361699190,3cyclictest3683731-21idleruntime-cro09:35:000
353361699190,2cyclictest3699736-21if_enp0s29f109:50:140
3533627991816,1cyclictest3616732-21kworker/u8:2+flush-259:009:25:153
3533627991816,1cyclictest3616732-21kworker/u8:2+flush-259:009:25:143
3533627991814,3cyclictest507-21in:imuxsock12:10:013
3533627991812,4cyclictest252-21systemd-journal11:00:003
3533623991815,2cyclictest509-21rs:main2
3533623991815,2cyclictest3740426-21kworker/u8:3+events_unbound12:10:222
3533623991814,3cyclictest3663358-21/usr/sbin/munin09:15:202
3533623991814,3cyclictest3632243-21kworker/u8:3+flush-259:009:00:002
3533623991812,4cyclictest3663120-21latency_hist09:15:002
3533620991815,2cyclictest3632243-21kworker/u8:3+flush-259:008:55:141
3533620991814,3cyclictest3740426-21kworker/u8:3+flush-259:012:10:151
3533620991813,4cyclictest3544089-21/usr/sbin/munin07:20:131
353362099180,17cyclictest252-21systemd-journal11:34:591
353361699186,11cyclictest3871050-21acpi12:35:130
353361699185,10cyclictest0-21swapper/008:05:010
353361699183,13cyclictest3867743-21cut12:30:180
353361699181,4cyclictest3761730-21latency_hist10:50:010
353361699181,3cyclictest252-21systemd-journal07:35:000
353361699181,2cyclictest3823057-21ntpq11:45:220
353361699181,2cyclictest3802449-21ntpq11:25:220
353361699181,2cyclictest3797292-21ntpq11:20:220
353361699181,2cyclictest3750564-21ntpq10:35:210
353361699181,2cyclictest3727024-21ps10:15:180
353361699181,2cyclictest3595267-21ntpq08:05:230
353361699181,2cyclictest3579437-21ntpq07:50:220
353361699181,2cyclictest3553246-21ntpq07:25:180
353361699180,6cyclictest3672507-21ntpq09:20:190
353361699180,5cyclictest3724834-21ntpq10:10:200
353361699180,5cyclictest3710299-21df_inode10:00:110
353361699180,4cyclictest3792624-21awk11:15:240
353361699180,4cyclictest3782343-21cron11:10:010
353361699180,3cyclictest3855323-21chrt12:20:000
353361699180,3cyclictest3761883-21latency_hist10:50:010
353361699180,3cyclictest3538702-21basename07:15:000
353361699180,1cyclictest0-21swapper/010:20:140
3533627991713,3cyclictest252-21systemd-journal07:40:003
3533627991712,3cyclictest3782532-21base6411:10:013
3533623991713,3cyclictest3740426-21kworker/u8:3+flush-259:011:49:592
3533620991714,2cyclictest3798884-21irqcore11:25:171
3533620991714,2cyclictest3632896-21irqcore08:45:191
3533620991713,3cyclictest3580321-21kworker/u8:3+flush-259:008:11:581
3533620991713,3cyclictest3528488-21kworker/u8:1+flush-259:007:10:251
3533620991712,4cyclictest3559584-21apt-get07:35:111
3533620991712,4cyclictest252-21systemd-journal11:10:531
353361699176,10cyclictest3603846-21tr08:15:180
353361699172,10cyclictest3839776-21apt-get12:05:120
353361699171,4cyclictest3698613-21/usr/sbin/munin09:45:270
3533616991712,4cyclictest3632243-21kworker/u8:3+events_unbound10:05:180
353361699171,1cyclictest3859655-21ntpq12:20:220
353361699171,1cyclictest3859655-21ntpq12:20:220
353361699171,1cyclictest3771176-21ntpq10:55:200
353361699170,5cyclictest3772487-21/usr/sbin/munin11:00:130
353361699170,5cyclictest3563616-21ntpq07:35:220
353361699170,4cyclictest3574188-21grep07:45:200
353361699170,3cyclictest3637995-21chrt08:50:150
353361699170,3cyclictest3580781-21fschecks_time07:55:130
353361699170,3cyclictest3533537-21latency_hist07:10:000
353361699170,2cyclictest3828319-21ntpq11:50:230
353361699170,1cyclictest3600427-21ntpq08:10:210
3533627991613,2cyclictest2971-21cron08:40:003
3533627991613,2cyclictest0-21swapper/311:30:003
3533627991612,3cyclictest3740426-21kworker/u8:3+flush-259:010:45:193
3533627991612,3cyclictest3616732-21kworker/u8:2+flush-259:010:10:533
3533627991612,2cyclictest3844727-21apt-key12:10:003
3533627991611,4cyclictest3632243-21kworker/u8:3+flush-259:009:15:133
3533627991611,4cyclictest3616732-21kworker/u8:2+flush-259:008:30:263
3533627991611,3cyclictest3720607-21kworker/u8:0+flush-259:010:15:243
3533627991610,4cyclictest3616732-21kworker/u8:2+events_unbound09:45:013
3533623991614,1cyclictest0-21swapper/208:05:002
3533623991613,2cyclictest3657983-21kworker/u8:1+flush-259:009:20:142
3533623991613,2cyclictest3616732-21kworker/u8:2+events_unbound09:50:172
3533623991613,2cyclictest3570251-21kworker/u8:0+flush-259:008:50:002
3533623991613,2cyclictest3570251-21kworker/u8:0+flush-259:008:35:252
353361699160,2cyclictest3808726-21find11:35:120
353362799158,4cyclictest252-21systemd-journal12:25:013
353362799158,4cyclictest252-21systemd-journal12:25:003
3533627991511,3cyclictest3616732-21kworker/u8:2+flush-259:008:55:213
3533627991511,2cyclictest3772726-21irqcore11:00:163
3533627991510,3cyclictest1-21systemd11:20:003
353362399154,3cyclictest3720587-21munin-run10:10:012
3533623991512,2cyclictest3538532-21kworker/u8:2+events_unbound07:15:172
3533623991510,4cyclictest3865688-21latency_hist12:30:012
3533623991510,4cyclictest3865688-21latency_hist12:30:012
3533620991513,1cyclictest3632243-21kworker/u8:3+flush-259:009:30:141
3533620991512,2cyclictest3668247-21cron09:20:001
3533620991510,3cyclictest252-21systemd-journal12:05:001
353362799149,3cyclictest3740426-21kworker/u8:3+flush-259:011:45:003
353362799149,3cyclictest3740426-21kworker/u8:3+flush-259:011:35:003
3533627991410,3cyclictest3699822-21tr09:50:163
3533627991410,3cyclictest3672968-21unixbench_singl09:20:243
3533627991410,3cyclictest3616732-21kworker/u8:2+flush-259:009:55:193
353362399149,4cyclictest3724855-21ntpq10:10:202
353362399149,4cyclictest3570251-21kworker/u8:0+flush-259:008:20:252
3533623991411,2cyclictest3804023-21irqcore11:30:182
3533623991411,2cyclictest3777204-21cron11:05:012
3533623991411,2cyclictest3564465-21munin-plugin-st07:40:002
3533623991411,2cyclictest0-21swapper/209:34:592
3533623991410,3cyclictest3761730-21latency_hist10:50:012
3533623991410,3cyclictest3580321-21kworker/u8:3+flush-259:008:29:592
353362099149,4cyclictest3585721-21apt-get08:00:111
353362099148,4cyclictest3866123-21diskstats12:30:151
3533620991412,1cyclictest3861220-21sed12:25:171
3533620991412,1cyclictest3861220-21sed12:25:171
3533620991411,2cyclictest500-21systemd-logind09:05:011
3533620991410,3cyclictest252-21systemd-journal10:15:531
353362799139,3cyclictest0-21swapper/307:45:003
353362799139,2cyclictest3570251-21kworker/u8:0+flush-259:007:55:253
353362799139,2cyclictest0-21swapper/307:10:163
353362799131,10cyclictest3699149-21awk09:49:593
353362399137,4cyclictest3824247-21apt-get11:50:132
353362399137,4cyclictest3817915-21ntpq11:40:212
3533623991311,1cyclictest0-21swapper/209:09:592
3533623991310,2cyclictest3834790-21fschecks_count12:00:142
3533623991310,2cyclictest3616732-21kworker/u8:2+flush-259:009:00:172
353362099139,3cyclictest3715683-21/usr/sbin/munin10:05:151
353362099138,4cyclictest3615832-21ntpq08:25:211
353362099138,3cyclictest252-21systemd-journal09:44:591
3533620991310,2cyclictest3829123-21systemd-journal11:55:001
3533620991310,2cyclictest3617180-21fschecks_count08:30:171
353362799129,2cyclictest3762224-21fschecks_time10:50:153
353362799129,1cyclictest0-21swapper/310:10:013
353362799129,1cyclictest0-21swapper/307:20:013
353362799127,3cyclictest3538532-21kworker/u8:2+flush-259:007:25:153
353362399129,2cyclictest3740426-21kworker/u8:3+flush-259:011:10:252
353362399129,2cyclictest3595125-21needreboot08:05:222
353362399129,2cyclictest3554159-21cron07:30:012
353362399128,3cyclictest3730910-21kworker/u8:1+flush-259:011:25:252
353362399121,4cyclictest832-21runrttasks09:45:222
3533623991210,1cyclictest0-21swapper/207:45:022
353362099129,2cyclictest3823114-21ntp_kernel_pll_11:45:221
353362099129,2cyclictest3740175-21needreboot10:25:211
353362099129,2cyclictest3607117-21irqcore08:20:171
353362099129,2cyclictest3574723-21unixbench_singl07:45:251
353362099129,2cyclictest3538532-21kworker/u8:2+flush-259:007:54:341
353362099129,2cyclictest0-21swapper/112:10:001
353362099129,2cyclictest0-21swapper/110:45:211
353362799118,2cyclictest3632552-21cpuspeed_turbos08:45:153
353362799117,3cyclictest3839527-21kworker/u8:0+events_unbound12:35:163
353362799117,3cyclictest3570251-21kworker/u8:0+flush-259:008:40:173
353362799116,4cyclictest3730910-21kworker/u8:1+flush-259:011:20:173
353362399119,1cyclictest0-21swapper/210:55:122
353362399118,2cyclictest3580626-21cpuspeed_turbos07:55:132
353362399117,3cyclictest3689099-21apt-get09:40:112
353362399110,5cyclictest3709891-21munin-run10:00:002
353362099118,2cyclictest3622605-21irqcore08:35:161
353362099118,2cyclictest3541954-21kthreadcore07:15:171
353362099118,2cyclictest0-21swapper/111:05:131
353362099117,3cyclictest3678558-21kworker/u8:0+flush-259:009:45:491
353362099117,3cyclictest3616732-21kworker/u8:2+flush-259:008:53:571
353362799108,1cyclictest0-21swapper/307:25:003
353362799107,2cyclictest0-21swapper/312:35:003
353362099108,1cyclictest0-21swapper/108:40:121
353362099106,3cyclictest311ktimers/112:15:141
353362099106,2cyclictest3812733-21ntpq11:35:221
353362099106,2cyclictest0-21swapper/110:15:001
353362099105,4cyclictest3761976-21/usr/sbin/munin10:50:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional