You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 22:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Jan 10, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
564453994237,4cyclictest745442-21latency_hist10:05:011
564447994034,5cyclictest252-21systemd-journal09:00:000
564461993528,4cyclictest252-21systemd-journal10:20:143
564447993428,4cyclictest509-21rs:main0
564457993226,4cyclictest604790-21ntpq07:45:222
564461993026,2cyclictest839486-21/usr/sbin/munin11:35:223
564453992924,4cyclictest599618-21ntpq07:40:201
564453992923,4cyclictest703046-21ntpq09:20:191
564447992924,4cyclictest864953-21kworker/u8:1+events_unbound12:30:180
564457992823,3cyclictest880386-21latency_hist12:15:012
564447992723,3cyclictest531897-21kworker/u8:2+events_unbound08:10:190
564461992520,3cyclictest564394-21latency_hist07:10:013
564457992520,4cyclictest900157-21ntpq12:30:242
564453992520,3cyclictest252-21systemd-journal11:58:541
564453992519,4cyclictest724495-21latency_hist09:45:001
564447992521,2cyclictest865165-21/usr/sbin/munin12:00:120
564447992520,4cyclictest901003-21kworker/u8:0+flush-259:012:40:010
564447992520,4cyclictest678149-21kworker/u8:3+flush-259:009:05:000
564453992420,3cyclictest745680-21/usr/sbin/munin10:05:141
564453992419,3cyclictest750842-21/usr/sbin/munin10:10:141
564447992421,2cyclictest771241-21kworker/u8:2+flush-259:010:35:220
564447992419,4cyclictest894971-21ntpq12:25:230
564447992419,4cyclictest894971-21ntpq12:25:220
564453992319,3cyclictest580144-21/usr/sbin/munin07:25:151
564447992319,3cyclictest812036-21kworker/u8:0+events_unbound11:10:200
564447992319,3cyclictest714234-21kworker/u8:2+events_unbound09:40:010
564447992319,2cyclictest740553-21kworker/u8:0+flush-259:010:25:190
564447992318,4cyclictest657357-21kworker/u8:0+events_unbound08:50:210
564447992317,4cyclictest771241-21kworker/u8:2+flush-259:010:45:000
564461992116,3cyclictest864928-21latency_hist12:00:003
564461992116,3cyclictest807729-21awk11:05:013
564461992115,4cyclictest252-21systemd-journal08:30:013
564461992112,8cyclictest0-21swapper/309:40:003
564457992118,2cyclictest864953-21kworker/u8:1+flush-259:012:05:002
564457992117,2cyclictest0-21swapper/210:20:002
564447992116,3cyclictest678149-21kworker/u8:3+flush-259:009:10:160
564461992016,3cyclictest564126-21kworker/u8:1+flush-259:007:25:013
564461992015,3cyclictest252-21systemd-journal09:45:173
564461992014,4cyclictest683283-21latency_hist09:05:003
564457992015,3cyclictest786618-21latency_hist10:45:012
564453992017,2cyclictest564443-21cyclictest10:40:001
564453992014,4cyclictest252-21systemd-journal09:35:561
564447992013,5cyclictest822716-21ntpq11:15:210
564447992013,5cyclictest728775-21ntpq09:45:210
564457991916,2cyclictest0-21swapper/212:10:002
564453991914,4cyclictest252-21systemd-journal09:17:561
564453991913,4cyclictest252-21systemd-journal11:55:001
564447991915,3cyclictest531897-21kworker/u8:2+events_unbound07:50:150
564447991913,4cyclictest252-21systemd-journal07:19:590
564461991814,3cyclictest507-21in:imuxsock09:10:003
564461991812,4cyclictest616378-21diskstats08:00:143
564457991815,2cyclictest859785-21sh11:54:592
564453991815,2cyclictest708992-21journal-offline09:30:001
564453991814,3cyclictest509-21rs:main1
564453991813,4cyclictest656203-21ntpq08:35:191
564453991812,4cyclictest802549-21latency_hist11:00:011
564447991815,2cyclictest724719-21kworker/u8:1+flush-259:009:50:210
564447991814,3cyclictest625758-21kworker/u8:1+events_unbound08:25:210
564447991814,2cyclictest0-21swapper/011:55:000
564447991813,3cyclictest252-21systemd-journal10:00:200
564447991811,5cyclictest662266-21latency_hist08:45:010
56446199179,7cyclictest796928-21cron10:55:003
56446199178,8cyclictest713719-21unixbench_multi09:30:253
56446199175,10cyclictest673251-21apt-get08:55:103
56446199171,3cyclictest569610-21turbostat.cron07:14:593
564461991712,4cyclictest812875-21latency_hist11:10:013
564461991711,5cyclictest0-21swapper/307:39:593
56445799177,8cyclictest657141-21basename08:40:012
564457991715,1cyclictest0-21swapper/209:50:002
564457991710,6cyclictest771630-21diskstats10:30:142
56445399173,9cyclictest688433-21basename09:10:001
564453991714,1cyclictest0-21swapper/109:55:001
564453991713,3cyclictest688693-21apt-get09:10:111
564447991714,2cyclictest60350irq/132-enp0s29f1:rx-011:59:590
564447991713,3cyclictest818043-21kworker/u8:2+events_unbound12:20:130
564447991713,3cyclictest818043-21kworker/u8:2+events_unbound12:20:130
56446199167,8cyclictest0-21swapper/312:10:193
56446199165,9cyclictest744732-21/usr/sbin/munin10:00:273
56446199165,9cyclictest0-21swapper/308:05:113
56446199164,4cyclictest801324-21cut10:55:183
56446199163,3cyclictest608865-21tr07:50:173
56446199163,1cyclictest0-21swapper/308:10:223
56446199161,3cyclictest513-21gdbus11:20:023
56446199161,2cyclictest890706-21apt12:25:013
56446199161,2cyclictest890706-21apt12:25:003
564461991610,5cyclictest776337-21latency_hist10:35:013
56446199160,4cyclictest865491-21hddtemp_smartct12:00:133
56446199160,4cyclictest818359-21mktemp11:15:013
56446199160,4cyclictest702574-21grep09:20:163
56446199160,4cyclictest672984-21awk08:55:003
56445799169,5cyclictest713356-21ntpq09:30:222
564457991614,1cyclictest0-21swapper/209:30:002
564457991613,2cyclictest609975-21ntp_offset07:50:222
564453991612,3cyclictest531897-21kworker/u8:2+flush-259:008:10:151
56444799169,5cyclictest667405-21latency_hist08:50:010
564447991614,1cyclictest60350irq/132-enp0s29f1:rx-012:06:290
564447991614,1cyclictest60350irq/132-enp0s29f1:rx-011:30:250
564447991614,1cyclictest60350irq/132-enp0s29f1:rx-011:25:130
564447991613,2cyclictest713234-21needreboot09:30:200
564447991613,2cyclictest60350irq/132-enp0s29f1:rx-012:13:520
564447991613,2cyclictest60350irq/132-enp0s29f1:rx-011:22:030
564447991613,2cyclictest60350irq/132-enp0s29f1:rx-011:05:170
564447991613,2cyclictest60350irq/132-enp0s29f1:rx-010:15:140
564447991613,2cyclictest60350irq/132-enp0s29f1:rx-010:10:190
564447991612,3cyclictest531897-21kworker/u8:2+events_unbound08:00:220
564447991612,2cyclictest576007-21kthreadcore07:20:170
564447991611,4cyclictest797631-21apt-get10:55:120
564447991611,4cyclictest678149-21kworker/u8:3+events_unbound09:05:240
564447991611,3cyclictest1-21systemd07:30:010
56446199153,3cyclictest655186-21cut08:35:173
56446199153,3cyclictest490-21dbus-daemon09:45:003
56446199152,2cyclictest845824-21tr11:40:173
56446199152,2cyclictest631678-21apt-get08:15:123
56446199151,3cyclictest901005-21systemd-run12:35:013
56446199151,2cyclictest574714-21idleruntime-cro07:20:003
56446199151,2cyclictest252-21systemd-journal10:30:003
56446199151,2cyclictest0-21swapper/307:40:243
56446199150,4cyclictest839259-21idleruntime-cro11:35:003
56446199150,4cyclictest833172-21ntpq11:25:183
56446199150,4cyclictest786627-21cut10:45:003
56446199150,4cyclictest756401-21if_enp0s30f410:15:153
56446199150,4cyclictest755768-21cron10:15:003
56446199150,4cyclictest705563-21grep09:25:183
56446199150,4cyclictest662531-21modprobe08:45:103
56446199150,4cyclictest604782-21ntpq07:45:223
56446199150,3cyclictest698693-21sh09:20:003
56446199150,2cyclictest895865-21apt12:30:003
56446199150,2cyclictest895865-21apt12:30:003
56446199150,2cyclictest252-21systemd-journal07:30:003
56445799159,4cyclictest734271-21ntpq09:50:202
56445399159,4cyclictest880655-21apt-get12:15:111
564453991513,1cyclictest660775-21kthreadcore08:40:201
564453991511,3cyclictest818043-21kworker/u8:2+flush-259:012:05:261
56445399151,13cyclictest500-21systemd-logind07:40:001
564453991511,2cyclictest0-21swapper/107:15:231
56444799158,5cyclictest589324-21ntpq07:30:220
56444799158,5cyclictest252-21systemd-journal10:24:550
564447991513,1cyclictest60350irq/132-enp0s29f1:rx-011:35:240
564447991513,1cyclictest60350irq/132-enp0s29f1:rx-011:00:150
564447991512,2cyclictest61150irq/133-enp0s29f1:tx-008:20:230
564447991512,2cyclictest60350irq/132-enp0s29f1:rx-012:15:240
564447991512,2cyclictest60350irq/132-enp0s29f1:rx-009:29:130
564447991512,2cyclictest60350irq/132-enp0s29f1:rx-009:19:150
564447991512,2cyclictest60350irq/132-enp0s29f1:rx-008:15:140
564447991512,2cyclictest60350irq/132-enp0s29f1:rx-007:56:570
564447991512,2cyclictest0-21swapper/008:10:000
564447991512,1cyclictest569610-21turbostat07:15:000
564447991511,3cyclictest812036-21kworker/u8:0+flush-259:011:40:200
564447991510,4cyclictest564126-21kworker/u8:1+flush-259:007:45:020
56446199141,2cyclictest854626-21idleruntime-cro11:49:593
56446199141,2cyclictest827904-21tail11:20:183
56446199141,2cyclictest739437-21ntpq09:55:193
56446199141,2cyclictest734355-21ntpq09:50:213
56446199141,2cyclictest0-21swapper/307:35:003
56446199140,6cyclictest657518-21cut08:40:153
56446199140,6cyclictest640784-21ntpq08:20:203
56446199140,4cyclictest905371-21/usr/sbin/munin12:35:243
56446199140,4cyclictest648393-21grep08:30:193
56446199140,3cyclictest787290-21grep10:45:163
56446199140,3cyclictest780991-21tail10:35:273
56446199140,2cyclictest690994-21ps09:10:173
56446199140,2cyclictest611471-21grep07:55:153
56446199140,2cyclictest252-21systemd-journal12:10:003
56445799148,4cyclictest844351-21latency_hist11:40:002
564457991412,1cyclictest0-21swapper/212:00:012
564457991411,2cyclictest903545-21tr12:35:192
564457991411,2cyclictest794340-21kthreadcore10:50:172
564457991411,2cyclictest765177-21ntp_kernel_pll_10:20:202
564457991411,2cyclictest723605-21ntp_kernel_pll_09:40:192
564457991411,2cyclictest252-21systemd-journal09:05:562
564457991410,3cyclictest252-21systemd-journal10:25:302
564457991410,2cyclictest0-21swapper/207:55:182
56445399149,3cyclictest0-21swapper/107:34:591
564453991411,2cyclictest852051-21kthreadcore11:45:161
56444799149,3cyclictest531897-21kworker/u8:2+flush-259:008:30:060
564447991412,1cyclictest60350irq/132-enp0s29f1:rx-009:36:460
564447991411,2cyclictest60350irq/132-enp0s29f1:rx-008:35:300
56446199130,4cyclictest880801-21df12:15:133
56446199130,2cyclictest857190-21grep11:50:173
56445799139,3cyclictest564126-21kworker/u8:1+flush-259:007:25:202
56445799138,4cyclictest818043-21kworker/u8:2+flush-259:012:15:202
564457991311,1cyclictest0-21swapper/208:50:012
564457991310,2cyclictest812036-21kworker/u8:0+flush-259:011:45:202
564457991310,2cyclictest803221-21irqcore11:00:182
564457991310,2cyclictest790776-21tail10:45:192
564457991310,2cyclictest699993-21kthreadcore09:20:142
564457991310,2cyclictest637263-21irqcore08:20:162
56445399139,3cyclictest531897-21kworker/u8:2+flush-259:008:07:271
56445399139,3cyclictest252-21systemd-journal11:33:541
56445399138,4cyclictest625758-21kworker/u8:1+flush-259:008:29:271
56445399138,3cyclictest252-21systemd-journal09:03:561
56445399134,6cyclictest513-21gdbus11:15:011
564453991310,2cyclictest789179-21kthreadcore10:45:171
564453991310,2cyclictest678122-21cron09:00:001
564447991312,1cyclictest60350irq/132-enp0s29f1:rx-011:45:060
564447991311,2cyclictest61150irq/133-enp0s29f1:tx-009:55:210
564447991311,1cyclictest60350irq/132-enp0s29f1:rx-009:22:540
564447991311,1cyclictest60350irq/132-enp0s29f1:rx-007:44:210
564447991311,1cyclictest60350irq/132-enp0s29f1:rx-007:35:190
564447991310,2cyclictest750168-21unixbench_singl10:05:240
56445799129,2cyclictest864953-21kworker/u8:1+flush-259:012:25:262
56445799129,2cyclictest864953-21kworker/u8:1+flush-259:012:25:252
56445799129,2cyclictest819770-21kthreadcore11:15:182
56445799129,2cyclictest808174-21fschecks_count11:05:142
56445799129,2cyclictest0-21swapper/209:55:202
56445799127,4cyclictest672987-21latency_hist08:55:002
56445799127,4cyclictest1-21systemd11:30:012
56445799120,5cyclictest579915-21munin-run07:24:592
56445399129,2cyclictest886014-21fschecks_count12:20:141
56445399129,2cyclictest886014-21fschecks_count12:20:131
56445399129,2cyclictest844682-21cpuspeed_turbos11:40:141
56445399129,2cyclictest668514-21irqcore08:50:151
56445399129,2cyclictest0-21swapper/108:35:011
56445399128,3cyclictest756014-21kworker/u8:3+flush-259:010:30:261
56445399128,3cyclictest636842-21apt-get08:20:121
56445399128,3cyclictest531897-21kworker/u8:2+flush-259:007:20:161
56445399128,3cyclictest0-21swapper/109:50:001
56445399128,2cyclictest864953-21kworker/u8:1+flush-259:012:35:261
564453991210,1cyclictest735430-21/usr/sbin/munin09:55:151
564453991210,1cyclictest0-21swapper/111:39:591
564453991210,1cyclictest0-21swapper/110:55:001
56444799129,2cyclictest60350irq/132-enp0s29f1:rx-010:30:460
564447991210,1cyclictest60350irq/132-enp0s29f1:rx-010:45:230
56445799119,1cyclictest0-21swapper/208:35:012
56445799118,2cyclictest673512-21fschecks_time08:55:132
56445799118,2cyclictest636571-21sh08:19:592
56445799116,1cyclictest590190-21idleruntime-cro07:35:002
56445399118,2cyclictest896196-21cpuspeed_turbos12:30:151
56445399118,2cyclictest802882-21cpuspeed_turbos11:00:131
56445399116,4cyclictest818043-21kworker/u8:2+flush-259:012:10:561
56445399114,4cyclictest766300-21/usr/sbin/munin10:25:121
56445399111,2cyclictest667405-21latency_hist08:50:011
56445799108,1cyclictest0-21swapper/208:25:132
56445799107,2cyclictest777015-21grep10:35:172
56445799107,2cyclictest0-21swapper/210:00:182
56445799106,2cyclictest0-21swapper/208:40:222
56445399105,3cyclictest569808-21sed07:15:011
5644579996,2cyclictest813207-21cpuspeed_turbos11:10:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional