You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-11 - 18:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Mar 11, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
229115993329,3cyclictest260031-21apt-get07:40:023
229115993226,4cyclictest252-21systemd-journal11:35:003
229115993127,3cyclictest322770-21gpgv08:40:013
229115992824,3cyclictest384942-21kworker/u8:2+flush-259:011:10:003
229108992724,2cyclictest410958-21kworker/u8:0+flush-259:010:25:231
229105992720,5cyclictest436386-21ntpq10:25:200
229111992618,6cyclictest252-21systemd-journal12:00:002
229108992622,3cyclictest431414-21kworker/u8:3+events_unbound11:35:011
229108992620,4cyclictest504698-21apt-get11:35:111
229108992619,5cyclictest509883-21apt-get11:40:121
229105992622,3cyclictest223840-21kworker/u8:0+events_unbound07:40:010
229105992620,4cyclictest380187-21diskstats09:35:120
229108992521,3cyclictest363999-21kworker/u8:4+events_unbound09:50:011
229108992521,3cyclictest218462-21kworker/u8:4+flush-259:007:20:231
229115992419,4cyclictest384942-21kworker/u8:2+flush-259:010:55:203
229115992419,3cyclictest252-21systemd-journal12:30:003
229115992418,4cyclictest384955-21latency_hist09:40:013
229115992418,4cyclictest379768-21latency_hist09:35:013
229108992420,3cyclictest463089-21kworker/u8:0+flush-259:010:55:231
229105992420,3cyclictest322654-21apt-get08:40:000
229105992419,4cyclictest545929-21/usr/sbin/munin12:15:140
229115992319,3cyclictest431414-21kworker/u8:3+flush-259:010:40:263
229108992320,2cyclictest494088-21kworker/u8:5+events_unbound12:20:191
229108992319,3cyclictest384942-21kworker/u8:2+events_unbound10:20:191
229108992318,3cyclictest410958-21kworker/u8:0+flush-259:010:19:591
229105992317,4cyclictest353153-21ntpq09:05:230
22910599231,20cyclictest252-21systemd-journal11:51:180
22910599230,22cyclictest452767-21true10:44:590
229108992219,2cyclictest566273-21kworker/u8:2+flush-259:012:30:231
229108992219,2cyclictest401066-21kworker/u8:1+flush-259:010:00:231
229108992219,2cyclictest359163-21kworker/u8:1+events_unbound09:30:141
229108992218,3cyclictest525300-21kworker/u8:1+events_unbound12:00:191
229108992217,4cyclictest489143-21apt-get11:20:121
229108992216,4cyclictest252-21systemd-journal07:17:001
229105992216,4cyclictest504439-21latency_hist11:35:010
229115992117,3cyclictest535414-21kworker/u8:0+flush-259:012:25:013
229115992117,3cyclictest401066-21kworker/u8:1+events_unbound10:00:183
229115992117,3cyclictest265862-21kworker/u8:3+flush-259:007:54:593
229115992115,4cyclictest482857-21ntpq11:10:223
229115992115,4cyclictest400597-21cp09:55:013
229108992117,3cyclictest252-21systemd-journal11:08:391
229105992119,1cyclictest0-21swapper/012:25:010
229105992115,4cyclictest358319-21ntpq09:10:220
229105992114,5cyclictest431721-21latency_hist10:25:010
229115992016,3cyclictest223840-21kworker/u8:0+flush-259:007:30:163
229115992016,2cyclictest494088-21kworker/u8:5+flush-259:012:05:013
229111992012,7cyclictest252-21systemd-journal11:19:542
229108992016,3cyclictest359163-21kworker/u8:1+events_unbound09:35:131
229108992015,3cyclictest252-21systemd-journal09:13:171
229105992014,4cyclictest473402-21turbostat11:05:000
229105992012,6cyclictest535816-21diskstats12:05:120
22911199197,10cyclictest0-21swapper/211:30:112
22911199196,11cyclictest513-21gdbus11:15:012
229111991915,3cyclictest525300-21kworker/u8:1+flush-259:012:05:182
229111991913,4cyclictest252-21systemd-journal08:55:182
229108991916,2cyclictest384942-21kworker/u8:2+flush-259:010:35:161
229108991915,3cyclictest384942-21kworker/u8:2+events_unbound11:00:121
229108991915,3cyclictest333668-21apt-get08:50:101
229108991915,3cyclictest307147-21kworker/u8:0+flush-259:009:15:201
229108991913,4cyclictest252-21systemd-journal09:55:001
22910599190,17cyclictest509-21rs:main0
229115991815,2cyclictest296724-21sh08:15:003
229115991814,2cyclictest437982-21irqcore10:30:153
229115991813,3cyclictest252-21systemd-journal10:24:593
22911199187,9cyclictest0-21swapper/208:30:232
22911199181,6cyclictest270818-21cron07:49:592
229111991814,3cyclictest571888-21turbostat.cron12:39:592
229111991810,7cyclictest500-21systemd-logind12:30:002
229108991814,2cyclictest0-21swapper/109:45:011
229108991814,2cyclictest0-21swapper/108:35:011
229108991812,4cyclictest333408-21latency_hist08:50:011
229105991811,5cyclictest410233-21ntpq10:00:200
22910599180,17cyclictest1-21systemd08:26:170
22910599180,16cyclictest442479-21munin-run10:34:590
229115991714,2cyclictest229724-21irqcore07:10:163
22911199177,8cyclictest0-21swapper/209:55:012
22911199176,10cyclictest551229-21cut12:20:122
22911199176,10cyclictest281482-21cpuspeed_turbos08:00:122
22911199175,10cyclictest304786-21cut08:20:182
22911199174,2cyclictest507-21in:imuxsock07:45:012
22911199171,5cyclictest395257-21awk09:50:012
22911199171,4cyclictest260055-21gpgv07:40:012
22911199171,3cyclictest252-21systemd-journal07:54:592
22911199170,5cyclictest348853-21awk09:05:012
22911199170,4cyclictest452744-21awk10:45:002
22911199170,14cyclictest514792-21latency_hist11:45:002
229108991714,2cyclictest494087-21kworker/u8:0+flush-259:011:55:001
229108991714,2cyclictest307147-21kworker/u8:0+events_unbound09:00:171
229108991714,2cyclictest265862-21kworker/u8:3+flush-259:007:55:231
229108991713,3cyclictest525300-21kworker/u8:1+flush-259:012:10:261
229108991713,3cyclictest431414-21kworker/u8:3+flush-259:010:45:001
229108991713,3cyclictest384942-21kworker/u8:2+events_unbound10:53:371
229108991713,3cyclictest359163-21kworker/u8:1+flush-259:009:25:211
229105991713,3cyclictest252-21systemd-journal08:10:000
229105991712,3cyclictest244593-21cron07:25:000
22910599171,15cyclictest1-21systemd11:16:180
229105991711,4cyclictest252-21systemd-journal12:30:180
229105991710,5cyclictest385376-21diskstats09:40:150
229115991610,4cyclictest252-21systemd-journal09:25:183
22911199165,4cyclictest513-21gdbus12:15:012
22911199164,3cyclictest436892-21awk10:25:242
22911199163,2cyclictest281148-21awk08:00:002
22911199161,4cyclictest443083-21/usr/sbin/munin10:35:162
22911199161,3cyclictest421405-21cron10:15:002
22911199161,3cyclictest328238-21cron08:45:002
22911199160,4cyclictest254927-21dump-pmu-power07:35:002
22911199160,3cyclictest473803-21cut11:05:162
22911199160,3cyclictest458263-21cpuspeed_turbos10:50:112
22911199160,3cyclictest431722-21sh10:24:592
22911199160,3cyclictest374970-21grep09:30:132
22911199160,3cyclictest322971-21apt-key08:40:012
22911199160,3cyclictest238943-21sed07:15:242
229108991614,1cyclictest281382-21kworker/u8:4+events_unbound08:35:201
229108991613,2cyclictest265862-21kworker/u8:3+events_unbound08:10:151
229108991613,2cyclictest265862-21kworker/u8:3+events_unbound07:50:141
229108991612,3cyclictest384942-21kworker/u8:2+events_unbound10:45:141
229108991612,3cyclictest276229-21kworker/u8:2+events_unbound08:05:131
229108991611,4cyclictest363999-21kworker/u8:4+events_unbound09:23:151
229105991614,1cyclictest60350irq/132-enp0s29f1:rx-009:52:340
229105991614,1cyclictest60350irq/132-enp0s29f1:rx-008:45:160
229105991613,2cyclictest60350irq/132-enp0s29f1:rx-011:12:250
229105991613,2cyclictest60350irq/132-enp0s29f1:rx-010:59:440
229105991613,2cyclictest60350irq/132-enp0s29f1:rx-010:10:200
229105991613,2cyclictest60350irq/132-enp0s29f1:rx-008:22:110
229105991613,2cyclictest60350irq/132-enp0s29f1:rx-008:01:410
229105991612,3cyclictest60350irq/132-enp0s29f1:rx-007:30:110
229105991612,3cyclictest462082-21needreboot10:50:180
229105991612,2cyclictest60350irq/132-enp0s29f1:rx-011:25:110
229105991612,2cyclictest249749-21turbostat07:30:000
229105991610,4cyclictest343962-21apt-get09:00:120
22911599158,5cyclictest296959-21kworker/u8:1+flush-259:008:45:153
22911599151,4cyclictest488888-21awk11:19:593
229115991512,2cyclictest571019-21ntp_kernel_pll_12:35:213
229115991512,2cyclictest359823-21sed09:15:173
229115991512,2cyclictest234891-21irqcore07:15:173
22911199156,8cyclictest0-21swapper/209:40:162
22911199156,7cyclictest0-21swapper/209:25:002
22911199154,4cyclictest328239-21cron08:45:012
22911199154,1cyclictest0-21swapper/209:55:162
22911199153,3cyclictest469704-21cut11:00:152
22911199153,3cyclictest356408-21cut09:10:172
22911199151,2cyclictest549951-21ntpq12:15:202
22911199151,2cyclictest0-21swapper/211:55:012
229111991512,2cyclictest515446-21irqcore11:45:162
229111991512,2cyclictest370726-21kthreadcore09:25:172
22911199150,5cyclictest337662-21ntpq08:50:202
22911199150,5cyclictest307387-21apt-get08:25:102
22911199150,4cyclictest562239-21grep12:30:132
22911199150,4cyclictest530668-21grep12:00:122
22911199150,4cyclictest504952-21cut11:35:142
22911199150,4cyclictest359825-21tr09:15:182
22911199150,4cyclictest297415-21cut08:15:152
22911199150,3cyclictest468199-21awk11:00:002
22911199150,3cyclictest442053-21unixbench_singl10:30:222
22911199150,3cyclictest426580-21sh10:20:002
22911199150,3cyclictest411104-21awk10:05:012
22911199150,3cyclictest349580-21irqcore09:05:192
22911199150,3cyclictest234208-21turbostat.cron07:15:002
229108991511,3cyclictest296959-21kworker/u8:1+flush-259:008:41:541
22910899150,9cyclictest488894-21awk11:20:001
22910599159,4cyclictest229315-21apt-get07:10:110
229105991513,2cyclictest61150irq/133-enp0s29f1:tx-011:40:160
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-012:10:160
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-011:49:290
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-011:35:180
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-009:25:150
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-007:55:150
229105991513,1cyclictest60350irq/132-enp0s29f1:rx-007:42:240
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-011:20:240
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-010:35:490
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-009:48:190
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-008:40:160
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-007:51:010
229105991512,2cyclictest60350irq/132-enp0s29f1:rx-007:15:130
22911599149,3cyclictest384942-21kworker/u8:2+events_unbound10:10:003
229115991412,1cyclictest447613-21munin-plugin-st10:39:593
229115991411,2cyclictest529311-21ntp_kernel_err11:55:183
229115991411,1cyclictest0-21swapper/307:25:003
229115991410,3cyclictest509-21rs:main3
229115991410,2cyclictest0-21swapper/309:55:163
22911199145,1cyclictest0-21swapper/209:35:152
22911199140,5cyclictest295871-21ntpq08:10:212
22911199140,4cyclictest415488-21smartctl10:05:212
22911199140,4cyclictest245278-21grep07:25:182
22911199140,3cyclictest455311-21kthreadcore10:45:172
22911199140,3cyclictest244631-21systemd-run07:25:012
229108991411,2cyclictest525300-21kworker/u8:1+events_unbound12:05:141
229108991411,2cyclictest411375-21apt-get10:05:111
229108991411,2cyclictest384942-21kworker/u8:2+flush-259:010:30:171
229108991411,2cyclictest333423-21kworker/u8:3+events_unbound09:05:231
229108991411,2cyclictest266313-21sed07:45:131
229108991411,2cyclictest265862-21kworker/u8:3+events_unbound08:00:141
229108991410,3cyclictest463089-21kworker/u8:0+events_unbound11:10:121
229108991410,3cyclictest401066-21kworker/u8:1+flush-259:009:55:211
229108991410,3cyclictest384942-21kworker/u8:2+flush-259:010:10:191
229108991410,3cyclictest249999-21apt-get07:30:101
229105991413,1cyclictest60350irq/132-enp0s29f1:rx-012:35:240
229105991413,1cyclictest60350irq/132-enp0s29f1:rx-008:55:130
229105991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:220
229105991412,1cyclictest60350irq/132-enp0s29f1:rx-008:50:140
229105991411,2cyclictest60350irq/132-enp0s29f1:rx-010:05:250
229105991411,2cyclictest60350irq/132-enp0s29f1:rx-009:55:220
229105991411,2cyclictest368479-21needreboot09:20:180
229105991410,2cyclictest60350irq/132-enp0s29f1:rx-007:45:100
22911599139,3cyclictest494088-21kworker/u8:5+flush-259:012:20:013
22911599139,3cyclictest322876-21kworker/u8:2+flush-259:008:50:153
22911599137,4cyclictest354011-21latency_hist09:10:013
22911599131,2cyclictest509612-21awk11:40:003
229115991311,1cyclictest0-21swapper/308:09:593
229115991310,2cyclictest519924-21cron11:50:003
229115991310,2cyclictest394353-21ntp_kernel_pll_09:45:223
22911199130,4cyclictest494820-21irqcore11:25:162
22910899139,3cyclictest535414-21kworker/u8:0+flush-259:012:25:211
22910899139,3cyclictest296959-21kworker/u8:1+flush-259:008:17:231
22910899139,3cyclictest229081-21kworker/u8:2+events_unbound07:10:231
229108991311,1cyclictest0-21swapper/107:40:011
229108991310,2cyclictest550366-21unixbench_multi12:15:241
229108991310,2cyclictest264626-21needreboot07:40:201
22910899130,12cyclictest1-21systemd07:26:191
22910599139,3cyclictest359600-21fschecks_count09:15:160
22910599132,2cyclictest312586-21apt-get08:30:120
229105991312,1cyclictest60350irq/132-enp0s29f1:rx-010:46:390
229105991311,1cyclictest60350irq/132-enp0s29f1:rx-011:57:390
229105991311,1cyclictest60350irq/132-enp0s29f1:rx-008:17:180
229105991311,1cyclictest60350irq/132-enp0s29f1:rx-008:10:130
229105991310,2cyclictest61150irq/133-enp0s29f1:tx-012:25:190
229105991310,2cyclictest530692-21fschecks_count12:00:120
22911599129,2cyclictest436880-21unixbench_singl10:25:243
22911599129,2cyclictest307147-21kworker/u8:0+flush-259:009:05:003
22911599129,2cyclictest301908-21idleruntime-cro08:19:593
22911599127,3cyclictest280910-21sshd07:57:183
229115991210,1cyclictest0-21swapper/311:55:013
229115991210,1cyclictest0-21swapper/307:40:013
22911199120,3cyclictest490036-21irqcore11:20:172
22910899129,2cyclictest525300-21kworker/u8:1+flush-259:011:55:161
22910899129,2cyclictest519533-21unixbench_singl11:45:251
22910899126,4cyclictest252-21systemd-journal08:29:591
22910899123,2cyclictest571895-21apt12:40:001
229108991210,1cyclictest494088-21kworker/u8:5+events_unbound11:25:191
22910599128,3cyclictest1-21systemd11:10:000
22911599119,1cyclictest481rcuc/308:25:183
22911599118,2cyclictest358178-21cut09:10:193
22911599116,4cyclictest463055-21awk10:54:593
22911599116,3cyclictest490-21dbus-daemon09:25:013
22911599115,4cyclictest307119-21latency_hist08:25:003
22911599115,4cyclictest252-21systemd-journal11:25:003
22911599110,10cyclictest339565-21irqcore08:55:163
22911599107,2cyclictest494751-21irqcore11:25:163
22911599107,2cyclictest385618-21irqcore09:40:173
22911599107,2cyclictest312651-21cpuspeed_turbos08:30:123
22911599107,2cyclictest0-21swapper/310:10:183
22911599101,8cyclictest61-21kcompactd010:16:443
2291159995,2cyclictest544797-21ntpq12:10:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional