You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 15:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun Sep 06, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2177190994439,4cyclictest2514244-21cp12:35:003
2177186994441,2cyclictest2249414-21kworker/u8:0+events_unbound08:25:212
2177186994036,3cyclictest2192635-21kworker/u8:1+flush-259:008:20:112
2177186993733,3cyclictest2348063-21kworker/u8:4+events_unbound10:04:012
2177186993431,2cyclictest2410208-21kworker/u8:0+flush-259:011:35:222
2177186993427,3cyclictest2384600-21kworker/u8:3+events_unbound10:55:142
2177181993428,4cyclictest2478102-21apt-get12:00:111
2177186993329,2cyclictest2358701-21kworker/u8:2+flush-259:010:05:152
2177179993329,3cyclictest2208353-21/usr/sbin/munin07:40:150
2177190993227,4cyclictest2457481-21/usr/sbin/munin11:40:163
2177179993229,2cyclictest2249414-21kworker/u8:0+flush-259:008:35:180
2177179993228,3cyclictest335557-21systemd-journal07:29:290
2177186993128,2cyclictest2457281-21kworker/u8:3+flush-259:011:55:252
2177181993126,3cyclictest2503950-21/usr/sbin/munin12:25:141
2177186993027,2cyclictest2228804-21kworker/u8:0+events_unbound08:10:202
2177181993024,4cyclictest2367950-21ntpq10:10:221
2177190992925,3cyclictest2457281-21kworker/u8:3+flush-259:012:10:253
2177186992926,2cyclictest2374292-21kworker/u8:2+flush-259:011:10:152
2177186992923,4cyclictest509-21rs:main2
2177181992923,4cyclictest2274324-21ntpq08:40:211
2177179992923,4cyclictest2322290-21latency_hist09:30:010
2177186992824,3cyclictest2374292-21kworker/u8:2+events_unbound10:25:142
2177186992824,3cyclictest2192635-21kworker/u8:1+events_unbound08:40:152
2177181992823,3cyclictest2249386-21latency_hist08:20:011
2177181992821,5cyclictest2347116-21ntpq09:50:211
2177190992723,3cyclictest2311703-21kworker/u8:3+flush-259:009:45:253
2177190992722,3cyclictest2483008-21latency_hist12:05:003
2177186992723,3cyclictest2374292-21kworker/u8:2+events_unbound10:45:142
2177179992720,5cyclictest2300337-21ntpq09:05:230
2177190992623,2cyclictest2348063-21kworker/u8:4+events_unbound10:40:193
2177186992622,3cyclictest2457281-21kworker/u8:3+events_unbound12:20:092
2177186992619,5cyclictest335557-21systemd-journal09:53:072
2177181992621,3cyclictest2192868-21/usr/sbin/munin07:25:201
2177179992623,2cyclictest2348306-21apt-get09:55:110
2177186992522,2cyclictest2384600-21kworker/u8:3+events_unbound10:40:152
2177186992522,2cyclictest2280414-21kworker/u8:0+events_unbound09:30:202
2177186992521,3cyclictest2384600-21kworker/u8:3+events_unbound10:30:152
2177186992520,3cyclictest2457281-21kworker/u8:3+flush-259:012:25:222
2177186992517,3cyclictest2280414-21kworker/u8:0+events_unbound09:35:222
2177181992521,3cyclictest2301897-21apt-get09:10:141
2177190992420,3cyclictest2374292-21kworker/u8:2+flush-259:010:45:203
2177190992419,4cyclictest2384638-21apt-get10:30:113
2177190992418,4cyclictest335557-21systemd-journal07:45:003
2177186992421,2cyclictest2374292-21kworker/u8:2+events_unbound11:15:232
2177186992421,2cyclictest2177156-21kworker/u8:2+flush-259:007:25:242
2177186992420,3cyclictest2177156-21kworker/u8:2+events_unbound07:20:212
2177186992417,3cyclictest2483252-21kworker/u8:0+events_unbound12:15:002
2177190992319,3cyclictest2249629-21/usr/sbin/munin08:20:133
2177190992319,3cyclictest2177404-21apt-get07:10:103
2177186992320,2cyclictest2457281-21kworker/u8:3+flush-259:012:05:222
2177186992319,3cyclictest2483252-21kworker/u8:0+events_unbound12:35:302
2177181992320,2cyclictest2177156-21kworker/u8:2+flush-259:007:30:211
2177181992319,3cyclictest509-21rs:main1
2177181992317,5cyclictest2289849-21ntpq08:55:231
2177179992320,2cyclictest2410208-21kworker/u8:0+flush-259:011:30:000
2177190992218,3cyclictest2415546-21gpgv11:00:003
2177190992217,3cyclictest509-21rs:main3
2177186992219,2cyclictest2374292-21kworker/u8:2+flush-259:011:00:252
2177186992219,2cyclictest2374292-21kworker/u8:2+events_unbound11:25:212
2177186992219,2cyclictest2192635-21kworker/u8:1+events_unbound08:15:142
2177186992219,2cyclictest2177156-21kworker/u8:2+flush-259:007:15:162
2177186992218,3cyclictest2337728-21kworker/u8:1+events_unbound10:10:292
2177186992218,3cyclictest2192635-21kworker/u8:1+flush-259:008:05:172
2177186992218,3cyclictest2192635-21kworker/u8:1+flush-259:008:05:172
2177186992218,3cyclictest2192635-21kworker/u8:1+events_unbound07:56:592
2177186992216,4cyclictest2202985-21latency_hist07:35:012
2177186992213,3cyclictest2374292-21kworker/u8:2+events_unbound11:30:202
2177181992216,5cyclictest2394993-21/usr/sbin/munin10:40:151
2177179992219,2cyclictest2467510-21kworker/u8:1+events_unbound11:50:140
2177179992218,3cyclictest2218715-21apt-get07:50:110
2177190992117,3cyclictest2192635-21kworker/u8:1+flush-259:008:35:003
2177190992117,2cyclictest2374292-21kworker/u8:2+flush-259:011:45:153
2177190992116,3cyclictest335557-21systemd-journal10:20:073
2177190992116,3cyclictest2333005-21diskstats09:40:143
2177190992115,4cyclictest2269149-21ntpq08:35:233
2177186992117,2cyclictest0-21swapper/209:10:002
2177186992116,4cyclictest2311703-21kworker/u8:3+flush-259:009:45:212
2177186992114,3cyclictest2374292-21kworker/u8:2+events_unbound11:20:242
2177179992118,2cyclictest2457281-21kworker/u8:3+events_unbound12:10:230
2177179992118,2cyclictest2243710-21kworker/u8:3+flush-259:008:15:130
2177179992114,5cyclictest2467540-21latency_hist11:50:010
2177179992114,5cyclictest2409419-21ntpq10:50:210
2177190992016,3cyclictest2467811-21apt-get11:50:113
2177190992014,5cyclictest2416026-21cat11:00:013
2177190992014,4cyclictest2239054-21latency_hist08:10:013
2177190992014,4cyclictest2239054-21latency_hist08:10:003
2177186992017,2cyclictest2311703-21kworker/u8:3+flush-259:009:20:202
2177186992017,2cyclictest2249414-21kworker/u8:0+events_unbound08:30:152
2177186992016,3cyclictest2139012-21kworker/u8:0+events_unbound07:35:202
2177186992015,4cyclictest507-21in:imuxsock11:50:002
2177181992016,3cyclictest2431564-21apt-get11:15:111
2177181992016,3cyclictest2182559-21apt-get07:15:131
2177181992013,5cyclictest335557-21systemd-journal08:00:001
2177179992017,2cyclictest2483252-21kworker/u8:0+flush-259:012:35:200
2177179992016,3cyclictest2384600-21kworker/u8:3+flush-259:010:30:210
2177179992016,3cyclictest2327689-21apt-get09:35:110
2177179992014,3cyclictest2177156-21kworker/u8:2+flush-259:007:30:140
2177190991917,1cyclictest2249394-21munin-plugin-st08:20:003
2177190991915,3cyclictest2264905-21kworker/u8:2+flush-259:008:45:003
2177186991916,2cyclictest2421008-21kworker/u8:1+events_unbound11:05:152
2177186991916,2cyclictest2311703-21kworker/u8:3+flush-259:009:40:162
2177186991916,2cyclictest2285846-21kworker/u8:1+events_unbound09:01:582
2177186991916,2cyclictest2122753-21kworker/u8:3+events_unbound07:14:092
2177186991915,3cyclictest2311703-21kworker/u8:3+events_unbound09:59:192
2177186991915,3cyclictest2280414-21kworker/u8:0+events_unbound08:55:012
2177186991915,3cyclictest2249414-21kworker/u8:0+events_unbound08:35:252
2177181991915,3cyclictest2509124-21/usr/sbin/munin12:30:151
2177179991916,2cyclictest2280414-21kworker/u8:0+flush-259:009:15:120
2177179991915,3cyclictest2177156-21kworker/u8:2+events_unbound07:10:210
2177179991914,3cyclictest2177156-21kworker/u8:2+flush-259:008:05:220
2177179991914,3cyclictest2177156-21kworker/u8:2+flush-259:008:05:220
2177179991913,5cyclictest2410208-21kworker/u8:0+flush-259:011:35:250
2177179991913,1cyclictest60350irq/132-enp0s29f1:rx-008:40:260
2177190991815,2cyclictest507-21in:imuxsock07:30:003
2177186991815,3cyclictest2374292-21kworker/u8:2+events_unbound11:40:102
2177186991814,3cyclictest2457281-21kworker/u8:3+flush-259:012:30:272
2177186991814,3cyclictest2139012-21kworker/u8:0+events_unbound07:41:082
2177186991812,4cyclictest2493608-21/usr/sbin/munin12:15:202
2177181991815,2cyclictest2192635-21kworker/u8:1+flush-259:008:30:001
2177181991814,3cyclictest335557-21systemd-journal09:21:071
2177181991812,4cyclictest2263966-21ntpq08:30:191
2177179991815,2cyclictest2483252-21kworker/u8:0+flush-259:012:30:170
2177179991815,2cyclictest2139012-21kworker/u8:0+flush-259:007:35:170
2177179991814,3cyclictest2384600-21kworker/u8:3+flush-259:010:55:150
2177179991814,3cyclictest2348063-21kworker/u8:4+events_unbound10:25:210
2177179991813,4cyclictest2280414-21kworker/u8:0+events_unbound09:30:220
2177190991714,2cyclictest2374292-21kworker/u8:2+flush-259:010:30:003
2177190991713,3cyclictest2343090-21kworker/u8:2+events_unbound09:50:233
2177186991714,2cyclictest2467510-21kworker/u8:1+flush-259:012:00:202
2177186991713,3cyclictest2311703-21kworker/u8:3+events_unbound09:25:092
217718699170,8cyclictest2223606-21munin-run07:54:592
2177181991711,5cyclictest2301203-21latency_hist09:10:011
2177179991714,2cyclictest60350irq/132-enp0s29f1:rx-008:30:110
2177179991714,2cyclictest2192635-21kworker/u8:1+events_unbound07:47:080
2177179991713,3cyclictest2374292-21kworker/u8:2+flush-259:011:15:200
2177179991712,3cyclictest2348063-21kworker/u8:4+flush-259:010:25:000
2177179991711,4cyclictest2467510-21kworker/u8:1+events_unbound12:00:200
2177179991711,4cyclictest2305901-21ntpq09:10:230
2177179991710,5cyclictest335557-21systemd-journal10:45:000
2177179991710,5cyclictest2228777-21latency_hist08:00:010
2177190991613,2cyclictest2421008-21kworker/u8:1+flush-259:011:05:243
2177190991613,2cyclictest2285846-21kworker/u8:1+flush-259:009:05:013
2177190991612,3cyclictest2213507-21/usr/sbin/munin07:45:233
2177190991612,3cyclictest2192635-21kworker/u8:1+flush-259:008:00:253
2177190991611,3cyclictest2498533-21cron12:19:593
2177190991610,4cyclictest2311965-21latency_hist09:20:013
2177186991613,2cyclictest2384600-21kworker/u8:3+flush-259:010:50:052
2177186991613,2cyclictest2264905-21kworker/u8:2+flush-259:009:14:492
2177186991613,2cyclictest2192635-21kworker/u8:1+events_unbound08:00:212
2177186991612,3cyclictest2348063-21kworker/u8:4+events_unbound10:35:202
2177186991612,3cyclictest2280414-21kworker/u8:0+events_unbound09:17:352
2177181991614,1cyclictest503-21tpm2-abrmd07:25:001
2177181991613,1cyclictest2353902-21irqcore10:00:171
2177181991612,3cyclictest2192635-21kworker/u8:1+flush-259:008:01:131
2177181991611,3cyclictest335557-21systemd-journal09:37:071
2177181991611,3cyclictest335557-21systemd-journal08:07:071
2177181991611,3cyclictest335557-21systemd-journal08:07:071
2177181991610,4cyclictest335557-21systemd-journal11:25:001
2177179991614,1cyclictest60350irq/132-enp0s29f1:rx-010:45:170
2177179991614,1cyclictest60350irq/132-enp0s29f1:rx-008:55:150
2177179991614,1cyclictest60350irq/132-enp0s29f1:rx-008:10:220
2177179991612,3cyclictest2457281-21kworker/u8:3+flush-259:012:05:200
2177179991611,3cyclictest2374292-21kworker/u8:2+flush-259:011:00:250
2177190991512,2cyclictest2202975-21sh07:35:003
2177190991511,3cyclictest2264905-21kworker/u8:2+flush-259:008:55:193
2177186991511,3cyclictest2348063-21kworker/u8:4+flush-259:010:24:512
2177186991511,3cyclictest2337728-21kworker/u8:1+events_unbound10:16:032
2177186991510,4cyclictest2192635-21kworker/u8:1+events_unbound07:47:152
217718199159,4cyclictest335557-21systemd-journal10:10:001
2177181991512,2cyclictest2456416-21ntp_offset11:35:221
2177181991510,4cyclictest2353217-21latency_hist10:00:001
2177179991513,1cyclictest60350irq/132-enp0s29f1:rx-012:15:250
2177179991513,1cyclictest60350irq/132-enp0s29f1:rx-011:20:160
2177179991513,1cyclictest60350irq/132-enp0s29f1:rx-011:10:230
2177179991513,1cyclictest60350irq/132-enp0s29f1:rx-008:01:140
2177179991512,2cyclictest60350irq/132-enp0s29f1:rx-010:04:090
2177179991512,2cyclictest60350irq/132-enp0s29f1:rx-009:00:170
2177179991512,2cyclictest0-21swapper/008:50:000
2177179991511,3cyclictest2421008-21kworker/u8:1+flush-259:011:05:210
217719099149,4cyclictest2363676-21latency_hist10:10:013
2177190991412,1cyclictest0-21swapper/308:30:003
2177190991411,2cyclictest2337728-21kworker/u8:1+flush-259:010:10:263
2177186991410,3cyclictest2467510-21kworker/u8:1+events_unbound11:51:002
2177186991410,3cyclictest2285846-21kworker/u8:1+events_unbound08:59:022
2177181991412,1cyclictest2515183-21irqcore12:35:191
2177181991411,2cyclictest2498939-21cpuspeed_turbos12:20:131
2177181991411,2cyclictest2348063-21kworker/u8:4+flush-259:010:50:171
2177181991411,2cyclictest2177470-21cpuspeed_turbos07:10:121
2177181991410,3cyclictest335557-21systemd-journal11:04:551
2177179991412,1cyclictest60350irq/132-enp0s29f1:rx-012:29:540
2177179991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:200
2177179991412,1cyclictest60350irq/132-enp0s29f1:rx-007:20:150
2177179991411,2cyclictest61150irq/133-enp0s29f1:tx-009:20:160
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-011:55:150
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-011:40:120
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-011:30:220
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-010:35:340
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-010:05:340
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-009:41:570
2177179991411,2cyclictest60350irq/132-enp0s29f1:rx-008:54:210
217719099139,3cyclictest2374292-21kworker/u8:2+flush-259:011:30:303
2177190991311,1cyclictest0-21swapper/309:10:013
2177190991310,2cyclictest2306382-21unixbench_singl09:10:263
2177190991310,2cyclictest2224001-21cpuspeed_turbos07:55:133
217718199139,3cyclictest2291012-21/usr/sbin/munin09:00:141
2177181991312,1cyclictest2348063-21kworker/u8:4+events_unbound10:35:161
2177181991311,1cyclictest2415637-21latency_hist11:00:011
2177181991310,2cyclictest2468828-21kthreadcore11:50:171
2177179991311,1cyclictest60350irq/132-enp0s29f1:rx-012:20:170
2177179991311,1cyclictest60350irq/132-enp0s29f1:rx-008:20:150
2177179991311,1cyclictest60350irq/132-enp0s29f1:rx-007:15:190
2177179991310,2cyclictest60350irq/132-enp0s29f1:rx-010:10:190
217719099129,2cyclictest500-21systemd-logind12:00:003
217719099129,2cyclictest2436003-21unixbench_multi11:15:253
217719099129,2cyclictest2430855-21unixbench_singl11:10:253
217719099129,2cyclictest2223114-21unixbench_multi07:50:233
217719099126,5cyclictest2326559-21ntpq09:30:223
2177190991210,1cyclictest0-21swapper/307:20:003
217718199129,2cyclictest2447173-21apt-get11:30:111
217718199129,2cyclictest2384600-21kworker/u8:3+flush-259:010:31:121
217718199129,2cyclictest2383362-21needreboot10:25:181
217718199129,2cyclictest2192635-21kworker/u8:1+events_unbound07:50:161
2177181991210,1cyclictest0-21swapper/111:45:151
2177181991210,1cyclictest0-21swapper/109:50:001
2177179991210,1cyclictest60350irq/132-enp0s29f1:rx-009:50:150
2177179991210,1cyclictest60350irq/132-enp0s29f1:rx-008:25:230
217719099118,2cyclictest2314531-21kthreadcore09:20:183
217719099118,2cyclictest0-21swapper/310:50:143
217719099117,3cyclictest2122753-21kworker/u8:3+flush-259:007:25:003
217718199119,1cyclictest0-21swapper/112:14:591
217718199118,2cyclictest2311496-21unixbench_multi09:15:251
217718199118,1cyclictest0-21swapper/108:45:161
217718199110,9cyclictest2477844-21awk11:59:591
217719099107,1cyclictest0-21swapper/308:50:013
217719099104,4cyclictest2446019-21ntpq11:25:203
217718199108,1cyclictest2217988-21unixbench_multi07:45:251
217718199107,2cyclictest2487297-21ntp_kernel_pll_12:05:221
217718199107,2cyclictest2253679-21ntp_kernel_pll_08:20:201
217718199107,2cyclictest2139012-21kworker/u8:0+flush-259:007:35:131
217717999107,2cyclictest60350irq/132-enp0s29f1:rx-010:15:130
21771909996,2cyclictest2374292-21kworker/u8:2+flush-259:011:20:153
21771819997,1cyclictest0-21swapper/108:35:201
21771819996,2cyclictest2208539-21diskstats07:40:151
21771819995,3cyclictest2325411-21cut09:30:191
21771909984,3cyclictest2203424-21uniq07:35:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional