You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 23:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 18, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2311105994225,12cyclictest2641022-21latency_hist12:25:011
2311105993218,9cyclictest2515752-21ntpq10:20:201
2311103993016,13cyclictest2526002-21ntpq10:30:220
2311113992922,5cyclictest2629846-21ntpq12:10:243
2311108992924,3cyclictest2490554-21base6410:00:012
2311113992826,1cyclictest2316045-21fwupd07:13:003
2311113992823,3cyclictest2395615-21latency_hist08:30:013
2311108992825,2cyclictest2609684-21kworker/u8:2+events_unbound12:10:252
2311113992721,4cyclictest2630926-21/usr/sbin/munin12:15:143
2311105992722,3cyclictest2369512-21/usr/sbin/munin08:05:141
2311113992620,4cyclictest2552643-21latency_hist11:00:013
2311103992621,4cyclictest2578471-21apt-get11:23:580
2311103992620,4cyclictest2395624-21apt-get08:30:020
2311113992519,4cyclictest2603652-21ntpq11:45:223
231110399259,4cyclictest252-21systemd-journal09:09:060
2311113992418,4cyclictest2479599-21kworker/u8:1+flush-259:011:00:183
2311108992319,3cyclictest2479599-21kworker/u8:1+flush-259:011:23:562
2311108992317,4cyclictest252-21systemd-journal11:10:572
2311105992317,4cyclictest2458977-21latency_hist09:30:001
2311113992218,3cyclictest2630705-21kworker/u8:3+flush-259:012:25:003
2311113992215,4cyclictest2619964-21latency_hist12:04:593
2311108992218,3cyclictest2609684-21kworker/u8:2+flush-259:012:10:002
2311113992116,3cyclictest2485017-21store09:55:013
2311108992116,3cyclictest252-21systemd-journal11:57:072
2311105992116,3cyclictest252-21systemd-journal09:50:061
2311113992016,2cyclictest2619965-21apt-get12:05:023
2311113992012,6cyclictest2489526-21ntpq09:55:213
2311108992016,3cyclictest2432877-21kworker/u8:3+events_unbound09:05:242
2311108992016,3cyclictest2374635-21kworker/u8:3+flush-259:008:20:382
2311105992016,3cyclictest2641287-21apt-get12:25:111
2311105992014,4cyclictest252-21systemd-journal08:30:051
2311105992010,9cyclictest503-21tpm2-abrmd12:10:001
2311103992014,4cyclictest2646209-21latency_hist12:30:010
2311113991915,3cyclictest2384819-21apt-config08:20:003
231110899196,6cyclictest2369277-21munin-run08:05:002
2311108991915,3cyclictest2290937-21kworker/u8:0+events_unbound07:13:072
2311108991914,3cyclictest2521760-21kworker/u8:3+flush-259:010:50:012
2311105991911,7cyclictest252-21systemd-journal12:36:091
2311103991916,2cyclictest2348405-21apt-get07:45:140
2311103991915,3cyclictest2578662-21kworker/u8:4+flush-259:011:50:000
2311103991915,3cyclictest2490383-21latency_hist10:00:010
2311103991913,1cyclictest60350irq/132-enp0s29f1:rx-011:55:230
2311113991815,2cyclictest2558466-21irqcore11:05:173
2311113991815,2cyclictest0-21swapper/308:05:003
2311113991810,6cyclictest2443669-21cp09:15:003
2311108991814,3cyclictest2630705-21kworker/u8:3+flush-259:012:32:422
2311108991814,3cyclictest2578662-21kworker/u8:4+events_unbound11:45:222
2311108991814,2cyclictest0-21swapper/211:10:002
231110599188,8cyclictest513-21gdbus09:20:001
2311105991814,3cyclictest2384814-21gpgv08:20:001
231110599180,15cyclictest2506324-21latency_hist10:15:011
2311103991814,3cyclictest2384824-21kworker/u8:0+flush-259:009:00:000
2311103991813,4cyclictest252-21systemd-journal08:13:060
2311103991813,1cyclictest60350irq/132-enp0s29f1:rx-009:33:080
2311103991812,4cyclictest2520813-21ntpq10:25:220
2311103991811,5cyclictest2480014-21diskstats09:50:130
2311113991712,3cyclictest2438148-21apt-get09:10:133
2311108991714,2cyclictest2464209-21kworker/u8:0+flush-259:010:00:222
2311108991714,2cyclictest2353314-21kworker/u8:1+flush-259:008:40:012
2311108991714,2cyclictest0-21swapper/209:00:002
231110599178,8cyclictest2427933-21cpuspeed_turbos09:00:141
2311105991711,4cyclictest2321546-21cp07:20:001
231110599170,4cyclictest2526576-21chrt10:31:031
2311103991713,3cyclictest2290937-21kworker/u8:0+flush-259:007:20:010
2311103991711,4cyclictest2651815-21diskstats12:35:120
2311103991710,5cyclictest2348139-21latency_hist07:45:010
2311103991710,5cyclictest2348139-21latency_hist07:45:000
2311113991613,2cyclictest2570740-21kthreadcore11:15:193
2311113991612,3cyclictest2971-21cron09:10:013
2311113991612,2cyclictest0-21swapper/308:50:003
2311113991610,4cyclictest252-21systemd-journal07:20:013
2311108991613,2cyclictest2609684-21kworker/u8:2+flush-259:012:00:202
2311108991613,2cyclictest2552641-21idleruntime-cro11:00:012
2311108991613,2cyclictest2521760-21kworker/u8:3+events_unbound10:40:232
2311108991613,2cyclictest2364325-21kworker/u8:4+flush-259:008:10:162
2311108991613,1cyclictest0-21swapper/209:15:002
2311108991612,3cyclictest509-21rs:main2
2311108991612,3cyclictest2578662-21kworker/u8:4+flush-259:011:25:172
2311108991612,3cyclictest2479599-21kworker/u8:1+flush-259:010:15:002
2311108991612,3cyclictest2384824-21kworker/u8:0+flush-259:008:50:062
231110599168,6cyclictest0-21swapper/112:05:011
231110599167,8cyclictest2547788-21cpuspeed_turbos10:55:121
231110599166,9cyclictest2336370-21sed07:30:211
231110599165,9cyclictest2390431-21basename08:25:011
231110599163,11cyclictest2594401-21latency_hist11:40:001
2311105991613,2cyclictest2490376-21cron10:00:001
231110599161,2cyclictest2468409-21ntpq09:35:211
2311105991612,3cyclictest2583990-21kworker/u8:1+flush-259:011:55:181
2311105991612,3cyclictest2443062-21gpgv09:14:591
231110599161,13cyclictest2479590-21munin-plugin-st09:50:001
231110599161,13cyclictest2395605-21cron08:30:001
2311105991610,5cyclictest2541396-21ntpq10:45:201
231110599160,5cyclictest2630701-21sh12:14:591
231110599160,5cyclictest2352395-21ntpq07:45:231
231110599160,3cyclictest2379605-21latency_hist08:15:001
231110599160,1cyclictest2331945-21seq07:26:421
231110599160,13cyclictest2599777-21tr11:45:141
231110599160,13cyclictest2311466-21cut07:10:091
231110399169,6cyclictest2589373-21/usr/sbin/munin11:35:150
2311103991614,1cyclictest60350irq/132-enp0s29f1:rx-010:45:150
2311103991614,1cyclictest60350irq/132-enp0s29f1:rx-007:35:180
2311103991614,1cyclictest60350irq/132-enp0s29f1:rx-007:10:110
2311103991613,2cyclictest60350irq/132-enp0s29f1:rx-012:15:170
2311103991613,2cyclictest60350irq/132-enp0s29f1:rx-012:10:170
2311103991613,2cyclictest2511420-21sh10:19:590
2311103991612,2cyclictest0-21swapper/011:15:010
231111399158,5cyclictest252-21systemd-journal11:40:003
2311113991512,2cyclictest2526915-21munin-plugin-st10:35:003
2311113991511,2cyclictest0-21swapper/311:24:023
2311113991510,4cyclictest2432667-21kworker/u8:2+flush-259:009:45:263
2311113991510,3cyclictest2464209-21kworker/u8:0+flush-259:009:35:173
2311108991513,2cyclictest2337410-21kworker/u8:0+events_unbound07:45:232
2311108991512,2cyclictest2630706-21kworker/u8:4+flush-259:012:35:182
2311108991512,2cyclictest2479599-21kworker/u8:1+events_unbound11:15:252
2311108991512,2cyclictest2257232-21kworker/u8:3+events_unbound07:40:132
2311108991512,2cyclictest2257232-21kworker/u8:3+events_unbound07:40:122
2311108991511,3cyclictest2479599-21kworker/u8:1+flush-259:010:05:172
2311108991511,3cyclictest2432667-21kworker/u8:2+flush-259:009:15:012
2311108991510,3cyclictest509-21rs:main2
2311108991510,3cyclictest2448665-21kworker/u8:3+events_unbound09:49:082
231110599158,6cyclictest0-21swapper/109:00:001
231110599157,7cyclictest2414140-21kthreadcore08:45:181
231110599152,9cyclictest2552908-21apt-get11:00:131
231110599152,10cyclictest252-21systemd-journal11:31:051
231110599152,10cyclictest2448895-21apt-get09:20:131
231110599151,2cyclictest2357639-21ntpq07:50:221
2311105991512,2cyclictest0-21swapper/111:45:001
231110599151,12cyclictest2577602-21ntpq11:20:211
231110599151,12cyclictest2573362-21dump-pmu-power11:20:001
231110599151,12cyclictest2568131-21cron11:15:011
231110599151,12cyclictest2562984-21apt11:10:011
231110599151,12cyclictest2368386-21ntpq08:00:201
231110599150,5cyclictest2605218-21ps11:50:181
231110599150,1cyclictest2347267-21ntpq07:40:231
231110599150,1cyclictest2347267-21ntpq07:40:221
2311103991514,1cyclictest60350irq/132-enp0s29f1:rx-008:53:060
2311103991513,2cyclictest60350irq/132-enp0s29f1:rx-010:52:380
2311103991513,2cyclictest60350irq/132-enp0s29f1:rx-007:30:480
2311103991513,1cyclictest60350irq/132-enp0s29f1:rx-012:30:160
2311103991513,1cyclictest60350irq/132-enp0s29f1:rx-011:05:170
2311103991513,1cyclictest60350irq/132-enp0s29f1:rx-010:55:240
2311103991513,1cyclictest60350irq/132-enp0s29f1:rx-008:38:400
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-011:41:580
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-011:32:580
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-010:37:570
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-010:20:240
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-010:12:000
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-010:05:080
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:240
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-009:35:170
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-008:41:290
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-008:27:060
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-008:00:150
2311103991512,2cyclictest60350irq/132-enp0s29f1:rx-007:26:260
2311103991511,3cyclictest2615089-21apt-get12:00:130
2311103991511,3cyclictest2454059-21apt-get09:25:120
2311103991510,4cyclictest2630706-21kworker/u8:4+flush-259:012:20:180
2311113991412,1cyclictest2427126-21munin-run09:00:003
2311113991410,3cyclictest2479599-21kworker/u8:1+flush-259:010:05:263
2311113991410,3cyclictest2453363-21unixbench_singl09:20:263
2311113991410,2cyclictest0-21swapper/308:20:143
231110899148,4cyclictest2384908-21apt-key08:20:002
2311108991411,2cyclictest0-21swapper/212:15:202
2311108991410,3cyclictest2583990-21kworker/u8:1+events_unbound11:30:232
2311108991410,3cyclictest2578662-21kworker/u8:4+flush-259:011:50:142
2311108991410,3cyclictest2448665-21kworker/u8:3+flush-259:009:40:302
2311108991410,3cyclictest2290937-21kworker/u8:0+events_unbound07:17:012
231110599141,9cyclictest2646470-21apt-get12:30:111
231110599141,4cyclictest2411563-21cron08:45:001
231110599141,2cyclictest2421031-21ntpq08:50:221
231110599141,11cyclictest2532703-21grep10:40:171
231110599141,10cyclictest2516974-21tr10:25:141
231110599141,10cyclictest0-21swapper/110:15:111
231110599140,5cyclictest2527176-21chrt10:35:131
231110599140,5cyclictest2495158-21/usr/sbin/munin10:00:231
231110599140,4cyclictest2632777-21cut12:15:191
231110599140,4cyclictest2433165-21tune2fs09:05:131
231110599140,4cyclictest2401727-21perl08:35:171
231110599140,3cyclictest2474466-21systemd-run09:45:011
231110599140,3cyclictest2327043-21latency_hist07:25:001
231110599140,2cyclictest2547454-21idleruntime-cro10:55:011
231110599140,2cyclictest252-21systemd-journal11:30:001
231110599140,2cyclictest2500289-21ntpq10:05:201
231110599140,2cyclictest2464171-21cron09:35:001
231110599140,2cyclictest2363671-21latency_hist08:00:001
231110599140,1cyclictest2338494-21ps07:35:171
231110399149,4cyclictest60350irq/132-enp0s29f1:rx-007:20:210
2311103991412,2cyclictest60350irq/132-enp0s29f1:rx-008:06:240
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-011:51:090
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-011:17:360
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-009:15:130
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-008:20:150
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-008:15:160
2311103991412,1cyclictest60350irq/132-enp0s29f1:rx-007:55:250
2311103991411,2cyclictest60350irq/132-enp0s29f1:rx-010:40:200
2311103991411,2cyclictest60350irq/132-enp0s29f1:rx-009:43:180
2311103991411,2cyclictest60350irq/132-enp0s29f1:rx-008:45:130
231111399139,2cyclictest0-21swapper/312:30:003
231111399139,2cyclictest0-21swapper/310:24:593
231111399139,2cyclictest0-21swapper/309:05:003
231111399139,2cyclictest0-21swapper/308:10:213
2311113991310,2cyclictest2589117-21cron11:35:003
2311113991310,2cyclictest2541848-21unixbench_multi10:45:253
231111399130,3cyclictest2537178-21awk10:45:013
231110899139,3cyclictest2630705-21kworker/u8:3+events_unbound12:20:282
231110899139,3cyclictest2521760-21kworker/u8:3+events_unbound10:50:142
231110899139,3cyclictest2479599-21kworker/u8:1+events_unbound11:00:242
231110899139,3cyclictest2432667-21kworker/u8:2+events_unbound09:50:022
231110899139,3cyclictest2432667-21kworker/u8:2+events_unbound09:30:212
231110899139,3cyclictest2257232-21kworker/u8:3+events_unbound07:55:022
231110899139,3cyclictest2257232-21kworker/u8:3+events_unbound07:25:212
231110899138,3cyclictest2511398-21cron10:20:002
231110899137,4cyclictest2406647-21apt-get08:40:122
2311108991310,2cyclictest2578662-21kworker/u8:4+events_unbound11:40:172
2311103991311,1cyclictest60350irq/132-enp0s29f1:rx-010:03:240
2311103991311,1cyclictest0-21swapper/009:24:590
231111399129,2cyclictest2583990-21kworker/u8:1+flush-259:011:41:113
231111399129,2cyclictest0-21swapper/312:35:213
231111399128,2cyclictest0-21swapper/307:30:173
231111399126,4cyclictest2410759-21ntpq08:40:243
231111399123,7cyclictest2474433-21idleruntime-cro09:44:593
2311113991210,1cyclictest0-21swapper/309:35:013
231110899129,2cyclictest2526011-21ntp_kernel_pll_10:30:232
231110899128,3cyclictest2464209-21kworker/u8:0+flush-259:009:36:072
231110899128,3cyclictest2432667-21kworker/u8:2+events_unbound09:22:082
231110899128,3cyclictest2358745-21kworker/u8:2+flush-259:008:05:182
231110899128,3cyclictest2353314-21kworker/u8:1+flush-259:008:25:112
231110899128,3cyclictest2257232-21kworker/u8:3+events_unbound07:31:082
231110899128,2cyclictest2406399-21kworker/u8:4+events_unbound08:45:232
2311108991210,1cyclictest2323425-21kthreadcore07:20:182
2311108991210,1cyclictest0-21swapper/209:25:162
231110399129,2cyclictest60350irq/132-enp0s29f1:rx-009:10:140
231110399129,2cyclictest60350irq/132-enp0s29f1:rx-009:00:210
2311103991210,1cyclictest60350irq/132-enp0s29f1:rx-012:07:280
2311103991210,1cyclictest60350irq/132-enp0s29f1:rx-011:00:070
231111399119,1cyclictest2311096-21cyclictest07:25:193
231111399118,2cyclictest2579051-21cpuspeed_turbos11:25:153
231111399117,3cyclictest2479599-21kworker/u8:1+flush-259:010:52:373
231111399116,4cyclictest2337385-21apt-get07:35:033
231111399114,5cyclictest490-21dbus-daemon08:35:003
231110899117,3cyclictest2495652-21kworker/u8:2+flush-259:010:25:162
231110899117,3cyclictest2406399-21kworker/u8:4+events_unbound09:03:092
231111399107,2cyclictest2405908-21unixbench_multi08:35:263
231111399103,4cyclictest2565241-21kthreadcore11:10:203
23111089995,3cyclictest2479599-21kworker/u8:1+events_unbound10:20:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional