You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 19:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Sep 10, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2357675995855,2cyclictest2342127-21kworker/u8:2+events_unbound07:20:242
2357675995855,2cyclictest2342127-21kworker/u8:2+events_unbound07:20:242
2357675994441,2cyclictest2529165-21kworker/u8:1+events_unbound10:45:142
2357675994441,2cyclictest2471562-21kworker/u8:3+events_unbound10:15:232
2357671994037,2cyclictest2471562-21kworker/u8:3+flush-259:010:35:251
2357675993936,2cyclictest2513320-21kworker/u8:2+events_unbound09:45:222
2357675993936,2cyclictest2430214-21kworker/u8:1+flush-259:008:40:262
2357675993935,3cyclictest2487769-21kworker/u8:1+events_unbound09:15:192
2357675993835,2cyclictest2618031-21kworker/u8:1+events_unbound12:00:242
2357675993734,2cyclictest2685468-21kworker/u8:0+flush-259:012:30:152
2357675993734,2cyclictest2471562-21kworker/u8:3+events_unbound10:40:142
2357675993734,2cyclictest2471562-21kworker/u8:3+events_unbound10:35:242
235766899370,35cyclictest1-21systemd08:32:270
2357675993634,2cyclictest2394060-21kworker/u8:4+flush-259:008:00:262
2357675993633,2cyclictest2659714-21kworker/u8:2+flush-259:012:25:232
2357675993532,2cyclictest2352737-21kworker/u8:0+events_unbound08:10:202
2357675993531,3cyclictest2529165-21kworker/u8:1+flush-259:010:05:192
2357675993526,3cyclictest2554717-21kworker/u8:2+events_unbound10:20:142
2357671993531,3cyclictest2644169-21kworker/u8:3+flush-259:011:45:211
2357675993432,2cyclictest2441038-21kworker/u8:3+events_unbound08:30:182
2357675993431,2cyclictest2607911-21kworker/u8:3+events_unbound11:25:142
2357675993430,3cyclictest2394060-21kworker/u8:4+events_unbound07:55:212
2357675993330,2cyclictest2644169-21kworker/u8:3+flush-259:011:45:242
2357675993330,2cyclictest2471562-21kworker/u8:3+flush-259:010:25:242
2357675993229,2cyclictest2580961-21kworker/u8:5+events_unbound11:40:142
2357675993229,2cyclictest2394060-21kworker/u8:4+events_unbound08:15:122
2357675993229,2cyclictest2378316-21kworker/u8:3+events_unbound07:35:112
2357675993229,2cyclictest2352737-21kworker/u8:0+events_unbound08:20:252
2357675993229,2cyclictest2336819-21kworker/u8:1+events_unbound07:15:142
2357671993229,2cyclictest2471562-21kworker/u8:3+events_unbound10:30:101
2357678993125,4cyclictest2628376-21latency_hist11:30:003
2357675993128,2cyclictest2618031-21kworker/u8:1+flush-259:012:10:152
2357675993128,2cyclictest2607911-21kworker/u8:3+events_unbound11:20:132
2357675993128,2cyclictest2607911-21kworker/u8:3+events_unbound11:15:222
2357675993125,5cyclictest2602080-21latency_hist11:05:002
2357671993128,2cyclictest2352737-21kworker/u8:0+events_unbound08:25:171
2357671993127,3cyclictest2607947-21apt-get11:10:131
2357671993127,3cyclictest2430214-21kworker/u8:1+flush-259:008:40:201
2357671993127,2cyclictest2618031-21kworker/u8:1+events_unbound12:20:241
2357671993127,2cyclictest2618031-21kworker/u8:1+events_unbound12:20:231
2357668993122,6cyclictest2685455-21latency_hist12:25:010
2357668993122,6cyclictest2685455-21latency_hist12:25:010
2357675993028,2cyclictest2498150-21kworker/u8:0+flush-259:010:00:152
2357671993024,4cyclictest2613119-21/usr/sbin/munin11:15:151
2357675992924,4cyclictest2524181-21/usr/sbin/munin09:50:132
2357671992925,3cyclictest2628076-21kworker/u8:0+flush-259:011:30:221
2357668992925,3cyclictest2352737-21kworker/u8:0+events_unbound07:45:010
2357668992922,5cyclictest2373133-21latency_hist07:25:000
2357668992922,5cyclictest2373133-21latency_hist07:25:000
2357678992822,4cyclictest335557-21systemd-journal10:20:013
2357675992826,2cyclictest2461141-21kworker/u8:0+flush-259:009:05:162
2357675992825,2cyclictest2665252-21kworker/u8:4+events_unbound12:05:252
2357675992822,3cyclictest2529165-21kworker/u8:1+events_unbound10:10:202
2357671992825,2cyclictest2352737-21kworker/u8:0+flush-259:008:05:241
2357675992724,2cyclictest2654536-21kworker/u8:0+events_unbound11:55:172
2357675992724,2cyclictest2378316-21kworker/u8:3+events_unbound07:40:062
2357671992723,3cyclictest2654536-21kworker/u8:0+flush-259:011:55:151
2357671992723,3cyclictest2618031-21kworker/u8:1+events_unbound12:00:181
2357671992722,3cyclictest2685468-21kworker/u8:0+flush-259:012:30:151
2357668992721,4cyclictest2581204-21apt-get10:45:120
2357678992622,3cyclictest2352737-21kworker/u8:0+flush-259:007:25:233
2357675992624,1cyclictest2471562-21kworker/u8:3+events_unbound09:30:122
2357675992623,2cyclictest2471562-21kworker/u8:3+events_unbound09:24:182
2357675992623,2cyclictest2430214-21kworker/u8:1+events_unbound08:55:172
2357675992622,3cyclictest2471562-21kworker/u8:3+events_unbound10:30:142
2357671992623,2cyclictest2487769-21kworker/u8:1+events_unbound09:20:201
2357678992522,2cyclictest2394060-21kworker/u8:4+flush-259:008:05:163
2357675992523,2cyclictest2394060-21kworker/u8:4+events_unbound07:50:272
2357675992522,2cyclictest2580961-21kworker/u8:5+events_unbound11:10:032
2357675992522,2cyclictest2554717-21kworker/u8:2+events_unbound10:50:172
2357675992521,3cyclictest2675297-21kworker/u8:0+events_unbound12:15:152
2357671992522,2cyclictest2665252-21kworker/u8:4+events_unbound12:05:281
2357671992522,2cyclictest2352737-21kworker/u8:0+flush-259:008:25:001
2357671992519,4cyclictest335557-21systemd-journal07:40:001
2357671992518,4cyclictest2471562-21kworker/u8:3+flush-259:009:10:201
2357668992520,3cyclictest2550077-21/usr/sbin/munin10:15:130
2357668992519,4cyclictest2570814-21/usr/sbin/munin10:35:160
2357668992519,4cyclictest2456583-21apt-get08:45:100
2357675992421,2cyclictest2607269-21apt-get11:10:012
2357675992421,2cyclictest2529165-21kworker/u8:1+events_unbound09:55:172
2357675992421,2cyclictest2529165-21kworker/u8:1+events_unbound09:55:162
2357675992421,2cyclictest2430214-21kworker/u8:1+events_unbound09:00:162
2357675992420,3cyclictest2430214-21kworker/u8:1+events_unbound08:25:162
2357671992421,2cyclictest2580961-21kworker/u8:5+flush-259:011:05:211
2357671992421,2cyclictest2513320-21kworker/u8:2+flush-259:009:40:231
2357671992421,2cyclictest2471562-21kworker/u8:3+events_unbound09:35:181
2357671992420,3cyclictest2685468-21kworker/u8:0+flush-259:012:35:231
2357671992420,3cyclictest2618031-21kworker/u8:1+flush-259:012:10:221
2357671992420,3cyclictest2529165-21kworker/u8:1+flush-259:009:55:201
2357671992420,3cyclictest2529165-21kworker/u8:1+flush-259:009:55:191
2357671992420,3cyclictest2409577-21gpgv08:00:011
2357668992419,3cyclictest2394060-21kworker/u8:4+flush-259:007:55:210
2357675992320,2cyclictest2352737-21kworker/u8:0+flush-259:008:05:262
2357675992319,3cyclictest2430214-21kworker/u8:1+flush-259:008:54:592
2357671992319,3cyclictest2471562-21kworker/u8:3+events_unbound09:25:241
2357668992317,4cyclictest2633555-21latency_hist11:35:010
2357668992316,5cyclictest335557-21systemd-journal07:35:000
2357675992219,2cyclictest2580961-21kworker/u8:5+events_unbound11:35:122
2357675992219,2cyclictest2352737-21kworker/u8:0+events_unbound07:30:122
2357675992218,3cyclictest2685468-21kworker/u8:0+events_unbound12:36:172
2357671992219,2cyclictest2580961-21kworker/u8:5+events_unbound10:50:151
2357671992219,2cyclictest2471562-21kworker/u8:3+events_unbound10:25:151
2357671992219,2cyclictest2441038-21kworker/u8:3+flush-259:008:35:211
2357671992219,2cyclictest2414949-21sh08:05:001
2357671992219,2cyclictest2378316-21kworker/u8:3+events_unbound07:40:181
2357671992219,2cyclictest2352737-21kworker/u8:0+flush-259:007:45:151
2357671992218,3cyclictest2680013-21kworker/u8:3+flush-259:012:20:001
2357671992218,3cyclictest2580961-21kworker/u8:5+flush-259:011:35:181
2357671992218,3cyclictest2441038-21kworker/u8:3+events_unbound08:45:211
2357671992216,2cyclictest2591337-21kworker/u8:0+events_unbound10:55:201
2357671992215,5cyclictest335557-21systemd-journal10:25:011
2357671992215,2cyclictest2659714-21kworker/u8:2+events_unbound12:25:221
2357668992215,5cyclictest2700971-21latency_hist12:40:010
2357668992215,5cyclictest2505674-21sshd09:30:180
2357675992118,2cyclictest2441038-21kworker/u8:3+flush-259:008:45:142
2357675992118,2cyclictest2441038-21kworker/u8:3+events_unbound08:35:162
2357675992118,2cyclictest2394060-21kworker/u8:4+events_unbound07:45:292
2357671992118,2cyclictest2580961-21kworker/u8:5+events_unbound11:50:121
2357671992118,2cyclictest2498150-21kworker/u8:0+flush-259:010:00:141
2357671992118,2cyclictest2430214-21kworker/u8:1+flush-259:008:55:141
2357671992118,2cyclictest2352737-21kworker/u8:0+events_unbound08:10:231
2357678992014,5cyclictest2628076-21kworker/u8:0+flush-259:011:30:213
2357675992017,2cyclictest2580961-21kworker/u8:5+events_unbound10:58:192
2357675992017,2cyclictest2513320-21kworker/u8:2+flush-259:009:40:022
2357675992017,2cyclictest2347400-21kworker/u8:3+flush-259:007:10:052
2357671992016,3cyclictest2430214-21kworker/u8:1+flush-259:008:50:221
2357671992012,3cyclictest2430214-21kworker/u8:1+events_unbound09:00:181
2357668992014,4cyclictest2675297-21kworker/u8:0+flush-259:012:15:230
2357668992014,4cyclictest2639006-21apt-get11:40:110
2357668992014,4cyclictest2467056-21diskstats08:55:130
2357668992013,5cyclictest335557-21systemd-journal10:30:170
2357675991915,3cyclictest2680013-21kworker/u8:3+events_unbound12:22:532
2357675991915,3cyclictest2680013-21kworker/u8:3+events_unbound12:22:522
2357671991916,2cyclictest2607911-21kworker/u8:3+events_unbound11:20:231
2357671991916,2cyclictest2529165-21kworker/u8:1+events_unbound10:45:201
2357671991915,3cyclictest2373398-21apt-get07:25:101
2357671991915,2cyclictest2524632-21irqcore09:50:151
2357668991915,2cyclictest2399030-21sh07:50:000
2357668991912,5cyclictest2477106-21gpgv09:05:010
2357668991912,1cyclictest2565390-21turbostat10:30:000
2357668991911,5cyclictest2549837-21latency_hist10:15:000
2357678991815,2cyclictest2665391-21irqcore12:05:173
2357678991815,2cyclictest2379579-21kthreadcore07:30:163
2357678991811,5cyclictest2513627-21latency_hist09:40:013
2357675991815,2cyclictest2580961-21kworker/u8:5+events_unbound11:52:532
2357675991814,3cyclictest2628076-21kworker/u8:0+events_unbound11:32:412
2357675991814,3cyclictest2471562-21kworker/u8:3+events_unbound09:10:142
2357668991815,2cyclictest2435652-21cron08:25:010
2357668991813,3cyclictest60350irq/132-enp0s29f1:rx-011:25:100
235767899176,2cyclictest0-21swapper/310:10:193
2357678991714,2cyclictest2430214-21kworker/u8:1+flush-259:008:25:223
2357678991712,3cyclictest2613144-21apt-get11:15:113
2357678991712,3cyclictest2493216-21apt-get09:20:103
2357678991711,4cyclictest335557-21systemd-journal12:40:003
2357678991711,4cyclictest2607700-21cat11:10:013
2357675991713,3cyclictest2471562-21kworker/u8:3+events_unbound09:29:072
2357671991714,3cyclictest2352737-21kworker/u8:0+events_unbound07:34:271
2357671991713,3cyclictest2638521-21kworker/u8:2+events_unbound11:43:171
2357671991713,3cyclictest2394060-21kworker/u8:4+events_unbound08:34:581
2357668991714,2cyclictest60350irq/132-enp0s29f1:rx-010:50:240
2357668991713,3cyclictest2430214-21kworker/u8:1+flush-259:008:45:000
2357668991711,4cyclictest335557-21systemd-journal12:32:170
2357668991711,4cyclictest2420446-21apt-get08:10:110
2357668991711,3cyclictest2414960-21turbostat08:05:000
235766899170,15cyclictest2365366-21sshd07:15:170
235767899167,5cyclictest0-21swapper/307:55:593
2357678991613,2cyclictest2971-21cron10:00:003
2357678991613,2cyclictest2971-21cron10:00:003
2357678991613,2cyclictest0-21swapper/309:29:593
2357678991612,3cyclictest2352737-21kworker/u8:0+flush-259:007:50:153
2357678991610,4cyclictest2675057-21awk12:14:593
2357675991613,2cyclictest2487769-21kworker/u8:1+events_unbound09:36:182
235767199169,5cyclictest335557-21systemd-journal09:19:591
2357671991614,1cyclictest0-21swapper/110:45:001
2357671991613,2cyclictest2597153-21kworker/u8:1+events_unbound11:00:141
2357671991613,2cyclictest2471562-21kworker/u8:3+events_unbound10:18:171
2357671991612,3cyclictest2539722-21/usr/sbin/munin10:05:141
2357668991614,1cyclictest60350irq/132-enp0s29f1:rx-011:15:150
2357668991614,1cyclictest60350irq/132-enp0s29f1:rx-008:05:230
2357668991613,2cyclictest60350irq/132-enp0s29f1:rx-009:45:140
2357668991612,3cyclictest2471562-21kworker/u8:3+flush-259:009:35:210
2357668991611,3cyclictest2430214-21kworker/u8:1+flush-259:008:50:180
235767899151,9cyclictest2653232-21memory11:50:193
235767899151,4cyclictest2367162-21collect207:15:213
2357678991511,3cyclictest507-21in:imuxsock10:55:003
2357678991510,3cyclictest2477429-21grep09:05:003
235767899150,5cyclictest2383727-21runrttasks07:35:073
235767899150,5cyclictest2362743-21sshd07:14:253
235767899150,4cyclictest2622314-21ld11:20:203
235767899150,4cyclictest2574878-21ntpq10:35:223
2357671991512,2cyclictest2471562-21kworker/u8:3+events_unbound10:12:191
2357671991512,2cyclictest2394060-21kworker/u8:4+flush-259:007:50:241
2357671991512,2cyclictest2342127-21kworker/u8:2+events_unbound07:14:281
2357671991510,4cyclictest2477552-21gpgv09:05:011
235766899159,4cyclictest509-21rs:main0
235766899158,5cyclictest335557-21systemd-journal11:10:010
235766899158,5cyclictest335557-21systemd-journal07:15:000
2357668991514,1cyclictest60350irq/132-enp0s29f1:rx-011:10:210
2357668991514,1cyclictest60350irq/132-enp0s29f1:rx-007:26:520
2357668991513,1cyclictest60350irq/132-enp0s29f1:rx-010:55:180
2357668991513,1cyclictest60350irq/132-enp0s29f1:rx-007:38:210
2357668991511,3cyclictest2597390-21fschecks_count11:00:140
2357668991511,3cyclictest2471562-21kworker/u8:3+flush-259:009:10:180
235766899151,12cyclictest335557-21systemd-journal08:40:010
2357668991510,4cyclictest2618031-21kworker/u8:1+flush-259:012:10:180
2357678991412,1cyclictest2664704-21sh12:05:003
2357678991412,1cyclictest0-21swapper/311:00:013
2357678991410,3cyclictest2580961-21kworker/u8:5+flush-259:011:35:123
235767899140,4cyclictest2689730-21ntp_12:25:213
235767899140,4cyclictest2555244-21https10:20:113
235767899140,4cyclictest2446242-21seq08:35:053
235767899140,3cyclictest2612800-21sshd11:14:183
2357668991413,1cyclictest60350irq/132-enp0s29f1:rx-007:51:070
2357668991412,1cyclictest60350irq/132-enp0s29f1:rx-011:50:130
2357668991412,1cyclictest60350irq/132-enp0s29f1:rx-010:44:300
2357668991412,1cyclictest60350irq/132-enp0s29f1:rx-008:26:270
2357668991411,2cyclictest60350irq/132-enp0s29f1:rx-009:15:580
2357668991411,2cyclictest2498150-21kworker/u8:0+flush-259:010:00:240
2357668991410,2cyclictest60350irq/132-enp0s29f1:rx-011:20:190
235767899138,3cyclictest2368422-21grep07:20:143
235767899138,3cyclictest2368422-21grep07:20:143
2357678991311,1cyclictest0-21swapper/308:05:013
2357678991310,2cyclictest2471562-21kworker/u8:3+flush-259:009:50:003
235767899130,6cyclictest2481756-21gcc09:05:213
235767199139,3cyclictest2394060-21kworker/u8:4+events_unbound08:17:171
235767199139,3cyclictest2336819-21kworker/u8:1+flush-259:007:20:261
235767199139,3cyclictest2336819-21kworker/u8:1+flush-259:007:20:251
2357671991311,1cyclictest2367992-21grep07:20:001
2357671991310,2cyclictest2607911-21kworker/u8:3+flush-259:011:25:181
2357671991310,2cyclictest0-21swapper/109:45:181
235766899139,3cyclictest2498128-21idleruntime-cro09:24:590
235766899138,3cyclictest2529410-21apt-get09:55:130
235766899138,3cyclictest2529410-21apt-get09:55:120
235766899137,4cyclictest2544656-21latency_hist10:10:010
2357668991311,1cyclictest60350irq/132-enp0s29f1:rx-012:05:240
2357668991310,2cyclictest60350irq/132-enp0s29f1:rx-011:55:220
2357668991310,2cyclictest60350irq/132-enp0s29f1:rx-011:45:180
2357668991310,2cyclictest60350irq/132-enp0s29f1:rx-010:20:210
2357668991310,2cyclictest60350irq/132-enp0s29f1:rx-008:15:160
235767899129,2cyclictest2482991-21cpuspeed_turbos09:10:163
235767899129,2cyclictest2398617-21unixbench_singl07:45:263
235767899129,2cyclictest2389335-21irqcore07:40:163
235767899121,2cyclictest2569753-21ntpq10:30:213
235767899121,2cyclictest2435660-21munin-run08:25:003
235767899120,5cyclictest2654538-21/usr/sbin/munin11:55:243
235767899120,3cyclictest2461423-21taskset08:49:543
235767899120,3cyclictest0-21swapper/308:30:153
235767899120,2cyclictest2694944-21ntpq12:30:213
235767899120,2cyclictest2465748-21ntpq08:50:213
235766899129,2cyclictest60350irq/132-enp0s29f1:rx-012:25:120
2357668991210,1cyclictest60350irq/132-enp0s29f1:rx-012:00:070
2357668991210,1cyclictest60350irq/132-enp0s29f1:rx-009:50:210
2357668991210,1cyclictest60350irq/132-enp0s29f1:rx-009:40:100
235767899118,2cyclictest2529165-21kworker/u8:1+flush-259:010:05:163
235767899111,2cyclictest2679341-21ntpq12:15:193
235767899111,2cyclictest2517718-21memory09:40:183
235767899110,7cyclictest2643549-21users11:40:233
235766899118,2cyclictest60350irq/132-enp0s29f1:rx-009:25:140
235767899100,9cyclictest0-21swapper/309:50:143
235767899100,8cyclictest0-21swapper/310:40:113
235767899100,2cyclictest2644365-21df_inode11:45:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional