You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 08:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Sep 06, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2677043994639,5cyclictest2886174-21cp22:30:001
2677039994642,2cyclictest2881122-21kworker/u8:4+flush-259:022:35:270
2677043994540,4cyclictest2859339-21latency_hist22:05:001
2677046994133,5cyclictest2687834-21cat19:20:012
2677043994136,3cyclictest2714347-21apt-get19:45:131
2677043993934,4cyclictest2838897-21apt-get21:45:111
2677043993934,3cyclictest2897115-21apt-get22:40:121
2677039993732,3cyclictest2792283-21apt-get21:00:110
2677049993620,5cyclictest335557-21systemd-journal20:25:003
2677046993629,4cyclictest2859358-21http22:05:002
2677039993629,5cyclictest335557-21systemd-journal00:35:010
2677043993532,2cyclictest2907223-21kworker/u8:2+events_unbound23:20:251
2677043993528,5cyclictest3006460-21cp00:25:011
2677043993430,2cyclictest2735428-21apt-get20:05:121
2677046993326,5cyclictest2827392-21ntpq21:30:262
2677039993226,4cyclictest2755778-21latency_hist20:25:000
2677049993125,4cyclictest2761224-21apt-get20:30:123
2677043993024,4cyclictest2958347-21ntpq23:35:201
2677039992925,3cyclictest2639346-21kworker/u8:2+flush-259:019:25:250
2677049992825,2cyclictest2922934-21apt-get23:05:133
2677043992825,2cyclictest2792040-21kworker/u8:3+events_unbound21:30:011
2677043992821,5cyclictest2869226-21ntpq22:10:221
2677049992722,3cyclictest2985357-21apt-get00:05:123
2677049992721,4cyclictest2870576-21cat22:15:013
2677039992719,5cyclictest335557-21systemd-journal21:52:030
2677049992621,4cyclictest2813042-21/usr/sbin/munin21:20:223
2677046992619,5cyclictest2734294-21ntpq20:00:202
2677046992520,4cyclictest2844044-21apt-get21:50:152
2677043992520,4cyclictest2949179-21apt-get23:30:101
2677043992518,5cyclictest335557-21systemd-journal21:45:011
2677039992522,2cyclictest2719467-21kworker/u8:3+flush-259:020:05:150
2677039992520,4cyclictest2791193-21ntpq20:55:220
2677039992519,4cyclictest2776462-21latency_hist20:45:010
2677049992419,4cyclictest2702905-21kworker/u8:1+flush-259:020:15:003
2677046992420,3cyclictest2719502-21apt-get19:50:142
2677043992419,4cyclictest335557-21systemd-journal00:23:071
2677043992419,4cyclictest2870808-21apt-get22:15:111
2677043992419,3cyclictest335557-21systemd-journal23:02:081
2677043992418,4cyclictest2766137-21latency_hist20:35:011
2677039992420,3cyclictest2639346-21kworker/u8:2+events_unbound19:40:180
2677049992319,3cyclictest2875367-21kworker/u8:1+flush-259:022:50:313
2677049992318,3cyclictest2781885-21/usr/sbin/munin20:50:223
2677039992317,4cyclictest2917483-21latency_hist23:00:000
2677049992216,4cyclictest2958997-21kworker/u8:0+flush-259:023:40:113
2677049992215,5cyclictest2876133-21diskstats22:20:143
2677046992216,4cyclictest335557-21systemd-journal22:54:552
2677043992218,3cyclictest2964467-21kworker/u8:1+flush-259:000:10:191
2677043992215,5cyclictest2963559-21ntpq23:40:201
2677049992117,3cyclictest335557-21systemd-journal22:05:033
2677049992115,4cyclictest335557-21systemd-journal21:48:033
2677046992115,4cyclictest3000673-21latency_hist00:20:002
2677039992117,3cyclictest2702905-21kworker/u8:1+flush-259:019:50:170
2677039992117,3cyclictest2687157-21kworker/u8:0+flush-259:019:20:000
2677049992014,4cyclictest2812798-21latency_hist21:20:013
2677039992017,2cyclictest2964467-21kworker/u8:1+flush-259:000:25:010
2677039992016,3cyclictest2854406-21apt-get22:00:130
2677049991915,3cyclictest2875367-21kworker/u8:1+flush-259:022:55:173
2677049991913,4cyclictest2687686-21cp19:20:003
2677046991914,4cyclictest3022044-21base6400:40:012
2677043991913,4cyclictest335557-21systemd-journal19:20:001
2677039991916,2cyclictest2886063-21idleruntime-cro22:30:000
2677039991914,3cyclictest2891679-21latency_hist22:35:020
2677049991813,4cyclictest2639346-21kworker/u8:2+flush-259:019:20:183
2677049991813,3cyclictest2990559-21apt-get00:10:123
2677049991813,3cyclictest2756050-21apt-get20:25:133
2677046991812,4cyclictest509-21rs:main2
2677043991814,3cyclictest2802416-21kworker/u8:5+flush-259:021:10:211
2677039991815,2cyclictest2907223-21kworker/u8:2+flush-259:023:30:200
2677039991815,2cyclictest2802393-21sh21:09:590
2677039991814,3cyclictest2907223-21kworker/u8:2+flush-259:000:35:260
2677039991812,4cyclictest2729750-21cp20:00:010
2677049991714,2cyclictest2897540-21irqcore22:40:173
2677049991713,3cyclictest2639346-21kworker/u8:2+flush-259:020:00:173
2677049991713,2cyclictest0-21swapper/323:54:593
2677049991710,5cyclictest2896847-21latency_hist22:40:003
2677046991714,2cyclictest2928512-21irqcore23:10:162
2677046991710,5cyclictest509-21rs:main2
2677043991715,1cyclictest0-21swapper/123:20:001
2677043991714,2cyclictest2702905-21kworker/u8:1+flush-259:019:55:171
2677043991713,3cyclictest2828523-21/usr/sbin/munin21:35:151
2677039991713,3cyclictest2639346-21kworker/u8:2+flush-259:019:30:260
2677039991712,4cyclictest2687325-21apt-get19:20:020
2677039991711,4cyclictest2853225-21ntpq21:55:220
2677049991613,2cyclictest3019272-21kthreadcore00:35:213
2677049991613,2cyclictest2875367-21kworker/u8:1+events_unbound22:25:133
2677049991611,4cyclictest2907223-21kworker/u8:2+flush-259:023:23:093
2677049991611,3cyclictest2844019-21/usr/sbin/munin21:50:173
2677049991610,4cyclictest2938130-21latency_hist23:20:003
2677046991614,1cyclictest2923356-21irqcore23:05:182
2677046991613,2cyclictest2984176-21ntp_kernel_err00:00:222
2677046991612,3cyclictest2792040-21kworker/u8:3+flush-259:021:20:202
2677046991612,3cyclictest2719467-21kworker/u8:3+flush-259:020:15:192
2677046991611,1cyclictest191rcu_preempt22:32:222
267704399168,7cyclictest0-21swapper/123:30:001
267704399168,7cyclictest0-21swapper/123:30:001
2677043991614,1cyclictest2752423-21kthreadcore20:20:181
2677043991613,2cyclictest2964467-21kworker/u8:1+flush-259:000:10:011
2677043991613,2cyclictest2907223-21kworker/u8:2+flush-259:000:15:121
2677043991613,2cyclictest2702905-21kworker/u8:1+flush-259:019:40:241
2677043991612,3cyclictest507-21in:imuxsock22:55:001
2677043991612,3cyclictest2964467-21kworker/u8:1+flush-259:023:45:121
2677043991612,3cyclictest2750408-21kworker/u8:0+flush-259:020:45:211
2677043991612,3cyclictest2719467-21kworker/u8:3+events_unbound19:50:241
2677043991612,3cyclictest2639346-21kworker/u8:2+events_unbound20:50:191
2677039991614,1cyclictest60350irq/132-enp0s29f1:rx-023:55:140
2677039991614,1cyclictest60350irq/132-enp0s29f1:rx-020:15:130
2677039991613,2cyclictest60350irq/132-enp0s29f1:rx-020:26:420
2677039991613,2cyclictest60350irq/132-enp0s29f1:rx-020:00:230
2677039991613,2cyclictest2880862-21cron22:25:010
2677039991613,2cyclictest2750408-21kworker/u8:0+events_unbound20:45:230
2677039991613,2cyclictest2744629-21ntp_kernel_pll_20:10:220
2677039991612,3cyclictest2639346-21kworker/u8:2+flush-259:019:45:260
2677039991611,4cyclictest2771333-21latency_hist20:40:000
2677049991512,2cyclictest2793303-21kthreadcore21:00:173
2677049991512,1cyclictest0-21swapper/322:05:013
2677049991511,2cyclictest3005852-21gpgv00:25:013
2677049991511,2cyclictest2755547-21kworker/u8:1+flush-259:020:35:173
267704699152,8cyclictest2942394-21ntpq23:20:222
267704699152,7cyclictest2833462-21munin-run21:40:002
2677046991513,1cyclictest0-21swapper/220:10:012
2677046991511,1cyclictest191rcu_preempt22:57:422
267704699150,4cyclictest2973915-21ntpq23:50:212
2677043991511,3cyclictest2907223-21kworker/u8:2+flush-259:000:30:161
2677043991511,3cyclictest2755547-21kworker/u8:1+flush-259:021:00:251
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-023:45:380
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:160
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-022:50:240
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-022:40:160
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-022:05:190
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:180
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-019:35:260
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-000:25:230
2677039991513,1cyclictest60350irq/132-enp0s29f1:rx-000:00:180
2677039991512,2cyclictest60350irq/132-enp0s29f1:rx-021:10:140
267704999149,3cyclictest2828276-21sh21:35:013
2677049991410,2cyclictest2886582-21base6422:30:013
2677049991410,2cyclictest0-21swapper/321:25:183
267704699148,4cyclictest0-21swapper/220:46:582
267704699147,5cyclictest2806735-21ntpq21:10:212
267704699147,4cyclictest0-21swapper/220:21:182
267704699145,1cyclictest191rcu_preempt22:25:172
267704699142,2cyclictest2948028-21ntpq23:25:212
267704699142,2cyclictest2948028-21ntpq23:25:202
267704699142,2cyclictest2686404-21ntpq19:15:202
267704699141,3cyclictest2791109-21ntpq20:55:202
2677046991411,2cyclictest2894248-21tr22:35:212
2677046991411,1cyclictest191rcu_preempt19:27:212
2677046991410,3cyclictest2729680-21gpgv20:00:002
2677046991410,2cyclictest2639346-21kworker/u8:2+flush-259:020:40:002
2677046991410,1cyclictest191rcu_preempt21:06:362
267704699140,4cyclictest3015836-21ntpq00:30:252
2677043991411,2cyclictest2979951-21idleruntime-cro00:00:001
2677043991410,3cyclictest2771600-21apt-get20:40:121
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-023:39:130
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-023:25:170
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-023:25:170
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-023:20:140
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-022:45:230
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-022:10:250
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:230
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-000:10:190
2677039991412,1cyclictest60350irq/132-enp0s29f1:rx-000:05:230
2677039991411,2cyclictest61150irq/133-enp0s29f1:tx-021:27:170
2677039991411,2cyclictest60350irq/132-enp0s29f1:rx-021:36:170
2677039991411,2cyclictest2921800-21ntp_kernel_pll_23:00:230
2677039991411,2cyclictest2639346-21kworker/u8:2+flush-259:020:50:280
267704999139,2cyclictest2965115-21irqcore23:45:153
267704999137,4cyclictest335557-21systemd-journal23:13:073
267704999137,4cyclictest2682141-21latency_hist19:15:003
2677049991310,2cyclictest2998074-21tr00:15:183
2677049991310,2cyclictest2875367-21kworker/u8:1+flush-259:022:45:313
2677049991310,2cyclictest0-21swapper/320:44:593
267704699138,1cyclictest191rcu_preempt00:10:222
267704699137,4cyclictest335557-21systemd-journal23:40:012
267704699136,1cyclictest191rcu_preempt22:15:152
267704699133,1cyclictest0-21swapper/220:30:212
267704699132,3cyclictest41-21ksoftirqd/223:45:192
267704699132,3cyclictest41-21ksoftirqd/222:45:232
267704699132,2cyclictest41-21ksoftirqd/223:35:002
267704699132,2cyclictest41-21ksoftirqd/222:40:232
267704699131,2cyclictest41-21ksoftirqd/222:20:202
267704699131,2cyclictest41-21ksoftirqd/219:40:232
267704699131,2cyclictest2811974-21ntpq21:15:232
2677046991312,1cyclictest391rcuc/220:13:262
2677046991310,2cyclictest2849628-21irqcore21:55:182
2677046991310,2cyclictest2818646-21irqcore21:25:172
2677043991310,2cyclictest2741005-21sed20:10:171
2677043991310,2cyclictest0-21swapper/122:30:211
2677043991310,2cyclictest0-21swapper/119:35:001
267703999139,3cyclictest2677415-21diskstats19:10:140
2677039991312,1cyclictest60350irq/132-enp0s29f1:rx-021:43:550
2677039991311,1cyclictest60350irq/132-enp0s29f1:rx-023:42:520
2677039991311,1cyclictest60350irq/132-enp0s29f1:rx-023:10:140
2677039991311,1cyclictest60350irq/132-enp0s29f1:rx-021:15:140
2677039991310,2cyclictest60350irq/132-enp0s29f1:rx-023:55:000
267704999128,3cyclictest2954076-21cron23:35:003
267704999126,4cyclictest3005827-21apt-get00:25:013
267704699129,2cyclictest2771717-21cpuspeed_turbos20:40:132
267704699128,1cyclictest191rcu_preempt22:05:172
267704699125,1cyclictest191rcu_preempt23:01:112
267704699125,1cyclictest191rcu_preempt21:45:242
267704699124,1cyclictest191rcu_preempt23:15:162
267704699123,6cyclictest41-21ksoftirqd/219:10:242
267704699123,1cyclictest191rcu_preempt23:55:232
267704699122,3cyclictest41-21ksoftirqd/220:50:232
267704699121,2cyclictest41-21ksoftirqd/219:30:202
267704699121,2cyclictest3010637-21ntpq00:25:232
267704699120,7cyclictest2985510-21df_inode00:05:142
267704699120,7cyclictest2792443-21diskstats21:00:142
267704699120,4cyclictest2703853-21fw_conntrack19:35:152
267704699120,2cyclictest2958376-21ntpq23:35:202
267704399129,2cyclictest2980473-21fschecks_time00:00:151
267704399121,2cyclictest2692924-21awk19:25:001
267703999129,2cyclictest60350irq/132-enp0s29f1:rx-021:20:140
2677039991210,1cyclictest60350irq/132-enp0s29f1:rx-022:16:250
267704999119,1cyclictest0-21swapper/319:40:003
267704999118,2cyclictest2984649-21unixbench_singl00:00:283
267704999118,2cyclictest2746190-21irqcore20:15:173
267704999118,2cyclictest2723971-21unixbench_multi19:50:263
267704999118,2cyclictest2711051-21kthreadcore19:40:183
267704999118,2cyclictest0-21swapper/321:05:213
267704999117,3cyclictest2849426-21fschecks_time21:55:163
267704999116,3cyclictest2838428-21sshd21:42:423
267704399119,1cyclictest0-21swapper/123:10:181
267704399119,1cyclictest0-21swapper/100:40:011
267704399115,4cyclictest2766402-21apt-get20:35:121
267704999107,2cyclictest2787171-21cpuspeed_turbos20:55:123
267704399108,1cyclictest0-21swapper/120:15:121
267704399107,2cyclictest0-21swapper/122:35:211
267704399106,3cyclictest335557-21systemd-journal23:50:081
267704399106,2cyclictest0-21swapper/119:15:001
26770499996,2cyclictest27009992sleep319:30:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional