You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 14:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Apr 16, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2823799994844,3cyclictest3037486-21kworker/u8:2+events_unbound23:00:220
2823799994742,4cyclictest3130270-21kworker/u8:2+flush-259:000:10:150
2823799994741,4cyclictest252-21systemd-journal22:20:130
2823809994437,5cyclictest3104483-21latency_hist23:40:013
2823805994337,4cyclictest509-21rs:main2
2823799994234,5cyclictest3077877-21ntpq23:10:200
2823805994035,3cyclictest3114783-21latency_hist23:50:022
2823799993833,3cyclictest3037486-21kworker/u8:2+flush-259:022:50:140
2823805993731,4cyclictest3079186-21diskstats23:15:132
2823799993733,3cyclictest2927419-21kworker/u8:4+flush-259:021:00:190
2823799993731,4cyclictest509-21rs:main0
2823799993629,5cyclictest3030975-21ntpq22:25:180
2823803993532,2cyclictest3037486-21kworker/u8:2+flush-259:023:00:181
2823803993531,3cyclictest3058405-21apt-get22:55:111
2823803993531,3cyclictest3058405-21apt-get22:55:111
2823799993531,3cyclictest2891030-21kworker/u8:1+flush-259:020:25:170
2823799993529,4cyclictest3047816-21latency_hist22:45:000
282379999341,31cyclictest1-21systemd20:40:000
282379999341,31cyclictest1-21systemd20:39:590
2823809993326,5cyclictest3119898-21latency_hist23:55:013
2823805993328,3cyclictest252-21systemd-journal21:45:122
2823803993327,4cyclictest3161068-21apt-get00:35:011
2823803993227,3cyclictest252-21systemd-journal20:50:131
2823799993227,3cyclictest3006140-21kworker/u8:0+flush-259:023:20:180
2823799993225,5cyclictest2875745-21latency_hist19:59:590
2823803993124,5cyclictest252-21systemd-journal21:25:001
2823799993126,3cyclictest490-21dbus-daemon20:35:000
2823803993027,2cyclictest3006140-21kworker/u8:0+flush-259:022:50:001
2823799993027,2cyclictest3130270-21kworker/u8:2+flush-259:000:15:210
2823799993025,3cyclictest2849738-21apt-get19:35:110
2823805992925,3cyclictest2891030-21kworker/u8:1+flush-259:021:05:002
2823809992825,2cyclictest2864984-21kworker/u8:2+flush-259:020:05:163
2823805992823,4cyclictest2964417-21/usr/sbin/munin21:25:142
2823805992823,3cyclictest3099586-21/usr/sbin/munin23:35:122
2823803992824,3cyclictest3068684-21kworker/u8:4+flush-259:023:35:211
2823799992823,3cyclictest3119889-21kworker/u8:1+events_unbound00:00:130
2823799992823,3cyclictest3109871-21kworker/u8:0+events_unbound23:50:220
2823805992724,2cyclictest2823983-21kworker/u8:3+flush-259:019:50:152
2823799992723,3cyclictest3068684-21kworker/u8:4+flush-259:023:35:190
2823799992722,4cyclictest2823983-21kworker/u8:3+events_unbound19:10:170
2823799992719,5cyclictest2843539-21ntpq19:25:230
2823799992622,3cyclictest2891030-21kworker/u8:1+flush-259:020:20:170
2823799992621,3cyclictest3140527-21kworker/u8:0+flush-259:000:25:210
2823799992621,3cyclictest3119889-21kworker/u8:1+flush-259:023:55:210
2823799992621,3cyclictest3016526-21apt-get22:15:120
2823799992620,4cyclictest252-21systemd-journal20:15:000
2823799992619,5cyclictest2994277-21ntpq21:50:180
2823809992521,3cyclictest3130270-21kworker/u8:2+flush-259:000:20:003
2823803992522,2cyclictest3026611-21apt-get22:24:551
2823803992520,3cyclictest3104741-21apt-get23:40:101
2823803992518,5cyclictest2984815-21latency_hist21:45:011
2823803992517,6cyclictest3088166-21ntpq23:20:211
2823799992521,3cyclictest3140527-21kworker/u8:0+events_unbound00:20:230
2823799992519,4cyclictest509-21rs:main0
2823809992420,2cyclictest2974508-21kworker/u8:2+flush-259:021:45:173
2823803992420,3cyclictest2860084-21apt-get19:45:121
2823799992420,3cyclictest2891030-21kworker/u8:1+flush-259:020:45:180
2823799992420,3cyclictest2864984-21kworker/u8:2+events_unbound20:05:130
2823799992420,3cyclictest2849497-21kworker/u8:0+events_unbound19:45:240
2823799992419,4cyclictest3006140-21kworker/u8:0+events_unbound22:05:220
2823809992319,3cyclictest2964971-21kworker/u8:1+flush-259:021:30:003
2823809992317,4cyclictest3037895-21diskstats22:35:133
2823805992318,3cyclictest2881365-21diskstats20:05:132
2823803992318,4cyclictest2953494-21gpgv21:15:011
2823803992317,4cyclictest252-21systemd-journal19:20:141
2823799992319,3cyclictest3130270-21kworker/u8:2+flush-259:000:35:190
2823799992319,3cyclictest2964971-21kworker/u8:1+flush-259:022:35:240
2823799992319,3cyclictest2906932-21kworker/u8:0+flush-259:021:10:230
2823799992319,3cyclictest2823983-21kworker/u8:3+events_unbound19:40:140
2823799992317,5cyclictest2870778-21kworker/u8:4+events_unbound20:00:110
2823799992316,5cyclictest2943220-21/usr/sbin/munin21:05:150
2823809992218,3cyclictest2927416-21kworker/u8:2+flush-259:020:50:003
2823805992218,3cyclictest2995170-21kworker/u8:2+flush-259:021:55:172
2823799992217,3cyclictest2906932-21kworker/u8:0+flush-259:021:35:160
2823799992216,4cyclictest3140527-21kworker/u8:0+flush-259:000:34:590
2823799992214,5cyclictest3109645-21latency_hist23:45:020
282380999216,5cyclictest2872474-21cat19:55:173
282380999216,3cyclictest0-21swapper/319:50:153
282380999215,5cyclictest2936971-21ld20:55:233
2823809992117,3cyclictest2870778-21kworker/u8:4+flush-259:020:30:003
2823809992117,3cyclictest2864984-21kworker/u8:2+flush-259:020:10:163
2823809992116,4cyclictest3037486-21kworker/u8:2+flush-259:022:45:243
2823809992114,5cyclictest3031892-21apt-get22:30:003
282380599219,4cyclictest509-21rs:main2
2823805992117,3cyclictest3068684-21kworker/u8:4+flush-259:023:40:152
2823805992114,5cyclictest2833197-21ntpq19:15:212
2823803992115,5cyclictest2870787-21/usr/sbin/munin19:55:211
282380999208,10cyclictest3005772-21cat22:05:013
282380999206,4cyclictest2939615-21cut21:00:183
282380999206,4cyclictest2893106-21cut20:15:173
282380999205,4cyclictest3139665-21ntpq00:10:213
2823809992015,3cyclictest2906966-21apt-get20:30:123
2823809992012,7cyclictest3058371-21kworker/u8:3+flush-259:023:00:013
2823809992012,7cyclictest3058371-21kworker/u8:3+flush-259:023:00:003
2823805992015,4cyclictest3068684-21kworker/u8:4+flush-259:023:25:232
2823803992016,3cyclictest3006140-21kworker/u8:0+flush-259:023:20:011
2823803992016,2cyclictest0-21swapper/120:35:001
2823803992015,4cyclictest3119889-21kworker/u8:1+flush-259:000:00:191
2823799992014,4cyclictest3109871-21kworker/u8:0+events_unbound23:45:260
2823799992014,4cyclictest252-21systemd-journal22:00:130
282380999195,4cyclictest2916215-21gcc20:35:223
282380999195,4cyclictest2916215-21gcc20:35:223
2823809991916,2cyclictest0-21swapper/321:45:003
2823809991915,3cyclictest3006173-21apt-get22:05:113
2823805991915,3cyclictest252-21systemd-journal21:16:592
2823803991915,3cyclictest2849497-21kworker/u8:0+flush-259:020:00:151
2823803991914,4cyclictest2974725-21/usr/sbin/munin21:35:131
2823803991914,4cyclictest2844621-21apt-get19:30:101
282380399190,16cyclictest1-21systemd21:05:001
2823799991916,2cyclictest2989926-21kworker/u8:3+events_unbound22:10:120
2823799991915,3cyclictest2995170-21kworker/u8:2+events_unbound21:55:170
2823799991915,3cyclictest2823983-21kworker/u8:3+events_unbound19:15:230
282380999186,7cyclictest3114381-21users23:45:253
282380999185,2cyclictest2973602-21ntpq21:30:213
2823803991814,2cyclictest0-21swapper/122:30:001
2823803991812,4cyclictest252-21systemd-journal00:11:141
2823799991814,3cyclictest3068684-21kworker/u8:4+events_unbound23:30:330
2823799991814,3cyclictest2906932-21kworker/u8:0+events_unbound21:45:150
2823799991814,3cyclictest2906932-21kworker/u8:0+events_unbound20:53:140
2823799991813,4cyclictest2959253-21apt-get21:20:120
282380999176,2cyclictest3016102-21modprobe22:13:133
2823809991711,5cyclictest0-21swapper/300:25:003
2823809991711,4cyclictest3084185-21apt-get23:20:123
282380599179,5cyclictest252-21systemd-journal19:25:002
2823805991713,3cyclictest509-21rs:main2
2823805991713,3cyclictest3135624-21/usr/sbin/munin00:10:142
2823805991710,5cyclictest3021439-21latency_hist22:20:002
2823803991713,3cyclictest3006140-21kworker/u8:0+flush-259:023:11:011
2823803991710,5cyclictest252-21systemd-journal22:41:131
2823799991714,2cyclictest60350irq/132-enp0s29f1:rx-019:32:560
2823799991714,2cyclictest60350irq/132-enp0s29f1:rx-000:05:370
282380999166,8cyclictest3058199-21cat22:55:003
2823809991614,1cyclictest0-21swapper/300:34:593
2823809991612,3cyclictest2927419-21kworker/u8:4+flush-259:021:15:113
2823809991610,4cyclictest2928088-21grep20:50:153
2823805991613,2cyclictest3141177-21irqcore00:15:142
2823803991613,2cyclictest0-21swapper/100:34:591
2823803991612,3cyclictest3020609-21ntp_offset22:15:221
2823803991612,3cyclictest2891030-21kworker/u8:1+flush-259:020:20:181
2823803991610,4cyclictest3042662-21latency_hist22:40:011
2823799991614,1cyclictest60350irq/132-enp0s29f1:rx-022:45:230
2823799991614,1cyclictest60350irq/132-enp0s29f1:rx-020:15:200
2823799991613,2cyclictest60350irq/132-enp0s29f1:rx-019:50:130
2823799991613,2cyclictest3058371-21kworker/u8:3+flush-259:023:05:200
2823799991612,3cyclictest2974508-21kworker/u8:2+events_unbound21:40:140
2823799991612,3cyclictest2917271-21kworker/u8:3+events_unbound20:55:090
2823799991612,2cyclictest3083920-21idleruntime-cro23:20:000
282380999159,4cyclictest2839465-21cat19:25:123
282380999158,5cyclictest3130619-21grep00:05:113
282380999158,5cyclictest3120272-21grep23:55:123
282380999158,5cyclictest2948335-21latency_hist21:10:013
282380999157,6cyclictest3094701-21tune2fs23:30:133
282380999157,6cyclictest3077694-21ls23:10:183
282380999157,6cyclictest2877021-21sed20:00:153
282380999157,5cyclictest3021702-21/usr/sbin/munin22:20:103
282380999156,7cyclictest3079281-21perl23:15:143
282380999155,2cyclictest3155017-21ntpq00:25:213
282380999155,2cyclictest2848662-21ntp_states19:30:223
282380999154,2cyclictest3125472-21df00:00:123
282380999154,2cyclictest2963342-21ntpq21:20:233
282380999154,2cyclictest2864151-21ntp_states19:45:223
282380999154,2cyclictest2855051-21df19:40:133
282380999154,2cyclictest2850024-21fw_conntrack19:35:133
282380999154,2cyclictest2827865-21memory19:10:193
2823809991510,3cyclictest0-21swapper/323:00:173
282380599159,4cyclictest2880928-21latency_hist20:05:002
2823805991511,3cyclictest2933048-21fschecks_time20:55:152
2823805991510,4cyclictest3130270-21kworker/u8:2+flush-259:000:05:242
2823805991510,4cyclictest2823983-21kworker/u8:3+flush-259:019:25:152
2823805991510,3cyclictest3016247-21cron22:15:002
2823803991512,2cyclictest3005480-21latency_hist22:05:011
2823803991511,3cyclictest252-21systemd-journal20:20:001
2823803991510,3cyclictest509-21rs:main1
2823799991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:230
2823799991512,2cyclictest60350irq/132-enp0s29f1:rx-021:17:060
2823799991512,2cyclictest3037454-21cron22:35:000
2823799991511,3cyclictest3037486-21kworker/u8:2+events_unbound22:55:180
2823799991511,3cyclictest3037486-21kworker/u8:2+events_unbound22:55:180
282380999149,3cyclictest2995152-21turbostat.cron21:55:003
282380999148,4cyclictest3021457-21seq22:20:003
282380999148,4cyclictest2953769-21grep21:15:013
282380999148,4cyclictest252-21systemd-journal20:40:133
282380999147,5cyclictest3036631-21ntpq22:30:223
282380999147,5cyclictest2838787-21head19:20:233
282380999147,5cyclictest2829339-21cut19:15:133
282380999146,5cyclictest3068879-21df_inode23:05:143
282380999146,5cyclictest2995414-21/usr/sbin/munin21:55:123
282380999146,5cyclictest2823952-21sed19:10:003
282380999145,6cyclictest2109693-21snmpd23:44:143
282380599149,4cyclictest3161770-21apt-get00:35:122
282380599149,4cyclictest2917271-21kworker/u8:3+flush-259:020:50:242
2823805991410,3cyclictest3037486-21kworker/u8:2+flush-259:022:50:252
2823805991410,3cyclictest252-21systemd-journal20:45:002
282380399149,3cyclictest252-21systemd-journal00:18:141
282380399148,5cyclictest2989926-21kworker/u8:3+flush-259:022:09:441
2823803991411,2cyclictest2937795-21idleruntime-cro21:00:001
2823799991412,1cyclictest60350irq/132-enp0s29f1:rx-023:25:140
282380999139,3cyclictest3162717-21sed00:35:163
282380999138,3cyclictest3046905-21ntpq22:40:203
282380999137,4cyclictest3089585-21sort23:25:133
282380999136,5cyclictest2975492-21irqcore21:35:153
282380999136,5cyclictest2896886-21fschecks_time20:20:143
282380599135,7cyclictest0-21swapper/223:05:002
2823805991311,1cyclictest0-21swapper/220:50:002
2823805991310,2cyclictest2906932-21kworker/u8:0+flush-259:021:40:182
282380399139,3cyclictest3006140-21kworker/u8:0+flush-259:022:13:441
282380399139,3cyclictest2927419-21kworker/u8:4+flush-259:021:09:441
282380399139,3cyclictest2891030-21kworker/u8:1+flush-259:020:45:001
282380399137,4cyclictest3135390-21latency_hist00:10:011
282380399137,4cyclictest3094425-21/usr/sbin/munin23:30:141
2823803991310,2cyclictest2829403-21fschecks_count19:15:131
2823803991310,2cyclictest0-21swapper/121:50:001
2823799991311,1cyclictest60350irq/132-enp0s29f1:rx-019:21:490
282380599129,2cyclictest0-21swapper/200:30:002
282380599129,1cyclictest0-21swapper/223:15:002
282380599128,3cyclictest3119889-21kworker/u8:1+flush-259:000:03:192
282380399129,2cyclictest2924802-21kthreadcore20:45:161
282380399128,3cyclictest2849497-21kworker/u8:0+flush-259:019:40:171
282380399128,3cyclictest2823983-21kworker/u8:3+flush-259:019:25:261
282380399128,2cyclictest2974489-21munin-plugin-st21:35:001
282380599118,2cyclictest3027344-21irqcore22:25:142
282380599118,2cyclictest2959666-21irqcore21:20:172
282380599118,2cyclictest2870878-21cpuspeed_turbos19:55:132
282380599110,5cyclictest3150768-21awk00:25:002
282380399118,2cyclictest0-21swapper/119:35:131
282380399118,1cyclictest0-21swapper/119:55:001
282380399117,3cyclictest3037486-21kworker/u8:2+flush-259:022:50:241
282380399116,4cyclictest3068684-21kworker/u8:4+flush-259:023:10:001
282380399115,4cyclictest3145884-21apt-get00:20:131
282380399108,1cyclictest0-21swapper/120:35:131
282380399108,1cyclictest0-21swapper/120:35:131
282380399105,4cyclictest2891286-21cat20:14:591
282380399104,2cyclictest191rcu_preempt23:55:171
28238059995,3cyclictest2906932-21kworker/u8:0+flush-259:020:30:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional