You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 05:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed May 20, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2550160994139,1cyclictest2773805-21fwupd22:44:053
2550156994137,3cyclictest2815366-21gpgv23:25:002
2550156994130,9cyclictest2639235-21/usr/sbin/munin20:35:192
2550160993832,4cyclictest2670340-21diskstats21:05:143
2550156993833,3cyclictest2586403-21kworker/u8:2+flush-259:019:55:152
2550154993731,4cyclictest335557-21systemd-journal20:47:571
2550160993629,5cyclictest2731011-21ntpq22:00:233
2550154993632,3cyclictest2618581-21kworker/u8:2+flush-259:020:25:171
2550156993529,4cyclictest2591992-21cp19:50:012
2550156993529,4cyclictest2591992-21cp19:50:012
2550154993530,3cyclictest2711440-21kworker/u8:4+flush-259:021:50:001
2550150993431,2cyclictest2649175-21kworker/u8:0+flush-259:021:51:030
2550150993430,3cyclictest335557-21systemd-journal20:18:570
2550154993327,4cyclictest2737657-21cp22:10:011
2550154993126,4cyclictest2613198-21kworker/u8:1+flush-259:020:45:011
2550154993126,3cyclictest2597437-21base6419:55:011
2550150993125,4cyclictest2825163-21ntpq23:30:200
2550156993025,4cyclictest2835474-21ntpq23:40:212
2550156993023,5cyclictest2591636-21cat19:50:002
2550156993016,8cyclictest2893253-21idleruntime-cro00:40:012
2550150993025,4cyclictest2597024-21cp19:55:000
2550150993025,4cyclictest2597024-21cp19:54:590
2550156992925,3cyclictest2565566-21kworker/u8:3+flush-259:021:30:012
2550154992923,4cyclictest2695696-21latency_hist21:30:011
2550156992821,5cyclictest2575874-21latency_hist19:35:012
2550154992825,2cyclictest2846343-21kworker/u8:1+flush-259:023:55:141
2550150992825,2cyclictest2862247-21kworker/u8:0+flush-259:000:15:180
2550160992724,2cyclictest2763211-21kworker/u8:1+events_unbound22:55:163
2550150992721,4cyclictest2847071-21diskstats23:55:110
2550160992620,4cyclictest2841493-21latency_hist23:50:013
2550160992620,4cyclictest2706055-21latency_hist21:40:003
2550160992619,5cyclictest335557-21systemd-journal22:55:003
2550150992621,4cyclictest2618581-21kworker/u8:2+flush-259:020:20:240
2550150992620,4cyclictest335557-21systemd-journal19:44:570
2550160992522,2cyclictest2862247-21kworker/u8:0+flush-259:000:15:013
2550160992521,3cyclictest2596942-21http19:55:003
2550160992521,3cyclictest2596942-21http19:55:003
255015699256,17cyclictest2618355-21awk20:15:002
255015699256,17cyclictest2618355-21awk20:15:002
2550154992521,3cyclictest2618581-21kworker/u8:2+events_unbound20:15:231
2550154992521,2cyclictest2726957-21kworker/u8:3+events_unbound00:00:191
2550154992519,4cyclictest2877726-21latency_hist00:25:011
2550154992519,4cyclictest2721806-21apt-get21:55:111
2550150992521,3cyclictest2804628-21kworker/u8:0+events_unbound23:20:140
2550150992521,3cyclictest2726957-21kworker/u8:3+flush-259:022:30:170
2550150992521,3cyclictest2711440-21kworker/u8:4+events_unbound21:45:190
2550150992521,3cyclictest2649175-21kworker/u8:0+events_unbound21:25:190
2550150992519,4cyclictest2675082-21systemd-run21:10:010
2550150992518,5cyclictest2752864-21latency_hist22:25:000
2550160992417,5cyclictest2825144-21ntpq23:30:213
2550160992417,5cyclictest2560435-21latency_hist19:20:003
2550154992421,2cyclictest2596927-21idleruntime-cro19:55:001
2550154992421,2cyclictest2596927-21idleruntime-cro19:55:001
2550150992420,3cyclictest2726957-21kworker/u8:3+events_unbound23:50:210
2550160992319,3cyclictest2862247-21kworker/u8:0+flush-259:000:15:153
2550160992319,3cyclictest2851896-21kworker/u8:2+flush-259:000:25:173
2550160992316,5cyclictest2768330-21latency_hist22:40:003
2550160992316,4cyclictest335557-21systemd-journal23:01:083
2550160992316,4cyclictest2550538-21diskstats19:10:163
2550156992317,4cyclictest335557-21systemd-journal00:27:072
2550154992319,3cyclictest2804628-21kworker/u8:0+events_unbound23:20:181
2550150992319,3cyclictest2851896-21kworker/u8:2+events_unbound00:25:210
2550150992319,3cyclictest2784464-21kworker/u8:2+events_unbound23:00:040
2550150992319,3cyclictest2565566-21kworker/u8:3+events_unbound20:30:230
2550150992319,3cyclictest2565566-21kworker/u8:3+events_unbound19:45:160
2550150992318,3cyclictest2768358-21gpgv22:40:000
2550156992213,8cyclictest2690528-21idleruntime-cro21:25:012
2550154992218,3cyclictest2690539-21kworker/u8:1+flush-259:022:20:171
2550154992218,3cyclictest2586404-21kworker/u8:4+events_unbound19:49:591
2550154992216,4cyclictest2585245-21ntpq19:40:201
2550150992219,2cyclictest2810089-21kworker/u8:1+events_unbound23:35:150
2550150992217,4cyclictest2753105-21kworker/u8:2+events_unbound22:25:180
2550150992215,5cyclictest335557-21systemd-journal20:44:560
2550160992117,2cyclictest2680194-21sh21:15:003
2550160992114,5cyclictest2618348-21latency_hist20:15:013
2550160992114,5cyclictest2618348-21latency_hist20:15:003
2550156992114,5cyclictest335557-21systemd-journal21:16:022
2550154992117,3cyclictest2649175-21kworker/u8:0+flush-259:021:50:141
2550154992117,2cyclictest2851896-21kworker/u8:2+events_unbound00:05:301
2550154992116,4cyclictest2564661-21ntpq19:20:191
255015499210,20cyclictest1-21systemd22:05:001
2550150992118,2cyclictest2726957-21kworker/u8:3+flush-259:023:05:210
2550150992118,2cyclictest2654459-21kworker/u8:2+flush-259:021:10:250
2550150992116,4cyclictest2654459-21kworker/u8:2+flush-259:021:40:260
2550150992114,5cyclictest2778243-21ntpq22:45:210
2550160992014,5cyclictest2574953-21ntpq19:30:193
2550156992017,2cyclictest509-21rs:main2
2550156992014,4cyclictest2737876-21diskstats22:10:132
2550156992014,4cyclictest2664757-21latency_hist21:00:012
2550154992017,2cyclictest2773570-21kworker/u8:0+flush-259:022:50:271
2550154992017,2cyclictest2763211-21kworker/u8:1+flush-259:022:55:181
2550154992016,3cyclictest2810089-21kworker/u8:1+flush-259:023:30:211
2550154992016,3cyclictest2726957-21kworker/u8:3+flush-259:023:15:051
2550154992016,3cyclictest2565566-21kworker/u8:3+flush-259:019:25:201
2550154992014,5cyclictest2819939-21ntpq23:25:211
2550150992017,2cyclictest2726397-21kworker/u8:2+flush-259:022:08:080
2550150992016,3cyclictest2763211-21kworker/u8:1+events_unbound23:11:070
2550150992016,3cyclictest2613198-21kworker/u8:1+events_unbound20:35:180
2550150992016,3cyclictest2602714-21kworker/u8:4+events_unbound20:05:000
2550150992016,3cyclictest2565566-21kworker/u8:3+events_unbound21:30:040
2550150992016,3cyclictest2565566-21kworker/u8:3+events_unbound21:15:190
2550150992015,4cyclictest2726957-21kworker/u8:3+events_unbound23:43:070
2550150992014,4cyclictest2836744-21diskstats23:45:140
2550160991913,4cyclictest2613198-21kworker/u8:1+flush-259:020:35:153
2550160991911,5cyclictest2654436-21latency_hist20:50:003
255015699198,1cyclictest2669905-21idleruntime-cro21:05:002
2550156991914,4cyclictest335557-21systemd-journal20:30:002
255015699190,3cyclictest2851782-21cron00:00:002
2550154991916,2cyclictest2654459-21kworker/u8:2+events_unbound21:10:181
2550154991915,3cyclictest2862247-21kworker/u8:0+flush-259:000:30:251
2550154991914,4cyclictest2685620-21/usr/sbin/munin21:20:221
2550154991913,4cyclictest2644155-21latency_hist20:40:011
2550150991916,2cyclictest2586403-21kworker/u8:2+flush-259:019:55:220
2550150991916,2cyclictest2565566-21kworker/u8:3+events_unbound20:10:200
2550150991916,2cyclictest2565566-21kworker/u8:3+events_unbound20:10:200
2550150991915,3cyclictest2613198-21kworker/u8:1+events_unbound20:54:590
255015699188,9cyclictest0-21swapper/223:50:222
255015699185,3cyclictest1-21systemd00:20:002
255015699183,4cyclictest2789404-21latency_hist23:00:012
2550156991810,6cyclictest2746839-21ntpq22:15:222
2550154991815,2cyclictest2842167-21irqcore23:50:151
2550154991814,3cyclictest2726957-21kworker/u8:3+flush-259:000:10:201
2550150991815,2cyclictest509-21rs:main0
2550150991815,2cyclictest2862247-21kworker/u8:0+events_unbound00:30:240
2550150991815,2cyclictest2613198-21kworker/u8:1+events_unbound20:25:220
2550150991815,2cyclictest2555479-21kworker/u8:1+flush-259:019:29:020
2550150991814,3cyclictest2726957-21kworker/u8:3+events_unbound23:26:130
2550150991814,3cyclictest2649175-21kworker/u8:0+events_unbound21:59:030
255016099179,5cyclictest2586175-21latency_hist19:45:013
255015699175,4cyclictest335557-21systemd-journal23:11:102
255015699174,4cyclictest2554486-21wc19:10:262
2550156991713,3cyclictest513-21gdbus19:20:002
2550156991711,4cyclictest2763616-21diskstats22:35:142
2550154991714,2cyclictest2826706-21irqcore23:35:161
2550154991714,2cyclictest0-21swapper/100:30:001
2550154991713,3cyclictest2565566-21kworker/u8:3+flush-259:019:35:191
2550154991713,3cyclictest2550105-21kworker/u8:0+events_unbound19:10:301
2550150991714,2cyclictest2862247-21kworker/u8:0+flush-259:000:20:210
2550150991714,2cyclictest2753105-21kworker/u8:2+flush-259:022:41:090
2550150991713,3cyclictest2726957-21kworker/u8:3+events_unbound23:15:260
2550150991713,3cyclictest2690539-21kworker/u8:1+flush-259:022:02:080
2550150991713,3cyclictest2596825-21kworker/u8:0+events_unbound20:00:270
2550160991614,1cyclictest2763175-21cron22:35:003
2550160991613,1cyclictest0-21swapper/322:30:003
2550160991612,2cyclictest0-21swapper/322:20:003
255016099161,14cyclictest2591347-21idleruntime-cro19:50:003
255015699165,6cyclictest2628660-21munin-run20:25:002
2550156991614,1cyclictest0-21swapper/219:45:002
2550156991613,2cyclictest2971-21cron22:45:002
2550156991612,3cyclictest2649957-21runrttasks20:45:152
2550154991613,2cyclictest2655100-21irqcore20:50:161
2550154991613,2cyclictest2603768-21tr20:00:171
2550154991612,3cyclictest2877744-21kworker/u8:3+events_unbound00:35:151
2550154991612,3cyclictest2773570-21kworker/u8:0+events_unbound22:46:091
2550154991612,3cyclictest2565566-21kworker/u8:3+events_unbound21:30:241
2550154991612,3cyclictest2565566-21kworker/u8:3+events_unbound20:30:151
2550154991610,4cyclictest2836322-21latency_hist23:45:021
2550150991614,1cyclictest60350irq/132-enp0s29f1:rx-019:10:180
2550150991613,2cyclictest60350irq/132-enp0s29f1:rx-022:13:080
2550150991613,2cyclictest60350irq/132-enp0s29f1:rx-000:13:320
2550150991613,2cyclictest2613198-21kworker/u8:1+flush-259:020:55:220
2550150991613,2cyclictest2550105-21kworker/u8:0+events_unbound19:15:190
2550150991612,3cyclictest2726957-21kworker/u8:3+events_unbound00:05:200
2550150991612,3cyclictest2690539-21kworker/u8:1+events_unbound21:37:030
2550150991610,4cyclictest2669912-21latency_hist21:05:010
2550160991512,2cyclictest2814352-21ntp_kernel_pll_23:20:213
2550160991512,2cyclictest0-21swapper/321:05:003
2550160991511,3cyclictest2654459-21kworker/u8:2+flush-259:021:20:163
255015699159,4cyclictest2784230-21latency_hist22:55:012
255015699155,2cyclictest0-21swapper/200:05:232
2550156991513,1cyclictest0-21swapper/222:50:002
2550156991512,2cyclictest0-21swapper/200:15:002
2550156991511,3cyclictest2690539-21kworker/u8:1+events_unbound22:20:142
2550154991511,3cyclictest335557-21systemd-journal22:36:041
2550150991513,1cyclictest60350irq/132-enp0s29f1:rx-019:30:230
2550150991513,1cyclictest60350irq/132-enp0s29f1:rx-019:20:230
2550150991512,2cyclictest60350irq/132-enp0s29f1:rx-021:20:150
2550150991511,2cyclictest2747731-21turbostat22:20:010
255016099149,3cyclictest335557-21systemd-journal20:59:553
255016099147,5cyclictest335557-21systemd-journal23:10:003
2550160991410,3cyclictest2737716-21apt-get22:10:113
2550160991410,3cyclictest2566070-21fschecks_time19:25:143
255015699147,5cyclictest2602908-21diskstats20:00:132
2550156991411,2cyclictest2805625-21irqcore23:15:192
2550156991411,2cyclictest2680195-21sh21:14:592
255015099149,3cyclictest60350irq/132-enp0s29f1:rx-020:45:120
2550150991413,1cyclictest60350irq/132-enp0s29f1:rx-000:36:140
2550150991411,2cyclictest60350irq/132-enp0s29f1:rx-022:55:190
255016099138,4cyclictest2649175-21kworker/u8:0+flush-259:021:50:253
255016099137,4cyclictest2711452-21/usr/sbin/munin21:45:173
255016099136,5cyclictest2580989-21latency_hist19:40:003
2550160991311,1cyclictest0-21swapper/300:30:223
255016099131,10cyclictest2649269-21awk20:45:003
2550160991310,2cyclictest2619013-21irqcore20:15:173
2550160991310,2cyclictest0-21swapper/323:10:153
255015499139,3cyclictest2617418-21ntp_kernel_err20:10:221
255015499139,3cyclictest2617418-21ntp_kernel_err20:10:221
255015499139,3cyclictest2613198-21kworker/u8:1+events_unbound20:59:561
2550154991310,2cyclictest2789740-21cpuspeed_turbos23:00:121
2550154991310,2cyclictest2763211-21kworker/u8:1+flush-259:023:10:151
2550154991310,2cyclictest2550105-21kworker/u8:0+flush-259:019:15:211
2550150991311,1cyclictest60350irq/132-enp0s29f1:rx-000:00:310
255016099126,3cyclictest2633823-21cron20:30:003
255015699129,2cyclictest2675039-21cron21:10:002
255015699127,3cyclictest2576297-21diskstats19:35:152
255015699120,10cyclictest2737269-21cat22:10:002
255015499128,3cyclictest2753105-21kworker/u8:2+events_unbound22:30:121
255015499126,4cyclictest2732154-21apt-get22:05:131
255015099128,3cyclictest2773570-21kworker/u8:0+flush-259:022:50:210
255016099119,1cyclictest2603762-21kthreadcore20:00:173
255016099118,2cyclictest0-21swapper/321:40:133
255016099117,3cyclictest2852101-21kworker/u8:4+flush-259:000:02:383
255016099115,4cyclictest335557-21systemd-journal23:45:003
255016099110,7cyclictest1-21systemd00:00:003
255015699118,2cyclictest2620907-21kthreadcore20:15:192
255015699118,1cyclictest0-21swapper/221:35:242
255015699114,5cyclictest2726729-21apt22:00:002
255015499118,2cyclictest2654459-21kworker/u8:2+flush-259:021:40:211
255015499116,4cyclictest335557-21systemd-journal22:20:001
255016099105,4cyclictest491ktimers/321:25:113
255016099104,4cyclictest335557-21systemd-journal00:09:593
255015699107,2cyclictest2888083-21idleruntime-cro00:35:002
255015699105,3cyclictest335557-21systemd-journal23:32:062
255015699104,4cyclictest490-21dbus-daemon23:50:002
255015699100,8cyclictest2565566-21kworker/u8:3+flush-259:019:30:002
25501569995,3cyclictest2564555-21cut19:20:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional