You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Mon Jun 01, 2026 12:44:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2738719994740,5cyclictest335557-21systemd-journal11:55:241
2738714994642,3cyclictest2966674-21latency_hist10:50:010
2738724994138,2cyclictest2993327-21kworker/u8:0+flush-259:011:25:172
2738719993733,3cyclictest2831657-21kworker/u8:3+flush-259:009:55:231
2738719993428,4cyclictest335557-21systemd-journal08:35:011
2738719993428,4cyclictest2993546-21/usr/sbin/munin11:15:151
2738719993428,4cyclictest2759549-21apt-get07:30:121
2738714993327,4cyclictest335557-21systemd-journal10:05:200
2738728993222,8cyclictest3034726-21latency_hist11:55:013
2738728993222,8cyclictest2915142-21cat10:00:013
2738724993226,4cyclictest335557-21systemd-journal09:35:292
2738719993227,4cyclictest2893484-21kworker/u8:1+flush-259:010:20:271
2738719993226,4cyclictest2899082-21latency_hist09:45:011
2738719993124,5cyclictest335557-21systemd-journal11:00:231
2738714993125,4cyclictest335557-21systemd-journal09:19:180
2738719993024,4cyclictest3019425-21/usr/sbin/munin11:40:121
2738719993024,4cyclictest2743844-21latency_hist07:15:001
2738714992926,2cyclictest2738710-21kworker/u8:1+flush-259:007:15:170
2738714992925,3cyclictest2716815-21kworker/u8:0+flush-259:007:10:170
2738714992924,3cyclictest2941077-21kworker/u8:3+flush-259:010:55:240
2738728992822,5cyclictest2764403-21latency_hist07:35:003
2738724992823,4cyclictest3003759-21latency_hist11:25:012
2738714992825,2cyclictest2841990-21kworker/u8:0+flush-259:010:00:160
2738728992721,4cyclictest3081071-21latency_hist12:40:003
2738728992720,5cyclictest335557-21systemd-journal11:04:593
2738714992721,4cyclictest2847148-21latency_hist08:55:000
2738728992620,4cyclictest3055618-21apt-get12:15:113
2738724992621,3cyclictest2893966-21base6409:40:022
2738719992621,3cyclictest2801061-21apt-get08:10:111
2738719992621,3cyclictest2801061-21apt-get08:10:111
2738719992618,6cyclictest2765074-21/usr/sbin/munin07:35:141
2738714992622,2cyclictest1-21systemd07:36:440
2738714992618,5cyclictest2830834-21ntpq08:35:220
2738724992520,4cyclictest335557-21systemd-journal10:36:232
2738724992520,4cyclictest335557-21systemd-journal10:36:222
2738724992519,4cyclictest2811363-21apt-get08:20:122
2738724992514,9cyclictest509-21rs:main2
2738719992418,4cyclictest2904242-21kworker/u8:2+flush-259:009:55:001
2738719992417,5cyclictest2993305-21latency_hist11:15:001
2738714992417,5cyclictest2758462-21ntpq07:25:230
2738728992319,3cyclictest2764929-21base6407:35:013
2738728992318,3cyclictest3060761-21apt-get12:20:123
2738728992314,8cyclictest2988809-21irqcore11:10:173
2738724992318,3cyclictest335557-21systemd-journal09:49:202
2738719992319,3cyclictest2841990-21kworker/u8:0+events_unbound10:30:241
2738719992319,3cyclictest2841990-21kworker/u8:0+events_unbound10:30:241
2738719992316,5cyclictest335557-21systemd-journal09:15:011
2738714992317,4cyclictest3040342-21diskstats12:00:130
2738714992317,4cyclictest2982128-21ntpq11:00:220
2738728992219,2cyclictest2790499-21kworker/u8:3+flush-259:008:15:173
2738728992216,4cyclictest2872848-21latency_hist09:20:013
2738728992214,7cyclictest2831657-21kworker/u8:3+flush-259:009:05:223
2738719992218,3cyclictest2993327-21kworker/u8:0+flush-259:011:35:141
2738719992218,3cyclictest2941077-21kworker/u8:3+events_unbound10:55:241
2738719992218,3cyclictest2831657-21kworker/u8:3+flush-259:009:45:211
2738719992214,5cyclictest2946030-21latency_hist10:30:001
2738714992218,3cyclictest2831657-21kworker/u8:3+flush-259:009:50:230
2738714992217,3cyclictest335557-21systemd-journal09:55:170
2738714992216,4cyclictest335557-21systemd-journal11:40:390
2738724992115,4cyclictest2946459-21diskstats10:30:152
2738724992115,4cyclictest2946459-21diskstats10:30:142
2738719992116,4cyclictest2738710-21kworker/u8:1+events_unbound07:15:241
2738719992116,3cyclictest2738710-21kworker/u8:1+events_unbound07:25:171
2738714992116,3cyclictest2805994-21kworker/u8:1+flush-259:009:00:010
2738714992114,5cyclictest2893452-21latency_hist09:40:020
2738728992016,3cyclictest2775105-21kworker/u8:2+flush-259:008:40:213
2738724992016,3cyclictest2941077-21kworker/u8:3+flush-259:011:15:242
2738724992016,3cyclictest2893484-21kworker/u8:1+flush-259:010:20:172
2738724992015,3cyclictest3029549-21awk11:50:012
2738724992011,8cyclictest0-21swapper/210:30:002
2738719992017,2cyclictest3081075-21latency_hist12:40:001
2738719992017,2cyclictest3066623-21irqcore12:25:171
2738719992016,3cyclictest2951237-21kworker/u8:4+events_unbound12:00:231
2738719992015,3cyclictest3029819-21apt-get11:50:111
2738719992014,4cyclictest2887443-21ntpq09:30:201
2738714992014,4cyclictest2941112-21apt-get10:25:110
2738714992013,5cyclictest3002892-21ntpq11:20:230
2738728991915,3cyclictest335557-21systemd-journal10:40:003
2738728991915,3cyclictest335557-21systemd-journal10:39:593
2738728991915,3cyclictest2738710-21kworker/u8:1+flush-259:007:45:143
2738728991914,4cyclictest335557-21systemd-journal12:30:003
2738728991910,8cyclictest0-21swapper/309:20:163
273872499193,4cyclictest2759292-21idleruntime-cro07:29:592
273871999192,8cyclictest2920160-21munin-run10:04:591
2738719991915,3cyclictest3045289-21kworker/u8:3+events_unbound12:05:231
2738719991915,3cyclictest2738710-21kworker/u8:1+events_unbound07:45:211
2738719991914,4cyclictest2888600-21apt-get09:35:121
2738719991913,4cyclictest2780464-21/usr/sbin/munin07:50:201
2738719991912,5cyclictest2799935-21ntpq08:05:251
2738719991912,5cyclictest2799935-21ntpq08:05:241
2738719991912,5cyclictest2738657-21latency_hist07:10:011
2738719991911,6cyclictest335557-21systemd-journal07:44:591
2738714991915,2cyclictest0-21swapper/008:35:010
2738714991912,5cyclictest3065632-21latency_hist12:25:000
2738728991816,1cyclictest0-21swapper/311:45:013
2738728991815,2cyclictest2738950-21apt-get07:10:133
2738728991812,4cyclictest2925622-21apt-get10:10:093
273872899180,2cyclictest2821541-21cat08:30:003
273872499187,10cyclictest2868141-21cpuspeed_turbos09:15:142
273872499186,1cyclictest0-21swapper/207:10:202
2738719991814,3cyclictest3075928-21sh12:35:001
2738719991814,3cyclictest2862615-21kworker/u8:2+flush-259:009:25:001
2738719991814,3cyclictest2831657-21kworker/u8:3+events_unbound08:50:201
2738719991814,2cyclictest0-21swapper/110:10:001
2738719991813,4cyclictest2957134-21kworker/u8:0+flush-259:010:45:151
2738719991813,3cyclictest2893484-21kworker/u8:1+events_unbound10:10:191
273871999181,15cyclictest1-21systemd08:02:441
2738714991814,3cyclictest2831657-21kworker/u8:3+flush-259:009:45:260
2738714991814,2cyclictest1-21systemd09:21:190
273872899174,11cyclictest2996006-21cut11:15:173
2738728991714,2cyclictest0-21swapper/307:19:593
2738728991713,3cyclictest507-21in:imuxsock12:10:003
2738728991711,4cyclictest335557-21systemd-journal11:49:263
273872899171,14cyclictest2759276-21apt07:29:593
2738728991710,6cyclictest3014000-21sh11:35:003
273872499176,1cyclictest0-21swapper/211:35:162
2738724991714,2cyclictest0-21swapper/210:45:002
273872499170,3cyclictest3076875-21sed12:35:152
2738719991714,2cyclictest2941077-21kworker/u8:3+events_unbound10:50:181
2738719991714,2cyclictest2775105-21kworker/u8:2+flush-259:008:20:271
2738719991713,3cyclictest2993327-21kworker/u8:0+events_unbound11:25:231
2738719991713,3cyclictest2805994-21kworker/u8:1+events_unbound09:05:131
2738719991712,4cyclictest3034966-21kworker/u8:0+events_unbound12:10:221
2738719991712,4cyclictest2716815-21kworker/u8:0+flush-259:007:25:001
2738714991714,2cyclictest2749295-21apt-get07:20:110
2738714991713,2cyclictest60350irq/132-enp0s29f1:rx-011:05:130
2738714991712,3cyclictest2775105-21kworker/u8:2+flush-259:008:30:010
2738714991711,4cyclictest2867754-21latency_hist09:15:010
273872899167,8cyclictest0-21swapper/308:00:003
2738728991612,3cyclictest2925592-21kworker/u8:4+flush-259:010:15:163
273872899161,1cyclictest2924476-21ntpq10:05:203
273872899161,14cyclictest2946018-21cron10:30:003
273872899161,13cyclictest335557-21systemd-journal10:50:003
273872899160,2cyclictest3054959-21users12:10:243
273872899160,2cyclictest3014485-21fschecks_time11:35:123
273872899160,2cyclictest1-21systemd12:35:003
273872899160,1cyclictest0-21swapper/308:45:153
273872499166,1cyclictest0-21swapper/209:55:192
2738724991612,2cyclictest2967117-21latency_hist10:50:002
2738724991611,4cyclictest335557-21systemd-journal08:50:012
273872499160,8cyclictest2930557-21basename10:15:002
2738719991612,3cyclictest2998727-21kworker/u8:1+flush-259:011:45:211
2738719991612,3cyclictest2816450-21kworker/u8:0+flush-259:008:25:211
2738714991613,2cyclictest60350irq/132-enp0s29f1:rx-011:31:180
2738714991613,2cyclictest60350irq/132-enp0s29f1:rx-010:14:590
2738714991613,2cyclictest0-21swapper/008:10:010
2738714991613,2cyclictest0-21swapper/008:10:000
2738714991612,3cyclictest60350irq/132-enp0s29f1:rx-012:10:110
2738714991612,3cyclictest2775105-21kworker/u8:2+flush-259:007:50:160
2738714991612,2cyclictest0-21swapper/011:30:010
2738714991611,4cyclictest335557-21systemd-journal09:08:180
2738714991610,4cyclictest335557-21systemd-journal07:56:450
273872899157,6cyclictest0-21swapper/308:50:133
273872899155,9cyclictest490-21dbus-daemon11:25:013
273872899154,9cyclictest2801283-21fschecks_time08:10:153
273872899154,9cyclictest2801283-21fschecks_time08:10:143
273872899153,11cyclictest2894887-21sed09:40:163
273872899152,11cyclictest2971814-21grep10:50:243
273872899152,10cyclictest0-21swapper/307:45:003
273872899151,3cyclictest2856566-21ntpq09:00:223
273872899151,2cyclictest3039079-21ntpq11:55:233
273872899151,1cyclictest2939987-21ntpq10:20:223
273872899151,1cyclictest2784467-21ntpq07:50:213
273872899151,1cyclictest0-21swapper/311:05:203
2738728991511,2cyclictest2972567-21cpuspeed_turbos10:55:143
273872899150,5cyclictest2913658-21ntpq09:55:233
273872899150,4cyclictest2863663-21sed09:10:153
273872899150,3cyclictest2799935-21ntpq08:05:243
273872899150,3cyclictest2799935-21ntpq08:05:243
273872899150,3cyclictest2792343-21tr08:00:173
273872899150,1cyclictest2815394-21ntpq08:20:233
273872499159,4cyclictest2765026-21cat07:35:002
273872499154,3cyclictest2992372-21nfs_client11:10:202
273872499151,5cyclictest2753230-21ntpq07:20:202
2738724991512,2cyclictest2967783-21sed10:50:162
2738724991512,2cyclictest2920830-21irqcore10:05:162
273872499150,2cyclictest2831668-21latency_hist08:40:012
2738719991511,3cyclictest3055380-21kworker/u8:1+flush-259:012:20:141
2738714991513,2cyclictest60350irq/132-enp0s29f1:rx-009:42:320
2738714991513,1cyclictest60350irq/132-enp0s29f1:rx-010:44:070
2738714991513,1cyclictest60350irq/132-enp0s29f1:rx-010:22:160
2738714991513,1cyclictest60350irq/132-enp0s29f1:rx-009:25:210
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-012:30:110
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-011:50:130
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-011:15:190
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-009:00:210
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-008:40:120
2738714991512,2cyclictest60350irq/132-enp0s29f1:rx-008:00:250
273871499150,14cyclictest1-21systemd08:13:430
273871499150,14cyclictest1-21systemd08:13:430
273872899145,7cyclictest0-21swapper/309:50:133
273872899144,9cyclictest2829475-21tr08:35:193
273872899142,10cyclictest2949960-21cut10:30:203
273872899142,10cyclictest2949960-21cut10:30:193
273872899141,2cyclictest2882296-21ntpq09:25:213
273872899141,1cyclictest2753266-21ntpq07:20:223
273872899141,11cyclictest2825662-21ntpq08:30:203
2738728991410,2cyclictest0-21swapper/309:00:003
273872899140,4cyclictest3004144-21tr11:25:143
273872899140,3cyclictest2899758-21tr09:45:163
273872899140,2cyclictest335557-21systemd-journal12:05:013
273872899140,2cyclictest2893446-21basename09:40:003
273872899140,1cyclictest2960632-21grep10:40:213
273872499146,2cyclictest3049848-21users12:05:262
2738724991412,1cyclictest2805994-21kworker/u8:1+flush-259:008:30:172
2738724991410,3cyclictest2716815-21kworker/u8:0+flush-259:007:30:052
2738719991412,1cyclictest61650irq/143-enp0s29f1:tx-509:27:551
273871999141,10cyclictest335557-21systemd-journal08:20:011
2738719991410,3cyclictest2816450-21kworker/u8:0+events_unbound08:35:151
2738714991413,1cyclictest60350irq/132-enp0s29f1:rx-011:59:400
2738714991412,2cyclictest60350irq/132-enp0s29f1:rx-012:35:140
2738714991412,2cyclictest60350irq/132-enp0s29f1:rx-008:48:030
2738714991412,1cyclictest60350irq/132-enp0s29f1:rx-011:45:240
2738714991412,1cyclictest60350irq/132-enp0s29f1:rx-011:10:230
2738714991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:190
2738714991412,1cyclictest60350irq/132-enp0s29f1:rx-007:45:160
2738714991411,2cyclictest60350irq/132-enp0s29f1:rx-008:20:160
2738714991411,2cyclictest60350irq/132-enp0s29f1:rx-007:30:140
2738714991411,2cyclictest2946470-21fschecks_count10:30:150
2738714991411,2cyclictest2946470-21fschecks_count10:30:150
273872499139,3cyclictest335557-21systemd-journal12:15:432
273872499139,3cyclictest2775105-21kworker/u8:2+flush-259:008:40:162
273872499131,5cyclictest2857864-21df09:05:122
2738724991311,1cyclictest0-21swapper/211:05:162
2738724991311,1cyclictest0-21swapper/209:15:012
2738724991310,2cyclictest2790818-21cpuspeed_turbos08:00:132
2738719991310,2cyclictest2983467-21fschecks_count11:05:141
2738719991310,2cyclictest2956705-21cpuspeed_turbos10:40:121
2738714991312,1cyclictest60350irq/132-enp0s29f1:rx-011:35:440
2738714991312,1cyclictest60350irq/132-enp0s29f1:rx-009:30:190
2738714991311,2cyclictest60350irq/132-enp0s29f1:rx-007:42:540
2738714991311,1cyclictest60350irq/132-enp0s29f1:rx-010:50:180
2738714991311,1cyclictest60350irq/132-enp0s29f1:rx-010:35:160
2738714991311,1cyclictest60350irq/132-enp0s29f1:rx-010:35:150
2738714991310,2cyclictest60350irq/132-enp0s29f1:rx-012:25:120
2738714991310,2cyclictest60350irq/132-enp0s29f1:rx-012:06:220
273872499129,2cyclictest3040305-21cpuspeed_turbos12:00:122
273872499129,2cyclictest2844546-21kthreadcore08:50:162
273872499126,4cyclictest2935688-21latency_hist10:20:002
273872499125,5cyclictest335557-21systemd-journal09:30:012
273872499121,9cyclictest2920165-21awk10:04:592
273871999129,1cyclictest0-21swapper/110:15:121
273871999129,1cyclictest0-21swapper/108:50:011
273871999128,2cyclictest0-21swapper/109:00:201
2738714991211,1cyclictest60350irq/132-enp0s29f1:rx-012:17:480
2738714991211,1cyclictest60350irq/132-enp0s29f1:rx-008:15:360
273872499119,1cyclictest0-21swapper/211:00:182
273872499118,2cyclictest2873536-21irqcore09:20:142
273872499118,2cyclictest2748574-21unixbench_multi07:15:262
273872499117,3cyclictest3034966-21kworker/u8:0+flush-259:012:30:002
273872499117,2cyclictest0-21swapper/208:10:152
273872499117,2cyclictest0-21swapper/208:10:142
273872499110,4cyclictest2779345-21ntpq07:45:222
273872499110,10cyclictest2971-21cron09:54:592
273871999117,3cyclictest2993327-21kworker/u8:0+flush-259:011:30:171
273871999117,3cyclictest0-21swapper/108:45:001
273871999116,3cyclictest3044213-21kworker/u8:2+flush-259:012:15:201
273872499107,2cyclictest3013076-21ntp_kernel_err11:30:212
273872499104,5cyclictest2851462-21ntpq08:55:212
273872499104,4cyclictest2806509-21fw_forwarded_lo08:15:152
273871999108,1cyclictest0-21swapper/109:15:131
273871999107,1cyclictest0-21swapper/107:55:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional