You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 05:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jul 31, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2410164995754,2cyclictest2711665-21kworker/u8:0+flush-259:000:05:001
2410169995548,4cyclictest2451932-21latency_hist19:50:013
2410169994844,3cyclictest2576621-21kworker/u8:2+flush-259:022:21:073
2410169994842,4cyclictest2684845-21kworker/u8:2+flush-259:023:40:003
2410164994845,2cyclictest2457116-21kworker/u8:0+events_unbound20:30:011
2410159994744,2cyclictest211rcuc/023:55:010
2410167994640,4cyclictest335557-21systemd-journal20:20:082
2410164994542,2cyclictest2498874-21kworker/u8:2+flush-259:020:45:241
2410164994541,3cyclictest2529707-21kworker/u8:3+flush-259:021:15:001
2410169994338,3cyclictest2737095-21gpgv00:25:003
2410164994138,2cyclictest2529707-21kworker/u8:3+flush-259:021:45:001
2410164993731,4cyclictest335557-21systemd-journal22:04:081
2410159993731,3cyclictest335557-21systemd-journal21:47:080
2410159993731,3cyclictest335557-21systemd-journal21:47:080
2410169993632,3cyclictest2721472-21kworker/u8:4+flush-259:000:20:003
2410167993631,3cyclictest2628512-21latency_hist22:40:012
2410164993529,4cyclictest2675076-21kworker/u8:1+flush-259:023:35:001
2410164993431,3cyclictest2675076-21kworker/u8:1+events_unbound00:38:111
2410164993430,3cyclictest2607228-21kworker/u8:1+events_unbound22:30:231
2410164993430,2cyclictest2576621-21kworker/u8:2+flush-259:022:20:211
2410159993431,2cyclictest2550593-21kworker/u8:1+flush-259:021:35:170
2410159993427,5cyclictest2685342-21latency_hist23:35:000
2410169993328,2cyclictest2638742-21kworker/u8:2+flush-259:022:50:113
2410164993329,2cyclictest2457116-21kworker/u8:0+events_unbound20:36:421
2410164993327,4cyclictest335557-21systemd-journal19:21:231
2410159993328,3cyclictest335557-21systemd-journal00:37:310
2410169993226,4cyclictest2597126-21latency_hist22:10:003
2410169993225,5cyclictest335557-21systemd-journal23:30:003
2410164993127,3cyclictest2654594-21apt-get23:05:141
2410164993127,3cyclictest2457116-21kworker/u8:0+events_unbound20:06:381
2410164993127,3cyclictest2421113-21kworker/u8:4+events_unbound19:40:091
2410164993124,5cyclictest335557-21systemd-journal22:30:001
2410159993127,3cyclictest2529707-21kworker/u8:3+flush-259:021:10:250
2410164993026,3cyclictest2457116-21kworker/u8:0+events_unbound20:30:091
2410164993026,3cyclictest2421113-21kworker/u8:4+events_unbound19:50:101
2410159993024,4cyclictest2540535-21apt-get21:15:130
2410169992925,3cyclictest2576621-21kworker/u8:2+flush-259:022:10:253
2410169992924,3cyclictest2607506-21gpgv22:20:003
2410164992926,2cyclictest2737020-21kworker/u8:0+flush-259:000:25:261
2410169992823,3cyclictest335557-21systemd-journal00:40:003
2410164992824,3cyclictest2695722-21kworker/u8:3+events_unbound23:55:191
241015999280,27cyclictest1-21systemd21:09:080
2410167992712,10cyclictest2457374-21/usr/sbin/munin19:55:152
2410164992723,3cyclictest2550593-21kworker/u8:1+events_unbound21:36:561
2410164992722,4cyclictest2669576-21kworker/u8:2+events_unbound23:22:141
2410159992722,3cyclictest335557-21systemd-journal20:55:080
2410167992622,2cyclictest2602565-21apt-get22:15:102
2410164992622,3cyclictest2675076-21kworker/u8:1+flush-259:000:10:011
2410164992622,3cyclictest2529960-21kworker/u8:4+events_unbound21:55:231
2410164992622,3cyclictest2519767-21kworker/u8:0+events_unbound20:56:091
2410159992622,3cyclictest2576621-21kworker/u8:2+flush-259:022:04:000
2410169992517,6cyclictest2654755-21diskstats23:05:163
2410164992522,2cyclictest2420925-21kworker/u8:2+events_unbound19:56:091
2410164992521,3cyclictest2576621-21kworker/u8:2+flush-259:022:40:001
2410164992520,3cyclictest2576621-21kworker/u8:2+flush-259:022:15:101
2410164992517,3cyclictest2659525-21kworker/u8:0+flush-259:023:25:161
2410159992522,2cyclictest2550593-21kworker/u8:1+events_unbound21:55:010
2410159992521,3cyclictest2659525-21kworker/u8:0+events_unbound23:44:090
2410159992519,4cyclictest335557-21systemd-journal23:46:080
2410169992420,3cyclictest2420925-21kworker/u8:2+flush-259:019:55:173
2410164992418,4cyclictest2529960-21kworker/u8:4+flush-259:021:25:001
2410164992418,4cyclictest2462550-21/usr/sbin/munin20:00:141
2410164992417,5cyclictest335557-21systemd-journal23:40:001
2410159992421,2cyclictest2695722-21kworker/u8:3+flush-259:000:00:150
2410159992420,3cyclictest2576621-21kworker/u8:2+events_unbound22:10:000
2410169992319,3cyclictest2529960-21kworker/u8:4+flush-259:021:10:173
2410169992318,3cyclictest2415725-21gpgv19:15:003
2410167992318,3cyclictest2716561-21apt-get00:05:132
2410167992317,5cyclictest2421113-21kworker/u8:4+flush-259:019:50:152
2410164992320,2cyclictest2421113-21kworker/u8:4+events_unbound19:30:141
2410164992320,2cyclictest2421113-21kworker/u8:4+events_unbound19:30:131
2410164992319,3cyclictest2684845-21kworker/u8:2+events_unbound23:40:241
2410164992319,3cyclictest2644226-21kworker/u8:3+events_unbound23:00:211
2410164992319,3cyclictest2638742-21kworker/u8:2+flush-259:022:55:271
2410164992319,3cyclictest2576621-21kworker/u8:2+events_unbound21:50:141
2410164992319,3cyclictest2576621-21kworker/u8:2+events_unbound21:50:131
2410159992319,3cyclictest2457116-21kworker/u8:0+events_unbound19:55:070
2410169992218,3cyclictest2659770-21apt-get23:10:123
2410169992215,5cyclictest2509160-21latency_hist20:45:013
2410169992214,5cyclictest2720535-21ntpq00:05:233
2410167992218,3cyclictest2695722-21kworker/u8:3+flush-259:000:15:012
2410164992218,3cyclictest2675076-21kworker/u8:1+flush-259:023:45:201
2410164992218,3cyclictest2529707-21kworker/u8:3+events_unbound21:29:111
2410164992218,3cyclictest2519767-21kworker/u8:0+events_unbound21:00:191
2410159992219,2cyclictest2555999-21apt-get21:30:130
2410159992219,2cyclictest2525004-21apt-get21:00:120
2410159992218,3cyclictest2659525-21kworker/u8:0+events_unbound23:17:160
2410159992218,3cyclictest2509182-21kworker/u8:1+events_unbound20:50:080
2410159992218,3cyclictest2457116-21kworker/u8:0+events_unbound20:48:090
2410169992116,3cyclictest2452172-21/usr/sbin/munin19:50:223
241016799215,5cyclictest2535357-21/usr/sbin/munin21:10:202
2410167992114,5cyclictest335557-21systemd-journal00:03:102
2410164992117,3cyclictest2659525-21kworker/u8:0+events_unbound23:15:231
2410164992117,3cyclictest2529960-21kworker/u8:4+events_unbound21:45:281
2410164992117,3cyclictest2529960-21kworker/u8:4+events_unbound21:45:281
2410164992117,3cyclictest2457116-21kworker/u8:0+flush-259:020:20:201
2410159992119,1cyclictest2607228-21kworker/u8:1+events_unbound22:22:040
2410159992118,2cyclictest2420925-21kworker/u8:2+flush-259:019:50:210
2410159992117,3cyclictest2644226-21kworker/u8:3+events_unbound23:02:130
2410159992117,3cyclictest2576621-21kworker/u8:2+events_unbound21:53:100
2410159992117,3cyclictest2576621-21kworker/u8:2+events_unbound21:53:100
2410159992117,3cyclictest2529960-21kworker/u8:4+events_unbound21:28:100
2410159992117,3cyclictest2421113-21kworker/u8:4+flush-259:019:30:150
2410159992117,3cyclictest2421113-21kworker/u8:4+flush-259:019:30:150
2410159992117,3cyclictest2420925-21kworker/u8:2+events_unbound19:20:060
2410169992014,4cyclictest2644226-21kworker/u8:3+flush-259:023:00:233
2410169992014,4cyclictest2441573-21latency_hist19:40:003
241016799205,4cyclictest2699974-21gcc23:45:202
241016799205,4cyclictest2482553-21collect220:15:212
2410167992016,3cyclictest2644226-21kworker/u8:3+flush-259:023:00:272
2410164992017,2cyclictest2576621-21kworker/u8:2+flush-259:022:05:151
2410164992016,3cyclictest2695722-21kworker/u8:3+events_unbound00:10:281
2410164992016,3cyclictest2675076-21kworker/u8:1+flush-259:023:52:251
2410164992016,3cyclictest2659732-21kworker/u8:4+events_unbound23:10:191
2410164992016,3cyclictest2472903-21apt-get20:10:111
2410164992016,3cyclictest2421113-21kworker/u8:4+events_unbound19:39:571
2410164992015,4cyclictest2478479-21apt-get20:15:111
2410159992016,3cyclictest2607228-21kworker/u8:1+flush-259:022:50:120
2410159992016,3cyclictest2587018-21kworker/u8:0+events_unbound22:17:100
2410159992016,3cyclictest2420925-21kworker/u8:2+events_unbound20:10:150
2410159992016,3cyclictest2420925-21kworker/u8:2+events_unbound19:43:100
2410159992014,4cyclictest2414475-21ntpq19:10:210
2410169991913,5cyclictest335557-21systemd-journal22:45:003
241016799194,4cyclictest2685174-21sshd23:33:092
241016799193,5cyclictest2534230-21ld21:05:192
2410164991916,2cyclictest2623400-21kworker/u8:0+flush-259:022:45:261
2410164991915,2cyclictest2519767-21kworker/u8:0+flush-259:021:05:151
2410159991916,2cyclictest2721472-21kworker/u8:4+events_unbound00:26:110
2410159991916,2cyclictest2472676-21kworker/u8:1+events_unbound20:29:080
2410159991915,3cyclictest2675076-21kworker/u8:1+flush-259:000:10:200
2410159991915,2cyclictest2675302-21apt-get23:25:140
2410169991815,2cyclictest2529960-21kworker/u8:4+flush-259:021:55:273
2410169991814,3cyclictest2509182-21kworker/u8:1+flush-259:020:55:013
2410169991813,4cyclictest2741708-21ntpq00:25:253
2410169991813,4cyclictest2695722-21kworker/u8:3+flush-259:023:55:003
2410169991813,4cyclictest2644425-21diskstats22:55:143
241016799185,3cyclictest2705030-21ntpq23:50:192
241016799184,3cyclictest2690753-21sshd23:40:082
2410167991815,2cyclictest2618223-21idleruntime-cro22:30:002
2410164991813,3cyclictest2504296-21apt-get20:40:111
2410164991812,2cyclictest2529960-21kworker/u8:4+events_unbound21:15:181
2410159991815,2cyclictest2669576-21kworker/u8:2+flush-259:023:20:210
2410159991814,3cyclictest2529960-21kworker/u8:4+flush-259:021:20:180
2410159991814,3cyclictest2457116-21kworker/u8:0+events_unbound20:15:100
2410159991814,2cyclictest60350irq/132-enp0s29f1:rx-021:40:180
2410169991711,4cyclictest2410071-21latency_hist19:10:013
241016799176,9cyclictest2514333-21latency_hist20:50:012
241016799175,4cyclictest2420025-21ldconfig19:15:232
241016799174,8cyclictest1-21systemd22:05:002
2410167991714,2cyclictest2493726-21idleruntime-cro20:30:002
2410167991713,3cyclictest2514567-21kworker/u8:4+flush-259:020:50:242
2410167991710,5cyclictest2602495-21latency_hist22:15:012
2410167991710,5cyclictest2446753-21cron19:45:002
2410164991714,2cyclictest2576621-21kworker/u8:2+flush-259:022:10:151
2410159991714,2cyclictest60350irq/132-enp0s29f1:rx-022:47:570
2410159991714,2cyclictest2484079-21irqcore20:20:170
2410159991713,3cyclictest2644226-21kworker/u8:3+events_unbound23:06:090
2410159991713,3cyclictest2607228-21kworker/u8:1+events_unbound22:59:100
2410159991713,3cyclictest2472676-21kworker/u8:1+flush-259:020:30:210
2410159991713,3cyclictest2457116-21kworker/u8:0+events_unbound20:41:440
2410159991713,3cyclictest2421113-21kworker/u8:4+events_unbound19:35:150
2410169991613,2cyclictest2613099-21kworker/u8:3+flush-259:022:31:123
2410169991612,3cyclictest3566127-21kworker/u8:1+flush-259:019:15:183
2410169991612,3cyclictest2519767-21kworker/u8:0+flush-259:021:05:133
2410169991612,3cyclictest2420925-21kworker/u8:2+flush-259:020:00:253
241016799169,6cyclictest2670514-21irqcore23:20:162
241016799169,5cyclictest2690513-21idleruntime-cro23:39:592
241016799169,5cyclictest2633654-21cron22:45:002
241016799168,6cyclictest2498444-21head20:30:242
241016799164,7cyclictest2608324-21df_inode22:20:132
241016799164,4cyclictest2747445-21chrt00:30:512
241016799164,3cyclictest832-21runrttasks23:07:202
241016799164,3cyclictest2476912-21ntpq20:10:212
241016799164,2cyclictest2570382-21ntpq21:40:222
241016799163,4cyclictest2585720-21memory21:55:202
2410167991613,2cyclictest0-21swapper/200:25:002
2410167991612,3cyclictest2420925-21kworker/u8:2+flush-259:019:25:002
2410167991610,4cyclictest2664691-21dump-pmu-power23:14:592
2410164991612,2cyclictest2420925-21kworker/u8:2+events_unbound19:27:091
2410164991611,4cyclictest2736975-21latency_hist00:25:011
2410159991614,1cyclictest60350irq/132-enp0s29f1:rx-000:15:190
2410159991613,2cyclictest60350irq/132-enp0s29f1:rx-020:05:230
2410159991612,3cyclictest2623400-21kworker/u8:0+flush-259:022:40:210
2410159991612,3cyclictest2420925-21kworker/u8:2+events_unbound20:01:350
2410159991611,4cyclictest2607228-21kworker/u8:1+events_unbound22:32:080
2410159991610,4cyclictest2732091-21/usr/sbin/munin00:20:160
241016999159,4cyclictest2618230-21latency_hist22:30:013
2410169991512,2cyclictest2638833-21idleruntime-cro22:50:003
2410169991512,2cyclictest2550593-21kworker/u8:1+flush-259:021:35:263
2410169991512,2cyclictest2431916-21irqcore19:30:193
2410169991512,2cyclictest2431916-21irqcore19:30:193
2410169991511,3cyclictest2560907-21sh21:35:003
241016799159,5cyclictest2680179-21awk23:29:592
241016799159,4cyclictest2644670-21ps22:55:172
241016799159,4cyclictest2623389-21latency_hist22:35:002
241016799159,4cyclictest2529937-21cron21:05:002
241016799159,4cyclictest2524724-21cron21:00:002
241016799158,5cyclictest2566179-21sed21:40:002
241016799158,5cyclictest2546733-21sort21:20:172
241016799158,5cyclictest2447203-21/usr/sbin/munin19:45:132
241016799158,5cyclictest2410482-21tr19:10:112
241016799157,6cyclictest832-21runrttasks00:25:142
241016799157,6cyclictest2556180-21grep21:30:142
241016799157,6cyclictest2471877-21ps20:05:232
241016799157,6cyclictest2410252-21cat19:10:012
241016799157,5cyclictest2634245-21/usr/sbin/munin22:45:162
241016799154,2cyclictest2665272-21hddtemp_smartct23:15:172
241016799154,2cyclictest2431656-21df19:30:162
241016799154,2cyclictest2431656-21df19:30:162
2410167991511,3cyclictest2420925-21kworker/u8:2+flush-259:019:25:252
2410167991511,2cyclictest335557-21systemd-journal20:40:012
2410167991510,3cyclictest2748380-21idleruntime00:35:162
2410164991512,2cyclictest2721472-21kworker/u8:4+flush-259:000:30:241
2410159991512,2cyclictest60350irq/132-enp0s29f1:rx-019:45:180
2410159991512,2cyclictest2420882-21turbostat19:20:000
2410159991511,3cyclictest2675076-21kworker/u8:1+events_unbound23:35:080
2410159991511,3cyclictest2410106-21kworker/u8:0+events_unbound19:25:080
241016999148,4cyclictest2711380-21apt-get00:00:133
2410169991411,2cyclictest2498874-21kworker/u8:2+flush-259:020:35:233
2410169991410,3cyclictest2576621-21kworker/u8:2+flush-259:022:00:133
2410169991410,3cyclictest2576621-21kworker/u8:2+flush-259:021:50:263
2410169991410,3cyclictest2576621-21kworker/u8:2+flush-259:021:50:263
2410169991410,3cyclictest2550593-21kworker/u8:1+flush-259:021:40:253
241016799149,3cyclictest2540957-21sed21:15:192
241016799148,4cyclictest2550936-21grep21:25:142
241016799148,4cyclictest2467487-21latency_hist20:05:002
241016799147,5cyclictest2731407-21head00:15:252
241016799147,5cyclictest2643089-21seq22:50:202
241016799147,5cyclictest2576756-21cpuspeed_turbos21:50:132
241016799147,5cyclictest2576756-21cpuspeed_turbos21:50:132
241016799146,6cyclictest2439534-21grep19:35:182
241016799146,5cyclictest2571481-21find21:45:112
241016799146,5cyclictest2571481-21find21:45:112
241016799144,2cyclictest2504452-21df_inode20:40:122
241016499149,3cyclictest2576621-21kworker/u8:2+events_unbound22:41:091
2410164991411,2cyclictest2695722-21kworker/u8:3+events_unbound00:15:271
2410164991410,3cyclictest3566127-21kworker/u8:1+events_unbound19:10:271
2410164991410,3cyclictest2447033-21apt-get19:45:101
2410159991412,1cyclictest60350irq/132-enp0s29f1:rx-023:55:120
2410159991411,2cyclictest2576621-21kworker/u8:2+events_unbound22:35:080
2410169991310,2cyclictest2540658-21cpuspeed_turbos21:15:153
2410169991310,1cyclictest0-21swapper/321:05:003
241016799133,3cyclictest0-21swapper/222:05:092
241016499139,3cyclictest2514567-21kworker/u8:4+flush-259:020:50:161
2410159991311,1cyclictest60350irq/132-enp0s29f1:rx-000:30:080
2410159991311,1cyclictest60350irq/132-enp0s29f1:rx-000:05:160
2410159991310,2cyclictest60350irq/132-enp0s29f1:rx-022:25:150
241016999128,3cyclictest2457116-21kworker/u8:0+flush-259:020:26:193
241016999126,4cyclictest335557-21systemd-journal21:26:083
2410164991210,1cyclictest123135-21kworker/u8:3+events_unbound19:20:001
241016999119,1cyclictest0-21swapper/323:20:173
241016999118,1cyclictest0-21swapper/323:30:233
241016999117,3cyclictest2675076-21kworker/u8:1+flush-259:000:10:263
241016999112,7cyclictest335557-21systemd-journal19:45:003
241016499117,3cyclictest2529960-21kworker/u8:4+flush-259:021:30:051
241016999107,2cyclictest2529960-21kworker/u8:4+flush-259:021:45:243
241016999107,2cyclictest2529960-21kworker/u8:4+flush-259:021:45:233
241016499106,2cyclictest2639304-21tune2fs22:50:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional