You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 23:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Tue Mar 17, 2026 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2924807995040,7cyclictest3033625-21grep12:20:231
2924807994638,4cyclictest2978639-21munin-plugin-st09:45:011
2924816994538,5cyclictest3023624-21kworker/u8:0+events_unbound12:35:013
2924816994538,5cyclictest3023624-21kworker/u8:0+events_unbound12:35:003
2924816994531,11cyclictest391ktimers/308:35:123
2924807994538,4cyclictest2971820-21munin-plugin-st09:25:001
2924807994518,9cyclictest3000861-21seq10:45:491
2924807994517,10cyclictest2968045-21chrt09:11:011
2924816994432,8cyclictest3018496-21sendmail11:40:003
2924816994431,9cyclictest391ktimers/310:25:143
2924812994433,6cyclictest3025517-21munin-plugin-st12:00:002
2924812994433,6cyclictest3025517-21munin-plugin-st11:59:592
2924807994435,5cyclictest3004640-21apt-get11:00:001
2924803994432,8cyclictest2936810-21grep07:40:250
2924812994330,10cyclictest3005664-21/usr/sbin/munin11:00:272
2924812994329,11cyclictest2936647-21/usr/sbin/munin07:40:252
2924812994328,9cyclictest0-21swapper/210:10:102
2924812994328,9cyclictest0-21swapper/210:10:102
2924807994324,15cyclictest2938506-21unixbench_multi07:45:261
2924812994220,11cyclictest0-21swapper/207:25:132
2924816994134,5cyclictest3005074-21kworker/u8:2+events_unbound11:09:493
2924816994130,6cyclictest2978178-21grep09:40:263
2924812994128,10cyclictest3035267-21/usr/sbin/munin12:25:262
2924812994128,10cyclictest3035267-21/usr/sbin/munin12:25:262
2924812994128,10cyclictest3027910-21/usr/sbin/munin12:05:202
2924812994128,10cyclictest2996569-21/usr/sbin/munin10:35:182
2924812994127,11cyclictest3016107-21/usr/sbin/munin11:30:232
2924812994126,11cyclictest2939550-21/usr/sbin/munin07:50:172
2924812994118,6cyclictest2947497-21apt-get08:15:032
2924807994126,11cyclictest2976278-21/usr/sbin/munin09:35:241
2924803994131,6cyclictest0-21swapper/011:35:000
2924812994030,7cyclictest3017008-21apt-get11:35:122
2924812994030,6cyclictest0-21swapper/212:35:002
2924812994027,10cyclictest3027049-21cron12:03:002
2924812994027,10cyclictest2925737-21/usr/sbin/munin07:10:262
2924812994026,11cyclictest2990666-21cron10:19:002
2924812994026,11cyclictest2990666-21cron10:19:002
2924807994026,11cyclictest2957218-21/usr/sbin/munin08:40:201
2924807994026,10cyclictest3022575-21/usr/sbin/munin11:50:161
2924807994021,13cyclictest2966015-21/usr/sbin/munin09:05:241
2924803994033,4cyclictest2947510-21munin-plugin-st08:15:010
2924803994029,7cyclictest2996278-21grep10:35:140
2924812993932,4cyclictest2935577-21sendmail07:40:002
2924812993932,4cyclictest2935577-21sendmail07:40:002
2924812993930,6cyclictest2960094-21apt-get08:50:112
2924812993930,6cyclictest2960094-21apt-get08:50:112
2924812993930,6cyclictest2951367-21apt-get08:25:152
2924812993930,5cyclictest3033628-21unixbench_singl12:20:222
2924812993929,8cyclictest2982457-21apt-get09:55:062
2924812993928,8cyclictest2992717-21munin-node10:25:122
2924812993927,9cyclictest2985889-21/usr/sbin/munin10:05:262
2924812993927,8cyclictest3023342-21grep11:50:252
2924812993927,7cyclictest3012487-21/usr/sbin/munin11:20:212
2924812993926,10cyclictest3011213-21/usr/sbin/munin11:15:252
2924812993926,10cyclictest2945456-21cron08:06:012
2924812993926,10cyclictest2930197-21cron07:22:002
2924812993925,11cyclictest3009003-21/usr/sbin/munin11:10:182
2924812993919,16cyclictest2954978-21runrttasks08:35:112
2924807993929,6cyclictest0-21swapper/111:30:001
2924807993929,6cyclictest0-21swapper/107:25:011
2924807993926,10cyclictest2984511-21/usr/sbin/munin10:00:161
2924807993925,10cyclictest231ktimers/111:35:111
2924807993923,10cyclictest2987704-21/usr/sbin/munin10:10:131
2924807993923,10cyclictest2987704-21/usr/sbin/munin10:10:131
2924807993921,14cyclictest3025924-21fschecks_count12:00:141
2924807993921,13cyclictest2951475-21df08:25:161
2924807993920,14cyclictest53-21kcompactd012:35:051
2924807993920,14cyclictest53-21kcompactd012:35:051
2924807993919,15cyclictest2948145-21tune2fs08:15:171
2924803993927,8cyclictest3017991-21grep11:35:240
2924812993833,2cyclictest2927268-21apt-get07:15:122
2924812993832,3cyclictest3020430-21apt-get11:45:122
2924812993831,4cyclictest2970095-21cron09:20:012
2924812993828,7cyclictest3001581-21/usr/sbin/munin10:50:152
2924812993827,8cyclictest2961853-21/usr/sbin/munin08:55:162
2924812993827,8cyclictest2933762-21sshd07:34:002
2924812993826,9cyclictest3039151-21sshd12:37:332
2924812993826,9cyclictest3039151-21sshd12:37:332
2924812993826,9cyclictest3029767-21pluginstate12:10:222
2924812993826,8cyclictest2942980-21/usr/sbin/munin08:00:182
2924812993825,10cyclictest2965756-21/usr/sbin/munin09:05:212
2924812993821,13cyclictest2997942-21df10:40:132
2924812993820,13cyclictest2954908-21latency_hist08:35:012
2924807993825,10cyclictest3031662-21/usr/sbin/munin12:15:231
2924807993825,10cyclictest2962320-21/usr/sbin/munin08:55:221
2924807993824,9cyclictest2931716-21/usr/sbin/munin07:25:281
2924807993824,10cyclictest2969292-21/usr/sbin/munin09:15:221
2924807993823,10cyclictest0-21swapper/108:30:121
2924807993822,11cyclictest0-21swapper/111:20:111
2924807993822,10cyclictest0-21swapper/107:10:131
2924807993821,13cyclictest3028923-21cron12:10:001
2924812993731,3cyclictest3018732-21apt-get11:40:152
2924812993727,7cyclictest3003598-21/usr/sbin/munin10:55:202
2924812993727,7cyclictest2984303-21/usr/sbin/munin10:00:132
2924812993727,7cyclictest2977753-21/usr/sbin/munin09:40:222
2924812993726,8cyclictest2976136-21/usr/sbin/munin09:35:232
2924812993720,13cyclictest2980538-21latency_hist09:50:012
2924812993720,13cyclictest2973634-21sed09:30:012
2924812993719,15cyclictest2951216-21latency_hist08:25:012
2924807993726,8cyclictest2958397-21/usr/sbin/munin08:45:211
2924807993726,7cyclictest3007303-21/usr/sbin/munin11:05:201
2924807993725,8cyclictest2947016-21grep08:10:271
2924807993724,9cyclictest2973167-21cron09:26:011
2924807993724,10cyclictest3002116-21/usr/sbin/munin10:50:231
2924807993724,10cyclictest2963963-21/usr/sbin/munin09:00:201
2924807993723,10cyclictest2982467-21/usr/sbin/munin09:55:211
2924807993722,9cyclictest2991187-21fschecks_count10:20:141
2924807993722,9cyclictest0-21swapper/109:45:141
2924807993720,14cyclictest2961036-21grep08:50:221
2924807993720,14cyclictest2961036-21grep08:50:221
2924807993720,13cyclictest2935089-21unixbench_multi07:35:251
2924807993720,13cyclictest2935089-21unixbench_multi07:35:241
2924816993626,5cyclictest0-21swapper/310:45:153
2924816993625,8cyclictest391ktimers/308:20:113
2924812993626,7cyclictest2973920-21/usr/sbin/munin09:30:162
2924812993626,6cyclictest3013947-21/usr/sbin/munin11:25:192
2924812993625,7cyclictest2991632-21/usr/sbin/munin10:20:202
2924812993624,8cyclictest2970970-21/usr/sbin/munin09:20:232
2924812993620,13cyclictest2946787-21expr08:10:262
2924812993620,12cyclictest3032551-21cat12:20:012
2924812993620,12cyclictest3008350-21date11:10:012
2924807993626,7cyclictest2942630-21apt-get08:00:121
2924807993625,7cyclictest2998161-21/usr/sbin/munin10:40:151
2924807993625,7cyclictest2973735-21/usr/sbin/munin09:30:191
2924807993624,9cyclictest2995707-21cron10:33:011
2924807993624,7cyclictest3035983-21munin-node12:30:111
2924807993623,10cyclictest3029216-21/usr/sbin/munin12:10:131
2924807993623,10cyclictest2990369-21/usr/sbin/munin10:15:261
2924807993623,10cyclictest2990369-21/usr/sbin/munin10:15:261
2924807993623,10cyclictest2986763-21/usr/sbin/munin10:05:241
2924807993621,7cyclictest3020538-21/usr/sbin/munin11:45:141
2924807993621,11cyclictest2944326-21/usr/sbin/munin08:05:181
2924807993620,13cyclictest3019724-21grep11:40:271
2924807993619,12cyclictest3024041-21/usr/sbin/munin11:55:181
2924807993619,12cyclictest3024041-21/usr/sbin/munin11:55:181
2924807993618,15cyclictest2997072-21timerwakeupswit10:35:231
2924807993613,10cyclictest645-21runrttasks10:26:021
2924803993629,4cyclictest2987361-21auditctl10:10:000
2924816993524,7cyclictest0-21swapper/312:15:243
2924812993530,2cyclictest2994420-21apt-get10:30:122
2924812993529,4cyclictest2980557-21apt-get09:50:132
2924812993526,6cyclictest2959382-21/usr/sbin/munin08:45:262
2924812993526,6cyclictest2937622-21df_inode07:45:152
2924812993525,7cyclictest2963633-21/usr/sbin/munin09:00:152
2924812993525,7cyclictest2956801-21df08:40:142
2924812993525,6cyclictest2941355-21meminfo07:55:172
2924812993519,12cyclictest2999965-21fschecks_time10:45:152
2924812993519,12cyclictest2967003-21ls09:10:132
2924807993524,8cyclictest2941488-21/usr/sbin/munin07:55:211
2924807993524,7cyclictest3005218-21/usr/sbin/munin11:00:151
2924807993524,7cyclictest2950951-21cron08:23:011
2924807993523,9cyclictest2939587-21grep07:50:171
2924807993523,9cyclictest2933139-21/usr/sbin/munin07:30:251
2924807993523,8cyclictest3015403-21/usr/sbin/munin11:30:121
2924807993523,7cyclictest3010314-21/usr/sbin/munin11:15:161
2924807993522,9cyclictest2980553-21/usr/sbin/munin09:50:231
2924807993519,11cyclictest645-21runrttasks08:35:101
2924807993519,11cyclictest2935862-21/usr/sbin/munin07:40:131
2924807993518,14cyclictest2928048-21switchtime07:15:241
2924807993518,13cyclictest3035334-21grep12:25:261
2924807993518,13cyclictest3035334-21grep12:25:261
2924807993424,7cyclictest3009149-21/usr/sbin/munin11:10:201
2924816993326,4cyclictest2980329-21kthreadd09:50:013
2924816993325,4cyclictest0-21swapper/310:55:163
2924816993323,6cyclictest0-21swapper/312:05:263
2924803993327,3cyclictest3035751-21kthreadd12:30:000
2924803993327,3cyclictest3035751-21kthreadd12:30:000
2924803993322,8cyclictest2942390-21sendmail08:00:010
2924803993322,7cyclictest2976934-21sendmail09:40:010
2924816993225,4cyclictest0-21swapper/310:00:003
2924816993214,6cyclictest2949282-21kworker/u8:3+events_unbound09:05:433
2924803993226,3cyclictest0-21swapper/011:20:010
2924803993221,7cyclictest2952363-21grep08:25:270
2924803993221,7cyclictest2933392-21grep07:30:270
2924803993221,6cyclictest0-21swapper/012:30:270
2924816993122,6cyclictest2971803-21sh09:25:013
2924803993123,5cyclictest2973507-21cron09:30:010
2924816993022,4cyclictest0-21swapper/310:35:243
2924816993022,4cyclictest0-21swapper/308:44:183
2924816993021,5cyclictest0-21swapper/309:50:253
2924816993021,4cyclictest0-21swapper/310:30:243
2924816993019,6cyclictest0-21swapper/309:00:003
2924816993018,8cyclictest2941095-21grep07:55:163
2924803993023,4cyclictest0-21swapper/011:10:230
2924803993023,3cyclictest0-21swapper/007:20:260
2924803993021,6cyclictest2979874-21grep09:45:280
2924803993019,7cyclictest2993704-21grep10:25:260
2924816992921,4cyclictest0-21swapper/308:55:003
2924816992921,4cyclictest0-21swapper/308:55:003
2924816992919,7cyclictest2975650-21fschecks_time09:35:163
2924803992922,4cyclictest0-21swapper/012:40:000
2924803992922,4cyclictest0-21swapper/012:40:000
2924803992922,3cyclictest0-21swapper/010:20:000
2924803992922,3cyclictest0-21swapper/010:20:000
2924803992921,4cyclictest0-21swapper/009:15:250
2924816992821,5cyclictest2921186-21kworker/u8:0+events_unbound07:34:033
2924816992821,3cyclictest0-21swapper/310:05:243
2924816992819,6cyclictest3006142-21grep11:00:323
2924816992817,7cyclictest204-21dbus-daemon10:24:393
2924816992815,5cyclictest3025347-21kworker/u8:2+events_unbound12:00:243
2924803992820,4cyclictest0-21swapper/008:10:010
2924803992818,7cyclictest2985241-21unixbench_singl10:00:230
2924803992818,6cyclictest0-21swapper/012:00:000
2924803992818,6cyclictest0-21swapper/012:00:000
2924803992818,6cyclictest0-21swapper/011:00:320
292481699272,3cyclictest131rcu_preempt12:25:143
292481699272,3cyclictest131rcu_preempt12:25:143
2924816992720,5cyclictest2983729-21kworker/u8:0+events_unbound10:10:223
2924816992720,5cyclictest2983729-21kworker/u8:0+events_unbound10:10:223
2924816992719,4cyclictest0-21swapper/308:30:273
2924816992718,6cyclictest2972219-21tune2fs09:25:163
292481699271,14cyclictest0-21swapper/310:15:243
292481699271,14cyclictest0-21swapper/310:15:243
292481699270,14cyclictest0-21swapper/307:15:123
292480399277,16cyclictest0-21swapper/011:00:000
2924803992719,5cyclictest0-21swapper/011:50:250
2924803992718,6cyclictest3013775-21fschecks_count11:25:170
2924803992717,7cyclictest2931739-21head07:25:280
2924816992620,4cyclictest2977697-21kworker/u8:1+events_unbound11:19:503
2924816992619,5cyclictest2926217-21kworker/u8:2+events_unbound08:29:213
2924816992619,3cyclictest0-21swapper/311:55:253
2924816992619,3cyclictest0-21swapper/311:55:253
2924816992619,3cyclictest0-21swapper/309:30:263
2924816992618,6cyclictest2926217-21kworker/u8:2+events_unbound09:14:463
2924816992618,4cyclictest0-21swapper/308:10:173
2924816992617,6cyclictest0-21swapper/312:35:003
2924816992614,8cyclictest0-21swapper/310:45:013
2924803992618,4cyclictest2926443-21munin-plugin-st07:15:010
292480399261,5cyclictest3025762-21apt-get12:00:120
2924816992518,5cyclictest2926217-21kworker/u8:2+events_unbound07:50:103
2924816992517,5cyclictest2935569-21munin-run07:40:003
2924816992517,5cyclictest2935569-21munin-run07:40:003
2924816992517,5cyclictest0-21swapper/311:40:273
2924816992517,4cyclictest0-21swapper/311:25:273
2924816992516,6cyclictest0-21swapper/308:45:213
2924816992515,7cyclictest3020585-21grep11:45:163
292481699251,13cyclictest0-21swapper/307:25:273
292481699251,12cyclictest0-21swapper/312:20:163
292481699251,12cyclictest0-21swapper/309:15:183
2924803992518,4cyclictest0-21swapper/011:20:240
2924803992518,4cyclictest0-21swapper/009:10:230
2924803992517,5cyclictest0-21swapper/007:15:160
2924803992510,12cyclictest121ktimers/009:50:110
2924816992420,2cyclictest0-21swapper/307:10:143
2924816992418,4cyclictest2926217-21kworker/u8:2+events_unbound07:24:343
2924816992418,3cyclictest0-21swapper/311:15:013
2924816992417,5cyclictest2936919-21kworker/u8:3+events_unbound07:45:033
2924816992417,5cyclictest2926217-21kworker/u8:2+events_unbound07:40:403
2924816992416,6cyclictest2956240-21kworker/u8:0+events_unbound09:00:113
2924816992416,4cyclictest0-21swapper/312:10:243
2924816992415,7cyclictest3018344-21kworker/u8:1+events_unbound11:50:413
292481699241,12cyclictest0-21swapper/311:30:103
2924803992418,4cyclictest0-21swapper/008:04:210
2924803992418,3cyclictest0-21swapper/012:15:240
2924803992418,2cyclictest0-21swapper/009:07:010
2924803992418,2cyclictest0-21swapper/007:53:580
2924803992417,3cyclictest0-21swapper/010:45:010
2924803992417,3cyclictest0-21swapper/009:30:400
2924803992416,4cyclictest0-21swapper/008:40:000
292480399241,20cyclictest3000747-21tail10:45:240
292480399241,19cyclictest0-21swapper/012:24:050
292480399241,19cyclictest0-21swapper/012:14:380
292480399241,19cyclictest0-21swapper/009:21:460
292480399241,19cyclictest0-21swapper/008:45:130
292480399241,19cyclictest0-21swapper/008:41:540
292480399240,20cyclictest0-21swapper/007:36:280
292480399240,20cyclictest0-21swapper/007:36:280
2924816992317,4cyclictest2983729-21kworker/u8:0+events_unbound10:02:483
2924816992317,4cyclictest2977697-21kworker/u8:1+events_unbound10:52:503
2924816992315,4cyclictest0-21swapper/308:15:163
292480399233,3cyclictest0-21swapper/010:10:560
292480399233,3cyclictest0-21swapper/010:10:560
2924803992317,3cyclictest13850irq/36-mmc112:10:000
2924803992315,6cyclictest121ktimers/010:20:130
2924803992314,6cyclictest121ktimers/011:45:150
292480399231,3cyclictest2964085-21ntp_states09:00:220
292480399231,3cyclictest2961800-21apt-get08:55:150
292480399231,18cyclictest0-21swapper/009:59:380
292480399230,19cyclictest0-21swapper/010:31:300
2924816992215,5cyclictest2926217-21kworker/u8:2+events_unbound08:00:263
292480399223,5cyclictest2949419-21munin-run08:20:010
2924803992216,2cyclictest0-21swapper/011:40:240
2924803992216,2cyclictest0-21swapper/009:40:220
2924803992215,3cyclictest0-21swapper/008:23:110
2924803992212,7cyclictest3002079-21wc10:50:220
292480399221,19cyclictest0-21swapper/008:50:110
292480399221,19cyclictest0-21swapper/008:50:110
292480399220,18cyclictest0-21swapper/011:08:020
2924816992114,5cyclictest2940454-21kworker/u8:1+events_unbound08:07:243
2924803992115,3cyclictest0-21swapper/007:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional