You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 04:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Nov 12, 2025 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3010981994941,4cyclictest3082302-21munin-plugin-st10:35:003
3010981994941,4cyclictest3082302-21munin-plugin-st10:35:003
3010981994839,6cyclictest391ktimers/307:10:143
3010981994820,10cyclictest3032422-21/usr/sbin/munin08:10:223
3010981994718,10cyclictest3061748-21df09:35:163
3010977994737,6cyclictest3123392-21unixbench_multi12:30:252
3010977994732,11cyclictest311ktimers/208:30:142
3010981994530,11cyclictest3029017-21munin-node08:00:113
3010981994524,12cyclictest3077322-21df10:20:183
3010981994517,9cyclictest3048059-21cat08:55:173
3010974994536,5cyclictest215-21dbus-daemon09:20:211
3010974994534,8cyclictest231ktimers/109:50:151
3010974994534,8cyclictest231ktimers/107:25:151
3010981994430,10cyclictest3017520-21/usr/sbin/munin07:25:223
3010977994436,4cyclictest3042615-21cron08:40:002
3010970994436,6cyclictest121ktimers/008:40:120
3010970994435,4cyclictest3121696-21grep12:25:250
3010981994333,7cyclictest3051854-21/usr/sbin/munin09:05:213
3010981994315,12cyclictest3120923-21fschecks_time12:25:153
3010974994335,4cyclictest3080379-21apt-get10:30:001
3010974994332,7cyclictest3100854-21grep11:25:251
3010970994336,4cyclictest3070139-21munin-plugin-st10:00:010
3010981994233,6cyclictest3031896-21sshd08:06:393
3010981994229,9cyclictest3043327-21/usr/sbin/munin08:40:193
3010981994228,10cyclictest3068715-21/usr/sbin/munin09:55:143
3010977994232,6cyclictest3092052-21unixbench_multi11:00:262
3010974994231,7cyclictest3021933-21sh07:40:001
3010974994231,7cyclictest3021933-21sh07:40:001
3010970994232,8cyclictest151rcuc/012:35:120
3010981994134,4cyclictest0-21swapper/307:50:273
3010981994130,7cyclictest3019211-21/usr/sbin/munin07:30:193
3010981994130,7cyclictest3019211-21/usr/sbin/munin07:30:193
3010981994127,10cyclictest3079464-21/usr/sbin/munin10:25:223
3010981994126,8cyclictest3094802-21/usr/sbin/munin11:10:173
3010981994126,11cyclictest3075490-21/usr/sbin/munin10:15:143
3010981994119,4cyclictest3053558-21meminfo09:10:213
3010977994134,4cyclictest3030475-21cron08:05:012
3010977994130,6cyclictest0-21swapper/210:50:002
3010974994133,4cyclictest3097675-21idleruntime-cro11:20:001
3010970994125,8cyclictest3042615-21cron08:40:000
3010981994033,4cyclictest3098105-21apt-get11:20:133
3010981994032,4cyclictest0-21swapper/309:30:243
3010981994030,6cyclictest3107662-21grep11:45:283
3010981994029,7cyclictest3100254-21/usr/sbin/munin11:25:193
3010981994029,7cyclictest3096282-21/usr/sbin/munin11:15:163
3010981994029,7cyclictest3091178-21/usr/sbin/munin11:00:153
3010981994029,7cyclictest3015677-21/usr/sbin/munin07:20:213
3010981994027,10cyclictest3124552-21/usr/sbin/munin12:35:203
3010981994027,10cyclictest3034229-21/usr/sbin/munin08:15:123
3010981994027,10cyclictest3022458-21/usr/sbin/munin07:40:173
3010981994027,10cyclictest3013779-21/usr/sbin/munin07:15:203
3010981994026,11cyclictest3065932-21/usr/sbin/munin09:45:233
3010981994020,6cyclictest3122786-21latency12:30:183
3010981994020,14cyclictest3050160-21grep09:00:213
3010977994033,4cyclictest3012594-21cron07:15:012
3010977994031,6cyclictest3125147-21unixbench_singl12:35:262
3010977994031,6cyclictest3079961-21unixbench_singl10:25:262
3010970994035,3cyclictest3028769-21sh08:00:000
3010970994028,9cyclictest121ktimers/010:00:150
3010981993930,6cyclictest3067650-21/usr/sbin/munin09:50:253
3010981993929,7cyclictest3038755-21cron08:27:013
3010981993929,6cyclictest3041145-21sshd08:35:073
3010981993928,8cyclictest3105518-21/usr/sbin/munin11:40:223
3010981993928,8cyclictest3105518-21/usr/sbin/munin11:40:223
3010981993928,7cyclictest3102047-21/usr/sbin/munin11:30:223
3010981993928,7cyclictest3086785-21/usr/sbin/munin10:45:253
3010981993928,7cyclictest3027300-21munin-node07:55:153
3010981993927,7cyclictest3115414-21/usr/sbin/munin12:10:143
3010981993926,10cyclictest3111452-21cron11:56:593
3010981993925,9cyclictest3072345-21/usr/sbin/munin10:05:173
3010981993828,7cyclictest3118083-21/usr/sbin/munin12:15:263
3010981993828,7cyclictest3085082-21/usr/sbin/munin10:40:243
3010981993828,7cyclictest3024239-21/usr/sbin/munin07:45:193
3010981993827,8cyclictest3083955-21sshd10:39:393
3010981993827,7cyclictest3046683-21/usr/sbin/munin08:50:203
3010981993827,7cyclictest3046683-21/usr/sbin/munin08:50:193
3010981993826,7cyclictest3058358-21forks09:25:163
3010981993825,10cyclictest3057740-21cron09:22:003
3010981993823,12cyclictest3114229-21/usr/sbin/munin12:05:213
3010981993823,12cyclictest3021315-21/usr/sbin/munin07:35:263
3010981993823,12cyclictest3021315-21/usr/sbin/munin07:35:263
3010981993822,11cyclictest3055861-21grep09:15:273
3010977993828,6cyclictest0-21swapper/209:45:152
3010977993824,10cyclictest311ktimers/211:55:172
3010974993831,5cyclictest3039441-21kworker/u8:1+events_unbound08:40:171
3010970993831,5cyclictest3090564-21kworker/u8:0+events_unbound11:35:430
3010981993727,7cyclictest3074126-21/usr/sbin/munin10:10:203
3010981993726,7cyclictest3103770-21ntp_states11:35:213
3010981993725,10cyclictest3039458-21apt-get08:30:123
3010981993723,10cyclictest3070906-21/usr/sbin/munin10:00:223
3010981993722,11cyclictest3063346-21/usr/sbin/munin09:40:253
3010981993719,15cyclictest3092967-21fschecks_time11:05:193
3010974993727,6cyclictest3066736-21munin-run09:50:001
3010970993729,4cyclictest0-21swapper/010:20:010
3010970993722,11cyclictest121ktimers/011:45:150
3010981993626,7cyclictest3119997-21cron12:22:003
3010981993626,7cyclictest3109702-21cron11:54:003
3010981993625,7cyclictest3087736-21/usr/sbin/munin10:50:143
3010981993621,11cyclictest3035838-21/usr/sbin/munin08:20:243
3010981993619,13cyclictest3112955-21grep12:00:263
3010977993628,4cyclictest0-21swapper/207:30:152
3010977993628,4cyclictest0-21swapper/207:30:152
3010977993625,7cyclictest0-21swapper/209:40:262
3010974993629,5cyclictest3023602-21kworker/u8:2+events_unbound08:00:091
3010974993628,4cyclictest0-21swapper/112:20:001
3010974993627,4cyclictest0-21swapper/112:00:161
301097099363,10cyclictest11-21ksoftirqd/008:30:140
301097099361,8cyclictest11-21ksoftirqd/010:25:140
3010981993525,7cyclictest3089457-21df_inode10:55:143
3010977993527,5cyclictest3047386-21kworker/u8:0+events_unbound09:00:192
3010977993525,7cyclictest3109345-21grep11:50:252
3010974993525,6cyclictest0-21swapper/107:50:061
301097099357,6cyclictest0-21swapper/011:15:140
3010970993528,5cyclictest3051030-21kworker/u8:2+events_unbound09:05:220
301097099351,9cyclictest11-21ksoftirqd/007:40:130
3010970993516,8cyclictest131rcu_preempt11:55:150
3010970993510,4cyclictest131rcu_preempt09:30:130
3010981993421,10cyclictest3045593-21sed08:45:243
3010977993427,5cyclictest3018402-21kworker/u8:1+events_unbound07:35:062
3010977993427,5cyclictest3018402-21kworker/u8:1+events_unbound07:35:062
3010977993427,4cyclictest3118999-21fschecks_time12:20:162
3010977993426,4cyclictest0-21swapper/208:40:262
3010977993424,6cyclictest0-21swapper/211:25:252
3010974993422,7cyclictest3042640-21tr08:40:001
3010970993427,5cyclictest3104401-21kworker/u8:2+events_unbound12:20:480
3010977993322,7cyclictest0-21swapper/212:30:002
3010974993326,5cyclictest3073308-21kworker/u8:1+events_unbound10:10:291
3010970993327,4cyclictest3090819-21cron11:00:000
3010970993327,3cyclictest3016785-21apt-key07:25:010
3010970993323,7cyclictest3109383-21grep11:50:250
3010970993315,4cyclictest131rcu_preempt10:35:190
3010977993224,4cyclictest0-21swapper/210:50:332
3010977993223,6cyclictest0-21swapper/211:20:002
3010977993222,7cyclictest0-21swapper/208:15:232
3010977993222,6cyclictest0-21swapper/211:10:252
3010974993225,3cyclictest0-21swapper/107:40:161
3010974993224,4cyclictest0-21swapper/111:05:281
3010974993222,7cyclictest3038560-21grep08:25:271
3010974993222,4cyclictest0-21swapper/109:35:261
3010970993225,4cyclictest0-21swapper/009:45:250
3010977993122,6cyclictest0-21swapper/207:40:242
3010977993122,5cyclictest0-21swapper/208:10:282
3010974993124,5cyclictest3073308-21kworker/u8:1+events_unbound10:55:351
3010974993123,5cyclictest3118501-21kworker/u8:0+events_unbound12:30:501
301097099317,5cyclictest0-21swapper/008:05:230
3010970993124,5cyclictest3102671-21kworker/u8:1+events_unbound11:40:440
3010970993124,5cyclictest3102671-21kworker/u8:1+events_unbound11:40:440
3010970993124,4cyclictest0-21swapper/008:25:010
3010970993124,3cyclictest0-21swapper/008:30:000
3010970993121,6cyclictest0-21swapper/010:20:280
3010970993112,12cyclictest11-21ksoftirqd/012:05:250
3010977993023,5cyclictest3026964-21kworker/u8:3+events_unbound07:55:102
3010977993023,4cyclictest0-21swapper/209:40:012
3010977993022,5cyclictest3049452-21cut08:59:592
3010977993022,4cyclictest3016650-21latency_hist07:25:012
3010977993022,4cyclictest0-21swapper/210:05:252
3010977993020,8cyclictest440-21cron11:50:002
3010977993020,6cyclictest0-21swapper/210:20:192
3010977993020,6cyclictest0-21swapper/209:50:272
3010977993020,6cyclictest0-21swapper/208:45:242
3010970993021,7cyclictest3093964-21kworker/u8:2+events_unbound11:09:320
301097799299,7cyclictest1-21systemd07:20:012
3010977992922,5cyclictest3056252-21kworker/u8:1+events_unbound09:20:232
3010977992922,4cyclictest3118103-21unixbench_singl12:15:262
3010977992920,5cyclictest0-21swapper/212:00:262
3010977992919,6cyclictest0-21swapper/211:00:002
3010977992919,6cyclictest0-21swapper/208:05:292
3010974992922,4cyclictest3085722-21idleruntime-cro10:45:011
3010974992921,3cyclictest0-21swapper/107:55:291
3010974992920,4cyclictest0-21swapper/109:00:001
3010974992919,6cyclictest3105089-21fschecks_count11:40:161
3010974992919,6cyclictest3105089-21fschecks_count11:40:161
3010974992919,6cyclictest3078276-21grep10:20:291
3010974992919,5cyclictest0-21swapper/110:40:001
3010974992917,8cyclictest1-21systemd08:50:171
3010974992917,8cyclictest1-21systemd08:50:171
301097099298,13cyclictest131rcu_preempt12:10:190
3010970992922,5cyclictest3014603-21kworker/u8:2+events_unbound07:18:460
3010970992921,4cyclictest0-21swapper/010:30:270
3010970992921,4cyclictest0-21swapper/010:30:270
3010970992919,5cyclictest131rcu_preempt09:28:520
3010970992916,8cyclictest0-21swapper/010:05:270
3010977992821,3cyclictest0-21swapper/209:20:012
3010977992819,5cyclictest0-21swapper/209:25:262
3010977992818,7cyclictest3038617-21unixbench_singl08:25:272
3010974992822,4cyclictest3090564-21kworker/u8:0+events_unbound11:30:171
3010974992819,6cyclictest0-21swapper/110:45:261
3010974992818,6cyclictest0-21swapper/112:25:011
301097099288,5cyclictest131rcu_preempt09:10:130
301097099286,12cyclictest11-21ksoftirqd/009:20:250
3010970992822,4cyclictest3107964-21kworker/u8:1+events_unbound12:02:420
3010970992820,6cyclictest3012252-21kworker/u8:0+events_unbound07:37:430
3010970992820,6cyclictest3012252-21kworker/u8:0+events_unbound07:37:420
3010970992820,5cyclictest13950irq/36-mmc109:20:000
3010970992820,5cyclictest0-21swapper/011:10:160
3010970992820,4cyclictest0-21swapper/010:45:010
3010970992819,6cyclictest3074802-21unixbench_multi10:10:280
3010970992812,7cyclictest11-21ksoftirqd/009:00:260
301097099281,13cyclictest0-21swapper/010:45:420
3010977992719,4cyclictest0-21swapper/210:10:182
3010977992717,6cyclictest0-21swapper/210:30:272
3010977992717,6cyclictest0-21swapper/210:30:272
3010977992717,6cyclictest0-21swapper/207:25:182
3010974992720,5cyclictest3047386-21kworker/u8:0+events_unbound09:10:221
3010974992719,4cyclictest0-21swapper/112:10:161
3010974992718,4cyclictest0-21swapper/112:35:261
301097099279,7cyclictest11-21ksoftirqd/008:15:190
301097099276,12cyclictest11-21ksoftirqd/007:10:270
3010970992721,4cyclictest3085418-21kworker/u8:2+events_unbound10:50:230
3010970992718,6cyclictest3117263-21grep12:15:160
3010970992712,4cyclictest0-21swapper/008:46:550
3010970992711,14cyclictest11-21ksoftirqd/008:00:130
3010970992710,7cyclictest11-21ksoftirqd/007:50:270
3010977992618,5cyclictest0-21swapper/211:20:262
3010977992617,6cyclictest0-21swapper/211:40:002
3010974992619,4cyclictest0-21swapper/107:20:271
3010974992619,3cyclictest0-21swapper/107:15:271
3010974992618,6cyclictest146-21jbd2/mmcblk1p2-807:45:231
3010974992618,5cyclictest0-21swapper/112:05:251
301097099269,6cyclictest11-21ksoftirqd/007:30:260
301097099269,6cyclictest11-21ksoftirqd/007:30:260
301097099265,12cyclictest131rcu_preempt09:40:160
301097099262,13cyclictest0-21swapper/012:30:480
3010970992619,5cyclictest3068123-21kworker/u8:2+events_unbound09:51:420
3010970992618,6cyclictest3023602-21kworker/u8:2+events_unbound08:11:420
3010970992616,7cyclictest3102968-21sh11:34:590
3010970992611,9cyclictest131rcu_preempt11:20:240
3010977992517,4cyclictest0-21swapper/209:30:262
3010977992516,5cyclictest0-21swapper/211:45:012
3010977992516,5cyclictest0-21swapper/211:45:002
3010974992518,5cyclictest3107964-21kworker/u8:1+events_unbound11:52:541
3010974992518,4cyclictest3047386-21kworker/u8:0+events_unbound09:00:251
3010974992517,5cyclictest0-21swapper/112:25:161
3010974992517,3cyclictest0-21swapper/109:25:171
301097099252,7cyclictest11-21ksoftirqd/009:35:180
3010970992519,4cyclictest3090564-21kworker/u8:0+events_unbound11:25:160
3010970992518,5cyclictest3090564-21kworker/u8:0+events_unbound11:04:500
3010970992516,7cyclictest3042655-21kworker/u8:2+events_unbound08:53:420
3010970992516,7cyclictest3042655-21kworker/u8:2+events_unbound08:53:410
3010970992513,4cyclictest131rcu_preempt08:55:170
3010977992417,5cyclictest311ktimers/210:00:152
3010977992417,4cyclictest0-21swapper/210:20:002
3010977992415,6cyclictest0-21swapper/212:10:172
3010977992413,8cyclictest3113857-21cut12:05:162
3010974992418,3cyclictest0-21swapper/108:10:011
3010974992417,5cyclictest3073308-21kworker/u8:1+events_unbound10:53:401
3010974992416,3cyclictest0-21swapper/110:30:251
3010974992416,3cyclictest0-21swapper/110:30:251
3010974992415,5cyclictest0-21swapper/111:35:261
3010974992414,6cyclictest0-21swapper/107:35:001
3010974992414,6cyclictest0-21swapper/107:34:591
3010974992412,7cyclictest0-21swapper/110:00:211
301097099247,10cyclictest11-21ksoftirqd/007:45:160
3010970992411,5cyclictest131rcu_preempt07:25:050
3010977992317,3cyclictest0-21swapper/208:50:272
3010977992317,3cyclictest0-21swapper/208:50:272
3010977992316,3cyclictest0-21swapper/208:20:252
301097499237,3cyclictest131rcu_preempt09:59:321
3010974992316,5cyclictest3033644-21kworker/u8:0+events_unbound08:32:541
3010974992316,5cyclictest3033644-21kworker/u8:0+events_unbound08:20:141
3010974992316,4cyclictest0-21swapper/109:30:171
3010974992316,4cyclictest0-21swapper/108:15:221
3010974992315,4cyclictest3056276-21idleruntime-cro09:20:011
3010974992314,4cyclictest0-21swapper/110:05:161
3010974992313,6cyclictest0-21swapper/111:20:001
3010977992215,3cyclictest0-21swapper/210:40:182
3010977992214,5cyclictest0-21swapper/211:30:172
3010977992214,5cyclictest0-21swapper/209:05:272
3010974992214,5cyclictest3044670-21wc08:45:131
3010974992212,6cyclictest231ktimers/112:00:011
3010974992211,9cyclictest3010961-21cyclictest11:45:151
3010977992115,3cyclictest0-21swapper/207:55:022
301097499212,4cyclictest131rcu_preempt09:40:131
3010974992115,3cyclictest0-21swapper/107:15:011
3010974992114,3cyclictest0-21swapper/111:00:261
3010974992112,5cyclictest231ktimers/110:15:271
3010977992013,3cyclictest3025352-21sh07:50:002
3010977992012,5cyclictest3054561-21cron09:15:012
3010977992010,6cyclictest0-21swapper/210:40:002
301097499209,7cyclictest231ktimers/111:10:121
301097499205,11cyclictest1-21systemd09:05:191
301097499201,12cyclictest3033494-21/usr/sbin/munin08:10:291
3010977991912,4cyclictest0-21swapper/211:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional