You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 11:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon Jul 20, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3123494995548,5cyclictest3183697-21kworker/u8:1+events_unbound22:17:061
3123497995428,21cyclictest0-21swapper/221:00:012
3123497995320,29cyclictest3173276-21grep21:30:282
3123494995241,8cyclictest231ktimers/123:30:121
3123497995127,19cyclictest0-21swapper/219:45:242
3123497995038,8cyclictest311ktimers/219:20:132
3123489994943,3cyclictest31447112sleep020:10:120
3123489994833,9cyclictest121ktimers/023:55:120
3123489994833,9cyclictest121ktimers/023:55:120
3123497994722,20cyclictest0-21swapper/200:20:012
3123502994621,9cyclictest3162337-21/usr/sbin/munin21:00:213
3123502994618,11cyclictest3183988-21latency_hist22:05:003
3123497994639,4cyclictest3154941-21sendmail20:40:012
3123502994531,10cyclictest3153450-21apt-get20:35:133
312349799452,13cyclictest3233581-21grep00:25:242
3123494994538,5cyclictest3216266-21kworker/u8:0+events_unbound23:37:081
3123494994437,5cyclictest3181967-21kworker/u8:3+events_unbound22:02:041
3123494994434,5cyclictest3218274-21munin-plugin-st23:45:001
312349799432,13cyclictest3126950-21unixbench_multi19:15:272
312349799431,10cyclictest438-21cron22:35:002
3123494994336,5cyclictest3230044-21kworker/u8:1+events_unbound00:23:451
3123489994331,9cyclictest121ktimers/021:40:120
3123502994228,11cyclictest3152358-21/usr/sbin/munin20:30:243
3123497994235,4cyclictest3180524-21idleruntime-cro21:55:002
3123497994235,4cyclictest3180524-21idleruntime-cro21:55:002
312349799421,8cyclictest3130362-21tail19:25:262
3123497994215,25cyclictest3143361-21kernelversion20:05:242
3123494994234,4cyclictest3190888-21latency_hist22:25:011
3123502994129,8cyclictest3148477-21/usr/sbin/munin20:20:193
3123502994127,10cyclictest3216899-21/usr/sbin/munin23:40:143
312349799413,6cyclictest3234894-21wc00:30:232
312349799413,6cyclictest3149604-21apt20:25:012
312349799412,8cyclictest3142312-21unixbench_multi20:00:242
312349799412,7cyclictest3195671-21head22:35:252
312349799412,7cyclictest3163453-21cron21:05:012
312349799412,7cyclictest3132536-21cron19:35:002
312349799412,7cyclictest3132536-21cron19:35:002
312349799412,5cyclictest3157084-21fschecks_time20:45:142
312349799412,13cyclictest3133973-21sshd19:36:502
312349799412,12cyclictest3220020-21sshd23:48:172
312349799412,12cyclictest3135955-21cron19:45:002
312349799412,11cyclictest3218252-21cron23:45:002
312349799412,11cyclictest3189858-21ntp_states22:20:222
3123497994121,16cyclictest0-21swapper/221:10:012
312349799411,14cyclictest3178412-21grep21:45:252
312349799411,10cyclictest438-21cron19:15:012
3123489994129,9cyclictest121ktimers/000:15:110
3123489994129,8cyclictest3192227-21grep22:25:270
3123502994034,4cyclictest3197788-21cron22:45:003
3123502994031,6cyclictest3138890-21unixbench_multi19:50:253
3123502994031,6cyclictest3135564-21unixbench_singl19:40:273
3123502994030,7cyclictest3186315-21/usr/sbin/munin22:10:193
3123502994026,11cyclictest3231402-21/usr/sbin/munin00:20:203
3123502994026,11cyclictest3172361-21/usr/sbin/munin21:30:173
3123502994025,12cyclictest3228251-21/usr/sbin/munin00:10:263
3123502994024,10cyclictest3139599-21apt-get19:55:143
3123502994014,8cyclictest0-21swapper/322:30:013
312349799403,10cyclictest3139703-21df19:55:162
312349799402,9cyclictest3159781-21sshd20:52:002
312349799402,8cyclictest3199060-21sed22:45:252
312349799402,12cyclictest3200774-21unixbench_singl22:50:252
312349799402,10cyclictest3167245-21fschecks_count21:15:172
312349799401,9cyclictest3228260-21grep00:10:262
312349799401,9cyclictest3225532-21/usr/sbin/munin00:05:252
3123497994016,20cyclictest3192676-21auditctl22:30:002
312349799401,12cyclictest3177106-21cron21:45:002
3123497994010,25cyclictest0-21swapper/223:20:002
3123494994033,4cyclictest3182232-21idleruntime-cro22:00:001
3123494994033,4cyclictest3182232-21idleruntime-cro22:00:001
3123489994031,6cyclictest3134262-21sendmail19:40:000
3123502993932,4cyclictest3189294-21apt-get22:20:123
3123502993928,8cyclictest3193779-21/usr/sbin/munin22:30:273
3123502993927,8cyclictest3208364-21/usr/sbin/munin23:15:143
3123502993927,8cyclictest3179048-21apt-get21:50:133
3123502993927,8cyclictest3179048-21apt-get21:50:133
3123502993927,8cyclictest3158671-21/usr/sbin/munin20:50:143
3123502993926,9cyclictest0-21swapper/323:05:263
3123502993926,9cyclictest0-21swapper/323:05:263
3123502993926,10cyclictest3161251-21/usr/sbin/munin20:55:273
3123502993925,9cyclictest3224180-21/usr/sbin/munin00:00:213
3123502993925,9cyclictest3224180-21/usr/sbin/munin00:00:213
3123502993925,12cyclictest3147796-21cron20:19:013
3123502993925,12cyclictest3133541-21/usr/sbin/munin19:35:273
3123502993925,11cyclictest3210026-21/usr/sbin/munin23:20:143
3123502993922,13cyclictest3215597-21tr23:35:203
3123502993918,15cyclictest3182367-21latency_hist22:00:013
3123502993918,15cyclictest3182367-21latency_hist22:00:013
3123502993918,12cyclictest0-21swapper/323:50:113
3123502993914,9cyclictest0-21swapper/323:00:223
312349799398,4cyclictest0-21swapper/223:10:172
312349799397,5cyclictest311ktimers/221:35:172
312349799397,4cyclictest0-21swapper/223:50:212
312349799393,9cyclictest3222669-21ntp_states23:55:212
312349799393,9cyclictest3222669-21ntp_states23:55:212
312349799393,5cyclictest3197370-21grep22:40:262
312349799392,6cyclictest164-21systemd-journal22:12:042
312349799392,10cyclictest164-21systemd-journal19:51:522
312349799391,9cyclictest3166455-21head21:10:282
312349799391,6cyclictest3184795-21ps22:05:242
312349799391,11cyclictest3168998-21fschecks_count21:20:252
312349799391,10cyclictest164-21systemd-journal00:23:432
3123494993927,9cyclictest231ktimers/121:40:111
3123502993828,7cyclictest3177888-21/usr/sbin/munin21:45:203
3123502993828,6cyclictest0-21swapper/323:25:273
3123502993827,8cyclictest3233168-21/usr/sbin/munin00:25:213
3123502993827,8cyclictest3207754-21cron23:12:003
3123502993827,8cyclictest3145787-21/usr/sbin/munin20:10:263
3123502993827,8cyclictest3129639-21/usr/sbin/munin19:25:173
3123502993826,9cyclictest3218259-21apt-get23:45:053
3123502993826,9cyclictest3213461-21/usr/sbin/munin23:30:143
3123502993826,9cyclictest3155473-21/usr/sbin/munin20:40:173
3123502993826,9cyclictest3123746-21/usr/sbin/munin19:10:143
3123502993825,9cyclictest3199847-21diskstats22:50:153
3123502993825,9cyclictest3198161-21/usr/sbin/munin22:45:163
3123502993824,11cyclictest3164145-21meminfo21:05:213
3123502993824,11cyclictest3156968-21/usr/sbin/munin20:45:123
3123502993823,12cyclictest3229332-21/usr/sbin/munin00:15:183
3123502993820,13cyclictest3175419-21sendmail21:40:003
312349799385,5cyclictest0-21swapper/200:00:282
312349799385,5cyclictest0-21swapper/200:00:282
3123497993830,4cyclictest0-21swapper/200:40:002
312349799382,6cyclictest3150671-21ps20:25:232
312349799381,10cyclictest3182923-21wc22:00:192
3123494993829,4cyclictest0-21swapper/120:25:161
3123494993821,13cyclictest231ktimers/120:45:101
3123502993729,5cyclictest3136195-21apt-get19:45:113
3123502993727,7cyclictest3143017-21munin-node20:05:173
3123502993726,8cyclictest3188246-21/usr/sbin/munin22:15:243
3123502993726,8cyclictest3167551-21/usr/sbin/munin21:15:203
3123502993726,8cyclictest3165458-21/usr/sbin/munin21:10:153
3123502993726,8cyclictest3127678-21apt-get19:20:153
3123502993726,8cyclictest3126549-21/usr/sbin/munin19:15:243
3123502993726,7cyclictest3142034-21/usr/sbin/munin20:00:233
3123502993726,7cyclictest0-21swapper/322:35:253
3123502993720,13cyclictest3132537-21cron19:35:003
3123502993720,13cyclictest3132537-21cron19:35:003
3123502993718,14cyclictest3223798-21latency_hist00:00:013
3123497993724,9cyclictest3147413-21grep20:15:262
3123497993711,2cyclictest0-21swapper/221:27:402
312349799371,10cyclictest3214907-21latency_hist23:35:012
312349799371,10cyclictest3202442-21tail22:55:252
3123497993710,3cyclictest0-21swapper/223:36:532
3123502993630,3cyclictest3235965-21apt-get00:35:113
3123502993630,3cyclictest3222105-21apt-get23:55:113
3123502993625,8cyclictest3151345-21cron20:28:003
3123502993624,8cyclictest3201951-21/usr/sbin/munin22:55:203
3123502993624,8cyclictest3175763-21diskstats21:40:143
3123502993624,10cyclictest3234703-21load00:30:193
312349799368,5cyclictest0-21swapper/222:20:012
312349799361,9cyclictest3210968-21grep23:20:262
312349799361,8cyclictest3205542-21find23:05:242
312349799361,8cyclictest3205542-21find23:05:242
312349799361,10cyclictest3213141-21apt23:30:012
312349799361,10cyclictest3156633-21cron20:45:012
3123497993610,3cyclictest0-21swapper/220:30:312
3123494993622,10cyclictest231ktimers/120:00:111
3123489993626,6cyclictest0-21swapper/020:45:240
3123502993529,3cyclictest3168827-21apt-get21:20:143
312349799351,9cyclictest3181763-21sed21:55:262
312349799351,9cyclictest3181763-21sed21:55:262
3123494993522,9cyclictest3175387-21cron21:40:001
3123489993527,6cyclictest3156642-21audit_send_reply20:45:010
3123489993527,4cyclictest0-21swapper/019:15:170
3123489993525,6cyclictest0-21swapper/020:35:000
3123502993428,3cyclictest3225535-21apt-get00:05:183
3123494993427,5cyclictest3133988-21kworker/u8:2+events_unbound19:41:511
3123494993423,7cyclictest0-21swapper/100:20:011
312348999348,20cyclictest0-21swapper/020:20:130
312348999346,17cyclictest11-21ksoftirqd/023:25:170
3123489993422,7cyclictest0-21swapper/020:55:270
3123494993325,4cyclictest0-21swapper/122:10:241
3123494993318,10cyclictest231ktimers/121:15:131
312349499331,6cyclictest0-21swapper/123:55:121
312349499331,6cyclictest0-21swapper/123:55:121
312349499331,5cyclictest0-21swapper/100:35:111
3123489993327,4cyclictest3130840-21sh19:30:000
3123489993323,4cyclictest131rcu_preempt19:42:070
3123489993322,8cyclictest121ktimers/023:45:010
312349499323,4cyclictest131rcu_preempt23:00:131
312348999329,17cyclictest11-21ksoftirqd/022:05:260
312348999323,22cyclictest11-21ksoftirqd/022:50:150
3123489993220,9cyclictest121ktimers/020:00:110
3123489993218,3cyclictest131rcu_preempt23:50:130
3123489993212,5cyclictest131rcu_preempt22:55:180
3123494993123,4cyclictest0-21swapper/121:50:261
3123494993123,4cyclictest0-21swapper/121:50:261
312348999316,15cyclictest11-21ksoftirqd/023:35:220
312348999314,8cyclictest11-21ksoftirqd/021:05:190
3123489993123,4cyclictest0-21swapper/000:30:000
3123489993121,6cyclictest0-21swapper/023:10:160
312348999311,22cyclictest0-21swapper/023:30:150
312349799301,10cyclictest3204631-21awk23:05:002
312349499306,14cyclictest3199939-21grep22:50:161
3123494993023,3cyclictest0-21swapper/120:00:011
3123494993022,4cyclictest0-21swapper/122:40:161
3123494993022,4cyclictest0-21swapper/120:20:011
3123494993018,5cyclictest3221667-21kworker/u8:0+events_unbound00:10:231
312348999309,4cyclictest131rcu_preempt22:10:470
312348999308,5cyclictest121ktimers/020:15:160
312348999307,5cyclictest151rcuc/023:50:000
312348999305,7cyclictest3141090-21idleruntime-cro20:00:010
312348999305,6cyclictest11-21ksoftirqd/020:25:200
312348999303,7cyclictest11-21ksoftirqd/000:05:290
3123489993023,4cyclictest3183947-21munin-plugin-st22:05:000
3123489993019,5cyclictest131rcu_preempt21:10:380
3123489993018,6cyclictest131rcu_preempt21:20:220
3123489993014,4cyclictest131rcu_preempt21:35:160
3123489993013,8cyclictest3197488-21kworker/u8:1-events_freezable_power_22:49:120
3123489993011,5cyclictest131rcu_preempt20:05:250
3123489993011,5cyclictest131rcu_preempt00:35:100
3123489993011,4cyclictest131rcu_preempt00:00:200
3123489993011,4cyclictest131rcu_preempt00:00:200
3123494992923,4cyclictest3123304-21kworker/u8:1+events_unbound19:11:501
3123494992921,4cyclictest0-21swapper/121:00:011
3123494992919,5cyclictest0-21swapper/120:31:591
3123494992914,3cyclictest380-21ntpd20:50:211
3123494992912,6cyclictest3204429-21kworker/u8:2+events_unbound23:10:191
312348999296,7cyclictest11-21ksoftirqd/019:45:120
312348999295,6cyclictest11-21ksoftirqd/000:20:190
312348999294,13cyclictest11-21ksoftirqd/019:35:010
312348999294,13cyclictest11-21ksoftirqd/019:35:010
3123489992922,5cyclictest151rcuc/021:33:250
3123489992918,8cyclictest121ktimers/023:20:000
3123489992912,11cyclictest151rcuc/019:50:140
3123494992822,4cyclictest3149244-21kworker/u8:2+events_unbound20:21:161
3123494992822,4cyclictest3137579-21kworker/u8:2+events_unbound19:54:411
3123494992821,5cyclictest3169972-21kworker/u8:2+events_unbound21:28:291
3123494992821,5cyclictest3159976-21kworker/u8:1+events_unbound21:07:031
3123494992820,4cyclictest3202903-21sendmail23:00:001
312349499281,12cyclictest0-21swapper/121:24:031
3123494992810,10cyclictest0-21swapper/120:13:171
312348999289,12cyclictest131rcu_preempt21:00:210
312348999285,7cyclictest11-21ksoftirqd/023:20:190
312348999283,6cyclictest11-21ksoftirqd/022:40:250
312348999283,6cyclictest11-21ksoftirqd/022:35:010
3123489992821,4cyclictest3182221-21cron22:00:000
3123489992821,4cyclictest3182221-21cron22:00:000
3123489992819,6cyclictest121ktimers/022:38:040
3123489992818,6cyclictest0-21swapper/022:20:010
312348999281,7cyclictest11-21ksoftirqd/020:35:180
3123489992817,7cyclictest3235321-21grep00:30:260
3123489992813,4cyclictest131rcu_preempt20:50:200
3123489992812,5cyclictest131rcu_preempt21:15:230
3123489992812,4cyclictest131rcu_preempt21:50:140
3123489992812,4cyclictest131rcu_preempt21:50:140
3123494992720,5cyclictest3195759-21kworker/u8:0+events_unbound22:39:231
3123494992719,6cyclictest3169972-21kworker/u8:2+events_unbound21:32:031
312349499271,4cyclictest131rcu_preempt23:05:131
312349499271,4cyclictest131rcu_preempt23:05:131
312349499271,4cyclictest131rcu_preempt00:30:121
3123494992713,3cyclictest0-21swapper/122:45:141
312348999276,6cyclictest11-21ksoftirqd/000:10:240
312348999276,4cyclictest0-21swapper/019:11:040
312348999274,5cyclictest11-21ksoftirqd/023:05:140
312348999274,5cyclictest11-21ksoftirqd/023:05:140
3123489992713,3cyclictest131rcu_preempt21:45:180
3123489992712,5cyclictest131rcu_preempt23:00:270
3123489992710,4cyclictest131rcu_preempt19:20:140
312349499268,10cyclictest0-21swapper/121:15:011
3123494992621,3cyclictest3135761-21kworker/u8:1-events_freezable_power_19:45:121
3123494992620,4cyclictest3187172-21kworker/u8:0+events_unbound22:27:061
3123494992620,4cyclictest3154543-21kworker/u8:1+events_unbound20:36:571
3123494992619,5cyclictest3204429-21kworker/u8:2+events_unbound23:15:251
3123494992619,5cyclictest3130496-21kworker/u8:2+events_unbound19:30:171
3123494992619,5cyclictest3130496-21kworker/u8:2+events_unbound19:30:171
312349499261,6cyclictest3162350-21/usr/sbin/munin21:00:221
312349499261,4cyclictest131rcu_preempt23:20:131
3123494992611,5cyclictest3137579-21kworker/u8:2+events_unbound20:06:071
312349499261,12cyclictest0-21swapper/119:39:091
312349499260,13cyclictest0-21swapper/123:25:401
3123489992619,3cyclictest0-21swapper/022:25:010
3123489992616,8cyclictest121ktimers/021:25:170
3123494992512,7cyclictest131rcu_preempt23:46:491
3123494992418,4cyclictest3154543-21kworker/u8:1+events_unbound20:40:541
3123494992418,4cyclictest3123304-21kworker/u8:1+events_unbound19:20:211
3123494992418,3cyclictest3123483-21cyclictest00:00:131
3123494992418,3cyclictest3123483-21cyclictest00:00:131
312349499241,4cyclictest131rcu_preempt00:25:101
3123494992413,3cyclictest0-21swapper/100:05:201
3123494992412,3cyclictest24-21ksoftirqd/123:50:241
312349499236,9cyclictest0-21swapper/122:30:221
312349499234,11cyclictest24-21ksoftirqd/119:29:421
3123494992316,5cyclictest3176976-21kworker/u8:1+events_unbound21:47:261
3123494992316,3cyclictest0-21swapper/119:15:021
3123494992314,6cyclictest24-21ksoftirqd/122:06:461
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional