You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 12:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Sun Jun 28, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3229923995226,21cyclictest0-21swapper/323:15:253
3229923995226,21cyclictest0-21swapper/323:15:253
3229915995242,6cyclictest3261579-21munin-plugin-st20:40:001
3229923995042,5cyclictest3312984-21munin-plugin-st23:10:013
3229923994940,5cyclictest3234313-21fschecks_time19:20:173
3229912994941,6cyclictest3311281-21munin-plugin-st23:05:010
3229912994938,7cyclictest3261580-21cut20:40:010
3229923994635,7cyclictest3240901-21sendmail19:40:003
3229912994635,5cyclictest3256936-21kworker/u8:3+events_unbound21:02:420
3229923994435,6cyclictest391ktimers/319:55:123
3229923994412,29cyclictest3288600-21unixbench_multi21:55:263
3229918994433,6cyclictest0-21swapper/223:38:232
3229912994438,4cyclictest3319636-21kworker/u8:2+events_unbound23:38:230
3229923994335,5cyclictest3280354-21munin-plugin-st21:35:013
3229923994335,5cyclictest3280354-21munin-plugin-st21:35:013
3229923994329,11cyclictest391ktimers/319:45:133
3229923994318,21cyclictest0-21swapper/321:45:243
322992399431,3cyclictest0-21swapper/321:50:133
322992399431,13cyclictest3305779-21taskset22:45:373
322992399431,11cyclictest3290413-21/usr/sbin/munin22:00:263
322992399431,11cyclictest3290413-21/usr/sbin/munin22:00:263
3229915994336,4cyclictest3229903-21cyclictest22:35:011
3229915994335,5cyclictest3275246-21sendmail21:20:011
322992399423,34cyclictest0-21swapper/322:15:033
322992399421,9cyclictest3319263-21/usr/sbin/munin23:25:243
322992399421,12cyclictest3326681-21taskset23:48:163
3229923994121,17cyclictest0-21swapper/321:25:293
322992399411,9cyclictest3245291-21/usr/sbin/munin19:50:203
322992399411,8cyclictest3265098-21wc20:50:013
322992399411,3cyclictest0-21swapper/322:25:123
322992399411,2cyclictest3275477-21apt-get21:20:143
3229912994135,4cyclictest3328233-21kworker/u8:3+events_unbound00:23:320
3229912994132,6cyclictest0-21swapper/019:15:000
322992399403,34cyclictest3233459-21users19:15:263
322992399403,31cyclictest3313207-21apt-get23:10:133
3229923994032,4cyclictest0-21swapper/321:00:003
3229923994028,7cyclictest0-21swapper/323:50:263
322992399401,7cyclictest3334627-21/usr/sbin/munin00:10:213
322992399401,7cyclictest3269403-21/usr/sbin/munin21:00:243
322992399401,3cyclictest3342252-21sshd00:33:243
322992399401,3cyclictest0-21swapper/321:40:133
322992399400,3cyclictest0-21swapper/323:20:123
322992399400,3cyclictest0-21swapper/323:20:113
322992399400,3cyclictest0-21swapper/321:15:133
322992399400,3cyclictest0-21swapper/320:40:133
3229915994030,6cyclictest3332622-21fschecks_time00:05:171
3229915994026,10cyclictest231ktimers/122:45:121
3229912994026,6cyclictest3338902-21kworker/u8:2+events_unbound00:25:410
322992399393,4cyclictest3302984-21apt-get22:40:133
322992399392,32cyclictest3256937-21/usr/sbin/munin20:25:163
322992399391,9cyclictest3236574-21/usr/sbin/munin19:25:253
322992399391,7cyclictest3255053-21/usr/sbin/munin20:20:143
322992399391,6cyclictest3308218-21/usr/sbin/munin22:55:163
322992399391,4cyclictest3322072-21/usr/sbin/munin23:35:163
322992399391,3cyclictest0-21swapper/321:05:153
322992399391,32cyclictest3291200-21fschecks_count22:05:163
322992399391,13cyclictest3323865-21grep23:40:153
322992399391,13cyclictest3259023-21proc_pri20:30:233
322992399391,12cyclictest3293317-21/usr/sbin/munin22:10:223
322992399391,10cyclictest3332852-21meminfo00:05:193
3229923993910,26cyclictest3260663-21ntp_states20:35:213
322992399390,3cyclictest0-21swapper/320:00:103
3229918993931,5cyclictest3246220-21auditctl19:55:022
3229915993929,7cyclictest438-21cron22:40:011
322992399383,29cyclictest3251734-21sort20:10:153
322992399382,4cyclictest3241138-21apt-get19:40:133
322992399381,9cyclictest3335810-21apt-get00:15:133
322992399381,6cyclictest3339391-21/usr/sbin/munin00:25:173
322992399381,6cyclictest3320357-21df_inode23:30:153
322992399381,3cyclictest3249979-21df20:05:143
322992399381,3cyclictest0-21swapper/322:50:143
322992399381,3cyclictest0-21swapper/321:10:123
322992399381,3cyclictest0-21swapper/320:50:103
322992399381,3cyclictest0-21swapper/300:20:133
3229918993827,7cyclictest311ktimers/223:25:122
3229918993826,9cyclictest311ktimers/220:10:122
3229915993831,4cyclictest3290797-21cron22:05:011
3229915993831,4cyclictest3290797-21cron22:05:001
3229915993829,6cyclictest3250890-21unixbench_multi20:05:271
3229912993818,4cyclictest131rcu_preempt20:31:340
322992399373,29cyclictest3230190-21df_inode19:10:163
322992399373,28cyclictest3280581-21apt-get21:35:133
322992399373,28cyclictest3280581-21apt-get21:35:133
322992399371,4cyclictest3301422-21/usr/sbin/munin22:35:153
322992399371,4cyclictest3297447-21sshd22:22:593
322992399371,3cyclictest3330613-21/usr/sbin/munin00:00:153
322992399370,3cyclictest0-21swapper/323:00:123
322992399370,3cyclictest0-21swapper/300:35:113
3229918993728,4cyclictest0-21swapper/200:25:172
3229912993729,4cyclictest3344117-21basename00:40:000
3229912993727,5cyclictest3304092-21kworker/u8:2+events_unbound22:42:420
3229912993725,7cyclictest0-21swapper/019:55:010
3229912993717,10cyclictest131rcu_preempt20:56:590
322992399361,3cyclictest0-21swapper/320:15:133
322992399360,3cyclictest0-21swapper/322:30:093
322991599366,15cyclictest24-21ksoftirqd/121:45:011
3229912993627,6cyclictest0-21swapper/020:45:250
3229912993626,6cyclictest0-21swapper/019:15:170
3229912993617,4cyclictest131rcu_preempt20:10:530
3229912993613,19cyclictest121ktimers/021:35:010
3229912993613,19cyclictest121ktimers/021:35:000
3229918993528,4cyclictest3261577-21munin-plugin-st20:40:002
322991299357,15cyclictest131rcu_preempt19:30:010
3229912993528,4cyclictest3331974-21munin-plugin-st00:05:000
3229912993527,5cyclictest3330248-21cron00:00:010
3229912993523,7cyclictest0-21swapper/020:25:240
3229912993519,5cyclictest131rcu_preempt22:30:220
3229912993512,13cyclictest3314615-21kworker/u8:1-events_freezable_power_23:27:360
3229918993428,4cyclictest3337290-21sendmail00:20:002
3229918993427,4cyclictest0-21swapper/220:30:252
3229918993426,4cyclictest0-21swapper/221:00:152
3229915993427,4cyclictest3233856-21latency_hist19:20:011
3229915993425,4cyclictest0-21swapper/120:17:311
3229915993415,12cyclictest131rcu_preempt21:05:121
3229915993414,13cyclictest131rcu_preempt22:15:001
3229912993428,4cyclictest3250890-21unixbench_multi20:05:270
3229912993427,3cyclictest0-21swapper/020:20:000
3229912993426,6cyclictest3242244-21kworker/u8:3+events_unbound19:44:430
3229912993426,4cyclictest0-21swapper/021:10:010
3229912993424,6cyclictest0-21swapper/020:50:230
3229912993420,4cyclictest131rcu_preempt22:38:020
3229912993420,4cyclictest131rcu_preempt21:56:150
3229912993417,5cyclictest131rcu_preempt23:45:000
3229912993416,11cyclictest131rcu_preempt23:50:180
3229912993415,5cyclictest131rcu_preempt21:15:260
3229912993415,5cyclictest131rcu_preempt20:40:260
3229912993415,5cyclictest131rcu_preempt00:10:200
3229912993415,10cyclictest131rcu_preempt22:10:000
3229912993413,5cyclictest131rcu_preempt19:30:230
3229912993413,13cyclictest131rcu_preempt23:15:010
3229918993326,4cyclictest0-21swapper/221:05:262
3229918993325,5cyclictest3231301-21kworker/u8:1+events_unbound19:35:202
3229918993324,6cyclictest3317196-21kworker/u8:0+events_unbound23:24:292
3229918993324,6cyclictest3317196-21kworker/u8:0+events_unbound23:24:292
3229915993323,6cyclictest0-21swapper/123:05:011
3229915993323,5cyclictest0-21swapper/120:00:151
3229915993323,4cyclictest0-21swapper/120:30:261
3229915993318,5cyclictest131rcu_preempt20:48:391
3229915993315,15cyclictest231ktimers/119:24:371
3229912993325,6cyclictest3240875-21cron19:40:000
3229912993321,4cyclictest131rcu_preempt23:20:210
3229912993321,4cyclictest131rcu_preempt23:20:200
3229912993317,5cyclictest131rcu_preempt23:45:150
3229912993316,4cyclictest131rcu_preempt21:45:230
3229912993315,4cyclictest131rcu_preempt23:20:000
3229912993315,4cyclictest131rcu_preempt23:20:000
3229912993312,4cyclictest131rcu_preempt19:55:120
3229918993225,4cyclictest0-21swapper/220:45:012
3229918993224,4cyclictest0-21swapper/223:55:002
3229918993224,4cyclictest0-21swapper/223:30:162
3229918993222,7cyclictest3255134-21fschecks_count20:20:172
322991599329,13cyclictest0-21swapper/120:50:251
322991599328,6cyclictest24-21ksoftirqd/121:30:011
322991599326,6cyclictest24-21ksoftirqd/121:50:011
322991599324,8cyclictest24-21ksoftirqd/100:00:221
3229915993223,5cyclictest0-21swapper/100:40:011
3229915993222,6cyclictest0-21swapper/121:00:251
3229915993216,9cyclictest131rcu_preempt22:18:531
3229912993226,3cyclictest0-21swapper/019:45:260
3229912993224,4cyclictest121ktimers/020:25:000
3229912993221,6cyclictest0-21swapper/022:55:260
3229912993217,4cyclictest131rcu_preempt19:22:470
3229912993216,5cyclictest131rcu_preempt00:09:290
3229912993215,4cyclictest131rcu_preempt23:30:240
3229912993215,4cyclictest131rcu_preempt21:40:210
3229912993214,5cyclictest131rcu_preempt22:55:000
3229912993214,10cyclictest131rcu_preempt22:10:020
322991899316,4cyclictest131rcu_preempt23:15:122
322991899316,4cyclictest131rcu_preempt23:15:112
3229918993123,4cyclictest0-21swapper/200:15:002
322991599317,7cyclictest3336196-21sed00:15:191
322991599316,8cyclictest24-21ksoftirqd/100:20:261
322991599316,6cyclictest24-21ksoftirqd/121:00:021
322991599315,6cyclictest24-21ksoftirqd/119:35:171
322991599315,13cyclictest0-21swapper/121:50:151
322991599314,14cyclictest24-21ksoftirqd/100:30:221
3229912993123,5cyclictest0-21swapper/022:20:000
3229912993117,4cyclictest131rcu_preempt21:25:000
3229912993116,5cyclictest131rcu_preempt21:50:270
3229912993112,10cyclictest131rcu_preempt00:18:260
3229912993111,4cyclictest131rcu_preempt20:02:040
322991299310,28cyclictest0-21swapper/000:30:130
3229918993024,3cyclictest3285177-21kworker/u8:2+events_unbound21:52:492
3229918993023,3cyclictest0-21swapper/200:00:272
3229918993022,4cyclictest0-21swapper/220:55:252
3229918993022,4cyclictest0-21swapper/219:30:232
3229918993019,6cyclictest0-21swapper/222:30:222
322991599305,8cyclictest24-21ksoftirqd/119:10:201
322991599304,7cyclictest24-21ksoftirqd/121:35:001
322991599304,7cyclictest24-21ksoftirqd/121:35:001
322991599303,9cyclictest24-21ksoftirqd/122:53:201
3229915993023,4cyclictest231ktimers/122:55:151
3229915993019,7cyclictest0-21swapper/123:25:011
3229915993019,7cyclictest0-21swapper/123:25:001
3229915993011,6cyclictest24-21ksoftirqd/123:05:241
3229912993020,5cyclictest131rcu_preempt21:35:220
3229912993020,5cyclictest131rcu_preempt21:35:220
3229912993014,5cyclictest131rcu_preempt22:45:180
3229912993012,10cyclictest131rcu_preempt22:20:140
3229918992922,4cyclictest0-21swapper/223:40:242
3229918992922,4cyclictest0-21swapper/222:20:252
3229918992921,6cyclictest3314676-21auditctl23:15:002
3229918992920,4cyclictest0-21swapper/200:35:152
322991599299,3cyclictest131rcu_preempt23:47:001
322991599298,5cyclictest24-21ksoftirqd/120:20:271
322991599296,6cyclictest24-21ksoftirqd/120:40:231
322991599294,8cyclictest3334177-21cpuspeed00:10:161
322991599294,15cyclictest24-21ksoftirqd/123:42:591
322991599292,7cyclictest3273000-21grep21:10:251
3229915992924,3cyclictest0-21swapper/123:50:121
3229915992921,4cyclictest0-21swapper/120:25:251
3229915992920,7cyclictest231ktimers/119:55:221
3229915992920,7cyclictest231ktimers/119:25:171
3229915992918,4cyclictest131rcu_preempt21:20:131
3229915992916,4cyclictest131rcu_preempt22:05:131
3229912992914,5cyclictest131rcu_preempt21:10:180
3229912992911,10cyclictest131rcu_preempt22:25:250
3229918992822,4cyclictest3307800-21kworker/u8:1+events_unbound22:58:092
3229918992822,3cyclictest0-21swapper/219:45:152
3229918992820,4cyclictest0-21swapper/222:25:262
3229918992820,4cyclictest0-21swapper/221:15:252
322991599286,6cyclictest24-21ksoftirqd/119:45:171
322991599285,6cyclictest24-21ksoftirqd/123:55:151
322991599285,5cyclictest24-21ksoftirqd/100:25:241
322991599284,7cyclictest24-21ksoftirqd/122:26:001
322991599284,6cyclictest24-21ksoftirqd/121:55:251
322991599283,8cyclictest24-21ksoftirqd/123:37:351
322991599283,8cyclictest24-21ksoftirqd/123:17:431
322991599283,8cyclictest24-21ksoftirqd/123:17:421
322991599283,8cyclictest24-21ksoftirqd/122:40:231
3229915992819,5cyclictest0-21swapper/119:30:231
3229915992815,4cyclictest131rcu_preempt23:30:121
3229915992811,5cyclictest131rcu_preempt19:53:261
3229915992810,12cyclictest221rcuc/122:20:211
3229912992813,5cyclictest131rcu_preempt23:05:200
3229912992811,5cyclictest131rcu_preempt21:25:190
3229918992720,5cyclictest3271754-21kworker/u8:0+events_unbound21:35:572
3229918992720,5cyclictest3271754-21kworker/u8:0+events_unbound21:35:562
3229918992720,4cyclictest0-21swapper/220:05:262
3229918992718,5cyclictest0-21swapper/223:05:002
322991599278,16cyclictest231ktimers/120:10:121
322991599274,7cyclictest24-21ksoftirqd/123:25:231
3229918992619,3cyclictest0-21swapper/222:35:252
3229918992619,3cyclictest0-21swapper/219:15:012
3229918992617,4cyclictest0-21swapper/200:00:002
322991599265,7cyclictest24-21ksoftirqd/123:10:211
322991599263,7cyclictest24-21ksoftirqd/121:40:011
322991599263,7cyclictest24-21ksoftirqd/121:40:001
3229918992519,4cyclictest3293994-21kworker/u8:2+events_unbound22:15:002
3229918992519,3cyclictest3333148-21mailstats00:05:252
3229918992518,5cyclictest3271754-21kworker/u8:0+events_unbound21:30:142
3229918992518,5cyclictest3271754-21kworker/u8:0+events_unbound21:30:132
3229918992518,4cyclictest0-21swapper/222:00:272
3229918992518,4cyclictest0-21swapper/222:00:272
3229918992517,4cyclictest3232538-21fschecks_count19:15:162
3229918992517,4cyclictest0-21swapper/222:15:232
3229918992516,5cyclictest0-21swapper/223:05:112
322991899251,13cyclictest0-21swapper/200:20:142
322991899242,15cyclictest3285489-21awk21:50:002
3229918992418,4cyclictest3326671-21kworker/u8:2+events_unbound23:48:172
3229918992418,3cyclictest0-21swapper/221:25:282
3229918992416,4cyclictest3236873-21grep19:25:272
3229918992414,7cyclictest204-21dbus-daemon00:33:322
3229918992317,3cyclictest0-21swapper/222:05:152
3229918992317,3cyclictest0-21swapper/220:15:232
3229918992315,4cyclictest0-21swapper/219:40:262
3229918992312,7cyclictest204-21dbus-daemon21:57:502
3229918992216,3cyclictest0-21swapper/222:45:272
3229918992215,5cyclictest3304092-21kworker/u8:2+events_unbound22:43:392
3229918992214,4cyclictest3235137-21grep19:20:262
3229918992116,3cyclictest3248265-21cstates20:00:122
3229918992115,3cyclictest0-21swapper/220:50:202
322991899210,7cyclictest3283304-21unixbench_multi21:40:252
3229918992015,3cyclictest0-21swapper/221:20:212
3229918992014,3cyclictest0-21swapper/222:50:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional