You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 10:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Sun May 10, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2726354995135,11cyclictest391ktimers/321:25:113
2726344995136,11cyclictest231ktimers/100:35:131
2726349995027,18cyclictest0-21swapper/200:05:242
2726340994943,4cyclictest2769452-21kworker/u8:2+events_unbound21:12:580
2726349994719,25cyclictest2813011-21cron23:20:002
2726354994637,6cyclictest2736646-21grep19:35:253
2726344994633,9cyclictest231ktimers/120:05:131
2726340994537,6cyclictest2835206-21kworker/u8:0+events_unbound00:28:240
272634999441,11cyclictest2836039-21/usr/sbin/munin00:25:202
2726349994330,8cyclictest2793672-21sshd22:23:032
2726349994314,26cyclictest2794644-21ntp_states22:25:202
272634999423,8cyclictest2776424-21sshd21:33:242
272634999423,7cyclictest2728650-21df19:15:042
272634999421,9cyclictest2788112-21/usr/sbin/munin22:05:262
272634999421,9cyclictest2788112-21/usr/sbin/munin22:05:262
272634999421,8cyclictest2817111-21/usr/sbin/munin23:30:182
272634999421,10cyclictest2789482-21/usr/sbin/munin22:10:212
272634999411,9cyclictest2826513-21cron23:56:012
272634999411,9cyclictest2766680-21/usr/sbin/munin21:05:162
272634999411,9cyclictest2747694-21/usr/sbin/munin20:10:132
272634999411,8cyclictest2820856-21/usr/sbin/munin23:40:192
272634999411,8cyclictest2818940-21/usr/sbin/munin23:35:172
272634999411,8cyclictest2734253-21/usr/sbin/munin19:30:172
272634999411,10cyclictest2758768-21/usr/sbin/munin20:40:252
272634999409,4cyclictest2781675-21cron21:50:002
2726349994030,7cyclictest0-21swapper/223:00:012
272634999401,9cyclictest2834541-21/usr/sbin/munin00:20:242
272634999401,8cyclictest2764269-21cron20:56:002
2726349994013,24cyclictest0-21swapper/223:20:162
2726349994010,4cyclictest2771903-21fschecks_time21:20:162
2726344994031,5cyclictest2795550-21munin-plugin-st22:30:011
2726344994027,9cyclictest231ktimers/122:35:111
2726344994027,9cyclictest231ktimers/122:35:111
2726340994033,4cyclictest2740540-21sh19:50:010
2726354993933,4cyclictest2728001-21idleruntime-cro19:15:003
2726354993929,7cyclictest2819621-21grep23:35:253
272634999398,5cyclictest2785052-21seq21:59:382
2726349993922,13cyclictest0-21swapper/200:39:592
272634999391,9cyclictest2733328-21users19:25:272
272634999391,8cyclictest2751705-21/usr/sbin/munin20:20:232
2726349993912,24cyclictest2833915-21sed00:20:002
272634999391,11cyclictest2760315-21ntp_states20:45:242
272634999391,11cyclictest2735986-21irqstats19:35:172
272634999391,10cyclictest2795543-21apt-get22:30:052
2726344993930,5cyclictest2802579-21munin-plugin-st22:50:001
272634999382,10cyclictest2808600-21sed23:05:202
272634999381,9cyclictest2827222-21/usr/sbin/munin00:00:152
272634999381,9cyclictest2816152-21/usr/sbin/munin23:25:262
272634999381,9cyclictest2740859-21/usr/sbin/munin19:50:142
272634999381,9cyclictest2740859-21/usr/sbin/munin19:50:132
272634999381,7cyclictest2764860-21/usr/sbin/munin21:00:232
272634999381,7cyclictest2742607-21grep19:55:172
2726349993816,19cyclictest2749445-21cut20:15:172
2726349993813,21cyclictest0-21swapper/222:00:262
2726349993813,21cyclictest0-21swapper/222:00:262
2726349993812,22cyclictest0-21swapper/221:40:172
272634999381,11cyclictest2837906-21ntp_states00:30:222
272634999381,11cyclictest2837906-21ntp_states00:30:212
272634999381,11cyclictest2782361-21meminfo21:50:172
272634999381,11cyclictest2773961-21ntp_states21:25:192
272634999381,10cyclictest2831798-21cron00:13:002
272634999381,10cyclictest2807433-21unixbench_multi23:00:252
272634999381,10cyclictest2800421-21tail22:40:242
2726349993810,25cyclictest0-21swapper/220:25:022
2726340993827,7cyclictest2810884-21grep23:10:240
2726340993822,6cyclictest131rcu_preempt23:25:010
2726340993821,11cyclictest131rcu_preempt23:19:260
2726340993810,21cyclictest131rcu_preempt23:55:170
2726354993729,5cyclictest2772679-21grep21:20:253
272634999379,9cyclictest2769264-21grep21:10:262
272634999378,7cyclictest2770362-21cat21:15:182
272634999372,8cyclictest0-21swapper/220:50:212
272634999372,8cyclictest0-21swapper/220:50:212
272634999372,5cyclictest2744724-21grep20:00:192
272634999371,9cyclictest2798008-21/usr/sbin/munin22:35:162
272634999371,9cyclictest2798008-21/usr/sbin/munin22:35:162
272634999371,9cyclictest2739515-21meminfo19:45:192
272634999371,8cyclictest2737914-21grep19:40:202
272634999371,7cyclictest2822172-21fschecks_time23:45:152
272634999371,10cyclictest2809832-21/usr/sbin/munin23:10:102
2726349993710,23cyclictest0-21swapper/219:15:012
2726340993729,3cyclictest0-21swapper/020:10:020
2726354993630,4cyclictest2702262-21kworker/u8:3+events_unbound19:22:433
2726354993626,6cyclictest0-21swapper/323:00:253
272634999369,3cyclictest2823684-21/usr/sbin/munin23:50:232
272634999361,9cyclictest2756894-21ntp_states20:35:212
272634999361,9cyclictest2755009-21sed20:30:182
272634999361,9cyclictest2746334-21meminfo20:05:182
272634999361,9cyclictest2731627-21users19:20:252
2726344993627,5cyclictest2790428-21idleruntime-cro22:15:001
2726344993626,7cyclictest2821316-21grep23:40:241
2726344993624,8cyclictest2814305-21grep23:20:261
2726354993527,4cyclictest0-21swapper/319:50:163
2726354993527,4cyclictest0-21swapper/319:50:153
272634999352,6cyclictest2792117-21cron22:20:012
272634999351,8cyclictest2802795-21/usr/sbin/munin22:50:122
2726340993528,4cyclictest2838815-21munin-plugin-st00:34:590
2726340993528,4cyclictest2838815-21munin-plugin-st00:34:590
2726340993520,5cyclictest131rcu_preempt23:05:000
2726340993517,10cyclictest131rcu_preempt00:37:140
2726340993515,4cyclictest131rcu_preempt20:15:590
2726340993515,3cyclictest131rcu_preempt22:47:120
2726340993510,18cyclictest0-21swapper/000:10:170
272635499343,5cyclictest131rcu_preempt00:15:133
2726354993423,7cyclictest0-21swapper/300:39:593
272634999341,9cyclictest2801102-21/usr/sbin/munin22:45:222
272634999341,8cyclictest2776792-21/usr/sbin/munin21:35:162
2726340993426,4cyclictest0-21swapper/019:45:010
2726340993424,6cyclictest2777825-21grep21:35:240
2726340993413,10cyclictest131rcu_preempt21:55:230
2726354993322,8cyclictest2809605-21idleruntime-cro23:10:003
2726344993325,4cyclictest0-21swapper/122:05:261
2726344993325,4cyclictest0-21swapper/122:05:261
2726344993325,4cyclictest0-21swapper/100:10:151
2726344993324,5cyclictest0-21swapper/121:50:241
2726344993323,5cyclictest0-21swapper/100:30:161
2726344993323,5cyclictest0-21swapper/100:30:161
2726344993321,7cyclictest0-21swapper/120:50:261
2726344993321,7cyclictest0-21swapper/120:50:261
272634099336,8cyclictest11-21ksoftirqd/022:15:010
2726340993315,12cyclictest131rcu_preempt21:25:020
2726340993314,4cyclictest131rcu_preempt21:41:160
2726340993312,9cyclictest131rcu_preempt19:36:190
2726340993312,5cyclictest131rcu_preempt20:51:300
2726340993312,5cyclictest131rcu_preempt20:51:290
2726340993310,3cyclictest131rcu_preempt20:30:070
2726340993310,12cyclictest131rcu_preempt23:50:220
2726354993223,7cyclictest438-21cron00:29:593
2726354993222,6cyclictest0-21swapper/323:35:003
2726344993225,4cyclictest2735418-21auditctl19:35:001
2726344993224,4cyclictest0-21swapper/100:05:251
2726344993223,4cyclictest0-21swapper/119:45:251
2726344993222,6cyclictest0-21swapper/122:15:251
2726344993221,7cyclictest2736637-21grep19:35:261
2726340993224,4cyclictest0-21swapper/019:12:380
2726340993222,7cyclictest2768060-21wc21:10:010
2726340993222,4cyclictest131rcu_preempt21:50:230
2726340993216,5cyclictest131rcu_preempt22:35:270
2726340993216,5cyclictest131rcu_preempt22:35:260
2726340993216,4cyclictest131rcu_preempt21:45:220
2726340993216,4cyclictest131rcu_preempt21:32:040
2726340993216,3cyclictest131rcu_preempt23:25:210
2726340993215,9cyclictest131rcu_preempt19:15:030
2726340993215,10cyclictest131rcu_preempt22:50:120
2726340993213,5cyclictest151rcuc/020:24:570
2726340993213,16cyclictest121ktimers/000:05:220
2726340993211,5cyclictest131rcu_preempt20:10:160
2726340993210,5cyclictest131rcu_preempt21:15:250
2726354993123,5cyclictest2822981-21unixbench_multi23:45:253
2726354993123,4cyclictest0-21swapper/320:20:243
2726344993123,4cyclictest0-21swapper/122:05:001
2726344993123,4cyclictest0-21swapper/122:05:001
2726344993123,4cyclictest0-21swapper/120:00:161
2726340993123,3cyclictest0-21swapper/021:00:260
2726340993121,4cyclictest131rcu_preempt00:18:220
2726340993120,4cyclictest131rcu_preempt19:25:340
2726340993114,5cyclictest131rcu_preempt19:55:140
2726340993113,4cyclictest131rcu_preempt19:33:230
2726340993111,5cyclictest131rcu_preempt21:00:010
2726340993111,5cyclictest131rcu_preempt19:20:180
2726340993111,4cyclictest151rcuc/022:19:360
2726354993021,6cyclictest2756103-21cut20:35:003
2726344993023,4cyclictest0-21swapper/121:40:281
2726344993021,4cyclictest0-21swapper/122:55:001
2726344993020,7cyclictest438-21cron23:50:001
2726344993018,9cyclictest2805978-21sendmail23:00:001
272634099308,6cyclictest11-21ksoftirqd/022:05:260
272634099308,6cyclictest11-21ksoftirqd/022:05:250
272634099304,15cyclictest11-21ksoftirqd/019:55:010
272634099304,15cyclictest11-21ksoftirqd/019:55:010
2726340993020,7cyclictest121ktimers/022:40:120
2726340993019,7cyclictest2774455-21grep21:25:240
2726340993013,10cyclictest131rcu_preempt20:45:160
2726340993011,4cyclictest151rcuc/023:09:570
2726354992923,4cyclictest2784835-21kworker/u8:3+events_unbound22:23:033
2726354992922,4cyclictest0-21swapper/322:55:173
2726354992921,4cyclictest0-21swapper/322:25:143
2726354992921,4cyclictest0-21swapper/322:13:033
2726354992920,5cyclictest0-21swapper/319:40:253
2726354992919,7cyclictest2764618-21sh21:00:013
2726354992919,7cyclictest2759505-21sh20:45:003
2726354992919,6cyclictest0-21swapper/323:20:273
2726354992918,6cyclictest0-21swapper/321:40:003
2726344992921,4cyclictest0-21swapper/123:10:001
2726344992921,4cyclictest0-21swapper/119:25:161
2726340992919,7cyclictest121ktimers/023:30:200
2726340992919,7cyclictest121ktimers/000:20:170
272634099291,7cyclictest11-21ksoftirqd/022:00:180
272634099291,7cyclictest11-21ksoftirqd/022:00:180
2726340992914,4cyclictest131rcu_preempt20:35:140
2726340992911,5cyclictest131rcu_preempt22:25:130
2726340992911,4cyclictest131rcu_preempt22:56:440
2726340992911,11cyclictest131rcu_preempt22:30:170
2726354992821,4cyclictest438-21cron22:35:003
2726354992818,7cyclictest2766329-21cron21:05:013
2726354992817,7cyclictest0-21swapper/320:40:013
2726344992822,4cyclictest231ktimers/119:20:111
2726344992820,4cyclictest0-21swapper/123:15:011
2726344992820,4cyclictest0-21swapper/123:00:151
2726344992819,5cyclictest2813020-21sendmail23:20:011
272634099289,3cyclictest131rcu_preempt22:20:220
2726340992813,4cyclictest131rcu_preempt20:40:250
2726340992812,9cyclictest131rcu_preempt23:40:180
2726340992812,4cyclictest131rcu_preempt23:35:210
2726354992718,7cyclictest2769452-21kworker/u8:2+events_unbound21:17:583
2726344992719,4cyclictest0-21swapper/121:00:001
2726344992717,7cyclictest2741765-21grep19:50:241
2726344992717,7cyclictest2741765-21grep19:50:241
2726340992714,4cyclictest131rcu_preempt23:45:240
272634099271,14cyclictest0-21swapper/020:25:200
2726340992710,4cyclictest131rcu_preempt20:00:140
2726354992620,4cyclictest597-21sshd00:05:233
2726354992619,4cyclictest2799149-21sh22:40:013
2726354992619,4cyclictest2799149-21sh22:40:013
2726354992619,4cyclictest0-21swapper/322:15:253
2726354992617,7cyclictest2754256-21latency_hist20:30:013
2726354992616,6cyclictest0-21swapper/323:50:253
2726344992620,3cyclictest0-21swapper/121:25:221
2726344992619,4cyclictest0-21swapper/121:05:261
2726344992619,3cyclictest0-21swapper/123:50:141
2726344992615,7cyclictest0-21swapper/121:30:261
2726344992615,7cyclictest0-21swapper/119:15:261
2726344992614,9cyclictest2750751-21cron20:20:001
2726354992518,3cyclictest0-21swapper/323:10:243
2726354992517,5cyclictest0-21swapper/322:45:243
2726354992517,4cyclictest0-21swapper/322:05:253
2726354992517,4cyclictest0-21swapper/322:05:253
2726354992517,4cyclictest0-21swapper/321:45:163
2726354992515,6cyclictest0-21swapper/320:20:003
272635499250,7cyclictest2783766-21fschecks_count21:55:163
2726344992519,4cyclictest2800882-21munin-plugin-st22:45:001
2726344992517,4cyclictest0-21swapper/100:29:591
2726344992516,6cyclictest2819611-21grep23:35:261
2726344992515,6cyclictest0-21swapper/122:20:241
2726344992515,6cyclictest0-21swapper/119:10:351
2726344992514,7cyclictest0-21swapper/120:35:151
2726354992418,3cyclictest0-21swapper/321:05:243
2726354992417,3cyclictest0-21swapper/319:45:173
2726354992416,5cyclictest0-21swapper/319:25:173
2726354992416,4cyclictest0-21swapper/320:10:163
2726354992416,4cyclictest0-21swapper/319:35:003
2726354992415,4cyclictest0-21swapper/319:20:003
2726344992418,3cyclictest0-21swapper/120:45:271
2726344992416,5cyclictest0-21swapper/120:25:171
2726344992415,6cyclictest0-21swapper/121:15:011
2726354992317,4cyclictest2783025-21kworker/u8:2+events_unbound22:03:033
2726354992317,4cyclictest2783025-21kworker/u8:2+events_unbound22:03:033
2726354992317,3cyclictest2831047-21ntpq00:10:193
2726354992317,3cyclictest0-21swapper/323:40:213
2726354992316,4cyclictest391ktimers/320:07:213
2726354992316,4cyclictest2833465-21kworker/u8:2+events_unbound00:23:233
2726354992316,4cyclictest2825523-21fschecks_count23:55:143
2726354992316,4cyclictest0-21swapper/322:50:223
2726354992314,5cyclictest2811718-21fschecks_time23:15:163
2726344992317,3cyclictest0-21swapper/120:40:231
2726344992316,4cyclictest2818349-21idleruntime-cro23:35:001
2726344992315,4cyclictest2815378-21fschecks_time23:25:161
2726344992315,4cyclictest0-21swapper/121:15:171
2726344992313,7cyclictest2752029-21sed20:20:241
272635499226,4cyclictest2799521-21diskstats22:40:143
2726344992216,3cyclictest0-21swapper/121:00:161
2726344992215,3cyclictest0-21swapper/120:30:261
2726344992214,5cyclictest0-21swapper/120:15:011
2726344992214,5cyclictest0-21swapper/100:15:251
2726344992211,7cyclictest0-21swapper/122:30:251
272635499215,13cyclictest2743504-21grep19:55:253
2726354992115,4cyclictest2838356-21grep00:30:243
2726354992115,4cyclictest2838356-21grep00:30:243
2726354992114,5cyclictest391ktimers/300:00:143
2726354992113,4cyclictest0-21swapper/321:50:143
2726354992110,8cyclictest2779995-21munin-run21:45:003
2726344992115,3cyclictest2737557-21fschecks_time19:40:171
2726344992115,3cyclictest0-21swapper/100:00:201
2726344992114,4cyclictest2835411-21munin-plugin-st00:25:001
2726344992113,5cyclictest0-21swapper/121:20:161
2726344992112,6cyclictest0-21swapper/121:45:271
2726354992014,3cyclictest0-21swapper/321:30:183
272635499200,5cyclictest0-21swapper/320:50:003
2726344992012,5cyclictest0-21swapper/121:35:241
2726354991912,5cyclictest391ktimers/320:50:133
2726354991912,5cyclictest391ktimers/320:50:133
2726354991910,6cyclictest2744649-21ls20:00:183
2726354991910,6cyclictest0-21swapper/321:10:203
272635499189,6cyclictest2815747-21awk23:25:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional