You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 06:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Jan 07, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3278835995447,4cyclictest3393356-21sendmail00:40:001
3278842994940,6cyclictest3287503-21grep19:30:263
3278842994915,30cyclictest3325357-21grep21:20:273
3278838994934,11cyclictest311ktimers/223:20:132
3278835994941,5cyclictest3337749-21munin-plugin-st22:00:001
3278831994935,10cyclictest121ktimers/019:20:130
3278842994828,16cyclictest0-21swapper/322:40:153
3278835994838,7cyclictest3342453-21grep22:10:251
3278838994740,4cyclictest3358701-21munin-plugin-st23:00:012
3278831994737,7cyclictest3303333-21sendmail20:20:010
3278838994638,6cyclictest311ktimers/220:00:132
3278838994532,9cyclictest311ktimers/223:45:112
3278831994534,8cyclictest121ktimers/019:30:130
327884299443,11cyclictest3299558-21cron20:06:003
327884299432,9cyclictest3330100-21/usr/sbin/munin21:35:223
3278838994333,6cyclictest0-21swapper/223:25:252
3278838994332,8cyclictest3344594-21sh22:20:012
327884299428,7cyclictest3296257-21seq19:57:323
3278842994235,4cyclictest3286218-21munin-plugin-st19:30:003
3278842994214,25cyclictest0-21swapper/322:30:233
3278842994214,25cyclictest0-21swapper/322:30:223
3278838994235,4cyclictest0-21swapper/221:50:002
327884299411,12cyclictest3294334-21head19:50:263
3278842994111,28cyclictest0-21swapper/320:50:183
3278842994111,28cyclictest0-21swapper/320:50:183
327884299407,3cyclictest0-21swapper/321:50:233
3278842994015,22cyclictest0-21swapper/321:10:003
327884299401,10cyclictest3326543-21ntpq21:25:203
3278831994028,8cyclictest3344610-21sendmail22:20:000
327884299396,5cyclictest0-21swapper/300:25:253
3278842993932,4cyclictest3334338-21idleruntime-cro21:50:003
327884299392,8cyclictest3385988-21sshd00:16:403
3278842993918,18cyclictest0-21swapper/323:20:003
327884299391,11cyclictest3311699-21users20:40:283
327884299391,11cyclictest3306570-21users20:25:323
3278842993910,3cyclictest0-21swapper/320:10:203
3278838993930,5cyclictest3294771-21idleruntime-cro19:55:002
3278838993929,7cyclictest3379253-21sendmail00:00:002
3278835993928,7cyclictest3387538-21grep00:20:251
3278831993917,14cyclictest0-21swapper/020:55:260
3278831993917,14cyclictest0-21swapper/020:55:260
327884299389,5cyclictest0-21swapper/300:35:193
327884299388,5cyclictest3338106-21/usr/sbin/munin22:00:163
327884299382,7cyclictest3292772-21sshd19:46:423
327884299382,11cyclictest3341567-21fschecks_count22:10:173
327884299382,10cyclictest3377861-21df_inode23:55:153
327884299381,9cyclictest3360805-21grep23:05:143
327884299381,9cyclictest3360805-21grep23:05:143
327884299381,9cyclictest3357336-21/usr/sbin/munin22:55:163
327884299381,9cyclictest3331336-21fschecks_time21:40:173
327884299381,11cyclictest3374440-21df23:45:143
327884299381,11cyclictest3317803-21irqstats21:00:203
327884299381,10cyclictest3365910-21diskstats23:20:153
327884299381,10cyclictest3328293-21ntp_states21:30:213
327884299381,10cyclictest3312441-21/usr/sbin/munin20:45:143
327884299381,10cyclictest3312441-21/usr/sbin/munin20:45:143
3278835993828,7cyclictest3280467-21auditctl19:15:011
3278835993827,8cyclictest3380961-21munin-plugin-st00:05:001
3278831993828,4cyclictest131rcu_preempt20:35:000
3278831993825,10cyclictest121ktimers/000:25:120
3278831993810,21cyclictest131rcu_preempt23:50:140
327884299378,4cyclictest3382269-21timerandwakeup00:05:243
327884299373,7cyclictest204-21dbus-daemon22:37:073
327884299373,7cyclictest204-21dbus-daemon22:37:073
327884299371,9cyclictest3356534-21grep22:50:263
327884299371,9cyclictest3355277-21cron22:50:003
327884299371,9cyclictest3344340-21sshd22:17:053
327884299371,10cyclictest3307751-21ntp_states20:30:213
3278842993710,2cyclictest0-21swapper/323:37:093
3278842993710,24cyclictest0-21swapper/322:25:003
3278835993727,7cyclictest231ktimers/121:00:121
3278835993727,5cyclictest0-21swapper/123:02:161
3278831993730,4cyclictest3286217-21munin-run19:30:000
3278831993720,3cyclictest131rcu_preempt19:57:320
327884299367,4cyclictest3340321-21proc_pri22:05:223
327884299363,8cyclictest3370475-21sshd23:32:403
327884299361,9cyclictest3290821-21grep19:40:253
327884299361,8cyclictest3368539-21grep23:25:253
327884299361,11cyclictest3322821-21/usr/sbin/munin21:15:173
327884299361,11cyclictest3297167-21meminfo20:00:193
327884299361,11cyclictest3283151-21diskstats19:20:173
3278842993610,3cyclictest3360421-21latency_hist23:05:003
3278842993610,3cyclictest3278822-21cyclictest00:05:023
3278842993610,2cyclictest0-21swapper/320:20:133
3278838993628,4cyclictest0-21swapper/200:17:342
3278838993625,7cyclictest3308201-21grep20:30:252
3278835993628,4cyclictest0-21swapper/119:40:161
3278835993624,7cyclictest0-21swapper/121:40:011
3278831993628,5cyclictest121ktimers/020:35:120
3278831993625,8cyclictest3358704-21sendmail23:00:000
3278831993612,4cyclictest131rcu_preempt23:25:000
327884299358,3cyclictest3337249-21unixbench_multi21:55:263
327884299355,5cyclictest0-21swapper/320:35:243
327884299353,10cyclictest3316341-21awk20:55:243
327884299353,10cyclictest3316341-21awk20:55:233
327884299352,9cyclictest3281749-21meminfo19:15:193
327884299351,8cyclictest3280459-21cron19:15:003
3278842993512,19cyclictest0-21swapper/323:52:273
3278838993527,4cyclictest0-21swapper/200:25:152
3278835993525,7cyclictest3369412-21cut23:30:161
3278835993525,5cyclictest0-21swapper/100:20:011
327883199358,21cyclictest11-21ksoftirqd/022:55:000
3278831993522,6cyclictest131rcu_preempt20:43:190
327884299342,11cyclictest3289615-21cron19:40:003
327884299341,11cyclictest3348232-21ls22:30:003
327884299341,10cyclictest3383791-21wc00:10:233
3278838993426,4cyclictest0-21swapper/222:30:252
3278838993426,4cyclictest0-21swapper/222:30:252
3278838993426,4cyclictest0-21swapper/219:55:172
3278835993427,5cyclictest3296419-21kworker/u8:0+events_unbound20:01:441
3278835993427,3cyclictest0-21swapper/120:30:001
3278835993425,7cyclictest231ktimers/100:30:131
327883199349,4cyclictest131rcu_preempt23:10:160
327883199344,4cyclictest11-21ksoftirqd/021:40:010
3278831993423,7cyclictest0-21swapper/019:50:150
3278831993422,4cyclictest131rcu_preempt23:02:400
3278831993414,5cyclictest131rcu_preempt00:40:000
327884299332,10cyclictest3320993-21df_inode21:10:153
327884299331,11cyclictest3390980-21sed00:30:243
327884299331,11cyclictest3386980-21meminfo00:20:193
327884299331,11cyclictest3302916-21users20:15:273
3278838993327,4cyclictest3387766-21kworker/u8:3+events_unbound00:22:372
3278838993322,8cyclictest3325367-21grep21:20:272
3278835993323,7cyclictest3351646-21munin-plugin-st22:40:001
3278835993323,7cyclictest3351646-21munin-plugin-st22:40:001
3278835993322,8cyclictest3365555-21sendmail23:20:001
327883199338,7cyclictest11-21ksoftirqd/021:00:140
3278831993326,4cyclictest3278822-21cyclictest19:15:010
3278831993325,4cyclictest0-21swapper/022:25:160
3278831993322,4cyclictest131rcu_preempt20:25:240
3278831993321,5cyclictest131rcu_preempt23:47:550
3278831993321,4cyclictest131rcu_preempt22:09:160
3278831993315,3cyclictest131rcu_preempt23:05:180
3278831993315,3cyclictest131rcu_preempt23:05:170
3278831993314,7cyclictest131rcu_preempt21:10:260
327884299329,5cyclictest391ktimers/323:10:153
3278838993224,4cyclictest3377092-21sed23:50:232
3278835993222,6cyclictest0-21swapper/120:15:251
3278835993220,8cyclictest3310384-21sendmail20:40:001
327883199323,6cyclictest11-21ksoftirqd/020:25:010
3278831993224,4cyclictest0-21swapper/023:35:010
3278831993215,10cyclictest131rcu_preempt21:50:180
3278831993214,4cyclictest131rcu_preempt21:26:270
3278831993213,11cyclictest131rcu_preempt00:05:110
3278838993124,3cyclictest3273272-21kworker/u8:0+events_unbound19:31:422
3278838993124,3cyclictest0-21swapper/219:10:192
3278838993123,4cyclictest0-21swapper/220:11:432
3278838993121,6cyclictest0-21swapper/223:30:252
3278838993119,6cyclictest0-21swapper/220:15:252
327883199319,6cyclictest131rcu_preempt21:55:210
327883199319,5cyclictest131rcu_preempt19:15:200
327883199318,9cyclictest11-21ksoftirqd/021:40:140
327883199315,7cyclictest11-21ksoftirqd/020:10:270
327883199314,8cyclictest11-21ksoftirqd/023:45:000
3278831993121,7cyclictest121ktimers/021:20:230
3278831993115,4cyclictest131rcu_preempt23:25:360
327884299309,5cyclictest3373417-21cat23:40:243
3278835993021,6cyclictest3327481-21cron21:30:001
3278831993024,3cyclictest121ktimers/019:40:000
3278831993023,3cyclictest0-21swapper/021:15:260
3278831993021,4cyclictest0-21swapper/022:10:250
3278831993020,7cyclictest121ktimers/000:15:170
3278831993016,4cyclictest131rcu_preempt00:25:010
3278831993014,4cyclictest131rcu_preempt22:02:000
3278831993013,9cyclictest131rcu_preempt22:40:130
3278831993013,4cyclictest131rcu_preempt23:55:230
3278831993011,11cyclictest131rcu_preempt21:30:180
3278831993010,4cyclictest131rcu_preempt00:10:200
327883199300,27cyclictest0-21swapper/022:20:120
3278838992922,5cyclictest3310084-21kworker/u8:1+events_unbound21:11:562
3278838992921,5cyclictest0-21swapper/220:50:242
3278838992921,5cyclictest0-21swapper/220:50:242
3278838992920,5cyclictest3374103-21idleruntime-cro23:45:012
3278838992920,5cyclictest0-21swapper/219:21:372
3278838992916,5cyclictest3323795-21kworker/u8:2+events_unbound21:27:032
327883899291,4cyclictest131rcu_preempt21:15:132
3278835992922,3cyclictest0-21swapper/121:30:201
3278835992920,5cyclictest0-21swapper/122:45:011
3278835992919,7cyclictest3288325-21tune2fs19:35:161
3278835992916,9cyclictest0-21swapper/121:25:001
327883199299,5cyclictest131rcu_preempt20:50:180
327883199299,5cyclictest131rcu_preempt20:50:170
3278831992918,7cyclictest3351198-21grep22:35:230
3278831992918,7cyclictest3351198-21grep22:35:230
3278831992915,4cyclictest131rcu_preempt21:08:280
3278831992914,5cyclictest131rcu_preempt22:45:150
3278831992914,3cyclictest131rcu_preempt23:35:170
3278831992913,10cyclictest0-21swapper/023:17:260
3278831992911,4cyclictest131rcu_preempt20:00:200
3278831992910,4cyclictest131rcu_preempt20:05:200
3278838992822,3cyclictest0-21swapper/200:00:252
3278838992820,6cyclictest3327478-21kworker/u8:0+events_unbound21:31:592
3278838992818,6cyclictest3359987-21grep23:00:262
3278835992820,6cyclictest231ktimers/100:05:131
3278835992820,4cyclictest0-21swapper/121:20:011
3278835992819,6cyclictest3294337-21grep19:50:251
3278835992818,6cyclictest0-21swapper/120:10:261
3278835992817,7cyclictest3349467-21grep22:30:251
3278835992817,7cyclictest3349467-21grep22:30:251
3278831992817,4cyclictest131rcu_preempt00:30:140
3278831992813,8cyclictest131rcu_preempt22:30:150
3278831992813,8cyclictest131rcu_preempt22:30:140
3278831992812,4cyclictest131rcu_preempt00:04:180
3278831992811,4cyclictest131rcu_preempt19:40:210
3278831992810,8cyclictest131rcu_preempt20:45:140
3278831992810,8cyclictest131rcu_preempt20:45:140
3278838992719,4cyclictest0-21swapper/222:45:162
3278838992719,4cyclictest0-21swapper/222:10:252
3278838992718,7cyclictest3294690-21kworker/u8:1+events_unbound20:08:242
3278838992717,6cyclictest3318512-21grep21:00:282
3278835992716,7cyclictest3320165-21grep21:05:261
3278835992716,7cyclictest0-21swapper/121:50:161
3278831992715,3cyclictest131rcu_preempt19:45:210
3278831992711,5cyclictest131rcu_preempt21:45:200
3278838992617,5cyclictest0-21swapper/222:20:252
3278838992616,7cyclictest3317234-21basename21:00:002
3278838992616,7cyclictest3317234-21basename21:00:002
3278835992619,4cyclictest3346519-21sh22:25:011
3278835992618,5cyclictest3280202-21kworker/u8:2+events_unbound19:15:111
3278835992618,4cyclictest3375398-21head23:45:261
3278835992618,4cyclictest0-21swapper/122:05:251
3278838992520,3cyclictest3337197-21runrttasks21:55:262
3278838992519,4cyclictest3320303-21kworker/u8:2+events_unbound21:06:522
3278838992519,3cyclictest0-21swapper/223:10:222
3278838992515,7cyclictest3371904-21grep23:35:252
3278838992515,6cyclictest0-21swapper/220:40:262
3278838992514,7cyclictest3292615-21grep19:45:252
3278835992518,4cyclictest0-21swapper/123:50:141
3278835992517,4cyclictest0-21swapper/121:45:231
3278835992515,7cyclictest3313312-21grep20:45:241
3278835992515,7cyclictest3313312-21grep20:45:241
3278835992515,7cyclictest0-21swapper/100:30:011
3278838992418,4cyclictest3360040-21kworker/u8:0+events_unbound23:06:412
3278838992418,4cyclictest3360040-21kworker/u8:0+events_unbound23:06:412
3278838992417,3cyclictest0-21swapper/220:25:022
3278838992417,3cyclictest0-21swapper/200:35:272
3278838992416,4cyclictest0-21swapper/221:36:582
327883899241,5cyclictest131rcu_preempt00:10:122
3278838992414,7cyclictest3391206-21grep00:30:252
3278835992416,4cyclictest0-21swapper/123:40:001
3278835992415,6cyclictest0-21swapper/123:40:251
3278835992415,6cyclictest0-21swapper/121:00:001
3278835992415,6cyclictest0-21swapper/121:00:001
3278835992415,4cyclictest0-21swapper/122:50:251
3278835992414,7cyclictest3344133-21grep22:15:241
3278835992414,6cyclictest0-21swapper/100:10:251
3278838992319,2cyclictest0-21swapper/219:15:212
3278838992317,3cyclictest0-21swapper/222:05:252
3278838992317,3cyclictest0-21swapper/222:05:002
3278838992317,3cyclictest0-21swapper/220:40:002
3278838992317,3cyclictest0-21swapper/219:40:252
3278838992315,5cyclictest3335562-21grep21:50:262
3278838992314,6cyclictest3332620-21sh21:45:002
3278838992313,7cyclictest204-21dbus-daemon20:26:462
3278835992317,3cyclictest0-21swapper/120:30:171
3278835992317,3cyclictest0-21swapper/119:30:221
3278835992316,3cyclictest0-21swapper/123:10:251
3278835992314,6cyclictest3354726-21grep22:45:251
3278835992314,5cyclictest0-21swapper/122:25:261
3278835992314,5cyclictest0-21swapper/120:55:011
3278835992314,5cyclictest0-21swapper/120:55:001
327883899222,16cyclictest3312869-21ntpq20:45:212
327883899222,16cyclictest3312869-21ntpq20:45:202
3278838992218,2cyclictest3346203-21kworker/u8:1+events_unbound22:37:092
3278838992218,2cyclictest3346203-21kworker/u8:1+events_unbound22:37:082
327883899221,4cyclictest3355959-21meminfo22:50:182
3278835992216,3cyclictest0-21swapper/123:55:141
3278835992215,4cyclictest3321106-21fschecks_time21:10:171
3278835992214,5cyclictest231ktimers/119:30:001
3278835992212,7cyclictest3283388-21tr19:20:191
3278835992210,8cyclictest3292596-21grep19:45:251
327883599220,12cyclictest0-21swapper/120:05:331
3278838992115,3cyclictest0-21swapper/219:35:132
3278838992115,3cyclictest0-21swapper/219:25:302
3278838992114,4cyclictest3364614-21ntpq23:15:212
3278835992115,3cyclictest0-21swapper/122:55:171
3278835992112,6cyclictest0-21swapper/119:55:221
3278838992012,5cyclictest0-21swapper/222:25:172
3278835992015,3cyclictest0-21swapper/122:00:241
3278835992012,5cyclictest0-21swapper/123:25:261
3278835992011,6cyclictest0-21swapper/123:20:241
3278835991910,5cyclictest0-21swapper/120:40:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional