You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 22:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu Apr 16, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3056548995039,7cyclictest3145748-21munin-plugin-st11:25:013
3056539995042,5cyclictest3149136-21idleruntime-cro11:35:001
3056548994934,13cyclictest391ktimers/307:55:133
3056544994937,8cyclictest3127418-21grep10:30:172
3056544994732,11cyclictest311ktimers/210:15:122
3056539994638,4cyclictest0-21swapper/109:10:241
3056544994538,5cyclictest3082150-21kworker/u8:0+events_unbound08:25:202
3056539994532,10cyclictest231ktimers/109:50:101
3056548994333,6cyclictest3088606-21sendmail08:40:013
3056548994331,8cyclictest3153820-21grep11:45:263
3056535994336,4cyclictest3084550-21unixbench_singl08:25:240
3056544994229,10cyclictest204-21dbus-daemon08:20:182
3056548994129,9cyclictest391ktimers/312:30:123
3056544994135,4cyclictest3079864-21idleruntime-cro08:15:002
3056539994131,5cyclictest0-21swapper/109:30:011
3056535994132,6cyclictest3072596-21kworker/u8:1+events_unbound08:10:150
3056535994126,12cyclictest121ktimers/011:20:150
3056535994126,12cyclictest0-21swapper/008:40:210
3056548994025,9cyclictest3067515-21latency_hist07:40:013
3056535994031,5cyclictest1-21systemd08:15:160
3056544993927,8cyclictest311ktimers/207:10:112
3056539993930,5cyclictest3060477-21latency_hist07:20:011
3056535993930,5cyclictest0-21swapper/010:45:000
3056535993930,5cyclictest0-21swapper/010:44:590
3056535993928,6cyclictest0-21swapper/011:25:270
3056548993828,5cyclictest3116540-21sendmail10:00:003
3056544993829,7cyclictest438-21cron08:50:012
3056544993828,6cyclictest0-21swapper/210:40:002
3056539993825,9cyclictest3155489-21grep11:50:241
3056544993729,4cyclictest0-21swapper/207:20:002
3056535993725,8cyclictest3091595-21grep08:45:260
305654899368,12cyclictest3133954-21date10:50:013
3056548993626,7cyclictest3135489-21sh10:55:003
3056548993611,6cyclictest40-21ksoftirqd/308:30:013
3056544993629,4cyclictest3121048-21kworker/u8:3+events_unbound11:40:432
3056544993628,4cyclictest0-21swapper/212:35:482
3056539993628,4cyclictest3112625-21unixbench_multi09:45:251
3056539993628,4cyclictest3112625-21unixbench_multi09:45:251
3056539993625,8cyclictest3130370-21sh10:39:591
3056535993629,4cyclictest3088583-21cron08:40:010
305654899357,11cyclictest3120953-21ntp_states10:10:213
3056548993527,5cyclictest1-21systemd08:20:183
3056548993527,4cyclictest0-21swapper/309:20:243
3056548993522,10cyclictest391ktimers/312:20:133
3056539993525,7cyclictest0-21swapper/110:00:001
305653999350,4cyclictest131rcu_preempt11:40:121
3056535993525,7cyclictest121ktimers/012:05:120
3056548993424,7cyclictest391ktimers/307:30:123
3056544993428,4cyclictest3086745-21kworker/u8:1+events_unbound09:30:272
3056544993426,4cyclictest0-21swapper/208:15:262
3056544993424,6cyclictest0-21swapper/211:00:262
3056539993426,4cyclictest0-21swapper/112:25:231
3056539993426,4cyclictest0-21swapper/107:40:241
3056535993426,4cyclictest0-21swapper/007:25:010
305654899339,6cyclictest40-21ksoftirqd/312:30:003
3056548993325,4cyclictest0-21swapper/312:15:243
3056548993322,8cyclictest391ktimers/312:35:133
3056544993326,4cyclictest0-21swapper/209:25:142
3056544993325,4cyclictest0-21swapper/208:40:002
3056544993325,4cyclictest0-21swapper/207:25:152
3056544993325,4cyclictest0-21swapper/207:25:152
3056544993324,6cyclictest0-21swapper/209:25:012
3056544993322,8cyclictest3090310-21audit_send_reply08:45:002
3056539993325,4cyclictest0-21swapper/110:05:271
3056539993324,6cyclictest3088609-21sendmail08:40:011
3056539993321,8cyclictest3104169-21sh09:25:011
3056548993225,4cyclictest0-21swapper/307:50:263
3056548993220,9cyclictest3138329-21timerandwakeup11:00:253
3056548993220,9cyclictest3122080-21munin-node10:15:103
3056544993224,4cyclictest0-21swapper/207:35:012
3056539993224,4cyclictest0-21swapper/108:20:161
3056535993224,4cyclictest0-21swapper/010:45:270
3056535993222,7cyclictest3140605-21cron11:10:010
3056535993221,8cyclictest3143540-21unixbench_multi11:15:250
3056548993125,4cyclictest3140618-21latency_hist11:10:003
3056548993124,3cyclictest0-21swapper/309:40:023
3056548993121,5cyclictest0-21swapper/309:15:153
3056548993118,10cyclictest3111081-21cron09:41:003
3056544993125,4cyclictest3121048-21kworker/u8:3+events_unbound10:45:342
3056544993125,3cyclictest0-21swapper/208:30:182
3056544993124,4cyclictest0-21swapper/211:50:002
3056544993124,4cyclictest0-21swapper/207:20:252
3056544993123,6cyclictest3072596-21kworker/u8:1+events_unbound08:00:132
3056539993123,4cyclictest0-21swapper/109:15:251
3056539993123,4cyclictest0-21swapper/109:05:261
3056539993122,5cyclictest0-21swapper/112:05:161
3056535993125,3cyclictest0-21swapper/012:40:000
3056535993124,4cyclictest0-21swapper/010:15:000
3056535993123,4cyclictest0-21swapper/010:00:000
3056535993121,7cyclictest3107586-21cut09:35:010
3056548993022,4cyclictest0-21swapper/312:10:243
3056548993021,5cyclictest0-21swapper/307:25:243
3056548993021,5cyclictest0-21swapper/307:25:233
3056548993020,7cyclictest3069688-21/usr/sbin/munin07:45:163
3056548993018,9cyclictest3118990-21/usr/sbin/munin10:05:193
3056548993017,10cyclictest3140956-21/usr/sbin/munin11:10:153
3056548993016,11cyclictest3147265-21cron11:28:013
3056548993016,10cyclictest3068748-21grep07:40:243
3056548993012,13cyclictest3094721-21grep08:55:223
3056544993024,4cyclictest3145351-21kworker/u8:1+events_unbound11:21:352
3056544993022,5cyclictest0-21swapper/212:20:142
3056544993022,4cyclictest0-21swapper/211:55:162
3056544993022,4cyclictest0-21swapper/208:50:262
3056539993023,4cyclictest0-21swapper/112:30:251
3056539993022,5cyclictest0-21swapper/112:05:001
3056539993022,5cyclictest0-21swapper/112:05:001
3056539993022,4cyclictest0-21swapper/108:00:251
3056539993020,7cyclictest3108204-21fschecks_time09:35:161
3056539993020,6cyclictest0-21swapper/110:20:241
3056535993020,7cyclictest3058162-21cron07:15:000
305654899297,4cyclictest30817222wc08:20:023
3056548992921,4cyclictest0-21swapper/309:35:013
3056548992920,7cyclictest391ktimers/311:30:113
3056548992918,7cyclictest0-21swapper/310:40:243
3056548992918,7cyclictest0-21swapper/310:40:243
3056548992917,9cyclictest3155325-21/usr/sbin/munin11:50:243
3056548992917,9cyclictest3150536-21sshd11:36:253
305654499298,4cyclictest31246022switchtime10:20:222
3056544992922,4cyclictest0-21swapper/207:40:232
3056544992921,5cyclictest3109056-21unixbench_singl09:35:262
3056544992921,4cyclictest0-21swapper/210:50:262
3056539992923,4cyclictest3089412-21kworker/u8:0+events_unbound08:45:231
3056539992922,4cyclictest0-21swapper/111:10:261
3056539992921,6cyclictest3051270-21kworker/u8:2+events_unbound07:45:121
3056539992920,6cyclictest0-21swapper/107:25:001
3056539992911,8cyclictest24-21ksoftirqd/111:25:131
3056535992922,5cyclictest3166480-21kworker/u8:4+events_unbound12:25:500
3056535992921,4cyclictest3130373-21auditctl10:40:000
3056535992921,4cyclictest0-21swapper/011:45:260
3056535992921,4cyclictest0-21swapper/008:30:170
3056535992911,15cyclictest3076321-21latency_hist08:05:010
3056548992822,3cyclictest3076697-21apt-get08:05:143
3056548992818,7cyclictest3089284-21/usr/sbin/munin08:40:193
3056548992818,7cyclictest3078385-21munin-node08:10:113
3056548992817,7cyclictest0-21swapper/307:20:123
3056548992816,9cyclictest3129143-21/usr/sbin/munin10:35:183
3056548992816,9cyclictest1-21systemd11:40:433
3056548992814,11cyclictest3093161-21/usr/sbin/munin08:50:263
305654499285,11cyclictest3126569-21unixbench_singl10:25:242
3056544992822,4cyclictest3166480-21kworker/u8:4+events_unbound12:30:212
3056544992821,5cyclictest3069207-21kworker/u8:3+events_unbound07:50:052
3056544992820,4cyclictest0-21swapper/210:05:272
305654499281,4cyclictest0-21swapper/208:55:122
3056544992811,5cyclictest3130300-21kworker/u8:0+events_unbound11:32:172
305653999289,11cyclictest24-21ksoftirqd/109:40:051
3056539992821,4cyclictest0-21swapper/110:40:161
3056539992821,4cyclictest0-21swapper/110:40:161
3056539992820,4cyclictest0-21swapper/108:00:011
3056539992819,6cyclictest0-21swapper/107:50:251
3056539992819,4cyclictest0-21swapper/111:00:261
3056539992814,8cyclictest131rcu_preempt08:59:431
3056535992820,5cyclictest3109075-21unixbench_singl09:35:260
3056535992820,5cyclictest0-21swapper/009:40:260
3056535992820,4cyclictest438-21cron07:40:010
3056548992717,7cyclictest3137141-21sshd10:59:263
3056548992717,7cyclictest3075990-21cron08:01:593
3056548992715,9cyclictest3104514-21df09:25:133
3056548992715,9cyclictest3059130-21/usr/sbin/munin07:15:163
3056548992713,11cyclictest3096108-21/usr/sbin/munin09:00:173
3056548992711,12cyclictest3158391-21wc12:00:203
3056548992711,12cyclictest3158391-21wc12:00:193
3056544992721,3cyclictest0-21swapper/208:00:012
3056544992719,6cyclictest3111135-21kworker/u8:2+events_unbound10:10:162
3056544992719,5cyclictest0-21swapper/210:00:242
3056544992718,6cyclictest3100572-21ls09:15:012
3056544992717,7cyclictest3164386-21sed12:15:242
3056544992715,4cyclictest131rcu_preempt12:15:012
305653999274,4cyclictest131rcu_preempt12:20:251
305653999272,4cyclictest131rcu_preempt08:30:261
3056539992720,4cyclictest3164909-21grep12:20:001
3056539992719,5cyclictest3121449-21head10:10:251
305653999271,13cyclictest0-21swapper/108:10:121
3056535992722,3cyclictest3135510-21idleruntime-cro10:55:000
3056535992720,4cyclictest3067965-21fschecks_time07:40:150
3056535992719,4cyclictest3163566-21grep12:15:150
3056535992719,4cyclictest0-21swapper/007:30:150
305654899266,12cyclictest0-21swapper/309:07:393
305654899265,5cyclictest3090660-21diskstats08:45:163
3056548992619,4cyclictest3113347-21apt-get09:50:113
3056548992617,7cyclictest3156307-21/usr/sbin/munin11:55:153
3056548992616,7cyclictest3142643-21/usr/sbin/munin11:15:143
3056548992616,7cyclictest3125561-21/usr/sbin/munin10:25:133
3056548992616,7cyclictest3117319-21/usr/sbin/munin10:00:173
3056548992616,7cyclictest3057212-21/usr/sbin/munin07:10:193
3056548992615,8cyclictest3127208-21apt-get10:30:133
3056548992612,11cyclictest3160159-21/usr/sbin/munin12:05:193
305654499265,6cyclictest131rcu_preempt08:05:122
305654499262,5cyclictest32-21ksoftirqd/207:35:092
3056544992619,5cyclictest3145351-21kworker/u8:1+events_unbound11:35:272
3056544992616,5cyclictest311ktimers/209:40:122
3056544992615,7cyclictest3168542-21grep12:30:002
305654499261,12cyclictest0-21swapper/209:56:232
3056539992620,3cyclictest0-21swapper/112:35:241
3056539992619,5cyclictest3130300-21kworker/u8:0+events_unbound11:36:321
3056539992619,4cyclictest0-21swapper/110:55:261
3056539992616,7cyclictest3058419-21sed07:15:011
3056535992620,3cyclictest3170397-21latency_hist12:35:010
3056535992619,5cyclictest0-21swapper/011:55:240
3056535992619,4cyclictest13850irq/36-mmc109:15:010
3056535992618,6cyclictest3086745-21kworker/u8:1+events_unbound09:00:250
305653599261,2cyclictest0-21swapper/008:20:120
305653599261,22cyclictest3161730-21seq12:10:110
3056548992515,7cyclictest3111954-21if_eth009:45:173
3056548992515,7cyclictest3111954-21if_eth009:45:163
3056544992519,4cyclictest3150822-21kworker/u8:2+events_unbound11:53:492
3056544992519,3cyclictest0-21swapper/212:05:132
3056544992519,3cyclictest0-21swapper/207:45:252
3056544992518,5cyclictest53-21kcompactd011:25:132
3056544992518,5cyclictest3130300-21kworker/u8:0+events_unbound11:05:302
3056544992516,4cyclictest3119808-21kworker/u8:1+events_unbound10:40:402
3056544992516,4cyclictest3119808-21kworker/u8:1+events_unbound10:40:392
305653999252,3cyclictest131rcu_preempt10:15:131
3056539992519,4cyclictest3123393-21kworker/u8:2+events_unbound10:25:311
3056539992519,4cyclictest3111135-21kworker/u8:2+events_unbound10:00:301
3056539992519,3cyclictest0-21swapper/110:30:251
3056539992518,5cyclictest3078515-21kworker/u8:3+events_unbound08:15:171
3056539992517,5cyclictest0-21swapper/111:50:001
3056539992515,7cyclictest3096462-21wc09:00:221
305653999250,3cyclictest131rcu_preempt11:20:131
3056535992519,3cyclictest0-21swapper/011:00:250
3056535992519,3cyclictest0-21swapper/009:45:240
3056535992519,3cyclictest0-21swapper/009:45:240
3056535992518,3cyclictest0-21swapper/010:20:240
3056535992518,3cyclictest0-21swapper/008:06:290
3056535992517,5cyclictest0-21swapper/009:20:160
3056535992517,4cyclictest0-21swapper/009:15:150
305653599251,20cyclictest0-21swapper/009:27:290
3056544992418,4cyclictest3130300-21kworker/u8:0+events_unbound10:56:582
3056544992418,4cyclictest3089412-21kworker/u8:0+events_unbound09:03:132
3056544992418,3cyclictest0-21swapper/211:10:222
3056544992411,6cyclictest131rcu_preempt09:45:252
3056544992411,6cyclictest131rcu_preempt09:45:252
305654499240,3cyclictest0-21swapper/209:15:132
305653999243,4cyclictest131rcu_preempt08:25:161
3056539992418,3cyclictest0-21swapper/108:05:221
3056539992414,7cyclictest3065111-21grep07:30:261
305653999241,12cyclictest0-21swapper/108:51:011
305653999241,12cyclictest0-21swapper/107:35:151
3056535992418,3cyclictest645-21runrttasks07:45:240
3056535992415,5cyclictest13550irq/42-30be0000.ethernet07:25:150
3056535992415,5cyclictest13550irq/42-30be0000.ethernet07:25:140
305653599241,3cyclictest3113347-21apt-get09:50:110
305653599241,19cyclictest0-21swapper/011:11:200
305653599241,19cyclictest0-21swapper/007:50:550
3056544992317,3cyclictest0-21swapper/209:05:202
305654499231,12cyclictest0-21swapper/209:51:592
305654499230,11cyclictest0-21swapper/211:16:532
305653999239,8cyclictest131rcu_preempt10:50:011
3056539992315,5cyclictest0-21swapper/107:25:151
3056539992315,5cyclictest0-21swapper/107:25:151
305653999231,4cyclictest131rcu_preempt09:30:251
305653999231,13cyclictest0-21swapper/111:55:231
3056535992318,2cyclictest0-21swapper/007:15:030
3056535992317,4cyclictest3117224-21idleruntime10:00:160
3056535992316,3cyclictest3123581-21sendmail10:20:010
3056535992316,3cyclictest0-21swapper/011:39:030
3056535992316,3cyclictest0-21swapper/009:09:430
3056535992316,3cyclictest0-21swapper/008:52:290
3056535992315,4cyclictest3149128-21cron11:34:590
3056535992314,7cyclictest3074594-21sed08:00:000
305653599230,19cyclictest0-21swapper/011:41:250
305653599230,19cyclictest0-21swapper/010:25:380
305653599230,19cyclictest0-21swapper/008:58:160
3056544992215,5cyclictest3121048-21kworker/u8:3+events_unbound12:05:002
3056544992215,5cyclictest3121048-21kworker/u8:3+events_unbound12:04:592
3056539992216,4cyclictest3121048-21kworker/u8:3+events_unbound11:05:201
3056539992215,5cyclictest231ktimers/110:50:121
305653999221,3cyclictest3144023-21munin-run11:20:011
3056535992215,3cyclictest0-21swapper/012:00:140
3056535992215,3cyclictest0-21swapper/012:00:140
3056535992215,3cyclictest0-21swapper/010:33:560
3056535992214,5cyclictest0-21swapper/010:55:180
3056535992213,5cyclictest0-21swapper/011:50:250
305653599213,13cyclictest13550irq/42-30be0000.ethernet10:05:130
3056535992115,3cyclictest0-21swapper/012:20:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional