You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 13:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Wed Jul 01, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2814085995243,6cyclictest2831896-21sendmail19:59:592
2814079994837,8cyclictest121ktimers/020:15:110
2814079994824,20cyclictest2893462-21unixbench_multi22:55:250
2814085994740,4cyclictest2861110-21idleruntime-cro21:25:002
2814089994640,4cyclictest2907385-21kworker/u8:0+events_unbound23:41:343
2814079994639,4cyclictest2895637-21cron23:05:010
2814089994537,5cyclictest2859390-21grep21:20:013
2814079994533,9cyclictest121ktimers/019:15:130
281407999451,6cyclictest2880150-21cat22:20:010
281407999451,4cyclictest2884293-21ntp_states22:30:220
281407999441,9cyclictest2844796-21/usr/sbin/munin20:35:200
2814079994416,25cyclictest0-21swapper/022:45:010
281407999431,5cyclictest2924028-21/usr/sbin/munin00:30:110
281407999431,13cyclictest2826572-21awk19:45:000
281407999431,10cyclictest2922446-21/usr/sbin/munin00:25:150
281407999431,10cyclictest2922446-21/usr/sbin/munin00:25:150
281407999431,10cyclictest2832889-21/usr/sbin/munin20:00:250
281407999423,5cyclictest2857914-21apt-get21:15:130
2814089994131,6cyclictest2895636-21cron23:05:013
2814079994135,3cyclictest2900764-21sendmail23:19:590
281407999413,4cyclictest2850515-21cron20:51:010
2814079994134,4cyclictest2864925-21fschecks_time21:35:140
2814079994124,14cyclictest2819729-21auditctl19:25:010
2814079994124,14cyclictest2819729-21auditctl19:25:000
281407999411,7cyclictest2872592-21/usr/sbin/munin21:55:260
281407999411,3cyclictest2906220-21chrt23:35:150
281407999411,3cyclictest2906220-21chrt23:35:150
2814081994034,4cyclictest2904593-21fschecks_time23:30:171
2814081994033,4cyclictest2833607-21munin-plugin-st20:05:011
281407999403,5cyclictest2915202-21irqstats00:00:170
281407999402,5cyclictest2880800-21/usr/sbin/munin22:20:210
281407999402,4cyclictest2869281-21cron21:46:000
281407999401,9cyclictest2861967-21/usr/sbin/munin21:25:220
281407999401,8cyclictest2885747-21/usr/sbin/munin22:35:150
281407999401,8cyclictest2822771-21sshd19:30:470
281407999401,7cyclictest2920933-21/usr/sbin/munin00:20:190
281407999401,4cyclictest2901684-21/usr/sbin/munin23:20:220
2814079994014,23cyclictest0-21swapper/021:01:000
281407999401,12cyclictest2821004-21head19:25:250
281407999401,10cyclictest2870306-21meminfo21:50:200
281407999399,27cyclictest2823920-21/usr/sbin/munin19:35:220
281407999393,5cyclictest2863154-21df_inode21:30:150
2814079993928,7cyclictest0-21swapper/022:50:170
281407999392,6cyclictest2843105-21/usr/sbin/munin20:30:210
281407999392,6cyclictest2843105-21/usr/sbin/munin20:30:210
281407999392,5cyclictest2829705-21grep19:50:250
281407999391,8cyclictest2852637-21latency_hist21:00:010
281407999391,7cyclictest2856449-21/usr/sbin/munin21:10:170
281407999391,7cyclictest2839401-21idleruntime20:20:180
281407999391,6cyclictest2831410-21grep19:55:260
281407999391,5cyclictest2913863-21/usr/sbin/munin23:55:220
281407999391,5cyclictest2827340-21irqstats19:45:170
281407999391,4cyclictest2814541-21/usr/sbin/munin19:10:190
281407999391,11cyclictest2846105-21diskstats20:40:170
281407999391,11cyclictest2835498-21cron20:10:010
281407999383,4cyclictest2847801-21df_inode20:45:150
281407999383,11cyclictest2911888-21memory23:50:200
2814079993827,6cyclictest0-21swapper/023:31:310
281407999382,5cyclictest2876453-21apt-get22:10:060
281407999381,9cyclictest164-21systemd-journal00:11:390
281407999381,8cyclictest2919241-21irqstats00:15:180
281407999381,8cyclictest2889054-21/usr/sbin/munin22:45:120
281407999381,6cyclictest2898646-21/usr/sbin/munin23:10:250
281407999381,5cyclictest2908274-21meminfo23:40:180
281407999381,5cyclictest2866555-21/usr/sbin/munin21:40:140
281407999381,5cyclictest2866555-21/usr/sbin/munin21:40:140
2814079993815,21cyclictest13550irq/42-30be0000.ethernet20:10:230
281407999381,4cyclictest2925738-21taskset00:35:140
281407999381,4cyclictest2874223-21/usr/sbin/munin22:00:270
281407999381,10cyclictest2903772-21sed23:25:250
2814089993727,7cyclictest2814071-21cyclictest19:50:253
2814085993727,7cyclictest2846970-21grep20:40:262
2814081993727,6cyclictest0-21swapper/123:00:011
2814081993725,7cyclictest0-21swapper/120:15:261
281407999372,6cyclictest2916962-21cron00:10:000
281407999371,5cyclictest2855800-21cron21:08:000
281407999371,4cyclictest2895877-21/usr/sbin/munin23:05:110
2814089993627,6cyclictest0-21swapper/322:30:003
2814089993625,8cyclictest2904167-21sh23:29:593
2814089993625,7cyclictest2888391-21grep22:40:263
2814085993626,6cyclictest2918201-21grep00:10:212
2814081993629,5cyclictest2870949-21kworker/u8:0+events_unbound21:51:111
2814081993626,6cyclictest0-21swapper/120:00:001
281407999363,5cyclictest2909636-21df23:45:150
281407999363,5cyclictest2842234-21taskset20:28:560
281407999361,7cyclictest2859735-21/usr/sbin/munin21:20:150
281407999361,4cyclictest2882020-21/usr/sbin/munin22:25:110
2814085993528,4cyclictest0-21swapper/219:35:262
2814085993525,7cyclictest2893501-21grep22:55:262
2814085993525,6cyclictest0-21swapper/222:15:002
2814085993525,5cyclictest2822733-21head19:30:262
281408199355,4cyclictest131rcu_preempt19:30:131
2814081993527,4cyclictest0-21swapper/120:50:261
281407999351,3cyclictest2875094-21df22:05:150
281408999343,5cyclictest131rcu_preempt21:45:133
2814089993428,4cyclictest2824531-21kworker/u8:0+events_unbound19:45:493
2814089993416,5cyclictest2867891-21kworker/u8:1+events_unbound21:50:103
2814085993427,4cyclictest2900768-21sendmail23:20:002
2814081993428,4cyclictest2845753-21sendmail20:40:001
2814081993425,5cyclictest0-21swapper/121:36:081
2814089993326,4cyclictest0-21swapper/322:50:013
2814089993314,3cyclictest131rcu_preempt23:10:123
2814085993324,6cyclictest2840624-21sh20:25:012
2814085993321,8cyclictest0-21swapper/219:45:012
2814081993327,4cyclictest2916939-21kworker/u8:2+events_unbound00:36:461
2814081993317,11cyclictest231ktimers/120:30:121
2814081993317,11cyclictest231ktimers/120:30:121
2814089993223,4cyclictest0-21swapper/319:15:163
2814089993222,7cyclictest2862799-21sh21:30:013
2814089993214,5cyclictest131rcu_preempt19:55:133
281408599328,4cyclictest28858132cut22:35:162
2814085993225,4cyclictest0-21swapper/223:50:262
2814085993222,6cyclictest0-21swapper/221:20:002
2814085993221,8cyclictest2904180-21idleruntime-cro23:30:002
2814081993223,5cyclictest0-21swapper/100:20:001
2814089993125,4cyclictest2847071-21kworker/u8:0+events_unbound20:42:413
2814089993125,3cyclictest2923797-21kthreadd00:30:003
2814089993125,3cyclictest2923797-21kthreadd00:30:003
2814089993124,4cyclictest0-21swapper/323:30:273
2814089993124,4cyclictest0-21swapper/321:40:263
2814089993124,4cyclictest0-21swapper/321:40:253
2814089993123,5cyclictest2850780-21kworker/u8:2+events_unbound21:00:133
2814089993121,7cyclictest2879605-21grep22:15:243
2814089993121,6cyclictest2840211-21chrt20:20:253
2814089993114,13cyclictest391ktimers/322:50:143
2814085993121,6cyclictest2911204-21sort23:50:012
2814081993125,4cyclictest2883121-21kworker/u8:2+events_unbound22:44:411
2814081993122,5cyclictest0-21swapper/122:50:281
2814089993023,4cyclictest2920385-21sh00:19:593
2814089993021,6cyclictest0-21swapper/321:55:273
2814089993021,6cyclictest0-21swapper/319:20:163
2814089993021,6cyclictest0-21swapper/319:20:163
2814089993020,7cyclictest438-21cron00:39:593
281408999301,13cyclictest0-21swapper/321:11:003
2814085993024,3cyclictest0-21swapper/220:25:152
2814085993023,4cyclictest0-21swapper/222:25:172
2814085993023,4cyclictest0-21swapper/221:25:252
2814085993023,4cyclictest0-21swapper/200:06:392
2814085993022,4cyclictest0-21swapper/221:45:262
281408199305,4cyclictest131rcu_preempt23:35:141
281408199305,4cyclictest131rcu_preempt23:35:131
2814081993023,5cyclictest2916939-21kworker/u8:2+events_unbound00:13:071
2814081993023,5cyclictest2878339-21kworker/u8:1+events_unbound22:23:401
2814081993022,4cyclictest0-21swapper/121:00:001
2814081993021,5cyclictest0-21swapper/100:30:161
281408999296,15cyclictest40-21ksoftirqd/321:30:263
281408999292,3cyclictest2849385-21apt-get20:50:123
2814089992922,5cyclictest2903877-21kworker/u8:3+events_unbound23:36:343
2814089992922,5cyclictest2903877-21kworker/u8:3+events_unbound23:36:343
2814089992922,3cyclictest0-21swapper/322:00:283
2814089992921,4cyclictest0-21swapper/323:16:273
2814089992921,4cyclictest0-21swapper/321:35:233
2814089992920,5cyclictest0-21swapper/322:11:143
2814089992917,9cyclictest391ktimers/320:55:133
2814085992922,4cyclictest2815884-21latency_hist19:15:012
2814085992921,5cyclictest0-21swapper/219:30:002
2814085992921,4cyclictest0-21swapper/221:51:112
2814081992923,4cyclictest2808513-21kworker/u8:2+events_unbound19:15:481
2814081992922,5cyclictest2837080-21kworker/u8:1+events_unbound20:20:541
2814081992922,4cyclictest0-21swapper/123:05:251
2814081992920,5cyclictest0-21swapper/121:15:281
2814081992919,6cyclictest0-21swapper/123:50:001
2814089992822,4cyclictest2884852-21kworker/u8:1+events_unbound22:36:223
281408999281,5cyclictest2925486-21cron00:35:003
2814085992821,4cyclictest0-21swapper/221:40:262
2814085992821,4cyclictest0-21swapper/221:40:252
2814081992821,5cyclictest2831616-21kworker/u8:2+events_unbound20:10:521
2814081992820,5cyclictest0-21swapper/122:10:011
2814081992819,5cyclictest0-21swapper/100:00:161
281408999272,4cyclictest131rcu_preempt21:05:133
2814089992721,4cyclictest2912836-21kworker/u8:0+events_unbound23:58:423
2814089992720,5cyclictest2837080-21kworker/u8:1+events_unbound20:35:113
2814089992718,6cyclictest2909690-21fschecks_count23:45:173
281408999271,4cyclictest131rcu_preempt19:30:133
2814085992721,3cyclictest0-21swapper/223:00:262
2814085992719,4cyclictest0-21swapper/223:35:262
2814085992719,4cyclictest0-21swapper/223:35:252
2814085992719,4cyclictest0-21swapper/200:15:172
2814085992718,7cyclictest2888540-21kworker/u8:0+events_unbound22:45:432
281408199273,4cyclictest131rcu_preempt23:20:121
2814081992719,5cyclictest0-21swapper/123:50:261
281408199271,6cyclictest131rcu_preempt19:45:121
281408999269,4cyclictest131rcu_preempt22:30:123
2814089992620,3cyclictest0-21swapper/323:05:233
2814089992619,5cyclictest2837080-21kworker/u8:1+events_unbound20:26:223
281408999261,8cyclictest0-21swapper/300:00:123
2814089992618,4cyclictest0-21swapper/322:05:263
2814089992617,7cyclictest391ktimers/320:05:043
2814089992617,6cyclictest0-21swapper/320:30:253
2814089992617,6cyclictest0-21swapper/320:30:253
2814089992616,7cyclictest2814814-21ntpq19:10:233
281408999261,14cyclictest0-21swapper/300:21:293
2814085992620,3cyclictest0-21swapper/222:15:232
2814085992619,4cyclictest0-21swapper/220:40:002
2814085992617,6cyclictest0-21swapper/200:00:152
2814085992617,5cyclictest0-21swapper/220:00:252
2814081992619,4cyclictest0-21swapper/100:20:271
2814081992618,6cyclictest231ktimers/119:35:111
2814081992617,6cyclictest0-21swapper/122:00:281
2814081992616,7cyclictest2872582-21grep21:55:271
281408199261,3cyclictest131rcu_preempt22:25:141
281408199261,3cyclictest131rcu_preempt19:45:001
281408999256,11cyclictest0-21swapper/321:20:253
2814089992519,3cyclictest0-21swapper/323:50:253
2814089992518,5cyclictest2897192-21kworker/u8:1+events_unbound23:20:113
2814089992518,5cyclictest2888540-21kworker/u8:0+events_unbound22:55:183
2814089992518,5cyclictest2815532-21kworker/u8:1+events_unbound19:29:183
2814085992519,3cyclictest0-21swapper/200:25:262
2814085992519,3cyclictest0-21swapper/200:25:252
2814085992518,4cyclictest0-21swapper/221:05:272
2814085992517,4cyclictest0-21swapper/219:45:252
2814085992516,6cyclictest311ktimers/223:55:092
2814085992516,6cyclictest0-21swapper/221:40:002
2814085992516,5cyclictest0-21swapper/222:05:152
281408199253,5cyclictest2867463-21sed21:40:261
281408199253,5cyclictest2867463-21sed21:40:261
2814081992517,6cyclictest2852895-21kworker/u8:1+events_unbound21:00:201
2814081992517,5cyclictest0-21swapper/119:25:011
2814081992517,5cyclictest0-21swapper/119:25:011
2814081992515,7cyclictest2908819-21grep23:40:231
281408999246,10cyclictest0-21swapper/320:03:013
2814089992418,4cyclictest2819627-21kworker/u8:2+events_unbound19:35:143
2814089992417,5cyclictest2878339-21kworker/u8:1+events_unbound22:23:103
2814085992416,4cyclictest0-21swapper/220:55:252
2814085992415,5cyclictest2844028-21auditctl20:35:012
2814085992415,5cyclictest2844028-21auditctl20:35:012
2814085992414,7cyclictest2883942-21cut22:30:152
2814081992418,3cyclictest629-21snmpd23:25:191
2814081992418,3cyclictest0-21swapper/123:10:181
2814081992418,3cyclictest0-21swapper/120:05:201
2814081992417,5cyclictest2916300-21kworker/u8:1+events_unbound00:06:151
2814081992417,4cyclictest0-21swapper/120:40:261
2814081992414,8cyclictest2816009-21sed19:15:011
2814081992414,7cyclictest0-21swapper/123:05:011
2814081992413,7cyclictest2880085-21sendmail22:20:001
281408999232,5cyclictest131rcu_preempt20:15:103
2814089992317,4cyclictest2831616-21kworker/u8:2+events_unbound20:10:453
2814089992317,3cyclictest0-21swapper/300:10:193
281408999230,12cyclictest0-21swapper/319:41:113
281408599232,12cyclictest2887707-21sed22:40:192
2814085992317,3cyclictest0-21swapper/220:10:012
2814085992315,5cyclictest0-21swapper/222:20:172
2814085992314,6cyclictest2902494-21grep23:25:012
2814085992314,6cyclictest0-21swapper/223:10:002
281408199238,8cyclictest131rcu_preempt21:10:011
281408199234,11cyclictest0-21swapper/121:25:151
2814081992316,5cyclictest2883121-21kworker/u8:2+events_unbound22:33:351
2814081992316,3cyclictest0-21swapper/122:45:181
2814081992315,5cyclictest0-21swapper/122:35:171
2814081992315,5cyclictest0-21swapper/121:10:211
2814081992315,4cyclictest0-21swapper/123:55:241
2814081992313,6cyclictest2869147-21grep21:45:271
2814089992216,4cyclictest2916300-21kworker/u8:1+events_unbound00:08:523
2814089992214,5cyclictest391ktimers/320:48:223
2814085992216,3cyclictest0-21swapper/200:20:242
2814085992215,5cyclictest311ktimers/220:55:012
2814085992214,5cyclictest2849154-21idleruntime-cro20:49:592
2814085992211,8cyclictest2904566-21grep23:30:162
281408199229,6cyclictest24-21ksoftirqd/119:25:141
281408199229,6cyclictest24-21ksoftirqd/100:25:161
281408199229,6cyclictest24-21ksoftirqd/100:25:151
281408199228,11cyclictest221rcuc/121:30:151
2814081992214,5cyclictest0-21swapper/122:10:181
2814081992211,8cyclictest2860442-21wc21:20:261
2814085992115,3cyclictest2818459-21fschecks_time19:20:172
2814085992115,3cyclictest2818459-21fschecks_time19:20:172
2814085992115,3cyclictest0-21swapper/221:00:252
2814085992114,4cyclictest0-21swapper/200:35:282
2814085992113,6cyclictest311ktimers/221:55:142
2814085992113,5cyclictest2816216-21wc19:15:032
281408199214,11cyclictest0-21swapper/120:25:211
281408599209,7cyclictest2899189-21wc23:15:012
2814085992014,3cyclictest0-21swapper/222:00:172
2814085992013,5cyclictest311ktimers/223:40:112
2814085992013,5cyclictest311ktimers/222:50:142
2814085992012,5cyclictest2836743-21grep20:10:272
2814085992011,6cyclictest2829758-21unixbench_singl19:50:262
2814085992010,7cyclictest2925480-21seq00:34:572
2814081992013,4cyclictest0-21swapper/119:50:211
2814081992012,5cyclictest28476822sleep120:45:121
2814085991913,3cyclictest0-21swapper/221:30:262
2814085991912,5cyclictest311ktimers/221:10:152
2814085991912,5cyclictest311ktimers/220:15:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional