You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 13:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Sat Jun 20, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2877441995424,6cyclictest2930182-21grep21:40:150
2877453994835,9cyclictest391ktimers/321:15:133
287744999471,6cyclictest0-21swapper/220:20:122
287744999471,6cyclictest0-21swapper/220:20:122
2877453994635,7cyclictest2978191-21sendmail00:00:003
2877441994538,4cyclictest2992290-21sendmail00:40:010
2877449994436,5cyclictest2913591-21unixbench_singl20:50:232
2877444994435,4cyclictest0-21swapper/122:35:001
2877444994431,9cyclictest2886502-21audit_send_reply19:35:021
2877449994333,6cyclictest2896283-21grep20:00:252
2877444994336,5cyclictest2881262-21kworker/u8:2+events_unbound19:40:391
2877441994234,5cyclictest2929782-21sendmail21:40:010
2877449994131,6cyclictest2884375-21grep19:25:252
2877449994130,7cyclictest2962818-21apt-key23:15:002
2877449994130,7cyclictest2962818-21apt-key23:15:002
2877444994134,4cyclictest645-21runrttasks23:40:001
2877444994128,10cyclictest231ktimers/123:15:091
2877453994027,5cyclictest131rcu_preempt00:25:163
2877449994030,6cyclictest2981336-21unixbench_multi00:05:262
2877441994033,4cyclictest2879326-21timerwakeupswit19:15:020
2877441994028,8cyclictest2979928-21sort00:05:000
2877453993931,4cyclictest0-21swapper/322:15:003
2877453993828,6cyclictest2899735-21grep20:10:253
2877453993824,8cyclictest131rcu_preempt22:45:243
2877444993830,4cyclictest0-21swapper/100:30:001
2877441993830,5cyclictest2971761-21fschecks_count23:40:160
2877453993726,7cyclictest2917066-21sshd21:00:553
2877453993721,4cyclictest131rcu_preempt21:20:493
2877449993731,4cyclictest2888151-21kworker/u8:3+events_unbound19:39:422
2877449993719,5cyclictest131rcu_preempt23:45:232
2877444993729,4cyclictest0-21swapper/100:15:001
2877441993730,4cyclictest0-21swapper/021:15:010
2877441993730,4cyclictest0-21swapper/021:15:010
2877453993627,6cyclictest2929769-21sh21:40:013
2877444993626,6cyclictest1-21systemd19:25:361
2877441993625,7cyclictest2964093-21grep23:15:260
2877453993528,3cyclictest0-21swapper/322:55:003
2877444993528,4cyclictest0-21swapper/121:30:011
2877441993527,4cyclictest0-21swapper/022:20:000
2877441993526,5cyclictest0-21swapper/021:10:000
2877453993424,6cyclictest0-21swapper/322:05:173
2877453993415,10cyclictest131rcu_preempt20:32:003
287744999346,5cyclictest131rcu_preempt23:50:122
287744999345,4cyclictest131rcu_preempt23:30:112
2877449993427,4cyclictest0-21swapper/222:50:242
2877449993426,5cyclictest2928065-21cron21:35:002
2877449993426,4cyclictest0-21swapper/223:36:282
2877444993427,4cyclictest2923014-21latency_hist21:20:011
2877444993426,4cyclictest0-21swapper/119:55:001
2877441993424,7cyclictest2956796-21grep22:55:250
2877441993424,6cyclictest0-21swapper/022:36:180
287745399338,13cyclictest391ktimers/321:45:153
287745399338,13cyclictest391ktimers/321:45:153
2877453993323,7cyclictest391ktimers/320:55:183
2877453993320,5cyclictest131rcu_preempt23:31:413
2877453993318,5cyclictest2952155-21kworker/u8:1+events_unbound22:57:153
2877453993317,4cyclictest131rcu_preempt20:27:583
2877453993316,4cyclictest131rcu_preempt20:51:203
2877453993313,5cyclictest131rcu_preempt21:05:233
2877453993311,7cyclictest40-21ksoftirqd/300:10:253
287744999334,6cyclictest131rcu_preempt21:15:132
2877449993327,4cyclictest2957286-21munin-plugin-st23:00:002
2877449993323,7cyclictest311ktimers/221:45:122
2877449993323,7cyclictest311ktimers/221:45:112
2877449993323,6cyclictest0-21swapper/221:00:012
2877444993326,4cyclictest0-21swapper/123:30:251
2877444993322,8cyclictest2901864-21sendmail20:20:011
2877444993322,8cyclictest2901864-21sendmail20:20:001
2877441993326,4cyclictest0-21swapper/023:25:260
2877441993325,5cyclictest0-21swapper/020:50:110
287745399325,16cyclictest40-21ksoftirqd/322:00:173
287745399323,17cyclictest40-21ksoftirqd/300:35:233
2877453993224,4cyclictest0-21swapper/319:15:023
2877453993223,6cyclictest391ktimers/319:40:583
2877453993223,5cyclictest391ktimers/322:15:033
2877453993221,7cyclictest381rcuc/321:30:243
2877453993221,5cyclictest2952155-21kworker/u8:1+events_unbound23:29:423
2877453993221,4cyclictest131rcu_preempt21:29:043
2877453993219,10cyclictest391ktimers/322:20:113
2877453993218,5cyclictest131rcu_preempt20:35:213
2877453993217,3cyclictest131rcu_preempt00:25:003
2877453993216,9cyclictest0-21swapper/300:05:013
2877453993216,4cyclictest131rcu_preempt23:10:223
2877453993216,4cyclictest131rcu_preempt23:10:223
2877453993215,9cyclictest131rcu_preempt00:30:213
2877453993215,5cyclictest131rcu_preempt23:47:533
2877453993210,5cyclictest131rcu_preempt22:30:233
2877449993224,5cyclictest0-21swapper/219:15:012
2877449993222,6cyclictest0-21swapper/223:20:002
287744499328,9cyclictest2895032-21idleruntime-cro20:00:011
2877444993225,4cyclictest0-21swapper/100:35:241
2877444993224,4cyclictest0-21swapper/122:15:151
2877444993224,4cyclictest0-21swapper/122:10:161
2877444993223,6cyclictest2977738-21grep23:55:251
2877441993226,3cyclictest2974771-21kthreadd23:50:000
2877441993223,5cyclictest0-21swapper/021:25:000
2877441993222,7cyclictest0-21swapper/019:40:020
287745399318,13cyclictest40-21ksoftirqd/300:00:233
287745399316,10cyclictest40-21ksoftirqd/323:50:223
2877453993121,7cyclictest391ktimers/323:15:143
2877453993121,7cyclictest391ktimers/321:50:183
2877453993121,7cyclictest391ktimers/319:15:223
2877453993121,5cyclictest0-21swapper/321:10:163
2877453993121,5cyclictest0-21swapper/321:10:153
2877453993120,4cyclictest131rcu_preempt23:20:193
2877453993118,7cyclictest131rcu_preempt20:18:493
2877453993118,7cyclictest131rcu_preempt20:18:493
2877453993112,10cyclictest2948987-21apt22:35:093
2877453993111,11cyclictest131rcu_preempt00:15:143
2877449993124,5cyclictest2931454-21kworker/u8:0+events_unbound21:53:422
2877449993121,7cyclictest0-21swapper/200:35:002
2877449993118,5cyclictest2871690-21kworker/u8:1+events_unbound20:09:432
287744999311,3cyclictest131rcu_preempt00:35:122
287744499317,5cyclictest24-21ksoftirqd/100:00:141
287744499316,7cyclictest2918048-21sed21:05:201
2877444993125,4cyclictest2985445-21sendmail00:20:001
2877444993121,7cyclictest438-21cron20:50:001
2877444993121,7cyclictest231ktimers/122:40:111
2877444993116,4cyclictest131rcu_preempt20:40:201
2877441993124,4cyclictest0-21swapper/000:15:250
287745399308,14cyclictest381rcuc/321:40:233
287745399307,19cyclictest391ktimers/319:35:153
287745399305,14cyclictest40-21ksoftirqd/319:55:153
287745399304,7cyclictest40-21ksoftirqd/321:55:263
287745399303,17cyclictest40-21ksoftirqd/323:35:233
2877453993020,8cyclictest391ktimers/320:40:253
2877453993020,7cyclictest391ktimers/320:20:223
2877453993020,7cyclictest391ktimers/320:20:213
2877453993019,3cyclictest131rcu_preempt20:45:273
2877453993014,5cyclictest131rcu_preempt19:45:183
2877453993013,9cyclictest131rcu_preempt22:25:173
2877453993013,10cyclictest131rcu_preempt23:00:153
2877453993011,11cyclictest131rcu_preempt23:45:013
2877449993024,4cyclictest2934818-21kworker/u8:3+events_unbound22:19:422
2877449993016,9cyclictest204-21dbus-daemon21:26:002
2877449993016,5cyclictest2871690-21kworker/u8:1+events_unbound20:15:422
2877449993016,5cyclictest2871690-21kworker/u8:1+events_unbound20:15:412
2877449993015,12cyclictest2877434-21cyclictest21:00:132
2877449993014,6cyclictest2934818-21kworker/u8:3+events_unbound22:07:582
287744499306,15cyclictest24-21ksoftirqd/119:20:321
287744499303,8cyclictest24-21ksoftirqd/122:50:231
287744499303,11cyclictest2957267-21seq22:59:561
2877444993022,5cyclictest0-21swapper/123:10:241
2877444993022,5cyclictest0-21swapper/123:10:231
2877444993022,4cyclictest2953883-21latency_hist22:50:011
2877444993020,7cyclictest231ktimers/100:33:291
2877444993019,8cyclictest231ktimers/121:59:551
2877444993017,4cyclictest131rcu_preempt00:09:131
287745399297,6cyclictest40-21ksoftirqd/320:00:243
2877453992919,4cyclictest131rcu_preempt19:50:223
2877453992918,8cyclictest391ktimers/319:25:203
2877453992918,3cyclictest131rcu_preempt23:05:013
2877453992918,3cyclictest131rcu_preempt23:05:013
2877453992917,4cyclictest2871690-21kworker/u8:1+events_unbound20:08:143
2877453992915,5cyclictest131rcu_preempt19:20:113
2877453992910,10cyclictest131rcu_preempt19:33:263
2877449992923,4cyclictest2907006-21kworker/u8:2+events_unbound20:40:522
2877449992923,3cyclictest2891625-21kthreadd19:50:012
2877449992921,4cyclictest0-21swapper/222:45:252
2877449992921,4cyclictest0-21swapper/221:20:252
2877449992921,4cyclictest0-21swapper/200:10:162
2877449992920,4cyclictest0-21swapper/222:45:002
2877449992915,5cyclictest2956956-21kworker/u8:0+events_unbound23:01:132
287744999291,3cyclictest131rcu_preempt23:20:122
2877449992912,8cyclictest131rcu_preempt22:22:132
287744499299,8cyclictest2971974-21wc23:40:171
287744499299,4cyclictest131rcu_preempt20:50:211
287744499298,6cyclictest0-21swapper/123:05:051
287744499298,6cyclictest0-21swapper/123:05:041
287744499297,7cyclictest24-21ksoftirqd/100:25:001
2877444992923,4cyclictest231ktimers/121:42:341
2877444992923,3cyclictest0-21swapper/119:15:011
2877444992920,7cyclictest231ktimers/122:26:181
2877444992919,7cyclictest231ktimers/121:00:491
2877444992918,3cyclictest131rcu_preempt21:47:171
2877444992918,3cyclictest131rcu_preempt21:47:161
2877441992919,6cyclictest2937901-21grep22:00:250
2877441992910,16cyclictest2909062-21date20:40:010
2877449992822,4cyclictest2881262-21kworker/u8:2+events_unbound19:20:352
2877449992817,8cyclictest2979938-21apt-key00:05:002
2877449992816,8cyclictest1-21systemd20:30:492
287744999281,5cyclictest131rcu_preempt22:30:112
287744999281,4cyclictest131rcu_preempt19:50:122
287744499287,6cyclictest24-21ksoftirqd/123:00:151
287744499285,7cyclictest24-21ksoftirqd/119:15:191
287744499283,7cyclictest24-21ksoftirqd/120:05:281
287744499283,5cyclictest24-21ksoftirqd/123:45:181
2877444992820,4cyclictest0-21swapper/122:20:251
2877441992822,3cyclictest2981810-21latency_hist00:10:000
2877441992818,7cyclictest2896737-21sh20:05:010
287744199281,4cyclictest0-21swapper/022:20:110
2877449992720,4cyclictest0-21swapper/223:40:252
287744999271,4cyclictest131rcu_preempt21:40:112
2877449992710,5cyclictest131rcu_preempt20:35:112
287744499278,4cyclictest131rcu_preempt21:22:291
287744499277,6cyclictest24-21ksoftirqd/121:31:011
287744499276,5cyclictest24-21ksoftirqd/122:05:161
287744499275,4cyclictest24-21ksoftirqd/122:00:231
287744499275,4cyclictest24-21ksoftirqd/121:55:011
2877444992717,7cyclictest231ktimers/123:20:251
287744499271,6cyclictest24-21ksoftirqd/120:30:181
2877444992712,7cyclictest0-21swapper/123:50:311
2877444992711,9cyclictest131rcu_preempt20:27:591
287744499271,18cyclictest0-21swapper/120:10:171
2877444992711,4cyclictest131rcu_preempt20:00:141
2877441992719,4cyclictest2922836-21latency_hist21:20:000
2877449992620,3cyclictest0-21swapper/219:40:172
2877449992619,5cyclictest2974725-21kworker/u8:1+events_unbound00:20:132
2877449992616,6cyclictest0-21swapper/222:01:082
287744499269,4cyclictest131rcu_preempt23:25:161
287744499264,5cyclictest24-21ksoftirqd/120:55:221
287744499263,5cyclictest24-21ksoftirqd/120:20:191
287744499263,5cyclictest24-21ksoftirqd/120:20:181
287744499262,8cyclictest24-21ksoftirqd/121:10:011
287744499262,8cyclictest24-21ksoftirqd/121:10:011
287744499261,18cyclictest0-21swapper/120:35:121
2877444992610,3cyclictest131rcu_preempt19:35:131
2877441992621,3cyclictest2940287-21latency_hist22:10:010
2877441992620,3cyclictest0-21swapper/021:55:240
2877441992620,3cyclictest0-21swapper/021:35:010
2877441992619,4cyclictest0-21swapper/023:40:010
2877441992618,4cyclictest0-21swapper/021:45:150
2877441992618,4cyclictest0-21swapper/021:45:150
2877441992618,4cyclictest0-21swapper/020:25:010
2877441992618,4cyclictest0-21swapper/020:25:000
2877441992616,6cyclictest0-21swapper/021:25:230
2877449992519,4cyclictest2918903-21kworker/u8:0+events_unbound21:10:002
2877449992518,4cyclictest0-21swapper/221:55:252
2877449992517,5cyclictest2981764-21kworker/u8:3+events_unbound00:17:012
2877449992514,7cyclictest32-21ksoftirqd/222:35:172
2877449992510,3cyclictest131rcu_preempt22:25:162
2877444992510,5cyclictest131rcu_preempt19:45:201
2877441992518,5cyclictest208-21systemd-logind23:30:400
2877441992518,5cyclictest164-21systemd-journal19:44:410
2877441992518,4cyclictest2915699-21latency_hist21:00:000
2877441992517,5cyclictest2987166-21munin-run00:25:000
2877441992517,4cyclictest2895016-21cron20:00:000
2877441992517,4cyclictest0-21swapper/020:15:440
2877441992517,4cyclictest0-21swapper/020:15:440
287744199251,20cyclictest0-21swapper/020:12:400
2877449992418,4cyclictest2728431-21kworker/u8:3+events_unbound19:15:032
2877449992418,3cyclictest0-21swapper/200:00:012
2877449992417,4cyclictest2986835-21kworker/u8:2+events_unbound00:26:432
2877449992415,7cyclictest438-21cron20:30:002
2877449992414,3cyclictest32-21ksoftirqd/221:35:122
287744999241,12cyclictest0-21swapper/223:25:372
287744999240,11cyclictest0-21swapper/219:33:042
2877441992419,3cyclictest13550irq/42-30be0000.ethernet23:55:260
2877441992418,4cyclictest2960440-21unixbench_multi23:05:230
2877441992418,4cyclictest2960440-21unixbench_multi23:05:230
2877441992418,3cyclictest0-21swapper/023:05:000
2877441992417,5cyclictest178-21systemd-udevd20:32:400
2877441992416,5cyclictest0-21swapper/022:25:260
2877441992416,4cyclictest2890287-21fschecks_count19:45:150
287744199241,5cyclictest2940017-21apt-get22:10:060
2877441992415,6cyclictest0-21swapper/000:15:010
2877441992415,5cyclictest438-21cron23:55:000
2877441992413,6cyclictest2962814-21sed23:15:010
2877441992413,6cyclictest2962814-21sed23:15:000
287744199241,19cyclictest0-21swapper/022:31:450
287744999238,5cyclictest2888151-21kworker/u8:3+events_unbound20:10:092
287744999237,3cyclictest131rcu_preempt23:05:272
287744999237,3cyclictest131rcu_preempt23:05:262
287744999232,5cyclictest2894970-21runrttasks19:59:292
2877449992317,4cyclictest2898101-21kworker/u8:0+events_unbound20:45:092
2877441992318,3cyclictest0-21swapper/020:10:020
2877441992317,3cyclictest0-21swapper/019:35:010
2877441992316,5cyclictest178-21systemd-udevd23:20:400
2877441992316,3cyclictest0-21swapper/022:42:010
2877441992315,5cyclictest121ktimers/021:00:180
287744199231,18cyclictest0-21swapper/020:47:420
287744199231,18cyclictest0-21swapper/000:31:590
287744199230,19cyclictest0-21swapper/020:28:110
287744999227,7cyclictest131rcu_preempt22:10:272
2877441992217,3cyclictest13550irq/42-30be0000.ethernet00:25:110
2877441992217,2cyclictest0-21swapper/022:50:220
2877441992216,2cyclictest0-21swapper/019:15:170
2877441992215,3cyclictest0-21swapper/019:50:240
287744199221,3cyclictest0-21swapper/021:51:050
2877441992212,7cyclictest2883562-21tune2fs19:25:160
287744199221,19cyclictest0-21swapper/020:41:030
287744199220,18cyclictest0-21swapper/022:46:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional