You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 19:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue May 12, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
752521540,3sleep20-21swapper/212:27:332
752521540,3sleep20-21swapper/212:27:332
399121050,3sleep2351rcuc/212:17:362
399121050,3sleep2351rcuc/212:17:362
155912970,3sleep20-21swapper/209:53:002
155912970,3sleep20-21swapper/209:53:002
37692780,4sleep30-21swapper/309:22:423
37692780,4sleep30-21swapper/309:22:423
291842750,3sleep30-21swapper/309:02:483
291842750,3sleep30-21swapper/309:02:483
93402710,2sleep20-21swapper/208:12:342
93402710,2sleep20-21swapper/208:12:342
239902710,2sleep30-21swapper/307:22:463
239902710,2sleep30-21swapper/307:22:463
215122660,2chrt0-21swapper/308:42:463
215122660,2chrt0-21swapper/308:42:463
305932650,2sleep30-21swapper/307:42:363
305932650,2sleep30-21swapper/307:42:363
314392620,2chrt0-21swapper/312:02:463
314392620,2chrt0-21swapper/312:02:463
314392620,2chrt0-21swapper/312:02:463
303072610,3chrt0-21swapper/209:07:382
118172610,3sleep30-21swapper/312:37:393
311112520,2sleep20-21swapper/210:37:352
311112520,2sleep20-21swapper/210:37:352
87372450,2chrt8738-21cpuspeed09:37:361
87372450,2chrt8738-21cpuspeed09:37:361
195062440,3sleep20-21swapper/208:37:442
195062440,3sleep20-21swapper/208:37:442
19298994119,20cyclictest22271-21kernelversion10:12:401
19298994119,20cyclictest22271-21kernelversion10:12:401
155872410,4chrt0-21swapper/208:27:422
54642400,2sleep05466-21date08:02:310
54642400,2sleep05466-21date08:02:310
257162400,2chrt25715-21pluginstate11:47:430
257162400,2chrt25715-21pluginstate11:47:430
257162400,2chrt25715-21pluginstate11:47:430
129272400,3sleep30-21swapper/309:47:393
129272400,3sleep30-21swapper/309:47:393
125712400,2sleep112674-21grep11:12:441
125712400,2sleep112674-21grep11:12:441
19298993932,5cyclictest26581-21latency_hist11:52:231
19298993932,5cyclictest26581-21latency_hist11:52:231
19298993932,5cyclictest26581-21latency_hist11:52:231
147332390,2sleep30-21swapper/308:27:283
147332390,2sleep30-21swapper/308:27:283
147332390,2sleep30-21swapper/308:27:283
61422380,6sleep16145-21unixbench_singl09:27:471
61422380,6sleep16145-21unixbench_singl09:27:471
238552380,4sleep20-21swapper/207:22:452
238552380,4sleep20-21swapper/207:22:452
19298993820,17cyclictest19288-21cyclictest07:57:391
19298993820,17cyclictest19288-21cyclictest07:57:391
1929899381,9cyclictest1222-21sed12:07:491
1929899381,9cyclictest1222-21sed12:07:491
1929899381,11cyclictest5732-21fschecks_time08:02:371
1929899381,11cyclictest5732-21fschecks_time08:02:371
1929899381,11cyclictest5732-21fschecks_time08:02:371
1929899381,11cyclictest17052-21grep08:32:351
1929899381,11cyclictest17052-21grep08:32:351
1929899371,9cyclictest9360-21users11:02:481
1929899371,9cyclictest9360-21users11:02:481
1929899371,9cyclictest9360-21users11:02:481
1929899371,9cyclictest4232-21grep09:22:461
1929899371,9cyclictest4232-21grep09:22:461
1929899371,8cyclictest749-21runrttasks12:13:521
1929899371,8cyclictest749-21runrttasks12:13:521
19298993717,17cyclictest0-21swapper/111:27:371
19298993717,17cyclictest0-21swapper/111:27:371
19298993717,17cyclictest0-21swapper/111:27:371
19298993711,24cyclictest4240-21ls12:17:391
19298993711,24cyclictest4240-21ls12:17:391
1929899371,11cyclictest27159-21cron10:27:231
1929899371,11cyclictest27159-21cron10:27:231
19297993727,8cyclictest14678-21systemd-journal08:22:230
19297993727,8cyclictest14678-21systemd-journal08:22:230
249732360,4sleep30-21swapper/311:47:343
249732360,4sleep30-21swapper/311:47:343
1929899361,9cyclictest1285-21taskset07:50:001
1929899361,9cyclictest1285-21taskset07:50:001
1929899361,8cyclictest24506-21taskset07:26:321
1929899361,8cyclictest24506-21taskset07:26:321
1929899361,7cyclictest24161-21ls11:42:461
1929899361,7cyclictest24161-21ls11:42:461
19298993614,20cyclictest0-21swapper/111:27:261
19298993614,20cyclictest0-21swapper/111:27:261
1929899361,10cyclictest7686-21uniq12:27:351
1929899361,10cyclictest32055-21latency_hist07:47:231
1929899361,10cyclictest32055-21latency_hist07:47:231
1929899361,10cyclictest27540-21ntp_states11:52:431
1929899361,10cyclictest27540-21ntp_states11:52:431
1929899361,10cyclictest27540-21ntp_states11:52:431
1929899361,10cyclictest24352-21fschecks_count08:52:361
1929899361,10cyclictest24352-21fschecks_count08:52:361
1929899361,10cyclictest14318-21missed_timers11:17:391
1929899361,10cyclictest14318-21missed_timers11:17:391
1929899361,10cyclictest1257-21sort09:17:361
1929899361,10cyclictest1027-21cat09:17:291
1929899361,10cyclictest1027-21cat09:17:291
1929899358,25cyclictest20252-21sed10:07:381
1929899358,25cyclictest20252-21sed10:07:381
1929899352,9cyclictest32329-21/usr/sbin/munin12:07:391
1929899352,9cyclictest32329-21/usr/sbin/munin12:07:391
1929899352,9cyclictest32329-21/usr/sbin/munin12:07:391
1929899351,9cyclictest19461-21taskset10:04:311
1929899351,9cyclictest19461-21taskset10:04:311
1929899351,9cyclictest19461-21taskset10:04:311
1929899351,8cyclictest26629-21grep08:57:381
1929899351,8cyclictest26629-21grep08:57:381
1929899351,8cyclictest15788-21taskset09:55:351
1929899351,8cyclictest15788-21taskset09:55:351
1929899351,8cyclictest14678-21systemd-journal08:16:591
1929899351,8cyclictest14678-21systemd-journal08:16:591
1929899351,7cyclictest29817-21sshd07:38:001
1929899351,7cyclictest29817-21sshd07:38:001
1929899351,7cyclictest2928-21unixbench_singl07:52:481
1929899351,7cyclictest2928-21unixbench_singl07:52:481
1929899351,7cyclictest26323-21sshd07:31:001
1929899351,7cyclictest23633-21/usr/sbin/munin10:17:461
1929899351,7cyclictest23633-21/usr/sbin/munin10:17:461
1929899351,7cyclictest21707-21unixbench_singl08:42:471
1929899351,7cyclictest21707-21unixbench_singl08:42:471
1929899351,7cyclictest10279-21mailstats12:32:441
1929899351,7cyclictest10279-21mailstats12:32:441
1929899351,10cyclictest816-21fschecks_time10:42:371
1929899351,10cyclictest816-21fschecks_time10:42:371
1929899351,10cyclictest816-21fschecks_time10:42:371
1929899351,10cyclictest6775-21sed12:22:461
1929899351,10cyclictest29809-21/usr/sbin/munin09:07:371
1929899351,10cyclictest29521-21rt-features11:57:471
1929899351,10cyclictest29521-21rt-features11:57:471
1929899351,10cyclictest29521-21rt-features11:57:471
1929899351,10cyclictest20410-21cat11:32:451
1929899351,10cyclictest20410-21cat11:32:451
1929899351,10cyclictest14534-21cut09:52:361
19297993529,4cyclictest545-21in:imuxsock10:02:240
130952350,2sleep00-21swapper/008:22:340
130952350,2sleep00-21swapper/008:22:340
130952350,2sleep00-21swapper/008:22:340
1929899348,24cyclictest13343-21grep08:22:351
1929899348,24cyclictest13343-21grep08:22:351
1929899348,24cyclictest13343-21grep08:22:351
19298993428,4cyclictest11429-21cut08:17:371
19298993428,4cyclictest11429-21cut08:17:371
1929899341,9cyclictest6733-21/usr/sbin/munin09:32:461
1929899341,9cyclictest6733-21/usr/sbin/munin09:32:461
1929899341,9cyclictest22171-21/usr/sbin/munin11:37:471
1929899341,9cyclictest22171-21/usr/sbin/munin11:37:471
1929899341,9cyclictest18600-21/usr/sbin/munin08:37:421
1929899341,9cyclictest18600-21/usr/sbin/munin08:37:421
1929899341,8cyclictest29422-21fschecks_count10:32:351
1929899341,8cyclictest29422-21fschecks_count10:32:351
1929899341,7cyclictest28264-21sshd07:37:001
1929899341,7cyclictest28264-21sshd07:37:001
1929899341,6cyclictest6927-21taskset10:57:421
1929899341,6cyclictest6927-21taskset10:57:421
1929899341,10cyclictest32755-21sh10:42:231
1929899341,10cyclictest32755-21sh10:42:231
1929899341,10cyclictest12890-21irqstats09:47:381
1929899341,10cyclictest11064-21grep11:07:461
1929899341,10cyclictest11064-21grep11:07:461
1929899341,10cyclictest11064-21grep11:07:461
1929899340,10cyclictest14868-21/usr/sbin/munin08:27:381
19297993428,5cyclictest5764-21cut08:02:380
19297993428,5cyclictest5764-21cut08:02:380
19297993428,5cyclictest5764-21cut08:02:380
19297993420,12cyclictest13079-21needreboot09:47:410
19297993420,12cyclictest13079-21needreboot09:47:410
1929899336,3cyclictest0-21swapper/107:17:391
1929899336,3cyclictest0-21swapper/107:17:391
1929899331,9cyclictest6035-21latency_hist10:57:261
1929899331,9cyclictest24163-21latency_hist08:52:301
1929899331,9cyclictest24163-21latency_hist08:52:301
1929899331,9cyclictest11239-21/usr/sbin/munin12:37:421
1929899331,9cyclictest10546-21/usr/sbin/munin09:42:351
1929899331,9cyclictest10546-21/usr/sbin/munin09:42:351
1929899331,8cyclictest29439-21seq09:05:271
1929899331,8cyclictest29439-21seq09:05:271
1929899331,7cyclictest17644-21runrttasks10:00:301
1929899331,10cyclictest7596-21fschecks_time08:07:361
1929899331,10cyclictest7596-21fschecks_time08:07:361
1929899331,10cyclictest7443-21/usr/sbin/munin12:27:441
1929899331,10cyclictest7443-21/usr/sbin/munin12:27:441
1929899331,10cyclictest7443-21/usr/sbin/munin12:27:441
1929899331,10cyclictest30300-21cat10:32:451
1929899331,10cyclictest30300-21cat10:32:451
1929899330,9cyclictest3686-21unixbench_singl10:47:471
1929899330,9cyclictest3686-21unixbench_singl10:47:471
1929899330,9cyclictest3686-21unixbench_singl10:47:471
42772320,4sleep30-21swapper/307:57:403
42772320,4sleep30-21swapper/307:57:403
224432320,3sleep20-21swapper/210:12:422
224432320,3sleep20-21swapper/210:12:422
1929899320,10cyclictest28235-21processes10:27:441
19297993226,4cyclictest7692-21head08:07:360
19297993226,4cyclictest7692-21head08:07:360
19297993226,4cyclictest4728-21if_eth010:52:370
19297993226,4cyclictest23299-21head07:22:380
19297993226,4cyclictest23299-21head07:22:380
260742310,3chrt0-21swapper/308:57:343
260742310,3chrt0-21swapper/308:57:343
19297993123,6cyclictest31839-21sshd12:03:000
19297993123,6cyclictest31839-21sshd12:03:000
19297993123,6cyclictest31839-21sshd12:03:000
19297993123,6cyclictest21088-21missed_timers08:42:400
19297993123,6cyclictest21088-21missed_timers08:42:400
19297993024,5cyclictest29708-21timerandwakeup11:57:480
19297993024,5cyclictest29708-21timerandwakeup11:57:480
19297993024,5cyclictest29708-21timerandwakeup11:57:480
19297993024,4cyclictest25893-21head10:22:360
19297993024,4cyclictest25893-21head10:22:360
19297993023,5cyclictest1977-21fschecks_time07:52:370
19297993023,5cyclictest1977-21fschecks_time07:52:370
19297992922,5cyclictest7152-21sshd12:26:000
19297992922,5cyclictest7152-21sshd12:26:000
19297992922,5cyclictest1346-21avahi-daemon09:37:460
19297992922,5cyclictest1346-21avahi-daemon09:37:460
287622280,3sleep20-21swapper/207:37:372
287622280,3sleep20-21swapper/207:37:372
19297992822,3cyclictest0-21swapper/011:52:440
19297992822,3cyclictest0-21swapper/011:52:440
19297992822,3cyclictest0-21swapper/011:52:440
19297992821,5cyclictest1488-21idleruntime-cro07:52:230
19297992821,5cyclictest1488-21idleruntime-cro07:52:230
19297992820,6cyclictest30777-21switchtime09:07:450
19297992820,6cyclictest25020-21latency_hist11:47:350
19297992820,6cyclictest25020-21latency_hist11:47:350
149622280,3chrt0-21swapper/309:52:413
149622280,3chrt0-21swapper/309:52:413
19297992723,3cyclictest25306-21sendmail10:22:230
19297992723,3cyclictest25306-21sendmail10:22:230
19297992721,5cyclictest572-21cups-browsed07:32:460
19297992721,5cyclictest572-21cups-browsed07:32:460
19297992721,4cyclictest22331-21latency10:12:400
19297992721,4cyclictest22331-21latency10:12:400
19297992720,5cyclictest22557-21fschecks_time08:47:350
19297992720,5cyclictest22557-21fschecks_time08:47:350
19297992718,7cyclictest3279-21fschecks_time09:22:360
19297992718,7cyclictest3279-21fschecks_time09:22:360
19297992718,7cyclictest18265-21processes11:27:450
19297992718,7cyclictest18265-21processes11:27:450
19297992718,7cyclictest18265-21processes11:27:450
19297992621,3cyclictest0-21swapper/011:42:240
19297992621,3cyclictest0-21swapper/011:42:240
19297992621,3cyclictest0-21swapper/011:12:370
19297992621,3cyclictest0-21swapper/011:12:370
19297992620,4cyclictest14678-21systemd-journal10:57:460
19297992620,4cyclictest14678-21systemd-journal10:57:460
19297992620,3cyclictest0-21swapper/012:17:450
19297992620,3cyclictest0-21swapper/012:17:450
19297992619,5cyclictest27819-21latency_hist09:02:260
19297992619,5cyclictest27819-21latency_hist09:02:260
19297992619,5cyclictest2097-21fschecks_time12:12:360
19297992619,5cyclictest2097-21fschecks_time12:12:360
141102260,3sleep20-21swapper/208:22:432
141102260,3sleep20-21swapper/208:22:432
141102260,3sleep20-21swapper/208:22:432
19297992520,4cyclictest749-21runrttasks10:47:380
19297992520,4cyclictest749-21runrttasks10:47:380
19297992520,4cyclictest749-21runrttasks10:47:380
19297992520,3cyclictest0-21swapper/010:37:410
19297992520,3cyclictest0-21swapper/010:37:410
19297992520,3cyclictest0-21swapper/009:17:370
19297992519,5cyclictest32192-21kernelversion09:12:390
19297992519,5cyclictest32192-21kernelversion09:12:390
19297992517,6cyclictest28524-21/usr/sbin/munin07:37:430
19297992517,6cyclictest28524-21/usr/sbin/munin07:37:430
19297992515,8cyclictest20404-21kernelversion10:07:400
19297992515,8cyclictest20404-21kernelversion10:07:400
63902240,3sleep20-21swapper/208:02:452
63902240,3sleep20-21swapper/208:02:452
63902240,3sleep20-21swapper/208:02:452
284302240,4sleep20-21swapper/210:27:462
284302240,4sleep20-21swapper/210:27:462
284302240,4sleep20-21swapper/210:27:462
19297992420,2cyclictest0-21swapper/009:27:440
19297992420,2cyclictest0-21swapper/009:27:440
19297992419,3cyclictest0-21swapper/011:07:450
19297992419,3cyclictest0-21swapper/011:07:450
19297992419,3cyclictest0-21swapper/011:07:450
19297992419,3cyclictest0-21swapper/008:52:460
19297992419,3cyclictest0-21swapper/008:52:460
19297992417,5cyclictest12688-21sshd12:40:000
19297992415,7cyclictest19887-21ls11:32:380
19297992415,7cyclictest19887-21ls11:32:380
19297992415,6cyclictest18698-21sort08:37:330
19297992415,6cyclictest18698-21sort08:37:330
1929799235,13cyclictest8070-21find11:02:350
1929799235,13cyclictest8070-21find11:02:350
1929799235,13cyclictest8070-21find11:02:350
19297992317,5cyclictest15448-21unixbench-2d09:52:460
19297992317,5cyclictest15448-21unixbench-2d09:52:460
19297992317,5cyclictest11839-21sshd09:43:000
19297992317,5cyclictest11839-21sshd09:43:000
19297992317,4cyclictest0-21swapper/009:07:230
19297992317,4cyclictest0-21swapper/009:07:230
19297992316,5cyclictest18355-21gltestperf10:02:380
19297992316,5cyclictest18355-21gltestperf10:02:380
19297992316,5cyclictest18355-21gltestperf10:02:380
19297992316,4cyclictest0-21swapper/012:12:230
19297992316,4cyclictest0-21swapper/012:12:230
19297992315,6cyclictest749-21runrttasks10:42:460
19297992315,6cyclictest749-21runrttasks10:42:460
19297992315,6cyclictest749-21runrttasks10:42:460
19297992315,6cyclictest29876-21sort10:32:390
19297992315,6cyclictest29876-21sort10:32:390
19297992314,7cyclictest16515-21latency_hist08:32:300
307832220,2chrt30785-21ls07:42:370
307832220,2chrt30785-21ls07:42:370
30252220,3sleep20-21swapper/210:47:392
30252220,3sleep20-21swapper/210:47:392
30252220,3sleep20-21swapper/210:47:392
19297992218,2cyclictest0-21swapper/009:32:440
19297992218,2cyclictest0-21swapper/009:32:440
19297992218,2cyclictest0-21swapper/007:17:390
19297992218,2cyclictest0-21swapper/007:17:390
19297992217,3cyclictest0-21swapper/012:27:450
19297992217,3cyclictest0-21swapper/012:27:450
19297992217,3cyclictest0-21swapper/012:27:450
19297992217,3cyclictest0-21swapper/008:32:420
19297992217,3cyclictest0-21swapper/008:32:420
19297992216,3cyclictest0-21swapper/007:27:480
327072210,3sleep30-21swapper/307:47:393
327072210,3sleep30-21swapper/307:47:393
1929799216,13cyclictest29061-21basename10:32:250
1929799216,13cyclictest29061-21basename10:32:250
1929799216,13cyclictest29061-21basename10:32:250
19297992116,3cyclictest0-21swapper/011:22:360
19297992116,3cyclictest0-21swapper/011:22:360
19297992115,5cyclictest19288-21cyclictest12:32:350
19297992115,5cyclictest19288-21cyclictest12:32:350
319432200,3sleep30-21swapper/309:12:373
319432200,3sleep30-21swapper/309:12:373
225052200,3sleep20-21swapper/207:20:002
225052200,3sleep20-21swapper/207:20:002
19297992013,5cyclictest15317-21sshd11:22:000
19297992013,5cyclictest15317-21sshd11:22:000
1930099192,13cyclictest16198-21chrt09:57:352
19297991914,3cyclictest0-21swapper/008:12:450
19297991914,3cyclictest0-21swapper/008:12:450
1930199181,12cyclictest24349-21chrt10:17:423
1930199181,12cyclictest24349-21chrt10:17:423
1930099183,9cyclictest0-21swapper/209:44:002
1930099183,9cyclictest0-21swapper/209:44:002
1930099181,12cyclictest20772-21chrt08:42:372
1930099181,12cyclictest20772-21chrt08:42:372
1930099181,12cyclictest13099-21chrt11:12:522
1930099181,12cyclictest13099-21chrt11:12:522
1930099181,11cyclictest10916-21taskset11:07:452
1930099181,11cyclictest10916-21taskset11:07:452
1930099181,11cyclictest10916-21taskset11:07:452
273322170,3sleep20-21swapper/207:32:422
273322170,3sleep20-21swapper/207:32:422
1930199173,10cyclictest0-21swapper/312:17:003
1930199173,10cyclictest0-21swapper/312:17:003
1930199171,11cyclictest29707-21taskset10:32:373
1930199171,11cyclictest29707-21taskset10:32:373
1930099174,7cyclictest0-21swapper/209:37:372
1930099174,7cyclictest0-21swapper/209:37:372
1930099171,12cyclictest1226-21chrt09:17:352
127672170,3sleep30-21swapper/311:12:453
127672170,3sleep30-21swapper/311:12:453
94682161,3sleep30-21swapper/311:04:023
94682161,3sleep30-21swapper/311:04:023
94682161,3sleep30-21swapper/311:04:023
1930199163,9cyclictest0-21swapper/309:39:003
1930199163,9cyclictest0-21swapper/309:39:003
1930199163,9cyclictest0-21swapper/307:53:593
1930199163,9cyclictest0-21swapper/307:53:593
1930199161,12cyclictest1635-21chrt10:42:463
1930199161,12cyclictest1635-21chrt10:42:463
1930199161,12cyclictest1635-21chrt10:42:463
1930199161,11cyclictest24703-21chrt08:52:393
1930199161,11cyclictest24703-21chrt08:52:393
1930099161,9cyclictest0-21swapper/212:07:362
1930099161,9cyclictest0-21swapper/212:07:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional