You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 21:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jul 05, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2076121360,2sleep20-21swapper/209:23:242
2076121360,2sleep20-21swapper/209:23:242
2076121360,2sleep20-21swapper/209:23:242
2987621200,3sleep30-21swapper/312:38:353
2987621200,3sleep30-21swapper/312:38:353
925021060,2chrt8954-21latency_hist08:53:241
925021060,2chrt8954-21latency_hist08:53:241
925021060,2chrt8954-21latency_hist08:53:241
2185021020,3chrt0-21swapper/012:18:300
2185021020,3chrt0-21swapper/012:18:300
2185021020,3chrt0-21swapper/012:18:300
129162840,2sleep10-21swapper/111:53:341
129162840,2sleep10-21swapper/111:53:341
165962670,3sleep30-21swapper/312:03:333
165962670,3sleep30-21swapper/312:03:333
165962670,3sleep30-21swapper/312:03:333
25302620,2sleep30-21swapper/308:33:333
95182600,3sleep30-21swapper/307:28:123
186192600,1sleep018711-21latency_hist09:18:250
186192600,1sleep018711-21latency_hist09:18:250
186192600,1sleep018711-21latency_hist09:18:250
321642580,3chrt0-21swapper/209:53:272
321642580,3chrt0-21swapper/209:53:272
212972570,3chrt0-21swapper/109:23:321
155852540,1sleep10-21swapper/109:08:321
155852540,1sleep10-21swapper/109:08:321
155852540,1sleep10-21swapper/109:08:321
231312520,4sleep30-21swapper/312:23:233
231312520,4sleep30-21swapper/312:23:233
231312520,4sleep30-21swapper/312:23:233
327282500,1sleep132753-21latency_hist11:23:121
327282500,1sleep132753-21latency_hist11:23:121
273552490,2sleep127357-21basename11:08:241
4191994617,8cyclictest24002-21swap12:23:331
4191994617,8cyclictest24002-21swap12:23:331
317932450,3chrt0-21swapper/211:18:302
317932450,3chrt0-21swapper/211:18:302
141872450,2sleep00-21swapper/009:03:370
141872450,2sleep00-21swapper/009:03:370
141872450,2sleep00-21swapper/009:03:370
65002440,2sleep06698-21unixbench_singl08:43:360
65002440,2sleep06698-21unixbench_singl08:43:360
4191994421,6cyclictest28053-21df09:43:231
4191994421,6cyclictest28053-21df09:43:231
4191994420,8cyclictest2992-21taskset08:34:061
4191994417,9cyclictest18637-21sed09:18:241
4191994417,9cyclictest18637-21sed09:18:241
4191994417,9cyclictest18637-21sed09:18:241
7592430,4sleep3431rcuc/311:23:253
4191994330,7cyclictest0-21swapper/110:59:501
4191994330,7cyclictest0-21swapper/110:59:501
4191994330,7cyclictest0-21swapper/110:59:501
4191994320,5cyclictest23503-21bingrep08:03:291
4191994320,5cyclictest23503-21bingrep08:03:291
4191994316,10cyclictest964-21cat09:58:171
4191994316,10cyclictest964-21cat09:58:171
131502430,4sleep1773-21apache210:28:321
131502430,4sleep1773-21apache210:28:321
117672430,1sleep111895-21/usr/sbin/munin11:53:231
117672430,1sleep111895-21/usr/sbin/munin11:53:231
117672430,1sleep111895-21/usr/sbin/munin11:53:231
4191994234,5cyclictest700-21192.168.115.40-manager09:41:121
4191994234,5cyclictest700-21192.168.115.40-manager09:41:121
4191994219,11cyclictest30075-21chrt08:22:371
4191994219,11cyclictest30075-21chrt08:22:371
4191994219,11cyclictest30075-21chrt08:22:371
4191994219,10cyclictest10967-21if_eth010:23:271
4191994219,10cyclictest10967-21if_eth010:23:271
4191994218,7cyclictest20649-21fschecks_time09:23:241
4191994218,7cyclictest20649-21fschecks_time09:23:241
4191994218,11cyclictest4177-21cat11:33:171
4191994218,11cyclictest28932-21taskset11:12:221
4191994218,11cyclictest28932-21taskset11:12:221
4191994216,11cyclictest6067-21taskset11:38:171
4191994216,11cyclictest6067-21taskset11:38:171
4191994134,4cyclictest700-21192.168.115.40-manager07:45:321
4191994134,4cyclictest700-21192.168.115.40-manager07:45:321
4191994130,9cyclictest14982-21fschecks_count09:08:261
4191994130,9cyclictest14982-21fschecks_count09:08:261
4191994130,9cyclictest14982-21fschecks_count09:08:261
4191994118,9cyclictest21832-21taskset09:24:301
4191994118,9cyclictest21832-21taskset09:24:301
4191994118,8cyclictest5615-21/usr/sbin/munin10:08:311
4191994118,8cyclictest5615-21/usr/sbin/munin10:08:311
4191994116,8cyclictest14717-21/usr/sbin/munin11:58:351
4191994116,8cyclictest14717-21/usr/sbin/munin11:58:351
4191994116,8cyclictest14717-21/usr/sbin/munin11:58:351
4191994116,12cyclictest24266-21/usr/sbin/munin09:33:241
4191994116,12cyclictest24266-21/usr/sbin/munin09:33:241
4191994115,8cyclictest4262-21ntpq08:38:311
4191994115,8cyclictest4262-21ntpq08:38:311
212252410,2sleep021295-21processes09:23:320
47882400,2chrt4791-21wc10:08:221
47882400,2chrt4791-21wc10:08:221
47882400,2chrt4791-21wc10:08:221
4191994029,8cyclictest700-21192.168.115.40-manager08:18:191
4191994029,8cyclictest700-21192.168.115.40-manager08:18:191
4191994016,10cyclictest18711-21/usr/sbin/munin12:08:351
4191994016,10cyclictest18711-21/usr/sbin/munin12:08:351
4191994014,12cyclictest17554-21gltestperf07:48:261
4191994014,12cyclictest17554-21gltestperf07:48:261
4191993925,12cyclictest700-21192.168.115.40-manager10:50:251
4191993925,12cyclictest700-21192.168.115.40-manager10:50:251
4191993925,12cyclictest700-21192.168.115.40-manager10:50:251
4191993925,12cyclictest700-21192.168.115.40-manager10:50:251
4191993915,10cyclictest31564-21grep08:23:351
4191993915,10cyclictest31564-21grep08:23:351
306102390,1sleep130745-21grep09:48:331
306102390,1sleep130745-21grep09:48:331
259702390,2sleep026164-21unixbench_singl12:28:360
259702390,2sleep026164-21unixbench_singl12:28:360
7052380,3sleep10-21swapper/108:28:321
7052380,3sleep10-21swapper/108:28:321
4191993828,7cyclictest700-21192.168.115.40-manager07:40:501
4191993828,7cyclictest700-21192.168.115.40-manager07:40:501
4191993824,10cyclictest12014-21/usr/sbin/munin07:33:271
4191993824,10cyclictest12014-21/usr/sbin/munin07:33:271
4191993824,10cyclictest12014-21/usr/sbin/munin07:33:271
4191993817,6cyclictest0-21swapper/110:17:121
4191993817,6cyclictest0-21swapper/110:17:121
4191993816,6cyclictest23642-21kworker/u10:0+rpciod10:38:121
4191993816,4cyclictest26772-21users11:03:381
4191993815,8cyclictest9815-21latency_hist11:48:161
4191993815,8cyclictest9815-21latency_hist11:48:161
4191993815,8cyclictest9815-21latency_hist11:48:161
4191993815,10cyclictest26698-21latency_hist12:33:191
4191993815,10cyclictest26698-21latency_hist12:33:191
4191993714,7cyclictest0-21swapper/107:57:271
4191993714,7cyclictest0-21swapper/107:57:271
4191993714,7cyclictest0-21swapper/107:57:271
4191993714,6cyclictest19798-21irqstats12:13:291
4191993714,6cyclictest19798-21irqstats12:13:291
61212360,3sleep30-21swapper/308:43:303
61212360,3sleep30-21swapper/308:43:303
4191993625,9cyclictest700-21192.168.115.40-manager07:31:251
4191993625,9cyclictest700-21192.168.115.40-manager07:31:251
4191993625,9cyclictest700-21192.168.115.40-manager07:31:251
4191993623,11cyclictest25183-21if_eth008:08:261
4191993623,11cyclictest25183-21if_eth008:08:261
4191993617,9cyclictest21408-21if_eth007:58:261
4191993617,9cyclictest21408-21if_eth007:58:261
4191993617,9cyclictest21408-21if_eth007:58:261
4191993616,8cyclictest2432-21taskset09:59:071
4191993616,8cyclictest2432-21taskset09:59:071
4191993612,9cyclictest12678-21sendmail-mta09:02:591
4191993612,9cyclictest12678-21sendmail-mta09:02:591
4191993612,9cyclictest12678-21sendmail-mta09:02:591
4191993526,7cyclictest22281-21/usr/sbin/munin09:28:331
4191993520,11cyclictest19528-21taskset10:46:271
4191993520,11cyclictest19528-21taskset10:46:271
4190993527,6cyclictest20239-21latency_hist09:23:130
4190993527,6cyclictest20239-21latency_hist09:23:130
4191993422,9cyclictest8707-21rt-features07:23:331
4191993422,9cyclictest28642-21/usr/sbin/munin12:38:231
4191993422,9cyclictest28642-21/usr/sbin/munin12:38:231
4191993422,9cyclictest0-21swapper/111:40:141
4191993422,9cyclictest0-21swapper/111:40:141
4191993422,10cyclictest9822-21awk10:18:341
4191993422,10cyclictest9822-21awk10:18:341
4191993422,10cyclictest9822-21awk10:18:341
4191993420,11cyclictest22345-21/usr/sbin/munin10:53:291
4191993420,11cyclictest22345-21/usr/sbin/munin10:53:291
4191993420,11cyclictest22345-21/usr/sbin/munin10:53:291
4191993420,11cyclictest22277-21/usr/sbin/munin12:18:341
4191993420,11cyclictest22277-21/usr/sbin/munin12:18:341
4191993420,11cyclictest22277-21/usr/sbin/munin12:18:341
4191993417,13cyclictest16810-21gzip10:38:301
4191993417,13cyclictest16810-21gzip10:38:301
15412340,3chrt0-21swapper/308:33:163
15412340,3chrt0-21swapper/308:33:163
4191993321,9cyclictest16340-21ntp_states12:03:311
4191993321,9cyclictest16340-21ntp_states12:03:311
4191993321,9cyclictest16340-21ntp_states12:03:311
4191993319,11cyclictest0-21swapper/108:53:591
4191993319,11cyclictest0-21swapper/108:53:591
4191993319,11cyclictest0-21swapper/108:53:591
4190993328,3cyclictest22086-21gltestperf10:53:270
4190993328,3cyclictest22086-21gltestperf10:53:270
4190993328,3cyclictest22086-21gltestperf10:53:270
4190993326,5cyclictest5209-21cron08:43:120
4190993326,5cyclictest5209-21cron08:43:120
4190993326,5cyclictest31929-21fschecks_time09:53:250
4190993326,5cyclictest31929-21fschecks_time09:53:250
4190993326,5cyclictest21636-21needreboot07:58:290
4190993326,5cyclictest21636-21needreboot07:58:290
4190993326,5cyclictest21636-21needreboot07:58:290
4190993326,5cyclictest1324-21fschecks_time09:58:260
4190993326,5cyclictest1324-21fschecks_time09:58:260
4190993325,6cyclictest31756-21needreboot11:18:300
4190993325,6cyclictest31756-21needreboot11:18:300
4190993325,5cyclictest550-21cron10:38:120
4190993323,7cyclictest31385-21threads08:23:340
4190993323,7cyclictest31385-21threads08:23:340
4191993227,3cyclictest5617-21fschecks_count08:43:251
4191993227,3cyclictest5617-21fschecks_count08:43:251
4191993219,10cyclictest26869-21cpuspeed12:33:241
4191993219,10cyclictest26869-21cpuspeed12:33:241
4190993216,12cyclictest60-21kcompactd012:09:130
4190993216,12cyclictest60-21kcompactd012:09:130
4191993119,10cyclictest2169-21taskset11:27:561
4191993119,10cyclictest2169-21taskset11:27:561
4191993119,10cyclictest2169-21taskset11:27:561
4191993117,11cyclictest29664-21taskset11:13:261
4191993117,11cyclictest29664-21taskset11:13:261
4190993124,5cyclictest29494-21fschecks_time11:13:250
4190993124,5cyclictest28640-21cut08:18:240
4190993124,5cyclictest28640-21cut08:18:240
4190993124,5cyclictest28640-21cut08:18:240
4190993123,6cyclictest22277-21latency_hist09:28:230
4190993123,6cyclictest22277-21latency_hist09:28:230
305002310,3sleep20-21swapper/211:13:352
305002310,3sleep20-21swapper/211:13:352
4190993023,5cyclictest23020-21needreboot09:28:300
4190993023,5cyclictest17022-21latency_hist07:48:120
4190993023,5cyclictest17022-21latency_hist07:48:120
4190993021,7cyclictest8386-21gltestperf11:43:250
4190993021,7cyclictest8386-21gltestperf11:43:250
4190993021,7cyclictest8386-21gltestperf11:43:250
149832300,3sleep30-21swapper/309:08:263
149832300,3sleep30-21swapper/309:08:263
149832300,3sleep30-21swapper/309:08:263
419599296,19cyclictest244682chrt08:07:283
419599296,19cyclictest244682chrt08:07:283
4195992919,7cyclictest12013-21chrt07:33:273
4195992919,7cyclictest12013-21chrt07:33:273
4195992919,7cyclictest12013-21chrt07:33:273
4190992924,4cyclictest19767-21if_wlan012:13:290
4190992924,4cyclictest19767-21if_wlan012:13:290
4190992924,4cyclictest14065-21tail11:58:260
4190992924,4cyclictest14065-21tail11:58:260
4190992924,4cyclictest14065-21tail11:58:260
4190992924,3cyclictest4184-21cyclictest11:53:230
4190992924,3cyclictest4184-21cyclictest11:53:230
4190992924,3cyclictest4184-21cyclictest11:53:230
4190992923,4cyclictest27004-21sshd11:06:300
4190992923,4cyclictest27004-21sshd11:06:300
4190992922,5cyclictest20677-21sshd07:57:290
4190992922,5cyclictest20677-21sshd07:57:290
4190992922,5cyclictest20677-21sshd07:57:290
4190992921,6cyclictest24074-21threads12:23:330
4190992921,6cyclictest24074-21threads12:23:330
4190992920,7cyclictest29903-21needreboot11:13:290
4190992920,7cyclictest29903-21needreboot11:13:290
4190992822,4cyclictest7557-21wc10:13:320
4190992822,4cyclictest7557-21wc10:13:320
4190992822,3cyclictest0-21swapper/012:40:270
4190992822,3cyclictest0-21swapper/012:40:270
4190992821,5cyclictest20394-21kernelversion10:48:280
4190992821,5cyclictest20394-21kernelversion10:48:280
4190992821,5cyclictest20394-21kernelversion10:48:280
4190992821,5cyclictest20394-21kernelversion10:48:280
4190992821,5cyclictest12703-21latency_hist09:03:120
4190992821,5cyclictest12703-21latency_hist09:03:120
4190992821,5cyclictest12703-21latency_hist09:03:120
4190992821,4cyclictest0-21swapper/010:08:260
4190992820,6cyclictest29769-21/usr/sbin/munin09:48:300
4190992820,6cyclictest29769-21/usr/sbin/munin09:48:300
4190992820,6cyclictest1346-21avahi-daemon11:38:290
4190992820,6cyclictest1346-21avahi-daemon11:38:290
419599271,9cyclictest26892-21chrt12:33:243
419599271,9cyclictest26892-21chrt12:33:243
4190992722,3cyclictest0-21swapper/010:03:280
4190992722,3cyclictest0-21swapper/010:03:280
4190992721,3cyclictest0-21swapper/010:13:230
4190992721,3cyclictest0-21swapper/010:13:230
4190992720,5cyclictest4635-21gltestperf11:33:260
4190992720,5cyclictest4635-21gltestperf11:33:260
4190992720,5cyclictest24241-21sshd08:04:300
4190992720,5cyclictest24241-21sshd08:04:300
419599267,14cyclictest287552sleep309:43:313
419599267,14cyclictest287552sleep309:43:313
419599261,8cyclictest8228-21chrt10:16:083
419599261,8cyclictest8228-21chrt10:16:083
419599261,7cyclictest19957-21chrt09:19:443
419599261,7cyclictest19957-21chrt09:19:443
419599261,7cyclictest19957-21chrt09:19:443
419599261,4cyclictest5920-21chrt11:37:483
419599261,4cyclictest5920-21chrt11:37:483
419599261,4cyclictest318772sleep308:26:333
419599261,4cyclictest318772sleep308:26:333
419599260,8cyclictest17846-21chrt10:43:133
419599260,8cyclictest17846-21chrt10:43:133
4190992620,5cyclictest14046-21latency07:38:290
4190992620,5cyclictest14046-21latency07:38:290
4190992619,5cyclictest4184-21cyclictest09:46:550
4190992619,5cyclictest4184-21cyclictest09:46:550
4190992619,5cyclictest4184-21cyclictest09:13:280
4190992619,5cyclictest4184-21cyclictest09:13:280
4190992619,5cyclictest4184-21cyclictest09:13:280
4190992619,5cyclictest17495-21sshd10:39:300
4190992619,5cyclictest17495-21sshd10:39:300
4190992619,5cyclictest0-21swapper/010:18:340
4190992619,5cyclictest0-21swapper/010:18:340
4190992619,5cyclictest0-21swapper/010:18:340
4190992618,6cyclictest11773-21/usr/sbin/munin11:53:300
4190992618,6cyclictest11773-21/usr/sbin/munin11:53:300
4190992616,8cyclictest12753-21unixbench-2d07:33:350
4190992616,8cyclictest12753-21unixbench-2d07:33:350
4190992616,8cyclictest12753-21unixbench-2d07:33:350
419599251,8cyclictest4328-21chrt10:04:103
419599251,8cyclictest4328-21chrt10:04:103
419599251,8cyclictest4328-21chrt10:04:103
419599251,8cyclictest31041-21chrt11:18:233
419599251,8cyclictest31041-21chrt11:18:233
419599251,7cyclictest15659-21chrt10:34:493
419599251,4cyclictest86192sleep308:48:563
419599251,4cyclictest86192sleep308:48:563
419599251,4cyclictest86192sleep308:48:563
419599251,4cyclictest50262sleep308:41:163
419599251,4cyclictest50262sleep308:41:163
419599251,4cyclictest324272sleep311:19:233
419599251,4cyclictest288872sleep311:11:473
419599251,4cyclictest288872sleep311:11:473
419599251,4cyclictest271112sleep311:07:533
419599251,4cyclictest271112sleep311:07:533
419599251,4cyclictest262782sleep312:29:343
419599251,4cyclictest262782sleep312:29:343
419599251,4cyclictest23119-21chrt10:54:433
419599251,4cyclictest23119-21chrt10:54:433
419599251,4cyclictest23119-21chrt10:54:433
419599251,4cyclictest22545-21chrt08:02:333
419599251,4cyclictest22545-21chrt08:02:333
419599251,4cyclictest22545-21chrt08:02:333
419599251,4cyclictest208082chrt12:16:283
419599251,4cyclictest208082chrt12:16:283
419599251,4cyclictest195082sleep310:46:113
419599251,4cyclictest195082sleep310:46:113
419599251,4cyclictest18842sleep311:24:143
419599251,4cyclictest18842sleep311:24:143
419599251,4cyclictest18842sleep311:24:143
419599251,4cyclictest151082sleep312:00:163
419599251,4cyclictest151082sleep312:00:163
419599251,4cyclictest151082sleep312:00:163
419599251,4cyclictest12047-21chrt10:26:373
419599251,4cyclictest12047-21chrt10:26:373
419599251,3cyclictest274352sleep309:39:463
419599251,3cyclictest274352sleep309:39:463
419599251,3cyclictest260402sleep308:08:403
419599251,3cyclictest260402sleep308:08:403
419599251,3cyclictest0-21swapper/308:18:313
419599251,3cyclictest0-21swapper/308:18:313
419599251,3cyclictest0-21swapper/308:18:313
419599251,3cyclictest0-21swapper/307:31:113
419599251,3cyclictest0-21swapper/307:31:113
419599251,3cyclictest0-21swapper/307:31:113
419599250,4cyclictest255652sleep309:34:393
419599250,4cyclictest255652sleep309:34:393
4190992518,5cyclictest0-21swapper/008:48:260
4190992518,5cyclictest0-21swapper/008:48:260
4190992518,5cyclictest0-21swapper/008:48:260
4190992518,4cyclictest0-21swapper/011:23:360
4190992518,4cyclictest0-21swapper/011:23:360
4190992518,4cyclictest0-21swapper/011:23:360
4190992517,6cyclictest9915-21fschecks_count07:28:250
4190992517,6cyclictest9915-21fschecks_count07:28:250
4190992517,6cyclictest9915-21fschecks_count07:28:250
4190992516,7cyclictest0-21swapper/011:28:330
4190992511,11cyclictest0-21swapper/012:33:250
4190992511,11cyclictest0-21swapper/012:33:250
419599241,8cyclictest21892-21chrt09:25:183
419599241,8cyclictest21892-21chrt09:25:183
419599241,8cyclictest18220-21chrt09:16:223
419599241,8cyclictest18220-21chrt09:16:223
419599241,8cyclictest18220-21chrt09:16:223
419599241,8cyclictest10127-21chrt10:21:103
419599241,8cyclictest10127-21chrt10:21:103
419599241,8cyclictest10127-21chrt10:21:103
419599241,7cyclictest31100-21chrt09:48:423
419599241,7cyclictest31100-21chrt09:48:423
419599241,7cyclictest13176-21chrt11:54:413
419599241,7cyclictest13176-21chrt11:54:413
419599241,4cyclictest7142chrt09:56:143
419599241,4cyclictest7142chrt09:56:143
419599241,4cyclictest61932sleep310:08:523
419599241,4cyclictest61932sleep310:08:523
419599241,4cyclictest3703-21chrt11:28:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional