You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 02:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Jun 29, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2952621120,2sleep029531-21unixbench_multi00:28:300
2952621120,2sleep029531-21unixbench_multi00:28:300
2827121120,3chrt28266-21gzip21:33:241
2827121120,3chrt28266-21gzip21:33:241
525321100,2sleep15255-21awk21:58:241
525321100,2sleep15255-21awk21:58:241
525321100,2sleep15255-21awk21:58:241
2757921030,2chrt27580-21timerwakeupswit00:23:291
2757921030,2chrt27580-21timerwakeupswit00:23:291
152332980,3sleep10-21swapper/120:58:261
152332980,3sleep10-21swapper/120:58:261
324712950,2chrt32473-21sort20:18:221
324712950,2chrt32473-21sort20:18:221
129802920,2sleep30-21swapper/319:28:073
143692850,2sleep014368-21sed19:28:290
143692850,2sleep014368-21sed19:28:290
178532830,2chrt0-21swapper/319:38:293
178532830,2chrt0-21swapper/319:38:293
250552750,3sleep30-21swapper/319:58:233
250552750,3sleep30-21swapper/319:58:233
157472640,5sleep115746-21needreboot19:33:241
157472640,5sleep115746-21needreboot19:33:241
157472640,5sleep115746-21needreboot19:33:241
255212630,2sleep20-21swapper/221:28:062
255212630,2sleep20-21swapper/221:28:062
209192600,3chrt20922-21fschecks_time19:48:201
209192600,3chrt20922-21fschecks_time19:48:201
209192600,3chrt20922-21fschecks_time19:48:201
191372570,3sleep30-21swapper/319:43:193
191372570,3sleep30-21swapper/319:43:193
109342510,3sleep30-21swapper/322:13:263
109342510,3sleep30-21swapper/322:13:263
7911994842,3cyclictest0-21swapper/100:03:121
7911994842,3cyclictest0-21swapper/100:03:121
7911994842,3cyclictest0-21swapper/100:03:121
243952450,2sleep10-21swapper/121:23:231
243952450,2sleep10-21swapper/121:23:231
229152430,3chrt0-21swapper/200:13:212
229152430,3chrt0-21swapper/200:13:212
212632420,3chrt0-21swapper/322:43:093
212632420,3chrt0-21swapper/322:43:093
212632420,3chrt0-21swapper/322:43:093
7911994033,5cyclictest14266-21latency_hist20:58:071
37182400,2sleep13721-21cat23:23:061
7910993927,6cyclictest18905-21cut00:03:150
7910993927,6cyclictest18905-21cut00:03:150
7910993927,6cyclictest18905-21cut00:03:150
791199362,28cyclictest19653-21strings00:03:251
791199362,28cyclictest19653-21strings00:03:251
791099363,27cyclictest19579-21memory00:03:230
791099363,27cyclictest19579-21memory00:03:230
125952360,3sleep10-21swapper/122:18:221
125952360,3sleep10-21swapper/122:18:221
7911993427,5cyclictest3604-21if_err_eth020:28:201
7911993427,5cyclictest3604-21if_err_eth020:28:201
7911993426,5cyclictest32557-21latency_hist23:13:141
7911993426,5cyclictest32557-21latency_hist23:13:141
7910993326,5cyclictest26619-21mailstats21:28:280
7910993326,5cyclictest26619-21mailstats21:28:280
279112330,8sleep1281ktimers/122:58:261
279112330,8sleep1281ktimers/122:58:261
279112330,8sleep1281ktimers/122:58:261
7911993225,5cyclictest7159-21fschecks_time20:38:191
7911993225,5cyclictest7159-21fschecks_time20:38:191
7911993225,5cyclictest29424-21sshd20:08:321
7911993225,5cyclictest29424-21sshd20:08:321
7911993225,5cyclictest29424-21sshd20:08:321
7910993227,4cyclictest545-21in:imuxsock22:23:070
7910993227,4cyclictest545-21in:imuxsock22:23:070
7910993224,6cyclictest15221-21/usr/sbin/munin23:53:180
277602320,3sleep30-21swapper/322:58:243
277602320,3sleep30-21swapper/322:58:243
277602320,3sleep30-21swapper/322:58:243
7911993124,5cyclictest13873-21kernelversion23:48:221
7911993124,5cyclictest13873-21kernelversion23:48:221
7910993125,3cyclictest0-21swapper/020:03:060
7910993125,3cyclictest0-21swapper/020:03:060
7910993122,7cyclictest27090-21processes20:03:260
7910993122,7cyclictest27090-21processes20:03:260
7910993122,7cyclictest27090-21processes20:03:260
117012310,1sleep10-21swapper/123:43:221
117012310,1sleep10-21swapper/123:43:221
101082310,1sleep2351rcuc/220:43:282
101082310,1sleep2351rcuc/220:43:282
7911993025,3cyclictest21652-21kworker/u9:3-xprtiod21:52:251
7911993025,3cyclictest21652-21kworker/u9:3-xprtiod21:52:251
7911993024,4cyclictest26231-21latency_hist00:23:071
7911993024,4cyclictest26231-21latency_hist00:23:071
7911993023,5cyclictest11317-21/usr/sbin/munin19:23:181
7911993022,6cyclictest30711-21latency_hist23:08:171
7911993022,6cyclictest30711-21latency_hist23:08:171
7910993025,3cyclictest0-21swapper/021:23:060
7910993025,3cyclictest0-21swapper/021:23:060
7910993024,5cyclictest15774-21netstat19:33:240
7910993024,5cyclictest15774-21netstat19:33:240
7910993024,5cyclictest15774-21netstat19:33:240
7910993024,4cyclictest716-21latency23:13:220
7910993024,4cyclictest716-21latency23:13:220
7910993024,4cyclictest22886-21gltestperf19:53:210
7910993024,4cyclictest22886-21gltestperf19:53:210
7910993024,4cyclictest22886-21gltestperf19:53:210
7910993021,7cyclictest29642-21ntp_offset23:03:270
7910993021,7cyclictest29642-21ntp_offset23:03:270
7911992925,2cyclictest0-21swapper/121:18:281
7911992925,2cyclictest0-21swapper/121:18:281
7911992922,5cyclictest31108-21sendmail21:43:061
7911992922,5cyclictest31108-21sendmail21:43:061
7911992921,6cyclictest19020-21fschecks_time19:43:181
7911992921,6cyclictest19020-21fschecks_time19:43:181
7910992925,3cyclictest10681-21/usr/sbin/munin20:48:240
7910992923,4cyclictest7909-21cyclictest20:33:210
7910992923,4cyclictest7909-21cyclictest20:33:210
791399282,15cyclictest18873-21taskset00:03:123
791399282,15cyclictest18873-21taskset00:03:123
791399282,15cyclictest18873-21taskset00:03:123
7912992820,4cyclictest191852sleep200:03:202
7912992820,4cyclictest191852sleep200:03:202
7912992820,4cyclictest191852sleep200:03:202
7911992823,3cyclictest0-21swapper/119:28:191
7911992823,3cyclictest0-21swapper/119:28:191
7911992822,3cyclictest0-21swapper/123:53:291
7911992822,3cyclictest0-21swapper/123:53:291
7911992821,5cyclictest28818-21kernelversion00:28:221
7911992821,5cyclictest28818-21kernelversion00:28:221
7910992824,3cyclictest6194-21latency_hist22:03:070
7910992824,3cyclictest6194-21latency_hist22:03:070
7910992824,3cyclictest6194-21latency_hist22:03:070
7910992824,3cyclictest1273-21latency_hist00:43:060
7910992824,3cyclictest1273-21latency_hist00:43:060
7911992723,2cyclictest0-21swapper/122:48:221
7911992723,2cyclictest0-21swapper/122:48:221
7911992723,2cyclictest0-21swapper/122:48:221
7911992722,4cyclictest14975-21threads22:23:291
7911992722,4cyclictest14975-21threads22:23:291
7911992722,4cyclictest14975-21threads22:23:291
7911992721,5cyclictest9601-21ntp_offset20:43:241
7911992721,5cyclictest9601-21ntp_offset20:43:241
7911992721,4cyclictest547-21rs:main1
7911992721,4cyclictest547-21rs:main1
7911992721,3cyclictest0-21swapper/121:58:061
7911992721,3cyclictest0-21swapper/121:58:061
7911992720,5cyclictest14678-21systemd-journal22:31:321
7911992720,5cyclictest14678-21systemd-journal22:31:321
7911992718,7cyclictest17870-21fschecks_count22:33:201
7911992718,7cyclictest17870-21fschecks_count22:33:201
7911992718,7cyclictest17870-21fschecks_count22:33:201
7910992721,4cyclictest0-21swapper/000:08:290
7910992721,4cyclictest0-21swapper/000:08:290
231892270,2sleep00-21swapper/000:13:230
231892270,2sleep00-21swapper/000:13:230
150402270,3chrt0-21swapper/223:53:062
150402270,3chrt0-21swapper/223:53:062
7911992622,2cyclictest0-21swapper/122:13:271
7911992622,2cyclictest0-21swapper/122:13:271
7911992622,2cyclictest0-21swapper/121:03:271
7911992622,2cyclictest0-21swapper/121:03:271
7911992622,2cyclictest0-21swapper/121:03:271
7911992622,2cyclictest0-21swapper/121:03:271
7911992621,3cyclictest0-21swapper/121:43:291
7911992621,3cyclictest0-21swapper/121:43:291
7911992621,3cyclictest0-21swapper/121:43:291
7911992620,5cyclictest20629-21kernelversion21:13:221
7911992620,5cyclictest20629-21kernelversion21:13:221
7911992620,5cyclictest20629-21kernelversion21:13:221
7911992620,4cyclictest0-21swapper/120:48:291
7911992618,6cyclictest25713-21ls22:53:211
7911992618,6cyclictest25713-21ls22:53:211
7911992618,6cyclictest25713-21ls22:53:211
7910992622,2cyclictest0-21swapper/022:58:070
7910992622,2cyclictest0-21swapper/022:58:070
7910992622,2cyclictest0-21swapper/022:58:070
7910992621,4cyclictest9435-21timerwakeupswit22:08:310
7910992621,4cyclictest9435-21timerwakeupswit22:08:310
7910992621,4cyclictest9435-21timerwakeupswit22:08:310
7910992621,4cyclictest12768-21fschecks_count20:53:190
7910992620,3cyclictest0-21swapper/023:03:060
7910992620,3cyclictest0-21swapper/023:03:060
7910992620,3cyclictest0-21swapper/023:03:060
7910992620,3cyclictest0-21swapper/022:33:210
7910992620,3cyclictest0-21swapper/022:33:210
7910992620,3cyclictest0-21swapper/022:33:210
7910992619,5cyclictest12297-21fschecks_time22:18:200
7910992617,7cyclictest23447-21cut22:48:190
7911992521,2cyclictest0-21swapper/123:28:281
7911992521,2cyclictest0-21swapper/123:28:281
7911992521,2cyclictest0-21swapper/123:28:061
7911992521,2cyclictest0-21swapper/123:28:061
7911992521,2cyclictest0-21swapper/123:28:061
7911992521,2cyclictest0-21swapper/121:28:201
7911992521,2cyclictest0-21swapper/121:28:201
7911992520,3cyclictest0-21swapper/122:08:081
7911992520,3cyclictest0-21swapper/122:08:081
7911992520,3cyclictest0-21swapper/122:08:081
7911992519,4cyclictest25073-21needreboot19:58:241
7911992519,4cyclictest25073-21needreboot19:58:241
7911992519,4cyclictest0-21swapper/122:43:071
7911992519,4cyclictest0-21swapper/122:43:071
7911992519,4cyclictest0-21swapper/122:43:071
7911992518,5cyclictest31473-21sshd00:33:311
7911992518,5cyclictest31473-21sshd00:33:311
7910992521,3cyclictest0-21swapper/021:03:240
7910992521,3cyclictest0-21swapper/021:03:240
7910992521,3cyclictest0-21swapper/021:03:240
7910992521,3cyclictest0-21swapper/021:03:240
7910992520,4cyclictest13537-21fschecks_count23:48:190
7910992518,5cyclictest7909-21cyclictest23:28:290
7910992518,5cyclictest7909-21cyclictest23:28:290
7910992518,5cyclictest30746-21ntp_offset20:13:260
7910992518,5cyclictest30746-21ntp_offset20:13:260
7910992518,5cyclictest12126-21processes19:23:270
7911992419,3cyclictest0-21swapper/123:33:281
7911992419,3cyclictest0-21swapper/123:33:281
7911992419,3cyclictest0-21swapper/123:33:281
7911992419,3cyclictest0-21swapper/123:33:281
7911992419,3cyclictest0-21swapper/120:03:201
7911992419,3cyclictest0-21swapper/120:03:201
7911992419,3cyclictest0-21swapper/120:03:201
7911992419,3cyclictest0-21swapper/100:08:191
7911992419,3cyclictest0-21swapper/100:08:191
7911992418,4cyclictest0-21swapper/122:43:291
7911992418,4cyclictest0-21swapper/122:43:291
7911992418,4cyclictest0-21swapper/121:13:201
7911992417,5cyclictest1477-21sshd23:13:331
7911992417,5cyclictest1477-21sshd23:13:331
7911992416,6cyclictest30446-21if_eth020:13:221
7911992416,6cyclictest30446-21if_eth020:13:221
7911992416,5cyclictest11252-21idleruntime-cro23:43:061
7911992416,5cyclictest11252-21idleruntime-cro23:43:061
7911992416,5cyclictest11252-21idleruntime-cro23:43:061
7910992419,4cyclictest1229-21latency_hist20:23:120
7910992419,4cyclictest1229-21latency_hist20:23:120
7910992419,3cyclictest0-21swapper/019:43:170
7910992419,3cyclictest0-21swapper/019:43:170
7910992418,5cyclictest28604-21latency_hist00:28:200
7910992418,4cyclictest0-21swapper/023:28:060
7910992418,4cyclictest0-21swapper/023:28:060
7910992418,4cyclictest0-21swapper/023:28:060
7910992418,4cyclictest0-21swapper/022:18:070
7910992417,4cyclictest0-21swapper/020:43:210
7910992417,4cyclictest0-21swapper/020:43:210
7910992416,6cyclictest3275-21missed_timers21:53:220
7910992416,6cyclictest3275-21missed_timers21:53:220
7911992319,2cyclictest0-21swapper/121:08:211
7911992319,2cyclictest0-21swapper/121:08:211
7911992318,4cyclictest14678-21systemd-journal00:41:531
7911992318,4cyclictest14678-21systemd-journal00:41:531
7911992317,3cyclictest0-21swapper/122:08:321
7911992317,3cyclictest0-21swapper/122:08:321
7911992317,3cyclictest0-21swapper/122:08:321
7910992319,3cyclictest31486-21sort21:43:200
7910992319,3cyclictest31486-21sort21:43:200
7910992319,3cyclictest31486-21sort21:43:200
7910992318,3cyclictest0-21swapper/022:43:200
7910992318,3cyclictest0-21swapper/021:48:200
7910992318,3cyclictest0-21swapper/021:48:200
7910992316,5cyclictest21198-21wc22:43:070
7910992316,5cyclictest21198-21wc22:43:070
7910992316,5cyclictest21198-21wc22:43:070
7910992314,7cyclictest8521-21processes23:33:270
7910992314,7cyclictest8521-21processes23:33:270
7910992314,7cyclictest8521-21processes23:33:270
7910992314,7cyclictest8521-21processes23:33:270
7910992314,7cyclictest28508-21gltestperf20:08:200
7910992314,7cyclictest28508-21gltestperf20:08:200
7910992314,7cyclictest28508-21gltestperf20:08:200
791099231,19cyclictest31063-21find00:33:280
791099231,19cyclictest31063-21find00:33:280
7910992310,11cyclictest27581-21ls00:23:290
7911992213,7cyclictest22780-21fschecks_time19:53:201
7911992213,7cyclictest22780-21fschecks_time19:53:201
7911992213,7cyclictest22780-21fschecks_time19:53:201
7910992218,3cyclictest21022-21gltestperf19:48:200
7910992218,3cyclictest21022-21gltestperf19:48:200
7910992218,3cyclictest21022-21gltestperf19:48:200
7910992218,2cyclictest0-21swapper/022:23:280
7910992218,2cyclictest0-21swapper/022:23:280
7910992218,2cyclictest0-21swapper/022:23:280
7910992218,2cyclictest0-21swapper/020:38:290
7910992218,2cyclictest0-21swapper/020:38:290
7910992217,4cyclictest7909-21cyclictest21:18:180
7910992217,4cyclictest7909-21cyclictest21:18:180
7910992217,4cyclictest17999-21cron21:08:190
7910992217,4cyclictest17999-21cron21:08:190
7910992217,4cyclictest17999-21cron21:08:190
7910992217,3cyclictest0-21swapper/022:48:290
7910992217,3cyclictest0-21swapper/022:48:290
7910992217,3cyclictest0-21swapper/022:48:290
7910992217,3cyclictest0-21swapper/022:28:280
7910992217,3cyclictest0-21swapper/022:28:280
7910992217,3cyclictest0-21swapper/021:23:290
7910992217,3cyclictest0-21swapper/021:23:290
7910992217,3cyclictest0-21swapper/000:18:200
7910992217,3cyclictest0-21swapper/000:18:200
7910992216,5cyclictest29822-21gltestperf21:38:200
7910992216,5cyclictest29822-21gltestperf21:38:200
7910992215,5cyclictest2564-21kernelversion23:18:230
7910992215,5cyclictest0-21swapper/019:38:200
7910992215,5cyclictest0-21swapper/019:38:200
791299213,10cyclictest0-21swapper/222:28:332
791299213,10cyclictest0-21swapper/222:28:332
7911992117,3cyclictest90-1pr/ttyS020:33:281
7911992117,3cyclictest90-1pr/ttyS020:33:281
7911992115,4cyclictest0-21swapper/100:13:211
7911992115,4cyclictest0-21swapper/100:13:211
7911992113,6cyclictest1329-21/usr/sbin/munin20:23:251
7911992113,6cyclictest1329-21/usr/sbin/munin20:23:251
7911992113,6cyclictest1329-21/usr/sbin/munin20:23:251
791099215,3cyclictest181rcu_preempt20:28:230
791099215,3cyclictest181rcu_preempt20:28:230
7910992116,4cyclictest28689-21timerwakeupswit21:33:290
7910992116,4cyclictest28689-21timerwakeupswit21:33:290
7910992116,3cyclictest6715-21grep22:03:210
7910992116,3cyclictest6715-21grep22:03:210
7910992116,3cyclictest6715-21grep22:03:210
7910992116,3cyclictest0-21swapper/023:55:580
7910992116,3cyclictest0-21swapper/023:55:580
7910992114,6cyclictest31503-21ntp_offset23:08:250
7910992114,6cyclictest31503-21ntp_offset23:08:250
7910992113,6cyclictest2555-21unixbench_singl20:23:290
7910992113,6cyclictest2555-21unixbench_singl20:23:290
7910992113,6cyclictest2555-21unixbench_singl20:23:290
7910992112,7cyclictest14678-21systemd-journal21:16:240
90672200,2sleep30-21swapper/322:08:273
90672200,2sleep30-21swapper/322:08:273
90672200,2sleep30-21swapper/322:08:273
7913992012,5cyclictest105972sleep320:48:123
7913992012,5cyclictest105972sleep320:48:123
7910992015,3cyclictest0-21swapper/023:43:200
7910992015,3cyclictest0-21swapper/023:43:200
7910992015,3cyclictest0-21swapper/020:58:270
7910992015,3cyclictest0-21swapper/020:58:270
7910992011,7cyclictest11235-21cron23:43:060
7910992011,7cyclictest11235-21cron23:43:060
7910992011,7cyclictest11235-21cron23:43:060
791399192,9cyclictest0-21swapper/300:33:313
791399192,9cyclictest0-21swapper/300:33:313
791299193,9cyclictest0-21swapper/220:04:322
791299193,9cyclictest0-21swapper/220:04:322
791299193,9cyclictest0-21swapper/220:04:322
791299193,10cyclictest0-21swapper/200:09:312
791299193,10cyclictest0-21swapper/200:09:312
791399183,9cyclictest0-21swapper/321:19:323
791399183,9cyclictest0-21swapper/321:19:323
791399182,10cyclictest0-21swapper/300:03:273
791399182,10cyclictest0-21swapper/300:03:273
791399181,13cyclictest15163-21chrt19:33:183
791399181,12cyclictest13104-21taskset20:53:223
791399181,11cyclictest29389-21taskset23:03:233
791399181,11cyclictest29389-21taskset23:03:233
791299183,9cyclictest0-21swapper/223:18:332
791299183,9cyclictest0-21swapper/222:39:322
791299183,9cyclictest0-21swapper/222:39:322
791299183,9cyclictest0-21swapper/222:39:322
791299183,9cyclictest0-21swapper/222:14:322
791299183,9cyclictest0-21swapper/222:14:322
791299183,9cyclictest0-21swapper/222:11:072
791299183,9cyclictest0-21swapper/222:11:072
791299183,9cyclictest0-21swapper/222:11:072
791299183,9cyclictest0-21swapper/221:54:322
791299183,9cyclictest0-21swapper/221:54:322
791299183,9cyclictest0-21swapper/221:49:322
791299183,9cyclictest0-21swapper/221:49:322
791299183,9cyclictest0-21swapper/221:44:322
791299183,9cyclictest0-21swapper/221:44:322
791299183,9cyclictest0-21swapper/221:44:322
791299183,9cyclictest0-21swapper/220:34:322
791299183,9cyclictest0-21swapper/220:34:322
791299183,9cyclictest0-21swapper/219:59:332
791299183,9cyclictest0-21swapper/219:59:332
791299183,9cyclictest0-21swapper/219:48:322
791299183,9cyclictest0-21swapper/219:48:322
791299183,9cyclictest0-21swapper/219:48:322
791299183,9cyclictest0-21swapper/219:44:322
791299183,9cyclictest0-21swapper/219:44:322
791299183,9cyclictest0-21swapper/200:39:312
791299183,9cyclictest0-21swapper/200:39:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional