You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 05:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Jun 24, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3117721340,2sleep10-21swapper/122:38:171
3117721340,2sleep10-21swapper/122:38:171
3117721340,2sleep10-21swapper/122:38:171
726121260,8sleep1281ktimers/121:33:251
726121260,8sleep1281ktimers/121:33:251
488221060,2sleep04884-21latency21:28:180
1788721020,3chrt0-21swapper/323:28:243
1788721020,3chrt0-21swapper/323:28:243
129692810,4chrt12971-21grep20:23:261
129692810,4chrt12971-21grep20:23:261
123322810,3sleep30-21swapper/300:42:463
123322810,3sleep30-21swapper/300:42:463
219792790,3chrt21981-21fschecks_count20:48:151
75212760,3sleep30-21swapper/321:33:273
75212760,3sleep30-21swapper/321:33:273
254812710,3chrt0-21swapper/323:48:263
268012670,3sleep30-21swapper/319:33:193
268012670,3sleep30-21swapper/319:33:193
268012670,3sleep30-21swapper/319:33:193
4362620,3chrt0-21swapper/321:18:083
4362620,3chrt0-21swapper/321:18:083
176112590,3sleep30-21swapper/322:03:143
176112590,3sleep30-21swapper/322:03:143
176112590,3sleep30-21swapper/322:03:143
241812580,3sleep20-21swapper/220:53:182
241812580,3sleep20-21swapper/220:53:182
48692560,3sleep10-21swapper/100:23:021
48692560,3sleep10-21swapper/100:23:021
48692560,3sleep10-21swapper/100:23:021
228212560,3sleep20-21swapper/220:48:242
228212560,3sleep20-21swapper/220:48:242
228212560,3sleep20-21swapper/220:48:242
228212560,3sleep20-21swapper/220:48:242
116792560,3chrt0-21swapper/321:48:023
116792560,3chrt0-21swapper/321:48:023
116792560,3chrt0-21swapper/321:48:023
313142540,3sleep20-21swapper/200:03:332
313142540,3sleep20-21swapper/200:03:332
313142540,3sleep20-21swapper/200:03:332
130342520,3chrt0-21swapper/220:23:272
130342520,3chrt0-21swapper/220:23:272
48282490,2sleep04868-21tr00:23:020
48282490,2sleep04868-21tr00:23:020
48282490,2sleep04868-21tr00:23:020
18915994835,7cyclictest0-21swapper/000:03:090
18915994835,7cyclictest0-21swapper/000:03:090
284942460,2chrt0-21swapper/319:38:183
284942460,2chrt0-21swapper/319:38:183
184892460,5sleep018663-21grep22:03:240
184892460,5sleep018663-21grep22:03:240
104942450,3chrt0-21swapper/123:08:241
104942450,3chrt0-21swapper/123:08:241
325562440,3chrt0-21swapper/222:43:132
325562440,3chrt0-21swapper/222:43:132
325562440,3chrt0-21swapper/222:43:132
36872430,3sleep3431rcuc/300:18:173
36872430,3sleep3431rcuc/300:18:173
36872430,3sleep3431rcuc/300:18:173
278652420,3sleep30-21swapper/322:28:273
278652420,3sleep30-21swapper/322:28:273
159492410,3sleep20-21swapper/221:58:192
159492410,3sleep20-21swapper/221:58:192
159492410,3sleep20-21swapper/221:58:192
88242400,3chrt0-21swapper/223:03:272
88242400,3chrt0-21swapper/223:03:272
57992390,4sleep00-21swapper/020:03:310
57992390,4sleep00-21swapper/020:03:310
1891799391,31cyclictest29924-21taskset00:03:092
1891799391,31cyclictest29924-21taskset00:03:092
18915993928,9cyclictest2281-21sendmail21:23:010
18915993928,9cyclictest2281-21sendmail21:23:010
18915993928,9cyclictest2281-21sendmail21:23:010
18915993629,5cyclictest9932-21unixbench_singl20:13:250
18915993629,5cyclictest9932-21unixbench_singl20:13:250
186612360,3sleep20-21swapper/223:33:062
186612360,3sleep20-21swapper/223:33:062
285412340,3sleep30-21swapper/321:03:243
285412340,3sleep30-21swapper/321:03:243
76502330,2sleep07736-21switchtime20:08:230
76502330,2sleep07736-21switchtime20:08:230
34002330,2chrt0-21swapper/219:58:152
34002330,2chrt0-21swapper/219:58:152
18916993326,5cyclictest572-21cups-browsed23:13:251
18916993326,5cyclictest572-21cups-browsed23:13:251
18916993326,5cyclictest572-21cups-browsed23:13:251
18915993327,4cyclictest17792-21gltestperf22:03:150
18915993327,4cyclictest17792-21gltestperf22:03:150
158032330,2sleep20-21swapper/223:23:212
158032330,2sleep20-21swapper/223:23:212
158032330,2sleep20-21swapper/223:23:212
219312320,3sleep20-21swapper/223:38:242
219312320,3sleep20-21swapper/223:38:242
219312320,3sleep20-21swapper/223:38:242
20472320,2sleep30-21swapper/300:13:193
20472320,2sleep30-21swapper/300:13:193
18916993227,3cyclictest30372-21sed00:03:171
18916993227,3cyclictest30372-21sed00:03:171
18916993227,3cyclictest30372-21sed00:03:171
18916993227,3cyclictest25240-21switchtime19:28:241
18916993227,3cyclictest25240-21switchtime19:28:241
18916993225,5cyclictest32597-21netstat00:08:241
18916993225,5cyclictest32597-21netstat00:08:241
18916993225,5cyclictest32222-21cron22:43:151
18916993225,5cyclictest32222-21cron22:43:151
18916993225,5cyclictest23906-21cut20:53:161
18916993225,5cyclictest23906-21cut20:53:161
18916993225,5cyclictest21330-21latency_hist22:13:131
18916993225,5cyclictest21330-21latency_hist22:13:131
18916993225,5cyclictest21330-21latency_hist22:13:131
18915993228,3cyclictest20441-21cron23:38:010
18915993228,3cyclictest20441-21cron23:38:010
18915993227,3cyclictest0-21swapper/000:43:140
18915993226,4cyclictest30378-21grep00:03:170
18915993226,4cyclictest30378-21grep00:03:170
18915993226,4cyclictest30378-21grep00:03:170
18915993226,4cyclictest26655-21latency_hist22:28:020
18915993226,4cyclictest26655-21latency_hist22:28:020
18915993226,4cyclictest21814-21unixbench-2d23:38:240
18915993226,4cyclictest21814-21unixbench-2d23:38:240
18915993226,4cyclictest21814-21unixbench-2d23:38:240
18916993125,5cyclictest3512-21kernelversion19:58:161
18916993125,5cyclictest3512-21kernelversion19:58:161
18916993125,4cyclictest1346-21avahi-daemon23:58:241
18916993125,4cyclictest1346-21avahi-daemon23:58:241
18916993125,4cyclictest1346-21avahi-daemon19:43:221
18916993125,4cyclictest1346-21avahi-daemon19:43:221
18915993126,4cyclictest545-21in:imuxsock20:03:010
18915993126,4cyclictest545-21in:imuxsock20:03:010
18915993125,4cyclictest17823-21cron20:38:020
18915993125,4cyclictest17823-21cron20:38:020
18915993125,4cyclictest17823-21cron20:38:020
18915993125,4cyclictest0-21swapper/021:03:020
18916993026,2cyclictest0-21swapper/123:08:021
18916993026,2cyclictest0-21swapper/123:08:021
18916993026,2cyclictest0-21swapper/120:18:121
18916993026,2cyclictest0-21swapper/120:18:121
18916993024,5cyclictest7306-21sendmail23:03:021
18916993024,5cyclictest7306-21sendmail23:03:021
18916993023,5cyclictest27309-21latency_hist21:03:071
18916993022,5cyclictest20341-21munin-node22:54:361
18916993022,5cyclictest20341-21munin-node22:54:361
18916993022,5cyclictest20341-21munin-node22:54:361
18915993024,5cyclictest15986-21mailstats23:23:240
18915993024,5cyclictest15986-21mailstats23:23:240
18915993024,5cyclictest15986-21mailstats23:23:240
18915993024,4cyclictest32355-21latency_hist22:43:080
18915993024,4cyclictest32355-21latency_hist22:43:080
18915993024,4cyclictest32355-21latency_hist22:43:080
18915993024,4cyclictest28315-21tail19:38:160
18915993024,4cyclictest28315-21tail19:38:160
18915993024,4cyclictest23977-21gltestperf20:53:160
18915993024,4cyclictest23977-21gltestperf20:53:160
18915993024,4cyclictest0-21swapper/020:28:020
18915993024,4cyclictest0-21swapper/020:28:020
18915993022,6cyclictest23673-21latency22:18:210
18915993022,6cyclictest23673-21latency22:18:210
18915993022,6cyclictest10558-21latency_hist00:38:050
18915993022,6cyclictest10558-21latency_hist00:38:050
18915993022,6cyclictest10558-21latency_hist00:38:050
295592290,3sleep30-21swapper/322:33:253
295592290,3sleep30-21swapper/322:33:253
18916992925,3cyclictest2992-21kernelversion21:23:181
18916992925,3cyclictest2992-21kernelversion21:23:181
18916992923,4cyclictest16712-21sh23:28:151
18916992923,4cyclictest16712-21sh23:28:151
18916992922,5cyclictest12540-21latency_hist00:43:101
18916992922,5cyclictest12540-21latency_hist00:43:101
18916992921,6cyclictest29974-21netstat21:08:201
18916992921,6cyclictest29974-21netstat21:08:201
18916992921,6cyclictest29974-21netstat21:08:201
18915992924,4cyclictest18768-21ntp_kernel_err20:38:210
18915992924,4cyclictest18768-21ntp_kernel_err20:38:210
18915992924,3cyclictest0-21swapper/022:58:020
18915992924,3cyclictest0-21swapper/022:58:020
18915992924,3cyclictest0-21swapper/022:58:020
18915992923,5cyclictest12927-21cron23:18:020
18915992923,5cyclictest12927-21cron23:18:020
18915992923,5cyclictest12927-21cron23:18:020
18915992923,4cyclictest30718-21fschecks_count22:38:140
18915992923,4cyclictest17176-21fschecks_time23:28:150
18915992923,4cyclictest17176-21fschecks_time23:28:150
18915992922,5cyclictest21767-21latency22:13:160
18915992922,5cyclictest21767-21latency22:13:160
18915992922,5cyclictest21767-21latency22:13:160
18915992920,7cyclictest25092-21needreboot23:48:220
18915992919,8cyclictest2792-21cron19:58:020
18915992919,8cyclictest2792-21cron19:58:020
18916992824,3cyclictest11094-21timerwakeupswit21:43:241
18916992824,3cyclictest11094-21timerwakeupswit21:43:241
18916992824,3cyclictest11094-21timerwakeupswit21:43:241
18916992823,4cyclictest572-21cups-browsed23:38:221
18916992823,4cyclictest572-21cups-browsed23:38:221
18916992823,4cyclictest572-21cups-browsed23:38:221
18916992822,5cyclictest14870-21timerwakeupswit21:53:391
18916992822,5cyclictest14870-21timerwakeupswit21:53:391
18916992822,5cyclictest14870-21timerwakeupswit21:53:391
18916992822,4cyclictest17273-21latency_hist22:03:021
18916992822,4cyclictest17273-21latency_hist22:03:021
18916992822,4cyclictest17273-21latency_hist22:03:021
18916992821,5cyclictest24866-21latency_hist22:23:061
18916992821,5cyclictest24866-21latency_hist22:23:061
18916992821,5cyclictest18577-21sh23:33:141
18916992821,5cyclictest18577-21sh23:33:141
18916992821,5cyclictest13652-21kernelversion23:18:171
18916992821,5cyclictest13652-21kernelversion23:18:171
18916992821,5cyclictest13652-21kernelversion23:18:171
18916992821,5cyclictest13652-21kernelversion23:18:171
18916992821,5cyclictest10440-21sshd00:37:241
18916992821,5cyclictest10440-21sshd00:37:241
18916992821,5cyclictest10440-21sshd00:37:241
18916992821,5cyclictest0-21swapper/123:58:021
18916992821,5cyclictest0-21swapper/123:58:021
18915992823,3cyclictest0-21swapper/022:08:200
18915992823,3cyclictest0-21swapper/022:08:200
18915992823,3cyclictest0-21swapper/022:08:200
18915992822,4cyclictest31030-21latency_hist21:13:070
18915992822,4cyclictest31030-21latency_hist21:13:070
18915992822,4cyclictest31030-21latency_hist21:13:070
18915992822,4cyclictest0-21swapper/019:33:220
18915992822,4cyclictest0-21swapper/019:33:220
18915992822,4cyclictest0-21swapper/019:33:220
18915992822,4cyclictest0-21swapper/000:08:280
18915992822,4cyclictest0-21swapper/000:08:280
18915992821,5cyclictest27073-21fschecks_time22:28:170
18915992821,5cyclictest27073-21fschecks_time22:28:170
18915992821,5cyclictest26635-21gltestperf23:53:160
18915992821,5cyclictest26635-21gltestperf23:53:160
18916992723,3cyclictest25264-21fschecks_time22:23:161
18916992723,3cyclictest25264-21fschecks_time22:23:161
18916992723,2cyclictest0-21swapper/120:18:221
18916992723,2cyclictest0-21swapper/120:18:221
18916992722,4cyclictest572-21cups-browsed00:28:231
18916992722,4cyclictest572-21cups-browsed00:28:231
18916992722,4cyclictest572-21cups-browsed00:28:231
18916992721,4cyclictest13645-21latency_hist21:53:101
18916992721,4cyclictest13645-21latency_hist21:53:101
18916992721,4cyclictest13645-21latency_hist21:53:101
18916992720,5cyclictest5436-21gltestperf00:23:161
18916992720,5cyclictest5436-21gltestperf00:23:161
18915992723,2cyclictest0-21swapper/021:03:240
18915992723,2cyclictest0-21swapper/021:03:240
18915992723,2cyclictest0-21swapper/000:28:150
18915992722,4cyclictest32151-21switchtime21:13:260
18915992722,4cyclictest32151-21switchtime21:13:260
18915992722,4cyclictest13022-21unixbench-2d21:48:250
18915992722,4cyclictest13022-21unixbench-2d21:48:250
18915992722,4cyclictest13022-21unixbench-2d21:48:250
18915992722,3cyclictest0-21swapper/021:48:160
18915992722,3cyclictest0-21swapper/021:38:160
18915992722,3cyclictest0-21swapper/021:38:160
18915992721,4cyclictest15978-21gltestperf21:58:190
18915992721,4cyclictest0-21swapper/000:18:020
18915992721,4cyclictest0-21swapper/000:18:020
18915992720,4cyclictest15559-21latency_hist21:58:110
18915992720,4cyclictest15559-21latency_hist21:58:110
18915992720,4cyclictest15559-21latency_hist21:58:110
18915992719,6cyclictest2507-21missed_timers22:48:180
18915992719,6cyclictest2507-21missed_timers22:48:180
18915992719,6cyclictest2507-21missed_timers22:48:180
18916992622,2cyclictest0-21swapper/119:23:151
18916992622,2cyclictest0-21swapper/119:23:151
18916992621,3cyclictest0-21swapper/119:38:241
18916992621,3cyclictest0-21swapper/119:38:241
18916992620,5cyclictest23563-21threads23:43:231
18916992620,5cyclictest23563-21threads23:43:231
18916992619,5cyclictest32729-21threads19:48:231
18916992619,5cyclictest32729-21threads19:48:231
18916992619,5cyclictest32729-21threads19:48:231
18915992622,2cyclictest0-21swapper/019:28:020
18915992622,2cyclictest0-21swapper/019:28:020
18915992621,4cyclictest7298-21latency_hist23:03:120
18915992621,4cyclictest7298-21latency_hist23:03:120
18915992621,3cyclictest32086-21gltestperf19:48:150
18915992621,3cyclictest32086-21gltestperf19:48:150
18915992621,3cyclictest32086-21gltestperf19:48:150
18915992621,3cyclictest0-21swapper/023:03:220
18915992621,3cyclictest0-21swapper/023:03:220
18915992620,5cyclictest2857-21gltestperf21:23:160
18915992620,5cyclictest2857-21gltestperf21:23:160
18915992620,4cyclictest6563-21gltestperf21:33:160
18915992620,4cyclictest6563-21gltestperf21:33:160
18915992620,4cyclictest0-21swapper/023:08:220
18915992620,4cyclictest0-21swapper/023:08:220
18915992620,4cyclictest0-21swapper/022:48:140
18915992618,6cyclictest10373-21gltestperf21:43:150
18915992618,6cyclictest10373-21gltestperf21:43:150
18917992518,4cyclictest22183-21chrt22:13:222
18917992518,4cyclictest22183-21chrt22:13:222
18917992518,4cyclictest22183-21chrt22:13:222
18916992521,2cyclictest0-21swapper/122:48:231
18916992521,2cyclictest0-21swapper/122:48:231
18916992521,2cyclictest0-21swapper/122:48:231
18916992520,3cyclictest29013-21gltestperf22:33:181
18916992520,3cyclictest29013-21gltestperf22:33:181
18916992520,3cyclictest27775-21cut21:03:151
18916992520,3cyclictest27775-21cut21:03:151
18916992520,3cyclictest0-21swapper/122:28:271
18916992520,3cyclictest0-21swapper/122:28:271
18916992520,3cyclictest0-21swapper/120:33:251
18916992520,3cyclictest0-21swapper/120:33:251
18916992520,3cyclictest0-21swapper/120:33:251
18916992520,3cyclictest0-21swapper/120:03:321
18916992520,3cyclictest0-21swapper/120:03:321
18915992521,2cyclictest0-21swapper/020:48:150
18915992521,2cyclictest0-21swapper/020:48:150
18915992521,2cyclictest0-21swapper/020:48:150
18915992521,2cyclictest0-21swapper/020:48:150
18915992520,4cyclictest25400-21unixbench-2d19:28:250
18915992520,4cyclictest25400-21unixbench-2d19:28:250
18915992520,3cyclictest0-21swapper/000:28:230
18915992520,3cyclictest0-21swapper/000:28:230
18915992520,3cyclictest0-21swapper/000:28:230
18915992519,4cyclictest0-21swapper/000:38:240
18915992518,5cyclictest0-21swapper/020:43:210
18915992518,5cyclictest0-21swapper/020:43:210
18915992518,4cyclictest0-21swapper/023:18:150
18915992518,4cyclictest0-21swapper/023:18:150
18915992518,4cyclictest0-21swapper/023:18:150
18915992518,4cyclictest0-21swapper/023:18:150
18915992517,6cyclictest0-21swapper/022:33:280
18915992514,9cyclictest0-21swapper/000:23:230
63412240,2sleep30-21swapper/322:58:213
63412240,2sleep30-21swapper/322:58:213
253542241,3sleep30-21swapper/319:28:263
253542241,3sleep30-21swapper/319:28:263
18917992418,4cyclictest351rcuc/220:14:482
18917992418,4cyclictest351rcuc/220:14:482
18916992420,2cyclictest0-21swapper/121:28:181
18916992419,3cyclictest0-21swapper/123:23:251
18916992419,3cyclictest0-21swapper/123:23:251
18916992419,3cyclictest0-21swapper/123:23:251
18916992418,5cyclictest18913-21cyclictest20:43:171
18916992418,4cyclictest2251-21latency_hist21:23:011
18916992418,4cyclictest2251-21latency_hist21:23:011
18916992418,4cyclictest2251-21latency_hist21:23:011
18916992417,5cyclictest26906-21proc_pri19:33:221
18916992417,5cyclictest26906-21proc_pri19:33:221
18916992417,5cyclictest26906-21proc_pri19:33:221
18916992417,5cyclictest15623-21sshd20:28:441
18916992417,5cyclictest15623-21sshd20:28:441
18916992417,5cyclictest15623-21sshd20:28:441
18915992419,3cyclictest0-21swapper/019:43:240
18915992419,3cyclictest0-21swapper/019:43:240
18915992417,5cyclictest0-21swapper/020:23:060
18915992417,5cyclictest0-21swapper/020:23:060
18915992417,4cyclictest0-21swapper/022:43:220
18915992417,4cyclictest0-21swapper/020:28:150
18915992417,4cyclictest0-21swapper/020:28:150
18915992417,4cyclictest0-21swapper/020:28:150
1891899232,15cyclictest0-21swapper/300:03:073
1891899232,15cyclictest0-21swapper/300:03:073
18917992316,5cyclictest351rcuc/222:37:112
18917992316,5cyclictest351rcuc/222:37:112
18916992319,2cyclictest0-21swapper/122:03:201
18916992319,2cyclictest0-21swapper/122:03:201
18916992317,4cyclictest0-21swapper/123:51:371
18916992317,4cyclictest0-21swapper/122:13:341
18916992317,4cyclictest0-21swapper/122:13:341
18916992317,4cyclictest0-21swapper/122:13:341
18916992315,6cyclictest18912-21grep20:38:241
18916992315,6cyclictest18912-21grep20:38:241
18915992319,2cyclictest0-21swapper/023:43:200
18915992319,2cyclictest0-21swapper/023:43:200
35722220,3sleep20-21swapper/221:23:252
35722220,3sleep20-21swapper/221:23:252
18916992216,4cyclictest2803-21latency_hist19:58:021
18916992216,4cyclictest2803-21latency_hist19:58:021
18918992115,4cyclictest7128-21taskset00:28:143
18916992116,3cyclictest0-21swapper/120:08:231
18916992116,3cyclictest0-21swapper/120:08:231
18916992115,4cyclictest542-21rngd20:52:331
18916992115,4cyclictest542-21rngd20:52:331
18916992115,4cyclictest542-21rngd20:52:331
18916992115,4cyclictest542-21rngd20:52:331
18916992114,5cyclictest31613-21cat21:13:181
18916992114,5cyclictest31613-21cat21:13:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional