You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 11:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue Jun 30, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3144621170,2chrt31445-21entropy22:48:181
3144621170,2chrt31445-21entropy22:48:181
36212940,2chrt0-21swapper/320:08:083
36212940,2chrt0-21swapper/320:08:083
106502850,3sleep30-21swapper/320:23:323
106502850,3sleep30-21swapper/320:23:323
79692780,3sleep07967-21sensors_temp21:43:290
79692780,3sleep07967-21sensors_temp21:43:290
79692780,3sleep07967-21sensors_temp21:43:290
101582710,3chrt0-21swapper/321:48:303
101582710,3chrt0-21swapper/321:48:303
101582710,3chrt0-21swapper/321:48:303
169192680,3sleep00-21swapper/023:33:270
169192680,3sleep00-21swapper/023:33:270
13292630,3sleep30-21swapper/321:28:163
13292630,3sleep30-21swapper/321:28:163
13292630,3sleep30-21swapper/321:28:163
23592600,3sleep20-21swapper/200:23:222
23592600,3sleep20-21swapper/200:23:222
15920996052,3cyclictest0-21swapper/100:03:241
15920996052,3cyclictest0-21swapper/100:03:241
15920996052,3cyclictest0-21swapper/100:03:241
232702590,3sleep30-21swapper/320:58:263
232702590,3sleep30-21swapper/320:58:263
15912995729,26cyclictest30773-21netstat21:18:270
15912995729,26cyclictest30773-21netstat21:18:270
15912995729,26cyclictest30773-21netstat21:18:270
149382560,2sleep20-21swapper/220:38:102
149382560,2sleep20-21swapper/220:38:102
15912995444,5cyclictest0-21swapper/000:03:150
15912995444,5cyclictest0-21swapper/000:03:150
15912995444,5cyclictest0-21swapper/000:03:150
43882500,3chrt0-21swapper/221:33:302
43882500,3chrt0-21swapper/221:33:302
284252450,3chrt0-21swapper/300:03:583
284252450,3chrt0-21swapper/300:03:583
284252450,3chrt0-21swapper/300:03:583
297262440,2chrt0-21swapper/222:43:202
297262440,2chrt0-21swapper/222:43:202
204772440,3sleep30-21swapper/320:53:083
204772440,3sleep30-21swapper/320:53:083
195852440,2sleep119580-21latency_hist23:43:171
246842430,3sleep20-21swapper/222:28:292
246842430,3sleep20-21swapper/222:28:292
40242420,5chrt4014-21cpu00:28:190
40242420,5chrt4014-21cpu00:28:190
40242420,5chrt4014-21cpu00:28:190
1591299411,35cyclictest27850-21cat00:03:250
1591299411,35cyclictest27850-21cat00:03:250
1591299411,35cyclictest27850-21cat00:03:250
1592799401,23cyclictest26982-21chrt00:03:133
1592799401,23cyclictest26982-21chrt00:03:133
1592799401,23cyclictest26982-21chrt00:03:133
19542390,2sleep02061-21df_inode20:03:200
19542390,2sleep02061-21df_inode20:03:200
107832390,3sleep10-21swapper/121:53:191
107832390,3sleep10-21swapper/121:53:191
15920993832,4cyclictest360-21cut19:58:201
15920993832,4cyclictest360-21cut19:58:201
89942370,1sleep09149-21fschecks_time21:48:200
89942370,1sleep09149-21fschecks_time21:48:200
89942370,1sleep09149-21fschecks_time21:48:200
201862350,1sleep120253-21processes19:23:291
201862350,1sleep120253-21processes19:23:291
15920993529,4cyclictest15901-21cyclictest23:33:191
15920993529,4cyclictest15901-21cyclictest23:33:191
15920993424,3cyclictest181rcu_preempt20:13:071
15920993424,3cyclictest181rcu_preempt20:13:071
15912993424,7cyclictest550-21cron23:23:090
15912993424,7cyclictest550-21cron23:23:090
15912993424,7cyclictest550-21cron23:23:090
267972330,2sleep00-21swapper/019:43:140
267972330,2sleep00-21swapper/019:43:140
1592099336,6cyclictest281ktimers/122:57:101
1592099336,6cyclictest281ktimers/122:57:101
1592099336,6cyclictest281ktimers/122:57:101
15920993312,18cyclictest271rcuc/121:04:381
15920993312,18cyclictest271rcuc/121:04:381
15912993326,5cyclictest4738-21cut23:03:190
81942320,3sleep00-21swapper/023:13:130
81942320,3sleep00-21swapper/023:13:130
15920993227,3cyclictest26932-21latency_hist00:03:091
15920993227,3cyclictest26932-21latency_hist00:03:091
15920993227,3cyclictest26932-21latency_hist00:03:091
15920993227,3cyclictest0-21swapper/123:23:071
15920993227,3cyclictest0-21swapper/123:23:071
15920993227,3cyclictest0-21swapper/123:23:071
15920993225,5cyclictest281ktimers/123:16:111
15920993225,5cyclictest281ktimers/123:16:111
15920993224,6cyclictest30216-21unixbench_singl00:08:361
15920993224,6cyclictest30216-21unixbench_singl00:08:361
15920993223,7cyclictest7707-21needreboot21:43:251
15920993223,7cyclictest7707-21needreboot21:43:251
15920993223,7cyclictest7707-21needreboot21:43:251
15920993223,7cyclictest27958-21latency_hist21:13:071
15920993223,7cyclictest27958-21latency_hist21:13:071
15920993223,7cyclictest27958-21latency_hist21:13:071
15912993227,3cyclictest0-21swapper/023:03:070
15912993227,3cyclictest0-21swapper/023:03:070
1592099319,14cyclictest29-21ksoftirqd/123:40:571
1592099318,11cyclictest90-1pr/ttyS022:43:131
15920993124,5cyclictest2339-21fschecks_count00:23:221
15920993124,5cyclictest2339-21fschecks_count00:23:221
15920993121,8cyclictest5285-21netstat23:03:251
15920993121,8cyclictest5285-21netstat23:03:251
15920993117,3cyclictest90-1pr/ttyS019:30:471
15920993117,3cyclictest90-1pr/ttyS019:30:471
15912993126,4cyclictest9245-21latency_hist20:23:070
15912993126,4cyclictest9245-21latency_hist20:23:070
15912993125,4cyclictest9756-21grep00:43:190
15912993125,4cyclictest9756-21grep00:43:190
15920993026,3cyclictest28550-21latency_hist19:48:071
15920993026,3cyclictest28550-21latency_hist19:48:071
15920993026,3cyclictest15901-21cyclictest20:33:071
15920993026,3cyclictest15901-21cyclictest20:33:071
15920993025,4cyclictest21833-21sendmail22:23:071
15920993025,4cyclictest21833-21sendmail22:23:071
15920993025,4cyclictest21833-21sendmail22:23:071
15920993023,5cyclictest17147-21tr22:08:261
15920993023,5cyclictest17147-21tr22:08:261
15920993023,5cyclictest17147-21tr22:08:261
15912993021,7cyclictest11603-21fschecks_time20:28:220
15912993021,7cyclictest11603-21fschecks_time20:28:220
15912993020,7cyclictest21158-21threads22:18:290
15912993020,7cyclictest21158-21threads22:18:290
15912993020,7cyclictest21158-21threads22:18:290
139842300,1sleep10-21swapper/120:33:271
139842300,1sleep10-21swapper/120:33:271
1592099299,8cyclictest29-21ksoftirqd/121:32:141
1592099299,8cyclictest29-21ksoftirqd/121:32:141
1592099297,4cyclictest281ktimers/122:07:391
1592099297,4cyclictest281ktimers/122:07:391
1592099297,4cyclictest281ktimers/121:02:371
1592099297,4cyclictest281ktimers/121:02:371
15920992923,5cyclictest25518-21kernelversion19:38:231
15920992923,5cyclictest25518-21kernelversion19:38:231
15920992923,4cyclictest595-21sshd23:09:471
15920992923,4cyclictest595-21sshd23:09:471
15920992923,3cyclictest0-21swapper/123:48:291
15920992923,3cyclictest0-21swapper/123:48:291
15920992923,3cyclictest0-21swapper/123:48:291
15920992923,3cyclictest0-21swapper/120:18:281
15920992923,3cyclictest0-21swapper/120:18:281
15920992918,5cyclictest601-21snmpd22:48:311
15920992918,5cyclictest601-21snmpd22:48:311
15920992918,3cyclictest617-21lldpd22:15:241
15920992918,3cyclictest617-21lldpd22:15:241
15920992918,3cyclictest617-21lldpd22:15:241
15920992914,4cyclictest181rcu_preempt20:16:081
15920992914,4cyclictest181rcu_preempt20:16:081
15912992924,3cyclictest0-21swapper/020:08:290
15912992924,3cyclictest0-21swapper/020:08:290
15912992922,5cyclictest9774-21gltestperf20:23:230
15912992922,5cyclictest9774-21gltestperf20:23:230
15912992922,5cyclictest23279-21ntp_kernel_err20:58:260
15912992922,5cyclictest23279-21ntp_kernel_err20:58:260
15912992922,5cyclictest16398-21sshd20:38:470
15912992921,6cyclictest31487-21fschecks_count22:48:180
15912992921,6cyclictest31487-21fschecks_count22:48:180
129252290,3sleep112619-21/usr/sbin/munin21:58:211
129252290,3sleep112619-21/usr/sbin/munin21:58:211
129252290,3sleep112619-21/usr/sbin/munin21:58:211
15920992823,4cyclictest545-21in:imuxsock20:08:071
15920992823,4cyclictest545-21in:imuxsock20:08:071
15920992823,3cyclictest31133-21gltestperf00:13:241
15920992823,3cyclictest31133-21gltestperf00:13:241
15920992822,5cyclictest4314-21sh23:03:091
15920992822,5cyclictest4314-21sh23:03:091
15920992822,5cyclictest4314-21sh23:03:091
15920992822,5cyclictest30302-21cut21:18:211
15920992822,5cyclictest30302-21cut21:18:211
15920992822,5cyclictest30302-21cut21:18:211
15920992821,5cyclictest12735-21latency_hist21:58:191
15920992821,4cyclictest0-21swapper/119:53:181
15920992820,3cyclictest181rcu_preempt21:13:241
15920992820,3cyclictest181rcu_preempt21:13:241
15920992819,7cyclictest281ktimers/120:23:361
15920992819,7cyclictest281ktimers/120:23:361
15920992818,3cyclictest181rcu_preempt19:57:311
15920992818,3cyclictest181rcu_preempt19:57:311
15920992818,3cyclictest181rcu_preempt19:57:311
15920992818,3cyclictest181rcu_preempt19:57:311
15920992816,3cyclictest90-1pr/ttyS020:53:371
15920992816,3cyclictest90-1pr/ttyS020:53:371
15920992815,4cyclictest181rcu_preempt23:53:531
15920992815,4cyclictest181rcu_preempt23:53:531
15920992815,4cyclictest181rcu_preempt23:53:531
15912992824,3cyclictest15901-21cyclictest19:48:070
15912992824,3cyclictest15901-21cyclictest19:48:070
15912992823,3cyclictest24796-21head21:03:210
15912992823,3cyclictest24796-21head21:03:210
15912992821,6cyclictest30232-21unixbench_singl00:08:360
15912992821,6cyclictest30232-21unixbench_singl00:08:360
15912992821,5cyclictest5887-21ntp_offset21:38:260
15912992821,5cyclictest5887-21ntp_offset21:38:260
15912992821,5cyclictest25351-21sshd22:30:450
15912992821,5cyclictest25351-21sshd22:30:450
15912992819,7cyclictest15183-21ls22:03:240
15912992819,7cyclictest15183-21ls22:03:240
143802280,3sleep10-21swapper/123:28:221
143802280,3sleep10-21swapper/123:28:221
106162280,2sleep10-21swapper/121:53:111
62222270,2chrt6225-21latency_hist23:08:070
62222270,2chrt6225-21latency_hist23:08:070
1592099278,9cyclictest29-21ksoftirqd/100:36:451
1592099278,9cyclictest29-21ksoftirqd/100:36:451
15920992719,6cyclictest24056-21grep19:33:281
15920992719,6cyclictest24056-21grep19:33:281
15920992717,2cyclictest181rcu_preempt00:23:431
15920992717,2cyclictest181rcu_preempt00:23:431
15920992717,2cyclictest181rcu_preempt00:23:431
15920992716,3cyclictest181rcu_preempt20:39:041
15912992722,3cyclictest0-21swapper/023:53:070
15912992722,3cyclictest0-21swapper/023:53:070
15912992722,3cyclictest0-21swapper/023:53:070
15912992721,4cyclictest15901-21cyclictest23:23:440
15912992721,4cyclictest15901-21cyclictest23:23:440
15912992721,4cyclictest15901-21cyclictest20:00:490
15912992721,4cyclictest15901-21cyclictest20:00:490
15912992720,5cyclictest31495-21needreboot00:13:280
15912992720,5cyclictest31495-21needreboot00:13:280
15912992720,5cyclictest18282-21sshd20:43:470
15912992720,5cyclictest18282-21sshd20:43:470
15912992720,5cyclictest18282-21sshd20:43:470
15912992719,6cyclictest17436-21threads22:08:300
15912992719,6cyclictest17436-21threads22:08:300
15912992719,6cyclictest17436-21threads22:08:300
280942260,2sleep20-21swapper/222:38:222
1592099264,3cyclictest181rcu_preempt21:36:121
1592099264,3cyclictest181rcu_preempt21:36:121
1592099263,5cyclictest271rcuc/123:46:121
1592099263,5cyclictest271rcuc/123:46:121
15920992619,6cyclictest6002-21rt-features21:38:271
15920992619,6cyclictest6002-21rt-features21:38:271
15920992618,6cyclictest939-21needreboot00:18:231
15920992616,2cyclictest181rcu_preempt20:43:281
15920992616,2cyclictest181rcu_preempt20:43:281
15920992616,2cyclictest181rcu_preempt20:43:281
15920992614,4cyclictest181rcu_preempt21:27:481
15920992614,4cyclictest181rcu_preempt21:27:481
15920992614,4cyclictest181rcu_preempt21:27:481
15912992622,2cyclictest0-21swapper/021:53:200
15912992622,2cyclictest0-21swapper/021:53:200
15912992621,3cyclictest0-21swapper/019:53:280
15912992621,3cyclictest0-21swapper/019:53:280
15912992621,3cyclictest0-21swapper/019:53:280
15912992621,3cyclictest0-21swapper/019:53:280
15912992621,3cyclictest0-21swapper/000:23:230
15912992620,4cyclictest1346-21avahi-daemon21:28:300
15912992620,4cyclictest1346-21avahi-daemon21:28:300
15912992620,4cyclictest0-21swapper/023:43:270
15912992620,4cyclictest0-21swapper/023:43:270
15912992619,5cyclictest30478-21latency_hist19:53:100
15912992619,5cyclictest30478-21latency_hist19:53:100
15912992618,6cyclictest14430-21latency_hist22:03:130
15912992618,6cyclictest14430-21latency_hist22:03:130
15912992618,6cyclictest14430-21latency_hist22:03:130
15912992616,8cyclictest7560-21latency_hist00:38:080
15912992616,8cyclictest7560-21latency_hist00:38:080
1592099259,3cyclictest281ktimers/100:31:131
1592099259,3cyclictest281ktimers/100:31:131
1592099259,3cyclictest281ktimers/100:31:131
1592099256,3cyclictest281ktimers/123:23:291
1592099256,3cyclictest281ktimers/123:23:291
1592099252,5cyclictest271rcuc/120:52:461
1592099252,5cyclictest271rcuc/120:52:461
15920992520,2cyclictest0-21swapper/100:38:301
15920992520,2cyclictest0-21swapper/100:38:301
1592099251,5cyclictest281ktimers/122:35:581
1592099251,5cyclictest281ktimers/122:35:581
15920992514,2cyclictest181rcu_preempt21:53:481
15912992518,4cyclictest0-21swapper/020:48:190
15912992518,4cyclictest0-21swapper/020:48:190
15912992518,4cyclictest0-21swapper/020:33:220
15912992518,4cyclictest0-21swapper/020:33:220
15912992517,6cyclictest31861-21kernelversion22:48:210
15912992517,6cyclictest31861-21kernelversion22:48:210
15912992516,7cyclictest23713-21basename22:28:080
15912992516,7cyclictest23713-21basename22:28:080
277832240,2sleep0171ktimers/022:38:200
15920992420,2cyclictest0-21swapper/122:23:271
15920992420,2cyclictest0-21swapper/122:23:271
15920992414,2cyclictest181rcu_preempt22:31:391
15920992414,2cyclictest181rcu_preempt22:31:391
15920992411,3cyclictest181rcu_preempt19:49:051
15912992420,2cyclictest0-21swapper/022:33:290
15912992420,2cyclictest0-21swapper/022:33:290
15912992419,4cyclictest24943-21latency_hist19:38:150
15912992419,4cyclictest24943-21latency_hist19:38:150
15912992418,4cyclictest0-21swapper/023:43:170
15912992416,6cyclictest14602-21ls23:28:240
15912992416,6cyclictest14602-21ls23:28:240
15912992412,10cyclictest0-21swapper/019:23:190
15912992412,10cyclictest0-21swapper/019:23:190
1591299241,20cyclictest14678-21systemd-journal19:29:470
1591299241,20cyclictest14678-21systemd-journal19:29:470
15912992410,12cyclictest32319-21gltestperf21:23:210
15912992410,12cyclictest32319-21gltestperf21:23:210
15912992410,12cyclictest32319-21gltestperf21:23:210
15927992317,3cyclictest172782sleep322:08:283
15927992317,3cyclictest172782sleep322:08:283
15927992317,3cyclictest172782sleep322:08:283
15912992317,4cyclictest749-21runrttasks23:38:290
15912992317,4cyclictest0-21swapper/023:53:290
15912992317,4cyclictest0-21swapper/023:53:290
15912992317,4cyclictest0-21swapper/023:53:290
15912992317,4cyclictest0-21swapper/020:13:190
15912992317,4cyclictest0-21swapper/020:13:190
15912992314,7cyclictest8258-21/usr/sbin/munin23:13:250
15912992314,7cyclictest8258-21/usr/sbin/munin23:13:250
15912992314,7cyclictest4184-21grep21:33:290
15912992314,7cyclictest4184-21grep21:33:290
15912992218,2cyclictest0-21swapper/020:53:240
15912992218,2cyclictest0-21swapper/020:53:240
15912992216,4cyclictest0-21swapper/021:13:300
15912992216,4cyclictest0-21swapper/021:13:300
15912992216,3cyclictest0-21swapper/000:23:210
32572210,2sleep20-21swapper/221:33:182
32572210,2sleep20-21swapper/221:33:182
15927992114,4cyclictest33592sleep321:33:193
15912992116,3cyclictest0-21swapper/021:08:200
15912992116,3cyclictest0-21swapper/021:08:200
15912992116,3cyclictest0-21swapper/021:08:200
15912992116,3cyclictest0-21swapper/000:23:070
15912992114,5cyclictest18885-21missed_timers22:13:230
15912992114,5cyclictest18885-21missed_timers22:13:230
15912992114,5cyclictest18885-21missed_timers22:13:230
15912992113,6cyclictest5106-21/usr/sbin/munin00:28:300
15912992113,6cyclictest5106-21/usr/sbin/munin00:28:300
15912992113,6cyclictest5106-21/usr/sbin/munin00:28:300
15912992113,6cyclictest1036-21cut22:53:220
15912992113,6cyclictest1036-21cut22:53:220
15912992113,6cyclictest1036-21cut22:53:220
15927992013,5cyclictest157612sleep320:38:233
38482190,3sleep30-21swapper/321:33:233
38482190,3sleep30-21swapper/321:33:233
1592799191,13cyclictest19799-21taskset23:43:193
1592799191,13cyclictest19799-21taskset23:43:193
1592799183,9cyclictest0-21swapper/322:44:473
1592799183,9cyclictest0-21swapper/322:44:473
1592799181,12cyclictest29412-21chrt19:48:233
1592799181,12cyclictest29412-21chrt19:48:233
1592799181,12cyclictest12388-21chrt21:57:473
1592799181,12cyclictest12388-21chrt21:57:473
1592799181,11cyclictest8645-21taskset23:13:203
1592799179,4cyclictest152262sleep322:03:263
1592799179,4cyclictest152262sleep322:03:263
1592799171,12cyclictest7782-21chrt00:38:193
1592799171,12cyclictest7782-21chrt00:38:193
1592799171,12cyclictest13720-21chrt23:27:483
1592799171,12cyclictest13720-21chrt23:27:483
1592799171,12cyclictest10939-21chrt23:18:273
1592799171,12cyclictest10939-21chrt23:18:273
1592799171,12cyclictest10939-21chrt23:18:273
1592799171,11cyclictest8198-21taskset20:18:223
1592799171,11cyclictest8198-21taskset20:18:223
1592799171,11cyclictest32655-21chrt19:58:203
1592799171,11cyclictest21892-21taskset23:48:223
1592799171,11cyclictest21892-21taskset23:48:223
1592799171,11cyclictest21892-21taskset23:48:223
1592799171,10cyclictest0-21swapper/319:48:193
1592699171,14cyclictest18263-21chrt20:43:332
1592699171,14cyclictest18263-21chrt20:43:332
1592699171,14cyclictest18263-21chrt20:43:332
1592699171,12cyclictest25206-21chrt21:03:272
1592699171,12cyclictest25206-21chrt21:03:272
1592799163,9cyclictest0-21swapper/319:44:473
1592799162,9cyclictest0-21swapper/321:58:223
1592799162,9cyclictest0-21swapper/321:58:223
1592799162,9cyclictest0-21swapper/321:58:223
1592799161,11cyclictest30191-21chrt00:08:363
1592799161,11cyclictest30191-21chrt00:08:363
1592799161,11cyclictest0-21swapper/323:28:303
1592799161,11cyclictest0-21swapper/323:28:303
15927991610,4cyclictest21962-21chrt19:28:253
15927991610,4cyclictest21962-21chrt19:28:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional