You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 02:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Apr 12, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226262790,3sleep00-21swapper/021:37:180
226262790,3sleep00-21swapper/021:37:180
226262790,3sleep00-21swapper/021:37:180
113462740,3chrt0-21swapper/321:07:193
113462740,3chrt0-21swapper/321:07:193
188342700,2sleep00-21swapper/000:22:080
188342700,2sleep00-21swapper/000:22:080
188342700,2sleep00-21swapper/000:22:080
182512670,3chrt0-21swapper/222:52:182
182512670,3chrt0-21swapper/222:52:182
305372620,1chrt0-21swapper/023:27:090
52052590,2sleep20-21swapper/223:47:062
52052590,2sleep20-21swapper/223:47:062
60932580,3sleep30-21swapper/322:22:083
60932580,3sleep30-21swapper/322:22:083
32116995751,3cyclictest0-21swapper/000:02:000
32116995751,3cyclictest0-21swapper/000:02:000
32116995751,3cyclictest0-21swapper/000:02:000
179252540,2sleep00-21swapper/022:52:160
179252540,2sleep00-21swapper/022:52:160
32116995326,25cyclictest16239-21latency_hist00:16:550
89142520,1sleep3431rcuc/321:02:123
89142520,1sleep3431rcuc/321:02:123
283152520,1sleep00-21swapper/020:27:160
170472520,3sleep20-21swapper/219:57:152
170472520,3sleep20-21swapper/219:57:152
170472520,3sleep20-21swapper/219:57:152
46242500,4sleep30-21swapper/320:52:073
137422500,6sleep0171ktimers/000:07:170
137422500,6sleep0171ktimers/000:07:170
57302470,2sleep20-21swapper/222:21:592
57302470,2sleep20-21swapper/222:21:592
192022440,2chrt0-21swapper/222:57:082
192022440,2chrt0-21swapper/222:57:082
121652440,2sleep30-21swapper/300:02:343
121652440,2sleep30-21swapper/300:02:343
305252430,2chrt30528-21timerwakeupswit20:32:191
305252430,2chrt30528-21timerwakeupswit20:32:191
170762430,3chrt0-21swapper/319:57:163
170762430,3chrt0-21swapper/319:57:163
170762430,3chrt0-21swapper/319:57:163
32120994226,3cyclictest181rcu_preempt00:30:211
32120994226,3cyclictest181rcu_preempt00:30:211
3211699423,33cyclictest11587-21ntpq00:02:150
3211699423,33cyclictest11587-21ntpq00:02:150
222842420,3sleep20-21swapper/221:37:162
222842420,3sleep20-21swapper/221:37:162
222842420,3sleep20-21swapper/221:37:162
32116994115,2cyclictest0-21swapper/021:02:090
32116994115,2cyclictest0-21swapper/021:02:090
32116994016,3cyclictest142-21systemd-journal20:06:550
32116994016,3cyclictest142-21systemd-journal20:06:550
32116994016,3cyclictest142-21systemd-journal20:06:550
263442400,4sleep30-21swapper/323:17:033
263442400,4sleep30-21swapper/323:17:033
50092390,3sleep20-21swapper/220:52:102
3213599392,26cyclictest10766-21chrt00:02:013
3213599392,26cyclictest10766-21chrt00:02:013
3213599392,26cyclictest10766-21chrt00:02:013
32116993917,2cyclictest0-21swapper/020:22:090
32116993917,2cyclictest0-21swapper/020:22:090
32116993917,2cyclictest0-21swapper/020:22:090
147202390,2chrt14705-21processes21:17:170
147202390,2chrt14705-21processes21:17:170
147202390,2chrt14705-21processes21:17:170
32116993812,4cyclictest0-21swapper/019:32:160
3212899371,26cyclictest10742-21chrt00:02:002
3212899371,26cyclictest10742-21chrt00:02:002
3212899371,26cyclictest10742-21chrt00:02:002
32116993718,17cyclictest0-21swapper/021:22:170
32116993718,17cyclictest0-21swapper/021:22:170
32116993718,17cyclictest0-21swapper/021:22:170
32116993714,3cyclictest0-21swapper/021:32:170
32116993714,3cyclictest0-21swapper/021:32:170
32116993714,3cyclictest0-21swapper/021:32:170
214262370,2sleep30-21swapper/323:02:113
214262370,2sleep30-21swapper/323:02:113
214262370,2sleep30-21swapper/323:02:113
3212899360,24cyclictest12174-21chrt00:02:412
3212899360,24cyclictest12174-21chrt00:02:412
317632360,3sleep20-21swapper/220:37:112
317632360,3sleep20-21swapper/220:37:112
196642360,2sleep00-21swapper/021:32:080
196642360,2sleep00-21swapper/021:32:080
32116993527,6cyclictest21749-21switchtime23:02:170
32116993527,6cyclictest21749-21switchtime23:02:170
32116993527,6cyclictest21749-21switchtime23:02:170
276032350,3chrt0-21swapper/223:17:182
276032350,3chrt0-21swapper/223:17:182
127032350,3sleep30-21swapper/321:12:133
127032350,3sleep30-21swapper/321:12:133
32116993414,2cyclictest0-21swapper/000:37:080
32116993414,2cyclictest0-21swapper/000:37:080
32120993327,4cyclictest13565-21fschecks_time22:42:081
32120993327,4cyclictest13565-21fschecks_time22:42:081
32120993327,4cyclictest13565-21fschecks_time22:42:081
32120993325,3cyclictest181rcu_preempt20:13:171
32120993324,7cyclictest9847-21threads19:37:151
32120993324,7cyclictest9847-21threads19:37:151
32116993328,3cyclictest12376-21cut21:12:090
32116993328,3cyclictest12376-21cut21:12:090
32116993325,6cyclictest2071-21tr19:17:120
32135993225,4cyclictest187852chrt20:02:103
32135993225,4cyclictest187852chrt20:02:103
32135993225,4cyclictest187852chrt20:02:103
32120993226,4cyclictest17083-21missed_timers00:17:111
32120993226,4cyclictest17083-21missed_timers00:17:111
32116993223,7cyclictest12400-21latency_hist19:46:560
32116993223,7cyclictest12400-21latency_hist19:46:560
32116993222,8cyclictest25409-21gltestperf21:47:100
32116993222,8cyclictest25409-21gltestperf21:47:100
3212099317,6cyclictest281ktimers/123:54:201
3212099317,6cyclictest281ktimers/123:54:201
32120993123,6cyclictest23016-21latency_hist21:41:551
32120993123,6cyclictest23016-21latency_hist21:41:551
32120993123,6cyclictest23016-21latency_hist21:41:551
32120993123,5cyclictest11532-21tail00:02:131
32120993123,5cyclictest11532-21tail00:02:131
32116993121,8cyclictest20757-21netstat20:07:130
32116993121,8cyclictest20757-21netstat20:07:130
3212099309,18cyclictest281ktimers/100:11:211
3212099309,18cyclictest281ktimers/100:11:211
32120993023,5cyclictest6661-21processes22:22:151
32120993023,5cyclictest6661-21processes22:22:151
32120993023,5cyclictest0-21swapper/123:32:151
32120993023,5cyclictest0-21swapper/123:32:151
32116993024,4cyclictest1816-21gltestperf23:37:080
32116993024,4cyclictest1816-21gltestperf23:37:080
32116993023,5cyclictest32107-21cyclictest23:35:180
32116993023,5cyclictest32107-21cyclictest23:35:180
32116993023,5cyclictest32107-21cyclictest23:00:370
32116993023,5cyclictest32107-21cyclictest23:00:370
32116993023,5cyclictest32107-21cyclictest22:34:420
32116993023,5cyclictest32107-21cyclictest22:34:420
32116993023,5cyclictest22013-21latency_hist00:32:020
32116993023,5cyclictest22013-21latency_hist00:32:020
32116993022,6cyclictest14528-21unixbench_multi22:42:170
32116993022,6cyclictest14528-21unixbench_multi22:42:170
32116993022,6cyclictest14528-21unixbench_multi22:42:170
32116993021,7cyclictest7392-21cut23:52:080
32116993021,7cyclictest7392-21cut23:52:080
32116993015,12cyclictest142-21systemd-journal20:51:550
32116993015,12cyclictest142-21systemd-journal20:51:550
32120992925,3cyclictest90-1pr/ttyS021:45:471
32120992925,3cyclictest90-1pr/ttyS021:45:471
32120992923,5cyclictest28042-21sendmail23:21:551
32120992923,5cyclictest28042-21sendmail23:21:551
32120992922,5cyclictest4191-21fschecks_time22:17:081
32120992922,5cyclictest4191-21fschecks_time22:17:081
32120992922,5cyclictest25582-21wc00:41:551
32120992922,5cyclictest25582-21wc00:41:551
32120992920,7cyclictest3951-21netstat19:22:111
32120992920,7cyclictest3951-21netstat19:22:111
32120992920,7cyclictest3951-21netstat19:22:111
32120992915,2cyclictest181rcu_preempt20:17:071
32120992915,2cyclictest181rcu_preempt20:17:071
32120992915,2cyclictest181rcu_preempt00:22:091
32120992915,2cyclictest181rcu_preempt00:22:091
32120992915,2cyclictest181rcu_preempt00:22:091
32120992913,8cyclictest90-1pr/ttyS022:17:001
32120992913,3cyclictest181rcu_preempt19:42:091
32120992913,3cyclictest181rcu_preempt19:42:091
32116992922,5cyclictest5634-21sendmail22:21:540
32116992922,5cyclictest5634-21sendmail22:21:540
32116992920,7cyclictest32107-21cyclictest23:42:110
32116992920,7cyclictest32107-21cyclictest23:42:110
168912290,2sleep20-21swapper/222:51:572
168912290,2sleep20-21swapper/222:51:572
3212099285,5cyclictest281ktimers/121:10:071
3212099285,5cyclictest281ktimers/121:10:071
3212099284,5cyclictest271rcuc/123:44:361
3212099284,5cyclictest271rcuc/123:44:361
32120992824,3cyclictest29583-21ntp_offset21:57:141
32120992824,3cyclictest29583-21ntp_offset21:57:141
32120992821,5cyclictest32107-21cyclictest20:27:131
32120992820,6cyclictest21065-21threads20:07:171
32120992820,6cyclictest21065-21threads20:07:171
32120992817,2cyclictest181rcu_preempt21:16:581
32120992817,2cyclictest181rcu_preempt21:16:581
32120992816,2cyclictest181rcu_preempt23:27:201
32120992814,11cyclictest142-21systemd-journal21:31:551
32120992814,11cyclictest142-21systemd-journal21:31:551
32120992813,3cyclictest181rcu_preempt22:52:081
32120992813,3cyclictest181rcu_preempt22:52:081
32120992813,3cyclictest181rcu_preempt22:37:071
32120992813,3cyclictest181rcu_preempt22:37:071
32116992822,4cyclictest13236-21needreboot19:47:130
32116992822,3cyclictest0-21swapper/022:37:190
32116992822,3cyclictest0-21swapper/022:37:190
32116992819,7cyclictest9326-21kernelversion19:37:090
32116992819,7cyclictest9326-21kernelversion19:37:090
32116992819,7cyclictest23605-21latency_hist20:16:570
165502280,3sleep00-21swapper/019:57:090
165502280,3sleep00-21swapper/019:57:090
165502280,3sleep00-21swapper/019:57:090
3212099278,4cyclictest281ktimers/122:06:081
3212099278,4cyclictest281ktimers/122:06:081
3212099277,5cyclictest281ktimers/100:01:511
3212099277,5cyclictest281ktimers/100:01:511
3212099277,5cyclictest281ktimers/100:01:511
3212099276,6cyclictest281ktimers/120:43:151
3212099276,6cyclictest281ktimers/120:43:151
3212099274,4cyclictest281ktimers/119:58:141
3212099274,4cyclictest281ktimers/119:58:141
3212099274,4cyclictest281ktimers/119:58:141
32120992721,4cyclictest281ktimers/120:52:461
32120992719,5cyclictest281ktimers/100:34:461
32120992719,5cyclictest281ktimers/100:34:461
32120992715,3cyclictest90-1pr/ttyS023:02:031
32120992715,3cyclictest90-1pr/ttyS023:02:031
32120992714,7cyclictest90-1pr/ttyS023:47:111
32120992714,7cyclictest90-1pr/ttyS023:47:111
32120992714,3cyclictest90-1pr/ttyS023:09:341
32120992714,3cyclictest90-1pr/ttyS023:09:341
32116992721,4cyclictest6507-21needreboot22:22:120
32116992721,4cyclictest6507-21needreboot22:22:120
32116992721,4cyclictest28637-21latency_hist21:56:550
32116992721,4cyclictest28637-21latency_hist21:56:550
32116992721,4cyclictest24505-21ntp_kernel_pll_20:17:140
32116992721,4cyclictest24505-21ntp_kernel_pll_20:17:140
3212099268,4cyclictest271rcuc/123:22:111
3212099268,4cyclictest271rcuc/123:22:111
3212099267,8cyclictest15675-21irqstats22:47:111
3212099267,8cyclictest15675-21irqstats22:47:111
3212099263,5cyclictest271rcuc/120:24:051
3212099263,5cyclictest271rcuc/120:24:051
3212099263,5cyclictest271rcuc/120:24:051
32120992621,4cyclictest1346-21avahi-daemon19:27:161
32120992621,4cyclictest1346-21avahi-daemon19:27:161
32120992621,3cyclictest0-21swapper/122:27:101
32120992621,3cyclictest0-21swapper/122:27:101
32120992620,5cyclictest20998-21sshd21:34:561
32120992620,5cyclictest20998-21sshd21:34:561
32120992620,5cyclictest20998-21sshd21:34:561
32120992620,5cyclictest14139-21sshd19:50:551
32120992620,4cyclictest271rcuc/122:34:561
32120992620,4cyclictest271rcuc/122:34:561
32120992615,3cyclictest90-1pr/ttyS022:07:121
32120992615,3cyclictest90-1pr/ttyS022:07:121
32120992614,2cyclictest181rcu_preempt23:02:101
32120992614,2cyclictest181rcu_preempt23:02:101
32120992614,2cyclictest181rcu_preempt23:02:101
32120992612,2cyclictest181rcu_preempt23:37:051
32120992612,2cyclictest181rcu_preempt23:37:051
32120992612,2cyclictest181rcu_preempt19:56:481
32120992610,3cyclictest181rcu_preempt21:01:561
32120992610,3cyclictest181rcu_preempt21:01:561
32116992622,3cyclictest700-21192.168.115.40-manager21:57:170
32116992622,3cyclictest700-21192.168.115.40-manager21:57:170
32116992622,3cyclictest15183-21ntp_offset19:52:140
32116992621,4cyclictest32107-21cyclictest23:22:090
32116992621,4cyclictest32107-21cyclictest23:22:090
32116992620,4cyclictest0-21swapper/023:12:130
32116992620,4cyclictest0-21swapper/023:12:130
3211699261,4cyclictest0-21swapper/022:08:180
3211699261,4cyclictest0-21swapper/022:08:180
3212099253,5cyclictest281ktimers/119:34:141
32120992520,4cyclictest1648-21ntpd21:02:091
32120992520,4cyclictest1648-21ntpd21:02:091
32120992519,4cyclictest18362-21fschecks_count20:02:071
32120992519,4cyclictest18362-21fschecks_count20:02:071
32120992519,4cyclictest18362-21fschecks_count20:02:071
32120992517,6cyclictest498-21cat20:41:561
32120992517,6cyclictest498-21cat20:41:561
32120992517,6cyclictest25262-21fschecks_count21:47:091
32120992517,6cyclictest25262-21fschecks_count21:47:091
32120992517,2cyclictest181rcu_preempt00:12:071
32120992516,7cyclictest4411-21latency_hist20:52:001
32120992516,7cyclictest4411-21latency_hist20:52:001
32120992512,3cyclictest90-1pr/ttyS021:55:471
32120992512,3cyclictest90-1pr/ttyS021:55:471
32116992520,3cyclictest0-21swapper/021:42:170
32116992520,3cyclictest0-21swapper/021:42:170
32116992520,2cyclictest0-21swapper/022:31:550
32116992520,2cyclictest0-21swapper/022:31:550
32116992519,5cyclictest3514-21fschecks_time19:22:060
32116992519,5cyclictest3514-21fschecks_time19:22:060
32116992519,5cyclictest3514-21fschecks_time19:22:060
32116992519,3cyclictest0-21swapper/020:41:550
32116992519,3cyclictest0-21swapper/020:41:550
32116992518,5cyclictest5910-21sshd20:53:560
32116992518,5cyclictest32107-21cyclictest23:10:010
32116992518,5cyclictest32107-21cyclictest23:10:010
32120992419,3cyclictest0-21swapper/119:17:111
32120992410,2cyclictest181rcu_preempt23:12:101
32120992410,2cyclictest181rcu_preempt23:12:101
32116992420,2cyclictest0-21swapper/022:12:090
32116992419,3cyclictest0-21swapper/020:57:160
32116992419,3cyclictest0-21swapper/020:57:160
32116992418,4cyclictest0-21swapper/020:32:120
32116992418,4cyclictest0-21swapper/020:32:120
32135992315,5cyclictest140692sleep322:42:143
32135992315,5cyclictest140692sleep322:42:143
32135992315,5cyclictest140692sleep322:42:143
3212099239,2cyclictest181rcu_preempt21:22:081
3212099239,2cyclictest181rcu_preempt21:22:081
3212099239,2cyclictest181rcu_preempt21:22:081
3212099235,4cyclictest281ktimers/121:17:171
3212099235,4cyclictest281ktimers/121:17:171
3212099235,4cyclictest281ktimers/121:17:171
32116992318,3cyclictest0-21swapper/021:07:180
32116992318,3cyclictest0-21swapper/021:07:180
32116992317,5cyclictest15680-21kernelversion22:47:110
32116992317,5cyclictest15680-21kernelversion22:47:110
32116992317,5cyclictest1098-21gltestperf20:42:080
32116992317,5cyclictest1098-21gltestperf20:42:080
32116992316,4cyclictest0-21swapper/022:02:150
32116992316,4cyclictest0-21swapper/022:02:150
32116992316,4cyclictest0-21swapper/000:32:180
32116992316,4cyclictest0-21swapper/000:32:180
219242230,3chrt0-21swapper/220:12:062
62542220,1sleep06257-21sed19:27:180
62542220,1sleep06257-21sed19:27:180
32116992217,3cyclictest0-21swapper/023:47:130
32116992217,3cyclictest0-21swapper/023:47:130
32116992216,4cyclictest26713-21gltestperf23:17:090
32116992216,4cyclictest26713-21gltestperf23:17:090
32116992116,3cyclictest0-21swapper/000:17:150
32116992116,3cyclictest0-21swapper/000:17:150
218202210,3sleep30-21swapper/321:37:093
218202210,3sleep30-21swapper/321:37:093
218202210,3sleep30-21swapper/321:37:093
32128992014,3cyclictest136972chrt22:42:092
32128992014,3cyclictest136972chrt22:42:092
32128992014,3cyclictest136972chrt22:42:092
32128992013,4cyclictest145162sleep200:12:012
32128992013,4cyclictest145162sleep200:12:012
3212899201,12cyclictest7894-21taskset23:52:152
3212899201,12cyclictest7894-21taskset23:52:152
231822200,3sleep30-21swapper/320:12:193
71022190,3sleep20-21swapper/223:52:032
3212899191,11cyclictest30662-21chrt20:32:202
3212899191,11cyclictest30662-21chrt20:32:202
3213599183,8cyclictest0-21swapper/321:27:563
3213599183,8cyclictest0-21swapper/321:27:563
3213599182,11cyclictest0-21swapper/323:47:073
3213599182,11cyclictest0-21swapper/323:47:073
3213599181,10cyclictest0-21swapper/320:27:303
3212899189,6cyclictest20754-21taskset21:32:182
3212899189,6cyclictest20754-21taskset21:32:182
3212899189,6cyclictest20754-21taskset21:32:182
3212899183,9cyclictest0-21swapper/220:07:562
3212899183,9cyclictest0-21swapper/220:07:562
3212899183,9cyclictest0-21swapper/200:38:582
3212899183,9cyclictest0-21swapper/200:38:582
32128991813,3cyclictest173892chrt00:17:162
32128991813,3cyclictest173892chrt00:17:162
3213599172,11cyclictest0-21swapper/322:32:193
3213599172,11cyclictest0-21swapper/322:32:193
3213599172,11cyclictest0-21swapper/322:02:163
3213599172,11cyclictest0-21swapper/322:02:163
3213599171,12cyclictest8995-21taskset19:37:063
3213599171,12cyclictest8995-21taskset19:37:063
3213599171,12cyclictest20932-21chrt00:27:143
3213599171,12cyclictest20932-21chrt00:27:143
3213599171,11cyclictest27944-21chrt21:52:173
3213599171,11cyclictest27944-21chrt21:52:173
3213599171,11cyclictest16500-21taskset21:22:143
3213599171,11cyclictest16500-21taskset21:22:143
3213599171,11cyclictest16500-21taskset21:22:143
3213599171,11cyclictest1527-21taskset20:42:143
3213599171,11cyclictest1527-21taskset20:42:143
3213599171,11cyclictest0-21swapper/323:52:093
3213599171,11cyclictest0-21swapper/323:52:093
3213599171,10cyclictest1971-21taskset23:37:093
3213599171,10cyclictest1971-21taskset23:37:093
3212899171,12cyclictest1491-21chrt23:37:042
3212899171,10cyclictest20443-21chrt00:27:092
3212899171,10cyclictest20443-21chrt00:27:092
76402160,3sleep20-21swapper/219:32:102
3213599163,9cyclictest0-21swapper/321:18:563
3213599163,9cyclictest0-21swapper/321:18:563
3213599163,9cyclictest0-21swapper/321:18:563
3213599162,10cyclictest0-21swapper/322:12:073
3213599162,10cyclictest0-21swapper/321:32:113
3213599162,10cyclictest0-21swapper/321:32:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional