You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 13:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Jun 08, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2580121100,2chrt25802-21/usr/sbin/munin22:33:080
2580121100,2chrt25802-21/usr/sbin/munin22:33:080
914221060,9sleep0171ktimers/020:23:030
914221060,9sleep0171ktimers/020:23:030
2711821000,2sleep30-21swapper/300:03:163
2711821000,2sleep30-21swapper/300:03:163
187821000,2sleep01876-21ntp_states20:03:090
187821000,2sleep01876-21ntp_states20:03:090
107702970,12sleep0201rcuc/023:19:470
107702970,12sleep0201rcuc/023:19:470
107702970,12sleep0201rcuc/023:19:470
31412960,3sleep20-21swapper/200:27:592
31412960,3sleep20-21swapper/200:27:592
28362960,3sleep30-21swapper/320:07:503
28362960,3sleep30-21swapper/320:07:503
280282960,1sleep028030-21latency_hist00:07:550
280282960,1sleep028030-21latency_hist00:07:550
275552950,2sleep00-21swapper/019:46:250
275552950,2sleep00-21swapper/019:46:250
275552950,2sleep00-21swapper/019:46:250
219692950,2chrt21968-21ntpq22:23:070
219692950,2chrt21968-21ntpq22:23:070
219692950,2chrt21968-21ntpq22:23:070
54842940,2sleep15488-21missed_timers20:13:031
54842940,2sleep15488-21missed_timers20:13:031
143782930,3sleep30-21swapper/322:03:043
143782930,3sleep30-21swapper/322:03:043
326092920,3sleep30-21swapper/322:53:013
326092920,3sleep30-21swapper/322:53:013
326092920,3sleep30-21swapper/322:53:013
281272910,4sleep30-21swapper/321:13:073
281272910,4sleep30-21swapper/321:13:073
281272910,4sleep30-21swapper/321:13:073
10092840,1sleep30-21swapper/320:02:523
10092840,1sleep30-21swapper/320:02:523
259942830,2chrt0-21swapper/300:02:473
259942830,2chrt0-21swapper/300:02:473
23572780,3sleep3431rcuc/321:32:573
23572780,3sleep3431rcuc/321:32:573
210322760,2chrt0-21swapper/222:22:492
210322760,2chrt0-21swapper/222:22:492
210322760,2chrt0-21swapper/222:22:492
271302730,3sleep30-21swapper/322:38:013
271302730,3sleep30-21swapper/322:38:013
79432680,3sleep30-21swapper/323:13:033
79432680,3sleep30-21swapper/323:13:033
221482640,7sleep1281ktimers/120:58:011
221482640,7sleep1281ktimers/120:58:011
153762630,3sleep30-21swapper/323:33:013
124162610,3sleep30-21swapper/323:23:123
124162610,3sleep30-21swapper/323:23:123
321472600,1chrt0-21swapper/019:58:020
321472600,1chrt0-21swapper/019:58:020
198802570,3sleep00-21swapper/023:43:110
198802570,3sleep00-21swapper/023:43:110
198802570,3sleep00-21swapper/023:43:110
42832540,3sleep20-21swapper/200:28:112
42832540,3sleep20-21swapper/200:28:112
169402480,3sleep20-21swapper/219:18:002
113432480,4sleep30-21swapper/323:23:003
113432480,4sleep30-21swapper/323:23:003
295132470,3sleep30-21swapper/321:18:013
295132470,3sleep30-21swapper/321:18:013
136092470,3sleep30-21swapper/323:28:023
71992460,2sleep30-21swapper/300:38:013
224092450,3sleep022403-21gzip20:58:040
224092450,3sleep022403-21gzip20:58:040
194202450,3sleep20-21swapper/223:43:052
194202450,3sleep20-21swapper/223:43:052
194202450,3sleep20-21swapper/223:43:052
213612420,3sleep20-21swapper/219:28:092
213612420,3sleep20-21swapper/219:28:092
1528899425,26cyclictest26122-21date00:02:530
1528899425,26cyclictest26122-21date00:02:530
130922400,3sleep20-21swapper/220:33:062
130922400,3sleep20-21swapper/220:33:062
235272390,4sleep023531-21head19:33:100
235272390,4sleep023531-21head19:33:100
205462390,3sleep20-21swapper/220:53:042
205462390,3sleep20-21swapper/220:53:042
15288993932,5cyclictest15286-21cyclictest21:32:480
15288993932,5cyclictest15286-21cyclictest21:32:480
15290993830,3cyclictest0-21swapper/221:53:182
15290993830,3cyclictest0-21swapper/221:53:182
15288993829,7cyclictest28810-21netstat00:08:070
15288993829,7cyclictest28810-21netstat00:08:070
15288993829,7cyclictest28810-21netstat00:08:070
253762370,4chrt25265-21latency_hist21:07:530
253762370,4chrt25265-21latency_hist21:07:530
15289993724,4cyclictest27115-21grep00:03:161
15289993724,4cyclictest27115-21grep00:03:161
137002370,2sleep013722-21latency_hist22:02:580
68502360,4chrt0-21swapper/221:43:052
68502360,4chrt0-21swapper/221:43:052
232432350,2sleep30-21swapper/323:53:073
232432350,2sleep30-21swapper/323:53:073
232432350,2sleep30-21swapper/323:53:073
15288993529,4cyclictest1346-21avahi-daemon20:48:080
15288993529,4cyclictest1346-21avahi-daemon20:48:080
142072350,4sleep20-21swapper/223:28:092
142072350,4sleep20-21swapper/223:28:092
247842340,3sleep30-21swapper/321:03:103
247842340,3sleep30-21swapper/321:03:103
15289993428,4cyclictest25992-21cron00:02:471
15289993428,4cyclictest25992-21cron00:02:471
15289993425,7cyclictest20134-21fschecks_time20:53:001
15289993425,7cyclictest20134-21fschecks_time20:53:001
15288993425,7cyclictest25793-21latency_hist19:42:470
52852330,3sleep30-21swapper/320:13:013
52852330,3sleep30-21swapper/320:13:013
15288993329,3cyclictest15286-21cyclictest21:43:120
15288993329,3cyclictest15286-21cyclictest21:43:120
287752320,3sleep30-21swapper/300:08:063
287752320,3sleep30-21swapper/300:08:063
287752320,3sleep30-21swapper/300:08:063
15288993226,4cyclictest14678-21systemd-journal22:17:480
15288993226,4cyclictest14678-21systemd-journal22:17:480
15288993226,4cyclictest12537-21ntp_kernel_pll_21:58:070
15288993223,7cyclictest3350-21gltestperf20:08:010
15288993223,7cyclictest3350-21gltestperf20:08:010
15288993223,7cyclictest3350-21gltestperf20:08:010
74792310,3chrt0-21swapper/220:18:062
74792310,3chrt0-21swapper/220:18:062
1529099317,12cyclictest0-21swapper/200:03:162
1529099317,12cyclictest0-21swapper/200:03:162
15289993124,5cyclictest3434-21fschecks_time00:28:021
15289993124,5cyclictest3434-21fschecks_time00:28:021
15289993124,5cyclictest29927-21uniq19:52:591
15289993124,5cyclictest29927-21uniq19:52:591
15288993125,5cyclictest4927-21latency_hist00:32:550
15288993125,5cyclictest4927-21latency_hist00:32:550
15288993125,5cyclictest2929-21kernelversion21:33:020
15288993125,5cyclictest2929-21kernelversion21:33:020
15288993125,5cyclictest2929-21kernelversion21:33:020
15288993123,6cyclictest7790-21sendmail_mailst00:38:090
15288993123,2cyclictest181rcu_preempt23:32:570
15288993123,2cyclictest181rcu_preempt23:32:570
77102300,3sleep20-21swapper/200:38:092
191422300,3chrt0-21swapper/219:23:032
191422300,3chrt0-21swapper/219:23:032
15290993019,7cyclictest16958-21chrt22:09:182
15290993019,7cyclictest16958-21chrt22:09:182
1529099301,23cyclictest26153-21taskset00:02:552
1529099301,23cyclictest26153-21taskset00:02:552
15289993024,5cyclictest24365-21ntp_offset21:03:061
15289993024,5cyclictest24365-21ntp_offset21:03:061
15289993023,4cyclictest20684-21fschecks_time19:28:011
15289993023,4cyclictest20684-21fschecks_time19:28:011
15288993026,3cyclictest15286-21cyclictest20:42:290
15288993026,3cyclictest15286-21cyclictest20:42:290
15288993024,4cyclictest32187-21latency_hist22:52:470
15288993024,4cyclictest32187-21latency_hist22:52:470
15288993024,4cyclictest32187-21latency_hist22:52:470
15288993023,2cyclictest181rcu_preempt20:27:590
15288993023,2cyclictest181rcu_preempt20:27:590
15288993022,6cyclictest5888-21cut23:08:020
15288993022,6cyclictest5888-21cut23:08:020
15290992919,7cyclictest31871-21chrt22:48:492
15290992919,7cyclictest31871-21chrt22:48:492
15290992919,7cyclictest31871-21chrt22:48:492
15290992918,8cyclictest12745-21chrt23:25:522
15290992918,8cyclictest12745-21chrt23:25:522
15290992918,7cyclictest23685-21chrt19:34:452
15290992918,7cyclictest23685-21chrt19:34:452
15290992918,7cyclictest22119-21chrt23:50:562
15290992918,7cyclictest22119-21chrt23:50:562
15289992921,6cyclictest430-21latency_hist21:27:551
15289992921,6cyclictest430-21latency_hist21:27:551
15289992920,7cyclictest25027-21latency_hist22:32:591
15289992920,7cyclictest25027-21latency_hist22:32:591
15288992923,4cyclictest9591-21fschecks_time23:17:590
15288992923,4cyclictest9591-21fschecks_time23:17:590
15288992923,4cyclictest15286-21cyclictest21:41:140
15288992923,4cyclictest15286-21cyclictest21:41:140
15288992922,4cyclictest0-21swapper/019:28:110
15288992922,4cyclictest0-21swapper/019:28:110
15290992818,7cyclictest32569-21chrt21:25:162
15290992818,7cyclictest32569-21chrt21:25:162
15290992818,7cyclictest22505-21chrt22:23:212
15290992818,7cyclictest22505-21chrt22:23:212
15290992818,7cyclictest22505-21chrt22:23:212
15290992818,7cyclictest13176-21chrt21:59:172
15290992818,7cyclictest13176-21chrt21:59:172
15290992818,6cyclictest23136-21chrt20:59:392
15290992818,6cyclictest23136-21chrt20:59:392
15290992818,6cyclictest1280-21chrt22:53:322
15290992818,6cyclictest1280-21chrt22:53:322
15290992818,6cyclictest1280-21chrt22:53:322
15290992817,7cyclictest4361-21chrt20:09:262
15290992817,7cyclictest4361-21chrt20:09:262
15290992817,7cyclictest4361-21chrt20:09:262
15290992817,7cyclictest15210-21chrt22:05:482
15290992817,7cyclictest15210-21chrt22:05:482
15290992817,6cyclictest5901-21chrt21:42:082
15290992817,6cyclictest5901-21chrt21:42:082
15289992821,5cyclictest8799-21ntp_offset21:48:061
15289992821,5cyclictest8799-21ntp_offset21:48:061
15289992821,5cyclictest4824-21latency_hist20:12:571
15289992821,5cyclictest4824-21latency_hist20:12:571
15289992821,5cyclictest12026-21fschecks_time21:58:011
15289992821,5cyclictest12026-21fschecks_time21:58:011
15289992820,6cyclictest32675-21threads19:58:101
15289992820,6cyclictest32675-21threads19:58:101
15289992820,6cyclictest28870-21fschecks_count22:43:011
15289992820,6cyclictest28870-21fschecks_count22:43:011
15289992820,6cyclictest28870-21fschecks_count22:43:011
15289992819,6cyclictest21945-21tr22:23:061
15289992819,6cyclictest21945-21tr22:23:061
15289992819,6cyclictest21945-21tr22:23:061
15289992819,6cyclictest14414-21fschecks_count20:38:011
15289992819,6cyclictest14414-21fschecks_count20:38:011
1528899283,6cyclictest171ktimers/020:36:460
1528899283,6cyclictest171ktimers/020:36:460
15288992823,3cyclictest29590-21latency_hist19:52:470
15288992823,3cyclictest29590-21latency_hist19:52:470
15288992823,3cyclictest29590-21latency_hist19:52:470
15288992822,5cyclictest30297-21kernelversion19:53:020
15288992821,5cyclictest15286-21cyclictest22:06:260
15288992821,5cyclictest15286-21cyclictest22:06:260
15288992820,6cyclictest30981-21threads00:13:100
15288992820,6cyclictest30981-21threads00:13:100
15288992820,6cyclictest30981-21threads00:13:100
15288992817,2cyclictest181rcu_preempt22:45:490
15288992817,2cyclictest181rcu_preempt22:45:490
15288992817,2cyclictest181rcu_preempt22:45:490
15288992813,3cyclictest181rcu_preempt20:22:370
15288992813,3cyclictest181rcu_preempt20:22:370
113252280,1sleep011326-21sshd21:54:250
113252280,1sleep011326-21sshd21:54:250
15290992718,6cyclictest679-21chrt20:00:302
15290992718,6cyclictest679-21chrt20:00:302
15290992718,6cyclictest3841-21chrt21:34:532
15290992718,6cyclictest3841-21chrt21:34:532
15290992718,6cyclictest3841-21chrt21:34:532
15290992718,6cyclictest2692-21chrt20:06:572
15290992718,6cyclictest2692-21chrt20:06:572
15290992717,7cyclictest6977-21chrt23:09:162
15290992717,7cyclictest6977-21chrt23:09:162
15290992717,7cyclictest31262-21chrt00:13:242
15290992717,7cyclictest31262-21chrt00:13:242
15290992717,7cyclictest31262-21chrt00:13:242
15290992717,7cyclictest30666-21chrt21:20:012
15290992717,7cyclictest30666-21chrt21:20:012
15290992717,6cyclictest31198-21chrt19:54:432
15290992717,6cyclictest29304-21chrt19:49:562
15290992717,6cyclictest29304-21chrt19:49:562
15290992717,6cyclictest29304-21chrt19:49:562
15290992717,6cyclictest28317-21chrt22:41:182
15290992717,6cyclictest28317-21chrt22:41:182
15290992717,6cyclictest27537-21chrt19:46:102
15290992717,6cyclictest27537-21chrt19:46:102
15290992717,6cyclictest27537-21chrt19:46:102
15290992717,5cyclictest25672-21chrt22:33:062
15290992717,5cyclictest25672-21chrt22:33:062
15290992716,7cyclictest6591-21chrt00:36:352
15290992716,7cyclictest6591-21chrt00:36:352
15290992716,7cyclictest10254-21chrt20:27:232
15290992716,7cyclictest10254-21chrt20:27:232
15290992716,6cyclictest6279-21chrt20:14:492
15290992716,6cyclictest6279-21chrt20:14:492
15289992721,5cyclictest13413-21gltestperf23:28:011
15289992721,5cyclictest13413-21gltestperf23:28:011
15289992721,4cyclictest21292-21needreboot23:48:041
15289992721,4cyclictest21292-21needreboot23:48:041
15289992721,3cyclictest0-21swapper/119:42:471
15289992720,5cyclictest15286-21cyclictest22:13:001
15289992719,6cyclictest26717-21latency_hist22:37:531
15289992719,6cyclictest26717-21latency_hist22:37:531
15289992719,6cyclictest20741-21fschecks_count23:47:591
15289992719,6cyclictest20741-21fschecks_count23:47:591
15289992719,6cyclictest20741-21fschecks_count23:47:591
15288992722,4cyclictest572-21cups-browsed22:58:100
15288992722,4cyclictest572-21cups-browsed22:58:100
15288992722,4cyclictest572-21cups-browsed22:58:100
15288992721,3cyclictest0-21swapper/022:22:470
15288992721,3cyclictest0-21swapper/022:22:470
15288992721,3cyclictest0-21swapper/022:22:470
15288992718,6cyclictest7641-21cut23:13:010
15288992713,2cyclictest181rcu_preempt00:36:380
15288992713,2cyclictest181rcu_preempt00:36:380
15288992711,3cyclictest181rcu_preempt20:44:470
15288992711,3cyclictest181rcu_preempt20:44:470
32662260,4sleep30-21swapper/300:28:003
15290992620,3cyclictest0-21swapper/223:39:562
15290992620,3cyclictest0-21swapper/223:39:562
15290992620,3cyclictest0-21swapper/223:39:562
15290992619,4cyclictest19029-21chrt22:16:582
15290992619,4cyclictest19029-21chrt22:16:582
15290992617,6cyclictest5087-21chrt23:04:132
15290992617,6cyclictest5087-21chrt23:04:132
15290992617,6cyclictest5087-21chrt23:04:132
15290992617,6cyclictest30053-21chrt22:44:232
15290992617,6cyclictest30053-21chrt22:44:232
15290992617,6cyclictest30053-21chrt22:44:232
15290992617,6cyclictest28759-21chrt00:08:052
15290992617,6cyclictest28759-21chrt00:08:052
15290992617,6cyclictest28759-21chrt00:08:052
15290992617,5cyclictest28740-21chrt21:14:182
15290992617,5cyclictest28740-21chrt21:14:182
15290992617,5cyclictest28740-21chrt21:14:182
15290992616,7cyclictest24958-21chrt21:03:532
15290992616,7cyclictest24958-21chrt21:03:532
15290992616,6cyclictest727-21chrt00:18:552
15290992616,6cyclictest727-21chrt00:18:552
15290992616,6cyclictest727-21chrt00:18:552
15290992616,6cyclictest3418-21chrt23:02:002
15290992616,6cyclictest3418-21chrt23:02:002
15290992616,6cyclictest3418-21chrt23:02:002
15290992616,6cyclictest26970-21chrt21:10:492
15290992616,6cyclictest26970-21chrt21:10:492
15290992616,6cyclictest26970-21chrt21:10:492
15290992616,6cyclictest24232-21chrt22:28:112
15290992616,6cyclictest24232-21chrt22:28:112
15289992621,3cyclictest0-21swapper/119:47:581
15289992621,3cyclictest0-21swapper/119:47:581
15289992620,5cyclictest14577-21processes22:03:081
15289992620,5cyclictest14577-21processes22:03:081
15289992620,5cyclictest13611-21kernelversion23:28:021
15289992620,5cyclictest13611-21kernelversion23:28:021
15289992620,4cyclictest15286-21cyclictest20:30:571
15289992620,4cyclictest15286-21cyclictest20:30:571
15289992619,5cyclictest28225-21gltestperf19:48:001
15289992618,6cyclictest6692-21sh00:37:491
15289992618,6cyclictest6692-21sh00:37:491
15289992618,6cyclictest4228-21latency_hist21:37:561
15289992618,6cyclictest4228-21latency_hist21:37:561
15289992618,6cyclictest4228-21latency_hist21:37:561
15288992621,3cyclictest0-21swapper/000:27:480
15288992621,3cyclictest0-21swapper/000:27:480
15288992620,4cyclictest0-21swapper/021:22:470
15288992620,4cyclictest0-21swapper/021:22:470
15288992619,5cyclictest6244-21sshd20:14:240
15288992619,5cyclictest6244-21sshd20:14:240
78782250,3sleep30-21swapper/321:47:483
78782250,3sleep30-21swapper/321:47:483
15291992519,4cyclictest74342sleep323:13:003
15291992519,4cyclictest74342sleep323:13:003
15290992520,3cyclictest8923-21chrt23:15:192
15290992520,3cyclictest8923-21chrt23:15:192
15290992518,5cyclictest15635-21chrt20:39:172
15290992518,5cyclictest15635-21chrt20:39:172
15290992518,5cyclictest10756-21chrt23:19:372
15290992518,5cyclictest10756-21chrt23:19:372
15290992518,5cyclictest10756-21chrt23:19:372
15290992518,4cyclictest23917-21chrt23:55:072
15290992518,4cyclictest23917-21chrt23:55:072
15290992518,4cyclictest23917-21chrt23:55:072
15290992517,6cyclictest19414-21chrt20:49:422
15290992517,6cyclictest19414-21chrt20:49:422
15290992516,6cyclictest16330-21chrt23:33:362
15290992516,6cyclictest16330-21chrt23:33:362
15290992515,6cyclictest1984-21taskset21:29:582
15290992515,6cyclictest1984-21taskset21:29:582
15289992520,3cyclictest0-21swapper/122:38:011
15289992520,3cyclictest0-21swapper/122:38:011
15289992520,3cyclictest0-21swapper/120:43:071
15289992520,3cyclictest0-21swapper/120:43:071
15289992519,4cyclictest547-21rs:main1
15289992519,4cyclictest1597-21gltestperf00:23:011
15289992519,4cyclictest1597-21gltestperf00:23:011
15289992518,5cyclictest7783-21timerandwakeup20:18:091
15289992518,5cyclictest7783-21timerandwakeup20:18:091
15289992517,6cyclictest749-21runrttasks22:19:471
15289992517,6cyclictest749-21runrttasks22:19:471
15289992517,6cyclictest749-21runrttasks22:19:471
15289992516,7cyclictest8288-21processes23:13:081
15289992516,7cyclictest8288-21processes23:13:081
1528899253,4cyclictest181rcu_preempt22:10:430
1528899253,4cyclictest181rcu_preempt22:10:430
15288992520,4cyclictest23568-21kernelversion22:28:030
15288992520,4cyclictest23568-21kernelversion22:28:030
15288992520,3cyclictest0-21swapper/023:08:080
15288992519,5cyclictest15643-21missed_timers23:33:030
15288992519,5cyclictest15643-21missed_timers23:33:030
15288992519,4cyclictest601-21snmpd22:38:570
15288992519,4cyclictest601-21snmpd22:38:570
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional