You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 05:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sat Aug 15, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72792980,2sleep10-21swapper/120:24:011
239382760,1sleep124044-21/usr/sbin/munin21:09:011
193822700,2sleep00-21swapper/023:49:040
176092650,1chrt17610-21sed22:18:511
176092650,1chrt17610-21sed22:18:511
136482570,7sleep113650-21switchtime20:39:131
136482570,7sleep113650-21switchtime20:39:131
25222540,2chrt2523-21/usr/sbin/munin00:29:110
150712530,1sleep115182-21sed20:44:041
10202530,2sleep11122-21gzip22:59:081
10202530,2sleep11122-21gzip22:59:081
10202530,2sleep11122-21gzip22:59:081
235602460,2sleep30-21swapper/319:39:093
235602460,2sleep30-21swapper/319:39:093
279762450,3sleep30-21swapper/322:44:103
321932430,3sleep20-21swapper/221:29:082
12522430,2sleep10-21swapper/120:08:501
135582420,2sleep013645-21switchtime20:39:130
135582420,2sleep013645-21switchtime20:39:130
6362410,3sleep30-21swapper/322:59:013
6362410,3sleep30-21swapper/322:59:013
6362410,3sleep30-21swapper/322:59:013
239942410,2sleep00-21swapper/022:34:050
239942410,2sleep00-21swapper/022:34:050
117862400,3sleep30-21swapper/320:34:083
304722380,3sleep30-21swapper/321:24:093
304722380,3sleep30-21swapper/321:24:093
252772370,3sleep30-21swapper/319:44:073
133642370,3sleep30-21swapper/322:04:133
212012360,1sleep121360-21unixbench_multi20:59:121
212012360,1sleep121360-21unixbench_multi20:59:121
13890993624,4cyclictest181rcu_preempt00:00:351
13890993624,4cyclictest181rcu_preempt00:00:351
13890993624,4cyclictest181rcu_preempt00:00:351
13889993611,5cyclictest201rcuc/022:02:000
13889993611,5cyclictest201rcuc/022:02:000
13889993611,5cyclictest201rcuc/022:02:000
120702360,2sleep20-21swapper/222:03:562
120702360,2sleep20-21swapper/222:03:562
120702360,2sleep20-21swapper/222:03:562
61952350,2sleep06355-21cat23:14:020
1389399352,23cyclictest24544-21chrt00:03:533
1389399352,23cyclictest24544-21chrt00:03:533
1389399352,23cyclictest24544-21chrt00:03:533
13889993526,7cyclictest25230-21netstat19:44:060
154852340,3chrt0-21swapper/320:44:093
13890993426,6cyclictest31564-21fschecks_count21:29:001
13890993426,6cyclictest18469-21fschecks_count20:54:021
13890993421,2cyclictest181rcu_preempt00:14:001
13890993415,6cyclictest281ktimers/100:29:101
13889993427,5cyclictest6831-21sshd00:42:020
13889993427,5cyclictest6831-21sshd00:42:020
13889993427,5cyclictest2004-21unixbench-2d21:34:090
13889993427,5cyclictest2004-21unixbench-2d21:34:090
13889993427,5cyclictest16841-21sshd23:42:290
13889993427,5cyclictest16841-21sshd23:42:290
13889993427,5cyclictest16841-21sshd23:42:290
1389399331,23cyclictest25973-21chrt00:04:333
13890993327,5cyclictest14678-21systemd-journal22:38:291
13890993327,5cyclictest14678-21systemd-journal22:38:291
13890993326,5cyclictest15044-21threads22:09:101
13890993325,6cyclictest13888-21cyclictest00:34:101
13889993328,4cyclictest729-21dbus-daemon21:23:490
13889993328,4cyclictest729-21dbus-daemon21:23:490
13889993328,4cyclictest729-21dbus-daemon21:23:490
13889993328,3cyclictest0-21swapper/019:34:110
13889993326,5cyclictest17527-21cron22:18:480
13889993326,5cyclictest17527-21cron22:18:480
13889993319,11cyclictest0-21swapper/023:09:020
13889993319,11cyclictest0-21swapper/023:09:020
13889993314,14cyclictest90-1pr/ttyS019:42:070
13889993314,14cyclictest90-1pr/ttyS019:42:070
11242330,2sleep20-21swapper/222:59:082
11242330,2sleep20-21swapper/222:59:082
11242330,2sleep20-21swapper/222:59:082
13890993227,4cyclictest14678-21systemd-journal00:25:011
13890993226,5cyclictest13888-21cyclictest21:25:291
13890993226,5cyclictest13888-21cyclictest21:25:291
13890993223,7cyclictest601-21snmpd22:46:591
13890993216,8cyclictest90-1pr/ttyS023:44:051
13889993227,3cyclictest0-21swapper/022:13:490
13889993225,5cyclictest1521-21latency_hist00:28:550
13889993225,2cyclictest181rcu_preempt20:59:510
13889993225,2cyclictest181rcu_preempt20:59:510
13890993127,3cyclictest13888-21cyclictest22:49:021
13890993125,4cyclictest9469-21latency_hist23:23:491
13890993125,4cyclictest9469-21latency_hist23:23:491
13890993125,4cyclictest1346-21avahi-daemon00:39:101
13890993125,4cyclictest1346-21avahi-daemon00:39:101
13890993124,5cyclictest31790-21sendmail20:03:491
13890993124,5cyclictest31790-21sendmail20:03:491
13890993124,5cyclictest25162-21kernelversion00:04:051
13890993124,5cyclictest12063-21unixbench_singl20:34:101
13890993123,2cyclictest181rcu_preempt19:57:141
13890993123,2cyclictest181rcu_preempt00:13:091
13890993123,2cyclictest181rcu_preempt00:13:091
13889993125,5cyclictest12473-21latency_hist20:38:490
13889993125,4cyclictest23166-21latency_hist22:33:490
13889993125,4cyclictest23166-21latency_hist22:33:490
13889993123,6cyclictest29703-21tr22:49:050
1389099308,4cyclictest281ktimers/121:38:291
1389099308,4cyclictest281ktimers/121:38:291
13890993025,4cyclictest10472-21fschecks_time21:59:031
13890993025,4cyclictest10472-21fschecks_time21:59:031
13890993025,4cyclictest10472-21fschecks_time21:59:031
13890993017,2cyclictest181rcu_preempt23:54:011
1388999309,3cyclictest201rcuc/000:03:540
1388999309,3cyclictest201rcuc/000:03:540
1388999309,3cyclictest201rcuc/000:03:540
1388999307,4cyclictest201rcuc/022:54:320
1388999307,4cyclictest201rcuc/022:54:320
13889993026,3cyclictest7363-21gltestperf20:24:020
13889993026,3cyclictest13888-21cyclictest19:29:080
13889993024,5cyclictest4631-21sshd20:14:560
13889993024,4cyclictest90-1pr/ttyS020:05:260
13889993023,5cyclictest18265-21latency_hist20:53:540
13889993022,6cyclictest441-21fschecks_time22:59:000
13889993022,6cyclictest441-21fschecks_time22:59:000
13889993022,6cyclictest441-21fschecks_time22:59:000
13890992924,3cyclictest9249-21gltestperf20:29:011
13890992923,4cyclictest601-21snmpd21:55:001
13890992923,4cyclictest601-21snmpd21:55:001
13890992922,5cyclictest31369-21latency22:54:041
13890992922,5cyclictest31369-21latency22:54:041
13890992917,2cyclictest181rcu_preempt23:09:001
13890992917,2cyclictest181rcu_preempt23:09:001
1388999298,4cyclictest171ktimers/000:15:230
13889992925,3cyclictest3075-21systemctl23:04:110
13889992925,3cyclictest3075-21systemctl23:04:110
13889992924,4cyclictest25649-21switchtime00:04:120
13889992923,3cyclictest0-21swapper/020:23:490
13889992923,3cyclictest0-21swapper/020:23:490
13889992922,6cyclictest24437-21kernelversion21:09:050
13889992922,6cyclictest11571-21cpu23:28:590
13889992922,6cyclictest11571-21cpu23:28:590
13889992922,2cyclictest181rcu_preempt21:49:070
13889992922,2cyclictest181rcu_preempt21:49:070
13889992921,2cyclictest181rcu_preempt00:34:010
13889992916,4cyclictest90-1pr/ttyS021:43:500
13889992916,4cyclictest90-1pr/ttyS021:43:500
13889992916,4cyclictest90-1pr/ttyS021:43:500
13889992916,4cyclictest13782-21cut23:34:020
13889992914,4cyclictest90-1pr/ttyS023:23:580
13889992914,4cyclictest90-1pr/ttyS021:28:570
13889992914,4cyclictest90-1pr/ttyS000:13:490
13889992914,4cyclictest90-1pr/ttyS000:13:490
13889992914,4cyclictest30253-21/usr/sbin/munin00:19:110
13889992914,4cyclictest30253-21/usr/sbin/munin00:19:110
13889992913,4cyclictest90-1pr/ttyS021:14:070
13889992913,4cyclictest90-1pr/ttyS021:14:070
13890992823,2cyclictest0-21swapper/119:33:491
13890992822,5cyclictest31308-21timerandwakeup00:19:111
13890992822,5cyclictest31308-21timerandwakeup00:19:111
13890992822,4cyclictest0-21swapper/123:39:031
13890992822,4cyclictest0-21swapper/123:39:031
13890992822,4cyclictest0-21swapper/123:39:031
13890992821,5cyclictest26591-21fschecks_time19:48:591
13890992814,4cyclictest90-1pr/ttyS022:04:091
13890992814,12cyclictest281ktimers/121:39:111
13890992814,12cyclictest281ktimers/121:39:111
13890992814,12cyclictest281ktimers/121:39:111
13890992813,4cyclictest281ktimers/120:19:101
13890992813,4cyclictest281ktimers/120:19:101
13890992810,4cyclictest0-21swapper/120:10:171
13890992810,4cyclictest0-21swapper/120:10:171
1388999288,4cyclictest201rcuc/022:20:010
1388999288,4cyclictest201rcuc/022:20:010
13889992823,4cyclictest90-1pr/ttyS022:43:010
13889992823,4cyclictest90-1pr/ttyS022:43:010
13889992822,4cyclictest28499-21fschecks_time19:54:010
13889992816,4cyclictest90-1pr/ttyS021:28:590
13889992815,4cyclictest90-1pr/ttyS021:04:070
13889992813,4cyclictest90-1pr/ttyS022:48:550
13890992721,5cyclictest26899-21timerwakeupswit21:14:121
13890992721,5cyclictest26899-21timerwakeupswit21:14:121
13890992721,3cyclictest0-21swapper/122:43:491
13890992721,3cyclictest0-21swapper/122:43:491
13890992719,6cyclictest6084-21fschecks_count23:14:011
13890992714,3cyclictest90-1pr/ttyS023:34:091
13890992713,2cyclictest181rcu_preempt23:30:391
13890992713,2cyclictest181rcu_preempt23:30:391
13890992711,4cyclictest281ktimers/123:24:071
13889992719,6cyclictest10805-21ls23:24:110
13889992717,2cyclictest181rcu_preempt23:44:010
13889992716,4cyclictest181rcu_preempt21:28:300
13889992713,3cyclictest90-1pr/ttyS020:44:090
13889992713,3cyclictest201rcuc/020:54:060
1389099269,3cyclictest0-21swapper/119:35:391
13890992622,3cyclictest90-1pr/ttyS020:16:571
13890992621,3cyclictest0-21swapper/123:49:031
13890992621,2cyclictest0-21swapper/120:49:101
13890992620,4cyclictest27892-21fschecks_time21:19:021
13890992620,4cyclictest27892-21fschecks_time21:19:021
13890992620,4cyclictest27892-21fschecks_time21:19:021
13890992620,4cyclictest0-21swapper/122:19:041
13890992620,4cyclictest0-21swapper/122:19:041
13890992613,3cyclictest281ktimers/121:04:111
13890992611,4cyclictest271rcuc/121:49:301
13890992611,4cyclictest271rcuc/121:49:301
13890992610,4cyclictest0-21swapper/119:45:441
13889992620,4cyclictest0-21swapper/022:04:040
13889992619,4cyclictest171ktimers/021:58:000
13889992619,4cyclictest171ktimers/021:58:000
13889992618,2cyclictest181rcu_preempt20:01:410
13889992618,2cyclictest181rcu_preempt20:01:410
13889992615,2cyclictest181rcu_preempt19:53:120
13889992613,4cyclictest90-1pr/ttyS022:24:000
13889992613,4cyclictest90-1pr/ttyS020:09:030
13889992613,4cyclictest90-1pr/ttyS020:09:030
13890992521,3cyclictest90-1pr/ttyS021:44:001
13890992521,3cyclictest90-1pr/ttyS021:44:001
13890992520,3cyclictest0-21swapper/122:28:491
13890992517,6cyclictest22921-21cut19:39:011
13890992517,6cyclictest22921-21cut19:39:011
13889992518,4cyclictest0-21swapper/020:29:100
13889992513,3cyclictest90-1pr/ttyS023:19:010
167792240,3chrt0-21swapper/322:14:103
167792240,3chrt0-21swapper/322:14:103
13890992411,4cyclictest90-1pr/ttyS022:29:111
13890992411,4cyclictest90-1pr/ttyS022:29:111
13889992417,5cyclictest21082-21cut23:54:010
1389199231,17cyclictest24534-21taskset00:03:522
1389199231,17cyclictest24534-21taskset00:03:522
1389199231,17cyclictest24534-21taskset00:03:522
1389099239,3cyclictest0-21swapper/123:08:461
1389099239,3cyclictest0-21swapper/123:08:461
1388999237,4cyclictest201rcuc/021:44:060
1388999237,4cyclictest201rcuc/021:44:060
88542210,3chrt0-21swapper/320:28:543
88352211,3sleep20-21swapper/220:28:542
34322210,3sleep30-21swapper/300:33:563
111092210,3sleep30-21swapper/321:59:113
111092210,3sleep30-21swapper/321:59:113
111092210,3sleep30-21swapper/321:59:113
73672200,3sleep30-21swapper/323:16:293
13893992012,4cyclictest99582sleep320:29:103
1389199205,10cyclictest259722sleep200:04:332
1389399193,9cyclictest0-21swapper/319:59:253
1389399193,9cyclictest0-21swapper/319:59:253
1389399181,11cyclictest4254-21chrt23:09:023
1389399181,11cyclictest4254-21chrt23:09:023
322922170,3sleep30-21swapper/321:29:083
213932170,3sleep30-21swapper/322:28:543
1389399171,12cyclictest5774-21taskset20:19:053
1389399171,12cyclictest5774-21taskset20:19:053
1389399171,11cyclictest24802-21chrt21:09:103
1389399171,11cyclictest15314-21taskset23:39:013
1389399171,11cyclictest15314-21taskset23:39:013
1389399171,11cyclictest15314-21taskset23:39:013
1389399171,11cyclictest0-21swapper/300:09:123
1389399171,11cyclictest0-21swapper/300:09:123
1389399171,10cyclictest16575-21taskset20:49:003
13891991712,3cyclictest13442sleep221:34:022
13891991712,3cyclictest13442sleep221:34:022
13891991710,4cyclictest138592sleep220:39:152
13891991710,4cyclictest138592sleep220:39:152
1389399169,4cyclictest260922sleep322:39:123
1389399169,4cyclictest260922sleep322:39:123
1389399163,9cyclictest0-21swapper/300:37:013
1389399161,11cyclictest24574-21chrt22:34:123
1389399161,11cyclictest24574-21chrt22:34:123
1389399161,11cyclictest14525-21chrt22:09:043
1389399161,11cyclictest0-21swapper/322:53:563
1389399161,11cyclictest0-21swapper/320:14:123
1389399161,10cyclictest3218-21chrt21:39:053
1389399161,10cyclictest3218-21chrt21:39:053
1389399161,10cyclictest3218-21chrt21:39:053
1389399161,10cyclictest0-21swapper/320:09:063
1389399161,10cyclictest0-21swapper/320:09:063
13891991611,3cyclictest157122sleep223:39:042
13891991611,3cyclictest157122sleep223:39:042
13891991611,3cyclictest157122sleep223:39:042
1389199161,10cyclictest7777-21chrt23:18:582
1389199161,10cyclictest7777-21chrt23:18:582
303692151,2sleep20-21swapper/222:50:402
1389399151,9cyclictest28877-21chrt00:14:023
1389399151,9cyclictest12798-21taskset20:39:033
1389399151,9cyclictest12798-21taskset20:39:033
1389399151,9cyclictest0-21swapper/321:49:043
1389399151,9cyclictest0-21swapper/321:49:043
1389399151,11cyclictest0-21swapper/322:29:123
1389399151,11cyclictest0-21swapper/322:29:123
1389399151,11cyclictest0-21swapper/300:28:563
1389399151,10cyclictest9815-21chrt23:24:013
1389399151,10cyclictest32535-21chrt20:04:053
1389399151,10cyclictest28363-21taskset19:54:003
1389399151,10cyclictest21456-21chrt20:59:133
1389399151,10cyclictest21456-21chrt20:59:133
1389399151,10cyclictest0-21swapper/323:04:013
1389399151,10cyclictest0-21swapper/323:04:013
1389399151,10cyclictest0-21swapper/321:34:093
1389399151,10cyclictest0-21swapper/321:34:093
1389399150,10cyclictest19965-21taskset19:29:123
1389199159,3cyclictest0-21swapper/220:04:122
1389199151,8cyclictest0-21swapper/219:53:542
13891991511,2cyclictest303472sleep200:19:002
13891991511,2cyclictest303472sleep200:19:002
1389199151,12cyclictest25588-21taskset19:44:112
1389199151,10cyclictest27882-21taskset21:19:022
1389199151,10cyclictest27882-21taskset21:19:022
1389199151,10cyclictest27882-21taskset21:19:022
138392150,3sleep30-21swapper/323:34:023
1389399149,3cyclictest220422chrt19:34:113
1389399141,8cyclictest25913-21chrt21:14:003
1389399141,8cyclictest25913-21chrt21:14:003
1389399141,8cyclictest0-21swapper/321:19:113
1389399141,8cyclictest0-21swapper/321:19:113
1389399141,8cyclictest0-21swapper/321:19:113
1389399141,8cyclictest0-21swapper/320:54:113
1389399141,8cyclictest0-21swapper/300:43:543
1389399141,8cyclictest0-21swapper/300:43:543
1389399141,7cyclictest0-21swapper/319:48:573
1389199149,3cyclictest198342sleep219:29:112
1389199149,3cyclictest180372chrt20:52:572
1389199149,3cyclictest0-21swapper/223:13:592
1389199141,9cyclictest30157-21chrt21:24:042
1389199141,9cyclictest30157-21chrt21:24:042
1389199141,9cyclictest17859-21chrt22:19:002
1389199141,9cyclictest17859-21chrt22:19:002
1389199141,9cyclictest0-21swapper/223:50:292
1389199141,9cyclictest0-21swapper/223:37:292
1389199141,9cyclictest0-21swapper/222:14:292
1389199141,9cyclictest0-21swapper/222:14:292
1389199141,9cyclictest0-21swapper/221:44:292
1389199141,9cyclictest0-21swapper/221:44:292
1389199141,9cyclictest0-21swapper/220:44:272
1389199141,9cyclictest0-21swapper/219:54:542
1389199141,8cyclictest28780-21chrt00:14:012
1389199141,8cyclictest0-21swapper/220:09:052
1389199141,8cyclictest0-21swapper/220:09:052
1389199141,11cyclictest9483-21chrt21:54:112
1389199141,11cyclictest9483-21chrt21:54:112
1389199141,11cyclictest27219-21chrt22:44:002
1389199141,11cyclictest0-21swapper/221:09:022
1389199141,10cyclictest22638-21chrt19:38:562
1389199141,10cyclictest22638-21chrt19:38:562
1389399138,3cyclictest59542chrt21:44:403
1389399138,3cyclictest59542chrt21:44:403
1389399138,3cyclictest324722chrt22:58:493
1389399138,3cyclictest324722chrt22:58:493
1389399138,3cyclictest130452chrt23:31:573
1389399138,3cyclictest130452chrt23:31:573
1389399131,9cyclictest19713-21chrt23:49:063
1389399131,8cyclictest18474-21chrt23:44:153
1389399131,8cyclictest0-21swapper/321:55:293
1389399131,8cyclictest0-21swapper/321:55:293
1389399130,9cyclictest30375-21chrt00:19:013
1389399130,9cyclictest30375-21chrt00:19:013
1389199139,2cyclictest0-21swapper/222:10:192
1389199139,2cyclictest0-21swapper/220:59:002
1389199139,2cyclictest0-21swapper/220:59:002
1389199139,2cyclictest0-21swapper/220:19:082
1389199139,2cyclictest0-21swapper/220:19:082
1389199138,3cyclictest35112chrt23:05:072
1389199138,3cyclictest35112chrt23:05:072
1389199138,3cyclictest235752sleep221:07:052
1389199138,3cyclictest187352chrt23:47:362
1389199138,3cyclictest139482chrt22:08:552
1389199138,3cyclictest10052chrt00:24:222
1389199132,10cyclictest0-21swapper/220:54:052
1389199131,9cyclictest5572-21taskset00:39:022
1389199131,9cyclictest5572-21taskset00:39:022
1389199131,9cyclictest26865-21chrt21:14:112
1389199131,9cyclictest26865-21chrt21:14:112
1389199131,9cyclictest20366-21chrt22:24:062
1389199131,9cyclictest13230-21taskset23:33:492
1389199131,9cyclictest13230-21taskset23:33:492
1389199131,9cyclictest0-21swapper/219:34:002
1389199131,8cyclictest0-21swapper/221:50:292
1389199131,8cyclictest0-21swapper/221:50:292
1389199131,8cyclictest0-21swapper/220:30:272
1389199131,10cyclictest4830-21taskset23:09:092
1389199131,10cyclictest4830-21taskset23:09:092
1389199131,10cyclictest31606-21taskset22:54:092
1389199131,10cyclictest31606-21taskset22:54:092
1389199131,10cyclictest31467-21chrt20:00:102
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional