You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 11:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Mar 26, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3097021160,4chrt0-21swapper/319:32:013
1040421160,3sleep20-21swapper/220:06:462
2920821100,3sleep20-21swapper/223:47:032
2920821100,3sleep20-21swapper/223:47:032
2607721070,1chrt0-21swapper/022:12:020
2607721070,1chrt0-21swapper/022:12:020
242252910,3sleep20-21swapper/220:41:552
242252910,3sleep20-21swapper/220:41:552
25362870,3sleep20-21swapper/200:02:352
25362870,3sleep20-21swapper/200:02:352
25362870,3sleep20-21swapper/200:02:352
49912840,3sleep20-21swapper/219:51:502
49912840,3sleep20-21swapper/219:51:502
65432750,3sleep20-21swapper/222:47:012
65432750,3sleep20-21swapper/222:47:012
157452670,2sleep00-21swapper/020:20:310
157452670,2sleep00-21swapper/020:20:310
172322630,3sleep30-21swapper/323:16:533
172322630,3sleep30-21swapper/323:16:533
172322630,3sleep30-21swapper/323:16:533
100282630,4sleep20-21swapper/222:56:582
100282630,4sleep20-21swapper/222:56:582
100282630,4sleep20-21swapper/222:56:582
22393996248,6cyclictest0-21swapper/100:01:581
22393996248,6cyclictest0-21swapper/100:01:581
22393996248,6cyclictest0-21swapper/100:01:581
35782550,1sleep13581-21grep21:12:011
35782550,1sleep13581-21grep21:12:011
252032540,3sleep30-21swapper/320:46:383
252032540,3sleep30-21swapper/320:46:383
243912530,3sleep30-21swapper/323:36:513
243912530,3sleep30-21swapper/323:36:513
68622480,5chrt0-21swapper/321:21:563
68622480,5chrt0-21swapper/321:21:563
156222480,3sleep20-21swapper/223:11:572
156222480,3sleep20-21swapper/223:11:572
2239199473,20cyclictest1069-21latency_hist00:01:440
2239199473,20cyclictest1069-21latency_hist00:01:440
22399994326,11cyclictest10612sleep300:01:443
22399994326,11cyclictest10612sleep300:01:443
196592430,3sleep00-21swapper/021:56:540
196592430,3sleep00-21swapper/021:56:540
2239899413,28cyclictest1107-21chrt00:01:462
2239899413,28cyclictest1107-21chrt00:01:462
166702410,1sleep00-21swapper/023:16:460
166702410,1sleep00-21swapper/023:16:460
165522410,3sleep20-21swapper/220:21:532
165522410,3sleep20-21swapper/220:21:532
74372400,3sleep30-21swapper/319:56:583
74372400,3sleep30-21swapper/319:56:583
22391994026,6cyclictest29809-21unixbench_singl22:22:000
22391994026,6cyclictest29809-21unixbench_singl22:22:000
22391994026,6cyclictest29809-21unixbench_singl22:22:000
42462390,3sleep20-21swapper/200:07:022
2239399381,32cyclictest1506-21tail00:01:511
76612370,3chrt0-21swapper/219:57:012
76612370,3chrt0-21swapper/219:57:012
22393993729,6cyclictest28499-21unixbench_singl20:52:021
22393993729,6cyclictest28499-21unixbench_singl20:52:021
2239199373,26cyclictest2226-21/usr/sbin/munin00:02:000
2239199373,26cyclictest2226-21/usr/sbin/munin00:02:000
2239199373,26cyclictest2226-21/usr/sbin/munin00:02:000
22391993630,3cyclictest0-21swapper/022:41:390
22391993630,3cyclictest0-21swapper/022:41:390
22391993627,7cyclictest24167-21latency_hist23:36:470
108252360,5sleep20-21swapper/221:32:002
108252360,5sleep20-21swapper/221:32:002
22391993530,4cyclictest16008-21latency21:46:530
22391993530,4cyclictest16008-21latency21:46:530
22391993528,5cyclictest20992-21kernelversion23:26:540
22391993528,5cyclictest20992-21kernelversion23:26:540
22391993526,7cyclictest6037-21kernelversion22:46:550
22391993526,7cyclictest6037-21kernelversion22:46:550
85662340,2sleep30-21swapper/322:52:013
85662340,2sleep30-21swapper/322:52:013
22391993425,7cyclictest1187-21needreboot21:06:560
22391993425,7cyclictest1187-21needreboot21:06:560
30812330,3sleep30-21swapper/319:46:493
30812330,3sleep30-21swapper/319:46:493
22393993326,5cyclictest25735-21mailstats22:12:001
22393993326,5cyclictest25735-21mailstats22:12:001
22393993325,6cyclictest28728-21fschecks_count22:21:501
22393993325,6cyclictest28728-21fschecks_count22:21:501
22391993324,7cyclictest20284-21idleruntime-cro23:26:390
22391993324,7cyclictest20284-21idleruntime-cro23:26:390
2239199330,9cyclictest700-21192.168.115.40-manager00:36:090
2239199330,9cyclictest700-21192.168.115.40-manager00:36:090
312222320,2sleep00-21swapper/022:26:580
312222320,2sleep00-21swapper/022:26:580
261482320,3sleep20-21swapper/219:21:512
261482320,3sleep20-21swapper/219:21:512
261482320,3sleep20-21swapper/219:21:512
22391993227,3cyclictest0-21swapper/022:02:000
22391993227,3cyclictest0-21swapper/022:02:000
22391993227,3cyclictest0-21swapper/022:02:000
22391993223,6cyclictest17969-21head23:17:010
22391993223,6cyclictest17969-21head23:17:010
22391993223,6cyclictest17969-21head23:17:010
45702310,3sleep20-21swapper/221:16:522
45702310,3sleep20-21swapper/221:16:522
45702310,3sleep20-21swapper/221:16:522
22393993123,6cyclictest31020-21/usr/sbin/munin21:01:561
22391993124,5cyclictest1346-21avahi-daemon00:12:330
22391993124,5cyclictest1346-21avahi-daemon00:12:330
22391993120,7cyclictest0-21swapper/023:47:010
22391993120,7cyclictest0-21swapper/023:47:010
22391993117,12cyclictest0-21swapper/020:26:520
22391993117,12cyclictest0-21swapper/020:26:520
2239199310,12cyclictest23622-21bingrep22:06:540
2239199310,10cyclictest0-21swapper/019:38:010
2239199310,10cyclictest0-21swapper/019:38:010
2239199310,10cyclictest0-21swapper/019:38:010
2239399306,4cyclictest90-1pr/ttyS019:52:311
2239399306,4cyclictest90-1pr/ttyS019:52:311
22393993026,2cyclictest0-21swapper/100:01:391
22393993024,4cyclictest8363-21gltestperf21:26:531
22393993024,4cyclictest8363-21gltestperf21:26:531
22393993023,5cyclictest8976-21cut00:21:531
22393993023,5cyclictest8976-21cut00:21:531
22393993023,5cyclictest8976-21cut00:21:531
22393993020,7cyclictest21303-21fschecks_time22:01:521
22393993020,7cyclictest21303-21fschecks_time22:01:521
22393993020,7cyclictest21303-21fschecks_time22:01:521
22391993026,3cyclictest28806-21cut22:21:500
22391993026,3cyclictest28806-21cut22:21:500
22391993024,4cyclictest0-21swapper/021:27:010
22391993024,4cyclictest0-21swapper/021:27:010
22391993023,5cyclictest27067-21switchtime23:42:010
22391993023,5cyclictest27067-21switchtime23:42:010
22391993023,5cyclictest27067-21switchtime23:42:010
22391993021,7cyclictest8110-21netstat22:51:560
22391993021,7cyclictest8110-21netstat22:51:560
22391993021,7cyclictest4854-21latency_hist00:11:460
22391993021,7cyclictest14666-21wc23:11:390
2239999291,9cyclictest22426-21taskset20:36:583
2239999291,9cyclictest22426-21taskset20:36:583
2239999291,9cyclictest22426-21taskset20:36:583
22393992925,3cyclictest21236-21ntp_offset23:26:581
22393992925,3cyclictest21236-21ntp_offset23:26:581
22393992924,4cyclictest22388-21cyclictest19:46:581
22393992924,4cyclictest22388-21cyclictest19:46:581
22393992912,9cyclictest90-1pr/ttyS019:36:551
22393992912,9cyclictest90-1pr/ttyS019:36:551
22393992912,9cyclictest90-1pr/ttyS019:36:551
22393992911,2cyclictest0-21swapper/119:32:011
22391992924,4cyclictest12101-21gltestperf21:36:520
22391992924,4cyclictest12101-21gltestperf21:36:520
22391992924,4cyclictest12101-21gltestperf21:36:520
22391992923,4cyclictest1648-21ntpd19:28:290
22391992922,5cyclictest22388-21cyclictest22:45:350
22391992922,5cyclictest22388-21cyclictest22:45:350
22391992922,4cyclictest0-21swapper/019:56:010
22391992922,4cyclictest0-21swapper/019:56:010
2239199290,11cyclictest602-21ntp_states22:31:560
2239199290,11cyclictest602-21ntp_states22:31:560
136842290,2sleep30-21swapper/323:06:563
2239999287,18cyclictest0-21swapper/320:28:173
2239999287,18cyclictest0-21swapper/320:28:173
2239999281,7cyclictest31403-21chrt23:55:483
2239999281,7cyclictest31403-21chrt23:55:483
2239999281,7cyclictest13149-21chrt21:38:443
2239999281,7cyclictest13149-21chrt21:38:443
2239999281,7cyclictest13149-21chrt21:38:443
2239999281,6cyclictest9607-21chrt21:31:043
2239999281,6cyclictest9607-21chrt21:31:043
2239999281,6cyclictest3859-21chrt21:14:203
2239999281,6cyclictest3859-21chrt21:14:203
2239999281,6cyclictest32635-21chrt21:06:163
2239999281,6cyclictest22427-21chrt22:02:363
2239999281,6cyclictest22427-21chrt22:02:363
2239999281,6cyclictest22427-21chrt22:02:363
2239999281,6cyclictest1433-21chrt22:34:563
2239999281,6cyclictest1433-21chrt22:34:563
2239399284,6cyclictest281ktimers/121:46:521
2239399284,6cyclictest281ktimers/121:46:521
2239399283,21cyclictest281ktimers/122:55:011
2239399283,21cyclictest281ktimers/122:55:011
22393992820,2cyclictest181rcu_preempt21:35:141
22393992820,2cyclictest181rcu_preempt21:35:141
22393992817,4cyclictest25565-21kworker/u9:3-xprtiod19:26:521
2239399281,10cyclictest2515-21ntp_kernel_err22:36:571
2239399281,10cyclictest2515-21ntp_kernel_err22:36:571
22391992823,3cyclictest27796-21kernelversion20:51:530
22391992823,3cyclictest27796-21kernelversion20:51:530
22391992823,3cyclictest0-21swapper/021:42:010
22391992823,3cyclictest0-21swapper/021:42:010
22391992822,4cyclictest30117-21head19:31:520
22391992822,4cyclictest0-21swapper/023:56:390
22391992822,4cyclictest0-21swapper/023:56:390
22391992821,5cyclictest11819-21sshd20:08:000
22391992821,5cyclictest11819-21sshd20:08:000
22391992817,9cyclictest0-21swapper/000:22:090
22391992817,9cyclictest0-21swapper/000:22:090
22391992817,9cyclictest0-21swapper/000:22:090
257422270,3sleep20-21swapper/222:12:002
257422270,3sleep20-21swapper/222:12:002
2239999271,9cyclictest12406-21taskset23:02:023
2239999271,9cyclictest12406-21taskset23:02:023
2239999271,6cyclictest9984-21chrt20:03:213
2239999271,6cyclictest8114-21chrt00:17:273
2239999271,6cyclictest8114-21chrt00:17:273
2239999271,6cyclictest30696-21chrt21:00:333
2239999271,6cyclictest30696-21chrt21:00:333
2239999271,6cyclictest29323-21chrt23:48:003
2239999271,6cyclictest29323-21chrt23:48:003
2239999271,6cyclictest28759-21chrt20:54:343
2239999271,6cyclictest28759-21chrt20:54:343
2239999271,6cyclictest24525-21chrt22:10:403
2239999271,6cyclictest2043-21chrt21:10:103
2239999271,6cyclictest2043-21chrt21:10:103
2239999271,6cyclictest17464-21chrt20:23:233
2239999271,6cyclictest17464-21chrt20:23:233
2239999271,6cyclictest13844-21chrt20:15:073
2239999271,6cyclictest10841-21chrt23:00:493
2239999271,6cyclictest10841-21chrt23:00:493
2239999271,6cyclictest10841-21chrt23:00:493
2239399276,3cyclictest181rcu_preempt23:33:081
22393992721,4cyclictest1346-21avahi-daemon00:32:001
22393992721,4cyclictest1346-21avahi-daemon00:32:001
22393992720,5cyclictest12472-21ntp_offset21:36:571
22393992720,5cyclictest12472-21ntp_offset21:36:571
22393992720,5cyclictest12472-21ntp_offset21:36:571
22393992716,5cyclictest22388-21cyclictest21:52:001
22393992716,5cyclictest22388-21cyclictest21:52:001
2239399271,4cyclictest700-21192.168.115.40-manager23:53:551
2239399271,4cyclictest700-21192.168.115.40-manager23:53:551
22391992719,6cyclictest9550-21cut22:56:510
22391992719,6cyclictest9550-21cut22:56:510
22391992719,6cyclictest9550-21cut22:56:510
22391992719,6cyclictest0-21swapper/021:11:590
22391992719,6cyclictest0-21swapper/021:11:590
2239199271,8cyclictest0-21swapper/000:40:540
2239199271,8cyclictest0-21swapper/000:40:540
2239199271,8cyclictest0-21swapper/000:40:540
22391992718,7cyclictest24234-21needreboot20:41:560
22391992718,7cyclictest24234-21needreboot20:41:560
2239199271,7cyclictest0-21swapper/020:16:150
22391992717,8cyclictest1283-21cut19:41:510
2239199270,12cyclictest32313-21taskset21:02:060
2239999267,16cyclictest0-21swapper/322:43:203
2239999267,16cyclictest0-21swapper/322:43:203
2239999261,8cyclictest31840-21chrt22:28:333
2239999261,8cyclictest31840-21chrt22:28:333
2239999261,6cyclictest6362-21chrt00:13:533
2239999261,6cyclictest6362-21chrt00:13:533
2239999261,6cyclictest5573-21chrt21:17:263
2239999261,6cyclictest5573-21chrt21:17:263
2239999261,6cyclictest5573-21chrt21:17:263
2239999261,6cyclictest29440-21chrt19:30:333
2239999261,6cyclictest19921-21chrt23:22:123
2239999261,6cyclictest19921-21chrt23:22:123
2239999261,6cyclictest17291-21chrt21:51:473
2239999261,6cyclictest17291-21chrt21:51:473
2239999261,6cyclictest15826-21chrt00:39:543
2239999261,6cyclictest15826-21chrt00:39:543
2239999261,6cyclictest15826-21chrt00:39:543
2239999261,6cyclictest11381-21chrt21:34:583
2239999261,6cyclictest11381-21chrt21:34:583
2239999260,6cyclictest29984-21chrt22:23:393
2239999260,6cyclictest29984-21chrt22:23:393
2239999260,6cyclictest29984-21chrt22:23:393
2239999260,6cyclictest21846-21chrt23:27:583
2239999260,6cyclictest21846-21chrt23:27:583
2239999260,6cyclictest21137-21chrt20:32:443
22393992620,5cyclictest22388-21cyclictest20:12:001
22393992620,4cyclictest1007-21timerwakeupswit22:31:591
22393992620,4cyclictest1007-21timerwakeupswit22:31:591
22393992620,3cyclictest0-21swapper/119:21:531
22393992620,3cyclictest0-21swapper/119:21:531
22393992620,3cyclictest0-21swapper/119:21:531
22393992619,5cyclictest31041-21acpi21:01:491
22393992619,5cyclictest29387-21processes22:21:571
22393992619,5cyclictest29387-21processes22:21:571
22393992619,5cyclictest29387-21processes22:21:571
22393992617,2cyclictest181rcu_preempt20:07:001
22393992617,2cyclictest181rcu_preempt20:07:001
22393992616,3cyclictest90-1pr/ttyS020:22:001
22393992616,3cyclictest90-1pr/ttyS020:22:001
22393992614,3cyclictest181rcu_preempt22:56:531
22393992614,3cyclictest181rcu_preempt22:56:531
22393992614,3cyclictest181rcu_preempt22:56:531
2239399260,9cyclictest8284-21runrttasks00:19:341
2239399260,9cyclictest8284-21runrttasks00:19:341
22391992621,3cyclictest0-21swapper/020:22:020
22391992621,3cyclictest0-21swapper/020:22:020
22391992620,3cyclictest0-21swapper/023:02:010
22391992620,3cyclictest0-21swapper/023:02:010
22391992619,5cyclictest17625-21cut21:51:510
22391992619,5cyclictest17625-21cut21:51:510
2239199261,8cyclictest6708-21grep21:21:530
2239199261,8cyclictest6708-21grep21:21:530
22391992614,10cyclictest0-21swapper/021:18:330
22391992614,10cyclictest0-21swapper/021:18:330
22391992614,10cyclictest0-21swapper/021:18:330
2239199261,14cyclictest749-21runrttasks00:27:330
2239199261,14cyclictest749-21runrttasks00:27:330
2239199261,12cyclictest749-21runrttasks20:04:230
2239199260,8cyclictest19904-21awk20:31:500
2239199260,11cyclictest0-21swapper/021:33:330
2239199260,11cyclictest0-21swapper/021:33:330
24952250,3chrt0-21swapper/300:02:033
24952250,3chrt0-21swapper/300:02:033
24952250,3chrt0-21swapper/300:02:033
230662250,4sleep30-21swapper/323:31:563
2239999258,14cyclictest0-21swapper/300:24:433
2239999258,14cyclictest0-21swapper/300:24:433
2239999258,14cyclictest0-21swapper/300:24:433
2239999252,4cyclictest0-21swapper/320:21:393
2239999252,4cyclictest0-21swapper/320:21:393
2239999251,8cyclictest6146-21chrt19:52:113
2239999251,8cyclictest6146-21chrt19:52:113
2239999251,7cyclictest4528-21chrt00:09:153
2239999251,7cyclictest3196-21chrt22:38:343
2239999251,7cyclictest3196-21chrt22:38:343
2239999251,7cyclictest16490-21chrt23:16:213
2239999251,7cyclictest16490-21chrt23:16:213
2239999251,7cyclictest13713-21chrt00:32:023
2239999251,7cyclictest13713-21chrt00:32:023
2239999251,6cyclictest28084-21chrt22:18:323
2239999251,6cyclictest28084-21chrt22:18:323
2239999251,6cyclictest27462-21chrt19:24:013
2239999251,6cyclictest27462-21chrt19:24:013
2239999251,6cyclictest27462-21chrt19:24:013
2239999251,6cyclictest2574-21chrt19:44:073
2239999251,6cyclictest11764-21chrt20:07:193
2239999251,6cyclictest11764-21chrt20:07:193
2239999250,7cyclictest14933-21chrt21:42:263
2239999250,7cyclictest14933-21chrt21:42:263
2239999250,6cyclictest443-21chrt19:37:123
2239999250,6cyclictest443-21chrt19:37:123
2239999250,6cyclictest443-21chrt19:37:123
2239999250,6cyclictest26152-21chrt22:12:373
2239999250,6cyclictest26152-21chrt22:12:373
2239399257,3cyclictest281ktimers/100:11:591
2239399257,3cyclictest281ktimers/100:11:591
2239399255,4cyclictest90-1pr/ttyS020:57:181
22393992521,2cyclictest0-21swapper/100:26:531
22393992521,2cyclictest0-21swapper/100:26:531
22393992519,4cyclictest90-1pr/ttyS019:44:201
22393992519,3cyclictest0-21swapper/121:41:521
22393992519,3cyclictest0-21swapper/121:41:521
22393992519,3cyclictest0-21swapper/120:32:001
22393992518,5cyclictest7421-21netstat19:56:571
22393992518,5cyclictest7421-21netstat19:56:571
22393992518,5cyclictest550-21cron22:41:501
22393992518,5cyclictest550-21cron22:41:501
22393992518,5cyclictest271rcuc/123:41:161
22393992518,5cyclictest271rcuc/123:41:161
22393992518,5cyclictest16532-21latency_hist23:16:391
22393992518,5cyclictest16532-21latency_hist23:16:391
22393992518,5cyclictest12740-21sshd23:06:011
22393992518,5cyclictest12740-21sshd23:06:011
22393992517,6cyclictest5255-21sort21:17:011
22393992517,6cyclictest5255-21sort21:17:011
22393992517,6cyclictest5255-21sort21:17:011
22393992516,7cyclictest19881-21needreboot21:56:581
22393992516,7cyclictest19881-21needreboot21:56:581
22393992515,3cyclictest90-1pr/ttyS020:37:011
22393992515,3cyclictest90-1pr/ttyS020:37:011
22393992515,3cyclictest90-1pr/ttyS020:37:011
22393992514,6cyclictest90-1pr/ttyS000:36:581
22393992514,6cyclictest90-1pr/ttyS000:36:581
22393992514,6cyclictest90-1pr/ttyS000:36:581
22393992510,9cyclictest90-1pr/ttyS023:42:021
22393992510,9cyclictest90-1pr/ttyS023:42:021
22393992510,9cyclictest90-1pr/ttyS023:42:021
22391992521,2cyclictest0-21swapper/019:21:530
22391992521,2cyclictest0-21swapper/019:21:530
22391992521,2cyclictest0-21swapper/019:21:530
2239199250,9cyclictest6964-21perl19:56:520
2239199250,9cyclictest6964-21perl19:56:520
2239199250,7cyclictest0-21swapper/000:21:000
2239199250,7cyclictest0-21swapper/000:21:000
2239199250,16cyclictest25776-21runrttasks23:40:340
2239199250,16cyclictest25776-21runrttasks23:40:340
2239199250,10cyclictest25418-21seq20:46:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional