You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 21:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue May 05, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3011321020,1chrt30115-21sed11:27:401
3011321020,1chrt30115-21sed11:27:401
3011321020,1chrt30115-21sed11:27:401
156602700,3chrt0-21swapper/107:42:361
156602700,3chrt0-21swapper/107:42:361
156602700,3chrt0-21swapper/107:42:361
229002670,2chrt22898-21users11:07:410
229002670,2chrt22898-21users11:07:410
159982560,2sleep20-21swapper/210:52:272
159982560,2sleep20-21swapper/210:52:272
211052550,2chrt0-21swapper/112:32:281
211052550,2chrt0-21swapper/112:32:281
297272460,3chrt0-21swapper/109:52:291
3428994426,15cyclictest12636-21gltestperf10:42:310
3428994426,15cyclictest12636-21gltestperf10:42:310
3428994425,17cyclictest3427-21cyclictest08:47:310
3428994425,17cyclictest3427-21cyclictest08:47:310
166662430,3sleep30-21swapper/312:17:383
166662430,3sleep30-21swapper/312:17:383
3428994219,21cyclictest30818-21irqstats09:57:320
3428994219,21cyclictest30818-21irqstats09:57:320
185062420,2sleep20-21swapper/209:17:322
185062420,2sleep20-21swapper/209:17:322
185062420,2sleep20-21swapper/209:17:322
185062420,2sleep20-21swapper/209:17:322
304422410,2sleep30-21swapper/309:57:303
304422410,2sleep30-21swapper/309:57:303
279512400,2sleep127989-21ntp_states11:22:341
279512400,2sleep127989-21ntp_states11:22:341
227452400,3sleep30-21swapper/312:37:163
5242390,3sleep30-21swapper/310:07:303
5242390,3sleep30-21swapper/310:07:303
5242390,3sleep30-21swapper/310:07:303
3429993930,7cyclictest1673-21netstat08:32:341
3429993930,7cyclictest1673-21netstat08:32:341
275102390,2sleep027511-21sshd08:16:490
275102390,2sleep027511-21sshd08:16:490
3428993827,8cyclictest1455-21kernelversion08:32:320
3428993827,8cyclictest1455-21kernelversion08:32:320
342899381,12cyclictest32687-21latency_hist10:07:210
342899381,12cyclictest32687-21latency_hist10:07:210
3428993729,6cyclictest17141-21sendmail12:22:170
3428993729,6cyclictest17141-21sendmail12:22:170
3428993717,17cyclictest0-21swapper/010:07:310
3428993717,17cyclictest0-21swapper/010:07:310
3428993717,17cyclictest0-21swapper/010:07:310
342899371,13cyclictest12004-21cron09:02:170
342899371,13cyclictest12004-21cron09:02:170
342899371,12cyclictest9755-21cat12:02:210
342899371,12cyclictest9755-21cat12:02:210
342899371,12cyclictest9755-21cat12:02:210
342899371,12cyclictest4827-21needreboot11:47:340
342899371,12cyclictest4827-21needreboot11:47:340
342899371,12cyclictest4827-21needreboot11:47:340
342899371,12cyclictest21541-21if_eth012:32:310
153522370,3sleep30-21swapper/309:07:403
153522370,3sleep30-21swapper/309:07:403
153522370,3sleep30-21swapper/309:07:403
3429993629,5cyclictest1-21systemd11:12:321
3428993615,19cyclictest0-21swapper/008:17:360
3428993615,19cyclictest0-21swapper/008:17:360
3428993615,19cyclictest0-21swapper/007:22:310
342899361,12cyclictest32342-21uname08:27:380
342899361,12cyclictest14678-21systemd-journal10:17:170
342899361,12cyclictest14678-21systemd-journal10:17:170
3428993611,2cyclictest0-21swapper/007:38:390
3428993611,2cyclictest0-21swapper/007:38:390
3428993611,2cyclictest0-21swapper/007:38:390
342899361,11cyclictest20122-21grep11:02:300
342899361,11cyclictest20122-21grep11:02:300
342899359,4cyclictest0-21swapper/009:17:350
342899359,4cyclictest0-21swapper/009:17:350
342899352,9cyclictest19689-21latency_hist11:02:230
342899352,9cyclictest19689-21latency_hist11:02:230
342899351,7cyclictest18223-21cron07:52:170
3428993515,18cyclictest0-21swapper/012:37:370
3428993515,18cyclictest0-21swapper/012:37:370
342899351,14cyclictest27989-21ntp_states11:22:340
342899351,14cyclictest27989-21ntp_states11:22:340
342899351,12cyclictest27260-21latency_hist09:42:250
342899351,11cyclictest25353-21/usr/sbin/munin11:17:360
342899351,11cyclictest25353-21/usr/sbin/munin11:17:360
342899351,11cyclictest25353-21/usr/sbin/munin11:17:360
342899351,11cyclictest22193-21/usr/sbin/munin08:02:400
342899351,11cyclictest22193-21/usr/sbin/munin08:02:400
342899351,11cyclictest22193-21/usr/sbin/munin08:02:400
342899351,11cyclictest17379-21/usr/sbin/munin12:22:410
342899351,11cyclictest17379-21/usr/sbin/munin12:22:410
342899351,11cyclictest15883-21grep07:42:380
342899351,11cyclictest15883-21grep07:42:380
342899351,11cyclictest15883-21grep07:42:380
33012350,3sleep30-21swapper/310:17:283
33012350,3sleep30-21swapper/310:17:283
3429993426,6cyclictest10166-21cron10:37:171
3429993426,6cyclictest10166-21cron10:37:171
342899341,12cyclictest994-21cat11:37:310
342899341,12cyclictest994-21cat11:37:310
342899341,12cyclictest5902-21grep08:42:410
342899341,12cyclictest5902-21grep08:42:410
342899341,12cyclictest19769-21latency12:27:320
342899341,12cyclictest19769-21latency12:27:320
342899341,12cyclictest12485-21tune2fs09:02:300
342899341,12cyclictest12485-21tune2fs09:02:300
342899341,12cyclictest12485-21tune2fs09:02:300
342899341,11cyclictest749-21runrttasks11:46:360
342899341,11cyclictest749-21runrttasks11:46:360
342899341,11cyclictest749-21runrttasks11:46:360
342899341,11cyclictest30842-21latency_hist11:32:220
342899341,11cyclictest30842-21latency_hist11:32:220
342899341,11cyclictest30842-21latency_hist11:32:220
342899340,13cyclictest26629-21unixbench_singl09:37:400
3429993327,5cyclictest7623-21threads08:47:381
3429993327,5cyclictest7623-21threads08:47:381
3429993325,6cyclictest25065-21threads08:07:401
3429993325,6cyclictest25065-21threads08:07:401
3429993324,7cyclictest17817-21/usr/sbin/munin09:17:291
3429993324,7cyclictest17817-21/usr/sbin/munin09:17:291
3429993324,7cyclictest17817-21/usr/sbin/munin09:17:291
3429993324,7cyclictest17817-21/usr/sbin/munin09:17:291
3428993314,17cyclictest0-21swapper/008:37:320
3428993314,17cyclictest0-21swapper/008:37:320
342899331,12cyclictest749-21runrttasks08:55:070
342899331,12cyclictest749-21runrttasks08:55:070
342899331,12cyclictest25371-21sed09:37:250
3428993310,2cyclictest0-21swapper/010:32:380
3428993310,2cyclictest0-21swapper/010:32:380
342899330,6cyclictest10454-21chrt07:28:480
342899330,6cyclictest10454-21chrt07:28:480
178162330,3sleep0201rcuc/009:17:260
178162330,3sleep0201rcuc/009:17:260
342899329,3cyclictest32710-21latency_hist11:37:210
342899329,3cyclictest32710-21latency_hist11:37:210
342899327,5cyclictest13416-21latency_hist12:12:190
342899327,5cyclictest13416-21latency_hist12:12:190
342899321,12cyclictest30474-21ps08:22:370
342899321,12cyclictest30474-21ps08:22:370
342899321,12cyclictest23631-21df09:32:280
342899321,11cyclictest21062-21ntp_states07:57:380
342899321,11cyclictest21062-21ntp_states07:57:380
342899320,12cyclictest29508-21latency_hist09:52:280
342899320,12cyclictest29508-21latency_hist09:52:280
3429993125,5cyclictest7380-21ntp_offset10:27:361
3429993125,5cyclictest7380-21ntp_offset10:27:361
3429993125,5cyclictest7380-21ntp_offset10:27:361
3429993125,4cyclictest31987-21sendmail10:02:171
3429993125,4cyclictest31987-21sendmail10:02:171
3429993124,5cyclictest9631-21latency_hist12:02:161
3429993124,5cyclictest9631-21latency_hist12:02:161
3429993124,5cyclictest9631-21latency_hist12:02:161
3429993122,7cyclictest18899-21threads10:57:381
3429993122,7cyclictest18899-21threads10:57:381
342899316,7cyclictest10554-21tune2fs10:37:280
342899316,7cyclictest10554-21tune2fs10:37:280
3428993121,8cyclictest16381-21gltestperf10:52:300
3428993121,8cyclictest16381-21gltestperf10:52:300
3429993024,4cyclictest3158-21gltestperf08:37:281
3429993024,4cyclictest3158-21gltestperf08:37:281
3429993024,4cyclictest0-21swapper/108:57:161
3429993024,4cyclictest0-21swapper/108:57:161
3429993023,5cyclictest19170-21tr07:52:341
3429993021,7cyclictest20814-21threads09:22:391
3429993021,7cyclictest20814-21threads09:22:391
342899303,10cyclictest749-21runrttasks09:31:340
342899303,10cyclictest749-21runrttasks09:31:340
3428993023,4cyclictest0-21swapper/009:47:270
3428993023,4cyclictest0-21swapper/009:47:270
3428993017,11cyclictest0-21swapper/007:52:290
342899301,11cyclictest749-21runrttasks09:10:110
342899301,11cyclictest749-21runrttasks09:10:110
342899301,11cyclictest749-21runrttasks09:10:110
3429992922,5cyclictest26823-21sendmail_mailtr08:12:391
3429992922,5cyclictest26823-21sendmail_mailtr08:12:391
3429992922,5cyclictest2517-21sshd10:15:511
3429992922,5cyclictest2517-21sshd10:15:511
3429992922,5cyclictest13859-21cut12:12:291
3429992921,6cyclictest4996-21cut10:22:271
3429992921,6cyclictest4996-21cut10:22:271
3429992920,7cyclictest20374-21missed_timers11:02:331
3429992920,7cyclictest20374-21missed_timers11:02:331
3428992921,6cyclictest10576-21netstat12:02:340
3428992921,6cyclictest10576-21netstat12:02:340
342899291,13cyclictest6094-21/usr/sbin/munin11:52:310
342899291,13cyclictest6094-21/usr/sbin/munin11:52:310
266332290,2sleep20-21swapper/211:17:402
266332290,2sleep20-21swapper/211:17:402
266332290,2sleep20-21swapper/211:17:402
77892280,3sleep30-21swapper/308:47:383
77892280,3sleep30-21swapper/308:47:383
3429992822,5cyclictest31697-21cut08:27:301
3429992822,5cyclictest13086-21processes10:42:381
3429992822,5cyclictest13086-21processes10:42:381
3429992822,4cyclictest31318-21latency_hist08:27:171
3429992822,4cyclictest31318-21latency_hist08:27:171
3429992822,3cyclictest0-21swapper/109:32:231
3429992822,3cyclictest0-21swapper/109:32:231
3429992821,5cyclictest4503-21gltestperf11:47:311
3429992821,5cyclictest4503-21gltestperf11:47:311
3429992821,5cyclictest4503-21gltestperf11:47:311
3429992821,5cyclictest12746-21kernelversion09:02:331
3429992821,5cyclictest12746-21kernelversion09:02:331
3429992821,5cyclictest12746-21kernelversion09:02:331
3429992819,7cyclictest3427-21cyclictest11:17:281
3429992819,7cyclictest3427-21cyclictest11:17:281
3429992819,7cyclictest3427-21cyclictest11:17:281
3429992721,4cyclictest0-21swapper/111:12:301
225562270,2sleep122557-21expr08:02:301
225562270,2sleep122557-21expr08:02:301
225562270,2sleep122557-21expr08:02:301
70022260,3sleep30-21swapper/311:52:383
70022260,3sleep30-21swapper/311:52:383
3429992622,3cyclictest14495-21latency_hist07:42:161
3429992622,3cyclictest14495-21latency_hist07:42:161
3429992622,3cyclictest14495-21latency_hist07:42:161
3429992622,2cyclictest0-21swapper/112:32:351
3429992621,2cyclictest0-21swapper/109:52:171
3429992620,3cyclictest0-21swapper/112:12:281
3429992619,5cyclictest32763-21latency_hist11:37:241
3429992619,5cyclictest32763-21latency_hist11:37:241
3429992619,5cyclictest30214-21latency_hist09:57:221
3429992619,5cyclictest30214-21latency_hist09:57:221
3428992620,3cyclictest0-21swapper/010:47:350
3428992620,3cyclictest0-21swapper/010:47:350
260542260,4sleep30-21swapper/311:17:333
260542260,4sleep30-21swapper/311:17:333
260542260,4sleep30-21swapper/311:17:333
343699252,19cyclictest0-21swapper/310:47:073
343699252,19cyclictest0-21swapper/310:47:073
3429992520,3cyclictest0-21swapper/109:12:341
3429992520,3cyclictest0-21swapper/109:12:341
3429992520,3cyclictest0-21swapper/109:12:341
3429992519,5cyclictest10827-21switchtime12:02:381
3429992519,5cyclictest10827-21switchtime12:02:381
3429992518,4cyclictest0-21swapper/110:22:161
3429992518,4cyclictest0-21swapper/110:22:161
3429992517,6cyclictest7090-21sort11:52:391
3429992517,6cyclictest7090-21sort11:52:391
3428992520,3cyclictest24230-21ntp_offset11:12:340
3428992517,5cyclictest0-21swapper/010:27:400
3428992517,5cyclictest0-21swapper/010:27:400
3428992517,5cyclictest0-21swapper/010:27:400
343699249,12cyclictest129212sleep307:37:283
343699249,12cyclictest129212sleep307:37:283
343699249,12cyclictest129212sleep307:37:283
343699246,14cyclictest17472chrt11:38:273
343699246,14cyclictest17472chrt11:38:273
343699243,17cyclictest293672chrt08:21:383
343699243,17cyclictest293672chrt08:21:383
343699241,9cyclictest32767-21chrt10:07:263
343699241,9cyclictest32767-21chrt10:07:263
343699241,6cyclictest11089-21chrt12:02:393
343699241,6cyclictest11089-21chrt12:02:393
343699241,2cyclictest0-21swapper/311:29:273
343699241,2cyclictest0-21swapper/311:29:273
343699241,2cyclictest0-21swapper/311:29:273
343699241,11cyclictest27271-21chrt08:13:423
343699241,11cyclictest27271-21chrt08:13:423
343699241,10cyclictest27351-21chrt11:22:273
343699241,10cyclictest27351-21chrt11:22:273
343699241,10cyclictest24989-21chrt09:35:063
343699241,10cyclictest24989-21chrt09:35:063
343699241,10cyclictest24449-21chrt12:40:223
343699241,10cyclictest24449-21chrt12:40:223
343699241,10cyclictest21331-21chrt11:06:193
343699241,10cyclictest21331-21chrt11:06:193
3429992420,3cyclictest16876-21gltestperf07:47:301
3429992420,2cyclictest0-21swapper/111:37:361
3429992420,2cyclictest0-21swapper/111:37:361
3429992419,2cyclictest0-21swapper/109:37:381
3429992419,2cyclictest0-21swapper/109:37:381
3429992416,6cyclictest8948-21sort07:27:221
3428992420,2cyclictest0-21swapper/010:22:260
3428992420,2cyclictest0-21swapper/010:22:260
3428992418,4cyclictest0-21swapper/009:27:310
3428992418,4cyclictest0-21swapper/009:27:310
3428992417,5cyclictest19361-21sshd09:19:500
3428992417,5cyclictest19361-21sshd09:19:500
3428992417,5cyclictest19361-21sshd09:19:500
343699236,13cyclictest117962chrt10:39:183
343699236,13cyclictest117962chrt10:39:183
343699235,15cyclictest152172chrt12:16:463
343699235,15cyclictest152172chrt12:16:463
343699234,15cyclictest118552chrt09:00:293
343699234,15cyclictest118552chrt09:00:293
343699232,3cyclictest0-21swapper/309:07:173
343699232,3cyclictest0-21swapper/309:07:173
343699232,3cyclictest0-21swapper/309:07:173
343699231,9cyclictest2274-21chrt10:12:413
343699231,9cyclictest2274-21chrt10:12:413
343699231,9cyclictest20003-21chrt07:56:173
343699231,9cyclictest18977-21chrt12:26:513
343699231,9cyclictest18977-21chrt12:26:513
343699231,9cyclictest10679-21chrt07:31:453
343699231,9cyclictest10679-21chrt07:31:453
343699231,8cyclictest5268-21chrt10:22:293
343699231,8cyclictest5268-21chrt10:22:293
343699231,8cyclictest31729-21chrt11:32:363
343699231,8cyclictest31729-21chrt11:32:363
343699231,8cyclictest27487-21chrt09:45:193
343699231,8cyclictest27487-21chrt09:45:193
343699231,8cyclictest13256-21chrt12:10:513
343699231,8cyclictest13256-21chrt12:10:513
343699231,6cyclictest29299-21chrt09:49:413
343699231,6cyclictest29299-21chrt09:49:413
343699231,2cyclictest0-21swapper/309:38:203
343699231,2cyclictest0-21swapper/309:38:203
343699231,2cyclictest0-21swapper/307:42:373
343699231,2cyclictest0-21swapper/307:42:373
343699231,2cyclictest0-21swapper/307:42:373
343699231,18cyclictest0-21swapper/310:48:103
343699231,18cyclictest0-21swapper/310:48:103
343699231,18cyclictest0-21swapper/310:30:593
343699231,18cyclictest0-21swapper/310:30:593
343699231,18cyclictest0-21swapper/310:30:593
343699231,18cyclictest0-21swapper/308:30:233
343699231,12cyclictest29822-21chrt09:53:393
343699231,12cyclictest29822-21chrt09:53:393
343699231,10cyclictest4209-21chrt08:38:503
343699231,10cyclictest4209-21chrt08:38:503
3429992319,3cyclictest617-21lldpd11:42:271
3429992319,3cyclictest617-21lldpd11:42:271
3429992319,3cyclictest617-21lldpd11:42:271
3429992319,3cyclictest15797-21gltestperf12:17:291
3429992319,3cyclictest15797-21gltestperf12:17:291
3429992318,3cyclictest21917-21gltestperf11:07:311
3429992318,3cyclictest0-21swapper/112:27:171
3429992318,3cyclictest0-21swapper/112:27:171
3429992317,5cyclictest32325-21sshd10:03:501
3429992317,5cyclictest32325-21sshd10:03:501
3429992317,5cyclictest2031-21latency_hist10:12:161
3429992317,5cyclictest2031-21latency_hist10:12:161
3429992317,5cyclictest2031-21latency_hist10:12:161
3429992317,3cyclictest0-21swapper/108:47:271
3429992317,3cyclictest0-21swapper/108:47:271
3429992315,6cyclictest12109-21latency_hist10:42:181
3429992315,6cyclictest12109-21latency_hist10:42:181
3428992316,5cyclictest14419-21proc_pri12:12:360
3428992316,5cyclictest14419-21proc_pri12:12:360
343699221,9cyclictest18134-21chrt07:51:103
343699221,8cyclictest25346-21chrt08:07:513
343699221,8cyclictest25346-21chrt08:07:513
343699221,8cyclictest23740-21chrt08:06:103
343699221,8cyclictest23740-21chrt08:06:103
343699221,8cyclictest23740-21chrt08:06:103
343699221,8cyclictest19259-21chrt10:58:513
343699221,8cyclictest19259-21chrt10:58:513
343699221,7cyclictest9436-21chrt11:59:503
343699221,7cyclictest9436-21chrt11:59:503
343699221,7cyclictest9436-21chrt11:59:503
343699221,7cyclictest9101-21chrt08:52:333
343699221,7cyclictest9101-21chrt08:52:333
343699221,7cyclictest6201-21chrt08:44:533
343699221,7cyclictest6201-21chrt08:44:533
343699221,7cyclictest23117-21chrt11:10:213
343699221,7cyclictest23117-21chrt11:10:213
343699221,7cyclictest17311-21chrt10:52:443
343699221,7cyclictest17311-21chrt10:52:443
343699221,7cyclictest17056-21chrt09:12:393
343699221,7cyclictest17056-21chrt09:12:393
343699221,7cyclictest17056-21chrt09:12:393
343699221,7cyclictest17056-21chrt09:12:393
343699221,2cyclictest0-21swapper/312:29:003
343699221,2cyclictest0-21swapper/312:29:003
343699221,2cyclictest0-21swapper/311:49:373
343699221,2cyclictest0-21swapper/311:49:373
343699221,2cyclictest0-21swapper/311:49:373
343699221,2cyclictest0-21swapper/311:13:353
343699221,2cyclictest0-21swapper/309:31:083
343699221,2cyclictest0-21swapper/309:31:083
343699221,2cyclictest0-21swapper/307:58:193
343699221,2cyclictest0-21swapper/307:58:193
343699220,9cyclictest9918-21chrt10:34:033
343699220,9cyclictest9918-21chrt10:34:033
343699220,9cyclictest3585-21chrt11:42:413
343699220,9cyclictest3585-21chrt11:42:413
343699220,9cyclictest3585-21chrt11:42:413
343699220,4cyclictest311112sleep308:24:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional