You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 04:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sat Jul 04, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
149821350,3sleep20-21swapper/221:28:222
149821350,3sleep20-21swapper/221:28:222
149821350,3sleep20-21swapper/221:28:222
1810021270,2sleep20-21swapper/223:38:232
1810021270,2sleep20-21swapper/223:38:232
98852990,3sleep20-21swapper/200:43:232
98852990,3sleep20-21swapper/200:43:232
29142800,3sleep30-21swapper/300:23:263
29142800,3sleep30-21swapper/300:23:263
73192770,2chrt7317-21ntpq23:08:300
73192770,2chrt7317-21ntpq23:08:300
151132740,2sleep0171ktimers/023:28:320
151132740,2sleep0171ktimers/023:28:320
317622720,3sleep30-21swapper/322:48:233
77302620,2sleep30-21swapper/321:43:273
77302620,2sleep30-21swapper/321:43:273
77302620,2sleep30-21swapper/321:43:273
81552590,3chrt0-21swapper/020:18:250
81552590,3chrt0-21swapper/020:18:250
14222570,4chrt0-21swapper/319:58:553
14222570,4chrt0-21swapper/319:58:553
157652530,2chrt15766-21cat23:33:111
157652530,2chrt15766-21cat23:33:111
140322530,1sleep014068-21/usr/sbin/munin20:33:300
140322530,1sleep014068-21/usr/sbin/munin20:33:300
140322530,1sleep014068-21/usr/sbin/munin20:33:300
129502530,4chrt0-21swapper/221:58:252
129502530,4chrt0-21swapper/221:58:252
15851995236,8cyclictest27118-21/usr/sbin/munin00:03:370
15851995236,8cyclictest27118-21/usr/sbin/munin00:03:370
15851995236,8cyclictest27118-21/usr/sbin/munin00:03:370
272392480,4sleep30-21swapper/321:08:333
272392480,4sleep30-21swapper/321:08:333
82192470,2sleep30-21swapper/323:13:113
82192470,2sleep30-21swapper/323:13:113
238712450,2sleep30-21swapper/319:33:263
238712450,2sleep30-21swapper/319:33:263
130942450,2sleep113038-21latency_hist20:33:151
130942450,2sleep113038-21latency_hist20:33:151
309752440,3chrt0-21swapper/319:53:243
309752440,3chrt0-21swapper/319:53:243
7812430,2sleep0776-21strings19:58:270
7812430,2sleep0776-21strings19:58:270
316292430,3sleep20-21swapper/200:13:282
282322430,1sleep128426-21grep00:03:351
282322430,1sleep128426-21grep00:03:351
282322430,1sleep128426-21grep00:03:351
15854994312,6cyclictest281ktimers/122:01:471
15854994312,6cyclictest281ktimers/122:01:471
15854993931,5cyclictest0-21swapper/100:03:181
15854993931,5cyclictest0-21swapper/100:03:181
15854993931,5cyclictest0-21swapper/100:03:181
257972370,3sleep00-21swapper/022:33:200
257972370,3sleep00-21swapper/022:33:200
1585499375,5cyclictest281ktimers/121:44:501
1585499375,5cyclictest281ktimers/121:44:501
1585499375,5cyclictest281ktimers/121:44:501
282502350,2chrt0-21swapper/322:38:263
282502350,2chrt0-21swapper/322:38:263
15851993522,10cyclictest0-21swapper/000:08:320
15851993522,10cyclictest0-21swapper/000:08:320
218652340,3sleep30-21swapper/320:53:343
218652340,3sleep30-21swapper/320:53:343
218652340,3sleep30-21swapper/320:53:343
196352340,3sleep10-21swapper/120:48:291
196352340,3sleep10-21swapper/120:48:291
1585499348,6cyclictest281ktimers/120:05:251
1585499348,6cyclictest281ktimers/120:05:251
15854993428,5cyclictest24796-21gltestperf21:03:241
15854993428,5cyclictest24796-21gltestperf21:03:241
80362330,2sleep08222-21idleruntime00:38:240
185452330,2sleep018732-21timerwakeupswit19:18:330
1585699331,8cyclictest30627-21chrt00:12:213
1585699331,8cyclictest30627-21chrt00:12:213
1585499339,6cyclictest281ktimers/119:19:581
1585499339,21cyclictest281ktimers/119:57:441
1585499339,21cyclictest281ktimers/119:57:441
1585499337,5cyclictest281ktimers/120:18:041
1585499337,5cyclictest281ktimers/120:18:041
1585499337,5cyclictest281ktimers/120:18:041
1585499337,4cyclictest281ktimers/123:05:491
1585499337,4cyclictest281ktimers/123:05:491
15854993328,3cyclictest28058-21idleruntime-cro21:13:121
15854993328,3cyclictest28058-21idleruntime-cro21:13:121
15854993326,5cyclictest281ktimers/123:19:051
15854993326,5cyclictest281ktimers/123:19:051
15854993326,5cyclictest281ktimers/120:58:451
15854993326,5cyclictest281ktimers/120:58:451
15854993310,11cyclictest29-21ksoftirqd/122:39:251
15854993310,11cyclictest29-21ksoftirqd/122:39:251
34642320,2sleep13460-21strings22:58:261
34642320,2sleep13460-21strings22:58:261
34642320,2sleep13460-21strings22:58:261
1585499328,5cyclictest281ktimers/121:30:161
1585499328,5cyclictest281ktimers/121:30:161
1585499328,5cyclictest281ktimers/121:30:161
15851993225,5cyclictest30800-21unixbench-2d22:43:350
15851993225,5cyclictest30800-21unixbench-2d22:43:350
15851993225,5cyclictest30800-21unixbench-2d22:43:350
15851993225,5cyclictest12519-21tail23:23:240
15851993225,5cyclictest12519-21tail23:23:240
15851993225,5cyclictest12519-21tail23:23:240
15851993223,7cyclictest27639-21needreboot19:43:290
15851993223,7cyclictest27639-21needreboot19:43:290
286272310,3sleep1271rcuc/119:48:111
286272310,3sleep1271rcuc/119:48:111
232262310,2chrt0-21swapper/323:53:113
232262310,2chrt0-21swapper/323:53:113
1585499319,9cyclictest29-21ksoftirqd/123:34:101
1585499319,9cyclictest29-21ksoftirqd/123:34:101
1585499319,9cyclictest29-21ksoftirqd/123:34:101
1585499319,6cyclictest281ktimers/121:23:201
1585499319,6cyclictest281ktimers/121:23:201
1585499319,4cyclictest281ktimers/120:08:441
1585499319,4cyclictest281ktimers/120:08:441
1585499319,4cyclictest281ktimers/120:08:441
15854993126,4cyclictest9880-21gltestperf20:23:241
15854993126,4cyclictest9880-21gltestperf20:23:241
15854993126,4cyclictest281ktimers/100:32:521
15854993126,4cyclictest281ktimers/100:32:521
15854993126,4cyclictest15850-21cyclictest00:18:341
15854993126,4cyclictest15850-21cyclictest00:18:341
15854993126,3cyclictest0-21swapper/121:23:311
15854993126,3cyclictest0-21swapper/121:23:311
15854993126,3cyclictest0-21swapper/121:23:311
15854993125,4cyclictest281ktimers/120:53:451
15854993125,4cyclictest281ktimers/120:53:451
15854993125,4cyclictest281ktimers/120:53:451
15854993124,5cyclictest281ktimers/119:33:431
15854993124,5cyclictest281ktimers/119:33:431
15854993123,6cyclictest23132-21threads22:23:321
15854993123,6cyclictest23132-21threads22:23:321
15854993111,4cyclictest271rcuc/120:35:021
15854993111,4cyclictest271rcuc/120:35:021
15854993111,4cyclictest271rcuc/120:35:021
15851993127,3cyclictest4452-21latency_hist23:03:110
15851993127,3cyclictest4452-21latency_hist23:03:110
15851993127,3cyclictest4452-21latency_hist23:03:110
15851993126,3cyclictest31036-21head19:53:240
15851993126,3cyclictest31036-21head19:53:240
15851993124,5cyclictest572-21cups-browsed22:23:310
15851993124,5cyclictest572-21cups-browsed22:23:310
84212300,1sleep08492-21/usr/sbin/munin23:13:210
84212300,1sleep08492-21/usr/sbin/munin23:13:210
64532300,3sleep30-21swapper/300:33:233
257472300,3sleep20-21swapper/221:03:342
257472300,3sleep20-21swapper/221:03:342
15854993024,4cyclictest281ktimers/123:15:031
15854993024,4cyclictest281ktimers/123:15:031
15854993024,4cyclictest20017-21grep23:43:241
15854993024,4cyclictest20017-21grep23:43:241
15854993024,4cyclictest20017-21grep23:43:241
15851993025,3cyclictest1084-21ntp_offset00:18:300
15851993025,3cyclictest1084-21ntp_offset00:18:300
15851993024,4cyclictest5573-21cut21:38:240
15851993024,4cyclictest5573-21cut21:38:240
15851993023,5cyclictest23939-21kernelversion23:53:260
15851993023,5cyclictest23939-21kernelversion23:53:260
15851993023,5cyclictest23939-21kernelversion23:53:260
15851993023,5cyclictest23939-21kernelversion23:53:260
15851993023,5cyclictest1584-21fschecks_count21:28:230
15851993023,5cyclictest1584-21fschecks_count21:28:230
15851993023,5cyclictest1584-21fschecks_count21:28:230
15851993022,5cyclictest4577-21ntp_kernel_err20:08:280
15851993022,5cyclictest4577-21ntp_kernel_err20:08:280
15851993022,5cyclictest4577-21ntp_kernel_err20:08:280
15851993020,8cyclictest9719-21ntp_kernel_pll_21:48:280
15851993020,8cyclictest9719-21ntp_kernel_pll_21:48:280
15851993020,8cyclictest9719-21ntp_kernel_pll_21:48:280
15851993016,12cyclictest0-21swapper/019:23:230
15851993016,12cyclictest0-21swapper/019:23:230
63662290,2sleep06369-21sed20:13:270
63662290,2sleep06369-21sed20:13:270
63662290,2sleep06369-21sed20:13:270
46002290,3sleep30-21swapper/321:33:343
46002290,3sleep30-21swapper/321:33:343
46002290,3sleep30-21swapper/321:33:343
1585699292,15cyclictest0-21swapper/300:03:163
1585699292,15cyclictest0-21swapper/300:03:163
1585699292,15cyclictest0-21swapper/300:03:163
1585699291,9cyclictest12470-21chrt21:57:503
1585699291,9cyclictest12470-21chrt21:57:503
1585699291,9cyclictest12470-21chrt21:57:503
1585699291,8cyclictest18971-21chrt19:19:473
1585499294,5cyclictest271rcuc/123:41:331
1585499294,5cyclictest271rcuc/123:41:331
15854992923,4cyclictest25464-21fschecks_count23:58:231
15854992923,4cyclictest25464-21fschecks_count23:58:231
15854992923,3cyclictest0-21swapper/123:28:111
15854992923,3cyclictest0-21swapper/123:28:111
15854992923,3cyclictest0-21swapper/123:28:111
15854992920,7cyclictest22631-21cat23:48:331
15854992920,7cyclictest22631-21cat23:48:331
15854992919,7cyclictest281ktimers/122:44:091
15854992919,7cyclictest281ktimers/122:44:091
15854992919,7cyclictest281ktimers/122:44:091
15854992915,2cyclictest181rcu_preempt22:50:491
15854992915,2cyclictest181rcu_preempt22:50:491
15851992922,5cyclictest19852-21switchtime20:48:320
15851992922,5cyclictest19852-21switchtime20:48:320
52822280,2sleep15336-21fschecks_count21:38:231
52822280,2sleep15336-21fschecks_count21:38:231
52822280,2sleep15336-21fschecks_count21:38:231
220912280,4sleep00-21swapper/022:23:190
220912280,4sleep00-21swapper/022:23:190
1585699281,12cyclictest22252-21taskset22:23:223
1585699281,12cyclictest22252-21taskset22:23:223
15855992822,4cyclictest351rcuc/219:39:012
15855992822,4cyclictest351rcuc/219:39:012
15855992822,4cyclictest351rcuc/219:39:012
1585499289,3cyclictest281ktimers/121:38:471
1585499289,3cyclictest281ktimers/121:38:471
1585499289,3cyclictest281ktimers/121:38:471
1585499288,17cyclictest281ktimers/120:38:491
1585499288,17cyclictest281ktimers/120:38:491
1585499288,17cyclictest281ktimers/119:30:011
1585499288,17cyclictest281ktimers/119:30:011
1585499287,10cyclictest29-21ksoftirqd/119:28:191
1585499287,10cyclictest29-21ksoftirqd/119:28:191
1585499286,5cyclictest281ktimers/122:54:331
1585499286,5cyclictest281ktimers/122:54:331
1585499286,19cyclictest281ktimers/121:14:461
1585499286,19cyclictest281ktimers/121:14:461
1585499282,6cyclictest281ktimers/121:56:011
1585499282,6cyclictest281ktimers/121:56:011
1585499282,6cyclictest281ktimers/121:56:011
15854992824,3cyclictest25087-21idleruntime-cro23:58:111
15854992824,3cyclictest25087-21idleruntime-cro23:58:111
15854992824,3cyclictest24717-21ntp_offset22:28:291
15854992824,3cyclictest24717-21ntp_offset22:28:291
15854992824,3cyclictest17616-21timerwakeupswit22:08:321
15854992824,3cyclictest17616-21timerwakeupswit22:08:321
15854992824,3cyclictest17616-21timerwakeupswit22:08:321
15854992824,3cyclictest15850-21cyclictest22:18:291
15854992824,3cyclictest15850-21cyclictest22:18:291
15854992821,4cyclictest1346-21avahi-daemon00:43:111
15854992821,4cyclictest1346-21avahi-daemon00:43:111
15854992821,4cyclictest1346-21avahi-daemon00:43:111
15854992820,6cyclictest3883-21cron00:28:111
15854992820,6cyclictest3883-21cron00:28:111
15854992818,2cyclictest181rcu_preempt19:58:271
15854992818,2cyclictest181rcu_preempt19:58:271
15851992822,4cyclictest5505-21gltestperf21:38:240
15851992821,4cyclictest0-21swapper/022:03:310
15851992821,4cyclictest0-21swapper/022:03:310
15851992821,4cyclictest0-21swapper/022:03:310
15851992819,7cyclictest18882-21cat23:38:320
15851992819,7cyclictest18882-21cat23:38:320
15851992818,8cyclictest1655-21ntp_kernel_pll_22:53:290
15851992818,8cyclictest1655-21ntp_kernel_pll_22:53:290
1585699276,3cyclictest203882chrt20:51:133
1585699276,3cyclictest203882chrt20:51:133
1585699271,9cyclictest3285-21chrt22:58:243
1585699271,9cyclictest3285-21chrt22:58:243
1585699271,9cyclictest3285-21chrt22:58:243
1585699271,8cyclictest27124-21chrt22:33:363
1585699271,8cyclictest27124-21chrt22:33:363
1585699271,8cyclictest27124-21chrt22:33:363
1585699271,8cyclictest1790-21taskset00:20:293
1585699271,8cyclictest1790-21taskset00:20:293
1585699271,10cyclictest0-21swapper/300:03:303
1585699271,10cyclictest0-21swapper/300:03:303
1585699271,10cyclictest0-21swapper/300:03:303
1585699270,8cyclictest20836-21chrt19:24:263
1585699270,8cyclictest20836-21chrt19:24:263
15854992713,2cyclictest181rcu_preempt22:13:251
15854992713,2cyclictest181rcu_preempt22:13:251
15854992713,2cyclictest181rcu_preempt22:13:251
15851992722,3cyclictest0-21swapper/022:18:210
15851992722,3cyclictest0-21swapper/022:18:210
15851992722,3cyclictest0-21swapper/022:18:210
15851992720,5cyclictest9287-21sshd00:40:120
15851992720,5cyclictest9287-21sshd00:40:120
15851992720,5cyclictest9287-21sshd00:40:120
15851992720,5cyclictest7533-21sshd00:36:550
15851992720,5cyclictest29130-21gltestperf19:48:240
15851992720,5cyclictest29130-21gltestperf19:48:240
15851992720,5cyclictest29130-21gltestperf19:48:240
15851992718,7cyclictest29245-21sort21:13:340
15851992718,7cyclictest29245-21sort21:13:340
1585699262,6cyclictest0-21swapper/300:18:183
1585699261,9cyclictest24313-21chrt23:53:313
1585699261,9cyclictest24313-21chrt23:53:313
1585699261,9cyclictest24313-21chrt23:53:313
1585699261,9cyclictest24313-21chrt23:53:313
1585699261,8cyclictest397-21chrt22:49:403
1585699261,8cyclictest397-21chrt22:49:403
1585499267,8cyclictest29-21ksoftirqd/122:03:451
1585499267,8cyclictest29-21ksoftirqd/122:03:451
1585499267,8cyclictest29-21ksoftirqd/122:03:451
1585499266,9cyclictest29-21ksoftirqd/119:51:311
1585499266,9cyclictest29-21ksoftirqd/119:51:311
1585499266,9cyclictest29-21ksoftirqd/119:51:311
15854992622,3cyclictest26585-21ntp_offset22:33:291
15854992622,3cyclictest26585-21ntp_offset22:33:291
15854992622,3cyclictest26585-21ntp_offset22:33:291
15854992618,2cyclictest181rcu_preempt00:15:511
15854992617,2cyclictest181rcu_preempt20:19:001
15854992617,2cyclictest181rcu_preempt20:19:001
15851992621,3cyclictest0-21swapper/022:38:110
15851992621,3cyclictest0-21swapper/022:38:110
15851992621,3cyclictest0-21swapper/022:38:110
15851992620,3cyclictest0-21swapper/019:38:320
15851992620,3cyclictest0-21swapper/019:38:320
15851992620,3cyclictest0-21swapper/019:38:320
15851992619,5cyclictest25232-21ntp_kernel_pll_21:03:290
15851992619,5cyclictest25232-21ntp_kernel_pll_21:03:290
15851992619,5cyclictest25076-21cron23:58:240
15851992619,5cyclictest25076-21cron23:58:240
15851992619,5cyclictest25076-21cron23:58:240
15851992618,6cyclictest4394-21gltestperf00:28:240
15851992618,6cyclictest4394-21gltestperf00:28:240
15851992617,7cyclictest30670-21cat21:18:250
15851992617,7cyclictest30670-21cat21:18:250
1585699258,2cyclictest0-21swapper/322:32:513
1585699258,2cyclictest0-21swapper/322:32:513
1585699251,9cyclictest24284-21taskset21:03:113
1585699251,9cyclictest24284-21taskset21:03:113
1585699251,9cyclictest1688-21chrt22:53:293
1585699251,9cyclictest1688-21chrt22:53:293
1585699251,8cyclictest10076-21chrt23:17:583
1585699251,8cyclictest10076-21chrt23:17:583
15854992519,4cyclictest0-21swapper/100:08:241
15854992519,4cyclictest0-21swapper/100:08:241
15854992516,2cyclictest181rcu_preempt19:38:361
15854992516,2cyclictest181rcu_preempt19:38:361
15854992516,2cyclictest181rcu_preempt19:38:361
15854992515,3cyclictest181rcu_preempt20:43:311
15854992515,3cyclictest181rcu_preempt20:43:311
15854992512,3cyclictest181rcu_preempt00:37:221
15854992512,3cyclictest181rcu_preempt00:37:221
15851992520,3cyclictest0-21swapper/021:53:330
15851992520,3cyclictest0-21swapper/021:53:330
15851992520,3cyclictest0-21swapper/021:53:330
15851992519,4cyclictest0-21swapper/021:43:240
15851992519,4cyclictest0-21swapper/021:43:240
15851992517,6cyclictest31418-21kernelversion00:13:250
15851992517,6cyclictest20864-21ls20:53:230
15851992517,6cyclictest20864-21ls20:53:230
15851992517,6cyclictest20864-21ls20:53:230
15851992517,6cyclictest10419-21sort23:18:220
15851992517,6cyclictest10419-21sort23:18:220
55042240,2sleep20-21swapper/223:03:292
55042240,2sleep20-21swapper/223:03:292
1585699247,3cyclictest184692chrt20:45:303
1585699247,3cyclictest184692chrt20:45:303
15854992413,2cyclictest181rcu_preempt21:48:531
15854992413,2cyclictest181rcu_preempt21:48:531
15854992413,2cyclictest181rcu_preempt21:48:531
15851992419,4cyclictest15850-21cyclictest23:33:260
15851992419,4cyclictest15850-21cyclictest23:33:260
15851992419,4cyclictest15850-21cyclictest23:33:260
15851992419,3cyclictest0-21swapper/019:33:220
15851992418,5cyclictest23741-21timerwakeupswit20:58:330
15851992418,5cyclictest23741-21timerwakeupswit20:58:330
15851992418,4cyclictest18716-21gltestperf22:13:240
15851992418,4cyclictest15850-21cyclictest22:02:260
15851992418,4cyclictest15850-21cyclictest22:02:260
15851992417,5cyclictest4274-21fschecks_count00:28:240
15851992417,5cyclictest1992-21/usr/sbin/munin20:03:260
15851992417,5cyclictest1992-21/usr/sbin/munin20:03:260
208472230,3sleep20-21swapper/223:43:332
208472230,3sleep20-21swapper/223:43:332
208472230,3sleep20-21swapper/223:43:332
1585699230,8cyclictest11910-21chrt23:22:283
1585699230,8cyclictest11910-21chrt23:22:283
15855992311,8cyclictest270562sleep200:03:162
15855992311,8cyclictest270562sleep200:03:162
15855992311,8cyclictest270562sleep200:03:162
15854992311,3cyclictest90-1pr/ttyS023:10:341
15854992311,3cyclictest90-1pr/ttyS023:10:341
1585199233,3cyclictest181rcu_preempt22:42:480
1585199233,3cyclictest181rcu_preempt22:42:480
15851992318,3cyclictest0-21swapper/023:43:240
15851992318,3cyclictest0-21swapper/023:43:240
15851992318,3cyclictest0-21swapper/023:43:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional