You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 10:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu Nov 20, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2089537995243,7cyclictest231ktimers/119:55:071
2089537995033,13cyclictest231ktimers/120:25:131
2089537994842,4cyclictest231ktimers/123:00:121
2089546994638,5cyclictest2132217-21munin-plugin-st21:40:003
2089537994539,4cyclictest2087661-21kworker/u8:0+events_unbound19:10:401
2089537994534,7cyclictest2163089-21grep23:05:241
2089537994427,6cyclictest131rcu_preempt20:50:011
2089546994336,4cyclictest2103196-21cron20:15:003
2089546994336,4cyclictest2103196-21cron20:14:593
2089541994234,5cyclictest2137309-21kworker/u8:2+events_unbound22:06:082
2089541994129,10cyclictest311ktimers/219:55:082
2089537994133,6cyclictest231ktimers/122:30:121
2089537994132,6cyclictest2118581-21sendmail21:00:011
2089533994132,6cyclictest53-21kcompactd000:20:010
2089533994033,5cyclictest0-21swapper/022:30:130
2089537993932,5cyclictest2130214-21kworker/u8:0+events_unbound21:36:061
2089537993919,10cyclictest131rcu_preempt22:10:191
2089546993829,6cyclictest2176764-21unixbench_multi23:45:303
2089546993828,6cyclictest2119842-21unixbench_singl21:00:263
2089537993831,5cyclictest2099461-21kworker/u8:0+events_unbound20:05:461
2089546993726,7cyclictest0-21swapper/320:40:003
2089541993726,7cyclictest0-21swapper/200:25:262
208953799374,15cyclictest24-21ksoftirqd/123:35:111
2089537993715,11cyclictest0-21swapper/121:24:351
2089546993626,8cyclictest2127101-21cron21:25:003
2089537993629,5cyclictest2183854-21kworker/u8:0+events_unbound00:13:161
2089537993626,7cyclictest2137328-21sh21:55:001
2089537993624,8cyclictest2183603-21grep00:05:251
2089537993613,12cyclictest2094789-21kworker/u8:2-events_freezable_power_19:48:541
2089533993628,4cyclictest0-21swapper/020:45:010
2089533993625,7cyclictest2104907-21sh20:20:000
2089541993526,5cyclictest0-21swapper/223:40:222
2089541993523,7cyclictest0-21swapper/221:45:002
2089541993523,7cyclictest0-21swapper/221:45:002
2089537993528,5cyclictest2144234-21kworker/u8:1+events_unbound22:16:101
2089537993524,8cyclictest231ktimers/123:20:111
2089537993524,8cyclictest231ktimers/121:10:141
2089533993527,4cyclictest0-21swapper/022:40:000
2089546993426,4cyclictest0-21swapper/322:05:253
2089546993426,4cyclictest0-21swapper/300:28:553
2089546993420,10cyclictest391ktimers/322:00:153
2089541993426,4cyclictest0-21swapper/220:50:012
2089541993426,4cyclictest0-21swapper/200:23:542
2089541993425,5cyclictest0-21swapper/220:15:182
2089541993423,7cyclictest0-21swapper/200:10:002
2089541993421,9cyclictest2121988-21diskmemload21:10:012
2089537993428,4cyclictest2180314-21kworker/u8:1+events_unbound00:03:441
2089537993425,6cyclictest2147779-21cron22:25:001
2089537993421,5cyclictest131rcu_preempt22:35:231
2089537993421,5cyclictest131rcu_preempt19:15:031
2089537993417,5cyclictest2135580-21kworker/u8:0-events_freezable_power_22:01:471
2089537993411,14cyclictest131rcu_preempt00:15:161
2089533993427,4cyclictest2125394-21cron21:20:000
2089533993426,4cyclictest0-21swapper/023:15:250
2089546993327,3cyclictest0-21swapper/322:00:013
2089546993325,4cyclictest0-21swapper/322:40:003
2089546993325,4cyclictest0-21swapper/322:30:263
2089546993325,4cyclictest0-21swapper/300:10:253
2089546993324,6cyclictest2188701-21unixbench_multi00:20:243
2089541993325,5cyclictest2118570-21idleruntime-cro21:00:012
2089537993326,4cyclictest0-21swapper/120:50:281
2089537993321,5cyclictest131rcu_preempt21:57:011
2089537993311,11cyclictest24-21ksoftirqd/121:30:151
2089537993310,6cyclictest24-21ksoftirqd/123:45:011
2089546993225,3cyclictest0-21swapper/323:00:003
2089546993225,3cyclictest0-21swapper/319:30:003
2089541993224,4cyclictest0-21swapper/220:05:012
2089541993221,7cyclictest2154821-21auditctl22:45:012
2089537993224,4cyclictest0-21swapper/122:45:251
2089537993222,6cyclictest0-21swapper/120:10:281
2089537993222,6cyclictest0-21swapper/120:10:281
2089537993220,5cyclictest131rcu_preempt20:40:191
2089537993216,5cyclictest131rcu_preempt23:50:151
2089537993214,11cyclictest0-21swapper/119:30:151
2089533993224,5cyclictest2142671-21ls22:10:000
2089533993223,6cyclictest0-21swapper/022:25:010
2089546993124,5cyclictest2131973-21kworker/u8:2+events_unbound21:46:053
2089546993124,4cyclictest0-21swapper/322:25:243
2089546993122,6cyclictest391ktimers/320:05:123
2089546993120,8cyclictest2187453-21awk00:20:013
2089546993120,7cyclictest2104488-21grep20:15:273
2089546993118,10cyclictest2089524-21cyclictest20:55:093
2089541993123,4cyclictest0-21swapper/220:30:292
2089541993122,5cyclictest0-21swapper/222:55:272
2089541993118,9cyclictest2147321-21grep22:20:242
2089537993121,4cyclictest131rcu_preempt20:30:201
2089537993116,4cyclictest131rcu_preempt00:35:211
2089537993115,9cyclictest131rcu_preempt23:28:111
2089537993115,9cyclictest131rcu_preempt00:30:211
2089537993115,5cyclictest131rcu_preempt22:10:011
2089537993115,13cyclictest231ktimers/120:00:361
2089537993114,9cyclictest131rcu_preempt22:55:201
2089537993112,12cyclictest24-21ksoftirqd/120:20:271
2089537993112,12cyclictest24-21ksoftirqd/120:20:261
2089537993112,11cyclictest131rcu_preempt22:26:131
2089537993111,5cyclictest131rcu_preempt21:47:441
2089537993111,5cyclictest131rcu_preempt20:35:321
2089533993122,4cyclictest0-21swapper/019:35:420
2089533993121,6cyclictest2187900-21grep00:20:160
2089533993121,6cyclictest2165268-21grep23:15:010
208954699306,3cyclictest21662422cat23:15:243
208954699302,5cyclictest131rcu_preempt22:10:093
2089546993020,6cyclictest0-21swapper/322:19:593
2089546993020,6cyclictest0-21swapper/320:20:283
2089546993020,6cyclictest0-21swapper/320:20:283
2089541993022,4cyclictest0-21swapper/223:45:302
2089541993022,4cyclictest0-21swapper/222:45:162
2089541993022,4cyclictest0-21swapper/221:40:002
208953799308,12cyclictest24-21ksoftirqd/123:55:201
208953799308,12cyclictest24-21ksoftirqd/123:55:201
208953799303,8cyclictest24-21ksoftirqd/121:30:001
2089537993024,3cyclictest0-21swapper/122:45:011
208953799301,9cyclictest24-21ksoftirqd/123:10:121
2089537993018,4cyclictest131rcu_preempt19:23:151
2089537993018,4cyclictest131rcu_preempt19:23:151
2089537993018,2cyclictest131rcu_preempt21:04:041
2089537993013,3cyclictest131rcu_preempt21:07:011
2089537993012,9cyclictest131rcu_preempt23:18:461
2089537993012,3cyclictest131rcu_preempt00:25:021
2089537993011,5cyclictest131rcu_preempt21:40:101
2089537993011,5cyclictest131rcu_preempt21:40:101
2089537993010,5cyclictest131rcu_preempt21:15:181
2089546992918,8cyclictest0-21swapper/323:35:003
2089541992921,4cyclictest0-21swapper/222:15:172
208953799299,5cyclictest131rcu_preempt22:50:241
208953799299,5cyclictest131rcu_preempt22:50:231
2089537992921,6cyclictest2176308-21kworker/u8:2+events_unbound23:47:371
2089537992918,4cyclictest131rcu_preempt19:37:591
2089537992916,4cyclictest131rcu_preempt23:30:121
2089537992915,5cyclictest131rcu_preempt19:27:511
208953799291,12cyclictest0-21swapper/120:15:151
2089533992923,3cyclictest2092584-21kthreadd19:30:000
2089533992919,7cyclictest121ktimers/021:50:140
2089533992919,6cyclictest2193873-21grep00:35:260
2089541992821,5cyclictest2102930-21kworker/u8:0+events_unbound20:20:482
2089541992821,5cyclictest2102930-21kworker/u8:0+events_unbound20:20:482
2089541992821,4cyclictest2187447-21munin-plugin-st00:20:012
2089541992821,4cyclictest2091071-21cron19:17:002
2089541992820,4cyclictest0-21swapper/223:35:242
2089541992820,4cyclictest0-21swapper/223:10:012
2089541992820,4cyclictest0-21swapper/200:35:002
2089541992819,5cyclictest0-21swapper/221:10:182
2089541992818,6cyclictest0-21swapper/219:15:012
2089541992817,6cyclictest0-21swapper/222:36:122
2089537992810,11cyclictest24-21ksoftirqd/119:51:591
2089533992821,4cyclictest2097898-21sendmail20:00:010
2089533992820,5cyclictest2125377-21kworker/u8:0+events_unbound21:21:010
2089533992818,7cyclictest0-21swapper/022:05:010
2089533992818,6cyclictest0-21swapper/019:15:010
2089546992719,5cyclictest2093498-21kworker/u8:0+events_unbound19:45:443
2089546992719,5cyclictest0-21swapper/323:35:243
2089546992719,5cyclictest0-21swapper/319:15:023
2089546992717,6cyclictest0-21swapper/300:05:003
2089541992720,5cyclictest2097711-21kworker/u8:2+events_unbound20:50:552
2089541992718,6cyclictest2166959-21sh23:20:012
2089541992716,7cyclictest2119793-21grep21:00:252
208953799276,5cyclictest24-21ksoftirqd/100:20:191
2089533992720,3cyclictest0-21swapper/019:15:030
2089546992618,5cyclictest0-21swapper/321:25:193
2089546992617,6cyclictest0-21swapper/323:50:283
2089546992617,6cyclictest0-21swapper/322:50:243
2089546992617,6cyclictest0-21swapper/322:50:243
2089546992615,7cyclictest0-21swapper/319:57:123
2089541992620,4cyclictest2123625-21kworker/u8:3+events_unbound21:27:382
2089541992620,4cyclictest2099461-21kworker/u8:0+events_unbound20:05:122
2089541992620,4cyclictest2091587-21kworker/u8:1+events_unbound19:21:162
2089541992620,4cyclictest2091587-21kworker/u8:1+events_unbound19:21:162
2089541992618,4cyclictest0-21swapper/220:10:282
2089541992618,4cyclictest0-21swapper/220:10:282
2089541992617,6cyclictest0-21swapper/223:15:012
2089537992611,4cyclictest131rcu_preempt19:41:561
2089533992618,5cyclictest0-21swapper/000:00:270
208953399261,22cyclictest2134021-21sed21:45:010
208953399261,22cyclictest2134021-21sed21:45:000
208953399261,22cyclictest0-21swapper/022:45:120
2089546992518,4cyclictest2093833-21munin-run19:40:013
2089546992517,5cyclictest0-21swapper/321:20:013
2089546992517,5cyclictest0-21swapper/320:45:253
2089546992516,6cyclictest0-21swapper/321:50:263
2089546992516,6cyclictest0-21swapper/320:05:013
2089546992516,5cyclictest0-21swapper/321:10:273
2089546992515,6cyclictest0-21swapper/323:55:253
2089546992515,6cyclictest0-21swapper/323:55:253
2089541992518,5cyclictest2094153-21kworker/u8:1+events_unbound19:46:362
2089541992518,3cyclictest0-21swapper/221:55:222
2089541992517,6cyclictest2160063-21kworker/u8:1+events_unbound23:01:002
2089541992517,4cyclictest0-21swapper/222:00:272
2089541992516,6cyclictest0-21swapper/222:10:132
2089541992516,5cyclictest0-21swapper/223:50:282
2089541992515,7cyclictest2171179-21ntp_states23:30:222
2089541992515,6cyclictest311ktimers/222:30:012
208954199251,3cyclictest2095320-21apt-get19:50:082
2089533992520,2cyclictest13950irq/36-mmc122:00:010
2089533992519,3cyclictest0-21swapper/020:35:250
2089533992518,4cyclictest2171652-21unixbench_singl23:30:260
2089533992517,4cyclictest0-21swapper/021:05:260
2089533992516,5cyclictest0-21swapper/022:25:240
2089546992418,3cyclictest0-21swapper/320:40:223
2089546992418,3cyclictest0-21swapper/319:15:013
2089546992417,4cyclictest0-21swapper/319:51:563
2089546992416,4cyclictest0-21swapper/300:40:003
2089546992415,6cyclictest0-21swapper/323:05:153
2089546992413,8cyclictest1-21systemd23:41:213
208954199247,8cyclictest131rcu_preempt00:00:152
2089541992418,4cyclictest2091587-21kworker/u8:1+events_unbound19:25:412
2089541992418,3cyclictest0-21swapper/223:20:232
208954199240,4cyclictest131rcu_preempt21:50:132
2089533992418,3cyclictest0-21swapper/023:40:000
2089533992417,3cyclictest0-21swapper/020:29:130
2089533992417,3cyclictest0-21swapper/000:15:010
2089533992415,7cyclictest2116470-1kworker/0:0H+mmc_complete22:10:120
2089533992412,9cyclictest440-21cron20:09:590
208953399241,19cyclictest0-21swapper/021:14:410
208953399241,19cyclictest0-21swapper/020:49:570
208953399241,19cyclictest0-21swapper/019:49:040
208953399241,19cyclictest0-21swapper/000:32:290
208953399240,20cyclictest0-21swapper/021:35:010
2089546992317,3cyclictest0-21swapper/323:25:013
2089546992315,4cyclictest0-21swapper/322:40:273
2089546992314,6cyclictest0-21swapper/320:50:283
208954699231,3cyclictest0-21swapper/300:31:153
2089546992313,7cyclictest0-21swapper/321:05:263
208954199239,3cyclictest131rcu_preempt20:35:232
208954199237,8cyclictest131rcu_preempt21:15:132
2089541992316,4cyclictest2179353-21fschecks_time23:55:162
2089541992316,4cyclictest2179353-21fschecks_time23:55:152
2089541992316,3cyclictest0-21swapper/222:50:192
2089541992316,3cyclictest0-21swapper/222:50:192
2089541992315,5cyclictest0-21swapper/220:40:262
2089541992314,5cyclictest0-21swapper/222:30:172
2089541992310,8cyclictest311ktimers/221:45:102
2089533992317,3cyclictest0-21swapper/021:25:280
2089533992316,3cyclictest0-21swapper/023:54:290
2089533992316,3cyclictest0-21swapper/023:45:320
2089533992314,5cyclictest0-21swapper/023:00:000
208953399231,18cyclictest0-21swapper/023:42:310
208953399231,18cyclictest0-21swapper/023:06:220
208953399231,18cyclictest0-21swapper/019:51:370
208953399231,18cyclictest0-21swapper/019:42:360
208953399231,18cyclictest0-21swapper/019:33:540
208953399231,18cyclictest0-21swapper/019:20:170
208953399231,18cyclictest0-21swapper/019:20:160
208953399231,18cyclictest0-21swapper/000:06:100
208953399230,19cyclictest0-21swapper/023:01:520
208953399230,19cyclictest0-21swapper/021:47:200
208953399230,19cyclictest0-21swapper/021:33:230
208953399230,19cyclictest0-21swapper/021:03:030
2089546992216,3cyclictest0-21swapper/300:05:213
2089546992215,3cyclictest0-21swapper/322:25:003
2089546992214,5cyclictest0-21swapper/320:29:593
2089546992213,5cyclictest0-21swapper/320:30:293
208954199228,3cyclictest131rcu_preempt23:25:212
208954199225,6cyclictest131rcu_preempt00:35:252
2089541992216,3cyclictest0-21swapper/219:45:012
2089541992214,4cyclictest0-21swapper/221:30:232
2089541992214,4cyclictest0-21swapper/200:10:162
2089541992210,9cyclictest32-21ksoftirqd/220:25:232
2089533992217,3cyclictest0-21swapper/020:50:140
2089533992215,3cyclictest0-21swapper/023:23:020
2089533992214,5cyclictest0-21swapper/000:26:460
208953399221,3cyclictest21569532sleep022:50:040
208953399221,3cyclictest21569532sleep022:50:040
208953399221,19cyclictest0-21swapper/022:17:160
208953399221,19cyclictest0-21swapper/020:31:590
208953399221,17cyclictest0-21swapper/020:56:180
208954699219,9cyclictest2094486-21basename19:45:013
2089546992113,6cyclictest391ktimers/323:10:203
2089546992113,5cyclictest0-21swapper/321:45:003
2089546992113,5cyclictest0-21swapper/321:45:003
2089541992114,5cyclictest2091587-21kworker/u8:1+events_unbound19:31:222
2089541992113,3cyclictest0-21swapper/221:20:222
2089533992114,4cyclictest0-21swapper/000:00:010
2089533992114,4cyclictest0-21swapper/000:00:010
2089533992114,3cyclictest0-21swapper/023:26:440
2089533992114,3cyclictest0-21swapper/020:21:310
2089533992114,3cyclictest0-21swapper/020:21:310
2089541992014,4cyclictest2090923-21kworker/u8:2+events_unbound19:39:292
208954699191,9cyclictest2129764-21/usr/sbin/munin21:30:223
2089546991813,3cyclictest391ktimers/319:20:323
2089546991813,3cyclictest391ktimers/319:20:323
208954699181,13cyclictest2093192-21cron19:35:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional