You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 20:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Thu Mar 19, 2026 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4047047995730,23cyclictest4093011-21kworker/u8:209:20:010
4047052994933,13cyclictest231ktimers/112:00:121
4047058994732,11cyclictest391ktimers/311:45:103
4047054994733,10cyclictest311ktimers/211:20:122
4047054994733,10cyclictest311ktimers/211:20:112
404704799471,41cyclictest4072007-21sed08:20:020
4047058994633,10cyclictest4114073-21latency_hist10:20:003
4047058994531,10cyclictest391ktimers/309:00:113
4047058994432,8cyclictest391ktimers/307:35:123
4047058994333,7cyclictest4075891-21fschecks_count08:30:153
404704799431,10cyclictest4074055-21/usr/sbin/munin08:25:160
4047054994233,5cyclictest4061422-21cron07:50:002
4047052994234,4cyclictest4117497-21munin-plugin-st10:30:011
404704799421,8cyclictest4095347-21/usr/sbin/munin09:25:110
404704799421,8cyclictest4083133-21/usr/sbin/munin08:50:170
404704799421,8cyclictest4083133-21/usr/sbin/munin08:50:170
404704799421,38cyclictest4105925-21fschecks_count09:55:150
404705499411,5cyclictest131rcu_preempt07:15:132
404705499411,5cyclictest131rcu_preempt07:15:132
4047047994133,5cyclictest4127944-21sendmail11:00:010
404704799411,9cyclictest4141933-21diskstats11:40:130
404704799411,9cyclictest4109533-21/usr/sbin/munin10:05:160
4047047994119,18cyclictest0-21swapper/010:35:240
404704799411,8cyclictest4065381-21/usr/sbin/munin08:00:120
404704799411,8cyclictest4065381-21/usr/sbin/munin08:00:120
404704799411,12cyclictest4053982-21head07:25:280
4047058994028,9cyclictest391ktimers/311:35:103
4047054994030,6cyclictest4146221-21unixbench_multi11:50:252
404704799409,25cyclictest4068633-21latency_hist08:10:010
404704799408,8cyclictest4052085-21expr07:20:250
404704799403,5cyclictest4085657-21sshd08:56:450
404704799403,5cyclictest4085657-21sshd08:56:450
404704799403,4cyclictest4136700-21apt-get11:25:120
404704799402,9cyclictest4104967-21sed09:50:230
404704799401,9cyclictest4132409-21/usr/sbin/munin11:10:220
404704799401,9cyclictest4108123-21ps10:00:240
404704799401,8cyclictest4118158-21/usr/sbin/munin10:30:180
404704799401,7cyclictest4111185-21/usr/sbin/munin10:10:140
404704799401,6cyclictest4126048-21cron10:53:010
404704799401,6cyclictest4126048-21cron10:53:000
404704799401,10cyclictest4141600-21awk11:40:010
404704799401,10cyclictest4100586-21diskstats09:40:140
4047058993931,4cyclictest0-21swapper/312:05:003
4047054993930,4cyclictest0-21swapper/210:25:002
4047054993922,6cyclictest4062745-21kworker/u8:3+events_unbound08:10:312
404704799393,4cyclictest4058260-21apt-get07:40:120
404704799391,7cyclictest4079964-21/usr/sbin/munin08:40:230
404704799391,7cyclictest4048668-21cron07:15:000
404704799391,7cyclictest4048668-21cron07:15:000
404704799391,6cyclictest4148745-21df12:00:140
404704799391,6cyclictest4136071-21users11:20:250
404704799391,6cyclictest4136071-21users11:20:250
404704799391,6cyclictest4116344-21if_eth110:25:160
404704799391,6cyclictest4057306-21/usr/sbin/munin07:35:230
404704799391,5cyclictest4161198-21ps12:35:210
404704799391,11cyclictest4138958-21ntp_states11:30:190
404704799391,11cyclictest4114425-21df10:20:130
404704799391,11cyclictest4102519-21fschecks_count09:45:160
404704799391,11cyclictest4077759-21diskstats08:35:150
404704799391,10cyclictest4098748-21apt-get09:35:100
404704799391,10cyclictest4064849-21dpkg08:00:010
404704799391,10cyclictest4064849-21dpkg08:00:000
4047058993830,5cyclictest4156532-21unixbench_singl12:20:233
4047058993827,7cyclictest4145752-21grep11:50:193
4047054993828,6cyclictest4088929-21grep09:05:262
4047052993829,6cyclictest4141588-21sendmail11:40:001
4047052993827,8cyclictest4159934-21grep12:30:241
404704799383,4cyclictest4157170-21apt-get12:25:100
404704799383,4cyclictest4122213-21users10:40:230
404704799382,5cyclictest4114063-21cron10:20:000
404704799382,5cyclictest4097383-21irqstats09:30:170
404704799382,3cyclictest4088457-21ntp_states09:05:230
404704799381,7cyclictest4156471-21unixbench_multi12:20:230
404704799381,6cyclictest4152160-21diskstats12:10:140
404704799381,6cyclictest4134035-21/usr/sbin/munin11:15:240
404704799381,6cyclictest4134035-21/usr/sbin/munin11:15:240
404704799381,6cyclictest4081401-21irqstats08:45:180
404704799381,32cyclictest4070287-21latency_hist08:15:010
404704799381,10cyclictest4159432-21ntp_states12:30:190
4047054993729,4cyclictest4085938-21cron09:00:002
4047054993729,4cyclictest4085938-21cron09:00:002
4047054993729,4cyclictest0-21swapper/210:10:002
4047054993727,7cyclictest4158640-21auditctl12:30:002
4047052993729,4cyclictest0-21swapper/107:25:001
4047052993728,6cyclictest0-21swapper/107:45:001
404704799373,3cyclictest4060075-21df_inode07:45:150
404704799372,12cyclictest4093558-21diskstats09:20:150
404704799371,9cyclictest4076173-21meminfo08:30:180
404704799371,6cyclictest4123621-21ntp_states10:45:210
404704799371,5cyclictest4128382-21grep11:00:140
404704799371,5cyclictest4072321-21tune2fs08:20:140
404704799371,3cyclictest4147035-21df_inode11:55:140
404704799371,10cyclictest4155209-21cron12:20:000
4047058993628,4cyclictest0-21swapper/312:35:153
4047054993627,4cyclictest0-21swapper/208:00:262
4047054993627,4cyclictest0-21swapper/208:00:252
404705499361,5cyclictest0-21swapper/209:50:112
4047052993628,4cyclictest0-21swapper/109:00:271
4047052993626,7cyclictest4108564-21grep10:00:251
4047052993624,9cyclictest231ktimers/109:50:111
4047052993624,8cyclictest231ktimers/108:00:121
4047052993624,8cyclictest231ktimers/108:00:111
404704799361,9cyclictest4145222-21/usr/sbin/munin11:50:120
404704799361,6cyclictest4150430-21/usr/sbin/munin12:05:120
404704799361,31cyclictest0-21swapper/011:10:010
404704799361,10cyclictest4049514-21apt-get07:15:120
404704799361,10cyclictest4049514-21apt-get07:15:120
4047058993528,4cyclictest0-21swapper/308:05:153
4047058993525,7cyclictest438-21cron10:00:003
4047054993528,3cyclictest0-21swapper/208:10:002
404705499351,5cyclictest131rcu_preempt12:10:112
4047052993527,4cyclictest0-21swapper/108:00:001
4047052993527,4cyclictest0-21swapper/108:00:001
4047052993526,4cyclictest0-21swapper/110:10:231
404704799352,8cyclictest4055628-21sed07:30:220
404704799352,5cyclictest4086948-21/usr/sbin/munin09:00:260
404704799351,9cyclictest4062636-21ls07:50:250
404704799351,5cyclictest4047070-21latency_hist07:10:010
4047058993427,4cyclictest0-21swapper/307:45:253
4047058993424,7cyclictest4134321-21grep11:15:263
4047058993424,7cyclictest4134321-21grep11:15:253
4047054993428,3cyclictest0-21swapper/211:00:002
4047054993427,3cyclictest0-21swapper/211:10:002
4047054993426,4cyclictest0-21swapper/209:15:252
4047052993427,4cyclictest0-21swapper/109:10:141
4047052993420,10cyclictest231ktimers/111:25:121
4047058993322,7cyclictest0-21swapper/310:50:263
4047058993322,7cyclictest0-21swapper/310:50:263
4047054993325,4cyclictest0-21swapper/210:50:172
4047054993325,4cyclictest0-21swapper/210:50:172
4047054993324,5cyclictest0-21swapper/212:30:232
4047054993324,4cyclictest0-21swapper/208:40:252
4047054993322,8cyclictest311ktimers/210:30:122
404705299333,5cyclictest131rcu_preempt08:25:111
4047052993326,5cyclictest4142972-21kworker/u8:2+events_unbound12:10:591
4047052993325,4cyclictest0-21swapper/108:40:161
4047052993325,4cyclictest0-21swapper/108:40:011
404705299332,4cyclictest131rcu_preempt12:20:091
4047058993222,7cyclictest4054464-21apt-config07:30:013
404705499327,5cyclictest0-21swapper/208:30:122
404705499326,3cyclictest131rcu_preempt12:15:112
4047054993224,4cyclictest4162049-21cron12:40:002
4047054993220,8cyclictest4073165-21grep08:20:242
4047052993223,5cyclictest0-21swapper/110:55:011
4047052993223,5cyclictest0-21swapper/110:55:001
4047052993220,7cyclictest0-21swapper/108:15:241
404705299321,5cyclictest131rcu_preempt09:20:131
4047058993125,4cyclictest4120893-21cron10:40:013
4047058993121,8cyclictest4108967-21auditctl10:05:013
4047054993124,4cyclictest4054450-21apt-key07:30:002
4047054993123,4cyclictest0-21swapper/210:15:252
4047054993122,6cyclictest0-21swapper/210:10:242
4047054993023,3cyclictest0-21swapper/212:00:002
4047054993022,6cyclictest4098294-21kworker/u8:3+events_unbound11:10:532
4047052993022,5cyclictest0-21swapper/108:10:251
4047052993022,4cyclictest0-21swapper/112:35:231
4047052993022,4cyclictest0-21swapper/109:35:241
4047052993022,4cyclictest0-21swapper/108:05:251
4047058992922,4cyclictest0-21swapper/307:40:153
4047058992920,6cyclictest391ktimers/309:40:123
4047054992921,6cyclictest4093856-21latency09:20:192
4047054992921,4cyclictest0-21swapper/208:35:252
4047054992920,6cyclictest4058417-21tune2fs07:40:162
4047054992920,4cyclictest0-21swapper/211:40:002
4047054992918,7cyclictest1-21systemd11:15:522
4047054992918,7cyclictest1-21systemd11:15:522
4047054992916,8cyclictest204-21dbus-daemon08:50:382
4047054992916,8cyclictest204-21dbus-daemon08:50:372
4047052992921,4cyclictest0-21swapper/111:00:001
4047052992920,4cyclictest0-21swapper/110:15:171
4047052992919,7cyclictest4055071-21grep07:30:141
4047052992916,9cyclictest438-21cron08:50:001
4047058992822,3cyclictest0-21swapper/308:30:013
4047058992821,4cyclictest0-21swapper/307:35:013
4047058992821,3cyclictest0-21swapper/310:45:003
4047058992820,4cyclictest0-21swapper/309:45:163
404705499288,7cyclictest32-21ksoftirqd/207:15:022
404705499288,7cyclictest32-21ksoftirqd/207:15:012
4047052992819,6cyclictest0-21swapper/110:45:001
4047052992819,5cyclictest0-21swapper/112:30:011
4047052992817,7cyclictest4101967-21cat09:45:001
4047058992720,4cyclictest0-21swapper/308:55:273
4047058992720,4cyclictest0-21swapper/308:55:273
4047058992720,3cyclictest0-21swapper/308:10:223
4047058992719,5cyclictest0-21swapper/312:00:003
4047058992718,6cyclictest0-21swapper/309:10:013
4047054992717,7cyclictest311ktimers/210:35:112
4047052992721,4cyclictest4132200-21kworker/u8:0+events_unbound12:07:561
4047052992720,4cyclictest0-21swapper/107:50:251
4047052992720,3cyclictest0-21swapper/109:15:241
4047052992719,4cyclictest0-21swapper/110:35:141
4047052992717,6cyclictest4118762-21grep10:30:241
4047058992620,3cyclictest0-21swapper/309:50:153
4047058992619,4cyclictest0-21swapper/309:10:253
4047058992618,4cyclictest0-21swapper/308:00:283
4047058992618,4cyclictest0-21swapper/308:00:283
4047058992617,6cyclictest0-21swapper/311:00:013
4047058992617,6cyclictest0-21swapper/309:20:013
4047058992615,6cyclictest0-21swapper/309:30:243
404705499262,3cyclictest0-21swapper/209:40:122
4047054992620,3cyclictest0-21swapper/209:05:002
4047054992619,3cyclictest0-21swapper/209:10:242
4047054992618,6cyclictest4077000-21kworker/u8:1+events_unbound09:27:582
4047054992617,4cyclictest0-21swapper/209:45:272
4047054992613,9cyclictest4145002-21basename11:50:012
4047052992618,4cyclictest0-21swapper/109:45:271
4047052992616,6cyclictest0-21swapper/107:30:001
404705299261,5cyclictest131rcu_preempt10:05:111
4047058992519,3cyclictest0-21swapper/307:25:003
4047058992518,4cyclictest4094496-21grep09:20:263
4047058992518,4cyclictest0-21swapper/311:00:233
4047058992517,6cyclictest4151638-21kworker/u8:1+events_unbound12:15:023
4047058992517,5cyclictest0-21swapper/311:05:243
4047058992514,8cyclictest4051005-21tr07:20:003
4047058992514,8cyclictest4051005-21tr07:20:003
4047054992518,4cyclictest4071849-21kworker/u8:2+events_unbound08:27:392
4047052992519,3cyclictest0-21swapper/111:45:001
4047052992517,6cyclictest4048045-21sed07:10:251
4047052992517,6cyclictest4048045-21sed07:10:251
4047052992517,5cyclictest0-21swapper/111:10:011
4047052992517,4cyclictest0-21swapper/111:00:231
4047052992516,5cyclictest4088932-21unixbench_singl09:05:261
4047052992515,6cyclictest0-21swapper/108:30:251
4047058992417,4cyclictest0-21swapper/312:05:143
4047058992416,5cyclictest0-21swapper/309:35:253
4047058992416,4cyclictest0-21swapper/307:10:273
4047058992416,4cyclictest0-21swapper/307:10:263
404705499248,9cyclictest131rcu_preempt08:00:002
404705499248,9cyclictest131rcu_preempt08:00:002
4047054992418,4cyclictest4115748-21kworker/u8:2+events_unbound10:47:552
4047054992417,5cyclictest4115748-21kworker/u8:2+events_unbound10:26:472
4047054992417,3cyclictest0-21swapper/211:40:202
4047054992413,6cyclictest0-21swapper/208:15:242
404705499240,13cyclictest0-21swapper/209:58:282
4047052992418,4cyclictest4146345-21kworker/u8:3+events_unbound12:16:341
4047052992418,3cyclictest0-21swapper/109:55:231
4047052992417,5cyclictest4098294-21kworker/u8:3+events_unbound11:20:191
4047052992417,5cyclictest4098294-21kworker/u8:3+events_unbound11:20:191
4047052992417,4cyclictest0-21swapper/109:30:231
4047052992417,4cyclictest0-21swapper/109:25:261
4047052992413,8cyclictest1-21systemd08:55:391
4047052992413,8cyclictest1-21systemd08:55:381
4047052992413,6cyclictest0-21swapper/111:45:251
404705899237,12cyclictest391ktimers/311:10:113
4047058992317,3cyclictest0-21swapper/310:05:253
4047058992316,4cyclictest4117033-21tail10:25:233
4047058992316,3cyclictest0-21swapper/308:15:163
4047058992311,9cyclictest4135567-21ntp_states11:20:223
4047058992311,9cyclictest4135567-21ntp_states11:20:213
4047054992317,3cyclictest0-21swapper/210:05:012
4047054992315,6cyclictest4068121-21kworker/u8:0+events_unbound08:45:212
4047054992315,5cyclictest0-21swapper/207:25:002
404705299234,3cyclictest131rcu_preempt08:20:231
4047052992318,3cyclictest0-21swapper/111:10:241
4047052992315,4cyclictest0-21swapper/108:50:151
4047052992315,4cyclictest0-21swapper/108:50:151
4047058992216,4cyclictest4068121-21kworker/u8:0-writeback08:50:163
4047058992216,4cyclictest4068121-21kworker/u8:0-writeback08:50:153
4047058992216,3cyclictest4138176-21basename11:29:593
4047058992216,3cyclictest0-21swapper/312:30:183
4047058992216,3cyclictest0-21swapper/308:40:253
4047058992216,3cyclictest0-21swapper/308:35:163
4047058992215,4cyclictest0-21swapper/310:45:243
4047058992215,4cyclictest0-21swapper/310:20:153
4047058992214,4cyclictest0-21swapper/310:30:253
4047058992213,7cyclictest4081232-21cut08:45:173
4047058992213,6cyclictest0-21swapper/311:45:013
4047058992212,7cyclictest4158205-21awk12:25:233
4047054992216,4cyclictest4062745-21kworker/u8:3+events_unbound07:53:142
4047054992216,3cyclictest0-21swapper/210:40:172
4047054992214,5cyclictest0-21swapper/212:00:252
4047054992212,6cyclictest0-21swapper/212:20:142
4047052992216,3cyclictest0-21swapper/111:30:241
4047052992216,3cyclictest0-21swapper/107:45:211
4047058992115,3cyclictest0-21swapper/310:10:203
4047058992115,3cyclictest0-21swapper/308:20:243
4047058992113,5cyclictest0-21swapper/311:30:143
404705499213,4cyclictest131rcu_preempt11:30:092
4047054992115,4cyclictest4098294-21kworker/u8:3+events_unbound11:00:242
4047054992115,3cyclictest0-21swapper/211:25:232
4047054992113,4cyclictest4151809-21sh12:10:002
404705299214,4cyclictest131rcu_preempt10:45:111
4047052992116,3cyclictest0-21swapper/107:15:261
4047052992116,3cyclictest0-21swapper/107:15:261
4047052992114,4cyclictest4057072-21ntpq07:35:201
404705299211,3cyclictest131rcu_preempt11:50:231
4047058992013,4cyclictest4096827-21munin-plugin-st09:30:003
4047052992014,4cyclictest4114318-21/usr/sbin/munin10:20:161
4047058991913,3cyclictest0-21swapper/312:10:163
4047058991912,5cyclictest391ktimers/307:50:143
4047054991913,4cyclictest2383435-1kworker/2:0H+kblockd09:40:012
4047054991913,3cyclictest0-21swapper/207:35:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional