You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 08:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu Dec 18, 2025 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
612674995627,24cyclictest636872-21grep20:15:261
612669995541,10cyclictest121ktimers/020:40:130
612682995448,4cyclictest698632-21kworker/u8:3+events_unbound23:15:533
612674995238,11cyclictest231ktimers/120:55:111
612678995144,4cyclictest632196-21munin-plugin-st20:05:012
612682994943,4cyclictest661023-21kworker/u8:1+events_unbound21:55:373
612682994937,5cyclictest689158-21kworker/u8:0+events_unbound23:00:503
612674994923,22cyclictest0-21swapper/100:00:261
612682994824,7cyclictest131rcu_preempt00:20:113
612669994639,4cyclictest673768-21auditctl22:05:000
612682994533,5cyclictest642050-21kworker/u8:0+events_unbound20:55:283
612669994433,8cyclictest685922-21sendmail22:40:000
61267499437,6cyclictest626372-21/usr/sbin/munin19:45:261
61267499437,6cyclictest626372-21/usr/sbin/munin19:45:261
612674994325,14cyclictest0-21swapper/119:55:001
612674994325,14cyclictest0-21swapper/119:55:001
612682994236,4cyclictest620721-21kworker/u8:2+events_unbound19:50:173
612682994236,4cyclictest620721-21kworker/u8:2+events_unbound19:50:173
61267499424,4cyclictest0-21swapper/120:50:171
61267499424,4cyclictest0-21swapper/120:50:171
612674994213,27cyclictest658230-21/usr/sbin/munin21:20:161
612669994231,7cyclictest647332-21grep20:45:250
612669994231,7cyclictest647332-21grep20:45:250
612682994133,6cyclictest676902-21kworker/u8:3+events_unbound22:15:423
612674994130,7cyclictest685925-21sendmail22:40:001
61267499411,9cyclictest678483-21users22:15:261
61267499411,9cyclictest644559-21apt-get20:40:061
61267499411,8cyclictest651677-21/usr/sbin/munin21:00:181
61267499411,11cyclictest667086-21diskstats21:45:151
612669994133,5cyclictest663321-21munin-plugin-st21:35:000
612669994133,5cyclictest663321-21munin-plugin-st21:35:000
612669994129,8cyclictest683755-21grep22:30:240
612682994034,4cyclictest623217-21kworker/u8:0+events_unbound19:38:413
612682994020,7cyclictest131rcu_preempt21:00:133
612678994025,11cyclictest311ktimers/220:50:122
612678994025,11cyclictest311ktimers/220:50:122
61267499409,3cyclictest683765-21unixbench_singl22:30:251
61267499405,5cyclictest687549-21sshd22:44:111
61267499403,7cyclictest621973-21apt19:35:121
61267499402,5cyclictest659931-21/usr/sbin/munin21:25:191
61267499402,5cyclictest659931-21/usr/sbin/munin21:25:191
61267499401,9cyclictest634108-21/usr/sbin/munin20:10:141
61267499401,10cyclictest698502-21/usr/sbin/munin23:15:251
61267499401,10cyclictest1-21systemd21:50:361
612682993927,9cyclictest727517-21sendmail00:40:003
612678993929,5cyclictest644099-21cron20:40:002
61267499391,9cyclictest695011-21diskstats23:05:151
61267499391,9cyclictest691226-21cron22:55:011
61267499391,9cyclictest620392-21diskstats19:30:161
61267499391,8cyclictest703763-21fschecks_count23:30:161
61267499391,8cyclictest665259-21/usr/sbin/munin21:40:221
61267499391,7cyclictest675918-21fschecks_time22:10:161
61267499391,6cyclictest643639-21unixbench_multi20:35:271
612674993914,21cyclictest0-21swapper/122:00:131
61267499391,12cyclictest721930-21grep00:20:251
61267499391,11cyclictest722815-21fschecks_time00:25:161
61267499391,11cyclictest681159-21diskstats22:25:151
61267499391,11cyclictest657988-21run-parts21:20:001
612669993929,6cyclictest0-21swapper/020:05:010
612682993832,4cyclictest391ktimers/321:35:123
612682993827,8cyclictest675044-21head22:05:263
612682993822,5cyclictest131rcu_preempt23:11:483
612682993822,5cyclictest131rcu_preempt23:11:483
612678993830,5cyclictest637282-21cron20:20:012
61267499388,6cyclictest720618-21seq00:19:351
612674993828,5cyclictest0-21swapper/123:30:001
61267499381,9cyclictest687964-21df22:45:141
61267499381,9cyclictest628991-21/usr/sbin/munin19:55:251
61267499381,8cyclictest726216-21fschecks_time00:35:171
61267499381,8cyclictest700533-21/usr/sbin/munin23:20:191
61267499381,7cyclictest663870-21/usr/sbin/munin21:35:171
612674993812,23cyclictest614616-21munin-node19:15:031
61267499381,10cyclictest713389-21unixbench_singl23:55:251
61267499381,10cyclictest691606-21grep22:55:151
612669993827,8cyclictest725793-21auditctl00:35:000
612682993731,4cyclictest685773-21kworker/u8:1+events_unbound22:54:413
612682993716,13cyclictest40-21ksoftirqd/321:55:003
612682993716,12cyclictest131rcu_preempt20:05:023
61267499372,8cyclictest613824-21grep19:10:221
61267499372,6cyclictest647259-21grep20:45:251
61267499372,6cyclictest647259-21grep20:45:251
61267499371,9cyclictest654921-21df_inode21:10:141
61267499371,9cyclictest639427-21grep20:25:161
61267499371,8cyclictest631640-21cat20:00:261
61267499371,6cyclictest708743-21/usr/sbin/munin23:45:241
61267499371,6cyclictest617865-21grep19:20:261
61267499371,5cyclictest715968-21grep00:05:181
61267499371,5cyclictest711711-21grep23:50:241
61267499371,5cyclictest711711-21grep23:50:241
61267499371,10cyclictest707152-21/usr/sbin/munin23:40:131
61267499371,10cyclictest674452-21memory22:05:201
61267499371,10cyclictest618574-21apt-get19:25:131
612669993728,6cyclictest694652-21sh23:05:000
612669993726,7cyclictest0-21swapper/023:35:010
61268299365,16cyclictest40-21ksoftirqd/320:30:253
612682993618,11cyclictest131rcu_preempt22:47:073
61267499362,10cyclictest1-21systemd00:13:341
61267499361,9cyclictest694240-21grep23:00:261
61267499361,9cyclictest671150-21ntp_states21:55:201
61267499361,12cyclictest706792-21auditctl23:40:011
61267499361,10cyclictest661399-21apt-get21:30:051
61267499361,10cyclictest661399-21apt-get21:30:051
612682993529,4cyclictest720494-21kworker/u8:2+events_unbound00:30:403
612682993529,4cyclictest706445-21kworker/u8:1+events_unbound23:35:573
612682993529,4cyclictest675104-21kworker/u8:1+events_unbound22:25:453
612682993523,5cyclictest708197-21kworker/u8:3+events_unbound23:45:373
612682993522,4cyclictest131rcu_preempt23:20:183
612682993519,5cyclictest667339-21latency21:45:183
612682993519,5cyclictest636936-21kworker/u8:2-events_power_efficient20:50:493
612682993519,5cyclictest636936-21kworker/u8:2-events_power_efficient20:50:493
612682993515,10cyclictest638692-21kworker/u8:3+events_unbound20:21:203
612682993515,10cyclictest131rcu_preempt20:35:253
612682993514,12cyclictest131rcu_preempt23:55:003
612682993514,12cyclictest131rcu_preempt23:55:003
612678993526,5cyclictest164-21systemd-journal19:55:152
612678993524,7cyclictest652863-21auditctl21:05:002
61267499352,6cyclictest0-21swapper/123:10:231
61267499352,6cyclictest0-21swapper/123:10:231
61267499351,8cyclictest654185-21sshd21:05:291
612674993511,2cyclictest0-21swapper/100:30:231
612669993527,4cyclictest0-21swapper/019:30:010
612669993524,9cyclictest121ktimers/021:10:120
612669993522,8cyclictest0-21swapper/019:40:260
612682993425,6cyclictest614233-21kworker/u8:1+events_unbound19:16:083
612682993422,4cyclictest131rcu_preempt21:15:223
612682993420,4cyclictest131rcu_preempt22:30:133
612682993417,3cyclictest131rcu_preempt21:33:133
612682993417,3cyclictest131rcu_preempt21:33:133
612682993414,3cyclictest131rcu_preempt22:12:293
61267899349,7cyclictest32-21ksoftirqd/223:50:162
61267899349,7cyclictest32-21ksoftirqd/223:50:162
61267899344,11cyclictest722549-21cat00:25:012
612678993426,4cyclictest727495-21auditctl00:39:592
612678993426,4cyclictest0-21swapper/221:30:282
612678993426,4cyclictest0-21swapper/221:30:282
612678993425,4cyclictest0-21swapper/222:10:152
612678993423,6cyclictest627309-21fschecks_time19:50:162
612678993423,6cyclictest627309-21fschecks_time19:50:162
612678993410,6cyclictest32-21ksoftirqd/223:15:182
61267499347,5cyclictest641216-21cat20:30:161
61267499341,8cyclictest680384-21grep22:20:261
61267499341,7cyclictest624726-21grep19:40:261
61267499341,10cyclictest632989-21ntpq20:05:231
612669993427,3cyclictest0-21swapper/019:55:000
612669993427,3cyclictest0-21swapper/019:55:000
612669993422,8cyclictest678437-21grep22:15:250
61268299339,11cyclictest40-21ksoftirqd/320:15:263
612682993327,4cyclictest689158-21kworker/u8:0+events_unbound23:25:563
612682993323,4cyclictest680415-21kworker/u8:2+events_unbound22:20:433
612682993316,8cyclictest612660-21cyclictest00:11:103
612682993315,4cyclictest131rcu_preempt21:11:283
612682993314,11cyclictest131rcu_preempt19:10:153
612682993313,11cyclictest131rcu_preempt00:25:143
61267899339,7cyclictest32-21ksoftirqd/219:45:252
61267899339,7cyclictest32-21ksoftirqd/219:45:252
612678993324,4cyclictest0-21swapper/220:10:242
612678993323,6cyclictest0-21swapper/223:45:152
61267499331,8cyclictest638541-21grep20:20:261
612669993326,5cyclictest701327-21kworker/u8:1+events_unbound23:20:550
612669993322,7cyclictest722390-21sh00:25:000
61266999331,2cyclictest0-21swapper/020:50:120
61266999331,2cyclictest0-21swapper/020:50:120
61268299328,14cyclictest40-21ksoftirqd/320:40:183
612682993225,4cyclictest0-21swapper/319:45:263
612682993225,4cyclictest0-21swapper/319:45:253
612682993221,4cyclictest131rcu_preempt21:40:163
612682993220,5cyclictest131rcu_preempt20:45:173
612682993220,5cyclictest131rcu_preempt20:45:173
612682993217,7cyclictest703085-21kworker/u8:2+events_unbound23:33:063
612682993216,5cyclictest131rcu_preempt22:55:253
612682993215,9cyclictest131rcu_preempt21:05:383
612682993215,5cyclictest131rcu_preempt23:59:513
61267899327,6cyclictest32-21ksoftirqd/219:35:012
612678993224,4cyclictest0-21swapper/223:30:262
612678993224,4cyclictest0-21swapper/219:20:162
612678993223,5cyclictest647266-21grep20:45:252
612678993223,5cyclictest647266-21grep20:45:252
612678993221,8cyclictest0-21swapper/222:20:002
612669993221,8cyclictest687619-21sh22:45:000
61268299319,14cyclictest40-21ksoftirqd/319:20:213
61268299319,13cyclictest131rcu_preempt00:05:273
612682993121,8cyclictest676902-21kworker/u8:3+events_unbound22:38:403
612682993116,7cyclictest657915-21kworker/u8:3+events_unbound21:28:453
612682993116,7cyclictest657915-21kworker/u8:3+events_unbound21:28:453
612682993112,12cyclictest40-21ksoftirqd/320:05:213
612682993111,3cyclictest131rcu_preempt00:04:143
61267899318,5cyclictest32-21ksoftirqd/221:00:002
61267899315,10cyclictest694662-21latency_hist23:05:012
61267899314,12cyclictest688041-21fschecks_time22:45:162
61267899313,12cyclictest659445-21auditctl21:22:002
612678993124,4cyclictest673767-21cron22:05:002
612678993124,4cyclictest661001-21unixbench_singl21:25:252
612678993124,4cyclictest661001-21unixbench_singl21:25:252
612678993123,4cyclictest0-21swapper/221:40:342
612678993123,4cyclictest0-21swapper/221:10:212
612678993119,7cyclictest0-21swapper/222:40:002
612678993114,3cyclictest0-21swapper/219:40:252
612678993110,12cyclictest32-21ksoftirqd/220:40:122
612669993123,4cyclictest0-21swapper/022:55:260
612669993122,6cyclictest0-21swapper/021:20:000
61268299309,6cyclictest131rcu_preempt19:40:223
61268299307,11cyclictest40-21ksoftirqd/322:00:183
612682993018,6cyclictest685773-21kworker/u8:1+events_unbound22:41:413
612682993015,4cyclictest131rcu_preempt20:25:173
612682993014,4cyclictest131rcu_preempt21:24:403
612682993012,3cyclictest131rcu_preempt19:34:443
612682993010,12cyclictest40-21ksoftirqd/323:40:153
612682993010,12cyclictest131rcu_preempt19:25:343
612678993022,4cyclictest0-21swapper/221:05:272
612678993020,7cyclictest715947-21fschecks_count00:05:172
612678993020,7cyclictest702088-21fschecks_count23:25:152
612669993024,4cyclictest0-21swapper/019:30:130
612682992912,5cyclictest131rcu_preempt20:10:113
612682992912,4cyclictest131rcu_preempt19:57:063
612678992919,6cyclictest0-21swapper/200:10:252
612678992916,10cyclictest633793-21cron20:09:012
612678992915,11cyclictest669601-21/usr/sbin/munin21:50:222
612669992923,4cyclictest674439-21latency22:05:180
612669992921,4cyclictest0-21swapper/023:20:000
612669992921,4cyclictest0-21swapper/022:25:260
61268299281,18cyclictest0-21swapper/300:15:213
612682992811,4cyclictest131rcu_preempt23:08:253
61267899288,6cyclictest0-21swapper/223:55:152
612678992821,4cyclictest614675-21fschecks_count19:15:022
612678992820,4cyclictest0-21swapper/223:10:242
612678992820,4cyclictest0-21swapper/223:10:242
612678992818,7cyclictest684112-21cron22:34:002
612678992818,6cyclictest0-21swapper/223:20:282
612669992821,4cyclictest668629-21apt-key21:50:000
612669992820,5cyclictest0-21swapper/023:10:000
612669992820,4cyclictest0-21swapper/022:45:260
61267899279,13cyclictest725312-21grep00:30:252
612678992721,3cyclictest0-21swapper/221:35:252
612678992720,5cyclictest311ktimers/221:15:122
612678992719,6cyclictest679892-21ntp_offset22:20:212
612678992719,4cyclictest719191-21apt-get00:15:132
612678992716,7cyclictest0-21swapper/200:05:002
612678992713,11cyclictest681284-21/usr/sbin/munin22:25:172
612678992711,12cyclictest640936-21/usr/sbin/munin20:30:252
61266999273,6cyclictest644118-21munin-run20:40:000
612669992719,4cyclictest0-21swapper/023:15:000
612669992719,4cyclictest0-21swapper/023:15:000
612669992719,4cyclictest0-21swapper/021:55:190
612669992719,4cyclictest0-21swapper/000:35:270
612669992718,4cyclictest0-21swapper/019:15:020
61266999271,7cyclictest438-21cron20:20:010
612669992717,7cyclictest0-21swapper/021:30:000
612669992717,7cyclictest0-21swapper/021:30:000
612669992717,6cyclictest0-21swapper/021:00:280
612669992713,9cyclictest642409-21cron20:35:000
612678992619,4cyclictest706802-21munin-plugin-st23:40:012
612678992619,4cyclictest0-21swapper/220:20:262
612678992618,4cyclictest0-21swapper/222:40:242
612678992616,7cyclictest695472-21/usr/sbin/munin23:05:222
612678992614,9cyclictest723352-21/usr/sbin/munin00:25:242
61266999261,9cyclictest6646252who21:35:260
612669992618,6cyclictest708621-21sort23:45:010
612669992617,6cyclictest121ktimers/022:50:060
61267899259,11cyclictest675596-21cut22:10:012
612678992516,6cyclictest0-21swapper/219:35:262
612678992512,3cyclictest131rcu_preempt22:52:102
612678992512,10cyclictest692380-21/usr/sbin/munin22:55:252
612669992519,3cyclictest0-21swapper/020:05:170
612669992518,3cyclictest0-21swapper/023:40:010
61266999251,6cyclictest670244-21runrttasks21:53:510
612669992514,7cyclictest0-21swapper/000:25:260
61267899248,12cyclictest707961-21wc23:40:242
612678992417,5cyclictest629-21snmpd21:59:512
612678992414,7cyclictest639615-21/usr/sbin/munin20:25:182
612678992414,7cyclictest613286-21/usr/sbin/munin19:10:162
612678992414,6cyclictest618571-21/usr/sbin/munin19:25:262
612669992420,2cyclictest13550irq/42-30be0000.ethernet21:20:140
612669992417,4cyclictest650717-21grep20:55:250
612669992415,6cyclictest617910-21grep19:20:260
612669992414,7cyclictest634942-21sed20:10:220
612669992412,8cyclictest676702-21grep22:10:250
61266999240,22cyclictest0-21swapper/023:45:380
612669992316,5cyclictest13550irq/42-30be0000.ethernet23:50:050
612669992316,5cyclictest13550irq/42-30be0000.ethernet23:50:050
612669992316,4cyclictest0-21swapper/000:00:250
612669992316,3cyclictest0-21swapper/019:47:540
612669992316,3cyclictest0-21swapper/019:47:540
612669992315,6cyclictest13550irq/42-30be0000.ethernet21:40:120
612669992315,6cyclictest121ktimers/019:55:040
612669992314,7cyclictest693425-1kworker/0:0H+mmc_complete00:20:000
61266999231,3cyclictest615151-21apt-get19:15:120
612669992312,8cyclictest622263-21/usr/sbin/munin19:35:170
61266999231,20cyclictest0-21swapper/000:05:130
61266999231,18cyclictest0-21swapper/023:28:310
61266999231,18cyclictest0-21swapper/022:20:020
612669992217,3cyclictest13550irq/42-30be0000.ethernet00:10:180
61266999220,18cyclictest0-21swapper/020:21:430
612669992114,5cyclictest121ktimers/023:55:120
612669992114,3cyclictest0-21swapper/020:26:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional