You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 11:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sat May 23, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3803056995445,6cyclictest3853447-21grep21:30:270
380305699531,32cyclictest3829399-21latency_hist20:25:000
3803064995144,4cyclictest3831104-21munin-plugin-st20:30:002
3803070994941,6cyclictest3895630-21kworker/u8:3+events_unbound23:31:593
3803061994941,5cyclictest3861342-21munin-plugin-st21:55:001
380305699491,30cyclictest3856412-21fschecks_count21:40:130
380305699491,30cyclictest0-21swapper/020:05:190
3803061994837,7cyclictest3905981-21grep00:00:261
3803056994834,11cyclictest121ktimers/022:45:110
3803070994740,4cyclictest3807000-21munin-plugin-st19:20:003
3803061994733,10cyclictest231ktimers/123:35:091
3803056994739,5cyclictest3827694-21idleruntime-cro20:20:010
3803056994728,15cyclictest0-21swapper/021:15:250
3803056994728,15cyclictest0-21swapper/000:05:250
380305699471,28cyclictest3807561-21if_eth119:20:180
380305699471,27cyclictest3878343-21cat22:40:240
380305699470,40cyclictest0-21swapper/020:55:110
3803061994620,9cyclictest131rcu_preempt00:20:121
380305699461,39cyclictest3912393-21/usr/sbin/munin00:20:220
3803070994536,6cyclictest3896329-21fschecks_count23:35:123
3803064994511,16cyclictest32-21ksoftirqd/222:10:012
3803056994537,4cyclictest3895969-21munin-plugin-st23:35:000
380305699450,36cyclictest0-21swapper/021:55:120
3803064994434,5cyclictest208-21systemd-logind21:01:252
3803064994434,5cyclictest208-21systemd-logind21:01:252
3803061994436,4cyclictest0-21swapper/121:30:171
380305699441,25cyclictest3865746-21ntpq22:05:200
380305699440,41cyclictest0-21swapper/023:10:120
3803064994337,4cyclictest3813621-21kworker/u8:2+events_unbound19:46:132
380305699433,36cyclictest3909111-21/usr/sbin/munin00:10:240
380305699432,35cyclictest13550irq/42-30be0000.ethernet21:05:120
380305699431,39cyclictest3890130-21/usr/sbin/munin23:15:240
380305699431,39cyclictest3804020-21/usr/sbin/munin19:10:250
380305699431,26cyclictest3829786-21tune2fs20:25:160
3803070994236,4cyclictest3842968-21kworker/u8:2+events_unbound21:41:463
3803061994236,4cyclictest3857692-21kworker/u8:3+events_unbound21:46:461
380305699427,32cyclictest3811018-21/usr/sbin/munin19:30:170
380305699421,38cyclictest3819900-21/usr/sbin/munin19:55:190
380305699421,38cyclictest3819900-21/usr/sbin/munin19:55:190
380305699421,37cyclictest3812796-21/usr/sbin/munin19:35:180
380305699421,36cyclictest3850209-21/usr/sbin/munin21:25:130
380305699421,24cyclictest3817790-21df_inode19:50:140
3803070994135,4cyclictest3869954-21kworker/u8:3+events_unbound22:16:473
3803061994125,11cyclictest0-21swapper/121:45:011
380305699412,33cyclictest3897914-21apt-get23:40:110
380305699412,22cyclictest3874338-21irqstats22:30:180
3803056994122,15cyclictest0-21swapper/021:50:170
380305699411,37cyclictest3910654-21/usr/sbin/munin00:15:220
380305699411,37cyclictest3886339-21/usr/sbin/munin23:05:170
380305699411,36cyclictest3836444-21sshd20:45:080
380305699411,35cyclictest3863550-21/usr/sbin/munin22:00:140
380305699411,34cyclictest3884266-21apt-get23:00:110
380305699410,32cyclictest0-21swapper/022:15:110
3803070994032,6cyclictest3810035-21kworker/u8:3+events_unbound19:26:133
3803070994032,6cyclictest3810035-21kworker/u8:3+events_unbound19:26:133
3803070994032,4cyclictest0-21swapper/321:40:013
3803070994028,6cyclictest3885409-21kworker/u8:0+events_unbound23:06:543
3803064994016,12cyclictest131rcu_preempt00:05:202
3803064994010,6cyclictest301rcuc/221:30:002
3803061994034,4cyclictest3842968-21kworker/u8:2+events_unbound21:36:431
380305699403,34cyclictest3806902-21cron19:19:000
380305699402,35cyclictest3846101-21cron21:11:000
380305699401,36cyclictest3914157-21/usr/sbin/munin00:25:230
380305699401,34cyclictest3905286-21/usr/sbin/munin00:00:180
380305699401,34cyclictest3903674-21/usr/sbin/munin23:55:170
380305699401,34cyclictest3891673-21/usr/sbin/munin23:20:230
380305699401,34cyclictest3831937-21/usr/sbin/munin20:30:220
380305699401,33cyclictest3834957-21grep20:40:160
380305699401,24cyclictest3808942-21/usr/sbin/munin19:25:170
380305699401,24cyclictest3808942-21/usr/sbin/munin19:25:170
380305699401,23cyclictest3901643-21df_inode23:50:170
3803070993933,4cyclictest3877072-21kworker/u8:2+events_unbound22:51:543
3803070993933,4cyclictest3861079-21kworker/u8:1+events_unbound22:26:503
3803070993919,5cyclictest131rcu_preempt00:35:243
380306499399,9cyclictest32-21ksoftirqd/221:05:232
3803061993916,7cyclictest131rcu_preempt21:15:121
3803061993916,6cyclictest3830751-21kworker/u8:0+events_unbound20:30:201
380305699393,33cyclictest3880957-21df_inode22:50:140
380305699393,33cyclictest3833029-21apt-get20:35:110
380305699393,20cyclictest3870635-21df22:20:140
3803056993927,9cyclictest438-21cron21:50:000
380305699391,35cyclictest3883154-21/usr/sbin/munin22:55:220
380305699391,35cyclictest3875834-21/usr/sbin/munin22:35:140
380305699391,35cyclictest3875834-21/usr/sbin/munin22:35:140
380305699391,35cyclictest3815189-21cron19:41:000
380305699391,33cyclictest3848399-21/usr/sbin/munin21:20:120
380305699391,32cyclictest0-21swapper/000:35:110
380305699390,31cyclictest0-21swapper/023:35:100
3803070993832,4cyclictest3842968-21kworker/u8:2+events_unbound21:11:273
3803070993822,9cyclictest131rcu_preempt19:50:013
3803070993820,10cyclictest131rcu_preempt20:54:553
3803070993818,10cyclictest131rcu_preempt00:26:493
380305699381,34cyclictest3900132-21/usr/sbin/munin23:45:200
380305699381,34cyclictest3854901-21/usr/sbin/munin21:35:180
380305699381,34cyclictest3821426-21/usr/sbin/munin20:00:160
380305699381,34cyclictest3817273-21cron19:48:000
380305699381,32cyclictest3824827-21/usr/sbin/munin20:10:170
380305699381,31cyclictest0-21swapper/020:50:120
3803070993731,4cyclictest3836146-21kworker/u8:3+events_unbound21:01:283
3803070993731,4cyclictest3836146-21kworker/u8:3+events_unbound21:01:283
3803070993731,3cyclictest0-21swapper/320:45:253
3803070993728,4cyclictest0-21swapper/300:15:163
3803070993717,10cyclictest131rcu_preempt20:13:463
3803064993715,8cyclictest131rcu_preempt21:10:232
3803061993731,4cyclictest3895630-21kworker/u8:3+events_unbound23:42:011
3803061993730,5cyclictest3875287-21kworker/u8:0+events_unbound22:46:521
3803061993728,5cyclictest0-21swapper/120:20:131
3803061993723,11cyclictest231ktimers/122:15:131
3803061993718,11cyclictest3878865-21kworker/u8:1+events_unbound23:17:351
380305699371,33cyclictest3915290-21diskstats00:30:140
380305699371,33cyclictest3893238-21meminfo23:25:170
380305699371,33cyclictest3842091-21/usr/sbin/munin21:00:210
380305699371,33cyclictest3842091-21/usr/sbin/munin21:00:210
3803070993624,7cyclictest0-21swapper/319:40:253
3803070993624,6cyclictest3820752-21kworker/u8:1+events_unbound20:00:403
3803070993621,4cyclictest131rcu_preempt23:40:203
3803070993618,5cyclictest131rcu_preempt20:40:233
3803070993618,5cyclictest131rcu_preempt19:50:133
3803070993617,12cyclictest3815497-21kworker/u8:0-ext4-rsv-conversion20:00:013
3803070993617,12cyclictest3815497-21kworker/u8:0-ext4-rsv-conversion20:00:003
3803070993613,13cyclictest131rcu_preempt19:35:013
3803064993629,4cyclictest301rcuc/200:35:212
3803064993628,4cyclictest3889130-21sh23:15:002
3803064993624,4cyclictest131rcu_preempt23:56:262
3803064993618,5cyclictest131rcu_preempt20:05:072
3803061993627,5cyclictest0-21swapper/120:45:141
3803061993620,5cyclictest131rcu_preempt19:15:051
3803061993616,11cyclictest131rcu_preempt19:55:241
3803061993616,11cyclictest131rcu_preempt19:55:231
3803070993518,5cyclictest131rcu_preempt21:30:013
3803070993515,5cyclictest131rcu_preempt20:15:163
380306499354,15cyclictest32-21ksoftirqd/200:00:202
3803064993528,4cyclictest3839611-21cron20:55:012
3803064993515,3cyclictest131rcu_preempt20:23:372
3803064993515,3cyclictest131rcu_preempt19:25:012
3803064993514,9cyclictest131rcu_preempt21:35:122
3803064993514,3cyclictest0-21swapper/219:15:032
3803061993518,6cyclictest131rcu_preempt20:25:131
3803061993518,10cyclictest3836146-21kworker/u8:3+events_unbound21:11:571
3803061993512,7cyclictest131rcu_preempt20:40:111
3803070993428,4cyclictest3878865-21kworker/u8:1+events_unbound23:53:163
3803070993425,5cyclictest208-21systemd-logind21:16:413
3803070993425,5cyclictest0-21swapper/323:00:253
3803070993424,6cyclictest0-21swapper/322:35:003
3803070993418,9cyclictest131rcu_preempt23:19:073
3803070993418,4cyclictest131rcu_preempt19:13:183
3803070993418,10cyclictest131rcu_preempt22:45:003
3803070993417,5cyclictest131rcu_preempt21:45:173
3803070993414,5cyclictest131rcu_preempt22:35:203
3803070993414,5cyclictest131rcu_preempt22:35:193
3803070993414,4cyclictest131rcu_preempt23:45:183
3803070993414,10cyclictest131rcu_preempt21:30:253
3803070993413,13cyclictest131rcu_preempt22:05:213
3803070993413,13cyclictest131rcu_preempt21:05:243
3803070993413,11cyclictest3830751-21kworker/u8:0-events_freezable_power_20:38:423
3803070993412,6cyclictest131rcu_preempt00:00:113
380306499346,7cyclictest32-21ksoftirqd/222:55:002
3803064993426,4cyclictest0-21swapper/219:30:252
3803064993410,7cyclictest32-21ksoftirqd/219:15:012
3803061993427,5cyclictest3899222-21kworker/u8:2+events_unbound00:13:361
3803061993425,6cyclictest3868166-21unixbench_singl22:10:251
3803061993424,6cyclictest0-21swapper/123:30:251
3803061993418,5cyclictest131rcu_preempt20:20:001
3803061993417,11cyclictest380-21ntpd23:10:211
380305699340,31cyclictest3803058-21latency_hist19:10:010
3803070993326,3cyclictest0-21swapper/300:30:253
3803070993318,4cyclictest131rcu_preempt22:45:253
3803070993314,6cyclictest3807301-21kworker/u8:2+events_unbound19:24:393
3803070993314,10cyclictest131rcu_preempt22:55:233
3803064993326,4cyclictest311ktimers/220:47:552
3803064993322,7cyclictest0-21swapper/222:40:002
3803064993322,7cyclictest0-21swapper/222:40:002
3803064993317,4cyclictest311ktimers/221:54:162
3803064993314,3cyclictest131rcu_preempt21:30:152
380306499331,16cyclictest0-21swapper/220:40:252
380306199338,6cyclictest24-21ksoftirqd/100:35:231
3803061993320,10cyclictest231ktimers/121:05:121
3803061993318,5cyclictest131rcu_preempt21:25:191
3803061993316,6cyclictest3899222-21kworker/u8:2+events_unbound23:59:361
3803070993222,6cyclictest0-21swapper/323:20:163
3803070993219,7cyclictest3904548-21kworker/u8:3+events_unbound00:08:383
3803070993217,9cyclictest3866391-21kworker/u8:0+events_unbound22:21:183
3803070993216,5cyclictest131rcu_preempt23:10:233
3803070993216,5cyclictest131rcu_preempt00:20:253
3803070993214,5cyclictest131rcu_preempt20:30:233
3803070993213,6cyclictest131rcu_preempt21:55:163
3803070993213,11cyclictest131rcu_preempt19:35:163
3803070993212,5cyclictest131rcu_preempt23:25:193
3803070993212,5cyclictest131rcu_preempt21:55:013
3803070993212,5cyclictest131rcu_preempt21:20:193
3803070993211,5cyclictest131rcu_preempt00:10:143
380306499326,6cyclictest32-21ksoftirqd/223:00:232
380306499324,7cyclictest32-21ksoftirqd/221:15:252
380306499324,7cyclictest32-21ksoftirqd/220:35:212
3803064993225,4cyclictest0-21swapper/200:15:162
3803064993222,7cyclictest311ktimers/223:35:012
3803064993219,5cyclictest131rcu_preempt22:40:242
3803064993213,4cyclictest0-21swapper/222:25:232
3803064993211,4cyclictest301rcuc/222:01:572
3803064993211,11cyclictest131rcu_preempt19:41:132
380306199325,6cyclictest24-21ksoftirqd/122:10:001
3803061993225,4cyclictest0-21swapper/122:29:591
3803061993217,9cyclictest131rcu_preempt00:35:011
3803061993216,10cyclictest131rcu_preempt20:55:581
3803070993114,5cyclictest131rcu_preempt20:55:213
3803070993113,5cyclictest0-21swapper/323:55:203
3803070993111,12cyclictest131rcu_preempt22:10:173
3803064993124,4cyclictest311ktimers/222:47:132
3803064993121,7cyclictest311ktimers/221:40:102
3803064993121,7cyclictest311ktimers/220:55:152
3803064993120,8cyclictest311ktimers/223:40:242
3803064993120,8cyclictest311ktimers/220:15:212
3803064993120,3cyclictest131rcu_preempt21:20:132
3803064993119,9cyclictest311ktimers/221:45:192
3803064993119,9cyclictest311ktimers/220:00:242
3803064993119,8cyclictest311ktimers/223:25:232
380306499311,5cyclictest3870283-21munin-run22:20:012
3803064993115,9cyclictest131rcu_preempt23:18:222
3803064993114,9cyclictest131rcu_preempt23:05:212
3803064993112,4cyclictest131rcu_preempt23:20:132
3803061993124,4cyclictest0-21swapper/119:45:151
3803061993122,6cyclictest0-21swapper/123:05:001
3803061993117,9cyclictest231ktimers/119:40:131
3803061993113,7cyclictest131rcu_preempt22:30:111
3803061993111,12cyclictest131rcu_preempt20:35:231
3803070993015,4cyclictest131rcu_preempt20:20:233
3803070993014,5cyclictest131rcu_preempt20:25:183
3803070993011,5cyclictest0-21swapper/320:05:243
3803070993011,4cyclictest131rcu_preempt22:00:163
380306499309,12cyclictest32-21ksoftirqd/221:55:412
380306499307,20cyclictest311ktimers/219:55:162
380306499307,20cyclictest311ktimers/219:55:162
380306499305,7cyclictest32-21ksoftirqd/223:50:182
380306499303,7cyclictest32-21ksoftirqd/223:00:012
3803064993024,3cyclictest301rcuc/222:30:182
3803064993020,8cyclictest311ktimers/200:20:242
3803064993020,7cyclictest301rcuc/219:52:162
3803064993014,4cyclictest131rcu_preempt20:10:162
3803064993013,10cyclictest131rcu_preempt00:28:502
380306199308,4cyclictest0-21swapper/120:05:131
380306199307,7cyclictest24-21ksoftirqd/100:15:241
3803061993024,4cyclictest3834361-21kworker/u8:2+events_unbound20:51:261
3803061993021,6cyclictest0-21swapper/121:05:001
3803061993021,6cyclictest0-21swapper/121:05:001
3803061993020,6cyclictest0-21swapper/100:05:251
3803061993018,4cyclictest3861079-21kworker/u8:1+events_unbound22:21:231
3803061993016,5cyclictest131rcu_preempt19:20:211
3803061993012,6cyclictest131rcu_preempt00:25:111
3803061993012,5cyclictest131rcu_preempt19:10:161
3803061993011,10cyclictest131rcu_preempt22:55:141
380306499298,13cyclictest32-21ksoftirqd/200:10:202
380306499295,5cyclictest301rcuc/220:32:472
3803064992912,4cyclictest131rcu_preempt19:35:112
3803064992910,5cyclictest131rcu_preempt23:48:362
380306199297,11cyclictest131rcu_preempt23:49:591
380306199296,7cyclictest24-21ksoftirqd/121:55:251
3803061992921,6cyclictest3820752-21kworker/u8:1+events_unbound20:01:171
3803061992918,7cyclictest0-21swapper/123:05:251
3803061992916,5cyclictest131rcu_preempt23:50:031
3803061992915,4cyclictest131rcu_preempt22:00:221
3803064992813,4cyclictest131rcu_preempt22:20:192
3803064992812,4cyclictest0-21swapper/200:30:122
3803064992811,9cyclictest131rcu_preempt23:35:232
380306199288,16cyclictest231ktimers/119:35:241
3803061992821,5cyclictest3796075-21kworker/u8:1+events_unbound19:50:121
3803061992818,7cyclictest0-21swapper/122:45:001
3803061992812,5cyclictest131rcu_preempt19:25:241
3803061992812,5cyclictest131rcu_preempt19:25:231
380306499273,5cyclictest301rcuc/219:25:202
380306499273,5cyclictest301rcuc/219:25:192
3803061992720,3cyclictest0-21swapper/122:40:001
3803061992720,3cyclictest0-21swapper/122:39:591
3803061992716,5cyclictest131rcu_preempt22:50:121
3803061992716,4cyclictest131rcu_preempt19:35:001
3803061992714,5cyclictest131rcu_preempt20:10:131
3803061992711,13cyclictest24-21ksoftirqd/123:20:251
380306199269,8cyclictest3803050-21cyclictest23:25:191
3803061992612,3cyclictest131rcu_preempt21:20:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional