You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 05:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed Sep 23, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2236499684,9cyclictest131rcu_preempt22:55:150
2239199662,31cyclictest35966irq/58-eth000:15:153
22384996321,33cyclictest25381-21ssh21:55:132
2237599601,47cyclictest27620-21ssh00:20:131
22384995426,21cyclictest21579-21ssh21:35:142
2236499526,36cyclictest11099-21ssh22:05:130
2236499515,7cyclictest131rcu_preempt00:10:150
2236499513,9cyclictest131rcu_preempt00:30:120
22391994812,26cyclictest35966irq/58-eth022:05:123
2236499482,38cyclictest13039-21ssh23:00:140
2239199472,28cyclictest35966irq/58-eth023:55:133
2238499476,32cyclictest0-21swapper/220:45:162
2239199441,25cyclictest35966irq/58-eth021:40:113
2236499431,31cyclictest8758-21ssh21:10:180
2238499422,20cyclictest591-21ssh00:05:122
22391994124,6cyclictest0-21swapper/319:15:123
22384994119,17cyclictest27346irq/40-dwc2_hso23:05:142
2237599414,29cyclictest14667-21ssh22:25:131
22375994126,10cyclictest0-21swapper/122:40:101
2236499414,29cyclictest31117-21ssh23:10:180
2236499412,31cyclictest15037-21ssh23:55:140
2238499394,22cyclictest0-21swapper/222:25:132
22384993928,7cyclictest23810-21apt-get19:45:122
2236499393,3cyclictest131rcu_preempt00:25:130
22384993827,7cyclictest5794-21apt-get20:00:132
2236499372,31cyclictest11272-21ssh23:35:120
2236499372,24cyclictest23009-1kworker/u9:221:25:150
22364993718,17cyclictest12-21ksoftirqd/023:50:130
2236499371,30cyclictest20157-21ssh22:10:180
2239199362,21cyclictest35966irq/58-eth023:25:123
2238499362,28cyclictest20861-21ssh23:56:392
2236499365,24cyclictest7635-21ssh23:15:140
2239199352,19cyclictest35966irq/58-eth021:25:143
2239199351,20cyclictest35966irq/58-eth023:45:143
2238499355,13cyclictest0-21swapper/221:48:402
22384993526,7cyclictest18571-21apt-get00:15:152
22384993525,7cyclictest29039-21apt-get22:15:132
2236499352,8cyclictest131rcu_preempt19:30:120
2239199343,15cyclictest35966irq/58-eth021:50:123
2239199343,15cyclictest35966irq/58-eth021:50:123
2238499343,14cyclictest27551-21ssh22:13:412
2238499336,13cyclictest14719-21ntp_192.168.11422:05:252
22364993319,9cyclictest0-21swapper/023:20:140
2236499331,23cyclictest23731-21ssh22:30:130
22391993215,11cyclictest35966irq/58-eth022:55:153
22384993215,10cyclictest2070-21apt-get22:00:142
2237599324,23cyclictest621-21ssh00:05:131
2236499321,20cyclictest29403-21rm23:45:140
2238499316,8cyclictest0-21swapper/221:34:572
2238499316,7cyclictest8282-21diskmemload22:23:572
22384993123,4cyclictest11262-21apt-get23:35:122
22384993111,17cyclictest0-21swapper/220:10:122
2239199302,14cyclictest35966irq/58-eth022:20:173
2238499309,19cyclictest22550-21ssh22:45:312
2238499305,10cyclictest0-21swapper/223:21:182
2238499304,17cyclictest0-21swapper/200:33:082
2238499304,12cyclictest13776-21rm23:54:002
22384993022,4cyclictest20325-21apt-get23:40:112
22384993022,4cyclictest20325-21apt-get23:40:112
22384993020,8cyclictest29389-21apt-get23:45:152
22384993017,9cyclictest0-21swapper/220:40:122
22384993013,13cyclictest3477-21ssh21:25:132
2237599300,17cyclictest21-21ksoftirqd/121:00:141
2239199292,16cyclictest35966irq/58-eth000:05:123
2238499299,17cyclictest32002-21kthreadcore20:25:202
2238499299,11cyclictest8282-21diskmemload21:14:402
2238499296,8cyclictest1961-21rm21:23:412
2238499295,11cyclictest0-21swapper/222:43:182
2238499294,12cyclictest0-21swapper/222:56:442
22384992912,13cyclictest30140-21grep20:55:282
2237599293,6cyclictest21-21ksoftirqd/100:28:221
22375992915,8cyclictest973-21apt-get19:55:141
2236499296,17cyclictest12-21ksoftirqd/021:15:160
2239199281,13cyclictest35966irq/58-eth021:20:163
2238499286,18cyclictest20336-21sort20:15:132
2238499284,16cyclictest27346irq/40-dwc2_hso21:50:122
2238499284,16cyclictest27346irq/40-dwc2_hso21:50:122
2238499282,17cyclictest22136-21apt-get00:35:122
22384992814,12cyclictest6930-21wc20:35:022
22384992814,12cyclictest0-21swapper/221:15:152
22384992811,9cyclictest27799-21/usr/sbin/munin22:30:312
2236499282,21cyclictest2267-21ssh23:30:120
2236499282,20cyclictest12-21ksoftirqd/021:45:150
2238499279,15cyclictest29445-21kthreadcore19:15:212
2238499276,9cyclictest4036-21rm00:25:052
2238499276,9cyclictest29980-21ssh00:01:452
2238499273,10cyclictest0-21swapper/222:53:212
22384992716,8cyclictest25654-21apt-get23:25:132
22384992713,11cyclictest22488-21/usr/sbin/munin19:10:232
2238499271,18cyclictest27796-21cut00:20:162
22384992711,8cyclictest14193-21ls19:35:162
22384992711,13cyclictest19393-21ssh23:02:192
22384992711,13cyclictest11618-21sshd20:39:382
22384992711,13cyclictest0-21swapper/221:40:102
22384992710,13cyclictest5839-21cut19:25:202
2237599272,19cyclictest13093-21sh21:46:391
2236499271,18cyclictest20358-21ssh23:40:130
2236499271,18cyclictest20358-21ssh23:40:120
2238499264,6cyclictest0-21swapper/223:13:262
2238499263,6cyclictest0-21swapper/200:13:032
22384992614,10cyclictest0-21swapper/222:35:132
22384992613,10cyclictest25382-21tail20:20:182
2237599267,5cyclictest21-21ksoftirqd/123:45:311
2237599266,9cyclictest0-21swapper/120:00:121
2237599265,3cyclictest131rcu_preempt00:32:341
22375992613,4cyclictest21-21ksoftirqd/123:44:421
22375992613,4cyclictest21-21ksoftirqd/123:44:421
22364992612,10cyclictest12-21ksoftirqd/000:00:140
2239199257,13cyclictest35966irq/58-eth022:00:163
2238499259,12cyclictest0-21swapper/219:50:132
2238499255,8cyclictest10476-21sh23:34:322
2238499254,17cyclictest13102-21rm23:16:182
2238499252,14cyclictest27346irq/40-dwc2_hso19:40:132
2237599257,9cyclictest21-21ksoftirqd/123:51:011
2237599257,9cyclictest21-21ksoftirqd/122:20:391
2237599257,8cyclictest21-21ksoftirqd/121:52:241
2237599257,8cyclictest21-21ksoftirqd/121:52:241
2237599257,7cyclictest21-21ksoftirqd/122:55:051
2237599256,9cyclictest21-21ksoftirqd/122:15:581
2237599256,3cyclictest131rcu_preempt21:23:051
2237599253,6cyclictest21-21ksoftirqd/121:31:161
2237599253,16cyclictest30613-21ssh21:40:111
2237599252,6cyclictest21-21ksoftirqd/100:00:141
2237599252,4cyclictest21-21ksoftirqd/122:35:171
22375992520,4cyclictest201rcuc/123:58:221
22375992519,4cyclictest201rcuc/122:33:371
2239199242,14cyclictest35966irq/58-eth000:10:163
22391992414,7cyclictest36066irq/59-eth023:35:123
2238499249,12cyclictest3685-21munin-node21:05:142
2238499244,17cyclictest1465-21kthreadcore19:20:202
2237599247,9cyclictest201rcuc/121:19:341
2237599247,7cyclictest21-21ksoftirqd/122:00:531
2237599247,10cyclictest21-21ksoftirqd/123:11:531
2237599246,8cyclictest21-21ksoftirqd/123:06:121
2237599246,8cyclictest21-21ksoftirqd/121:25:121
2237599245,7cyclictest21-21ksoftirqd/123:21:491
22375992420,3cyclictest201rcuc/123:02:031
22375992414,6cyclictest31004-21ssh22:50:251
2237599241,17cyclictest18602-21ssh00:15:151
2236499241,19cyclictest32736-21ssh22:35:130
22391992311,5cyclictest0-21swapper/322:35:163
2238499238,12cyclictest10640-21/usr/sbin/munin20:05:122
2238499236,13cyclictest976-21/usr/sbin/munin19:55:142
2238499235,9cyclictest9843-21irqrtprio19:30:182
2237599236,9cyclictest21-21ksoftirqd/123:18:441
2237599236,8cyclictest21-21ksoftirqd/122:47:391
2237599236,8cyclictest21-21ksoftirqd/121:12:021
2237599236,8cyclictest21-21ksoftirqd/100:12:341
2237599236,6cyclictest21-21ksoftirqd/122:10:161
2237599236,3cyclictest131rcu_preempt19:35:201
2237599234,5cyclictest21-21ksoftirqd/121:35:181
2237599234,3cyclictest131rcu_preempt23:25:351
2237599233,3cyclictest131rcu_preempt19:10:271
2237599233,3cyclictest131rcu_preempt00:39:191
22375992310,4cyclictest21-21ksoftirqd/123:32:071
2237599230,6cyclictest21-21ksoftirqd/120:45:011
2236499232,17cyclictest26841-21sh21:20:160
2236499231,17cyclictest3703-21apt-get21:05:150
2239199229,9cyclictest35966irq/58-eth020:35:113
2239199222,9cyclictest35966irq/58-eth022:15:123
2239199222,15cyclictest0-21swapper/323:40:113
2239199222,15cyclictest0-21swapper/323:40:113
22391992210,6cyclictest0-21swapper/300:38:393
2238499225,9cyclictest22243-21cat20:50:202
2237599226,9cyclictest21-21ksoftirqd/119:45:311
2237599226,7cyclictest21-21ksoftirqd/123:38:031
2237599224,2cyclictest131rcu_preempt22:05:111
2237599223,4cyclictest21-21ksoftirqd/120:55:181
22375992217,3cyclictest0-21swapper/121:05:141
22375992216,4cyclictest201rcuc/121:55:031
22375992216,3cyclictest29858-21apt-get20:25:121
2236499224,10cyclictest1213-21grep00:05:180
22364992213,4cyclictest12-21ksoftirqd/022:50:130
2236499221,16cyclictest11327-21kthreadcore20:05:190
2239199213,15cyclictest35966irq/58-eth021:30:123
22391992111,7cyclictest0-21swapper/300:00:153
22384992113,6cyclictest31230-21apt-get21:00:132
2237599215,4cyclictest21-21ksoftirqd/119:20:181
2237599214,3cyclictest131rcu_preempt20:50:011
2237599213,5cyclictest21-21ksoftirqd/120:15:021
2237599213,5cyclictest21-21ksoftirqd/119:30:161
22375992112,5cyclictest4512-21apt-get19:25:131
2237599211,10cyclictest29657-21latency_hist20:25:011
2236499218,10cyclictest12-21ksoftirqd/000:35:140
2236499212,15cyclictest18338-21ssh22:45:140
22364992114,4cyclictest0-21swapper/022:40:240
22364992113,5cyclictest0-21swapper/020:55:120
2236499211,17cyclictest9030-21ls20:00:230
2236499211,16cyclictest18934-21apt-get19:40:120
2239199208,9cyclictest0-21swapper/322:10:553
2239199203,10cyclictest0-21swapper/323:05:383
2239199203,10cyclictest0-21swapper/321:35:143
2239199202,4cyclictest0-21swapper/319:20:143
2239199201,15cyclictest0-21swapper/319:10:223
22391992010,8cyclictest0-21swapper/322:32:163
2237599204,3cyclictest131rcu_preempt19:51:011
2237599204,2cyclictest131rcu_preempt20:05:161
2237599203,4cyclictest21-21ksoftirqd/119:15:171
2236499203,5cyclictest30137-21kthreadcore19:15:200
2236499201,9cyclictest25900-21fschecks_time23:25:170
2236499201,14cyclictest29049-21ssh22:15:120
22364992010,7cyclictest1651-21apt22:00:010
22364992010,6cyclictest31466-21kthreadcore21:40:180
22364992010,5cyclictest0-21swapper/021:30:120
2236499200,14cyclictest28529-21ls19:50:000
2239199199,8cyclictest0-21swapper/323:14:413
2239199198,7cyclictest0-21swapper/323:30:323
2239199195,7cyclictest0-21swapper/323:00:143
2239199193,11cyclictest0-21swapper/321:46:243
2239199193,11cyclictest0-21swapper/300:31:563
2239199192,11cyclictest0-21swapper/300:25:393
2239199191,12cyclictest0-21swapper/320:50:163
2237599194,3cyclictest131rcu_preempt19:40:241
2237599191,6cyclictest21-21ksoftirqd/120:55:021
2236499196,9cyclictest2090-21ssh22:00:150
2236499193,9cyclictest0-21swapper/023:05:390
2236499191,9cyclictest20059-21kthreadcore00:15:210
22364991913,4cyclictest0-21swapper/022:20:190
22364991911,5cyclictest8282-21diskmemload21:35:160
2236499191,15cyclictest7132-21cut20:35:140
2236499191,14cyclictest17247-21tr22:25:220
2236499190,16cyclictest1051-21ls19:55:150
2236499190,11cyclictest733-21latency_hist19:55:020
2239199189,7cyclictest0-21swapper/300:24:303
2239199189,6cyclictest0-21swapper/321:17:273
2239199187,7cyclictest0-21swapper/322:45:213
2239199183,11cyclictest0-21swapper/323:50:153
2239199183,10cyclictest0-21swapper/322:44:393
2239199182,11cyclictest0-21swapper/322:25:143
2239199182,10cyclictest0-21swapper/320:45:173
2239199182,10cyclictest0-21swapper/320:10:123
2239199182,10cyclictest0-21swapper/320:05:043
2239199182,10cyclictest0-21swapper/319:45:053
2239199182,10cyclictest0-21swapper/319:30:143
22391991810,5cyclictest35966irq/58-eth023:15:293
2238499189,7cyclictest22266-21latency_hist19:10:012
2237599184,3cyclictest131rcu_preempt20:35:231
2237599180,3cyclictest20260-21apt-get20:15:131
2236499186,10cyclictest614-21dbus-daemon20:41:130
2236499182,10cyclictest20490-21ls20:15:170
2236499181,3cyclictest0-21swapper/019:10:170
2239199179,6cyclictest0-21swapper/322:52:443
2239199179,6cyclictest0-21swapper/321:59:153
2239199179,6cyclictest0-21swapper/321:10:473
2239199177,8cyclictest0-21swapper/323:20:333
2239199172,10cyclictest0-21swapper/320:00:133
2239199172,10cyclictest0-21swapper/319:55:143
2239199172,10cyclictest0-21swapper/319:25:143
2239199171,12cyclictest0-21swapper/320:15:123
2237599171,5cyclictest21-21ksoftirqd/120:30:221
2236499179,6cyclictest16498-21fschecks_count21:50:160
2236499179,6cyclictest16498-21fschecks_count21:50:160
2236499177,7cyclictest27373-21cut20:20:220
2236499176,3cyclictest131rcu_preempt20:50:010
2236499175,9cyclictest24776-21missed_timers20:50:220
2236499174,10cyclictest3351-21cat19:20:280
2236499172,11cyclictest0-21swapper/020:10:130
2236499172,10cyclictest0-21swapper/019:25:130
2236499171,13cyclictest3537-21cp21:05:020
2236499171,13cyclictest1982-21munin-run20:30:020
2236499171,12cyclictest28525-21tr00:20:190
2239199162,10cyclictest0-21swapper/319:35:153
2239199161,9cyclictest0-21swapper/321:05:023
2237599163,2cyclictest131rcu_preempt19:10:011
2236499161,11cyclictest17515-21ntp_192.168.11419:35:250
2239199151,11cyclictest0-21swapper/320:55:223
2239199151,11cyclictest0-21swapper/319:50:313
2236499152,10cyclictest2409-21grep20:30:170
2239199147,5cyclictest0-21swapper/321:08:563
2239199138,3cyclictest0-21swapper/320:24:213
2239199132,9cyclictest0-21swapper/320:30:183
2239199131,5cyclictest0-21swapper/320:25:193
2239199131,10cyclictest0-21swapper/320:40:263
2236499131,9cyclictest22368-21latency_hist19:10:000
2239199113,6cyclictest0-21swapper/319:40:233
2239199111,7cyclictest0-21swapper/319:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional