You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 06:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Apr 14, 2026 00:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234399816,26cyclictest27-21ksoftirqd/219:15:132
234399816,26cyclictest27-21ksoftirqd/219:15:132
2352996649,10cyclictest35966irq/58-eth021:35:123
234399634,34cyclictest25586-21ssh23:15:142
2323996328,11cyclictest12-21ksoftirqd/022:05:120
232399624,44cyclictest15715-21ssh00:20:130
2323996031,20cyclictest11209-21ssh00:00:130
232399562,45cyclictest6511-21ssh00:15:120
232399543,42cyclictest7196-21ssh21:55:120
2352995221,19cyclictest35966irq/58-eth000:35:133
235299505,25cyclictest35966irq/58-eth000:30:143
232399492,32cyclictest2704-21ssh22:10:130
234399475,30cyclictest7025-21ssh23:05:132
232399473,29cyclictest11735-21ssh22:50:120
232399462,9cyclictest131rcu_preempt00:35:140
233399454,33cyclictest24929-21ssh00:25:121
233399454,33cyclictest24929-21ssh00:25:121
235299443,24cyclictest35966irq/58-eth022:30:123
235299423,24cyclictest35966irq/58-eth000:10:133
233399425,28cyclictest30231-21ssh23:00:131
2323994214,19cyclictest12-21ksoftirqd/020:55:130
232399421,32cyclictest25215-21ssh23:50:140
232399421,32cyclictest25215-21ssh23:50:140
235299414,26cyclictest35966irq/58-eth022:10:143
234399401,33cyclictest30410-21ssh22:25:122
234399401,33cyclictest30410-21ssh22:25:122
232399404,26cyclictest20932-21ssh21:10:140
232399404,26cyclictest20932-21ssh21:10:140
232399401,6cyclictest131rcu_preempt00:30:150
235299385,21cyclictest35966irq/58-eth021:55:123
233399386,19cyclictest8335-21kworker/u8:021:25:131
235299375,17cyclictest35966irq/58-eth022:15:143
235299373,22cyclictest35966irq/58-eth000:20:133
235299372,27cyclictest35966irq/58-eth023:50:143
235299372,27cyclictest35966irq/58-eth023:50:143
235299353,19cyclictest35966irq/58-eth022:35:153
235299352,20cyclictest35966irq/58-eth022:50:143
234399357,20cyclictest27-21ksoftirqd/221:50:122
234399355,16cyclictest0-21swapper/220:50:132
232399354,20cyclictest21225-21ssh22:20:130
232399351,9cyclictest131rcu_preempt22:40:120
2352993420,7cyclictest35966irq/58-eth000:25:133
2352993420,7cyclictest35966irq/58-eth000:25:133
232399342,26cyclictest16383-21ssh21:25:130
235299331,20cyclictest35966irq/58-eth021:25:133
233399332,23cyclictest6842-21ssh23:40:141
234399315,12cyclictest25643-21ssh21:30:142
234399313,23cyclictest16465-21ssh22:35:152
234399313,20cyclictest0-21swapper/222:50:122
2333993121,6cyclictest0-21swapper/100:35:151
232399312,22cyclictest20834-21ssh23:30:140
235299306,17cyclictest0-21swapper/300:15:133
2333993015,8cyclictest0-21swapper/119:15:151
2333993015,8cyclictest0-21swapper/119:15:141
232399307,15cyclictest12-21ksoftirqd/023:40:140
232399302,21cyclictest29732-21ssh00:10:140
232399301,21cyclictest16460-21ssh22:35:140
234399293,18cyclictest27346irq/40-dwc2_hso21:05:122
234399293,10cyclictest27-21ksoftirqd/221:15:142
234399293,10cyclictest27-21ksoftirqd/221:15:142
2352992813,6cyclictest35966irq/58-eth022:00:133
234399286,15cyclictest131rcu_preempt19:45:162
234399281,16cyclictest11860-21ssh21:40:142
2343992811,12cyclictest261rcuc/200:35:152
234399279,8cyclictest27-21ksoftirqd/222:42:522
234399275,17cyclictest27-21ksoftirqd/200:20:132
234399274,11cyclictest2309-21cyclictest23:25:112
234399273,16cyclictest27346irq/40-dwc2_hso20:45:142
234399272,20cyclictest0-21swapper/222:00:152
234399272,19cyclictest0-21swapper/223:00:152
2343992716,8cyclictest5595-21sh23:55:242
234399271,20cyclictest2728-21ssh22:10:142
234399271,15cyclictest7257-21rm22:30:142
233399276,10cyclictest16344-21ssh23:10:131
233399276,10cyclictest16344-21ssh23:10:131
233399273,19cyclictest30224-21ssh21:15:151
233399273,19cyclictest30224-21ssh21:15:151
233399272,19cyclictest20505-21diskmemload00:05:141
2333992711,4cyclictest21-21ksoftirqd/122:24:521
232399272,18cyclictest7256-21sh22:30:140
232399271,20cyclictest2560-21ssh22:45:140
232399270,18cyclictest12-21ksoftirqd/021:00:140
235299263,15cyclictest35966irq/58-eth021:45:123
2352992612,10cyclictest35966irq/58-eth022:20:133
234399268,12cyclictest0-21swapper/219:35:152
234399264,8cyclictest0-21swapper/222:05:132
234399262,18cyclictest8335-21kworker/u8:020:40:052
234399262,10cyclictest0-21swapper/220:20:152
2343992617,4cyclictest20462-21sh00:05:142
2343992610,4cyclictest27-21ksoftirqd/221:56:472
2333992617,5cyclictest0-21swapper/120:05:141
2333992615,7cyclictest0-21swapper/120:45:141
2333992614,3cyclictest21-21ksoftirqd/122:09:071
2333992610,11cyclictest21-21ksoftirqd/121:30:151
234399257,7cyclictest261rcuc/223:45:152
234399253,17cyclictest27346irq/40-dwc2_hso21:35:122
234399251,8cyclictest32569-21find20:15:132
234399251,21cyclictest11497-21ls00:00:172
234399251,17cyclictest27346irq/40-dwc2_hso20:35:142
233399258,5cyclictest131rcu_preempt23:05:151
233399256,9cyclictest21-21ksoftirqd/121:57:591
2333992512,4cyclictest0-21swapper/123:33:011
2333992510,8cyclictest21-21ksoftirqd/121:20:231
232399259,7cyclictest12-21ksoftirqd/020:40:130
2323992515,6cyclictest20505-21diskmemload23:20:130
2343992419,3cyclictest0-21swapper/219:20:252
234399241,11cyclictest6836-21ssh23:40:142
2343992410,8cyclictest27-21ksoftirqd/220:00:282
234399240,14cyclictest13010-21kthreadcore21:00:212
233399248,8cyclictest21-21ksoftirqd/123:48:131
233399247,8cyclictest21-21ksoftirqd/121:51:001
233399246,9cyclictest21-21ksoftirqd/123:55:501
233399244,16cyclictest29946-21ssh23:35:131
233399243,6cyclictest21-21ksoftirqd/121:15:011
233399243,6cyclictest21-21ksoftirqd/121:15:001
233399241,5cyclictest21-21ksoftirqd/122:55:241
2333992413,3cyclictest0-21swapper/121:00:181
232399242,3cyclictest131rcu_preempt21:15:140
232399242,3cyclictest131rcu_preempt21:15:140
235299239,7cyclictest0-21swapper/321:00:133
235299235,12cyclictest0-21swapper/322:45:143
2352992316,3cyclictest0-21swapper/321:10:123
2352992316,3cyclictest0-21swapper/321:10:123
234399237,6cyclictest131rcu_preempt21:20:512
234399236,4cyclictest131rcu_preempt22:57:062
234399235,10cyclictest24786-21ssh21:45:262
2343992311,4cyclictest27-21ksoftirqd/222:49:042
234399231,13cyclictest2407-21ssh23:20:142
234399230,16cyclictest27346irq/40-dwc2_hso21:25:142
234399230,11cyclictest5273-21ls00:30:242
233399238,4cyclictest21-21ksoftirqd/123:21:301
233399236,9cyclictest21-21ksoftirqd/121:50:011
233399236,7cyclictest21-21ksoftirqd/100:32:001
233399236,7cyclictest21-21ksoftirqd/100:21:021
233399235,8cyclictest21-21ksoftirqd/121:37:061
233399234,8cyclictest21-21ksoftirqd/100:03:581
233399233,8cyclictest21-21ksoftirqd/122:31:521
2333992315,3cyclictest0-21swapper/120:15:131
2333992312,3cyclictest0-21swapper/122:36:061
2333992311,4cyclictest21-21ksoftirqd/100:13:381
2333992311,4cyclictest0-21swapper/123:51:151
2333992311,4cyclictest0-21swapper/123:51:151
2333992310,4cyclictest21-21ksoftirqd/122:53:051
2333992310,4cyclictest21-21ksoftirqd/122:48:491
232399232,16cyclictest7050-21ssh23:05:150
2323992317,2cyclictest0-21swapper/021:50:130
2323992312,7cyclictest614-21dbus-daemon20:12:160
232399231,17cyclictest11624-21ssh23:25:140
234399222,11cyclictest32328-21kthreadcore23:35:212
2343992210,4cyclictest27-21ksoftirqd/200:10:212
234399220,5cyclictest6808-21ls00:15:172
233399228,4cyclictest21-21ksoftirqd/122:16:431
233399227,9cyclictest21-21ksoftirqd/119:45:291
233399227,7cyclictest21-21ksoftirqd/123:27:161
233399227,6cyclictest21-21ksoftirqd/122:00:571
233399227,4cyclictest21-21ksoftirqd/123:16:191
233399227,3cyclictest131rcu_preempt22:10:311
233399226,7cyclictest21-21ksoftirqd/122:40:461
233399226,7cyclictest21-21ksoftirqd/122:27:291
233399226,7cyclictest21-21ksoftirqd/122:27:291
233399225,4cyclictest21-21ksoftirqd/120:20:261
233399223,9cyclictest21-21ksoftirqd/120:00:291
2333992210,3cyclictest131rcu_preempt20:40:021
232399229,4cyclictest131rcu_preempt19:25:280
232399221,18cyclictest21294-21cut21:45:160
232399221,18cyclictest18273-21ls20:00:170
232399221,16cyclictest19928-21ls22:00:230
232399220,17cyclictest30714-21kthreadcore23:35:190
235299211,10cyclictest0-21swapper/321:20:133
2352992110,8cyclictest0-21swapper/322:44:533
234399218,3cyclictest131rcu_preempt23:33:232
234399217,5cyclictest15316-21kthreadcore19:55:202
234399216,12cyclictest131rcu_preempt20:10:142
234399216,12cyclictest131rcu_preempt19:30:262
234399212,10cyclictest23707-21kthreadcore22:20:202
2343992115,4cyclictest0-21swapper/223:50:302
2343992115,4cyclictest0-21swapper/223:50:302
2343992113,6cyclictest11823-21sshd22:15:052
234399211,18cyclictest24937-21kthreadcore20:05:212
2343992111,4cyclictest0-21swapper/200:25:242
2343992111,4cyclictest0-21swapper/200:25:242
234399211,11cyclictest16334-21ssh23:10:122
234399211,11cyclictest16334-21ssh23:10:122
2343992110,9cyclictest27299-21ssh21:12:162
2343992110,9cyclictest27299-21ssh21:12:162
233399217,8cyclictest21-21ksoftirqd/100:19:191
233399215,8cyclictest21-21ksoftirqd/119:35:011
233399214,4cyclictest21-21ksoftirqd/120:50:231
233399211,17cyclictest8618-21cut19:50:161
233399210,18cyclictest6151-21ls19:10:281
233399210,16cyclictest20619-21ls19:25:271
235299204,12cyclictest0-21swapper/323:20:143
2352992011,6cyclictest35966irq/58-eth000:00:143
2352992010,8cyclictest0-21swapper/323:46:353
234399209,9cyclictest14343-21sshd20:27:092
234399201,9cyclictest20668-21grep19:25:282
234399201,16cyclictest14819-21wc20:30:182
234399201,10cyclictest3738-21kthreadcore19:10:202
233399206,9cyclictest21-21ksoftirqd/120:25:221
233399206,8cyclictest21-21ksoftirqd/120:35:011
233399204,10cyclictest0-21swapper/119:55:131
233399203,8cyclictest21-21ksoftirqd/121:40:361
232399207,10cyclictest3440-21sh19:45:010
232399203,7cyclictest25505-21irqrtprio00:25:180
232399203,7cyclictest25505-21irqrtprio00:25:170
232399201,15cyclictest7242-21apt-get19:15:140
232399201,15cyclictest7242-21apt-get19:15:130
232399201,15cyclictest7125-21ssh21:20:140
232399201,10cyclictest30197-21/usr/sbin/munin23:00:150
235299199,7cyclictest0-21swapper/323:01:243
235299199,7cyclictest0-21swapper/300:05:213
235299194,9cyclictest35966irq/58-eth023:35:123
2352991910,6cyclictest35966irq/58-eth023:05:153
2352991910,6cyclictest0-21swapper/321:52:523
2352991910,3cyclictest0-21swapper/323:15:283
234399191,9cyclictest8905-21kthreadcore20:55:212
2343991913,4cyclictest0-21swapper/219:50:152
234399191,15cyclictest32266-21kthreadcore19:40:192
233399197,4cyclictest0-21swapper/120:10:021
233399192,9cyclictest21-21ksoftirqd/119:20:261
232399197,9cyclictest8201-21cron19:50:010
232399195,7cyclictest0-21swapper/020:50:130
232399194,5cyclictest131rcu_preempt00:00:010
232399191,6cyclictest13015-21munin-run19:55:010
232399191,14cyclictest20951-21diskmemload22:55:130
232399191,11cyclictest20471-21grep00:05:140
235299189,6cyclictest0-21swapper/323:43:073
235299188,7cyclictest0-21swapper/321:30:193
235299182,12cyclictest0-21swapper/322:55:133
235299182,10cyclictest0-21swapper/320:25:143
235299182,10cyclictest0-21swapper/320:20:153
235299182,10cyclictest0-21swapper/319:50:143
235299182,10cyclictest0-21swapper/319:25:123
2352991813,3cyclictest33-21ksoftirqd/321:40:053
235299181,11cyclictest35966irq/58-eth023:25:143
233399181,9cyclictest21-21ksoftirqd/119:40:141
233399181,5cyclictest21-21ksoftirqd/119:35:211
233399181,3cyclictest131rcu_preempt20:55:181
232399189,6cyclictest18480-21kthreadcore21:05:210
232399188,4cyclictest12-21ksoftirqd/019:10:140
232399187,8cyclictest12-21ksoftirqd/023:45:150
232399182,13cyclictest4786-21kthreadcore21:35:200
232399181,2cyclictest131rcu_preempt20:15:130
232399181,11cyclictest29142-21ssh21:30:250
2323991810,5cyclictest31076-21latency_hist19:40:010
2323991810,5cyclictest20505-21diskmemload22:15:290
235299179,6cyclictest0-21swapper/323:55:313
235299179,6cyclictest0-21swapper/322:26:373
235299179,6cyclictest0-21swapper/322:26:373
235299179,6cyclictest0-21swapper/322:05:343
235299172,10cyclictest0-21swapper/320:55:143
235299172,10cyclictest0-21swapper/320:50:143
235299172,10cyclictest0-21swapper/320:30:143
235299172,10cyclictest0-21swapper/319:40:143
2352991711,4cyclictest0-21swapper/323:10:233
2352991711,4cyclictest0-21swapper/323:10:233
235299171,10cyclictest0-21swapper/321:15:303
235299171,10cyclictest0-21swapper/321:15:303
233399177,7cyclictest2309-21cyclictest21:05:121
232399178,6cyclictest12-21ksoftirqd/020:05:120
232399174,11cyclictest29863-21kthreadcore20:45:200
232399172,12cyclictest20657-21idleruntime-cro21:45:010
232399172,10cyclictest0-21swapper/020:30:150
232399172,10cyclictest0-21swapper/019:30:150
232399171,7cyclictest18013-21tr23:10:220
232399171,7cyclictest18013-21tr23:10:220
2323991711,4cyclictest29265-21sh23:15:270
2323991711,4cyclictest12675-21kernelversion19:20:170
232399171,13cyclictest14036-21kthreadcore19:55:190
232399171,12cyclictest11463-21kthreadcore20:25:210
235299169,5cyclictest35966irq/58-eth023:30:133
235299169,3cyclictest0-21swapper/321:05:123
234399161,6cyclictest2385-21latency_hist19:10:022
233399164,4cyclictest21-21ksoftirqd/120:40:051
233399163,4cyclictest21-21ksoftirqd/119:10:021
232399162,9cyclictest0-21swapper/020:35:150
232399162,9cyclictest0-21swapper/020:20:150
232399161,12cyclictest31703-21ssh22:25:200
232399161,12cyclictest31703-21ssh22:25:200
235299151,9cyclictest0-21swapper/319:20:263
235299151,7cyclictest0-21swapper/319:15:193
235299151,7cyclictest0-21swapper/319:15:193
235299149,3cyclictest0-21swapper/320:10:183
235299146,6cyclictest0-21swapper/319:55:043
235299142,8cyclictest0-21swapper/320:10:013
235299141,7cyclictest0-21swapper/319:50:023
232399142,10cyclictest2325-21latency_hist19:10:020
235299132,9cyclictest0-21swapper/320:45:293
235299131,5cyclictest0-21swapper/319:35:153
235299131,5cyclictest0-21swapper/319:10:223
235299124,6cyclictest0-21swapper/320:03:543
235299122,6cyclictest0-21swapper/320:35:153
235299122,6cyclictest0-21swapper/320:15:043
235299122,6cyclictest0-21swapper/319:30:153
235299122,4cyclictest0-21swapper/320:40:123
235299101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional