You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 01:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 01, 2026 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1332299690,40cyclictest23559-21ssh10:35:162
1330399639,44cyclictest5420-21ssh10:25:140
1330399556,41cyclictest21731-21ssh11:10:150
1330399556,41cyclictest21731-21ssh11:10:140
13322995317,18cyclictest27-21ksoftirqd/207:45:142
1331499532,41cyclictest10985-21ssh11:40:131
1331499532,41cyclictest10985-21ssh11:40:121
1331499532,40cyclictest25065-21ssh10:00:151
1330399517,8cyclictest131rcu_preempt08:35:130
1330399515,7cyclictest131rcu_preempt07:15:110
1330399503,7cyclictest131rcu_preempt08:50:130
13303995025,12cyclictest131rcu_preempt07:35:130
1333199493,29cyclictest35966irq/58-eth010:35:173
1332299474,28cyclictest26536-21kworker/u8:311:20:122
1333199453,25cyclictest35966irq/58-eth009:45:143
13331994514,22cyclictest35966irq/58-eth010:00:153
1330399453,4cyclictest131rcu_preempt11:55:140
1331499443,36cyclictest12952-21ssh12:35:131
1331499443,31cyclictest21426-21ssh09:40:151
13331994314,20cyclictest35966irq/58-eth011:05:153
1331499431,37cyclictest29033-21sh11:50:121
1330399422,32cyclictest31970-21ssh09:10:120
1333199411,25cyclictest35966irq/58-eth009:25:153
1333199405,24cyclictest35966irq/58-eth011:25:123
1332299382,19cyclictest767-21ssh11:34:032
13314993711,19cyclictest21-21ksoftirqd/109:05:131
13331993617,11cyclictest0-21swapper/310:50:153
13331993615,13cyclictest35966irq/58-eth010:45:133
1332299363,23cyclictest12968-21ssh12:35:152
1332299351,22cyclictest16387-21ssh11:25:132
1332299351,21cyclictest29042-21ssh11:50:122
13303993515,10cyclictest131rcu_preempt07:45:130
1332299346,20cyclictest0-21swapper/208:45:132
1333199337,18cyclictest35966irq/58-eth009:20:123
1333199337,18cyclictest35966irq/58-eth009:20:113
1333199332,22cyclictest35966irq/58-eth010:05:123
1331499332,23cyclictest30396-21sh09:45:121
13303993321,7cyclictest14698-21ssh12:00:130
1333199321,21cyclictest35966irq/58-eth011:15:133
1333199321,21cyclictest35966irq/58-eth011:15:123
1332299324,9cyclictest27-21ksoftirqd/208:50:132
1332299321,15cyclictest5326-21ssh11:18:042
1332299321,15cyclictest5326-21ssh11:18:042
13322993211,15cyclictest23663-21ssh12:05:152
1331499322,22cyclictest19731-21ssh10:15:131
13303993225,4cyclictest131rcu_preempt09:55:170
1330399322,18cyclictest151rcuc/011:20:120
1333199311,7cyclictest0-21swapper/312:20:143
1333199311,7cyclictest0-21swapper/312:20:133
13331993117,6cyclictest0-21swapper/309:00:153
1332299313,17cyclictest27346irq/40-dwc2_hso07:30:132
13314993122,5cyclictest0-21swapper/109:20:111
13314993122,5cyclictest0-21swapper/109:20:101
1331499311,20cyclictest0-21swapper/109:35:131
1330399312,20cyclictest8625-21ssh09:15:140
1333199306,16cyclictest0-21swapper/311:45:143
1332299302,21cyclictest0-21swapper/208:35:142
1331499302,23cyclictest27090-21ssh10:55:161
1333199299,12cyclictest35966irq/58-eth011:35:123
13331992917,7cyclictest0-21swapper/312:00:133
1332299292,21cyclictest0-21swapper/207:35:152
1332299292,20cyclictest0-21swapper/208:05:152
1330399291,24cyclictest3654-21ssh11:00:130
1333199285,14cyclictest35966irq/58-eth010:40:143
1333199282,16cyclictest36066irq/59-eth012:25:163
1333199282,16cyclictest35966irq/58-eth007:45:143
1332299282,20cyclictest3326-21ssh09:30:132
1331499283,15cyclictest14499-21ssh10:30:121
1331499281,21cyclictest10729-21ssh10:10:161
1333199278,4cyclictest0-21swapper/309:18:133
1333199272,3cyclictest0-21swapper/311:10:153
1333199272,3cyclictest0-21swapper/311:10:143
1333199272,15cyclictest35966irq/58-eth012:10:153
1333199271,6cyclictest0-21swapper/311:55:173
1333199271,4cyclictest0-21swapper/310:55:313
1332299273,16cyclictest27346irq/40-dwc2_hso10:45:132
1332299271,19cyclictest0-21swapper/207:25:142
1332299271,19cyclictest0-21swapper/207:25:132
13314992716,4cyclictest21-21ksoftirqd/112:15:171
13314992714,4cyclictest21-21ksoftirqd/108:40:271
1330399279,11cyclictest630-21Async09:36:310
1330399273,20cyclictest21399-21ssh09:40:120
1330399272,7cyclictest131rcu_preempt11:16:040
1330399272,7cyclictest131rcu_preempt11:16:030
13303992716,8cyclictest12-21ksoftirqd/010:10:150
1333199262,7cyclictest0-21swapper/310:10:323
1333199262,4cyclictest0-21swapper/312:15:193
1333199261,7cyclictest35966irq/58-eth011:20:123
1333199261,6cyclictest0-21swapper/310:25:203
1333199261,6cyclictest0-21swapper/310:20:253
13331992610,10cyclictest35966irq/58-eth012:30:033
1332299262,9cyclictest16299-21ssh11:41:022
1332299262,9cyclictest16299-21ssh11:41:012
1332299262,8cyclictest7021-21ssh09:50:132
13322992614,5cyclictest12754-21ssh10:28:322
1331499262,15cyclictest30758-21ssh11:15:131
1331499262,15cyclictest30758-21ssh11:15:121
13314992611,9cyclictest201rcuc/111:46:021
1330399266,17cyclictest12-21ksoftirqd/009:25:140
1330399263,19cyclictest12345-21ssh09:35:140
1333199255,3cyclictest0-21swapper/311:50:193
1333199253,10cyclictest0-21swapper/311:41:023
1333199253,10cyclictest0-21swapper/311:41:013
1333199251,7cyclictest0-21swapper/309:14:043
1333199251,21cyclictest0-21swapper/309:40:303
1332299259,7cyclictest131rcu_preempt10:35:022
1332299259,13cyclictest0-21swapper/209:40:172
13322992511,9cyclictest0-21swapper/212:25:152
1332299251,15cyclictest20078-21ssh11:45:152
1332299251,13cyclictest17687-21ssh09:20:132
1332299251,13cyclictest17687-21ssh09:20:122
1331499259,8cyclictest21-21ksoftirqd/110:22:201
1331499258,9cyclictest21-21ksoftirqd/110:50:221
1331499256,10cyclictest21-21ksoftirqd/111:37:011
13314992512,4cyclictest21-21ksoftirqd/107:40:271
1330399251,20cyclictest25075-21ssh10:00:150
1333199245,11cyclictest35966irq/58-eth008:40:143
1333199241,9cyclictest35966irq/58-eth012:05:133
1333199241,8cyclictest0-21swapper/310:15:303
1333199241,7cyclictest0-21swapper/309:57:583
1333199241,6cyclictest0-21swapper/312:35:183
1333199241,6cyclictest0-21swapper/308:45:273
1332299249,7cyclictest131rcu_preempt10:20:022
1332299242,16cyclictest0-21swapper/211:55:042
1332299242,10cyclictest0-21swapper/207:20:132
1332299241,18cyclictest9306-21ssh12:15:152
1331499249,8cyclictest21-21ksoftirqd/108:05:281
1331499249,4cyclictest21-21ksoftirqd/112:24:091
1331499249,4cyclictest21-21ksoftirqd/112:24:081
1331499245,3cyclictest131rcu_preempt09:15:211
1331499242,6cyclictest21-21ksoftirqd/109:50:271
13314992410,4cyclictest21-21ksoftirqd/111:20:481
1330399243,16cyclictest151rcuc/012:35:120
1333199233,3cyclictest0-21swapper/309:30:163
1333199232,7cyclictest0-21swapper/311:30:163
1333199232,6cyclictest0-21swapper/310:30:293
1333199231,7cyclictest0-21swapper/308:35:153
1333199231,6cyclictest0-21swapper/311:00:233
1333199231,6cyclictest0-21swapper/308:35:013
1333199231,6cyclictest0-21swapper/308:25:023
1333199231,6cyclictest0-21swapper/308:20:253
1333199231,6cyclictest0-21swapper/308:05:163
1333199231,6cyclictest0-21swapper/307:35:203
1333199231,5cyclictest0-21swapper/309:05:263
1333199231,5cyclictest0-21swapper/308:00:213
1332299237,4cyclictest0-21swapper/212:10:132
1332299232,6cyclictest0-21swapper/211:36:002
1332299232,16cyclictest0-21swapper/212:21:042
1332299232,16cyclictest0-21swapper/212:21:032
1332299232,16cyclictest0-21swapper/211:06:032
1332299232,16cyclictest0-21swapper/209:10:132
13322992312,9cyclictest24734-21ssh10:52:412
1332299231,12cyclictest1792-21ssh09:45:282
1332299231,11cyclictest14723-21ssh12:00:132
1331499237,2cyclictest21-21ksoftirqd/111:12:311
1331499237,2cyclictest21-21ksoftirqd/111:12:311
1331499236,7cyclictest21-21ksoftirqd/112:26:501
1331499236,3cyclictest131rcu_preempt09:10:151
1331499235,8cyclictest131rcu_preempt12:05:261
1331499235,3cyclictest131rcu_preempt11:30:241
1331499234,5cyclictest21-21ksoftirqd/108:50:251
1331499234,4cyclictest21-21ksoftirqd/108:25:311
1331499233,6cyclictest21-21ksoftirqd/111:55:191
1331499233,5cyclictest21-21ksoftirqd/111:25:201
1331499233,12cyclictest0-21swapper/110:35:151
1331499231,4cyclictest21-21ksoftirqd/107:10:021
13314992312,4cyclictest21-21ksoftirqd/108:20:021
13314992312,4cyclictest21-21ksoftirqd/108:20:011
13314992311,9cyclictest1-21systemd09:25:411
13314992311,10cyclictest0-21swapper/111:07:391
13314992311,10cyclictest0-21swapper/111:04:101
13314992311,10cyclictest0-21swapper/110:09:181
1330399238,3cyclictest131rcu_preempt12:15:140
1330399235,10cyclictest0-21swapper/007:20:130
13303992316,4cyclictest3932-21sh12:30:140
1330399231,5cyclictest4239-21threads12:10:310
1330399231,3cyclictest131rcu_preempt10:20:150
1330399231,18cyclictest18292-21ssh12:20:120
1330399231,18cyclictest18292-21ssh12:20:120
1333199222,6cyclictest0-21swapper/309:35:203
1333199221,6cyclictest0-21swapper/309:50:203
1333199221,6cyclictest0-21swapper/308:55:213
1333199221,6cyclictest0-21swapper/308:50:173
1333199221,6cyclictest0-21swapper/308:15:293
1333199221,6cyclictest0-21swapper/308:15:283
1333199221,6cyclictest0-21swapper/307:55:183
1333199221,6cyclictest0-21swapper/307:35:013
1333199221,6cyclictest0-21swapper/307:20:153
1333199221,6cyclictest0-21swapper/307:15:173
1333199221,6cyclictest0-21swapper/307:10:193
1332299229,10cyclictest0-21swapper/208:00:202
1332299222,6cyclictest0-21swapper/210:05:052
1332299222,15cyclictest0-21swapper/211:12:032
1332299222,15cyclictest0-21swapper/211:12:022
1332299221,19cyclictest29107-21ssh10:55:232
1331499229,7cyclictest21-21ksoftirqd/109:56:161
1331499229,5cyclictest21-21ksoftirqd/109:30:581
1331499227,10cyclictest14722-21apt-get07:45:151
1331499226,9cyclictest21-21ksoftirqd/108:25:011
1331499223,9cyclictest21-21ksoftirqd/112:30:191
1331499223,12cyclictest21-21ksoftirqd/110:46:381
13314992215,5cyclictest201rcuc/110:25:131
1331499221,18cyclictest26300-21ls07:20:201
1331499221,15cyclictest32570-21sh10:40:141
1330399224,3cyclictest0-21swapper/009:50:210
1330399224,16cyclictest12663-21ssh11:05:140
1330399222,16cyclictest30413-21ssh09:45:140
1330399221,3cyclictest131rcu_preempt10:15:230
1330399221,2cyclictest131rcu_preempt07:40:320
1330399221,15cyclictest12-21ksoftirqd/008:20:110
1333199211,6cyclictest0-21swapper/308:15:023
1333199211,6cyclictest0-21swapper/307:50:243
1333199211,6cyclictest0-21swapper/307:50:233
1333199211,6cyclictest0-21swapper/307:40:223
1333199211,6cyclictest0-21swapper/307:25:213
1333199211,6cyclictest0-21swapper/307:25:203
1332299215,8cyclictest540-21sh10:40:192
1332299212,16cyclictest28997-21timerandwakeup10:00:302
1332299212,10cyclictest29861-21kthreadcore09:25:252
1332299211,16cyclictest12518-21latency_hist12:35:012
1332299211,10cyclictest31328-21kthreadcore10:20:242
1332299211,10cyclictest14485-21munin-run07:45:022
13322992110,9cyclictest8790-21ssh11:00:532
1331499218,7cyclictest21-21ksoftirqd/112:11:531
1331499218,7cyclictest21-21ksoftirqd/112:02:121
1331499217,11cyclictest18910-21sort07:15:181
1331499213,2cyclictest131rcu_preempt07:55:181
1331499211,4cyclictest21-21ksoftirqd/107:30:291
1330399217,4cyclictest131rcu_preempt10:30:170
1330399212,5cyclictest14294-21cut07:10:210
1330399211,8cyclictest24286-21/usr/sbin/munin07:55:240
1330399211,7cyclictest20020-21ntp_192.168.11411:25:240
1330399211,6cyclictest4007-21kthreadcore11:35:200
1330399211,5cyclictest0-21swapper/009:21:290
1330399211,5cyclictest0-21swapper/009:21:280
1330399211,2cyclictest131rcu_preempt10:45:250
1330399211,17cyclictest6188-21ls08:40:230
13303992110,9cyclictest151rcuc/011:40:150
13303992110,9cyclictest151rcuc/011:40:140
1332299207,6cyclictest27174-21fw_packets09:05:162
1332299201,9cyclictest8812-21sort09:15:172
1332299201,9cyclictest20973-21switchtime08:55:292
1332299201,9cyclictest0-21swapper/210:10:152
1332299201,13cyclictest0-21swapper/207:50:222
1332299201,13cyclictest0-21swapper/207:50:212
1332299201,10cyclictest30379-21wc08:35:012
1332299201,10cyclictest22800-21sendmail07:20:012
1332299201,10cyclictest19693-21ntpq09:55:292
1332299201,10cyclictest12545-21fschecks_time09:35:172
1331499208,3cyclictest131rcu_preempt07:50:171
1331499208,3cyclictest131rcu_preempt07:50:161
1331499208,3cyclictest131rcu_preempt07:10:191
1331499204,2cyclictest131rcu_preempt08:00:331
1331499203,4cyclictest131rcu_preempt08:10:221
1331499201,14cyclictest30484-21cpuspeed08:35:131
1330399208,7cyclictest131rcu_preempt11:31:030
1330399207,10cyclictest15748-21cron08:20:010
1330399207,10cyclictest15748-21cron08:20:000
1330399206,4cyclictest131rcu_preempt08:25:270
1330399202,15cyclictest4143-21kthreadcore08:05:230
1330399201,8cyclictest821-21irqstats10:40:200
13303992014,4cyclictest0-21swapper/010:35:160
1330399201,3cyclictest131rcu_preempt09:30:130
13303992013,5cyclictest0-21swapper/008:45:220
1330399201,15cyclictest20054-21apt-get11:45:150
1332299196,10cyclictest0-21swapper/208:20:132
1332299195,11cyclictest27346irq/40-dwc2_hso08:40:132
1332299193,10cyclictest20847-21acpi08:25:132
1332299191,9cyclictest24636-21irqcore07:55:172
1332299191,9cyclictest15789-21kthreadcore07:10:222
1332299191,11cyclictest10264-21unixbench_singl08:10:302
1331499197,7cyclictest21-21ksoftirqd/108:45:291
1331499197,7cyclictest21-21ksoftirqd/107:35:191
1331499194,3cyclictest131rcu_preempt08:55:201
1331499192,4cyclictest131rcu_preempt07:25:201
1331499192,4cyclictest131rcu_preempt07:25:191
1330399199,7cyclictest28514-21cat12:05:320
1330399199,7cyclictest18131-21apt-get10:50:150
1330399197,9cyclictest2308-21sed10:05:180
1330399195,4cyclictest131rcu_preempt07:50:220
1330399195,4cyclictest131rcu_preempt07:50:210
1330399192,13cyclictest32546-21memory11:50:230
1330399191,9cyclictest8971-21tr08:10:230
1330399191,14cyclictest0-21swapper/008:00:150
1330399191,11cyclictest25656-21find08:30:120
1332299181,7cyclictest11406-21wc08:15:172
1332299181,7cyclictest11406-21wc08:15:172
1332299180,11cyclictest22183-21apt-get09:00:132
1331499187,3cyclictest131rcu_preempt08:30:201
1331499181,8cyclictest21-21ksoftirqd/109:00:161
1330399186,3cyclictest131rcu_preempt10:55:220
1330399181,6cyclictest17337-21/usr/sbin/munin08:55:290
13303991812,4cyclictest0-21swapper/012:25:280
1330399181,14cyclictest22367-21uniq09:00:170
1332299171,8cyclictest13371-21latency_hist07:10:022
13303991710,3cyclictest131rcu_preempt07:25:130
13303991710,3cyclictest131rcu_preempt07:25:120
1330399170,7cyclictest30763-21timerwakeupswit09:05:280
1330399141,10cyclictest13380-21latency_hist07:10:020
1333199111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional