You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 15:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 09, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16600996314,5cyclictest131rcu_preempt11:40:132
1660099612,43cyclictest930-21snmpd10:15:132
1660099582,33cyclictest29300-21ssh12:25:142
1660099560,32cyclictest22285-21ssh11:45:142
1660099543,6cyclictest131rcu_preempt11:35:112
16610995130,13cyclictest35966irq/58-eth009:55:123
1658099514,8cyclictest131rcu_preempt11:10:140
16600994924,14cyclictest131rcu_preempt07:55:132
1658099495,36cyclictest20772-21ssh09:20:130
1661099484,27cyclictest35966irq/58-eth007:25:133
1658099482,27cyclictest12-21ksoftirqd/008:10:110
16600994716,17cyclictest27-21ksoftirqd/208:40:122
16600994631,9cyclictest27-21ksoftirqd/210:00:142
16600994512,17cyclictest131rcu_preempt10:40:122
1659199454,33cyclictest25665-21ssh12:05:121
1658099443,23cyclictest31574-21ssh10:20:140
1658099433,30cyclictest24385-21ssh09:40:120
1661099422,25cyclictest35966irq/58-eth009:35:113
16600994217,20cyclictest0-21swapper/211:30:122
1659199422,36cyclictest28038-21ssh10:00:121
1658099421,31cyclictest28059-21ssh10:00:140
1658099413,18cyclictest12-21ksoftirqd/012:15:140
1661099404,24cyclictest35966irq/58-eth010:50:153
1659199404,32cyclictest2689-21ssh09:10:131
1658099404,22cyclictest2799-21ssh10:40:120
1658099402,35cyclictest20847-21ssh10:50:130
1660099392,27cyclictest20292-21ssh12:20:122
1660099391,3cyclictest131rcu_preempt09:55:122
16591993916,19cyclictest21-21ksoftirqd/108:55:131
1658099396,21cyclictest12-21ksoftirqd/009:45:140
16580993916,7cyclictest131rcu_preempt08:15:130
1661099388,18cyclictest35966irq/58-eth011:50:133
1661099382,25cyclictest35966irq/58-eth011:10:143
1660099371,24cyclictest15382-21ssh11:05:122
1659199371,28cyclictest10014-21sh09:50:131
16600993626,5cyclictest15379-21ssh09:35:102
16600993519,6cyclictest2287-21diskmemload11:10:142
1659199358,21cyclictest2246-21ssh12:10:121
1661099342,21cyclictest35966irq/58-eth010:20:143
1659199331,3cyclictest21-21ksoftirqd/111:25:141
1658099338,11cyclictest0-21swapper/011:30:140
1660099325,16cyclictest27346irq/40-dwc2_hso09:50:142
1659199323,19cyclictest21-21ksoftirqd/110:05:131
1659199323,19cyclictest21-21ksoftirqd/110:05:131
1659199322,25cyclictest29811-21ssh10:55:121
1661099312,19cyclictest35966irq/58-eth010:10:133
1661099312,14cyclictest35966irq/58-eth008:30:123
1660099314,11cyclictest18867-21ssh11:25:122
1660099313,20cyclictest27346irq/40-dwc2_hso11:00:122
16591993120,6cyclictest2287-21diskmemload09:55:141
16591993116,10cyclictest31207-21apt-get11:50:131
1658099312,24cyclictest15363-21ssh11:05:110
1661099302,17cyclictest35966irq/58-eth012:15:153
1660099303,16cyclictest27346irq/40-dwc2_hso07:30:122
16600993011,14cyclictest27-21ksoftirqd/209:25:132
16600993010,5cyclictest131rcu_preempt07:25:112
1659199302,22cyclictest24413-21ssh09:20:261
1660099294,17cyclictest27346irq/40-dwc2_hso12:35:132
1660099293,11cyclictest0-21swapper/207:35:132
1659199292,21cyclictest22584-21ssh10:15:151
16580992910,4cyclictest131rcu_preempt11:25:130
1661099282,17cyclictest35966irq/58-eth011:30:133
1660099284,12cyclictest27346irq/40-dwc2_hso07:50:132
1660099284,12cyclictest27346irq/40-dwc2_hso07:50:122
1660099283,20cyclictest27346irq/40-dwc2_hso08:35:122
1659199282,20cyclictest8202-21ssh10:25:151
1660099273,13cyclictest27346irq/40-dwc2_hso08:10:122
1660099272,17cyclictest27346irq/40-dwc2_hso10:10:132
1660099272,15cyclictest27346irq/40-dwc2_hso12:10:112
1660099271,19cyclictest27346irq/40-dwc2_hso11:15:132
16591992714,8cyclictest11279-21apt-get12:15:131
1658099272,9cyclictest0-21swapper/007:45:120
1658099272,20cyclictest881-21ssh11:15:130
1658099271,8cyclictest12-21ksoftirqd/009:05:120
1658099271,3cyclictest0-21swapper/011:50:140
16610992615,8cyclictest0-21swapper/310:05:143
16610992615,8cyclictest0-21swapper/310:05:133
16610992615,7cyclictest35966irq/58-eth009:40:123
1661099261,17cyclictest0-21swapper/310:25:123
1660099263,15cyclictest27346irq/40-dwc2_hso08:20:122
1660099263,14cyclictest27346irq/40-dwc2_hso07:40:132
1659199264,4cyclictest21-21ksoftirqd/110:35:181
1659199263,5cyclictest21-21ksoftirqd/110:55:011
1659199262,4cyclictest21-21ksoftirqd/111:10:391
1659199262,18cyclictest17210-21ssh10:30:131
1659199262,18cyclictest17210-21ssh10:30:121
1659199262,16cyclictest6377-21sh09:30:131
1659199261,19cyclictest22283-21sh11:45:151
16591992610,11cyclictest13260-21apt-get11:40:121
1658099262,18cyclictest5904-21ssh12:30:130
16580992612,5cyclictest0-21swapper/008:25:120
1658099261,19cyclictest29762-21ssh09:25:140
1660099253,7cyclictest31215-21ssh11:50:132
1660099251,21cyclictest29882-21cpuspeed10:55:132
1660099250,17cyclictest4948-21ls09:45:302
1659199258,5cyclictest21-21ksoftirqd/110:10:311
1659199256,6cyclictest21-21ksoftirqd/111:15:221
1659199256,11cyclictest0-21swapper/111:20:111
1659199254,6cyclictest21-21ksoftirqd/110:45:251
1659199254,5cyclictest21-21ksoftirqd/111:05:201
1659199253,6cyclictest19962-21wc07:10:211
1659199253,4cyclictest21-21ksoftirqd/108:05:021
1659199252,4cyclictest18059-21diskstats07:45:141
16591992518,3cyclictest27717-21apt-get11:30:131
16591992516,4cyclictest22800-21apt-get07:50:121
16591992516,4cyclictest22800-21apt-get07:50:121
1658099258,4cyclictest12-21ksoftirqd/011:42:090
1658099252,18cyclictest11703-21ssh09:15:160
1658099251,18cyclictest12-21ksoftirqd/011:35:110
1661099242,17cyclictest35966irq/58-eth008:50:133
1661099242,17cyclictest35966irq/58-eth008:50:133
1660099244,10cyclictest31578-21sh10:20:142
1660099243,13cyclictest27346irq/40-dwc2_hso07:20:132
1660099241,8cyclictest9846-21ssh11:20:122
1660099240,21cyclictest24402-21ls10:50:242
1659199244,5cyclictest21-21ksoftirqd/109:29:471
1659199244,4cyclictest21-21ksoftirqd/111:05:021
1659199243,4cyclictest21-21ksoftirqd/111:35:261
1659199242,5cyclictest21-21ksoftirqd/112:25:131
1659199242,5cyclictest21-21ksoftirqd/107:20:141
1659199242,15cyclictest24407-21rm09:40:131
1659199241,7cyclictest32292-21ssh10:20:201
16591992417,4cyclictest11679-21apt-get09:15:131
1658099244,9cyclictest131rcu_preempt12:05:210
1658099244,10cyclictest0-21swapper/012:20:120
1658099243,3cyclictest131rcu_preempt12:10:310
1658099241,17cyclictest11827-21ssh10:45:140
1661099239,8cyclictest0-21swapper/309:15:163
1661099232,13cyclictest0-21swapper/310:00:143
16610992311,9cyclictest0-21swapper/312:30:213
1660099238,11cyclictest0-21swapper/208:15:302
1660099234,14cyclictest27346irq/40-dwc2_hso09:30:112
1660099232,11cyclictest8178-21sh10:25:122
16600992316,4cyclictest29918-21ssh10:35:272
1659199234,6cyclictest21-21ksoftirqd/112:00:101
1659199234,4cyclictest201rcuc/111:56:161
1659199233,5cyclictest21-21ksoftirqd/110:40:141
1659199233,5cyclictest21-21ksoftirqd/108:35:011
1659199233,4cyclictest21-21ksoftirqd/112:22:491
1659199233,4cyclictest21-21ksoftirqd/109:35:121
1659199231,6cyclictest21-21ksoftirqd/112:30:181
1659199231,4cyclictest21-21ksoftirqd/112:35:041
1658099239,4cyclictest131rcu_preempt11:00:150
1658099232,13cyclictest2287-21diskmemload10:05:130
1658099232,13cyclictest2287-21diskmemload10:05:130
1661099229,10cyclictest0-21swapper/311:45:163
1661099226,10cyclictest35966irq/58-eth011:40:143
1660099222,11cyclictest15022-21ssh12:15:282
1660099221,9cyclictest14010-21kthreadcore09:15:232
1660099221,9cyclictest0-21swapper/212:06:252
1660099221,5cyclictest0-21swapper/208:00:122
1659199226,7cyclictest1030-21apt-get09:45:141
1659199224,4cyclictest21-21ksoftirqd/107:35:191
1659199222,6cyclictest21-21ksoftirqd/108:45:141
1659199221,7cyclictest24751-21kthreadcore08:25:221
1659199221,5cyclictest21-21ksoftirqd/107:15:121
16591992211,7cyclictest1265-21apt-get08:35:131
1659199221,16cyclictest23794-21apt-key08:25:011
1659199220,7cyclictest10069-21cut08:10:161
1658099228,10cyclictest1-21systemd10:35:320
1658099225,4cyclictest131rcu_preempt07:30:010
1658099222,15cyclictest10021-21ssh09:50:140
1658099221,9cyclictest11966-21kthreadcore11:20:200
1658099221,2cyclictest131rcu_preempt08:45:130
1658099221,17cyclictest15565-21cut09:35:150
1661099216,6cyclictest35966irq/58-eth010:35:133
1661099212,12cyclictest35966irq/58-eth012:10:123
16610992112,6cyclictest35966irq/58-eth010:55:133
1661099211,16cyclictest0-21swapper/309:05:173
1660099218,10cyclictest10538-21ssh12:30:312
1660099212,16cyclictest8158-21irqcore11:55:152
1660099212,10cyclictest3886-21kthreadcore09:10:202
1660099211,17cyclictest19404-21wc08:50:292
1660099211,17cyclictest19404-21wc08:50:282
1660099211,11cyclictest19790-21kthreadcore07:10:202
1660099211,10cyclictest32537-21switchtime08:30:282
16600992110,9cyclictest28622-21ssh09:24:082
1659199218,3cyclictest131rcu_preempt07:40:141
1659199217,4cyclictest0-21swapper/107:30:311
1659199213,5cyclictest21-21ksoftirqd/108:15:201
1659199213,4cyclictest21-21ksoftirqd/108:55:021
1659199213,4cyclictest21-21ksoftirqd/108:55:011
1659199213,4cyclictest21-21ksoftirqd/107:55:291
1659199212,5cyclictest25477-21fschecks_count09:00:151
1659199212,5cyclictest21-21ksoftirqd/108:40:181
1659199211,4cyclictest201rcuc/108:05:241
1658099219,9cyclictest9841-21diskmemload09:30:210
1658099212,15cyclictest12-21ksoftirqd/008:50:150
1658099212,15cyclictest12-21ksoftirqd/008:50:140
16580992112,6cyclictest151rcuc/007:15:130
1658099211,16cyclictest7713-21ssh11:55:100
16580992110,3cyclictest131rcu_preempt12:25:140
1661099209,8cyclictest0-21swapper/312:35:273
1661099205,8cyclictest0-21swapper/311:20:123
1661099204,13cyclictest0-21swapper/310:30:283
1661099204,13cyclictest0-21swapper/310:30:283
16610992011,5cyclictest35966irq/58-eth011:55:593
16610992010,7cyclictest0-21swapper/311:05:153
16610992010,6cyclictest0-21swapper/309:20:173
1660099202,9cyclictest24562-21fschecks_count09:40:162
1660099202,11cyclictest0-21swapper/209:00:142
1660099202,10cyclictest27-21ksoftirqd/212:00:112
1660099201,9cyclictest7400-21kthreadcore10:05:222
1660099201,9cyclictest7400-21kthreadcore10:05:212
1660099201,15cyclictest31071-21kthreadcore09:05:192
1660099201,11cyclictest23834-21memory08:55:212
1660099201,11cyclictest11935-21diskstats10:45:172
1660099201,10cyclictest4941-21fschecks_time08:05:162
1660099201,10cyclictest12957-21kthreadcore08:45:202
1660099200,8cyclictest18122-21uniq07:45:152
1659199201,4cyclictest21-21ksoftirqd/109:10:021
1659199200,8cyclictest2952-21cat07:25:291
1658099208,4cyclictest131rcu_preempt10:15:010
1658099203,2cyclictest131rcu_preempt12:35:180
1658099201,3cyclictest131rcu_preempt10:25:140
1658099201,14cyclictest16682-21apt-get12:00:110
1658099200,2cyclictest131rcu_preempt09:57:150
1661099199,8cyclictest0-21swapper/309:54:313
1661099197,7cyclictest0-21swapper/311:35:123
1661099194,9cyclictest0-21swapper/308:15:133
16610991912,5cyclictest33-21ksoftirqd/311:00:163
1661099191,13cyclictest0-21swapper/309:45:203
1661099191,13cyclictest0-21swapper/308:10:213
1661099191,12cyclictest0-21swapper/311:25:123
1660099191,9cyclictest25503-21kthreadcore08:25:212
1660099191,9cyclictest24574-21kthreadcore07:15:222
1660099191,7cyclictest2305466ssh10:31:382
1660099191,7cyclictest2305466ssh10:31:382
1658099194,8cyclictest0-21swapper/010:55:110
1658099191,5cyclictest7520-21ssh09:10:320
1658099191,3cyclictest131rcu_preempt10:33:510
1658099191,3cyclictest131rcu_preempt10:33:500
16580991913,3cyclictest12-21ksoftirqd/008:05:280
16580991911,6cyclictest12864-21sshd07:38:250
1661099189,6cyclictest35966irq/58-eth012:05:113
1661099189,6cyclictest0-21swapper/312:28:253
1661099188,6cyclictest0-21swapper/311:19:453
1661099184,7cyclictest35966irq/58-eth009:30:123
1661099182,11cyclictest0-21swapper/310:45:143
1661099182,10cyclictest0-21swapper/307:45:143
1661099182,10cyclictest0-21swapper/307:20:143
16610991811,4cyclictest0-21swapper/312:20:123
1661099181,10cyclictest35966irq/58-eth007:10:123
16610991810,6cyclictest0-21swapper/309:25:133
1660099186,10cyclictest0-21swapper/207:10:012
1658099189,6cyclictest1594-21ps08:35:170
1658099187,8cyclictest27437-21awk10:15:340
1658099184,3cyclictest131rcu_preempt07:40:290
16580991813,3cyclictest0-21swapper/008:55:240
1658099181,2cyclictest131rcu_preempt08:20:180
1658099181,2cyclictest131rcu_preempt07:10:190
1658099181,15cyclictest30217-21kthreadcore07:55:210
1658099181,14cyclictest22420-21cut11:45:160
16580991810,6cyclictest29934-21swap07:20:270
1661099179,6cyclictest0-21swapper/312:00:473
1661099179,6cyclictest0-21swapper/310:40:383
1661099179,6cyclictest0-21swapper/310:17:433
1661099172,9cyclictest0-21swapper/308:25:153
1661099172,10cyclictest0-21swapper/307:55:133
1661099172,10cyclictest0-21swapper/307:40:143
1661099172,10cyclictest0-21swapper/307:30:143
1661099171,10cyclictest35966irq/58-eth009:10:133
1661099171,10cyclictest0-21swapper/308:35:233
1661099171,10cyclictest0-21swapper/307:50:293
1661099171,10cyclictest0-21swapper/307:50:283
1658099174,10cyclictest7386-21grep07:30:280
16580991712,3cyclictest0-21swapper/009:00:140
1658099171,14cyclictest6223-21fschecks_count08:40:150
16580991710,5cyclictest23133-21sed07:50:190
16580991710,5cyclictest23133-21sed07:50:180
1658099162,9cyclictest0-21swapper/008:00:130
1658099161,5cyclictest28973-21df_inode08:30:150
1661099151,7cyclictest0-21swapper/309:00:123
1661099151,11cyclictest0-21swapper/307:15:173
16610991510,3cyclictest0-21swapper/308:41:253
1658099152,10cyclictest16637-21latency_hist07:10:020
1661099149,3cyclictest0-21swapper/308:00:213
1661099141,7cyclictest0-21swapper/308:05:153
1661099141,10cyclictest0-21swapper/307:35:163
1659199141,10cyclictest16463-21latency_hist07:10:011
1661099131,7cyclictest0-21swapper/308:55:153
1661099128,2cyclictest33-21ksoftirqd/308:20:303
1661099127,3cyclictest0-21swapper/308:45:183
166109991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional