You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-07 - 13:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun Jun 07, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2011099864,10cyclictest131rcu_preempt22:45:132
2008799664,45cyclictest23449-21ssh23:25:120
2011099581,32cyclictest15236-21ssh21:15:142
2011099527,31cyclictest27-21ksoftirqd/221:35:102
2009599521,7cyclictest131rcu_preempt23:45:121
2012599517,26cyclictest35966irq/58-eth000:10:133
2009599501,38cyclictest12170-21ssh22:25:131
2012599472,31cyclictest35966irq/58-eth021:05:133
2009599471,13cyclictest9290-21ssh23:35:131
2012599463,25cyclictest35966irq/58-eth023:50:123
20110994525,14cyclictest271ksoftirqd/223:40:132
2009599442,9cyclictest21086-21apt-get22:30:111
2012599434,23cyclictest35966irq/58-eth000:25:113
2011099433,21cyclictest0-21swapper/219:50:112
2009599432,36cyclictest13756-21diskmemload21:50:141
2012599425,22cyclictest35966irq/58-eth023:25:143
2011099423,28cyclictest27-21ksoftirqd/222:30:112
2008799426,24cyclictest13117-21ssh23:55:140
20125994019,8cyclictest0-21swapper/320:00:123
2011099402,29cyclictest0-21swapper/223:45:132
2009599402,32cyclictest18948-21rm21:35:111
2012599394,20cyclictest35966irq/58-eth021:45:123
20125993917,12cyclictest35966irq/58-eth021:15:133
2011099392,19cyclictest28851-21ssh00:39:362
2008799392,27cyclictest11579-21ssh00:30:130
2008799383,26cyclictest14372-21ssh23:20:120
20087993813,17cyclictest12-21ksoftirqd/020:05:130
20087993813,15cyclictest12-21ksoftirqd/000:05:150
2011099378,18cyclictest131rcu_preempt22:35:132
2009599379,4cyclictest13363-21fschecks_time23:55:171
2008799373,5cyclictest131rcu_preempt19:50:110
2012599362,22cyclictest35966irq/58-eth000:00:113
2009599364,8cyclictest131rcu_preempt21:40:121
2009599362,27cyclictest8389-21ssh22:05:131
2009599361,11cyclictest8770-21grep23:15:231
20087993622,8cyclictest12-21ksoftirqd/023:54:350
2012599352,23cyclictest35966irq/58-eth021:55:153
2009599351,11cyclictest3134-21tr23:30:221
20087993515,14cyclictest12-21ksoftirqd/019:20:120
2012599341,22cyclictest35966irq/58-eth000:30:133
2011099342,25cyclictest9984-21ssh00:28:352
2009599344,20cyclictest18407-21ssh23:40:141
2009599342,20cyclictest21-21ksoftirqd/119:45:131
2009599342,10cyclictest2912-21kthreadcore22:55:191
2009599341,9cyclictest31106-21latency_hist00:05:011
2009599341,9cyclictest28377-21unixbench_multi21:20:321
2009599341,10cyclictest14479-21tr00:30:201
2008799343,20cyclictest12-21ksoftirqd/020:10:120
2009599337,5cyclictest9472-21sed20:40:021
2009599333,8cyclictest30125-21ssh22:35:121
2009599332,8cyclictest28749-21ssh23:10:121
2009599331,9cyclictest27426-21grep20:25:021
2009599331,9cyclictest15285-21kthreadcore20:45:191
2009599331,8cyclictest9397-21cut21:10:221
2009599331,26cyclictest14387-21ssh23:20:121
2009599331,10cyclictest21622-21kthreadcore00:35:201
20087993319,9cyclictest12-21ksoftirqd/023:38:360
2012599323,18cyclictest35966irq/58-eth020:55:143
20125993223,6cyclictest35966irq/58-eth023:05:133
2012599322,21cyclictest35966irq/58-eth000:20:143
2011099322,14cyclictest32584-21ssh00:22:352
2011099321,3cyclictest131rcu_preempt20:35:132
2009599322,9cyclictest8253-21unixbench_singl23:50:291
2009599322,10cyclictest29613-21latency_hist19:20:021
2009599321,9cyclictest8516-21irqrtprio00:10:191
2009599321,9cyclictest20663-21grep00:15:251
2009599321,11cyclictest532-21kthreadcore20:30:201
2009599321,10cyclictest6919-21cut22:40:171
2009599321,10cyclictest15659-21irqcore21:15:181
2009599320,11cyclictest13918-21cut19:35:201
2008799323,20cyclictest7947-21ssh00:10:140
2008799321,23cyclictest6179-21ssh21:10:110
20125993124,4cyclictest35966irq/58-eth021:35:113
20125993116,10cyclictest0-21swapper/323:30:133
2011099314,20cyclictest27-21ksoftirqd/220:45:142
20110993113,8cyclictest24815-21cron19:15:012
2009599315,6cyclictest29543-21kthreadcore22:15:241
2009599315,5cyclictest22107-21usage_EthcTask22:10:311
2009599311,9cyclictest6703-21sed00:25:281
2009599311,9cyclictest24378-21grep21:55:231
2009599311,9cyclictest2213-21grep21:25:201
2009599311,9cyclictest18622-21latency_hist21:35:011
2009599311,9cyclictest17722-21cat20:15:021
2009599311,10cyclictest5118-21sh22:20:221
2009599311,10cyclictest5118-21sh22:20:211
2009599311,10cyclictest26266-21apt-get19:50:131
2008799313,21cyclictest1382-21ssh22:55:120
2012599301,18cyclictest35966irq/58-eth021:50:143
2011099302,11cyclictest4077-21sh23:50:142
2011099300,24cyclictest17598-21ssh23:39:352
2009599303,21cyclictest24788-21ssh22:50:161
2009599302,9cyclictest30312-21kthreadcore20:25:211
2009599301,9cyclictest9962-21kthreadcore20:05:201
2009599301,9cyclictest32345-21irqrtprio22:00:171
2009599301,9cyclictest20876-21irqstats19:10:181
2009599301,9cyclictest19545-21cut20:50:191
2009599301,9cyclictest19545-21cut20:50:181
2009599301,8cyclictest6454-21kthreadcore21:45:191
2009599301,8cyclictest29820-21kthreadcore21:00:211
2009599301,8cyclictest11778-21latency_hist19:35:021
2009599301,10cyclictest26896-21kthreadcore00:20:211
2012599292,16cyclictest35966irq/58-eth022:50:143
2011099293,12cyclictest20512-21kworker/2:023:25:122
2011099292,20cyclictest0-21swapper/223:00:042
20110992918,5cyclictest5796-21diskmemload21:25:142
20110992913,12cyclictest0-21swapper/200:30:122
2009599292,9cyclictest1411-21fschecks_time21:05:161
2009599292,6cyclictest0-21swapper/123:28:351
2009599291,9cyclictest6183-21kthreadcore20:00:221
2009599291,9cyclictest20624-21kthreadcore23:05:201
2009599291,13cyclictest10529-21sh23:00:141
2009599291,11cyclictest7663-21cut00:10:021
2009599291,10cyclictest19493-21/usr/sbin/munin22:45:271
2009599291,10cyclictest17775-21kthreadcore19:40:191
2008799292,22cyclictest5328-21ssh23:15:130
2008799292,20cyclictest2581-21ssh00:25:130
2008799292,18cyclictest15746-21ssh22:45:130
2012599282,20cyclictest35966irq/58-eth022:35:123
20110992814,9cyclictest0-21swapper/222:05:122
2009599281,9cyclictest31254-21grep19:55:161
2009599281,9cyclictest25430-21kthreadcore20:55:191
2009599281,9cyclictest20513-21kthreadcore20:15:221
2009599281,9cyclictest10193-21irqstats20:40:181
20125992714,8cyclictest0-21swapper/322:20:283
20125992714,8cyclictest0-21swapper/322:20:273
20125992710,9cyclictest35966irq/58-eth023:00:143
2011099272,15cyclictest27346irq/40-dwc2_hso21:00:132
2011099272,14cyclictest6767-21ssh22:40:142
2011099272,10cyclictest0-21swapper/219:30:142
2011099271,19cyclictest27346irq/40-dwc2_hso22:10:122
2009599271,9cyclictest5562-21missed_timers19:25:211
2008799272,19cyclictest25827-21ssh23:43:350
2008799272,18cyclictest22251-21ssh00:00:110
20087992715,6cyclictest5796-21diskmemload23:05:130
2008799271,19cyclictest21113-21rm22:30:120
20125992613,8cyclictest35966irq/58-eth022:55:133
20125992611,10cyclictest35966irq/58-eth000:15:123
2011099263,16cyclictest27346irq/40-dwc2_hso00:10:142
2011099262,19cyclictest0-21swapper/200:08:292
2011099262,16cyclictest27346irq/40-dwc2_hso19:55:142
2011099262,16cyclictest27346irq/40-dwc2_hso19:20:142
2011099262,10cyclictest5796-21diskmemload00:15:352
2011099261,15cyclictest27346irq/40-dwc2_hso23:15:132
2008799261,21cyclictest25984-21ssh00:20:140
2012599252,15cyclictest35966irq/58-eth023:20:133
2011099254,14cyclictest27346irq/40-dwc2_hso20:55:132
2011099254,14cyclictest27346irq/40-dwc2_hso19:15:112
2011099252,16cyclictest27-21ksoftirqd/223:55:142
2011099251,12cyclictest628-21tcf-agent21:55:152
2011099250,16cyclictest19684-21sh23:05:142
2009599256,4cyclictest2026-21awk19:25:021
2008799253,14cyclictest0-21swapper/021:00:140
2008799251,21cyclictest31782-21ssh22:00:120
2008799251,20cyclictest26461-21ssh22:15:130
2011099245,12cyclictest0-21swapper/219:40:122
2008799243,17cyclictest24792-21ssh22:50:160
2008799242,17cyclictest32506-21ssh23:30:120
2008799242,17cyclictest13777-21ssh21:50:160
2008799241,19cyclictest10528-21ssh23:00:140
2008799241,18cyclictest20686-21rm00:35:130
20087992411,8cyclictest151rcuc/020:25:130
2012599234,10cyclictest35966irq/58-eth023:35:143
20125992310,8cyclictest0-21swapper/322:40:153
2011099236,14cyclictest0-21swapper/220:50:292
2011099236,14cyclictest0-21swapper/220:50:292
2011099232,16cyclictest27346irq/40-dwc2_hso23:30:132
2011099232,16cyclictest0-21swapper/200:03:302
20110992311,9cyclictest29738-21ssh22:50:382
2011099231,10cyclictest0-21swapper/222:25:142
2011099229,9cyclictest0-21swapper/220:30:162
2011099229,8cyclictest0-21swapper/223:10:122
2011099226,6cyclictest27-21ksoftirqd/222:55:142
2011099223,10cyclictest3558-21kthreadcore21:05:222
2011099222,12cyclictest27346irq/40-dwc2_hso20:10:142
20110992211,9cyclictest21667-21ssh21:54:172
20110992211,9cyclictest1599-21ssh21:41:452
2011099221,12cyclictest27284-21cut22:15:202
2011099221,10cyclictest6434-21hwlatdetect21:10:152
2008799224,12cyclictest630-21Async09:36:310
2008799223,16cyclictest4686-21ssh21:45:110
20087992213,5cyclictest12-21ksoftirqd/020:00:130
2008799221,18cyclictest9795-21ls20:40:170
2012599218,8cyclictest35966irq/58-eth021:20:163
2012599217,8cyclictest35966irq/58-eth023:10:133
2012599216,10cyclictest0-21swapper/322:25:133
2012599213,10cyclictest35966irq/58-eth020:40:133
2012599212,13cyclictest0-21swapper/300:36:353
20125992110,8cyclictest0-21swapper/322:04:093
20125992110,6cyclictest0-21swapper/321:27:023
2011099215,13cyclictest2565-21tail19:25:162
2011099211,12cyclictest24318-21ssh21:20:152
2011099211,10cyclictest2256-21tr22:00:222
2011099211,10cyclictest14119-21cut20:45:002
20110992110,8cyclictest0-21swapper/220:05:182
2008799218,3cyclictest131rcu_preempt23:45:270
2008799212,17cyclictest22790-21ls21:55:160
20087992114,5cyclictest0-21swapper/023:10:150
20087992111,7cyclictest18939-21apt-get21:35:090
2008799210,4cyclictest131rcu_preempt22:28:030
2012599209,8cyclictest0-21swapper/322:10:453
2012599209,8cyclictest0-21swapper/322:08:303
20125992011,6cyclictest35966irq/58-eth021:10:123
20125992010,6cyclictest35966irq/58-eth022:15:133
2011099205,6cyclictest32050-21cron20:30:012
2011099201,9cyclictest6210-21kthreadcore22:20:222
2011099201,9cyclictest6210-21kthreadcore22:20:222
2011099201,9cyclictest5189-21sed21:45:162
2011099201,9cyclictest18164-21uname23:20:272
2011099201,9cyclictest13138-21kthreadcore21:30:232
2011099201,16cyclictest18000-21fschecks_count20:15:172
2011099201,15cyclictest930-21snmpd19:35:142
2011099201,10cyclictest5333-21ps20:00:202
2011099201,10cyclictest27270-21grep20:25:012
2011099201,10cyclictest22181-21kthreadcore19:45:192
2008799208,9cyclictest12-21ksoftirqd/020:55:130
2008799202,4cyclictest131rcu_preempt20:30:170
2008799202,2cyclictest131rcu_preempt21:15:060
2008799201,4cyclictest131rcu_preempt22:40:210
20087992014,4cyclictest0-21swapper/021:45:010
2012599193,9cyclictest0-21swapper/323:56:363
2012599193,11cyclictest0-21swapper/323:42:353
20125991911,6cyclictest0-21swapper/321:30:343
20087991913,4cyclictest14660-21irqcore20:45:170
20087991913,4cyclictest0-21swapper/019:30:110
20087991911,6cyclictest0-21swapper/020:00:020
20087991911,5cyclictest4468-21sh21:25:230
2008799191,15cyclictest3116-21ssh22:20:130
2008799191,15cyclictest3116-21ssh22:20:120
2008799191,14cyclictest30116-21ssh22:35:120
2008799191,13cyclictest19110-21kthreadcore00:15:200
2012599189,6cyclictest35966irq/58-eth022:45:133
2012599183,10cyclictest0-21swapper/323:47:353
2012599182,10cyclictest0-21swapper/320:10:143
2012599182,10cyclictest0-21swapper/319:25:133
2012599182,10cyclictest0-21swapper/319:20:133
20125991810,6cyclictest0-21swapper/322:30:073
20125991810,5cyclictest35966irq/58-eth000:05:153
2008799184,9cyclictest0-21swapper/021:30:140
2008799183,12cyclictest0-21swapper/020:20:120
2008799182,12cyclictest844-21munin-node22:05:110
2008799181,14cyclictest26962-21tr21:20:240
2008799181,10cyclictest23786-21ls20:55:010
2008799181,10cyclictest23786-21ls20:55:000
2008799180,3cyclictest131rcu_preempt19:45:170
2012599179,3cyclictest0-21swapper/321:41:593
2012599178,6cyclictest0-21swapper/323:15:473
2012599172,10cyclictest0-21swapper/320:45:133
2012599172,10cyclictest0-21swapper/320:15:153
2012599172,10cyclictest0-21swapper/319:40:133
2008799174,10cyclictest3909-21kthreadcore19:25:200
20087991710,4cyclictest4829-21cut20:35:150
20087991710,4cyclictest4028-21cut21:05:220
2012599162,9cyclictest0-21swapper/320:35:133
2012599162,11cyclictest0-21swapper/320:20:133
2012599162,10cyclictest0-21swapper/319:55:153
2012599161,12cyclictest0-21swapper/320:30:173
2008799164,9cyclictest26111-21kthreadcore19:15:190
2008799164,8cyclictest0-21swapper/019:10:120
2008799162,2cyclictest131rcu_preempt19:10:020
2008799162,11cyclictest18417-21irqrtprio20:15:190
2008799162,11cyclictest11916-21cpuspeed19:35:150
2008799161,11cyclictest16757-21diskstats19:40:140
2012599152,8cyclictest0-21swapper/320:50:053
2012599152,8cyclictest0-21swapper/320:50:043
2012599142,5cyclictest0-21swapper/319:30:133
2012599141,9cyclictest35966irq/58-eth020:05:133
2012599141,8cyclictest0-21swapper/320:25:163
2012599141,7cyclictest0-21swapper/321:05:013
2011099141,10cyclictest20149-21latency_hist19:10:012
2009599141,10cyclictest20188-21grep19:10:011
2012599139,2cyclictest33-21ksoftirqd/319:15:223
2012599131,7cyclictest0-21swapper/319:45:163
2012599131,6cyclictest0-21swapper/319:35:233
2012599121,9cyclictest0-21swapper/319:10:283
2012599111,8cyclictest0-21swapper/319:50:143
2012599111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional