You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 16:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Tue Jun 02, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
465499604,8cyclictest131rcu_preempt10:00:140
465499591,45cyclictest8785-21ssh09:20:120
4670995536,11cyclictest21-21ksoftirqd/109:20:121
4689995218,23cyclictest35966irq/58-eth010:05:123
465499521,42cyclictest6711-21ssh11:25:140
467899513,28cyclictest0-21swapper/212:35:132
4678995119,26cyclictest27-21ksoftirqd/209:15:132
465499513,4cyclictest131rcu_preempt09:40:120
465499512,38cyclictest1455-21ssh10:10:130
467899502,32cyclictest22784-21ssh12:10:122
467899501,26cyclictest19451-21ssh10:20:122
4678995012,30cyclictest27-21ksoftirqd/211:10:132
468999494,26cyclictest35966irq/58-eth011:40:133
4678994913,23cyclictest27-21ksoftirqd/211:55:142
468999465,27cyclictest35966irq/58-eth011:25:133
468999453,26cyclictest35966irq/58-eth009:45:133
468999453,26cyclictest35966irq/58-eth009:45:123
465499442,7cyclictest131rcu_preempt08:15:120
4670994216,20cyclictest21-21ksoftirqd/107:40:151
467099413,8cyclictest131rcu_preempt10:25:151
467899394,25cyclictest27-21ksoftirqd/209:55:132
467899392,28cyclictest12370-21ssh09:40:122
467099392,30cyclictest20997-21ssh11:15:141
4670993918,17cyclictest131rcu_preempt09:30:121
467899383,26cyclictest26815-21ssh09:30:132
468999372,20cyclictest35966irq/58-eth011:10:123
467899372,28cyclictest28527-21ssh10:25:142
465499373,20cyclictest22800-21ssh12:10:130
468999366,21cyclictest35966irq/58-eth012:05:133
468999362,23cyclictest35966irq/58-eth011:00:133
468999361,20cyclictest35966irq/58-eth009:10:133
467899363,25cyclictest261rcuc/212:00:132
467899363,25cyclictest261rcuc/212:00:132
467899362,18cyclictest930-21snmpd12:25:282
467099362,31cyclictest21424-21ssh09:45:131
467099362,31cyclictest21424-21ssh09:45:131
467099361,24cyclictest14027-21ssh10:35:151
468999353,23cyclictest35966irq/58-eth009:55:133
467899352,28cyclictest28323-21grep11:35:272
4678993515,15cyclictest4642-21cyclictest09:50:112
467099353,27cyclictest1477-21sshd10:10:141
467099351,30cyclictest19472-21ssh10:20:141
468999342,20cyclictest35966irq/58-eth012:20:133
467899342,19cyclictest17453-21ssh10:55:142
467899342,14cyclictest26413-21ssh12:30:152
467099342,25cyclictest15939-21sh10:00:131
468999334,24cyclictest0-21swapper/311:55:143
468999333,6cyclictest0-21swapper/308:30:133
468999331,6cyclictest0-21swapper/310:55:143
468999331,18cyclictest35966irq/58-eth008:55:133
467899335,19cyclictest27346irq/40-dwc2_hso08:35:142
467899335,19cyclictest27346irq/40-dwc2_hso08:35:142
467899331,25cyclictest15960-21ssh10:00:142
4670993322,7cyclictest0-21swapper/111:45:131
468999324,18cyclictest0-21swapper/312:30:153
468999322,21cyclictest35966irq/58-eth009:25:123
4689993219,9cyclictest35966irq/58-eth010:15:153
467099324,25cyclictest12038-21ssh11:10:121
467099323,23cyclictest24853-21ssh10:05:131
465499322,19cyclictest26442-21ssh11:00:130
468999319,4cyclictest0-21swapper/309:15:243
468999311,7cyclictest35966irq/58-eth008:00:123
467099316,15cyclictest21-21ksoftirqd/108:00:121
467099312,23cyclictest8333-21ssh12:20:131
467099311,24cyclictest10472-21ssh10:15:151
465499312,24cyclictest3132-21ssh11:05:150
467899306,13cyclictest27346irq/40-dwc2_hso08:40:132
467899304,18cyclictest27346irq/40-dwc2_hso08:05:112
467899302,16cyclictest27346irq/40-dwc2_hso08:55:142
4678993013,12cyclictest0-21swapper/209:10:142
467099302,24cyclictest3400-21ssh09:35:131
467099302,20cyclictest17348-21ssh12:25:121
467099301,4cyclictest131rcu_preempt08:25:121
467099301,22cyclictest8496-21sh10:50:131
465499303,3cyclictest0-21swapper/010:20:130
465499303,23cyclictest12-21ksoftirqd/010:05:130
465499303,22cyclictest19200-21ssh11:50:140
465499302,20cyclictest26380-21/usr/sbin/munin12:30:280
465499300,24cyclictest0-21swapper/007:25:120
467899292,22cyclictest27-21ksoftirqd/211:00:122
467899292,20cyclictest0-21swapper/207:45:132
467899292,16cyclictest27346irq/40-dwc2_hso07:50:142
467899292,16cyclictest27346irq/40-dwc2_hso07:50:142
4670992914,3cyclictest21-21ksoftirqd/112:15:121
465499292,21cyclictest9203-21cat12:00:280
465499292,21cyclictest9203-21cat12:00:280
465499292,16cyclictest31737-21ssh12:15:120
468999287,3cyclictest0-21swapper/311:50:233
468999283,16cyclictest0-21swapper/312:15:283
467099284,19cyclictest4817-21ssh12:00:131
467099284,19cyclictest4817-21ssh12:00:131
467099283,19cyclictest32284-21ssh09:15:151
468999279,4cyclictest0-21swapper/310:35:243
468999273,7cyclictest0-21swapper/312:12:283
468999273,16cyclictest0-21swapper/312:00:283
468999273,16cyclictest0-21swapper/312:00:273
468999271,12cyclictest35966irq/58-eth009:35:123
467899276,14cyclictest261rcuc/207:20:122
467899272,15cyclictest27346irq/40-dwc2_hso07:15:132
467899272,15cyclictest27346irq/40-dwc2_hso07:15:132
4670992712,12cyclictest21-21ksoftirqd/109:25:111
465499272,19cyclictest18781-21ssh12:05:410
465499271,21cyclictest1547-21ssh09:50:240
468999269,3cyclictest0-21swapper/310:42:483
468999268,3cyclictest0-21swapper/309:21:443
468999261,9cyclictest0-21swapper/310:45:293
467899265,14cyclictest0-21swapper/211:25:142
467899263,11cyclictest27346irq/40-dwc2_hso08:50:142
467899262,14cyclictest4172-21ssh12:15:302
467899261,17cyclictest10849-21grep10:15:192
467899261,11cyclictest22837-21diskmemload11:40:342
467099264,13cyclictest21-21ksoftirqd/109:55:131
467099262,20cyclictest5135-21kthreadcore08:15:211
467099262,18cyclictest28223-21ssh11:55:141
467099262,17cyclictest5055-21ssh10:30:131
467099262,17cyclictest1134-21ssh11:40:131
467099261,19cyclictest2960-21ssh12:35:141
465499262,16cyclictest12-21ksoftirqd/008:35:140
465499262,16cyclictest12-21ksoftirqd/008:35:140
468999259,10cyclictest35966irq/58-eth010:30:243
468999258,11cyclictest35966irq/58-eth012:37:283
468999251,7cyclictest0-21swapper/311:45:233
4689992510,4cyclictest0-21swapper/311:39:143
468999250,10cyclictest35966irq/58-eth011:30:193
467899252,18cyclictest21948-21ssh12:09:272
467899251,17cyclictest27346irq/40-dwc2_hso08:10:102
4678992510,11cyclictest10341-21fschecks_time11:45:152
465499253,16cyclictest6907-21apt-get09:55:120
468999242,7cyclictest0-21swapper/309:05:133
468999242,6cyclictest0-21swapper/312:25:163
468999241,4cyclictest0-21swapper/309:40:203
467899249,12cyclictest25118-21latency_hist08:05:012
467899244,11cyclictest0-21swapper/209:45:132
467899244,11cyclictest0-21swapper/209:45:132
467899241,14cyclictest20142-21tr09:05:192
467899241,13cyclictest21004-21sh11:15:142
467899240,9cyclictest3381-21ssh09:35:112
467899240,20cyclictest4978-21ls10:10:242
467099242,11cyclictest24484-21ssh11:35:131
468999239,3cyclictest0-21swapper/310:12:043
468999239,3cyclictest0-21swapper/310:02:343
468999237,4cyclictest35966irq/58-eth011:20:203
468999237,3cyclictest0-21swapper/310:53:133
468999236,6cyclictest0-21swapper/309:34:123
468999236,3cyclictest0-21swapper/310:25:293
468999231,7cyclictest0-21swapper/310:20:143
468999231,6cyclictest0-21swapper/309:50:303
468999231,10cyclictest0-21swapper/308:15:143
4689992310,5cyclictest0-21swapper/311:15:143
467899234,15cyclictest27346irq/40-dwc2_hso12:20:132
467899233,11cyclictest3710-21usage_Anybus10:45:322
467899231,18cyclictest22547-21latency_hist10:40:022
467899231,17cyclictest19576-21tr11:50:182
467899231,14cyclictest27346irq/40-dwc2_hso07:55:142
467899231,14cyclictest27346irq/40-dwc2_hso07:55:132
467899231,14cyclictest26471-21latency_hist09:30:012
467899231,12cyclictest26250-21tr10:40:232
467099237,6cyclictest22837-21diskmemload11:05:321
467099233,10cyclictest13825-21ssh12:05:131
467099231,18cyclictest30122-21ssh11:20:151
465499233,15cyclictest8515-21ssh10:50:150
465499232,17cyclictest21700-21irqstats11:15:190
468999221,7cyclictest0-21swapper/309:00:173
468999221,7cyclictest0-21swapper/308:45:213
467899223,9cyclictest131rcu_preempt11:23:432
467899222,3cyclictest131rcu_preempt10:51:372
467899222,10cyclictest6259-21kthreadcore10:30:192
467899221,12cyclictest16302-21irqrtprio11:30:192
467899221,10cyclictest25249-21irqcore10:05:182
465499224,13cyclictest0-21swapper/009:25:130
4654992215,4cyclictest29176-21sed07:35:180
4654992214,5cyclictest8285-21/usr/sbin/munin12:20:160
4654992214,5cyclictest0-21swapper/010:55:300
4654992213,6cyclictest21470-21grep12:25:260
465499221,16cyclictest10156-21sh11:45:140
468999212,6cyclictest0-21swapper/308:40:143
468999211,6cyclictest0-21swapper/311:05:173
468999211,6cyclictest0-21swapper/308:35:143
468999211,6cyclictest0-21swapper/308:35:143
468999211,6cyclictest0-21swapper/307:20:213
467899216,12cyclictest4794-21apt-get07:10:132
467899212,10cyclictest3140-21grep11:05:162
467899211,16cyclictest2361-21wc08:45:232
467099212,9cyclictest0-21swapper/107:50:231
467099212,9cyclictest0-21swapper/107:50:221
467099212,17cyclictest19199-21sh11:50:141
467099212,15cyclictest26418-21ssh12:30:161
467099211,16cyclictest14409-21apt-get07:20:121
467099210,18cyclictest26644-21ls08:40:011
467099210,18cyclictest26644-21ls08:40:001
465499217,8cyclictest131rcu_preempt07:20:130
465499213,10cyclictest0-21swapper/010:45:150
465499212,15cyclictest24512-21ssh11:35:150
465499211,16cyclictest5382-21grep09:35:200
465499211,16cyclictest23999-21apt-get07:30:120
468999202,6cyclictest0-21swapper/307:25:193
467899206,11cyclictest18682-21kthreadcore08:30:202
467899206,10cyclictest29359-21irqrtprio07:35:192
467899202,9cyclictest10618-21kthreadcore09:20:202
467899202,9cyclictest0-21swapper/209:00:152
467899201,16cyclictest23768-21cron07:30:012
467899201,15cyclictest7565-21fschecks_time08:20:142
467899201,10cyclictest1757-21sed07:40:192
467899200,11cyclictest27607-21switchtime07:30:282
467099204,10cyclictest0-21swapper/107:45:121
467099201,5cyclictest2689-21sed11:05:011
467099201,16cyclictest29085-21ls07:35:171
467099201,15cyclictest9574-21apt-get07:15:111
467099201,15cyclictest9574-21apt-get07:15:111
467099201,15cyclictest30383-21ssh09:50:111
4670992010,7cyclictest16373-21sed09:40:291
465499202,12cyclictest9299-21sh11:44:290
465499201,15cyclictest22954-21ssh10:40:140
465499201,14cyclictest31873-21latency_hist09:15:020
4654992010,7cyclictest0-21swapper/009:45:180
4654992010,7cyclictest0-21swapper/009:45:180
465499200,17cyclictest4859-21ls10:30:020
468999191,11cyclictest0-21swapper/307:10:153
467899192,9cyclictest27-21ksoftirqd/208:15:122
467899191,9cyclictest13691-21kthreadcore08:25:212
467099196,9cyclictest0-21swapper/108:40:131
467099192,12cyclictest15622-21ssh11:30:141
4670991912,5cyclictest28558-21ssh10:41:271
467099191,16cyclictest27378-21unixbench_singl09:10:291
4670991911,6cyclictest19264-21sshd07:55:281
4670991911,6cyclictest19264-21sshd07:55:281
467099191,14cyclictest22810-21ssh12:10:141
465499197,9cyclictest24084-21cron11:35:010
465499194,3cyclictest131rcu_preempt07:45:010
465499192,11cyclictest0-21swapper/008:55:150
465499191,3cyclictest131rcu_preempt11:15:020
465499191,15cyclictest25122-21munin-run08:05:010
468999183,10cyclictest0-21swapper/307:50:143
468999183,10cyclictest0-21swapper/307:50:143
468999182,10cyclictest0-21swapper/308:25:153
468999182,10cyclictest0-21swapper/308:05:043
468999182,10cyclictest0-21swapper/307:55:153
468999182,10cyclictest0-21swapper/307:55:153
468999182,10cyclictest0-21swapper/307:45:133
468999182,10cyclictest0-21swapper/307:15:133
468999182,10cyclictest0-21swapper/307:15:133
467099186,8cyclictest0-21swapper/111:25:201
467099181,8cyclictest20600-21ps08:30:261
4670991812,4cyclictest0-21swapper/108:20:131
467099181,15cyclictest31596-21ls08:45:151
467099180,10cyclictest0-21swapper/110:56:081
465499183,4cyclictest131rcu_preempt07:50:230
465499183,4cyclictest131rcu_preempt07:50:230
465499182,9cyclictest0-21swapper/011:55:280
465499181,8cyclictest10712-21kthreadcore07:15:180
465499181,8cyclictest10712-21kthreadcore07:15:180
465499181,4cyclictest131rcu_preempt08:40:220
465499181,15cyclictest3698-21ps12:35:190
465499181,15cyclictest30260-21ls09:30:220
465499181,15cyclictest17221-21cat09:00:290
4654991811,5cyclictest0-21swapper/007:55:270
4654991811,5cyclictest0-21swapper/007:55:270
465499181,13cyclictest32283-21sh09:15:150
465499181,13cyclictest30123-21sh11:20:150
468999172,10cyclictest0-21swapper/308:10:103
467099178,7cyclictest8804-21fschecks_count08:55:171
467099171,9cyclictest19682-21ps07:25:171
467099171,8cyclictest821-21kthreadcore08:10:191
467099171,12cyclictest614-21dbus-daemon07:14:361
467099171,12cyclictest3238-21ssh10:45:291
467099171,12cyclictest0-21swapper/107:30:141
465499174,11cyclictest7351-21/usr/sbin/munin08:50:260
465499172,10cyclictest0-21swapper/010:35:130
465499171,12cyclictest8523-21memory10:30:220
465499171,12cyclictest12283-21apt-get08:25:140
465499171,12cyclictest10592-21diskstats10:15:160
4654991710,4cyclictest21664-21sh08:35:010
467899160,9cyclictest4642-21cyclictest07:10:022
467099167,6cyclictest26321-21kthreadcore08:05:191
467099162,9cyclictest0-21swapper/109:00:151
467099161,12cyclictest4083-21cut08:50:171
467099161,12cyclictest20925-21kthreadcore09:05:201
465499165,8cyclictest18787-21irqrtprio09:05:170
465499163,9cyclictest0-21swapper/008:10:110
465499162,9cyclictest0-21swapper/007:10:130
465499161,12cyclictest9892-21grep07:45:270
465499161,11cyclictest7506-21diskstats08:20:120
465499161,10cyclictest27922-21kthreadcore08:05:200
468999152,10cyclictest0-21swapper/308:20:113
465499151,11cyclictest31674-21fschecks_time08:45:170
468999149,3cyclictest0-21swapper/308:50:173
467099141,10cyclictest4553-21latency_hist07:10:021
468999131,9cyclictest0-21swapper/307:40:133
465499131,9cyclictest4655-21latency_hist07:10:020
468999127,3cyclictest0-21swapper/307:38:373
468999122,6cyclictest0-21swapper/307:30:143
468999101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional