You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 21:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu Sep 17, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5364995952,4cyclictest32376-21sh09:50:121
535699583,44cyclictest22482-21ssh12:25:130
535699523,40cyclictest21108-21ssh10:55:120
535699522,7cyclictest131rcu_preempt07:55:120
535699512,43cyclictest27699-21ssh12:10:110
535699503,35cyclictest23197-21ssh09:45:130
536499484,37cyclictest7912-21ssh10:30:141
5381994633,10cyclictest35966irq/58-eth009:35:123
5381994633,10cyclictest35966irq/58-eth009:35:123
5374994629,8cyclictest261rcuc/209:15:112
536499464,37cyclictest5704-21sh11:40:121
536499462,33cyclictest0-21swapper/107:50:121
5356994627,17cyclictest12050-21ssh10:50:140
5381994528,14cyclictest35966irq/58-eth010:30:133
535699442,6cyclictest131rcu_preempt08:45:130
537499413,31cyclictest32392-21ssh09:50:132
537499412,34cyclictest29017-21ssh11:35:132
536499391,33cyclictest14154-21ssh09:40:141
5374993819,13cyclictest23663-21diskmemload10:15:122
535699381,31cyclictest2952-21ssh10:45:130
537499372,18cyclictest15004-21ssh12:37:502
5374993712,15cyclictest27-21ksoftirqd/210:55:132
5356993719,10cyclictest12-21ksoftirqd/011:55:110
5356993719,10cyclictest12-21ksoftirqd/011:55:100
5374993622,8cyclictest27-21ksoftirqd/208:25:112
5374993619,9cyclictest27-21ksoftirqd/208:55:112
5364993612,17cyclictest0-21swapper/111:35:101
535699362,25cyclictest15923-21ssh11:10:140
538199357,22cyclictest35966irq/58-eth010:55:123
537499353,27cyclictest31541-21ssh12:30:122
537499351,26cyclictest13506-21ssh12:20:122
536499351,3cyclictest0-21swapper/108:10:141
538199349,14cyclictest35966irq/58-eth008:30:123
538199347,13cyclictest0-21swapper/310:25:253
538199343,22cyclictest35966irq/58-eth010:05:133
538199342,17cyclictest35966irq/58-eth009:25:123
5374993421,5cyclictest30206-21ssh11:00:122
535699344,19cyclictest3963-21ssh10:10:130
5381993317,12cyclictest35966irq/58-eth012:35:133
537499332,17cyclictest27-21ksoftirqd/208:10:122
537499332,15cyclictest24072-21ssh09:10:142
5374993316,11cyclictest0-21swapper/209:20:122
536499331,21cyclictest427-21sh11:55:111
536499331,21cyclictest427-21sh11:55:101
538199321,18cyclictest35966irq/58-eth011:50:133
536499326,20cyclictest18707-21ssh12:05:141
535699324,25cyclictest23663-21diskmemload11:20:130
535699320,23cyclictest4424-21ssh12:15:150
538199313,22cyclictest0-21swapper/311:05:153
538199311,22cyclictest35966irq/58-eth009:50:143
535699312,26cyclictest18197-21ssh10:00:120
535699312,21cyclictest572066ssh11:40:130
538199303,14cyclictest35966irq/58-eth012:30:113
5381993015,12cyclictest35966irq/58-eth011:20:143
538199300,10cyclictest0-21swapper/307:30:133
537499303,21cyclictest0-21swapper/208:15:142
537499301,19cyclictest27346irq/40-dwc2_hso11:55:112
537499301,19cyclictest27346irq/40-dwc2_hso11:55:112
536499302,20cyclictest28210-21ssh09:30:101
538199294,21cyclictest35966irq/58-eth011:40:133
537499299,15cyclictest27713-21sh12:10:132
537499294,20cyclictest28233-21ssh09:30:122
537499291,16cyclictest19966-21ssh11:30:142
536499291,5cyclictest21-21ksoftirqd/109:15:241
536499290,7cyclictest25870-21grep09:45:211
536499290,10cyclictest31518-21apt-get12:30:131
538199282,13cyclictest0-21swapper/310:40:103
538199281,21cyclictest35966irq/58-eth012:15:153
537499283,18cyclictest31250-21ssh10:25:112
537499283,17cyclictest27346irq/40-dwc2_hso07:55:122
537499283,16cyclictest27346irq/40-dwc2_hso11:10:132
536499289,6cyclictest21-21ksoftirqd/109:40:011
536499289,6cyclictest21-21ksoftirqd/109:40:001
536499284,5cyclictest21-21ksoftirqd/112:00:211
5364992819,4cyclictest22471-21apt-get12:25:141
536499281,21cyclictest26214-21ssh10:40:131
535699282,20cyclictest5053-21ssh09:35:130
535699282,20cyclictest5053-21ssh09:35:130
5356992815,6cyclictest121ksoftirqd/007:20:120
535699281,21cyclictest9958-21ssh09:20:130
535699281,11cyclictest19965-21ssh11:30:130
537499279,14cyclictest14468-21tune2fs09:00:152
5374992712,6cyclictest0-21swapper/211:05:152
5364992715,7cyclictest0-21swapper/108:55:111
535699277,12cyclictest5355-21cyclictest10:20:130
5356992715,9cyclictest1197-21irqcore09:15:170
535699271,19cyclictest9619-21ssh12:00:130
537499263,16cyclictest27346irq/40-dwc2_hso08:45:132
537499261,17cyclictest18702-21ssh12:05:132
536499268,13cyclictest9931-21apt-get09:20:111
536499267,6cyclictest21-21ksoftirqd/110:50:231
536499264,5cyclictest21-21ksoftirqd/112:35:131
536499264,5cyclictest21-21ksoftirqd/111:20:201
536499263,5cyclictest21-21ksoftirqd/112:20:131
536499263,5cyclictest21-21ksoftirqd/110:20:171
536499262,4cyclictest21-21ksoftirqd/109:30:011
536499262,20cyclictest4428-21ssh12:15:151
536499262,18cyclictest25096-21ssh11:15:131
536499262,18cyclictest18200-21diskmemload10:00:141
536499261,4cyclictest21-21ksoftirqd/110:10:201
536499261,22cyclictest2946-21ssh10:45:131
535699262,19cyclictest24076-21ssh09:10:140
535699262,19cyclictest18958-21kworker/u8:110:35:170
535699262,16cyclictest27313-21ssh10:05:130
537499251,16cyclictest14873-21ls11:45:142
537499251,10cyclictest0-21swapper/207:50:122
536499256,5cyclictest21-21ksoftirqd/108:25:191
536499256,5cyclictest21-21ksoftirqd/108:15:161
536499255,5cyclictest21-21ksoftirqd/111:00:251
536499254,9cyclictest201rcuc/107:55:201
536499253,6cyclictest9210-21ssh09:55:161
536499253,4cyclictest21-21ksoftirqd/108:55:021
536499253,11cyclictest0-21swapper/108:01:011
536499252,6cyclictest32732-21latency_hist11:55:021
536499252,5cyclictest21-21ksoftirqd/110:55:191
536499252,4cyclictest21-21ksoftirqd/111:05:221
5364992517,4cyclictest19080-21apt-get09:05:131
536499251,17cyclictest30984-21ssh10:05:271
538199244,11cyclictest35966irq/58-eth012:05:133
538199243,13cyclictest35966irq/58-eth011:45:123
538199242,14cyclictest35966irq/58-eth011:55:113
538199242,14cyclictest35966irq/58-eth011:55:113
538199241,15cyclictest35966irq/58-eth010:45:153
537499243,16cyclictest27346irq/40-dwc2_hso11:40:112
537499242,9cyclictest0-21swapper/209:40:132
537499242,14cyclictest27346irq/40-dwc2_hso08:00:102
537499241,8cyclictest26373-21timerandwakeup08:35:282
537499241,6cyclictest19623-21ls11:30:012
537499241,13cyclictest930-21snmpd10:40:132
537499240,20cyclictest10566-21ls07:15:162
537499240,15cyclictest27346irq/40-dwc2_hso11:50:132
536499246,9cyclictest131rcu_preempt07:45:291
536499246,5cyclictest21-21ksoftirqd/111:45:171
536499243,5cyclictest21-21ksoftirqd/107:25:181
536499243,4cyclictest21-21ksoftirqd/112:10:121
536499243,4cyclictest21-21ksoftirqd/107:20:011
536499242,6cyclictest21930-21cat10:35:351
536499241,8cyclictest2131-21grep08:45:201
5364992417,4cyclictest10851-21apt-get11:25:111
536499241,4cyclictest21-21ksoftirqd/111:10:251
5364992412,7cyclictest27512-21apt-get08:40:131
535699242,16cyclictest31259-21ssh10:25:130
5356992412,7cyclictest15143-21ssh12:38:000
5356992411,5cyclictest0-21swapper/010:45:010
538199232,12cyclictest35966irq/58-eth008:35:133
537499237,7cyclictest12495-21diskmemload10:50:192
537499236,15cyclictest5627-21sh10:10:212
537499234,15cyclictest23160-21/usr/sbin/munin09:45:322
5374992312,9cyclictest16574-21ssh10:34:592
5374992312,9cyclictest0-21swapper/211:24:152
537499231,18cyclictest22652-21grep12:25:162
536499236,5cyclictest21-21ksoftirqd/110:15:301
536499235,5cyclictest21-21ksoftirqd/111:30:191
535699234,15cyclictest12-21ksoftirqd/009:30:120
535699231,16cyclictest31545-21ssh12:30:130
538199229,4cyclictest0-21swapper/312:00:233
538199222,15cyclictest0-21swapper/309:20:113
5381992210,9cyclictest0-21swapper/312:10:193
537499226,13cyclictest4880-21fschecks_time08:50:162
537499223,10cyclictest18815-21irqrtprio10:00:182
537499221,8cyclictest22181-21kthreadcore07:25:202
5374992211,9cyclictest8583-21ssh10:46:262
537499221,12cyclictest5043-21ssh09:35:122
537499221,12cyclictest5043-21ssh09:35:112
537499220,12cyclictest17709-21tr10:35:222
536499225,5cyclictest21-21ksoftirqd/107:10:021
536499223,6cyclictest21-21ksoftirqd/108:20:311
536499223,5cyclictest21-21ksoftirqd/110:25:291
536499222,6cyclictest10298-21latency_hist07:15:021
536499222,5cyclictest21-21ksoftirqd/108:10:011
536499222,4cyclictest21-21ksoftirqd/109:10:271
536499221,6cyclictest27746-21cut07:30:221
536499221,18cyclictest2206-21cut07:40:131
535699224,11cyclictest19108-21sh09:25:130
535699222,15cyclictest9079-21ssh09:55:130
535699220,17cyclictest3534-21cut08:15:160
538199213,10cyclictest0-21swapper/311:35:123
538199211,10cyclictest0-21swapper/312:20:133
537499216,12cyclictest27508-21fschecks_time10:05:172
537499214,9cyclictest9687-21irqrtprio09:55:192
537499212,9cyclictest23583-21kthreadcore10:20:202
537499212,15cyclictest25064-21apt-get11:15:122
537499212,11cyclictest26894-21grep08:05:182
5374992112,6cyclictest10098-21cron07:15:012
537499211,10cyclictest21390-21ps08:30:242
537499210,6cyclictest19078-21apt-get09:25:122
536499215,4cyclictest21-21ksoftirqd/107:35:201
536499212,5cyclictest21-21ksoftirqd/107:20:311
536499212,4cyclictest21-21ksoftirqd/109:00:131
5364992112,6cyclictest22688-21apt-get08:35:131
535699211,3cyclictest131rcu_preempt11:00:120
535699211,16cyclictest29827-21cut07:35:150
535699211,14cyclictest15017-21cut11:45:160
535699211,10cyclictest30042-21grep11:35:190
5356992110,4cyclictest0-21swapper/008:50:140
538199203,14cyclictest0-21swapper/309:40:143
538199201,13cyclictest0-21swapper/308:30:023
537499203,14cyclictest10107-21kthreadcore07:45:222
537499201,9cyclictest26470-21kthreadcore07:30:202
537499201,9cyclictest19309-21idleruntime09:05:172
537499201,9cyclictest12979-21latency_hist08:25:002
5374992011,6cyclictest0-21swapper/212:15:132
537499201,15cyclictest27638-21fschecks_count08:40:152
537499201,11cyclictest10365-21ps12:00:192
537499201,10cyclictest6684-21latency_hist07:45:012
537499200,11cyclictest30914-21kthreadcore07:35:192
5364992015,3cyclictest201rcuc/108:30:141
535699207,6cyclictest16828-21date10:35:010
535699203,13cyclictest0-21swapper/011:25:120
5356992013,4cyclictest0-21swapper/008:05:110
5356992010,7cyclictest27316-21cron12:10:020
535699200,15cyclictest22743-21ls08:35:150
535699200,14cyclictest21909-21ls10:20:010
538199197,9cyclictest0-21swapper/311:01:293
5381991912,4cyclictest0-21swapper/309:30:123
538199191,13cyclictest0-21swapper/310:05:023
538199191,13cyclictest0-21swapper/309:10:293
5381991910,6cyclictest0-21swapper/311:12:443
5381991910,6cyclictest0-21swapper/310:24:403
537499197,9cyclictest15217-21sshd07:20:102
537499191,10cyclictest5481-21cat07:10:012
535699199,8cyclictest18016-21df_inode08:30:140
535699195,3cyclictest131rcu_preempt09:10:010
535699192,2cyclictest131rcu_preempt11:50:540
535699192,13cyclictest131rcu_preempt07:10:120
535699191,9cyclictest15525-21tr09:40:200
535699191,7cyclictest15179-21latency_hist07:20:020
535699191,5cyclictest12623-21cat08:20:320
5356991913,4cyclictest12-21ksoftirqd/011:07:160
535699191,2cyclictest131rcu_preempt09:50:130
535699191,2cyclictest131rcu_preempt07:30:240
5356991912,5cyclictest0-21swapper/011:15:130
538199189,6cyclictest35966irq/58-eth009:55:123
538199189,6cyclictest0-21swapper/311:15:403
538199189,6cyclictest0-21swapper/310:53:413
538199182,8cyclictest0-21swapper/308:00:103
538199182,10cyclictest0-21swapper/309:00:133
538199182,10cyclictest0-21swapper/308:55:123
538199182,10cyclictest0-21swapper/307:55:133
5381991812,4cyclictest0-21swapper/310:15:283
538199181,13cyclictest35966irq/58-eth010:10:563
538199181,12cyclictest0-21swapper/308:10:023
5381991810,6cyclictest0-21swapper/309:47:213
535699189,7cyclictest16440-21ntp_192.168.11408:25:240
535699189,6cyclictest0-21swapper/008:55:170
535699184,4cyclictest131rcu_preempt07:50:230
5356991812,4cyclictest0-21swapper/012:20:280
535699181,12cyclictest1-21systemd07:25:120
538199179,6cyclictest0-21swapper/311:29:103
538199179,6cyclictest0-21swapper/309:17:463
538199176,7cyclictest0-21swapper/311:30:143
538199172,9cyclictest0-21swapper/307:20:143
538199172,10cyclictest0-21swapper/312:25:143
538199172,10cyclictest0-21swapper/310:35:053
538199172,10cyclictest0-21swapper/309:05:143
538199172,10cyclictest0-21swapper/307:50:123
538199172,10cyclictest0-21swapper/307:35:133
538199172,10cyclictest0-21swapper/307:25:133
538199171,9cyclictest0-21swapper/308:40:133
538199171,10cyclictest0-21swapper/307:40:243
535699171,5cyclictest5524-21ntp_192.168.11407:40:210
535699171,2cyclictest131rcu_preempt09:00:140
5356991712,3cyclictest0-21swapper/008:10:150
535699170,8cyclictest131rcu_preempt07:45:160
538199162,10cyclictest0-21swapper/308:15:143
538199162,10cyclictest0-21swapper/308:10:143
535699162,3cyclictest131rcu_preempt08:00:220
5356991610,2cyclictest12-21ksoftirqd/008:40:130
538199141,10cyclictest0-21swapper/308:45:173
538199131,8cyclictest0-21swapper/308:20:213
538199131,10cyclictest0-21swapper/307:45:253
535699130,10cyclictest5367-21latency_hist07:10:010
538199127,3cyclictest0-21swapper/307:14:503
538199122,6cyclictest0-21swapper/308:50:153
538199112,5cyclictest0-21swapper/307:15:133
538199111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional