You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 05:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Sep 13, 2026 00:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2508995522,22cyclictest27-21ksoftirqd/222:10:132
249399484,42cyclictest29543-21ssh00:15:151
247499482,37cyclictest2721-21ssh22:30:120
249399474,33cyclictest21267-21ssh21:10:141
249399471,38cyclictest23993-21ssh00:30:171
250899461,13cyclictest0-21swapper/220:30:142
250899452,33cyclictest17188-21ssh22:20:122
2508994521,15cyclictest27-21ksoftirqd/221:35:142
249399443,34cyclictest31499-21ssh23:40:121
252399435,23cyclictest35966irq/58-eth023:55:153
249399434,10cyclictest21-21ksoftirqd/123:20:121
247499422,6cyclictest131rcu_preempt20:30:130
250899411,29cyclictest20606-21ssh00:10:182
249399411,29cyclictest4749-21sh21:55:131
2493994022,11cyclictest0-21swapper/119:15:141
250899382,12cyclictest384-21ssh00:35:142
2508993818,15cyclictest27-21ksoftirqd/221:20:152
2508993812,10cyclictest8475-1kworker/u9:121:50:122
247499387,23cyclictest12-21ksoftirqd/022:05:130
2474993833,2cyclictest12-21ksoftirqd/022:35:140
247499382,30cyclictest6748-21ssh21:20:130
247499382,25cyclictest26180-21ssh23:55:160
247499381,33cyclictest27906-21ssh23:20:120
252399373,24cyclictest35966irq/58-eth023:50:153
252399372,21cyclictest35966irq/58-eth021:40:123
249399372,3cyclictest0-21swapper/120:55:141
249399372,3cyclictest0-21swapper/120:55:131
249399361,29cyclictest6489-21sh22:50:181
247499361,5cyclictest131rcu_preempt21:25:110
247499361,28cyclictest17173-21ssh22:20:110
250899352,20cyclictest2723-21ssh00:00:132
252399342,19cyclictest35966irq/58-eth022:30:133
2508993422,6cyclictest21273-21ssh21:10:152
250899341,21cyclictest13772-21ssh22:00:132
247499341,23cyclictest13483-21ssh23:30:130
247499340,27cyclictest0-21swapper/020:35:140
252399331,18cyclictest35966irq/58-eth019:50:143
250899336,16cyclictest0-21swapper/223:40:132
250899332,24cyclictest8176-21ntp_states23:25:252
247499331,21cyclictest8133-21ssh23:45:130
252399323,17cyclictest35966irq/58-eth022:05:123
250899323,7cyclictest0-21swapper/222:38:052
249399323,24cyclictest15434-21ssh22:55:161
247499324,17cyclictest6508-21ssh22:50:190
247499321,25cyclictest4731-21ssh21:55:120
252399319,12cyclictest35966irq/58-eth000:20:143
250899313,21cyclictest0-21swapper/200:30:182
250899312,13cyclictest22636-21ssh21:27:582
250899311,18cyclictest8270-21ssh22:15:142
2493993111,13cyclictest0-21swapper/119:30:131
250899304,15cyclictest27346irq/40-dwc2_hso00:15:162
2508993014,13cyclictest0-21swapper/219:40:022
2508993014,13cyclictest0-21swapper/219:40:012
249399303,17cyclictest28204-21ssh21:50:151
2493993020,7cyclictest21-21ksoftirqd/122:05:121
2493993012,13cyclictest21-21ksoftirqd/123:00:131
250899295,13cyclictest27346irq/40-dwc2_hso21:30:172
250899293,17cyclictest2461-21cyclictest23:45:132
250899292,21cyclictest13260-21ssh22:52:572
250899292,17cyclictest27346irq/40-dwc2_hso00:05:142
250899292,17cyclictest27346irq/40-dwc2_hso00:05:132
2508992918,7cyclictest27-21ksoftirqd/222:25:132
249399290,22cyclictest0-21swapper/120:20:151
247499295,13cyclictest10162-21ssh21:40:110
247499292,20cyclictest390-21ssh00:35:140
252399282,19cyclictest0-21swapper/321:25:263
2523992810,10cyclictest35966irq/58-eth000:10:203
250899286,13cyclictest27346irq/40-dwc2_hso22:40:132
250899283,15cyclictest27346irq/40-dwc2_hso20:10:142
249399284,13cyclictest21-21ksoftirqd/119:35:121
249399284,13cyclictest21-21ksoftirqd/119:35:111
247499282,16cyclictest22448-21ssh23:35:120
2474992816,7cyclictest0-21swapper/000:00:320
252399277,14cyclictest35966irq/58-eth022:55:183
2523992713,9cyclictest35966irq/58-eth022:15:133
250899273,18cyclictest1047-21ssh23:05:122
250899273,18cyclictest1047-21ssh23:05:122
250899272,19cyclictest0-21swapper/220:50:142
250899272,10cyclictest0-21swapper/219:50:142
2508992713,11cyclictest25009-21cron23:17:012
250899271,23cyclictest930-21snmpd19:20:172
250899271,18cyclictest19853-21uniq20:35:162
249399271,17cyclictest19246-21ssh21:45:131
247499275,14cyclictest11638-21ssh00:05:120
247499275,14cyclictest11638-21ssh00:05:120
2523992620,4cyclictest35966irq/58-eth023:40:113
250899262,20cyclictest4750-21ssh21:55:152
249399265,7cyclictest21-21ksoftirqd/100:00:161
249399262,7cyclictest21-21ksoftirqd/100:05:131
249399262,7cyclictest21-21ksoftirqd/100:05:121
2493992618,5cyclictest201rcuc/121:20:131
247499263,20cyclictest6072-21ssh00:20:130
247499262,15cyclictest17125-21ssh23:50:150
247499262,14cyclictest31717-21ssh22:10:130
247499261,19cyclictest20956-21df_inode22:40:150
250899254,13cyclictest27346irq/40-dwc2_hso20:45:142
250899253,17cyclictest27346irq/40-dwc2_hso21:00:152
250899251,11cyclictest31558-21find19:40:122
247499258,10cyclictest12-21ksoftirqd/023:15:140
247499253,15cyclictest29886-21ssh22:45:160
247499253,15cyclictest29886-21ssh22:45:160
252399243,14cyclictest0-21swapper/322:38:043
252399241,14cyclictest35966irq/58-eth022:10:153
252399241,13cyclictest35966irq/58-eth021:20:133
250899244,14cyclictest27346irq/40-dwc2_hso21:15:152
250899242,16cyclictest2708-21ssh22:30:122
2508992416,5cyclictest31526-21ssh22:10:012
2508992412,9cyclictest17329-21ssh23:13:342
249399249,3cyclictest0-21swapper/123:16:001
249399242,16cyclictest6091-21sh00:20:151
247499248,3cyclictest131rcu_preempt21:35:140
247499242,17cyclictest7542-21apt-get19:15:130
247499241,16cyclictest2086666diskmemload00:25:130
252399233,13cyclictest0-21swapper/323:10:163
252399232,15cyclictest0-21swapper/322:20:263
250899239,8cyclictest27-21ksoftirqd/221:40:112
250899233,13cyclictest0-21swapper/219:10:142
250899232,16cyclictest0-21swapper/200:27:272
250899231,19cyclictest28626-21sed23:00:312
250899230,16cyclictest11252-21latency_hist21:00:012
250899230,16cyclictest11252-21latency_hist21:00:012
249399231,17cyclictest1253-21ssh21:35:151
249399231,16cyclictest17168-21find19:25:151
2493992311,10cyclictest0-21swapper/123:54:341
2493992310,6cyclictest131rcu_preempt23:08:261
2493992310,6cyclictest131rcu_preempt23:08:261
2493992310,11cyclictest0-21swapper/120:25:161
247499235,10cyclictest1056-21ssh23:05:130
247499235,10cyclictest1056-21ssh23:05:130
247499232,16cyclictest20866-21diskmemload22:29:250
247499232,14cyclictest844-21munin-node20:00:130
247499232,14cyclictest844-21munin-node20:00:120
252399223,6cyclictest0-21swapper/323:20:113
252399223,12cyclictest0-21swapper/300:35:153
252399222,13cyclictest35966irq/58-eth000:00:133
250899228,11cyclictest6673-21irqrtprio00:20:202
250899222,13cyclictest27346irq/40-dwc2_hso20:05:132
249399229,10cyclictest21-21ksoftirqd/123:35:421
249399222,4cyclictest131rcu_preempt19:45:141
249399222,16cyclictest8260-21apt-get22:15:121
249399222,15cyclictest15669-21apt-get21:25:101
2493992214,5cyclictest21-21ksoftirqd/121:40:281
2474992210,3cyclictest131rcu_preempt23:25:190
252399211,11cyclictest0-21swapper/321:10:133
250899218,4cyclictest2314-21latency_hist00:00:022
250899214,11cyclictest27346irq/40-dwc2_hso19:25:142
250899213,15cyclictest27346irq/40-dwc2_hso20:15:162
250899213,15cyclictest27346irq/40-dwc2_hso20:15:152
250899212,15cyclictest0-21swapper/219:30:152
250899212,12cyclictest27346irq/40-dwc2_hso19:45:152
250899211,14cyclictest2682-21ls19:10:032
250899211,11cyclictest32149-21cut22:45:242
250899211,11cyclictest32149-21cut22:45:232
250899211,11cyclictest25934-21cat23:55:012
250899211,10cyclictest4140-21latency_hist23:25:022
250899211,10cyclictest19338-21ssh21:45:162
250899211,10cyclictest15457-21ssh22:55:172
249399219,3cyclictest21-21ksoftirqd/123:45:141
249399219,3cyclictest21-21ksoftirqd/122:32:231
249399219,3cyclictest21-21ksoftirqd/122:27:441
249399219,2cyclictest21-21ksoftirqd/120:40:121
249399219,2cyclictest131rcu_preempt22:42:341
249399217,7cyclictest0-21swapper/119:10:141
249399212,14cyclictest24806-21ssh21:30:171
249399212,14cyclictest16887-21ssh23:13:021
2493992115,4cyclictest0-21swapper/123:30:131
2493992110,3cyclictest21-21ksoftirqd/122:47:541
2493992110,3cyclictest21-21ksoftirqd/122:47:531
247499219,7cyclictest131rcu_preempt20:05:140
247499211,6cyclictest31604-21df_inode23:40:130
247499211,2cyclictest131rcu_preempt21:50:120
252399208,9cyclictest0-21swapper/321:15:313
2523992011,6cyclictest35966irq/58-eth023:25:123
2523992011,6cyclictest35966irq/58-eth021:35:143
252399201,14cyclictest35966irq/58-eth023:30:123
250899205,6cyclictest0-21swapper/221:05:282
250899203,9cyclictest16122-21kthreadcore19:55:242
250899202,11cyclictest5803-21irqrtprio20:20:202
250899202,11cyclictest25813-21kthreadcore20:40:182
250899201,9cyclictest10235-21fschecks_count20:25:172
250899201,6cyclictest13709-21fschecks_time23:30:152
250899201,15cyclictest24069-21kthreadcore23:35:202
250899201,11cyclictest8632-21kthreadcore19:15:202
249399209,8cyclictest30947-21cat23:55:321
249399209,2cyclictest21-21ksoftirqd/100:15:011
249399208,3cyclictest21-21ksoftirqd/122:10:131
249399208,3cyclictest21-21ksoftirqd/121:18:551
249399207,7cyclictest21-21ksoftirqd/122:04:561
249399207,4cyclictest131rcu_preempt00:25:221
249399207,3cyclictest0-21swapper/122:37:021
2493992014,4cyclictest21-21ksoftirqd/120:40:011
247499209,3cyclictest131rcu_preempt20:15:010
247499207,4cyclictest131rcu_preempt22:55:260
247499203,3cyclictest131rcu_preempt21:45:140
247499201,9cyclictest0-21swapper/000:15:240
2474992013,5cyclictest28561-21ssh00:14:230
2474992012,6cyclictest15676-21ssh22:18:390
247499201,14cyclictest29125-21apt-get20:45:010
252399199,7cyclictest35966irq/58-eth023:15:143
252399199,6cyclictest35966irq/58-eth021:30:243
252399193,13cyclictest35966irq/58-eth020:55:133
252399193,13cyclictest35966irq/58-eth020:55:133
252399193,11cyclictest0-21swapper/300:28:263
252399192,11cyclictest0-21swapper/323:00:043
252399192,11cyclictest0-21swapper/319:10:053
252399191,9cyclictest35966irq/58-eth023:05:143
252399191,9cyclictest35966irq/58-eth023:05:133
2523991913,4cyclictest0-21swapper/321:49:013
2523991912,5cyclictest35966irq/58-eth000:05:143
2523991912,5cyclictest35966irq/58-eth000:05:143
2523991911,5cyclictest35966irq/58-eth022:00:133
252399191,12cyclictest0-21swapper/322:25:113
252399191,11cyclictest0-21swapper/319:45:023
2523991910,6cyclictest35966irq/58-eth021:55:133
250899191,10cyclictest20444-21cut20:00:182
250899191,10cyclictest20444-21cut20:00:182
249399198,9cyclictest21-21ksoftirqd/100:35:381
249399197,3cyclictest21-21ksoftirqd/122:20:131
249399191,9cyclictest16460-21fschecks_time21:05:171
2493991913,4cyclictest21-21ksoftirqd/119:55:231
2493991913,3cyclictest21-21ksoftirqd/121:00:231
2493991912,4cyclictest21-21ksoftirqd/119:25:011
247499193,5cyclictest53646usb-storage20:15:040
247499193,5cyclictest53646usb-storage20:15:040
2474991911,6cyclictest31486-21ssh23:03:050
2474991911,6cyclictest26995-21ssh21:11:390
247499191,15cyclictest32224-21kernelversion19:40:190
252399189,6cyclictest35966irq/58-eth000:15:163
252399189,6cyclictest0-21swapper/300:31:343
252399183,12cyclictest35966irq/58-eth019:30:143
252399182,11cyclictest0-21swapper/323:38:243
252399182,11cyclictest0-21swapper/322:54:253
252399182,10cyclictest0-21swapper/320:30:143
252399182,10cyclictest0-21swapper/320:10:153
252399182,10cyclictest0-21swapper/320:00:123
252399182,10cyclictest0-21swapper/320:00:123
2523991813,3cyclictest33-21ksoftirqd/319:25:283
2523991812,4cyclictest0-21swapper/323:45:263
2523991811,4cyclictest0-21swapper/322:45:233
2523991811,4cyclictest0-21swapper/322:45:233
252399181,12cyclictest35966irq/58-eth021:50:123
252399181,12cyclictest0-21swapper/320:20:243
252399181,11cyclictest0-21swapper/320:25:243
249399187,8cyclictest21-21ksoftirqd/120:00:241
249399187,8cyclictest21-21ksoftirqd/120:00:231
249399187,3cyclictest21-21ksoftirqd/120:15:271
249399187,3cyclictest21-21ksoftirqd/120:15:271
249399185,9cyclictest201rcuc/123:25:121
249399185,5cyclictest21-21ksoftirqd/119:50:191
249399185,10cyclictest29207-21kthreadcore20:10:191
2493991811,5cyclictest29767-21irqcore20:45:191
247499188,7cyclictest12428-21wc19:50:300
247499186,4cyclictest131rcu_preempt21:00:200
247499184,7cyclictest121ksoftirqd/023:10:160
247499184,11cyclictest20097-21unixbench_multi21:05:290
2474991811,5cyclictest28991-21ssh00:30:460
2474991811,5cyclictest13651-21ssh22:00:060
247499181,14cyclictest30502-21ls21:15:180
2474991810,6cyclictest8500-21sshd19:49:250
252399172,9cyclictest0-21swapper/320:50:143
252399172,9cyclictest0-21swapper/319:20:173
252399172,10cyclictest0-21swapper/320:35:163
252399172,10cyclictest0-21swapper/320:15:163
252399172,10cyclictest0-21swapper/320:15:153
249399177,8cyclictest19506-21cron20:35:011
249399175,3cyclictest21-21ksoftirqd/120:05:121
249399171,3cyclictest0-21swapper/120:50:391
2493991710,4cyclictest32128-21cut19:40:181
247499179,6cyclictest9885-21sshd20:24:250
247499179,6cyclictest14674-21cron20:30:010
247499179,5cyclictest23329-21kthreadcore19:30:200
247499178,7cyclictest6379-21users20:50:300
247499171,5cyclictest818-21sed20:45:310
247499171,5cyclictest21740-21munin-run19:30:010
247499171,13cyclictest6645-21apt-get20:55:130
247499171,13cyclictest6645-21apt-get20:55:120
247499171,13cyclictest28350-21ntp_192.168.11421:30:280
252399168,6cyclictest0-21swapper/322:43:313
252399164,9cyclictest35966irq/58-eth019:15:133
252399162,10cyclictest0-21swapper/320:45:153
249399163,10cyclictest2541-21latency_hist19:10:011
247499166,7cyclictest13629-21df19:55:170
247499163,10cyclictest2405-21latency_hist19:10:010
247499162,10cyclictest0-21swapper/019:20:180
247499161,13cyclictest7491-21latency_hist19:15:010
252399153,9cyclictest0-21swapper/319:55:233
2523991510,3cyclictest33-21ksoftirqd/320:05:273
247499151,11cyclictest26791-21apt-get19:35:110
247499151,11cyclictest26791-21apt-get19:35:110
252399149,3cyclictest0-21swapper/319:47:423
252399142,10cyclictest0-21swapper/321:05:153
252399141,7cyclictest0-21swapper/319:40:023
252399141,7cyclictest0-21swapper/319:40:013
252399131,6cyclictest0-21swapper/321:05:013
252399128,3cyclictest0-21swapper/320:42:353
252399121,8cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional