You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 03:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed Apr 22, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15322998020,12cyclictest12-21ksoftirqd/022:00:140
1533499584,47cyclictest3514-21ssh23:35:131
1532299554,11cyclictest131rcu_preempt19:25:120
15322995535,9cyclictest12-21ksoftirqd/022:40:120
1533499541,44cyclictest10762-21rm22:45:121
1532299535,38cyclictest17705-21ssh21:55:130
15322995334,8cyclictest12-21ksoftirqd/021:35:140
15354995036,7cyclictest35966irq/58-eth021:50:133
1532299474,4cyclictest131rcu_preempt21:30:120
1533499463,34cyclictest6963-21ssh23:55:121
1532299465,4cyclictest131rcu_preempt20:10:130
1533499451,38cyclictest19620-21ssh21:20:151
15322994531,5cyclictest12-21ksoftirqd/021:15:140
1533499433,33cyclictest32255-21sh23:15:131
1533499430,32cyclictest12423-21ssh22:10:121
15334994216,17cyclictest21-21ksoftirqd/122:55:141
1532299401,31cyclictest14373-21ssh23:05:140
1533499381,25cyclictest17972-21ssh23:25:131
1532299385,19cyclictest12-21ksoftirqd/022:05:150
1534399372,27cyclictest3461-21cpuspeed22:05:152
1534399372,18cyclictest12546-21df_inode22:10:152
1535499363,20cyclictest35966irq/58-eth020:50:123
15354993619,11cyclictest0-21swapper/321:15:143
15343993622,9cyclictest0-21swapper/219:30:122
15322993616,9cyclictest12-21ksoftirqd/023:40:140
1535499349,13cyclictest35966irq/58-eth023:10:123
1533499342,24cyclictest25063-21ssh00:05:151
1535499336,18cyclictest35966irq/58-eth022:00:143
1534399332,4cyclictest0-21swapper/219:25:112
1534399332,21cyclictest0-21swapper/221:50:132
1533499331,24cyclictest23145-21ssh21:40:121
15322993319,8cyclictest15308-21cyclictest21:50:140
1535499323,19cyclictest35966irq/58-eth021:55:143
1535499322,19cyclictest35966irq/58-eth000:30:133
1535499321,16cyclictest35966irq/58-eth019:30:123
1533499323,14cyclictest1555-21ssh00:10:131
1534399315,18cyclictest27346irq/40-dwc2_hso20:05:142
1534399314,19cyclictest261rcuc/220:55:122
15343993113,12cyclictest0-21swapper/200:30:132
1533499314,15cyclictest0-21swapper/122:50:101
1533499314,15cyclictest0-21swapper/122:50:101
1533499312,23cyclictest32181-21ssh21:45:121
1532299311,2cyclictest131rcu_preempt00:10:120
1535499306,11cyclictest35966irq/58-eth022:05:143
1535499301,13cyclictest35966irq/58-eth020:30:123
1533499301,23cyclictest1599-21sshd21:10:161
1532299301,22cyclictest21404-21ssh23:45:140
1533499292,15cyclictest30407-21ssh23:50:121
15334992916,8cyclictest14362-21apt-get23:05:141
15354992813,6cyclictest35966irq/58-eth022:35:123
1534399284,17cyclictest27346irq/40-dwc2_hso20:10:132
1534399284,14cyclictest0-21swapper/223:05:152
1534399282,19cyclictest0-21swapper/220:50:142
1534399281,13cyclictest1584766ssh00:00:032
1533499282,20cyclictest3849-21ssh22:21:381
1533499282,20cyclictest3849-21ssh22:21:381
1533499282,20cyclictest24050-21ssh21:57:161
1532299282,19cyclictest5022-21ssh00:30:140
1535499272,18cyclictest0-21swapper/323:18:163
15354992714,9cyclictest35966irq/58-eth022:40:143
1534399273,17cyclictest27346irq/40-dwc2_hso19:15:132
1534399272,18cyclictest27346irq/40-dwc2_hso23:45:132
15343992712,12cyclictest0-21swapper/222:35:122
15343992712,11cyclictest0-21swapper/200:20:162
1534399271,15cyclictest93066snmpd22:55:002
1534399271,15cyclictest93066snmpd22:54:592
1533499272,20cyclictest2327-21ssh22:04:151
15334992718,5cyclictest19302-21apt-get20:55:131
1533499271,20cyclictest21439-21ssh22:15:121
15334992711,10cyclictest25070-21apt-get19:20:121
1532299271,3cyclictest131rcu_preempt19:55:130
1535499264,12cyclictest35966irq/58-eth019:15:143
1535499262,18cyclictest0-21swapper/322:31:453
1534399263,11cyclictest5174-21ssh21:30:132
1534399261,8cyclictest115866diskmemload23:20:272
1533499262,19cyclictest21894-21ssh00:39:161
1533499262,19cyclictest11381-21ssh22:09:161
1533499262,18cyclictest25153-21df22:35:151
15334992610,11cyclictest5000-21apt-get00:30:111
15322992610,11cyclictest12-21ksoftirqd/000:23:150
1535499251,16cyclictest0-21swapper/322:21:393
1535499251,16cyclictest0-21swapper/322:21:383
1535499251,12cyclictest35966irq/58-eth000:35:143
1534399252,15cyclictest0-21swapper/220:25:132
1534399251,6cyclictest1596266/usr/sbin/munin22:30:282
15343992513,7cyclictest12918-21ssh22:26:452
1534399251,20cyclictest2495-21ssh23:34:152
1534399251,13cyclictest30225-21grep22:20:012
1533499257,7cyclictest21-21ksoftirqd/122:29:471
1535499243,13cyclictest0-21swapper/300:25:043
1534399243,17cyclictest27346irq/40-dwc2_hso21:35:132
1534399241,7cyclictest3082866ssh23:13:472
1534399241,7cyclictest2824366ssh21:40:442
1534399241,6cyclictest1677366rm22:46:532
1534399241,18cyclictest26751-21ssh22:00:132
1534399240,19cyclictest6589-21latency_hist23:55:012
1534399240,12cyclictest364566ssh22:40:202
1533499246,9cyclictest21-21ksoftirqd/123:48:301
1533499246,8cyclictest21-21ksoftirqd/121:29:131
1533499245,8cyclictest21-21ksoftirqd/121:54:151
1533499245,3cyclictest131rcu_preempt21:15:371
15334992419,3cyclictest201rcuc/122:40:281
1533499241,8cyclictest21863-21cut19:50:171
15334992418,4cyclictest201rcuc/123:44:061
15334992418,4cyclictest201rcuc/100:20:301
1533499241,19cyclictest10528-21ssh00:15:131
15334992410,9cyclictest5146-21apt-get21:30:121
1532299241,4cyclictest131rcu_preempt00:35:140
1535499233,13cyclictest0-21swapper/322:27:153
1535499232,10cyclictest0-21swapper/322:11:153
1534399232,16cyclictest27346irq/40-dwc2_hso20:00:132
1534399232,11cyclictest15410-21ssh00:15:322
1534399231,7cyclictest697166ssh00:11:122
1534399231,6cyclictest722666ssh21:11:272
1534399231,6cyclictest3222066ssh23:15:122
1534399231,6cyclictest2431466ssh21:57:342
1534399231,6cyclictest2376566ssh23:26:392
15343992312,9cyclictest3785-21ssh00:28:592
1534399231,19cyclictest3202-21latency21:45:232
1534399231,13cyclictest340666ssh23:35:062
1534399231,12cyclictest28978-21cut21:25:192
1534399231,11cyclictest930-21snmpd21:15:222
1533499236,9cyclictest21-21ksoftirqd/100:29:191
1533499235,9cyclictest21-21ksoftirqd/123:01:061
1533499235,7cyclictest21-21ksoftirqd/123:30:301
1533499233,5cyclictest21-21ksoftirqd/122:31:471
1533499232,5cyclictest21-21ksoftirqd/123:10:211
15334992318,3cyclictest201rcuc/121:38:531
1535499225,10cyclictest35966irq/58-eth000:05:163
1534399223,10cyclictest28879-21grep19:20:272
1534399222,10cyclictest2211-21grep22:20:292
1534399222,10cyclictest2211-21grep22:20:292
1534399221,6cyclictest961366kworker/u8:123:42:112
1534399221,11cyclictest8957-21kthreadcore23:55:192
1533499223,7cyclictest21-21ksoftirqd/119:45:211
1533499222,7cyclictest21-21ksoftirqd/120:15:251
15334992217,3cyclictest201rcuc/120:05:281
15334992215,4cyclictest0-21swapper/100:00:141
1533499221,17cyclictest8253-21ls20:40:221
1533499221,17cyclictest26302-21apt-get19:55:121
1533499221,16cyclictest8966-21ssh23:20:141
1533499221,16cyclictest20395-21cut19:15:161
1532299228,4cyclictest12-21ksoftirqd/023:15:180
1532299222,15cyclictest19659-21ssh00:00:250
1532299221,17cyclictest7242-21ssh23:35:270
15322992211,4cyclictest1158-21diskmemload22:47:460
1532299221,11cyclictest0-21swapper/021:47:460
15354992110,5cyclictest35966irq/58-eth023:30:183
1534399217,4cyclictest22015-21grep19:50:182
1534399216,13cyclictest29335-21irqcore21:05:182
1534399216,12cyclictest16738-21apt-get19:45:122
1534399212,9cyclictest5597-21cut23:00:172
1534399212,9cyclictest5503-21irqrtprio20:40:172
1534399211,9cyclictest16528-21wc20:15:222
1534399211,9cyclictest13214-21grep20:45:262
1534399211,13cyclictest2201666ssh00:39:232
1534399211,11cyclictest28034-21ssh00:05:242
1534399211,11cyclictest10646-21grep19:35:272
1534399211,10cyclictest29221-21irqcore22:55:192
1534399211,10cyclictest26429-21fschecks_count19:55:162
1533499217,9cyclictest2266-21apt-get19:30:141
1533499215,3cyclictest131rcu_preempt20:10:201
1533499213,9cyclictest29878-21apt-get19:25:111
1533499213,4cyclictest21-21ksoftirqd/120:00:181
1533499212,10cyclictest19152-21timerandwakeup19:10:281
1533499212,10cyclictest19152-21timerandwakeup19:10:281
15334992112,6cyclictest0-21swapper/120:35:301
1532299216,3cyclictest131rcu_preempt22:50:200
1532299216,3cyclictest131rcu_preempt22:50:200
1532299214,6cyclictest12-21ksoftirqd/023:25:190
1532299211,7cyclictest28709-21ntp_states22:35:260
1532299211,4cyclictest0-21swapper/020:25:200
1532299211,3cyclictest0-21swapper/023:22:250
1532299211,17cyclictest5629-21ls21:10:300
15322992111,7cyclictest21159-21grep22:15:020
1532299211,14cyclictest22327-21ssh22:32:150
1535499209,9cyclictest0-21swapper/300:14:103
1535499209,8cyclictest0-21swapper/323:24:163
15354992014,3cyclictest36066irq/59-eth023:35:133
15354992010,8cyclictest0-21swapper/322:48:033
1534399203,14cyclictest12954-21cut19:40:192
1534399203,14cyclictest12954-21cut19:40:192
1534399202,4cyclictest0-21swapper/220:20:142
1534399201,9cyclictest25728-21kthreadcore21:00:202
1534399201,9cyclictest15836-21irqcore19:10:182
1534399201,9cyclictest15836-21irqcore19:10:172
1534399201,8cyclictest19694-21sort21:20:152
1534399201,10cyclictest674-21cut20:35:192
1533499203,3cyclictest131rcu_preempt21:00:211
15334992015,3cyclictest201rcuc/120:20:141
1533499201,12cyclictest0-21swapper/119:40:181
1533499201,12cyclictest0-21swapper/119:40:171
1532299208,8cyclictest25032-21apt-get00:05:140
1532299201,8cyclictest31538-21irqcore20:00:190
1532299201,4cyclictest0-21swapper/021:41:160
1532299201,16cyclictest5364-21find23:00:110
1532299201,10cyclictest24637-21cut21:00:170
1532299201,10cyclictest22526-21kthreadcore20:55:230
1532299201,10cyclictest0-21swapper/023:58:090
1535499193,11cyclictest0-21swapper/300:23:163
1535499191,12cyclictest0-21swapper/320:15:223
1535499191,11cyclictest0-21swapper/319:35:283
1534399190,9cyclictest27971-21fschecks_time20:30:172
1533499195,3cyclictest131rcu_preempt19:10:011
1533499193,4cyclictest21-21ksoftirqd/120:45:151
15334991914,3cyclictest201rcuc/120:50:291
1533499191,15cyclictest23126-21ls20:25:181
1532299191,9cyclictest6818-21munin-plugin-st19:35:010
1532299191,7cyclictest25794-21usage_Anybus22:15:280
1532299191,4cyclictest24828-21munin-run19:20:000
1532299191,4cyclictest0-21swapper/022:58:550
1532299191,3cyclictest0-21swapper/021:26:390
1532299191,3cyclictest0-21swapper/021:05:210
15322991911,4cyclictest10577-21sh22:25:250
1532299191,12cyclictest19637-21sshd21:20:150
1532299191,10cyclictest14508-21kthreadcore20:15:200
1535499189,6cyclictest35966irq/58-eth023:05:163
1535499189,6cyclictest35966irq/58-eth021:40:133
1535499189,6cyclictest0-21swapper/323:50:503
1535499188,8cyclictest0-21swapper/321:22:373
1535499187,8cyclictest0-21swapper/321:45:263
1535499185,8cyclictest35966irq/58-eth023:25:153
1535499184,6cyclictest0-21swapper/320:55:133
1535499183,9cyclictest0-21swapper/320:25:133
1535499183,10cyclictest0-21swapper/321:30:143
1535499182,11cyclictest0-21swapper/323:45:143
1535499182,10cyclictest0-21swapper/322:55:063
1535499182,10cyclictest0-21swapper/320:20:143
1535499182,10cyclictest0-21swapper/320:10:153
15354991813,3cyclictest36066irq/59-eth023:03:473
15354991811,5cyclictest36066irq/59-eth021:10:263
1535499181,13cyclictest35966irq/58-eth000:15:123
15354991810,6cyclictest0-21swapper/323:43:063
15354991810,6cyclictest0-21swapper/321:37:343
15354991810,5cyclictest0-21swapper/323:55:293
15354991810,4cyclictest0-21swapper/321:25:173
1534399180,9cyclictest15394-21latency_hist19:10:022
1533499182,4cyclictest21-21ksoftirqd/121:05:281
15334991811,4cyclictest7432-21if_eth119:35:171
1532299185,10cyclictest13254-21grep20:45:270
1532299183,12cyclictest32323-21cron20:35:020
1532299182,11cyclictest3522-21ssh22:21:160
1532299182,11cyclictest3522-21ssh22:21:150
1532299181,6cyclictest20174-21latency_hist19:15:010
1532299181,6cyclictest20174-21latency_hist19:15:010
1532299181,4cyclictest0-21swapper/020:20:140
1532299181,2cyclictest0-21swapper/023:50:030
15322991812,4cyclictest18037-21processes20:50:270
15322991812,4cyclictest1158-21diskmemload00:25:040
1532299181,14cyclictest29642-21cut23:30:210
1532299181,14cyclictest25124-21ls19:50:280
1532299181,11cyclictest26727-21ls23:10:220
1532299180,9cyclictest844-21munin-node20:35:130
1535499179,6cyclictest0-21swapper/322:52:003
1535499179,6cyclictest0-21swapper/322:52:003
1535499179,6cyclictest0-21swapper/322:16:193
1535499178,6cyclictest0-21swapper/300:00:063
1535499173,6cyclictest0-21swapper/319:25:133
1535499172,10cyclictest0-21swapper/320:05:153
1535499171,11cyclictest0-21swapper/319:45:133
15354991710,5cyclictest0-21swapper/321:05:153
1532299173,11cyclictest6072-21tr20:40:190
1532299172,11cyclictest15447-21ps19:40:270
1532299172,11cyclictest15447-21ps19:40:260
1532299172,10cyclictest0-21swapper/020:05:150
15322991711,4cyclictest17009-21if_err_eth119:45:160
15322991711,4cyclictest0-21swapper/019:35:280
15322991711,3cyclictest0-21swapper/000:15:130
1535499162,9cyclictest0-21swapper/320:00:153
1535499161,12cyclictest0-21swapper/321:00:163
1533499162,9cyclictest0-21swapper/120:30:131
1535499151,12cyclictest0-21swapper/319:20:273
1532299151,11cyclictest25215-21fschecks_count19:20:150
1535499142,8cyclictest0-21swapper/319:55:143
1532299141,10cyclictest15363-21latency_hist19:10:010
1535499138,3cyclictest0-21swapper/319:40:153
1535499138,3cyclictest0-21swapper/319:40:143
1535499135,6cyclictest0-21swapper/319:10:193
1535499135,6cyclictest0-21swapper/319:10:193
1535499131,9cyclictest0-21swapper/320:40:293
1535499131,10cyclictest0-21swapper/319:50:293
1535499121,9cyclictest0-21swapper/320:45:153
1535499111,7cyclictest0-21swapper/320:35:303
1535499111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional