You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Fri Jun 05, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2437799609,10cyclictest131rcu_preempt12:05:130
2438999592,51cyclictest4733-21ssh10:55:131
2438999543,6cyclictest131rcu_preempt10:05:131
2437799531,49cyclictest24556-21ssh10:30:110
2437799505,31cyclictest15526-21ssh11:55:120
2437799493,8cyclictest131rcu_preempt07:20:120
24406994838,5cyclictest35966irq/58-eth011:20:113
2440699473,29cyclictest35966irq/58-eth011:15:143
2438999461,33cyclictest29971-21ssh10:15:111
2438999451,35cyclictest10127-21ssh10:40:111
2437799453,37cyclictest19114-21ssh10:45:120
2437799453,37cyclictest19114-21ssh10:45:120
2440699442,26cyclictest35966irq/58-eth009:45:123
2437799442,6cyclictest131rcu_preempt08:50:110
24406994322,12cyclictest35966irq/58-eth010:15:123
2437799425,26cyclictest12-21ksoftirqd/008:00:120
2437799423,30cyclictest28165-21ssh10:50:140
24377994210,23cyclictest12-21ksoftirqd/007:30:120
2438999413,29cyclictest10386-21ssh09:10:111
2438999412,34cyclictest29826-21ssh11:45:121
24377994120,12cyclictest8442-21ssh09:45:120
2438999401,33cyclictest22468-21sh12:35:131
2437799381,26cyclictest17241-21ssh11:20:120
2438999371,24cyclictest17496-21ssh09:50:101
2439799352,21cyclictest27-21ksoftirqd/209:50:112
2437799336,18cyclictest12-21ksoftirqd/009:00:110
2437799336,16cyclictest12-21ksoftirqd/008:25:140
2437799333,19cyclictest12-21ksoftirqd/008:55:110
2440699329,14cyclictest0-21swapper/312:10:113
2440699329,14cyclictest0-21swapper/312:10:103
2440699323,17cyclictest35966irq/58-eth011:55:113
2439799323,19cyclictest27346irq/40-dwc2_hso08:05:142
2439799322,19cyclictest20664-1kworker/u9:210:35:142
2437799324,6cyclictest131rcu_preempt07:25:130
24406993114,12cyclictest0-21swapper/309:40:123
2439799312,12cyclictest14334-21rm10:23:572
2438999312,21cyclictest13554-21ssh12:30:131
2440699303,17cyclictest35966irq/58-eth009:15:163
2439799306,19cyclictest12518-21ssh09:28:522
2439799302,12cyclictest3503-21ssh09:23:522
2438999301,17cyclictest15592-21ssh10:25:131
2437799302,21cyclictest8182-21ssh11:15:140
2440699293,20cyclictest35966irq/58-eth008:05:133
2440699292,16cyclictest35966irq/58-eth010:00:123
2439799294,17cyclictest27346irq/40-dwc2_hso07:40:142
2439799292,21cyclictest0-21swapper/209:05:132
2439799292,11cyclictest0-21swapper/210:12:302
2439799292,11cyclictest0-21swapper/210:12:302
2437799292,21cyclictest16485-21usage_Anybus10:05:300
2440699285,16cyclictest0-21swapper/310:50:143
24406992818,6cyclictest35966irq/58-eth011:10:133
2439799282,18cyclictest27346irq/40-dwc2_hso07:55:122
2439799281,22cyclictest3197066ssh09:40:122
24397992810,12cyclictest27-21ksoftirqd/211:30:152
2437799287,7cyclictest131rcu_preempt09:25:310
2437799282,18cyclictest2867-21ssh11:30:150
2440699273,20cyclictest35966irq/58-eth008:35:153
2440699272,18cyclictest0-21swapper/310:30:133
2440699272,16cyclictest0-21swapper/311:05:133
24406992713,9cyclictest35966irq/58-eth012:20:233
2439799274,16cyclictest27346irq/40-dwc2_hso08:20:122
2439799272,16cyclictest27346irq/40-dwc2_hso12:30:122
2439799271,21cyclictest31615-21ssh11:10:132
24397992712,11cyclictest27-21ksoftirqd/212:10:112
24397992712,11cyclictest27-21ksoftirqd/212:10:102
2438999271,20cyclictest26265-21ssh11:25:131
2438999271,19cyclictest22601-21ssh12:15:251
2437799272,16cyclictest22690-21ssh11:05:130
2440699263,10cyclictest35966irq/58-eth010:40:113
2440699260,17cyclictest35966irq/58-eth010:45:123
2440699260,17cyclictest35966irq/58-eth010:45:123
2439799263,10cyclictest22469-21ssh12:35:132
2439799261,10cyclictest22897-21ssh10:28:302
2437799267,14cyclictest12-21ksoftirqd/011:51:290
2437799262,7cyclictest131rcu_preempt10:10:300
2437799262,7cyclictest131rcu_preempt10:10:290
2440699253,19cyclictest0-21swapper/309:28:523
2440699253,18cyclictest35966irq/58-eth010:10:113
2440699253,18cyclictest35966irq/58-eth010:10:113
2440699253,15cyclictest0-21swapper/309:59:303
2440699252,16cyclictest35966irq/58-eth009:50:103
2440699252,15cyclictest35966irq/58-eth010:35:143
2440699251,15cyclictest0-21swapper/311:30:153
2439799257,14cyclictest24527-21diskmemload12:00:112
2439799252,19cyclictest13834-21ssh09:30:102
2439799252,18cyclictest5577-21diskmemload11:49:302
2439799252,16cyclictest27346irq/40-dwc2_hso10:40:122
2439799250,14cyclictest7704-21find08:00:112
2439799250,14cyclictest6551-21ls07:25:152
2438999252,9cyclictest131rcu_preempt12:00:111
2438999252,6cyclictest21-21ksoftirqd/111:25:011
2438999251,19cyclictest28319-21ssh09:20:131
2438999251,19cyclictest24576-21ssh10:30:131
24377992514,8cyclictest11188-21ssh09:10:170
2440699244,16cyclictest35966irq/58-eth009:30:103
2440699243,19cyclictest0-21swapper/311:49:303
2440699242,18cyclictest35966irq/58-eth012:35:123
2440699242,18cyclictest35966irq/58-eth009:10:123
24406992410,8cyclictest35966irq/58-eth012:30:133
2439799242,19cyclictest0-21swapper/208:35:152
2439799242,17cyclictest0-21swapper/211:54:062
2439799242,15cyclictest27346irq/40-dwc2_hso07:35:152
24397992416,5cyclictest26772-21ssh09:55:162
2438999242,14cyclictest0-21swapper/110:10:111
2438999242,14cyclictest0-21swapper/110:10:111
24389992414,7cyclictest21-21ksoftirqd/108:00:011
2437799247,9cyclictest0-21swapper/012:25:120
2437799242,14cyclictest19331-21ssh09:15:150
24377992415,6cyclictest3179-21ssh10:00:130
2437799241,2cyclictest131rcu_preempt10:20:220
2437799241,18cyclictest13729-21ssh11:00:140
2440699234,8cyclictest0-21swapper/308:15:123
2440699234,13cyclictest0-21swapper/310:25:133
2440699233,18cyclictest0-21swapper/310:23:573
2439799234,15cyclictest27346irq/40-dwc2_hso07:45:142
2439799233,6cyclictest0-21swapper/209:49:302
24397992312,9cyclictest24084-21ssh11:22:532
2439799231,19cyclictest670-21kthreadcore08:25:202
2438999231,18cyclictest15696-21cut11:55:151
2438999231,18cyclictest10018-21diskmemload11:50:131
2437799239,11cyclictest18014-21grep09:30:260
2437799233,11cyclictest0-21swapper/010:15:120
2437799231,17cyclictest20935-21apt-get07:40:130
2439799228,11cyclictest17331-21apt-get08:10:122
2439799228,11cyclictest0-21swapper/207:15:142
2439799227,12cyclictest0-21swapper/208:20:012
2439799224,15cyclictest19615-21wc09:15:182
2439799221,5cyclictest2373766kworker/u8:212:20:382
2439799221,5cyclictest1859466kworker/u8:011:02:432
24397992214,5cyclictest5779-21ssh12:05:312
24397992213,5cyclictest0-21swapper/211:25:142
24397992211,9cyclictest18201-21ssh10:07:022
24397992211,9cyclictest16038-21sshd11:19:042
24397992211,8cyclictest22655-21/usr/sbin/munin11:05:262
2439799221,13cyclictest946866ssh10:02:162
2439799221,13cyclictest1172966ssh10:58:082
2439799221,12cyclictest5389-21grep07:20:282
2438999224,13cyclictest3153-21apt-get10:00:121
2438999223,3cyclictest131rcu_preempt09:45:131
24389992214,5cyclictest0-21swapper/111:00:191
2438999221,17cyclictest13838-21sh09:30:121
2438999221,15cyclictest31797-21cut11:10:161
2437799222,14cyclictest31330-21users09:55:300
24377992211,8cyclictest1914-21ssh11:45:300
2437799221,17cyclictest13931-21grep08:40:160
2437799221,16cyclictest26255-21ssh11:25:130
2437799220,18cyclictest7927-21ls11:15:020
2440699217,8cyclictest35966irq/58-eth011:40:123
2440699213,14cyclictest35966irq/58-eth009:20:123
2440699213,13cyclictest0-21swapper/311:00:143
24406992114,4cyclictest33-21ksoftirqd/311:35:163
2439799219,9cyclictest23169-21fschecks_time09:35:162
2439799218,9cyclictest24725-21wc07:10:172
2439799217,11cyclictest30152-21kthreadcore08:55:192
2439799214,14cyclictest30668-21ssh10:50:222
2439799213,15cyclictest27346irq/40-dwc2_hso10:30:132
2439799212,15cyclictest27346irq/40-dwc2_hso09:10:102
2439799211,11cyclictest30716-21grep07:50:172
2439799211,11cyclictest30716-21grep07:50:172
2439799211,11cyclictest20759-21ssh11:40:122
2439799211,11cyclictest19107-21ssh10:45:112
2439799211,11cyclictest19107-21ssh10:45:112
2438999213,3cyclictest1698-21apt-get07:20:121
2438999213,3cyclictest10018-21diskmemload11:40:361
2437799219,8cyclictest12-21ksoftirqd/012:20:110
2437799213,14cyclictest151rcuc/012:10:110
2437799213,14cyclictest151rcuc/012:10:110
2437799211,3cyclictest131rcu_preempt10:25:120
2437799211,17cyclictest24515-21ssh11:40:250
2437799211,15cyclictest16082-21apt-get07:35:150
2437799211,14cyclictest12112-21ls11:35:170
2437799211,13cyclictest13501-21latency_hist08:40:020
2437799211,12cyclictest26261-21ls09:55:010
2440699205,10cyclictest0-21swapper/312:25:133
2440699203,10cyclictest35966irq/58-eth007:45:133
24406992013,5cyclictest0-21swapper/310:05:413
2440699201,14cyclictest35966irq/58-eth010:55:133
2439799208,5cyclictest0-21swapper/208:40:172
2439799201,9cyclictest733-21kthreadcore10:15:202
2439799201,9cyclictest4917-21kthreadcore08:30:192
2439799201,9cyclictest28152-21cat08:55:022
2439799201,9cyclictest24131-21latency_hist12:00:022
24397992014,4cyclictest21950-21ssh12:15:202
2439799201,15cyclictest4557-21ssh12:25:122
2439799201,10cyclictest12591-21kthreadcore11:35:212
2438999202,15cyclictest12940-21irqcore08:05:181
24389992011,6cyclictest29048-21cron07:15:011
2438999201,16cyclictest23583-21kthreadcore08:15:201
2438999201,16cyclictest18435-21kthreadcore07:35:231
2438999201,16cyclictest15106-21sed07:30:281
2438999201,14cyclictest4567-21ssh12:25:131
2438999201,14cyclictest18762-21ls08:45:171
2438999201,13cyclictest10018-21diskmemload09:35:141
2437799209,3cyclictest12-21ksoftirqd/012:30:440
24377992012,6cyclictest13769-21ssh10:40:240
2440699193,9cyclictest0-21swapper/311:51:303
24406991910,7cyclictest0-21swapper/312:04:213
2439799191,9cyclictest11718-21irqcore07:30:172
2439799191,8cyclictest2044-21kthreadcore09:00:182
2439799191,14cyclictest19279-21kthreadcore08:45:192
2438999198,8cyclictest30653-21grep09:55:281
2438999192,3cyclictest131rcu_preempt12:23:051
2438999192,15cyclictest23267-21irqrtprio11:05:191
2438999191,3cyclictest0-21swapper/109:15:181
24389991913,4cyclictest24366-21cyclictest09:40:121
2438999191,16cyclictest32054-21grep08:55:271
2438999191,15cyclictest6595-21tr09:25:201
2438999191,15cyclictest6006-21kthreadcore11:30:241
2438999191,15cyclictest2821-21kthreadcore09:00:191
2438999191,15cyclictest19521-21tail10:45:171
2438999191,15cyclictest19521-21tail10:45:171
2437799198,8cyclictest9728-21cron10:40:010
2437799191,7cyclictest10187-21latency_hist09:10:020
2437799191,13cyclictest27868-21ssh09:35:330
2440699188,6cyclictest35966irq/58-eth009:35:143
2440699182,9cyclictest0-21swapper/308:00:133
2440699182,10cyclictest0-21swapper/308:50:043
2440699182,10cyclictest0-21swapper/307:25:143
2440699181,10cyclictest0-21swapper/308:40:163
24406991810,6cyclictest0-21swapper/312:15:293
2438999188,2cyclictest21-21ksoftirqd/107:20:011
2438999182,11cyclictest14393-21ssh10:24:011
2438999181,15cyclictest13457-21missed_timers12:10:221
2438999181,15cyclictest13457-21missed_timers12:10:221
24389991811,5cyclictest1148-21sh10:35:131
2438999181,14cyclictest5181-21kthreadcore08:30:191
2438999181,14cyclictest23615-21fschecks_time08:50:151
24389991811,4cyclictest0-21swapper/107:45:241
2438999181,10cyclictest12512-21awk08:35:291
24389991810,5cyclictest3293-21kthreadcore12:05:201
2438999180,16cyclictest24038-21kthreadcore07:40:221
2438999180,15cyclictest14355-21irqrtprio08:40:181
2437799188,8cyclictest8070-21apt09:45:010
2437799186,9cyclictest20881-21kthreadcore12:15:190
2437799183,3cyclictest131rcu_preempt12:00:120
2437799183,12cyclictest7744-21sh10:55:220
2437799182,2cyclictest131rcu_preempt09:20:120
2437799181,5cyclictest0-21swapper/007:55:230
2437799181,13cyclictest27886-21ntp_192.168.11407:10:240
2437799180,15cyclictest13155-21cut08:05:190
2440699176,4cyclictest0-21swapper/311:25:133
2440699172,9cyclictest0-21swapper/307:50:153
2440699172,9cyclictest0-21swapper/307:50:143
2440699172,10cyclictest0-21swapper/308:25:143
2440699172,10cyclictest0-21swapper/308:20:143
2440699172,10cyclictest0-21swapper/307:40:143
2440699172,10cyclictest0-21swapper/307:30:133
2440699172,10cyclictest0-21swapper/307:15:133
2440699171,13cyclictest0-21swapper/312:05:203
2440699171,10cyclictest0-21swapper/308:45:143
2439799171,8cyclictest24258-21latency_hist07:10:012
2438999174,10cyclictest10827-21kthreadcore08:00:211
2438999173,11cyclictest20372-21latency_hist11:40:021
2438999173,10cyclictest22088-21latency_hist08:15:021
2438999171,7cyclictest28234-21cut08:20:201
2438999171,13cyclictest5971-21sed09:05:171
2438999171,12cyclictest8294-21diskstats11:15:151
2437799179,5cyclictest0-21swapper/012:35:230
2437799178,6cyclictest685-21in:imuxsock07:20:010
2437799172,10cyclictest0-21swapper/008:10:130
2440699166,6cyclictest33-21ksoftirqd/308:55:253
2438999169,5cyclictest30836-21modprobe07:50:181
2438999169,5cyclictest30836-21modprobe07:50:181
2438999169,4cyclictest7544-21kthreadcore07:25:211
2438999161,3cyclictest0-21swapper/110:50:121
2438999160,8cyclictest32346-21irqrtprio08:25:181
2437799167,7cyclictest23103-21users08:45:310
2437799163,10cyclictest30301-21sort08:20:230
2437799163,10cyclictest28211-21kthreadcore07:45:220
2437799162,9cyclictest0-21swapper/008:30:130
2437799162,8cyclictest0-21swapper/008:15:120
2437799161,12cyclictest31838-21tr07:50:210
2437799161,12cyclictest31838-21tr07:50:210
2440699152,10cyclictest0-21swapper/307:35:163
2440699151,12cyclictest0-21swapper/308:10:153
2440699151,11cyclictest0-21swapper/307:20:283
2437799151,11cyclictest24408-21grep07:10:000
2440699149,3cyclictest0-21swapper/309:00:293
2438999141,10cyclictest167-21systemd-network07:10:011
2440699138,3cyclictest0-21swapper/308:30:273
2440699123,6cyclictest0-21swapper/309:05:113
2440699121,8cyclictest35966irq/58-eth007:10:143
2440699121,8cyclictest0-21swapper/307:55:133
2440699103,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional