You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 23:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Aug 30, 2026 12:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1763099653,49cyclictest24990-21ssh11:45:111
1762199601,42cyclictest32523-21ssh12:25:130
1762199584,39cyclictest14337-21ssh12:15:140
1762199572,7cyclictest12-21ksoftirqd/010:10:110
17640995615,21cyclictest27-21ksoftirqd/210:10:112
1763099543,8cyclictest131rcu_preempt09:45:131
17630995417,24cyclictest21-21ksoftirqd/112:20:121
1763099541,5cyclictest131rcu_preempt10:00:131
1763099541,5cyclictest131rcu_preempt10:00:121
1764099521,28cyclictest24346-21ssh10:15:122
1762199461,33cyclictest22887-21ssh10:50:130
1763099454,33cyclictest28664-21ssh12:05:121
17648994335,5cyclictest35966irq/58-eth009:15:133
1763099413,24cyclictest12722-21sh09:15:131
17630994119,8cyclictest21-21ksoftirqd/108:50:121
1763099411,32cyclictest8512-21sshd11:00:151
17630994111,22cyclictest21-21ksoftirqd/112:35:141
1762199393,26cyclictest25733-21sshd09:40:150
1764899384,24cyclictest35966irq/58-eth011:40:123
1764899383,30cyclictest0-21swapper/307:20:113
17648993827,7cyclictest35966irq/58-eth010:10:113
17648993723,10cyclictest35966irq/58-eth011:50:133
17648993719,11cyclictest35966irq/58-eth012:25:133
1763099372,23cyclictest19624-21ssh12:00:141
1764899366,22cyclictest0-21swapper/310:00:133
1764899366,22cyclictest0-21swapper/310:00:133
1764899361,23cyclictest35966irq/58-eth010:45:123
1764099369,8cyclictest27-21ksoftirqd/209:30:132
1764099369,8cyclictest27-21ksoftirqd/209:30:122
1764099361,28cyclictest3103-21ssh11:15:152
1763099367,18cyclictest21-21ksoftirqd/108:35:131
1763099363,21cyclictest15885-21ssh11:40:131
1764899353,21cyclictest35966irq/58-eth010:35:143
1764899351,26cyclictest0-21swapper/310:25:133
1763099354,21cyclictest9183-21ssh12:30:151
1764899345,17cyclictest35966irq/58-eth012:00:133
17648993426,5cyclictest35966irq/58-eth007:50:103
1764899341,2cyclictest0-21swapper/310:40:123
1764099341,23cyclictest20419-21ssh09:55:142
17640993411,16cyclictest0-21swapper/209:23:102
1763099349,18cyclictest21-21ksoftirqd/107:50:111
17630993421,7cyclictest3266-21diskmemload12:25:151
1763099340,26cyclictest0-21swapper/109:50:121
1764099331,17cyclictest25732-21ssh09:40:152
1764899322,20cyclictest35966irq/58-eth007:55:133
1764099328,15cyclictest27-21ksoftirqd/212:25:022
17640993214,12cyclictest0-21swapper/211:37:382
1762199322,23cyclictest20436-21ssh09:55:150
1764099315,16cyclictest27346irq/40-dwc2_hso09:05:142
17640993113,13cyclictest0-21swapper/212:00:122
1763099313,11cyclictest30257-21ssh11:30:131
1762199311,19cyclictest24344-21rm10:15:110
1764899302,24cyclictest0-21swapper/311:20:123
1764899301,2cyclictest0-21swapper/311:10:153
1764899300,2cyclictest0-21swapper/311:45:123
1764099303,17cyclictest27346irq/40-dwc2_hso07:45:122
1764099302,21cyclictest0-21swapper/207:30:152
1763099304,18cyclictest21-21ksoftirqd/111:25:131
1763099304,18cyclictest21-21ksoftirqd/111:25:121
1763099303,19cyclictest21-21ksoftirqd/107:20:111
1764899293,22cyclictest36066irq/59-eth009:50:133
1764899292,24cyclictest0-21swapper/312:05:123
1764899291,18cyclictest35966irq/58-eth008:10:113
1764899290,2cyclictest0-21swapper/309:25:123
1764899290,1cyclictest0-21swapper/312:15:133
1764899290,1cyclictest0-21swapper/311:25:123
1764899290,1cyclictest0-21swapper/311:25:113
17640992912,8cyclictest0-21swapper/209:28:402
17640992912,11cyclictest0-21swapper/210:42:402
1763099296,17cyclictest21-21ksoftirqd/111:35:131
17630992920,6cyclictest21-21ksoftirqd/110:45:131
1764899282,17cyclictest35966irq/58-eth008:00:123
1764899281,2cyclictest0-21swapper/310:20:143
17648992812,9cyclictest35966irq/58-eth012:35:143
1764899280,2cyclictest0-21swapper/311:30:143
1764899280,2cyclictest0-21swapper/309:30:133
1764899280,2cyclictest0-21swapper/309:30:123
1764099289,13cyclictest27-21ksoftirqd/211:56:042
1764099289,11cyclictest0-21swapper/210:50:142
1764099285,14cyclictest27346irq/40-dwc2_hso08:00:132
1764099282,20cyclictest0-21swapper/208:55:142
1764099282,20cyclictest0-21swapper/208:55:132
1764099282,18cyclictest30283-21ssh11:30:152
17640992812,10cyclictest0-21swapper/212:33:262
1763099282,21cyclictest26314-21sshd09:00:141
1762199283,6cyclictest12-21ksoftirqd/011:35:180
17621992816,5cyclictest0-21swapper/008:00:120
1764899271,2cyclictest0-21swapper/309:35:133
1764899271,2cyclictest0-21swapper/309:35:123
1764899271,2cyclictest0-21swapper/308:40:143
1764899271,12cyclictest35966irq/58-eth007:35:123
1764899270,2cyclictest0-21swapper/307:30:123
1764099272,19cyclictest0-21swapper/208:10:142
1764099272,16cyclictest27346irq/40-dwc2_hso12:15:142
1764099272,12cyclictest12152-21ssh11:20:112
17640992712,7cyclictest0-21swapper/212:12:192
1764099271,12cyclictest17509-21ssh11:05:132
17640992710,11cyclictest0-21swapper/209:17:152
17621992718,6cyclictest3690-21ssh09:10:150
17621992715,9cyclictest151rcuc/008:55:120
17621992715,9cyclictest151rcuc/008:55:120
1764899265,13cyclictest0-21swapper/311:05:133
1764899263,19cyclictest0-21swapper/309:55:143
1764899261,7cyclictest0-21swapper/309:45:133
1764899261,5cyclictest35966irq/58-eth010:15:123
1764899261,3cyclictest0-21swapper/311:15:133
1764899261,22cyclictest0-21swapper/312:30:213
1764899261,21cyclictest0-21swapper/310:55:233
1764899261,1cyclictest0-21swapper/309:10:133
1764899260,3cyclictest0-21swapper/310:50:133
1764899260,2cyclictest0-21swapper/312:10:123
1764899260,2cyclictest0-21swapper/308:30:133
1764099268,12cyclictest0-21swapper/211:44:102
1764099264,3cyclictest131rcu_preempt08:30:152
1764099262,15cyclictest27346irq/40-dwc2_hso10:30:132
17640992612,8cyclictest0-21swapper/211:13:032
1763099263,10cyclictest1016-21ssh10:20:111
1762199261,19cyclictest19163-21sh10:30:140
1764899252,19cyclictest0-21swapper/311:35:113
1764899251,2cyclictest0-21swapper/307:10:133
1764899251,21cyclictest0-21swapper/309:43:303
1764899251,20cyclictest0-21swapper/312:25:003
1764899251,20cyclictest0-21swapper/308:20:123
1764099259,13cyclictest11981-21fschecks_time08:45:152
1764099259,13cyclictest11981-21fschecks_time08:45:142
1764099255,12cyclictest27-21ksoftirqd/210:47:502
17640992514,7cyclictest1594-21sh11:50:122
17640992512,3cyclictest0-21swapper/210:27:372
17640992511,3cyclictest0-21swapper/210:36:512
17640992510,7cyclictest0-21swapper/211:03:522
17640992510,12cyclictest27-21ksoftirqd/210:05:232
1763099253,11cyclictest0-21swapper/107:55:141
1763099252,18cyclictest5313-21ssh12:10:141
1763099252,17cyclictest21876-21ssh09:20:161
1763099251,18cyclictest17517-21sh11:05:131
1762199259,8cyclictest12-21ksoftirqd/009:51:080
1762199259,8cyclictest0-21swapper/011:50:120
1762199257,3cyclictest131rcu_preempt08:30:150
1762199252,19cyclictest16691-21apt-get08:50:120
1762199252,17cyclictest30911-21ssh09:25:130
17621992519,4cyclictest0-21swapper/010:25:140
1764899241,6cyclictest0-21swapper/311:55:223
1764899241,2cyclictest0-21swapper/310:30:243
1764899241,2cyclictest0-21swapper/308:15:283
1764899241,2cyclictest0-21swapper/308:15:273
1764899241,2cyclictest0-21swapper/307:15:233
1764899241,20cyclictest0-21swapper/309:20:333
1764899240,2cyclictest0-21swapper/310:05:143
1764899240,2cyclictest0-21swapper/309:05:123
1764899240,2cyclictest0-21swapper/308:25:133
1764899240,2cyclictest0-21swapper/307:40:123
1764099249,9cyclictest131rcu_preempt09:53:372
1764099249,7cyclictest27-21ksoftirqd/211:25:142
1764099249,7cyclictest27-21ksoftirqd/211:25:132
1764099249,5cyclictest0-21swapper/212:36:452
1764099245,12cyclictest27-21ksoftirqd/210:24:492
17640992414,8cyclictest0-21swapper/211:45:182
17640992413,9cyclictest0-21swapper/209:10:122
1763099243,15cyclictest6229-21ssh10:05:141
1763099242,19cyclictest10775-21uniq08:10:151
1763099242,16cyclictest10135-21ssh10:25:141
1763099241,20cyclictest0-21swapper/107:10:141
17621992414,4cyclictest12-21ksoftirqd/012:11:580
1764899231,3cyclictest0-21swapper/309:00:013
1764899231,3cyclictest0-21swapper/309:00:003
1764899231,2cyclictest0-21swapper/308:35:113
1764899231,19cyclictest0-21swapper/308:50:183
1764899230,2cyclictest0-21swapper/311:00:253
1764899230,2cyclictest0-21swapper/309:00:133
1764899230,2cyclictest0-21swapper/308:05:123
1764899230,1cyclictest0-21swapper/307:25:143
1764099237,10cyclictest0-21swapper/210:00:122
1764099237,10cyclictest0-21swapper/210:00:122
1764099237,10cyclictest0-21swapper/208:40:292
1764099236,5cyclictest27-21ksoftirqd/209:45:552
1764099235,11cyclictest27-21ksoftirqd/212:07:312
1764099234,3cyclictest131rcu_preempt10:55:252
17640992310,3cyclictest21631-21ssh09:35:452
17640992310,3cyclictest21631-21ssh09:35:442
17630992315,6cyclictest0-21swapper/110:55:161
17630992312,6cyclictest21-21ksoftirqd/107:15:121
1763099231,17cyclictest10606-21ssh11:55:121
1762199238,8cyclictest12-21ksoftirqd/009:23:200
17621992317,3cyclictest0-21swapper/008:10:130
17621992310,6cyclictest12-21ksoftirqd/010:59:090
1764899220,2cyclictest0-21swapper/308:45:143
1764899220,2cyclictest0-21swapper/308:45:143
1764099224,6cyclictest27-21ksoftirqd/212:29:452
17640992213,6cyclictest29613-21cron08:30:022
17640992212,8cyclictest0-21swapper/207:25:152
1763099226,4cyclictest131rcu_preempt10:45:001
1763099223,9cyclictest4463-21sendmail-msp10:40:011
1762199229,4cyclictest12-21ksoftirqd/012:23:250
1762199229,4cyclictest12-21ksoftirqd/011:07:590
1762199228,8cyclictest12-21ksoftirqd/011:04:380
1762199228,7cyclictest12-21ksoftirqd/011:45:390
1762199228,5cyclictest12-21ksoftirqd/011:42:380
1762199225,13cyclictest19614-21ssh12:00:130
1762199221,19cyclictest3296-21cut11:15:170
1762199221,10cyclictest151rcuc/010:00:300
1762199221,10cyclictest151rcuc/010:00:290
17621992210,9cyclictest151rcuc/010:40:130
1764899211,3cyclictest0-21swapper/307:45:133
1764099212,6cyclictest0-21swapper/208:05:112
1764099211,16cyclictest29306-21kthreadcore07:55:192
1762199219,4cyclictest12-21ksoftirqd/011:14:100
1762199219,4cyclictest12-21ksoftirqd/010:47:310
1762199218,5cyclictest12-21ksoftirqd/012:06:230
1762199217,8cyclictest12-21ksoftirqd/009:49:000
1762199216,7cyclictest12-21ksoftirqd/009:31:550
1762199216,7cyclictest12-21ksoftirqd/009:31:550
1762199213,3cyclictest131rcu_preempt10:35:550
1762199213,2cyclictest131rcu_preempt10:20:140
1762199213,12cyclictest0-21swapper/008:05:140
1762199212,14cyclictest22589-21apt-get07:15:110
1762199211,3cyclictest131rcu_preempt07:40:150
1762199211,16cyclictest21765-21ssh12:35:240
17621992111,4cyclictest12-21ksoftirqd/012:31:460
17621992110,4cyclictest12-21ksoftirqd/010:08:580
17621992110,3cyclictest131rcu_preempt11:21:250
1762199210,16cyclictest4757-21ls07:30:170
1764099205,6cyclictest17268-21irqrtprio08:50:182
1764099201,6cyclictest131rcu_preempt07:20:022
1764099201,13cyclictest31151-21timerwakeupswit07:20:282
1764099201,10cyclictest23921-21awk08:20:262
17640992010,8cyclictest0-21swapper/208:37:142
17640992010,8cyclictest0-21swapper/208:15:142
17640992010,8cyclictest0-21swapper/208:15:132
1764099200,12cyclictest0-21swapper/207:40:302
1763099202,14cyclictest14151-21apt-get07:40:131
1763099201,7cyclictest23868-21kthreadcore10:50:201
17630992010,7cyclictest42-21kauditd11:20:001
1762199208,4cyclictest12-21ksoftirqd/011:58:150
1762199208,3cyclictest131rcu_preempt09:00:320
17621992013,5cyclictest3258-21sh09:10:020
1762199201,15cyclictest6793-21latency_hist08:40:010
1764099198,3cyclictest25343-21kthreadcore07:50:182
1764099193,13cyclictest27-21ksoftirqd/207:35:142
1764099191,10cyclictest29597-21grep09:00:212
1764099191,10cyclictest17886-21cut07:10:172
1763099199,7cyclictest23734-21cat10:30:311
1763099193,13cyclictest15642-21grep11:20:221
1763099192,12cyclictest0-21swapper/109:25:141
1763099191,9cyclictest12352-21irqrtprio08:45:171
1763099191,9cyclictest12352-21irqrtprio08:45:161
1763099191,7cyclictest22041-21cut09:55:221
17630991912,4cyclictest0-21swapper/109:40:031
17630991911,6cyclictest5283-21ntp_states11:50:271
1763099191,11cyclictest25181-21grep08:25:171
1763099191,10cyclictest19204-21cut09:35:211
1763099191,10cyclictest19204-21cut09:35:211
17630991910,6cyclictest28512-21kthreadcore11:10:241
1763099190,15cyclictest17254-21ps10:10:181
1762199195,3cyclictest131rcu_preempt09:35:110
1762199195,3cyclictest131rcu_preempt09:35:110
1762199194,7cyclictest12-21ksoftirqd/009:16:540
1762199193,8cyclictest12-21ksoftirqd/007:50:010
1762199193,7cyclictest131rcu_preempt11:32:360
1762199192,3cyclictest131rcu_preempt08:40:190
17621991913,4cyclictest0-21swapper/011:25:250
17621991913,4cyclictest0-21swapper/011:25:250
1762199191,15cyclictest22423-21ls07:15:010
1763099186,9cyclictest19409-21irqcore07:45:191
1763099183,11cyclictest0-21swapper/108:20:121
1763099181,9cyclictest7132-21kthreadcore07:30:211
1763099181,14cyclictest10097-21kthreadcore08:40:231
1763099181,11cyclictest14481-21grep12:15:151
17630991810,5cyclictest7514-21/usr/sbin/munin09:30:281
17630991810,5cyclictest7514-21/usr/sbin/munin09:30:281
1762199187,3cyclictest131rcu_preempt07:25:150
1762199181,8cyclictest23258-21kthreadcore08:20:210
17621991812,4cyclictest0-21swapper/007:35:150
1763099176,8cyclictest31121-21apt-get09:05:131
1763099171,7cyclictest5595-21cat08:05:011
1763099171,7cyclictest25085-21kernelversion10:15:191
1763099171,13cyclictest6677-21tr08:05:191
1762199178,6cyclictest151rcuc/007:55:130
1762199176,3cyclictest131rcu_preempt08:25:270
17621991711,4cyclictest0-21swapper/007:20:160
1764899161,2cyclictest0-21swapper/307:10:023
1764899161,2cyclictest0-21swapper/307:10:013
1763099168,5cyclictest31093-21kthreadcore08:30:201
1763099165,5cyclictest0-21swapper/109:10:221
1763099164,9cyclictest24200-21kthreadcore08:55:211
1763099164,9cyclictest24200-21kthreadcore08:55:211
1763099163,10cyclictest0-21swapper/107:35:131
1763099161,13cyclictest15533-21fschecks_count08:15:151
1763099161,13cyclictest15533-21fschecks_count08:15:151
1763099161,12cyclictest32188-21apt-get07:25:141
1762199166,3cyclictest131rcu_preempt08:15:160
1762199166,3cyclictest131rcu_preempt08:15:160
1762199161,9cyclictest0-21swapper/008:45:110
1762199161,9cyclictest0-21swapper/008:45:110
1762199151,11cyclictest17703-21latency_hist07:10:020
1762199151,11cyclictest17703-21latency_hist07:10:020
1762199151,10cyclictest23781-21apt-get07:50:120
1764099141,10cyclictest17702-21latency_hist07:10:022
1764099141,10cyclictest17702-21latency_hist07:10:012
1763099141,10cyclictest17690-21latency_hist07:10:021
1763099141,10cyclictest17690-21latency_hist07:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional