You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 23:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun Jun 14, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26444995338,7cyclictest35966irq/58-eth010:10:133
26444995338,7cyclictest35966irq/58-eth010:10:133
2643699511,34cyclictest15773-21ssh12:30:122
2642799487,6cyclictest131rcu_preempt10:10:131
2642799487,6cyclictest131rcu_preempt10:10:131
2641999481,34cyclictest6791-21ssh12:25:110
2643699471,22cyclictest25074-21ssh09:35:122
2641999472,33cyclictest3259-21ssh12:05:120
2643699462,33cyclictest1683-21ssh11:10:122
2644499446,22cyclictest35966irq/58-eth011:25:133
2643699432,34cyclictest17716-21memory11:35:222
2643699424,28cyclictest23206-21ssh10:10:132
2643699424,28cyclictest23206-21ssh10:10:132
26436994215,22cyclictest630-21Async09:36:312
2643699414,27cyclictest5206-21ssh10:00:122
2643699412,36cyclictest11443-21kthreadcore11:15:182
2644499402,34cyclictest0-21swapper/311:35:123
2644499391,23cyclictest35966irq/58-eth010:45:123
2643699382,18cyclictest22422-21ssh10:09:342
2641999383,8cyclictest12-21ksoftirqd/009:05:130
26444993717,9cyclictest0-21swapper/309:20:123
2643699372,29cyclictest1168-21ssh11:25:342
2643699372,18cyclictest16581-21sed11:00:172
2644499365,28cyclictest35966irq/58-eth009:10:143
2644499362,2cyclictest0-21swapper/307:55:133
2644499362,2cyclictest0-21swapper/307:55:133
2643699362,17cyclictest15447-21sh11:52:352
2643699362,17cyclictest12358-21diskmemload12:03:342
2641999364,25cyclictest8812-21ssh10:20:130
2644499351,2cyclictest0-21swapper/307:45:133
2643699352,26cyclictest11599-21usage_EthcTask12:25:302
2643699344,22cyclictest0-21swapper/210:55:172
26436993424,7cyclictest27-21ksoftirqd/210:20:122
2642799341,22cyclictest26812-21ssh10:30:101
2644499332,26cyclictest0-21swapper/311:40:123
2644499329,12cyclictest35966irq/58-eth012:30:113
2644499322,2cyclictest0-21swapper/310:20:123
2644499322,16cyclictest35966irq/58-eth012:00:133
2643699329,16cyclictest0-21swapper/212:20:272
2643699324,17cyclictest27346irq/40-dwc2_hso09:30:132
2642799322,22cyclictest19740-21ssh11:20:121
2642799320,3cyclictest131rcu_preempt07:40:121
2641999321,22cyclictest19696-21ssh09:50:130
2644499313,25cyclictest0-21swapper/308:55:103
2644499311,2cyclictest0-21swapper/307:40:123
2644499311,1cyclictest0-21swapper/309:50:143
2643699313,17cyclictest27346irq/40-dwc2_hso08:45:122
2643699312,4cyclictest271ksoftirqd/208:05:212
2643699312,22cyclictest8828-21ssh11:30:212
2642799319,8cyclictest21-21ksoftirqd/111:30:101
2642799312,23cyclictest15963-21ssh12:10:271
2641999318,19cyclictest12-21ksoftirqd/008:50:130
2641999315,19cyclictest21659-21ssh09:15:130
2644499305,12cyclictest35966irq/58-eth010:05:163
2644499304,16cyclictest0-21swapper/309:15:143
26444993014,11cyclictest35966irq/58-eth011:45:143
26444993014,10cyclictest35966irq/58-eth011:50:133
2644499301,3cyclictest0-21swapper/311:00:123
2643699307,14cyclictest27-21ksoftirqd/209:59:372
2643699303,16cyclictest27346irq/40-dwc2_hso07:10:112
2643699302,12cyclictest11604-21ssh10:39:342
2641999304,5cyclictest131rcu_preempt10:55:110
26444992915,11cyclictest35966irq/58-eth010:55:123
2644499291,3cyclictest0-21swapper/308:35:133
2644499291,2cyclictest0-21swapper/307:30:133
2644499291,25cyclictest0-21swapper/311:05:193
2644499291,23cyclictest0-21swapper/310:30:113
2644499291,22cyclictest0-21swapper/310:50:133
2643699294,6cyclictest131rcu_preempt09:00:112
2643699294,16cyclictest27346irq/40-dwc2_hso08:20:132
2643699294,16cyclictest27346irq/40-dwc2_hso08:20:132
2643699293,21cyclictest0-21swapper/212:19:342
2643699293,10cyclictest26694-21grep11:40:232
2643699292,21cyclictest0-21swapper/211:47:342
2643699292,20cyclictest0-21swapper/207:45:142
2642799291,5cyclictest131rcu_preempt10:55:121
2641999294,17cyclictest17854-21ssh11:55:130
2641999292,19cyclictest3206-21kthreadcore11:10:180
26419992913,2cyclictest131rcu_preempt07:15:130
2641999291,2cyclictest7423-1kworker/u9:009:10:140
2644499284,15cyclictest0-21swapper/311:10:133
2644499283,2cyclictest0-21swapper/311:15:123
2644499283,18cyclictest35966irq/58-eth010:40:143
2644499282,5cyclictest0-21swapper/312:10:133
2644499281,3cyclictest0-21swapper/309:00:123
2643699289,16cyclictest26337-21ssh09:15:322
2643699285,6cyclictest0-21swapper/207:20:152
2643699285,6cyclictest0-21swapper/207:20:142
2643699283,20cyclictest27-21ksoftirqd/210:25:152
2643699282,21cyclictest25724-21apt-get08:50:132
2642799282,19cyclictest3207-21kthreadcore11:10:181
2642799281,3cyclictest131rcu_preempt08:55:101
26419992822,3cyclictest131rcu_preempt10:35:110
2644499274,12cyclictest0-21swapper/312:35:123
2644499273,12cyclictest0-21swapper/312:09:353
2644499271,3cyclictest0-21swapper/309:28:393
2644499271,2cyclictest0-21swapper/309:30:143
2644499271,2cyclictest0-21swapper/308:00:133
2644499271,2cyclictest0-21swapper/307:20:143
2644499271,2cyclictest0-21swapper/307:20:143
2644499271,22cyclictest0-21swapper/310:00:313
2643699277,12cyclictest27-21ksoftirqd/212:35:532
2643699276,11cyclictest27346irq/40-dwc2_hso08:30:142
2643699276,11cyclictest27-21ksoftirqd/210:15:522
2643699274,13cyclictest27-21ksoftirqd/209:10:222
2643699274,10cyclictest27-21ksoftirqd/209:25:052
2643699272,19cyclictest0-21swapper/212:06:342
2643699272,10cyclictest20489-21ssh12:14:342
26436992716,5cyclictest261rcuc/209:45:102
2642799272,15cyclictest12358-21diskmemload11:05:131
26427992715,10cyclictest0-21swapper/109:25:261
2642799271,2cyclictest131rcu_preempt10:35:101
2641999271,19cyclictest32237-21ssh10:15:130
2644499263,18cyclictest0-21swapper/311:30:213
2644499262,21cyclictest0-21swapper/310:35:163
2644499261,8cyclictest0-21swapper/309:40:133
2644499261,3cyclictest0-21swapper/311:55:143
2644499261,3cyclictest0-21swapper/310:25:253
2644499261,2cyclictest0-21swapper/309:55:173
2644499261,22cyclictest0-21swapper/309:38:233
2644499261,22cyclictest0-21swapper/308:50:293
2644499261,21cyclictest0-21swapper/312:15:113
2643699265,11cyclictest27-21ksoftirqd/210:34:442
2643699264,10cyclictest27-21ksoftirqd/209:42:522
2643699263,11cyclictest27-21ksoftirqd/211:58:362
26436992613,9cyclictest27-21ksoftirqd/207:20:012
2642799262,18cyclictest22591-21ssh11:02:341
2641999266,4cyclictest12-21ksoftirqd/007:30:130
2641999265,13cyclictest7218-21ssh09:25:130
26419992613,8cyclictest12-21ksoftirqd/011:40:120
2644499252,2cyclictest0-21swapper/307:15:133
2644499252,18cyclictest0-21swapper/310:15:043
2644499251,4cyclictest0-21swapper/312:25:113
2644499251,3cyclictest0-21swapper/309:46:073
2644499251,2cyclictest0-21swapper/312:20:293
2644499251,2cyclictest0-21swapper/311:20:123
2644499251,21cyclictest0-21swapper/308:50:013
2643699256,12cyclictest27-21ksoftirqd/211:09:272
2643699255,12cyclictest131rcu_preempt09:50:262
2643699252,16cyclictest27-21ksoftirqd/208:25:132
26436992517,5cyclictest24546irq/40-fe98000007:30:132
26436992514,8cyclictest261rcuc/210:40:122
2643699251,21cyclictest0-21swapper/209:05:142
2643699250,7cyclictest0-21swapper/208:55:102
2642799254,4cyclictest131rcu_preempt11:35:121
2642799252,19cyclictest32256-21ssh10:15:141
2642799252,18cyclictest21463-21ntp_192.168.11411:55:241
2642799251,3cyclictest131rcu_preempt08:00:191
2642799251,19cyclictest24727-21ssh12:35:121
2642799251,16cyclictest0-21swapper/109:52:501
2641999258,3cyclictest131rcu_preempt09:41:180
2641999257,3cyclictest131rcu_preempt11:15:120
2644499242,7cyclictest0-21swapper/307:25:133
2644499241,3cyclictest0-21swapper/308:25:133
2644499241,3cyclictest0-21swapper/307:10:103
2644499241,2cyclictest0-21swapper/309:05:133
2644499241,20cyclictest0-21swapper/308:20:163
2644499241,20cyclictest0-21swapper/308:20:163
2643699246,6cyclictest131rcu_preempt11:21:532
2643699244,13cyclictest27-21ksoftirqd/207:55:182
2643699244,13cyclictest27-21ksoftirqd/207:55:182
26436992415,7cyclictest261rcuc/209:22:132
26436992413,4cyclictest0-21swapper/210:46:102
2642799246,3cyclictest131rcu_preempt10:00:121
2642799243,9cyclictest131rcu_preempt08:20:151
2642799243,9cyclictest131rcu_preempt08:20:141
2642799241,8cyclictest29226-21latency_hist08:20:011
2642799241,3cyclictest0-21swapper/112:18:191
26419992412,8cyclictest12-21ksoftirqd/007:45:130
2644499231,3cyclictest0-21swapper/308:15:183
2644499231,3cyclictest0-21swapper/307:35:153
2644499231,2cyclictest0-21swapper/308:30:243
2644499231,2cyclictest0-21swapper/307:50:123
2644499231,19cyclictest0-21swapper/308:10:013
2643699236,8cyclictest27-21ksoftirqd/208:40:272
2643699233,14cyclictest27346irq/40-dwc2_hso07:40:132
2643699231,5cyclictest27-21ksoftirqd/208:15:172
2643699231,4cyclictest27-21ksoftirqd/207:35:212
2642799231,2cyclictest131rcu_preempt12:25:121
2642799231,2cyclictest131rcu_preempt12:20:181
2642799231,2cyclictest131rcu_preempt12:00:121
26427992311,10cyclictest1657-21kthreadcore11:45:201
2642799230,17cyclictest10885-21kthreadcore11:50:211
2641999233,13cyclictest25085-21kworker/u8:009:45:120
2641999232,16cyclictest12358-21diskmemload10:11:340
2641999232,16cyclictest12358-21diskmemload10:11:340
2641999231,19cyclictest2109-21ls09:20:280
2644499221,3cyclictest0-21swapper/308:10:143
2644499221,2cyclictest0-21swapper/308:40:153
2643699223,6cyclictest27-21ksoftirqd/208:35:242
2642799228,3cyclictest21-21ksoftirqd/111:20:011
2642799226,6cyclictest131rcu_preempt10:40:371
2642799221,8cyclictest21069-21ssh12:31:041
2642799221,4cyclictest131rcu_preempt10:05:311
2642799221,4cyclictest0-21swapper/109:35:311
2642799221,2cyclictest131rcu_preempt11:40:271
2642799221,16cyclictest8891-21ls10:20:141
2641999222,15cyclictest5220-21ssh11:30:120
2641999222,14cyclictest20820-21kthreadcore11:20:190
2643699214,11cyclictest27346irq/40-dwc2_hso08:10:132
2643699211,12cyclictest4922-21sh07:55:012
2643699211,11cyclictest131rcu_preempt07:25:252
2642799212,3cyclictest131rcu_preempt10:25:271
2642799211,8cyclictest22857-21cut09:15:201
2642799211,7cyclictest26067-21usage_EthcTask10:45:311
2642799211,6cyclictest1843-21grep08:25:171
26427992115,4cyclictest0-21swapper/111:25:301
2642799211,4cyclictest0-21swapper/109:20:301
2642799210,3cyclictest131rcu_preempt09:40:131
2641999219,4cyclictest12-21ksoftirqd/011:29:440
2641999214,13cyclictest12418-21ssh10:40:140
2641999211,3cyclictest131rcu_preempt09:34:120
2641999211,2cyclictest131rcu_preempt11:52:590
26419992112,6cyclictest19423-21cron08:10:010
26419992110,8cyclictest24715-21apt-get12:35:120
26419992110,3cyclictest131rcu_preempt12:15:260
2644499201,2cyclictest0-21swapper/307:10:023
2643699201,14cyclictest10135-21grep08:00:162
2642799206,3cyclictest131rcu_preempt09:05:181
2642799205,2cyclictest131rcu_preempt07:25:281
2642799204,3cyclictest131rcu_preempt09:45:261
2642799201,9cyclictest0-21swapper/109:10:221
2642799201,6cyclictest3539-21munin-run07:20:011
2642799201,4cyclictest131rcu_preempt10:51:441
2642799201,4cyclictest0-21swapper/108:05:191
2642799201,2cyclictest131rcu_preempt07:50:121
26427992011,6cyclictest0-21swapper/108:10:171
2642799201,13cyclictest0-21swapper/109:58:051
2642799201,12cyclictest0-21swapper/108:45:181
2642799201,10cyclictest16134-21grep09:30:141
2641999204,4cyclictest131rcu_preempt09:00:120
2641999202,3cyclictest131rcu_preempt11:35:160
2641999201,3cyclictest131rcu_preempt12:20:190
2641999201,3cyclictest131rcu_preempt09:35:210
2641999201,16cyclictest25115-21sh11:05:140
26419992011,3cyclictest12-21ksoftirqd/008:55:110
2641999200,18cyclictest30342-21find10:50:110
2641999200,15cyclictest31788-21apt-get11:45:040
2642799195,7cyclictest21-21ksoftirqd/109:00:131
2642799191,6cyclictest30331-21cat08:55:021
2642799191,3cyclictest0-21swapper/112:05:191
2642799190,4cyclictest131rcu_preempt07:45:221
2642799190,3cyclictest131rcu_preempt07:30:221
2641999196,3cyclictest131rcu_preempt08:00:220
2641999193,3cyclictest131rcu_preempt10:30:160
26419991915,3cyclictest12-21ksoftirqd/011:00:130
26419991913,4cyclictest752-21usage_EthcTask09:55:310
2641999191,2cyclictest131rcu_preempt12:30:150
2641999191,2cyclictest131rcu_preempt08:35:150
2641999191,2cyclictest131rcu_preempt07:55:150
2641999191,2cyclictest131rcu_preempt07:55:150
26419991911,6cyclictest0-21swapper/012:10:130
2641999191,13cyclictest1662-21apt-get08:25:130
2643699181,9cyclictest26452-21cat07:10:022
2642799181,11cyclictest26992-21cut07:10:161
2642799181,11cyclictest19558-21tr07:35:191
2642799181,11cyclictest0-21swapper/108:35:251
2641999182,3cyclictest131rcu_preempt10:49:030
2641999182,2cyclictest131rcu_preempt08:30:210
2641999181,9cyclictest17824-21/usr/sbin/munin10:25:270
2641999181,6cyclictest27444-21kthreadcore08:15:210
2641999181,4cyclictest131rcu_preempt12:00:230
26419991813,3cyclictest22823-21latency_hist07:40:010
2641999181,2cyclictest131rcu_preempt07:25:270
26419991812,4cyclictest5183-21acpi10:00:110
26419991810,5cyclictest12358-21diskmemload10:05:160
2642799171,7cyclictest5429-21if_eth007:55:171
2642799171,7cyclictest5429-21if_eth007:55:171
2642799171,3cyclictest0-21swapper/108:30:201
2642799171,2cyclictest131rcu_preempt07:20:271
2642799171,2cyclictest131rcu_preempt07:20:271
2642799170,6cyclictest16345-21hwlatdetect08:40:161
2642799170,2cyclictest131rcu_preempt07:10:021
2641999178,7cyclictest23152-21ntp_states08:10:260
2641999172,4cyclictest12-21ksoftirqd/007:40:100
2641999171,3cyclictest131rcu_preempt07:10:120
2641999161,3cyclictest131rcu_preempt08:20:170
2641999161,3cyclictest131rcu_preempt08:20:170
2641999161,3cyclictest0-21swapper/007:50:180
2641999161,2cyclictest131rcu_preempt08:45:190
2641999161,11cyclictest19526-21ntp_192.168.11408:40:240
2641999157,6cyclictest12-21ksoftirqd/007:20:050
2641999157,6cyclictest12-21ksoftirqd/007:20:050
2641999141,9cyclictest26420-21cut07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional