You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 09:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Mon Sep 07, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25090997261,8cyclictest35966irq/58-eth021:45:153
25090996251,6cyclictest35966irq/58-eth023:25:153
2505599626,40cyclictest23828-21ssh21:35:140
2505599615,42cyclictest27079-21ssh00:00:160
2506599552,39cyclictest20102-21ssh21:15:131
2506599501,36cyclictest16568-21ssh00:30:141
2506599494,9cyclictest131rcu_preempt22:35:141
2506599493,10cyclictest131rcu_preempt22:00:111
2509099483,15cyclictest35966irq/58-eth020:05:123
25075994813,14cyclictest27-21ksoftirqd/222:40:142
25065994811,26cyclictest9476-21ssh21:45:141
25090994511,10cyclictest35966irq/58-eth000:10:143
2506599443,6cyclictest131rcu_preempt19:30:151
2505599444,29cyclictest3765-21ssh00:05:150
2509099432,26cyclictest35966irq/58-eth020:10:123
2506599435,4cyclictest131rcu_preempt22:25:121
2505599432,33cyclictest22386-21ssh22:10:150
2505599420,33cyclictest24811-21rm23:05:130
2509099414,22cyclictest35966irq/58-eth000:05:153
2507599411,22cyclictest4257-21ssh22:00:132
2509099404,22cyclictest35966irq/58-eth023:15:163
2506599402,31cyclictest7519-21ssh00:25:151
2507599392,29cyclictest21937-21ssh00:31:092
25090993817,10cyclictest35966irq/58-eth023:40:153
25055993820,11cyclictest12-21ksoftirqd/020:00:150
2509099372,22cyclictest35966irq/58-eth022:10:153
25065993622,6cyclictest21-21ksoftirqd/120:45:131
2509099352,18cyclictest35966irq/58-eth000:15:123
2505599352,27cyclictest29169-21sh21:20:130
2509099346,16cyclictest35966irq/58-eth022:05:133
2509099342,16cyclictest35966irq/58-eth019:40:143
2509099334,18cyclictest35966irq/58-eth022:45:143
2506599338,11cyclictest131rcu_preempt22:15:121
25065993326,4cyclictest10662-21diskmemload21:20:121
2505599332,25cyclictest30109-21ssh22:50:160
2509099323,15cyclictest35966irq/58-eth019:30:143
2507599325,9cyclictest0-21swapper/222:35:142
2507599324,19cyclictest27346irq/40-dwc2_hso21:05:142
2507599322,16cyclictest14841-21ssh21:30:172
2505599327,18cyclictest12-21ksoftirqd/019:35:150
2505599325,20cyclictest12-21ksoftirqd/000:35:130
2505599321,25cyclictest8903-21ssh23:50:140
25065993120,4cyclictest21-21ksoftirqd/123:00:151
25065993120,4cyclictest21-21ksoftirqd/123:00:141
2505599313,20cyclictest8151-21ssh22:20:130
2509099301,18cyclictest35966irq/58-eth023:30:133
2507599302,21cyclictest0-21swapper/220:10:142
2506599302,20cyclictest28598-21ssh23:25:141
2506599302,18cyclictest21-21ksoftirqd/121:10:151
25065993013,4cyclictest21-21ksoftirqd/122:55:111
2505599303,19cyclictest5789-21ssh21:25:150
2505599303,19cyclictest5789-21ssh21:25:140
25055993020,7cyclictest0-21swapper/022:45:140
2509099292,15cyclictest35966irq/58-eth000:25:133
2507599294,16cyclictest27346irq/40-dwc2_hso23:35:142
2507599294,16cyclictest27346irq/40-dwc2_hso23:35:132
2507599292,17cyclictest27346irq/40-dwc2_hso23:25:142
25075992921,3cyclictest23252-21ssh23:40:142
2506599294,5cyclictest131rcu_preempt00:20:131
2505599293,21cyclictest449-21ssh21:40:150
2507599283,20cyclictest0-21swapper/200:25:142
2507599282,21cyclictest21804-21ssh00:15:132
2507599282,20cyclictest0-21swapper/220:30:142
2506599287,15cyclictest21-21ksoftirqd/123:20:131
2506599282,20cyclictest31802-21ssh00:37:091
2506599282,20cyclictest1463-21ssh23:10:151
2506599281,24cyclictest24810-21sh23:05:141
2509099273,15cyclictest35966irq/58-eth000:30:133
25090992710,11cyclictest35966irq/58-eth021:15:143
2507599273,14cyclictest27346irq/40-dwc2_hso20:20:142
2507599272,19cyclictest0-21swapper/221:45:152
2506599270,19cyclictest21785-21ssh00:15:121
2505599272,20cyclictest6665-21ssh22:55:140
2509099261,16cyclictest35966irq/58-eth021:25:143
2509099261,16cyclictest35966irq/58-eth021:25:133
2507599262,9cyclictest30958-21ssh00:36:082
2507599262,9cyclictest0-21swapper/221:35:142
2507599262,19cyclictest13259-21apt-get22:05:122
2507599262,19cyclictest0-21swapper/221:15:142
2507599262,10cyclictest0-21swapper/200:05:152
2507599261,7cyclictest11084-21ssh21:10:142
2507599261,18cyclictest27346irq/40-dwc2_hso19:20:122
2506599261,20cyclictest5268-21ssh23:30:151
2505599262,20cyclictest23253-21ssh23:40:140
2509099253,17cyclictest0-21swapper/300:37:083
2509099251,13cyclictest35966irq/58-eth023:20:143
2507599253,15cyclictest0-21swapper/222:55:122
2507599252,15cyclictest0-21swapper/220:00:142
25075992517,5cyclictest0-21swapper/222:15:122
2507599251,14cyclictest22607-21ls20:45:232
2507599251,14cyclictest12531-21ssh23:50:252
2507599251,13cyclictest16880-21cut19:35:162
2507599251,12cyclictest26237-21ssh22:30:152
2506599259,11cyclictest21-21ksoftirqd/119:25:131
2505599256,10cyclictest0-21swapper/021:50:150
2505599252,18cyclictest14831-21ssh21:30:150
2509099241,8cyclictest0-21swapper/320:50:143
2509099241,13cyclictest0-21swapper/321:35:143
25090992410,11cyclictest0-21swapper/321:40:243
25090992410,11cyclictest0-21swapper/300:00:243
2507599243,16cyclictest27346irq/40-dwc2_hso23:15:142
2507599243,16cyclictest27346irq/40-dwc2_hso20:15:122
2507599243,16cyclictest25039-21cyclictest19:10:142
2507599241,18cyclictest27346irq/40-dwc2_hso23:55:152
2506599242,15cyclictest8799-21ssh23:50:081
2505599247,8cyclictest12-21ksoftirqd/021:12:120
2505599243,12cyclictest0-21swapper/021:00:130
2505599242,18cyclictest19570-21ssh23:20:140
2509099232,12cyclictest35966irq/58-eth023:10:143
2507599233,6cyclictest0-21swapper/200:20:092
2507599232,18cyclictest0-21swapper/219:50:132
2507599232,17cyclictest27346irq/40-dwc2_hso00:10:132
2507599231,10cyclictest20988-21ssh22:45:122
2506599234,14cyclictest0-21swapper/121:40:151
2506599233,15cyclictest27529-21ssh21:55:121
2506599231,17cyclictest13292-21ssh22:05:131
2509099227,8cyclictest35966irq/58-eth022:40:153
2509099221,18cyclictest0-21swapper/322:20:293
2507599224,9cyclictest11970-21cpu19:30:152
2507599223,15cyclictest27346irq/40-dwc2_hso23:10:162
25075992212,8cyclictest0-21swapper/223:50:012
2507599221,16cyclictest17174-21ssh22:25:132
25075992210,9cyclictest12967-21ssh23:33:592
2507599220,5cyclictest13281-21find20:05:132
2507599220,19cyclictest6911-21ls19:55:242
2506599222,16cyclictest23366-21ssh23:40:161
2506599222,14cyclictest27087-21sh00:00:151
25065992213,7cyclictest21-21ksoftirqd/122:30:151
2506599221,18cyclictest18575-21ssh21:50:171
2506599220,19cyclictest10817-21ls23:15:181
2505599227,9cyclictest12-21ksoftirqd/021:15:260
2505599227,7cyclictest12-21ksoftirqd/022:34:500
2505599226,2cyclictest131rcu_preempt00:34:090
2505599222,16cyclictest2825-21apt-get22:35:130
25055992212,4cyclictest0-21swapper/023:59:060
2505599221,18cyclictest21659-21grep19:40:170
2505599221,16cyclictest21775-21apt-get00:15:120
2509099214,9cyclictest35966irq/58-eth022:50:173
2509099211,13cyclictest35966irq/58-eth021:20:133
2507599218,10cyclictest19536-21ssh21:50:242
2507599216,12cyclictest17315-21kthreadcore23:00:232
2507599216,12cyclictest17315-21kthreadcore23:00:222
2507599211,5cyclictest27505-21find21:55:112
25075992115,3cyclictest29770-21ssh21:20:182
25075992114,4cyclictest3548-21ssh00:05:012
2507599211,17cyclictest0-21swapper/219:40:242
2507599211,14cyclictest0-21swapper/221:45:012
2507599211,11cyclictest24976-21ssh23:05:172
2507599211,11cyclictest19670-21df_inode23:20:162
2507599211,10cyclictest30265-21fschecks_count22:50:192
25075992110,8cyclictest12118-21sh22:20:302
2506599213,9cyclictest131rcu_preempt00:10:141
2506599213,12cyclictest0-21swapper/121:00:131
25065992111,7cyclictest4624-21cron20:30:011
25065992111,5cyclictest10662-21diskmemload23:45:131
2505599219,2cyclictest131rcu_preempt23:16:530
2505599217,4cyclictest12-21ksoftirqd/022:16:470
2505599213,15cyclictest8997-21wc23:30:280
2505599211,8cyclictest131rcu_preempt22:06:230
25055992113,6cyclictest0-21swapper/020:10:140
25055992113,4cyclictest12-21ksoftirqd/019:10:140
2505599211,17cyclictest3977-21ls22:00:010
2505599211,14cyclictest24056-21find20:50:130
2509099209,8cyclictest0-21swapper/322:01:353
2509099202,11cyclictest0-21swapper/319:55:043
25090992011,6cyclictest0-21swapper/322:35:013
25090992010,8cyclictest0-21swapper/322:58:513
25090992010,7cyclictest0-21swapper/323:47:043
2507599206,11cyclictest28666-21cron20:55:012
2507599202,7cyclictest0-21swapper/219:25:132
2507599202,11cyclictest11866-21kthreadcore20:35:212
2507599201,17cyclictest14807-21cut20:40:172
2507599201,15cyclictest30110-21uniq19:15:162
2507599201,15cyclictest1417-21sort21:00:162
2507599201,11cyclictest30922-21idleruntime-cro19:50:012
2507599201,11cyclictest23842-21rm22:10:232
2506599205,12cyclictest0-21swapper/122:45:131
2506599202,13cyclictest844-21munin-node23:55:151
2506599202,12cyclictest14247-21ssh23:35:131
2506599202,12cyclictest14247-21ssh23:35:131
25065992013,4cyclictest23025-21irqrtprio22:10:211
25065992012,5cyclictest19209-21grep20:45:021
25065992012,5cyclictest0-21swapper/119:55:291
2506599201,11cyclictest8143-21tr21:25:211
2506599201,11cyclictest8143-21tr21:25:211
25065992010,7cyclictest7258-21cat00:05:241
2505599209,2cyclictest131rcu_preempt19:25:280
2505599208,4cyclictest12-21ksoftirqd/023:48:590
2505599207,10cyclictest4966-21cat23:10:250
2505599206,7cyclictest12-21ksoftirqd/022:42:470
2505599206,5cyclictest12-21ksoftirqd/020:45:120
2505599201,3cyclictest131rcu_preempt00:10:190
25055992013,5cyclictest0-21swapper/020:20:250
25055992013,4cyclictest12-21ksoftirqd/022:00:300
2505599201,13cyclictest3610-21find19:55:120
2505599201,10cyclictest12-21ksoftirqd/019:50:020
25055992010,6cyclictest12-21ksoftirqd/020:25:120
25055992010,4cyclictest12-21ksoftirqd/022:27:270
2509099198,8cyclictest0-21swapper/321:58:343
2509099193,9cyclictest35966irq/58-eth021:30:163
2509099192,11cyclictest0-21swapper/323:55:053
2509099192,11cyclictest0-21swapper/300:20:083
2509099191,13cyclictest0-21swapper/321:05:153
2509099191,13cyclictest0-21swapper/319:45:113
25075991911,5cyclictest6179-21tail21:25:182
25075991911,5cyclictest6179-21tail21:25:182
2507599191,15cyclictest27346irq/40-dwc2_hso20:55:142
2506599192,4cyclictest131rcu_preempt19:15:011
2506599192,13cyclictest11926-21ssh22:40:151
25065991911,5cyclictest0-21swapper/120:55:281
2505599195,4cyclictest151rcuc/019:15:230
2505599194,11cyclictest25039-21cyclictest00:20:140
2505599193,3cyclictest131rcu_preempt21:05:150
2505599191,9cyclictest3792-21kthreadcore19:20:180
25055991913,4cyclictest0-21swapper/020:35:150
2505599190,6cyclictest13282-21/usr/sbin/munin21:45:290
2505599190,12cyclictest26551-21timerandwakeup20:15:280
2509099189,6cyclictest0-21swapper/323:07:553
2509099188,6cyclictest35966irq/58-eth023:00:153
2509099188,6cyclictest35966irq/58-eth023:00:153
2509099182,10cyclictest0-21swapper/320:35:143
2509099182,10cyclictest0-21swapper/320:15:143
2509099182,10cyclictest0-21swapper/320:00:153
25090991812,4cyclictest0-21swapper/321:10:093
2509099181,12cyclictest0-21swapper/322:30:033
25090991810,6cyclictest0-21swapper/323:53:493
25090991810,6cyclictest0-21swapper/322:35:283
2507599181,8cyclictest25184-21latency_hist19:10:012
2507599180,9cyclictest32465-21/usr/sbin/munin20:25:152
2506599181,9cyclictest21218-21kthreadcore20:10:231
2506599181,14cyclictest3476-21sed19:55:021
25065991811,4cyclictest24573-21kthreadcore21:35:201
2506599181,10cyclictest31368-21ls20:20:301
2506599181,10cyclictest13741-21irqcore20:05:181
2505599188,8cyclictest3303-21cron19:55:010
2505599188,7cyclictest14569-21df_inode20:40:160
2505599188,7cyclictest11967-21apt-get19:30:140
2505599186,9cyclictest7535-21cpu00:25:140
2505599186,8cyclictest0-21swapper/023:25:130
2505599182,3cyclictest131rcu_preempt20:30:140
2505599181,3cyclictest131rcu_preempt23:35:120
2505599181,3cyclictest131rcu_preempt23:35:120
25055991812,4cyclictest16485-21kthreadcore23:00:230
25055991812,4cyclictest16485-21kthreadcore23:00:220
25055991812,4cyclictest0-21swapper/020:05:240
2509099179,6cyclictest0-21swapper/323:35:273
2509099179,6cyclictest0-21swapper/323:35:263
2509099179,6cyclictest0-21swapper/322:16:303
2509099172,12cyclictest0-21swapper/321:50:333
2509099172,10cyclictest0-21swapper/320:55:143
2509099172,10cyclictest0-21swapper/320:45:143
2509099172,10cyclictest0-21swapper/320:30:153
2509099172,10cyclictest0-21swapper/320:20:153
2509099171,10cyclictest0-21swapper/319:15:223
2506599179,5cyclictest0-21swapper/119:20:151
2506599178,6cyclictest27227-21kthreadcore19:45:181
2506599171,6cyclictest14996-21fschecks_count21:30:181
2506599171,12cyclictest8714-21grep20:30:301
25065991710,4cyclictest27470-21idleruntime-cro20:20:011
2505599177,6cyclictest12-21ksoftirqd/020:55:120
2509099162,9cyclictest0-21swapper/319:50:133
2506599166,8cyclictest30082-21apt-get22:50:151
2506599163,10cyclictest24836-21latency19:40:251
2506599162,2cyclictest131rcu_preempt21:05:121
2506599161,13cyclictest30632-21cut19:15:181
2506599161,12cyclictest28676-21munin-run20:55:011
2506599161,11cyclictest8263-21diskstats22:20:151
2509099151,12cyclictest0-21swapper/319:15:023
25090991510,3cyclictest0-21swapper/320:25:263
2506599153,9cyclictest9784-21fschecks_count20:35:161
2506599151,11cyclictest8454-21apt-get20:00:151
2506599151,11cyclictest25191-21latency_hist19:10:011
2506599151,11cyclictest17070-21irqcore19:35:181
2505599151,10cyclictest25124-21cat19:10:010
2509099147,4cyclictest0-21swapper/319:25:143
2509099141,6cyclictest0-21swapper/319:20:173
2509099131,10cyclictest0-21swapper/321:05:033
2509099122,6cyclictest0-21swapper/319:35:143
2509099112,5cyclictest0-21swapper/320:40:153
2509099111,5cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional