You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 10:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Jul 19, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
534499572,42cyclictest2617-21ssh22:10:130
535799542,40cyclictest17013-21ssh22:00:131
538599534,12cyclictest131rcu_preempt19:40:122
534499502,4cyclictest131rcu_preempt23:35:160
534499483,10cyclictest131rcu_preempt22:30:130
535799452,34cyclictest12151-21ssh23:45:141
538599441,28cyclictest21161-21ssh23:50:162
535799441,34cyclictest13744-21ssh23:10:121
534499443,4cyclictest131rcu_preempt21:15:160
538599439,23cyclictest27-21ksoftirqd/223:30:162
538599430,24cyclictest1079-21ssh22:45:142
535799434,34cyclictest8573-21ssh23:25:121
534499431,32cyclictest22419-21ssh21:45:160
535799421,35cyclictest27600-21ssh21:30:131
534499411,27cyclictest30153-21ssh23:55:130
538599396,17cyclictest131rcu_preempt22:02:542
538599392,31cyclictest13269-21ssh00:39:312
5344993919,12cyclictest664-21ntpd00:00:140
534499390,26cyclictest0-21swapper/022:35:140
541699385,29cyclictest35966irq/58-eth021:30:133
541699382,27cyclictest35966irq/58-eth023:20:143
538599383,29cyclictest0-21swapper/223:13:312
538599382,19cyclictest23771-21cat22:55:302
538599380,6cyclictest131rcu_preempt00:20:132
534499384,5cyclictest131rcu_preempt23:10:140
541699372,20cyclictest35966irq/58-eth021:05:143
541699371,21cyclictest35966irq/58-eth021:15:153
538599362,18cyclictest6007-21cat00:15:292
535799367,23cyclictest21-21ksoftirqd/123:55:151
5344993618,13cyclictest5332-21cyclictest23:20:150
541699353,21cyclictest35966irq/58-eth023:50:143
538599353,9cyclictest0-21swapper/223:08:082
538599352,17cyclictest23652-21diskmemload21:57:542
534499352,19cyclictest4749-21ssh23:05:130
538599342,16cyclictest6551-21ssh23:23:122
535799342,23cyclictest1060-21ssh22:45:131
5344993419,12cyclictest12-21ksoftirqd/019:20:130
541699335,23cyclictest35966irq/58-eth022:25:133
541699335,15cyclictest35966irq/58-eth000:35:123
541699333,25cyclictest35966irq/58-eth020:30:133
541699333,25cyclictest35966irq/58-eth020:30:123
538599335,18cyclictest261rcuc/220:30:122
538599335,18cyclictest261rcuc/220:30:112
538599332,15cyclictest30181-21sh23:18:302
5385993313,15cyclictest29730-21sh22:25:132
535799334,5cyclictest131rcu_preempt20:05:121
5357993314,11cyclictest21-21ksoftirqd/100:20:151
5344993316,11cyclictest12-21ksoftirqd/022:05:120
534499331,22cyclictest18618-21ssh21:25:130
5416993212,13cyclictest0-21swapper/320:25:133
538599326,14cyclictest27346irq/40-dwc2_hso19:10:122
538599325,15cyclictest27346irq/40-dwc2_hso21:50:132
534499321,25cyclictest0-21swapper/019:15:140
5416993111,11cyclictest35966irq/58-eth000:20:133
538599311,22cyclictest0-21swapper/221:30:142
535799312,23cyclictest11806-21ssh23:08:091
534499313,19cyclictest22847-21ssh23:15:140
541699302,18cyclictest35966irq/58-eth022:35:143
538599306,12cyclictest27346irq/40-dwc2_hso20:25:122
538599303,19cyclictest27-21ksoftirqd/223:35:162
535799302,22cyclictest19078-21ssh22:55:141
535799302,21cyclictest8731-21ssh00:18:301
5357993011,13cyclictest21-21ksoftirqd/120:40:131
534499302,21cyclictest12412-21ssh00:38:290
534499302,20cyclictest12788-21cat21:55:310
538599295,15cyclictest27346irq/40-dwc2_hso20:20:132
538599292,21cyclictest0-21swapper/222:05:122
538599292,16cyclictest27346irq/40-dwc2_hso19:15:142
538599292,11cyclictest23079-21ssh00:08:302
535799293,20cyclictest15746-21ssh00:05:151
535799292,21cyclictest9029-21ssh21:35:301
535799292,21cyclictest24474-21ssh22:40:141
534499291,5cyclictest17737-21apt-get20:30:110
534499291,5cyclictest17737-21apt-get20:30:110
541699283,17cyclictest0-21swapper/323:13:303
5416992817,7cyclictest35966irq/58-eth000:10:143
538599283,14cyclictest6426-1kworker/u9:122:30:132
538599282,21cyclictest27-21ksoftirqd/222:13:312
538599281,18cyclictest27346irq/40-dwc2_hso21:05:132
535799282,19cyclictest26971-21sed23:15:311
5357992814,7cyclictest23652-21diskmemload22:25:131
534499282,20cyclictest9099-21cat21:35:310
534499282,19cyclictest31996-21rm00:13:240
541699273,16cyclictest0-21swapper/323:08:083
541699273,16cyclictest0-21swapper/300:18:303
541699272,16cyclictest35966irq/58-eth022:55:133
538599273,16cyclictest27346irq/40-dwc2_hso20:10:142
538599272,9cyclictest17383-21ssh22:53:302
538599272,9cyclictest17383-21ssh22:53:302
538599271,19cyclictest3396-21grep23:40:172
535799272,19cyclictest24750-21timerwakeupswit22:20:301
535799271,23cyclictest10068-21sh22:50:141
535799271,23cyclictest10068-21sh22:50:131
534499272,19cyclictest2605-21ssh21:33:300
534499271,20cyclictest15910-21kworker/u8:123:50:150
538599262,19cyclictest29309-21ssh00:10:312
538599262,17cyclictest27346irq/40-dwc2_hso21:45:152
538599262,16cyclictest27346irq/40-dwc2_hso19:50:142
538599260,18cyclictest23507-21timerwakeupswit00:25:312
535799265,11cyclictest21-21ksoftirqd/121:23:001
535799265,11cyclictest21-21ksoftirqd/121:22:591
5357992613,8cyclictest201rcuc/123:20:141
535799261,21cyclictest7968-21ssh21:55:131
534499265,14cyclictest11645-21ssh22:15:110
534499262,19cyclictest24748-21timerwakeupswit22:20:300
534499262,18cyclictest21835-21/usr/sbin/munin22:00:300
534499261,19cyclictest24064-21sh21:10:140
541699255,12cyclictest0-21swapper/323:45:153
541699252,18cyclictest0-21swapper/319:25:153
541699252,15cyclictest36066irq/59-eth023:15:153
5416992514,7cyclictest0-21swapper/323:40:263
541699251,17cyclictest0-21swapper/323:35:163
541699251,13cyclictest0-21swapper/322:10:133
538599252,4cyclictest131rcu_preempt20:50:132
538599252,20cyclictest28503-21ssh00:30:132
538599252,20cyclictest0-21swapper/219:35:122
538599252,19cyclictest24065-21ssh21:10:142
538599252,15cyclictest27346irq/40-dwc2_hso00:00:152
538599251,9cyclictest9595-21ssh21:20:142
538599251,9cyclictest9595-21ssh21:20:132
5385992516,6cyclictest14910-21ssh21:40:172
538599251,21cyclictest6315-21latency_hist00:00:012
535799252,17cyclictest585-21apt-get21:15:151
5357992517,4cyclictest32095-21ntp_192.168.11400:30:241
534499254,3cyclictest131rcu_preempt00:20:130
534499252,18cyclictest8147-21apt-get20:20:130
534499252,17cyclictest3213-21ssh23:40:140
534499252,17cyclictest2630-21ssh21:50:260
534499251,19cyclictest10069-21ssh22:50:140
534499251,19cyclictest10069-21ssh22:50:130
541699244,13cyclictest0-21swapper/322:45:143
538599245,13cyclictest27-21ksoftirqd/221:25:152
538599243,17cyclictest20911-21cron20:00:012
538599241,9cyclictest15425-21ssh22:35:122
5385992416,5cyclictest9347-21ssh23:25:192
5385992412,7cyclictest0-21swapper/223:00:142
538599241,15cyclictest18785-21apt-config21:05:012
538599241,13cyclictest24045-21kthreadcore22:20:242
535799245,3cyclictest131rcu_preempt21:27:271
541699237,10cyclictest35966irq/58-eth023:25:263
541699232,7cyclictest0-21swapper/321:55:143
538599232,17cyclictest22212-21kthreadcore20:00:202
538599232,16cyclictest0-21swapper/222:17:302
538599231,13cyclictest15573-21cut23:45:232
5357992314,6cyclictest0-21swapper/119:10:171
535799231,18cyclictest25983-21find22:05:101
535799231,18cyclictest21239-21find20:00:121
535799231,17cyclictest9537-21ssh22:12:561
535799231,15cyclictest19492-21ssh00:25:131
535799230,19cyclictest16338-21find19:55:121
534499239,11cyclictest12-21ksoftirqd/022:40:110
538599225,10cyclictest29310-21latency20:05:232
538599224,10cyclictest5427-21sed19:10:012
538599224,10cyclictest28491-21kthreadcore20:40:202
538599223,11cyclictest27346irq/40-dwc2_hso19:30:132
538599222,12cyclictest9129-21kthreadcore19:45:232
538599222,10cyclictest24628-21fschecks_count22:40:152
538599221,17cyclictest14116-21latency_hist21:00:012
538599221,13cyclictest15782-21irqrtprio19:20:192
538599221,13cyclictest1017-21irqcore21:15:222
535799229,7cyclictest21-21ksoftirqd/100:00:251
535799229,4cyclictest21-21ksoftirqd/120:35:181
535799228,7cyclictest21-21ksoftirqd/122:39:191
535799227,8cyclictest21-21ksoftirqd/121:12:091
535799226,8cyclictest21-21ksoftirqd/123:01:481
535799225,3cyclictest131rcu_preempt00:12:081
5357992213,5cyclictest22385-21/usr/sbin/munin21:45:181
5357992212,6cyclictest0-21swapper/120:45:121
535799221,16cyclictest17649-21sh23:30:151
535799221,14cyclictest6650-21ls20:15:221
534499221,18cyclictest22954-21ls22:55:280
541699219,9cyclictest0-21swapper/322:18:313
538599217,10cyclictest0-21swapper/219:25:182
538599212,6cyclictest0-21swapper/220:45:122
538599211,9cyclictest8117-21cat21:35:272
538599211,9cyclictest4429-21kthreadcore20:15:192
538599211,10cyclictest23008-21irqcore20:35:192
535799216,5cyclictest21-21ksoftirqd/123:35:211
535799215,9cyclictest21-21ksoftirqd/121:43:551
5357992113,6cyclictest13341-21sshd22:33:001
535799211,2cyclictest131rcu_preempt21:51:491
5357992112,6cyclictest0-21swapper/120:20:131
5357992112,3cyclictest0-21swapper/100:35:281
535799211,15cyclictest6612-21latency23:40:241
534499212,14cyclictest844-21munin-node19:25:160
534499211,6cyclictest23222-21kthreadcore20:35:210
534499211,16cyclictest9594-21ssh21:20:140
534499211,16cyclictest9594-21ssh21:20:130
534499211,16cyclictest5552-21apt-get19:10:130
534499211,16cyclictest21372-21ssh23:30:280
541699209,6cyclictest0-21swapper/322:54:083
541699209,6cyclictest0-21swapper/322:54:073
541699208,9cyclictest0-21swapper/300:02:353
541699206,9cyclictest35966irq/58-eth000:08:313
541699203,11cyclictest0-21swapper/321:40:103
5416992013,4cyclictest35966irq/58-eth021:25:153
541699201,11cyclictest0-21swapper/319:10:123
535799209,4cyclictest21-21ksoftirqd/120:25:261
535799208,9cyclictest30841-21apt-get20:10:141
535799206,4cyclictest21-21ksoftirqd/119:40:151
5357992014,4cyclictest0-21swapper/121:00:161
535799201,10cyclictest12860-21kthreadcore22:15:171
534499209,8cyclictest614-21dbus-daemon19:32:160
534499203,13cyclictest0-21swapper/021:05:150
534499201,5cyclictest6534-21munin-run19:45:010
5344992012,5cyclictest17034-21kthreadcore00:05:200
534499201,15cyclictest28140-21rm23:00:150
534499201,10cyclictest9129-21cron20:55:010
541699193,11cyclictest0-21swapper/323:03:313
541699193,11cyclictest0-21swapper/322:05:313
541699193,11cyclictest0-21swapper/322:02:543
541699192,8cyclictest0-21swapper/321:45:163
541699192,11cyclictest0-21swapper/322:40:053
541699192,10cyclictest0-21swapper/320:15:043
541699191,13cyclictest0-21swapper/319:20:143
541699191,12cyclictest0-21swapper/321:05:013
541699191,12cyclictest0-21swapper/319:55:283
5416991910,6cyclictest0-21swapper/321:50:223
5416991910,6cyclictest0-21swapper/321:35:273
5416991910,6cyclictest0-21swapper/321:13:043
535799199,7cyclictest19985-21apt-get19:25:161
535799196,7cyclictest21-21ksoftirqd/123:53:221
5357991914,3cyclictest0-21swapper/120:55:151
534499193,7cyclictest13973-21kthreadcore23:45:220
534499192,13cyclictest20920-21apt-get20:00:020
5344991912,5cyclictest30123-21ps22:25:180
5344991912,4cyclictest1280-21/usr/sbin/munin00:15:140
534499190,13cyclictest28460-21find00:30:110
541699189,6cyclictest0-21swapper/300:26:443
541699182,10cyclictest0-21swapper/320:45:133
541699182,10cyclictest0-21swapper/320:20:153
541699182,10cyclictest0-21swapper/320:00:053
541699182,10cyclictest0-21swapper/319:30:133
5416991810,6cyclictest0-21swapper/322:24:543
5416991810,5cyclictest35966irq/58-eth021:20:123
5416991810,5cyclictest35966irq/58-eth021:20:123
535799188,3cyclictest131rcu_preempt19:30:151
535799183,13cyclictest9159-21kthreadcore19:45:221
535799181,8cyclictest21-21ksoftirqd/119:15:241
5357991813,3cyclictest0-21swapper/120:50:151
5357991813,3cyclictest0-21swapper/120:30:141
5357991813,3cyclictest0-21swapper/120:30:141
5357991812,4cyclictest0-21swapper/119:35:221
5344991811,5cyclictest21275-21kthreadcore00:25:230
534499181,14cyclictest5229-21apt-get19:10:020
541699179,6cyclictest0-21swapper/323:56:413
541699179,6cyclictest0-21swapper/323:30:313
541699172,11cyclictest0-21swapper/322:30:043
541699172,10cyclictest0-21swapper/319:50:153
541699171,8cyclictest0-21swapper/319:40:013
5416991710,4cyclictest0-21swapper/300:30:473
535799179,6cyclictest17960-21cut19:20:211
535799176,2cyclictest131rcu_preempt19:50:241
5357991711,4cyclictest0-21swapper/121:10:011
534499178,6cyclictest3068-21ps22:45:230
534499178,6cyclictest17791-21wc21:00:280
534499177,8cyclictest27431-21df20:40:150
534499176,7cyclictest0-21swapper/020:15:110
534499174,10cyclictest9944-21kthreadcore19:45:230
534499172,12cyclictest21974-21ssh21:45:010
534499172,10cyclictest8559-21ssh23:25:120
534499172,10cyclictest0-21swapper/020:10:160
5344991711,4cyclictest0-21swapper/020:25:150
5344991711,4cyclictest0-21swapper/019:35:280
541699162,7cyclictest0-21swapper/319:45:143
541699161,10cyclictest0-21swapper/320:10:163
534499165,9cyclictest20108-21timerwakeupswit19:55:280
534499161,13cyclictest29844-21unixbench_multi20:05:280
534499161,13cyclictest12052-21irqcore19:50:180
534499161,12cyclictest2315-21tr20:45:210
534499161,11cyclictest9458-21diskstats20:55:160
541699149,3cyclictest33-21ksoftirqd/320:05:123
541699136,3cyclictest0-21swapper/320:35:293
541699132,6cyclictest0-21swapper/320:55:143
535799135,5cyclictest5363-21date19:10:011
541699128,2cyclictest33-21ksoftirqd/320:40:173
541699127,3cyclictest33-21ksoftirqd/319:40:143
541699122,4cyclictest0-21swapper/319:15:143
541699121,9cyclictest0-21swapper/320:50:183
541699111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional