You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun May 31, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7321996427,18cyclictest27-21ksoftirqd/200:25:132
7333996139,17cyclictest35966irq/58-eth022:35:133
732199591,31cyclictest30728-21ssh22:25:132
732199572,28cyclictest18065-21ssh23:30:122
730199551,19cyclictest1794-21sh22:45:140
7301995444,4cyclictest5465-21ssh23:05:140
732199520,28cyclictest0-21swapper/223:00:122
730199466,16cyclictest3514-21ssh23:40:120
733399446,22cyclictest35966irq/58-eth023:10:123
732199420,31cyclictest0-21swapper/221:45:132
7321994114,22cyclictest27-21ksoftirqd/220:00:132
7321994014,10cyclictest2966-21ssh00:33:112
730199403,7cyclictest131rcu_preempt21:25:140
730199401,16cyclictest12788-21ssh22:15:130
732199393,27cyclictest30554-21ssh23:55:152
732199391,30cyclictest16393-21ssh22:35:132
730199395,26cyclictest32766-21ssh21:50:130
730199393,33cyclictest21643-21ssh23:50:140
730199392,34cyclictest27027-21ssh23:35:140
7333993823,10cyclictest35966irq/58-eth000:15:123
733399373,26cyclictest35966irq/58-eth021:20:153
732199373,21cyclictest11422-21ssh21:20:152
7312993726,8cyclictest29128-21apt-get20:40:131
730199374,28cyclictest23525-21ssh23:15:140
730199373,7cyclictest131rcu_preempt22:25:130
730199372,32cyclictest21770-21ssh22:20:120
732199361,24cyclictest29269-21ssh21:30:132
730199363,30cyclictest28838-21ssh23:00:130
731299353,26cyclictest19304-21ssh00:25:141
731299351,27cyclictest12533-21ssh23:45:141
7301993518,13cyclictest12-21ksoftirqd/020:45:140
732199349,15cyclictest27-21ksoftirqd/200:15:132
732199347,15cyclictest261rcuc/222:20:142
732199343,25cyclictest0-21swapper/223:48:052
7321993418,10cyclictest5463-21sh23:05:142
732199341,29cyclictest21657-21sh23:50:152
730199345,26cyclictest14498-21ssh23:10:120
730199341,29cyclictest18383-21ssh22:00:140
732199331,4cyclictest131rcu_preempt22:15:132
731299335,13cyclictest0-21swapper/121:50:111
7312993325,3cyclictest10683-21apt-get22:50:121
7333993214,10cyclictest0-21swapper/322:20:133
732199323,12cyclictest9312-21ssh21:55:102
7312993226,3cyclictest20330-21apt-get21:25:131
7333993113,11cyclictest35966irq/58-eth023:35:143
732199314,16cyclictest261rcuc/221:50:122
731299315,17cyclictest29259-21ssh21:30:121
7312993120,8cyclictest3825-21apt-get19:40:121
7301993121,6cyclictest25363-21diskmemload00:00:120
733399306,18cyclictest35966irq/58-eth023:05:123
7333993022,5cyclictest35966irq/58-eth021:30:113
732199304,19cyclictest27346irq/40-dwc2_hso20:15:122
732199302,21cyclictest0-21swapper/219:10:122
732199302,19cyclictest3845-21ssh22:10:132
731299302,22cyclictest25406-21rm23:33:321
731299302,22cyclictest15648-21sh23:27:331
7312993018,7cyclictest0-21swapper/121:40:141
7312993016,10cyclictest1339-21ssh00:15:121
733399293,18cyclictest35966irq/58-eth022:10:123
732199299,10cyclictest7293-21cyclictest23:10:122
732199293,16cyclictest27346irq/40-dwc2_hso20:30:112
731299294,20cyclictest19730-21ssh22:55:111
7312992924,2cyclictest21760-21apt-get22:20:141
730199291,9cyclictest18084-21ssh23:30:130
7333992814,9cyclictest35966irq/58-eth022:45:143
732199285,14cyclictest27346irq/40-dwc2_hso21:00:132
732199285,14cyclictest27346irq/40-dwc2_hso21:00:122
732199283,20cyclictest0-21swapper/223:28:332
732199283,17cyclictest27346irq/40-dwc2_hso20:10:132
732199282,11cyclictest0-21swapper/219:50:142
732199282,10cyclictest10557-21ssh00:36:322
7321992811,14cyclictest26409-21cron19:30:022
731299282,21cyclictest996-21ssh22:44:331
731299282,20cyclictest3746-21ssh23:03:331
731299282,11cyclictest7448-21ssh23:23:331
731299282,11cyclictest7448-21ssh23:23:331
7312992812,12cyclictest5438-21apt-get23:05:131
730199282,20cyclictest19547-21ssh23:48:060
7301992811,5cyclictest14845-21ssh22:33:370
732199272,20cyclictest8440-21ssh23:40:332
7312992720,4cyclictest25838-21apt-get21:10:141
730199274,14cyclictest22937-21kworker/u8:100:10:130
730199274,11cyclictest5833-21ssh21:35:120
730199272,19cyclictest26738-21sh22:58:050
730199271,21cyclictest27337-21ssh22:05:120
733399263,16cyclictest0-21swapper/323:48:063
733399262,17cyclictest0-21swapper/323:34:333
732199263,20cyclictest18387-21ssh22:00:142
732199263,15cyclictest27346irq/40-dwc2_hso19:20:122
7321992616,6cyclictest27-21ksoftirqd/222:59:002
7321992614,8cyclictest365-21if_eth123:20:182
7321992614,8cyclictest365-21if_eth123:20:182
731299262,18cyclictest21527-21ssh23:13:061
7312992620,3cyclictest30717-21apt-get22:25:131
7312992614,7cyclictest11400-21apt-get21:20:151
7312992613,9cyclictest0-21swapper/121:55:111
7312992613,9cyclictest0-21swapper/100:30:111
7312992611,11cyclictest21-21ksoftirqd/122:12:341
730199262,19cyclictest17292-21ssh23:29:340
730199262,18cyclictest17227-21ssh22:52:330
7301992613,10cyclictest21073-21ssh22:35:310
7301992612,10cyclictest630-21Async09:36:310
732199252,18cyclictest27346irq/40-dwc2_hso22:45:142
732199251,21cyclictest25414-21cut22:40:152
732199251,20cyclictest10700-21ssh22:50:112
731299254,5cyclictest21-21ksoftirqd/123:40:171
731299252,18cyclictest0-21swapper/120:55:121
730199255,15cyclictest12-21ksoftirqd/019:55:110
732199246,9cyclictest0-21swapper/219:55:112
732199244,14cyclictest13167-21missed_timers20:20:222
732199243,15cyclictest25862-21ssh21:10:142
732199240,13cyclictest6100-21latency_hist20:50:022
731299248,13cyclictest0-21swapper/100:05:131
731299243,3cyclictest21-21ksoftirqd/123:56:081
7312992418,3cyclictest1776-21apt-get22:45:121
731299241,17cyclictest0-21swapper/100:20:121
731299241,15cyclictest24846-21rm00:10:141
730199244,9cyclictest30961-1kworker/u9:221:20:210
730199242,15cyclictest14943-21ssh21:40:150
7301992420,2cyclictest0-21swapper/019:45:120
733399239,9cyclictest0-21swapper/322:15:143
733399233,13cyclictest0-21swapper/322:52:343
732199238,5cyclictest0-21swapper/200:00:132
732199233,14cyclictest27346irq/40-dwc2_hso23:15:152
732199231,19cyclictest27683-21irqrtprio23:35:182
731299237,8cyclictest0-21swapper/122:08:241
731299237,4cyclictest0-21swapper/120:15:141
731299236,11cyclictest0-21swapper/119:25:131
731299234,4cyclictest21-21ksoftirqd/123:38:211
731299234,4cyclictest21-21ksoftirqd/121:46:391
7312992316,4cyclictest19501-21apt-get20:30:121
7312992316,4cyclictest1491-21apt-get20:45:141
731299231,16cyclictest7399-21ssh22:30:141
730199237,6cyclictest131rcu_preempt20:10:120
7301992316,4cyclictest7293-21cyclictest19:35:120
7301992315,6cyclictest19375-21ssh00:05:240
7301992311,9cyclictest28645-21/usr/sbin/munin00:30:170
730199231,15cyclictest844-21munin-node21:15:170
733399227,9cyclictest35966irq/58-eth022:05:123
733399223,13cyclictest0-21swapper/300:27:343
733399223,11cyclictest0-21swapper/321:35:123
7333992211,8cyclictest35966irq/58-eth022:30:143
733399221,14cyclictest0-21swapper/322:58:053
732199225,9cyclictest0-21swapper/219:15:122
732199223,15cyclictest27346irq/40-dwc2_hso00:20:122
732199222,15cyclictest15906-21ssh00:05:132
732199221,11cyclictest11233-21cpuspeed20:55:152
732199221,10cyclictest24483-21grep21:25:312
7321992210,5cyclictest30747-21ssh00:11:472
7321992210,5cyclictest1087-21ssh22:07:082
731299229,10cyclictest6144-21cut00:35:201
731299226,4cyclictest0-21swapper/122:36:181
731299225,13cyclictest0-21swapper/119:10:111
731299222,5cyclictest21-21ksoftirqd/123:53:331
7312992215,4cyclictest26645-21apt-get19:30:121
7312992213,6cyclictest0-21swapper/119:50:231
7312992212,7cyclictest12777-21apt-get22:15:121
733399215,13cyclictest35966irq/58-eth021:50:123
733399213,13cyclictest0-21swapper/323:40:133
7333992110,7cyclictest35966irq/58-eth021:10:133
732199213,6cyclictest14836-21ssh21:40:132
732199212,15cyclictest0-21swapper/221:35:122
732199211,7cyclictest7388-21ssh22:30:122
732199211,11cyclictest27654-21cat20:35:242
732199210,18cyclictest14700-21find20:25:132
731299216,3cyclictest0-21swapper/121:15:331
731299213,6cyclictest21-21ksoftirqd/122:02:011
731299212,4cyclictest21-21ksoftirqd/100:01:391
7312992114,4cyclictest27813-21apt-get20:05:141
731299211,13cyclictest24259-21grep23:15:191
731299211,11cyclictest12221-21find19:15:111
730199211,17cyclictest24879-21ps20:00:200
733399209,6cyclictest0-21swapper/323:58:543
733399206,9cyclictest35966irq/58-eth022:40:133
733399205,9cyclictest35966irq/58-eth021:40:123
733399203,12cyclictest0-21swapper/323:27:333
7333992012,3cyclictest0-21swapper/300:10:023
732199202,7cyclictest0-21swapper/221:05:122
732199202,15cyclictest1548-21kthreadcore19:35:212
732199202,10cyclictest29950-21kthreadcore20:05:202
732199201,9cyclictest3946-21fschecks_count19:40:162
7321992012,4cyclictest27-21ksoftirqd/220:50:032
732199201,16cyclictest2369-21ssh21:15:162
732199201,16cyclictest1436-21latency_hist20:45:012
731299201,16cyclictest8633-21find19:45:121
731299201,16cyclictest6403-21find20:50:111
730199208,3cyclictest131rcu_preempt23:55:170
730199205,11cyclictest3666-21ssh23:20:240
730199205,11cyclictest3666-21ssh23:20:230
7301992012,5cyclictest0-21swapper/020:25:010
7301992011,7cyclictest28871-21cron21:30:010
730199201,14cyclictest630-21Async09:36:310
733399193,11cyclictest35966irq/58-eth019:10:123
733399193,11cyclictest0-21swapper/323:23:063
733399193,11cyclictest0-21swapper/323:23:053
733399193,11cyclictest0-21swapper/323:00:333
733399193,10cyclictest0-21swapper/300:00:123
733399191,13cyclictest0-21swapper/320:40:163
733399191,13cyclictest0-21swapper/300:35:233
7333991910,6cyclictest35966irq/58-eth021:25:153
7333991910,6cyclictest0-21swapper/322:00:113
732199191,9cyclictest8816-21cut19:45:162
732199191,9cyclictest27757-21kthreadcore19:30:192
731299195,4cyclictest131rcu_preempt20:40:001
731299194,12cyclictest24490-21timerandwakeup21:05:281
731299193,4cyclictest21-21ksoftirqd/121:36:111
731299191,8cyclictest15736-21kthreadcore20:25:191
731299191,7cyclictest27586-21munin-run20:05:001
7312991912,5cyclictest0-21swapper/119:20:281
730199199,7cyclictest16096-21df_inode21:00:150
730199199,7cyclictest16096-21df_inode21:00:140
730199197,9cyclictest7161-21cut00:35:200
730199191,9cyclictest8100-21uname19:10:180
730199191,9cyclictest27629-21kthreadcore21:10:200
730199191,8cyclictest10026-21kernelversion21:55:170
730199191,15cyclictest23833-21apt-get21:45:140
7301991911,5cyclictest1680-21if_eth100:15:180
733399188,7cyclictest35966irq/58-eth021:15:163
733399183,9cyclictest0-21swapper/322:26:333
733399183,10cyclictest35966irq/58-eth000:10:143
733399182,11cyclictest0-21swapper/300:21:333
733399182,10cyclictest0-21swapper/320:05:143
7333991811,5cyclictest0-21swapper/323:15:133
733399181,13cyclictest0-21swapper/319:30:023
7333991810,6cyclictest0-21swapper/323:52:593
731299188,7cyclictest13293-21ntpq20:20:251
731299186,9cyclictest2032-21cut19:35:211
731299185,10cyclictest18307-21df19:55:141
731299181,15cyclictest18193-21kthreadcore21:00:211
731299181,15cyclictest18193-21kthreadcore21:00:211
730199189,7cyclictest10329-21apt-get00:20:120
730199189,6cyclictest8406-21cron19:45:010
730199189,5cyclictest20044-21kthreadcore00:25:200
730199181,6cyclictest25363-21diskmemload22:43:330
7301991813,2cyclictest12-21ksoftirqd/020:05:270
7301991811,5cyclictest15757-21cron21:00:010
733399179,6cyclictest0-21swapper/321:55:043
733399179,6cyclictest0-21swapper/321:45:593
733399176,7cyclictest0-21swapper/300:30:143
733399172,10cyclictest35966irq/58-eth020:10:113
733399172,10cyclictest0-21swapper/320:25:153
733399172,10cyclictest0-21swapper/319:40:143
731299174,10cyclictest568-21cut20:10:171
730199175,9cyclictest27230-21kthreadcore20:35:220
730199172,10cyclictest0-21swapper/020:30:110
7301991712,3cyclictest10067-21ls20:50:270
730199171,12cyclictest25369-21pluginstate21:10:010
733399162,11cyclictest0-21swapper/319:20:133
733399162,10cyclictest0-21swapper/321:05:143
733399162,10cyclictest0-21swapper/320:35:143
733399161,9cyclictest0-21swapper/319:35:153
730199168,5cyclictest13531-21fschecks_count19:50:160
730199163,10cyclictest14101-21kthreadcore19:15:190
730199162,9cyclictest0-21swapper/020:40:130
730199161,11cyclictest22935-21kthreadcore19:25:180
730199161,10cyclictest5172-21fschecks_time20:15:160
733399152,9cyclictest0-21swapper/319:55:143
733399152,10cyclictest0-21swapper/320:55:143
733399152,10cyclictest0-21swapper/320:00:133
733399152,10cyclictest0-21swapper/319:50:143
733399152,10cyclictest0-21swapper/319:15:113
733399151,9cyclictest0-21swapper/320:25:013
733399151,7cyclictest0-21swapper/320:35:023
731299151,10cyclictest7342-21latency_hist19:10:011
730199151,11cyclictest17880-21cut19:20:180
730199151,10cyclictest15176-21irqcore20:25:190
732199141,9cyclictest7341-21latency_hist19:10:022
730199141,10cyclictest7293-21cyclictest19:10:020
733399132,8cyclictest0-21swapper/320:15:183
733399131,8cyclictest0-21swapper/319:50:003
733399131,5cyclictest0-21swapper/321:00:223
733399131,5cyclictest0-21swapper/321:00:213
733399127,3cyclictest0-21swapper/319:34:063
733399122,6cyclictest0-21swapper/320:50:133
733399121,7cyclictest0-21swapper/320:45:143
733399111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional