You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 11:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Aug 11, 2026 00:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21076995924,8cyclictest131rcu_preempt23:25:132
21084995338,11cyclictest35966irq/58-eth021:40:133
21076994839,6cyclictest1638-21ssh21:25:122
2106199423,28cyclictest14085-21ssh21:50:121
2107699412,36cyclictest0-21swapper/221:35:132
2106199412,21cyclictest10669-21ssh21:30:151
2105299403,10cyclictest131rcu_preempt23:35:120
2106199391,29cyclictest5175-21ssh21:45:131
21052993818,3cyclictest131rcu_preempt20:25:110
21084993714,17cyclictest321rcuc/323:50:133
2105299372,20cyclictest11850-21ssh23:55:110
2106199361,27cyclictest19560-21ssh23:05:131
2106199361,27cyclictest19560-21ssh23:05:131
2106199360,18cyclictest19691-21ssh23:59:061
2108499353,25cyclictest35966irq/58-eth000:20:173
2108499346,19cyclictest33-21ksoftirqd/323:40:123
2108499345,17cyclictest35966irq/58-eth022:55:133
2108499342,18cyclictest35966irq/58-eth000:05:173
2107699342,12cyclictest131rcu_preempt19:55:122
2108499332,20cyclictest35966irq/58-eth019:40:143
2108499327,15cyclictest0-21swapper/323:20:153
2108499326,21cyclictest0-21swapper/322:20:143
2106199320,23cyclictest0-21swapper/122:30:121
2108499316,16cyclictest35966irq/58-eth000:35:133
2108499314,17cyclictest35966irq/58-eth022:15:133
2108499311,17cyclictest35966irq/58-eth021:25:133
21076993118,8cyclictest27-21ksoftirqd/220:10:122
21076993118,11cyclictest0-21swapper/223:35:132
21052993116,9cyclictest12-21ksoftirqd/022:40:140
2108499302,20cyclictest35966irq/58-eth022:45:153
2108499301,19cyclictest0-21swapper/321:45:143
2107699306,13cyclictest261rcuc/223:50:122
21076993022,6cyclictest27-21ksoftirqd/223:30:512
2105299301,19cyclictest10686-21rm21:30:160
2108499293,17cyclictest0-21swapper/322:12:333
2107699292,20cyclictest5190-21ssh21:45:152
2107699292,18cyclictest27346irq/40-dwc2_hso20:45:142
2107699292,18cyclictest21391-21ssh22:30:132
2107699291,19cyclictest27346irq/40-dwc2_hso20:20:132
2107699291,19cyclictest27346irq/40-dwc2_hso20:20:132
21061992915,8cyclictest28579-21rm21:40:141
2105299292,19cyclictest6497-21diskstats00:10:160
2108499283,16cyclictest0-21swapper/300:02:493
2108499282,16cyclictest35966irq/58-eth020:55:113
2107699289,12cyclictest27-21ksoftirqd/221:40:132
2107699283,19cyclictest27346irq/40-dwc2_hso21:15:162
2107699282,16cyclictest27346irq/40-dwc2_hso21:00:122
2107699282,14cyclictest19580-21ssh23:05:152
2107699282,14cyclictest19580-21ssh23:05:142
2106199281,19cyclictest3487-21ssh22:20:141
2106199281,18cyclictest15382-21ssh00:15:151
2108499276,15cyclictest0-21swapper/300:10:133
2108499274,16cyclictest0-21swapper/321:35:143
21084992718,6cyclictest35966irq/58-eth000:15:153
2107699272,16cyclictest27346irq/40-dwc2_hso23:10:122
2107699272,11cyclictest0-21swapper/220:25:122
21076992719,5cyclictest27-21ksoftirqd/222:20:142
2105299272,19cyclictest25442-21ssh00:37:330
2107699263,17cyclictest27346irq/40-dwc2_hso20:15:142
2107699262,20cyclictest0-21swapper/220:30:122
21076992616,7cyclictest1426-21ssh22:55:132
21076992614,4cyclictest17854-21/usr/sbin/munin22:10:222
2107699261,11cyclictest930-21snmpd21:10:142
2106199262,18cyclictest29078-21cat00:20:341
2108499251,15cyclictest0-21swapper/322:00:153
2107699257,15cyclictest3091-21ssh22:20:022
21076992513,9cyclictest0-21swapper/222:40:192
2107699251,17cyclictest12435-21ssh22:25:132
2106199253,3cyclictest131rcu_preempt23:10:131
21052992513,3cyclictest131rcu_preempt22:25:250
2108499244,12cyclictest0-21swapper/300:30:143
2108499242,15cyclictest33-21ksoftirqd/323:35:123
2108499242,12cyclictest35966irq/58-eth021:10:133
21084992412,8cyclictest35966irq/58-eth023:55:123
21084992410,6cyclictest0-21swapper/323:30:243
2107699242,15cyclictest27346irq/40-dwc2_hso23:15:152
21076992417,4cyclictest3489-21ssh00:25:222
21076992417,4cyclictest3489-21ssh00:25:212
2107699241,20cyclictest31804-21kthreadcore19:20:202
2106199241,4cyclictest0-21swapper/123:30:141
21061992413,2cyclictest0-21swapper/123:42:011
2106199241,17cyclictest27552-21ssh00:02:491
2105299246,3cyclictest131rcu_preempt22:07:310
2105299243,18cyclictest7184-21ssh21:10:130
21052992420,2cyclictest14027-21ssh23:20:140
2108499234,14cyclictest0-21swapper/323:05:153
2108499234,14cyclictest0-21swapper/323:05:143
2108499231,12cyclictest35966irq/58-eth020:45:133
2107699236,9cyclictest31013-21cut22:35:172
2107699236,14cyclictest15985-21sh22:45:162
2107699234,9cyclictest24950-21ssh22:50:152
2107699234,5cyclictest17479-21ssh23:40:142
2107699232,15cyclictest27346irq/40-dwc2_hso20:50:132
2107699232,12cyclictest26395-21ssh23:45:152
21076992313,7cyclictest0-21swapper/200:00:162
21076992311,9cyclictest0-21swapper/220:55:222
21076992311,5cyclictest13620-21ssh00:13:232
2106199234,15cyclictest10534-21ssh23:00:131
2106199234,15cyclictest10534-21ssh23:00:131
2105299231,7cyclictest5188-21proc_pri22:55:270
21084992212,6cyclictest35966irq/58-eth022:30:123
2107699222,6cyclictest0-21swapper/221:31:462
2107699222,2cyclictest0-21swapper/200:36:142
2107699222,15cyclictest0-21swapper/221:54:462
2107699222,14cyclictest0-21swapper/219:15:152
2107699222,13cyclictest0-21swapper/221:05:142
2107699222,13cyclictest0-21swapper/221:05:142
2107699221,5cyclictest14037-21ssh23:20:142
21076992212,8cyclictest261rcuc/200:20:172
2107699221,12cyclictest7061-21grep19:25:292
2107699221,12cyclictest31025-21grep19:50:302
2107699221,12cyclictest19543-21unixbench_singl00:15:322
2107699221,12cyclictest10318-21cron20:40:012
2107699220,5cyclictest9888-21find00:30:122
2106199221,16cyclictest2869-21ssh23:50:121
2105299227,11cyclictest32332-21ssh22:00:140
2105299222,3cyclictest131rcu_preempt22:35:120
21052992215,5cyclictest12-21ksoftirqd/021:45:140
2105299221,4cyclictest0-21swapper/023:16:380
2108499213,13cyclictest0-21swapper/321:20:163
21084992114,4cyclictest0-21swapper/321:55:123
2108499211,15cyclictest0-21swapper/320:35:023
2107699216,11cyclictest0-21swapper/219:45:122
2107699212,16cyclictest930-21snmpd21:20:162
2107699211,11cyclictest23129-21/usr/sbin/munin21:55:142
2107699211,10cyclictest8649-21latency_hist22:05:022
2107699211,10cyclictest13025-21wc19:35:142
2107699211,10cyclictest12113-21hwlatdetect23:55:152
21076992110,9cyclictest11246-21ssh23:00:192
21076992110,9cyclictest11246-21ssh23:00:182
2106199217,4cyclictest131rcu_preempt21:55:271
2106199211,15cyclictest12451-21ssh22:25:141
2105299213,15cyclictest25997-21apt-get19:15:140
21052992115,3cyclictest0-21swapper/021:25:120
2108499207,7cyclictest321rcuc/320:25:113
2108499204,8cyclictest0-21swapper/322:25:133
21084992012,6cyclictest35966irq/58-eth023:15:173
2108499201,14cyclictest321rcuc/322:05:123
21084992010,6cyclictest0-21swapper/321:50:223
2107699205,10cyclictest261rcuc/200:05:172
2107699202,9cyclictest24514-21missed_timers19:10:542
2107699202,15cyclictest9536-21irqcore20:05:182
2107699201,16cyclictest8756-21kthreadcore19:30:202
2107699201,16cyclictest17714-21fschecks_count19:40:162
2107699201,10cyclictest9063-21fschecks_count22:05:172
2107699201,10cyclictest11427-21kthreadcore20:40:202
2106199202,14cyclictest6941-21apt-get22:40:131
2106199201,7cyclictest4718-21processes00:25:261
2106199201,7cyclictest4718-21processes00:25:261
21061992014,4cyclictest0-21swapper/119:25:281
2106199201,3cyclictest131rcu_preempt21:15:151
2106199201,16cyclictest26179-21cut19:15:181
2106199201,13cyclictest30009-21ssh23:45:271
2106199200,8cyclictest0-21swapper/120:15:251
2105299209,8cyclictest29406-21cut22:15:210
2105299208,9cyclictest31958-21apt-get23:30:130
2105299204,2cyclictest131rcu_preempt22:46:470
2105299202,13cyclictest9128-21ssh00:29:330
2105299202,13cyclictest9128-21ssh00:29:330
2105299201,7cyclictest131rcu_preempt21:40:180
21052992013,4cyclictest15086-21usage_EthcTask23:00:310
21052992013,4cyclictest15086-21usage_EthcTask23:00:310
21052992012,5cyclictest29551-21sh21:00:020
2108499198,7cyclictest0-21swapper/323:25:143
2108499198,4cyclictest33-21ksoftirqd/323:14:403
2108499193,10cyclictest36066irq/59-eth022:50:153
2108499192,10cyclictest0-21swapper/321:30:163
2108499192,10cyclictest0-21swapper/320:00:153
2108499192,10cyclictest0-21swapper/320:00:143
2108499191,15cyclictest35966irq/58-eth022:35:113
21084991911,4cyclictest0-21swapper/300:26:033
21084991911,4cyclictest0-21swapper/300:26:023
2108499191,13cyclictest0-21swapper/323:45:273
2107699192,11cyclictest0-21swapper/220:00:142
2107699192,11cyclictest0-21swapper/220:00:132
2106199199,7cyclictest32513-21ssh22:00:161
2106199198,9cyclictest19218-21tr22:45:251
2106199198,8cyclictest8150-21fschecks_count19:30:171
2106199192,7cyclictest0-21swapper/122:15:181
2106199191,9cyclictest7980-21kthreadcore00:10:201
2106199191,6cyclictest18724-21missed_timers20:45:241
2106199191,3cyclictest0-21swapper/120:55:111
21061991913,4cyclictest0-21swapper/119:45:231
21061991912,5cyclictest22970-21/usr/sbin/munin23:25:261
2106199191,16cyclictest28312-21ls20:25:011
2106199191,16cyclictest28312-21ls20:25:011
2106199191,15cyclictest30538-21uniq22:35:141
2106199191,15cyclictest1621-21apt-get21:25:111
2106199191,15cyclictest11184-21kthreadcore20:05:211
2106199191,14cyclictest25158-21ssh21:20:161
2106199191,12cyclictest28840-21tr20:25:151
2106199191,11cyclictest31752-21awk19:50:311
2106199191,10cyclictest15167-21ssh21:14:201
2106199190,9cyclictest0-21swapper/121:40:011
2106199190,16cyclictest20197-21find20:50:121
2105299199,7cyclictest20299-21uptime21:15:320
2105299193,4cyclictest17606-21apt-get19:40:140
2105299192,4cyclictest12-21ksoftirqd/022:20:260
2105299192,14cyclictest17889-21ssh22:10:130
2105299192,12cyclictest27934-21ssh21:39:460
2105299191,9cyclictest21372-21fschecks_time19:10:160
2105299191,4cyclictest0-21swapper/019:55:120
2105299191,10cyclictest25605-21tr19:45:210
2105299191,10cyclictest0-21swapper/021:10:010
2105299191,10cyclictest0-21swapper/021:10:010
2105299190,16cyclictest20001-21ls20:50:020
2105299190,14cyclictest31882-21ls19:55:010
2108499188,6cyclictest0-21swapper/322:41:313
2108499182,10cyclictest0-21swapper/320:20:143
2108499182,10cyclictest0-21swapper/320:20:143
2108499182,10cyclictest0-21swapper/319:20:153
21084991811,5cyclictest35966irq/58-eth023:00:133
21084991811,5cyclictest35966irq/58-eth023:00:123
2108499181,12cyclictest0-21swapper/319:50:023
2108499181,11cyclictest0-21swapper/319:30:233
2107699181,9cyclictest21109-21wc19:10:012
2106199188,8cyclictest12109-21cut00:30:211
2106199186,2cyclictest131rcu_preempt20:30:141
2106199181,8cyclictest0-21swapper/120:00:261
2106199181,8cyclictest0-21swapper/120:00:261
2106199181,6cyclictest0-21swapper/100:07:221
2106199181,4cyclictest131rcu_preempt22:50:181
2106199181,14cyclictest18944-21cpuspeed00:35:151
2106199181,13cyclictest8944-21mailstats23:15:311
2106199181,13cyclictest6773-21diskmemload22:05:141
2106199180,3cyclictest131rcu_preempt19:20:201
2105299189,6cyclictest28919-21sshd22:33:470
2105299188,7cyclictest19337-21find00:15:310
2105299188,3cyclictest131rcu_preempt20:15:020
2105299185,10cyclictest30257-21tr00:05:220
2105299183,3cyclictest131rcu_preempt20:55:010
2105299182,10cyclictest6773-21diskmemload21:55:120
2105299181,9cyclictest20127-21irqcore23:05:190
2105299181,9cyclictest20127-21irqcore23:05:190
21052991811,5cyclictest6847-21timerandwakeup23:50:290
21052991811,4cyclictest25794-21kthreadcore00:20:230
2105299181,14cyclictest14104-21ssh21:50:130
2105299181,13cyclictest3193-21apt-get19:25:110
2105299181,13cyclictest17454-21apt-get23:40:120
21052991810,6cyclictest5466-21pluginstate20:35:010
2108499172,10cyclictest0-21swapper/321:00:133
2108499172,10cyclictest0-21swapper/319:35:123
2108499171,11cyclictest0-21swapper/321:10:013
2108499171,11cyclictest0-21swapper/321:10:013
2108499171,10cyclictest0-21swapper/320:15:153
2106199179,6cyclictest10441-21wc20:40:011
2106199172,6cyclictest12440-21kthreadcore20:40:221
2106199171,9cyclictest22495-21usage_EthcTask22:10:291
2106199171,7cyclictest0-21swapper/123:00:021
2106199171,7cyclictest0-21swapper/121:05:201
2106199171,7cyclictest0-21swapper/121:05:201
2106199171,6cyclictest14188-21fschecks_time20:10:161
21061991711,4cyclictest0-21swapper/119:40:231
2106199171,12cyclictest12173-21ntp_states23:35:251
2106199171,10cyclictest0-21swapper/123:22:531
2105299179,6cyclictest614-21dbus-daemon20:07:160
2105299178,6cyclictest22310-21sendmail_mailqu20:15:290
2105299174,2cyclictest131rcu_preempt20:00:260
2105299174,2cyclictest131rcu_preempt20:00:260
2105299174,11cyclictest12-21ksoftirqd/022:50:160
2105299173,11cyclictest20894-21diskmemload00:00:170
2105299172,2cyclictest131rcu_preempt19:30:140
2105299171,13cyclictest30802-21apt-get19:20:140
2105299171,13cyclictest28557-21rm23:10:130
21052991710,5cyclictest23586-21cut23:25:180
21052991710,4cyclictest25886-21kthreadcore21:20:220
2108499169,5cyclictest0-21swapper/321:15:173
2108499162,9cyclictest0-21swapper/320:35:143
2106199164,4cyclictest21-21ksoftirqd/119:35:111
2106199162,9cyclictest0-21swapper/119:10:141
2106199162,3cyclictest32009-21apt-get19:55:121
2105299166,7cyclictest29883-21memory23:45:250
21052991611,3cyclictest17355-21sh19:40:010
2105299160,5cyclictest10665-21tail00:30:190
2108499152,9cyclictest0-21swapper/319:15:153
2106199150,13cyclictest29803-21apt-get21:00:121
2105299152,10cyclictest11237-21kernelversion20:40:190
2105299151,11cyclictest5772-21apt-get20:35:150
2105299151,11cyclictest30307-21sed21:00:170
2108499141,9cyclictest0-21swapper/320:10:143
2105299149,3cyclictest23779-21entropy20:20:160
2105299149,3cyclictest23779-21entropy20:20:150
2105299141,9cyclictest21107-21latency_hist19:10:010
2108499138,3cyclictest0-21swapper/320:43:123
2108499133,8cyclictest0-21swapper/319:50:223
2108499133,7cyclictest0-21swapper/319:10:133
2108499131,10cyclictest0-21swapper/320:50:283
2106199130,10cyclictest21068-21latency_hist19:10:011
2108499122,6cyclictest0-21swapper/320:05:143
2108499122,6cyclictest0-21swapper/319:55:133
2108499121,8cyclictest0-21swapper/319:25:133
2108499111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional