You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 17:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jun 13, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1784299592,5cyclictest131rcu_preempt11:20:111
17858995534,15cyclictest35966irq/58-eth010:20:153
1784299491,37cyclictest25891-21ssh09:40:121
1783099464,32cyclictest7760-21ssh09:30:130
1785899451,39cyclictest0-21swapper/310:15:123
1784299451,37cyclictest28184-21ssh10:35:131
1783099452,36cyclictest4804-21ssh12:10:120
1783099452,36cyclictest4804-21ssh12:10:120
17830994422,9cyclictest131rcu_preempt07:35:130
1785899431,39cyclictest0-21swapper/310:50:133
1783099433,36cyclictest4890-21ssh10:40:130
1783099433,33cyclictest3534-21diskmemload10:45:120
1785099414,13cyclictest0-21swapper/210:05:152
1784299411,22cyclictest6359-21ssh10:05:141
1783099417,10cyclictest131rcu_preempt11:40:150
17858994017,14cyclictest35966irq/58-eth012:25:123
1783099405,26cyclictest5952-21sh10:20:320
1783099403,30cyclictest23021-21ssh10:50:130
17858993925,7cyclictest35966irq/58-eth009:30:133
1785899382,26cyclictest35966irq/58-eth011:45:133
1785899381,2cyclictest0-21swapper/311:55:123
1785899381,2cyclictest0-21swapper/308:25:123
1785899381,27cyclictest35966irq/58-eth010:55:123
1785099382,20cyclictest0-21swapper/210:55:552
1784299381,30cyclictest13934-21ssh10:45:111
1783099382,24cyclictest2497-21ssh09:45:130
1785899374,21cyclictest35966irq/58-eth009:10:133
1785899363,23cyclictest35966irq/58-eth007:20:133
1783099364,23cyclictest22924-21ssh12:20:120
1785899354,18cyclictest35966irq/58-eth010:30:123
1784299351,26cyclictest10118-21ssh10:25:141
1785899341,28cyclictest0-21swapper/309:15:123
17830993421,8cyclictest12-21ksoftirqd/010:10:120
1785899332,27cyclictest0-21swapper/307:25:123
1784299332,11cyclictest31218-21ssh10:54:331
1785899321,27cyclictest0-21swapper/311:20:123
1785899320,2cyclictest0-21swapper/312:05:123
1783099328,14cyclictest12-21ksoftirqd/011:10:140
1785899311,1cyclictest0-21swapper/309:40:123
17850993117,7cyclictest27-21ksoftirqd/208:00:152
1784299311,18cyclictest31141-21ssh09:25:141
1784299311,12cyclictest20405-21ssh10:10:451
1783099312,23cyclictest17462-21ssh12:35:130
1783099312,23cyclictest15073-21ssh11:39:320
1783099312,21cyclictest21310-21ssh11:25:140
1785899303,16cyclictest35966irq/58-eth012:30:143
1785899301,2cyclictest0-21swapper/312:11:503
1785899301,2cyclictest0-21swapper/312:11:493
1785899301,25cyclictest0-21swapper/308:40:163
1785899301,24cyclictest0-21swapper/307:50:143
1785899301,24cyclictest0-21swapper/307:50:133
1785899301,23cyclictest0-21swapper/309:55:123
1785099302,23cyclictest10639-21ssh10:41:312
1785099302,22cyclictest30377-21ssh10:53:322
1783099302,22cyclictest32236-21ssh11:48:320
1783099302,20cyclictest30167-21ssh10:16:340
1783099302,18cyclictest31856-21ssh12:25:120
1785899292,23cyclictest0-21swapper/307:55:123
1785899292,22cyclictest0-21swapper/311:40:143
1785899291,2cyclictest0-21swapper/312:35:123
1785899291,24cyclictest0-21swapper/311:30:143
1785899291,23cyclictest0-21swapper/307:35:133
1785899291,22cyclictest0-21swapper/310:35:143
1785899291,21cyclictest0-21swapper/308:35:133
1785899291,20cyclictest35966irq/58-eth012:20:133
1785899291,1cyclictest0-21swapper/311:50:113
1785099295,15cyclictest27346irq/40-dwc2_hso10:15:132
17850992915,11cyclictest0-21swapper/209:30:122
1783099292,22cyclictest10648-21sh11:18:310
1783099291,21cyclictest13942-21ssh12:15:140
1785899281,2cyclictest0-21swapper/310:25:133
1785899281,2cyclictest0-21swapper/309:20:133
1785899281,23cyclictest0-21swapper/311:00:123
1785899281,23cyclictest0-21swapper/307:15:133
1785899281,15cyclictest35966irq/58-eth008:20:153
1785099283,20cyclictest0-21swapper/210:36:342
1785099283,18cyclictest27346irq/40-dwc2_hso12:00:132
1784299283,13cyclictest21-21ksoftirqd/108:55:131
17842992813,8cyclictest12907-21ssh09:15:121
1783099289,10cyclictest131rcu_preempt07:15:130
1783099284,7cyclictest131rcu_preempt10:30:340
1783099281,19cyclictest10137-21ssh10:25:150
17830992810,12cyclictest630-21Async09:36:310
1785899276,18cyclictest35966irq/58-eth011:10:143
1785899273,17cyclictest0-21swapper/309:50:413
1785899271,2cyclictest0-21swapper/310:40:143
1785899271,22cyclictest0-21swapper/312:00:113
1785099274,15cyclictest27346irq/40-dwc2_hso07:55:122
1785099273,18cyclictest27346irq/40-dwc2_hso08:40:142
17850992717,7cyclictest930-21snmpd07:15:132
1784299272,19cyclictest24898-21ssh11:45:151
17842992715,4cyclictest0-21swapper/109:20:131
1784299271,20cyclictest8423-21ssh12:30:131
1784299270,14cyclictest25942-21cat11:25:311
1783099277,9cyclictest131rcu_preempt09:05:140
1783099272,19cyclictest3534-21diskmemload11:31:010
1783099272,19cyclictest13897-21ssh11:00:550
17830992715,6cyclictest0-21swapper/012:30:130
1785899261,3cyclictest0-21swapper/311:27:553
1785899261,2cyclictest0-21swapper/311:05:143
1785899261,2cyclictest0-21swapper/309:05:143
1785899261,2cyclictest0-21swapper/307:40:233
1785899261,22cyclictest0-21swapper/308:45:123
1785899261,21cyclictest0-21swapper/309:29:573
1785899260,2cyclictest0-21swapper/311:15:123
1785099262,15cyclictest13928-21ssh12:15:142
17850992620,2cyclictest2746ksoftirqd/211:10:132
1785099261,19cyclictest0-21swapper/209:40:142
1784299262,19cyclictest11475-21ssh10:42:321
1784299261,20cyclictest14412-21ssh11:01:321
1784299261,19cyclictest29709-21ssh10:00:151
1783099261,18cyclictest28236-21ssh12:05:120
1785899251,5cyclictest0-21swapper/309:35:133
1785899251,3cyclictest0-21swapper/309:45:143
1785899251,2cyclictest0-21swapper/312:20:023
1785899251,2cyclictest0-21swapper/307:30:133
1785099253,10cyclictest8422-21ssh12:30:132
1785099251,9cyclictest15246-21ssh11:02:312
1784299259,11cyclictest21-21ksoftirqd/110:19:321
1784299250,19cyclictest4339-21ssh12:25:331
1785899241,5cyclictest0-21swapper/307:10:283
1785899241,3cyclictest0-21swapper/310:10:263
1785899241,2cyclictest0-21swapper/311:35:133
1785899241,2cyclictest0-21swapper/310:46:253
1785899241,2cyclictest0-21swapper/310:05:143
1785899241,2cyclictest0-21swapper/307:45:163
1785899241,20cyclictest0-21swapper/310:00:203
1785899240,2cyclictest0-21swapper/308:05:113
1785099242,17cyclictest27346irq/40-dwc2_hso10:25:152
17850992412,9cyclictest30340-21ssh11:46:132
1785099241,15cyclictest10322-21ssh11:55:132
1784299242,17cyclictest25905-21ssh11:09:321
1783099246,12cyclictest151rcuc/011:55:120
1783099243,11cyclictest30190-21kworker/u8:109:10:150
1783099241,18cyclictest3976-21ssh10:39:310
1785899231,2cyclictest0-21swapper/308:50:113
1785899231,2cyclictest0-21swapper/308:30:123
1785899231,2cyclictest0-21swapper/308:10:273
1785899231,2cyclictest0-21swapper/308:00:143
1785899230,2cyclictest0-21swapper/308:15:253
1785099239,11cyclictest17259-21kthreadcore11:40:202
1785099233,16cyclictest0-21swapper/210:24:322
1785099232,6cyclictest0-21swapper/211:08:332
1785099232,5cyclictest0-21swapper/207:50:142
1785099232,5cyclictest0-21swapper/207:50:132
1785099232,16cyclictest0-21swapper/211:29:322
17850992314,7cyclictest261rcuc/209:00:112
17850992312,9cyclictest15083-21ssh11:39:332
17850992311,9cyclictest3534-21diskmemload12:20:112
1785099231,13cyclictest13450-21latency_hist12:15:012
1785099231,13cyclictest13450-21latency_hist12:15:002
1784299231,15cyclictest5073-21ssh11:50:241
1783099231,17cyclictest26620-21ssh12:03:330
1783099231,16cyclictest16801-21ssh09:35:120
1785899221,18cyclictest0-21swapper/308:55:133
1785899220,2cyclictest0-21swapper/309:00:123
1785099228,11cyclictest13137-21perl09:15:162
1785099223,15cyclictest2482-21ssh09:45:122
1785099223,11cyclictest24934-21tr09:20:222
1785099222,16cyclictest0-21swapper/212:29:312
17850992211,9cyclictest4474-21ssh10:03:142
1785099221,15cyclictest27346irq/40-dwc2_hso09:25:042
1785099221,12cyclictest4934-21fschecks_count07:30:172
1784299226,10cyclictest201rcuc/112:35:131
1784299221,18cyclictest20478-21ls08:50:251
1784299221,17cyclictest23786-21ls08:20:231
1783099229,9cyclictest20252-21cron08:20:000
1783099229,10cyclictest3886-21tmpfiles10:02:320
1783099224,6cyclictest131rcu_preempt08:05:120
1783099222,15cyclictest20645-21ssh09:55:140
1783099221,4cyclictest131rcu_preempt08:10:120
1783099221,17cyclictest5033-21ls07:30:180
17830992210,8cyclictest121ksoftirqd/007:40:140
1783099220,18cyclictest30150-21ls08:30:150
1785099218,5cyclictest2428-21open_files09:05:282
1785099217,4cyclictest31181-21cat07:20:292
1785099215,13cyclictest0-21swapper/209:50:172
1785099214,9cyclictest17880-21latency_hist07:10:022
1785099212,9cyclictest22811-21latency_hist10:50:022
1785099212,9cyclictest16077-21kthreadcore10:10:192
1785099212,9cyclictest12524-21munin-plugin-st09:15:012
1785099211,6cyclictest1930866ssh12:35:202
17850992116,3cyclictest261rcuc/207:40:142
17850992112,6cyclictest2461-21ssh11:30:292
1785099211,15cyclictest10959-21diskstats08:10:142
1785099210,11cyclictest2280-21cut07:25:212
1784299212,14cyclictest32035-21ssh10:55:131
1784299211,17cyclictest15244-21ssh09:50:251
17830992111,7cyclictest20897-21cron11:25:020
1785099209,4cyclictest22467-21grep07:45:232
1785099203,9cyclictest31056-21cut12:05:222
1785099203,9cyclictest0-21swapper/208:35:142
1785099202,8cyclictest18031-21kthreadcore09:35:192
1785099202,15cyclictest18114-21fschecks_count07:10:152
1785099202,12cyclictest27346irq/40-dwc2_hso11:50:132
1785099201,9cyclictest14675-21kthreadcore08:45:232
1785099201,9cyclictest13236-21wc07:35:282
1785099201,16cyclictest23431-21kthreadcore08:55:222
1785099201,16cyclictest12821-21ssh11:20:172
1785099201,15cyclictest20641-21ssh09:55:122
1785099201,11cyclictest6509-21cut11:15:222
1785099201,10cyclictest20808-21grep08:20:192
1785099201,10cyclictest18822-21kthreadcore08:15:212
1784299202,6cyclictest26598-21ntp_states12:20:261
1784299201,7cyclictest17658-21ntpq12:15:261
1784299200,14cyclictest5951-21sh10:20:331
1783099207,3cyclictest131rcu_preempt09:40:270
1783099201,16cyclictest20480-21find08:20:130
1783099200,17cyclictest21706-21ls08:55:150
1785099199,8cyclictest0-21swapper/208:25:332
1785099191,9cyclictest6188-21fschecks_count08:05:162
1785099191,9cyclictest30226-21fschecks_count08:30:152
1785099191,9cyclictest17458-21irqrtprio08:50:162
1785099191,8cyclictest19236-21diskstats10:30:142
1784299191,8cyclictest131rcu_preempt09:33:261
17842991911,6cyclictest3497-21sshd09:05:321
1784299191,16cyclictest30427-21df_inode11:30:141
1784299191,12cyclictest10649-21sh11:18:311
1783099194,12cyclictest8161-21kthreadcore08:40:220
1783099193,3cyclictest131rcu_preempt10:55:130
1783099191,3cyclictest0-21swapper/007:20:150
1783099191,16cyclictest6206-21ls08:35:270
1783099191,15cyclictest13486-21sed09:15:170
17830991911,5cyclictest0-21swapper/007:50:160
17830991911,5cyclictest0-21swapper/007:50:150
1785899181,4cyclictest0-21swapper/307:10:013
1784299189,6cyclictest12899-21load07:35:241
1784299188,7cyclictest22974-21awk10:30:271
1784299186,9cyclictest20582-21processes09:35:281
1784299182,7cyclictest131rcu_preempt11:37:251
1784299181,5cyclictest11116-21munin-run09:50:021
1784299181,15cyclictest22796-21ls12:00:231
1784299181,14cyclictest19198-21find07:45:131
1784299181,11cyclictest21174-21ssh11:41:011
1784299180,9cyclictest29519-21kernelversion07:55:181
1783099189,6cyclictest22523-21cron07:15:010
1783099189,6cyclictest2118-21kthreadcore07:25:210
1783099188,7cyclictest20574-21tr11:05:220
1783099182,13cyclictest30650-21apt-get09:25:020
1783099181,9cyclictest26563-21kthreadcore08:25:200
1783099181,13cyclictest22431-21irqcore09:20:190
17830991810,6cyclictest21073-21tail07:45:210
17830991810,5cyclictest6322-21/usr/sbin/munin10:05:280
17830991810,5cyclictest3583-21kthreadcore11:50:200
1784299173,11cyclictest9154-21tr08:05:211
1784299172,12cyclictest30313-21ntp_192.168.11411:10:241
1784299171,7cyclictest15849-21grep08:15:161
1784299171,6cyclictest131rcu_preempt09:10:071
1784299171,2cyclictest0-21swapper/112:00:021
17842991711,4cyclictest4511-21apt-config07:30:021
1784299171,13cyclictest4485-21apt-get07:30:041
1784299171,13cyclictest26823-21kthreadcore08:25:211
1784299171,12cyclictest14988-21sed07:40:181
1784299171,11cyclictest0-21swapper/112:10:141
1784299171,11cyclictest0-21swapper/112:10:131
1784299171,10cyclictest2334-21latency_hist08:35:011
1783099179,6cyclictest29787-21sed09:00:220
1783099176,6cyclictest12-21ksoftirqd/008:50:120
1783099171,13cyclictest32110-21ls07:55:210
17830991710,5cyclictest11780-21cron08:45:010
1784299169,4cyclictest25019-21kthreadcore07:15:221
1784299168,6cyclictest32127-21cron07:25:001
1784299168,6cyclictest28909-21cut12:05:191
1784299168,6cyclictest2164-21cat08:00:211
1784299168,6cyclictest10850-21find08:10:111
1784299166,8cyclictest21170-21sed09:55:171
1784299162,9cyclictest0-21swapper/108:35:131
1784299162,10cyclictest0-21swapper/107:50:141
1784299162,10cyclictest0-21swapper/107:50:131
17842991611,3cyclictest15441-21sh08:45:251
1783099167,6cyclictest1579-21irqcore08:00:190
1784299158,2cyclictest0-21swapper/108:40:141
1784299151,11cyclictest27115-21irqstats09:00:171
1784299151,11cyclictest19676-21tr07:10:201
1784299146,6cyclictest0-21swapper/107:10:011
1783099141,9cyclictest17878-21cat07:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional