You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 03:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Aug 20, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27151996840,12cyclictest35966irq/58-eth021:20:143
2710399523,39cyclictest13409-21ssh23:15:111
2715199513,30cyclictest35966irq/58-eth021:15:133
2707699514,38cyclictest14654-21ssh22:40:100
2707699503,38cyclictest7964-21ssh23:30:100
2710399483,33cyclictest18990-21ssh00:30:151
2710399483,33cyclictest18990-21ssh00:30:151
27151994733,9cyclictest35966irq/58-eth022:50:133
2710399451,30cyclictest22454-21ssh21:15:121
27151994422,14cyclictest35966irq/58-eth023:15:123
2707699431,34cyclictest17028-21ssh23:35:120
27151994221,10cyclictest35966irq/58-eth022:45:153
2713099424,30cyclictest27346irq/40-dwc2_hso23:20:122
2715199413,21cyclictest35966irq/58-eth023:10:133
27151994129,6cyclictest35966irq/58-eth000:30:143
27151994129,6cyclictest35966irq/58-eth000:30:143
2710399412,5cyclictest131rcu_preempt22:25:101
2707699412,30cyclictest2707-21ssh23:45:120
2715199402,23cyclictest35966irq/58-eth021:50:133
2707699402,28cyclictest18678-21ssh23:00:130
2715199392,25cyclictest35966irq/58-eth023:20:133
27151993910,18cyclictest35966irq/58-eth023:55:133
2713099391,18cyclictest31517-21ssh21:20:142
2713099381,22cyclictest22445-21sh21:15:122
2715199374,20cyclictest35966irq/58-eth021:30:143
2707699372,28cyclictest27756-21ssh23:05:140
2707699372,28cyclictest0-21swapper/023:25:130
27151993620,9cyclictest0-21swapper/322:15:113
27076993421,6cyclictest0-21swapper/000:15:120
2713099331,26cyclictest20725-21ssh23:55:142
2710399332,23cyclictest23768-21ssh22:45:141
27103993312,5cyclictest131rcu_preempt00:10:121
2707699332,22cyclictest12088-21sh21:45:110
2707699332,21cyclictest13342-21ssh21:10:140
2713099321,27cyclictest12079-21ssh21:45:112
2707699323,25cyclictest26135-21ssh23:40:150
2713099313,21cyclictest0-21swapper/219:35:132
27151993018,7cyclictest0-21swapper/322:00:133
2715199301,18cyclictest35966irq/58-eth000:35:133
2715199301,15cyclictest35966irq/58-eth020:10:133
2713099304,19cyclictest27346irq/40-dwc2_hso21:55:132
2713099304,17cyclictest27-21ksoftirqd/222:25:132
2713099303,18cyclictest27346irq/40-dwc2_hso22:50:142
2713099302,6cyclictest947-21ssh00:20:122
2713099302,22cyclictest7978-21ssh23:30:102
2713099302,21cyclictest0-21swapper/220:30:122
2713099302,14cyclictest27-21ksoftirqd/220:35:122
2713099301,24cyclictest0-21swapper/200:15:132
27103993018,3cyclictest21-21ksoftirqd/122:15:111
2707699305,17cyclictest0-21swapper/020:45:100
2713099294,18cyclictest27-21ksoftirqd/221:40:132
2713099293,19cyclictest27346irq/40-dwc2_hso19:50:132
2713099292,8cyclictest17296-21ssh21:30:152
2713099292,21cyclictest0-21swapper/220:20:132
2713099292,21cyclictest0-21swapper/220:20:132
2713099292,21cyclictest0-21swapper/219:30:122
2710399294,10cyclictest131rcu_preempt00:25:131
2710399293,18cyclictest964-21ssh00:20:131
2710399292,20cyclictest21195-21ssh21:50:131
27103992913,7cyclictest0-21swapper/122:50:131
2713099285,16cyclictest27346irq/40-dwc2_hso22:15:132
2713099282,21cyclictest4617-21ssh22:34:212
2713099282,21cyclictest4617-21ssh22:34:202
2713099282,21cyclictest20096-21kworker/u8:022:42:212
2713099282,20cyclictest23449-21ssh23:37:202
2713099282,15cyclictest21185-21ssh21:50:132
2713099282,11cyclictest7906-21ssh22:00:192
2713099281,13cyclictest4155-21apt00:40:002
2710399283,15cyclictest630-21Async09:36:311
2710399282,18cyclictest26920-21cat23:40:201
2710399281,24cyclictest15930-21ssh22:05:141
2710399281,17cyclictest17287-21ssh21:30:141
2713099275,15cyclictest27346irq/40-dwc2_hso23:10:122
2713099273,17cyclictest27346irq/40-dwc2_hso21:25:132
2713099272,23cyclictest31456-21ssh23:25:132
2713099271,17cyclictest27346irq/40-dwc2_hso19:55:112
2713099271,17cyclictest27346irq/40-dwc2_hso19:55:102
2710399272,18cyclictest12928-21diskmemload23:20:111
2710399271,22cyclictest27737-21ssh23:05:121
2710399271,21cyclictest21-21ksoftirqd/122:30:141
2710399271,21cyclictest21-21ksoftirqd/122:30:141
2707699272,18cyclictest2975-21ssh21:40:130
2713099263,6cyclictest0-21swapper/222:10:202
27103992616,4cyclictest31536-21ssh21:20:151
2707699266,14cyclictest12-21ksoftirqd/019:20:140
2715199252,17cyclictest0-21swapper/322:34:213
2715199252,17cyclictest0-21swapper/322:34:203
2713099255,16cyclictest0-21swapper/223:15:122
2710399251,16cyclictest22340-21ssh23:00:241
2707699252,17cyclictest2064-21ssh22:50:200
27076992513,6cyclictest0-21swapper/000:27:240
2715199246,10cyclictest35966irq/58-eth021:55:153
27130992413,8cyclictest0-21swapper/221:15:012
2713099241,13cyclictest11323-21apt23:50:012
27130992411,10cyclictest20810-21/usr/sbin/munin22:05:322
2710399242,18cyclictest6377-21ssh00:05:141
2710399242,18cyclictest11740-21ssh23:50:141
27103992414,4cyclictest0-21swapper/123:55:131
2710399241,19cyclictest28555-21find19:45:121
2710399241,18cyclictest6849-21sh22:00:131
2707699242,18cyclictest10919-21ssh22:20:130
27076992418,4cyclictest0-21swapper/000:20:220
2707699241,19cyclictest8210-21sh21:25:140
2707699241,18cyclictest15918-21sh22:05:140
2713099236,14cyclictest32106-21grep19:45:262
2713099235,9cyclictest8641-21diskmemload22:35:222
2713099232,6cyclictest0-21swapper/200:27:252
2713099232,16cyclictest0-21swapper/200:03:322
2713099232,15cyclictest0-21swapper/200:30:162
2713099232,15cyclictest0-21swapper/200:30:152
27130992312,9cyclictest0-21swapper/220:55:282
27130992311,9cyclictest0-21swapper/219:25:232
27130992311,10cyclictest23182-21ssh00:14:042
27130992310,11cyclictest19001-21ssh23:53:262
2707699239,8cyclictest12-21ksoftirqd/020:35:230
2707699239,4cyclictest12-21ksoftirqd/022:30:200
2707699239,4cyclictest12-21ksoftirqd/022:30:200
2707699238,8cyclictest12-21ksoftirqd/000:08:130
2707699238,5cyclictest131rcu_preempt20:20:010
2707699232,17cyclictest844-21munin-node19:50:140
2707699231,18cyclictest7207-21cut20:30:140
2715199224,12cyclictest35966irq/58-eth023:45:133
2715199224,11cyclictest35966irq/58-eth021:00:123
27151992211,9cyclictest0-21swapper/323:04:203
2713099223,16cyclictest27346irq/40-dwc2_hso21:00:132
2713099221,9cyclictest930-21snmpd23:00:312
2713099221,17cyclictest30551-21kthreadcore23:05:212
2713099221,12cyclictest135-21systemd-journal20:05:002
2713099221,11cyclictest27872-21kthreadcore21:35:202
2713099221,11cyclictest27261-21kthreadcore20:50:182
2713099221,10cyclictest8675-21irqcore21:05:172
2713099221,10cyclictest8675-21irqcore21:05:172
2713099221,10cyclictest26283-21fschecks_count23:40:172
2713099221,10cyclictest20301-21fschecks_count20:10:162
2710399224,10cyclictest2984-21wc23:25:281
2710399222,17cyclictest2715-21rm23:45:131
2710399222,16cyclictest8216-21ssh21:25:151
2707699222,15cyclictest29764-21ssh00:00:160
27076992215,4cyclictest0-21swapper/019:45:120
27076992211,3cyclictest0-21swapper/022:46:100
27076992210,9cyclictest26576-21awk21:15:280
2715199214,11cyclictest35966irq/58-eth022:20:263
27151992111,7cyclictest35966irq/58-eth021:10:133
2713099212,16cyclictest4625-21cpuspeed19:20:132
2713099212,16cyclictest25452-21kthreadcore22:45:212
2713099211,9cyclictest27472-21fschecks_count19:10:152
2713099211,9cyclictest15700-21tail20:05:182
2713099211,9cyclictest11215-21tr22:55:212
2713099211,17cyclictest14858-21wc22:20:282
2713099211,16cyclictest6548-21cut00:05:182
2713099211,16cyclictest27346irq/40-dwc2_hso20:25:142
2713099211,16cyclictest27243-21ntp_states19:40:242
2713099211,11cyclictest18677-21kthreadcore20:40:202
2713099211,10cyclictest28294-21grep20:15:222
2713099211,10cyclictest21748-21grep20:45:142
2710399218,4cyclictest131rcu_preempt20:55:181
27103992116,3cyclictest13046-21sh23:15:021
27103992112,6cyclictest0-21swapper/120:35:231
27103992110,4cyclictest131rcu_preempt22:55:131
2707699219,4cyclictest12-21ksoftirqd/022:13:420
2707699216,5cyclictest12-21ksoftirqd/021:52:270
27076992117,2cyclictest0-21swapper/023:50:120
27076992115,4cyclictest0-21swapper/022:55:250
27076992114,4cyclictest0-21swapper/021:57:090
27076992112,7cyclictest0-21swapper/021:05:130
27076992112,7cyclictest0-21swapper/021:05:120
27076992112,6cyclictest26371-21cpuspeed21:35:140
2707699211,17cyclictest10401-21ls22:00:230
2707699211,15cyclictest1080-21ls20:20:290
2707699211,15cyclictest1080-21ls20:20:280
2707699211,14cyclictest8295-21ls21:05:010
27076992111,4cyclictest0-21swapper/019:40:120
2715199203,12cyclictest0-21swapper/322:40:103
27151992013,4cyclictest0-21swapper/322:37:203
27151992011,6cyclictest0-21swapper/321:45:223
2715199201,12cyclictest35966irq/58-eth021:40:133
27151992010,8cyclictest0-21swapper/322:14:033
2713099201,8cyclictest32306-21fschecks_time19:15:152
2713099201,7cyclictest26997-21latency_hist19:10:022
2710399202,16cyclictest13674-21ssh22:39:201
2710399202,14cyclictest24362-21ssh00:15:131
2710399201,5cyclictest12928-21diskmemload22:24:351
2710399200,17cyclictest30052-21ls20:50:271
2710399200,14cyclictest30108-21ssh22:10:501
2707699209,4cyclictest12-21ksoftirqd/020:55:170
2707699209,3cyclictest12-21ksoftirqd/019:25:130
2707699208,4cyclictest12-21ksoftirqd/023:59:350
2707699208,4cyclictest0-21swapper/000:31:540
2707699208,4cyclictest0-21swapper/000:31:540
2707699208,3cyclictest0-21swapper/021:32:460
2707699205,4cyclictest131rcu_preempt20:00:310
2707699203,11cyclictest0-21swapper/021:20:150
2707699202,3cyclictest131rcu_preempt23:10:260
27076992014,4cyclictest12-21ksoftirqd/022:15:200
27076992014,4cyclictest0-21swapper/023:15:140
2707699201,3cyclictest0-21swapper/000:11:140
27076992013,4cyclictest5985-21irqcore22:35:180
27076992013,4cyclictest30480-21kthreadcore00:35:210
2707699201,12cyclictest23903-21ssh22:25:270
2715199192,10cyclictest0-21swapper/321:35:143
2715199191,13cyclictest0-21swapper/320:40:163
2715199191,13cyclictest0-21swapper/319:50:013
2715199191,12cyclictest0-21swapper/319:25:163
27151991910,6cyclictest0-21swapper/300:25:303
2710399199,6cyclictest15395-21kthreadcore22:40:171
2710399194,4cyclictest131rcu_preempt20:50:021
2710399193,11cyclictest0-21swapper/119:15:121
27103991912,5cyclictest73-21kswapd023:30:231
2707699194,5cyclictest12-21ksoftirqd/020:05:210
27076991912,4cyclictest0-21swapper/019:10:020
27076991911,5cyclictest151rcuc/020:50:110
2715199189,6cyclictest35966irq/58-eth023:35:133
2715199189,6cyclictest0-21swapper/323:50:313
2715199189,6cyclictest0-21swapper/323:40:583
2715199189,6cyclictest0-21swapper/323:05:453
2715199189,6cyclictest0-21swapper/322:27:283
2715199189,6cyclictest0-21swapper/300:11:383
2715199182,10cyclictest0-21swapper/320:45:053
2715199182,10cyclictest0-21swapper/320:20:143
2715199182,10cyclictest0-21swapper/320:20:133
2715199181,13cyclictest0-21swapper/300:25:013
2715199181,12cyclictest0-21swapper/319:40:123
27151991810,6cyclictest0-21swapper/323:30:143
27151991810,6cyclictest0-21swapper/322:57:193
27151991810,6cyclictest0-21swapper/322:09:423
2710399189,6cyclictest9151-21cron19:25:011
2710399188,8cyclictest4401-21/usr/sbin/munin19:50:241
2710399186,9cyclictest15145-21cron20:05:011
2710399184,10cyclictest4060-21irqrtprio21:00:181
2710399182,13cyclictest31934-21kthreadcore20:20:211
2710399182,13cyclictest31934-21kthreadcore20:20:211
2710399182,12cyclictest30336-21df_inode21:55:161
2710399181,9cyclictest31920-21latency_hist19:15:001
2710399181,8cyclictest29034-21kthreadcore21:35:231
2710399181,3cyclictest0-21swapper/121:14:401
27103991812,4cyclictest21-21ksoftirqd/120:45:011
2710399181,14cyclictest2698-21cut20:25:191
27103991810,6cyclictest17775-21kthreadcore23:35:211
27103991810,5cyclictest3721-21kthreadcore21:40:201
27103991810,5cyclictest0-21swapper/121:05:151
27103991810,5cyclictest0-21swapper/121:05:151
2707699188,7cyclictest30937-21mailstats19:10:270
2707699188,4cyclictest12-21ksoftirqd/019:30:260
2707699187,7cyclictest151rcuc/023:20:120
2707699185,4cyclictest12-21ksoftirqd/019:35:210
27076991812,4cyclictest0-21swapper/019:55:130
27076991812,4cyclictest0-21swapper/019:55:130
2715199179,6cyclictest36066irq/59-eth000:15:123
2715199179,6cyclictest0-21swapper/321:28:283
2715199179,6cyclictest0-21swapper/300:00:163
2715199172,9cyclictest0-21swapper/320:55:143
2715199172,9cyclictest0-21swapper/319:10:053
2715199172,10cyclictest0-21swapper/323:25:133
2715199172,10cyclictest0-21swapper/319:35:133
2715199172,10cyclictest0-21swapper/319:30:123
2715199171,12cyclictest0-21swapper/320:25:303
2715199171,11cyclictest0-21swapper/300:05:313
2710399179,6cyclictest5960-21cron00:05:011
2710399178,5cyclictest0-21swapper/120:05:121
2710399176,8cyclictest18997-21apt-get19:35:131
2710399172,10cyclictest0-21swapper/119:55:121
2710399172,10cyclictest0-21swapper/119:55:121
2710399171,6cyclictest7354-21ps20:30:161
2710399171,13cyclictest20369-21cut20:10:161
2710399171,13cyclictest14628-21kthreadcore21:45:191
2710399171,12cyclictest0-21swapper/119:25:121
2710399170,12cyclictest24978-21apt-get20:15:131
2707699177,7cyclictest32148-21apt-get19:15:120
2707699174,10cyclictest3006-21kthreadcore20:25:210
2707699171,2cyclictest131rcu_preempt20:10:230
2715199161,8cyclictest0-21swapper/320:05:113
2715199161,12cyclictest0-21swapper/321:05:153
2715199161,12cyclictest0-21swapper/321:05:153
2710399167,6cyclictest27278-21cut19:10:021
2710399162,11cyclictest24157-21irqcore19:40:161
27076991611,3cyclictest0-21swapper/020:40:160
2715199152,9cyclictest0-21swapper/319:55:123
2715199152,9cyclictest0-21swapper/319:55:123
2710399151,11cyclictest14341-21fschecks_count19:30:141
2715199141,6cyclictest0-21swapper/320:30:163
2715199141,11cyclictest0-21swapper/319:20:293
2715199141,10cyclictest0-21swapper/320:35:143
2715199138,3cyclictest0-21swapper/319:50:243
2715199133,4cyclictest0-21swapper/320:15:133
2715199132,6cyclictest0-21swapper/320:00:153
2715199131,8cyclictest0-21swapper/319:15:133
2715199121,7cyclictest0-21swapper/320:50:113
2715199101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional