You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 23:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Sep 04, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30761996021,7cyclictest131rcu_preempt09:30:110
3076199594,12cyclictest131rcu_preempt12:10:130
3076199525,6cyclictest131rcu_preempt07:20:120
3076999506,39cyclictest30038-21ssh09:35:111
30789994932,11cyclictest35966irq/58-eth011:40:143
3077899471,28cyclictest29128-21ssh10:10:172
3076199472,42cyclictest21809-21ssh11:35:140
3076199463,5cyclictest131rcu_preempt10:15:140
30769994532,8cyclictest0-21swapper/110:00:131
30769994532,8cyclictest0-21swapper/110:00:131
30789994438,3cyclictest35966irq/58-eth010:55:123
3078999441,25cyclictest35966irq/58-eth007:15:123
3076199434,31cyclictest18988-21ssh10:40:130
3076199421,33cyclictest22920-21ssh11:00:120
30789994130,8cyclictest35966irq/58-eth009:35:113
30769994117,17cyclictest21-21ksoftirqd/110:10:171
30761994125,9cyclictest12-21ksoftirqd/010:48:340
3076999404,29cyclictest25931-21ssh09:15:111
30761993915,4cyclictest12-21ksoftirqd/008:55:120
3076999383,24cyclictest16432-21diskmemload09:20:131
3077899371,18cyclictest25919-21ssh09:15:102
3076999365,23cyclictest12718-21ssh11:30:161
3076199363,22cyclictest679-21ssh10:30:120
30778993523,6cyclictest0-21swapper/210:50:162
3076999353,26cyclictest5960-21ssh12:20:151
3076999353,18cyclictest10844-21kworker/u8:312:25:141
3076199352,24cyclictest16213-21kthreadcore10:55:200
3076199351,23cyclictest24073-21ssh10:25:130
3077899349,10cyclictest27-21ksoftirqd/209:40:012
3077899346,20cyclictest27-21ksoftirqd/212:36:092
3076999343,24cyclictest753-21sendmail-mta08:30:131
3076199342,5cyclictest11807-21apt-get09:25:120
3076199342,27cyclictest15676-21sed09:45:010
3078999332,20cyclictest35966irq/58-eth012:20:153
3077899338,17cyclictest0-21swapper/211:56:022
3076199332,23cyclictest9849-21grep11:10:230
30778993213,6cyclictest27-21ksoftirqd/211:50:332
30778993213,13cyclictest0-21swapper/212:09:162
30778993212,12cyclictest0-21swapper/211:16:402
3077899321,19cyclictest930-21snmpd09:20:132
30778993211,14cyclictest0-21swapper/212:30:342
3076999324,15cyclictest11810-21ssh09:25:111
3076999322,20cyclictest6781-21ssh09:40:141
3077899316,17cyclictest27346irq/40-dwc2_hso11:10:182
3077899316,14cyclictest131rcu_preempt08:30:272
3077899314,16cyclictest27346irq/40-dwc2_hso10:55:122
3077899313,7cyclictest30751-21cyclictest10:00:122
3077899313,7cyclictest30751-21cyclictest10:00:122
3077899312,21cyclictest16355-21ntp_192.168.11411:30:282
3077899311,18cyclictest27346irq/40-dwc2_hso07:40:122
30778993110,15cyclictest0-21swapper/209:31:302
30769993119,6cyclictest16432-21diskmemload11:10:181
3076199312,21cyclictest30937-21ssh11:40:160
30789993018,8cyclictest35966irq/58-eth009:15:133
3078999301,20cyclictest35966irq/58-eth008:55:113
3077899306,17cyclictest27346irq/40-dwc2_hso12:00:132
3077899301,24cyclictest7499-21ssh11:45:132
30778993011,13cyclictest0-21swapper/212:23:422
3076199301,4cyclictest10603-21apt-get08:30:130
3078999291,19cyclictest35966irq/58-eth009:40:123
3077899299,14cyclictest0-21swapper/211:27:322
3077899299,13cyclictest0-21swapper/210:19:582
3077899298,4cyclictest27-21ksoftirqd/209:57:012
3077899295,13cyclictest27346irq/40-dwc2_hso08:55:122
3077899291,19cyclictest26301-21ssh11:00:212
30769992917,6cyclictest16432-21diskmemload12:10:121
30761992923,4cyclictest12-21ksoftirqd/009:10:010
3076199292,21cyclictest25569-21ssh11:55:150
3076199292,19cyclictest12153-21ssh10:53:340
3076199291,4cyclictest5932-21apt-get12:20:140
3078999283,16cyclictest0-21swapper/311:08:343
3078999282,16cyclictest35966irq/58-eth008:30:133
30789992816,8cyclictest0-21swapper/310:35:143
3077899289,4cyclictest27-21ksoftirqd/210:26:052
3077899288,8cyclictest27-21ksoftirqd/212:15:312
3077899282,21cyclictest5334-21ssh11:06:352
3077899282,16cyclictest27346irq/40-dwc2_hso08:20:122
30778992813,8cyclictest0-21swapper/210:42:542
30778992811,11cyclictest0-21swapper/211:37:572
30761992822,2cyclictest121ksoftirqd/012:05:110
3076199282,19cyclictest30576-21/usr/sbin/munin11:20:260
3078999273,18cyclictest0-21swapper/311:13:343
3078999271,13cyclictest35966irq/58-eth012:30:113
3077899277,13cyclictest27-21ksoftirqd/208:45:272
3077899276,7cyclictest27-21ksoftirqd/209:13:102
3077899276,3cyclictest131rcu_preempt07:35:172
3077899276,3cyclictest131rcu_preempt07:35:172
3077899276,12cyclictest27-21ksoftirqd/209:45:112
3077899274,15cyclictest131rcu_preempt07:55:252
3077899274,13cyclictest27-21ksoftirqd/210:05:542
3077899273,17cyclictest20299-21diskmemload12:10:122
3077899272,19cyclictest0-21swapper/207:45:132
3077899272,17cyclictest6780-21diskmemload09:40:132
3077899272,15cyclictest27346irq/40-dwc2_hso09:00:122
3076999278,9cyclictest21-21ksoftirqd/110:30:081
3076999272,19cyclictest16837-21ssh09:10:131
3076199278,8cyclictest12-21ksoftirqd/008:05:120
30761992712,9cyclictest630-21Async09:36:310
3078999268,10cyclictest36066irq/59-eth009:20:133
3077899269,11cyclictest0-21swapper/212:28:292
3077899268,7cyclictest27-21ksoftirqd/209:05:162
3077899268,12cyclictest27-21ksoftirqd/211:20:332
3077899267,7cyclictest27-21ksoftirqd/209:53:372
3077899267,3cyclictest131rcu_preempt07:50:282
3077899265,13cyclictest27-21ksoftirqd/209:25:462
3077899265,12cyclictest131rcu_preempt08:35:252
3077899263,12cyclictest131rcu_preempt10:35:222
3077899262,9cyclictest27-21ksoftirqd/210:20:282
3076999268,10cyclictest201rcuc/109:49:541
3076999264,4cyclictest21-21ksoftirqd/112:30:231
3076999262,19cyclictest10642-21ssh10:15:341
30769992620,4cyclictest201rcuc/111:05:301
3076199263,18cyclictest12-21ksoftirqd/009:50:140
3076199263,17cyclictest2178-21ssh12:00:140
3076199262,8cyclictest131rcu_preempt12:18:340
3076199262,3cyclictest131rcu_preempt11:05:160
3078999254,14cyclictest35966irq/58-eth010:45:113
3078999253,12cyclictest0-21swapper/311:25:123
3078999252,19cyclictest35966irq/58-eth012:25:133
3078999252,17cyclictest0-21swapper/310:12:343
3078999252,10cyclictest35966irq/58-eth009:30:113
30789992510,10cyclictest35966irq/58-eth009:10:133
3077899257,4cyclictest27-21ksoftirqd/211:40:222
30778992511,8cyclictest0-21swapper/210:31:192
3076999259,8cyclictest21-21ksoftirqd/110:38:141
3076999253,5cyclictest21-21ksoftirqd/111:00:011
3076999252,18cyclictest4903-21ssh10:50:181
30769992514,8cyclictest21-21ksoftirqd/111:25:271
30769992512,3cyclictest0-21swapper/111:51:461
3076199251,19cyclictest678-21ssh12:35:120
3077899245,12cyclictest27-21ksoftirqd/207:20:262
3077899245,11cyclictest131rcu_preempt08:00:202
3077899245,11cyclictest131rcu_preempt08:00:202
3077899244,5cyclictest27-21ksoftirqd/208:40:202
3077899244,12cyclictest131rcu_preempt07:25:142
3077899244,11cyclictest27-21ksoftirqd/210:48:042
3076999248,5cyclictest21-21ksoftirqd/112:05:021
3076999248,3cyclictest131rcu_preempt11:38:001
3076999246,9cyclictest21-21ksoftirqd/110:06:001
3076999245,8cyclictest21-21ksoftirqd/110:28:461
3076999244,4cyclictest21-21ksoftirqd/109:30:161
30769992412,8cyclictest18599-21needreboot10:20:241
3076199242,6cyclictest0-21swapper/011:18:340
3076199242,19cyclictest2280-21ls09:20:010
30761992411,8cyclictest12-21ksoftirqd/007:50:120
3078999233,13cyclictest0-21swapper/311:30:343
3077899237,4cyclictest27-21ksoftirqd/208:50:122
3076999237,8cyclictest21-21ksoftirqd/111:57:231
3076999237,8cyclictest21-21ksoftirqd/108:10:241
3076999237,7cyclictest21-21ksoftirqd/111:43:221
3076999236,8cyclictest21-21ksoftirqd/112:08:091
3076999236,8cyclictest21-21ksoftirqd/111:04:381
3076999236,8cyclictest21-21ksoftirqd/110:45:521
3076999236,8cyclictest21-21ksoftirqd/109:52:131
3076999234,9cyclictest21-21ksoftirqd/111:16:251
3076999234,8cyclictest131rcu_preempt09:55:201
3076999234,5cyclictest21-21ksoftirqd/107:25:231
3076999234,4cyclictest21-21ksoftirqd/108:00:121
3076999234,4cyclictest21-21ksoftirqd/108:00:121
3076999233,6cyclictest21-21ksoftirqd/108:30:011
3076999232,17cyclictest3281-21apt-get07:15:121
3076999231,7cyclictest21-21ksoftirqd/110:40:191
3076999231,5cyclictest21-21ksoftirqd/109:05:161
3076999231,16cyclictest7497-21sh11:45:131
3076999231,11cyclictest21-21ksoftirqd/107:35:181
3076999231,11cyclictest21-21ksoftirqd/107:35:181
3076199235,8cyclictest131rcu_preempt10:05:190
30761992316,4cyclictest12-21ksoftirqd/008:10:120
3076199231,16cyclictest14992-21sh12:25:140
30761992310,8cyclictest536-21usb-storage09:45:040
3078999225,12cyclictest321rcuc/310:40:123
3078999223,13cyclictest0-21swapper/310:08:343
3078999223,12cyclictest0-21swapper/310:04:353
3078999223,12cyclictest0-21swapper/310:04:343
3077899225,9cyclictest27-21ksoftirqd/208:15:122
3077899223,4cyclictest27-21ksoftirqd/208:10:192
3077899222,7cyclictest131rcu_preempt08:05:162
30778992212,8cyclictest0-21swapper/208:25:282
3077899221,11cyclictest7005-21cat07:15:282
30778992210,4cyclictest27-21ksoftirqd/207:10:112
3076999227,8cyclictest21-21ksoftirqd/111:22:541
3076999225,8cyclictest21-21ksoftirqd/112:37:321
3076999225,7cyclictest25828-21kworker/1:212:15:101
3076199221,6cyclictest13253-21kthreadcore10:00:210
3076199221,6cyclictest13253-21kthreadcore10:00:210
3076199221,3cyclictest0-21swapper/010:20:120
3076199221,17cyclictest18771-21latency_hist08:05:010
3076199221,17cyclictest18771-21latency_hist08:05:010
3076199221,15cyclictest13452-21ssh11:30:230
3076199220,3cyclictest131rcu_preempt08:35:170
3078999213,9cyclictest0-21swapper/311:00:213
3078999213,12cyclictest35966irq/58-eth012:35:113
3078999211,14cyclictest0-21swapper/310:30:113
3077899211,11cyclictest21578-21tail07:30:272
3076999215,7cyclictest21-21ksoftirqd/108:50:161
3076999214,8cyclictest21-21ksoftirqd/107:20:141
3076999214,6cyclictest21-21ksoftirqd/108:55:151
3076999214,5cyclictest21-21ksoftirqd/107:50:221
30761992112,5cyclictest24044-21/usr/sbin/munin12:30:150
30761992112,5cyclictest12-21ksoftirqd/007:40:130
3076199211,16cyclictest28615-21apt-get08:15:100
3076199210,8cyclictest9754-21/usr/sbin/munin10:35:280
3078999209,8cyclictest0-21swapper/312:02:223
3078999202,13cyclictest321rcuc/310:50:163
3078999202,12cyclictest0-21swapper/310:15:103
30789992012,5cyclictest0-21swapper/312:05:223
3076999203,5cyclictest21-21ksoftirqd/109:00:181
3076999201,4cyclictest21-21ksoftirqd/108:20:181
3076999200,5cyclictest21-21ksoftirqd/107:55:131
3076199204,9cyclictest32712-21kthreadcore08:50:220
3076199201,9cyclictest22352-21latency_hist07:35:010
3076199201,7cyclictest29093-21/usr/sbin/munin10:10:310
3076199201,6cyclictest17666-21grep09:10:180
3076199201,2cyclictest131rcu_preempt09:55:220
3078999193,11cyclictest0-21swapper/312:18:343
3078999193,11cyclictest0-21swapper/311:21:343
30789991913,4cyclictest0-21swapper/311:15:143
30789991913,4cyclictest0-21swapper/309:25:283
3078999191,15cyclictest0-21swapper/311:45:193
3077899191,13cyclictest30837-21latency_hist07:10:022
3076999197,3cyclictest131rcu_preempt07:30:171
3076999194,3cyclictest131rcu_preempt08:10:011
3076999191,8cyclictest27783-21kthreadcore08:45:211
3076999191,5cyclictest21-21ksoftirqd/107:10:161
30769991914,3cyclictest201rcuc/108:40:241
30769991912,4cyclictest201rcuc/108:40:011
3076199199,7cyclictest12-21ksoftirqd/008:40:140
3076199192,2cyclictest131rcu_preempt11:25:250
3076199192,15cyclictest2219-21kthreadcore08:20:180
3076199191,3cyclictest131rcu_preempt07:55:170
3078999188,8cyclictest0-21swapper/311:53:523
3078999183,10cyclictest0-21swapper/311:59:343
3078999182,11cyclictest0-21swapper/307:45:133
3078999182,10cyclictest0-21swapper/308:15:123
3078999182,10cyclictest0-21swapper/307:25:153
3078999182,10cyclictest0-21swapper/307:10:133
30789991812,4cyclictest0-21swapper/312:10:573
30789991811,5cyclictest0-21swapper/309:45:233
3078999181,12cyclictest0-21swapper/311:39:343
3078999181,12cyclictest0-21swapper/308:05:163
30789991810,6cyclictest0-21swapper/309:55:063
3076999183,11cyclictest844-21munin-node07:40:141
30769991812,3cyclictest0-21swapper/108:15:271
3076199189,5cyclictest12-21ksoftirqd/011:50:160
3076199183,8cyclictest0-21swapper/007:25:140
3076199182,13cyclictest3776-21kthreadcore09:20:190
3076199181,8cyclictest5980-21fschecks_time08:25:150
3076199181,6cyclictest131rcu_preempt09:40:010
3076199181,4cyclictest131rcu_preempt07:35:200
3076199181,4cyclictest131rcu_preempt07:35:200
3076199181,2cyclictest131rcu_preempt07:15:280
3078999179,6cyclictest0-21swapper/309:53:303
3078999179,5cyclictest35966irq/58-eth010:25:133
3078999172,10cyclictest0-21swapper/310:20:043
3078999172,10cyclictest0-21swapper/308:00:143
3078999172,10cyclictest0-21swapper/308:00:143
3078999172,10cyclictest0-21swapper/307:40:143
3078999172,10cyclictest0-21swapper/307:20:133
3078999171,12cyclictest0-21swapper/307:50:273
3076199179,5cyclictest12-21ksoftirqd/011:45:120
3076199172,4cyclictest131rcu_preempt07:45:260
3076199171,11cyclictest0-21swapper/007:10:130
3078999162,9cyclictest0-21swapper/308:50:143
3078999162,9cyclictest0-21swapper/308:35:153
3078999162,9cyclictest0-21swapper/308:20:143
3078999161,10cyclictest0-21swapper/308:45:153
3076999169,5cyclictest32679-21sed07:45:181
3076199161,11cyclictest10399-21wc09:00:210
3078999152,4cyclictest0-21swapper/308:25:133
3078999152,10cyclictest0-21swapper/309:05:133
3078999152,10cyclictest0-21swapper/307:35:113
3078999152,10cyclictest0-21swapper/307:35:113
3078999149,3cyclictest0-21swapper/307:55:233
3078999141,10cyclictest0-21swapper/307:30:203
3076999141,9cyclictest30823-21cat07:10:021
3078999131,10cyclictest0-21swapper/308:10:193
3076199131,8cyclictest30822-21latency_hist07:10:020
3078999125,5cyclictest0-21swapper/308:45:013
3078999122,8cyclictest0-21swapper/309:00:123
307899991,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional