You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 23:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Aug 18, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11224996453,8cyclictest29569-21diskmemload11:15:142
1122499563,15cyclictest27346irq/40-dwc2_hso08:55:132
1121599553,40cyclictest28773-21ssh11:15:131
1121599551,41cyclictest23585-21ssh11:30:141
1121599512,36cyclictest7368-21ssh12:15:141
11215994828,8cyclictest21-21ksoftirqd/109:35:141
1121599481,39cyclictest27241-21ssh11:50:141
1120799481,23cyclictest26759-21ssh10:20:130
1121599474,3cyclictest131rcu_preempt08:35:111
1121599474,3cyclictest131rcu_preempt08:35:111
1121599452,6cyclictest131rcu_preempt08:50:111
1120799453,31cyclictest15129-21kworker/u8:011:00:140
1123199443,24cyclictest35966irq/58-eth009:50:103
11224994412,26cyclictest31442-21sh11:34:092
1120799442,5cyclictest12-21ksoftirqd/009:55:140
1121599431,32cyclictest0-21swapper/112:25:131
1123199423,26cyclictest35966irq/58-eth011:15:123
1123199421,25cyclictest35966irq/58-eth012:35:133
1122499421,21cyclictest19385-21ssh09:40:132
1121599421,33cyclictest0-21swapper/108:05:121
1122499411,32cyclictest1060-21sh11:52:102
1120799412,31cyclictest6932-21ssh10:45:130
1120799411,33cyclictest10764-21ssh11:05:120
1122499402,31cyclictest26109-21ssh11:49:092
11231993928,6cyclictest35966irq/58-eth010:25:133
1121599392,30cyclictest1974-21ssh12:30:131
1123199382,24cyclictest35966irq/58-eth007:45:133
1122499381,17cyclictest32065-21ssh10:05:142
1120799383,26cyclictest3422-21ssh10:25:130
1122499372,18cyclictest7114-21ssh11:01:092
1121599361,30cyclictest17580-21ssh10:15:091
1123199352,19cyclictest35966irq/58-eth012:05:133
1122499352,16cyclictest0-21swapper/211:41:082
1122499351,18cyclictest29569-21diskmemload11:56:102
1122499350,20cyclictest930-21snmpd07:50:132
1123199332,18cyclictest35966irq/58-eth010:55:133
1122499333,19cyclictest27346irq/40-dwc2_hso09:00:122
1121599332,20cyclictest16387-21ssh12:20:131
1123199323,20cyclictest35966irq/58-eth012:15:143
1122499322,8cyclictest0-21swapper/211:29:302
1120799323,21cyclictest32582-21ssh11:35:130
1122499314,20cyclictest27346irq/40-dwc2_hso08:45:142
1121599313,21cyclictest24725-21ssh09:25:141
11207993114,9cyclictest29569-21diskmemload09:10:140
1122499304,20cyclictest0-21swapper/211:35:132
11207993013,10cyclictest0-21swapper/010:55:320
1123199293,19cyclictest0-21swapper/310:52:093
1122499293,21cyclictest0-21swapper/211:07:092
1122499293,17cyclictest27346irq/40-dwc2_hso12:30:122
1122499293,16cyclictest27346irq/40-dwc2_hso09:15:142
1122499292,21cyclictest11633-21sh11:22:092
1122499292,20cyclictest0-21swapper/207:35:142
1121599292,20cyclictest22717-21ssh11:29:301
1123199284,13cyclictest0-21swapper/312:25:133
1123199283,12cyclictest35966irq/58-eth011:50:143
1123199280,17cyclictest35966irq/58-eth008:35:113
1123199280,17cyclictest35966irq/58-eth008:35:103
1122499283,20cyclictest0-21swapper/210:48:092
1122499283,15cyclictest27346irq/40-dwc2_hso09:25:132
1122499282,22cyclictest30371-21ssh10:40:152
1121599287,12cyclictest630-21Async09:36:311
1121599285,18cyclictest12899-21ssh12:00:131
1121599283,22cyclictest6960-21ssh10:45:141
1121599281,19cyclictest1648-21ssh11:00:101
1120799282,20cyclictest30946-21ssh11:50:270
1123199273,18cyclictest0-21swapper/311:00:103
1123199272,11cyclictest35966irq/58-eth009:40:133
1122499274,13cyclictest27346irq/40-dwc2_hso08:35:102
1122499274,13cyclictest27346irq/40-dwc2_hso08:35:092
1122499272,20cyclictest30460-21ssh10:56:102
1121599279,12cyclictest4479-1kworker/u9:010:50:091
1120799273,16cyclictest3847-21ssh11:55:120
1123199262,15cyclictest35966irq/58-eth012:20:123
1122499263,16cyclictest27346irq/40-dwc2_hso08:00:132
1122499262,9cyclictest24079-21ssh10:54:202
1122499262,15cyclictest27346irq/40-dwc2_hso08:05:132
1122499262,10cyclictest0-21swapper/207:10:132
11224992614,6cyclictest2746ksoftirqd/210:10:132
11215992615,5cyclictest11199-21cyclictest07:40:141
1121599261,18cyclictest25085-21ssh10:55:131
1121599261,18cyclictest23122-21ssh10:00:151
1120799269,11cyclictest12-21ksoftirqd/011:41:090
1123199253,14cyclictest0-21swapper/311:29:293
11231992511,9cyclictest35966irq/58-eth011:45:133
1122499255,14cyclictest27346irq/40-dwc2_hso08:10:122
1122499254,17cyclictest27346irq/40-dwc2_hso08:50:132
1121599258,12cyclictest21-21ksoftirqd/111:14:241
1121599254,13cyclictest0-21swapper/108:30:141
1121599253,17cyclictest3439-21ssh10:25:151
1121599252,18cyclictest5396-21ssh11:20:131
11215992515,8cyclictest0-21swapper/109:45:131
1120799259,8cyclictest12-21ksoftirqd/009:28:080
1120799258,4cyclictest131rcu_preempt10:50:270
1120799257,10cyclictest12-21ksoftirqd/011:30:110
1120799253,5cyclictest12-21ksoftirqd/008:25:310
1123199245,11cyclictest35966irq/58-eth010:00:153
1123199243,14cyclictest0-21swapper/311:30:093
1122499249,11cyclictest5432-21cron08:45:012
1122499242,9cyclictest12402-21ssh10:30:142
1122499242,14cyclictest30872-21apt-get12:10:122
1120799248,7cyclictest12-21ksoftirqd/012:05:240
1120799248,10cyclictest12-21ksoftirqd/010:06:570
1120799242,16cyclictest1367-21ssh09:30:140
11207992414,7cyclictest12-21ksoftirqd/011:20:140
1123199232,14cyclictest35966irq/58-eth012:00:143
11231992312,5cyclictest0-21swapper/310:40:293
1123199230,15cyclictest35966irq/58-eth010:15:103
1122499238,5cyclictest29569-21diskmemload12:05:132
1122499233,12cyclictest10482-21kworker/2:012:00:142
1122499232,6cyclictest0-21swapper/210:38:102
1122499232,6cyclictest0-21swapper/210:38:102
1122499232,16cyclictest29969-21ssh09:10:132
1122499232,16cyclictest0-21swapper/212:23:092
1122499232,16cyclictest0-21swapper/211:10:092
1121599235,10cyclictest0-21swapper/107:35:131
11215992312,3cyclictest21-21ksoftirqd/109:50:291
1121599231,16cyclictest30437-21ssh10:20:261
1120799239,4cyclictest131rcu_preempt12:39:190
1120799238,5cyclictest12-21ksoftirqd/009:17:570
1120799236,7cyclictest12-21ksoftirqd/009:24:500
1120799235,11cyclictest12-21ksoftirqd/010:17:080
1120799235,10cyclictest12-21ksoftirqd/012:14:390
11207992313,7cyclictest0-21swapper/009:05:140
11207992312,5cyclictest0-21swapper/009:46:010
1123199223,11cyclictest0-21swapper/312:30:143
11231992213,6cyclictest35966irq/58-eth009:45:133
1122499226,13cyclictest7496-21unixbench_multi10:25:312
1122499223,11cyclictest11726-21kthreadcore09:35:202
1122499221,17cyclictest25955-21cut07:25:172
1122499221,17cyclictest17957-21head08:20:292
1122499221,11cyclictest24845-21grep07:20:272
11224992210,9cyclictest16365-21ssh12:36:092
11224992210,5cyclictest13976-21ssh09:55:102
1121599224,13cyclictest29569-21diskmemload11:45:141
1121599223,14cyclictest844-21munin-node10:40:161
11215992213,6cyclictest1945-21unixbench_multi07:30:301
1121599221,16cyclictest30110-21cut09:10:161
1120799229,3cyclictest131rcu_preempt07:10:250
1120799226,4cyclictest131rcu_preempt12:00:190
1120799226,4cyclictest131rcu_preempt10:35:300
1120799226,4cyclictest131rcu_preempt10:35:290
1120799225,7cyclictest12-21ksoftirqd/011:45:260
1120799225,10cyclictest12-21ksoftirqd/010:02:260
1120799224,11cyclictest12-21ksoftirqd/012:30:460
11207992214,5cyclictest0-21swapper/011:10:130
1120799221,16cyclictest16548-21cut12:20:160
1123199212,11cyclictest35966irq/58-eth010:05:133
11231992112,6cyclictest35966irq/58-eth010:10:153
1122499213,13cyclictest28560-21cpuspeed09:45:142
1122499211,9cyclictest17834-21kthreadcore09:20:222
1122499211,17cyclictest0-21swapper/212:25:212
1122499211,16cyclictest5356-21if_err_eth109:50:162
1122499211,13cyclictest29569-21diskmemload12:15:142
1122499211,12cyclictest28241-21cron08:35:002
1122499211,10cyclictest31664-21sh10:05:012
1121599219,7cyclictest21-21ksoftirqd/111:40:131
1121599219,3cyclictest131rcu_preempt10:30:261
1121599215,4cyclictest131rcu_preempt09:55:251
11215992114,5cyclictest8408-21ssh09:33:091
1121599210,18cyclictest21151-21ls07:50:281
1120799219,7cyclictest12-21ksoftirqd/012:15:360
1120799219,4cyclictest0-21swapper/009:52:040
1120799218,8cyclictest12-21ksoftirqd/009:35:190
1120799218,4cyclictest12-21ksoftirqd/012:26:480
1120799214,9cyclictest12-21ksoftirqd/011:25:180
11207992112,4cyclictest0-21swapper/007:55:150
11207992110,5cyclictest12-21ksoftirqd/011:15:370
1123199209,8cyclictest0-21swapper/310:32:443
1123199209,8cyclictest0-21swapper/309:38:283
1123199206,10cyclictest35966irq/58-eth009:10:133
1123199203,11cyclictest0-21swapper/310:45:093
1123199201,11cyclictest0-21swapper/307:15:133
1122499208,8cyclictest8033-21/usr/sbin/munin07:40:152
1122499204,13cyclictest25902-21cat07:55:282
1122499202,5cyclictest0-21swapper/208:25:152
1122499202,11cyclictest16790-21irqrtprio07:15:182
1122499201,9cyclictest30712-21fschecks_time07:30:162
1122499201,9cyclictest16421-21grep07:45:292
1122499201,15cyclictest1452-21df09:30:152
1122499201,11cyclictest26773-21ssh10:20:132
1122499201,11cyclictest25432-21irqrtprio09:05:182
1122499201,11cyclictest19647-21cut10:15:202
1121599209,8cyclictest6990-21irqcore09:15:201
1121599207,10cyclictest19517-21entropy09:40:151
1121599204,4cyclictest21-21ksoftirqd/111:55:291
1121599202,13cyclictest32555-21apt-get11:35:131
11215992011,5cyclictest21-21ksoftirqd/108:10:131
1121599201,13cyclictest8925-21idleruntime10:10:181
11215992010,7cyclictest21-21ksoftirqd/109:10:011
1121599200,17cyclictest24742-21ls07:20:281
1121599200,16cyclictest7038-21ls12:15:011
1121599200,14cyclictest19779-21switchtime07:15:281
1120799209,5cyclictest0-21swapper/010:10:080
1120799208,4cyclictest12-21ksoftirqd/007:30:240
1120799208,4cyclictest12-21ksoftirqd/007:15:260
11207992011,4cyclictest12-21ksoftirqd/007:45:250
11207992011,3cyclictest12-21ksoftirqd/008:10:020
1120799201,12cyclictest14086-21find08:20:130
11207992010,4cyclictest12-21ksoftirqd/009:40:240
1120799200,16cyclictest25643-21ls07:25:020
1123199199,7cyclictest0-21swapper/310:20:163
1123199196,6cyclictest35966irq/58-eth011:20:153
1123199195,9cyclictest35966irq/58-eth011:05:123
1123199193,11cyclictest0-21swapper/311:57:083
1123199193,11cyclictest0-21swapper/311:43:093
1123199193,11cyclictest0-21swapper/310:39:103
1123199193,11cyclictest0-21swapper/310:39:093
1123199192,13cyclictest35966irq/58-eth011:10:133
1123199192,11cyclictest0-21swapper/308:55:053
1123199192,11cyclictest0-21swapper/307:40:053
11231991911,6cyclictest35966irq/58-eth011:35:133
1123199191,12cyclictest35966irq/58-eth012:10:143
1123199191,11cyclictest0-21swapper/307:50:143
1122499193,9cyclictest11377-21kthreadcore08:15:212
1122499190,10cyclictest11316-21grep07:10:022
1121599199,8cyclictest9440-21timerwakeupswit08:45:301
1121599191,9cyclictest1194-21kthreadcore10:05:211
11215991912,5cyclictest18043-21ssh11:08:251
1121599191,11cyclictest10225-21kworker/u8:112:35:131
1120799197,7cyclictest12-21ksoftirqd/007:35:150
1120799195,8cyclictest12-21ksoftirqd/010:30:520
1120799194,3cyclictest131rcu_preempt10:43:180
1120799191,6cyclictest12-21ksoftirqd/007:50:130
11207991913,4cyclictest0-21swapper/008:10:220
1123199183,8cyclictest0-21swapper/307:30:143
1123199182,10cyclictest0-21swapper/308:30:143
1123199182,10cyclictest0-21swapper/308:05:133
1123199182,10cyclictest0-21swapper/308:00:053
11231991811,4cyclictest0-21swapper/309:25:233
11231991811,4cyclictest0-21swapper/309:15:243
1123199181,10cyclictest0-21swapper/308:10:053
11231991810,6cyclictest0-21swapper/309:30:073
1121599185,10cyclictest1-21systemd08:59:051
1121599182,14cyclictest21804-21kthreadcore09:00:191
1121599181,9cyclictest11670-21cut08:15:201
1121599181,7cyclictest21806-21irqcore10:35:191
1121599181,7cyclictest21806-21irqcore10:35:181
1120799189,6cyclictest0-21swapper/008:15:230
1120799184,4cyclictest12-21ksoftirqd/008:30:280
1120799182,3cyclictest131rcu_preempt08:55:280
1120799182,3cyclictest131rcu_preempt07:10:010
1120799182,13cyclictest26882-21kthreadcore07:25:210
1120799182,11cyclictest0-21swapper/008:45:150
1120799182,11cyclictest0-21swapper/008:40:150
1120799181,6cyclictest12-21ksoftirqd/007:40:190
1123199171,9cyclictest35966irq/58-eth008:20:143
1123199171,11cyclictest0-21swapper/308:15:153
1121599177,7cyclictest22845-21irqrtprio07:55:181
1121599172,6cyclictest15645-21cat09:20:121
1121599172,10cyclictest0-21swapper/107:45:131
1121599171,12cyclictest20517-21kthreadcore08:25:211
11215991710,5cyclictest31058-21grep08:00:291
11215991710,5cyclictest30169-21ssh12:09:401
1120799173,12cyclictest28827-21irqcore08:35:160
1120799173,12cyclictest28827-21irqcore08:35:160
11207991711,4cyclictest26888-21apt-get08:00:050
11207991711,4cyclictest14116-21threads08:50:280
11207991711,4cyclictest0-21swapper/009:00:300
1123199168,6cyclictest0-21swapper/309:20:453
1123199162,9cyclictest35966irq/58-eth007:55:143
1123199162,9cyclictest0-21swapper/307:35:133
1123199162,10cyclictest0-21swapper/307:20:133
11231991610,4cyclictest0-21swapper/309:55:173
1121599163,9cyclictest11315-21cat07:10:021
1121599163,10cyclictest29562-21unixbench_multi07:25:301
1121599161,8cyclictest12343-21kthreadcore07:10:201
1121599160,9cyclictest1050-21fschecks_time08:40:161
1123199151,9cyclictest0-21swapper/307:10:143
1123199151,11cyclictest0-21swapper/308:25:173
1123199149,3cyclictest0-21swapper/309:05:273
1123199132,9cyclictest0-21swapper/308:45:153
1123199132,7cyclictest0-21swapper/309:00:133
1123199122,7cyclictest0-21swapper/307:25:223
1123199122,5cyclictest0-21swapper/308:40:143
1123199112,7cyclictest0-21swapper/308:50:133
1123199101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional