You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 01:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun May 31, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1185199879,59cyclictest23945-21ssh10:00:131
1185199775,59cyclictest29968-21ssh11:15:121
1185199614,44cyclictest4412-21ssh10:25:121
1185799555,47cyclictest31505-21ssh10:40:122
1185799553,34cyclictest930-21snmpd09:10:142
1184399545,7cyclictest131rcu_preempt10:05:140
1185799531,5cyclictest0-21swapper/207:20:112
1184399502,8cyclictest131rcu_preempt10:40:120
1185199491,35cyclictest1172-21ssh11:35:141
1185799484,39cyclictest0-21swapper/210:05:142
1185799482,5cyclictest0-21swapper/208:00:132
1185799471,4cyclictest0-21swapper/210:35:142
1184399472,39cyclictest0-21swapper/012:25:140
1184399461,41cyclictest0-21swapper/007:20:110
1184399461,2cyclictest0-21swapper/009:30:110
1185799454,3cyclictest22961-21apt-get12:05:142
1185799454,2cyclictest29910-21diskmemload12:30:112
1185799454,2cyclictest29910-21diskmemload12:30:112
1185799452,5cyclictest0-21swapper/208:10:112
1185799452,5cyclictest0-21swapper/207:15:142
11851994528,11cyclictest10108-21ssh11:40:121
1184399452,2cyclictest0-21swapper/007:40:130
1185799441,5cyclictest0-21swapper/212:25:142
1184399441,39cyclictest0-21swapper/007:25:120
1184399441,37cyclictest0-21swapper/011:30:130
1184399433,33cyclictest0-21swapper/009:40:110
1184399431,35cyclictest17679-21ssh12:20:120
1185799423,3cyclictest0-21swapper/211:30:132
1185799421,8cyclictest0-21swapper/209:55:132
1185799421,4cyclictest0-21swapper/208:55:142
1185799421,4cyclictest0-21swapper/207:25:122
1185799421,30cyclictest0-21swapper/211:45:132
1184399421,2cyclictest14732-21apt-get09:55:140
1184399421,10cyclictest4703-21ssh11:55:110
1186799413,27cyclictest35966irq/58-eth010:10:113
1185799413,2cyclictest0-21swapper/207:10:122
1185799411,7cyclictest0-21swapper/210:20:122
1185799411,4cyclictest0-21swapper/209:40:122
1185799410,4cyclictest4724-21apt-get11:55:132
1185199412,29cyclictest22510-21ssh10:35:131
1184399412,3cyclictest12129-21apt-get12:35:130
1184399410,7cyclictest19120-21ssh09:39:350
1185799402,4cyclictest0-21swapper/208:20:122
1185799401,5cyclictest0-21swapper/207:35:142
1184399402,5cyclictest0-21swapper/008:00:130
1184399402,34cyclictest0-21swapper/011:45:130
1185799393,5cyclictest28164-21kthreadcore09:05:212
1185799391,5cyclictest0-21swapper/207:45:142
1185799391,5cyclictest0-21swapper/207:45:142
1185799391,4cyclictest18706-21apt-get10:15:162
1185799390,9cyclictest3355-21cut11:00:172
1184399391,3cyclictest27754-21apt-get10:20:120
1184399391,34cyclictest0-21swapper/009:05:120
11843993911,22cyclictest12-21ksoftirqd/008:45:130
1186799383,22cyclictest35966irq/58-eth008:50:143
1185199382,26cyclictest13777-21ssh12:00:101
1184399381,8cyclictest9679-21ssh11:40:000
1184399381,3cyclictest0-21swapper/007:10:140
1184399380,2cyclictest32555-21apt-get08:05:150
1186799371,10cyclictest0-21swapper/310:40:123
1185799371,6cyclictest0-21swapper/211:10:112
1185799371,4cyclictest0-21swapper/211:40:122
1185799371,33cyclictest0-21swapper/208:50:232
1185799370,8cyclictest18087-21ssh09:20:222
1185799370,2cyclictest0-21swapper/207:30:122
1186799362,23cyclictest35966irq/58-eth009:05:133
1185799365,3cyclictest7462-21rm12:14:112
1185799361,9cyclictest14482-21ssh10:10:302
1185799361,8cyclictest12623-21ssh10:45:322
1185799361,7cyclictest28626-21irqcore11:50:182
1185799361,6cyclictest6273-21latency_hist11:20:022
1185799361,6cyclictest27700-21kthreadcore10:55:212
1185799361,2cyclictest0-21swapper/210:30:262
1185799360,6cyclictest0-21swapper/212:35:142
1185799360,5cyclictest17664-21ssh12:20:122
11851993618,10cyclictest0-21swapper/110:05:131
1185199360,30cyclictest28135-21ssh11:50:121
1184399361,3cyclictest22493-21apt-get10:35:130
1184399361,2cyclictest9721-21apt-get08:15:130
1184399361,2cyclictest20875-21apt-get11:10:140
1184399361,2cyclictest0-21swapper/011:50:120
1184399361,2cyclictest0-21swapper/010:55:130
1184399361,26cyclictest25147-21ssh09:25:110
11867993518,12cyclictest35966irq/58-eth012:25:153
1185799351,8cyclictest21985-21grep07:50:282
1185799351,7cyclictest8550-21unixbench_singl10:25:302
1185799351,6cyclictest13757-21acpi12:00:102
1185799351,6cyclictest12770-21unixbench_singl12:15:282
1185799351,3cyclictest0-21swapper/207:40:052
1185799351,31cyclictest15244-21latency_hist11:25:022
1185799351,31cyclictest15244-21latency_hist11:25:012
1185799351,30cyclictest11775-21/usr/sbin/munin11:05:152
1185799350,2cyclictest0-21swapper/209:15:122
1185199355,5cyclictest2767-21ssh11:00:131
1184399353,24cyclictest4431-21ssh10:25:140
1184399351,3cyclictest0-21swapper/012:05:160
1184399351,3cyclictest0-21swapper/010:30:130
1184399351,2cyclictest3082-21apt-get12:30:120
1184399351,2cyclictest3082-21apt-get12:30:110
1184399351,2cyclictest13768-21apt-get12:00:110
1184399351,2cyclictest0-21swapper/011:00:130
1184399351,22cyclictest29910-21diskmemload09:23:460
1184399351,10cyclictest2737-21tr08:40:200
1184399350,2cyclictest15614-21apt-get11:25:140
1186799342,17cyclictest35966irq/58-eth008:25:133
1185799348,23cyclictest28778-21cat10:00:322
1185799342,10cyclictest0-21swapper/209:00:142
1185799341,9cyclictest2656-21kthreadcore08:05:222
1185799341,8cyclictest1295-21/usr/sbin/munin08:40:232
1185799341,6cyclictest28024-21/usr/sbin/munin08:30:322
1185799341,6cyclictest21083-21timerandwakeup10:50:292
1185799341,6cyclictest20369-21ssh11:25:312
1185799340,8cyclictest9277-21missed_timers09:50:232
1184399343,28cyclictest9585-21mailstats09:50:270
1184399343,28cyclictest17140-21irqcore07:15:170
1184399342,27cyclictest10097-21apt-get11:40:110
1184399341,7cyclictest282566ssh09:10:360
1184399341,5cyclictest24127-21apt-get08:30:150
1185799331,8cyclictest11968-21latency_hist07:10:022
1185799331,6cyclictest23908-21idleruntime-cro08:30:022
1185799331,6cyclictest0-21swapper/209:45:132
1185799331,29cyclictest29576-21uname08:35:192
1185799331,10cyclictest4834-21ntpq11:35:272
1185199334,21cyclictest29910-21diskmemload12:05:161
1184399334,21cyclictest0-21swapper/008:10:120
1184399332,28cyclictest17283-21kthreadcore08:55:200
1184399331,9cyclictest715966ssh12:13:480
1184399331,4cyclictest4340-21ssh11:17:470
1184399331,3cyclictest18108-21apt-get07:50:120
1184399331,3cyclictest0-21swapper/008:20:140
1184399331,2cyclictest0-21swapper/009:45:130
1184399330,5cyclictest11816-21sh11:05:130
1185799321,7cyclictest30205-21ssh09:25:442
1185799321,5cyclictest5495-21ps09:30:252
1184399324,25cyclictest8339-21cat07:40:010
1184399321,3cyclictest0-21swapper/007:55:130
1184399321,2cyclictest0-21swapper/010:15:150
1184399321,10cyclictest20724-21/usr/sbin/munin09:00:160
1186799313,20cyclictest0-21swapper/309:15:133
11867993117,8cyclictest35966irq/58-eth009:25:123
1185799311,8cyclictest8765-21tr08:45:222
1185799311,8cyclictest11168-21cut08:15:212
1185799311,6cyclictest26387-21ntp_states07:55:262
1185799311,2cyclictest0-21swapper/209:35:122
1185199312,21cyclictest8554-21ssh12:15:121
1184399311,8cyclictest28583-21cat10:00:300
1184399311,7cyclictest31516-21irqcore07:30:180
1184399311,22cyclictest6961-21ssh09:15:120
1184399310,4cyclictest8072-21apt-get10:45:140
1186799302,7cyclictest35966irq/58-eth010:05:143
1185199302,19cyclictest6612-21ssh11:20:111
1185199302,19cyclictest6612-21ssh11:20:101
11851993017,7cyclictest201rcuc/107:35:151
1184399301,9cyclictest22166-21ssh10:50:450
1184399301,7cyclictest21567-21kthreadcore08:25:210
1184399301,6cyclictest11116-21kthreadcore12:15:210
1184399301,2cyclictest0-21swapper/010:10:100
1184399301,2cyclictest0-21swapper/008:50:130
11851992914,4cyclictest21-21ksoftirqd/109:25:201
1185199291,18cyclictest0-21swapper/111:25:131
1184399291,8cyclictest17314-21cat07:45:310
1184399291,8cyclictest17314-21cat07:45:300
1184399291,6cyclictest675-21cat08:35:290
1184399291,12cyclictest10229-21ssh11:20:230
1184399291,12cyclictest10229-21ssh11:20:220
1186799282,6cyclictest0-21swapper/310:35:143
1185199282,19cyclictest15712-21ntp_kernel_err12:35:261
11851992817,6cyclictest201rcuc/111:45:131
1186799272,9cyclictest0-21swapper/309:10:143
1186799272,8cyclictest0-21swapper/312:30:123
1186799272,8cyclictest0-21swapper/312:30:123
1186799272,8cyclictest0-21swapper/307:55:133
11851992713,10cyclictest11829-21cyclictest10:15:131
1185199271,20cyclictest11828-21ssh11:05:141
1184399271,6cyclictest11849-21cat07:10:020
1186799269,4cyclictest0-21swapper/310:53:263
1186799268,5cyclictest35966irq/58-eth011:26:483
1186799262,9cyclictest0-21swapper/310:00:213
1186799262,8cyclictest0-21swapper/310:25:143
1186799261,11cyclictest0-21swapper/308:15:203
11867992610,4cyclictest0-21swapper/311:43:053
1185199263,14cyclictest20889-21ssh11:10:121
11851992613,4cyclictest21-21ksoftirqd/112:30:001
1185199261,18cyclictest0-21swapper/108:05:131
11851992610,3cyclictest131rcu_preempt08:15:141
1186799259,3cyclictest0-21swapper/312:15:343
1186799259,13cyclictest0-21swapper/311:45:533
1186799258,6cyclictest36066irq/59-eth010:30:363
1186799258,3cyclictest0-21swapper/312:10:073
1186799255,15cyclictest0-21swapper/309:55:123
1186799253,6cyclictest0-21swapper/309:21:363
1186799253,11cyclictest0-21swapper/309:30:103
1186799252,8cyclictest0-21swapper/311:50:183
1186799252,12cyclictest35966irq/58-eth011:30:123
1186799249,3cyclictest0-21swapper/311:59:443
1186799249,3cyclictest0-21swapper/311:23:413
1186799249,3cyclictest0-21swapper/311:23:403
1186799248,6cyclictest0-21swapper/312:08:033
1186799247,7cyclictest0-21swapper/310:49:263
1186799242,7cyclictest0-21swapper/312:00:153
1186799241,8cyclictest35966irq/58-eth012:35:263
1186799241,7cyclictest0-21swapper/310:20:163
1185199249,5cyclictest21-21ksoftirqd/110:50:261
1185199248,8cyclictest21-21ksoftirqd/109:50:301
1185199246,3cyclictest131rcu_preempt08:50:141
1185199243,15cyclictest3093-21ssh12:30:121
1185199243,15cyclictest3093-21ssh12:30:111
1185199243,14cyclictest13488-21ssh10:30:141
1186799233,9cyclictest0-21swapper/309:35:153
1186799233,8cyclictest0-21swapper/311:05:283
1186799232,7cyclictest0-21swapper/309:40:193
1186799232,7cyclictest0-21swapper/307:20:133
1186799232,6cyclictest0-21swapper/311:00:133
1185199239,8cyclictest21-21ksoftirqd/111:58:141
1185199236,3cyclictest131rcu_preempt09:10:121
1185199234,11cyclictest31990-21ssh12:10:121
1185199233,14cyclictest16073-21ssh09:20:141
1186799227,6cyclictest0-21swapper/309:51:283
1186799222,6cyclictest0-21swapper/312:20:123
1186799222,6cyclictest0-21swapper/311:10:143
1186799222,12cyclictest0-21swapper/307:25:133
1185199229,3cyclictest131rcu_preempt09:45:341
1185199222,14cyclictest8095-21sh10:45:141
1186799212,6cyclictest0-21swapper/307:30:263
1185199218,4cyclictest21-21ksoftirqd/112:24:401
1185199217,8cyclictest21-21ksoftirqd/109:58:421
1185199216,3cyclictest131rcu_preempt07:35:011
1185199215,4cyclictest131rcu_preempt07:40:221
1185199213,7cyclictest21-21ksoftirqd/109:41:231
1185199211,17cyclictest5049-21cut08:10:151
1185199211,17cyclictest10809-21apt-get09:35:131
1185199211,16cyclictest27897-21ssh10:20:161
1185199210,8cyclictest30908-21usage_EthcTask07:25:311
1186799203,13cyclictest0-21swapper/309:45:153
1185199209,3cyclictest21-21ksoftirqd/110:56:271
1185199209,3cyclictest21-21ksoftirqd/110:13:211
1185199209,3cyclictest21-21ksoftirqd/109:05:141
1185199203,7cyclictest21-21ksoftirqd/109:00:271
1185199203,4cyclictest131rcu_preempt07:50:281
1185199202,11cyclictest21-21ksoftirqd/108:40:141
1185199201,15cyclictest15844-21latency_hist09:20:031
1185199201,10cyclictest29276-21cut08:00:211
11851992010,5cyclictest21-21ksoftirqd/108:25:241
1185199200,13cyclictest12143-21cut07:10:161
1186799199,7cyclictest0-21swapper/310:55:243
11867991913,4cyclictest36066irq/59-eth011:40:033
11867991911,5cyclictest35966irq/58-eth010:15:153
1186799191,11cyclictest0-21swapper/308:45:023
1185199195,4cyclictest21-21ksoftirqd/108:40:011
1185199192,3cyclictest131rcu_preempt07:15:181
11851991914,3cyclictest0-21swapper/110:40:271
1185199191,15cyclictest24558-21apt-get11:30:131
11851991911,5cyclictest0-21swapper/109:30:091
1186799182,10cyclictest0-21swapper/308:20:143
1186799182,10cyclictest0-21swapper/307:40:053
1186799182,10cyclictest0-21swapper/307:10:143
11867991810,5cyclictest35966irq/58-eth011:15:123
1185199187,3cyclictest131rcu_preempt08:30:211
1185199186,4cyclictest21-21ksoftirqd/107:20:251
1185199184,11cyclictest18358-21tail08:20:291
11851991813,3cyclictest0-21swapper/107:55:251
11851991810,5cyclictest0-21swapper/107:45:191
11851991810,5cyclictest0-21swapper/107:45:181
1186799173,7cyclictest0-21swapper/308:05:123
1186799172,10cyclictest0-21swapper/308:55:143
1186799172,10cyclictest0-21swapper/307:45:163
1186799172,10cyclictest0-21swapper/307:45:153
11851991712,3cyclictest0-21swapper/108:55:261
1186799162,9cyclictest0-21swapper/308:10:133
1186799162,9cyclictest0-21swapper/307:15:133
1186799162,11cyclictest0-21swapper/308:00:143
1185199164,3cyclictest131rcu_preempt07:10:011
1185199162,11cyclictest6303-21fschecks_time08:45:171
1186799152,9cyclictest0-21swapper/308:45:133
1186799141,9cyclictest0-21swapper/309:00:153
1186799138,3cyclictest0-21swapper/308:34:033
1186799135,6cyclictest35966irq/58-eth008:35:123
1186799131,9cyclictest0-21swapper/307:55:033
1186799122,7cyclictest0-21swapper/307:40:003
1186799111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional