You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 23:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri Jun 19, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21526998435,13cyclictest27-21ksoftirqd/208:25:122
2151899725,57cyclictest16131-21ssh11:20:141
2152699662,42cyclictest30401-21ssh11:10:122
21534995939,13cyclictest35966irq/58-eth007:50:123
2150799541,39cyclictest7037-21ssh11:15:140
2150799524,36cyclictest30386-21ssh11:10:110
2150799512,43cyclictest5473-21ssh11:50:130
2153499504,25cyclictest35966irq/58-eth009:20:143
2152699509,31cyclictest0-21swapper/212:10:132
2151899506,4cyclictest131rcu_preempt12:25:121
2153499477,26cyclictest35966irq/58-eth009:10:133
2150799462,4cyclictest131rcu_preempt08:30:150
2152699432,35cyclictest14483-21ssh11:55:132
2152699432,24cyclictest15213-21sh09:50:132
2151899422,27cyclictest13072-21ssh12:30:131
2153499412,23cyclictest35966irq/58-eth007:30:133
2151899412,30cyclictest14499-21ssh11:55:131
21507994118,16cyclictest24291-21ssh09:55:110
2153499404,21cyclictest35966irq/58-eth009:45:123
2151899391,30cyclictest24286-21ssh09:55:121
2150799391,23cyclictest15200-21diskmemload09:50:130
2153499383,20cyclictest35966irq/58-eth009:35:123
2150799381,29cyclictest31332-21sshd07:20:140
2152699372,18cyclictest15057-21rm10:42:322
21534993618,11cyclictest35966irq/58-eth012:25:113
2150799363,6cyclictest131rcu_preempt07:45:150
2150799363,24cyclictest12-21ksoftirqd/008:20:130
21507993621,9cyclictest12-21ksoftirqd/011:34:210
2153499352,17cyclictest35966irq/58-eth008:10:123
2151899352,21cyclictest11442-21ssh09:30:121
21518993515,12cyclictest0-21swapper/110:20:141
21507993510,17cyclictest12-21ksoftirqd/008:25:140
2153499341,20cyclictest35966irq/58-eth009:00:133
2152699346,22cyclictest27-21ksoftirqd/212:01:322
2152699343,23cyclictest0-21swapper/210:49:152
2151899341,24cyclictest22025-21ssh12:35:141
2150799345,18cyclictest12-21ksoftirqd/007:25:090
2150799332,23cyclictest7626-21ssh09:10:130
2153499321,21cyclictest35966irq/58-eth010:05:123
21534993212,10cyclictest0-21swapper/308:55:133
2153499320,21cyclictest35966irq/58-eth009:40:133
2152699322,8cyclictest930-21snmpd11:05:122
2151899322,16cyclictest19080-21ssh10:10:131
2150799322,24cyclictest13733-21ssh10:25:140
21507993212,14cyclictest12-21ksoftirqd/012:15:140
2152699319,17cyclictest27-21ksoftirqd/210:33:322
2150799312,4cyclictest131rcu_preempt09:00:140
2152699305,5cyclictest27-21ksoftirqd/210:58:392
21526993016,12cyclictest261rcuc/212:15:132
21518993012,11cyclictest630-21Async09:36:311
2150799302,3cyclictest131rcu_preempt08:45:140
2150799302,22cyclictest27713-21ssh12:36:330
2153499293,17cyclictest0-21swapper/310:42:333
2152699294,10cyclictest27-21ksoftirqd/211:43:322
2152699293,19cyclictest0-21swapper/211:54:232
2152699293,19cyclictest0-21swapper/211:15:322
2152699292,19cyclictest11466-21ssh09:30:132
2152699291,21cyclictest23497-21ssh11:23:322
2152699290,21cyclictest0-21swapper/209:55:132
2150799292,21cyclictest25503-21ssh10:49:150
21507992916,7cyclictest0-21swapper/009:15:130
2153499283,18cyclictest35966irq/58-eth012:10:143
2153499283,18cyclictest0-21swapper/312:00:333
21534992811,9cyclictest35966irq/58-eth011:50:143
2152699285,16cyclictest27-21ksoftirqd/210:52:142
2152699283,18cyclictest0-21swapper/212:36:322
2152699282,20cyclictest0-21swapper/208:20:152
2152699281,18cyclictest28980-21cut11:45:152
2151899282,20cyclictest28850-21ssh11:45:121
2150799283,2cyclictest131rcu_preempt12:00:120
2150799283,16cyclictest5670-1kworker/u9:010:05:130
2150799282,19cyclictest6180-21ssh09:45:130
21507992812,10cyclictest0-21swapper/010:35:120
2152699275,11cyclictest131rcu_preempt11:38:122
2152699274,16cyclictest27346irq/40-dwc2_hso08:40:142
2152699272,22cyclictest7214-21diskmemload12:05:152
2152699272,19cyclictest0-21swapper/209:00:152
2152699272,15cyclictest28122-21ssh10:15:132
2152699271,22cyclictest29577-21ssh09:40:122
21526992711,4cyclictest27-21ksoftirqd/211:27:092
2151899273,9cyclictest0-21swapper/110:32:321
2151899272,19cyclictest29589-21rm11:09:311
2151899272,18cyclictest8500-21ssh10:40:141
2150799277,8cyclictest0-21swapper/008:00:130
2150799277,8cyclictest0-21swapper/008:00:120
2150799272,19cyclictest28126-21ssh10:15:140
2150799271,19cyclictest20474-21ssh09:35:120
2153499260,16cyclictest35966irq/58-eth010:45:123
2152699268,10cyclictest12229-21apt-get11:00:112
2152699266,11cyclictest27-21ksoftirqd/212:25:272
2152699264,8cyclictest27-21ksoftirqd/210:01:482
2152699264,8cyclictest0-21swapper/209:10:132
2152699264,15cyclictest27346irq/40-dwc2_hso11:30:112
2152699262,12cyclictest854-21sshd07:20:132
2152699262,10cyclictest0-21swapper/208:10:152
2151899262,17cyclictest25126-21ssh11:25:111
2151899260,18cyclictest73-21kswapd010:55:111
2150799262,19cyclictest8424-21ssh12:09:310
2153499257,10cyclictest35966irq/58-eth009:55:113
2153499256,16cyclictest35966irq/58-eth011:35:123
2153499253,11cyclictest0-21swapper/311:03:323
2152699256,8cyclictest27-21ksoftirqd/212:32:542
2152699252,18cyclictest20249-21ssh10:27:302
2152699252,12cyclictest16678-21ssh09:15:142
21526992513,9cyclictest12346-21ssh09:47:052
21526992512,4cyclictest27-21ksoftirqd/210:23:592
2150799253,17cyclictest30025-21ssh11:25:330
21507992514,3cyclictest12-21ksoftirqd/011:56:450
2153499244,12cyclictest35966irq/58-eth011:15:133
2153499243,13cyclictest0-21swapper/309:30:143
21534992414,7cyclictest35966irq/58-eth010:20:143
2152699248,7cyclictest27-21ksoftirqd/210:08:142
2152699244,7cyclictest27-21ksoftirqd/210:38:002
2152699243,8cyclictest27-21ksoftirqd/212:21:202
2152699243,17cyclictest23743-21kthreadcore09:35:222
2152699242,13cyclictest0-21swapper/208:00:132
2152699242,13cyclictest0-21swapper/208:00:122
21526992413,7cyclictest261rcuc/208:40:022
2152699241,14cyclictest27346irq/40-dwc2_hso09:05:132
2152699240,21cyclictest20697-21grep08:50:152
2151899245,11cyclictest21-21ksoftirqd/109:35:121
2151899243,11cyclictest664-21ntpd10:05:121
2151899241,18cyclictest983-21ssh10:00:131
2150799249,8cyclictest12-21ksoftirqd/008:40:270
21507992414,3cyclictest12-21ksoftirqd/007:45:000
2153499239,8cyclictest35966irq/58-eth012:05:153
21534992315,5cyclictest35966irq/58-eth010:35:123
21534992311,10cyclictest321rcuc/311:50:023
21526992313,8cyclictest0-21swapper/209:28:262
2152699231,15cyclictest27346irq/40-dwc2_hso07:25:112
2152699231,14cyclictest0-21swapper/210:12:092
2151899232,16cyclictest15132-21ssh11:19:191
2151899232,15cyclictest5954-21ssh10:37:311
2151899231,3cyclictest131rcu_preempt09:20:141
21518992312,8cyclictest0-21swapper/107:35:141
21518992312,8cyclictest0-21swapper/107:35:131
2151899231,17cyclictest28341-21cut10:15:171
2150799237,12cyclictest12-21ksoftirqd/010:33:310
2150799231,18cyclictest31024-21ssh12:20:230
2150799231,15cyclictest12227-21ssh11:00:110
21507992310,3cyclictest12-21ksoftirqd/009:30:140
2153499223,13cyclictest35966irq/58-eth010:25:143
2152699224,9cyclictest29226-21tail08:55:282
2152699224,3cyclictest131rcu_preempt07:10:162
2152699224,3cyclictest131rcu_preempt07:10:152
2152699224,12cyclictest27-21ksoftirqd/208:05:262
2152699222,16cyclictest23074-21unixbench_singl08:15:272
2152699222,13cyclictest0-21swapper/207:30:122
2152699222,12cyclictest28559-21tr09:20:222
2152699221,14cyclictest27346irq/40-dwc2_hso08:45:132
2151899222,15cyclictest19641-21ssh11:03:331
2151899221,18cyclictest5551-21ls11:50:151
2151899221,15cyclictest6209-21cut08:35:161
2151899221,15cyclictest167-21systemd-network12:05:161
2150799222,14cyclictest11217-21ssh10:59:180
21507992213,7cyclictest12-21ksoftirqd/009:25:130
21507992210,3cyclictest12-21ksoftirqd/010:23:430
21507992210,10cyclictest131rcu_preempt10:01:300
2150799220,19cyclictest17193-21ls10:45:010
2153499215,12cyclictest35966irq/58-eth012:15:133
21534992114,4cyclictest0-21swapper/309:50:203
2152699216,12cyclictest27-21ksoftirqd/207:50:282
2152699215,5cyclictest28366-21ps07:15:212
2152699212,13cyclictest27346irq/40-dwc2_hso07:35:142
2152699212,13cyclictest27346irq/40-dwc2_hso07:35:132
2152699210,17cyclictest1802-21irqrtprio08:30:222
2151899212,14cyclictest15939-21ssh09:50:201
2151899212,13cyclictest26582-21ssh12:19:331
2150799219,3cyclictest12-21ksoftirqd/008:10:290
2150799218,4cyclictest12-21ksoftirqd/007:30:300
21507992116,3cyclictest0-21swapper/010:10:250
21507992112,6cyclictest0-21swapper/009:05:300
21507992111,7cyclictest0-21swapper/009:20:250
2150799211,14cyclictest21279-21ssh11:05:130
2153499208,8cyclictest35966irq/58-eth010:30:243
2153499208,5cyclictest0-21swapper/310:15:213
2153499207,10cyclictest0-21swapper/311:09:083
2153499206,10cyclictest35966irq/58-eth011:20:133
2153499204,9cyclictest0-21swapper/309:15:143
2153499203,12cyclictest35966irq/58-eth008:05:113
2153499202,12cyclictest0-21swapper/310:59:183
21534992011,7cyclictest0-21swapper/310:50:093
2153499201,12cyclictest35966irq/58-eth012:20:123
2153499201,11cyclictest35966irq/58-eth007:45:133
2152699203,11cyclictest0-21swapper/207:45:142
21526992011,7cyclictest0-21swapper/207:55:172
2152699201,10cyclictest18631-21irqrtprio07:40:182
2151899209,8cyclictest31076-21cron07:20:011
2151899209,3cyclictest131rcu_preempt10:50:191
2151899206,9cyclictest21-21ksoftirqd/112:20:131
2150799209,4cyclictest131rcu_preempt11:22:590
2150799209,3cyclictest12-21ksoftirqd/007:55:200
2150799208,3cyclictest131rcu_preempt12:12:160
2150799207,3cyclictest12-21ksoftirqd/007:15:010
2150799207,3cyclictest12-21ksoftirqd/007:15:000
2150799207,11cyclictest131rcu_preempt12:26:230
2150799207,11cyclictest131rcu_preempt11:35:400
21507992014,3cyclictest12-21ksoftirqd/009:43:570
2150799201,16cyclictest20131-21ls11:40:150
2153499198,9cyclictest0-21swapper/310:10:163
2153499198,8cyclictest0-21swapper/311:30:193
2153499193,11cyclictest0-21swapper/312:39:303
2153499193,11cyclictest0-21swapper/311:55:313
2153499193,11cyclictest0-21swapper/311:25:333
2153499193,11cyclictest0-21swapper/311:14:313
2153499191,10cyclictest0-21swapper/307:40:153
2151899198,7cyclictest0-21swapper/112:00:211
2151899191,9cyclictest16378-21tr08:10:201
2151899191,3cyclictest131rcu_preempt11:10:141
2151899191,15cyclictest25397-21apt-get08:55:111
2151899191,14cyclictest32254-21cron07:55:021
2151899191,13cyclictest5784-21cron09:45:021
2151899191,12cyclictest20173-21latency09:15:231
2151899191,10cyclictest16500-21kthreadcore10:25:211
21518991910,6cyclictest22673-21cron07:45:011
2151899190,15cyclictest8436-21ls07:30:011
2150799194,4cyclictest131rcu_preempt08:35:170
2150799190,13cyclictest28743-21ls08:55:230
2153499182,10cyclictest0-21swapper/308:45:143
2153499182,10cyclictest0-21swapper/308:20:143
2153499182,10cyclictest0-21swapper/308:15:133
2153499182,10cyclictest0-21swapper/307:20:143
2153499181,9cyclictest35966irq/58-eth012:30:143
2153499181,12cyclictest0-21swapper/308:35:013
2153499181,11cyclictest0-21swapper/308:40:033
2152699181,8cyclictest21558-21latency_hist07:10:012
2151899189,6cyclictest11144-21kthreadcore08:05:211
2151899188,8cyclictest20590-21apt-get08:50:131
2151899188,3cyclictest0-21swapper/112:10:361
2151899186,9cyclictest9491-21cut09:45:221
2151899182,11cyclictest2385-21ssh09:25:121
2151899181,7cyclictest26578-21wc07:45:281
2151899181,14cyclictest23470-21kthreadcore07:10:221
2151899181,14cyclictest23470-21kthreadcore07:10:211
2151899181,13cyclictest5363-21sh11:30:241
2151899181,10cyclictest28626-21latency_hist08:25:011
21518991810,5cyclictest22010-21usage_EthcTask10:45:291
2150799188,2cyclictest12-21ksoftirqd/007:50:280
2150799185,11cyclictest131rcu_preempt10:51:490
21507991811,4cyclictest12-21ksoftirqd/011:47:350
2153499179,6cyclictest0-21swapper/310:04:483
2153499171,13cyclictest0-21swapper/309:25:293
2153499171,11cyclictest0-21swapper/311:40:133
2151899177,7cyclictest20410-21sed11:40:181
2151899174,10cyclictest9148-21kernelversion07:30:191
2151899173,10cyclictest5936-21latency_hist08:35:021
2151899172,12cyclictest2907-21hwlatdetect09:05:171
2151899172,10cyclictest0-21swapper/108:40:061
2151899171,9cyclictest5301-21irqcore08:00:171
2151899171,9cyclictest5301-21irqcore08:00:161
2151899171,12cyclictest11249-21ssh09:10:251
2151899170,12cyclictest2184-21ls07:20:231
2150799177,4cyclictest131rcu_preempt07:15:140
2150799177,3cyclictest12-21ksoftirqd/007:35:160
2150799177,3cyclictest12-21ksoftirqd/007:35:150
2150799175,3cyclictest12-21ksoftirqd/008:55:010
2150799171,3cyclictest0-21swapper/008:15:160
2150799171,13cyclictest16634-21nfs_client12:30:240
2153499161,12cyclictest0-21swapper/307:35:173
2153499161,12cyclictest0-21swapper/307:35:163
2153499161,11cyclictest0-21swapper/309:10:023
2153499161,10cyclictest0-21swapper/307:25:103
2151899168,6cyclictest2868-21sshd07:55:211
2151899162,11cyclictest20376-21kthreadcore08:15:191
2151899161,13cyclictest29374-21irqrtprio08:25:181
2151899161,11cyclictest15840-21sort08:45:131
2153499151,10cyclictest35966irq/58-eth008:35:133
2151899152,11cyclictest30795-21sed09:00:201
2151899151,11cyclictest21568-21latency_hist07:10:011
2150799151,11cyclictest11825-21kthreadcore08:05:200
2153499141,11cyclictest0-21swapper/307:55:213
2150799141,10cyclictest21514-21wc07:10:020
2153499139,2cyclictest33-21ksoftirqd/307:12:503
2153499139,2cyclictest33-21ksoftirqd/307:12:493
2153499131,4cyclictest0-21swapper/308:00:273
2153499131,4cyclictest0-21swapper/308:00:263
2153499131,10cyclictest0-21swapper/308:25:293
2153499111,8cyclictest0-21swapper/308:50:153
2153499111,7cyclictest0-21swapper/307:15:173
2153499111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional