You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 19:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 29, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7502995813,8cyclictest12-21ksoftirqd/010:20:140
7502995813,8cyclictest12-21ksoftirqd/010:20:140
751099564,40cyclictest9972-21ssh11:25:121
751599499,12cyclictest131rcu_preempt12:25:172
751599493,36cyclictest31561-21ssh11:55:132
750299494,39cyclictest18934-21ssh11:30:130
752399483,26cyclictest35966irq/58-eth010:10:113
7515994411,19cyclictest630-21Async09:36:312
7510994213,22cyclictest21-21ksoftirqd/107:35:131
751099411,28cyclictest31580-21ssh11:55:141
7502994118,9cyclictest7494-21cyclictest09:15:140
751599404,24cyclictest27-21ksoftirqd/210:00:132
751599401,32cyclictest22565-21ssh11:50:122
7502994028,7cyclictest12-21ksoftirqd/011:50:120
751099394,25cyclictest25762-21diskmemload12:05:131
751599372,18cyclictest18949-21ssh11:30:142
752399363,24cyclictest35966irq/58-eth009:55:143
750299361,27cyclictest11841-21ssh09:20:120
750299361,27cyclictest11841-21ssh09:20:120
751599353,11cyclictest0-21swapper/207:50:132
751599353,11cyclictest0-21swapper/207:50:132
752399343,14cyclictest35966irq/58-eth009:05:143
752399342,15cyclictest35966irq/58-eth012:05:123
751599342,15cyclictest29932-21ssh12:30:132
7502993320,8cyclictest151rcuc/007:10:130
7523993221,7cyclictest35966irq/58-eth011:10:143
751099322,2cyclictest131rcu_preempt12:35:141
751099321,28cyclictest6408-21ssh09:35:141
751099321,24cyclictest990-21ssh09:50:131
751599313,21cyclictest17276-21ssh12:05:122
751599312,23cyclictest8332-1kworker/u9:211:25:082
751099313,25cyclictest29812-21ssh09:30:131
751099311,20cyclictest31630-21ssh10:25:131
750299315,4cyclictest131rcu_preempt07:45:130
750299312,23cyclictest11782-21ssh10:50:130
750299312,23cyclictest11782-21ssh10:50:130
751099302,25cyclictest20769-21ssh09:25:131
751099302,21cyclictest20745-21ssh10:55:141
750299301,3cyclictest0-21swapper/010:10:110
751599294,19cyclictest0-21swapper/207:35:142
751599293,11cyclictest0-21swapper/208:35:152
751599292,16cyclictest0-21swapper/209:00:142
7515992914,10cyclictest0-21swapper/209:45:132
7515992914,10cyclictest0-21swapper/209:45:122
751099293,22cyclictest927-21ssh11:20:121
751099293,22cyclictest927-21ssh11:20:121
750299292,4cyclictest0-21swapper/007:55:140
7502992919,6cyclictest25762-21diskmemload09:50:150
752399284,16cyclictest35966irq/58-eth010:20:153
752399284,16cyclictest35966irq/58-eth010:20:153
751599284,4cyclictest22588-21apt-get10:20:142
751599284,4cyclictest22588-21apt-get10:20:142
751599282,18cyclictest8219-21sh12:00:122
751599281,20cyclictest26301-21ssh12:10:132
751599281,18cyclictest27346irq/40-dwc2_hso08:05:152
751099282,18cyclictest11923-21ssh12:20:121
751099281,21cyclictest19006-21rm10:00:141
7502992822,3cyclictest0-21swapper/009:10:120
750299282,21cyclictest28072-21ssh10:05:160
752399279,7cyclictest35966irq/58-eth008:55:133
752399272,16cyclictest35966irq/58-eth012:15:133
751599276,4cyclictest0-21swapper/211:35:132
751599275,14cyclictest0-21swapper/212:20:132
751599272,10cyclictest6424-21ssh12:35:082
751599271,21cyclictest1016-21diskmemload09:50:152
751599271,17cyclictest0-21swapper/210:55:122
751599271,14cyclictest28955-21ssh11:15:332
751099272,18cyclictest18954-21ssh11:30:141
750299276,5cyclictest0-21swapper/007:15:130
750299276,10cyclictest12-21ksoftirqd/011:05:130
750299274,2cyclictest131rcu_preempt12:15:140
750299271,23cyclictest6552-21ssh12:35:150
750299271,19cyclictest20780-21ssh09:25:140
752399263,13cyclictest35966irq/58-eth011:15:153
7523992611,8cyclictest0-21swapper/310:45:153
751599262,9cyclictest27783-21sh09:28:082
751599262,9cyclictest261rcuc/211:47:082
751599262,18cyclictest0-21swapper/208:20:142
751599261,20cyclictest9938-21ssh12:18:092
751599261,20cyclictest26167-21ssh09:10:142
7510992619,3cyclictest9992-21ssh09:55:151
751099261,17cyclictest844-21munin-node09:00:151
750299264,7cyclictest12-21ksoftirqd/008:50:150
750299264,18cyclictest8225-21ssh12:00:120
750299263,19cyclictest24371-21ssh09:45:130
750299263,19cyclictest24371-21ssh09:45:130
750299262,8cyclictest0-21swapper/009:40:120
750299262,15cyclictest12-21ksoftirqd/008:55:140
750299261,18cyclictest12-21ksoftirqd/010:15:140
752399257,12cyclictest0-21swapper/307:40:133
752399253,11cyclictest0-21swapper/309:28:083
751599254,15cyclictest27346irq/40-dwc2_hso10:35:152
751599253,3cyclictest0-21swapper/208:45:152
751599252,8cyclictest17840-21ssh09:59:082
751599251,14cyclictest961-21ssh11:20:142
751599251,14cyclictest961-21ssh11:20:142
7510992520,3cyclictest21-21ksoftirqd/111:10:081
750299255,16cyclictest12-21ksoftirqd/007:30:130
750299254,5cyclictest12-21ksoftirqd/010:55:140
750299253,14cyclictest4616-21ssh11:40:130
750299252,18cyclictest31774-21apt-get07:35:130
750299251,20cyclictest9993-21sh11:25:130
7502992511,11cyclictest12-21ksoftirqd/011:10:140
752399243,15cyclictest35966irq/58-eth009:15:143
752399243,15cyclictest0-21swapper/310:25:153
752399242,17cyclictest35966irq/58-eth008:05:133
752399242,13cyclictest35966irq/58-eth011:25:133
752399241,13cyclictest35966irq/58-eth012:00:113
751599243,15cyclictest27346irq/40-dwc2_hso08:10:142
751599242,16cyclictest27346irq/40-dwc2_hso10:45:142
751599240,16cyclictest12753-21ls07:45:282
751099244,14cyclictest24385-21ssh11:15:151
751099242,19cyclictest26174-21ssh10:40:121
751099241,7cyclictest131rcu_preempt11:48:081
751099241,6cyclictest27778-21kthreadcore07:30:181
751099241,17cyclictest11870-21ssh09:20:141
751099241,17cyclictest11870-21ssh09:20:141
750299244,15cyclictest630-21Async09:36:310
750299242,18cyclictest29777-21sh11:00:150
7502992413,8cyclictest12-21ksoftirqd/008:10:270
7502992412,3cyclictest12-21ksoftirqd/010:00:140
750299241,18cyclictest29821-21sh09:30:130
7502992411,4cyclictest31563-21apt-get11:55:130
7502992411,3cyclictest12-21ksoftirqd/010:35:270
752399233,14cyclictest35966irq/58-eth012:20:143
752399233,13cyclictest0-21swapper/311:48:083
752399233,13cyclictest0-21swapper/307:35:053
751599234,10cyclictest27-21ksoftirqd/209:40:122
751599232,14cyclictest27346irq/40-dwc2_hso07:20:142
7515992312,8cyclictest0-21swapper/209:20:292
7515992312,8cyclictest0-21swapper/209:20:292
751599231,19cyclictest6965-21kthreadcore08:15:192
7515992311,9cyclictest0-21swapper/209:15:262
751599231,18cyclictest692-21kthreadcore09:30:222
751599231,10cyclictest31252-21kthreadcore10:05:252
751099231,6cyclictest4561-21apt-get10:10:121
7510992314,6cyclictest21-21ksoftirqd/112:15:131
7510992313,7cyclictest21-21ksoftirqd/110:05:161
7510992312,9cyclictest201rcuc/112:25:161
750299239,4cyclictest12-21ksoftirqd/009:55:200
750299232,16cyclictest2782-21apt-get10:45:140
750299230,3cyclictest131rcu_preempt08:05:010
752399229,9cyclictest35966irq/58-eth011:55:143
752399223,13cyclictest0-21swapper/311:54:083
751599222,6cyclictest0-21swapper/211:05:082
751599221,11cyclictest14920-21ssh10:50:242
751599221,11cyclictest14920-21ssh10:50:242
7515992210,9cyclictest2360-21ssh11:00:442
751099223,16cyclictest201rcuc/107:20:141
751099223,12cyclictest131rcu_preempt11:05:011
751099222,18cyclictest21431-21ssh10:19:081
751099221,3cyclictest0-21swapper/107:55:201
751099221,17cyclictest22621-21sh10:20:151
751099221,17cyclictest22621-21sh10:20:151
751099221,14cyclictest10567-21grep08:20:181
751099221,10cyclictest8293-21ntp_offset11:40:261
7510992210,8cyclictest7494-21cyclictest12:10:111
750299229,6cyclictest12-21ksoftirqd/012:08:270
750299221,16cyclictest21262-21ls09:05:150
7502992211,3cyclictest12-21ksoftirqd/007:20:230
7502992211,2cyclictest12-21ksoftirqd/011:15:140
752399219,9cyclictest0-21swapper/311:23:033
752399219,9cyclictest0-21swapper/311:23:033
751599219,4cyclictest8225-21ssh10:30:132
751599215,13cyclictest5315-21kthreadcore10:10:182
751599211,15cyclictest6638-21cut09:35:182
751599211,13cyclictest17173-21kthreadcore08:25:222
7515992110,9cyclictest15290-21ssh11:10:132
751099219,4cyclictest131rcu_preempt10:45:151
751099216,5cyclictest0-21swapper/111:35:301
751099211,17cyclictest525-21ls08:10:021
750299219,3cyclictest12-21ksoftirqd/012:30:290
750299219,3cyclictest12-21ksoftirqd/012:10:170
750299219,3cyclictest12-21ksoftirqd/011:46:170
750299219,3cyclictest12-21ksoftirqd/010:41:530
750299217,3cyclictest12-21ksoftirqd/012:20:130
750299212,14cyclictest28040-21ssh11:35:140
7502992115,4cyclictest0-21swapper/011:20:140
7502992115,4cyclictest0-21swapper/011:20:130
750299211,4cyclictest131rcu_preempt10:25:170
750299211,3cyclictest0-21swapper/007:10:020
7502992110,4cyclictest12-21ksoftirqd/009:05:010
7502992110,3cyclictest12-21ksoftirqd/012:25:180
752399209,7cyclictest35966irq/58-eth011:05:133
752399203,11cyclictest0-21swapper/312:11:083
752399203,11cyclictest0-21swapper/310:19:083
752399202,13cyclictest0-21swapper/307:20:143
752399202,11cyclictest0-21swapper/310:05:153
7523992013,4cyclictest0-21swapper/311:32:423
7523992010,4cyclictest0-21swapper/311:40:163
751599203,14cyclictest27009-21fschecks_count07:30:152
751599201,8cyclictest13580-21ssh10:15:132
751599201,5cyclictest0-21swapper/208:30:142
751599201,10cyclictest29621-21fschecks_count08:40:172
751599201,10cyclictest26144-21/usr/sbin/munin10:40:172
751599201,10cyclictest17158-21cat07:20:012
751599200,9cyclictest91-21pr/ttyS009:00:012
751099203,12cyclictest844-21munin-node08:30:141
751099201,3cyclictest131rcu_preempt10:50:161
751099201,3cyclictest131rcu_preempt10:50:151
7510992012,5cyclictest8176-21/usr/sbin/munin12:00:111
7510992012,5cyclictest0-21swapper/110:35:371
750299209,3cyclictest12-21ksoftirqd/008:05:310
750299207,4cyclictest12-21ksoftirqd/008:30:160
750299206,4cyclictest12-21ksoftirqd/008:25:020
750299206,3cyclictest0-21swapper/010:30:110
750299201,16cyclictest12-21ksoftirqd/008:40:150
7502992011,6cyclictest0-21swapper/007:50:110
7502992011,6cyclictest0-21swapper/007:50:110
7502992011,6cyclictest0-21swapper/007:25:150
752399199,5cyclictest0-21swapper/311:00:333
752399198,5cyclictest0-21swapper/310:30:193
752399193,11cyclictest0-21swapper/310:01:083
752399193,11cyclictest0-21swapper/309:50:413
752399192,11cyclictest0-21swapper/310:55:143
752399192,11cyclictest0-21swapper/307:30:133
752399191,13cyclictest0-21swapper/312:35:313
752399191,13cyclictest0-21swapper/312:25:333
752399191,13cyclictest0-21swapper/308:00:233
752399191,12cyclictest0-21swapper/309:10:153
752399191,12cyclictest0-21swapper/307:15:413
752399191,12cyclictest0-21swapper/307:10:023
752399191,11cyclictest0-21swapper/307:50:273
752399191,11cyclictest0-21swapper/307:50:273
7523991910,6cyclictest0-21swapper/310:38:393
751599198,9cyclictest22080-21sshd07:25:082
751599194,11cyclictest27346irq/40-dwc2_hso08:50:152
751599191,9cyclictest7882-21kthreadcore11:40:222
751599191,9cyclictest4264-21grep07:40:172
751599191,9cyclictest18795-21fschecks_time07:55:172
751599191,8cyclictest7799-21fschecks_count07:10:162
751599191,15cyclictest23515-21kthreadcore09:05:212
751599191,10cyclictest27231-21unixbench_multi08:00:292
751099197,4cyclictest0-21swapper/109:40:301
751099196,10cyclictest29976-21cat09:10:281
751099195,5cyclictest131rcu_preempt07:25:151
751099193,4cyclictest131rcu_preempt07:45:011
751099193,12cyclictest201rcuc/108:35:141
751099192,3cyclictest0-21swapper/109:45:151
751099192,3cyclictest0-21swapper/109:45:151
751099191,9cyclictest23594-21fschecks_time08:00:171
751099191,2cyclictest131rcu_preempt11:50:371
751099191,16cyclictest12777-21irqcore07:15:171
751099191,15cyclictest31367-21ssh12:30:211
7510991911,5cyclictest0-21swapper/108:50:171
751099191,14cyclictest2817-21ssh09:15:151
751099191,13cyclictest6450-21ssh11:05:151
7510991910,6cyclictest21-21ksoftirqd/108:15:121
750299191,9cyclictest4679-21kthreadcore08:45:220
750299191,9cyclictest17520-21tr08:25:230
7502991913,4cyclictest0-21swapper/009:35:140
752399183,10cyclictest0-21swapper/310:43:083
752399182,11cyclictest0-21swapper/311:35:143
752399182,11cyclictest0-21swapper/308:10:143
752399182,10cyclictest0-21swapper/308:30:143
752399182,10cyclictest0-21swapper/308:25:163
752399182,10cyclictest0-21swapper/307:25:153
7523991810,6cyclictest0-21swapper/312:32:273
7523991810,6cyclictest0-21swapper/309:35:023
7523991810,6cyclictest0-21swapper/309:30:103
752399180,11cyclictest35966irq/58-eth009:20:123
752399180,11cyclictest35966irq/58-eth009:20:123
751599181,8cyclictest7407-21latency_hist07:10:022
751099187,4cyclictest131rcu_preempt08:10:161
751099181,9cyclictest0-21swapper/110:30:221
750299188,3cyclictest12-21ksoftirqd/008:15:170
750299182,13cyclictest24639-21apt-get08:35:140
752399179,6cyclictest0-21swapper/310:54:333
752399179,6cyclictest0-21swapper/310:54:333
752399179,6cyclictest0-21swapper/309:46:223
752399179,6cyclictest0-21swapper/309:46:223
752399179,6cyclictest0-21swapper/309:40:443
752399172,10cyclictest0-21swapper/309:00:153
752399172,10cyclictest0-21swapper/308:50:153
752399172,10cyclictest0-21swapper/308:20:143
751099177,8cyclictest13880-21diskstats07:50:151
751099177,8cyclictest13880-21diskstats07:50:151
751099172,2cyclictest131rcu_preempt09:05:251
7510991711,4cyclictest0-21swapper/107:45:271
752399161,12cyclictest0-21swapper/308:45:283
752399161,12cyclictest0-21swapper/308:35:293
751099162,9cyclictest0-21swapper/107:10:141
751099162,12cyclictest0-21swapper/108:40:151
751099161,6cyclictest11710-21fschecks_time08:55:161
751099161,13cyclictest16211-21kthreadcore08:25:211
7510991610,4cyclictest5565-21timerandwakeup08:45:291
752399151,8cyclictest0-21swapper/308:45:013
752399151,10cyclictest0-21swapper/307:55:183
751099151,2cyclictest131rcu_preempt07:10:021
752399141,11cyclictest0-21swapper/308:15:273
752399131,8cyclictest0-21swapper/307:10:233
752399131,7cyclictest0-21swapper/307:45:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional