You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 14:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 01, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1164099772,9cyclictest131rcu_preempt08:20:130
1164599554,9cyclictest131rcu_preempt08:00:141
1164099542,40cyclictest26637-21ssh10:20:120
1164099525,7cyclictest131rcu_preempt11:35:110
11645994828,12cyclictest0-21swapper/112:20:131
1164599472,36cyclictest3195-21ssh10:25:141
1164599472,34cyclictest12131-21ssh10:30:131
1164599461,36cyclictest24840-21ssh12:25:141
1164099451,3cyclictest0-21swapper/010:40:110
1164099451,3cyclictest0-21swapper/010:40:100
1164099451,3cyclictest0-21swapper/008:50:140
1164099443,36cyclictest0-21swapper/009:25:130
1164099441,37cyclictest0-21swapper/010:35:130
1164599412,30cyclictest6743-21ssh10:45:131
1164099412,5cyclictest0-21swapper/009:05:130
1164099412,35cyclictest0-21swapper/007:40:130
1164599403,32cyclictest24702-21ssh10:55:121
1164099402,2cyclictest3448-21apt-get07:35:140
1164099401,2cyclictest31092-21apt-get07:30:120
1166599392,26cyclictest35966irq/58-eth009:40:143
1165799394,30cyclictest19408-21ssh09:40:132
1164099391,3cyclictest0-21swapper/009:00:130
1164099391,3cyclictest0-21swapper/009:00:130
1164099391,2cyclictest28406-21apt-get11:15:120
1164099391,2cyclictest22732-21apt-get07:55:140
1164099391,2cyclictest12132-21apt-get10:30:130
1164099390,4cyclictest15825-21apt-get12:20:140
1164599381,32cyclictest23080-21ssh10:00:131
11645993813,19cyclictest21-21ksoftirqd/112:00:111
1164099382,7cyclictest0-21swapper/008:35:140
1164099382,7cyclictest0-21swapper/008:35:130
1164099382,2cyclictest11859-21apt-get07:10:130
1164099382,2cyclictest11859-21apt-get07:10:130
1164099381,3cyclictest0-21swapper/012:25:130
1164099381,2cyclictest17651-21apt-get10:15:120
11657993726,5cyclictest27-21ksoftirqd/209:00:122
11657993726,5cyclictest27-21ksoftirqd/209:00:122
1164599379,20cyclictest21-21ksoftirqd/109:45:121
1164599371,28cyclictest10338-21ssh09:35:121
1164099372,2cyclictest8660-21apt-get10:10:130
11640993722,7cyclictest0-21swapper/010:55:120
1164099371,6cyclictest0-21swapper/008:10:130
1164099371,3cyclictest14010-21ssh11:25:130
1164099370,4cyclictest30076-21apt-get09:10:130
1164099370,32cyclictest16692-21cut09:55:220
11657993616,4cyclictest0-21swapper/208:10:132
1164099362,2cyclictest17553-21apt-get11:45:150
1164099361,3cyclictest0-21swapper/009:45:130
1164099361,3cyclictest0-21swapper/007:25:160
1164099361,2cyclictest0-21swapper/010:05:130
1164099361,22cyclictest0-21swapper/009:42:230
1164099361,11cyclictest32420-21ssh11:51:420
1164099351,2cyclictest9537-21apt-get08:15:110
1164099351,10cyclictest3307-21diskstats10:25:170
1164099342,29cyclictest16666-21apt-get07:15:120
1164099341,9cyclictest19129-21cut09:20:240
1164099341,3cyclictest0-21swapper/011:00:120
1164099341,3cyclictest0-21swapper/008:45:130
1164099341,3cyclictest0-21swapper/008:05:130
1164099341,3cyclictest0-21swapper/008:00:140
1164099341,3cyclictest0-21swapper/007:50:140
1164099341,2cyclictest0-21swapper/009:30:120
1166599333,17cyclictest35966irq/58-eth008:45:113
11657993327,2cyclictest27346irq/40-dwc2_hso10:00:122
1165799332,18cyclictest19401-21ssh11:10:132
1165799332,15cyclictest0-21swapper/211:33:112
1164099336,24cyclictest23508-21irqcore11:30:160
1164099334,24cyclictest10289-21apt-get12:35:130
1164099331,9cyclictest6774-21df09:15:150
1164099331,9cyclictest10405-21ntpq10:45:260
1164099331,7cyclictest23223-21cut12:05:210
1164099331,3cyclictest0-21swapper/009:50:130
1164099331,3cyclictest0-21swapper/009:50:130
1164099331,3cyclictest0-21swapper/007:45:130
1164099331,2cyclictest23063-21apt-get10:00:120
1164099331,2cyclictest12146-21apt-get12:00:120
1165799325,15cyclictest27346irq/40-dwc2_hso10:40:122
1165799325,15cyclictest27346irq/40-dwc2_hso10:40:122
1165799321,19cyclictest10352-21ssh09:35:132
1164599326,16cyclictest21-21ksoftirqd/107:45:141
1164599324,20cyclictest4937-21ssh09:50:131
1164599324,20cyclictest4937-21ssh09:50:131
11645993212,15cyclictest21-21ksoftirqd/112:12:101
1164099321,9cyclictest8331-21ssh12:15:200
1164099321,9cyclictest10316-21find09:35:110
1164099321,6cyclictest5195-21fschecks_count11:20:150
1164099321,6cyclictest21594-21/usr/sbin/munin07:20:150
1164099321,4cyclictest12138-21ntp_192.168.11411:40:230
1164099321,3cyclictest3178-21apt-get11:55:120
1164099321,2cyclictest0-21swapper/008:40:130
11665993120,5cyclictest35966irq/58-eth010:05:133
11657993123,6cyclictest27346irq/40-dwc2_hso09:10:012
11657993122,4cyclictest261rcuc/209:25:132
1165799312,17cyclictest3203-21ssh11:55:142
1164599318,18cyclictest5007-21ssh11:20:111
1164599313,18cyclictest3477-21kworker/u8:109:55:121
1164099311,7cyclictest10534-21fschecks_count11:05:160
1164099311,3cyclictest0-21swapper/012:10:120
1164099311,3cyclictest0-21swapper/008:30:140
1164099311,2cyclictest0-21swapper/008:25:130
1165799305,14cyclictest27346irq/40-dwc2_hso09:30:142
1164599303,20cyclictest10306-21rm12:35:141
1164599301,20cyclictest1364-21ssh09:30:141
1164099301,6cyclictest1602-21cut12:30:160
1164099301,6cyclictest1602-21cut12:30:160
1164099301,6cyclictest15679-21/usr/sbin/munin10:50:150
1166599293,18cyclictest35966irq/58-eth009:25:133
1166599293,15cyclictest35966irq/58-eth009:10:143
1166599292,15cyclictest35966irq/58-eth010:00:123
1166599291,20cyclictest35966irq/58-eth008:55:163
1166599291,14cyclictest35966irq/58-eth010:55:123
1165799295,14cyclictest27346irq/40-dwc2_hso08:50:132
1165799292,21cyclictest5791-21ssh11:37:112
1164099291,7cyclictest27576-21ssh11:14:300
11665992813,9cyclictest35966irq/58-eth011:05:143
1165799283,15cyclictest27346irq/40-dwc2_hso11:15:132
11657992813,12cyclictest27-21ksoftirqd/210:25:142
1164599282,19cyclictest1392-21ssh11:00:131
1164099281,4cyclictest15649-21fschecks_count08:55:180
1166599273,15cyclictest35966irq/58-eth011:50:123
1166599272,17cyclictest35966irq/58-eth010:30:133
1166599272,16cyclictest35966irq/58-eth008:10:133
1166599272,15cyclictest0-21swapper/310:25:143
1165799276,11cyclictest27346irq/40-dwc2_hso07:50:122
1165799273,21cyclictest0-21swapper/212:06:112
1165799273,16cyclictest27346irq/40-dwc2_hso07:30:112
1165799273,15cyclictest27346irq/40-dwc2_hso10:30:142
11657992712,13cyclictest32686-21diskmemload12:29:112
1165799271,20cyclictest30101-21ssh09:10:142
1164599274,19cyclictest17661-21ssh10:15:121
1164599272,19cyclictest26603-21ssh12:06:111
1164599272,11cyclictest131rcu_preempt11:55:131
11645992720,3cyclictest131rcu_preempt12:30:121
11645992720,3cyclictest131rcu_preempt12:30:121
11645992715,6cyclictest21-21ksoftirqd/111:45:151
11645992713,9cyclictest0-21swapper/108:15:111
1166599267,11cyclictest35966irq/58-eth009:30:143
1166599262,14cyclictest35966irq/58-eth009:35:133
1165799261,15cyclictest10308-21ssh12:35:142
1164599262,19cyclictest32144-21ssh10:05:141
1164599262,14cyclictest630-21Async09:36:311
1166599252,13cyclictest35966irq/58-eth011:15:133
1165799254,12cyclictest27346irq/40-dwc2_hso10:50:132
1165799252,8cyclictest23001-21ssh11:46:112
1165799252,21cyclictest16168-21ssh11:43:492
1165799252,18cyclictest20072-21ssh12:04:112
1165799251,9cyclictest19395-21ls08:25:182
1165799251,15cyclictest27346irq/40-dwc2_hso11:50:112
1164599257,4cyclictest131rcu_preempt08:25:141
1164599251,16cyclictest6682-21ssh09:15:131
11645992510,10cyclictest0-21swapper/107:35:131
1164099251,8cyclictest11724-21wc07:10:010
1166599246,8cyclictest35966irq/58-eth010:35:133
1166599242,12cyclictest35966irq/58-eth007:20:123
11657992416,5cyclictest2058-21ssh11:00:182
1165799241,18cyclictest27346irq/40-dwc2_hso09:15:132
1165799241,14cyclictest29777-21latency_hist10:40:012
1164599242,18cyclictest6834-21ssh12:15:141
1164599242,18cyclictest14014-21ssh11:25:131
1166599231,15cyclictest0-21swapper/309:05:123
1165799239,11cyclictest0-21swapper/210:05:122
1165799235,9cyclictest15849-21ssh12:20:142
1165799232,19cyclictest6803-21apt-get12:15:132
1165799232,17cyclictest32474-21fschecks_count08:05:182
1165799232,16cyclictest0-21swapper/212:13:562
1165799232,16cyclictest0-21swapper/207:55:052
1164599232,2cyclictest131rcu_preempt08:10:131
1164599232,11cyclictest8546-21ssh11:40:131
1164599231,3cyclictest131rcu_preempt07:25:141
1164599231,17cyclictest32048-21ssh11:15:251
11665992210,9cyclictest0-21swapper/312:32:513
11665992210,9cyclictest0-21swapper/312:32:513
1165799223,16cyclictest15296-21kthreadcore09:55:202
1165799223,10cyclictest9670-21fschecks_count08:15:142
1165799222,14cyclictest27346irq/40-dwc2_hso10:20:142
1165799222,13cyclictest27346irq/40-dwc2_hso07:10:122
1165799222,13cyclictest27346irq/40-dwc2_hso07:10:122
1165799221,11cyclictest14358-21awk11:25:162
1165799221,11cyclictest10931-21kthreadcore10:10:212
1164599224,11cyclictest8674-21ssh10:10:131
1164599223,8cyclictest131rcu_preempt08:40:141
1164599222,16cyclictest15522-21apt-get08:55:151
1164599222,15cyclictest25804-21kthreadcore09:05:191
1164599221,17cyclictest11808-21ls07:10:021
1166599219,9cyclictest0-21swapper/312:23:533
1166599219,9cyclictest0-21swapper/309:24:013
1166599213,12cyclictest0-21swapper/312:29:113
1165799218,11cyclictest29216-21irqcore08:35:192
1165799218,11cyclictest29216-21irqcore08:35:192
1165799217,11cyclictest19475-21grep09:20:282
1165799216,5cyclictest12146-21kthreadcore11:05:212
1165799215,13cyclictest24833-21kthreadcore08:30:202
1165799212,10cyclictest12790-21cron07:45:022
1165799211,12cyclictest7999-21tr11:20:222
1165799211,11cyclictest25452-21ps10:55:182
1165799211,10cyclictest1370-21cpu12:30:142
1165799211,10cyclictest1370-21cpu12:30:132
11645992114,4cyclictest31640-21latency_hist11:35:011
11645992113,5cyclictest16433-21kernelversion10:50:181
11645992113,3cyclictest21-21ksoftirqd/107:50:131
1164599211,16cyclictest30090-21ssh09:10:141
1164599211,15cyclictest20309-21find09:00:121
1164599211,15cyclictest20309-21find09:00:121
1164599210,16cyclictest22887-21cut07:55:161
1166599209,7cyclictest0-21swapper/307:50:143
1166599203,8cyclictest35966irq/58-eth007:10:123
1166599203,8cyclictest35966irq/58-eth007:10:123
1166599203,12cyclictest0-21swapper/311:43:503
1166599203,11cyclictest0-21swapper/311:55:113
1166599202,13cyclictest35966irq/58-eth010:15:133
1166599202,13cyclictest0-21swapper/310:50:143
1166599202,11cyclictest0-21swapper/311:45:113
1165799207,11cyclictest2443-21kthreadcore08:40:202
1165799206,11cyclictest13353-21sort07:45:182
1165799202,9cyclictest21273-21awk07:20:012
1165799202,10cyclictest5640-21irqrtprio09:50:192
1165799202,10cyclictest5640-21irqrtprio09:50:192
1165799201,9cyclictest9556-21sh10:45:232
1165799201,9cyclictest26419-21fschecks_count07:25:182
1165799201,9cyclictest19552-21ssh10:15:202
1165799201,16cyclictest17516-21kthreadcore08:55:242
1165799201,11cyclictest6481-21kthreadcore07:35:222
1165799201,10cyclictest9680-21cat08:45:292
1165799201,10cyclictest29020-21irqrtprio09:45:182
1165799201,10cyclictest24755-21grep07:20:222
1165799201,10cyclictest14508-21fschecks_time08:20:162
11645992011,7cyclictest614-21dbus-daemon08:33:101
1164599201,16cyclictest23445-21ssh09:40:301
1164599201,15cyclictest17973-21ls08:20:281
1164599201,13cyclictest14007-21ssh11:05:261
1164599200,5cyclictest21-21ksoftirqd/107:40:291
1164599200,17cyclictest2357-21timerandwakeup07:30:271
1166599199,8cyclictest0-21swapper/311:03:073
11665991912,4cyclictest0-21swapper/312:35:263
11665991912,4cyclictest0-21swapper/312:10:283
11665991911,6cyclictest0-21swapper/310:41:113
11665991911,6cyclictest0-21swapper/310:41:113
11665991911,5cyclictest35966irq/58-eth009:50:143
11665991911,5cyclictest35966irq/58-eth009:50:143
11665991911,5cyclictest0-21swapper/312:05:013
1166599191,11cyclictest35966irq/58-eth010:45:123
11665991910,6cyclictest35966irq/58-eth012:05:123
1165799191,9cyclictest28235-21sed08:00:202
1164599195,11cyclictest10887-21fschecks_time08:50:171
11645991912,4cyclictest31947-21/usr/sbin/munin11:35:131
1164599190,15cyclictest2957-21ls10:25:021
1166599189,6cyclictest0-21swapper/311:13:013
1166599189,6cyclictest0-21swapper/309:49:413
1166599182,11cyclictest0-21swapper/311:36:113
1166599182,11cyclictest0-21swapper/311:31:113
1166599182,11cyclictest0-21swapper/308:50:153
1166599182,10cyclictest0-21swapper/309:00:143
1166599182,10cyclictest0-21swapper/309:00:143
1166599182,10cyclictest0-21swapper/308:20:143
1166599182,10cyclictest0-21swapper/308:15:123
11665991811,4cyclictest0-21swapper/311:25:023
11665991810,5cyclictest35966irq/58-eth011:25:273
11665991810,4cyclictest0-21swapper/307:35:143
11665991810,3cyclictest0-21swapper/312:16:273
1164599187,6cyclictest211ksoftirqd/107:15:121
1164599184,9cyclictest0-21swapper/109:20:141
1164599183,12cyclictest24721-21ssh10:35:241
1164599183,12cyclictest23464-21cat11:10:291
1164599181,4cyclictest131rcu_preempt09:25:141
11645991811,5cyclictest0-21swapper/107:10:231
11645991811,5cyclictest0-21swapper/107:10:221
11645991811,4cyclictest0-21swapper/108:05:191
11645991810,5cyclictest0-21swapper/107:20:181
1164599180,9cyclictest32590-21grep08:35:301
1164599180,9cyclictest32590-21grep08:35:291
1166599179,6cyclictest0-21swapper/309:15:163
1166599178,6cyclictest0-21swapper/310:20:113
1166599172,10cyclictest0-21swapper/307:55:053
11665991711,4cyclictest35966irq/58-eth010:10:133
11665991711,4cyclictest0-21swapper/309:55:173
1165799171,9cyclictest11726-21cat07:10:012
1164599179,6cyclictest5703-21cron08:45:011
1164599171,12cyclictest30263-21df10:40:131
1164599171,12cyclictest30263-21df10:40:131
1166599161,13cyclictest0-21swapper/308:10:023
1166599152,10cyclictest0-21swapper/307:30:133
1166599151,12cyclictest0-21swapper/307:40:143
1166599151,12cyclictest0-21swapper/307:25:243
1166599151,11cyclictest0-21swapper/308:30:153
1166599151,11cyclictest0-21swapper/308:25:193
1166599141,11cyclictest0-21swapper/307:15:163
1166599138,2cyclictest0-21swapper/308:35:153
1166599138,2cyclictest0-21swapper/308:35:153
1166599131,6cyclictest0-21swapper/307:45:193
1166599111,7cyclictest0-21swapper/308:40:213
1166599111,7cyclictest0-21swapper/308:00:163
1166599101,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional