You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 12:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Tue Sep 22, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1959199811,74cyclictest8993-21ssh00:10:142
1958199574,15cyclictest21-21ksoftirqd/100:30:151
1959199562,48cyclictest13703-21apt-get21:50:132
1959199553,35cyclictest9783-21ssh21:30:172
19607995332,17cyclictest35966irq/58-eth023:45:153
1959199522,45cyclictest0-21swapper/223:55:132
1959199521,45cyclictest0-21swapper/222:05:132
19607995119,21cyclictest35966irq/58-eth021:45:153
19607995119,21cyclictest35966irq/58-eth021:45:143
1959199511,37cyclictest0-21swapper/222:45:132
19581995123,4cyclictest131rcu_preempt22:40:121
1959199502,34cyclictest0-21swapper/223:30:122
1959199501,43cyclictest25751-21apt-get22:50:132
1959199501,33cyclictest0-21swapper/222:00:132
1958199508,11cyclictest21-21ksoftirqd/100:00:141
1959199494,38cyclictest19249-21ssh23:40:122
1959199494,31cyclictest3847-21ssh00:25:152
1959199493,37cyclictest32312-21ssh00:05:122
1959199481,27cyclictest30324-21/usr/sbin/munin22:15:232
1958199481,33cyclictest22887-21ssh21:55:141
1959199471,41cyclictest0-21swapper/219:50:112
1958199468,3cyclictest131rcu_preempt21:10:141
1956999461,33cyclictest14836-21ssh21:15:130
1959199453,36cyclictest0-21swapper/219:40:132
1959199451,38cyclictest0-21swapper/220:50:132
1959199450,41cyclictest11571-21apt-get23:00:102
1959199441,39cyclictest0-21swapper/220:25:142
1959199441,37cyclictest0-21swapper/223:45:122
1959199441,29cyclictest15386-21cut00:30:242
1959199440,34cyclictest26106-21usage_Anybus22:30:342
1959199432,37cyclictest27346irq/40-dwc2_hso20:30:142
1959199431,34cyclictest5762-21apt-get21:10:142
1956999430,8cyclictest131rcu_preempt21:50:120
1959199420,36cyclictest28122-21apt-get21:00:042
1959199420,28cyclictest17544-21vmstat22:25:322
19569994220,10cyclictest12-21ksoftirqd/022:10:150
1959199413,30cyclictest22855-21apt-get21:55:152
1959199412,34cyclictest0-21swapper/221:20:132
1959199411,32cyclictest27176-21apt-get00:20:132
1959199411,27cyclictest5359-21diskmemload23:16:452
1959199411,27cyclictest22961-21users00:15:312
1959199411,26cyclictest26628-21cat00:35:302
1959199410,36cyclictest0-21swapper/220:05:142
1959199410,36cyclictest0-21swapper/220:05:132
1959199410,29cyclictest1820-21ssh21:42:122
1959199410,28cyclictest13064-21kthreadcore19:35:222
1959199410,27cyclictest7744-21sed20:05:012
19607994027,8cyclictest35966irq/58-eth022:40:143
1959199401,36cyclictest0-21swapper/221:35:122
1959199401,27cyclictest7655-21memory20:35:232
1959199401,22cyclictest18193-21irqstats20:15:202
1959199401,22cyclictest0-21swapper/222:37:592
1959199401,22cyclictest0-21swapper/222:37:582
1959199400,27cyclictest14447-21cut20:45:182
1959199400,26cyclictest18963-21ssh21:15:292
1958199401,12cyclictest131rcu_preempt21:40:131
1959199391,34cyclictest9250-21tail23:50:312
1959199391,33cyclictest24361-21ssh23:05:282
1959199390,27cyclictest5892-21kthreadcore21:45:212
1959199390,27cyclictest5892-21kthreadcore21:45:202
1956999393,25cyclictest2508-21ssh22:55:140
1956999391,32cyclictest4639-21ssh21:45:130
1956999391,32cyclictest4639-21ssh21:45:130
1960799381,19cyclictest35966irq/58-eth022:25:133
1959199383,32cyclictest0-21swapper/222:40:122
1959199381,33cyclictest0-21swapper/223:10:122
1959199381,31cyclictest0-21swapper/221:05:112
1959199371,33cyclictest5290-21awk19:25:272
1959199371,33cyclictest19271-21cut22:10:212
1959199371,25cyclictest26193-21kthreadcore00:00:232
1959199370,30cyclictest9168-21apt-get20:40:132
1958199372,27cyclictest0-21swapper/120:40:141
1956999372,23cyclictest25765-21ssh22:50:130
19569993718,8cyclictest12-21ksoftirqd/021:40:130
1959199361,32cyclictest14493-21cat23:35:292
1959199360,32cyclictest0-21swapper/221:25:112
1959199360,23cyclictest0-21swapper/220:55:162
1960799352,21cyclictest35966irq/58-eth000:00:143
1959199352,20cyclictest31603-21ssh23:28:142
1959199351,31cyclictest20968-21acpi19:45:122
1959199351,25cyclictest7085-21missed_timers22:20:212
1959199350,32cyclictest319-21sed19:20:232
1959199350,32cyclictest12773-21apt20:10:112
1959199350,31cyclictest2805-21sshd19:57:092
1956999354,18cyclictest27187-21ssh00:20:140
1956999351,25cyclictest21774-21ssh22:30:160
19607993418,12cyclictest35966irq/58-eth021:10:153
1959199341,28cyclictest0-21swapper/219:30:142
1959199340,31cyclictest15392-21munin-node23:20:122
1959199340,26cyclictest8900-21ssh22:57:192
1956999332,26cyclictest32322-21ssh00:05:130
1956999332,24cyclictest10240-21ssh23:35:130
1960799327,21cyclictest35966irq/58-eth023:10:133
1960799322,21cyclictest35966irq/58-eth022:20:123
19607993216,11cyclictest35966irq/58-eth022:55:153
19607993216,11cyclictest35966irq/58-eth022:00:153
1959199320,28cyclictest19986-21if_err_eth019:10:182
1956999321,25cyclictest12681-21ssh22:25:140
1960799317,15cyclictest35966irq/58-eth023:20:133
1960799313,18cyclictest35966irq/58-eth000:25:133
1960799313,17cyclictest35966irq/58-eth022:05:113
1959199313,24cyclictest22650-21if_err_eth020:20:172
1958199314,21cyclictest11600-21ssh23:00:121
1958199313,21cyclictest753-21sendmail-mta21:15:121
1959199305,22cyclictest24889-21sed19:15:172
19581993024,2cyclictest21-21ksoftirqd/120:50:131
1956999302,2cyclictest131rcu_preempt20:30:140
1960799292,17cyclictest35966irq/58-eth022:30:173
19607992912,12cyclictest0-21swapper/322:45:133
1960799291,15cyclictest35966irq/58-eth000:05:113
19607992910,15cyclictest0-21swapper/323:15:143
1958199292,23cyclictest16702-21ssh22:45:141
1960799283,20cyclictest35966irq/58-eth022:15:143
19607992814,10cyclictest35966irq/58-eth023:30:143
1958199283,16cyclictest22076-21ssh00:35:141
19569992814,11cyclictest12-21ksoftirqd/019:30:150
1956999281,19cyclictest5359-21diskmemload00:25:130
1960799271,12cyclictest35966irq/58-eth021:25:123
19607992711,12cyclictest0-21swapper/321:35:133
1958199272,19cyclictest31425-21usage_Anybus00:20:301
19581992720,4cyclictest0-21swapper/121:45:141
19581992720,4cyclictest0-21swapper/121:45:141
1956999271,21cyclictest26900-21ssh22:15:140
1956999271,19cyclictest15442-21ssh23:20:160
19569992711,11cyclictest12-21ksoftirqd/020:15:140
1960799264,12cyclictest35966irq/58-eth021:20:123
1960799261,15cyclictest35966irq/58-eth022:50:133
1958199265,5cyclictest131rcu_preempt22:20:141
1958199263,13cyclictest16343-1kworker/1:2H19:55:111
1958199262,21cyclictest0-21swapper/120:35:131
1956999269,10cyclictest28109-21latency_hist21:00:010
1956999262,18cyclictest16842-21cat00:30:290
19607992511,6cyclictest0-21swapper/323:25:133
1958199252,20cyclictest18848-21ssh21:35:121
1956999252,15cyclictest14157-21ssh23:55:140
19569992516,6cyclictest6413-21ssh23:15:140
1956999251,18cyclictest1115-21ssh23:30:140
1956999251,16cyclictest844-21munin-node21:00:150
1960799244,10cyclictest35966irq/58-eth020:00:173
19607992412,8cyclictest35966irq/58-eth000:15:153
1960799241,15cyclictest0-21swapper/300:30:293
1960799241,14cyclictest35966irq/58-eth020:30:133
1958199248,4cyclictest21-21ksoftirqd/100:10:161
1958199241,19cyclictest18121-21ssh00:15:151
1956999242,18cyclictest131rcu_preempt22:45:140
1956999242,17cyclictest29731-21ssh23:10:130
1956999242,13cyclictest9788-21ssh21:30:180
1956999241,14cyclictest0-21swapper/023:05:150
1960799235,11cyclictest0-21swapper/323:35:133
19607992312,7cyclictest35966irq/58-eth023:50:153
1960799231,14cyclictest35966irq/58-eth000:35:133
1958199239,4cyclictest21-21ksoftirqd/122:05:171
1958199236,13cyclictest21-21ksoftirqd/119:40:141
1958199232,18cyclictest0-21swapper/121:30:171
1958199231,4cyclictest30814-21ssh22:35:131
1958199231,4cyclictest30814-21ssh22:35:121
19581992311,4cyclictest21-21ksoftirqd/119:25:211
19581992311,3cyclictest21-21ksoftirqd/122:17:181
1956999233,4cyclictest0-21swapper/021:10:140
1960799224,10cyclictest35966irq/58-eth023:40:123
1958199229,4cyclictest21-21ksoftirqd/120:10:011
1958199229,4cyclictest21-21ksoftirqd/120:10:011
1958199228,11cyclictest21-21ksoftirqd/123:25:311
1958199222,18cyclictest21-21ksoftirqd/123:30:121
19581992216,4cyclictest0-21swapper/120:20:131
19581992216,3cyclictest0-21swapper/123:50:161
19581992212,3cyclictest21-21ksoftirqd/121:00:291
1958199221,17cyclictest17557-21apt-get20:15:131
19581992211,3cyclictest21-21ksoftirqd/122:50:231
19581992211,2cyclictest21-21ksoftirqd/123:16:591
1956999225,6cyclictest10747-21cut20:40:220
1956999224,13cyclictest131rcu_preempt20:05:130
1956999224,13cyclictest131rcu_preempt20:05:130
19569992214,5cyclictest0-21swapper/000:20:000
19569992214,4cyclictest12-21ksoftirqd/019:25:140
1956999221,3cyclictest131rcu_preempt22:05:120
1960799212,8cyclictest35966irq/58-eth020:15:133
1959199212,17cyclictest27346irq/40-dwc2_hso19:10:012
1958199219,10cyclictest0-21swapper/122:02:521
1958199218,3cyclictest21-21ksoftirqd/123:58:511
1958199212,17cyclictest28382-21ssh23:45:151
1958199211,9cyclictest201rcuc/119:15:191
19581992111,7cyclictest12542-21cron00:30:021
1958199211,16cyclictest5359-21diskmemload21:20:251
1958199211,14cyclictest20711-21find23:05:141
19581992111,2cyclictest21-21ksoftirqd/100:05:241
19581992110,9cyclictest0-21swapper/123:14:341
19569992112,4cyclictest0-21swapper/019:15:120
1956999211,16cyclictest3205266ssh22:00:140
1956999211,16cyclictest30891-21ls00:40:010
1956999211,16cyclictest20985-21apt-get19:45:120
1956999211,15cyclictest23999-21apt-get21:20:120
1960799208,7cyclictest35966irq/58-eth023:05:153
1958199209,9cyclictest0-21swapper/123:24:011
1958199209,9cyclictest0-21swapper/122:58:271
1958199209,8cyclictest201rcuc/120:30:141
1958199208,10cyclictest0-21swapper/122:29:261
1958199207,6cyclictest21-21ksoftirqd/122:11:011
1958199206,8cyclictest19557-21cyclictest21:50:131
1958199201,6cyclictest27354-21cut19:50:191
1958199201,6cyclictest25699-21cat22:30:321
19581992014,4cyclictest0-21swapper/121:25:121
1958199201,3cyclictest0-21swapper/121:05:181
1958199201,2cyclictest131rcu_preempt23:43:151
19581992010,2cyclictest21-21ksoftirqd/123:37:431
1956999202,9cyclictest131rcu_preempt23:40:280
19569992014,4cyclictest0-21swapper/021:35:120
19569992013,4cyclictest0-21swapper/020:45:250
19569992012,4cyclictest32046-21sh23:45:260
1956999201,10cyclictest25999-21wc20:20:280
1960799193,11cyclictest0-21swapper/300:23:303
1960799192,8cyclictest0-21swapper/321:30:173
1960799192,13cyclictest0-21swapper/319:40:133
1960799192,10cyclictest0-21swapper/320:40:143
1960799192,10cyclictest0-21swapper/320:10:043
1960799192,10cyclictest0-21swapper/319:55:053
1960799191,13cyclictest0-21swapper/320:20:143
1960799191,13cyclictest0-21swapper/320:10:023
1960799191,13cyclictest0-21swapper/320:10:013
1960799191,13cyclictest0-21swapper/319:20:263
1960799191,10cyclictest35966irq/58-eth023:00:123
1960799191,10cyclictest0-21swapper/320:30:013
19607991910,6cyclictest35966irq/58-eth023:55:123
1958199196,9cyclictest21-21ksoftirqd/119:45:131
1958199193,12cyclictest21-21ksoftirqd/119:10:141
1958199192,14cyclictest630-21Async09:36:311
19581991911,6cyclictest11101-21sshd19:33:301
1958199191,15cyclictest29416-21cpuspeed19:20:131
19581991910,6cyclictest0-21swapper/119:35:191
1956999191,6cyclictest30816-21apt-get22:35:120
1956999191,6cyclictest30816-21apt-get22:35:120
1956999191,5cyclictest0-21swapper/020:50:170
19569991913,4cyclictest0-21swapper/000:10:280
19569991911,6cyclictest0-21swapper/019:50:140
19569991911,5cyclictest5359-21diskmemload21:25:210
19569991910,6cyclictest12-21ksoftirqd/019:20:130
1960799189,6cyclictest0-21swapper/322:36:353
1960799189,6cyclictest0-21swapper/322:36:353
1960799188,6cyclictest0-21swapper/321:15:313
1960799183,11cyclictest0-21swapper/300:14:313
1960799182,10cyclictest0-21swapper/319:45:133
1960799182,10cyclictest0-21swapper/319:35:143
1960799182,10cyclictest0-21swapper/319:30:153
1960799182,10cyclictest0-21swapper/319:25:153
1960799181,14cyclictest0-21swapper/321:50:193
1960799181,10cyclictest0-21swapper/322:15:033
19607991810,5cyclictest0-21swapper/320:50:123
19581991812,4cyclictest0-21swapper/120:25:171
19581991812,4cyclictest0-21swapper/120:10:181
19581991812,3cyclictest21-21ksoftirqd/120:55:261
1956999186,6cyclictest131rcu_preempt21:10:020
1956999184,2cyclictest131rcu_preempt23:00:090
1956999182,3cyclictest0-21swapper/020:35:190
1956999182,2cyclictest131rcu_preempt19:35:140
1956999181,7cyclictest23233-21/usr/sbin/munin00:00:130
1956999181,4cyclictest131rcu_preempt20:30:010
19569991812,4cyclictest0-21swapper/023:25:240
19569991812,4cyclictest0-21swapper/022:40:160
1956999181,12cyclictest27810-21ssh21:55:340
19569991810,5cyclictest6569-21kthreadcore22:20:210
1960799179,6cyclictest0-21swapper/321:41:103
1960799178,6cyclictest0-21swapper/321:55:233
1960799175,5cyclictest0-21swapper/319:10:133
1960799171,9cyclictest0-21swapper/320:35:163
1956999178,6cyclictest2836-21basename20:00:010
1956999174,4cyclictest4640-21apt-get23:50:040
1956999171,5cyclictest19901-21tune2fs19:10:170
1956999171,5cyclictest0-21swapper/020:14:260
1956999171,13cyclictest17473-21kthreadcore19:40:190
19607991610,2cyclictest0-21swapper/321:05:123
1958199164,9cyclictest17500-21grep20:45:281
1960799152,7cyclictest0-21swapper/320:55:153
1960799151,11cyclictest0-21swapper/320:45:203
19607991510,2cyclictest0-21swapper/319:50:123
1956999152,5cyclictest3117-21/usr/sbin/munin20:00:170
1960799149,3cyclictest321rcuc/321:02:223
1958199141,10cyclictest19620-21latency_hist19:10:011
1960799138,2cyclictest0-21swapper/319:15:133
1956999131,9cyclictest19601-21ls19:10:010
1960799101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional