You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 01:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 30, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
205499756,49cyclictest10080-21ssh09:40:131
205499744,54cyclictest4590-21ssh11:25:141
206399542,17cyclictest13042-21ssh12:22:592
204699505,37cyclictest20653-21ssh12:10:130
205499494,8cyclictest131rcu_preempt08:15:121
2046994913,23cyclictest12-21ksoftirqd/007:30:130
206399482,27cyclictest29578-21ssh12:15:122
204699482,39cyclictest19082-21ssh09:45:150
205499468,2cyclictest131rcu_preempt12:00:141
205499462,6cyclictest131rcu_preempt08:45:161
205499455,33cyclictest20589-21ssh09:10:141
2046994527,11cyclictest6014-21apt-get08:55:140
207199443,24cyclictest35966irq/58-eth009:10:143
205499442,31cyclictest8185-21ssh10:15:111
204699446,29cyclictest28060-21ssh09:50:120
204699446,29cyclictest28060-21ssh09:50:120
2046994326,11cyclictest8165-21ssh11:45:110
204699431,32cyclictest22573-21ssh11:35:120
2071994230,5cyclictest35966irq/58-eth012:15:113
206399422,30cyclictest4574-21ssh11:25:122
2046994120,15cyclictest4745-21apt-get08:20:120
206399392,19cyclictest24342-21kthreadcore11:35:182
204699391,31cyclictest15331-21ssh09:25:120
207199384,21cyclictest35966irq/58-eth009:45:153
2071993819,11cyclictest35966irq/58-eth011:35:123
2046993818,13cyclictest13635-21apt-get10:00:110
2071993714,11cyclictest35966irq/58-eth010:05:123
206399375,25cyclictest27-21ksoftirqd/210:55:112
206399372,18cyclictest5717-21cat11:05:312
204699379,12cyclictest32021-21ssh12:34:060
204699379,10cyclictest31685-21diskmemload11:40:140
206399361,19cyclictest930-21snmpd11:10:132
205499362,24cyclictest24392-21ssh09:30:141
205499362,22cyclictest21-21ksoftirqd/108:20:121
204699363,24cyclictest20598-21ssh09:10:140
2046993628,4cyclictest6972-21apt-get07:15:120
2046993628,4cyclictest14390-21apt-get08:30:140
2046993625,8cyclictest0-21swapper/010:55:100
204699358,11cyclictest2999-21ssh09:16:400
2046993511,10cyclictest11752-21entropy12:05:150
204699349,10cyclictest11077-21ssh09:57:240
204699348,8cyclictest23032-21ssh11:51:510
204699348,8cyclictest23032-21ssh11:51:510
204699346,9cyclictest6138-21ssh12:20:110
204699343,23cyclictest4825-21ssh11:25:150
204699342,20cyclictest27980-21ssh11:20:130
204699339,10cyclictest3706-21rm10:47:230
204699337,10cyclictest2733-21/usr/sbin/munin10:30:270
204699333,17cyclictest2596-21ssh12:00:130
2046993326,3cyclictest17937-21apt-get08:00:120
2046993315,15cyclictest0-21swapper/011:55:130
207199320,21cyclictest35966irq/58-eth011:30:133
206399322,19cyclictest17121-21ssh11:50:152
206399322,19cyclictest17121-21ssh11:50:152
205499327,15cyclictest21-21ksoftirqd/108:55:131
205499324,19cyclictest978-21ssh09:35:121
205499323,19cyclictest15106-21ssh12:25:111
205499322,24cyclictest6307-21ssh10:50:151
205499322,21cyclictest24341-21cut11:35:181
205499322,21cyclictest24186-21/usr/sbin/munin12:10:241
205499321,23cyclictest29707-21ssh10:45:121
204699329,10cyclictest26923-21ssh11:19:130
204699327,10cyclictest21118-21ssh12:26:500
204699326,8cyclictest9883-21apt-get11:10:130
2046993225,3cyclictest22747-21apt-get08:05:130
2046993221,6cyclictest13567-21ssh11:30:130
207199319,10cyclictest35966irq/58-eth012:10:123
206399312,13cyclictest1304-21ssh11:58:582
204699319,8cyclictest30957-21ssh10:09:430
204699318,9cyclictest442-21ssh09:34:550
204699317,8cyclictest20173-21diskmemload11:01:100
204699317,8cyclictest20173-21diskmemload10:20:090
204699316,8cyclictest19070-21ntp_states09:05:250
204699316,8cyclictest18649-21munin-run09:45:020
204699312,21cyclictest31633-21ssh10:10:130
2046993118,10cyclictest0-21swapper/007:50:100
207199303,14cyclictest35966irq/58-eth010:10:133
207199302,16cyclictest35966irq/58-eth010:55:103
207199302,13cyclictest35966irq/58-eth011:00:113
206399303,18cyclictest27346irq/40-dwc2_hso07:45:122
206399302,25cyclictest616-21systemd-logind10:35:122
206399302,25cyclictest616-21systemd-logind10:35:122
205499302,22cyclictest25069-21sh11:16:581
205499302,21cyclictest10026-21diskstats11:10:151
204699309,8cyclictest14845-21ssh09:24:550
204699309,6cyclictest12898-21cron07:55:020
204699307,7cyclictest1997-21memory07:40:220
204699307,10cyclictest28253-21ssh10:43:430
204699305,8cyclictest3823-21ssh12:17:410
2046993022,4cyclictest32340-21apt-get08:15:120
2046993020,6cyclictest729-21apt-get12:35:130
204699301,26cyclictest6331-21ssh10:50:160
2046993010,8cyclictest26519-21ssh10:25:150
207199292,17cyclictest35966irq/58-eth010:30:133
207199291,16cyclictest35966irq/58-eth010:15:113
206399294,19cyclictest27346irq/40-dwc2_hso08:10:132
206399293,19cyclictest27346irq/40-dwc2_hso07:10:112
206399292,11cyclictest26232-21ssh10:25:132
205499292,20cyclictest22677-21ssh10:05:141
205499291,20cyclictest26211-21sh10:25:111
204699297,13cyclictest0-21swapper/010:36:100
204699297,13cyclictest0-21swapper/010:36:090
207199283,18cyclictest0-21swapper/311:06:583
207199282,15cyclictest35966irq/58-eth008:50:143
2071992813,9cyclictest35966irq/58-eth011:10:143
207199281,17cyclictest35966irq/58-eth011:45:103
2054992819,5cyclictest0-21swapper/110:20:271
205499281,21cyclictest11636-21ssh12:05:141
204699289,6cyclictest20173-21diskmemload10:18:200
204699286,8cyclictest12476-21kernelversion07:20:180
204699286,8cyclictest12476-21kernelversion07:20:180
2071992711,10cyclictest35966irq/58-eth011:58:583
206399273,11cyclictest0-21swapper/209:40:142
206399271,16cyclictest8195-21ssh10:15:122
205499272,20cyclictest732-21diskmemload12:35:111
2054992714,9cyclictest0-21swapper/107:15:141
2054992712,10cyclictest21-21ksoftirqd/110:35:141
2054992712,10cyclictest21-21ksoftirqd/110:35:141
205499271,18cyclictest31689-21ssh11:40:141
204699274,19cyclictest9543-21find08:25:090
206399263,7cyclictest739-21ssh12:35:112
206399263,18cyclictest27346irq/40-dwc2_hso11:15:122
206399262,20cyclictest0-21swapper/207:20:132
206399262,20cyclictest0-21swapper/207:20:132
206399262,18cyclictest0-21swapper/208:25:112
205499264,8cyclictest131rcu_preempt08:05:131
2054992611,8cyclictest21-21ksoftirqd/111:00:121
204699267,15cyclictest1608-21cut09:35:180
2046992619,3cyclictest26270-21apt-get07:35:120
2071992513,7cyclictest35966irq/58-eth009:35:133
2063992511,8cyclictest0-21swapper/211:20:122
2046992515,7cyclictest0-21swapper/008:45:160
2046992513,9cyclictest1035-21df_inode11:05:150
207199243,13cyclictest0-21swapper/310:25:123
206399246,11cyclictest27346irq/40-dwc2_hso08:05:112
206399243,14cyclictest27346irq/40-dwc2_hso09:50:132
206399243,14cyclictest27346irq/40-dwc2_hso09:50:132
206399242,18cyclictest31257-21ssh11:02:582
206399242,15cyclictest27346irq/40-dwc2_hso09:05:122
2063992415,5cyclictest27-21ksoftirqd/209:45:152
206399241,12cyclictest20747-21ssh10:40:122
2063992410,11cyclictest0-21swapper/210:05:072
205499241,17cyclictest13662-21ssh10:00:121
2054992410,3cyclictest0-21swapper/110:40:211
2046992415,7cyclictest0-21swapper/008:50:130
2046992415,6cyclictest16689-21diskstats07:25:140
2046992414,7cyclictest14071-21wc09:00:200
2046992413,8cyclictest5482-21memory07:10:230
2046992412,9cyclictest8052-21sshd07:46:480
2071992310,10cyclictest0-21swapper/312:35:213
206399239,11cyclictest17507-21cat10:00:262
206399236,9cyclictest11635-21diskmemload12:05:142
206399233,17cyclictest0-21swapper/208:00:132
206399233,14cyclictest27346irq/40-dwc2_hso08:20:122
206399232,6cyclictest0-21swapper/212:25:452
206399232,6cyclictest0-21swapper/211:33:592
206399232,15cyclictest0-21swapper/207:35:052
2063992312,9cyclictest11338-21ssh09:20:432
206399231,19cyclictest8455-21fschecks_count07:50:142
206399231,10cyclictest844-21munin-node10:10:132
2063992310,11cyclictest4730-21wc09:35:282
205499234,14cyclictest0-21swapper/111:05:141
2054992314,4cyclictest21-21ksoftirqd/111:55:121
2054992313,6cyclictest28316-21if_eth111:20:151
204699236,14cyclictest25829-21cut08:40:210
204699236,14cyclictest25829-21cut08:40:210
2046992311,9cyclictest2051-21cut07:10:010
207199229,9cyclictest0-21swapper/309:15:213
207199223,13cyclictest35966irq/58-eth011:40:143
207199223,13cyclictest0-21swapper/312:21:583
2071992216,4cyclictest0-21swapper/309:20:493
2071992215,5cyclictest0-21swapper/311:25:113
2071992214,6cyclictest0-21swapper/312:02:533
2071992214,5cyclictest35966irq/58-eth012:05:133
2071992214,5cyclictest0-21swapper/309:50:133
2071992214,5cyclictest0-21swapper/309:50:133
2071992213,6cyclictest0-21swapper/311:24:123
2071992213,4cyclictest0-21swapper/312:25:203
2071992212,7cyclictest0-21swapper/309:33:253
207199221,15cyclictest35966irq/58-eth010:45:123
206399229,10cyclictest30425-21uname10:45:182
206399223,12cyclictest0-21swapper/208:55:142
206399223,12cyclictest0-21swapper/208:35:132
206399222,15cyclictest0-21swapper/212:10:242
206399222,12cyclictest24644-21hwlatdetect09:30:182
206399221,9cyclictest73-21kswapd012:30:122
206399221,11cyclictest32341-21kthreadcore09:15:212
205499222,14cyclictest6270-21sh09:20:151
205499221,18cyclictest8176-21ssh11:45:111
204699229,9cyclictest0-21swapper/008:10:130
2071992115,4cyclictest0-21swapper/307:10:153
2071992112,7cyclictest0-21swapper/310:20:293
2071992112,6cyclictest0-21swapper/307:50:233
2071992111,7cyclictest0-21swapper/310:00:183
206399219,9cyclictest15497-21fschecks_count09:25:152
206399217,6cyclictest1623-21kthreadcore11:40:232
206399213,10cyclictest21752-21usage_EthcTask10:20:312
2063992114,5cyclictest11815-21ssh10:51:202
206399211,16cyclictest6078-21tr09:55:202
206399211,16cyclictest4367-21kthreadcore12:00:212
206399211,16cyclictest18182-21grep08:30:302
206399211,11cyclictest21060-21cut09:10:182
205499217,5cyclictest28207-21kthreadcore08:10:191
205499211,15cyclictest15494-21cut09:25:151
205499211,14cyclictest6806-21ls07:15:011
2046992112,6cyclictest22661-21ntp_states08:35:250
2071992014,4cyclictest0-21swapper/311:50:153
2071992014,4cyclictest0-21swapper/311:50:153
2071992014,4cyclictest0-21swapper/310:51:023
2071992014,4cyclictest0-21swapper/310:40:233
2071992014,4cyclictest0-21swapper/308:25:163
2071992013,5cyclictest0-21swapper/311:15:143
2071992013,5cyclictest0-21swapper/309:55:323
2071992013,5cyclictest0-21swapper/309:40:123
2071992011,6cyclictest0-21swapper/309:25:153
206399208,10cyclictest32488-21fschecks_count08:15:162
206399207,5cyclictest24138-21fschecks_count08:40:162
206399207,5cyclictest24138-21fschecks_count08:40:162
206399202,12cyclictest0-21swapper/211:45:102
206399201,9cyclictest5776-21cron08:55:012
206399201,5cyclictest0-21swapper/207:40:132
206399201,15cyclictest28813-21cpu08:45:162
206399201,11cyclictest16664-21grep07:55:272
206399201,10cyclictest21248-21latency_hist07:30:012
2063992010,7cyclictest0-21swapper/210:30:112
205499208,4cyclictest131rcu_preempt11:30:311
205499202,13cyclictest21-21ksoftirqd/112:22:581
205499201,8cyclictest12426-21kthreadcore09:00:171
205499201,15cyclictest27852-21ls09:50:021
2071991914,3cyclictest0-21swapper/308:00:243
2071991913,4cyclictest0-21swapper/312:30:193
2071991913,4cyclictest0-21swapper/310:35:023
2071991913,4cyclictest0-21swapper/310:35:023
2071991913,4cyclictest0-21swapper/308:35:143
2071991913,4cyclictest0-21swapper/308:05:293
207199191,13cyclictest0-21swapper/309:00:133
206399191,9cyclictest7079-21fschecks_count07:15:152
206399191,15cyclictest21514-21uniq07:30:152
206399190,17cyclictest10895-21ls09:00:112
205499192,6cyclictest29600-21apt-get09:15:121
205499191,9cyclictest26377-21fschecks_count07:35:151
205499191,7cyclictest15357-21df10:55:121
205499191,16cyclictest5568-21tr07:45:201
205499191,14cyclictest30181-21cut12:15:171
205499191,11cyclictest2872-21ls10:30:131
207199186,8cyclictest321rcuc/308:10:133
207199182,10cyclictest0-21swapper/308:40:153
207199182,10cyclictest0-21swapper/308:40:143
207199182,10cyclictest0-21swapper/307:35:053
2071991812,4cyclictest0-21swapper/308:45:203
2071991812,4cyclictest0-21swapper/308:30:173
2071991812,4cyclictest0-21swapper/308:20:193
2071991812,4cyclictest0-21swapper/307:55:203
2071991812,4cyclictest0-21swapper/307:45:203
2071991812,4cyclictest0-21swapper/307:40:193
2071991812,4cyclictest0-21swapper/307:30:303
2071991812,4cyclictest0-21swapper/307:15:173
205499187,7cyclictest614-21dbus-daemon10:10:081
205499186,3cyclictest131rcu_preempt12:30:181
205499181,9cyclictest32521-21cat09:50:321
205499181,9cyclictest32521-21cat09:50:311
205499181,7cyclictest21324-21usage_Anybus11:50:321
205499181,7cyclictest21324-21usage_Anybus11:50:311
205499181,11cyclictest17885-21ls08:00:021
207199178,4cyclictest0-21swapper/309:05:133
207199174,8cyclictest0-21swapper/307:20:123
207199174,8cyclictest0-21swapper/307:20:123
207199172,10cyclictest0-21swapper/307:25:133
2071991712,3cyclictest0-21swapper/308:55:203
2071991711,4cyclictest0-21swapper/308:15:143
205499175,9cyclictest23993-21kthreadcore07:30:221
205499174,5cyclictest18258-21sed08:00:171
205499172,4cyclictest17269-21kthreadcore07:25:181
205499171,6cyclictest4649-21apt-get09:55:141
2054991710,5cyclictest211ksoftirqd/108:50:141
205499163,10cyclictest14883-21ps08:30:191
205499162,12cyclictest22046-21kthreadcore08:35:211
205499162,11cyclictest9004-21kthreadcore07:50:171
205499161,12cyclictest31101-21apt-get07:40:131
205499161,12cyclictest24405-21sed08:40:181
205499161,12cyclictest24405-21sed08:40:181
205499161,12cyclictest16429-21tr09:05:181
205499161,12cyclictest13090-21proc_pri08:25:241
205499151,11cyclictest11828-21apt-get07:20:121
205499151,11cyclictest11828-21apt-get07:20:111
206399142,9cyclictest1965-21latency_hist07:10:022
205499141,10cyclictest2119-21ls07:10:021
207199103,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional