You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 11:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Sep 19, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2461199663,49cyclictest27510-21ssh00:20:111
2461199654,47cyclictest520-21ssh00:05:131
24633995840,13cyclictest35966irq/58-eth021:25:113
24622995417,20cyclictest27-21ksoftirqd/200:25:132
2460099521,37cyclictest26383-21ssh21:55:120
2463399476,24cyclictest35966irq/58-eth023:25:123
2463399448,21cyclictest35966irq/58-eth023:50:133
24633994433,7cyclictest35966irq/58-eth000:35:153
2463399441,27cyclictest35966irq/58-eth020:20:123
2461199441,34cyclictest9579-21ssh00:10:121
2461199441,34cyclictest9579-21ssh00:10:121
24633994321,17cyclictest35966irq/58-eth023:40:123
24611994223,14cyclictest21-21ksoftirqd/121:40:131
2463399414,22cyclictest35966irq/58-eth000:20:123
2462299412,28cyclictest527-21sh00:05:142
2461199403,25cyclictest5979-21ssh23:50:131
2463399382,22cyclictest35966irq/58-eth022:55:123
2460099387,23cyclictest6479-21kworker/u8:021:15:140
2463399372,24cyclictest35966irq/58-eth022:10:133
2462299378,21cyclictest630-21Async09:36:312
24611993710,17cyclictest21-21ksoftirqd/120:20:121
24600993721,10cyclictest0-21swapper/022:54:240
24622993611,12cyclictest11968-21sh22:05:132
2461199362,23cyclictest23088-21ssh21:35:141
24633993517,11cyclictest35966irq/58-eth021:10:133
2462299351,24cyclictest25901-21ssh23:25:132
24633993413,11cyclictest35966irq/58-eth021:15:143
2462299346,18cyclictest27346irq/40-dwc2_hso19:35:152
2462299341,20cyclictest15036-21ssh23:55:152
24633993213,13cyclictest35966irq/58-eth020:30:133
2460099323,21cyclictest24431-21ssh22:30:130
2462299312,22cyclictest20510-21ssh21:32:232
2461199313,4cyclictest131rcu_preempt23:45:121
24611993122,4cyclictest31457-21ssh23:10:131
2461199312,21cyclictest14924-21/usr/sbin/munin19:30:211
2460099318,9cyclictest12-21ksoftirqd/023:40:230
2460099312,22cyclictest28137-21ssh22:13:250
2463399304,15cyclictest35966irq/58-eth000:15:133
2463399304,15cyclictest35966irq/58-eth000:15:133
2463399303,20cyclictest35966irq/58-eth021:20:123
2462299302,21cyclictest0-21swapper/222:10:142
2463399293,15cyclictest35966irq/58-eth021:55:113
24633992914,9cyclictest35966irq/58-eth022:30:133
2462299291,16cyclictest4504-21ssh22:55:132
2461199292,21cyclictest21-21ksoftirqd/122:55:121
2461199290,25cyclictest17439-21ssh21:50:131
2460099292,19cyclictest5235-21memory19:20:210
2460099292,19cyclictest29255-21sed19:45:210
2462299282,15cyclictest13454-21ssh23:00:132
2462299282,11cyclictest0-21swapper/219:55:142
2462299281,20cyclictest27346irq/40-dwc2_hso20:00:122
2461199282,20cyclictest15724-21ssh21:48:241
2463399277,12cyclictest35966irq/58-eth000:30:133
2463399274,10cyclictest35966irq/58-eth023:35:153
2463399272,16cyclictest35966irq/58-eth021:35:143
24633992710,12cyclictest35966irq/58-eth021:45:143
2462299273,9cyclictest0-21swapper/223:19:252
2462299272,16cyclictest27346irq/40-dwc2_hso00:20:132
2461199272,19cyclictest2990-21ssh22:00:131
2460099273,18cyclictest29412-21ssh23:45:140
2460099273,17cyclictest7940-21sh23:15:130
2460099273,16cyclictest22102-21ssh00:35:160
2463399262,18cyclictest0-21swapper/322:26:243
2462299262,16cyclictest5668-21find21:05:122
24611992614,6cyclictest10257-21diskmemload00:25:151
24600992615,6cyclictest0-21swapper/020:45:140
2460099261,19cyclictest5050-21ssh21:25:120
24600992610,8cyclictest0-21swapper/000:04:530
24633992512,8cyclictest33-21ksoftirqd/321:31:153
2462299252,16cyclictest0-21swapper/219:25:212
2462299251,8cyclictest688466ssh21:43:302
2462299251,6cyclictest2320666ssh21:15:252
24622992512,10cyclictest8236-21sshd23:31:392
2462299250,13cyclictest4548-21timerandwakeup21:00:272
2462299250,13cyclictest4548-21timerandwakeup21:00:272
2461199255,12cyclictest11447-21ssh23:35:151
2461199251,18cyclictest20929-21ssh22:10:141
2460099254,17cyclictest12-21ksoftirqd/022:20:120
24600992516,5cyclictest151rcuc/022:05:150
24600992512,7cyclictest0-21swapper/021:20:130
24600992511,7cyclictest0-21swapper/019:50:100
24633992410,10cyclictest0-21swapper/300:10:283
24633992410,10cyclictest0-21swapper/300:10:273
2462299246,11cyclictest27346irq/40-dwc2_hso20:05:132
2462299242,14cyclictest0-21swapper/219:45:132
2462299241,6cyclictest261166rm22:53:162
2462299241,6cyclictest2300066rm22:28:472
2462299241,6cyclictest1167366ssh23:51:232
24622992412,9cyclictest0-21swapper/220:10:192
2462299241,19cyclictest0-21swapper/223:05:132
2462299241,13cyclictest18697-21ssh00:15:162
2462299241,13cyclictest18697-21ssh00:15:162
2462299240,21cyclictest6470-21ls22:00:232
2461199242,17cyclictest2481-21ssh23:30:151
2461199242,16cyclictest10652-21ssh21:10:141
2461199241,18cyclictest5677-21apt-get21:05:121
2460099242,16cyclictest19065-21ntp_192.168.11422:25:240
24600992417,4cyclictest2981-21ssh23:10:290
2460099241,17cyclictest17448-21ssh21:50:130
24600992411,7cyclictest0-21swapper/000:05:550
2463399239,10cyclictest35966irq/58-eth022:50:123
2463399232,14cyclictest35966irq/58-eth022:20:123
2463399231,15cyclictest0-21swapper/321:00:143
2463399231,15cyclictest0-21swapper/321:00:133
2462299232,5cyclictest0-21swapper/219:10:212
2462299232,16cyclictest0-21swapper/223:22:242
2462299232,16cyclictest0-21swapper/219:30:212
2462299232,16cyclictest0-21swapper/219:15:212
2462299232,16cyclictest0-21swapper/200:38:252
2462299231,7cyclictest93066snmpd21:45:112
2462299231,6cyclictest3139766ssh21:55:432
24622992313,7cyclictest11218-21grep21:10:172
24622992311,5cyclictest18080-21sshd22:44:252
2462299231,12cyclictest32100-21irqrtprio23:10:192
24622992310,10cyclictest27-21ksoftirqd/200:30:142
2462299230,7cyclictest1279966ssh22:22:172
2461199239,4cyclictest131rcu_preempt00:30:171
2461199237,4cyclictest21-21ksoftirqd/123:02:081
2461199231,15cyclictest15035-21sh23:55:141
2460099239,8cyclictest12-21ksoftirqd/023:21:420
24600992317,4cyclictest12-21ksoftirqd/000:30:300
2460099231,18cyclictest23826-21ls20:50:170
24600992311,4cyclictest12-21ksoftirqd/021:37:470
2463399222,13cyclictest35966irq/58-eth023:15:123
24633992210,9cyclictest0-21swapper/323:45:193
2462299224,7cyclictest0-21swapper/200:10:132
2462299224,7cyclictest0-21swapper/200:10:122
2462299222,11cyclictest537-21ntp_192.168.11423:45:242
2462299221,11cyclictest8594-21missed_timers21:25:212
2461199222,14cyclictest17776-21ntp_states21:30:251
2461199221,7cyclictest18683-21cut20:45:011
24611992215,5cyclictest0-21swapper/120:05:121
24611992214,5cyclictest0-21swapper/121:00:021
2461199221,11cyclictest23987-21ssh00:00:121
2460099228,5cyclictest131rcu_preempt23:55:540
2460099227,5cyclictest12-21ksoftirqd/023:34:010
2460099227,4cyclictest12-21ksoftirqd/023:38:370
2460099223,10cyclictest12-21ksoftirqd/022:42:510
24600992210,6cyclictest12-21ksoftirqd/021:30:250
2463399217,10cyclictest35966irq/58-eth023:00:133
2463399216,11cyclictest0-21swapper/322:40:303
2463399212,13cyclictest35966irq/58-eth022:00:133
24633992110,6cyclictest0-21swapper/322:17:103
2462299217,10cyclictest30659-21ssh21:20:202
2462299216,3cyclictest877-21lldpd21:51:132
2462299216,12cyclictest27-21ksoftirqd/200:00:222
2462299214,14cyclictest24134-21kthreadcore20:15:202
2462299213,9cyclictest27158-21/usr/sbin/munin20:20:132
2462299211,9cyclictest13763-21idleruntime-cro20:40:012
2462299211,16cyclictest0-21swapper/222:35:032
2462299211,11cyclictest9624-21cron22:40:012
2462299211,11cyclictest30783-21kthreadcore22:15:202
2462299211,11cyclictest11547-21df_inode23:35:162
2462299211,10cyclictest21154-21/usr/sbin/munin19:40:282
2462299211,10cyclictest21154-21/usr/sbin/munin19:40:282
2461199219,2cyclictest131rcu_preempt20:40:011
2461199214,10cyclictest21-21ksoftirqd/119:15:131
2461199211,7cyclictest23111-21ntp_192.168.11421:15:231
2461199211,5cyclictest24519-21df22:30:141
24611992111,7cyclictest16495-21cron23:20:011
2461199211,10cyclictest0-21swapper/121:26:481
24611992110,8cyclictest29299-21cron19:15:011
2460099218,4cyclictest12-21ksoftirqd/000:15:130
2460099218,4cyclictest12-21ksoftirqd/000:15:120
2460099218,10cyclictest13040-21cron23:00:010
2460099217,2cyclictest131rcu_preempt00:20:250
2460099212,14cyclictest29891-21ssh22:15:130
24600992115,4cyclictest0-21swapper/021:45:270
24600992113,5cyclictest151rcuc/022:45:110
2460099211,17cyclictest26199-21if_err_eth023:25:160
2460099211,16cyclictest11396-21ps21:10:180
2463399209,8cyclictest0-21swapper/323:59:533
2463399208,7cyclictest35966irq/58-eth023:10:133
2463399205,8cyclictest0-21swapper/319:30:103
2463399203,11cyclictest0-21swapper/322:05:243
2463399203,11cyclictest0-21swapper/319:10:223
2463399202,12cyclictest0-21swapper/300:25:143
2463399202,11cyclictest0-21swapper/321:40:123
24633992011,6cyclictest0-21swapper/300:00:313
2462299209,3cyclictest0-21swapper/223:40:122
2462299201,15cyclictest32167-21fschecks_time20:25:162
2462299201,10cyclictest32461-21cut19:50:172
2462299201,10cyclictest22668-21ntp_states22:45:252
2461199208,3cyclictest131rcu_preempt22:45:171
2461199202,11cyclictest21-21ksoftirqd/122:05:141
24611992013,4cyclictest27086-21kernelversion21:55:171
24611992012,6cyclictest1909-21sshd22:52:231
2461199201,14cyclictest1736-21ntp_192.168.11419:50:211
2460099209,3cyclictest0-21swapper/019:55:200
2460099206,4cyclictest12-21ksoftirqd/022:01:530
2460099206,4cyclictest12-21ksoftirqd/020:20:180
2460099206,11cyclictest22310-21latency_hist23:05:020
2460099202,2cyclictest131rcu_preempt23:05:060
2460099202,13cyclictest4426-21users21:40:320
24600992014,4cyclictest0-21swapper/023:50:590
24600992014,4cyclictest0-21swapper/000:10:180
24600992014,4cyclictest0-21swapper/000:10:180
24600992013,5cyclictest0-21swapper/022:38:430
24600992011,6cyclictest0-21swapper/019:39:200
2460099201,14cyclictest14891-21ls19:30:220
2463399193,11cyclictest0-21swapper/323:21:343
2463399193,11cyclictest0-21swapper/319:50:213
2463399193,10cyclictest0-21swapper/319:20:203
2463399192,7cyclictest0-21swapper/319:25:133
24633991913,4cyclictest0-21swapper/323:05:223
24633991912,4cyclictest0-21swapper/322:35:133
2463399191,11cyclictest0-21swapper/319:45:153
2462299194,6cyclictest27-21ksoftirqd/220:45:202
2462299192,9cyclictest15838-21kthreadcore20:40:192
2462299192,8cyclictest28399-21apt-get20:55:132
2462299191,9cyclictest4352-21/usr/sbin/munin20:30:112
2462299191,9cyclictest1902-21/usr/sbin/munin19:20:142
2462299191,15cyclictest23596-21apt-get20:50:132
2461199193,3cyclictest10257-21diskmemload23:29:281
2461199193,11cyclictest22474-21acpi23:05:111
2461199192,13cyclictest16928-21ssh23:20:151
2461199191,9cyclictest0-21swapper/123:41:521
2461199191,9cyclictest0-21swapper/122:42:321
2461199191,9cyclictest0-21swapper/121:20:461
2461199191,3cyclictest131rcu_preempt22:35:141
2461199191,3cyclictest131rcu_preempt22:15:241
2461199191,3cyclictest0-21swapper/100:35:231
2461199191,10cyclictest8734-21kthreadcore20:00:191
2460099199,7cyclictest8675-21cat00:25:310
2460099195,4cyclictest131rcu_preempt20:40:290
2460099195,11cyclictest0-21swapper/019:40:130
2460099195,11cyclictest0-21swapper/019:40:120
2460099192,4cyclictest12-21ksoftirqd/020:00:240
24600991913,4cyclictest0-21swapper/019:10:250
2463399188,6cyclictest35966irq/58-eth022:45:123
2463399182,10cyclictest0-21swapper/320:15:133
24633991810,6cyclictest0-21swapper/323:30:023
24633991810,5cyclictest0-21swapper/300:05:333
2462299180,9cyclictest24689-21latency_hist19:10:032
2461199188,8cyclictest16427-21df19:35:171
2461199184,3cyclictest131rcu_preempt21:05:011
2461199184,3cyclictest131rcu_preempt21:05:011
2461199181,9cyclictest23409-21latency_hist20:50:011
2461199181,6cyclictest22135-21awk20:15:021
2461199181,14cyclictest6130-21kthreadcore19:55:201
2461199181,14cyclictest21001-21cut00:15:231
2461199181,14cyclictest21001-21cut00:15:221
2461199181,13cyclictest4370-21apt-get20:30:121
2460099188,2cyclictest12-21ksoftirqd/020:10:150
2460099185,7cyclictest12-21ksoftirqd/020:05:230
2460099182,13cyclictest32344-21irqcore20:25:160
24600991812,4cyclictest0-21swapper/021:00:280
24600991812,4cyclictest0-21swapper/021:00:270
2460099181,13cyclictest7840-21ntp_states20:30:240
2463399178,4cyclictest0-21swapper/321:52:443
2463399171,13cyclictest0-21swapper/319:55:213
2461199179,5cyclictest930-21snmpd19:30:031
2461199171,7cyclictest32339-21irqcore20:25:161
2461199171,13cyclictest21753-21irqrtprio19:40:181
2461199171,13cyclictest21753-21irqrtprio19:40:171
24611991710,5cyclictest15819-21grep22:25:181
2460099175,9cyclictest131rcu_preempt19:15:170
2460099174,7cyclictest12-21ksoftirqd/020:55:150
2460099172,10cyclictest0-21swapper/020:15:130
2460099171,3cyclictest131rcu_preempt20:35:170
24600991711,3cyclictest0-21swapper/019:25:130
2460099171,12cyclictest10263-21awk21:10:010
2463399162,9cyclictest0-21swapper/320:40:143
2463399162,9cyclictest0-21swapper/320:00:133
2463399162,7cyclictest0-21swapper/321:05:143
2463399162,10cyclictest0-21swapper/319:40:133
2463399162,10cyclictest0-21swapper/319:40:133
2461199162,11cyclictest6488-21ssh22:20:131
2461199162,11cyclictest27420-21grep20:50:281
2461199161,12cyclictest3456-21cut19:20:181
2461199161,12cyclictest26518-21sed19:45:171
2461199161,12cyclictest22517-21cut20:15:151
2463399153,8cyclictest0-21swapper/319:35:143
2463399148,4cyclictest0-21swapper/320:45:233
2463399142,8cyclictest0-21swapper/320:25:123
2463399142,8cyclictest0-21swapper/320:10:133
2460099141,10cyclictest24601-21latency_hist19:10:010
2463399137,4cyclictest0-21swapper/320:55:293
2463399133,7cyclictest0-21swapper/320:55:013
2463399133,5cyclictest0-21swapper/320:35:143
2463399132,8cyclictest0-21swapper/320:10:003
2463399132,6cyclictest0-21swapper/319:15:213
2461199134,6cyclictest24675-21latency_hist19:10:011
2463399101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional