You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 11:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 20, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
81799725,50cyclictest28618-21ssh22:45:130
842996441,17cyclictest35966irq/58-eth021:15:133
81799642,53cyclictest24834-21ssh22:25:120
842996241,12cyclictest35966irq/58-eth022:40:133
81799574,40cyclictest19578-21ssh22:40:130
82599554,8cyclictest131rcu_preempt22:20:131
81799532,38cyclictest5196-21ssh22:50:130
825995112,8cyclictest131rcu_preempt00:10:141
817994928,14cyclictest0-21swapper/022:20:120
82599485,39cyclictest17816-21ssh23:15:131
82599483,7cyclictest131rcu_preempt21:00:161
817994831,11cyclictest0-21swapper/000:05:180
81799482,28cyclictest19320-21ssh21:10:150
825994710,10cyclictest131rcu_preempt19:55:141
83499464,29cyclictest15865-21ssh22:20:142
84299453,26cyclictest35966irq/58-eth021:50:133
81799453,6cyclictest131rcu_preempt20:05:160
81799453,30cyclictest7234-21ssh23:45:140
82599447,34cyclictest14167-21ssh22:55:131
82599431,28cyclictest31937-21ssh21:35:141
84299425,26cyclictest35966irq/58-eth020:40:123
84299423,28cyclictest35966irq/58-eth021:30:153
842994222,9cyclictest35966irq/58-eth022:45:133
81799424,26cyclictest32240-21ssh23:05:140
81799422,34cyclictest32384-21ssh00:35:120
84299413,23cyclictest35966irq/58-eth000:35:123
84299391,23cyclictest35966irq/58-eth000:05:183
825993913,17cyclictest21-21ksoftirqd/120:20:131
81799399,25cyclictest12-21ksoftirqd/000:15:140
834993822,10cyclictest27154-21ssh23:36:312
834993723,11cyclictest27-21ksoftirqd/222:35:132
83499372,19cyclictest7473-21ssh00:38:422
83499371,28cyclictest16178-21ssh23:50:122
825993711,10cyclictest21-21ksoftirqd/100:18:421
81799371,28cyclictest8803-21ssh23:10:120
82599362,22cyclictest17566-21ssh21:45:131
84299355,20cyclictest35966irq/58-eth023:00:143
82599359,21cyclictest21-21ksoftirqd/120:25:131
82599351,26cyclictest26787-21ssh23:20:151
81799354,17cyclictest3224-21ssh21:55:140
84299333,23cyclictest0-21swapper/300:20:423
84299332,21cyclictest35966irq/58-eth023:35:133
84299332,15cyclictest35966irq/58-eth021:20:143
842993320,9cyclictest35966irq/58-eth022:20:133
83499333,19cyclictest27-21ksoftirqd/223:10:122
842993213,15cyclictest0-21swapper/322:15:143
83499323,22cyclictest27-21ksoftirqd/222:25:132
83499322,22cyclictest930-21snmpd21:35:142
84299316,16cyclictest35966irq/58-eth023:50:133
842993114,10cyclictest35966irq/58-eth022:50:143
842993113,11cyclictest0-21swapper/300:33:403
83499316,13cyclictest27346irq/40-dwc2_hso20:40:122
83499315,14cyclictest27346irq/40-dwc2_hso21:00:162
83499313,16cyclictest30606-21sh23:40:132
82599313,12cyclictest131rcu_preempt19:10:141
84299307,13cyclictest35966irq/58-eth023:20:143
84299302,21cyclictest0-21swapper/322:00:133
842993010,12cyclictest0-21swapper/322:25:133
83499308,17cyclictest31274-21kworker/2:200:34:422
83499307,13cyclictest27-21ksoftirqd/219:55:142
83499304,13cyclictest27346irq/40-dwc2_hso21:05:142
83499302,21cyclictest10433-21ssh00:20:422
83499302,21cyclictest0-21swapper/220:00:132
834993018,10cyclictest27-21ksoftirqd/223:45:142
83499301,18cyclictest3375-21ssh23:25:142
834993011,3cyclictest27-21ksoftirqd/221:23:482
82599302,22cyclictest27608-21ssh23:37:041
82599302,19cyclictest21-21ksoftirqd/120:00:121
842992913,10cyclictest35966irq/58-eth021:25:133
82599293,16cyclictest18900-21diskmemload00:30:121
82599292,16cyclictest32374-21ssh00:35:111
81799292,21cyclictest10740-21ssh23:28:310
817992914,7cyclictest131rcu_preempt00:30:130
817992913,6cyclictest12-21ksoftirqd/022:35:120
84299283,18cyclictest35966irq/58-eth021:35:143
83499283,18cyclictest167-21systemd-network20:55:132
83499283,14cyclictest27346irq/40-dwc2_hso00:10:122
83499282,21cyclictest0-21swapper/200:18:422
834992818,3cyclictest27346irq/40-dwc2_hso19:10:142
83499281,15cyclictest12261-21ssh22:00:132
82599282,19cyclictest9058-21ssh23:26:311
82599282,19cyclictest24850-21ssh22:25:131
817992817,7cyclictest0-21swapper/022:05:160
817992817,7cyclictest0-21swapper/022:05:160
83499272,18cyclictest27346irq/40-dwc2_hso23:30:132
83499271,16cyclictest32241-21ssh23:05:142
84299261,15cyclictest35966irq/58-eth000:25:113
83499266,11cyclictest27346irq/40-dwc2_hso22:40:122
82599266,14cyclictest19300-21ssh21:10:141
83499253,14cyclictest27346irq/40-dwc2_hso19:25:132
83499253,14cyclictest27346irq/40-dwc2_hso19:25:132
83499252,20cyclictest0-21swapper/221:25:132
83499252,20cyclictest0-21swapper/219:30:152
83499252,13cyclictest26833-21ssh23:00:262
83499251,17cyclictest0-21swapper/222:30:142
83499251,13cyclictest930-21snmpd19:15:152
82599252,18cyclictest8639-21ssh21:40:141
82599252,18cyclictest18950-21ssh23:32:311
82599252,16cyclictest23250-21ssh23:00:141
81799253,15cyclictest26673-21kworker/u8:222:15:140
817992512,9cyclictest8211-21cron21:40:020
81799251,14cyclictest135-21systemd-journal20:00:020
84299242,18cyclictest35966irq/58-eth021:45:133
83499243,15cyclictest0-21swapper/222:15:152
83499242,20cyclictest0-21swapper/220:10:142
83499242,19cyclictest0-21swapper/220:35:142
83499242,12cyclictest22884-21ssh21:30:142
83499241,8cyclictest21892-21ssh00:28:422
82599243,12cyclictest21-21ksoftirqd/100:20:141
82599242,17cyclictest25109-21ssh23:55:131
82599242,17cyclictest1485-21apt-get22:30:141
825992413,9cyclictest7689-21cat21:55:321
825992411,11cyclictest4501-21cron21:20:011
81799244,12cyclictest30604-21sh23:40:120
81799242,18cyclictest0-21swapper/000:00:150
81799242,10cyclictest26591-21ssh21:50:140
81799241,18cyclictest26313-21ssh23:35:310
817992410,10cyclictest23238-21diskmemload23:00:140
842992315,6cyclictest0-21swapper/321:40:133
842992312,7cyclictest35966irq/58-eth022:05:143
842992312,7cyclictest35966irq/58-eth022:05:143
842992311,4cyclictest0-21swapper/300:15:143
83499235,9cyclictest11756-21kthreadcore00:05:252
83499233,10cyclictest870-21ssh22:10:222
834992313,8cyclictest27-21ksoftirqd/221:17:592
834992312,5cyclictest9540-21ssh21:57:182
834992311,9cyclictest0-21swapper/220:25:142
84299225,9cyclictest0-21swapper/323:10:123
84299223,13cyclictest0-21swapper/323:26:313
842992214,2cyclictest0-21swapper/323:55:123
84299221,15cyclictest0-21swapper/321:10:143
84299221,11cyclictest35966irq/58-eth000:00:133
842992210,9cyclictest0-21swapper/300:10:143
83499228,4cyclictest27-21ksoftirqd/221:43:042
83499225,9cyclictest28938-21kthreadcore20:45:182
83499225,9cyclictest25040-21cpu19:35:162
83499224,10cyclictest18559-21kthreadcore23:15:192
83499223,16cyclictest26743-21acpi23:20:132
83499222,15cyclictest0-21swapper/222:55:202
834992211,9cyclictest3908-21ssh22:48:572
83499221,11cyclictest7722-21kthreadcore22:50:212
83499221,10cyclictest28623-21cut21:50:212
83499221,10cyclictest2318-21ssh00:00:202
82599221,14cyclictest4916-21grep20:55:171
82599221,12cyclictest23618-21kernelversion21:30:191
81799223,15cyclictest12397-21ssh23:30:130
84299214,8cyclictest0-21swapper/323:40:133
842992116,2cyclictest0-21swapper/322:35:133
842992111,6cyclictest35966irq/58-eth023:05:143
83499211,11cyclictest25343-21fschecks_time23:55:162
83499211,11cyclictest14235-21kthreadcore20:30:202
82599213,13cyclictest12250-21ssh22:00:121
825992113,5cyclictest21671-21irqcore22:05:201
825992113,5cyclictest21671-21irqcore22:05:201
82599211,15cyclictest32231-21ssh23:05:141
82599210,18cyclictest10713-21ls19:20:151
82599210,11cyclictest19757-21kworker/1:100:28:401
81799214,14cyclictest4943-21ssh21:20:150
81799213,12cyclictest0-21swapper/019:15:150
817992114,4cyclictest17692-21forks21:45:150
817992114,4cyclictest0-21swapper/021:35:140
817992112,6cyclictest2008-21cron19:45:010
84299207,10cyclictest0-21swapper/323:15:133
842992014,2cyclictest0-21swapper/319:55:143
842992012,3cyclictest0-21swapper/320:00:113
842992011,6cyclictest0-21swapper/322:10:213
83499209,4cyclictest23425-21grep21:10:322
83499204,8cyclictest8295-21kthreadcore19:50:192
83499203,14cyclictest11000-21irqcore19:20:192
83499202,9cyclictest3558-21cpu20:20:142
83499201,9cyclictest22304-21kthreadcore20:05:242
83499201,16cyclictest31363-21fschecks_time20:15:162
83499201,16cyclictest2406-21fschecks_time19:45:142
83499201,10cyclictest20121-21kthreadcore21:45:202
82599201,9cyclictest23532-21cat22:40:271
82599201,15cyclictest28633-21ssh22:45:141
82599201,12cyclictest0-21swapper/121:28:531
82599201,10cyclictest0-21swapper/119:50:191
81799205,2cyclictest9539-21apt-get21:00:160
81799204,6cyclictest12-21ksoftirqd/020:35:220
817992012,6cyclictest24779-21cron19:35:010
81799201,12cyclictest30247-21ssh22:10:120
81799200,3cyclictest131rcu_preempt00:20:120
84299199,6cyclictest35966irq/58-eth023:45:143
84299199,6cyclictest0-21swapper/322:59:343
84299193,11cyclictest0-21swapper/323:30:313
84299192,14cyclictest35966irq/58-eth021:55:143
84299192,10cyclictest0-21swapper/320:35:143
842991914,3cyclictest0-21swapper/320:15:133
842991911,3cyclictest0-21swapper/320:05:183
84299191,12cyclictest0-21swapper/321:00:323
83499196,6cyclictest32629-21if_eth020:50:182
834991911,3cyclictest0-21swapper/222:05:152
834991911,3cyclictest0-21swapper/222:05:152
83499191,10cyclictest30471-21kernelversion19:40:192
82599197,9cyclictest2170-21sed23:40:291
82599192,14cyclictest6066-21sed19:45:261
82599191,6cyclictest10623-21ssh22:35:161
825991912,5cyclictest30221-21/usr/sbin/munin22:10:141
82599191,14cyclictest1741-21ssh00:00:151
82599190,10cyclictest16647-21cut19:25:201
82599190,10cyclictest16647-21cut19:25:201
81799199,7cyclictest18886-21usage_EthcTask00:25:280
81799198,8cyclictest27702-21cut23:20:210
81799192,9cyclictest806-21cyclictest23:50:130
81799192,7cyclictest22541-21tr00:10:220
817991911,6cyclictest5580-21sshd19:11:310
81799191,15cyclictest28445-21cut21:15:160
81799191,12cyclictest16394-21latency20:30:230
84299182,10cyclictest0-21swapper/320:55:153
84299182,10cyclictest0-21swapper/320:20:143
842991812,3cyclictest0-21swapper/319:15:133
84299181,13cyclictest0-21swapper/322:30:183
84299181,13cyclictest0-21swapper/320:45:263
83499181,9cyclictest900-21cat19:10:012
82599183,10cyclictest26575-21ssh21:50:131
82599182,10cyclictest10892-21apt-get00:05:181
82599181,8cyclictest7839-21kthreadcore19:15:221
825991812,4cyclictest0-21swapper/120:45:211
825991811,5cyclictest8771-21/usr/sbin/munin23:10:271
825991811,5cyclictest0-21swapper/123:50:111
82599181,11cyclictest11239-21ssh23:45:301
825991810,6cyclictest32108-21cron20:50:021
825991810,5cyclictest5785-21kthreadcore21:20:301
81799189,6cyclictest26389-21grep23:20:010
81799185,10cyclictest28315-21missed_timers19:35:240
81799184,9cyclictest7021-21find19:50:120
81799182,13cyclictest1419-21latency_hist00:00:020
81799182,12cyclictest13914-21ssh21:25:120
81799181,7cyclictest24603-21cat21:30:220
81799181,5cyclictest12823-21irqrtprio22:00:170
84299172,10cyclictest0-21swapper/320:25:133
842991712,3cyclictest0-21swapper/320:50:123
842991712,3cyclictest0-21swapper/319:40:133
84299171,13cyclictest0-21swapper/319:50:163
82599171,8cyclictest23878-21kthreadcore20:40:191
82599171,12cyclictest9718-21tr22:15:231
82599171,12cyclictest30874-21awk20:15:001
82599171,12cyclictest24998-21/usr/sbin/munin19:35:341
81799179,6cyclictest9287-21sh21:00:010
81799173,11cyclictest17943-21switchtime21:05:280
81799172,12cyclictest1494-21ssh22:30:140
81799172,11cyclictest375-21irqrtprio20:50:180
81799172,11cyclictest0-21swapper/020:45:110
81799172,11cyclictest0-21swapper/019:45:120
81799172,10cyclictest0-21swapper/020:10:140
81799171,13cyclictest8193-21latency_hist20:25:020
81799171,12cyclictest14352-21fschecks_time22:55:160
84299161,12cyclictest0-21swapper/319:35:183
84299161,11cyclictest0-21swapper/319:30:013
84299161,11cyclictest0-21swapper/319:30:013
82599162,11cyclictest32471-21kthreadcore19:40:231
82599161,8cyclictest23340-21sed20:05:261
82599161,12cyclictest5853-21sed22:50:181
82599161,12cyclictest20220-21apt-get19:30:151
82599161,12cyclictest15765-21kthreadcore21:05:201
82599161,12cyclictest14972-21cut20:30:211
82599161,11cyclictest20631-21cut20:35:221
81799167,7cyclictest15012-21usage_EthcTask19:20:320
81799162,9cyclictest0-21swapper/020:15:140
81799162,2cyclictest131rcu_preempt20:00:160
817991611,3cyclictest22986-21hwlatdetect20:40:160
81799161,11cyclictest18961-21grep19:25:270
81799161,11cyclictest18961-21grep19:25:270
84299159,4cyclictest0-21swapper/319:10:123
84299151,8cyclictest0-21swapper/321:05:163
82599151,11cyclictest31332-21fschecks_time20:15:161
81799151,11cyclictest8364-21apt-get20:25:130
81799150,7cyclictest902-21latency_hist19:10:010
84299149,3cyclictest0-21swapper/320:10:143
84299149,2cyclictest0-21swapper/320:30:143
84299142,6cyclictest0-21swapper/319:45:113
84299141,9cyclictest0-21swapper/319:20:173
842991410,2cyclictest0-21swapper/319:30:143
82599141,10cyclictest943-21wc19:10:011
84299111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional