You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 22:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri May 22, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2431399591,46cyclictest25033-21rm12:35:132
2427899513,40cyclictest7917-21ssh10:20:131
2427899512,4cyclictest0-21swapper/109:20:111
2427899503,38cyclictest8656-21ssh11:50:121
2427899501,5cyclictest1256-21ssh11:10:121
2427899493,37cyclictest24709-21ssh11:05:131
2427899456,6cyclictest10078-21kworker/u8:009:15:131
2427899451,10cyclictest14507-21ssh09:30:111
2427199443,28cyclictest12251-21ssh12:10:120
2427199442,31cyclictest25989-21ssh10:30:140
2427899432,4cyclictest0-21swapper/107:55:111
2427899422,5cyclictest0-21swapper/107:35:161
2433999413,23cyclictest35966irq/58-eth011:55:143
24313994124,11cyclictest261rcuc/211:10:122
2427899413,30cyclictest17639-21ssh11:55:131
2433999403,24cyclictest35966irq/58-eth010:10:123
24339994011,18cyclictest35966irq/58-eth011:20:123
24313994033,4cyclictest27346irq/40-dwc2_hso12:30:142
2427899405,25cyclictest28380-21ssh11:25:121
2427899402,3cyclictest0-21swapper/107:10:131
2427899401,8cyclictest12248-21apt-get12:10:121
2431399399,25cyclictest27346irq/40-dwc2_hso09:10:122
2427899392,3cyclictest0-21swapper/108:25:131
2427199391,29cyclictest5360-21ssh09:25:120
2427199391,29cyclictest5360-21ssh09:25:120
2427899381,5cyclictest0-21swapper/109:00:131
2427899381,4cyclictest0-21swapper/108:20:121
2427199381,31cyclictest20792-21ssh10:45:120
2427899371,9cyclictest4260-21fschecks_count10:00:141
2427899361,9cyclictest12302-21tr09:10:201
2427899361,6cyclictest29118-21memory07:45:231
2427899361,16cyclictest25962-21apt-get10:30:121
2427199361,31cyclictest3286-21ssh12:05:140
2427199361,31cyclictest3286-21ssh12:05:130
24339993518,11cyclictest35966irq/58-eth011:30:143
2427899351,8cyclictest2048-21kthreadcore09:40:191
2427899351,8cyclictest18868-21kthreadcore08:10:201
2427899351,12cyclictest6347-21ntp_states10:35:261
2433999344,20cyclictest35966irq/58-eth009:40:113
2427899347,24cyclictest31528-21cut08:55:221
2427899347,24cyclictest31528-21cut08:55:211
2427899341,9cyclictest30623-21rm12:20:151
2427899341,9cyclictest13183-21tr07:30:191
2427899341,8cyclictest30039-21kthreadcore12:00:241
2427899341,8cyclictest12559-21kthreadcore11:15:221
2427899341,7cyclictest9671-21sed08:00:191
2427899341,7cyclictest9671-21sed08:00:191
2427899341,5cyclictest31383-21tr10:50:201
2427899340,9cyclictest25504-21kthreadcore09:35:221
2433999331,25cyclictest35966irq/58-eth010:25:123
2433999331,20cyclictest35966irq/58-eth008:30:123
2427899338,22cyclictest17773-21ntp_states11:35:261
2427899337,23cyclictest22354-21fschecks_count10:10:161
2427899332,9cyclictest19364-21sed08:45:191
2427899331,9cyclictest8725-21cut09:05:221
2427899331,9cyclictest8725-21cut09:05:221
2427899331,9cyclictest2654-21tr07:20:181
2427899331,9cyclictest2654-21tr07:20:181
2427899331,9cyclictest24290-21cmp07:10:011
2427899331,9cyclictest22369-21cut12:15:191
2427899331,9cyclictest22369-21cut12:15:181
2427899331,8cyclictest9908-21ntp_kernel_err07:25:231
2427899331,8cyclictest5187-21ssh11:30:161
2427899331,8cyclictest31402-21fschecks_count10:15:151
2427899331,8cyclictest30721-21fschecks_count07:50:151
2427899331,8cyclictest25202-21fschecks_count12:35:161
2427899331,8cyclictest13289-21sort08:05:211
2427899331,8cyclictest12937-21ntp_192.168.11409:45:241
2427899331,8cyclictest11551-21ssh09:27:021
2427899331,8cyclictest11551-21ssh09:27:011
2427899331,7cyclictest17356-21ntp_192.168.11408:40:241
2427899331,7cyclictest13367-21tr10:40:211
2427899331,6cyclictest9535-21sed08:35:181
2427199333,28cyclictest11724-21ssh10:40:120
24313993218,9cyclictest261rcuc/211:45:112
2427899325,18cyclictest21-21ksoftirqd/110:05:121
2427899322,8cyclictest3753-21latency_hist10:00:011
2427899322,7cyclictest15851-21/usr/sbin/munin12:30:171
2427899322,3cyclictest0-21swapper/108:50:241
2427899321,9cyclictest6894-21ps10:55:171
2427899321,9cyclictest6028-21kthreadcore12:05:221
2427899321,9cyclictest6028-21kthreadcore12:05:221
2427899321,8cyclictest20867-21timerandwakeup10:25:271
2427899321,7cyclictest23399-21cut07:40:191
2427899321,7cyclictest18913-21irqcore09:50:181
2427899321,6cyclictest8880-21/usr/sbin/munin08:30:311
2427899321,10cyclictest8876-21grep12:25:211
2427899321,10cyclictest22772-21irqrtprio08:15:171
2427199323,22cyclictest10273-21ssh11:15:120
24271993221,6cyclictest121ksoftirqd/007:20:120
24271993221,6cyclictest121ksoftirqd/007:20:110
24271993217,4cyclictest29970-21apt-config12:20:030
24271993217,4cyclictest29970-21apt-config12:20:020
24271993211,10cyclictest12-21ksoftirqd/010:20:120
2427199320,22cyclictest15637-21ssh11:00:130
2433999311,16cyclictest35966irq/58-eth009:20:113
2433999311,14cyclictest35966irq/58-eth009:35:133
2427899311,7cyclictest26714-21netstat11:40:241
2427899311,7cyclictest23631-21usage_Anybus11:20:301
2427899311,6cyclictest29392-21df_inode07:15:151
24278993112,16cyclictest21066-21idleruntime10:45:161
2427199312,23cyclictest16908-21rm10:25:100
2427199311,21cyclictest0-21swapper/008:55:130
2427199311,21cyclictest0-21swapper/008:55:130
24339993010,7cyclictest0-21swapper/311:50:253
2431399303,15cyclictest27346irq/40-dwc2_hso08:45:122
24313993012,7cyclictest0-21swapper/209:25:132
24313993012,7cyclictest0-21swapper/209:25:132
2427899304,5cyclictest32130-21cpu11:45:121
2427899301,5cyclictest16330-21irqrtprio11:00:191
2427199304,21cyclictest6958-21ssh12:25:130
24271993022,5cyclictest28272-21kworker/u8:011:30:140
2433999294,15cyclictest35966irq/58-eth009:30:123
2431399295,14cyclictest27346irq/40-dwc2_hso11:50:122
2431399293,9cyclictest22225-21ssh10:10:132
2431399293,17cyclictest27346irq/40-dwc2_hso07:40:122
2431399292,20cyclictest0-21swapper/208:15:122
2427199292,20cyclictest10089-21diskmemload11:45:130
2433999282,16cyclictest35966irq/58-eth010:40:133
2431399282,19cyclictest0-21swapper/208:30:152
2427199283,16cyclictest10564-21ssh09:10:120
2431399273,16cyclictest27346irq/40-dwc2_hso10:55:122
2431399272,20cyclictest0-21swapper/208:55:142
2431399272,20cyclictest0-21swapper/208:55:132
2431399272,19cyclictest0-21swapper/208:20:132
2431399271,14cyclictest13160-21ssh10:05:132
24313992711,12cyclictest3963-21kthreadcore10:35:202
2427199272,19cyclictest2361966ssh09:35:140
2433999265,14cyclictest35966irq/58-eth011:00:143
2433999262,12cyclictest35966irq/58-eth010:05:123
2431399265,14cyclictest27346irq/40-dwc2_hso07:55:112
2433999253,15cyclictest35966irq/58-eth007:45:143
2433999253,12cyclictest35966irq/58-eth011:05:123
24339992513,8cyclictest35966irq/58-eth009:05:143
24339992513,8cyclictest35966irq/58-eth009:05:133
2431399258,14cyclictest1690-21ntpq12:20:262
2431399252,15cyclictest27346irq/40-dwc2_hso09:00:132
24313992513,9cyclictest0-21swapper/208:05:172
24313992512,9cyclictest14481-21/usr/sbin/munin09:30:262
24313992512,6cyclictest27-21ksoftirqd/209:50:132
24313992511,9cyclictest25991-21rm10:30:132
2427199254,11cyclictest0-21swapper/008:10:140
2427199252,19cyclictest753-21sendmail-mta09:45:130
24339992414,5cyclictest0-21swapper/312:30:133
2433999241,10cyclictest35966irq/58-eth007:15:143
2431399244,13cyclictest27346irq/40-dwc2_hso07:20:122
2431399244,13cyclictest27346irq/40-dwc2_hso07:20:112
2431399243,7cyclictest0-21swapper/209:40:132
2431399242,17cyclictest27346irq/40-dwc2_hso10:15:142
2431399241,9cyclictest0-21swapper/212:00:142
2431399241,7cyclictest131rcu_preempt11:25:122
2431399241,15cyclictest27346irq/40-dwc2_hso10:00:112
2427199242,16cyclictest31252-21ssh10:15:120
2431399233,11cyclictest32760-21timerwakeupswit09:20:282
24313992316,4cyclictest27-21ksoftirqd/212:10:122
24313992312,9cyclictest29570-21ssh11:05:332
2431399231,18cyclictest27249-21kthreadcore07:45:202
24313992311,5cyclictest14606-21ssh10:22:432
2427199235,9cyclictest12-21ksoftirqd/010:55:110
2427199232,13cyclictest19490-21diskstats11:20:150
2427199231,18cyclictest22169-21apt-get08:15:110
2433999223,9cyclictest35966irq/58-eth008:45:143
2433999222,15cyclictest35966irq/58-eth012:05:133
2433999222,15cyclictest35966irq/58-eth012:05:123
2433999221,14cyclictest35966irq/58-eth011:15:123
2431399229,6cyclictest10072-21sshd09:09:082
2431399229,6cyclictest10072-21sshd09:09:072
2431399226,13cyclictest29980-21kthreadcore07:15:192
2431399226,13cyclictest29066-21cron07:15:022
2431399223,16cyclictest24133-21kthreadcore12:15:212
2431399223,16cyclictest24133-21kthreadcore12:15:212
24313992212,8cyclictest0-21swapper/208:50:292
2431399221,18cyclictest19778-21cut09:15:152
2431399221,14cyclictest0-21swapper/211:15:112
2431399221,12cyclictest3509-21cut12:05:172
2431399221,12cyclictest3509-21cut12:05:162
2431399220,15cyclictest8031-21ls08:00:162
2431399220,15cyclictest8031-21ls08:00:152
2427199222,15cyclictest2652-21ssh10:35:120
2427199221,17cyclictest11153-21ls08:00:230
2427199221,17cyclictest11153-21ls08:00:220
2427199221,16cyclictest30277-21ssh12:20:060
2427199221,16cyclictest14504-21apt-get09:30:130
24271992210,9cyclictest26253-21cron12:00:020
2433999219,9cyclictest0-21swapper/310:49:103
2433999218,10cyclictest0-21swapper/309:25:153
2433999218,10cyclictest0-21swapper/309:25:143
2433999214,9cyclictest35966irq/58-eth012:35:143
2431399219,9cyclictest23543-21ssh11:20:302
2431399217,11cyclictest0-21swapper/207:30:192
2431399215,14cyclictest30171-21cat10:50:162
2431399211,17cyclictest16881-21/usr/sbin/munin10:25:182
24313992111,7cyclictest13802-21sh11:35:032
2431399211,12cyclictest30344-21kthreadcore09:55:222
2431399211,11cyclictest9391-21cut12:25:212
2431399211,11cyclictest6629-21fschecks_count07:25:132
2431399211,11cyclictest21240-21netstat11:55:242
24313992110,9cyclictest26082-21ssh10:46:002
24313992110,6cyclictest27346irq/40-dwc2_hso11:40:142
2427199213,3cyclictest10089-21diskmemload12:02:180
2427199211,17cyclictest24680-21uniq07:10:160
2427199211,16cyclictest16084-21apt-get07:35:140
2433999209,9cyclictest0-21swapper/310:21:343
2433999202,12cyclictest35966irq/58-eth011:25:133
2433999201,14cyclictest0-21swapper/312:10:163
2431399201,9cyclictest17653-21needreboot11:35:232
2431399201,8cyclictest20837-21latency_hist07:40:012
2431399201,17cyclictest13694-21tr10:40:202
2431399201,15cyclictest32096-21hwlatdetect08:25:162
2431399201,11cyclictest16037-21ssh11:00:192
2431399201,11cyclictest12946-21ssh09:45:242
2431399201,10cyclictest26121-21kthreadcore09:35:222
2427199203,14cyclictest29843-21ssh10:50:110
2427199201,15cyclictest7515-21cut09:05:200
2427199201,15cyclictest7515-21cut09:05:200
2427199201,15cyclictest27506-21ssh09:55:130
2427199201,11cyclictest28510-21entropy11:25:140
24271992010,8cyclictest614-21dbus-daemon07:34:130
24271992010,8cyclictest29541-21users11:05:320
2433999198,9cyclictest0-21swapper/309:13:253
2433999197,5cyclictest0-21swapper/311:10:133
2433999192,10cyclictest0-21swapper/307:35:153
24339991910,6cyclictest35966irq/58-eth012:20:133
24339991910,6cyclictest0-21swapper/312:02:573
24339991910,6cyclictest0-21swapper/311:41:373
24339991910,6cyclictest0-21swapper/310:30:173
24339991910,6cyclictest0-21swapper/309:49:213
2431399191,9cyclictest17566-21fschecks_time08:10:172
2431399191,9cyclictest14169-21hwlatdetect08:40:172
2431399191,8cyclictest30772-21fschecks_time07:50:162
2427199195,10cyclictest20125-21irqcore09:15:170
2427199194,10cyclictest0-21swapper/008:35:130
2427199193,12cyclictest0-21swapper/007:15:140
2427199193,10cyclictest0-21swapper/007:45:140
2427199192,7cyclictest1384-21diskstats11:10:150
2427199192,15cyclictest5228-21kthreadcore10:00:180
2427199192,14cyclictest32213-21memory09:20:210
2427199191,3cyclictest131rcu_preempt11:40:130
2427199191,15cyclictest13113-21find10:05:120
2427199191,14cyclictest24547-21apt-get12:35:060
2427199191,10cyclictest20378-21kthreadcore08:45:190
2433999189,6cyclictest35966irq/58-eth011:45:133
2433999189,6cyclictest0-21swapper/310:39:593
2433999189,6cyclictest0-21swapper/310:00:033
2433999189,6cyclictest0-21swapper/309:50:193
2433999189,6cyclictest0-21swapper/309:17:483
2433999188,6cyclictest0-21swapper/310:50:473
2433999187,8cyclictest0-21swapper/312:25:143
2433999183,10cyclictest0-21swapper/309:55:143
2433999182,10cyclictest0-21swapper/308:40:143
2433999182,10cyclictest0-21swapper/308:15:123
24339991812,4cyclictest0-21swapper/311:35:203
2433999181,13cyclictest0-21swapper/310:15:133
2433999181,12cyclictest0-21swapper/308:55:013
24339991810,6cyclictest0-21swapper/310:55:033
2431399181,9cyclictest9277-21fschecks_count08:35:162
2427199189,6cyclictest18622-21sort09:50:160
2427199188,8cyclictest31875-21apt-get08:25:120
2427199182,13cyclictest32653-21apt-get09:40:110
2427199181,8cyclictest10478-21tr11:50:200
2427199181,13cyclictest16618-21cut12:30:170
24271991810,6cyclictest4657-21cut07:55:190
2433999179,6cyclictest0-21swapper/312:15:263
2433999179,6cyclictest0-21swapper/312:15:253
2433999174,8cyclictest0-21swapper/308:00:143
2433999174,8cyclictest0-21swapper/308:00:133
2433999172,10cyclictest0-21swapper/307:30:133
2431399171,13cyclictest24382-21latency_hist07:10:012
2427199177,8cyclictest14001-21sshd11:35:080
2427199173,3cyclictest131rcu_preempt10:10:250
2427199172,12cyclictest30786-21tune2fs07:50:160
2427199172,12cyclictest3052-21kthreadcore09:00:200
2427199171,8cyclictest14921-21kthreadcore08:40:190
2427199171,14cyclictest12954-21irqcore08:05:190
2427199170,10cyclictest21038-21cut07:40:140
2433999163,7cyclictest0-21swapper/308:55:133
2433999163,7cyclictest0-21swapper/308:55:133
2433999162,10cyclictest0-21swapper/307:25:113
2427199168,6cyclictest4935-21sed08:30:180
2427199161,12cyclictest31684-21latency_hist08:25:020
2427199161,11cyclictest24405-21latency_hist07:10:020
2427199161,10cyclictest8348-21kthreadcore07:25:180
2427199160,13cyclictest26635-21kthreadcore08:50:210
2433999155,5cyclictest0-21swapper/308:20:123
2433999151,9cyclictest0-21swapper/308:10:133
2433999151,12cyclictest0-21swapper/307:55:013
2433999149,3cyclictest0-21swapper/307:10:293
2433999143,6cyclictest0-21swapper/307:55:133
2433999142,8cyclictest35966irq/58-eth009:00:143
2433999142,8cyclictest35966irq/58-eth007:40:113
2433999141,7cyclictest0-21swapper/308:05:303
2433999131,9cyclictest0-21swapper/307:20:253
2433999131,9cyclictest0-21swapper/307:20:243
2433999127,3cyclictest0-21swapper/308:36:393
2433999122,6cyclictest0-21swapper/308:25:143
2433999101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional