You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 00:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot4s.osadl.org (updated Sat Dec 20, 2025 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22921996524,23cyclictest27-21ksoftirqd/209:20:132
2291299531,25cyclictest16981-21ssh09:50:161
2291299481,4cyclictest131rcu_preempt09:55:121
2293399437,26cyclictest28366irq/58-eth010:20:163
2292199431,33cyclictest3767-21ssh09:25:132
2293399422,27cyclictest28366irq/58-eth008:55:133
22921994232,4cyclictest12538-21ssh11:35:142
22921994232,4cyclictest12538-21ssh11:35:142
2292199401,10cyclictest12396-21kthreadcore07:30:232
2291299402,15cyclictest20542-21latency_hist08:15:011
2292199391,10cyclictest27646-21wc07:15:002
2292199391,10cyclictest27646-21wc07:15:002
2293399385,17cyclictest28366irq/58-eth011:00:143
2292199381,14cyclictest4120-21kthreadcore07:55:222
2292199381,10cyclictest26881-21kthreadcore09:55:192
2291299381,15cyclictest26919-21ssh11:05:261
2293399377,18cyclictest0-21swapper/312:05:113
2292199377,4cyclictest10977-21ssh09:10:212
2292199377,4cyclictest10977-21ssh09:10:212
2292199371,14cyclictest8349-21kthreadcore08:35:212
2292199371,12cyclictest29606-21grep07:50:182
2291299371,13cyclictest22277-21ssh11:40:201
2293399361,20cyclictest28366irq/58-eth011:40:143
2292199362,6cyclictest0-21swapper/212:16:002
2292199361,6cyclictest3258-21latency_hist11:30:012
2292199361,13cyclictest28943-21apt-get12:20:162
2292199361,12cyclictest15573-21users08:05:302
2292199361,10cyclictest17754-21kthreadcore07:35:222
2291299364,11cyclictest22863-21cyclictest07:25:161
2291299362,6cyclictest0-21swapper/112:21:001
2291299362,28cyclictest1801-21ssh12:05:101
2291299361,11cyclictest2999-21fschecks_time10:00:161
2291299361,11cyclictest26875-21cut08:55:171
2292199353,11cyclictest1245-21kthreadcore09:40:242
2292199351,9cyclictest18540-21kthreadcore10:25:222
2292199351,6cyclictest25555-21diskstats08:20:162
2292199351,6cyclictest20545-21ntp_192.168.11409:50:282
22921993512,20cyclictest3383-21ssh11:30:032
2292199351,16cyclictest17223-21/usr/sbin/munin08:45:172
2292199351,14cyclictest25683-21grep09:35:242
2292199351,10cyclictest31995-21missed_timers10:50:232
2292199350,7cyclictest1225-21ntp_192.168.11408:25:242
2292199350,12cyclictest26406-21users08:50:322
2291299352,10cyclictest15467-21latency_hist10:25:011
2291299351,9cyclictest12927-21kthreadcore08:40:181
2291299351,10cyclictest309-21ssh12:03:441
2291299351,10cyclictest20399-21tr09:15:211
2291299351,10cyclictest14232-21cut10:05:221
2292199342,11cyclictest10025-21kthreadcore10:20:252
2291299341,9cyclictest30309-21grep10:15:191
2291299341,10cyclictest22948-21ntp_states07:40:261
2293399337,15cyclictest0-21swapper/310:50:143
22933993314,10cyclictest28366irq/58-eth010:45:133
2292199335,5cyclictest5708-21irqrtprio10:55:192
2292199331,9cyclictest0-21swapper/210:12:542
2292199331,8cyclictest26688-21sendmail09:20:002
2292199331,7cyclictest24883-21df_inode10:30:162
2292199331,6cyclictest8447-21diskmemload11:20:362
2292199331,11cyclictest32105-21ssh12:02:572
2292199331,11cyclictest2365-21tr12:05:162
2292199331,10cyclictest5144-21diskstats07:25:182
2292199331,10cyclictest14799-21timerandwakeup12:10:282
2292199331,10cyclictest0-21swapper/210:45:542
2292199330,5cyclictest16725-21irqcore11:55:182
2292199330,11cyclictest26776-21needreboot11:05:232
2292199330,10cyclictest3185-21grep10:00:182
2291299332,12cyclictest9202-21cut08:00:221
2291299332,12cyclictest9202-21cut08:00:221
2291299331,9cyclictest447-21tr09:00:181
2291299331,9cyclictest20615-21/usr/sbin/munin08:15:251
2291299331,9cyclictest18458-21timerandwakeup12:30:291
2291299331,11cyclictest14936-21kthreadcore11:00:201
2291299331,11cyclictest14288-21latency12:10:221
2291299331,10cyclictest32669-21/usr/sbin/munin07:20:261
2291299330,9cyclictest19354-21kthreadcore08:45:211
2293399323,20cyclictest28366irq/58-eth011:55:133
2292199322,10cyclictest6705-21kthreadcore09:05:222
2292199321,7cyclictest9286-21/usr/sbin/munin11:15:172
2292199321,7cyclictest11762-21ssh10:05:132
2292199321,5cyclictest9639-21/usr/sbin/munin08:00:242
2292199321,5cyclictest9639-21/usr/sbin/munin08:00:242
2292199321,13cyclictest9191-21sed12:25:252
2292199321,13cyclictest30521-21/usr/sbin/munin11:45:152
2292199321,10cyclictest926-21kthreadcore07:20:212
2292199321,10cyclictest22285-21ssh12:34:032
2292199321,10cyclictest0-21swapper/211:12:562
2291299327,5cyclictest19808-21sed10:45:161
2291299321,9cyclictest8752-21kthreadcore09:45:191
2291299321,9cyclictest4078-21cat09:25:181
2291299321,9cyclictest17987-21/usr/sbin/munin11:20:121
2291299321,8cyclictest31928-21latency10:50:221
2291299321,12cyclictest28138-21unixbench_multi07:45:291
2291299321,11cyclictest14659-21cpu07:35:151
2293399316,15cyclictest0-21swapper/311:50:163
2292199312,12cyclictest8447-21diskmemload11:04:392
2292199311,9cyclictest32400-21users07:15:302
2292199311,6cyclictest29738-21ssh10:15:032
2292199311,5cyclictest28525-21rm12:35:452
2292199311,5cyclictest15974-21sh09:49:232
2291299318,3cyclictest3394-21kthreadcore10:35:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional