You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 22:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Jun 26, 2026 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
150193996241,13cyclictest289609-40mandb11:56:370
150194995951,5cyclictest289609-40mandb11:56:361
150194995332,5cyclictest171rcu_preempt08:15:311
150194995141,6cyclictest313617-21unixbench-2d12:35:461
150194995141,6cyclictest313617-21unixbench-2d12:35:461
150193995022,22cyclictest254973-21sshd11:00:320
150194994940,3cyclictest0-21swapper/109:50:301
150195994838,7cyclictest289609-40mandb11:56:412
150195994739,3cyclictest0-21swapper/209:35:012
150194994739,4cyclictest1003-21mta-sts-daemon11:20:351
150193994740,3cyclictest0-21swapper/010:00:330
150193994733,11cyclictest184763-21systemd-run09:05:010
150194994638,4cyclictest0-21swapper/111:10:021
15019399462,23cyclictest187333-21ssh09:10:330
150193994619,10cyclictest12650irq/32-eth010:20:300
150195994538,4cyclictest303039-21idleruntime-cro12:20:012
150194994537,5cyclictest151699-21systemd-run07:15:011
150194994534,8cyclictest160764-21systemd-run07:45:011
150193994517,9cyclictest186233-21diskmemload10:50:300
150193994517,9cyclictest186233-21diskmemload10:50:300
150194994436,4cyclictest210964-21grep09:50:011
15019399441,6cyclictest171rcu_preempt12:15:310
150193994416,21cyclictest267248-21sshd11:20:320
150195994337,3cyclictest0-21swapper/209:00:012
150195994332,7cyclictest0-21swapper/211:40:362
150194994335,4cyclictest0-21swapper/111:00:011
150194994335,4cyclictest0-21swapper/111:00:011
150193994335,3cyclictest195556-21kworker/u8:16-rpciod09:25:220
15019599424,34cyclictest258024-21ssh11:05:322
150195994234,5cyclictest1-21systemd12:35:012
150194994235,4cyclictest278438-21cron11:40:011
15019399423,32cyclictest258025-21sshd11:05:320
150193994232,8cyclictest217103-21grep10:00:010
150193994225,11cyclictest12650irq/32-eth012:20:400
150195994133,5cyclictest0-21swapper/209:55:332
150195994133,3cyclictest0-21swapper/212:10:322
150194994130,4cyclictest0-21swapper/109:55:331
150194994129,9cyclictest196445-21missed_timers09:25:281
150193994133,6cyclictest207901-21grep09:45:010
150194994032,5cyclictest227942-21unixbench_singl10:15:431
150194994031,6cyclictest263119-21sh11:15:011
150194994031,6cyclictest256975-21ssh11:05:001
150193994030,6cyclictest151ktimers/012:10:360
150194993931,5cyclictest1-21systemd08:50:021
150193993917,14cyclictest12650irq/32-eth008:40:310
150194993831,5cyclictest312069-21latency_hist12:35:001
150194993830,5cyclictest178655-21unixbench_singl08:40:421
150193993824,4cyclictest0-21swapper/008:20:010
150195993729,5cyclictest192448-21sendmail09:20:012
150195993729,4cyclictest0-21swapper/210:50:372
150195993729,4cyclictest0-21swapper/210:50:372
150195993727,7cyclictest421-21dbus-daemon09:49:232
150195993726,7cyclictest420-21avahi-daemon12:25:312
150195993726,7cyclictest420-21avahi-daemon12:25:312
150194993731,3cyclictest0-21swapper/111:40:321
150194993729,5cyclictest1-21systemd12:05:021
150194993727,6cyclictest3545678-21cups-browsed10:20:571
15019499372,29cyclictest218117-21ssh10:00:301
150194993710,20cyclictest665-21snmpd09:35:351
150193993731,4cyclictest315090-21munin-run12:40:000
150193993731,4cyclictest315090-21munin-run12:40:000
150193993731,4cyclictest160764-21systemd-run07:45:010
150193993730,4cyclictest189968-21fschecks_count09:15:240
150193993726,8cyclictest309961-21missed_timers12:30:270
15019399371,31cyclictest264156-21ntpq11:15:320
15019599363,29cyclictest294862-21sshd12:05:322
150195993628,5cyclictest186252-21systemd-run09:10:012
150195993624,9cyclictest245465-21sh10:45:282
150195993620,11cyclictest3545678-21cups-browsed11:35:362
150194993627,6cyclictest166778-21latency_hist08:05:011
150194993627,6cyclictest166778-21latency_hist08:05:011
150194993626,7cyclictest206277-21timerandwakeup09:40:431
150193993628,5cyclictest308885-21sshd12:29:310
150193993628,5cyclictest308885-21sshd12:29:310
150193993624,10cyclictest151681-21cut07:15:010
150195993526,5cyclictest1003-21mta-sts-daemon09:35:442
150195993524,7cyclictest312393-21/usr/sbin/munin12:35:272
150195993524,7cyclictest312393-21/usr/sbin/munin12:35:272
150195993523,8cyclictest266181-21cron11:20:012
150194993528,3cyclictest0-21swapper/109:25:011
150194993526,5cyclictest162746-21fschecks_count07:50:261
150193993525,6cyclictest3545678-21cups-browsed10:40:350
15019599344,20cyclictest270268-21ssh11:25:332
150195993426,5cyclictest306104-21latency_hist12:25:022
150195993426,5cyclictest244859-21latency_hist10:45:022
150195993426,5cyclictest0-21swapper/207:20:012
150195993425,6cyclictest187446-21/usr/sbin/munin09:10:382
150195993423,7cyclictest232923-21aten_r2power_po10:25:212
15019599342,23cyclictest218152-21sshd10:00:322
15019499343,25cyclictest242721-21ssh10:40:351
150194993427,3cyclictest0-21swapper/112:10:251
150194993427,3cyclictest0-21swapper/109:10:021
150194993426,5cyclictest287818-21latency_hist11:55:021
150194993426,5cyclictest264753-21unixbench_singl11:15:441
150194993423,8cyclictest183774-21gltestperf09:00:241
150193993428,3cyclictest1003-21mta-sts-daemon11:10:370
150193993424,6cyclictest420-21avahi-daemon11:37:150
150193993419,9cyclictest12650irq/32-eth011:45:310
150195993326,4cyclictest220025-21kworker/u8:8-nfsiod10:15:012
150195993325,5cyclictest291217-21aten_r2power_po12:00:212
150195993324,6cyclictest205902-21ssh09:40:342
150195993323,7cyclictest227378-21sh10:15:332
15019599332,26cyclictest282571-21ssh11:45:312
15019599332,26cyclictest193499-21ssh09:20:322
150195993322,6cyclictest0-21swapper/211:54:182
150194993326,4cyclictest236853-21switchtime10:30:401
150194993326,3cyclictest0-21swapper/110:05:211
150194993326,3cyclictest0-21swapper/110:05:211
150194993325,5cyclictest249382-21unixbench_multi10:50:431
150194993325,5cyclictest249382-21unixbench_multi10:50:431
150194993325,5cyclictest224888-21unixbench_singl10:10:431
150194993323,8cyclictest155460-21missed_timers07:25:261
150194993322,8cyclictest157057-21ntp_kernel_pll_07:30:331
15019499331,24cyclictest199689-21ssh09:30:321
150194993312,11cyclictest271ktimers/111:25:401
150193993327,3cyclictest0-21swapper/007:25:020
150193993325,5cyclictest3545678-21cups-browsed10:40:020
150193993325,4cyclictest0-21swapper/010:05:300
150193993325,4cyclictest0-21swapper/010:05:300
150193993323,7cyclictest272838-21cpu11:30:240
150193993318,12cyclictest0-21swapper/010:18:310
150195993224,5cyclictest232456-21sh10:25:002
150195993223,6cyclictest241605-21cron10:40:012
150195993223,5cyclictest160757-21systemd-run07:45:012
150195993222,7cyclictest178825-21latency_hist08:45:022
15019599322,23cyclictest186233-21diskmemload11:00:302
150194993226,3cyclictest0-21swapper/110:35:421
150194993225,4cyclictest304544-21timerwakeupswit12:20:551
150194993225,3cyclictest0-21swapper/109:15:021
150194993222,7cyclictest300829-21missed_timers12:15:271
150194993222,7cyclictest153187-21munin-run07:20:011
150193993226,3cyclictest0-21swapper/009:40:010
15019399321,26cyclictest209006-21ssh09:45:340
150195993125,3cyclictest0-21swapper/208:35:012
150195993125,2cyclictest0-21swapper/211:00:012
150195993125,2cyclictest0-21swapper/211:00:012
150195993124,3cyclictest0-21swapper/207:40:012
150195993123,5cyclictest221848-21unixbench_singl10:05:432
150195993123,5cyclictest221848-21unixbench_singl10:05:432
150195993123,5cyclictest174261-21systemd-run08:30:022
150195993123,5cyclictest0-21swapper/209:25:392
150195993122,7cyclictest155997-21timerwakeupswit07:25:412
150195993122,6cyclictest420-21avahi-daemon07:15:012
150194993125,3cyclictest0-21swapper/111:50:011
150194993124,4cyclictest307445-21threads12:25:431
150194993124,4cyclictest307445-21threads12:25:431
150194993124,4cyclictest161761-21proc_pri07:45:381
150194993123,5cyclictest265-21systemd-journal08:25:011
150194993123,3cyclictest0-21swapper/110:25:301
15019399312,23cyclictest3545678-21cups-browsed08:25:320
150195993021,6cyclictest181593-21unixbench_multi08:50:422
15019599301,21cyclictest267298-21ssh11:20:342
15019499304,17cyclictest294880-21ssh12:05:331
150194993023,4cyclictest246962-21ssh10:47:321
150194993020,8cyclictest176626-21ntp_kernel_pll_08:35:331
150194993011,15cyclictest271ktimers/108:55:341
150193993025,3cyclictest247841-21latency_hist10:50:020
150193993024,3cyclictest0-21swapper/011:25:230
150193993023,5cyclictest421-21dbus-daemon11:45:010
150193993023,5cyclictest233845-21switchtime10:25:420
15019399302,24cyclictest224288-21ntpq10:10:310
150193993020,5cyclictest151ktimers/008:05:310
150193993020,3cyclictest1000-21mta-sts-daemon12:06:140
150193993018,9cyclictest0-21swapper/012:00:210
150193993011,14cyclictest3545678-21cups-browsed07:25:330
15019599294,17cyclictest443-21gpm07:55:352
15019599294,17cyclictest443-21gpm07:55:352
15019599293,18cyclictest172135-21sshd08:20:312
150195992922,4cyclictest212518-21unixbench-2d09:50:432
150195992921,5cyclictest421-21dbus-daemon11:35:012
150195992920,6cyclictest1-21systemd08:15:022
15019499292,22cyclictest181106-21sshd08:50:311
150193992923,3cyclictest0-21swapper/011:00:000
150193992923,3cyclictest0-21swapper/011:00:000
150193992921,6cyclictest153188-21grep07:20:010
150193992918,4cyclictest12650irq/32-eth009:20:320
15019599282,21cyclictest179586-21sshd08:45:302
150195992820,6cyclictest168306-21latency_hist08:10:012
150195992819,6cyclictest420-21avahi-daemon08:40:012
150195992811,11cyclictest1002-21mta-sts-daemon07:45:312
15019499285,17cyclictest428-21mta-sts-daemon08:10:301
150194992822,3cyclictest0-21swapper/111:30:231
150194992822,3cyclictest0-21swapper/107:25:011
15019599273,18cyclictest261116-21ssh11:10:332
150195992722,2cyclictest0-21swapper/209:05:002
150195992720,4cyclictest186260-21kworker/u8:10-xprtiod10:30:362
150195992718,6cyclictest567-21gdbus07:25:002
15019499272,4cyclictest0-21swapper/108:25:331
150194992721,3cyclictest0-21swapper/108:00:011
150194992721,3cyclictest0-21swapper/108:00:011
15019499271,17cyclictest190364-21ntpq09:15:311
150193992721,4cyclictest169728-21latency_hist08:15:010
150193992720,3cyclictest0-21swapper/010:33:030
150193992720,3cyclictest0-21swapper/009:50:360
150193992718,6cyclictest0-21swapper/009:00:010
150193992718,6cyclictest0-21swapper/008:20:350
150193992712,11cyclictest12650irq/32-eth011:50:460
150193992712,11cyclictest12650irq/32-eth009:30:290
15019399271,18cyclictest3545675-21cupsd08:00:350
15019399271,18cyclictest3545675-21cupsd08:00:350
150195992619,4cyclictest0-21swapper/208:05:002
150195992619,4cyclictest0-21swapper/208:05:002
150195992619,3cyclictest0-21swapper/207:55:002
150195992618,5cyclictest157687-21idleruntime-cro07:35:012
150193992621,2cyclictest0-21swapper/008:45:300
150193992620,4cyclictest185759-21wc09:05:360
150193992618,5cyclictest0-21swapper/007:35:010
150193992612,11cyclictest0-21swapper/008:50:420
150195992519,3cyclictest0-21swapper/208:20:012
15019499252,16cyclictest420-21avahi-daemon08:30:401
150194992514,8cyclictest421-21dbus-daemon07:40:011
150193992515,8cyclictest421-21dbus-daemon07:38:310
150194992414,6cyclictest420-21avahi-daemon08:05:321
15019399242,16cyclictest176680-21ntp_states08:35:360
15019399242,16cyclictest161585-21sshd07:45:300
150193992418,4cyclictest623-21sshd07:54:300
150193992317,4cyclictest0-21swapper/008:30:310
150193992217,2cyclictest0-21swapper/007:55:260
150193992217,2cyclictest0-21swapper/007:55:260
15019699202,12cyclictest0-21swapper/309:35:353
150194992015,2cyclictest0-21swapper/107:10:021
15019699193,9cyclictest0-21swapper/311:30:333
15019699193,9cyclictest0-21swapper/310:20:333
15019699193,10cyclictest0-21swapper/311:05:333
15019699193,10cyclictest0-21swapper/310:35:493
15019699192,9cyclictest0-21swapper/309:45:313
15019699192,10cyclictest0-21swapper/310:15:303
15019699184,7cyclictest0-21swapper/311:45:313
15019699184,10cyclictest0-21swapper/311:15:313
15019699183,9cyclictest0-21swapper/310:25:353
15019699183,10cyclictest0-21swapper/310:40:353
15019699183,10cyclictest0-21swapper/310:00:323
15019699182,9cyclictest0-21swapper/312:30:313
15019699182,11cyclictest0-21swapper/311:55:313
15019699182,10cyclictest0-21swapper/311:10:363
15019699182,10cyclictest0-21swapper/309:25:343
15019699181,10cyclictest0-21swapper/311:50:303
15019699181,10cyclictest0-21swapper/310:30:323
150193991811,5cyclictest12650irq/32-eth007:10:010
15019699174,7cyclictest0-21swapper/311:25:423
15019699174,7cyclictest0-21swapper/308:55:303
15019699172,9cyclictest0-21swapper/312:25:323
15019699172,9cyclictest0-21swapper/312:25:323
15019699172,9cyclictest0-21swapper/310:10:323
15019699172,9cyclictest0-21swapper/309:55:313
15019699172,8cyclictest0-21swapper/310:45:273
15019699172,8cyclictest0-21swapper/307:55:353
15019699172,8cyclictest0-21swapper/307:55:353
15019699172,7cyclictest0-21swapper/308:05:313
15019699172,10cyclictest0-21swapper/312:20:403
150195991711,3cyclictest0-21swapper/207:10:022
15019699165,6cyclictest0-21swapper/309:00:333
15019699163,8cyclictest0-21swapper/311:00:013
15019699163,8cyclictest0-21swapper/311:00:013
15019699163,8cyclictest0-21swapper/309:10:383
15019699163,7cyclictest0-21swapper/307:10:313
15019699162,9cyclictest0-21swapper/312:10:293
15019699162,9cyclictest0-21swapper/311:35:313
15019699162,8cyclictest0-21swapper/311:20:313
15019699162,8cyclictest0-21swapper/311:00:303
15019699162,8cyclictest0-21swapper/309:20:313
15019699162,7cyclictest0-21swapper/307:30:013
15019699162,6cyclictest0-21swapper/308:15:323
15019699161,10cyclictest0-21swapper/309:50:343
15019699153,5cyclictest0-21swapper/308:25:323
15019699152,9cyclictest0-21swapper/308:40:343
15019699152,8cyclictest0-21swapper/312:35:293
15019699152,8cyclictest0-21swapper/312:35:293
15019699152,8cyclictest0-21swapper/311:40:223
15019699152,8cyclictest0-21swapper/310:50:333
15019699152,8cyclictest0-21swapper/310:50:333
15019699152,8cyclictest0-21swapper/309:30:173
15019699152,7cyclictest0-21swapper/309:40:323
15019699151,7cyclictest0-21swapper/308:35:303
15019699151,6cyclictest0-21swapper/308:30:323
15019699151,10cyclictest0-21swapper/312:05:253
15019699143,8cyclictest0-21swapper/310:05:213
15019699143,8cyclictest0-21swapper/310:05:213
15019699142,8cyclictest0-21swapper/309:15:313
15019699142,6cyclictest0-21swapper/308:10:323
15019699141,8cyclictest0-21swapper/312:15:223
15019699141,8cyclictest0-21swapper/312:00:363
15019699141,7cyclictest0-21swapper/307:40:323
15019699133,5cyclictest0-21swapper/308:50:313
15019699132,7cyclictest0-21swapper/307:15:373
15019699131,8cyclictest0-21swapper/309:10:023
15019699131,7cyclictest0-21swapper/307:20:253
15019699123,5cyclictest0-21swapper/308:00:313
15019699123,5cyclictest0-21swapper/308:00:313
15019699122,6cyclictest0-21swapper/308:20:313
15019699121,6cyclictest0-21swapper/307:45:303
15019699121,6cyclictest0-21swapper/307:35:303
15019699112,6cyclictest0-21swapper/307:30:263
15019699111,6cyclictest0-21swapper/308:45:273
15019699105,3cyclictest0-21swapper/307:50:283
1501969981,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional