You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 19:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Jul 16, 2026 12:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
63637995138,10cyclictest185753-21ps11:30:480
63638995037,8cyclictest0-21swapper/109:45:431
6364099475,37cyclictest203817-21ssh12:00:472
63640994738,5cyclictest96687-21systemd-run09:00:022
63637994734,9cyclictest12650irq/32-eth010:55:480
63638994634,7cyclictest3451713-21cups-browsed10:20:541
63637994641,3cyclictest1003-21mta-sts-daemon10:53:060
63637994639,3cyclictest0-21swapper/007:40:010
63640994435,6cyclictest134380-21mailstats10:05:502
63640994435,5cyclictest84648-21systemd-run08:20:012
6364099443,35cyclictest125069-21ssh09:50:462
63638994234,5cyclictest185731-21/usr/sbin/munin11:30:481
63638994232,6cyclictest140723-21unixbench_multi10:15:531
63637994234,6cyclictest0-21swapper/012:30:020
63637994228,11cyclictest98194-21systemd-run09:05:020
63640994133,5cyclictest202671-21systemd12:00:022
6363799414,30cyclictest140165-21ssh10:15:410
63637994134,4cyclictest92110-21cron08:45:000
63637994134,4cyclictest92110-21cron08:45:000
63637994133,5cyclictest207345-21unixbench_singl12:05:550
63637994128,10cyclictest175405-21munin-plugin-st11:15:000
6363799411,30cyclictest112982-21ssh09:30:440
63638994031,6cyclictest265-21systemd-journal11:15:001
63638994030,7cyclictest215251-21aten_r2power_po12:20:291
63638994030,6cyclictest65144-21systemd-run07:15:011
63638994030,6cyclictest65144-21systemd-run07:15:011
63637994032,5cyclictest67265-21sshd07:20:380
63637994030,4cyclictest1000-21mta-sts-daemon09:45:590
63637994028,9cyclictest420-21avahi-daemon10:32:490
63640993931,5cyclictest209395-21ssh12:10:372
63640993930,5cyclictest142103-21munin-run10:20:002
63640993928,8cyclictest157281-21idleruntime-cro10:45:012
63638993931,5cyclictest1-21systemd08:45:011
63638993931,5cyclictest1-21systemd08:45:011
63637993930,5cyclictest188977-21threads11:35:520
63637993928,7cyclictest78774-21latency_hist08:00:020
63638993829,6cyclictest1-21systemd12:15:011
63638993829,5cyclictest158568-21sendmail_mailqu10:45:491
63637993831,4cyclictest1003-21mta-sts-daemon11:45:460
63637993831,4cyclictest1003-21mta-sts-daemon11:45:460
63637993830,5cyclictest420-21avahi-daemon09:17:410
63637993830,5cyclictest178469-21systemd-run11:20:010
63637993829,6cyclictest151ktimers/008:50:420
63640993729,5cyclictest77137-21systemd-run07:55:012
63640993729,5cyclictest265-21systemd-journal11:10:002
63640993729,5cyclictest265-21systemd-journal11:10:002
63638993728,6cyclictest70153-21fschecks_time07:30:331
63637993730,4cyclictest127121-21latency_hist09:55:020
63637993729,5cyclictest160294-21munin-run10:50:010
63637993729,4cyclictest0-21swapper/011:55:460
63637993725,7cyclictest121014-21sort09:45:020
63637993720,3cyclictest171rcu_preempt10:35:460
63637993714,16cyclictest81113-21sshd08:05:480
63640993631,2cyclictest0-21swapper/212:40:012
63640993628,5cyclictest116411-21timerwakeupswit09:35:542
63638993630,4cyclictest63636-21cyclictest12:30:011
63638993629,5cyclictest136053-21cron10:10:011
63638993628,5cyclictest95154-21latency_hist08:55:001
63638993628,5cyclictest130422-21aten_r2power_cu10:00:311
63638993628,4cyclictest0-21swapper/111:00:431
63638993610,5cyclictest28-21ksoftirqd/111:40:411
63637993628,6cyclictest3451713-21cups-browsed10:41:590
63637993627,6cyclictest420-21avahi-daemon12:32:050
63637993627,6cyclictest3451713-21cups-browsed09:10:320
63637993621,2cyclictest171rcu_preempt07:50:450
63640993530,2cyclictest0-21swapper/211:20:012
63640993526,6cyclictest487-21rs:main2
63640993526,6cyclictest487-21rs:main2
63640993526,6cyclictest0-21swapper/210:40:012
63638993527,5cyclictest98195-21unin-run09:05:021
63638993527,5cyclictest175915-21aten_r2power_po11:15:311
63638993527,5cyclictest0-21swapper/108:40:011
63638993525,7cyclictest93665-21/usr/sbin/munin08:50:011
6363799353,22cyclictest109971-21ssh09:25:470
63637993527,5cyclictest90604-21munin-run08:40:000
63637993525,8cyclictest1000-21mta-sts-daemon10:07:180
63637993525,7cyclictest193588-21cut11:45:020
63640993425,6cyclictest421-21dbus-daemon08:10:012
63638993427,4cyclictest137763-21unixbench_singl10:10:541
63638993424,6cyclictest104408-21unixbench_singl09:15:541
63637993427,3cyclictest0-21swapper/012:15:300
63637993424,7cyclictest198014-21switchtime11:50:510
63637993420,9cyclictest12650irq/32-eth010:25:430
63637993412,3cyclictest171rcu_preempt08:45:450
63640993327,3cyclictest0-21swapper/209:50:012
63640993326,4cyclictest114845-21ssh09:35:012
63640993324,6cyclictest420-21avahi-daemon09:10:432
6364099332,24cyclictest118990-21ssh09:40:432
63638993326,4cyclictest188465-21missed_timers11:35:371
63638993326,4cyclictest1001-21mta-sts-daemon08:15:441
63638993326,4cyclictest1000-21mta-sts-daemon09:11:451
63638993325,5cyclictest73147-21fschecks_count07:40:341
63638993325,5cyclictest66676-21latency_hist07:20:011
63638993325,5cyclictest66676-21latency_hist07:20:011
6363899331,28cyclictest164419-21ssh10:55:441
6363899331,22cyclictest206786-21ssh12:05:441
63637993326,5cyclictest63636-21cyclictest12:10:470
63637993325,5cyclictest439-21systemd-logind10:00:010
63637993325,4cyclictest0-21swapper/011:00:520
63637993324,7cyclictest64329-21sshd07:10:380
63637993324,7cyclictest64329-21sshd07:10:380
63637993314,3cyclictest171rcu_preempt08:00:470
63640993226,3cyclictest0-21swapper/212:05:402
63640993226,3cyclictest0-21swapper/208:35:482
63640993226,3cyclictest0-21swapper/208:00:012
63640993226,2cyclictest0-21swapper/209:20:012
63640993225,4cyclictest143830-21unixbench_singl10:20:542
63640993225,4cyclictest109961-21sh09:25:472
63640993225,4cyclictest1000-21mta-sts-daemon11:53:162
6364099322,4cyclictest171rcu_preempt11:40:412
63640993224,5cyclictest213006-21mailstats12:15:492
63640993224,5cyclictest129986-21munin-run10:00:012
63640993224,4cyclictest74648-21fschecks_count07:45:342
63640993222,7cyclictest137002-21missed_timers10:10:382
6363899323,23cyclictest119049-21ssh09:40:451
63638993227,2cyclictest0-21swapper/108:00:011
63638993225,4cyclictest148172-21ssh10:30:001
63638993224,5cyclictest155341-21ntp_kernel_pll_10:40:421
63638993223,6cyclictest85278-21kernelversion08:20:371
63638993223,6cyclictest1003-21mta-sts-daemon10:30:121
63638993222,6cyclictest1003-21mta-sts-daemon11:29:091
63638993218,10cyclictest109684-21latency09:25:371
6363899321,24cyclictest197853-21ssh11:50:481
6363799328,6cyclictest0-21swapper/008:15:480
63637993226,4cyclictest131692-21unixbench_singl10:00:550
63637993222,7cyclictest420-21avahi-daemon11:06:240
63637993222,7cyclictest420-21avahi-daemon11:06:240
63637993219,3cyclictest171rcu_preempt09:05:460
63637993218,9cyclictest12650irq/32-eth012:20:420
63640993124,4cyclictest168012-21unixbench_singl11:00:542
63640993123,5cyclictest421-21dbus-daemon10:00:082
63640993122,6cyclictest421-21dbus-daemon12:20:482
63640993119,9cyclictest173218-21kernelversion11:10:372
63638993124,4cyclictest181482-21idleruntime-cro11:25:011
63638993124,4cyclictest118027-21ssh09:40:021
63638993123,5cyclictest99668-21munin-run09:10:011
63638993123,5cyclictest127081-21latency_hist09:55:021
63638993123,4cyclictest0-21swapper/111:55:471
63638993122,3cyclictest0-21swapper/110:35:451
63637993124,5cyclictest191rcuc/008:20:430
6363799312,22cyclictest203853-21ssh12:00:490
63637993121,7cyclictest82454-21ntp_kernel_err08:10:420
63637993118,7cyclictest151ktimers/011:25:440
6363799310,13cyclictest107740-21kworker/0:2+events_power_efficient09:24:130
6363799310,13cyclictest107740-21kworker/0:2+events_power_efficient09:24:130
63640993025,2cyclictest0-21swapper/207:40:472
63640993021,6cyclictest1000-21mta-sts-daemon08:20:362
63638993024,4cyclictest0-21swapper/112:40:021
63638993024,3cyclictest0-21swapper/111:45:351
63638993024,3cyclictest0-21swapper/111:45:351
63638993023,4cyclictest1003-21mta-sts-daemon11:09:481
63638993023,4cyclictest1003-21mta-sts-daemon11:09:481
63638993021,6cyclictest75558-21unixbench_singl07:45:521
63638993021,6cyclictest420-21avahi-daemon12:31:321
63638993020,8cyclictest108810-21sh09:25:021
63638993020,8cyclictest108810-21sh09:25:021
63637993023,3cyclictest0-21swapper/010:20:390
63637993022,5cyclictest0-21swapper/011:20:100
63637993015,12cyclictest0-21swapper/007:25:370
63637993014,10cyclictest224167-21find12:35:310
63637993012,10cyclictest12650irq/32-eth007:40:470
63640992922,3cyclictest0-21swapper/211:40:012
63640992921,5cyclictest1000-21mta-sts-daemon10:46:482
63640992919,8cyclictest150243-21sh10:31:592
63640992919,5cyclictest0-21swapper/207:35:412
63640992918,7cyclictest221979-21sh12:30:492
63638992922,4cyclictest63698-21grep07:10:021
63638992922,4cyclictest63698-21grep07:10:021
63638992922,3cyclictest0-21swapper/112:00:561
6363899292,20cyclictest128076-21ssh09:55:431
63637992915,11cyclictest191rcuc/007:30:450
63640992822,3cyclictest0-21swapper/211:25:342
63640992822,3cyclictest0-21swapper/209:10:012
6364099282,22cyclictest106878-21ssh09:20:472
6364099282,22cyclictest106878-21ssh09:20:472
63640992822,2cyclictest0-21swapper/211:30:542
6364099282,20cyclictest79591-21sshd08:00:462
6363899282,20cyclictest79594-21sshd08:00:461
63638992818,6cyclictest95668-21fschecks_count08:55:351
6363799284,19cyclictest665-21snmpd08:55:450
6363799282,19cyclictest12650irq/32-eth010:10:450
63637992821,3cyclictest0-21swapper/009:35:400
63637992816,7cyclictest1-21systemd08:32:470
63637992816,7cyclictest1-21systemd08:32:470
63637992813,3cyclictest171rcu_preempt07:45:420
63640992721,3cyclictest0-21swapper/208:45:002
63640992721,3cyclictest0-21swapper/208:45:002
63640992721,3cyclictest0-21swapper/207:20:002
63640992721,3cyclictest0-21swapper/207:20:002
63640992718,5cyclictest164521-21sshd10:55:482
63640992717,6cyclictest421-21dbus-daemon07:15:012
63640992717,6cyclictest421-21dbus-daemon07:15:012
63640992713,10cyclictest265-21systemd-journal10:25:472
63638992722,2cyclictest0-21swapper/110:55:021
63638992720,4cyclictest1002-21mta-sts-daemon09:30:461
63638992719,5cyclictest3451713-21cups-browsed07:35:191
63637992716,3cyclictest171rcu_preempt08:25:440
6364199261,4cyclictest0-21swapper/308:47:033
6364199261,20cyclictest0-21swapper/312:25:453
6364199261,19cyclictest0-21swapper/310:05:433
63640992621,2cyclictest0-21swapper/210:50:412
63640992619,4cyclictest69510-21unixbench_multi07:25:532
63640992619,4cyclictest0-21swapper/209:05:012
63640992615,3cyclictest0-21swapper/207:35:012
6363899264,13cyclictest428-21mta-sts-daemon07:50:421
63638992620,3cyclictest1000-21mta-sts-daemon07:20:441
63638992617,6cyclictest441-21thd12:19:291
6364199252,19cyclictest0-21swapper/309:05:463
6364199251,3cyclictest0-21swapper/310:40:473
63640992520,2cyclictest0-21swapper/208:55:002
63640992520,2cyclictest0-21swapper/208:30:422
63640992520,2cyclictest0-21swapper/208:30:422
63640992520,2cyclictest0-21swapper/208:10:442
63640992518,4cyclictest68143-21idleruntime-cro07:25:002
63640992517,5cyclictest218243-21head12:25:302
63640992515,7cyclictest179444-21needreboot11:20:382
6364199242,4cyclictest0-21swapper/312:30:443
63640992417,4cyclictest71107-21kworker/u8:3+xprtiod08:30:012
6363799246,3cyclictest171rcu_preempt07:15:400
6363799246,3cyclictest171rcu_preempt07:15:400
6364199232,14cyclictest0-21swapper/309:55:413
6364199231,4cyclictest0-21swapper/310:30:463
6364199231,4cyclictest0-21swapper/308:10:433
6364199231,4cyclictest0-21swapper/307:20:413
6364199231,3cyclictest0-21swapper/311:25:463
6364199231,3cyclictest0-21swapper/309:30:403
63640992319,2cyclictest0-21swapper/208:50:002
6363899235,13cyclictest428-21mta-sts-daemon08:10:421
63638992317,3cyclictest1000-21mta-sts-daemon08:30:331
63638992317,3cyclictest1000-21mta-sts-daemon08:30:331
63638992314,6cyclictest68970-21awk07:25:391
6364199222,4cyclictest0-21swapper/309:00:413
6364199222,16cyclictest0-21swapper/309:45:423
6364199221,6cyclictest0-21swapper/308:35:393
6364199221,5cyclictest0-21swapper/312:35:443
6364199221,5cyclictest0-21swapper/310:50:423
6364199221,4cyclictest0-21swapper/311:50:413
6364199221,4cyclictest0-21swapper/311:35:303
6364199221,4cyclictest0-21swapper/311:20:433
6364199221,4cyclictest0-21swapper/311:05:403
6364199221,4cyclictest0-21swapper/311:05:403
6364199221,4cyclictest0-21swapper/309:25:403
6364199221,4cyclictest0-21swapper/309:15:433
6364199221,4cyclictest0-21swapper/308:55:413
6364199221,4cyclictest0-21swapper/308:05:473
6364199221,4cyclictest0-21swapper/307:15:403
6364199221,4cyclictest0-21swapper/307:15:403
6364199221,3cyclictest0-21swapper/310:20:463
6364199221,3cyclictest0-21swapper/308:20:453
6364199221,3cyclictest0-21swapper/307:55:443
6364199221,3cyclictest0-21swapper/307:45:443
6364199221,3cyclictest0-21swapper/307:40:463
6364199221,3cyclictest0-21swapper/307:35:393
6364199221,3cyclictest0-21swapper/307:30:443
6364199221,17cyclictest0-21swapper/309:40:013
63638992216,3cyclictest0-21swapper/108:05:481
6364199212,5cyclictest0-21swapper/310:45:433
6364199212,13cyclictest0-21swapper/309:40:453
6364199212,13cyclictest0-21swapper/309:20:473
6364199212,13cyclictest0-21swapper/309:20:473
6364199211,5cyclictest0-21swapper/312:20:413
6364199211,4cyclictest0-21swapper/312:10:153
6364199211,4cyclictest0-21swapper/312:05:433
6364199211,4cyclictest0-21swapper/311:45:393
6364199211,4cyclictest0-21swapper/311:45:393
6364199211,4cyclictest0-21swapper/311:40:453
6364199211,4cyclictest0-21swapper/310:25:403
6364199211,4cyclictest0-21swapper/310:15:423
6364199211,4cyclictest0-21swapper/310:10:423
6364199211,4cyclictest0-21swapper/310:00:433
6364199211,4cyclictest0-21swapper/309:10:423
6364199211,4cyclictest0-21swapper/308:50:393
6364199211,4cyclictest0-21swapper/308:40:443
6364199211,4cyclictest0-21swapper/308:40:443
6364199211,4cyclictest0-21swapper/308:30:443
6364199211,4cyclictest0-21swapper/308:30:443
6364199211,4cyclictest0-21swapper/308:15:483
6364199211,4cyclictest0-21swapper/307:50:403
6364199211,4cyclictest0-21swapper/307:10:413
6364199211,4cyclictest0-21swapper/307:10:413
6364199211,3cyclictest0-21swapper/312:00:453
6364199211,3cyclictest0-21swapper/311:55:453
6364199211,3cyclictest0-21swapper/311:30:443
6364199211,3cyclictest0-21swapper/310:35:423
6364199211,3cyclictest0-21swapper/309:50:353
6364199211,3cyclictest0-21swapper/308:25:393
6364199211,3cyclictest0-21swapper/307:25:463
6364199211,14cyclictest0-21swapper/311:15:393
6364199210,4cyclictest0-21swapper/312:15:333
6364199210,3cyclictest0-21swapper/311:00:423
6363899212,14cyclictest86994-21ntpq08:25:411
6363799216,6cyclictest14-21ksoftirqd/007:10:010
6363799216,6cyclictest14-21ksoftirqd/007:10:010
6364199201,3cyclictest0-21swapper/311:10:413
6364199201,3cyclictest0-21swapper/310:55:403
6364199201,3cyclictest0-21swapper/308:00:413
6364199191,4cyclictest0-21swapper/307:10:013
6364199191,4cyclictest0-21swapper/307:10:013
6364099186,8cyclictest421-21dbus-daemon07:10:012
6364099186,8cyclictest421-21dbus-daemon07:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional