You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 00:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu May 21, 2026 12:43:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
354175995143,5cyclictest540558-21grep12:20:012
354174995041,5cyclictest449020-21grep10:25:011
354174995041,5cyclictest449020-21grep10:25:011
354174995040,7cyclictest357159-21munin-run07:20:011
354174994824,5cyclictest171rcu_preempt12:06:001
354174994739,5cyclictest444970-21grep10:20:011
354174994637,6cyclictest382682-21systemd-run08:45:021
354174994635,8cyclictest1-21systemd10:15:011
354173994636,8cyclictest1-21systemd11:35:010
354175994436,5cyclictest476979-21idleruntime-cro11:00:012
354173994424,17cyclictest659-21snmpd07:30:590
354175994335,5cyclictest1-21systemd12:25:022
354175994334,6cyclictest421-21dbus-daemon09:35:012
354173994326,14cyclictest12350irq/32-eth010:25:540
354173994326,14cyclictest12350irq/32-eth010:25:540
35417399432,31cyclictest391529-21ssh09:10:590
354175994234,5cyclictest372148-21cron08:10:002
354174994233,5cyclictest406556-21sshd09:31:081
354174994222,4cyclictest171rcu_preempt08:26:021
354175994131,7cyclictest508666-21munin-plugin-st11:40:012
35417599412,32cyclictest418094-21ssh09:45:582
354173994135,4cyclictest1-21systemd10:25:020
354173994135,4cyclictest1-21systemd10:25:020
354173994126,12cyclictest494960-21sshd11:21:270
354175994033,4cyclictest0-21swapper/212:25:552
354175994033,4cyclictest0-21swapper/212:25:552
354175994022,12cyclictest351ktimers/207:45:552
35417499405,27cyclictest502014-21ssh11:30:561
354174994034,3cyclictest0-21swapper/111:50:011
354174994034,3cyclictest0-21swapper/109:40:011
354173994032,6cyclictest151ktimers/010:35:550
354173994032,5cyclictest430169-21ssh10:00:560
354173994030,7cyclictest405764-21latency09:30:520
354173994026,7cyclictest12350irq/32-eth010:55:550
354173994019,15cyclictest12350irq/32-eth010:50:590
354175993934,2cyclictest0-21swapper/211:25:012
35417599393,33cyclictest514076-21ssh11:46:002
354175993931,5cyclictest438459-21proc_pri10:11:022
354175993931,5cyclictest373648-21idleruntime-cro08:15:002
35417599392,33cyclictest525919-21ssh12:00:542
35417599392,30cyclictest391475-21ssh09:10:572
354174993931,5cyclictest369183-21systemd-run08:00:011
354174993930,6cyclictest421697-21fschecks_count09:50:501
354174993929,7cyclictest420-21avahi-daemon11:45:021
354174993926,5cyclictest1051-21mta-sts-daemon10:00:011
354174993926,5cyclictest1051-21mta-sts-daemon10:00:011
35417499392,30cyclictest391550-21ssh09:11:001
354173993932,4cyclictest420767-21cut09:50:010
354173993912,23cyclictest426152-21ssh09:55:550
354173993912,23cyclictest426152-21ssh09:55:550
354175993831,4cyclictest1051-21mta-sts-daemon10:16:462
354175993830,4cyclictest0-21swapper/210:00:582
354175993828,7cyclictest469023-21systemd-run10:50:012
354175993827,7cyclictest461221-21latency_hist10:40:022
354174993828,7cyclictest484866-21cron11:10:001
354174993828,7cyclictest484866-21cron11:10:001
354174993827,7cyclictest525562-21fschecks_count12:00:491
354174993814,4cyclictest0-21swapper/107:45:581
354173993830,5cyclictest533804-21missed_timers12:10:570
354173993827,8cyclictest514653-21unixbench_singl11:46:100
35417399382,27cyclictest510048-21ssh11:40:570
354173993822,12cyclictest151ktimers/011:06:080
354173993822,12cyclictest151ktimers/011:06:080
354175993730,4cyclictest388767-21kworker/u8:14-rpciod10:05:482
354175993729,5cyclictest421-21dbus-daemon10:35:012
354175993729,5cyclictest421-21dbus-daemon07:20:012
354175993727,6cyclictest3738109-21cups-browsed11:55:502
354175993727,6cyclictest3738109-21cups-browsed11:55:502
354174993729,4cyclictest0-21swapper/109:15:511
354174993728,6cyclictest478949-21grep11:01:081
354174993728,6cyclictest478949-21grep11:01:081
354174993726,7cyclictest0-21swapper/109:41:071
354173993731,4cyclictest151ktimers/007:16:020
354173993728,7cyclictest421-21dbus-daemon08:35:020
354173993725,9cyclictest0-21swapper/012:05:440
354173993719,12cyclictest12350irq/32-eth011:01:010
354173993719,12cyclictest12350irq/32-eth011:01:010
354174993628,6cyclictest354172-21cyclictest12:10:571
354174993628,4cyclictest388692-21latency_hist09:05:021
354174993618,6cyclictest461086-21kworker/u8:4-xprtiod11:55:591
354174993618,6cyclictest461086-21kworker/u8:4-xprtiod11:55:591
354174993618,5cyclictest171rcu_preempt11:50:541
354173993627,6cyclictest381239-21latency_hist08:40:020
354173993627,5cyclictest0-21swapper/012:30:580
354175993529,3cyclictest0-21swapper/212:10:002
354175993529,3cyclictest0-21swapper/210:21:012
354175993529,3cyclictest0-21swapper/210:21:012
354175993527,5cyclictest512687-21ssh11:45:012
354175993527,5cyclictest388676-21systemd-run09:05:012
35417499353,23cyclictest3738081-21cupsd12:15:591
354174993525,7cyclictest354973-21needreboot07:10:531
354174993523,9cyclictest506667-21unixbench_singl11:36:091
354174993516,4cyclictest0-21swapper/108:46:021
35417399359,19cyclictest422164-21ssh09:50:590
354173993529,4cyclictest524622-21/usr/sbin/munin12:00:020
354173993529,4cyclictest524622-21/usr/sbin/munin12:00:020
354173993529,3cyclictest0-21swapper/011:20:020
354173993528,4cyclictest553418-21fschecks_count12:35:480
35417399352,28cyclictest541936-21ssh12:20:570
35417499347,19cyclictest418196-21ssh09:46:011
354174993427,4cyclictest375161-21systemctl08:20:001
354174993427,4cyclictest0-21swapper/109:00:011
354174993426,5cyclictest450952-21unixbench_multi10:26:081
354174993426,5cyclictest450952-21unixbench_multi10:26:081
354174993425,6cyclictest363193-21unin-run07:40:011
354174993425,5cyclictest401219-21ssh09:25:021
354174993425,5cyclictest401219-21ssh09:25:021
354174993424,7cyclictest1055-21mta-sts-daemon12:21:091
354174993419,4cyclictest171rcu_preempt11:15:591
354173993426,5cyclictest383562-21sshd08:46:000
354173993425,5cyclictest151ktimers/011:50:560
354173993424,7cyclictest537719-21latency12:15:540
354173993422,4cyclictest151ktimers/010:16:000
354173993415,11cyclictest151ktimers/009:05:560
35417399341,29cyclictest462341-21ssh10:40:560
354175993327,3cyclictest0-21swapper/210:51:032
354175993327,3cyclictest0-21swapper/207:45:012
354175993326,3cyclictest0-21swapper/210:00:022
354175993326,3cyclictest0-21swapper/210:00:022
354175993325,5cyclictest1-21systemd09:25:022
354175993325,5cyclictest1-21systemd09:25:022
354175993324,6cyclictest420-21avahi-daemon09:35:282
354175993323,5cyclictest0-21swapper/210:40:582
35417599331,24cyclictest413989-21ssh09:40:532
354174993328,3cyclictest0-21swapper/108:40:021
354174993327,3cyclictest28-21ksoftirqd/110:00:451
354174993326,4cyclictest466956-21unixbench_multi10:46:121
354174993326,4cyclictest461037-21munin-run10:40:011
354174993325,5cyclictest472988-21cut10:55:011
354174993325,5cyclictest370652-21cron08:05:011
35417399333,25cyclictest395443-21ssh09:16:010
354173993325,6cyclictest753-21ntpd12:25:490
354173993325,6cyclictest753-21ntpd12:25:490
354173993325,5cyclictest0-21swapper/012:00:470
354173993317,11cyclictest12350irq/32-eth009:35:580
354175993225,3cyclictest0-21swapper/212:35:592
354175993225,3cyclictest0-21swapper/210:25:552
354175993225,3cyclictest0-21swapper/210:25:552
354175993223,6cyclictest1051-21mta-sts-daemon11:09:552
354175993223,6cyclictest1051-21mta-sts-daemon11:09:552
35417499323,23cyclictest434301-21ssh10:05:591
354174993225,4cyclictest1055-21mta-sts-daemon10:34:311
354174993224,5cyclictest494726-21tail11:21:081
354174993224,5cyclictest389191-21fschecks_time09:05:491
354174993221,7cyclictest545794-21needreboot12:25:521
354174993221,7cyclictest545794-21needreboot12:25:521
354174993217,7cyclictest354172-21cyclictest10:57:111
35417499321,23cyclictest462390-21ssh10:40:581
35417399323,25cyclictest498177-21ssh11:26:020
354173993221,8cyclictest0-21swapper/009:21:070
354173993221,8cyclictest0-21swapper/009:21:070
354175993126,2cyclictest0-21swapper/209:00:012
354175993125,3cyclictest0-21swapper/207:40:012
354175993124,3cyclictest0-21swapper/211:15:012
354175993123,4cyclictest0-21swapper/212:11:032
354175993123,3cyclictest0-21swapper/211:35:012
354175993121,7cyclictest478978-21grep11:01:082
354175993121,7cyclictest478978-21grep11:01:082
35417499316,18cyclictest428-21mta-sts-daemon07:20:561
35417399314,18cyclictest12350irq/32-eth011:36:020
354173993122,5cyclictest0-21swapper/009:26:020
354175993024,3cyclictest0-21swapper/208:35:012
354175993023,3cyclictest0-21swapper/209:50:592
354175993022,5cyclictest383297-21kernelversion08:45:542
354175993022,5cyclictest376656-21cron08:25:002
354175993022,5cyclictest376656-21cron08:25:002
354175993021,6cyclictest420-21avahi-daemon07:10:022
354175993021,5cyclictest359043-21aten_r2power_po07:25:462
354175993020,7cyclictest549921-21ntp_kernel_pll_12:30:582
354175993020,7cyclictest382579-21unixbench_singl08:41:072
354174993024,3cyclictest0-21swapper/112:35:001
354174993022,5cyclictest486771-21sed11:11:131
35417499302,21cyclictest553953-21ssh12:36:001
354173993023,5cyclictest0-21swapper/010:30:480
354173993021,6cyclictest441-21gpm10:07:260
35417599293,21cyclictest518095-21ssh11:51:002
354175992922,4cyclictest401951-21cpuspeed09:25:462
354175992922,4cyclictest369023-21unixbench_multi07:56:082
354175992922,3cyclictest0-21swapper/207:15:012
354175992921,5cyclictest390155-21munin-run09:10:012
354175992918,6cyclictest0-21swapper/207:51:012
35417599291,22cyclictest377610-21sshd08:25:582
354174992922,4cyclictest1056-21mta-sts-daemon09:29:131
354174992920,6cyclictest497398-21aten_r2power_en11:25:461
354173992924,2cyclictest0-21swapper/010:45:570
354173992923,4cyclictest416732-21munin-run09:45:010
354173992922,3cyclictest0-21swapper/008:55:540
354173992921,6cyclictest0-21swapper/011:11:120
354173992920,7cyclictest371537-21ntp_kernel_pll_08:05:560
35417399291,24cyclictest440964-21systemd-run10:15:010
354175992821,4cyclictest1051-21mta-sts-daemon11:15:422
354175992820,3cyclictest0-21swapper/209:15:572
354175992819,6cyclictest3738109-21cups-browsed08:52:582
354174992818,3cyclictest370929-21aten_r2power_cu08:05:451
354173992822,4cyclictest518-21rs:main0
354173992819,7cyclictest1-21systemd07:55:460
354175992721,2cyclictest0-21swapper/208:40:012
354175992720,5cyclictest360707-21gltestperf07:30:492
354174992716,7cyclictest359736-21find07:26:021
354174992710,4cyclictest0-21swapper/107:40:581
354173992721,4cyclictest354172-21cyclictest07:11:030
354173992718,7cyclictest357990-21needreboot07:20:520
354173992711,3cyclictest0-21swapper/007:46:060
35417599262,20cyclictest498218-21ssh11:26:042
354175992621,3cyclictest0-21swapper/208:20:002
354175992620,2cyclictest0-21swapper/208:02:432
354175992619,4cyclictest358624-21unixbench_singl07:21:062
354174992617,7cyclictest372891-21missed_timers08:10:521
354173992620,4cyclictest518-21rs:main0
354173992615,9cyclictest0-21swapper/008:45:010
35417499256,3cyclictest171rcu_preempt07:30:571
354174992519,3cyclictest0-21swapper/108:51:071
354173992521,2cyclictest0-21swapper/007:40:010
354173992520,3cyclictest1051-21mta-sts-daemon07:40:490
354173992519,4cyclictest12450irq/33-eth008:50:590
354173992518,3cyclictest0-21swapper/008:15:560
354173992515,8cyclictest439-21systemd-logind08:15:010
354173992420,3cyclictest354172-21cyclictest08:01:030
354173992415,7cyclictest0-21swapper/007:55:010
354173992412,9cyclictest151ktimers/008:21:000
354173992412,9cyclictest151ktimers/008:21:000
35417499239,5cyclictest171rcu_preempt08:30:571
35417499237,11cyclictest261rcuc/108:20:571
35417499237,11cyclictest261rcuc/108:20:571
354174992317,3cyclictest1055-21mta-sts-daemon07:50:471
354173992210,10cyclictest0-21swapper/009:01:030
35417699203,9cyclictest0-21swapper/310:01:013
35417699195,8cyclictest0-21swapper/310:06:043
35417699192,10cyclictest0-21swapper/311:46:003
35417399196,7cyclictest11systemd07:10:010
35417699184,7cyclictest0-21swapper/311:00:553
35417699184,7cyclictest0-21swapper/311:00:553
35417699184,6cyclictest0-21swapper/309:10:573
35417699182,8cyclictest0-21swapper/310:21:003
35417699182,8cyclictest0-21swapper/310:21:003
35417699174,7cyclictest0-21swapper/309:16:023
35417699173,9cyclictest0-21swapper/311:35:563
35417699173,8cyclictest0-21swapper/311:30:563
35417699173,8cyclictest0-21swapper/310:15:563
35417699172,9cyclictest0-21swapper/310:56:023
35417699172,9cyclictest0-21swapper/310:51:033
35417699172,8cyclictest0-21swapper/311:55:543
35417699172,8cyclictest0-21swapper/311:55:543
35417699172,8cyclictest0-21swapper/309:20:573
35417699172,8cyclictest0-21swapper/309:20:573
35417699171,8cyclictest0-21swapper/312:25:543
35417699171,8cyclictest0-21swapper/312:25:543
35417499173,10cyclictest1-21systemd07:10:011
35417699164,7cyclictest0-21swapper/312:36:023
35417699164,7cyclictest0-21swapper/311:10:013
35417699164,7cyclictest0-21swapper/311:10:013
35417699164,4cyclictest0-21swapper/307:50:573
35417699163,8cyclictest0-21swapper/310:41:023
35417699163,8cyclictest0-21swapper/310:10:563
35417699163,7cyclictest0-21swapper/311:50:433
35417699162,8cyclictest0-21swapper/311:40:543
35417699162,8cyclictest0-21swapper/309:40:563
35417699162,8cyclictest0-21swapper/309:35:553
35417699162,8cyclictest0-21swapper/309:25:553
35417699162,6cyclictest0-21swapper/308:55:583
35417699162,6cyclictest0-21swapper/307:45:553
35417699161,9cyclictest0-21swapper/312:30:573
35417699161,9cyclictest0-21swapper/311:21:023
35417699161,9cyclictest0-21swapper/311:15:553
35417699161,9cyclictest0-21swapper/309:31:003
35417699161,7cyclictest0-21swapper/307:16:003
35417699154,5cyclictest0-21swapper/308:30:583
35417699154,4cyclictest0-21swapper/308:05:563
35417699153,7cyclictest0-21swapper/312:20:583
35417699152,8cyclictest0-21swapper/310:35:523
35417699152,8cyclictest0-21swapper/310:25:553
35417699152,8cyclictest0-21swapper/310:25:553
35417699152,8cyclictest0-21swapper/309:50:593
35417699152,7cyclictest0-21swapper/312:00:553
35417699152,6cyclictest0-21swapper/309:05:563
35417699152,6cyclictest0-21swapper/308:25:553
35417699152,6cyclictest0-21swapper/307:35:533
35417699152,6cyclictest0-21swapper/307:25:583
35417699151,9cyclictest0-21swapper/312:05:543
35417699151,9cyclictest0-21swapper/310:46:053
35417699151,9cyclictest0-21swapper/310:30:583
35417699151,8cyclictest0-21swapper/309:50:033
35417699144,6cyclictest0-21swapper/309:55:503
35417699144,6cyclictest0-21swapper/309:55:503
35417699144,4cyclictest0-21swapper/309:00:573
35417699143,7cyclictest0-21swapper/312:15:593
35417699143,4cyclictest0-21swapper/308:40:553
35417699142,7cyclictest0-21swapper/311:25:593
35417699142,7cyclictest0-21swapper/311:10:593
35417699142,6cyclictest0-21swapper/312:10:453
35417699142,6cyclictest0-21swapper/307:45:013
35417699141,7cyclictest0-21swapper/308:45:593
35417699141,7cyclictest0-21swapper/308:21:093
35417699141,7cyclictest0-21swapper/308:21:093
35417699134,5cyclictest0-21swapper/308:01:003
35417699133,5cyclictest0-21swapper/308:40:013
35417699132,5cyclictest0-21swapper/308:50:553
35417699132,5cyclictest0-21swapper/307:30:573
35417699122,6cyclictest0-21swapper/308:15:553
35417699121,6cyclictest0-21swapper/308:10:593
35417699121,6cyclictest0-21swapper/307:55:583
35417699101,6cyclictest0-21swapper/307:10:493
35417699101,4cyclictest0-21swapper/307:25:023
3541769992,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional