You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 22:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 30, 2026 12:43:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
439577995547,5cyclictest504517-21/usr/sbin/munin09:45:570
439579995343,6cyclictest441077-21systemd-run07:15:012
439579995143,5cyclictest442580-21sendmail07:20:012
439578994938,8cyclictest474330-21diskmemload10:35:511
439578994838,5cyclictest0-21swapper/110:45:491
43957899483,34cyclictest499947-21ssh09:40:471
439578994739,4cyclictest0-21swapper/110:15:431
439578994739,4cyclictest0-21swapper/109:00:011
439578994736,7cyclictest534841-21cron10:25:011
439578994637,5cyclictest1055-21mta-sts-daemon07:25:501
439578994637,5cyclictest1055-21mta-sts-daemon07:25:501
439578994636,6cyclictest632173-21latency_hist12:25:021
439577994633,10cyclictest482479-21systemctl09:20:010
439578994534,6cyclictest0-21swapper/110:06:091
439577994527,9cyclictest151ktimers/007:50:450
439579994433,8cyclictest421-21dbus-daemon12:21:372
439578994436,4cyclictest0-21swapper/111:00:491
439577994436,5cyclictest547040-21cron10:40:010
439579994335,4cyclictest0-21swapper/209:10:422
439579994334,6cyclictest499205-21aten_r2power_cu09:40:342
439578994332,7cyclictest637703-21threads12:30:551
439577994336,4cyclictest599720-21latency_hist11:45:020
439577994330,5cyclictest421-21dbus-daemon11:00:490
439577994328,12cyclictest475304-21ssh09:10:390
439578994233,5cyclictest607756-21ssh11:55:021
439578994233,5cyclictest607756-21ssh11:55:021
439578994233,4cyclictest0-21swapper/111:20:431
439578994232,6cyclictest1055-21mta-sts-daemon11:35:461
439578994229,8cyclictest420-21avahi-daemon12:00:411
43957799425,29cyclictest609036-21ssh11:55:470
439579994130,7cyclictest641619-21mailstats12:35:542
439579994128,9cyclictest492298-21timerandwakeup09:31:012
439578994133,5cyclictest543080-21latency_hist10:35:021
43957899412,31cyclictest633219-21ssh12:25:431
439577994135,3cyclictest432970-21kworker/u8:13-xprtiod07:20:020
43957999405,28cyclictest516174-21sh10:00:522
439579994031,6cyclictest619854-21grep12:10:002
439579994030,7cyclictest456072-21/usr/sbin/munin08:05:022
439579994030,5cyclictest0-21swapper/209:45:492
439579994029,7cyclictest695-21cron10:45:012
439578994031,6cyclictest0-21swapper/112:00:001
439577994034,4cyclictest633167-21needreboot12:25:420
439577994032,5cyclictest569290-21unixbench_singl11:05:570
43957999393,30cyclictest576790-21ssh11:15:492
439579993915,5cyclictest36-21ksoftirqd/211:05:442
439579993913,4cyclictest36-21ksoftirqd/211:50:462
439579993913,4cyclictest36-21ksoftirqd/211:50:462
439578993932,5cyclictest439576-21cyclictest10:50:451
439578993930,5cyclictest453076-21systemd-run07:55:011
439578993930,5cyclictest1054-21mta-sts-daemon08:15:451
439579993833,2cyclictest0-21swapper/208:40:002
439579993815,6cyclictest36-21ksoftirqd/210:55:472
439579993815,6cyclictest36-21ksoftirqd/210:55:472
439578993826,8cyclictest641205-21missed_timers12:35:411
43957899382,30cyclictest516117-21ssh10:00:491
439577993832,4cyclictest644165-21/usr/sbin/munin12:40:010
439577993830,5cyclictest421-21dbus-daemon10:46:590
439579993729,5cyclictest261-21systemd-journal12:20:012
439579993729,4cyclictest0-21swapper/211:40:012
439579993727,7cyclictest623896-21cut12:15:012
43957999372,29cyclictest520258-21ssh10:06:042
439578993729,5cyclictest589394-21grep11:30:571
439578993729,5cyclictest524716-21tail10:11:041
439578993728,6cyclictest476144-21unixbench-2d09:10:551
439578993727,7cyclictest3838849-21cups-browsed11:45:501
439578993727,7cyclictest3838849-21cups-browsed10:25:581
439578993724,10cyclictest484285-21timerandwakeup09:21:031
43957899372,30cyclictest504042-21ssh09:45:481
43957899372,29cyclictest548382-21ssh10:40:471
439577993731,4cyclictest487708-21grep09:25:430
439577993731,4cyclictest487708-21grep09:25:430
439577993730,4cyclictest516658-21unixbench_multi10:00:590
439577993729,6cyclictest563326-21munin-run11:00:010
439577993729,6cyclictest563326-21munin-run11:00:010
439577993727,7cyclictest595581-21grep11:40:010
43957999363,28cyclictest596873-21ssh11:40:452
439579993629,4cyclictest583508-21cron11:25:012
439579993628,5cyclictest564844-21find11:00:542
439579993628,4cyclictest0-21swapper/209:25:532
439579993628,4cyclictest0-21swapper/209:25:532
439579993627,6cyclictest543594-21aten_r2power_po10:35:342
439579993625,8cyclictest483548-21kernelversion09:20:472
439578993630,3cyclictest0-21swapper/112:20:011
439578993627,5cyclictest465078-21kthreadd08:35:011
439578993627,5cyclictest0-21swapper/112:10:471
439577993629,4cyclictest531756-21/usr/sbin/munin10:20:390
439577993624,10cyclictest3838849-21cups-browsed10:40:500
439579993527,5cyclictest421-21dbus-daemon11:15:012
439579993527,5cyclictest261-21systemd-journal11:31:082
439579993525,4cyclictest0-21swapper/211:45:512
43957999352,26cyclictest556560-21ssh10:50:472
43957999352,24cyclictest532178-21ssh10:20:472
439578993526,6cyclictest1055-21mta-sts-daemon09:33:351
439578993525,7cyclictest261-21systemd-journal11:00:031
439578993525,7cyclictest261-21systemd-journal11:00:031
43957799355,27cyclictest0-21swapper/009:20:580
439577993529,3cyclictest0-21swapper/011:10:550
439577993528,4cyclictest457711-21latency_hist08:10:020
439577993527,5cyclictest589203-21timerandwakeup11:30:560
439577993526,7cyclictest616495-21sh12:05:370
439577993524,7cyclictest536605-21timerwakeupswit10:25:570
43957999349,4cyclictest36-21ksoftirqd/210:15:472
43957999346,7cyclictest633343-21ntpq12:25:482
43957999343,25cyclictest536134-21ssh10:25:472
439579993428,3cyclictest0-21swapper/208:25:012
439579993427,4cyclictest1051-21mta-sts-daemon07:30:442
439579993427,3cyclictest0-21swapper/209:35:502
439579993426,4cyclictest0-21swapper/209:55:452
439579993426,4cyclictest0-21swapper/209:55:452
439579993425,4cyclictest0-21swapper/212:30:502
43957999342,25cyclictest540181-21ssh10:30:452
43957999342,23cyclictest474330-21diskmemload11:25:512
439579993416,12cyclictest548-21gdbus09:15:062
439578993427,3cyclictest0-21swapper/111:20:011
439578993426,5cyclictest616527-21telnet12:05:371
439578993426,5cyclictest448610-21latency_hist07:40:021
439578993424,5cyclictest0-21swapper/108:40:451
439577993427,4cyclictest420-21avahi-daemon09:00:520
439577993427,4cyclictest0-21swapper/012:30:470
439577993427,3cyclictest0-21swapper/009:35:060
439577993426,5cyclictest1-21systemd11:15:080
439577993423,4cyclictest469711-21latency_hist08:50:020
439579993326,4cyclictest0-21swapper/207:10:012
439579993326,3cyclictest0-21swapper/212:00:002
439579993325,5cyclictest450067-21cron07:45:012
439579993325,5cyclictest1-21systemd10:46:592
439579993311,5cyclictest36-21ksoftirqd/209:05:462
439578993327,3cyclictest0-21swapper/111:05:521
439578993325,5cyclictest495988-21ssh09:35:511
43957799334,3cyclictest0-21swapper/010:15:470
439577993327,3cyclictest0-21swapper/010:55:010
439577993325,5cyclictest421-21dbus-daemon12:05:010
439577993324,6cyclictest474330-21diskmemload11:45:500
43957799332,26cyclictest659-21snmpd07:55:490
43957799332,23cyclictest508048-21ssh09:50:480
439577993310,18cyclictest12350irq/32-eth011:50:480
439577993310,18cyclictest12350irq/32-eth011:50:480
439579993225,3cyclictest0-21swapper/212:00:492
439579993222,5cyclictest0-21swapper/210:10:562
439579993219,5cyclictest462698-21interrupts08:25:392
439579993219,5cyclictest462698-21interrupts08:25:392
43957899323,21cyclictest479770-21ssh09:15:491
439578993224,5cyclictest487249-21aten_r2power_vo09:25:361
439578993224,5cyclictest487249-21aten_r2power_vo09:25:361
439578993224,5cyclictest463572-21idleruntime-cro08:30:001
439578993224,5cyclictest463572-21idleruntime-cro08:30:001
439578993222,6cyclictest510919-21latency_hist09:55:031
439577993224,3cyclictest0-21swapper/010:10:550
439577993223,6cyclictest420-21avahi-daemon12:17:570
439577993220,9cyclictest0-21swapper/009:34:550
439579993123,5cyclictest445590-21systemd-run07:30:012
439579993123,5cyclictest445590-21systemd-run07:30:012
439578993125,3cyclictest0-21swapper/107:20:011
439578993124,3cyclictest0-21swapper/111:25:491
43957799317,16cyclictest151ktimers/008:35:450
439577993125,2cyclictest0-21swapper/011:30:010
439577993124,5cyclictest620687-21fschecks_count12:10:400
439577993122,4cyclictest0-21swapper/010:34:350
439577993118,10cyclictest261-21systemd-journal09:45:010
439577993113,14cyclictest624-21lldpd09:55:440
439577993113,14cyclictest624-21lldpd09:55:440
439579993023,4cyclictest448548-21cron07:40:012
439579993010,5cyclictest36-21ksoftirqd/209:50:502
439578993023,5cyclictest1-21systemd08:25:011
439578993023,4cyclictest572816-21processes11:10:521
439578993022,4cyclictest0-21swapper/109:55:541
439578993022,4cyclictest0-21swapper/109:55:541
43957799306,5cyclictest171rcu_preempt07:25:470
43957799306,5cyclictest171rcu_preempt07:25:470
439577993023,5cyclictest420-21avahi-daemon11:22:410
439577993022,4cyclictest0-21swapper/012:21:480
439577993013,13cyclictest151ktimers/008:40:420
43957999299,4cyclictest36-21ksoftirqd/208:15:352
439579992911,4cyclictest36-21ksoftirqd/208:55:432
439579992910,6cyclictest36-21ksoftirqd/208:05:362
439579992910,4cyclictest36-21ksoftirqd/209:00:492
439578992921,5cyclictest444009-21unixbench_singl07:20:571
439577992923,4cyclictest459583-21fschecks_time08:15:370
439577992911,14cyclictest151ktimers/007:45:450
43957999289,4cyclictest36-21ksoftirqd/208:45:472
43957999289,4cyclictest36-21ksoftirqd/207:55:382
439579992822,3cyclictest0-21swapper/207:25:012
439579992810,4cyclictest36-21ksoftirqd/208:10:482
439579992810,4cyclictest36-21ksoftirqd/207:45:432
439578992822,3cyclictest0-21swapper/107:35:001
439578992820,5cyclictest456584-21fschecks_count08:05:381
439578992820,5cyclictest441053-21cron07:15:011
439578992819,6cyclictest597470-21unixbench_singl11:40:551
439578992812,11cyclictest421-21dbus-daemon08:35:441
43957799283,21cyclictest520217-21ssh10:06:030
439577992821,5cyclictest443984-21unixbench_multi07:20:570
439577992818,6cyclictest12350irq/32-eth007:30:480
43957999279,5cyclictest36-21ksoftirqd/207:50:362
43957999278,5cyclictest36-21ksoftirqd/208:40:442
43957999274,17cyclictest1056-21mta-sts-daemon08:30:452
439578992721,3cyclictest0-21swapper/109:10:021
439578992720,4cyclictest0-21swapper/108:00:011
439578992718,7cyclictest459057-21/usr/sbin/munin08:15:021
439579992619,3cyclictest0-21swapper/208:55:012
439578992620,3cyclictest0-21swapper/108:55:021
439577992620,3cyclictest0-21swapper/007:40:010
43957899255,12cyclictest420-21avahi-daemon09:00:461
439578992520,3cyclictest0-21swapper/107:50:011
439578992518,5cyclictest456064-21cron08:05:011
439578992517,5cyclictest469509-21unixbench_singl08:45:561
439577992519,4cyclictest1051-21mta-sts-daemon08:20:430
439577992513,9cyclictest465063-21munin-plugin-st08:35:000
439577992415,4cyclictest0-21swapper/008:55:420
439578992318,2cyclictest0-21swapper/107:45:011
43957799236,15cyclictest460604-21kworker/u8:18+events_unbound08:53:580
439577992318,2cyclictest0-21swapper/008:00:360
439577992317,4cyclictest436533-21kworker/u8:9-xprtiod07:10:020
439577992317,3cyclictest1056-21mta-sts-daemon07:10:500
439577992315,6cyclictest3838849-21cups-browsed08:25:390
439577992315,6cyclictest3838849-21cups-browsed08:25:390
439578992211,7cyclictest439576-21cyclictest07:10:011
439577992215,5cyclictest449115-21fschecks_time07:40:380
43957799221,18cyclictest458425-21ntpq08:10:430
43957799212,9cyclictest473536-21wc09:05:400
43958099183,8cyclictest0-21swapper/310:00:513
43958099173,9cyclictest0-21swapper/310:06:063
43958099172,7cyclictest0-21swapper/311:05:343
43958099163,6cyclictest0-21swapper/309:30:523
43958099162,9cyclictest0-21swapper/312:30:533
43958099161,9cyclictest0-21swapper/311:30:433
43958099161,8cyclictest0-21swapper/311:25:503
43958099161,8cyclictest0-21swapper/311:15:433
43958099161,7cyclictest0-21swapper/309:25:463
43958099161,7cyclictest0-21swapper/309:25:463
43958099154,7cyclictest0-21swapper/311:45:463
43958099154,5cyclictest0-21swapper/309:15:473
43958099153,8cyclictest0-21swapper/309:35:513
43958099153,7cyclictest0-21swapper/311:40:423
43958099152,8cyclictest0-21swapper/310:45:453
43958099152,7cyclictest0-21swapper/311:35:403
43958099151,8cyclictest0-21swapper/312:20:453
43958099151,8cyclictest0-21swapper/309:50:453
43958099151,8cyclictest0-21swapper/309:20:563
43958099151,7cyclictest0-21swapper/310:40:453
43958099151,6cyclictest0-21swapper/308:40:443
43958099151,6cyclictest0-21swapper/307:20:443
43958099143,8cyclictest0-21swapper/308:35:443
43958099143,6cyclictest0-21swapper/312:35:443
43958099142,8cyclictest0-21swapper/310:50:443
43958099142,7cyclictest0-21swapper/310:55:573
43958099142,7cyclictest0-21swapper/310:55:573
43958099142,7cyclictest0-21swapper/310:25:013
43958099142,7cyclictest0-21swapper/309:55:443
43958099142,7cyclictest0-21swapper/309:55:443
43958099142,5cyclictest0-21swapper/307:25:543
43958099142,5cyclictest0-21swapper/307:25:543
43958099141,8cyclictest0-21swapper/312:25:443
43958099141,8cyclictest0-21swapper/312:05:503
43958099141,8cyclictest0-21swapper/312:00:433
43958099141,8cyclictest0-21swapper/311:50:463
43958099141,8cyclictest0-21swapper/311:50:463
43958099141,8cyclictest0-21swapper/310:25:473
43958099141,8cyclictest0-21swapper/310:10:373
43958099141,8cyclictest0-21swapper/309:10:433
43958099141,6cyclictest0-21swapper/308:20:433
43958099141,6cyclictest0-21swapper/307:55:423
43958099141,6cyclictest0-21swapper/307:35:473
43958099133,6cyclictest0-21swapper/311:55:383
43958099133,4cyclictest0-21swapper/307:45:453
43958099132,7cyclictest0-21swapper/312:10:403
43958099132,6cyclictest0-21swapper/308:00:443
43958099131,8cyclictest0-21swapper/310:30:223
43958099131,7cyclictest0-21swapper/310:15:443
43958099131,7cyclictest0-21swapper/309:45:493
43958099131,7cyclictest0-21swapper/309:40:453
43958099131,6cyclictest0-21swapper/308:05:453
43958099131,6cyclictest0-21swapper/307:10:463
43958099131,5cyclictest0-21swapper/308:15:423
43958099123,7cyclictest0-21swapper/310:35:383
43958099123,5cyclictest0-21swapper/311:00:053
43958099121,7cyclictest0-21swapper/312:15:453
43958099121,7cyclictest0-21swapper/311:20:373
43958099121,6cyclictest0-21swapper/311:10:453
43958099121,6cyclictest0-21swapper/308:10:443
43958099121,5cyclictest0-21swapper/309:00:443
43958099111,6cyclictest0-21swapper/309:10:033
43958099111,6cyclictest0-21swapper/308:25:423
43958099111,6cyclictest0-21swapper/308:25:423
43958099111,5cyclictest0-21swapper/308:30:453
43958099111,5cyclictest0-21swapper/307:15:413
43958099101,5cyclictest0-21swapper/308:55:313
43958099100,6cyclictest0-21swapper/307:30:363
4395809993,3cyclictest0-21swapper/308:45:463
4395809991,5cyclictest0-21swapper/307:50:343
4395809991,5cyclictest0-21swapper/307:40:343
4395809991,4cyclictest0-21swapper/308:55:033
4395809991,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional