You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 18:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu May 07, 2026 12:43:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1593074995751,4cyclictest1637192-21grep09:20:010
1593075995439,11cyclictest1055-21mta-sts-daemon09:20:501
1593074995446,5cyclictest1-21systemd11:15:020
1593074995142,6cyclictest790599-21cups-browsed10:20:490
1593075994636,6cyclictest1718243-21latency_hist11:00:031
1593074994538,4cyclictest1633117-21kworker/u8:8-events_unbound09:55:490
1593074994534,7cyclictest421-21dbus-daemon07:30:010
1593076994436,5cyclictest1637181-21sendmail09:20:012
1593075994436,5cyclictest1713997-21munin-plugin-st10:55:011
1593074994332,9cyclictest1726095-21munin-plugin-st11:10:000
1593076994234,4cyclictest0-21swapper/210:35:012
1593074994235,4cyclictest1-21systemd07:45:020
1593075994133,5cyclictest1600558-21/usr/sbin/munin07:35:011
1593076994033,4cyclictest1728318-21kworker/u8:1-xprtiod12:00:542
1593076994033,3cyclictest0-21swapper/210:50:502
1593076994032,6cyclictest1603582-21systemd-run07:45:012
1593076994032,5cyclictest1794666-21idleruntime-cro12:35:012
1593076994030,8cyclictest1718074-21ssh11:00:022
1593075994032,5cyclictest1681747-21systemd-run10:15:011
1593075994032,5cyclictest1681747-21systemd-run10:15:011
1593075994031,6cyclictest1629049-21grep09:10:011
1593075994029,8cyclictest0-21swapper/111:40:501
1593074994033,5cyclictest0-21swapper/010:10:410
1593074994033,5cyclictest0-21swapper/010:10:410
1593074994032,5cyclictest790599-21cups-browsed11:36:520
1593074994032,5cyclictest790599-21cups-browsed11:36:520
1593074994032,5cyclictest659-21snmpd11:16:140
1593074994032,5cyclictest1-21systemd12:40:020
1593074994030,7cyclictest1665556-21cut09:55:020
1593076993932,3cyclictest0-21swapper/210:25:552
1593076993931,5cyclictest1626075-21systemd-run09:00:012
1593076993930,6cyclictest420-21avahi-daemon09:13:312
1593076993929,7cyclictest421-21dbus-daemon12:35:552
1593076993929,7cyclictest1735990-21timerwakeupswit11:21:022
1593075993930,6cyclictest1798695-21sendmail12:40:011
1593075993930,6cyclictest1-21systemd11:05:011
159307599392,28cyclictest1650627-21ssh09:35:481
159307599391,30cyclictest1743509-21ssh11:30:471
1593074993933,3cyclictest0-21swapper/012:25:480
1593074993932,5cyclictest421-21dbus-daemon10:50:020
1593074993932,4cyclictest1646019-21aten_r2power_vo09:30:380
1593074993932,4cyclictest1646019-21aten_r2power_vo09:30:380
1593074993930,6cyclictest790599-21cups-browsed09:13:420
1593074993930,6cyclictest420-21avahi-daemon10:02:260
1593074993930,5cyclictest1776902-21kworker/u8:0-xprtiod12:20:530
1593074993929,7cyclictest1691389-21mailstats10:25:570
1593076993830,5cyclictest1701912-21cron10:40:012
159307699381,31cyclictest1747634-21ssh11:35:512
159307699381,31cyclictest1747634-21ssh11:35:512
1593075993832,4cyclictest1593073-21cyclictest11:40:001
1593075993832,4cyclictest1593073-21cyclictest11:40:001
1593075993830,5cyclictest1794658-21ssh12:35:011
1593075993830,5cyclictest1689816-21cron10:25:011
1593075993824,10cyclictest1727856-21timerwakeupswit11:10:581
1593074993832,4cyclictest1770560-21cut12:05:010
1593074993826,9cyclictest0-21swapper/009:35:570
1593076993730,4cyclictest1669607-21idleruntime-cro10:00:012
1593076993728,6cyclictest790599-21cups-browsed12:07:182
1593076993728,6cyclictest790599-21cups-browsed12:07:182
1593076993726,8cyclictest1751580-21ntp_kernel_err11:40:472
1593075993728,6cyclictest421-21dbus-daemon09:55:011
1593075993728,6cyclictest1627544-21cron09:05:011
1593075993727,6cyclictest420-21avahi-daemon09:15:561
1593075993722,11cyclictest1658369-21idleruntime09:45:411
159307499374,18cyclictest171rcu_preempt10:35:530
1593074993730,4cyclictest1722051-21cron11:05:000
1593076993630,4cyclictest1744129-21unixbench_multi11:31:002
1593076993630,3cyclictest0-21swapper/211:55:572
1593076993629,3cyclictest0-21swapper/209:25:492
1593076993628,5cyclictest1601931-21unixbench_multi07:36:002
1593075993630,3cyclictest0-21swapper/107:55:011
1593074993619,14cyclictest1766581-21latency_hist12:00:020
1593076993529,3cyclictest0-21swapper/211:50:542
1593076993529,3cyclictest0-21swapper/210:20:002
1593076993527,6cyclictest695-21cron12:20:002
1593076993527,5cyclictest695-21cron09:25:012
1593076993527,5cyclictest1719947-21unixbench_singl11:01:002
1593076993525,7cyclictest1653458-21systemctl09:40:002
1593075993528,4cyclictest1705987-21kworker/u8:4-xprtiod11:55:461
1593075993528,4cyclictest0-21swapper/110:00:011
1593075993527,5cyclictest1767475-21if_eth012:00:421
1593075993527,5cyclictest1643127-21grep09:26:001
1593075993526,6cyclictest1690697-21sort10:25:411
1593074993527,5cyclictest1714529-21/usr/sbin/munin10:55:380
1593074993526,7cyclictest12350irq/32-eth011:20:510
1593076993428,3cyclictest0-21swapper/210:11:322
1593076993428,3cyclictest0-21swapper/210:11:322
1593076993426,5cyclictest1627449-21tail09:01:012
1593076993425,6cyclictest1675310-21switchtime10:06:002
1593076993425,6cyclictest1675310-21switchtime10:06:002
1593075993427,4cyclictest1051-21mta-sts-daemon11:15:361
1593075993424,6cyclictest1710078-21latency_hist10:50:021
159307599342,26cyclictest1695142-21sh10:30:481
159307499343,24cyclictest1642475-21ssh09:25:460
1593074993428,3cyclictest0-21swapper/007:20:010
1593074993427,5cyclictest790599-21cups-browsed08:45:500
1593074993426,6cyclictest1683385-21switchtime10:15:590
159307499342,28cyclictest1771948-21rm12:05:530
159307499342,28cyclictest1771948-21rm12:05:530
1593074993414,5cyclictest0-21swapper/011:45:540
1593076993327,3cyclictest0-21swapper/212:25:012
1593076993327,3cyclictest0-21swapper/209:55:022
1593076993324,6cyclictest790599-21cups-browsed08:42:322
1593075993324,6cyclictest790599-21cups-browsed09:35:011
1593075993324,6cyclictest790599-21cups-browsed09:35:011
1593075993324,6cyclictest1596051-21cron07:20:001
159307599331,26cyclictest1675104-21ssh10:05:551
159307599331,26cyclictest1675104-21ssh10:05:551
159307599331,22cyclictest1670841-21ssh10:00:451
1593074993326,5cyclictest790599-21cups-browsed09:22:330
1593074993326,5cyclictest421-21dbus-daemon08:00:010
1593074993326,5cyclictest0-21swapper/012:35:010
1593074993323,7cyclictest1658807-21ntpq09:45:500
1593074993323,6cyclictest0-21swapper/010:06:000
1593074993323,6cyclictest0-21swapper/010:06:000
1593074993318,11cyclictest12350irq/32-eth011:30:540
1593074993313,6cyclictest171rcu_preempt07:10:520
1593074993313,6cyclictest171rcu_preempt07:10:520
1593076993224,5cyclictest1617059-21munin-plugin-st08:30:002
1593076993224,5cyclictest1054-21mta-sts-daemon09:05:492
1593076993224,5cyclictest0-21swapper/211:10:532
1593075993227,2cyclictest0-21swapper/111:50:021
1593075993225,4cyclictest0-21swapper/110:20:001
1593075993224,5cyclictest1784167-21systemctl12:20:561
1593075993224,5cyclictest1603558-21cron07:45:011
1593075993223,6cyclictest439-21systemd-logind11:55:021
1593075993223,4cyclictest0-21swapper/112:05:511
1593075993223,4cyclictest0-21swapper/112:05:511
159307599321,21cyclictest1699331-21ssh10:35:581
1593074993224,6cyclictest1752294-21unixbench_singl11:40:590
159307499322,27cyclictest1702952-21uname10:40:450
1593076993125,4cyclictest1593073-21cyclictest08:10:472
1593076993124,5cyclictest1605062-21idleruntime-cro07:50:012
1593076993124,4cyclictest1788233-21sendmail_mailtr12:25:572
1593076993124,4cyclictest1054-21mta-sts-daemon11:18:032
1593076993124,3cyclictest0-21swapper/211:45:582
1593076993124,3cyclictest0-21swapper/211:25:552
1593076993123,5cyclictest1687024-21ntp_kernel_err10:20:462
1593075993123,5cyclictest1788599-21unixbench_singl12:26:011
1593075993122,6cyclictest1605078-21systemd-run07:50:011
159307599312,25cyclictest1735457-21ssh11:20:481
1593075993121,7cyclictest1617882-21netstat08:30:451
1593074993126,2cyclictest0-21swapper/012:15:000
1593074993125,3cyclictest0-21swapper/011:25:410
1593074993124,4cyclictest420-21avahi-daemon10:34:360
1593074993122,7cyclictest1619355-21needreboot08:35:450
159307499312,24cyclictest421-21dbus-daemon07:45:460
159307499312,21cyclictest1779887-21ssh12:15:490
1593074993118,10cyclictest0-21swapper/010:52:270
1593076993024,3cyclictest0-21swapper/210:45:422
1593076993024,2cyclictest0-21swapper/208:10:012
1593076993022,5cyclictest1654201-21aten_r2power_vo09:40:392
159307699302,21cyclictest1703378-21ssh10:40:562
1593076993021,6cyclictest421-21dbus-daemon10:05:022
1593076993021,6cyclictest1596051-21cron07:20:002
1593075993022,5cyclictest1630974-21unixbench_singl09:11:011
1593075993021,6cyclictest790599-21cups-browsed08:05:591
1593074993022,5cyclictest1-21systemd07:51:310
1593076992923,3cyclictest0-21swapper/212:10:532
1593076992921,5cyclictest1622930-21sed08:46:002
1593076992921,5cyclictest1607952-21unixbench_multi07:56:012
1593076992920,7cyclictest1605902-21ntp_kernel_err07:50:462
1593075992923,3cyclictest0-21swapper/112:10:481
1593075992921,5cyclictest1054-21mta-sts-daemon12:15:371
1593074992920,7cyclictest1629039-21cron09:10:010
1593076992820,5cyclictest1599080-21systemd-run07:30:012
1593075992823,3cyclictest1593073-21cyclictest08:10:501
1593075992822,3cyclictest0-21swapper/108:00:011
1593075992820,5cyclictest1738873-21aten_r2power_fr11:25:371
1593074992821,5cyclictest790599-21cups-browsed08:03:250
1593074992820,6cyclictest1614911-21ntp_kernel_err08:20:450
1593074992820,6cyclictest1614911-21ntp_kernel_err08:20:450
1593074992820,6cyclictest151ktimers/009:00:500
1593074992820,5cyclictest1054-21mta-sts-daemon07:38:480
1593074992817,8cyclictest790599-21cups-browsed07:20:490
1593076992722,2cyclictest0-21swapper/208:20:582
1593076992722,2cyclictest0-21swapper/208:20:582
1593076992721,3cyclictest0-21swapper/209:45:512
1593076992719,5cyclictest695-21cron08:55:012
1593076992719,5cyclictest1051-21mta-sts-daemon08:04:202
1593076992712,10cyclictest1723365-21ntpq11:05:502
1593075992722,2cyclictest0-21swapper/107:40:011
159307599272,18cyclictest1654716-21ssh09:40:481
1593075992721,3cyclictest0-21swapper/109:00:011
1593075992720,4cyclictest1613393-21ntp_kernel_err08:15:451
1593075992720,4cyclictest1054-21mta-sts-daemon11:08:201
1593075992718,6cyclictest790599-21cups-browsed08:50:021
1593075992718,6cyclictest548-21gdbus08:38:051
1593074992721,4cyclictest421-21dbus-daemon08:30:010
1593074992720,5cyclictest1623065-21kworker/u8:14-xprtiod09:40:380
1593074992718,6cyclictest1760354-21grep11:51:000
159307699263,12cyclictest1629051-21diskmemload09:35:012
159307699263,12cyclictest1629051-21diskmemload09:35:012
1593076992618,5cyclictest421-21dbus-daemon08:20:012
159307599264,12cyclictest1629051-21diskmemload10:40:351
1593075992619,4cyclictest1617072-21true08:30:011
1593075992618,5cyclictest1608429-21aten_r2power_po08:00:381
1593075992615,8cyclictest1-21systemd07:20:421
1593074992620,4cyclictest0-21swapper/008:30:490
1593074992617,7cyclictest1-21systemd08:09:320
159307499260,4cyclictest171rcu_preempt08:15:460
1593076992517,5cyclictest1618536-21cron08:35:012
1593076992516,7cyclictest1593934-21ntp_kernel_pll_07:10:482
1593076992516,7cyclictest1593934-21ntp_kernel_pll_07:10:482
159307599253,14cyclictest428-21mta-sts-daemon08:50:501
159307599252,16cyclictest428-21mta-sts-daemon08:20:471
159307599252,16cyclictest428-21mta-sts-daemon08:20:471
1593075992518,4cyclictest1620432-21aten2.4-expect08:40:391
1593074992518,4cyclictest1591737-21kworker/u8:8-xprtiod07:30:430
1593074992511,9cyclictest1612031-21sshd08:10:540
1593076992419,2cyclictest0-21swapper/208:40:012
1593076992413,8cyclictest1599964-21ntp_kernel_pll_07:30:482
1593075992417,5cyclictest548-21gdbus07:27:251
1593075992416,5cyclictest1593591-21fschecks_time07:10:421
1593075992416,5cyclictest1593591-21fschecks_time07:10:421
1593074992417,4cyclictest1625904-21tail08:55:590
159307499241,11cyclictest1623414-21aten_r2power_po08:50:380
159307499235,12cyclictest1620521-21ls08:40:390
159307799225,8cyclictest0-21swapper/311:25:503
159307799224,11cyclictest0-21swapper/311:30:493
159307799222,13cyclictest0-21swapper/311:35:533
159307799222,13cyclictest0-21swapper/311:35:533
159307699221,16cyclictest659-21snmpd07:20:492
159307799213,11cyclictest0-21swapper/309:55:503
159307799213,11cyclictest0-21swapper/309:10:483
159307799212,12cyclictest0-21swapper/311:50:483
159307799212,11cyclictest0-21swapper/309:15:463
159307799204,11cyclictest0-21swapper/310:50:513
159307799204,10cyclictest0-21swapper/310:55:523
159307799203,12cyclictest0-21swapper/311:15:503
159307799203,11cyclictest0-21swapper/311:49:323
159307799203,11cyclictest0-21swapper/309:45:473
159307799202,13cyclictest0-21swapper/310:10:483
159307799202,13cyclictest0-21swapper/310:10:483
159307799202,12cyclictest0-21swapper/311:00:483
159307799202,12cyclictest0-21swapper/309:40:543
159307799202,12cyclictest0-21swapper/309:30:473
159307799202,12cyclictest0-21swapper/309:30:473
159307799202,11cyclictest0-21swapper/310:45:493
159307799202,11cyclictest0-21swapper/310:20:493
159307799202,11cyclictest0-21swapper/310:05:483
159307799202,11cyclictest0-21swapper/310:05:483
159307799202,10cyclictest0-21swapper/310:25:483
159307799201,15cyclictest0-21swapper/312:25:483
159307799201,12cyclictest0-21swapper/307:20:513
159307499209,7cyclictest261-21systemd-journal07:10:010
159307499209,7cyclictest261-21systemd-journal07:10:010
159307799193,9cyclictest0-21swapper/311:20:503
159307799193,9cyclictest0-21swapper/310:00:453
159307799193,8cyclictest0-21swapper/307:45:463
159307799193,10cyclictest0-21swapper/310:40:513
159307799193,10cyclictest0-21swapper/310:35:533
159307799192,9cyclictest0-21swapper/307:35:483
159307799192,8cyclictest0-21swapper/308:05:503
159307799192,12cyclictest0-21swapper/312:35:583
159307799192,12cyclictest0-21swapper/312:30:543
159307799192,11cyclictest0-21swapper/312:10:483
159307799192,11cyclictest0-21swapper/310:31:013
159307799192,11cyclictest0-21swapper/310:15:463
159307799192,11cyclictest0-21swapper/309:50:483
159307799192,11cyclictest0-21swapper/309:25:463
159307799192,11cyclictest0-21swapper/309:20:523
159307799192,10cyclictest0-21swapper/312:00:483
159307799192,10cyclictest0-21swapper/311:10:483
159307599192,12cyclictest1593027-21/usr/sbin/munin07:10:011
159307599192,12cyclictest1593027-21/usr/sbin/munin07:10:011
159307799185,9cyclictest0-21swapper/311:05:513
159307799183,8cyclictest0-21swapper/308:45:483
159307799183,11cyclictest0-21swapper/309:10:013
159307799183,10cyclictest0-21swapper/311:55:503
159307799182,11cyclictest0-21swapper/312:20:423
159307799182,11cyclictest0-21swapper/312:15:463
159307799182,10cyclictest0-21swapper/309:35:513
159307799182,10cyclictest0-21swapper/307:25:493
159307799181,8cyclictest0-21swapper/308:20:473
159307799181,8cyclictest0-21swapper/308:20:473
159307799181,11cyclictest0-21swapper/312:05:503
159307799181,11cyclictest0-21swapper/312:05:503
159307799180,9cyclictest0-21swapper/307:55:523
159307799173,9cyclictest0-21swapper/308:25:493
159307799173,7cyclictest0-21swapper/308:00:463
159307799173,10cyclictest0-21swapper/309:00:513
159307799172,9cyclictest0-21swapper/307:40:473
159307799172,8cyclictest0-21swapper/307:10:513
159307799172,8cyclictest0-21swapper/307:10:513
159307799172,10cyclictest0-21swapper/311:40:393
159307799171,10cyclictest0-21swapper/308:15:473
1593076991713,2cyclictest0-21swapper/207:10:022
1593076991713,2cyclictest0-21swapper/207:10:022
159307799163,9cyclictest0-21swapper/308:35:513
159307799162,8cyclictest0-21swapper/308:50:493
159307799162,8cyclictest0-21swapper/307:15:493
159307799153,7cyclictest0-21swapper/308:40:473
159307799152,7cyclictest0-21swapper/308:55:413
159307799151,9cyclictest0-21swapper/308:30:523
159307799151,9cyclictest0-21swapper/307:50:473
159307799143,7cyclictest0-21swapper/308:10:473
159307799141,8cyclictest0-21swapper/307:30:453
159307799111,7cyclictest0-21swapper/307:10:013
159307799111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional