You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 00:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Aug 13, 2026 12:43:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1714976996353,6cyclictest1854482-21idleruntime-cro12:00:012
1714976996050,6cyclictest1782699-21date10:01:012
1714976995343,5cyclictest1851472-21systemd11:55:022
1714976995041,5cyclictest1741983-21kworker/u8:11+xprtiod08:40:022
1714976994836,7cyclictest420-21avahi-daemon11:01:402
1714974994839,5cyclictest1781489-21awk10:00:010
1714976994737,6cyclictest1792331-21grep10:16:082
1714975994738,6cyclictest1749477-21systemd-run09:05:011
1714975994737,6cyclictest1853177-21tail11:56:081
171497699461,30cyclictest1822309-21ssh11:05:592
1714976994536,5cyclictest1729956-21systemctl08:00:012
1714976994531,7cyclictest1808926-21sh10:45:012
1714975994539,3cyclictest0-21swapper/112:25:541
1714975994539,3cyclictest0-21swapper/112:25:541
1714975994537,5cyclictest1744223-21/usr/sbin/munin08:45:511
1714976994432,5cyclictest0-21swapper/212:30:532
1714976994422,4cyclictest171rcu_preempt08:40:542
1714975994437,4cyclictest0-21swapper/110:40:011
1714976994335,5cyclictest1815584-21aten_r2power_po10:55:462
1714974994333,6cyclictest926384-21cups-browsed10:55:590
171497699423,29cyclictest1855643-21ssh12:00:552
171497699423,29cyclictest1001-21mta-sts-daemon12:21:022
171497699423,29cyclictest1001-21mta-sts-daemon12:21:022
1714976994229,7cyclictest0-21swapper/212:05:582
1714975994234,5cyclictest1848419-21ssh11:50:011
1714975994234,4cyclictest0-21swapper/111:30:561
1714976994132,6cyclictest351ktimers/211:11:022
1714976994131,6cyclictest1767392-21ntp_kernel_pll_09:35:562
1714975994134,4cyclictest1741973-21sendmail08:40:011
1714976994031,6cyclictest1805849-21munin-run10:40:012
1714975994033,4cyclictest1784550-21munin-plugin-st10:05:011
1714975994032,5cyclictest1719497-21systemd-run07:25:021
171497599401,33cyclictest1788841-21ssh10:11:001
1714974994030,7cyclictest1822323-21ntpq11:06:000
1714976993928,7cyclictest926384-21cups-browsed09:58:192
1714976993928,7cyclictest1001-21mta-sts-daemon09:13:442
1714976993926,8cyclictest420-21avahi-daemon10:08:032
171497699392,29cyclictest1794803-21ssh10:20:532
171497699392,29cyclictest1794803-21ssh10:20:532
1714974993919,14cyclictest151ktimers/007:15:550
1714976993830,5cyclictest1842487-21latency_hist11:40:022
1714976993830,5cyclictest1737477-21idleruntime-cro08:25:002
1714976993829,5cyclictest1846736-21sendmail_mailst11:46:002
1714976993829,3cyclictest0-21swapper/212:40:012
1714974993833,3cyclictest1778443-21grep09:55:010
1714974993826,9cyclictest1787603-21kworker/u8:8+xprtiod10:40:010
1714974993826,10cyclictest1793675-21sendmail10:20:010
1714976993730,3cyclictest0-21swapper/208:20:002
1714976993727,6cyclictest421-21dbus-daemon08:10:012
171497699371,15cyclictest175523950ssh09:15:582
1714975993729,5cyclictest1794461-21fschecks_time10:20:481
1714975993729,5cyclictest1794461-21fschecks_time10:20:481
1714975993729,5cyclictest1757117-21ssh09:20:021
1714975993724,10cyclictest1801504-21unixbench_multi10:31:091
1714974993729,5cyclictest1840539-21ld11:36:000
1714974993725,10cyclictest1745477-21fschecks_time08:50:470
1714976993630,4cyclictest1714973-21cyclictest10:50:472
1714976993629,4cyclictest1831161-21missed_timers11:20:512
1714976993629,4cyclictest1831161-21missed_timers11:20:512
1714976993628,5cyclictest1830158-21idleruntime-cro11:20:012
1714976993628,5cyclictest1830158-21idleruntime-cro11:20:012
1714976993628,4cyclictest0-21swapper/209:10:012
171497699362,26cyclictest1773602-21ssh09:45:592
171497599366,18cyclictest428-21mta-sts-daemon08:21:011
1714975993628,5cyclictest1865319-21unixbench-2d12:16:091
171497499362,25cyclictest665-21snmpd11:56:000
1714974993622,11cyclictest1755795-21head09:16:080
1714976993523,9cyclictest1745792-21ntp_kernel_err08:50:522
1714975993528,4cyclictest1001-21mta-sts-daemon10:19:131
1714975993528,3cyclictest0-21swapper/112:10:551
1714975993528,3cyclictest0-21swapper/109:11:001
1714975993527,5cyclictest1824675-21aten_r2power_po11:10:451
1714975993527,5cyclictest1799796-21latency_hist10:30:011
1714975993527,5cyclictest1750954-21diskmemload09:35:481
1714975993527,5cyclictest1750945-21cron09:10:011
1714975993527,4cyclictest1000-21mta-sts-daemon09:42:521
1714975993527,4cyclictest0-21swapper/109:50:531
1714974993530,3cyclictest1818072-21kworker/u8:6-xprtiod12:20:010
1714974993529,4cyclictest1834200-21missed_timers11:25:520
1714974993529,4cyclictest1834200-21missed_timers11:25:520
1714974993526,7cyclictest1-21systemd12:10:010
171497699343,22cyclictest1864832-21ssh12:15:592
1714976993428,3cyclictest0-21swapper/211:30:002
1714976993428,3cyclictest0-21swapper/211:30:002
1714976993426,5cyclictest1812024-21latency_hist10:50:022
1714976993426,4cyclictest1799788-21systemd10:30:012
1714976993426,4cyclictest0-21swapper/209:21:152
1714976993426,4cyclictest0-21swapper/209:21:152
1714976993425,5cyclictest420-21avahi-daemon08:30:372
1714975993428,3cyclictest0-21swapper/111:36:011
1714974993429,3cyclictest0-21swapper/008:15:010
1714974993427,4cyclictest1797374-21cpu10:25:460
1714974993427,4cyclictest12650irq/32-eth009:11:020
1714974993426,5cyclictest420-21avahi-daemon11:03:530
1714974993426,5cyclictest12650irq/32-eth011:21:010
1714974993426,5cyclictest12650irq/32-eth011:21:010
1714974993421,9cyclictest1813094-21sh10:50:550
1714976993324,6cyclictest420-21avahi-daemon07:40:012
1714976993322,7cyclictest1772374-21systemd-run09:45:012
1714976993321,8cyclictest1870888-21ntpq12:25:552
1714976993321,8cyclictest1870888-21ntpq12:25:552
171497699331,22cyclictest1764451-21ssh09:31:012
1714975993325,5cyclictest1830187-21/usr/sbin/munin11:20:021
1714975993325,5cyclictest1830187-21/usr/sbin/munin11:20:021
1714975993325,5cyclictest1818026-21munin-plugin-st11:00:011
1714974993328,3cyclictest1848404-21munin-plugin-st11:50:010
1714974993328,3cyclictest0-21swapper/012:40:020
1714974993325,3cyclictest0-21swapper/009:36:010
1714974993322,8cyclictest1736785-21needreboot08:20:530
1714976993225,3cyclictest0-21swapper/211:30:502
1714976993225,3cyclictest0-21swapper/209:30:012
1714976993225,3cyclictest0-21swapper/209:30:012
1714976993224,5cyclictest1-21systemd08:50:022
171497599323,23cyclictest926384-21cups-browsed10:51:021
1714975993225,4cyclictest1845369-21munin-run11:45:011
1714975993225,4cyclictest1773523-21ntp_kernel_pll_09:45:561
1714975993224,5cyclictest926384-21cups-browsed09:57:351
1714975993224,5cyclictest1763191-21ssh09:30:011
1714975993224,5cyclictest1763191-21ssh09:30:011
1714975993224,5cyclictest0-21swapper/110:45:511
1714975993223,6cyclictest1722810-21aten_r2power_po07:35:451
1714975993221,7cyclictest421-21dbus-daemon09:20:531
1714975993221,7cyclictest421-21dbus-daemon09:20:531
171497499323,22cyclictest1758228-21ssh09:20:540
171497499323,22cyclictest1758228-21ssh09:20:540
1714974993224,6cyclictest1855722-21ntp_offset12:00:590
1714974993224,4cyclictest1002-21mta-sts-daemon11:12:080
171497499321,28cyclictest1811957-21awk10:50:010
1714976993123,3cyclictest0-21swapper/212:10:552
1714976993122,7cyclictest0-21swapper/207:21:012
1714976993122,6cyclictest1726974-21systemd-run07:50:012
171497699312,18cyclictest1788787-21ntpq10:10:582
1714976993121,4cyclictest0-21swapper/209:54:502
1714976993119,9cyclictest1715864-21ntp_kernel_pll_07:10:562
1714976993119,8cyclictest1000-21mta-sts-daemon10:34:512
1714976993118,10cyclictest1730841-21ntp_kernel_pll_08:00:582
1714975993125,3cyclictest0-21swapper/111:51:091
1714975993123,4cyclictest1001-21mta-sts-daemon12:37:181
1714974993124,5cyclictest926384-21cups-browsed07:28:570
1714974993124,5cyclictest1782774-21/usr/sbin/munin10:01:040
1714974993124,4cyclictest1736010-21latency_hist08:20:020
171497499312,24cyclictest1873915-21ssh12:30:540
1714974993118,10cyclictest12650irq/32-eth011:16:020
1714974993118,10cyclictest12650irq/32-eth011:16:020
1714974993115,4cyclictest1750954-21diskmemload10:31:200
1714976993024,3cyclictest0-21swapper/207:20:012
1714975993024,3cyclictest0-21swapper/111:30:001
1714975993024,3cyclictest0-21swapper/111:30:001
1714975993024,3cyclictest0-21swapper/108:00:011
1714975993023,3cyclictest0-21swapper/107:15:561
1714975993022,5cyclictest1747945-21basename09:00:011
1714974993024,4cyclictest1870621-21kernelversion12:25:500
1714974993024,4cyclictest1870621-21kernelversion12:25:500
1714974993023,5cyclictest265-21systemd-journal08:00:010
1714974993020,7cyclictest1731995-21fschecks_time08:05:470
171497499301,20cyclictest12650irq/32-eth011:40:580
171497699293,18cyclictest428-21mta-sts-daemon07:40:572
1714976992922,4cyclictest1000-21mta-sts-daemon07:51:002
1714976992921,3cyclictest0-21swapper/211:40:552
1714975992922,3cyclictest0-21swapper/110:06:031
1714975992921,5cyclictest1732981-21systemctl08:10:011
1714975992921,3cyclictest0-21swapper/110:41:011
1714975992921,3cyclictest0-21swapper/109:31:081
1714975992916,10cyclictest1868180-21switchtime12:21:061
1714975992916,10cyclictest1868180-21switchtime12:21:061
1714974992923,3cyclictest1866653-21cron12:20:030
1714974992923,3cyclictest1866653-21cron12:20:030
1714974992920,7cyclictest694-21cron08:50:010
1714974992919,8cyclictest1003-21mta-sts-daemon09:45:310
1714974992918,8cyclictest1723274-21needreboot07:35:520
171497699281,5cyclictest0-21swapper/208:56:002
171497599283,19cyclictest1831371-21ssh11:20:571
171497599283,19cyclictest1831371-21ssh11:20:571
1714975992821,5cyclictest0-21swapper/107:41:071
1714975992820,5cyclictest1855671-21collect212:00:571
1714975992820,5cyclictest1746469-21systemd-run08:55:021
1714975992820,5cyclictest1720577-21systemctl07:26:061
1714974992822,3cyclictest1722478-21kworker/u8:10-xprtiod07:35:010
1714974992822,3cyclictest0-21swapper/009:30:450
1714974992822,3cyclictest0-21swapper/008:45:010
1714974992821,5cyclictest566-21gmain09:29:070
1714974992821,5cyclictest566-21gmain09:29:070
1714974992821,4cyclictest1719464-21cron07:25:010
1714974992820,5cyclictest1-21systemd10:10:010
1714974992819,6cyclictest0-21swapper/010:25:000
1714974992819,6cyclictest0-21swapper/010:25:000
1714976992719,5cyclictest1749453-21cron09:05:012
171497699271,5cyclictest171rcu_preempt08:10:582
1714975992721,3cyclictest0-21swapper/108:45:021
1714975992720,4cyclictest1000-21mta-sts-daemon11:08:251
1714975992718,4cyclictest0-21swapper/111:00:471
1714974992722,2cyclictest0-21swapper/007:15:010
1714974992721,4cyclictest926384-21cups-browsed08:57:230
1714974992720,4cyclictest1788923-21ps10:11:030
1714974992719,5cyclictest926384-21cups-browsed08:35:460
171497499271,20cyclictest665-21snmpd12:14:110
171497699265,11cyclictest1001-21mta-sts-daemon07:26:002
171497699263,15cyclictest627-21lldpd07:30:592
1714976992618,5cyclictest1002-21mta-sts-daemon08:29:172
1714976992618,5cyclictest1002-21mta-sts-daemon08:29:172
1714975992618,5cyclictest420-21avahi-daemon07:11:011
1714975992616,7cyclictest265-21systemd-journal07:55:011
171497499262,21cyclictest1849538-21ssh11:50:550
171497499262,19cyclictest1730579-21awk08:00:510
1714974992614,8cyclictest12650irq/32-eth010:40:590
171497599254,13cyclictest1003-21mta-sts-daemon12:05:561
1714975992519,4cyclictest1714973-21cyclictest07:45:591
1714975992519,4cyclictest0-21swapper/108:20:011
1714975992518,4cyclictest1733330-21aten2.4-expect08:10:451
1714975992518,4cyclictest1003-21mta-sts-daemon08:03:171
1714975992517,4cyclictest0-21swapper/112:30:401
1714974992520,3cyclictest1003-21mta-sts-daemon09:43:110
1714974992519,3cyclictest0-21swapper/011:30:440
171497599244,12cyclictest428-21mta-sts-daemon07:31:001
1714975992416,5cyclictest1739821-21ntp_kernel_err08:30:551
171497499249,11cyclictest1750988-21latency_hist09:10:020
1714974992419,2cyclictest0-21swapper/009:01:070
1714974992417,4cyclictest926384-21cups-browsed07:50:030
1714974992416,6cyclictest1726743-21timerwakeupswit07:46:060
1714974992414,7cyclictest1737983-21fschecks_count08:25:470
1714974992414,7cyclictest1737983-21fschecks_count08:25:470
171497799233,4cyclictest0-21swapper/311:15:593
171497799233,4cyclictest0-21swapper/311:15:593
171497799232,6cyclictest0-21swapper/308:40:543
171497799232,5cyclictest0-21swapper/308:50:523
171497799231,6cyclictest0-21swapper/310:25:543
1714975992318,2cyclictest0-21swapper/107:10:021
1714974992317,4cyclictest1724247-21aten2.4-expect07:40:460
171497499231,17cyclictest1739570-21awk08:30:500
171497799223,6cyclictest0-21swapper/312:26:073
171497799223,6cyclictest0-21swapper/312:26:073
171497799222,7cyclictest0-21swapper/310:50:493
171497799222,7cyclictest0-21swapper/310:04:233
171497799222,6cyclictest0-21swapper/309:31:003
171497799222,5cyclictest0-21swapper/312:30:473
171497799222,5cyclictest0-21swapper/309:50:583
171497799221,7cyclictest0-21swapper/310:06:043
171497799221,6cyclictest0-21swapper/312:01:023
171497799221,6cyclictest0-21swapper/309:35:463
171497799221,4cyclictest0-21swapper/312:11:033
1714975992216,3cyclictest0-21swapper/108:25:551
1714975992216,3cyclictest0-21swapper/108:25:551
171497799213,6cyclictest0-21swapper/311:45:463
171497799212,6cyclictest0-21swapper/312:20:573
171497799212,6cyclictest0-21swapper/312:20:573
171497799212,6cyclictest0-21swapper/309:15:563
171497799212,5cyclictest0-21swapper/310:35:523
171497799212,4cyclictest0-21swapper/312:05:543
171497799212,4cyclictest0-21swapper/309:55:563
171497799211,6cyclictest0-21swapper/312:16:003
171497799211,6cyclictest0-21swapper/310:16:023
171497799211,6cyclictest0-21swapper/309:41:033
171497799211,5cyclictest0-21swapper/309:20:553
171497799211,5cyclictest0-21swapper/309:20:553
171497799211,5cyclictest0-21swapper/307:55:553
171497799202,5cyclictest0-21swapper/310:55:543
171497799202,4cyclictest0-21swapper/312:00:013
171497799201,5cyclictest0-21swapper/312:40:013
171497799201,5cyclictest0-21swapper/311:35:583
171497799201,5cyclictest0-21swapper/310:41:083
171497799201,5cyclictest0-21swapper/308:26:003
171497799201,5cyclictest0-21swapper/308:26:003
171497799201,4cyclictest0-21swapper/311:11:023
171497799201,4cyclictest0-21swapper/308:15:013
171497799192,5cyclictest0-21swapper/307:20:023
171497799191,6cyclictest0-21swapper/311:40:593
171497799191,6cyclictest0-21swapper/311:26:013
171497799191,6cyclictest0-21swapper/311:26:013
171497799191,6cyclictest0-21swapper/311:20:543
171497799191,6cyclictest0-21swapper/311:20:543
171497799191,6cyclictest0-21swapper/309:26:133
171497799191,6cyclictest0-21swapper/309:26:133
171497799191,5cyclictest0-21swapper/310:20:553
171497799191,5cyclictest0-21swapper/310:20:553
171497799191,5cyclictest0-21swapper/309:10:563
171497799191,5cyclictest0-21swapper/307:30:583
171497799191,4cyclictest0-21swapper/311:01:033
171497799191,4cyclictest0-21swapper/310:31:023
171497799184,11cyclictest0-21swapper/309:00:563
171497799183,5cyclictest0-21swapper/311:31:063
171497799182,5cyclictest0-21swapper/311:06:023
171497799181,6cyclictest0-21swapper/310:11:083
171497799181,6cyclictest0-21swapper/309:46:003
171497799173,5cyclictest0-21swapper/308:45:593
171497799171,5cyclictest0-21swapper/310:45:543
171497799171,4cyclictest0-21swapper/311:51:023
171497799171,4cyclictest0-21swapper/309:05:583
171497499177,7cyclictest421-21dbus-daemon07:10:020
171497799153,7cyclictest0-21swapper/309:00:013
171497799151,7cyclictest0-21swapper/308:01:023
171497799151,6cyclictest0-21swapper/307:40:553
1714976991510,3cyclictest36-21ksoftirqd/207:10:022
171497799142,6cyclictest0-21swapper/307:35:563
171497799141,9cyclictest0-21swapper/308:35:573
171497799131,7cyclictest0-21swapper/307:26:033
171497799131,6cyclictest0-21swapper/308:05:523
171497799115,4cyclictest0-21swapper/308:20:563
171497799112,5cyclictest0-21swapper/307:10:493
171497799104,5cyclictest0-21swapper/308:30:583
171497799102,5cyclictest0-21swapper/307:50:483
171497799101,5cyclictest0-21swapper/308:20:023
17149779991,4cyclictest0-21swapper/307:45:573
17149779991,4cyclictest0-21swapper/307:20:543
17149779981,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional