You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 07:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon May 04, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1629919994436,5cyclictest1738885-21latency_hist20:55:003
1629913994441,2cyclictest1744098-21kworker/u8:3+events_unbound21:30:202
1629907994440,3cyclictest1718329-21kworker/u8:2+flush-259:020:35:000
1629913994340,2cyclictest1931016-21kworker/u8:0+flush-259:000:20:222
1629913994339,3cyclictest1692582-21kworker/u8:0+flush-259:020:25:212
1629907994338,3cyclictest1624745-21kworker/u8:2+events_unbound19:20:010
1629919994134,5cyclictest1862753-21ntpq22:50:213
1629913993933,4cyclictest1635147-21cp19:15:012
1629919993831,5cyclictest1784895-21ntpq21:35:233
1629910993828,2cyclictest1785580-21kworker/u8:0+events_unbound22:05:011
1629913993731,4cyclictest1759594-21latency_hist21:15:012
1629910993734,2cyclictest1874151-21kworker/u8:2+events_unbound23:05:211
1629913993633,2cyclictest1931016-21kworker/u8:0+flush-259:000:10:012
1629913993632,3cyclictest1796298-21kworker/u8:3+flush-259:022:10:212
1629913993631,3cyclictest252-21systemd-journal00:03:332
1629910993633,2cyclictest1785580-21kworker/u8:0+flush-259:021:50:211
1629907993629,5cyclictest1821025-21ntpq22:10:200
1629919993530,3cyclictest1656803-21apt-get19:35:133
1629913993431,2cyclictest1904846-21kworker/u8:2+flush-259:023:50:002
1629913993431,2cyclictest1744098-21kworker/u8:3+flush-259:021:20:002
1629910993427,5cyclictest1639811-21ntpq19:15:211
1629913993330,2cyclictest1692582-21kworker/u8:0+events_unbound20:30:222
1629913993330,2cyclictest1645868-21kworker/u8:3+flush-259:020:00:002
1629913993329,3cyclictest1848399-21kworker/u8:3+events_unbound22:45:002
1629910993330,2cyclictest1738755-21kworker/u8:4+flush-259:021:10:001
1629919993227,3cyclictest1962151-21apt-get00:30:113
1629910993229,2cyclictest1790973-21kworker/u8:2+flush-259:021:45:211
1629907993228,3cyclictest1894520-21kworker/u8:0+flush-259:023:35:180
1629919993126,4cyclictest509-21rs:main3
1629913993128,2cyclictest1848399-21kworker/u8:3+events_unbound22:45:242
1629910993129,1cyclictest3654797-21kworker/u8:0+flush-259:019:20:211
1629919993024,4cyclictest1661940-21/usr/sbin/munin19:40:233
1629913993027,2cyclictest1790973-21kworker/u8:2+events_unbound21:45:182
1629913993026,3cyclictest1785580-21kworker/u8:0+flush-259:022:00:212
1629913993026,3cyclictest1758688-21kworker/u8:1+events_unbound21:20:242
1629910993027,2cyclictest1738755-21kworker/u8:4+flush-259:020:55:211
1629907993026,3cyclictest3654797-21kworker/u8:0+flush-259:019:10:140
1629907993026,3cyclictest1925764-21kworker/u8:1+flush-259:000:05:250
1629907993026,3cyclictest1744098-21kworker/u8:3+events_unbound21:20:120
1629907993026,3cyclictest1708045-21kworker/u8:3+events_unbound20:40:170
1629907993022,6cyclictest1650508-21ntpq19:25:210
1629913992926,2cyclictest1682301-21kworker/u8:1+flush-259:020:20:012
1629910992926,2cyclictest1941303-21kworker/u8:3+flush-259:000:15:211
1629910992926,2cyclictest1708045-21kworker/u8:3+flush-259:020:40:201
1629910992925,3cyclictest1785580-21kworker/u8:0+events_unbound21:38:121
1629913992825,2cyclictest1848399-21kworker/u8:3+events_unbound22:50:232
1629913992825,2cyclictest1821971-21kworker/u8:1+events_unbound22:30:252
1629913992825,2cyclictest1744098-21kworker/u8:3+flush-259:021:10:212
1629913992824,3cyclictest1692582-21kworker/u8:0+flush-259:020:35:192
1629910992826,1cyclictest1931016-21kworker/u8:0+flush-259:000:30:221
1629910992825,2cyclictest1838101-21kworker/u8:2+flush-259:022:35:001
1629910992825,2cyclictest1758688-21kworker/u8:1+events_unbound21:15:151
1629910992824,3cyclictest1910255-21apt-get23:40:121
1629907992825,2cyclictest1879308-21kworker/u8:1+flush-259:023:15:170
1629907992823,4cyclictest1801254-21latency_hist21:55:010
1629919992722,3cyclictest1651636-21/usr/sbin/munin19:30:143
1629913992724,2cyclictest1888422-21kworker/u8:3+events_unbound23:50:212
1629910992724,2cyclictest1692582-21kworker/u8:0+events_unbound20:15:151
1629910992723,3cyclictest1888422-21kworker/u8:3+flush-259:023:25:001
1629910992723,3cyclictest1848399-21kworker/u8:3+events_unbound23:02:101
1629910992723,3cyclictest1827355-21kworker/u8:4+events_unbound22:21:111
1629910992723,3cyclictest1692582-21kworker/u8:0+events_unbound20:30:241
1629907992723,3cyclictest1888422-21kworker/u8:3+flush-259:023:20:220
1629907992722,3cyclictest1821971-21kworker/u8:1+events_unbound22:30:230
1629913992623,2cyclictest1931016-21kworker/u8:0+events_unbound00:33:452
1629913992623,2cyclictest1692582-21kworker/u8:0+events_unbound20:57:132
1629913992623,2cyclictest1645868-21kworker/u8:3+events_unbound19:40:232
1629913992622,3cyclictest1941303-21kworker/u8:3+events_unbound00:15:132
1629910992623,2cyclictest1682301-21kworker/u8:1+flush-259:020:20:241
1629907992621,4cyclictest1899949-21apt-get23:30:100
1629919992520,4cyclictest1941303-21kworker/u8:3+flush-259:000:10:183
1629919992520,4cyclictest1837450-21latency_hist22:30:023
1629919992518,4cyclictest509-21rs:main3
1629913992523,2cyclictest1619532-21kworker/u8:1+flush-259:019:50:232
1629913992521,3cyclictest1708045-21kworker/u8:3+events_unbound20:53:102
1629910992522,2cyclictest1925764-21kworker/u8:1+flush-259:000:05:001
1629910992522,2cyclictest1682301-21kworker/u8:1+flush-259:020:00:251
1629910992521,3cyclictest1894520-21kworker/u8:0+events_unbound23:26:111
1629910992521,3cyclictest1894520-21kworker/u8:0+events_unbound23:26:111
1629910992521,3cyclictest1888422-21kworker/u8:3+flush-259:023:55:211
1629907992522,2cyclictest1619532-21kworker/u8:1+flush-259:019:30:230
1629907992521,3cyclictest1931016-21kworker/u8:0+events_unbound00:30:250
1629907992521,3cyclictest1843280-21apt-get22:35:120
1629907992518,5cyclictest1790072-21ntpq21:40:220
1629919992419,4cyclictest1936143-21latency_hist00:05:003
1629919992419,3cyclictest1863883-21apt-get22:55:103
1629913992421,2cyclictest1888422-21kworker/u8:3+events_unbound23:55:272
1629913992421,2cyclictest1811868-21kworker/u8:2+events_unbound22:05:212
1629913992418,4cyclictest252-21systemd-journal22:18:102
1629910992423,1cyclictest1645868-21kworker/u8:3+events_unbound19:56:121
1629910992421,3cyclictest1888422-21kworker/u8:3+events_unbound23:45:221
1629910992421,2cyclictest1925764-21kworker/u8:1+flush-259:023:55:011
1629910992421,2cyclictest1821971-21kworker/u8:1+events_unbound22:37:101
1629910992421,2cyclictest1785580-21kworker/u8:0+flush-259:022:40:111
1629910992421,2cyclictest1758688-21kworker/u8:1+events_unbound21:13:121
1629910992421,2cyclictest1758688-21kworker/u8:1+events_unbound21:13:121
1629907992420,3cyclictest1931016-21kworker/u8:0+flush-259:000:35:350
1629907992417,5cyclictest1758698-21ntpq21:10:210
1629907992417,5cyclictest1758698-21ntpq21:10:210
162990799240,22cyclictest587621-21sshd00:11:200
1629919992317,4cyclictest252-21systemd-journal22:01:103
1629913992320,2cyclictest1931016-21kworker/u8:0+flush-259:000:35:172
1629913992320,2cyclictest1888422-21kworker/u8:3+flush-259:023:25:222
1629913992320,2cyclictest1888422-21kworker/u8:3+flush-259:023:25:212
1629913992320,2cyclictest1619532-21kworker/u8:1+events_unbound19:30:242
1629913992319,3cyclictest1738755-21kworker/u8:4+events_unbound21:42:122
1629910992320,2cyclictest3654797-21kworker/u8:0+events_unbound19:13:101
1629910992320,2cyclictest1672267-21apt-get19:50:111
1629910992320,2cyclictest1660767-21kworker/u8:0+flush-259:019:50:011
1629910992319,3cyclictest1888422-21kworker/u8:3+flush-259:023:30:201
1629910992319,3cyclictest1848399-21kworker/u8:3+events_unbound22:45:141
1629910992318,4cyclictest1936377-21/usr/sbin/munin00:05:241
1629907992319,3cyclictest1785580-21kworker/u8:0+flush-259:022:15:210
1629907992319,3cyclictest1769406-21kworker/u8:0+events_unbound21:25:220
1629919992216,4cyclictest1967269-21/usr/sbin/munin00:35:243
1629913992219,2cyclictest1879308-21kworker/u8:1+flush-259:023:10:202
1629913992219,2cyclictest1785580-21kworker/u8:0+flush-259:022:20:212
1629913992219,2cyclictest1785580-21kworker/u8:0+flush-259:021:50:162
1629913992219,2cyclictest1692582-21kworker/u8:0+flush-259:021:05:262
1629913992218,3cyclictest1858684-21kworker/u8:1+events_unbound22:57:342
1629913992218,3cyclictest1619532-21kworker/u8:1+events_unbound19:45:112
1629910992219,3cyclictest1931016-21kworker/u8:0+events_unbound00:23:201
1629910992219,2cyclictest1956751-21kworker/u8:3+events_unbound00:28:211
1629910992219,2cyclictest1879308-21kworker/u8:1+flush-259:023:15:161
1629910992219,2cyclictest1676882-21kworker/u8:2+events_unbound20:12:031
1629910992218,3cyclictest1941303-21kworker/u8:3+events_unbound00:12:211
1629910992218,3cyclictest1769406-21kworker/u8:0+events_unbound21:29:111
1629910992218,3cyclictest1708045-21kworker/u8:3+flush-259:020:50:251
1629907992219,2cyclictest1827355-21kworker/u8:4+events_unbound22:25:230
1629907992218,3cyclictest1848399-21kworker/u8:3+flush-259:022:52:310
1629907992218,3cyclictest1785580-21kworker/u8:0+events_unbound21:48:110
1629907992218,3cyclictest1692582-21kworker/u8:0+events_unbound20:25:160
1629907992217,4cyclictest1645431-21ntpq19:20:210
1629919992116,3cyclictest1707993-21sh20:25:003
1629919992114,5cyclictest1815946-21ntpq22:05:223
1629919992114,5cyclictest1682687-21diskstats20:00:143
1629913992117,3cyclictest1894520-21kworker/u8:0+flush-259:023:30:152
1629913992115,4cyclictest1894495-21latency_hist23:25:002
1629910992117,3cyclictest1785580-21kworker/u8:0+events_unbound22:25:121
1629907992118,2cyclictest1780243-21kworker/u8:1+flush-259:021:55:210
1629907992117,3cyclictest1848399-21kworker/u8:3+events_unbound22:49:130
1629907992117,3cyclictest1692582-21kworker/u8:0+flush-259:020:50:150
1629907992117,3cyclictest1692582-21kworker/u8:0+flush-259:020:10:250
1629907992117,3cyclictest1682301-21kworker/u8:1+events_unbound20:07:020
1629907992115,5cyclictest1852402-21ntpq22:40:210
1629919992017,2cyclictest1947127-21irqcore00:15:153
1629919992014,4cyclictest1640690-21latency_hist19:20:003
1629913992017,2cyclictest1769406-21kworker/u8:0+flush-259:021:25:232
1629913992017,2cyclictest1624745-21kworker/u8:2+flush-259:019:25:152
1629910992016,3cyclictest1796298-21kworker/u8:3+flush-259:022:10:241
1629907992017,2cyclictest1692582-21kworker/u8:0+events_unbound21:00:000
1629907992016,3cyclictest1754457-21latency_hist21:10:010
1629907992015,4cyclictest1692582-21kworker/u8:0+events_unbound20:21:120
1629919991915,3cyclictest1682301-21kworker/u8:1+flush-259:020:40:003
1629919991914,3cyclictest1894520-21kworker/u8:0+flush-259:023:40:003
1629913991916,2cyclictest1888422-21kworker/u8:3+events_unbound23:41:112
1629913991916,2cyclictest1708045-21kworker/u8:3+events_unbound20:40:232
1629913991916,2cyclictest1682301-21kworker/u8:1+flush-259:020:05:012
1629913991915,3cyclictest1785580-21kworker/u8:0+events_unbound22:25:142
1629910991916,2cyclictest1708045-21kworker/u8:3+flush-259:020:39:591
1629910991915,3cyclictest1785580-21kworker/u8:0+flush-259:021:40:241
1629907991916,2cyclictest1951304-21kworker/u8:2+events_unbound00:20:210
1629907991915,3cyclictest1931016-21kworker/u8:0+events_unbound00:00:230
1629919991815,2cyclictest2971-21cron19:15:003
1629919991815,2cyclictest1713860-21irqcore20:30:173
1629919991814,3cyclictest1676882-21kworker/u8:2+flush-259:020:05:143
1629919991813,4cyclictest1744098-21kworker/u8:3+flush-259:021:20:153
1629919991812,4cyclictest1957150-21diskstats00:25:123
1629919991812,4cyclictest1838303-21diskstats22:30:153
1629913991815,2cyclictest1733772-21kworker/u8:1+events_unbound21:04:102
1629913991814,3cyclictest1676882-21kworker/u8:2+events_unbound20:12:102
1629913991814,3cyclictest1676882-21kworker/u8:2+events_unbound20:06:152
1629913991814,2cyclictest0-21swapper/223:20:002
1629910991815,2cyclictest1879308-21kworker/u8:1+events_unbound23:10:161
1629910991814,3cyclictest1769925-21gpgv21:25:001
1629907991816,2cyclictest1744098-21kworker/u8:3+events_unbound21:30:140
1629907991815,2cyclictest1879308-21kworker/u8:1+events_unbound23:10:200
1629907991814,3cyclictest1692582-21kworker/u8:0+events_unbound21:02:120
1629907991814,3cyclictest1645868-21kworker/u8:3+events_unbound20:02:030
1629919991713,2cyclictest1905029-21latency_hist23:35:003
162991399175,6cyclictest1879083-21apt23:10:002
1629913991714,2cyclictest3654797-21kworker/u8:0+events_unbound19:20:172
1629913991714,2cyclictest1858684-21kworker/u8:1+flush-259:023:00:202
1629913991714,2cyclictest1708045-21kworker/u8:3+events_unbound20:45:172
1629913991713,3cyclictest3654797-21kworker/u8:0+events_unbound19:16:002
1629913991713,3cyclictest1744098-21kworker/u8:3+flush-259:021:35:212
1629910991713,3cyclictest1848399-21kworker/u8:3+events_unbound22:52:311
162991099171,15cyclictest1666863-21unin-run19:45:001
1629910991710,5cyclictest1805535-21ntpq21:55:211
1629907991713,3cyclictest1925764-21kworker/u8:1+events_unbound23:55:130
1629907991713,3cyclictest1645868-21kworker/u8:3+events_unbound19:40:220
1629907991712,4cyclictest1723474-21latency_hist20:40:000
1629919991613,2cyclictest0-21swapper/319:25:003
1629913991613,2cyclictest1925764-21kworker/u8:1+flush-259:000:25:232
1629913991613,2cyclictest1838101-21kworker/u8:2+events_unbound22:35:132
1629913991612,3cyclictest1805553-21kworker/u8:2+events_unbound22:00:012
1629910991613,2cyclictest1660767-21kworker/u8:0+flush-259:019:35:261
1629907991614,1cyclictest60350irq/132-enp0s29f1:rx-021:37:590
1629907991614,1cyclictest60350irq/132-enp0s29f1:rx-020:45:250
1629907991613,2cyclictest60350irq/132-enp0s29f1:rx-019:39:470
1629907991613,2cyclictest1811868-21kworker/u8:2+flush-259:022:05:210
1629907991613,2cyclictest1619532-21kworker/u8:1+flush-259:019:50:250
1629907991613,1cyclictest0-21swapper/020:20:010
162991999156,6cyclictest1915128-21apt-get23:45:013
1629919991512,2cyclictest1926523-21irqcore23:55:173
1629919991512,2cyclictest0-21swapper/321:15:003
1629919991512,2cyclictest0-21swapper/321:15:003
1629919991511,2cyclictest0-21swapper/321:20:003
1629913991511,3cyclictest1682301-21kworker/u8:1+flush-259:020:20:162
1629910991512,2cyclictest1904846-21kworker/u8:2+flush-259:023:35:251
1629910991512,2cyclictest1749275-21idleruntime-cro21:05:001
1629910991512,2cyclictest1624745-21kworker/u8:2+flush-259:019:25:221
1629910991511,3cyclictest509-21rs:main1
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-023:45:110
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-023:28:200
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-023:28:190
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:200
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-023:01:090
1629907991513,1cyclictest60350irq/132-enp0s29f1:rx-000:25:130
1629907991512,2cyclictest60350irq/132-enp0s29f1:rx-023:50:120
1629907991512,2cyclictest60350irq/132-enp0s29f1:rx-000:15:140
1629919991412,1cyclictest1941272-21idleruntime-cro00:10:003
1629919991411,2cyclictest1795181-21ntp_kernel_pll_21:45:213
1629919991410,3cyclictest1743206-21ntp_kernel_pll_20:55:213
1629913991410,3cyclictest1925764-21kworker/u8:1+events_unbound00:13:462
1629910991411,2cyclictest1785580-21kworker/u8:0+events_unbound22:15:221
1629910991410,3cyclictest1744098-21kworker/u8:3+flush-259:021:30:161
1629907991412,2cyclictest60350irq/132-enp0s29f1:rx-022:56:510
1629907991412,1cyclictest60350irq/132-enp0s29f1:rx-019:45:140
1629907991411,2cyclictest60350irq/132-enp0s29f1:rx-022:03:110
162991999137,4cyclictest252-21systemd-journal23:07:113
162991999136,5cyclictest252-21systemd-journal23:20:003
1629919991310,2cyclictest0-21swapper/322:45:003
162991099139,3cyclictest1728875-21apt-get20:45:121
162991099138,3cyclictest252-21systemd-journal20:10:011
1629910991310,3cyclictest1624745-21kworker/u8:2+events_unbound19:31:581
1629910991310,2cyclictest1682301-21kworker/u8:1+flush-259:020:25:111
1629907991311,1cyclictest60350irq/132-enp0s29f1:rx-021:15:230
162991999129,2cyclictest1853714-21cpuspeed_turbos22:45:133
162991999128,3cyclictest1884175-21cron23:15:003
162991999126,4cyclictest1817035-21apt-get22:10:113
162991999126,4cyclictest1801021-21sshd21:52:193
162991399128,3cyclictest1660767-21kworker/u8:0+flush-259:019:37:172
162991999119,1cyclictest0-21swapper/322:20:013
162991999116,3cyclictest1728850-21/usr/sbin/munin20:45:153
162991399118,1cyclictest0-21swapper/223:35:182
162991999106,2cyclictest0-21swapper/320:40:153
16299199990,8cyclictest1677801-21irqcore19:55:173
16299109997,1cyclictest0-21swapper/122:05:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional