You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 04:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Mar 28, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191699922080,2sleep375-21kworker/3:1-events23:41:283
18281402960,4chrt0-21swapper/222:11:302
17239542660,0chrt281rcuc/120:26:311
17296622640,2sleep3471rcuc/320:31:353
17296622640,2sleep3471rcuc/320:31:353
19183172600,4chrt0-21swapper/123:41:341
1646962996050,7cyclictest1646792-21latency_hist19:10:010
1646968995749,6cyclictest1585386-21kworker/u8:3+events_unbound19:21:391
17117892560,8sleep375-21kworker/3:1+mm_percpu_wq20:11:363
1646962995649,4cyclictest1797650-21latency_hist21:40:010
1646968995549,5cyclictest296449-21kworker/u8:2+events_unbound23:16:301
1646968995549,4cyclictest1847480-21kworker/u8:3+events_unbound22:46:361
1646968995549,4cyclictest1847480-21kworker/u8:3+events_unbound22:46:361
1646968995241,8cyclictest1782755-21latency_hist21:25:011
1646968995145,4cyclictest1971517-21kworker/u8:1+flush-8:000:36:441
1646968995040,7cyclictest1881845-21latency_hist23:05:011
16513502490,3chrt0-21swapper/119:11:471
1646968994943,4cyclictest1753059-21kworker/u8:1+events_unbound21:06:441
1646968994841,5cyclictest296449-21kworker/u8:2+flush-8:021:41:401
1646968994739,6cyclictest1862091-21latency_hist22:45:001
1646968994440,3cyclictest1792554-21kworker/u8:0+flush-8:022:05:311
1646968994338,3cyclictest296449-21kworker/u8:2+events_unbound20:21:331
1646968994337,5cyclictest1585386-21kworker/u8:3+flush-8:019:26:451
1646962994334,6cyclictest1827520-21latency_hist22:10:010
1646968994237,4cyclictest1697730-21kworker/u8:3+flush-8:020:46:421
1646968994137,3cyclictest296449-21kworker/u8:2+events_unbound20:41:391
1646968994135,4cyclictest1646817-21kworker/u8:1+events_unbound19:46:291
1646968994134,5cyclictest1787730-21latency_hist21:30:011
1646968994133,5cyclictest1708398-21latency_hist20:10:011
1646968994034,5cyclictest1697730-21kworker/u8:3+flush-8:020:31:401
1646968994034,5cyclictest1697730-21kworker/u8:3+flush-8:020:31:401
1646968994034,4cyclictest296449-21kworker/u8:2+flush-8:021:51:551
1646968994034,3cyclictest1966074-21kworker/u8:0+flush-8:000:31:311
1646970993929,7cyclictest280-21systemd-journal23:40:002
1646968993934,3cyclictest296449-21kworker/u8:2+events_unbound21:01:291
1646962993932,5cyclictest280-21systemd-journal21:58:030
1646962993930,7cyclictest1654385-21diskstats19:16:290
1646968993834,3cyclictest296449-21kworker/u8:2+events_unbound23:11:451
1646968993833,3cyclictest1955618-21kworker/u8:3+events_unbound00:26:291
1646968993833,3cyclictest1896895-21kworker/u8:0+flush-8:023:51:301
1646968993833,3cyclictest1697730-21kworker/u8:3+flush-8:020:36:441
1646968993832,4cyclictest296449-21kworker/u8:2+events_unbound22:31:291
1646968993832,4cyclictest1792554-21kworker/u8:0+events_unbound21:56:411
164223023825,7sleep30-21swapper/319:06:123
1646968993731,4cyclictest296449-21kworker/u8:2+flush-8:023:06:441
1646968993725,9cyclictest280-21systemd-journal22:00:301
1646962993726,9cyclictest1976481-21apt-get00:40:000
1646962993726,8cyclictest280-21systemd-journal22:43:280
16715552360,2sleep3481ktimers/319:31:353
1646968993630,4cyclictest296449-21kworker/u8:2+events_unbound21:36:461
1646968993630,4cyclictest1871950-21kworker/u8:1+flush-8:023:21:301
1646968993630,4cyclictest1646817-21kworker/u8:1+events_unbound19:55:001
1646968993630,3cyclictest1905800-21kworker/u8:2+events_unbound23:36:431
1646962993626,8cyclictest1966327-21awk00:30:000
1646968993530,3cyclictest1792554-21kworker/u8:0+events_unbound22:21:441
1646968993529,4cyclictest1697730-21kworker/u8:3+flush-8:020:01:411
1646968993529,4cyclictest1674004-21kworker/u8:0+events_unbound20:16:391
1646968993529,3cyclictest1652160-21kthreadcore19:15:021
1646968993528,5cyclictest1646817-21kworker/u8:1+events_unbound19:36:401
1646962993530,4cyclictest296449-21kworker/u8:2+flush-8:023:06:450
164696299351,32cyclictest1-21systemd23:40:000
1646968993429,4cyclictest1936797-21kworker/u8:2+flush-8:000:00:061
1646968993429,3cyclictest1792554-21kworker/u8:0+events_unbound22:16:291
1646968993428,4cyclictest1871950-21kworker/u8:1+events_unbound23:26:391
1646962993428,4cyclictest296449-21kworker/u8:2+flush-8:023:16:330
164696299340,33cyclictest1862897-21sshd22:46:290
164696299340,33cyclictest1862897-21sshd22:46:290
16467622344,9sleep20-21swapper/219:09:482
1646970993329,2cyclictest1935957-21grep23:56:462
1646970993328,4cyclictest1697730-21kworker/u8:3+flush-8:020:55:002
1646968993328,4cyclictest1847480-21kworker/u8:3+events_unbound22:51:291
1646968993328,3cyclictest280-21systemd-journal23:31:281
1646968993327,4cyclictest296449-21kworker/u8:2+events_unbound21:46:411
1646968993326,5cyclictest1936797-21kworker/u8:2+flush-8:000:16:381
1646968993325,6cyclictest1698489-21latency_hist20:00:011
1646968993324,5cyclictest0-21swapper/120:51:281
1646962993324,7cyclictest280-21systemd-journal00:30:010
1646962993320,2cyclictest421rcuog/222:21:340
1646968993227,3cyclictest1753059-21kworker/u8:1+flush-8:021:16:471
1646968993226,4cyclictest296449-21kworker/u8:2+flush-8:019:40:011
1646968993224,6cyclictest1936797-21kworker/u8:2+events_unbound00:06:271
1646962993227,4cyclictest1585386-21kworker/u8:3+events_unbound19:11:450
1646968993126,3cyclictest1872403-21aten2_r5power_v22:56:291
1646968993125,4cyclictest296449-21kworker/u8:2+flush-8:021:31:381
1646968993125,4cyclictest1674004-21kworker/u8:0+events_unbound20:11:301
1646968993124,5cyclictest1847480-21kworker/u8:3+events_unbound22:36:301
1646962993127,2cyclictest1702576-21needreboot20:01:380
1646962993125,4cyclictest1777853-21latency_hist21:20:010
1646962993123,6cyclictest1931326-21latency_hist23:55:010
17613492300,2sleep3471rcuc/321:01:363
1646970993027,2cyclictest1777566-21apt-get21:20:002
1646970993026,2cyclictest1783337-21tr21:26:272
1646970993024,4cyclictest1957073-21fschecks_time00:21:292
1646968993022,6cyclictest1936797-21kworker/u8:2+flush-8:000:21:371
1646962993027,2cyclictest1875999-21needreboot22:56:390
1646962993025,3cyclictest1833033-21cpuspeed_turbos22:16:290
1646962993023,5cyclictest1832086-21grep22:11:460
1646970992924,3cyclictest1728799-21fschecks_count20:31:302
1646970992924,3cyclictest1728799-21fschecks_count20:31:302
1646968992923,4cyclictest1792554-21kworker/u8:0+events_unbound22:11:291
1646968992922,5cyclictest1926365-21latency_hist23:50:011
1646962992925,2cyclictest0-21swapper/022:36:460
1646962992924,4cyclictest296449-21kworker/u8:2+flush-8:020:51:320
1646962992924,4cyclictest1674004-21kworker/u8:0+flush-8:020:26:300
1646962992924,4cyclictest1646817-21kworker/u8:1+events_unbound19:50:000
1646962992924,3cyclictest1821691-21needreboot22:01:370
1646962992923,4cyclictest1936797-21kworker/u8:2+flush-8:000:01:290
1646962992923,4cyclictest1674004-21kworker/u8:0+events_unbound20:15:010
1646970992824,2cyclictest1886493-21grep23:06:452
1646970992823,3cyclictest1699181-21fschecks_time20:01:312
1646970992823,3cyclictest1663543-21unixbench_multi19:21:462
1646968992823,4cyclictest296449-21kworker/u8:2+flush-8:022:26:361
1646962992825,2cyclictest1778706-21irqcore21:21:320
1646962992824,3cyclictest1753059-21kworker/u8:1+events_unbound21:11:310
1646962992823,3cyclictest1936319-21munin-run00:00:000
1646962992822,4cyclictest296449-21kworker/u8:2+flush-8:021:46:410
1646962992822,4cyclictest1708080-21grep20:06:450
1646970992723,2cyclictest1747534-21grep20:46:432
1646970992718,7cyclictest1917145-21fschecks_time23:41:302
1646970992718,7cyclictest1838051-21fschecks_time22:21:292
1646970992717,8cyclictest1742964-21/usr/sbin/munin20:45:002
1646968992721,4cyclictest1768994-21kthreadcore21:11:321
1646962992723,2cyclictest1758530-21fschecks_time21:01:290
1646962992722,4cyclictest1697730-21kworker/u8:3+flush-8:020:46:410
1646970992623,2cyclictest1952725-21kthreadcore00:16:322
1646970992622,2cyclictest0-21swapper/222:06:372
1646970992621,3cyclictest1842989-21fschecks_time22:26:302
1646970992620,4cyclictest1976520-21gpgv00:40:002
1646970992618,6cyclictest1906205-21unixbench_multi23:26:452
1646968992622,2cyclictest0-21swapper/100:00:001
1646968992621,3cyclictest1756786-21latency20:56:361
1646962992622,3cyclictest1921453-21systemd-run23:45:000
1646962992622,2cyclictest1946230-21unixbench_singl00:06:470
1646962992622,2cyclictest1693540-21cron19:55:000
1646962992622,2cyclictest0-21swapper/021:51:280
1646962992620,4cyclictest1792315-21unixbench_multi21:31:450
1646962992620,4cyclictest1674004-21kworker/u8:0+events_unbound21:09:300
1646962992616,7cyclictest1723203-21latency_hist20:25:010
1646970992521,2cyclictest1966708-21apt-key00:30:012
1646970992519,4cyclictest296449-21kworker/u8:2+flush-8:019:16:302
1646970992519,4cyclictest1807883-21apt-get21:51:252
164696299259,15cyclictest1842360-21kworker/0:0+events22:51:320
1646962992521,3cyclictest1911536-21systemd-run23:35:000
1646962992521,3cyclictest1852258-21systemd-run22:35:000
1646962992520,4cyclictest401-21dbus-daemon23:30:010
1646962992520,3cyclictest1901623-21latency_hist23:25:010
1646962992518,5cyclictest1733479-21wget20:36:270
17397882240,5sleep0201rcuc/020:41:350
1646970992420,2cyclictest1792297-21unixbench_multi21:31:452
1646970992416,6cyclictest1738003-21awk20:40:002
1646968992418,4cyclictest1936797-21kworker/u8:2+flush-8:000:11:351
1646962992420,2cyclictest1753895-21irqrtprio20:56:330
1646962992419,4cyclictest1674004-21kworker/u8:0+flush-8:020:16:300
1646962992418,4cyclictest1926019-21unixbench_multi23:46:440
1646970992319,3cyclictest1646817-21kworker/u8:1+flush-8:019:41:442
1646970992316,5cyclictest1877136-21kworker/u8:3+events_unbound23:46:372
1646968992317,4cyclictest1669194-21apt-get19:31:251
1646962992319,3cyclictest1961378-21systemd-run00:25:000
1646962992319,2cyclictest1694073-21cpuspeed19:56:280
1646962992318,3cyclictest1798243-21tr21:41:280
1646962992318,3cyclictest0-21swapper/019:36:470
1646962992316,5cyclictest589-21runrttasks00:16:250
1646970992217,4cyclictest1930973-21sed23:51:452
1646970992217,4cyclictest1673763-21cron19:35:002
164696299222,19cyclictest0-21swapper/021:26:430
1646962992220,1cyclictest0-21swapper/000:11:450
1646962992218,3cyclictest1697730-21kworker/u8:3+flush-8:020:31:440
1646962992218,3cyclictest1697730-21kworker/u8:3+flush-8:020:31:440
1646962992217,4cyclictest1871950-21kworker/u8:1+flush-8:023:01:420
1646962992216,5cyclictest1877136-21kworker/u8:3+events_unbound23:11:310
16422182226,10sleep10-21swapper/119:06:021
1646970992117,3cyclictest1857166-21apt22:40:002
1646970992117,3cyclictest1818271-21fschecks_count22:01:292
1646970992117,2cyclictest1753210-21aten2_r5power_c20:56:272
1646970992116,3cyclictest1-21systemd19:55:002
1646970992112,6cyclictest1669011-21sed19:30:012
164697099211,18cyclictest1891697-21cron23:15:002
164696299211,3cyclictest407-21mta-sts-daemon19:41:450
1646962992112,7cyclictest1846409-21ntpq22:26:390
164697099202,13cyclictest1758179-21apt-get21:01:272
164697099202,13cyclictest1723554-21apt-get20:26:272
164697099202,12cyclictest1802948-21apt-get21:46:252
1646970992017,2cyclictest1862470-21aten2_r5power_c22:46:242
1646970992017,2cyclictest1862470-21aten2_r5power_c22:46:242
1646970992017,2cyclictest1674352-21cpuspeed_turbos19:36:302
1646970992016,3cyclictest1895728-21tail23:16:382
1646970992015,4cyclictest1792554-21kworker/u8:0+flush-8:022:16:432
1646970992015,4cyclictest1651511-21unixbench_multi19:11:472
1646970992011,7cyclictest1938749-21cut00:01:342
1646970992011,7cyclictest1767777-21date21:10:012
164696299201,9cyclictest1658745-21apt-get19:20:010
1646962992016,2cyclictest0-21swapper/019:26:420
1646962992014,4cyclictest0-21swapper/019:31:330
1646970991915,2cyclictest0-21swapper/221:56:292
1646970991914,3cyclictest1876394-21xargs22:56:462
1646970991914,3cyclictest1772626-21turbostat.cron21:15:002
1646970991914,3cyclictest0-21swapper/222:55:002
1646970991912,5cyclictest0-21swapper/223:01:272
1646970991911,6cyclictest1897192-21ls23:21:282
1646970991911,6cyclictest1849284-21cut22:31:342
1646970991910,7cyclictest1909735-21tr23:31:342
1646970991910,7cyclictest1696839-21tr19:56:352
164697099189,7cyclictest1949628-21cut00:11:372
164697099189,7cyclictest1800009-21cut21:41:342
164697099183,8cyclictest0-21swapper/220:21:362
1646970991814,2cyclictest0-21swapper/221:21:272
1646970991814,2cyclictest0-21swapper/220:11:282
1646970991813,3cyclictest0-21swapper/220:06:342
1646970991810,6cyclictest1858821-21cut22:41:332
1646970991810,6cyclictest1795896-21tr21:36:362
1646970991712,3cyclictest0-21swapper/200:31:362
1646970991712,3cyclictest0-21swapper/200:06:472
18078572162,9sleep30-21swapper/321:51:263
164697099162,13cyclictest0-21swapper/220:15:452
1646970991613,2cyclictest1684332-21fschecks_count19:46:302
19663392142,6sleep30-21swapper/300:30:013
18871352146,5sleep30-21swapper/323:11:283
19090232133,6sleep30-21swapper/323:31:343
18958402132,7sleep30-21swapper/323:16:413
18818582132,6sleep30-21swapper/323:05:023
18672752132,7sleep30-21swapper/322:50:293
18522192132,7sleep30-21swapper/322:35:003
17977832132,6sleep30-21swapper/321:40:023
17668592133,6sleep30-21swapper/321:06:393
16829502132,7sleep30-21swapper/319:41:433
16737892132,6sleep30-21swapper/319:35:013
16587262132,7sleep30-21swapper/319:20:003
19764642121,7sleep30-21swapper/300:40:003
19621532122,6sleep30-21swapper/300:26:323
19567602122,6sleep30-21swapper/300:21:263
19370872122,6sleep30-21swapper/300:01:283
19314662122,5sleep30-21swapper/323:55:023
19020662122,6sleep30-21swapper/323:26:283
18572822122,5sleep30-21swapper/322:40:013
18378242122,6sleep30-21swapper/322:21:273
18324742122,5sleep30-21swapper/322:15:013
18316322123,5sleep30-21swapper/322:11:403
18139382122,6sleep30-21swapper/321:56:353
17953502123,6sleep30-21swapper/321:36:373
17782612123,6sleep30-21swapper/321:21:283
17726552122,5sleep30-21swapper/321:15:023
17687952122,6sleep30-21swapper/321:11:333
17520172122,6sleep30-21swapper/320:51:423
17392432122,6sleep30-21swapper/320:41:353
17372742123,5sleep30-21swapper/320:36:453
17235282122,6sleep30-21swapper/320:26:263
17185922122,6sleep30-21swapper/320:21:243
17182292122,7sleep30-21swapper/320:20:003
17038242122,6sleep30-21swapper/320:06:273
16985322122,5sleep30-21swapper/320:00:013
16649732122,6sleep30-21swapper/319:26:343
16472202122,6sleep30-21swapper/319:11:303
19516942112,6sleep30-21swapper/300:15:243
19475792112,5sleep30-21swapper/300:11:353
19457072112,5sleep30-21swapper/300:06:423
19307452112,5sleep30-21swapper/323:51:453
19222582112,6sleep30-21swapper/323:46:313
19122222112,6sleep30-21swapper/323:36:323
18968892112,5sleep30-21swapper/323:20:023
18772212112,6sleep30-21swapper/323:01:123
18721852112,6sleep30-21swapper/322:55:263
18626922112,5sleep30-21swapper/322:46:273
18626922112,5sleep30-21swapper/322:46:273
18525952112,5sleep30-21swapper/322:36:283
18464552112,5sleep30-21swapper/322:26:413
18228062112,5sleep30-21swapper/322:05:023
18215392112,5sleep30-21swapper/322:01:373
18071102112,5sleep30-21swapper/321:46:443
17880852112,5sleep30-21swapper/321:31:273
17869942112,5sleep30-21swapper/321:26:423
17535722112,5sleep30-21swapper/320:56:323
16938152112,6sleep30-21swapper/319:55:323
16888412112,5sleep30-21swapper/319:50:023
16880392112,5sleep30-21swapper/319:46:453
16592282113,5sleep30-21swapper/319:20:123
17431812102,5sleep30-21swapper/320:45:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional