You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 04:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Jun 14, 2026 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1720113996035,21cyclictest1885045-21grep00:35:012
1720111995939,14cyclictest12650irq/32-eth023:00:280
1720113995725,29cyclictest1748612-21sh20:45:012
172011299557,34cyclictest1835911-21ssh23:15:301
1720113995342,8cyclictest1797210-21ssh22:15:022
172011199521,45cyclictest0-21swapper/019:20:250
1720113995026,5cyclictest1888326-21latency_hist00:40:022
172011199502,41cyclictest1867352-21ntpq00:05:320
172011199491,44cyclictest0-21swapper/022:20:330
1720113994838,6cyclictest1831683-21latency_hist23:10:022
172011399482,24cyclictest4189336-21cupsd21:00:282
172011399481,19cyclictest1810763-21sh22:35:282
1720111994839,6cyclictest1739603-21systemd-run20:15:010
1720113994738,6cyclictest1730615-21systemd-run19:45:012
1720111994731,14cyclictest1839182-21wc23:20:370
1720113994635,4cyclictest0-21swapper/221:35:302
172011199464,36cyclictest1740547-21ntpq20:15:310
1720111994637,6cyclictest265-21systemd-journal22:01:270
1720113994538,4cyclictest1841155-21latency_hist23:25:022
1720113994534,8cyclictest1798479-21mailstats22:15:352
172011399453,21cyclictest1726992-21sshd19:30:262
172011399453,14cyclictest428-21mta-sts-daemon23:40:302
172011299452,36cyclictest1842088-21ssh23:25:261
1720112994522,12cyclictest0-21swapper/120:30:301
1720111994514,27cyclictest1817200-21ssh22:45:330
1720112994438,4cyclictest0-21swapper/122:55:241
1720112994438,4cyclictest0-21swapper/122:55:241
172011299443,36cyclictest1782413-21ssh21:50:251
1720112994427,12cyclictest1826208-21missed_timers23:00:231
1720111994436,5cyclictest12650irq/32-eth023:50:240
1720111994433,7cyclictest151ktimers/020:30:230
172011199442,36cyclictest1743525-21ntpq20:25:280
172011199441,40cyclictest1734493-21ntpq19:55:260
172011199441,40cyclictest1734493-21ntpq19:55:260
172011199441,39cyclictest1886120-21ntpq00:35:290
172011199441,38cyclictest1728449-21ntpq19:35:250
172011399433,32cyclictest421-21dbus-daemon19:20:252
1720112994335,5cyclictest1863087-21ssh00:00:021
1720112994335,5cyclictest1863087-21ssh00:00:021
1720112994332,6cyclictest1769897-21ssh21:30:281
1720112994310,6cyclictest171rcu_preempt00:15:291
172011199435,31cyclictest1863043-21munin-run00:00:010
172011199435,31cyclictest1863043-21munin-run00:00:010
172011199431,38cyclictest1722442-21ntpq19:15:230
172011199431,37cyclictest1845230-21ntpq23:30:250
1720113994234,5cyclictest1788771-21sshd22:00:262
1720112994232,5cyclictest0-21swapper/121:25:271
172011199427,32cyclictest1752437-21ntpq20:55:240
172011199421,38cyclictest1779303-21modprobe21:45:250
172011199421,37cyclictest1747968-21ntpq20:40:260
172011199421,36cyclictest1763562-21ntpq21:20:290
172011199421,35cyclictest1864217-21ntpq00:00:330
172011199420,38cyclictest1776158-21ntpq21:40:260
1720113994130,7cyclictest4189339-21cups-browsed20:30:022
172011399412,33cyclictest1776295-21ssh21:40:312
1720112994132,6cyclictest439-21systemd-logind00:40:021
172011199412,34cyclictest1813918-21ntpq22:40:270
172011199411,37cyclictest1782415-21ntpq21:50:250
172011199411,37cyclictest1769796-21ntpq21:30:250
172011199411,37cyclictest1721061-21ntpq19:10:290
172011199411,36cyclictest1811401-21ssh22:35:420
172011199411,36cyclictest1798237-21ntpq22:15:270
172011199411,36cyclictest1729953-21ntpq19:40:240
172011199411,30cyclictest1760375-21ntp_states21:15:300
1720113994031,5cyclictest0-21swapper/221:20:272
172011199402,34cyclictest1832616-21ntpq23:10:220
172011199402,34cyclictest1786858-21apt-get21:57:280
172011199402,34cyclictest1737574-21ntpq20:05:350
172011199402,34cyclictest1726976-21ntpq19:30:250
172011199402,26cyclictest1851514-21ntpq23:40:280
172011199401,36cyclictest1835778-21ntpq23:15:250
172011199401,36cyclictest1757105-21ntpq21:10:260
172011199401,36cyclictest1746548-21ntpq20:35:300
172011199401,35cyclictest1795014-21ntpq22:10:220
172011199401,35cyclictest1753944-21ntpq21:00:250
172011199401,35cyclictest1741986-21ntpq20:20:270
172011199401,34cyclictest1879859-21ntpq00:25:280
172011199401,34cyclictest1804496-21ntpq22:25:270
172011199401,34cyclictest1791954-21ntpq22:05:270
172011199401,34cyclictest1731575-21ntpq19:45:310
172011199401,28cyclictest1848426-21ntpq23:35:290
1720111994012,22cyclictest1876653-21sshd00:20:270
1720113993933,3cyclictest1856794-21kworker/u8:6+xprtiod00:00:012
1720113993933,3cyclictest1856794-21kworker/u8:6+xprtiod00:00:012
172011299393,12cyclictest1883007-21ntpq00:30:301
172011199391,35cyclictest1808607-21ssh22:31:450
172011199391,35cyclictest1754819-21munin-node21:05:150
172011199391,35cyclictest1735941-21ntpq20:00:250
172011199391,35cyclictest1735941-21ntpq20:00:250
172011199391,35cyclictest1732822-21memory19:50:230
172011199391,34cyclictest1750934-21ntpq20:50:250
172011199391,34cyclictest1749451-21ntpq20:45:230
172011199390,34cyclictest1825243-21ssh22:59:460
172011199390,34cyclictest1825243-21ssh22:59:460
1720113993831,4cyclictest1825348-21idleruntime-cro23:00:002
1720113993831,4cyclictest1825348-21idleruntime-cro23:00:002
1720113993826,8cyclictest1855133-21timerwakeupswit23:45:382
172011399382,27cyclictest1779367-21ssh21:45:272
1720112993828,5cyclictest0-21swapper/121:45:261
1720112993826,5cyclictest0-21swapper/120:45:261
172011299382,30cyclictest1795169-21ssh22:10:291
172011199381,34cyclictest1842041-21ntpq23:25:240
172011199381,34cyclictest1829630-21ntpq23:05:300
172011199381,34cyclictest1821259-21ssh22:52:140
172011199381,34cyclictest1725485-21ntpq19:25:240
172011199381,32cyclictest1870370-21ntpq00:10:320
172011199381,32cyclictest1766607-21ntpq21:25:240
172011199381,24cyclictest1873598-21ntpq00:15:320
1720113993730,3cyclictest0-21swapper/222:30:332
1720113993725,8cyclictest420-21avahi-daemon23:25:152
1720113993724,10cyclictest1834831-21latency_hist23:15:022
1720112993731,3cyclictest0-21swapper/122:00:011
1720112993730,4cyclictest1002-21mta-sts-daemon21:05:291
1720112993729,5cyclictest1850516-21idleruntime-cro23:40:011
1720112993729,5cyclictest1763378-21needreboot21:20:221
1720112993728,6cyclictest421-21dbus-daemon22:50:011
1720112993722,9cyclictest4189339-21cups-browsed20:25:261
172011199372,30cyclictest1855858-21ssh23:47:210
172011199371,33cyclictest1772537-21diskstats21:35:160
172011399363,25cyclictest1835819-21ssh23:15:262
1720113993628,5cyclictest1746108-21fschecks_time20:35:192
1720113993628,5cyclictest1737412-21needreboot20:05:272
1720113993626,6cyclictest1863635-21cpuspeed00:00:182
172011399362,26cyclictest1845269-21sshd23:30:272
1720112993629,4cyclictest1794065-21idleruntime-cro22:10:011
1720112993629,3cyclictest0-21swapper/122:50:331
1720112993625,8cyclictest694-21cron22:17:011
172011199362,31cyclictest0-21swapper/000:30:260
1720113993528,4cyclictest1003-21mta-sts-daemon00:20:022
1720113993528,3cyclictest0-21swapper/223:35:322
1720113993527,5cyclictest1731119-21fschecks_count19:45:192
1720113993527,5cyclictest1002-21mta-sts-daemon20:50:252
1720113993523,6cyclictest0-21swapper/222:50:292
1720112993528,4cyclictest1869990-21fschecks_count00:10:241
1720112993527,5cyclictest1856757-21cron23:50:011
1720112993526,6cyclictest1-21systemd23:15:011
1720112993525,7cyclictest1727989-21aten_r2power_vo19:35:161
172011299352,25cyclictest1864287-21ssh00:00:361
172011399343,27cyclictest1760248-21ssh21:15:252
172011399341,27cyclictest1757211-21ssh21:10:312
1720112993427,4cyclictest1753130-21latency_hist21:00:021
1720112993424,6cyclictest420-21avahi-daemon00:28:411
1720113993325,5cyclictest4189339-21cups-browsed21:09:402
172011299335,17cyclictest1851485-21ssh23:40:271
1720112993326,4cyclictest1869393-21systemd00:10:011
1720112993326,4cyclictest1727043-21ntp_offset19:30:281
172011299332,22cyclictest1804501-21ssh22:25:271
172011399328,19cyclictest1879825-21ssh00:25:272
1720113993224,5cyclictest1723097-21munin-plugin-st19:20:012
1720113993224,3cyclictest0-21swapper/221:55:272
1720113993223,7cyclictest1826107-21kernelversion23:00:222
1720113993223,6cyclictest1857538-21latency23:50:212
1720113993221,8cyclictest1804257-21latency22:25:212
1720113993221,8cyclictest1734601-21proc_pri19:55:312
1720113993221,8cyclictest1734601-21proc_pri19:55:312
1720113993221,7cyclictest1867113-21missed_timers00:05:242
172011299324,20cyclictest1813863-21rm22:40:251
172011299324,19cyclictest1807630-21ssh22:30:311
1720112993226,3cyclictest0-21swapper/100:25:011
1720112993225,4cyclictest1838947-21ntp_kernel_pll_23:20:271
1720112993223,5cyclictest1732962-21sshd19:50:261
1720112993222,8cyclictest1857721-21ntp_kernel_err23:50:251
1720112993219,10cyclictest1732315-21kworker/u8:9+nfsiod20:40:291
1720112993215,14cyclictest271ktimers/119:45:321
172011299321,23cyclictest1773100-21rm21:35:301
1720113993124,4cyclictest1787812-21kworker/u8:6-nfsiod22:45:292
1720113993122,4cyclictest0-21swapper/220:45:262
172011399312,22cyclictest1801337-21ssh22:20:292
1720112993124,4cyclictest1825483-21kworker/u8:9-xprtiod23:30:151
172011299312,22cyclictest1829492-21ssh23:05:241
1720112993119,9cyclictest1750951-21ntp_kernel_pll_20:50:261
172011299311,10cyclictest1757236-21ssh21:10:321
1720113993023,4cyclictest1002-21mta-sts-daemon21:50:302
172011399302,22cyclictest1769816-21ssh21:30:252
1720113993020,8cyclictest1741944-21ntp_kernel_err20:20:262
1720113993020,7cyclictest1766102-21aten2.4-expect21:25:162
1720112993024,3cyclictest0-21swapper/120:40:011
1720112993019,7cyclictest1760702-21timerandwakeup21:15:371
1720112993012,14cyclictest421-21dbus-daemon19:25:251
1720113992922,4cyclictest0-21swapper/219:25:272
1720113992922,3cyclictest0-21swapper/219:55:012
172011399292,19cyclictest1813865-21awk22:40:252
1720113992921,4cyclictest0-21swapper/219:40:012
1720113992917,8cyclictest1791909-21ntp_kernel_pll_22:05:252
172011199291,25cyclictest1-21systemd19:10:010
1720113992822,3cyclictest0-21swapper/200:25:012
1720113992820,5cyclictest1741256-21latency_hist20:20:032
1720112992821,4cyclictest1773898-21kworker/u8:0-xprtiod22:00:241
1720112992721,4cyclictest1738547-21cpuspeed20:10:171
1720112992720,4cyclictest1001-21mta-sts-daemon22:20:401
1720112992720,3cyclictest0-21swapper/121:00:281
1720112992719,5cyclictest1729401-21aten_r2power_en19:40:151
1720112992719,4cyclictest0-21swapper/122:35:241
1720113992621,2cyclictest0-21swapper/200:10:332
1720112992620,3cyclictest0-21swapper/120:20:261
172011299261,20cyclictest1776223-21ssh21:40:281
1720113992519,4cyclictest0-21swapper/220:35:022
1720113992519,3cyclictest0-21swapper/221:00:012
1720113992517,5cyclictest0-21swapper/220:15:012
1720112992517,5cyclictest1002-21mta-sts-daemon20:00:041
1720112992517,5cyclictest1002-21mta-sts-daemon20:00:041
1720112992516,7cyclictest1723100-21sendmail19:20:011
1720113992417,4cyclictest1720047-21kworker/u8:21-xprtiod19:10:362
1720112992417,4cyclictest1724602-21cron19:25:011
1720112992412,9cyclictest1734426-21ntp_kernel_err19:55:241
1720112992412,9cyclictest1734426-21ntp_kernel_err19:55:241
1720113992318,2cyclictest0-21swapper/219:10:012
1720113992316,4cyclictest1736529-21sed20:00:402
1720113992316,4cyclictest1736529-21sed20:00:402
1720112992318,2cyclictest0-21swapper/119:15:021
1720112992316,4cyclictest1003-21mta-sts-daemon20:15:151
1720112992311,9cyclictest1737443-21ntp_kernel_err20:05:281
1720112991812,3cyclictest0-21swapper/119:10:021
172011499173,8cyclictest0-21swapper/300:15:263
172011499171,9cyclictest0-21swapper/323:05:273
172011499167,7cyclictest0-21swapper/321:35:023
172011499163,7cyclictest0-21swapper/322:45:243
172011499162,8cyclictest0-21swapper/300:30:023
172011499161,8cyclictest0-21swapper/323:25:273
172011499161,8cyclictest0-21swapper/320:05:283
172011499161,6cyclictest0-21swapper/321:00:293
172011499156,5cyclictest0-21swapper/323:33:193
172011499156,5cyclictest0-21swapper/322:35:173
172011499156,5cyclictest0-21swapper/319:20:183
172011499154,6cyclictest0-21swapper/321:20:273
172011499153,7cyclictest0-21swapper/321:45:273
172011499153,4cyclictest0-21swapper/319:25:243
172011499152,9cyclictest0-21swapper/300:30:313
172011499152,7cyclictest0-21swapper/323:20:273
172011499152,7cyclictest0-21swapper/322:50:403
172011499151,7cyclictest0-21swapper/321:25:273
172011499151,6cyclictest0-21swapper/319:45:303
172011499151,5cyclictest0-21swapper/320:10:243
172011499148,4cyclictest0-21swapper/319:15:243
172011499147,5cyclictest0-21swapper/321:55:273
172011499146,5cyclictest0-21swapper/300:20:283
172011499146,4cyclictest0-21swapper/323:17:523
172011499143,7cyclictest0-21swapper/322:00:263
172011499142,8cyclictest0-21swapper/323:40:273
172011499142,6cyclictest0-21swapper/320:55:243
172011499142,5cyclictest0-21swapper/319:55:243
172011499142,5cyclictest0-21swapper/319:55:243
172011499141,8cyclictest0-21swapper/323:45:323
172011499141,8cyclictest0-21swapper/322:15:253
172011499141,7cyclictest0-21swapper/321:35:263
172011499141,6cyclictest0-21swapper/323:55:243
172011499141,6cyclictest0-21swapper/323:55:243
172011499141,6cyclictest0-21swapper/323:50:243
172011499141,6cyclictest0-21swapper/320:20:253
172011499141,6cyclictest0-21swapper/320:00:283
172011499141,6cyclictest0-21swapper/320:00:283
172011499141,6cyclictest0-21swapper/319:10:233
172011499141,5cyclictest0-21swapper/321:05:243
172011499141,4cyclictest0-21swapper/322:10:253
172011499137,4cyclictest0-21swapper/319:50:323
172011499137,4cyclictest0-21swapper/300:07:453
172011499136,4cyclictest0-21swapper/323:35:323
172011499136,4cyclictest0-21swapper/323:10:303
172011499136,4cyclictest0-21swapper/323:05:023
172011499136,4cyclictest0-21swapper/322:40:393
172011499136,4cyclictest0-21swapper/320:30:263
172011499136,4cyclictest0-21swapper/300:00:283
172011499133,5cyclictest0-21swapper/319:10:013
172011499132,6cyclictest0-21swapper/321:40:163
172011499131,7cyclictest0-21swapper/322:55:253
172011499131,7cyclictest0-21swapper/322:55:253
172011499131,6cyclictest0-21swapper/322:05:243
172011499131,6cyclictest0-21swapper/320:40:273
172011499131,6cyclictest0-21swapper/320:15:293
172011499131,6cyclictest0-21swapper/319:40:263
172011499131,5cyclictest0-21swapper/320:50:243
172011499131,5cyclictest0-21swapper/320:45:233
172011499127,3cyclictest0-21swapper/322:20:293
172011499127,3cyclictest0-21swapper/321:15:213
172011499126,4cyclictest0-21swapper/322:35:023
172011499122,7cyclictest0-21swapper/300:10:473
172011499121,7cyclictest0-21swapper/322:25:243
172011499121,7cyclictest0-21swapper/321:15:013
172011499121,6cyclictest0-21swapper/321:50:273
172011499121,6cyclictest0-21swapper/300:35:203
172011499116,3cyclictest0-21swapper/320:25:173
172011499116,3cyclictest0-21swapper/319:35:173
172011499106,3cyclictest0-21swapper/319:30:263
172011499101,5cyclictest0-21swapper/320:35:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional