You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 13:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun Dec 14, 2025 00:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17968452870,2sleep3575-21kworker/3:2-events23:54:173
15768632830,5sleep3451irq_work/320:14:063
15229832500,5chrt0-21swapper/119:19:201
1513581995041,6cyclictest278-21systemd-journal21:14:071
1513578994943,4cyclictest1733060-21kworker/u8:1+events_unbound23:09:170
1513578994828,18cyclictest1549314-21ntpq19:44:110
16619222450,2sleep3471rcuc/321:39:083
1513581994533,9cyclictest278-21systemd-journal21:29:061
1513578994516,15cyclictest0-21swapper/019:59:010
1513581994436,5cyclictest278-21systemd-journal20:39:061
1513578994427,15cyclictest1786573-21ntpq23:44:100
1513581994334,6cyclictest1727899-21latency_hist22:45:011
1513578994319,21cyclictest1653796-21latency_hist21:30:010
1513581994234,6cyclictest1559765-21kworker/u8:2+flush-8:020:29:021
1513581994233,6cyclictest1628877-21latency_hist21:05:001
1513578994227,13cyclictest1688207-21cron22:04:590
1513578994227,13cyclictest1688207-21cron22:04:590
1513578994221,19cyclictest1732934-21unixbench_singl22:49:170
1513584994137,3cyclictest1762632-21kworker/u8:0+events_unbound00:14:172
1513584994135,4cyclictest1663614-21kworker/u8:3+flush-8:022:20:362
1513584994134,5cyclictest278-21systemd-journal19:45:002
1513581994130,8cyclictest1554839-21awk19:50:001
151357899412,38cyclictest0-21swapper/021:58:560
1513578994119,19cyclictest1535975-21diskstats19:34:030
1513578994033,5cyclictest1574387-21grep20:09:190
1513578994031,7cyclictest1792117-21latency_hist23:50:010
1513584993934,4cyclictest1584392-21kworker/u8:0+flush-8:021:10:062
1513584993933,4cyclictest1703274-21kworker/u8:0+events_unbound22:35:012
1513578993923,14cyclictest1796913-21unixbench_singl23:54:170
1513578993917,21cyclictest1699579-21sshd22:19:060
1513581993828,8cyclictest1633920-21cp21:10:011
151357899382,9cyclictest1708730-21acpi22:28:580
15387192370,3sleep20-21swapper/219:34:092
1513578993722,14cyclictest1812684-21cpuspeed_turbos00:14:020
1513578993719,15cyclictest278-21systemd-journal20:14:010
1513584993631,3cyclictest1817069-21kworker/u8:3+events_unbound00:20:112
1513581993629,5cyclictest1584392-21kworker/u8:0+flush-8:021:49:111
1513578993617,17cyclictest1599057-21unixbench_singl20:34:210
1513578993615,19cyclictest1518117-21switchtime19:14:130
1513578993612,9cyclictest1531038-21diskstats19:29:020
1513584993530,4cyclictest1747558-21kworker/u8:2+flush-8:000:04:122
1513584993530,3cyclictest1733060-21kworker/u8:1+flush-8:023:59:192
1513584993530,3cyclictest1733060-21kworker/u8:1+flush-8:023:59:192
1513581993527,5cyclictest1559616-21grep19:54:191
1513581993526,6cyclictest278-21systemd-journal22:09:061
1513578993520,13cyclictest907-21mta-sts-daemon00:10:000
1513584993428,4cyclictest1644179-21kworker/u8:1+events_unbound22:00:072
1513584993428,4cyclictest1644179-21kworker/u8:1+events_unbound22:00:072
1513581993429,3cyclictest1679625-21irqrtprio21:59:041
1513581993429,3cyclictest1549936-21cron19:45:001
1513581993427,5cyclictest1644179-21kworker/u8:1+flush-8:021:39:031
1513578993422,9cyclictest278-21systemd-journal20:19:010
1513578993419,13cyclictest0-21swapper/000:24:120
1513578993418,14cyclictest0-21swapper/022:09:020
1513578993417,15cyclictest0-21swapper/021:14:180
1513578993416,16cyclictest0-21swapper/021:24:150
1513578993415,16cyclictest278-21systemd-journal23:25:000
1513578993414,18cyclictest1827581-21tr00:29:010
1513578993414,18cyclictest1747821-21turbostat.cron23:05:000
1513578993413,2cyclictest0-21swapper/019:50:010
151357899341,31cyclictest1623782-21munin-run21:00:000
15133382343,9sleep20-21swapper/219:09:272
15133382343,9sleep20-21swapper/219:09:272
1513581993323,8cyclictest1752100-21ntpq23:09:111
151357899332,10cyclictest1613289-21ntpq20:49:150
1513578993318,13cyclictest0-21swapper/021:49:070
1513578993317,14cyclictest0-21swapper/019:15:030
1513578993316,16cyclictest1757556-21unixbench_multi23:14:190
1513578993316,15cyclictest0-21swapper/000:39:190
1513578993314,2cyclictest0-21swapper/021:35:010
1513578993314,17cyclictest0-21swapper/022:24:060
1513584993228,3cyclictest1559765-21kworker/u8:2+events_unbound20:20:012
1513584993227,3cyclictest1817069-21kworker/u8:3+events_unbound00:35:062
1513581993226,4cyclictest1585655-21irqrtprio20:24:051
1513581993226,4cyclictest1530493-21kworker/u8:1+flush-8:019:59:031
1513581993225,5cyclictest1688246-21apt-get22:05:001
1513581993225,5cyclictest1688246-21apt-get22:05:001
1513581993222,6cyclictest278-21systemd-journal19:26:001
151357899323,7cyclictest1677750-21ntpq21:54:120
1513578993214,17cyclictest1713638-21acpi22:33:590
1513578993212,16cyclictest576-21runrttasks21:39:000
1513581993126,3cyclictest1703233-21idleruntime-cro22:20:001
1513581993124,5cyclictest1644179-21kworker/u8:1+events_unbound22:14:061
1513581993124,5cyclictest1530493-21kworker/u8:1+flush-8:020:04:061
151357899311,9cyclictest1737343-21ntpq22:54:110
151357899311,9cyclictest1603403-21ntpq20:39:120
1513578993117,12cyclictest0-21swapper/020:04:070
1513578993115,14cyclictest0-21swapper/023:34:010
1513578993114,16cyclictest1633181-21runrttasks21:09:130
1513578993114,15cyclictest1758593-21cut23:19:010
1513578993114,15cyclictest0-21swapper/000:34:030
1513578993112,4cyclictest1557087-21kthreadcore19:54:080
1513578993111,3cyclictest0-21swapper/023:57:060
1513578993111,3cyclictest0-21swapper/023:57:050
151357899311,11cyclictest1604620-21apt-get20:43:590
151357899310,6cyclictest1723012-21sendmail22:40:000
1513584993026,3cyclictest1559765-21kworker/u8:2+flush-8:019:59:132
1513581993025,3cyclictest1729412-21irqrtprio22:49:051
1513581993024,4cyclictest1663614-21kworker/u8:3+flush-8:022:24:101
151357899309,7cyclictest1614832-21ls20:54:040
151357899309,6cyclictest1740302-21cut22:59:080
151357899309,6cyclictest1590425-21sed20:29:040
1513578993014,14cyclictest1639550-21apt-get21:18:590
1513578993013,3cyclictest1697595-21/usr/sbin/munin22:14:080
1513578993013,2cyclictest0-21swapper/021:44:070
1513578993013,2cyclictest0-21swapper/019:39:050
1513578993013,15cyclictest0-21swapper/019:25:010
1513578993012,2cyclictest0-21swapper/023:29:100
1513578993010,6cyclictest1817788-21cat00:19:050
1513581992924,3cyclictest1735057-21kthreadcore22:54:051
1513581992920,6cyclictest1668556-21latency_hist21:45:011
1513578992923,4cyclictest1589198-21unixbench_singl20:24:190
151357899291,10cyclictest1723578-21apt-get22:44:020
1513584992824,2cyclictest1600065-21fschecks_time20:39:032
1513584992823,3cyclictest1742751-21unixbench_multi22:59:182
1513581992824,2cyclictest0-21swapper/100:35:001
151357899287,7cyclictest1627402-21cut21:04:090
151357899287,6cyclictest1805437-21cut00:04:080
1513578992811,2cyclictest0-21swapper/023:39:010
16152612270,4sleep3575-21kworker/3:2-events20:54:063
1513584992723,3cyclictest1733060-21kworker/u8:1+flush-8:023:39:172
1513581992723,3cyclictest1723003-21awk22:40:001
1513581992722,4cyclictest1613791-21unixbench_singl20:49:211
1513581992722,3cyclictest1807517-21apt-get00:08:591
1513581992722,3cyclictest1599059-21unixbench_singl20:34:211
1513581992722,3cyclictest0-21swapper/123:05:001
1513584992621,4cyclictest1747558-21kworker/u8:2+flush-8:023:29:192
1513581992621,4cyclictest1559765-21kworker/u8:2+flush-8:020:14:031
1513581992621,3cyclictest1777391-21cat23:35:001
1513581992621,3cyclictest1623869-21sort21:00:001
1513581992621,3cyclictest0-21swapper/122:59:061
1513581992617,6cyclictest1518440-21latency_hist19:15:001
1513584992522,2cyclictest1689137-21fschecks_time22:09:042
1513584992522,2cyclictest1550795-21fschecks_count19:49:042
1513584992520,3cyclictest836676-21kworker/u8:3+events_unbound19:22:192
1513581992521,3cyclictest1658555-21unixbench_multi21:34:171
1513581992521,2cyclictest0-21swapper/120:55:001
1513581992520,4cyclictest1792018-21unixbench_singl23:49:181
1513581992520,3cyclictest1822831-21irqcore00:24:051
1513581992520,3cyclictest1772191-21timerandwakeup23:29:191
1513581992520,3cyclictest0-21swapper/100:40:001
17190802240,6sleep20-21swapper/222:39:052
151358499242,10cyclictest15560022cut19:54:062
1513581992421,2cyclictest1797032-21latency_hist23:55:001
1513581992420,2cyclictest1484769-21kworker/u8:0+flush-8:019:24:041
1513581992420,2cyclictest0-21swapper/122:34:191
1513581992419,3cyclictest1827758-21fschecks_count00:29:041
1513581992419,3cyclictest1819513-21kthreadcore00:19:091
1513581992418,4cyclictest1559765-21kworker/u8:2+flush-8:020:19:041
1513581992415,6cyclictest1513400-21latency_hist19:10:011
1513581992415,6cyclictest1513400-21latency_hist19:10:001
1513581992414,7cyclictest1762588-21latency_hist23:20:011
1513581992412,6cyclictest15406672chrt19:39:031
16992592230,2sleep3481ktimers/322:19:043
151358499233,6cyclictest0-21swapper/221:47:112
1513584992320,2cyclictest1639608-21aten2_r5power_e21:19:022
1513584992320,2cyclictest1618685-21unixbench_singl20:54:192
1513584992319,3cyclictest1584392-21kworker/u8:0+flush-8:020:34:142
1513584992319,2cyclictest0-21swapper/221:00:002
1513584992318,3cyclictest1683281-21turbostat.cron22:00:002
1513584992316,5cyclictest278-21systemd-journal21:55:002
1513581992319,2cyclictest0-21swapper/119:34:111
1513581992319,2cyclictest0-21swapper/100:04:171
1513581992318,3cyclictest0-21swapper/123:59:071
1513581992318,3cyclictest0-21swapper/123:59:071
1513581992318,2cyclictest0-21swapper/123:14:071
1513581992317,3cyclictest0-21swapper/123:44:021
1513581992315,5cyclictest0-21swapper/120:44:041
151358199231,20cyclictest1767422-21unixbench_singl23:24:191
15089522233,14sleep00-21swapper/019:07:240
15089522233,14sleep00-21swapper/019:07:240
151358499227,3cyclictest17535122wc23:14:022
151358499222,14cyclictest1792675-21apt-get23:53:592
1513584992218,3cyclictest1484769-21kworker/u8:0+events_unbound19:14:102
1513584992218,3cyclictest0-21swapper/221:10:002
1513584992218,2cyclictest1694208-21cpuspeed_turbos22:14:002
1513584992218,2cyclictest1521048-21aten2_r5power_v19:19:142
1513584992217,3cyclictest1484769-21kworker/u8:0+events_unbound19:25:012
151358199229,4cyclictest0-21swapper/122:27:301
1513581992218,2cyclictest907-21mta-sts-daemon21:24:091
1513581992218,2cyclictest904-21mta-sts-daemon21:51:061
1513581992216,3cyclictest1778850-21tr23:39:071
1513581992214,6cyclictest1812423-21/usr/sbin/munin00:14:181
1513584992117,3cyclictest1484769-21kworker/u8:0+flush-8:019:39:082
1513584992117,2cyclictest1743572-21aten2_r5power_p23:04:012
1513584992016,3cyclictest1831816-21turbostat.cron00:30:002
1513584992016,2cyclictest0-21swapper/222:45:012
1513584992014,4cyclictest1584392-21kworker/u8:0+events_unbound20:49:202
151358499201,2cyclictest17022881kthreadcore22:19:092
151358499192,12cyclictest1836153-21ntpq00:34:152
151358499192,12cyclictest1766906-21ntpq23:24:112
1513584991916,2cyclictest1791966-21unixbench_multi23:49:182
151358499191,3cyclictest161rcu_preempt20:24:052
1513581991915,2cyclictest0-21swapper/121:19:131
1513581991913,4cyclictest291ktimers/120:09:021
15087922195,8sleep30-21swapper/319:05:223
15087922195,8sleep30-21swapper/319:05:223
16925912189,5sleep30-21swapper/322:09:143
16137622180,5sleep3451irq_work/320:49:213
151358499182,15cyclictest0-21swapper/220:14:022
1513584991815,2cyclictest1624813-21fschecks_count21:04:032
1513584991814,3cyclictest1733060-21kworker/u8:1+events_unbound23:40:012
1513584991814,3cyclictest1594187-21cron20:30:012
1513584991814,3cyclictest0-21swapper/223:09:062
1513584991814,2cyclictest0-21swapper/222:54:042
1513584991814,2cyclictest0-21swapper/221:24:132
1513584991812,4cyclictest0-21swapper/222:49:082
151358499174,8cyclictest1712511-21ntpq22:29:122
1513584991713,2cyclictest0-21swapper/223:34:052
1513584991712,4cyclictest1747558-21kworker/u8:2+flush-8:000:06:112
1513584991712,4cyclictest1644179-21kworker/u8:1+flush-8:021:44:312
1513584991711,4cyclictest0-21swapper/200:19:092
18173692162,9sleep30-21swapper/300:19:023
151358499167,7cyclictest1652745-21tr21:29:092
151358499163,6cyclictest0-21swapper/223:16:262
151358499163,1cyclictest0-21swapper/221:34:002
151358499162,13cyclictest0-21swapper/220:40:272
1513584991613,2cyclictest0-21swapper/221:35:502
1513584991611,3cyclictest0-21swapper/220:09:092
17877912152,8sleep30-21swapper/323:49:003
151358499152,1cyclictest0-21swapper/220:01:392
18277522142,8sleep30-21swapper/300:29:043
16239492140,2chrt451irq_work/321:00:003
18335952133,6sleep30-21swapper/300:34:103
17407022132,7sleep30-21swapper/322:59:093
17379622132,7sleep30-21swapper/322:55:003
16740292132,7sleep30-21swapper/321:54:013
15814962133,6sleep30-21swapper/320:19:093
15696522132,6sleep30-21swapper/320:05:013
18217552122,6sleep30-21swapper/300:20:013
18122622122,6sleep30-21swapper/300:11:553
18069842122,5sleep30-21swapper/300:05:013
18029892122,6sleep30-21swapper/300:04:043
18015802122,6sleep30-21swapper/323:59:193
18015802122,6sleep30-21swapper/323:59:193
17866832123,6sleep30-21swapper/323:44:153
17783192122,6sleep30-21swapper/323:39:053
17734512123,6sleep30-21swapper/323:34:033
17582522122,6sleep30-21swapper/323:18:593
17434602122,6sleep30-21swapper/323:03:583
17236222122,6sleep30-21swapper/322:44:043
17227952122,6sleep30-21swapper/322:39:193
17078142122,6sleep30-21swapper/322:24:193
16792832122,6sleep30-21swapper/321:59:033
16704582122,6sleep30-21swapper/321:49:083
16448372121,7sleep30-21swapper/321:24:043
16400582122,6sleep30-21swapper/321:19:083
16239492122,5sleep30-21swapper/321:00:023
16081132123,6sleep30-21swapper/320:44:083
15950032122,6sleep30-21swapper/320:34:053
15903432123,5sleep30-21swapper/320:29:043
15843392122,5sleep30-21swapper/320:20:013
15513892122,6sleep30-21swapper/319:49:083
15262422122,6sleep30-21swapper/319:24:043
15251152122,6sleep30-21swapper/319:19:313
18377732112,6sleep30-21swapper/300:39:073
17679102112,5sleep30-21swapper/323:26:543
17631642112,5sleep30-21swapper/323:24:003
17536872112,5sleep30-21swapper/323:14:063
17486282112,5sleep30-21swapper/323:09:043
17284832112,5sleep30-21swapper/322:46:413
17136652112,5sleep30-21swapper/322:34:013
17085302112,5sleep30-21swapper/322:26:313
16936592112,5sleep30-21swapper/322:10:063
16838642112,5sleep30-21swapper/322:04:003
16838642112,5sleep30-21swapper/322:04:003
16641242112,5sleep30-21swapper/321:43:583
16540242112,5sleep30-21swapper/321:30:073
16494452112,5sleep30-21swapper/321:29:023
16343222112,5sleep30-21swapper/321:10:363
16292372112,6sleep30-21swapper/321:06:433
15996872112,5sleep30-21swapper/320:38:593
15652612112,5sleep30-21swapper/320:04:003
15602772112,5sleep30-21swapper/319:58:593
15553942112,5sleep30-21swapper/319:54:013
15450262112,5sleep30-21swapper/319:40:023
15406352112,5sleep30-21swapper/319:39:013
15354642112,5sleep30-21swapper/319:30:583
15309452112,5sleep30-21swapper/319:29:033
15166342112,5sleep30-21swapper/319:14:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional