You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 14:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sun Nov 23, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1743612994635,6cyclictest1810386-21unixbench_multi22:15:253
1743612994633,10cyclictest391ktimers/300:30:143
1743612994435,5cyclictest1756567-21latency_hist19:45:023
1743612994435,5cyclictest1747761-21idleruntime-cro19:20:013
1743598994436,6cyclictest1810574-21kworker/u8:1+events_unbound22:17:540
1743612994332,8cyclictest1841968-21latency_hist23:50:013
1743607994133,4cyclictest1838366-21sendmail23:40:002
1743607994133,4cyclictest1803619-21sendmail22:00:002
1743598994134,4cyclictest1801909-21munin-plugin-st21:55:010
1743603994033,4cyclictest1793159-21sh21:30:011
1743612993929,7cyclictest1808703-21grep22:10:253
1743612993926,9cyclictest1821118-21sshd22:47:563
1743607993932,4cyclictest1766928-21apt-get20:15:012
1743612993830,5cyclictest1814486-21idleruntime-cro22:30:003
1743607993828,5cyclictest0-21swapper/222:42:562
1743603993832,4cyclictest1773616-21kworker/u8:1+events_unbound20:42:341
1743603993831,3cyclictest0-21swapper/119:40:011
1743603993830,5cyclictest1775682-21munin-plugin-st20:40:011
1743598993830,4cyclictest0-21swapper/022:25:170
1743598993830,4cyclictest0-21swapper/021:05:010
1743612993727,6cyclictest0-21swapper/322:00:003
1743607993728,4cyclictest0-21swapper/220:00:002
1743603993727,6cyclictest0-21swapper/119:20:011
1743612993626,6cyclictest1792716-21grep21:25:283
1743612993625,8cyclictest0-21swapper/300:05:013
1743607993628,5cyclictest311ktimers/222:15:132
1743607993628,5cyclictest1839942-21kworker/u8:0+events_unbound00:03:092
1743607993625,8cyclictest1782510-21sendmail21:00:002
1743603993625,6cyclictest0-21swapper/100:20:001
1743612993527,5cyclictest1828126-21latency_hist23:10:013
1743612993523,8cyclictest1775249-21grep20:35:283
1743607993527,4cyclictest0-21swapper/219:10:172
1743607993524,7cyclictest0-21swapper/200:10:012
1743598993522,11cyclictest1799909-21kworker/u8:1+events_unbound21:55:270
1743612993428,4cyclictest1797974-21kworker/u8:0+events_unbound21:51:423
1743612993424,7cyclictest391ktimers/321:15:153
1743612993421,9cyclictest1817899-21sendmail22:40:003
1743607993425,5cyclictest1796575-21sendmail21:40:012
1743607993425,4cyclictest0-21swapper/219:20:012
1743607993424,5cyclictest0-21swapper/222:07:502
1743607993423,7cyclictest1771797-21grep20:25:292
1743607993423,7cyclictest0-21swapper/200:35:262
1743603993427,3cyclictest0-21swapper/122:40:161
1743603993426,5cyclictest1761604-21cat20:00:001
1743603993425,6cyclictest0-21swapper/122:55:001
1743598993422,8cyclictest1796144-21grep21:35:290
1743612993326,4cyclictest0-21swapper/321:15:003
1743612993326,4cyclictest0-21swapper/320:05:273
1743607993324,5cyclictest0-21swapper/219:40:272
1743607993323,6cyclictest0-21swapper/222:55:252
1743603993326,3cyclictest0-21swapper/123:25:001
1743603993323,6cyclictest0-21swapper/100:25:011
1743603993322,7cyclictest0-21swapper/122:25:261
1743598993326,3cyclictest0-21swapper/022:35:280
1743598993324,4cyclictest0-21swapper/021:25:280
1743598993323,6cyclictest0-21swapper/019:20:260
1743612993223,6cyclictest0-21swapper/322:35:003
1743612993221,7cyclictest0-21swapper/300:15:253
1743607993222,5cyclictest0-21swapper/222:37:552
1743603993224,4cyclictest0-21swapper/121:20:011
1743603993224,4cyclictest0-21swapper/100:05:151
1743603993221,8cyclictest1857654-21apt-config00:35:011
1743598993225,3cyclictest0-21swapper/023:50:000
1743598993225,3cyclictest0-21swapper/020:45:270
1743598993223,6cyclictest11-21ksoftirqd/000:20:160
1743598993221,8cyclictest1782542-21latency_hist21:00:010
1743598993221,7cyclictest1829351-21grep23:10:260
1743612993125,3cyclictest0-21swapper/320:59:593
1743612993123,5cyclictest1845379-21sh00:00:013
1743612993123,4cyclictest0-21swapper/322:10:003
1743612993123,4cyclictest0-21swapper/320:15:283
1743607993124,5cyclictest1845041-21kworker/u8:2+events_unbound23:58:092
1743607993123,4cyclictest0-21swapper/221:50:162
1743607993122,4cyclictest0-21swapper/221:10:162
1743607993121,6cyclictest0-21swapper/200:30:282
1743603993123,5cyclictest1749041-21unixbench_singl19:20:261
1743603993123,3cyclictest0-21swapper/122:35:001
1743603993122,6cyclictest0-21swapper/119:45:011
1743603993121,7cyclictest1791458-21sh21:25:011
1743598993125,3cyclictest0-21swapper/019:35:000
1743598993124,4cyclictest1745585-21apt-key19:15:010
1743598993124,3cyclictest0-21swapper/020:30:000
1743598993124,3cyclictest0-21swapper/000:05:240
1743598993123,4cyclictest0-21swapper/019:15:040
1743612993023,4cyclictest0-21swapper/323:10:263
1743607993023,3cyclictest0-21swapper/222:05:002
1743607993022,6cyclictest1849192-21fschecks_count00:10:162
174360799302,25cyclictest1752856-21cron19:35:012
1743607993020,6cyclictest0-21swapper/219:35:302
1743607993020,4cyclictest0-21swapper/220:55:012
1743603993022,4cyclictest1796564-21auditctl21:40:011
1743603993022,3cyclictest0-21swapper/100:00:011
1743598993020,8cyclictest11-21ksoftirqd/022:00:130
1743598993020,7cyclictest1786121-21cron21:10:000
174361299295,6cyclictest40-21ksoftirqd/320:50:263
1743612992921,4cyclictest0-21swapper/323:05:013
1743612992915,10cyclictest1745502-21munin-node19:15:013
1743607992922,5cyclictest1749103-21kworker/u8:2+events_unbound19:51:062
1743607992920,4cyclictest0-21swapper/219:20:272
1743603992918,8cyclictest1-21systemd21:52:481
1743598992921,4cyclictest0-21swapper/020:40:010
1743598992918,8cyclictest1828130-21munin-plugin-st23:10:010
1743598992918,6cyclictest1789525-21sh21:20:000
174361299287,5cyclictest381rcuc/322:20:243
174361299284,7cyclictest40-21ksoftirqd/319:27:193
174361299284,13cyclictest40-21ksoftirqd/323:20:223
174361299282,7cyclictest40-21ksoftirqd/319:30:223
1743612992822,3cyclictest0-21swapper/322:00:163
1743612992821,4cyclictest0-21swapper/323:40:153
1743612992820,5cyclictest0-21swapper/323:50:253
1743612992814,3cyclictest131rcu_preempt21:47:473
1743607992821,3cyclictest0-21swapper/223:10:262
1743607992818,6cyclictest0-21swapper/219:50:012
1743603992820,4cyclictest0-21swapper/123:50:251
1743598992821,3cyclictest0-21swapper/023:50:240
1743598992820,4cyclictest0-21swapper/023:25:270
1743598992820,3cyclictest0-21swapper/020:40:240
1743598992816,9cyclictest0-21swapper/019:40:010
174361299278,5cyclictest40-21ksoftirqd/323:15:143
174361299276,3cyclictest0-21swapper/320:49:083
174361299276,12cyclictest40-21ksoftirqd/319:55:223
174361299274,5cyclictest40-21ksoftirqd/320:30:263
1743612992720,4cyclictest1766960-21latency_hist20:15:013
1743612992720,3cyclictest0-21swapper/321:40:243
1743612992719,4cyclictest0-21swapper/300:10:253
1743612992711,9cyclictest0-21swapper/323:30:153
1743607992721,4cyclictest1794569-21kworker/u8:1+events_unbound21:32:442
1743607992720,4cyclictest0-21swapper/221:20:212
1743607992719,4cyclictest0-21swapper/222:20:262
1743607992719,4cyclictest0-21swapper/220:20:192
1743607992716,7cyclictest0-21swapper/221:00:272
1743603992720,5cyclictest299-21systemd-logind23:19:411
1743603992713,11cyclictest1766984-21latency_hist20:15:011
1743598992721,4cyclictest1838200-21kworker/u8:2+events_unbound23:43:060
1743598992720,3cyclictest0-21swapper/023:55:280
1743598992719,5cyclictest1839942-21kworker/u8:0-events_freezable_power_00:10:220
1743598992719,4cyclictest0-21swapper/019:55:180
1743598992716,7cyclictest1811934-21grep22:20:230
1743598992713,4cyclictest0-21swapper/021:44:280
174361299269,5cyclictest40-21ksoftirqd/300:35:063
174361299267,5cyclictest40-21ksoftirqd/319:50:123
174361299266,6cyclictest40-21ksoftirqd/321:00:223
174361299266,5cyclictest40-21ksoftirqd/300:25:023
174361299266,4cyclictest40-21ksoftirqd/319:35:153
174361299265,5cyclictest40-21ksoftirqd/320:05:023
174361299264,5cyclictest40-21ksoftirqd/320:40:143
1743612992620,3cyclictest391ktimers/320:28:583
1743612992616,7cyclictest391ktimers/319:21:133
1743612992611,8cyclictest131rcu_preempt22:59:293
1743612992610,4cyclictest0-21swapper/321:10:013
1743607992620,4cyclictest1810574-21kworker/u8:1+events_unbound22:32:562
1743607992620,3cyclictest0-21swapper/221:45:012
1743607992619,4cyclictest0-21swapper/220:15:022
1743603992617,6cyclictest0-21swapper/123:10:261
1743603992616,7cyclictest1779497-21grep20:50:171
1743603992614,9cyclictest0-21swapper/122:00:121
1743598992619,4cyclictest0-21swapper/023:20:000
1743598992616,7cyclictest1838356-21cat23:40:000
174361299255,5cyclictest40-21ksoftirqd/321:40:013
174361299253,4cyclictest40-21ksoftirqd/321:20:023
174361299251,13cyclictest0-21swapper/300:06:143
1743612992510,4cyclictest131rcu_preempt23:27:403
1743607992518,5cyclictest1838200-21kworker/u8:2+events_unbound23:47:582
1743607992518,3cyclictest0-21swapper/200:20:202
1743607992516,6cyclictest1751158-21sh19:30:012
1743607992514,7cyclictest1843229-21/usr/sbin/munin23:50:232
1743603992518,5cyclictest165-21systemd-journal20:31:411
1743603992518,4cyclictest0-21swapper/122:15:261
1743603992518,4cyclictest0-21swapper/121:55:271
1743603992517,4cyclictest0-21swapper/123:28:021
1743603992517,4cyclictest0-21swapper/123:05:141
1743603992515,6cyclictest0-21swapper/100:30:001
1743603992514,8cyclictest1-21systemd23:03:001
1743603992513,8cyclictest1745436-21latency_hist19:15:011
174359899259,4cyclictest151rcuc/021:20:250
1743598992518,5cyclictest1799909-21kworker/u8:1+events_unbound21:47:470
1743598992518,4cyclictest13950irq/36-mmc119:30:010
1743598992517,4cyclictest1780780-21cron20:54:590
1743598992517,4cyclictest0-21swapper/020:30:260
1743598992516,5cyclictest0-21swapper/019:45:010
174361299249,8cyclictest131rcu_preempt20:20:183
174361299244,3cyclictest0-21swapper/322:41:263
1743612992416,5cyclictest1837344-21sed23:35:193
1743612992416,4cyclictest0-21swapper/321:30:223
1743612992412,4cyclictest131rcu_preempt22:50:143
1743607992419,3cyclictest1785067-21latency21:05:202
1743607992418,4cyclictest1765104-21kworker/u8:0+events_unbound20:35:322
1743607992418,3cyclictest0-21swapper/222:50:192
1743607992417,5cyclictest1829455-21kworker/u8:2+events_unbound23:20:102
174360799241,5cyclictest131rcu_preempt20:05:142
1743607992413,7cyclictest1776902-21grep20:40:272
1743607992411,8cyclictest0-21swapper/221:15:272
174360799241,12cyclictest0-21swapper/223:40:242
174360399244,16cyclictest1798276-21cron21:45:011
1743603992418,3cyclictest0-21swapper/122:20:171
1743603992418,3cyclictest0-21swapper/121:00:171
1743603992416,4cyclictest0-21swapper/119:32:201
1743598992417,3cyclictest0-21swapper/022:30:170
1743598992416,4cyclictest1855839-21auditctl00:30:010
1743598992416,4cyclictest0-21swapper/022:45:250
1743598992414,7cyclictest1836168-21sed23:30:250
1743598992413,8cyclictest121ktimers/020:15:160
174359899241,19cyclictest0-21swapper/022:53:030
174359899241,19cyclictest0-21swapper/021:31:540
174359899241,19cyclictest0-21swapper/020:14:190
174359899240,18cyclictest0-21swapper/021:10:230
174361299232,7cyclictest40-21ksoftirqd/319:45:033
174361299230,3cyclictest0-21swapper/300:20:573
1743607992317,4cyclictest1810574-21kworker/u8:1+events_unbound22:26:512
1743607992317,3cyclictest0-21swapper/220:35:012
174360799231,5cyclictest0-21swapper/222:11:432
1743607992315,6cyclictest311ktimers/220:00:152
174360799231,14cyclictest0-21swapper/223:25:202
174360399239,11cyclictest231ktimers/100:00:151
1743603992317,3cyclictest0-21swapper/120:20:181
1743603992317,3cyclictest0-21swapper/100:35:241
1743603992317,3cyclictest0-21swapper/100:10:211
1743603992316,5cyclictest181-21systemd-udevd20:05:421
1743603992316,5cyclictest165-21systemd-journal20:25:411
1743603992315,4cyclictest0-21swapper/119:45:261
1743603992313,6cyclictest1782514-21basename21:00:001
174360399231,11cyclictest0-21swapper/121:49:101
174360399230,4cyclictest0-21swapper/122:45:171
1743598992316,4cyclictest1859457-21munin-plugin-st00:40:000
1743598992316,4cyclictest0-21swapper/022:55:260
1743598992316,3cyclictest0-21swapper/022:05:250
1743598992315,6cyclictest1770048-21sed20:20:270
1743598992315,4cyclictest0-21swapper/019:50:170
1743598992313,7cyclictest1808383-21tr22:10:240
1743598992312,8cyclictest1832535-21expr23:20:250
1743607992215,3cyclictest0-21swapper/223:00:232
1743603992216,4cyclictest181-21systemd-udevd19:53:421
1743603992215,4cyclictest1806753-21tail22:05:251
1743603992214,5cyclictest0-21swapper/123:35:151
1743603992214,5cyclictest0-21swapper/122:15:001
1743603992210,9cyclictest215-21dbus-daemon23:48:061
1743598992216,2cyclictest0-21swapper/022:42:180
1743598992215,3cyclictest0-21swapper/020:09:250
174359899221,19cyclictest0-21swapper/020:01:240
1743598992211,7cyclictest0-21swapper/000:00:230
174360799211,5cyclictest1853924-21apt-get00:25:022
1743607992113,5cyclictest1835712-21ntpq23:30:202
1743603992116,3cyclictest0-21swapper/123:40:231
1743603992114,4cyclictest0-21swapper/121:05:261
1743603992112,6cyclictest1751155-21cron19:30:011
1743603992112,6cyclictest0-21swapper/122:35:281
1743598992114,3cyclictest0-21swapper/019:45:170
1743607992014,3cyclictest0-21swapper/223:05:162
1743607992011,6cyclictest1820036-21cut22:45:162
174360799201,14cyclictest1798284-21apt-get21:45:072
174360399209,8cyclictest1835262-21awk23:30:121
1743603992014,3cyclictest1823232-21/usr/sbin/munin22:55:231
1743607991912,5cyclictest311ktimers/221:25:142
1743603991913,3cyclictest0-21swapper/120:45:141
1743603991913,3cyclictest0-21swapper/120:15:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional