You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 16:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu May 21, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1092693994238,3cyclictest1196909-21kworker/u8:1+flush-259:009:14:593
1092685993935,3cyclictest211rcuc/009:15:000
1092688993835,2cyclictest1082522-21kworker/u8:1+events_unbound07:20:001
1092689993731,4cyclictest1181649-21/usr/sbin/munin08:35:142
1092689993531,3cyclictest1305964-21kworker/u8:3+flush-259:011:35:002
1092689993528,5cyclictest1123949-21latency_hist07:40:002
1092688993228,3cyclictest1378901-21kworker/u8:0+flush-259:011:50:001
1092688993228,3cyclictest1358217-21kworker/u8:2+flush-259:011:45:001
1092685993227,4cyclictest1404039-21ntpq12:05:190
1092693993128,2cyclictest1305964-21kworker/u8:3+events_unbound12:00:183
1092688993127,3cyclictest1186595-21latency_hist08:40:011
1092685993127,3cyclictest1389241-21apt-get11:55:110
1092685993126,3cyclictest1347945-21apt-get11:15:120
1092693993027,2cyclictest1207201-21kworker/u8:2+flush-259:009:29:593
1092689993026,3cyclictest1113217-21apt-get07:30:002
1092689993026,3cyclictest1113217-21apt-get07:30:002
1092689993025,4cyclictest1378674-21latency_hist11:45:002
1092685993026,3cyclictest1196909-21kworker/u8:1+flush-259:010:55:150
1092688992925,2cyclictest1108109-21kworker/u8:2+flush-259:007:45:131
1092685992924,3cyclictest509-21rs:main0
1092689992825,2cyclictest1233110-21kworker/u8:0+flush-259:009:39:592
1092688992725,1cyclictest1227934-21grep09:20:001
1092688992724,2cyclictest1233110-21kworker/u8:0+flush-259:009:45:171
1092688992723,3cyclictest1134823-21base6407:50:021
1092693992622,3cyclictest509-21rs:main3
1092693992620,4cyclictest1321883-21diskstats10:50:143
1092688992622,3cyclictest1305966-21kworker/u8:4+flush-259:011:00:111
1092688992622,3cyclictest1186860-21apt-get08:40:101
1092689992522,2cyclictest1196909-21kworker/u8:1+flush-259:010:50:002
1092689992522,2cyclictest1196909-21kworker/u8:1+flush-259:010:15:012
1092688992522,2cyclictest1357869-21kworker/u8:1+events_unbound11:25:011
1092688992522,2cyclictest1233110-21kworker/u8:0+events_unbound10:00:181
1092688992522,2cyclictest1129112-21kworker/u8:0+flush-259:008:20:101
1092688992520,4cyclictest1191999-21/usr/sbin/munin08:45:251
1092688992519,4cyclictest1279646-21latency_hist10:10:011
1092693992419,4cyclictest1336955-21apt-get11:05:003
1092689992420,3cyclictest1039568-21kworker/u8:3+events_unbound07:15:002
1092688992421,2cyclictest1347697-21systemd-run11:15:001
1092688992420,3cyclictest1363462-21apt-get11:30:101
1092688992420,2cyclictest1-21systemd10:15:011
1092688992418,4cyclictest1290447-21gpgv10:20:001
1092689992320,2cyclictest1284871-21kworker/u8:2+flush-259:010:35:002
1092689992319,3cyclictest335557-21systemd-journal09:50:302
1092689992319,3cyclictest1129112-21kworker/u8:0+flush-259:008:25:242
1092688992320,2cyclictest1196909-21kworker/u8:1+flush-259:010:40:101
1092688992319,3cyclictest1305964-21kworker/u8:3+flush-259:012:15:001
1092688992319,3cyclictest1108109-21kworker/u8:2+events_unbound07:35:111
1092688992318,3cyclictest1196909-21kworker/u8:1+events_unbound08:50:171
1092688992316,5cyclictest1305076-21ntpq10:30:211
1092685992319,3cyclictest1196909-21kworker/u8:1+flush-259:010:35:250
1092693992219,2cyclictest1378901-21kworker/u8:0+flush-259:011:55:123
1092689992219,2cyclictest1233110-21kworker/u8:0+flush-259:009:25:152
1092689992218,3cyclictest1394827-21/usr/sbin/munin12:00:202
1092689992217,3cyclictest1394158-21latency_hist12:00:002
1092685992219,2cyclictest1389596-21kworker/u8:4+flush-259:012:20:120
1092685992219,2cyclictest1305964-21kworker/u8:3+flush-259:011:10:000
1092685992218,3cyclictest1108109-21kworker/u8:2+flush-259:007:55:000
1092685992216,4cyclictest335557-21systemd-journal09:24:590
1092693992118,2cyclictest1196909-21kworker/u8:1+flush-259:009:05:223
1092689992118,2cyclictest1196909-21kworker/u8:1+flush-259:010:35:172
1092689992117,3cyclictest335557-21systemd-journal10:25:232
1092689992117,3cyclictest1196909-21kworker/u8:1+flush-259:009:00:162
1092688992118,2cyclictest1108109-21kworker/u8:2+flush-259:007:55:171
1092688992117,2cyclictest1306208-21apt-get10:35:121
1092688992116,4cyclictest1389596-21kworker/u8:4+flush-259:012:09:591
1092693992017,2cyclictest1207176-21sh09:00:003
1092693992017,2cyclictest1129112-21kworker/u8:0+events_unbound08:40:203
1092693992015,3cyclictest1113746-21base6407:30:003
1092689992017,2cyclictest1196909-21kworker/u8:1+flush-259:010:55:142
1092688992018,1cyclictest1305964-21kworker/u8:3+events_unbound10:45:111
1092688992016,3cyclictest1305964-21kworker/u8:3+events_unbound11:40:001
1092688992015,4cyclictest1181409-21latency_hist08:35:021
1092688992014,4cyclictest335557-21systemd-journal09:55:001
1092685992017,2cyclictest1108109-21kworker/u8:2+flush-259:007:30:130
1092685992016,3cyclictest335557-21systemd-journal08:30:190
1092685992015,3cyclictest1284009-21ntpq10:10:200
1092685992014,4cyclictest1269310-21latency_hist10:00:000
1092693991916,2cyclictest1165052-21kworker/u8:2+events_unbound08:50:023
1092689991912,5cyclictest1103374-21diskstats07:20:142
1092688991915,3cyclictest1196909-21kworker/u8:1+events_unbound09:30:191
1092688991915,3cyclictest1171091-21kworker/u8:3+flush-259:009:40:211
1092688991914,3cyclictest1171091-21kworker/u8:3+flush-259:009:10:161
1092688991914,3cyclictest1108109-21kworker/u8:2+events_unbound08:00:131
1092688991913,4cyclictest1394796-21cp12:00:011
1092688991912,6cyclictest1389596-21kworker/u8:4+flush-259:012:30:261
1092685991916,2cyclictest0-21swapper/011:05:010
1092685991914,4cyclictest1377750-21ntpq11:40:200
1092685991913,4cyclictest1409220-21ntpq12:10:220
1092693991815,2cyclictest1082522-21kworker/u8:1+flush-259:007:25:163
1092693991815,2cyclictest1082522-21kworker/u8:1+flush-259:007:25:163
1092693991814,3cyclictest500-21systemd-logind08:20:003
1092693991814,3cyclictest1363183-21sh11:30:003
1092693991814,3cyclictest1233110-21kworker/u8:0+flush-259:009:50:173
1092693991814,3cyclictest1171091-21kworker/u8:3+flush-259:009:15:173
1092688991814,3cyclictest1358217-21kworker/u8:2+flush-259:011:50:221
1092688991814,3cyclictest1196909-21kworker/u8:1+flush-259:010:20:141
1092688991814,3cyclictest1129112-21kworker/u8:0+flush-259:008:15:161
1092688991813,3cyclictest1290671-21kworker/u8:0+flush-259:010:30:011
1092685991815,2cyclictest1165052-21kworker/u8:2+flush-259:008:40:000
1092685991813,4cyclictest1285115-21apt-get10:15:120
1092685991812,5cyclictest1129085-21latency_hist07:45:000
1092693991714,2cyclictest1410777-21irqcore12:15:183
1092693991712,3cyclictest1285526-21sed10:15:183
1092689991715,1cyclictest0-21swapper/208:05:002
1092689991714,2cyclictest1357869-21kworker/u8:1+flush-259:011:35:142
1092689991714,2cyclictest1305964-21kworker/u8:3+flush-259:011:15:142
1092689991712,3cyclictest1165493-21latency_hist08:20:002
1092689991711,4cyclictest1295601-21latency_hist10:25:012
1092688991714,2cyclictest1305964-21kworker/u8:3+flush-259:010:50:211
1092688991713,3cyclictest1420012-21kworker/u8:0+events_unbound12:35:161
1092688991713,3cyclictest1400012-21kworker/u8:1+events_unbound12:15:191
1092688991712,3cyclictest1196909-21kworker/u8:1+flush-259:009:20:151
1092688991712,3cyclictest1196909-21kworker/u8:1+events_unbound09:55:131
1092688991712,3cyclictest1082522-21kworker/u8:1+flush-259:007:20:121
1092685991713,3cyclictest60350irq/132-enp0s29f1:rx-009:50:140
1092685991713,3cyclictest1108109-21kworker/u8:2+events_unbound07:45:200
1092693991613,2cyclictest1196909-21kworker/u8:1+events_unbound09:55:173
1092693991612,3cyclictest1384077-21apt-get11:50:103
1092693991611,4cyclictest1378917-21/usr/sbin/munin11:45:223
1092693991611,3cyclictest1166158-21/usr/sbin/munin08:20:143
1092689991614,1cyclictest0-21swapper/210:09:592
1092689991613,2cyclictest1405314-21cpuspeed_turbos12:10:132
1092689991613,2cyclictest1384489-21irqcore11:50:152
1092689991613,2cyclictest1305964-21kworker/u8:3+flush-259:010:40:132
1092689991613,2cyclictest1150729-21irqcore08:05:172
1092689991613,2cyclictest1129112-21kworker/u8:0+flush-259:007:50:002
1092689991613,1cyclictest0-21swapper/208:35:002
1092689991611,4cyclictest335557-21systemd-journal08:56:072
1092689991611,4cyclictest1400038-21apt-get12:05:112
109268899169,5cyclictest335557-21systemd-journal09:01:091
1092688991612,3cyclictest0-21swapper/109:09:591
1092685991614,1cyclictest60350irq/132-enp0s29f1:rx-009:15:160
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-012:25:200
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-012:00:220
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-011:30:190
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-011:25:180
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-008:50:200
1092685991613,2cyclictest60350irq/132-enp0s29f1:rx-007:10:030
1092685991612,3cyclictest1342515-21kworker/u8:1+flush-259:011:10:160
1092685991612,3cyclictest1274738-21apt-get10:05:130
1092685991612,2cyclictest1322132-21irqcore10:50:170
1092685991611,4cyclictest1149127-21ntpq08:00:200
1092693991513,2cyclictest1119018-21kworker/u8:3+events_unbound08:10:143
1092693991513,1cyclictest0-21swapper/311:40:003
1092693991512,2cyclictest1108109-21kworker/u8:2+flush-259:007:40:143
1092693991512,2cyclictest1108109-21kworker/u8:2+flush-259:007:40:133
1092693991512,2cyclictest1108109-21kworker/u8:2+flush-259:007:35:203
1092693991511,3cyclictest335557-21systemd-journal11:34:543
1092693991511,3cyclictest1097790-21sh07:15:013
1092693991510,3cyclictest1284844-21latency_hist10:15:003
109269399150,14cyclictest1-21systemd09:25:003
1092689991511,3cyclictest1-21systemd12:20:002
1092689991510,3cyclictest335557-21systemd-journal08:22:092
109268899158,5cyclictest335557-21systemd-journal08:15:001
1092688991512,2cyclictest1108109-21kworker/u8:2+events_unbound08:05:111
1092688991511,3cyclictest1358217-21kworker/u8:2+events_unbound11:25:141
1092688991511,3cyclictest1305966-21kworker/u8:4+flush-259:011:05:161
1092685991514,1cyclictest60350irq/132-enp0s29f1:rx-008:18:240
1092685991513,2cyclictest60350irq/132-enp0s29f1:rx-009:43:120
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-012:15:250
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-011:45:230
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-011:23:040
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-009:30:180
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:350
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-007:26:050
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-007:26:040
1092685991513,1cyclictest60350irq/132-enp0s29f1:rx-007:15:210
1092685991512,2cyclictest60350irq/132-enp0s29f1:rx-009:09:590
1092685991512,2cyclictest1233110-21kworker/u8:0+flush-259:010:00:200
1092685991512,2cyclictest1108109-21kworker/u8:2+flush-259:007:35:140
1092685991512,2cyclictest1082522-21kworker/u8:1+flush-259:007:20:200
1092685991511,3cyclictest1233110-21kworker/u8:0+flush-259:009:45:220
109268599150,14cyclictest1-21systemd09:29:090
1092693991411,2cyclictest1233110-21kworker/u8:0+flush-259:009:35:163
1092693991410,3cyclictest509-21rs:main3
1092693991410,3cyclictest1389596-21kworker/u8:4+flush-259:012:20:163
1092689991411,2cyclictest1305964-21kworker/u8:3+events_unbound10:50:252
1092689991411,2cyclictest1289994-21idleruntime-cro10:19:592
1092689991411,2cyclictest1140153-21cpuspeed_turbos07:55:132
1092689991411,2cyclictest1138810-21needreboot07:50:192
1092689991411,2cyclictest1129112-21kworker/u8:0+events_unbound08:45:002
1092689991411,2cyclictest1049591-21kworker/u8:0+flush-259:007:30:172
1092688991411,2cyclictest1415528-21apt-get12:20:101
1092688991411,2cyclictest1233110-21kworker/u8:0+events_unbound09:35:161
1092688991410,3cyclictest1113915-21apt-get07:30:111
1092688991410,2cyclictest0-21swapper/107:30:001
1092688991410,2cyclictest0-21swapper/107:30:001
1092685991413,1cyclictest60350irq/132-enp0s29f1:rx-012:35:100
1092685991413,1cyclictest60350irq/132-enp0s29f1:rx-010:46:430
1092685991413,1cyclictest60350irq/132-enp0s29f1:rx-010:22:090
1092685991413,1cyclictest60350irq/132-enp0s29f1:rx-009:35:130
1092685991412,1cyclictest60350irq/132-enp0s29f1:rx-010:30:200
1092685991412,1cyclictest60350irq/132-enp0s29f1:rx-008:05:190
1092685991412,1cyclictest60350irq/132-enp0s29f1:rx-007:55:160
1092685991411,2cyclictest60350irq/132-enp0s29f1:rx-012:34:470
1092685991411,2cyclictest60350irq/132-enp0s29f1:rx-008:55:110
1092685991411,1cyclictest1160352-21turbostat08:15:000
109269399139,3cyclictest335557-21systemd-journal11:10:303
109269399139,3cyclictest1144909-21latency_hist08:00:013
109269399138,4cyclictest335557-21systemd-journal08:40:013
1092693991311,1cyclictest1406802-21kthreadcore12:10:183
1092693991310,2cyclictest1316763-21fschecks_count10:45:123
1092693991310,2cyclictest1171091-21kworker/u8:3+flush-259:009:40:143
1092693991310,2cyclictest1119018-21kworker/u8:3+flush-259:008:05:133
109268999139,3cyclictest1389596-21kworker/u8:4+flush-259:012:25:162
1092689991311,1cyclictest1379356-21irqcore11:45:192
1092689991310,2cyclictest1171091-21kworker/u8:3+flush-259:009:10:192
1092689991310,2cyclictest1171091-21kworker/u8:3+events_unbound09:15:222
1092688991310,2cyclictest1093152-21fschecks_time07:10:131
109268599137,5cyclictest1124219-21apt-get07:40:130
109269399129,2cyclictest1435892-21cron12:40:003
109269399129,2cyclictest1176230-21idleruntime-cro08:30:003
109269399129,2cyclictest0-21swapper/311:00:013
109269399128,3cyclictest1196909-21kworker/u8:1+flush-259:010:30:133
109269399127,4cyclictest1258948-21latency_hist09:50:013
109269399126,4cyclictest1180498-21ntpq08:30:193
109268999129,2cyclictest1358217-21kworker/u8:2+flush-259:011:25:262
109268999129,2cyclictest1305966-21kworker/u8:4+events_unbound11:05:202
109268999129,2cyclictest1165052-21kworker/u8:2+events_unbound08:45:222
109268999129,2cyclictest1102526-21unixbench_singl07:15:252
109268999128,3cyclictest1353374-21fw_conntrack11:20:152
1092689991210,1cyclictest1239549-21kthreadcore09:30:182
109268899128,2cyclictest0-21swapper/111:15:251
109268899127,3cyclictest1171091-21kworker/u8:3+flush-259:008:28:121
1092685991211,1cyclictest60350irq/132-enp0s29f1:rx-011:54:380
1092685991211,1cyclictest60350irq/132-enp0s29f1:rx-008:23:190
1092685991210,1cyclictest60350irq/132-enp0s29f1:rx-008:40:090
109269399119,1cyclictest1149072-21needreboot08:00:193
109269399118,2cyclictest1098188-21cpuspeed_turbos07:15:143
109269399117,3cyclictest1389596-21kworker/u8:4+events_unbound12:25:153
109269399117,3cyclictest1134224-21apt-get07:50:003
109269399115,4cyclictest1107246-21ntpq07:20:213
109268999118,2cyclictest1196909-21kworker/u8:1+flush-259:009:20:162
109268999118,2cyclictest0-21swapper/211:15:002
109268999117,3cyclictest1159461-21ntpq08:10:202
109268999110,3cyclictest12544342ethtool09:45:172
109268899110,4cyclictest1129085-21latency_hist07:45:011
109268899110,4cyclictest1129085-21latency_hist07:45:001
109269399108,1cyclictest0-21swapper/309:00:123
109269399107,2cyclictest1192442-21irqcore08:45:173
109268999108,1cyclictest1420375-21munin-run12:25:002
109268999108,1cyclictest0-21swapper/209:40:152
109268999108,1cyclictest0-21swapper/209:09:592
109268999107,2cyclictest0-21swapper/207:40:192
109268999107,2cyclictest0-21swapper/207:40:192
109268999106,3cyclictest1430985-21munin-node12:35:102
109268999106,3cyclictest1268389-21ntp_kernel_err09:55:192
109268999100,2cyclictest490-21dbus-daemon10:05:002
10926939997,1cyclictest1389596-21kworker/u8:4+flush-259:012:05:103
10926939984,3cyclictest1274737-21apt10:05:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional