You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Dec 23, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1504253993831,5cyclictest1737595-21latency_hist10:55:012
1504255993327,4cyclictest252-21systemd-journal09:50:003
1504253993326,5cyclictest1700132-21ntpq10:15:222
1504245993331,1cyclictest1799875-21kworker/0:0+pm11:57:120
1504255993227,3cyclictest1494191-21kworker/u8:1+flush-259:007:15:003
1504253993023,5cyclictest1711744-21diskstats10:30:122
1504253993022,5cyclictest1582093-21latency_hist08:25:012
1504255992925,3cyclictest1768951-21kworker/u8:0+flush-259:011:40:183
1504253992924,4cyclictest1810212-21kworker/u8:1+flush-259:012:05:162
1504249992825,2cyclictest1555844-21kworker/u8:1+flush-259:008:55:221
1504255992722,4cyclictest1841239-21latency_hist12:35:003
1504255992620,4cyclictest1623313-21latency_hist09:05:013
1504249992623,2cyclictest1794942-21kworker/u8:3+events_unbound12:10:181
1504249992622,3cyclictest1794942-21kworker/u8:3+events_unbound11:55:231
1504249992622,3cyclictest1727036-21kworker/u8:4+flush-259:011:10:211
1504249992622,3cyclictest1455076-21kworker/u8:3+flush-259:007:35:221
1504249992622,3cyclictest1455076-21kworker/u8:3+flush-259:007:35:221
1504249992522,2cyclictest1768951-21kworker/u8:0+flush-259:011:35:301
1504245992522,2cyclictest1460263-21kworker/u8:5+flush-259:007:25:210
1504245992520,4cyclictest1654683-21diskstats09:35:150
1504249992421,2cyclictest1566409-21kworker/u8:0+flush-259:009:20:201
1504249992420,3cyclictest1794942-21kworker/u8:3+flush-259:012:25:231
1504245992421,2cyclictest1825438-21kworker/u8:0+flush-259:012:20:150
1504255992319,3cyclictest252-21systemd-journal09:35:183
1504253992315,6cyclictest252-21systemd-journal11:39:592
1504249992321,2cyclictest1758625-21kworker/u8:1+flush-259:011:50:251
1504249992320,2cyclictest1566409-21kworker/u8:0+events_unbound09:50:141
1504249992319,3cyclictest1555844-21kworker/u8:1+flush-259:008:25:001
1504255992220,1cyclictest1804751-21fwupd11:57:103
1504255992218,3cyclictest1768951-21kworker/u8:0+flush-259:011:25:213
1504255992217,3cyclictest1664583-21latency_hist09:45:013
1504255992217,3cyclictest1498662-21kworker/u8:2+flush-259:007:45:003
1504253992215,5cyclictest252-21systemd-journal08:58:172
1504249992220,2cyclictest1607727-21kworker/u8:2+events_unbound09:15:221
1504249992218,3cyclictest1810212-21kworker/u8:1+flush-259:012:05:211
1504249992218,3cyclictest1498662-21kworker/u8:2+flush-259:007:20:251
150425399214,4cyclictest1710415-21ntpq10:25:212
1504249992117,3cyclictest1524481-21kworker/u8:0+events_unbound07:50:201
1504249992115,4cyclictest1561278-21/usr/sbin/munin08:05:151
1504245992117,3cyclictest1530062-21kworker/u8:4+events_unbound07:45:220
1504255992017,2cyclictest1711550-21kworker/u8:4+flush-259:010:35:163
1504255992016,3cyclictest1669957-21apt-get09:50:113
1504249992016,3cyclictest1654487-21kworker/u8:1+events_unbound09:35:231
1504249992016,3cyclictest1566409-21kworker/u8:0+flush-259:010:25:191
1504249992016,3cyclictest1566409-21kworker/u8:0+flush-259:008:30:191
1504249992015,3cyclictest1566418-21/usr/sbin/munin08:10:151
1504249992014,5cyclictest1663695-21ntpq09:40:201
1504245992017,2cyclictest1690308-21kworker/u8:2+flush-259:010:45:170
1504255991914,3cyclictest1726714-21kworker/u8:3+flush-259:011:00:173
150425399194,4cyclictest1674012-21ld09:50:212
1504253991914,4cyclictest1727036-21kworker/u8:4+flush-259:011:15:002
1504253991913,4cyclictest1820526-21latency_hist12:15:012
1504253991912,5cyclictest1695855-21latency_hist10:15:002
150424999196,7cyclictest1700962-21idleruntime-cro10:20:001
1504249991916,2cyclictest1810212-21kworker/u8:1+flush-259:012:30:151
1504249991916,2cyclictest1768951-21kworker/u8:0+flush-259:011:40:241
1504249991916,2cyclictest1726714-21kworker/u8:3+flush-259:011:00:001
1504249991916,2cyclictest1690308-21kworker/u8:2+flush-259:010:35:171
1504249991916,2cyclictest1455076-21kworker/u8:3+events_unbound08:00:171
1504249991915,3cyclictest1784618-21kworker/u8:4+flush-259:011:49:591
1504249991915,3cyclictest1455076-21kworker/u8:3+flush-259:008:20:001
1504249991914,4cyclictest1732485-21base6410:50:011
1504249991914,3cyclictest1841262-21kworker/u8:4+flush-259:012:35:221
1504245991917,1cyclictest1638523-21kworker/u8:3+flush-259:009:45:180
1504245991916,3cyclictest1524481-21kworker/u8:0+events_unbound07:30:200
1504245991916,2cyclictest1607727-21kworker/u8:2+flush-259:008:50:240
1504245991916,2cyclictest1530062-21kworker/u8:4+flush-259:008:00:150
1504245991915,3cyclictest1810212-21kworker/u8:1+flush-259:012:10:190
1504245991915,3cyclictest1706388-21kworker/u8:1+events_unbound10:55:200
1504245991915,3cyclictest1566409-21kworker/u8:0+flush-259:010:25:150
1504245991913,4cyclictest1690268-21latency_hist10:10:000
1504255991815,3cyclictest1758625-21kworker/u8:1+flush-259:011:45:233
1504255991815,2cyclictest1810212-21kworker/u8:1+flush-259:012:15:253
1504255991815,1cyclictest0-21swapper/310:50:013
1504255991814,3cyclictest1566409-21kworker/u8:0+flush-259:009:25:183
1504255991814,3cyclictest1455076-21kworker/u8:3+flush-259:008:45:163
1504255991813,3cyclictest252-21systemd-journal07:21:163
1504253991816,1cyclictest0-21swapper/209:10:002
1504253991814,2cyclictest1555844-21kworker/u8:1+events_unbound08:15:182
1504253991814,2cyclictest0-21swapper/208:45:002
1504253991813,3cyclictest1690295-21gpgv10:10:012
1504249991816,2cyclictest1530062-21kworker/u8:4+events_unbound07:55:221
1504249991815,3cyclictest1460263-21kworker/u8:5+events_unbound07:25:221
1504249991814,3cyclictest1711584-21apt-get10:30:111
1504249991814,3cyclictest1566409-21kworker/u8:0+flush-259:010:20:151
1504249991813,3cyclictest1721883-21apt-get10:40:101
1504249991813,3cyclictest1524481-21kworker/u8:0+flush-259:007:40:171
1504245991816,2cyclictest1706388-21kworker/u8:1+events_unbound10:50:270
1504245991815,2cyclictest1841665-21diskstats12:35:160
1504245991815,2cyclictest1669636-21kworker/u8:1+flush-259:010:00:210
1504245991814,3cyclictest1669636-21kworker/u8:1+events_unbound10:15:240
1504245991814,2cyclictest1768951-21kworker/u8:0+flush-259:011:35:200
1504255991715,1cyclictest1571467-21cp08:15:003
1504255991714,2cyclictest1607727-21kworker/u8:2+events_unbound08:55:153
1504255991713,3cyclictest1455076-21kworker/u8:3+flush-259:008:05:173
1504255991712,3cyclictest252-21systemd-journal11:31:203
150425399174,7cyclictest1603119-21df08:45:142
150425399172,4cyclictest1596731-21ld08:35:222
1504249991714,2cyclictest1494191-21kworker/u8:1+events_unbound07:10:231
1504249991713,3cyclictest1810212-21kworker/u8:1+flush-259:012:15:191
1504249991713,3cyclictest1804760-21kworker/u8:2+events_unbound12:01:311
1504249991713,3cyclictest1726714-21kworker/u8:3+events_unbound11:05:241
1504249991713,3cyclictest1690308-21kworker/u8:2+events_unbound10:45:241
1504249991713,3cyclictest1669636-21kworker/u8:1+events_unbound09:55:131
1504249991713,3cyclictest1623325-21kworker/u8:1+flush-259:009:10:521
1504249991713,3cyclictest1566409-21kworker/u8:0+flush-259:010:05:151
1504249991713,3cyclictest1524481-21kworker/u8:0+flush-259:007:45:131
1504249991711,4cyclictest252-21systemd-journal09:00:491
1504245991714,2cyclictest60350irq/132-enp0s29f1:rx-011:15:140
1504245991714,2cyclictest1726714-21kworker/u8:3+flush-259:011:10:160
1504245991714,2cyclictest1638523-21kworker/u8:3+events_unbound09:20:180
1504245991713,3cyclictest500-21systemd-logind08:30:000
1504245991713,3cyclictest1768951-21kworker/u8:0+events_unbound11:25:210
1504245991713,3cyclictest1711550-21kworker/u8:4+events_unbound10:30:230
1504245991713,3cyclictest1623325-21kworker/u8:1+events_unbound09:10:160
1504245991713,3cyclictest1607727-21kworker/u8:2+events_unbound09:00:010
1504245991713,3cyclictest1524481-21kworker/u8:0+events_unbound07:50:160
1504255991613,2cyclictest1623325-21kworker/u8:1+events_unbound09:05:243
1504255991612,3cyclictest2971-21cron10:20:003
1504255991612,3cyclictest1566409-21kworker/u8:0+flush-259:008:30:273
150425399168,4cyclictest1679799-21taskset09:57:452
150425399164,9cyclictest1753490-21latency_hist11:10:012
150425399164,3cyclictest1705304-21ntpq10:20:242
1504253991613,2cyclictest0-21swapper/208:30:002
1504253991610,4cyclictest1623325-21kworker/u8:1+flush-259:009:10:192
1504249991613,2cyclictest1498662-21kworker/u8:2+events_unbound07:15:211
1504249991613,2cyclictest1455076-21kworker/u8:3+events_unbound08:45:201
1504249991612,3cyclictest1768951-21kworker/u8:0+events_unbound11:31:201
1504249991612,3cyclictest1649097-21kworker/u8:2+events_unbound09:30:171
1504249991612,3cyclictest1638523-21kworker/u8:3+flush-259:009:45:101
1504249991612,3cyclictest1597543-21awk08:40:001
1504249991612,2cyclictest1768951-21kworker/u8:0-events_unbound11:30:001
1504249991610,4cyclictest1530035-21latency_hist07:35:011
1504245991614,1cyclictest60350irq/132-enp0s29f1:rx-010:35:240
1504245991613,3cyclictest1690308-21kworker/u8:2+events_unbound10:40:120
1504245991613,2cyclictest60350irq/132-enp0s29f1:rx-012:00:190
1504245991613,2cyclictest60350irq/132-enp0s29f1:rx-011:20:180
1504245991613,2cyclictest1810212-21kworker/u8:1+events_unbound12:30:220
1504245991613,2cyclictest1810212-21kworker/u8:1+events_unbound12:05:220
1504245991613,2cyclictest1623325-21kworker/u8:1+events_unbound09:05:270
1504245991613,2cyclictest1498662-21kworker/u8:2+flush-259:007:20:210
1504245991612,3cyclictest1758625-21kworker/u8:1+flush-259:011:45:190
1504245991612,3cyclictest1638523-21kworker/u8:3+events_unbound09:40:200
1504245991612,3cyclictest1460263-21kworker/u8:5+flush-259:007:20:000
1504245991612,2cyclictest1748325-21sh11:05:000
1504255991513,2cyclictest491ktimers/310:10:003
1504255991513,2cyclictest1530062-21kworker/u8:4+flush-259:008:02:173
1504255991512,2cyclictest1726714-21kworker/u8:3+flush-259:011:05:173
1504255991512,2cyclictest1629167-21irqcore09:10:163
1504255991512,2cyclictest1566409-21kworker/u8:0+flush-259:008:30:003
150425399158,5cyclictest1799851-21cron11:54:592
150425399158,5cyclictest1731974-21latency_hist10:50:002
1504253991512,2cyclictest1810203-21idleruntime-cro12:05:002
1504253991511,3cyclictest1774552-21fschecks_time11:30:152
1504253991511,3cyclictest1498662-21kworker/u8:2+flush-259:007:10:192
1504253991510,3cyclictest1738288-21apt-get10:55:142
150425399150,2cyclictest1566183-21awk08:10:002
1504249991511,3cyclictest1607727-21kworker/u8:2+events_unbound08:50:221
1504245991513,1cyclictest61150irq/133-enp0s29f1:tx-007:40:150
1504245991513,1cyclictest60350irq/132-enp0s29f1:rx-012:25:140
1504245991513,1cyclictest60350irq/132-enp0s29f1:rx-012:16:270
1504245991513,1cyclictest60350irq/132-enp0s29f1:rx-011:52:130
1504245991513,1cyclictest60350irq/132-enp0s29f1:rx-011:40:240
1504245991513,1cyclictest60350irq/132-enp0s29f1:rx-008:21:350
1504245991512,2cyclictest60350irq/132-enp0s29f1:rx-011:30:240
1504245991512,2cyclictest60350irq/132-enp0s29f1:rx-009:55:200
1504245991512,2cyclictest60350irq/132-enp0s29f1:rx-009:17:110
1504245991512,2cyclictest60350irq/132-enp0s29f1:rx-007:38:300
1504245991512,2cyclictest60350irq/132-enp0s29f1:rx-007:38:300
1504245991512,2cyclictest1498662-21kworker/u8:2+events_unbound07:10:170
1504255991411,2cyclictest1669636-21kworker/u8:1+events_unbound09:55:193
1504255991410,3cyclictest252-21systemd-journal12:00:193
1504255991410,3cyclictest1810212-21kworker/u8:1+flush-259:012:10:223
1504255991410,3cyclictest1669636-21kworker/u8:1+flush-259:010:15:003
1504255991410,3cyclictest1455076-21kworker/u8:3+events_unbound07:35:193
1504255991410,3cyclictest1455076-21kworker/u8:3+events_unbound07:35:193
150425399148,4cyclictest1825684-21awk12:19:592
150425399143,9cyclictest0-21swapper/208:50:152
150425399142,3cyclictest1800088-21/usr/sbin/munin11:55:232
150425399142,10cyclictest1638774-21latency_hist09:20:002
1504253991411,2cyclictest1769258-21cpuspeed_turbos11:25:132
1504253991411,1cyclictest0-21swapper/210:05:002
1504253991410,2cyclictest1725983-21ntp_offset10:40:212
150424999149,4cyclictest1764055-21apt-get11:20:121
1504249991411,2cyclictest1825438-21kworker/u8:0+events_unbound12:20:191
1504249991410,3cyclictest1726714-21kworker/u8:3+flush-259:011:15:191
1504249991410,3cyclictest1669636-21kworker/u8:1+flush-259:010:11:321
1504249991410,3cyclictest1623325-21kworker/u8:1+events_unbound09:05:191
1504245991413,1cyclictest60350irq/132-enp0s29f1:rx-010:13:490
1504245991412,1cyclictest60350irq/132-enp0s29f1:rx-011:08:330
1504245991412,1cyclictest60350irq/132-enp0s29f1:rx-010:20:230
1504245991411,2cyclictest60350irq/132-enp0s29f1:rx-008:12:030
1504245991411,2cyclictest1566409-21kworker/u8:0+flush-259:008:30:120
1504245991410,3cyclictest1566409-21kworker/u8:0+flush-259:008:35:130
150424599140,12cyclictest252-21systemd-journal09:52:180
150425599139,3cyclictest1758625-21kworker/u8:1+flush-259:011:50:223
1504255991310,2cyclictest1706388-21kworker/u8:1+flush-259:010:25:253
1504255991310,2cyclictest0-21swapper/307:35:003
150425399139,3cyclictest1833457-21kthreadcore12:25:192
150425399139,2cyclictest1550990-21cpuspeed_turbos07:55:132
150425399137,4cyclictest1841250-21cut12:35:002
150425399136,5cyclictest1758983-21grep11:15:122
150425399136,5cyclictest1623301-21awk09:05:002
150425399133,6cyclictest1528968-21memory07:30:212
150425399132,2cyclictest1545915-21df_inode07:50:122
150425399132,2cyclictest1520921-21irqstats07:25:152
1504253991310,2cyclictest1788694-21ntp_kernel_pll_11:40:232
1504253991310,2cyclictest1651658-21kthreadcore09:30:202
1504253991310,1cyclictest1825949-21apt-get12:20:102
150424999139,3cyclictest1726714-21kworker/u8:3+events_unbound11:01:131
150424999139,3cyclictest1566409-21kworker/u8:0+events_unbound09:25:171
1504245991310,2cyclictest60350irq/132-enp0s29f1:rx-009:32:580
1504245991310,2cyclictest60350irq/132-enp0s29f1:rx-009:25:210
1504245991310,2cyclictest60350irq/132-enp0s29f1:rx-008:05:160
1504245991310,2cyclictest1598031-21fschecks_time08:40:140
150425599129,2cyclictest0-21swapper/309:25:003
150425599128,3cyclictest1810212-21kworker/u8:1+flush-259:012:05:203
150425599128,2cyclictest0-21swapper/307:25:143
150425599127,4cyclictest1555844-21kworker/u8:1+events_unbound08:15:023
150425399129,1cyclictest0-21swapper/207:20:152
150425399127,3cyclictest1768922-21sh11:24:592
150425399127,3cyclictest1592404-21apt08:35:002
150425399127,3cyclictest1510063-21irqcore07:15:152
150425399126,4cyclictest1658967-21unixbench_singl09:35:262
150425399125,5cyclictest1716844-21tr10:35:132
150425399125,5cyclictest1644376-21uniq09:25:172
150425399124,6cyclictest1793024-21grep11:45:192
150425399122,2cyclictest1535631-21df07:40:122
150425399121,2cyclictest1841657-21df_inode12:35:152
150425399121,2cyclictest1743598-21df_inode11:00:152
1504253991210,0cyclictest1664840-21apt-get09:45:102
150424999127,3cyclictest1669636-21kworker/u8:1+flush-259:010:00:151
1504245991210,1cyclictest60350irq/132-enp0s29f1:rx-008:15:140
1504245991210,1cyclictest60350irq/132-enp0s29f1:rx-007:55:180
150425599118,2cyclictest1701314-21cpuspeed_turbos10:20:153
150425599118,2cyclictest1509763-21cpuspeed_turbos07:15:123
150425599118,1cyclictest0-21swapper/312:40:013
150425599117,3cyclictest1727036-21kworker/u8:4+flush-259:011:10:183
150425599117,2cyclictest252-21systemd-journal08:24:553
150425599116,3cyclictest1725789-21sshd10:40:183
150425599113,6cyclictest1711550-21kworker/u8:4+flush-259:010:35:003
150424999117,3cyclictest1566409-21kworker/u8:0+events_unbound08:43:181
150424599119,1cyclictest60350irq/132-enp0s29f1:rx-009:03:570
1504245991110,1cyclictest60350irq/132-enp0s29f1:rx-008:45:120
150425599108,1cyclictest1636518-21kthreadcore09:15:173
150425599106,3cyclictest1758625-21kworker/u8:1+flush-259:011:20:213
150424999105,3cyclictest1586413-21ntpq08:25:231
15042559996,2cyclictest0-21swapper/307:45:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional