You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 20:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Sat Nov 22, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1753129999978,17cyclictest1859436-21latency_hist12:40:010
1753132999689,5cyclictest1820698-21kworker/u8:0+events_unbound10:40:251
1753132999588,4cyclictest1737754-21kworker/u8:0+events_unbound07:35:251
1753129999583,9cyclictest1828678-21latency_hist11:05:010
1753129999583,9cyclictest1828678-21latency_hist11:05:000
1753132998878,7cyclictest1792050-21kworker/u8:1+flush-179:009:20:011
1753129998777,7cyclictest271-21systemd-journal09:00:000
1753132998477,5cyclictest1753057-21kworker/u8:3+events_unbound07:35:011
1753129998474,7cyclictest1773733-21latency_hist08:15:010
1753132997972,5cyclictest1831973-21kworker/u8:2+events_unbound11:20:251
1753132997869,7cyclictest1820698-21kworker/u8:0+flush-179:011:20:001
1753132997869,6cyclictest1819112-21kworker/u8:1+flush-179:010:35:011
1753129997842,34cyclictest723-21Xorg11:55:580
1753132997768,7cyclictest1827329-21/usr/sbin/munin11:00:251
1753132997768,7cyclictest1827329-21/usr/sbin/munin11:00:241
1753132997669,5cyclictest1783901-21kworker/u8:3+events_unbound09:10:231
1753129997670,4cyclictest1737754-21kworker/u8:0+flush-179:007:20:010
1753132997366,5cyclictest1788879-21kworker/u8:0+events_unbound09:05:001
1753132997267,3cyclictest1803206-21kworker/u8:0+events_unbound09:55:261
1753129997267,3cyclictest1831973-21kworker/u8:2+flush-179:011:25:260
1753132997164,6cyclictest1803206-21kworker/u8:0+flush-179:009:50:261
1753132997162,6cyclictest1851495-21kworker/u8:0+flush-179:012:25:151
1753132997065,3cyclictest1737754-21kworker/u8:0+events_unbound07:20:181
1753129997060,7cyclictest1801358-21apt-get09:39:590
1753129997059,8cyclictest1765788-21latency_hist07:50:010
1753132996857,7cyclictest1793404-21kworker/u8:2+flush-179:009:25:211
1753132996761,4cyclictest1840118-21kworker/u8:0+flush-179:011:45:261
1753132996760,4cyclictest1753057-21kworker/u8:3+flush-179:007:15:211
1753132996759,6cyclictest1820698-21kworker/u8:0+flush-179:010:50:281
1753132996759,6cyclictest1767605-21kworker/u8:1+flush-179:008:00:001
1753129996734,24cyclictest1768031-21fschecks_time07:55:170
1753132996656,7cyclictest1820698-21kworker/u8:0+events_unbound10:45:271
1753129996634,1cyclictest0-21swapper/012:22:590
1753132996560,4cyclictest1840118-21kworker/u8:0+flush-179:011:54:551
1753132996560,4cyclictest1840118-21kworker/u8:0+flush-179:011:54:551
1753132996555,8cyclictest1808236-21/usr/sbin/munin10:00:251
1753129996557,6cyclictest1753057-21kworker/u8:3+flush-179:007:45:000
1753132996454,8cyclictest1840118-21kworker/u8:0+flush-179:012:00:011
1753132996358,4cyclictest1849860-21kworker/u8:2+flush-179:012:30:261
1753132996355,5cyclictest1753057-21kworker/u8:3+events_unbound08:25:251
1753132996352,7cyclictest1793404-21kworker/u8:2+flush-179:009:20:211
1753132996352,7cyclictest1793404-21kworker/u8:2+flush-179:009:20:211
1753132996349,10cyclictest1773749-21apt-get08:15:001
1753132996256,4cyclictest1848011-21kworker/u8:1+flush-179:012:10:001
1753129996251,8cyclictest1851462-21latency_hist12:15:010
1753133996158,2cyclictest0-21swapper/207:14:582
1753132996154,5cyclictest1783901-21kworker/u8:3+flush-179:008:45:241
1753132996152,6cyclictest1759394-21latency_hist07:30:021
1753132996149,8cyclictest1780362-21latency_hist08:35:011
1753132996055,4cyclictest1753057-21kworker/u8:3+flush-179:008:20:001
1753132996050,7cyclictest1820698-21kworker/u8:0+flush-179:011:10:211
1753129996051,6cyclictest271-21systemd-journal09:29:420
1753132995950,7cyclictest1778258-21kworker/u8:1+events_unbound08:35:261
1753137995855,2cyclictest0-21swapper/307:14:583
1753132995851,4cyclictest1803257-21kworker/u8:2+events_unbound09:45:231
1753132995848,7cyclictest1764197-21latency_hist07:45:001
1753132995751,4cyclictest1795246-21kworker/u8:0+events_unbound09:35:001
1753132995742,6cyclictest1753057-21kworker/u8:3+events_unbound08:10:001
1753132995650,4cyclictest1856288-21kworker/u8:3+events_unbound12:35:191
1753129995645,8cyclictest1799767-21latency_hist09:35:010
1753132995550,4cyclictest1831973-21kworker/u8:2+flush-179:011:39:551
1753132995544,9cyclictest1787290-21apt-get08:55:121
1753129995550,4cyclictest1848011-21kworker/u8:1+flush-179:012:05:260
1753132995348,4cyclictest1803257-21kworker/u8:2+flush-179:010:05:231
1753132995347,4cyclictest1848011-21kworker/u8:1+events_unbound12:10:261
1753132995346,5cyclictest271-21systemd-journal12:20:011
1753129995347,4cyclictest1778258-21kworker/u8:1+flush-179:008:40:230
1753129995344,7cyclictest1778258-21kworker/u8:1+flush-179:008:50:170
1753129995342,9cyclictest1841924-21apt-get11:45:180
1753129995338,11cyclictest1780372-21apt-get08:35:000
1753132995249,2cyclictest0-21swapper/110:26:581
1753132995245,5cyclictest1851495-21kworker/u8:0+flush-179:012:25:001
1753129995245,4cyclictest1843245-21kworker/u8:1+flush-179:011:50:270
1753129995245,4cyclictest1843245-21kworker/u8:1+flush-179:011:50:260
1753132995147,3cyclictest1803206-21kworker/u8:0+events_unbound10:25:001
1753132995145,4cyclictest1828713-21kworker/u8:1+flush-179:011:40:161
1753129995142,7cyclictest1833953-21/usr/sbin/munin11:20:240
1753129995142,7cyclictest1820698-21kworker/u8:0+flush-179:011:10:170
1753137994947,1cyclictest0-21swapper/310:29:593
1753132994943,4cyclictest1753057-21kworker/u8:3+flush-179:007:10:221
1753132994940,6cyclictest271-21systemd-journal08:45:001
1753132994919,28cyclictest723-21Xorg12:04:001
1753129994933,15cyclictest0-21swapper/012:18:590
4670791480,13ptp4l441ktimers/307:10:003
1753129994838,7cyclictest1801817-21cat09:40:020
1753132994742,4cyclictest1797643-21kworker/u8:1+flush-179:009:35:271
175312999470,29cyclictest0-21swapper/008:35:040
1753129994641,3cyclictest1803206-21kworker/u8:0+flush-179:009:45:290
1753129994637,7cyclictest1803206-21kworker/u8:0+flush-179:010:30:000
1753129994630,15cyclictest0-21swapper/012:30:590
175312999460,2cyclictest723-21Xorg10:40:590
1753132994540,4cyclictest1819112-21kworker/u8:1+events_unbound10:35:271
1753132994538,5cyclictest1764232-21kworker/u8:0+flush-179:007:45:261
175312999450,42cyclictest0-21swapper/009:55:160
1753132994439,4cyclictest1753057-21kworker/u8:3+events_unbound08:25:021
1753132994438,5cyclictest1803257-21kworker/u8:2+events_unbound10:15:261
1753132994436,5cyclictest1788879-21kworker/u8:0+flush-179:009:10:001
1753129994420,21cyclictest1827070-21latency_hist11:00:020
175312999441,3cyclictest1770819-21apt-get08:05:150
175312999441,29cyclictest1831856-21apt-get11:15:030
1753129994336,6cyclictest1803257-21kworker/u8:2+flush-179:010:15:010
1753129994336,5cyclictest1737754-21kworker/u8:0+events_unbound07:10:260
1753133994238,2cyclictest0-21swapper/207:38:592
175313399420,2cyclictest0-21swapper/212:31:002
1753129994233,6cyclictest271-21systemd-journal12:28:400
175312999422,39cyclictest1819138-21turbostat10:39:560
1753137994134,5cyclictest1830276-21turbostat11:09:593
175313799410,40cyclictest1804822-21turbostat09:50:013
175313799410,40cyclictest0-21swapper/312:30:593
1753133994138,2cyclictest0-21swapper/211:37:002
1753133994137,2cyclictest0-21swapper/209:11:002
175313399410,39cyclictest0-21swapper/212:17:592
1753132994137,3cyclictest1753057-21kworker/u8:3+events_unbound08:00:201
1753129994137,2cyclictest1841705-21systemd-run11:45:000
1753129994135,5cyclictest1836902-21turbostat11:34:550
1753129994036,3cyclictest1795025-21munin-run09:20:010
175313399390,2cyclictest0-21swapper/207:48:592
1753132993934,4cyclictest1822551-21kworker/u8:2+events_unbound10:55:211
1753129993935,3cyclictest1777194-21cron08:25:000
175312999390,38cyclictest0-21swapper/012:02:000
175312999390,2cyclictest723-21Xorg09:25:000
175312999390,2cyclictest723-21Xorg09:25:000
1753137993835,2cyclictest0-21swapper/307:35:003
1753133993834,2cyclictest0-21swapper/208:36:592
175313399380,36cyclictest0-21swapper/208:58:592
175313399380,36cyclictest0-21swapper/207:34:592
1753132993832,4cyclictest1831973-21kworker/u8:2+flush-179:011:25:221
1753132993830,4cyclictest0-21swapper/107:50:591
1753129993835,2cyclictest0-21swapper/010:24:550
1753129993820,16cyclictest1829760-21phc2sys11:05:230
1753137993735,1cyclictest0-21swapper/311:37:003
175313799370,36cyclictest0-21swapper/311:33:003
175313399370,35cyclictest0-21swapper/210:56:582
175313399370,35cyclictest0-21swapper/210:53:002
175313399370,35cyclictest0-21swapper/209:47:592
175313399370,35cyclictest0-21swapper/208:27:582
175313399370,1cyclictest0-21swapper/210:16:592
1753129993716,19cyclictest0-21swapper/010:10:210
1753129993713,22cyclictest475-21dbus-daemon07:30:000
175312999370,2cyclictest0-21swapper/008:47:590
1753137993634,1cyclictest0-21swapper/309:11:003
1753137993634,1cyclictest0-21swapper/308:58:593
1753137993634,1cyclictest0-21swapper/307:38:593
1753137993633,2cyclictest0-21swapper/310:52:593
1753137993630,5cyclictest1768968-21turbostat08:00:003
175313799360,35cyclictest0-21swapper/308:01:593
175313799360,35cyclictest0-21swapper/307:40:593
1753132993632,3cyclictest1803217-21cron09:45:001
1753132993629,5cyclictest1828713-21kworker/u8:1+events_unbound11:05:201
1753129993628,6cyclictest1757809-21latency_hist07:25:010
1753129993619,16cyclictest1806245-21unixbench-2d09:50:280
1753129993618,17cyclictest0-21swapper/010:45:270
1753129993610,25cyclictest1839374-21cat11:35:200
1753137993533,1cyclictest0-21swapper/308:08:593
175313799350,1cyclictest0-21swapper/310:10:593
1753133993533,1cyclictest0-21swapper/208:00:012
1753133993531,2cyclictest0-21swapper/208:44:592
1753132993531,2cyclictest723-21Xorg08:54:591
1753129993531,3cyclictest1818655-21grep10:30:240
1753129993530,4cyclictest1791850-21systemd-run09:10:010
1753129993529,5cyclictest1765783-21turbostat07:54:560
175312999352,30cyclictest1809823-21/usr/sbin/munin10:05:220
1753129993518,16cyclictest0-21swapper/010:00:240
1753137993432,1cyclictest0-21swapper/312:35:593
1753137993432,1cyclictest0-21swapper/310:15:593
1753137993432,1cyclictest0-21swapper/309:01:593
1753133993431,2cyclictest0-21swapper/211:21:002
1753133993431,2cyclictest0-21swapper/208:48:592
1753133993430,2cyclictest0-21swapper/209:19:592
1753129993416,16cyclictest0-21swapper/010:55:000
175312999341,4cyclictest0-21swapper/007:35:140
1753129993411,22cyclictest1792127-21cat09:10:130
175312999340,32cyclictest723-21Xorg08:01:590
1753137993331,1cyclictest0-21swapper/312:19:553
1753133993329,2cyclictest0-21swapper/211:48:592
1753133993329,2cyclictest0-21swapper/207:54:592
1753129993310,21cyclictest1775400-21cat08:15:250
1753137993230,1cyclictest0-21swapper/308:36:593
175313799320,31cyclictest0-21swapper/310:10:003
1753133993228,2cyclictest0-21swapper/212:00:592
1753129993214,16cyclictest1778773-21latency_hist08:30:020
1753133993128,2cyclictest0-21swapper/211:11:582
1753133993128,2cyclictest0-21swapper/209:09:592
1753133993128,2cyclictest0-21swapper/207:43:592
1753133993127,2cyclictest0-21swapper/209:39:002
1753132993127,2cyclictest0-21swapper/111:30:131
1753129993110,19cyclictest1789845-21mailstats09:00:250
1753137993028,1cyclictest0-21swapper/311:20:593
1753137993028,1cyclictest0-21swapper/308:49:003
1753133993027,2cyclictest0-21swapper/211:51:002
1753133993027,2cyclictest0-21swapper/211:50:592
1753133993027,2cyclictest0-21swapper/207:20:182
1753133993026,2cyclictest0-21swapper/211:55:582
1753133993026,2cyclictest0-21swapper/210:37:592
175313399300,2cyclictest0-21swapper/209:00:152
175312999301,4cyclictest0-21swapper/007:30:590
1753137992927,1cyclictest0-21swapper/308:44:593
1753137992926,2cyclictest0-21swapper/310:40:143
1753137992926,2cyclictest0-21swapper/307:55:003
1753137992926,2cyclictest0-21swapper/307:45:173
175313799290,2cyclictest0-21swapper/308:31:593
175313799290,28cyclictest0-21swapper/311:50:163
175313799290,28cyclictest0-21swapper/311:50:153
175313799290,27cyclictest0-21swapper/311:25:133
175313799290,27cyclictest0-21swapper/311:00:143
175313799290,27cyclictest0-21swapper/311:00:133
175313799290,27cyclictest0-21swapper/309:05:133
1753133992926,2cyclictest0-21swapper/209:43:592
1753133992926,2cyclictest0-21swapper/208:19:002
175313399290,2cyclictest0-21swapper/210:30:162
175313399290,27cyclictest0-21swapper/211:05:152
175313399290,27cyclictest0-21swapper/211:00:142
175313399290,27cyclictest0-21swapper/211:00:142
175313399290,27cyclictest0-21swapper/210:40:142
175313399290,27cyclictest0-21swapper/210:05:152
175313399290,27cyclictest0-21swapper/209:55:592
175313399290,27cyclictest0-21swapper/209:25:142
175313399290,27cyclictest0-21swapper/209:20:132
175313399290,27cyclictest0-21swapper/209:20:132
175313399290,27cyclictest0-21swapper/208:00:142
4670791280,6ptp4l441ktimers/309:35:003
1753137992826,1cyclictest0-21swapper/311:48:593
1753137992826,1cyclictest0-21swapper/309:25:143
1753137992826,1cyclictest0-21swapper/309:20:133
1753137992826,1cyclictest0-21swapper/309:20:133
1753137992826,1cyclictest0-21swapper/309:19:593
1753137992825,2cyclictest0-21swapper/311:11:593
175313799280,27cyclictest0-21swapper/307:20:173
175313799280,26cyclictest0-21swapper/312:10:153
175313799280,21cyclictest0-21swapper/310:30:153
1753133992825,2cyclictest0-21swapper/208:10:592
175313399280,26cyclictest0-21swapper/211:25:132
175313399280,26cyclictest0-21swapper/209:31:592
175313399280,25cyclictest0-21swapper/212:23:582
4670791270,4ptp4l441ktimers/312:25:013
1753137992724,2cyclictest0-21swapper/307:25:593
175313799270,26cyclictest0-21swapper/310:38:003
175313799270,25cyclictest0-21swapper/310:59:593
1753133992723,2cyclictest0-21swapper/210:26:592
175313399270,25cyclictest0-21swapper/212:07:002
1753132992717,4cyclictest1803257-21kworker/u8:2+events_unbound10:10:351
4670791260,5ptp4l441ktimers/312:24:553
4670791260,5ptp4l441ktimers/312:10:013
4670791260,5ptp4l441ktimers/310:05:003
4670791260,5ptp4l441ktimers/309:39:593
4670791260,5ptp4l441ktimers/308:14:553
1753137992625,1cyclictest0-21swapper/309:55:583
1753137992624,1cyclictest0-21swapper/312:01:003
1753137992624,1cyclictest0-21swapper/308:29:593
1753137992623,2cyclictest0-21swapper/309:40:023
175313799260,25cyclictest0-21swapper/308:52:593
175313799260,24cyclictest0-21swapper/307:15:233
1753133992623,2cyclictest0-21swapper/210:50:002
1753133992623,2cyclictest0-21swapper/208:05:152
1753133992623,2cyclictest0-21swapper/207:25:142
175313399260,2cyclictest0-21swapper/208:53:592
4670791250,6ptp4l61690irq/136-eno1-TxRx-010:50:013
4670791250,5ptp4l441ktimers/311:44:553
4670791250,5ptp4l441ktimers/310:25:013
4670791250,4ptp4l441ktimers/311:59:543
202199250,13rtkit-daemon1751720-21apt-get07:05:171
1753137992522,2cyclictest0-21swapper/311:16:593
1753137992522,2cyclictest0-21swapper/308:15:593
1753133992522,2cyclictest0-21swapper/211:34:592
1753133992521,2cyclictest0-21swapper/211:43:592
4670791240,5ptp4l61690irq/136-eno1-TxRx-009:54:563
1753133992420,2cyclictest0-21swapper/212:11:592
175312999230,3cyclictest1753158-21ls07:10:010
1753133992219,2cyclictest0-21swapper/212:25:582
1753133992118,2cyclictest0-21swapper/210:20:132
1753133992117,2cyclictest0-21swapper/209:52:592
1753133992117,2cyclictest0-21swapper/207:15:142
1753133992017,2cyclictest0-21swapper/210:12:592
1753133991916,2cyclictest0-21swapper/211:18:002
1753133991916,2cyclictest0-21swapper/208:31:592
1753133991815,2cyclictest0-21swapper/210:04:582
1753133991815,2cyclictest0-21swapper/208:20:592
1753133991513,1cyclictest0-21swapper/212:39:582
4670791140,3ptp4l441ktimers/308:20:193
17531339930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional