You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 18:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Fri Nov 28, 2025 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18019789910496,6cyclictest1816532-21kworker/u8:0+events_unbound08:20:011
1801976999275,12cyclictest1887759-21latency_hist11:35:000
1801976998972,13cyclictest1882756-21apt-get11:20:000
1801978998681,4cyclictest1891272-21kworker/u8:2+events_unbound11:55:231
1801978998680,4cyclictest1872916-21kworker/u8:0+flush-179:010:50:241
1801978998478,4cyclictest1826908-21kworker/u8:2+events_unbound08:30:301
1801978998379,3cyclictest1818961-21kworker/u8:2+events_unbound08:00:241
1801978998378,4cyclictest1903477-21kworker/u8:1+flush-179:012:35:261
1801978998374,7cyclictest1837272-21latency_hist09:00:011
1801978998276,4cyclictest1880893-21kworker/u8:0+events_unbound11:40:011
1801978998075,4cyclictest1854799-21kworker/u8:2+events_unbound10:15:231
1801978997868,8cyclictest1840679-21/usr/sbin/munin09:10:221
1801978997770,4cyclictest1861426-21kworker/u8:1+flush-179:011:05:241
1801976997767,7cyclictest1817885-21latency_hist08:00:010
1801978997668,5cyclictest1789531-21kworker/u8:3+events_unbound07:10:161
1801978997570,4cyclictest1903477-21kworker/u8:1+flush-179:012:25:221
1801978997469,4cyclictest1789531-21kworker/u8:3+flush-179:007:15:201
1801976997262,7cyclictest1827449-21latency_hist08:30:010
1801978997167,3cyclictest1854799-21kworker/u8:2+flush-179:010:10:241
1801978997167,3cyclictest1816532-21kworker/u8:0+flush-179:008:10:251
1801978997160,8cyclictest1851593-21latency_hist09:45:011
1801978997062,6cyclictest1818961-21kworker/u8:2+events_unbound08:05:201
1801978996964,3cyclictest1875578-21kworker/u8:2+flush-179:011:25:231
1801978996963,4cyclictest1805115-21kworker/u8:2+events_unbound07:40:221
1801978996960,7cyclictest1808250-21latency_hist07:30:001
1801976996960,6cyclictest1848432-21kworker/u8:3+flush-179:010:05:240
1801976996958,8cyclictest1899751-21diskstats12:10:140
1801978996858,8cyclictest1842266-21/usr/sbin/munin09:15:251
1801978996858,8cyclictest1842266-21/usr/sbin/munin09:15:241
1801978996856,4cyclictest1851830-21kworker/u8:0+flush-179:009:50:181
202199670,18rtkit-daemon1801289-21ntp_192.168.11407:05:270
1801983996761,4cyclictest1880885-21turbostat11:15:003
1801978996762,4cyclictest1891272-21kworker/u8:2+events_unbound12:10:221
1801978996762,3cyclictest1835703-21kworker/u8:0+events_unbound09:05:241
1801978996733,33cyclictest723-21Xorg09:04:591
1801978996662,3cyclictest1805115-21kworker/u8:2+events_unbound07:35:261
1801978996660,4cyclictest1813103-21kworker/u8:1+events_unbound07:55:131
1801978996655,9cyclictest1901336-21diskstats12:15:141
1801976996657,7cyclictest1824256-21latency_hist08:20:000
1801978996561,3cyclictest1821058-21kworker/u8:1+flush-179:008:24:561
1801978996560,4cyclictest1875578-21kworker/u8:2+events_unbound10:55:231
1801978996557,6cyclictest1835703-21kworker/u8:0+events_unbound09:20:251
1801978996456,6cyclictest1854799-21kworker/u8:2+events_unbound09:55:201
1801976996461,2cyclictest0-21swapper/009:13:590
1801976996453,8cyclictest1845222-21latency_hist09:25:000
1801976996451,10cyclictest1857961-21latency_hist10:05:010
1801978996354,7cyclictest1871554-21cp10:45:011
1801978996257,4cyclictest1861426-21kworker/u8:1+flush-179:010:25:231
1801976996257,3cyclictest1880893-21kworker/u8:0+flush-179:011:40:260
1801978996155,4cyclictest1843595-21kworker/u8:1+events_unbound09:25:181
1801978996155,4cyclictest1805034-21kworker/u8:1+events_unbound07:20:141
1801978996154,5cyclictest1851830-21kworker/u8:0+events_unbound09:45:201
1801978996153,5cyclictest1869479-21latency_hist10:40:001
1801976996153,5cyclictest1789531-21kworker/u8:3+flush-179:007:15:150
1801976996152,6cyclictest1896277-21/usr/sbin/munin12:00:230
1801978996054,4cyclictest1805115-21kworker/u8:2+events_unbound07:45:141
1801978996053,5cyclictest1880893-21kworker/u8:0+flush-179:011:45:351
1801976996047,10cyclictest1863006-21apt-get10:17:300
1801978995955,3cyclictest1821058-21kworker/u8:1+events_unbound08:50:241
1801978995954,3cyclictest1877738-21kworker/u8:3+events_unbound11:20:181
1801976995949,8cyclictest1832747-21apt-get08:45:110
1801978995753,3cyclictest1880893-21kworker/u8:0+flush-179:011:15:151
1801976995747,6cyclictest1899874-21kworker/u8:3+flush-179:012:15:240
1801978995652,3cyclictest1861426-21kworker/u8:1+flush-179:010:30:221
1801976995647,6cyclictest1871081-21latency_hist10:45:000
1801978995550,4cyclictest1848432-21kworker/u8:3+events_unbound10:20:191
1801983995452,1cyclictest0-21swapper/311:27:593
1801978995450,3cyclictest1897671-21kworker/u8:0+flush-179:012:20:211
1801978995450,3cyclictest1891299-21kworker/u8:3+flush-179:011:50:221
1801978995450,3cyclictest1891299-21kworker/u8:3+flush-179:011:50:211
1801978995445,7cyclictest1880871-21latency_hist11:15:001
1801976995450,2cyclictest723-21Xorg09:34:590
1801976995446,6cyclictest1816532-21kworker/u8:0+flush-179:008:35:000
1801978995347,4cyclictest1816532-21kworker/u8:0+events_unbound08:40:211
1801978995242,3cyclictest1880893-21kworker/u8:0+events_unbound11:45:001
1801978995147,3cyclictest1832370-21kworker/u8:2+flush-179:008:45:201
1801978995146,4cyclictest1816532-21kworker/u8:0+flush-179:008:25:201
1801978995146,4cyclictest1805115-21kworker/u8:2+events_unbound07:30:151
1801976995145,4cyclictest1822666-21latency_hist08:15:010
1801976995144,5cyclictest1891272-21kworker/u8:2+flush-179:012:05:250
1801976995139,9cyclictest271-21systemd-journal10:29:560
1801978995045,4cyclictest1827485-21kworker/u8:3+flush-179:008:35:211
1801978994945,3cyclictest1848432-21kworker/u8:3+flush-179:010:05:221
1801978994938,10cyclictest1846788-21kworker/u8:2+flush-179:009:35:011
180197699492,30cyclictest1830901-21apt-get08:40:130
4670791480,13ptp4l441ktimers/307:09:563
1801978994842,4cyclictest1854799-21kworker/u8:2+events_unbound10:00:231
1801978994826,20cyclictest1871081-21cron10:45:001
1801976994844,2cyclictest1877738-21kworker/u8:3+events_unbound11:35:260
1801976994840,6cyclictest1904245-21apt-get12:25:090
1801976994838,8cyclictest1834331-21apt-get08:50:110
1801976994636,7cyclictest1907165-21latency_hist12:35:010
1801978994536,4cyclictest1805115-21kworker/u8:2+events_unbound07:50:151
1801976994540,3cyclictest1880913-21systemd-run11:15:000
1801976994537,6cyclictest271-21systemd-journal07:15:000
1801976994536,7cyclictest1816302-21munin-run07:55:010
180197699451,42cyclictest1813135-21basename07:44:590
1801978994436,3cyclictest1861426-21kworker/u8:1+events_unbound11:00:281
1801976994335,6cyclictest1856385-21sendmail10:00:010
1801976994335,5cyclictest1814709-21latency_hist07:50:010
180197699433,3cyclictest1872931-21turbostat10:54:560
180197699432,34cyclictest2523597-21snmpd12:35:150
180198299420,40cyclictest0-21swapper/211:49:552
180198299420,40cyclictest0-21swapper/209:11:002
1801978994238,2cyclictest723-21Xorg12:08:591
1801978994237,4cyclictest1903477-21kworker/u8:1+events_unbound12:30:181
180197899420,40cyclictest723-21Xorg11:32:591
1801976994230,9cyclictest271-21systemd-journal11:25:000
180197699420,41cyclictest0-21swapper/009:05:590
1801978994131,3cyclictest1846788-21kworker/u8:2+events_unbound09:35:221
1801976994136,4cyclictest723-21Xorg10:35:590
1801976994135,4cyclictest1830624-21sendmail08:39:590
1801976994033,4cyclictest1802554-21kworker/u8:0+flush-179:007:40:000
1801982993936,2cyclictest0-21swapper/211:54:552
1801982993936,2cyclictest0-21swapper/211:54:552
180198299390,36cyclictest0-21swapper/207:46:592
1801976993934,3cyclictest1879304-21perf11:10:000
1801976993929,7cyclictest1806666-21latency_hist07:25:010
180198399380,37cyclictest1834075-21turbostat08:50:013
1801982993832,5cyclictest0-21swapper/209:23:002
180198299380,36cyclictest0-21swapper/211:26:582
180198299380,36cyclictest0-21swapper/210:23:002
180198299380,36cyclictest0-21swapper/210:05:592
1801976993835,2cyclictest0-21swapper/011:00:580
180198399370,36cyclictest0-21swapper/307:55:593
1801982993734,2cyclictest0-21swapper/208:29:002
180198299370,1cyclictest0-21swapper/212:20:592
1801976993734,2cyclictest0-21swapper/009:04:550
1801976993734,2cyclictest0-21swapper/008:09:590
1801976993733,3cyclictest2523597-21snmpd09:44:580
1801976993730,4cyclictest1835703-21kworker/u8:0+flush-179:009:15:160
1801976993730,4cyclictest1835703-21kworker/u8:0+flush-179:009:15:160
1801976993728,7cyclictest1867888-21latency_hist10:35:020
180198399360,35cyclictest0-21swapper/310:59:553
1801982993634,1cyclictest0-21swapper/210:40:002
1801976993631,3cyclictest1809850-21awk07:35:000
1801976993630,4cyclictest1886296-21latency_hist11:30:010
4670791350,6ptp4l441ktimers/311:40:003
1801983993533,1cyclictest0-21swapper/312:38:593
1801983993533,1cyclictest0-21swapper/308:42:003
1801983993533,1cyclictest0-21swapper/308:05:593
1801983993533,1cyclictest0-21swapper/308:00:593
180198399350,34cyclictest0-21swapper/312:12:593
180198399350,1cyclictest0-21swapper/310:21:593
1801982993532,2cyclictest0-21swapper/210:52:592
1801982993531,2cyclictest0-21swapper/209:47:582
1801978993531,2cyclictest0-21swapper/112:02:591
1801976993532,2cyclictest723-21Xorg09:37:000
1801976993528,5cyclictest1848432-21kworker/u8:3+flush-179:010:20:190
1801983993432,1cyclictest0-21swapper/312:16:593
1801983993432,1cyclictest0-21swapper/308:15:583
1801983993432,1cyclictest0-21swapper/308:14:553
1801982993431,2cyclictest0-21swapper/212:11:592
1801982993431,2cyclictest0-21swapper/211:39:552
1801982993431,2cyclictest0-21swapper/209:34:542
1801982993428,5cyclictest0-21swapper/211:59:592
180198299340,2cyclictest0-21swapper/210:17:382
1801976993426,6cyclictest1902659-21apt-get12:20:120
1801983993331,1cyclictest0-21swapper/311:51:593
1801983993331,1cyclictest0-21swapper/311:51:593
1801983993331,1cyclictest0-21swapper/311:32:593
1801983993331,1cyclictest0-21swapper/308:54:593
1801983993331,1cyclictest0-21swapper/307:54:553
1801982993330,2cyclictest0-21swapper/212:06:592
1801982993327,5cyclictest0-21swapper/212:04:592
180198299330,2cyclictest0-21swapper/210:25:142
180198299330,26cyclictest0-21swapper/208:40:122
180197699331,29cyclictest0-21swapper/011:45:160
1801983993230,1cyclictest0-21swapper/311:18:593
1801983993230,1cyclictest0-21swapper/307:31:003
1801983993229,2cyclictest0-21swapper/311:04:593
180198399320,31cyclictest0-21swapper/308:39:583
1801982993229,2cyclictest0-21swapper/210:46:582
1801982993229,2cyclictest0-21swapper/209:17:582
1801982993229,2cyclictest0-21swapper/209:17:582
1801982993229,2cyclictest0-21swapper/208:23:582
1801978993226,4cyclictest1802046-21latency_hist07:10:011
1801976993229,2cyclictest0-21swapper/008:24:590
1801976993227,3cyclictest1896077-21systemd-run12:00:010
1801976993227,3cyclictest0-21swapper/009:00:010
1801976993224,6cyclictest1818142-21/usr/sbin/munin08:00:160
1801976993223,7cyclictest1846812-21latency_hist09:30:000
180197699320,31cyclictest1801972-21cyclictest11:51:590
180197699320,31cyclictest1801972-21cyclictest11:51:590
1801983993129,1cyclictest0-21swapper/312:08:593
1801983993129,1cyclictest0-21swapper/311:10:003
1801982993128,2cyclictest0-21swapper/211:23:592
1801982993125,5cyclictest0-21swapper/208:05:122
180197699310,28cyclictest0-21swapper/010:55:120
1801983993028,1cyclictest0-21swapper/312:31:593
1801983993027,2cyclictest0-21swapper/308:34:593
180198399300,2cyclictest0-21swapper/309:05:153
180198399300,28cyclictest1827469-21turbostat08:30:003
1801982993028,1cyclictest0-21swapper/211:15:002
1801982993027,2cyclictest0-21swapper/209:57:002
1801982993027,2cyclictest0-21swapper/207:54:592
1801982993027,2cyclictest0-21swapper/207:33:592
1801982993023,5cyclictest0-21swapper/209:05:592
180198299300,2cyclictest0-21swapper/209:04:102
180198299300,2cyclictest0-21swapper/207:40:152
1801976993028,1cyclictest0-21swapper/009:54:560
1801983992927,1cyclictest0-21swapper/312:27:593
1801983992927,1cyclictest0-21swapper/309:50:143
1801983992926,2cyclictest0-21swapper/310:17:353
180198399290,28cyclictest0-21swapper/312:20:133
180198399290,28cyclictest0-21swapper/310:50:153
180198399290,27cyclictest0-21swapper/310:45:023
1801982992926,2cyclictest0-21swapper/209:40:132
180198299290,28cyclictest0-21swapper/210:55:132
180198299290,27cyclictest0-21swapper/212:35:152
180198299290,27cyclictest0-21swapper/208:45:022
180198299290,27cyclictest0-21swapper/207:20:142
1801983992826,1cyclictest0-21swapper/309:04:593
1801983992825,2cyclictest1871094-21turbostat10:45:003
1801983992825,2cyclictest0-21swapper/310:26:593
1801983992825,2cyclictest0-21swapper/307:13:593
1801983992824,3cyclictest0-21swapper/309:29:363
180198399282,25cyclictest1856381-21turbostat10:00:013
180198399280,26cyclictest0-21swapper/311:20:013
180198399280,26cyclictest0-21swapper/307:25:123
1801982992826,1cyclictest0-21swapper/210:40:112
1801982992826,1cyclictest0-21swapper/208:10:112
1801982992825,2cyclictest0-21swapper/211:30:142
1801982992825,2cyclictest0-21swapper/211:15:042
1801982992825,2cyclictest0-21swapper/207:16:592
1801982992825,2cyclictest0-21swapper/207:10:142
180198299280,2cyclictest0-21swapper/210:30:142
180198299280,2cyclictest0-21swapper/208:50:132
180198299280,26cyclictest0-21swapper/212:30:162
180198299280,26cyclictest0-21swapper/208:15:132
1801976992824,3cyclictest1860409-21fschecks_count10:10:170
1801976992823,4cyclictest723-21Xorg10:50:000
1801976992821,5cyclictest1851852-21apt-get09:45:120
4670791270,7ptp4l441ktimers/311:50:013
1801983992725,1cyclictest0-21swapper/307:39:593
1801982992724,2cyclictest0-21swapper/207:35:132
1801982992723,2cyclictest0-21swapper/208:58:262
1801982992722,3cyclictest0-21swapper/211:03:382
180198299270,25cyclictest0-21swapper/211:41:582
180198299270,25cyclictest0-21swapper/210:13:442
180198299270,25cyclictest0-21swapper/207:57:012
180198299270,1cyclictest0-21swapper/211:07:592
1801976992719,5cyclictest1805034-21kworker/u8:1+events_unbound07:25:190
4670791260,6ptp4l61690irq/136-eno1-TxRx-010:35:013
4670791260,6ptp4l61690irq/136-eno1-TxRx-009:35:013
4670791260,6ptp4l61690irq/136-eno1-TxRx-008:25:003
4670791260,6ptp4l61690irq/136-eno1-TxRx-007:24:553
4670791260,6ptp4l441ktimers/310:14:563
4670791260,6ptp4l441ktimers/309:20:013
4670791260,6ptp4l441ktimers/309:20:013
1801983992624,1cyclictest0-21swapper/308:55:123
180198399260,2cyclictest0-21swapper/309:43:363
180198399260,25cyclictest0-21swapper/307:15:143
180198399260,24cyclictest0-21swapper/309:45:323
180198399260,19cyclictest1857975-21turbostat10:05:003
1801982992623,2cyclictest0-21swapper/208:36:412
1801982992622,3cyclictest0-21swapper/209:50:442
180198299260,24cyclictest0-21swapper/209:36:072
180198299260,24cyclictest0-21swapper/209:28:202
180198299260,24cyclictest0-21swapper/208:30:432
180198299260,24cyclictest0-21swapper/208:00:442
1801983992523,1cyclictest0-21swapper/311:58:083
1801983992523,1cyclictest0-21swapper/311:41:363
1801983992523,1cyclictest0-21swapper/310:06:333
1801983992523,1cyclictest0-21swapper/307:43:483
1801983992522,2cyclictest0-21swapper/312:02:593
180198399250,24cyclictest0-21swapper/310:36:193
180198399250,24cyclictest0-21swapper/309:21:263
180198399250,24cyclictest0-21swapper/309:12:073
180198399250,24cyclictest0-21swapper/307:48:433
180198399250,23cyclictest0-21swapper/309:35:123
1801982992522,2cyclictest0-21swapper/210:04:342
180198299250,2cyclictest0-21swapper/212:27:592
180198299250,2cyclictest0-21swapper/207:27:222
1801982991815,2cyclictest0-21swapper/212:19:552
18019829930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional