You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-03 - 20:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Wed Sep 03, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6881299910698,6cyclictest712225-21kworker/u8:1+flush-179:009:40:010
6881309910594,8cyclictest751259-21latency_hist10:25:011
6881309910390,10cyclictest720678-21cp08:50:021
688130999990,7cyclictest709082-21kworker/u8:0+flush-179:008:20:001
688130999786,8cyclictest744625-21latency_hist10:05:011
688130999687,7cyclictest775198-21kworker/u8:3+flush-179:011:50:201
688130999579,15cyclictest792900-21kworker/u8:2+flush-179:012:40:001
688130999184,5cyclictest784741-21kworker/u8:2+flush-179:012:10:221
688130998980,7cyclictest694665-21/usr/sbin/munin07:30:251
688130998882,4cyclictest725294-21kworker/u8:2+flush-179:009:20:201
688130998875,10cyclictest601706-21systemd-journal09:00:321
688130998873,13cyclictest694664-21kworker/u8:1+flush-179:007:40:001
688130998781,4cyclictest787915-21kworker/u8:1+flush-179:012:25:211
688130998776,9cyclictest780161-21/usr/sbin/munin11:55:231
688130998573,10cyclictest730052-21apt-get09:20:001
688129998575,7cyclictest744862-21/usr/sbin/munin10:05:230
688129998574,8cyclictest686504-21kworker/u8:0+flush-179:007:30:260
688130998376,5cyclictest712225-21kworker/u8:1+flush-179:008:45:161
688130998376,5cyclictest686504-21kworker/u8:0+events_unbound07:20:221
688130998371,9cyclictest791593-21apt-get12:30:111
688130998274,6cyclictest717076-21kworker/u8:2+flush-179:008:40:241
688129998171,8cyclictest765592-21latency_hist11:10:020
688129998072,6cyclictest733725-21apt-get09:30:110
688129997952,24cyclictest759221-21latency_hist10:50:010
688129997952,24cyclictest759221-21latency_hist10:50:000
688129997767,8cyclictest772220-21apt-get11:30:110
688129997751,22cyclictest743028-21latency_hist10:00:020
688130997668,7cyclictest751294-21kworker/u8:1+flush-179:010:55:001
688129997565,7cyclictest747802-21latency_hist10:15:010
688130997466,6cyclictest762674-21apt-get11:00:121
688130997363,7cyclictest694664-21kworker/u8:1+events_unbound08:10:211
688129997360,10cyclictest712500-21apt-get08:25:120
688130997266,4cyclictest765627-21kworker/u8:3+events_unbound11:20:231
688130997260,9cyclictest735039-21kworker/u8:0+flush-179:009:40:131
688130997162,7cyclictest713867-21kworker/u8:3+events_unbound08:40:011
688130997162,6cyclictest746238-21kworker/u8:3+flush-179:010:10:121
688130997161,7cyclictest756587-21diskstats10:40:191
688129997162,7cyclictest775198-21kworker/u8:3+flush-179:011:40:140
688129997161,6cyclictest689896-21apt-get07:15:120
688129997161,6cyclictest689896-21apt-get07:15:110
688130997059,8cyclictest786548-21apt-get12:15:141
688129997058,10cyclictest761063-21apt-get10:55:120
688130996958,8cyclictest789492-21latency_hist12:25:021
688129996960,7cyclictest701343-21apt-get07:50:080
688129996958,8cyclictest730037-21latency_hist09:20:010
688130996862,4cyclictest730074-21kworker/u8:0+events_unbound09:25:201
688130996659,5cyclictest712225-21kworker/u8:1+events_unbound09:50:141
688130996658,6cyclictest601706-21systemd-journal08:26:321
688129996658,5cyclictest779115-21kworker/u8:0+events_unbound12:25:020
688129996657,7cyclictest779115-21kworker/u8:0+flush-179:011:50:250
688129996657,7cyclictest727092-21apt-get09:10:130
688129996656,7cyclictest720701-21kworker/u8:2+flush-179:008:50:130
688130996559,5cyclictest765627-21kworker/u8:3+flush-179:011:15:221
688130996458,5cyclictest713867-21kworker/u8:3+flush-179:008:35:001
688130996453,8cyclictest699756-21apt-get07:45:111
688129996456,6cyclictest688594-21diskstats07:10:150
688129996452,8cyclictest730074-21kworker/u8:0+flush-179:009:25:220
688130996358,4cyclictest704520-21kworker/u8:2+flush-179:008:04:551
688130996351,10cyclictest749395-21apt-get10:20:011
688129996355,5cyclictest704276-21latency_hist08:00:010
688130996252,7cyclictest777004-21apt-get11:45:111
688129996255,6cyclictest725294-21kworker/u8:2+flush-179:009:20:130
688129996252,7cyclictest779115-21kworker/u8:0+flush-179:011:55:210
688129996250,10cyclictest749362-21kworker/u8:0+flush-179:010:35:140
688130996155,5cyclictest704520-21kworker/u8:2+events_unbound08:20:251
688130996155,4cyclictest752848-21latency_hist10:30:001
688129996150,7cyclictest771994-21kworker/u8:1+flush-179:012:00:220
688130996053,5cyclictest762655-21kworker/u8:2+events_unbound11:25:151
688130995953,4cyclictest712225-21kworker/u8:1+flush-179:008:55:221
688130995952,6cyclictest735039-21kworker/u8:0+flush-179:009:35:211
688130995951,6cyclictest779115-21kworker/u8:0+flush-179:012:05:221
688129995953,3cyclictest704520-21kworker/u8:2+flush-179:008:15:030
688129995951,6cyclictest601706-21systemd-journal12:09:300
688129995949,7cyclictest601706-21systemd-journal09:53:320
688130995855,2cyclictest722-21Xorg09:48:001
688130995852,4cyclictest764263-21apt-get11:05:131
688129995851,5cyclictest779115-21kworker/u8:0+flush-179:012:10:210
688129995849,6cyclictest746243-21kworker/u8:4+flush-179:010:25:200
688129995824,27cyclictest1-21systemd08:50:010
688130995749,6cyclictest771994-21kworker/u8:1+events_unbound11:35:211
688129995753,3cyclictest765627-21kworker/u8:3+flush-179:011:30:010
688129995750,5cyclictest735039-21kworker/u8:0+flush-179:009:50:010
688130995650,5cyclictest746238-21kworker/u8:3+flush-179:010:30:181
688130995648,6cyclictest733446-21kworker/u8:3+events_unbound09:35:011
688129995645,9cyclictest769046-21apt-get11:20:110
688130995552,2cyclictest765627-21kworker/u8:3+events_unbound11:10:241
688130995448,5cyclictest686504-21kworker/u8:0+flush-179:007:25:211
688130995445,7cyclictest741666-21/usr/sbin/munin09:55:181
688130995445,6cyclictest771994-21kworker/u8:1+events_unbound11:30:201
688130995345,6cyclictest601706-21systemd-journal12:00:321
688129995345,6cyclictest779115-21kworker/u8:0+flush-179:012:20:160
688129995318,20cyclictest0-21swapper/010:50:150
688130995238,11cyclictest601706-21systemd-journal10:10:001
688129995247,4cyclictest712225-21kworker/u8:1+flush-179:009:05:260
688129995245,5cyclictest712225-21kworker/u8:1+flush-179:008:40:000
688129995241,9cyclictest704539-21apt-get08:00:150
688129995145,5cyclictest762655-21kworker/u8:2+flush-179:011:00:210
688129995145,5cyclictest746243-21kworker/u8:4+flush-179:010:20:210
688129995144,6cyclictest674208-21kworker/u8:1+flush-179:007:10:020
688129995141,8cyclictest771994-21kworker/u8:1+flush-179:011:45:150
688129995120,21cyclictest692845-21turbostat07:29:540
688130995043,5cyclictest674208-21kworker/u8:1+flush-179:007:15:001
688130994940,7cyclictest601706-21systemd-journal07:19:331
688130994940,7cyclictest601706-21systemd-journal07:19:321
688129994939,8cyclictest787888-21latency_hist12:20:010
688129994938,8cyclictest696327-21apt-key07:35:010
4505291490,13ptp4l441ktimers/307:10:013
688129994838,8cyclictest783100-21kworker/u8:3+flush-179:012:35:130
688129994838,8cyclictest601706-21systemd-journal11:19:590
688129994837,9cyclictest601706-21systemd-journal07:49:590
688130994742,4cyclictest725294-21kworker/u8:2+flush-179:009:10:211
68812999471,3cyclictest748056-21apt-get10:15:120
688130994640,5cyclictest776838-21latency_hist11:45:001
688130994637,7cyclictest725294-21kworker/u8:2+flush-179:009:10:011
688130994539,4cyclictest476-21dbus-daemon10:50:001
688130994539,4cyclictest476-21dbus-daemon10:50:001
688129994541,3cyclictest713843-21turbostat08:34:550
688129994541,3cyclictest704520-21kworker/u8:2+flush-179:008:20:260
688129994539,4cyclictest722-21Xorg07:41:580
688129994539,4cyclictest722-21Xorg07:41:580
688133994339,2cyclictest0-21swapper/208:48:592
688130994335,6cyclictest704520-21kworker/u8:2+flush-179:008:05:151
688130994335,5cyclictest698145-21/usr/sbin/munin07:40:181
688130994335,5cyclictest698145-21/usr/sbin/munin07:40:181
688130994333,7cyclictest754673-21/usr/sbin/munin10:35:151
688129994328,14cyclictest0-21swapper/008:56:590
68812999430,42cyclictest722-21Xorg11:39:000
68812999430,2cyclictest0-21swapper/008:43:590
688136994135,5cyclictest0-21swapper/308:56:593
688136994134,5cyclictest713843-21turbostat08:30:003
688130994138,2cyclictest0-21swapper/107:51:591
688130994137,3cyclictest699477-21kworker/u8:0+events_unbound07:55:261
688129994135,4cyclictest722-21Xorg08:07:590
688129994129,9cyclictest736924-21apt-get09:40:110
68812999411,32cyclictest692852-21dump-pmu-power07:24:590
68813699400,38cyclictest787893-21turbostat12:20:003
68813699400,1cyclictest0-21swapper/307:41:583
68813699400,1cyclictest0-21swapper/307:41:583
68813099401,2cyclictest722-21Xorg10:56:591
688129994032,5cyclictest1-21systemd09:00:320
688129993935,3cyclictest694664-21kworker/u8:1+flush-179:008:10:080
688129993920,18cyclictest792298-21needreboot12:30:190
688136993836,1cyclictest0-21swapper/310:39:593
688133993835,2cyclictest0-21swapper/212:25:002
68813399380,36cyclictest0-21swapper/210:31:002
68813399380,36cyclictest0-21swapper/210:12:592
688129993819,17cyclictest0-21swapper/011:15:010
688133993734,2cyclictest0-21swapper/210:44:002
688133993734,2cyclictest0-21swapper/209:29:552
688133993734,2cyclictest0-21swapper/209:10:592
688133993734,2cyclictest0-21swapper/208:09:562
68813399370,35cyclictest0-21swapper/209:31:582
68813399370,35cyclictest0-21swapper/209:06:592
688136993634,1cyclictest0-21swapper/312:36:593
688136993634,1cyclictest0-21swapper/310:46:593
688136993634,1cyclictest0-21swapper/310:46:593
68813699360,1cyclictest0-21swapper/309:14:593
688133993634,1cyclictest0-21swapper/208:29:592
688133993633,2cyclictest0-21swapper/207:38:592
688133993633,2cyclictest0-21swapper/207:14:592
68813399360,1cyclictest0-21swapper/211:44:542
68813399360,1cyclictest0-21swapper/211:38:592
68813399360,1cyclictest0-21swapper/208:03:592
68812999361,33cyclictest753103-21apt-get10:30:140
688136993533,1cyclictest0-21swapper/310:44:563
68813699350,2cyclictest775149-21turbostat11:40:003
688136993431,2cyclictest0-21swapper/311:41:003
688133993431,2cyclictest0-21swapper/210:35:582
68812999341,31cyclictest0-21swapper/010:40:180
688136993330,2cyclictest0-21swapper/310:54:553
688136993330,2cyclictest0-21swapper/310:33:583
68813399330,31cyclictest0-21swapper/209:55:592
688136993230,1cyclictest0-21swapper/309:56:593
688136993229,2cyclictest0-21swapper/307:53:593
68813699320,26cyclictest0-21swapper/308:39:003
688133993229,2cyclictest0-21swapper/212:14:552
688133993229,2cyclictest0-21swapper/212:00:592
688133993229,2cyclictest0-21swapper/211:29:002
688133993229,2cyclictest0-21swapper/207:59:552
68812999320,2cyclictest0-21swapper/010:04:000
688136993129,1cyclictest0-21swapper/308:49:593
688136993128,2cyclictest0-21swapper/309:37:593
688136993128,2cyclictest0-21swapper/309:30:593
688136993128,2cyclictest0-21swapper/309:26:593
68813699310,1cyclictest0-21swapper/312:10:143
688133993127,2cyclictest0-21swapper/211:53:592
68813399310,2cyclictest0-21swapper/211:24:592
688136993027,2cyclictest0-21swapper/311:02:583
68813699300,2cyclictest0-21swapper/309:15:133
68813699300,29cyclictest0-21swapper/307:29:543
68813699300,27cyclictest0-21swapper/310:25:163
68813699300,26cyclictest0-21swapper/308:15:133
68813399300,29cyclictest0-21swapper/211:10:142
688136992927,1cyclictest0-21swapper/311:50:583
688136992926,2cyclictest0-21swapper/308:02:003
68813699290,27cyclictest0-21swapper/309:00:153
688133992926,2cyclictest0-21swapper/212:05:132
688133992926,2cyclictest0-21swapper/211:05:582
68813399290,27cyclictest0-21swapper/212:35:122
68813399290,27cyclictest0-21swapper/212:25:032
68813399290,27cyclictest0-21swapper/210:20:032
68813399290,27cyclictest0-21swapper/210:15:132
68813399290,27cyclictest0-21swapper/208:50:122
68813399290,27cyclictest0-21swapper/208:35:132
68813399290,27cyclictest0-21swapper/208:20:132
68813399290,27cyclictest0-21swapper/208:10:122
68813399290,27cyclictest0-21swapper/207:15:122
68813399290,27cyclictest0-21swapper/207:15:122
68813399290,1cyclictest0-21swapper/211:45:132
4505291290,6ptp4l441ktimers/311:05:023
688136992826,1cyclictest0-21swapper/311:22:593
688136992826,1cyclictest0-21swapper/308:50:033
688136992825,2cyclictest0-21swapper/312:30:143
68813699280,27cyclictest0-21swapper/310:09:593
68813699280,26cyclictest0-21swapper/312:20:133
68813699280,26cyclictest0-21swapper/310:10:103
68813699280,21cyclictest0-21swapper/309:20:043
688133992825,2cyclictest0-21swapper/210:55:592
688133992825,2cyclictest0-21swapper/210:50:102
688133992825,2cyclictest0-21swapper/209:50:132
688133992825,2cyclictest0-21swapper/208:40:162
688133992825,2cyclictest0-21swapper/207:30:592
688133992824,2cyclictest0-21swapper/209:40:142
68813399280,2cyclictest0-21swapper/211:15:132
68813399280,26cyclictest0-21swapper/207:40:582
68813399280,26cyclictest0-21swapper/207:40:582
68813399280,21cyclictest0-21swapper/212:15:172
4505291280,6ptp4l61790irq/134-eno1-TxRx-009:55:013
4505291280,6ptp4l441ktimers/311:30:013
4505291280,6ptp4l441ktimers/311:00:013
4505291280,6ptp4l441ktimers/309:45:003
4505291280,6ptp4l441ktimers/307:25:003
688136992725,1cyclictest0-21swapper/311:30:123
688136992725,1cyclictest0-21swapper/310:20:013
688136992725,1cyclictest0-21swapper/307:14:553
688136992722,4cyclictest697907-21turbostat07:40:003
68813699270,26cyclictest781517-21turbostat12:00:013
68813699270,1cyclictest0-21swapper/312:25:023
4505291270,6ptp4l61790irq/134-eno1-TxRx-010:05:003
4505291270,6ptp4l61790irq/134-eno1-TxRx-007:50:013
4505291270,6ptp4l441ktimers/312:10:003
4505291270,6ptp4l441ktimers/311:15:023
4505291270,5ptp4l441ktimers/309:50:013
4505291270,5ptp4l441ktimers/307:20:023
4505291270,5ptp4l441ktimers/307:20:013
688136992623,2cyclictest0-21swapper/310:24:593
68813699260,24cyclictest0-21swapper/308:10:133
688133992623,2cyclictest0-21swapper/212:30:592
688133992623,2cyclictest0-21swapper/209:02:592
688133992623,2cyclictest0-21swapper/208:57:592
688133992623,2cyclictest0-21swapper/208:18:592
688133992623,2cyclictest0-21swapper/207:21:002
688133992622,2cyclictest0-21swapper/207:54:592
68813399260,2cyclictest0-21swapper/210:46:002
68813399260,2cyclictest0-21swapper/210:46:002
68813399260,24cyclictest0-21swapper/211:30:132
68813399260,24cyclictest0-21swapper/209:49:592
4505291260,6ptp4l61790irq/134-eno1-TxRx-011:19:553
4505291260,5ptp4l441ktimers/308:44:553
4505291260,4ptp4l441ktimers/309:10:013
688136992522,2cyclictest0-21swapper/307:30:153
68813699250,1cyclictest0-21swapper/311:46:583
688133992523,1cyclictest0-21swapper/211:05:012
688133992522,2cyclictest0-21swapper/210:03:002
4505291250,5ptp4l61790irq/134-eno1-TxRx-012:04:593
4505291250,5ptp4l441ktimers/307:59:553
4505291250,4ptp4l61790irq/134-eno1-TxRx-008:25:003
4505291250,4ptp4l441ktimers/308:34:553
688133992421,2cyclictest0-21swapper/208:34:592
688133992421,2cyclictest0-21swapper/207:45:592
4505291240,4ptp4l61790irq/134-eno1-TxRx-008:09:563
688133992220,1cyclictest0-21swapper/209:40:002
688133992219,2cyclictest0-21swapper/207:26:582
68813099220,4cyclictest688139-21grep07:10:011
688133992118,2cyclictest0-21swapper/209:16:592
688133991917,1cyclictest0-21swapper/210:05:592
688133991917,1cyclictest0-21swapper/209:20:132
688133991915,2cyclictest0-21swapper/211:55:592
688133991715,1cyclictest0-21swapper/210:26:592
6881339920,1cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional