You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 14:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sun Nov 16, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1736069910093,4cyclictest173523-21kworker/u8:3+flush-179:019:20:001
173606999685,8cyclictest273311-21latency_hist00:15:001
173604999683,10cyclictest183053-21apt-get19:40:000
173606999588,4cyclictest173523-21kworker/u8:3+events_unbound19:15:011
173606998780,5cyclictest224480-21kworker/u8:1+flush-179:022:30:011
173606998175,5cyclictest240152-21kworker/u8:0+events_unbound22:40:011
173606998069,8cyclictest223947-21latency_hist21:45:011
173604998073,5cyclictest207747-21kworker/u8:2+events_unbound21:00:240
173606997973,4cyclictest240152-21kworker/u8:0+events_unbound23:00:261
173604997972,5cyclictest207747-21kworker/u8:2+flush-179:021:15:010
173604997870,6cyclictest246488-21apt-get22:55:030
173606997770,5cyclictest198251-21kworker/u8:3+flush-179:021:10:011
173604997769,6cyclictest271-21systemd-journal20:29:590
173604997665,9cyclictest275178-21apt-get00:20:130
173606997569,4cyclictest219141-21kworker/u8:0+flush-179:021:40:001
173606997568,5cyclictest272391-21kworker/u8:2+flush-179:000:20:001
173604997567,6cyclictest258667-21latency_hist23:30:010
173606997467,5cyclictest228685-21kworker/u8:2+flush-179:022:00:001
173606997368,4cyclictest198251-21kworker/u8:3+flush-179:020:45:011
173606997368,4cyclictest160810-21kworker/u8:1+flush-179:019:55:271
173604997362,8cyclictest250197-21latency_hist23:05:020
173606997267,4cyclictest198194-21kworker/u8:1+flush-179:020:25:001
173606997266,4cyclictest227097-21kworker/u8:0+events_unbound22:10:001
173606997265,6cyclictest220732-21kworker/u8:2+flush-179:021:45:221
173606997165,4cyclictest189953-21kworker/u8:2+flush-179:020:35:011
173606997164,5cyclictest224480-21kworker/u8:1+flush-179:022:10:161
173606997162,6cyclictest260297-21kworker/u8:3+flush-179:023:45:211
173606997161,7cyclictest202975-21latency_hist20:40:001
173606997156,11cyclictest188144-21apt-get19:55:011
173604997165,4cyclictest201424-21kworker/u8:0+flush-179:021:10:000
173604997057,5cyclictest264244-21kworker/u8:1+events_unbound00:00:000
173606996963,4cyclictest227097-21kworker/u8:0+flush-179:022:05:011
173606996862,4cyclictest173523-21kworker/u8:3+flush-179:019:40:001
173606996858,8cyclictest195018-21latency_hist20:15:011
173604996861,4cyclictest227097-21kworker/u8:0+flush-179:021:55:220
173604996857,8cyclictest186546-21latency_hist19:50:000
173606996659,5cyclictest252024-21kworker/u8:1+events_unbound23:10:001
173606996558,5cyclictest240152-21kworker/u8:0+flush-179:023:20:011
173606996558,5cyclictest214156-21kworker/u8:1+flush-179:021:25:211
173604996554,8cyclictest271-21systemd-journal22:15:000
173606996459,4cyclictest160810-21kworker/u8:1+events_unbound19:45:171
173606996458,5cyclictest268540-21kworker/u8:3+flush-179:000:35:001
173604996459,4cyclictest220732-21kworker/u8:2+flush-179:021:40:010
173604996459,3cyclictest214156-21kworker/u8:1+flush-179:021:15:230
173606996358,4cyclictest198251-21kworker/u8:3+flush-179:020:59:541
173606996357,3cyclictest201424-21kworker/u8:0+events_unbound21:10:171
173604996355,6cyclictest189953-21kworker/u8:2+flush-179:020:20:250
173604996354,6cyclictest253642-21apt-get23:15:000
173604996353,8cyclictest188644-21apt-get19:55:150
173604996253,6cyclictest235084-21latency_hist22:20:010
173604996251,9cyclictest268775-21apt-get00:00:120
173604996155,4cyclictest273566-21apt-get00:15:140
173604996150,8cyclictest260257-21latency_hist23:35:010
173604996139,12cyclictest178260-21kworker/u8:0+events_unbound19:30:260
173606996041,12cyclictest178260-21kworker/u8:0+events_unbound19:25:261
173604996054,4cyclictest220732-21kworker/u8:2+events_unbound21:45:170
173610995956,2cyclictest0-21swapper/222:31:002
173606995954,3cyclictest198251-21kworker/u8:3+events_unbound21:05:011
173604995955,3cyclictest268540-21kworker/u8:3+flush-179:000:30:010
173604995951,6cyclictest265295-21latency_hist23:50:010
173606995854,2cyclictest723-21Xorg21:24:591
173604995852,4cyclictest240152-21kworker/u8:0+flush-179:023:10:000
173604995850,7cyclictest236693-21apt-get22:25:030
173604995849,7cyclictest271-21systemd-journal22:45:010
173606995752,4cyclictest240152-21kworker/u8:0+flush-179:000:09:561
173606995750,6cyclictest231936-21kworker/u8:3+flush-179:022:45:001
173606995650,4cyclictest189953-21kworker/u8:2+flush-179:020:05:251
173604995650,5cyclictest253759-21kworker/u8:2+events_unbound23:40:010
173606995549,4cyclictest240152-21kworker/u8:0+events_unbound00:00:241
173606995544,7cyclictest240152-21kworker/u8:0+flush-179:023:30:221
173604995546,6cyclictest173468-21latency_hist19:10:020
173606995449,4cyclictest240152-21kworker/u8:0+events_unbound23:15:011
173606995448,4cyclictest207747-21kworker/u8:2+flush-179:021:20:001
173604995449,4cyclictest253759-21kworker/u8:2+events_unbound23:40:250
173606995348,4cyclictest189953-21kworker/u8:2+flush-179:020:15:231
173606995343,8cyclictest255734-21/usr/sbin/munin23:20:241
173604995347,4cyclictest189953-21kworker/u8:2+flush-179:020:15:250
173604995347,4cyclictest189953-21kworker/u8:2+events_unbound20:30:310
173604995345,6cyclictest207747-21kworker/u8:2+events_unbound20:55:010
173604995246,4cyclictest214156-21kworker/u8:1+flush-179:021:25:170
173604995245,5cyclictest225790-21apt-get21:50:120
173606995146,4cyclictest224480-21kworker/u8:1+events_unbound22:30:211
173606995142,7cyclictest271-21systemd-journal19:35:011
173604995146,4cyclictest198251-21kworker/u8:3+events_unbound20:40:270
173604995144,5cyclictest219141-21kworker/u8:0+flush-179:021:35:010
173606995044,4cyclictest253759-21kworker/u8:2+flush-179:023:25:231
17360499501,47cyclictest1-21systemd19:54:460
173606994945,3cyclictest160810-21kworker/u8:1+events_unbound19:35:261
173604994944,4cyclictest231936-21kworker/u8:3+flush-179:022:35:160
173604994944,4cyclictest160810-21kworker/u8:1+events_unbound20:00:240
173604994942,5cyclictest224480-21kworker/u8:1+flush-179:022:30:220
173604994938,9cyclictest278349-21apt-get00:30:150
173606994844,3cyclictest240152-21kworker/u8:0+flush-179:022:55:001
173606994843,4cyclictest201424-21kworker/u8:0+events_unbound20:45:241
173606994842,5cyclictest243345-21kworker/u8:2+flush-179:022:50:001
173606994840,6cyclictest160810-21kworker/u8:1+events_unbound19:20:161
173604994843,3cyclictest178260-21kworker/u8:0+flush-179:019:30:010
17360699479,34cyclictest235354-21apt-get22:20:151
173606994740,5cyclictest231936-21kworker/u8:3+events_unbound22:20:011
173604994741,4cyclictest230572-21apt-get22:05:130
4670791460,12ptp4l441ktimers/319:09:563
173606994636,9cyclictest248334-21apt-get23:00:011
173604994641,4cyclictest173523-21kworker/u8:3+flush-179:019:15:200
173606994539,5cyclictest160810-21kworker/u8:1+events_unbound20:00:131
173604994540,4cyclictest264244-21kworker/u8:1+flush-179:023:50:230
173604994539,5cyclictest240152-21kworker/u8:0+flush-179:000:05:250
17361099440,41cyclictest0-21swapper/219:43:592
173606994437,3cyclictest278106-21awk00:30:001
173604994337,4cyclictest214156-21kworker/u8:1+events_unbound21:20:240
173604994336,5cyclictest207747-21kworker/u8:2+events_unbound21:00:010
173604994336,5cyclictest198251-21kworker/u8:3+flush-179:020:45:240
17361299421,40cyclictest431rcuc/300:24:553
173606994237,4cyclictest227097-21kworker/u8:0+events_unbound21:55:001
17360699420,41cyclictest0-21swapper/100:23:591
173612994140,1cyclictest0-21swapper/322:54:553
17360499411,3cyclictest244913-21perf22:50:000
173612994038,1cyclictest0-21swapper/319:50:593
173610994033,6cyclictest0-21swapper/222:22:592
173606994033,5cyclictest253759-21kworker/u8:2+flush-179:023:40:161
173604994036,3cyclictest173523-21kworker/u8:3+flush-179:019:40:240
173604994035,4cyclictest252024-21kworker/u8:1+flush-179:023:15:040
173604993934,4cyclictest175086-21munin-run19:14:590
17360499391,35cyclictest3145351-21munin-node19:20:140
17360499390,37cyclictest723-21Xorg23:21:000
173610993831,6cyclictest0-21swapper/223:10:592
17361099380,36cyclictest0-21swapper/200:38:592
17361099380,36cyclictest0-21swapper/200:12:592
17361099380,35cyclictest0-21swapper/221:14:582
173606993835,2cyclictest271rcuc/123:36:591
173606993833,3cyclictest214156-21kworker/u8:1+flush-179:021:30:221
173606993831,6cyclictest272391-21kworker/u8:2+flush-179:000:35:161
173604993836,1cyclictest0-21swapper/022:21:590
173604993834,3cyclictest173600-21cyclictest00:13:590
173604993832,5cyclictest189954-21kworker/u8:3+events_unbound20:10:000
17361299370,36cyclictest0-21swapper/323:34:593
173610993734,2cyclictest0-21swapper/223:19:592
173610993734,2cyclictest0-21swapper/222:01:592
173610993734,2cyclictest0-21swapper/200:18:592
17361099370,35cyclictest0-21swapper/222:13:002
173606993730,6cyclictest189953-21kworker/u8:2+events_unbound20:30:011
173604993734,2cyclictest0-21swapper/020:37:000
173604993732,3cyclictest246918-21apt-key22:55:010
17360499371,34cyclictest475-21dbus-daemon22:05:000
173612993634,2cyclictest0-21swapper/322:32:593
173612993634,1cyclictest0-21swapper/321:09:553
173612993634,1cyclictest0-21swapper/320:41:583
173606993631,4cyclictest268472-21systemctl00:00:001
173612993533,1cyclictest0-21swapper/323:53:593
173612993533,1cyclictest0-21swapper/300:04:553
173610993532,2cyclictest0-21swapper/223:50:592
173610993532,2cyclictest0-21swapper/219:49:552
173610993526,7cyclictest0-21swapper/219:12:592
173612993432,1cyclictest0-21swapper/323:57:593
173612993432,1cyclictest0-21swapper/320:20:003
173612993432,1cyclictest0-21swapper/320:02:593
173610993431,2cyclictest0-21swapper/222:56:592
173610993431,2cyclictest0-21swapper/221:21:002
173610993431,2cyclictest0-21swapper/220:45:582
173610993431,2cyclictest0-21swapper/219:56:592
173610993431,2cyclictest0-21swapper/219:15:582
173606993429,4cyclictest198251-21kworker/u8:3+flush-179:020:50:211
173604993431,2cyclictest723-21Xorg21:44:590
173604993429,4cyclictest268540-21kworker/u8:3+flush-179:000:35:250
173612993331,1cyclictest0-21swapper/319:32:583
173610993331,1cyclictest0-21swapper/221:20:002
173612993230,1cyclictest0-21swapper/323:05:593
173610993229,2cyclictest0-21swapper/221:45:592
173610993229,2cyclictest0-21swapper/221:00:592
173610993229,2cyclictest0-21swapper/219:51:592
173610993229,2cyclictest0-21swapper/200:00:592
173612993129,1cyclictest0-21swapper/323:35:593
173610993128,2cyclictest0-21swapper/223:21:592
17361099310,29cyclictest0-21swapper/223:55:142
17361099310,29cyclictest0-21swapper/220:20:162
173612993028,1cyclictest0-21swapper/320:58:593
173612993025,4cyclictest207767-21turbostat20:55:003
17361299300,29cyclictest0-21swapper/323:40:143
17361299300,28cyclictest0-21swapper/321:20:033
173610993027,2cyclictest0-21swapper/222:45:142
173610993027,2cyclictest0-21swapper/221:30:142
17361099300,2cyclictest0-21swapper/220:25:152
17361099300,28cyclictest0-21swapper/223:05:022
173612992927,1cyclictest0-21swapper/322:19:593
173612992926,2cyclictest0-21swapper/323:00:013
173612992926,2cyclictest0-21swapper/322:45:143
173612992926,2cyclictest0-21swapper/322:00:163
173612992926,2cyclictest0-21swapper/319:40:163
17361299290,28cyclictest0-21swapper/300:35:153
17361299290,27cyclictest0-21swapper/319:55:173
173610992926,2cyclictest0-21swapper/223:00:012
173610992926,2cyclictest0-21swapper/222:50:152
173610992926,2cyclictest0-21swapper/222:41:592
173610992926,2cyclictest0-21swapper/220:40:162
173610992926,2cyclictest0-21swapper/219:40:002
173610992926,2cyclictest0-21swapper/200:25:142
17361099290,2cyclictest0-21swapper/221:55:162
17361099290,27cyclictest0-21swapper/223:40:142
17361099290,27cyclictest0-21swapper/223:35:012
17361099290,27cyclictest0-21swapper/222:25:152
17361099290,27cyclictest0-21swapper/220:35:172
17361099290,27cyclictest0-21swapper/220:00:032
17361099290,27cyclictest0-21swapper/200:20:142
173604992923,4cyclictest0-21swapper/020:15:010
173612992826,1cyclictest0-21swapper/321:10:133
173612992825,2cyclictest0-21swapper/319:40:013
17361299280,27cyclictest0-21swapper/323:10:143
17361299280,27cyclictest0-21swapper/300:10:183
17361299280,26cyclictest0-21swapper/322:35:143
17361299280,26cyclictest0-21swapper/322:25:043
17361299280,21cyclictest0-21swapper/320:05:133
173610992826,1cyclictest0-21swapper/221:39:592
173610992825,2cyclictest0-21swapper/220:30:182
173610992825,2cyclictest0-21swapper/220:10:152
173610992825,2cyclictest0-21swapper/200:05:132
173606992822,4cyclictest0-21swapper/123:50:221
173612992725,1cyclictest0-21swapper/323:25:173
173612992725,1cyclictest0-21swapper/323:19:593
173612992725,1cyclictest0-21swapper/322:43:593
173612992725,1cyclictest0-21swapper/321:30:153
17361299270,26cyclictest0-21swapper/323:00:013
17361299270,26cyclictest0-21swapper/320:35:173
17361299270,26cyclictest0-21swapper/320:20:153
17361299270,26cyclictest0-21swapper/300:07:593
173610992724,2cyclictest0-21swapper/221:05:592
17361099270,25cyclictest0-21swapper/222:06:592
4670791260,5ptp4l441ktimers/322:10:013
173612992623,2cyclictest178289-21turbostat19:25:003
17361299260,25cyclictest0-21swapper/321:03:593
173610992622,2cyclictest0-21swapper/200:30:162
17361099260,24cyclictest0-21swapper/223:45:592
17361099260,24cyclictest0-21swapper/220:09:552
4670791250,6ptp4l61690irq/136-eno1-TxRx-022:25:003
173612992523,1cyclictest0-21swapper/321:59:593
173612992523,1cyclictest0-21swapper/321:53:003
17361299250,23cyclictest0-21swapper/319:10:213
173610992522,2cyclictest0-21swapper/221:40:582
17361099250,1cyclictest0-21swapper/223:29:592
4670791240,4ptp4l61690irq/136-eno1-TxRx-019:49:593
4670791240,4ptp4l441ktimers/323:25:003
4670791240,3ptp4l441ktimers/320:15:013
4670791240,3ptp4l441ktimers/300:20:003
173612992422,1cyclictest0-21swapper/322:12:593
173612992422,1cyclictest0-21swapper/321:37:593
173612992422,1cyclictest0-21swapper/321:28:583
173612992422,1cyclictest0-21swapper/319:29:593
173610992421,2cyclictest0-21swapper/223:32:592
173610992421,2cyclictest0-21swapper/220:15:162
4670791230,3ptp4l61690irq/136-eno1-TxRx-021:50:013
4670791230,3ptp4l441ktimers/323:49:563
4670791230,3ptp4l441ktimers/320:35:013
4670791230,3ptp4l441ktimers/320:30:003
4670791230,3ptp4l441ktimers/300:35:013
173612992321,1cyclictest0-21swapper/321:44:553
173612992321,1cyclictest0-21swapper/319:16:593
173610992321,1cyclictest0-21swapper/221:54:592
173610992320,2cyclictest0-21swapper/219:21:592
173610992316,6cyclictest0-21swapper/222:38:582
4670791220,3ptp4l441ktimers/321:20:003
202199220,13rtkit-daemon685-21ntpd19:09:241
173612992217,4cyclictest276526-21turbostat00:29:553
4670791210,2ptp4l441ktimers/320:49:553
173610992016,2cyclictest0-21swapper/219:28:592
173610991512,2cyclictest0-21swapper/222:18:592
173610991512,2cyclictest0-21swapper/220:50:592
173610991412,1cyclictest0-21swapper/219:34:592
173610991310,2cyclictest0-21swapper/220:56:592
1736109990,8cyclictest0-21swapper/221:26:452
1736109930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional