You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 15:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Thu Nov 27, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31862969910190,8cyclictest3215127-21latency_hist08:40:000
3186297999884,11cyclictest3224679-21latency_hist09:10:011
3186296998979,8cyclictest3192553-21latency_hist07:30:010
3186296998780,5cyclictest3283566-21kworker/u8:2+events_unbound12:15:010
3186296998681,4cyclictest3228181-21kworker/u8:0+flush-179:009:25:010
3186297998346,34cyclictest723-21Xorg08:58:581
3186297998346,34cyclictest723-21Xorg08:58:581
3186296998373,8cyclictest3281688-21latency_hist12:05:010
3186297998273,6cyclictest3206903-21latency_hist08:15:011
3186297998273,6cyclictest3206903-21latency_hist08:15:011
3186296998277,4cyclictest3246403-21kworker/u8:2+flush-179:010:45:000
3186296998173,6cyclictest3276943-21kworker/u8:0+events_unbound12:00:000
3186296998075,4cyclictest3184696-21kworker/u8:3+flush-179:007:55:230
3186296998073,5cyclictest3234734-21kworker/u8:2+flush-179:009:50:140
3186296997867,8cyclictest3290162-21latency_hist12:30:000
3186297997771,4cyclictest3187807-21kworker/u8:2+flush-179:007:25:011
3186297997564,8cyclictest271-21systemd-journal07:20:011
3186297997366,4cyclictest3200569-21kworker/u8:1+flush-179:008:20:231
3186296997366,5cyclictest3246403-21kworker/u8:2+events_unbound10:30:230
3186297997252,9cyclictest271-21systemd-journal10:56:271
3186296997268,3cyclictest3246403-21kworker/u8:2+flush-179:010:55:350
3186296997266,4cyclictest3246403-21kworker/u8:2+flush-179:011:30:180
3186296997261,8cyclictest3267611-21/usr/sbin/munin11:20:250
3186297997161,7cyclictest3219900-21latency_hist08:55:011
318629699710,69cyclictest723-21Xorg09:28:580
3186297997063,5cyclictest3200569-21kworker/u8:1+flush-179:008:19:591
3186296997065,3cyclictest3184643-21kworker/u8:1+events_unbound07:20:000
3186296997060,7cyclictest3262544-21latency_hist11:05:020
3186296996863,4cyclictest3260723-21kworker/u8:1+events_unbound11:05:240
3186296996863,4cyclictest3200569-21kworker/u8:1+events_unbound08:20:210
318629699680,66cyclictest3273776-21munin-run11:40:010
3186297996761,5cyclictest3239824-21kworker/u8:3+events_unbound10:00:011
3186296996764,2cyclictest3187807-21kworker/u8:2+flush-179:007:40:000
3186296996762,4cyclictest3256014-21kworker/u8:3+flush-179:010:50:210
3186296996760,5cyclictest3246403-21kworker/u8:2+flush-179:011:50:140
3186296996662,3cyclictest3260723-21kworker/u8:1+flush-179:011:10:230
3186297996458,4cyclictest3262577-21kworker/u8:3+flush-179:011:25:261
3186296996356,5cyclictest3184696-21kworker/u8:3+events_unbound08:30:190
3186296996355,6cyclictest3246403-21kworker/u8:2+flush-179:010:35:210
3186296996352,9cyclictest3236363-21apt-get09:45:000
3186296996352,9cyclictest3236363-21apt-get09:45:000
3186297996156,3cyclictest3236487-21kworker/u8:0+flush-179:010:10:181
3186296996051,6cyclictest3218312-21latency_hist08:50:010
3186297995948,9cyclictest3230266-21diskstats09:25:151
3186297995948,7cyclictest3246403-21kworker/u8:2+flush-179:010:20:161
3186297995945,13cyclictest3236487-21kworker/u8:0+flush-179:009:55:001
3186296995953,5cyclictest3262577-21kworker/u8:3+flush-179:011:25:290
3186296995953,4cyclictest3275315-21kworker/u8:3+flush-179:011:45:250
3186297995849,7cyclictest3246403-21kworker/u8:2+flush-179:011:40:211
3186297995751,4cyclictest3184696-21kworker/u8:3+events_unbound08:30:241
3186296995749,7cyclictest3281679-21kworker/u8:1+flush-179:012:05:130
3186296995748,5cyclictest3283566-21kworker/u8:2+flush-179:012:15:190
3186297995651,3cyclictest3246403-21kworker/u8:2+flush-179:010:45:251
3186297995650,4cyclictest3246403-21kworker/u8:2+flush-179:010:35:171
3186296995651,4cyclictest3224693-21kworker/u8:2+flush-179:009:10:240
3186296995650,4cyclictest3210763-21kworker/u8:0+events_unbound09:05:000
3186297995547,6cyclictest3290419-21apt-get12:30:131
3186296995548,5cyclictest3210763-21kworker/u8:0+flush-179:008:55:140
3186296995548,5cyclictest3210763-21kworker/u8:0+flush-179:008:55:140
3186297995445,6cyclictest3236353-21latency_hist09:45:011
3186297995445,6cyclictest3236353-21latency_hist09:45:011
3186297995443,8cyclictest3236867-21apt-get09:45:111
3186296995443,7cyclictest3184696-21kworker/u8:3+events_unbound08:15:150
3186297995347,4cyclictest3283566-21kworker/u8:2+flush-179:012:10:251
3186296995345,5cyclictest3236487-21kworker/u8:0+flush-179:010:05:230
3186297995247,4cyclictest3200569-21kworker/u8:1+flush-179:008:45:311
3186297995244,5cyclictest3248066-21kworker/u8:1+events_unbound10:25:231
3186296995245,5cyclictest3200569-21kworker/u8:1+flush-179:009:40:000
3186296995243,6cyclictest3200569-21kworker/u8:1+flush-179:008:05:190
3186297995147,3cyclictest3200569-21kworker/u8:1+flush-179:009:30:261
3186297995144,6cyclictest3246403-21kworker/u8:2+events_unbound11:35:001
3186297995142,6cyclictest3239824-21kworker/u8:3+flush-179:010:00:191
3186297995140,5cyclictest3281679-21kworker/u8:1+events_unbound12:35:221
3186299995047,2cyclictest0-21swapper/212:00:592
3186297995045,3cyclictest3184696-21kworker/u8:3+flush-179:007:50:271
3186297995044,4cyclictest3184696-21kworker/u8:3+events_unbound07:45:221
3186296995045,4cyclictest3210763-21kworker/u8:0+flush-179:008:55:010
3186296995040,8cyclictest3192811-21apt-get07:30:110
3186297994946,2cyclictest723-21Xorg11:55:001
3186297994943,4cyclictest3184696-21kworker/u8:3+flush-179:007:15:001
3186296994941,6cyclictest3200524-21kworker/u8:0+flush-179:008:00:190
4670791480,13ptp4l441ktimers/307:10:013
3186297994842,5cyclictest3256014-21kworker/u8:3+flush-179:010:50:241
3186296994844,3cyclictest3210763-21kworker/u8:0+flush-179:009:05:200
3186296994842,4cyclictest3246403-21kworker/u8:2+flush-179:011:40:140
3186297994741,4cyclictest3262577-21kworker/u8:3+flush-179:011:15:241
3186297994640,5cyclictest3260723-21kworker/u8:1+flush-179:011:05:211
3186296994641,4cyclictest3246403-21kworker/u8:2+events_unbound10:20:000
3186296994637,7cyclictest3286712-21apt-get12:20:030
3186296994637,6cyclictest3186157-21latency_hist07:10:020
3186296994635,8cyclictest3228126-21latency_hist09:20:010
3186297994540,4cyclictest3210763-21kworker/u8:0+flush-179:008:40:211
3186297994538,5cyclictest3246403-21kworker/u8:2+flush-179:011:35:171
3186297994537,6cyclictest3200569-21kworker/u8:1+flush-179:008:10:001
3186296994537,6cyclictest3210763-21kworker/u8:0+events_unbound08:25:170
318629699450,43cyclictest1-21systemd10:15:000
3186297994440,3cyclictest3283566-21kworker/u8:2+events_unbound12:24:591
3186297994440,2cyclictest723-21Xorg11:56:591
3186297994435,7cyclictest3210763-21kworker/u8:0+flush-179:008:25:191
3186297994434,6cyclictest3224721-21kworker/u8:3+flush-179:009:10:201
3186296994441,2cyclictest191rcuc/010:30:000
3186296994438,4cyclictest3234734-21kworker/u8:2+events_unbound10:00:230
3186297994334,7cyclictest3234744-21latency_hist09:40:021
318629799430,41cyclictest723-21Xorg07:41:001
3186296994338,4cyclictest3200569-21kworker/u8:1+flush-179:009:30:250
3186296994338,4cyclictest3187807-21kworker/u8:2+flush-179:007:45:010
3186296994335,5cyclictest3210763-21kworker/u8:0+flush-179:008:40:140
318629699430,41cyclictest3236367-21turbostat09:49:550
3186296994237,4cyclictest3246403-21kworker/u8:2+flush-179:010:20:220
3186296994233,6cyclictest3190968-21latency_hist07:25:010
318629699420,40cyclictest3291789-21munin-run12:35:010
3186302994138,2cyclictest0-21swapper/309:28:593
3186302994134,5cyclictest3285135-21turbostat12:15:013
3186297994136,4cyclictest1-21systemd09:20:011
3186296994137,3cyclictest3184643-21kworker/u8:1+flush-179:007:10:190
3186297994031,7cyclictest3260723-21kworker/u8:1+flush-179:011:00:221
3186296994035,3cyclictest3241367-21cron10:00:010
3186302993937,1cyclictest0-21swapper/312:05:003
3186299993934,2cyclictest0-21swapper/209:49:552
318629999390,36cyclictest0-21swapper/211:39:542
318629999390,36cyclictest0-21swapper/208:50:002
3186297993933,5cyclictest3187807-21kworker/u8:2+flush-179:007:30:221
318629799390,2cyclictest723-21Xorg12:19:591
3186297993831,4cyclictest3200569-21kworker/u8:1+flush-179:009:20:191
3186297993828,7cyclictest271-21systemd-journal09:05:011
318629799380,2cyclictest723-21Xorg10:18:591
3186296993834,3cyclictest723-21Xorg07:53:590
3186296993831,5cyclictest3246403-21kworker/u8:2+flush-179:010:45:190
318630299370,1cyclictest0-21swapper/308:33:593
3186299993734,2cyclictest0-21swapper/209:29:552
3186299993734,2cyclictest0-21swapper/207:33:592
3186299993733,2cyclictest0-21swapper/211:14:552
318629799370,36cyclictest0-21swapper/110:09:591
3186296993733,3cyclictest3267379-21perf11:20:000
318629699373,32cyclictest3184696-21kworker/u8:3+events_unbound08:10:130
318629699373,32cyclictest3184696-21kworker/u8:3+events_unbound08:10:120
3186302993633,2cyclictest0-21swapper/310:41:593
318630299360,35cyclictest0-21swapper/308:44:583
318630299360,35cyclictest0-21swapper/308:39:003
3186299993634,1cyclictest0-21swapper/211:25:002
3186299993633,2cyclictest0-21swapper/209:32:002
3186299993633,2cyclictest0-21swapper/209:20:592
3186299993633,2cyclictest0-21swapper/208:29:552
3186299993632,2cyclictest0-21swapper/212:21:002
318629999360,1cyclictest0-21swapper/209:15:592
3186297993633,2cyclictest0-21swapper/111:11:001
3186297993632,3cyclictest3195756-21systemctl07:40:011
3186302993533,1cyclictest0-21swapper/311:24:003
3186302993533,1cyclictest0-21swapper/309:11:593
3186302993533,1cyclictest0-21swapper/307:45:593
3186302993532,2cyclictest0-21swapper/308:49:593
3186302993531,3cyclictest0-21swapper/308:53:593
318630299350,34cyclictest0-21swapper/311:51:593
318630299350,1cyclictest0-21swapper/312:31:593
3186299993533,1cyclictest0-21swapper/212:14:582
3186299993532,2cyclictest0-21swapper/212:15:582
3186297993532,2cyclictest3200524-21kworker/u8:0+events_unbound08:00:131
3186296993527,6cyclictest3184696-21kworker/u8:3+flush-179:007:45:190
4670791340,6ptp4l441ktimers/310:05:003
3186299993432,1cyclictest0-21swapper/207:30:002
3186299993431,2cyclictest0-21swapper/207:49:562
3186297993424,9cyclictest3281694-21idleruntime-cro12:04:591
318629799342,30cyclictest3275579-21apt-get11:45:151
4670791330,27ptp4l441ktimers/310:56:313
3186302993331,1cyclictest0-21swapper/309:39:553
3186302993331,1cyclictest0-21swapper/307:44:563
3186299993330,2cyclictest0-21swapper/210:25:592
3186302993229,2cyclictest0-21swapper/312:36:003
3186302993229,2cyclictest0-21swapper/309:34:593
3186302993229,2cyclictest0-21swapper/307:16:583
3186299993229,2cyclictest0-21swapper/211:55:012
3186299993229,2cyclictest0-21swapper/211:09:562
3186299993229,2cyclictest0-21swapper/210:13:592
3186297993229,2cyclictest281ktimers/112:05:021
3186302993128,2cyclictest0-21swapper/309:18:593
3186302993125,4cyclictest3259163-21turbostat10:55:003
3186299993128,2cyclictest0-21swapper/210:43:582
3186299993128,2cyclictest0-21swapper/207:54:002
318629999310,29cyclictest0-21swapper/210:00:132
3186297993121,8cyclictest3191224-21apt-get07:25:101
318629799310,29cyclictest0-21swapper/111:20:151
318629699311,29cyclictest3292006-21apt-get12:35:130
3186302993027,2cyclictest0-21swapper/310:30:123
3186302993027,2cyclictest0-21swapper/310:15:153
3186302993027,2cyclictest0-21swapper/310:00:013
318630299300,28cyclictest0-21swapper/307:35:123
3186299993027,2cyclictest0-21swapper/210:33:592
3186299993027,2cyclictest0-21swapper/208:39:552
3186299993026,2cyclictest0-21swapper/212:25:122
3186299993025,2cyclictest0-21swapper/212:05:032
318629999300,27cyclictest0-21swapper/210:00:012
318629999300,27cyclictest0-21swapper/208:10:132
318629999300,27cyclictest0-21swapper/208:10:122
3186297993026,3cyclictest3255082-21hwlatdetect10:40:161
4670791290,6ptp4l441ktimers/311:30:013
3186302992927,1cyclictest0-21swapper/310:48:593
3186302992926,2cyclictest0-21swapper/312:25:113
318630299290,28cyclictest0-21swapper/311:55:143
318630299290,28cyclictest0-21swapper/309:00:113
318630299290,27cyclictest0-21swapper/310:10:153
318630299290,1cyclictest0-21swapper/312:05:013
3186299992926,2cyclictest0-21swapper/211:40:132
3186299992926,2cyclictest0-21swapper/210:55:132
3186299992926,2cyclictest0-21swapper/209:05:102
3186299992925,2cyclictest0-21swapper/212:35:132
318629999290,27cyclictest0-21swapper/211:15:132
318629999290,27cyclictest0-21swapper/210:50:132
318629999290,27cyclictest0-21swapper/209:10:022
318629999290,27cyclictest0-21swapper/208:40:132
318629999290,27cyclictest0-21swapper/208:07:002
318629999290,27cyclictest0-21swapper/207:40:152
318629999290,25cyclictest0-21swapper/211:25:162
3186297992927,1cyclictest3251451-21apt-get10:30:121
4670791280,7ptp4l61690irq/136-eno1-TxRx-009:00:013
4670791280,7ptp4l61690irq/136-eno1-TxRx-009:00:013
4670791280,7ptp4l441ktimers/311:34:593
4670791280,6ptp4l441ktimers/312:25:003
4670791280,6ptp4l441ktimers/308:25:003
3186302992826,1cyclictest0-21swapper/307:34:543
3186302992825,2cyclictest0-21swapper/311:05:133
3186302992825,2cyclictest0-21swapper/309:40:133
3186302992825,2cyclictest0-21swapper/309:40:133
3186302992825,2cyclictest0-21swapper/308:10:123
3186302992825,2cyclictest0-21swapper/308:10:123
318630299280,26cyclictest0-21swapper/311:45:133
3186299992825,2cyclictest0-21swapper/211:30:132
3186299992825,2cyclictest0-21swapper/207:22:582
318629999280,27cyclictest0-21swapper/210:15:152
318629999280,27cyclictest0-21swapper/207:10:142
4670791270,6ptp4l441ktimers/312:20:013
4670791270,6ptp4l441ktimers/311:40:013
3186302992725,1cyclictest0-21swapper/311:40:133
3186302992725,1cyclictest0-21swapper/308:09:593
3186302992725,1cyclictest0-21swapper/307:25:123
3186302992724,2cyclictest0-21swapper/308:02:593
318630299270,26cyclictest0-21swapper/311:15:593
318630299270,26cyclictest0-21swapper/310:28:593
318630299270,1cyclictest0-21swapper/309:07:003
318630299270,1cyclictest0-21swapper/307:25:003
3186299992724,2cyclictest0-21swapper/207:15:142
318629999270,2cyclictest0-21swapper/212:33:592
318629999270,2cyclictest0-21swapper/208:58:592
318629999270,2cyclictest0-21swapper/208:58:582
318629999270,25cyclictest0-21swapper/210:38:592
3186297992723,3cyclictest3283566-21kworker/u8:2+flush-179:012:25:231
3186297992722,4cyclictest3214200-21gltestperf08:35:141
3186297992722,3cyclictest0-21swapper/108:00:011
4670791260,6ptp4l61690irq/136-eno1-TxRx-010:10:013
4670791260,6ptp4l61690irq/136-eno1-TxRx-008:19:553
4670791260,6ptp4l441ktimers/311:04:563
4670791260,5ptp4l441ktimers/307:15:013
4670791260,4ptp4l441ktimers/310:40:013
4670791260,4ptp4l441ktimers/309:50:003
3186302992624,1cyclictest0-21swapper/309:54:553
3186302992623,2cyclictest0-21swapper/307:55:593
3186299992623,2cyclictest0-21swapper/209:35:012
4670791250,5ptp4l61690irq/136-eno1-TxRx-010:25:003
4670791250,5ptp4l441ktimers/309:25:013
4670791250,5ptp4l441ktimers/308:29:553
4670791250,5ptp4l441ktimers/307:54:543
3186299992522,2cyclictest0-21swapper/210:49:552
3186299992522,2cyclictest0-21swapper/210:22:592
3186299992522,2cyclictest0-21swapper/210:09:592
318629999250,2cyclictest0-21swapper/208:22:002
4670791240,4ptp4l441ktimers/311:14:543
3186299992320,2cyclictest0-21swapper/211:59:552
3186299992320,2cyclictest0-21swapper/209:53:592
3186299992319,2cyclictest0-21swapper/211:04:592
3186299992319,2cyclictest0-21swapper/208:04:582
3186299992220,1cyclictest0-21swapper/208:15:582
3186299992219,2cyclictest0-21swapper/208:34:552
3186299992218,2cyclictest0-21swapper/209:03:592
3186299991916,2cyclictest0-21swapper/209:40:592
3186299991916,2cyclictest0-21swapper/209:40:592
318629799192,15cyclictest3186326-21cat07:10:011
3186299991714,2cyclictest0-21swapper/207:36:582
3186299991512,2cyclictest0-21swapper/211:48:592
3186299991412,1cyclictest0-21swapper/208:55:002
318629999118,2cyclictest0-21swapper/207:55:592
31862999930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional