You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 20:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Fri Dec 19, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3064392999384,6cyclictest3067457-21latency_hist07:20:010
3064395999283,7cyclictest3070651-21latency_hist07:30:001
3064395998982,5cyclictest3122637-21kworker/u8:0+events_unbound10:20:241
3064395998879,7cyclictest3133554-21latency_hist10:45:011
3064395998175,4cyclictest3122637-21kworker/u8:0+events_unbound10:50:231
3064395998066,5cyclictest3085776-21kworker/u8:3+events_unbound09:15:231
3064395997974,4cyclictest3107856-21kworker/u8:3+flush-179:010:05:241
3064395997874,3cyclictest3159265-21kworker/u8:0+events_unbound12:05:261
3064395997871,5cyclictest3125553-21kworker/u8:1+flush-179:010:20:011
3064395997671,4cyclictest3067522-21kworker/u8:2+flush-179:007:40:281
3064392997569,4cyclictest3165916-21kworker/u8:1+events_unbound12:35:250
3064392997567,5cyclictest3085776-21kworker/u8:3+events_unbound08:20:000
3064395997466,6cyclictest3143766-21kworker/u8:1+events_unbound11:45:001
3064395997456,4cyclictest3136770-21kworker/u8:2+flush-179:011:15:011
3064392997465,7cyclictest3093491-21latency_hist08:40:010
3064395997165,4cyclictest3080784-21kworker/u8:3+events_unbound08:00:241
3064395997163,6cyclictest3131936-21kworker/u8:1+flush-179:010:40:001
3064395996963,4cyclictest3107810-21kworker/u8:1+flush-179:009:25:001
3064395996760,5cyclictest3165916-21kworker/u8:1+flush-179:012:40:001
3064395996760,5cyclictest3143766-21kworker/u8:1+events_unbound11:35:241
3064395996760,5cyclictest3143766-21kworker/u8:1+events_unbound11:35:241
3064392996760,5cyclictest3148118-21/usr/sbin/munin11:30:250
3064395996661,3cyclictest3114165-21kworker/u8:1+flush-179:009:50:001
3064395996559,5cyclictest3078326-21kworker/u8:0+events_unbound08:20:251
3064392996557,5cyclictest3122637-21kworker/u8:0+flush-179:010:45:230
306439299650,64cyclictest0-21swapper/009:56:580
306439299650,63cyclictest0-21swapper/009:02:590
3064395996458,5cyclictest3145586-21kworker/u8:2+flush-179:011:25:251
3064395996458,4cyclictest3122637-21kworker/u8:0+events_unbound10:55:151
3064392996360,2cyclictest0-21swapper/008:12:590
306439999620,1cyclictest0-21swapper/308:22:593
3064395996257,3cyclictest3110972-21kworker/u8:0+flush-179:009:40:161
3064395996256,4cyclictest3160909-21kworker/u8:4+flush-179:012:15:201
3064395996256,4cyclictest3085776-21kworker/u8:3+flush-179:008:30:211
3064395996252,7cyclictest3078887-21latency_hist07:55:001
3064399996159,1cyclictest0-21swapper/312:20:593
3064399996159,1cyclictest0-21swapper/309:56:593
3064395995955,3cyclictest3098494-21kworker/u8:2+flush-179:009:05:141
3064395995953,4cyclictest3136770-21kworker/u8:2+events_unbound11:05:241
3064395995953,4cyclictest3107810-21kworker/u8:1+events_unbound09:25:201
3064392995954,4cyclictest3059719-21kworker/u8:3+flush-179:007:15:000
3064392995953,4cyclictest3159326-21kworker/u8:3+flush-179:012:10:000
3064392995951,6cyclictest3127218-21kworker/u8:2+flush-179:010:30:010
3064392995950,7cyclictest3145586-21kworker/u8:2+flush-179:011:20:240
3064392995850,6cyclictest3147876-21latency_hist11:30:010
3064399995755,1cyclictest0-21swapper/309:11:593
3064395995651,3cyclictest3059660-21kworker/u8:1+flush-179:007:45:151
3064395995650,5cyclictest3059660-21kworker/u8:1+flush-179:008:00:001
3064395995650,4cyclictest3107856-21kworker/u8:3+events_unbound10:10:231
3064392995653,2cyclictest0-21swapper/012:04:590
3064392995653,2cyclictest0-21swapper/012:04:580
3064395995549,4cyclictest3067522-21kworker/u8:2+flush-179:007:35:201
3064395995545,6cyclictest3165916-21kworker/u8:1+events_unbound12:35:011
3064395995449,4cyclictest3065852-21kworker/u8:0+flush-179:007:20:201
3064395995448,5cyclictest3116832-21kworker/u8:0+flush-179:010:00:201
3064395995446,6cyclictest3085776-21kworker/u8:3+flush-179:008:15:151
3064395995344,6cyclictest3088750-21kworker/u8:2+events_unbound08:40:211
3064395995344,6cyclictest3088750-21kworker/u8:2+events_unbound08:40:201
3064392995245,5cyclictest3160909-21kworker/u8:4+events_unbound12:15:230
3064395995147,3cyclictest3096703-21kworker/u8:1+events_unbound08:55:151
3064395995146,4cyclictest3096139-21kworker/u8:0+events_unbound08:50:151
3064395995145,5cyclictest3088750-21kworker/u8:2+events_unbound08:45:141
3064395995143,5cyclictest3114165-21kworker/u8:1+flush-179:009:50:181
3064395995142,4cyclictest3059660-21kworker/u8:1+events_unbound07:30:121
3064392995148,2cyclictest191rcuc/012:25:000
3064392995144,5cyclictest3122637-21kworker/u8:0+flush-179:010:39:590
3064399995047,2cyclictest0-21swapper/309:02:593
3064395995044,4cyclictest3078326-21kworker/u8:0+flush-179:008:10:191
3064392995042,6cyclictest3088715-21latency_hist08:25:000
4577091490,14ptp4l61490irq/134-eno1-TxRx-007:10:013
3064395994938,4cyclictest3078326-21kworker/u8:0+flush-179:008:30:001
3064395994844,3cyclictest3098494-21kworker/u8:2+events_unbound09:10:241
3064395994841,4cyclictest3153327-21kworker/u8:2+events_unbound12:00:151
3064395994841,4cyclictest3153327-21kworker/u8:2+events_unbound12:00:151
3064395994840,6cyclictest3136770-21kworker/u8:2+events_unbound11:15:141
3064392994843,4cyclictest3088750-21kworker/u8:2+events_unbound08:40:240
3064392994843,4cyclictest3088750-21kworker/u8:2+events_unbound08:40:240
3064392994842,5cyclictest3143766-21kworker/u8:1+events_unbound11:15:230
3064392994842,4cyclictest3165916-21kworker/u8:1+events_unbound12:25:230
3064392994840,6cyclictest3077299-21latency_hist07:50:020
3064395994741,3cyclictest3122637-21kworker/u8:0+events_unbound11:00:171
306439999460,44cyclictest3147888-21turbostat11:30:013
3064395994639,4cyclictest3153327-21kworker/u8:2+events_unbound12:25:011
3064392994640,4cyclictest3153327-21kworker/u8:2+flush-179:012:00:020
3064392994640,4cyclictest3085776-21kworker/u8:3+events_unbound08:45:220
3064392994638,6cyclictest3153327-21kworker/u8:2+flush-179:011:50:160
306439299460,44cyclictest3078896-21turbostat07:59:540
3064395994540,4cyclictest3147914-21kworker/u8:3+flush-179:011:35:001
3064395994538,5cyclictest3122637-21kworker/u8:0+events_unbound10:30:221
3064392994539,4cyclictest3096703-21kworker/u8:1+flush-179:008:55:230
3064392994537,6cyclictest3106213-21latency_hist09:20:010
3064395994440,3cyclictest3127218-21kworker/u8:2+flush-179:010:45:201
3064395994439,4cyclictest3122637-21kworker/u8:0+flush-179:010:25:251
3064392994438,5cyclictest3059660-21kworker/u8:1+flush-179:007:50:220
3064392994435,7cyclictest3165916-21kworker/u8:1+flush-179:012:30:220
306439299440,2cyclictest3133567-21dump-pmu-power10:45:000
3064395994340,2cyclictest0-21swapper/109:02:591
3064395994338,4cyclictest3153327-21kworker/u8:2+events_unbound11:55:191
3064395994336,5cyclictest3143766-21kworker/u8:1+flush-179:011:45:251
3064392994337,4cyclictest3136770-21kworker/u8:2+flush-179:011:05:000
3064392994337,4cyclictest3109404-21latency_hist09:30:010
3064392994336,5cyclictest3159265-21kworker/u8:0+events_unbound12:10:210
3064392994336,5cyclictest3065852-21kworker/u8:0+flush-179:007:20:150
306439299430,41cyclictest714-21Xorg11:44:000
3064399994240,2cyclictest0-21swapper/310:36:583
3064399994236,5cyclictest3162471-21turbostat12:15:003
3064395994236,4cyclictest3078326-21kworker/u8:0+flush-179:008:35:191
306439599420,40cyclictest3064259-21turbostat07:14:541
3064392994237,3cyclictest714-21Xorg09:39:590
3064392994237,3cyclictest714-21Xorg09:39:590
3064392994236,4cyclictest3122637-21kworker/u8:0+flush-179:010:30:200
306439299420,41cyclictest3135148-21turbostat10:54:560
306439299420,40cyclictest3125572-21cron10:19:590
3064399994138,2cyclictest0-21swapper/308:12:593
3064399994135,5cyclictest3070663-21turbostat07:30:003
3064395994137,3cyclictest3153327-21kworker/u8:2+flush-179:011:50:211
3064392994136,4cyclictest3067522-21kworker/u8:2+events_unbound07:30:230
306439299411,39cyclictest3127199-21apt10:25:010
306439299411,38cyclictest714-21Xorg09:06:590
306439299411,38cyclictest3123994-21apt10:15:000
306439999400,32cyclictest3155846-21turbostat11:55:003
3064397994037,2cyclictest0-21swapper/209:56:592
3064397994035,2cyclictest0-21swapper/209:00:592
3064395994037,2cyclictest0-21swapper/109:56:591
3064392994035,4cyclictest3106722-21cat09:20:150
3064392994035,4cyclictest3085776-21kworker/u8:3+events_unbound08:50:210
3064392994034,5cyclictest3067522-21kworker/u8:2+flush-179:007:40:170
3064392994034,4cyclictest3143766-21kworker/u8:1+flush-179:011:40:010
3064392994034,4cyclictest3143766-21kworker/u8:1+flush-179:011:40:000
306439299401,2cyclictest714-21Xorg08:08:000
306439299400,38cyclictest714-21Xorg10:55:580
3064399993934,4cyclictest3160880-21turbostat12:09:593
306439999390,38cyclictest3130370-21turbostat10:35:003
306439999390,1cyclictest0-21swapper/307:52:593
306439799390,37cyclictest0-21swapper/209:32:002
306439799390,1cyclictest0-21swapper/212:08:002
3064395993933,5cyclictest3065852-21kworker/u8:0+flush-179:007:15:241
3064392993935,2cyclictest714-21Xorg09:50:590
3064392993932,5cyclictest266-21systemd-journal09:33:100
306439299390,37cyclictest714-21Xorg09:40:590
3064399993833,4cyclictest3083932-21turbostat08:10:013
3064397993835,2cyclictest0-21swapper/212:12:592
3064397993835,2cyclictest0-21swapper/211:42:592
3064397993834,2cyclictest0-21swapper/207:22:582
306439799380,36cyclictest0-21swapper/211:22:582
306439799380,36cyclictest0-21swapper/211:09:582
306439799380,35cyclictest0-21swapper/207:50:002
3064395993832,4cyclictest3107856-21kworker/u8:3+flush-179:009:30:161
3064392993833,4cyclictest3103051-21turbostat09:14:550
3064399993735,1cyclictest0-21swapper/312:28:593
306439999370,30cyclictest0-21swapper/309:17:593
3064397993734,2cyclictest0-21swapper/212:18:582
3064397993734,2cyclictest0-21swapper/211:58:592
3064397993734,2cyclictest0-21swapper/208:38:592
306439799370,35cyclictest0-21swapper/211:46:592
306439799370,35cyclictest0-21swapper/209:19:592
306439799370,35cyclictest0-21swapper/208:22:592
306439799370,35cyclictest0-21swapper/208:01:592
306439799370,1cyclictest0-21swapper/207:14:552
3064395993734,2cyclictest714-21Xorg12:13:001
3064392993731,4cyclictest3122637-21kworker/u8:0+events_unbound11:10:180
3064392993730,5cyclictest3141508-21latency_hist11:10:010
306439299370,34cyclictest3069320-21apt-get07:25:140
3064399993634,1cyclictest0-21swapper/311:13:003
3064399993634,1cyclictest0-21swapper/311:02:593
3064399993633,2cyclictest0-21swapper/308:54:593
3064399993632,3cyclictest3107830-21turbostat09:25:003
306439999360,35cyclictest0-21swapper/311:34:563
3064397993634,1cyclictest0-21swapper/209:54:592
3064397993633,2cyclictest0-21swapper/210:36:592
3064397993633,2cyclictest0-21swapper/208:54:002
3064397993633,2cyclictest0-21swapper/207:17:002
3064397993632,2cyclictest0-21swapper/210:23:592
3064397993632,2cyclictest0-21swapper/207:53:592
306439799360,2cyclictest0-21swapper/210:05:592
306439799360,1cyclictest0-21swapper/211:51:592
3064399993533,1cyclictest0-21swapper/312:01:593
3064399993533,1cyclictest0-21swapper/312:01:593
3064399993533,1cyclictest0-21swapper/308:04:553
3064397993532,2cyclictest0-21swapper/210:43:592
3064397993532,2cyclictest0-21swapper/207:43:592
3064397993532,2cyclictest0-21swapper/207:35:592
306439799350,1cyclictest0-21swapper/208:29:552
3064392993532,2cyclictest714-21Xorg08:04:580
3064399993432,1cyclictest0-21swapper/311:57:593
3064397993432,1cyclictest0-21swapper/207:29:592
3064397993431,2cyclictest0-21swapper/210:33:592
3064397993431,2cyclictest0-21swapper/210:16:002
3064395993431,2cyclictest0-21swapper/111:24:551
3064399993331,1cyclictest0-21swapper/311:42:003
3064392993329,3cyclictest714-21Xorg09:49:590
306439299330,31cyclictest3152921-21apt-get11:45:120
3064399993230,1cyclictest0-21swapper/311:49:593
3064399993229,2cyclictest0-21swapper/312:19:593
3064399993227,4cyclictest3090315-21turbostat08:30:003
3064397993230,1cyclictest0-21swapper/212:25:002
3064392993228,3cyclictest3122438-21systemd-run10:10:000
3064392993227,4cyclictest3120940-21latency_hist10:05:010
3064392993227,3cyclictest714-21Xorg08:33:580
3064392993226,5cyclictest3067522-21kworker/u8:2+events_unbound07:35:250
4577091310,10ptp4l61490irq/134-eno1-TxRx-007:39:553
3064399993127,3cyclictest3111002-21turbostat09:35:003
3064399993126,4cyclictest3133561-21turbostat10:45:003
3064399993125,5cyclictest3138321-21turbostat10:59:593
3064397993128,2cyclictest0-21swapper/211:17:592
3064397993128,2cyclictest0-21swapper/207:34:552
3064395993127,3cyclictest3165916-21kworker/u8:1+flush-179:012:25:211
3064395993126,4cyclictest3098494-21kworker/u8:2+events_unbound09:35:161
3064395993126,4cyclictest3098494-21kworker/u8:2+events_unbound09:35:151
3064399993028,1cyclictest0-21swapper/308:59:593
3064399993025,4cyclictest3120822-21turbostat10:05:003
3064397993028,1cyclictest0-21swapper/212:04:592
3064397993028,1cyclictest0-21swapper/212:04:582
3064397993026,2cyclictest0-21swapper/209:26:592
306439799300,28cyclictest0-21swapper/211:13:002
3064395993025,4cyclictest3084136-21ls08:10:011
4577091290,7ptp4l441ktimers/310:25:013
4577091290,7ptp4l441ktimers/308:50:003
4577091290,7ptp4l441ktimers/308:00:003
4577091290,7ptp4l441ktimers/307:35:003
4577091290,6ptp4l441ktimers/309:40:003
4577091290,6ptp4l441ktimers/309:40:003
3064399992927,1cyclictest0-21swapper/310:14:553
3064399992927,1cyclictest0-21swapper/308:32:593
3064399992925,3cyclictest3065879-21turbostat07:15:003
306439999290,2cyclictest0-21swapper/311:15:123
306439999290,28cyclictest0-21swapper/311:35:133
306439999290,28cyclictest0-21swapper/311:35:123
3064397992926,2cyclictest0-21swapper/207:55:032
306439799290,2cyclictest0-21swapper/211:00:582
306439799290,21cyclictest0-21swapper/208:40:132
306439799290,21cyclictest0-21swapper/208:40:132
4577091280,6ptp4l441ktimers/308:20:003
4577091280,6ptp4l441ktimers/307:20:003
3064399992825,2cyclictest0-21swapper/310:15:123
306439999280,27cyclictest0-21swapper/310:45:143
306439999280,27cyclictest0-21swapper/309:40:153
306439999280,27cyclictest0-21swapper/307:40:143
306439999280,26cyclictest0-21swapper/311:05:123
306439999280,26cyclictest0-21swapper/307:20:113
3064397992826,1cyclictest0-21swapper/212:39:592
3064397992826,1cyclictest0-21swapper/212:30:132
3064397992825,2cyclictest0-21swapper/208:15:122
3064397992824,2cyclictest0-21swapper/210:03:592
306439799280,2cyclictest0-21swapper/208:30:122
306439799280,26cyclictest0-21swapper/212:27:002
306439799280,26cyclictest0-21swapper/208:45:132
3064392992825,2cyclictest0-21swapper/008:27:000
45782912724,1phc2sys0-21swapper/309:45:143
4577091270,6ptp4l61490irq/134-eno1-TxRx-010:55:013
4577091270,6ptp4l441ktimers/307:50:013
4577091270,5ptp4l441ktimers/310:10:003
3064399992725,1cyclictest0-21swapper/309:05:133
306439999270,26cyclictest0-21swapper/309:51:593
306439999270,26cyclictest0-21swapper/309:25:133
3064397992726,1cyclictest0-21swapper/209:00:002
306439799270,24cyclictest0-21swapper/209:13:592
306439799270,1cyclictest0-21swapper/211:29:592
4577091260,5ptp4l61490irq/134-eno1-TxRx-012:35:003
4577091260,5ptp4l61490irq/134-eno1-TxRx-010:30:013
4577091260,5ptp4l441ktimers/308:44:563
4577091260,5ptp4l441ktimers/308:44:563
306439799260,2cyclictest0-21swapper/210:52:592
4577091250,4ptp4l61490irq/134-eno1-TxRx-012:40:013
4577091250,3ptp4l441ktimers/308:40:013
3064397992522,2cyclictest0-21swapper/211:39:002
3064397992522,2cyclictest0-21swapper/211:38:592
3064397992421,2cyclictest0-21swapper/208:07:592
231999240,13rtkit-daemon3063170-21cstates07:05:151
3064397992320,2cyclictest0-21swapper/209:05:592
4577091220,3ptp4l61490irq/134-eno1-TxRx-011:24:563
3064397992220,1cyclictest0-21swapper/210:30:012
3064397992017,2cyclictest0-21swapper/210:55:592
306439299182,12cyclictest3064253-21latency_hist07:10:020
3064397991714,2cyclictest0-21swapper/209:40:592
3064397991614,1cyclictest0-21swapper/210:15:002
3064397991310,2cyclictest0-21swapper/209:35:142
3064397991310,2cyclictest0-21swapper/209:35:142
306439799129,2cyclictest0-21swapper/208:14:552
306439799107,2cyclictest0-21swapper/210:46:002
30643979997,1cyclictest0-21swapper/209:49:592
30643979996,2cyclictest0-21swapper/209:20:192
30643979985,2cyclictest0-21swapper/211:34:052
30643979940,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional