You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 16:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Nov 24, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3181900999988,8cyclictest3278856-21latency_hist12:10:021
3181900999982,15cyclictest3234262-21kworker/u8:1+flush-179:010:00:261
3181896999282,6cyclictest3202219-21kworker/u8:2+flush-179:009:05:240
3181900998679,5cyclictest3228518-21kworker/u8:4+flush-179:010:30:251
3181900998678,6cyclictest3283635-21latency_hist12:25:001
3181900998676,7cyclictest3199831-21/usr/sbin/munin08:05:241
3181896998665,19cyclictest3217082-21latency_hist09:00:010
4670791800,21ptp4l61690irq/136-eno1-TxRx-007:05:103
4670791800,21ptp4l61690irq/136-eno1-TxRx-007:05:093
3181896997867,9cyclictest3202219-21kworker/u8:2+flush-179:008:40:010
3181900997769,6cyclictest3236408-21kworker/u8:0+flush-179:010:25:261
3181900997361,9cyclictest271-21systemd-journal11:30:211
3181900997361,9cyclictest271-21systemd-journal11:30:201
3181896997262,7cyclictest3184988-21latency_hist07:20:020
3181896997059,9cyclictest3273815-21latency_hist11:55:010
3181896997049,19cyclictest271-21systemd-journal07:59:190
3181896996964,2cyclictest723-21Xorg09:01:000
3181900996863,3cyclictest3287212-21gpgv12:35:011
3181896996857,8cyclictest3260824-21latency_hist11:15:000
3181900996757,7cyclictest3257654-21latency_hist11:05:021
3181900996660,4cyclictest3212357-21kworker/u8:3+flush-179:008:50:001
3181900996658,6cyclictest3252907-21kworker/u8:3+events_unbound11:30:001
3181896996456,6cyclictest3282083-21kworker/u8:2+flush-179:012:25:240
3181900996354,6cyclictest3212357-21kworker/u8:3+events_unbound09:00:031
3181901996259,2cyclictest0-21swapper/209:01:002
3181900996256,3cyclictest723-21Xorg12:31:591
3181900996252,8cyclictest271-21systemd-journal07:30:001
3181905996158,2cyclictest0-21swapper/312:31:593
3181900996155,4cyclictest3249715-21kworker/u8:2+flush-179:010:50:001
3181900996154,5cyclictest3191354-21kworker/u8:1+flush-179:007:40:161
3181900996053,5cyclictest3202219-21kworker/u8:2+flush-179:008:20:231
3181896996050,7cyclictest3256054-21latency_hist11:00:010
3181900995953,4cyclictest3234262-21kworker/u8:1+flush-179:009:55:221
3181900995952,5cyclictest3212357-21kworker/u8:3+events_unbound08:55:291
3181896995948,8cyclictest3288677-21latency_hist12:40:010
3181900995849,7cyclictest3204354-21latency_hist08:20:011
3181896995853,4cyclictest3202219-21kworker/u8:2+flush-179:008:40:250
3181896995850,5cyclictest3188201-21kworker/u8:3+flush-179:007:35:260
3181900995752,4cyclictest3191354-21kworker/u8:1+flush-179:007:55:251
3181896995752,4cyclictest3197970-21kworker/u8:0+events_unbound08:14:560
3181900995648,5cyclictest3181828-21kworker/u8:3+events_unbound07:10:161
3181900995647,7cyclictest3223706-21latency_hist09:20:001
3181896995650,4cyclictest3204329-21kworker/u8:1+flush-179:008:20:000
3181896995550,4cyclictest3202219-21kworker/u8:2+flush-179:008:20:250
3181900995448,4cyclictest3195071-21apt-get07:50:121
3181900995447,6cyclictest3164564-21kworker/u8:0+flush-179:007:30:251
3181900995447,5cyclictest3188201-21kworker/u8:3+events_unbound07:40:011
3181896995449,4cyclictest3252907-21kworker/u8:3+flush-179:012:10:250
3181896995447,5cyclictest3202219-21kworker/u8:2+events_unbound08:30:250
3181896995445,7cyclictest3282048-21latency_hist12:20:020
3181900995346,5cyclictest3218409-21kworker/u8:0+events_unbound09:25:011
3181900995346,5cyclictest3202219-21kworker/u8:2+events_unbound09:39:591
3181896995344,7cyclictest3262463-21http11:20:020
3181896995343,7cyclictest3202219-21kworker/u8:2+flush-179:009:40:240
3181900995248,3cyclictest3272244-21kworker/u8:0+flush-179:011:50:271
3181900995246,5cyclictest3228518-21kworker/u8:4+events_unbound10:35:251
3181900995242,8cyclictest3240166-21diskstats10:10:171
3181900995240,11cyclictest3252905-21kworker/u8:1+flush-179:010:55:001
318190099521,47cyclictest1-21systemd11:20:011
3181896995247,4cyclictest3262441-21kworker/u8:1+flush-179:011:40:010
3181896995245,5cyclictest3191354-21kworker/u8:1+events_unbound08:00:230
3181900995146,4cyclictest3204329-21kworker/u8:1+flush-179:008:35:231
3181900995143,5cyclictest3212322-21latency_hist08:45:001
3181900995045,4cyclictest3236408-21kworker/u8:0+events_unbound10:20:011
3181900994943,5cyclictest3202219-21kworker/u8:2+flush-179:009:10:261
3181900994943,4cyclictest3164564-21kworker/u8:0+flush-179:007:15:161
3181896994939,8cyclictest3186582-21latency_hist07:25:000
3181900994840,6cyclictest3226894-21latency_hist09:30:011
3181896994840,7cyclictest3272244-21kworker/u8:0+flush-179:012:00:200
3181900994740,5cyclictest3270900-21apt-get11:45:131
3181900994735,11cyclictest3268830-21kworker/u8:0+flush-179:011:40:171
3181900994642,3cyclictest3279093-21kworker/u8:0+flush-179:012:10:231
3181900994640,4cyclictest271-21systemd-journal09:05:521
3181896994641,4cyclictest3218409-21kworker/u8:0+flush-179:009:20:170
3181900994539,4cyclictest3188201-21kworker/u8:3+events_unbound07:45:131
3181900994539,4cyclictest3188201-21kworker/u8:3+events_unbound07:45:131
3181900994538,5cyclictest3252907-21kworker/u8:3+flush-179:011:05:171
318190099450,42cyclictest3256062-21grep11:00:011
3181896994538,5cyclictest3246747-21kworker/u8:1+events_unbound10:40:250
3181896994537,6cyclictest3254459-21latency_hist10:55:000
3181900994440,3cyclictest3262441-21kworker/u8:1+flush-179:011:20:171
3181900994432,10cyclictest3228518-21kworker/u8:4+flush-179:009:50:011
3181896994438,5cyclictest3202219-21kworker/u8:2+flush-179:009:25:180
3181896994436,6cyclictest3193468-21/usr/sbin/munin07:45:160
3181896994436,6cyclictest3193468-21/usr/sbin/munin07:45:160
3181896994435,7cyclictest3228518-21kworker/u8:4+flush-179:010:00:000
3181905994341,1cyclictest0-21swapper/310:31:593
3181900994340,2cyclictest3202219-21kworker/u8:2+events_unbound08:30:191
3181900994339,3cyclictest3249715-21kworker/u8:2+flush-179:011:10:261
3181900994338,3cyclictest3197970-21kworker/u8:0+events_unbound08:15:001
3181896994339,3cyclictest3228518-21kworker/u8:4+flush-179:010:30:260
318189699431,40cyclictest723-21Xorg08:06:590
318189699430,41cyclictest3212336-21turbostat08:49:540
3181901994239,2cyclictest0-21swapper/210:14:562
3181901994239,2cyclictest0-21swapper/210:03:592
3181900994231,9cyclictest3238295-21apt-get10:05:131
3181896994236,4cyclictest3233514-21apt-get09:50:130
3181896994232,7cyclictest3239623-21latency_hist10:10:010
318189699420,41cyclictest0-21swapper/010:14:560
318189699420,39cyclictest723-21Xorg07:30:590
3181905994135,4cyclictest3246527-21turbostat10:30:013
3181900994136,4cyclictest3202219-21kworker/u8:2+flush-179:008:25:221
3181896994136,3cyclictest723-21Xorg11:27:000
318189699411,36cyclictest3227150-21apt-get09:30:130
3181905994038,1cyclictest0-21swapper/308:25:593
3181905994033,5cyclictest3202781-21turbostat08:15:003
3181905994031,6cyclictest3231694-21turbostat09:45:003
3181900994036,3cyclictest723-21Xorg07:24:591
3181900994034,4cyclictest3252907-21kworker/u8:3+flush-179:012:15:201
3181900994033,5cyclictest3204329-21kworker/u8:1+events_unbound08:50:121
318190099401,2cyclictest0-21swapper/111:56:591
3181905993936,2cyclictest0-21swapper/311:47:583
3181900993935,3cyclictest3282083-21kworker/u8:2+flush-179:012:25:231
3181900993934,4cyclictest3246747-21kworker/u8:1+flush-179:010:40:231
3181896993933,4cyclictest723-21Xorg11:03:590
3181896993932,6cyclictest3268830-21kworker/u8:0+events_unbound11:40:310
318189699390,38cyclictest0-21swapper/010:28:000
3181905993836,1cyclictest0-21swapper/309:46:593
3181905993833,4cyclictest3218950-21turbostat09:05:003
3181901993834,2cyclictest0-21swapper/207:30:592
318190199380,35cyclictest0-21swapper/209:50:592
3181896993834,3cyclictest3202219-21kworker/u8:2+flush-179:008:25:260
3181896993834,3cyclictest151ktimers/009:16:000
3181896993833,4cyclictest3252907-21kworker/u8:3+flush-179:011:55:260
3181905993734,2cyclictest0-21swapper/310:01:583
3181905993730,5cyclictest3257665-21turbostat11:05:013
318190599370,36cyclictest0-21swapper/311:50:593
3181901993734,2cyclictest0-21swapper/210:27:592
3181900993732,4cyclictest3202219-21kworker/u8:2+events_unbound09:30:241
3181900993729,5cyclictest1-21systemd09:45:011
318190099370,36cyclictest0-21swapper/110:20:591
3181896993732,4cyclictest3181892-21cyclictest07:45:010
3181896993732,4cyclictest3164564-21kworker/u8:0+events_unbound07:10:230
3181896993732,3cyclictest3259270-21systemd-run11:10:000
3181896993729,6cyclictest3218409-21kworker/u8:0+flush-179:009:10:210
3181896993725,8cyclictest3267451-21latency_hist11:35:000
3181896993725,8cyclictest3267451-21latency_hist11:35:000
318190599360,35cyclictest0-21swapper/310:36:583
318190599360,35cyclictest0-21swapper/310:24:553
318190599360,35cyclictest0-21swapper/307:24:593
318190599360,1cyclictest0-21swapper/311:59:563
3181900993632,3cyclictest3191354-21kworker/u8:1+flush-179:008:00:211
3181896993633,2cyclictest0-21swapper/010:15:580
318189699362,30cyclictest3277527-21apt-get12:05:130
202199360,4rtkit-daemon517-21gmain10:03:580
202199360,3rtkit-daemon685-21ntpd08:53:570
202199360,2rtkit-daemon0-21swapper/012:33:090
3181905993532,2cyclictest0-21swapper/308:54:593
3181901993531,2cyclictest0-21swapper/212:37:592
3181900993529,4cyclictest3234262-21kworker/u8:1+events_unbound09:50:251
202199350,2rtkit-daemon0-21swapper/012:23:350
3181901993431,2cyclictest0-21swapper/211:27:002
3181896993422,10cyclictest0-21swapper/009:40:000
4670791330,6ptp4l441ktimers/308:40:003
3181901993330,2cyclictest0-21swapper/211:24:552
3181900993323,6cyclictest0-21swapper/111:35:121
3181896993330,2cyclictest0-21swapper/007:51:000
3181905993230,1cyclictest0-21swapper/312:29:583
3181905993229,2cyclictest0-21swapper/312:14:583
3181905993227,3cyclictest3243082-21turbostat10:20:003
3181905993226,4cyclictest3217083-21turbostat09:00:013
3181901993229,2cyclictest0-21swapper/211:03:582
3181901993228,2cyclictest0-21swapper/210:15:592
3181900993223,6cyclictest0-21swapper/112:00:111
3181896993228,2cyclictest3233262-21sh09:50:010
3181896993223,7cyclictest502-21rs:main0
3181905993129,1cyclictest0-21swapper/311:33:583
3181905993129,1cyclictest0-21swapper/311:33:583
3181905993129,1cyclictest0-21swapper/307:10:593
3181905993128,2cyclictest0-21swapper/310:54:003
3181905993128,2cyclictest0-21swapper/307:39:003
318190199310,27cyclictest0-21swapper/212:05:132
318190199310,27cyclictest0-21swapper/211:30:132
318190199310,27cyclictest0-21swapper/211:30:122
318190199310,27cyclictest0-21swapper/210:40:142
318190199310,27cyclictest0-21swapper/207:48:582
318190199310,27cyclictest0-21swapper/207:48:582
3181896993128,2cyclictest0-21swapper/010:47:590
3181896993127,3cyclictest3181892-21cyclictest11:15:160
3181896993126,3cyclictest0-21swapper/011:45:140
3181905993028,1cyclictest0-21swapper/312:00:583
3181905993028,1cyclictest0-21swapper/309:33:593
318190599300,29cyclictest3212336-21turbostat08:45:003
3181901993028,1cyclictest0-21swapper/209:59:002
3181901993026,2cyclictest0-21swapper/211:57:592
3181901993026,2cyclictest0-21swapper/207:50:592
3181901993025,2cyclictest0-21swapper/207:20:112
318190199300,2cyclictest0-21swapper/209:15:142
318190199300,23cyclictest0-21swapper/212:20:142
318190199300,23cyclictest0-21swapper/209:35:152
318190199300,1cyclictest0-21swapper/208:45:152
3181905992926,2cyclictest0-21swapper/310:10:003
318190599290,5cyclictest3201184-21turbostat08:10:003
318190599290,28cyclictest0-21swapper/312:35:023
318190599290,27cyclictest0-21swapper/311:35:133
318190599290,27cyclictest0-21swapper/308:45:153
3181901992926,2cyclictest0-21swapper/210:50:592
3181901992926,2cyclictest0-21swapper/209:30:112
3181901992925,2cyclictest0-21swapper/207:40:592
3181901992925,2cyclictest0-21swapper/207:15:132
318190199290,27cyclictest0-21swapper/210:05:142
3181896992926,2cyclictest0-21swapper/010:35:580
3181896992920,6cyclictest3181773-21latency_hist07:10:010
3181896992920,6cyclictest3181773-21latency_hist07:10:010
3181905992826,1cyclictest0-21swapper/309:05:583
3181905992825,2cyclictest0-21swapper/309:25:593
318190599280,27cyclictest3259244-21turbostat11:10:003
318190599280,26cyclictest0-21swapper/312:20:133
318190599280,26cyclictest0-21swapper/310:45:143
3181901992823,3cyclictest0-21swapper/212:31:592
318190199280,2cyclictest0-21swapper/212:10:152
318190199280,2cyclictest0-21swapper/211:08:592
318190199280,26cyclictest0-21swapper/212:18:592
4670791270,6ptp4l441ktimers/308:00:013
4670791270,6ptp4l441ktimers/307:45:013
3181905992725,1cyclictest0-21swapper/310:13:003
3181905992725,1cyclictest0-21swapper/308:03:593
3181905992724,2cyclictest0-21swapper/309:50:143
3181905992724,2cyclictest0-21swapper/308:24:593
318190599270,26cyclictest0-21swapper/311:29:003
318190599270,25cyclictest3251300-21turbostat10:44:593
318190599270,25cyclictest0-21swapper/307:25:443
318190599270,1cyclictest0-21swapper/307:19:583
3181901992724,2cyclictest0-21swapper/211:45:592
3181901992724,2cyclictest0-21swapper/211:12:002
3181901992724,2cyclictest0-21swapper/210:24:002
3181901992723,2cyclictest0-21swapper/210:47:582
318190199270,25cyclictest0-21swapper/210:37:592
318190199270,24cyclictest0-21swapper/208:29:592
3181896992724,2cyclictest0-21swapper/007:26:270
4670791260,6ptp4l441ktimers/308:34:563
3181905992624,1cyclictest0-21swapper/309:10:143
3181905992624,1cyclictest0-21swapper/307:45:133
3181905992624,1cyclictest0-21swapper/307:45:133
3181905992623,2cyclictest3270657-21turbostat11:45:003
318190599260,24cyclictest3278869-21turbostat12:10:003
3181901992623,2cyclictest0-21swapper/209:10:582
3181901992623,2cyclictest0-21swapper/208:02:592
318190199260,2cyclictest0-21swapper/209:45:002
318190199260,2cyclictest0-21swapper/208:35:142
318190199260,2cyclictest0-21swapper/207:13:592
318190199260,24cyclictest0-21swapper/210:34:592
318190199260,24cyclictest0-21swapper/207:25:402
3181900992619,5cyclictest0-21swapper/107:10:011
4670791250,6ptp4l441ktimers/312:20:013
4670791250,6ptp4l441ktimers/309:39:553
4670791250,3ptp4l441ktimers/309:25:013
318190599250,24cyclictest3262427-21turbostat11:20:003
318190599250,24cyclictest0-21swapper/311:23:003
318190599250,24cyclictest0-21swapper/308:18:593
318190599250,18cyclictest0-21swapper/307:32:143
3181901992523,1cyclictest0-21swapper/209:50:002
3181901992522,1cyclictest0-21swapper/211:54:592
318190199250,23cyclictest0-21swapper/208:34:592
4670791240,5ptp4l61690irq/136-eno1-TxRx-011:14:543
4670791240,4ptp4l441ktimers/309:59:553
4670791240,4ptp4l441ktimers/307:54:563
3181901992421,2cyclictest0-21swapper/211:38:592
3181901992421,2cyclictest0-21swapper/211:15:582
3181901992421,2cyclictest0-21swapper/208:24:552
3181901992421,2cyclictest0-21swapper/207:57:592
4670791230,4ptp4l61690irq/136-eno1-TxRx-009:19:563
4670791230,4ptp4l441ktimers/310:59:553
3181901992321,1cyclictest0-21swapper/209:30:002
3181901992320,2cyclictest0-21swapper/208:55:592
3181901992219,2cyclictest0-21swapper/210:58:002
3181901992119,1cyclictest0-21swapper/211:44:592
3181901992118,2cyclictest0-21swapper/208:50:592
3181901992017,2cyclictest0-21swapper/208:40:592
3181901992017,2cyclictest0-21swapper/207:35:592
3181900991915,2cyclictest3182015-21latency_hist07:10:011
3181901991815,2cyclictest0-21swapper/209:06:592
3181901991714,2cyclictest0-21swapper/208:07:592
3181901991612,2cyclictest0-21swapper/208:16:592
318190199129,2cyclictest0-21swapper/212:30:002
31819019996,2cyclictest0-21swapper/208:10:152
31819019980,7cyclictest0-21swapper/209:22:452
31819019970,1cyclictest0-21swapper/212:02:302
31819019940,3cyclictest0-21swapper/207:10:022
31819019940,3cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional