You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 09:21
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Thu Dec 11, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31205459910089,3cyclictest3120485-21kworker/u8:3+events_unbound19:20:261
3120545999691,4cyclictest3185387-21kworker/u8:3+events_unbound22:30:251
3120545999488,4cyclictest3174292-21kworker/u8:1+events_unbound23:00:231
3120544999183,5cyclictest3159204-21kworker/u8:3+flush-179:021:35:260
3120545998983,5cyclictest3206867-21kworker/u8:0+events_unbound23:35:241
3120545998881,5cyclictest3203940-21kworker/u8:3+events_unbound23:30:221
3120545998780,5cyclictest3203940-21kworker/u8:3+flush-179:000:00:241
3120545998780,5cyclictest3155218-21kworker/u8:0+events_unbound21:10:241
3120544998678,5cyclictest3222865-21latency_hist00:25:010
3120544998574,7cyclictest3131843-21latency_hist19:45:010
3120545998479,4cyclictest3123607-21latency_hist19:20:021
3120545998469,5cyclictest3177174-21kworker/u8:2+events_unbound22:05:001
3120544998473,8cyclictest3157574-21latency_hist21:05:010
3120545998371,4cyclictest3152465-21kworker/u8:2+events_unbound20:55:231
3120545998265,5cyclictest3120485-21kworker/u8:3+events_unbound20:20:141
3120545998175,4cyclictest3120485-21kworker/u8:3+events_unbound19:35:021
3120545998074,4cyclictest3225343-21kworker/u8:0+flush-179:000:35:261
3120545998063,5cyclictest3174292-21kworker/u8:1+events_unbound22:50:151
3120545997962,4cyclictest3203940-21kworker/u8:3+events_unbound00:25:211
3120545997870,6cyclictest3131888-21kworker/u8:1+flush-179:019:55:011
3120545997772,4cyclictest3218074-21kworker/u8:0+flush-179:000:10:251
3120545997771,4cyclictest3174292-21kworker/u8:1+flush-179:022:35:201
3120545997767,7cyclictest3226099-21latency_hist00:35:011
3120544997769,5cyclictest3174292-21kworker/u8:1+flush-179:022:55:260
3120545997667,7cyclictest3168720-21latency_hist21:40:011
3120544997670,4cyclictest3140833-21kworker/u8:0+flush-179:020:40:000
3120550997563,10cyclictest3227708-21turbostat00:40:003
3120545997567,5cyclictest3155218-21kworker/u8:0+events_unbound21:00:151
3120544997565,7cyclictest3159204-21kworker/u8:3+flush-179:021:30:250
3120545997469,3cyclictest3170288-21kworker/u8:0+flush-179:021:50:011
3120544997460,11cyclictest3198143-21latency_hist23:10:010
3120545997364,7cyclictest3172188-21kworker/u8:2+events_unbound21:55:161
3120545997362,8cyclictest3130244-21latency_hist19:40:011
3120544997367,4cyclictest3118945-21kworker/u8:2+flush-179:019:20:000
3120544997362,8cyclictest3193101-21latency_hist22:55:010
3120544997266,4cyclictest3170288-21kworker/u8:0+events_unbound22:00:000
3120545997167,3cyclictest3120485-21kworker/u8:3+events_unbound20:45:151
3120545997163,5cyclictest3140833-21kworker/u8:0+events_unbound20:35:151
3120545997065,4cyclictest3203940-21kworker/u8:3+events_unbound23:55:231
3120545997065,4cyclictest3140833-21kworker/u8:0+flush-179:020:40:231
3120545997062,6cyclictest3203940-21kworker/u8:3+events_unbound23:25:211
3120545997062,6cyclictest3174292-21kworker/u8:1+events_unbound22:20:201
3120545996863,3cyclictest3170288-21kworker/u8:0+flush-179:022:25:161
3120545996862,3cyclictest3120485-21kworker/u8:3+events_unbound20:25:171
312054599680,2cyclictest723-21Xorg21:09:591
3120544996861,5cyclictest3183598-21kworker/u8:2+events_unbound22:40:120
3120544996861,5cyclictest3174292-21kworker/u8:1+flush-179:022:00:150
3120545996762,4cyclictest3120485-21kworker/u8:3+events_unbound19:10:231
3120545996660,4cyclictest3170288-21kworker/u8:0+events_unbound22:15:161
3120545996659,6cyclictest3155218-21kworker/u8:0+flush-179:021:15:201
3120545996659,5cyclictest3159204-21kworker/u8:3+events_unbound21:25:191
3120545996658,5cyclictest3120485-21kworker/u8:3+flush-179:020:50:221
3120545996657,5cyclictest3167097-21kworker/u8:2+flush-179:021:40:211
3120544996657,4cyclictest3138257-21kworker/u8:1+flush-179:020:10:000
3120545996560,4cyclictest3174292-21kworker/u8:1+events_unbound22:45:241
3120545996559,4cyclictest3188388-21kworker/u8:0+events_unbound23:05:251
3120545996556,6cyclictest3177174-21kworker/u8:2+flush-179:022:05:161
3120545996548,15cyclictest3120485-21kworker/u8:3+flush-179:020:00:251
3120545996460,3cyclictest3135565-21kworker/u8:0+events_unbound19:55:241
3120545996459,4cyclictest3118945-21kworker/u8:2+events_unbound19:25:221
3120545996455,7cyclictest3159204-21kworker/u8:3+events_unbound21:20:231
3120544996451,4cyclictest3225343-21kworker/u8:0+flush-179:000:35:150
3120545996356,5cyclictest3136587-21kworker/u8:2+events_unbound20:05:211
3120545996258,3cyclictest3193137-21kworker/u8:2+flush-179:023:10:241
3120545996256,4cyclictest3214807-21kworker/u8:2+events_unbound00:05:171
3120544996153,5cyclictest3170288-21kworker/u8:0+events_unbound22:25:240
3120544996149,9cyclictest271-21systemd-journal21:45:000
3120544996054,4cyclictest3203940-21kworker/u8:3+events_unbound23:45:190
3120544996054,4cyclictest3159204-21kworker/u8:3+events_unbound21:50:000
3120544996051,6cyclictest3213243-21latency_hist23:55:010
3120544995950,6cyclictest3186737-21latency_hist22:35:010
3120545995851,5cyclictest3188388-21kworker/u8:0+events_unbound23:20:271
3120544995848,8cyclictest3216752-21apt-get00:05:150
3120545995751,4cyclictest3138257-21kworker/u8:1+flush-179:020:10:221
3120545995749,4cyclictest3120485-21kworker/u8:3+events_unbound20:15:261
3120545995651,4cyclictest3120485-21kworker/u8:3+events_unbound19:40:171
3120545995647,7cyclictest3183598-21kworker/u8:2+events_unbound22:40:191
3120544995649,5cyclictest3135565-21kworker/u8:0+events_unbound20:00:020
3120545995551,3cyclictest3170288-21kworker/u8:0+flush-179:021:50:261
3120544995548,5cyclictest3155218-21kworker/u8:0+events_unbound21:25:010
3120544995544,9cyclictest3125728-21apt-get19:25:130
3120544995449,3cyclictest3174292-21kworker/u8:1+flush-179:023:20:170
3120544995448,4cyclictest3205288-21latency_hist23:30:020
3120544995447,5cyclictest3174292-21kworker/u8:1+flush-179:022:45:210
3120545995348,4cyclictest3218074-21kworker/u8:0+flush-179:000:15:221
3120544995247,4cyclictest3120485-21kworker/u8:3+flush-179:020:50:180
3120544995244,6cyclictest3120485-21kworker/u8:3+events_unbound19:45:220
3120545995145,5cyclictest3203940-21kworker/u8:3+events_unbound23:45:241
3120544995145,4cyclictest3188388-21kworker/u8:0+flush-179:023:05:000
3120544995144,6cyclictest3193137-21kworker/u8:2+flush-179:023:10:220
3120544995143,5cyclictest3120485-21kworker/u8:3+flush-179:019:10:160
3120545995045,4cyclictest3203940-21kworker/u8:3+flush-179:023:50:211
3120544995044,4cyclictest3203940-21kworker/u8:3+flush-179:000:30:170
4670791490,13ptp4l441ktimers/319:10:013
3120544994943,4cyclictest3120485-21kworker/u8:3+flush-179:020:20:010
3120545994844,3cyclictest3218074-21kworker/u8:0+flush-179:000:20:161
3120545994844,3cyclictest3188388-21kworker/u8:0+flush-179:022:55:251
3120545994844,3cyclictest3159204-21kworker/u8:3+flush-179:021:30:291
3120544994844,3cyclictest3120485-21kworker/u8:3+flush-179:019:20:270
3120544994842,5cyclictest3120485-21kworker/u8:3+events_unbound20:45:160
3120545994739,6cyclictest3203940-21kworker/u8:3+flush-179:023:40:181
3120545994642,3cyclictest3170288-21kworker/u8:0+flush-179:022:10:261
3120544994640,4cyclictest3140833-21kworker/u8:0+flush-179:020:30:170
3120544994640,4cyclictest3131888-21kworker/u8:1+events_unbound19:55:000
3120544994540,4cyclictest3174292-21kworker/u8:1+events_unbound22:35:230
3120544994540,4cyclictest3136587-21kworker/u8:2+events_unbound20:10:240
3120544994538,6cyclictest3155218-21kworker/u8:0+events_unbound21:05:200
3120550994442,1cyclictest0-21swapper/321:16:003
3120548994441,2cyclictest0-21swapper/223:15:592
3120550994341,1cyclictest0-21swapper/323:29:593
3120545994338,4cyclictest3147159-21kworker/u8:1+flush-179:020:30:261
3120545994334,6cyclictest502-21rs:main1
3120544994338,4cyclictest3218074-21kworker/u8:0+events_unbound00:10:240
312054499431,5cyclictest3224520-21perf00:30:010
312054499431,40cyclictest3155966-21sendmail21:00:010
312055099420,5cyclictest0-21swapper/319:58:593
3120544994237,4cyclictest3177174-21kworker/u8:2+flush-179:022:05:220
3120544994232,8cyclictest3219924-21apt-get00:15:120
312055099410,38cyclictest3180376-21turbostat22:15:013
312054899410,39cyclictest0-21swapper/221:36:582
3120544994138,2cyclictest0-21swapper/023:55:590
3120544994136,3cyclictest3130264-21sendmail-msp19:40:000
312055099400,39cyclictest0-21swapper/300:24:593
3120544994035,4cyclictest3120485-21kworker/u8:3+flush-179:019:30:240
3120544994034,4cyclictest3206867-21kworker/u8:0+events_unbound23:43:230
3120544994031,7cyclictest3179037-21apt-get22:10:140
312054499400,3cyclictest3165540-21ls21:30:000
312054499400,38cyclictest723-21Xorg21:16:590
3120550993937,1cyclictest0-21swapper/300:34:593
3120548993935,2cyclictest0-21swapper/222:07:592
3120545993934,4cyclictest3120485-21kworker/u8:3+flush-179:019:45:141
3120544993933,5cyclictest3140833-21kworker/u8:0+flush-179:020:40:200
3120544993933,4cyclictest3160748-21munin-run21:14:590
3120548993834,2cyclictest0-21swapper/219:48:002
312054899380,36cyclictest0-21swapper/219:19:552
312054899380,35cyclictest0-21swapper/221:59:552
312054899380,29cyclictest0-21swapper/221:35:002
3120544993832,4cyclictest3136587-21kworker/u8:2+flush-179:020:05:000
4670791370,4ptp4l431rcuc/319:49:593
3120550993734,2cyclictest0-21swapper/323:54:563
3120548993734,2cyclictest0-21swapper/222:32:582
3120548993734,2cyclictest0-21swapper/221:11:592
3120548993734,2cyclictest0-21swapper/220:41:592
312054899370,35cyclictest0-21swapper/220:29:592
312054899370,35cyclictest0-21swapper/219:26:002
3120544993730,5cyclictest3174292-21kworker/u8:1+events_unbound23:15:190
3120544993726,9cyclictest3144834-21apt-get20:25:140
3120550993634,1cyclictest0-21swapper/321:20:593
3120550993633,2cyclictest0-21swapper/320:19:563
3120550993633,2cyclictest0-21swapper/319:19:583
312055099360,35cyclictest0-21swapper/323:34:563
312055099360,1cyclictest0-21swapper/321:35:593
312055099360,1cyclictest0-21swapper/320:38:593
3120548993633,2cyclictest0-21swapper/220:05:592
3120548993633,2cyclictest0-21swapper/219:51:592
312054899360,1cyclictest0-21swapper/200:23:592
3120544993627,7cyclictest3216498-21latency_hist00:05:000
3120550993532,2cyclictest0-21swapper/320:47:593
3120550993532,2cyclictest0-21swapper/320:01:593
312055099350,33cyclictest0-21swapper/300:15:143
3120548993532,2cyclictest0-21swapper/200:03:592
3120544993530,4cyclictest3203940-21kworker/u8:3+events_unbound23:35:170
3120544993527,6cyclictest3203940-21kworker/u8:3+events_unbound23:30:150
3120550993432,1cyclictest0-21swapper/320:52:593
3120548993431,2cyclictest0-21swapper/223:23:592
3120550993331,1cyclictest0-21swapper/323:15:583
3120550993331,1cyclictest0-21swapper/323:11:003
3120550993331,1cyclictest0-21swapper/321:44:553
312055099330,31cyclictest0-21swapper/323:35:133
3120548993331,1cyclictest0-21swapper/219:39:592
3120550993230,1cyclictest0-21swapper/323:45:593
3120548993230,1cyclictest0-21swapper/222:19:592
3120548993230,1cyclictest0-21swapper/221:24:592
3120548993229,2cyclictest0-21swapper/222:29:552
3120548993228,2cyclictest0-21swapper/223:08:592
202199320,23rtkit-daemon1926816-21munin-node19:06:550
3120550993129,1cyclictest0-21swapper/322:26:593
3120550993129,1cyclictest0-21swapper/319:44:003
3120550993129,1cyclictest0-21swapper/300:29:593
3120550993128,2cyclictest0-21swapper/322:21:593
312055099310,29cyclictest0-21swapper/323:00:583
3120548993128,2cyclictest0-21swapper/222:45:132
312054899310,23cyclictest0-21swapper/222:00:032
3120544993125,5cyclictest3170288-21kworker/u8:0+events_unbound21:50:230
3120550993023,5cyclictest3146171-21turbostat20:30:003
312055099300,28cyclictest0-21swapper/319:35:123
3120548993027,2cyclictest0-21swapper/223:35:142
3120548993027,2cyclictest0-21swapper/220:20:122
3120548993027,2cyclictest0-21swapper/219:30:132
3120548993027,2cyclictest0-21swapper/219:24:592
312054899300,2cyclictest0-21swapper/223:25:022
312054899300,28cyclictest0-21swapper/223:45:122
4670791290,7ptp4l61690irq/136-eno1-TxRx-019:30:003
4670791290,7ptp4l441ktimers/320:10:013
3120550992927,1cyclictest0-21swapper/321:59:003
3120550992927,1cyclictest0-21swapper/320:23:593
3120550992927,1cyclictest0-21swapper/319:20:153
3120550992927,1cyclictest0-21swapper/319:14:593
3120550992926,2cyclictest3194701-21turbostat23:00:013
3120550992926,2cyclictest0-21swapper/320:55:153
312055099290,1cyclictest0-21swapper/323:20:143
3120548992926,2cyclictest0-21swapper/223:33:592
3120548992926,2cyclictest0-21swapper/221:50:152
3120548992926,2cyclictest0-21swapper/221:03:592
3120548992926,2cyclictest0-21swapper/220:00:152
3120548992926,2cyclictest0-21swapper/200:15:142
312054899290,27cyclictest0-21swapper/223:57:592
312054899290,27cyclictest0-21swapper/223:50:162
312054899290,27cyclictest0-21swapper/222:20:132
312054899290,27cyclictest0-21swapper/221:25:142
312054899290,27cyclictest0-21swapper/220:35:142
312054899290,27cyclictest0-21swapper/220:30:142
3120544992922,5cyclictest3170288-21kworker/u8:0+flush-179:022:24:590
3120550992825,2cyclictest0-21swapper/321:32:593
3120550992825,2cyclictest0-21swapper/321:05:593
312055099280,27cyclictest0-21swapper/321:01:003
312055099280,26cyclictest0-21swapper/322:45:143
312055099280,26cyclictest0-21swapper/322:05:133
312055099280,26cyclictest0-21swapper/319:50:113
3120548992825,2cyclictest0-21swapper/222:13:592
312054899280,26cyclictest0-21swapper/220:55:032
312054899280,26cyclictest0-21swapper/200:40:002
312054899280,26cyclictest0-21swapper/200:10:142
4670791270,7ptp4l441ktimers/322:35:013
4670791270,6ptp4l441ktimers/323:45:003
4670791270,6ptp4l441ktimers/322:20:003
3120550992725,1cyclictest0-21swapper/323:55:123
3120550992725,1cyclictest0-21swapper/320:13:593
312055099270,26cyclictest0-21swapper/321:51:593
312055099270,1cyclictest0-21swapper/322:00:153
3120548992724,2cyclictest0-21swapper/223:41:592
312054899270,26cyclictest0-21swapper/221:49:592
3120544992721,4cyclictest0-21swapper/020:20:110
4670791260,6ptp4l441ktimers/320:35:013
3120550992624,1cyclictest0-21swapper/322:50:593
312055099260,25cyclictest0-21swapper/321:30:003
3120548992625,1cyclictest0-21swapper/223:04:592
3120548992623,2cyclictest0-21swapper/219:44:592
3120548992623,1cyclictest0-21swapper/222:45:012
312054899260,24cyclictest0-21swapper/219:10:262
312054899260,1cyclictest0-21swapper/200:30:002
4670791250,6ptp4l61690irq/136-eno1-TxRx-021:15:013
4670791250,6ptp4l441ktimers/300:09:563
4670791250,5ptp4l61690irq/136-eno1-TxRx-019:34:553
4670791250,5ptp4l441ktimers/321:50:013
3120550992523,1cyclictest0-21swapper/323:07:003
3120550992523,1cyclictest0-21swapper/300:12:583
3120548992522,2cyclictest0-21swapper/222:51:592
3120548992522,2cyclictest0-21swapper/220:17:582
3120548992522,2cyclictest0-21swapper/219:55:592
4670791240,6ptp4l441ktimers/322:40:013
4670791230,4ptp4l441ktimers/322:45:003
4670791230,4ptp4l441ktimers/300:04:553
3120548992321,1cyclictest0-21swapper/221:45:002
3120548992321,1cyclictest0-21swapper/200:35:002
4670791220,4ptp4l441ktimers/320:44:553
3120544992220,1cyclictest0-21swapper/022:15:120
3120548992119,1cyclictest0-21swapper/222:36:592
3120548992118,2cyclictest0-21swapper/221:06:592
3120548992018,1cyclictest0-21swapper/220:50:002
3120548992018,1cyclictest0-21swapper/220:12:592
3120548991916,2cyclictest0-21swapper/223:13:002
3120548991613,2cyclictest0-21swapper/200:05:592
312054599160,15cyclictest3120404-21latency_hist19:10:011
3120548991311,1cyclictest0-21swapper/222:58:592
312054899129,2cyclictest0-21swapper/220:50:592
312054899128,3cyclictest0-21swapper/221:16:002
31205489930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional