You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Sat Jul 12, 2025 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
210993199131119,5cyclictest2204742-21kworker/u8:1+events_unbound00:04:020
210993199111105,5cyclictest2135585-21kworker/u8:0+events_unbound20:30:020
21099339910289,10cyclictest2203151-40logrotate00:00:011
21099339910289,10cyclictest2203151-40logrotate00:00:011
21099319910095,4cyclictest2186947-21kworker/u8:0+events_unbound23:10:280
2109933999889,6cyclictest2119420-21kworker/u8:2+events_unbound19:45:291
2109933999889,6cyclictest2119420-21kworker/u8:2+events_unbound19:45:281
2109931999686,8cyclictest2207961-21latency_hist00:15:010
2109933999584,8cyclictest2121197-21kworker/u8:1+flush-179:020:00:001
2109933999584,4cyclictest2174276-21kworker/u8:2+events_unbound22:30:151
2109933999486,6cyclictest2109827-21kworker/u8:0+events_unbound19:25:251
2109933999486,6cyclictest2109827-21kworker/u8:0+events_unbound19:20:001
2109933999385,5cyclictest2211199-21kworker/u8:2+flush-179:000:30:231
2109933999181,4cyclictest2165483-21kworker/u8:0+events_unbound22:15:291
2109931999185,4cyclictest2156770-21kworker/u8:2+flush-179:021:35:230
2109933999083,5cyclictest2190126-21kworker/u8:3+events_unbound23:25:131
2109933999081,7cyclictest3105085-21systemd-journal20:45:011
2109933999077,10cyclictest2150361-21apt-get21:15:131
2109931999085,4cyclictest2179063-21kworker/u8:3+events_unbound22:50:190
2109931999085,4cyclictest2109827-21kworker/u8:0+events_unbound19:10:240
2109931999081,6cyclictest2190126-21kworker/u8:3+events_unbound23:25:190
2109931998978,9cyclictest2195435-21apt-get23:35:160
2109931998884,3cyclictest2135589-21kworker/u8:1+events_unbound21:20:280
2109931998876,10cyclictest2130930-21cat20:15:020
2109931998876,10cyclictest2113266-21apt-get19:20:090
2109933998781,4cyclictest2109827-21kworker/u8:0+events_unbound19:20:131
2109933998777,8cyclictest2174276-21kworker/u8:2+events_unbound23:00:191
2109933998775,6cyclictest2195160-21kworker/u8:2+events_unbound23:45:011
2109933998774,5cyclictest2179063-21kworker/u8:3+events_unbound22:50:241
2109931998776,8cyclictest2151704-21latency_hist21:20:020
2109933998676,8cyclictest2184458-21kworker/u8:1+flush-179:023:15:001
2109933998676,8cyclictest2183797-21latency_hist23:00:011
2109933998675,8cyclictest2124130-21latency_hist19:55:011
2109933998674,6cyclictest2133934-21kworker/u8:2+events_unbound20:25:241
2109933998574,8cyclictest2212741-21latency_hist00:30:001
2109931998581,3cyclictest2172060-21kworker/u8:3+events_unbound22:25:230
2109933998472,9cyclictest2130619-21gpgv20:15:021
2109931998479,4cyclictest2148565-21kworker/u8:0+flush-179:021:25:280
2109931998472,10cyclictest2122800-21apt-get19:50:120
2109933998378,3cyclictest2143787-21kworker/u8:3+flush-179:021:20:251
2109933998372,8cyclictest2159905-21kworker/u8:1+flush-179:021:45:221
2109931998375,6cyclictest2106841-21kworker/u8:3+events_unbound20:25:190
2109931998371,10cyclictest2139243-21apt-get20:40:130
2109933998276,5cyclictest2143787-21kworker/u8:3+flush-179:021:05:241
2109933998276,4cyclictest2163133-21kworker/u8:1+flush-179:022:40:241
2109933998176,3cyclictest2190126-21kworker/u8:3+events_unbound23:35:281
2109933997968,9cyclictest2110071-21apt-get19:10:111
2109931997974,3cyclictest2156770-21kworker/u8:2+events_unbound21:40:150
2109931997968,4cyclictest2124164-21kworker/u8:0+events_unbound19:55:280
2109933997867,5cyclictest2199987-21kworker/u8:3+events_unbound00:15:211
2109931997873,4cyclictest2106841-21kworker/u8:3+flush-179:019:35:210
2109933997771,5cyclictest2165483-21kworker/u8:0+flush-179:022:05:201
2109931997772,3cyclictest2106841-21kworker/u8:3+events_unbound19:30:160
2109931997771,5cyclictest2148565-21kworker/u8:0+flush-179:021:35:010
2109931997771,4cyclictest2124164-21kworker/u8:0+events_unbound20:20:230
2109933997666,5cyclictest2163133-21kworker/u8:1+events_unbound22:00:241
2109933997665,8cyclictest2143743-21latency_hist20:55:011
2109931997672,3cyclictest2182424-21kworker/u8:0+events_unbound22:55:260
2109933997567,6cyclictest2184458-21kworker/u8:1+events_unbound23:45:231
2109931997571,3cyclictest2109827-21kworker/u8:0+events_unbound19:15:230
2109931997569,4cyclictest2143787-21kworker/u8:3+events_unbound21:05:140
2109931997566,3cyclictest2161503-21kworker/u8:0+events_unbound21:50:250
2109933997469,3cyclictest2109827-21kworker/u8:0+flush-179:019:40:251
2109933997463,4cyclictest2161503-21kworker/u8:0+flush-179:021:50:271
2109931997470,3cyclictest2199987-21kworker/u8:3+events_unbound00:15:250
210993199740,73cyclictest1-21systemd22:50:000
2109931997367,4cyclictest2156770-21kworker/u8:2+events_unbound22:05:140
2109933997267,4cyclictest2156770-21kworker/u8:2+events_unbound21:55:181
2109933997265,6cyclictest2135589-21kworker/u8:1+flush-179:021:15:011
2109933997263,6cyclictest2211199-21kworker/u8:2+events_unbound00:35:171
2109931997264,6cyclictest2184458-21kworker/u8:1+events_unbound23:20:130
2109931997262,8cyclictest2177432-21latency_hist22:40:000
2109933997165,4cyclictest2174276-21kworker/u8:2+events_unbound23:15:151
2109933997164,5cyclictest2190126-21kworker/u8:3+events_unbound23:30:221
2109933997163,5cyclictest2174276-21kworker/u8:2+events_unbound23:05:141
2109933997158,5cyclictest2106841-21kworker/u8:3+events_unbound20:15:221
2109933997157,6cyclictest2195160-21kworker/u8:2+flush-179:000:00:151
2109931997164,5cyclictest2199987-21kworker/u8:3+events_unbound00:20:120
2109933997065,4cyclictest2139023-21kworker/u8:2+flush-179:020:45:261
2109931997066,3cyclictest2143787-21kworker/u8:3+flush-179:021:10:210
2109931996965,3cyclictest2184458-21kworker/u8:1+events_unbound23:55:200
2109931996965,3cyclictest2184458-21kworker/u8:1+events_unbound23:55:200
2109931996965,3cyclictest2156770-21kworker/u8:2+events_unbound22:20:140
2109931996964,4cyclictest2203111-21kworker/u8:0+events_unbound00:25:220
2109931996957,9cyclictest2201539-21latency_hist23:55:010
2109933996862,4cyclictest2106841-21kworker/u8:3+events_unbound19:30:231
2109933996861,5cyclictest2184458-21kworker/u8:1+flush-179:023:20:211
2109931996864,3cyclictest2163133-21kworker/u8:1+flush-179:022:30:280
2109931996864,3cyclictest2121197-21kworker/u8:1+flush-179:020:05:270
2109931996862,5cyclictest2124164-21kworker/u8:0+events_unbound20:12:100
2109931996862,4cyclictest2121197-21kworker/u8:1+events_unbound20:00:180
2109933996756,9cyclictest3105085-21systemd-journal20:35:001
2109931996760,5cyclictest2106841-21kworker/u8:3+flush-179:020:40:000
2109933996661,4cyclictest2209531-21kworker/u8:1+events_unbound00:20:181
2109933996661,3cyclictest2179063-21kworker/u8:3+flush-179:022:45:211
2109933996656,6cyclictest2172060-21kworker/u8:3+flush-179:022:25:201
2109933996654,4cyclictest2135589-21kworker/u8:1+flush-179:020:35:151
2109931996662,3cyclictest2195160-21kworker/u8:2+events_unbound23:40:130
2109931996656,7cyclictest2206377-21latency_hist00:10:010
2109937996561,2cyclictest0-21swapper/220:11:592
2109933996560,4cyclictest2148565-21kworker/u8:0+flush-179:021:35:151
2109933996550,5cyclictest2148565-21kworker/u8:0+flush-179:021:25:211
2109931996561,3cyclictest2190126-21kworker/u8:3+events_unbound23:30:200
2109931996561,3cyclictest2163139-21kworker/u8:3+flush-179:021:57:280
2109931996561,3cyclictest2156770-21kworker/u8:2+events_unbound22:10:260
2109931996455,7cyclictest2215921-21latency_hist00:40:010
2109931996453,4cyclictest2174276-21kworker/u8:2+events_unbound23:06:280
2109933996358,4cyclictest2121197-21kworker/u8:1+flush-179:020:05:231
2109931996358,4cyclictest2135589-21kworker/u8:1+events_unbound20:47:170
2109933996257,4cyclictest2163133-21kworker/u8:1+flush-179:022:35:271
2109933996256,4cyclictest2156770-21kworker/u8:2+flush-179:022:20:151
2109933996252,8cyclictest2169460-21latency_hist22:15:001
2109933996251,4cyclictest2109827-21kworker/u8:0+events_unbound19:35:241
2109931996258,3cyclictest2163133-21kworker/u8:1+events_unbound22:00:290
2109931996258,3cyclictest2143787-21kworker/u8:3+flush-179:020:59:560
2109931996258,3cyclictest2143787-21kworker/u8:3+flush-179:020:59:550
2109931996257,4cyclictest2142136-21kworker/u8:0+events_unbound20:50:240
2109931996257,3cyclictest2184458-21kworker/u8:1+events_unbound23:45:100
2109931996256,4cyclictest2159905-21kworker/u8:1+events_unbound21:45:110
2109931996156,4cyclictest2174276-21kworker/u8:2+events_unbound23:15:170
2109931996156,4cyclictest2114640-21kworker/u8:1+flush-179:019:26:290
2109931996056,3cyclictest2213255-21kworker/u8:1+flush-179:000:30:270
2109931996048,10cyclictest2145604-21apt-get21:00:140
2109931995955,3cyclictest2174276-21kworker/u8:2+flush-179:022:44:550
2109933995854,3cyclictest2142136-21kworker/u8:0+flush-179:021:00:151
2109942995552,2cyclictest0-21swapper/320:12:003
2109933995550,4cyclictest2184458-21kworker/u8:1+flush-179:023:50:231
2109933995448,4cyclictest2106841-21kworker/u8:3+events_unbound20:20:181
2109937995350,2cyclictest0-21swapper/223:06:592
2109933995247,4cyclictest2205029-21kworker/u8:4+events_unbound00:06:291
2109931994945,3cyclictest2109827-21kworker/u8:0+events_unbound19:40:290
4505291480,3ptp4l61790irq/134-eno1-TxRx-021:27:583
4505291480,13ptp4l441ktimers/319:09:563
2109933994843,4cyclictest2142136-21kworker/u8:0+events_unbound20:57:181
2109933994843,4cyclictest2142136-21kworker/u8:0+events_unbound20:57:181
2109933994843,4cyclictest2121197-21kworker/u8:1+events_unbound20:02:081
2109931994743,3cyclictest2106841-21kworker/u8:3+events_unbound19:48:290
2109931994743,3cyclictest2106841-21kworker/u8:3+events_unbound19:48:290
2109931994535,8cyclictest2184051-21apt-get23:00:110
2109942994441,2cyclictest0-21swapper/323:06:593
210994299430,42cyclictest0-21swapper/323:02:593
2109933994235,5cyclictest2205029-21kworker/u8:4+flush-179:000:10:161
210994299410,40cyclictest0-21swapper/323:39:593
210993199411,37cyclictest2171048-21sendmail22:20:000
2109942994038,1cyclictest0-21swapper/321:19:563
2109942994034,5cyclictest2142165-21turbostat20:50:003
210993799400,1cyclictest0-21swapper/223:32:592
210993799390,36cyclictest0-21swapper/222:48:002
210993799390,36cyclictest0-21swapper/219:11:592
2109942993836,1cyclictest0-21swapper/320:54:003
2109937993834,2cyclictest0-21swapper/223:38:002
210993799380,36cyclictest0-21swapper/222:28:592
210993799380,36cyclictest0-21swapper/200:35:592
210993799380,35cyclictest0-21swapper/200:07:592
2109933993835,1cyclictest0-21swapper/121:30:581
2109933993832,5cyclictest2159905-21kworker/u8:1+flush-179:021:44:551
2109942993735,1cyclictest0-21swapper/323:32:593
2109942993735,1cyclictest0-21swapper/320:43:583
2109942993735,1cyclictest0-21swapper/320:27:593
210994299370,1cyclictest0-21swapper/322:17:593
210993799370,35cyclictest0-21swapper/220:10:002
2109942993634,1cyclictest0-21swapper/323:49:583
2109942993634,1cyclictest0-21swapper/323:29:553
2109942993634,1cyclictest0-21swapper/323:18:593
2109942993634,1cyclictest0-21swapper/300:17:583
210994299360,35cyclictest0-21swapper/322:04:553
2109937993634,1cyclictest0-21swapper/221:59:592
2109937993633,2cyclictest0-21swapper/222:15:592
210993799360,1cyclictest0-21swapper/223:42:592
210993799360,1cyclictest0-21swapper/221:49:592
4505291350,3ptp4l441ktimers/320:30:033
2109937993533,1cyclictest0-21swapper/220:49:582
2109937993532,2cyclictest0-21swapper/220:02:592
2109942993432,1cyclictest0-21swapper/322:31:593
2109937993431,2cyclictest0-21swapper/222:39:542
2109937993431,2cyclictest0-21swapper/221:00:582
2109942993330,2cyclictest0-21swapper/319:11:593
210993799330,31cyclictest0-21swapper/219:35:592
2109942993230,1cyclictest0-21swapper/323:42:593
2109942993230,1cyclictest0-21swapper/321:49:593
2109937993229,2cyclictest0-21swapper/221:36:002
2109937993229,2cyclictest0-21swapper/219:25:012
2109942993129,1cyclictest0-21swapper/322:47:593
2109942993129,1cyclictest0-21swapper/320:09:593
2109942993129,1cyclictest0-21swapper/300:07:593
210994299310,1cyclictest0-21swapper/320:04:553
2109937993128,2cyclictest0-21swapper/222:55:002
2109937993128,2cyclictest0-21swapper/220:24:002
2109937993128,2cyclictest0-21swapper/219:55:592
2109942993028,1cyclictest0-21swapper/322:28:593
2109942993028,1cyclictest0-21swapper/319:34:553
2109942993028,1cyclictest0-21swapper/300:36:003
2109942993028,1cyclictest0-21swapper/300:25:593
2109942993027,2cyclictest0-21swapper/322:10:143
2109942993027,2cyclictest0-21swapper/321:55:173
210994299300,29cyclictest0-21swapper/319:23:583
210994299300,28cyclictest0-21swapper/323:55:143
210994299300,28cyclictest0-21swapper/323:55:143
210994299300,28cyclictest0-21swapper/323:50:153
210994299300,28cyclictest0-21swapper/322:05:153
2109937993027,2cyclictest0-21swapper/223:58:592
2109937993027,2cyclictest0-21swapper/223:58:592
2109937993027,2cyclictest0-21swapper/221:30:582
2109937993027,2cyclictest0-21swapper/200:22:582
2109937993025,2cyclictest0-21swapper/222:55:142
210993799300,28cyclictest0-21swapper/223:50:152
210993799300,28cyclictest0-21swapper/219:45:172
210993799300,28cyclictest0-21swapper/219:45:172
2109942992926,2cyclictest0-21swapper/300:00:143
210994299290,28cyclictest0-21swapper/320:20:153
210994299290,27cyclictest0-21swapper/322:35:143
210994299290,27cyclictest0-21swapper/321:00:153
210994299290,27cyclictest0-21swapper/320:15:023
2109937992926,2cyclictest0-21swapper/219:25:132
2109937992924,2cyclictest0-21swapper/221:10:132
210993799290,27cyclictest0-21swapper/223:29:592
210993799290,1cyclictest0-21swapper/220:15:032
4505291280,7ptp4l441ktimers/322:55:013
2109942992826,1cyclictest0-21swapper/319:45:173
2109942992826,1cyclictest0-21swapper/319:45:173
2109942992826,1cyclictest0-21swapper/319:25:143
2109942992825,2cyclictest0-21swapper/321:35:163
210994299280,26cyclictest0-21swapper/321:50:173
210994299280,26cyclictest0-21swapper/321:10:133
210994299280,26cyclictest0-21swapper/300:20:133
2109937992825,2cyclictest0-21swapper/220:50:142
2109937992825,2cyclictest0-21swapper/220:40:152
2109937992824,2cyclictest0-21swapper/221:42:592
210993799280,2cyclictest0-21swapper/200:00:142
210993799280,26cyclictest0-21swapper/221:25:152
4505291270,7ptp4l441ktimers/323:14:553
4505291270,6ptp4l61790irq/134-eno1-TxRx-022:25:003
2109942992725,1cyclictest0-21swapper/322:45:003
2109942992725,1cyclictest0-21swapper/319:35:583
210994299270,1cyclictest0-21swapper/321:20:173
210994299270,1cyclictest0-21swapper/319:50:143
2109937992724,2cyclictest0-21swapper/200:30:152
210993799270,25cyclictest0-21swapper/220:39:002
4505291260,6ptp4l61790irq/134-eno1-TxRx-023:00:013
4505291260,6ptp4l61790irq/134-eno1-TxRx-000:14:563
4505291260,6ptp4l441ktimers/321:45:003
4505291260,6ptp4l441ktimers/319:59:563
4505291260,6ptp4l441ktimers/300:34:553
4505291260,5ptp4l441ktimers/323:25:003
4505291260,5ptp4l441ktimers/321:35:013
2109942992624,1cyclictest0-21swapper/319:16:593
210994299260,24cyclictest0-21swapper/320:56:003
210994299260,24cyclictest0-21swapper/320:55:593
2109937992623,2cyclictest0-21swapper/220:29:582
2109937992623,2cyclictest0-21swapper/200:19:552
2109937992622,2cyclictest0-21swapper/223:12:592
210993799260,2cyclictest0-21swapper/222:40:582
210993799260,24cyclictest0-21swapper/223:24:592
210993799260,24cyclictest0-21swapper/222:24:592
210993799260,24cyclictest0-21swapper/219:15:152
201699260,8rtkit-daemon1383751-21avahi-daemon19:09:211
4505291250,5ptp4l61790irq/134-eno1-TxRx-021:09:543
4505291250,5ptp4l441ktimers/319:44:553
210994299250,1cyclictest0-21swapper/320:37:003
2109937992522,2cyclictest0-21swapper/200:12:592
210993799250,2cyclictest0-21swapper/219:42:592
210993799250,24cyclictest0-21swapper/220:31:582
2109937992421,2cyclictest0-21swapper/222:34:552
2109937992421,2cyclictest0-21swapper/221:16:582
2109937992320,2cyclictest0-21swapper/221:08:002
2109937992219,2cyclictest0-21swapper/223:01:592
2109937992118,2cyclictest0-21swapper/221:24:562
2109931992013,2cyclictest171rcu_preempt19:10:010
2109937991916,2cyclictest0-21swapper/222:06:592
2109937991815,2cyclictest0-21swapper/219:53:592
2109937991712,2cyclictest0-21swapper/223:45:132
2109937991613,2cyclictest0-21swapper/221:54:592
2109937991613,2cyclictest0-21swapper/219:34:592
2109937991511,2cyclictest2190160-21turbostat23:20:002
210993799129,2cyclictest0-21swapper/222:04:592
21099379996,2cyclictest0-21swapper/220:56:002
21099379996,2cyclictest0-21swapper/220:55:592
21099379985,2cyclictest0-21swapper/200:25:152
21099379980,7cyclictest0-21swapper/222:10:312
21099379931,1cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional