You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 00:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Mon Jun 30, 2025 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21191099107101,4cyclictest295663-21gpgv11:30:011
2119089910191,7cyclictest295498-21kworker/u8:1+events_unbound11:30:020
2119089910191,7cyclictest295498-21kworker/u8:1+events_unbound11:30:010
2119089910088,9cyclictest223382-21/usr/sbin/munin07:45:260
211908999487,5cyclictest271365-21kworker/u8:1+events_unbound10:50:290
211908999379,5cyclictest261585-21kworker/u8:3+events_unbound10:25:310
211908999282,4cyclictest244328-21kworker/u8:0+events_unbound09:25:290
211908999185,4cyclictest288547-21kworker/u8:0+events_unbound11:15:250
211908998976,10cyclictest258654-21apt-get09:35:180
211910998878,8cyclictest227905-21latency_hist08:00:001
211908998779,6cyclictest261585-21kworker/u8:3+flush-179:009:45:250
211910998682,3cyclictest316448-21kworker/u8:0+events_unbound12:35:151
211908998579,4cyclictest210231-21kworker/u8:1+events_unbound07:15:250
211908998576,6cyclictest277755-21latency_hist10:35:010
211908998573,9cyclictest242239-21latency_hist08:45:010
211908998573,9cyclictest242239-21latency_hist08:45:000
211908998372,8cyclictest502-21rs:main0
211910998272,7cyclictest211690-21latency_hist07:10:021
211908998277,4cyclictest248838-21kworker/u8:3+events_unbound09:10:270
211910998171,8cyclictest3105085-21systemd-journal07:45:011
211910998168,11cyclictest273252-21apt-get10:20:161
211908998176,3cyclictest231080-21kworker/u8:1+events_unbound08:20:180
211908998169,9cyclictest265016-21apt-get09:55:120
211910998068,10cyclictest240908-21apt-get08:40:161
211910998068,10cyclictest240908-21apt-get08:40:161
211908998076,3cyclictest253597-21kworker/u8:2+events_unbound09:25:000
211910997970,7cyclictest251773-21latency_hist09:15:001
211910997874,3cyclictest316448-21kworker/u8:0+events_unbound12:33:481
211908997863,6cyclictest240889-21kworker/u8:1+events_unbound09:05:300
211910997760,7cyclictest290483-21latency_hist11:15:011
211908997772,3cyclictest297362-21kworker/u8:3+events_unbound11:40:150
211908997666,7cyclictest243823-21latency_hist08:50:000
211910997569,4cyclictest240889-21kworker/u8:1+flush-179:009:00:181
211908997571,3cyclictest305375-21kworker/u8:3+flush-179:012:10:000
211908997570,4cyclictest271365-21kworker/u8:1+events_unbound10:45:290
211908997570,4cyclictest231080-21kworker/u8:1+events_unbound08:15:250
211910997470,3cyclictest269582-21kworker/u8:0+events_unbound10:05:481
211910997469,4cyclictest261628-21kworker/u8:4+events_unbound10:04:471
211908997469,4cyclictest277744-21kworker/u8:2+events_unbound10:35:160
211908997464,8cyclictest3105085-21systemd-journal07:29:480
211908997464,8cyclictest3105085-21systemd-journal07:29:470
211908997463,4cyclictest223130-21kworker/u8:2+events_unbound08:00:260
211910997369,3cyclictest305375-21kworker/u8:3+events_unbound12:11:201
211910997369,3cyclictest261628-21kworker/u8:4+events_unbound09:46:491
211910997369,3cyclictest223129-21kworker/u8:1+events_unbound07:46:481
211908997368,4cyclictest239080-21kworker/u8:2+flush-179:008:35:280
211908997367,4cyclictest227018-21kworker/u8:0+events_unbound08:05:150
211910997262,8cyclictest218579-21/usr/sbin/munin07:30:281
211908997267,4cyclictest271365-21kworker/u8:1+events_unbound10:20:190
211908997261,9cyclictest237714-21apt-get08:30:120
211910997167,3cyclictest310334-21kworker/u8:2+events_unbound12:21:211
211908997166,3cyclictest261585-21kworker/u8:3+flush-179:010:10:250
211908997165,4cyclictest257050-21kworker/u8:1+events_unbound09:40:210
211908997163,4cyclictest308748-21kworker/u8:0+events_unbound12:10:260
211908997162,7cyclictest219913-21kworker/u8:3+events_unbound07:40:140
211908997065,4cyclictest261585-21kworker/u8:3+flush-179:011:05:230
211910996961,6cyclictest240889-21kworker/u8:1+flush-179:008:55:001
211908996956,10cyclictest267956-21latency_hist10:05:010
211910996857,8cyclictest305375-21kworker/u8:3+flush-179:012:05:211
211908996862,4cyclictest240889-21kworker/u8:1+events_unbound08:55:220
211908996861,5cyclictest261585-21kworker/u8:3+events_unbound09:50:200
211908996859,6cyclictest303731-21latency_hist11:55:010
211910996753,13cyclictest244328-21kworker/u8:0+flush-179:009:25:261
211910996656,8cyclictest3105085-21systemd-journal07:25:001
211910996655,9cyclictest257069-21apt-get09:30:191
211908996660,4cyclictest231080-21kworker/u8:1+flush-179:008:25:190
211908996659,4cyclictest219909-21kworker/u8:0+events_unbound07:35:130
211910996455,6cyclictest279354-21latency_hist10:40:011
211908996459,4cyclictest227018-21kworker/u8:0+flush-179:007:55:250
211908996456,6cyclictest285726-21latency_hist11:00:010
211910996352,8cyclictest247021-21latency_hist09:00:011
211908996358,4cyclictest232133-21kworker/u8:3+events_unbound08:10:220
211910996252,8cyclictest260262-21apt-get09:40:171
211908996258,3cyclictest295498-21kworker/u8:1+flush-179:011:55:230
211908996257,4cyclictest305375-21kworker/u8:3+flush-179:012:20:260
211908996254,6cyclictest295498-21kworker/u8:1+events_unbound12:30:150
211910996157,3cyclictest305375-21kworker/u8:3+events_unbound12:03:481
211908996153,3cyclictest300438-21kworker/u8:2+events_unbound11:48:460
211908996152,4cyclictest261585-21kworker/u8:3+flush-179:010:15:250
211908996150,8cyclictest258399-21latency_hist09:35:000
211910996055,3cyclictest239080-21kworker/u8:2+events_unbound08:35:031
211910996050,5cyclictest223130-21kworker/u8:2+flush-179:008:00:311
211908996054,5cyclictest261628-21kworker/u8:4+events_unbound10:05:110
211908996053,5cyclictest210231-21kworker/u8:1+events_unbound07:30:160
211908996050,8cyclictest289135-21/usr/sbin/munin11:10:280
211910995956,2cyclictest0-21swapper/111:08:591
211908995949,4cyclictest210231-21kworker/u8:1+events_unbound07:22:490
211920995851,5cyclictest277774-21turbostat10:35:003
211910995853,3cyclictest231080-21kworker/u8:1+events_unbound08:25:021
211910995847,9cyclictest252286-21apt-get09:15:151
211908995852,4cyclictest240889-21kworker/u8:1+flush-179:009:00:290
211908995752,4cyclictest288547-21kworker/u8:0+events_unbound11:25:240
211908995749,3cyclictest271365-21kworker/u8:1+events_unbound11:00:480
211916995653,2cyclictest0-21swapper/208:40:592
211916995653,2cyclictest0-21swapper/208:40:582
211910995649,6cyclictest261585-21kworker/u8:3+flush-179:011:00:271
211910995648,6cyclictest299230-21apt-get11:40:121
211908995652,3cyclictest316448-21kworker/u8:0+flush-179:012:35:250
211910995551,3cyclictest271365-21kworker/u8:1+flush-179:010:40:261
211910995551,3cyclictest261628-21kworker/u8:4+events_unbound09:58:481
211910995543,9cyclictest226315-21latency_hist07:55:001
211908995550,4cyclictest207166-21kworker/u8:2+events_unbound07:10:200
211910995445,7cyclictest260003-21latency_hist09:40:021
211910995435,5cyclictest292607-21apt-get11:20:151
211908995450,3cyclictest244328-21kworker/u8:0+flush-179:009:15:230
211910995247,4cyclictest210231-21kworker/u8:1+events_unbound07:14:491
211908995248,3cyclictest305375-21kworker/u8:3+events_unbound12:15:220
211908995248,3cyclictest223129-21kworker/u8:1+flush-179:007:50:310
211910995143,7cyclictest231080-21kworker/u8:1+flush-179:008:30:261
211908995146,4cyclictest240889-21kworker/u8:1+events_unbound08:50:190
211910995038,11cyclictest210231-21kworker/u8:1+flush-179:007:15:271
211920994947,1cyclictest0-21swapper/311:21:593
211920994943,4cyclictest234275-21turbostat08:19:593
211910994945,3cyclictest261585-21kworker/u8:3+events_unbound09:50:031
211910994943,5cyclictest310334-21kworker/u8:2+flush-179:012:25:241
211910994937,9cyclictest250188-21latency_hist09:10:001
211908994940,3cyclictest271365-21kworker/u8:1+events_unbound10:40:300
211916994845,2cyclictest0-21swapper/209:47:002
211910994844,3cyclictest300438-21kworker/u8:2+events_unbound11:45:011
211910994844,3cyclictest231080-21kworker/u8:1+events_unbound08:19:561
211910994838,7cyclictest221559-21latency_hist07:40:001
211908994837,9cyclictest298977-21dump-pmu-power11:40:000
4505291470,13ptp4l441ktimers/307:09:563
211910994742,4cyclictest231080-21kworker/u8:1+events_unbound08:21:081
211910994739,6cyclictest281203-21apt-get10:45:151
211908994742,4cyclictest295498-21kworker/u8:1+flush-179:012:25:490
211910994639,5cyclictest3105085-21systemd-journal10:59:471
211910994537,6cyclictest297362-21kworker/u8:3+flush-179:011:35:251
211910994534,9cyclictest290737-21apt-get11:15:121
211910994439,4cyclictest310334-21kworker/u8:2+events_unbound12:17:001
211910994439,4cyclictest207166-21kworker/u8:2+events_unbound07:25:361
211910994439,4cyclictest207166-21kworker/u8:2+events_unbound07:25:351
211910994433,10cyclictest253604-1kworker/1:2H+kblockd09:25:001
211908994439,4cyclictest295498-21kworker/u8:1+events_unbound12:00:210
211920994235,5cyclictest213572-21turbostat07:14:593
21191099421,39cyclictest722-21Xorg08:14:591
211910994136,4cyclictest271365-21kworker/u8:1+events_unbound10:17:541
211910994136,3cyclictest722-21Xorg11:31:001
211910994136,3cyclictest722-21Xorg11:30:591
211910994135,4cyclictest476-21dbus-daemon08:50:011
211910994036,3cyclictest271365-21kworker/u8:1+events_unbound10:13:541
211910994029,9cyclictest3105085-21systemd-journal12:00:001
211920993936,2cyclictest0-21swapper/310:36:593
211920993931,7cyclictest0-21swapper/311:37:593
21192099390,36cyclictest0-21swapper/309:55:593
211916993936,2cyclictest0-21swapper/211:31:002
211916993936,2cyclictest0-21swapper/211:31:002
211910993936,2cyclictest0-21swapper/110:27:591
211910993934,4cyclictest271365-21kworker/u8:1+events_unbound10:30:541
211920993835,2cyclictest0-21swapper/310:24:593
21192099380,2cyclictest0-21swapper/308:08:003
211916993834,2cyclictest0-21swapper/210:55:002
21191699380,36cyclictest0-21swapper/212:10:002
211920993735,1cyclictest0-21swapper/311:33:593
211920993735,1cyclictest0-21swapper/311:33:583
211920993734,2cyclictest0-21swapper/312:00:003
211916993734,2cyclictest0-21swapper/211:19:552
211916993733,2cyclictest0-21swapper/212:13:592
211916993733,2cyclictest0-21swapper/208:54:552
21191699370,35cyclictest0-21swapper/210:34:592
211910993732,4cyclictest271365-21kworker/u8:1+events_unbound10:54:231
211910993730,4cyclictest0-21swapper/111:55:011
211920993633,2cyclictest0-21swapper/309:53:593
211920993633,2cyclictest0-21swapper/307:51:003
211920993628,7cyclictest0-21swapper/311:04:553
211920993623,11cyclictest302163-21turbostat11:54:553
21192099360,35cyclictest0-21swapper/310:01:593
211916993633,2cyclictest0-21swapper/210:10:002
211910993625,9cyclictest229764-21apt-get08:05:141
211920993532,2cyclictest0-21swapper/308:45:593
211920993529,5cyclictest0-21swapper/311:40:593
21192099350,34cyclictest0-21swapper/310:48:593
21192099350,1cyclictest0-21swapper/311:08:593
211916993532,2cyclictest0-21swapper/209:39:542
21191699350,1cyclictest0-21swapper/212:40:012
21192099340,33cyclictest0-21swapper/308:21:003
21192099340,27cyclictest0-21swapper/307:45:163
211916993431,2cyclictest0-21swapper/209:35:002
211916993431,2cyclictest0-21swapper/208:33:002
211908993430,3cyclictest211923-21latency_hist07:10:030
211920993331,1cyclictest0-21swapper/310:09:553
211920993331,1cyclictest0-21swapper/309:37:583
211920993331,1cyclictest0-21swapper/309:27:003
211920993331,1cyclictest0-21swapper/308:25:593
211920993329,2cyclictest0-21swapper/309:16:593
211920993325,7cyclictest0-21swapper/309:22:583
211916993331,1cyclictest0-21swapper/212:35:002
211916993330,2cyclictest0-21swapper/209:00:012
211916993330,2cyclictest0-21swapper/208:20:592
211916993330,2cyclictest0-21swapper/207:20:002
211920993225,6cyclictest0-21swapper/312:27:003
211920993225,5cyclictest0-21swapper/310:15:183
211916993129,1cyclictest0-21swapper/207:45:012
211916993128,2cyclictest0-21swapper/211:54:582
211916993128,2cyclictest0-21swapper/207:27:592
211916993128,2cyclictest0-21swapper/207:27:592
211920993028,1cyclictest0-21swapper/311:46:593
211920993027,2cyclictest0-21swapper/312:15:143
21192099300,27cyclictest0-21swapper/310:25:183
211916993028,1cyclictest0-21swapper/207:34:592
211916993027,2cyclictest0-21swapper/211:35:582
211920992927,1cyclictest0-21swapper/307:40:143
211920992923,5cyclictest0-21swapper/312:03:593
211920992921,7cyclictest0-21swapper/312:24:553
21192099290,28cyclictest0-21swapper/309:15:013
211916992927,1cyclictest0-21swapper/209:01:592
211916992927,1cyclictest0-21swapper/208:19:592
211916992926,2cyclictest0-21swapper/210:45:172
211916992926,2cyclictest0-21swapper/210:15:172
211916992926,2cyclictest0-21swapper/210:00:182
211916992926,2cyclictest0-21swapper/208:30:002
211916992926,2cyclictest0-21swapper/207:38:002
21191699290,28cyclictest0-21swapper/211:00:152
21191699290,27cyclictest0-21swapper/212:01:002
21191699290,27cyclictest0-21swapper/210:35:172
21191699290,27cyclictest0-21swapper/208:45:152
4505291280,7ptp4l441ktimers/310:15:013
4505291280,7ptp4l441ktimers/309:35:003
4505291280,6ptp4l441ktimers/308:00:003
4505291280,5ptp4l441ktimers/308:55:013
211920992826,1cyclictest0-21swapper/312:30:163
211920992826,1cyclictest0-21swapper/310:40:203
211920992826,1cyclictest0-21swapper/308:00:153
211920992825,2cyclictest0-21swapper/310:50:173
21192099280,26cyclictest0-21swapper/312:10:153
21192099280,26cyclictest0-21swapper/312:05:183
21192099280,26cyclictest0-21swapper/311:25:173
21192099280,26cyclictest0-21swapper/308:55:193
21192099280,26cyclictest0-21swapper/308:10:123
211916992825,2cyclictest0-21swapper/212:17:592
211916992825,2cyclictest0-21swapper/211:45:002
211916992825,2cyclictest0-21swapper/211:10:162
211916992825,2cyclictest0-21swapper/209:05:172
211916992825,2cyclictest0-21swapper/208:04:542
211916992824,2cyclictest0-21swapper/209:50:582
211916992824,2cyclictest0-21swapper/209:16:592
21191699280,27cyclictest0-21swapper/211:20:022
21191699280,26cyclictest0-21swapper/212:25:142
21191699280,26cyclictest0-21swapper/207:55:182
21191699280,1cyclictest0-21swapper/210:56:592
4505291270,7ptp4l61790irq/134-eno1-TxRx-011:20:003
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:45:013
211920992725,1cyclictest0-21swapper/309:07:583
211920992725,1cyclictest0-21swapper/309:00:163
211920992725,1cyclictest0-21swapper/307:35:143
211920992725,1cyclictest0-21swapper/307:29:593
211920992725,1cyclictest0-21swapper/307:29:583
211916992724,2cyclictest0-21swapper/210:41:592
211916992724,2cyclictest0-21swapper/209:43:592
211916992724,2cyclictest0-21swapper/208:13:592
211916992724,2cyclictest0-21swapper/208:05:152
4505291260,5ptp4l441ktimers/308:35:003
21192099260,1cyclictest0-21swapper/311:12:593
211916992624,1cyclictest0-21swapper/209:57:592
211916992623,2cyclictest0-21swapper/210:10:172
21191699260,24cyclictest0-21swapper/211:55:592
21191699260,24cyclictest0-21swapper/211:47:592
21191699260,24cyclictest0-21swapper/210:25:192
4505291250,5ptp4l441ktimers/312:39:553
4505291250,5ptp4l441ktimers/309:49:553
4505291250,5ptp4l441ktimers/307:35:003
4505291250,4ptp4l61790irq/134-eno1-TxRx-007:24:563
211920992523,1cyclictest0-21swapper/310:56:593
211916992522,2cyclictest0-21swapper/209:23:582
211916992522,2cyclictest0-21swapper/207:20:592
211916992522,2cyclictest0-21swapper/207:10:592
4505291240,5ptp4l61790irq/134-eno1-TxRx-008:44:573
4505291240,5ptp4l61790irq/134-eno1-TxRx-008:44:563
4505291240,5ptp4l441ktimers/308:39:553
4505291240,3ptp4l61790irq/134-eno1-TxRx-007:20:003
211916992422,1cyclictest0-21swapper/207:50:002
211916992421,2cyclictest0-21swapper/210:21:592
211916992219,2cyclictest0-21swapper/211:26:582
211916992218,2cyclictest0-21swapper/208:36:592
211916992017,2cyclictest0-21swapper/211:07:592
211916991814,2cyclictest0-21swapper/209:25:162
211916991613,1cyclictest0-21swapper/212:25:002
21191699119,1cyclictest0-21swapper/207:54:592
2119169986,1cyclictest0-21swapper/209:15:002
2119169930,2cyclictest0-21swapper/207:10:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional