You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 18:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Nov 17, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220881999789,6cyclictest257693-21latency_hist09:05:020
220881999685,8cyclictest327658-21latency_hist12:40:010
220885998977,8cyclictest322181-21kworker/u8:1+flush-179:012:40:001
220881998979,7cyclictest311485-21latency_hist11:50:020
220881998474,7cyclictest502-21rs:main0
220881998275,4cyclictest200318-21kworker/u8:0+events_unbound07:40:010
220885997767,7cyclictest264041-21kworker/u8:0+flush-179:009:30:011
220881997670,5cyclictest277030-21kworker/u8:3+flush-179:010:10:010
220885997463,8cyclictest231958-21latency_hist07:45:011
220881997464,8cyclictest265886-21/usr/sbin/munin09:30:250
220881997464,8cyclictest265886-21/usr/sbin/munin09:30:250
220881997366,5cyclictest322181-21kworker/u8:1+flush-179:012:30:270
220885997262,8cyclictest324731-21apt-get12:30:141
220885997261,8cyclictest319695-21latency_hist12:15:011
220885997260,9cyclictest271-21systemd-journal10:15:281
220881997060,8cyclictest269102-21apt-get09:40:120
220885996960,6cyclictest227153-21latency_hist07:30:011
220881996861,5cyclictest200318-21kworker/u8:0+flush-179:007:55:000
220881996858,7cyclictest306450-21latency_hist11:35:010
220885996754,10cyclictest317849-21latency_hist12:10:011
220881996759,5cyclictest251325-21latency_hist08:45:010
220881996658,6cyclictest271-21systemd-journal08:55:010
220881996657,7cyclictest251562-21/usr/sbin/munin08:45:250
220881996560,3cyclictest191rcuc/008:25:250
220881996553,10cyclictest232213-21apt-get07:45:140
220885996454,8cyclictest283699-21kworker/u8:0+flush-179:011:10:011
220885996453,9cyclictest283919-21apt-get10:25:131
220881996457,6cyclictest316962-21kworker/u8:1+flush-179:012:09:560
220885996350,10cyclictest276831-21apt-get10:03:031
220881996354,6cyclictest277030-21kworker/u8:3+flush-179:010:25:010
220881996354,6cyclictest277030-21kworker/u8:3+flush-179:010:25:010
220885996253,7cyclictest261100-21/usr/sbin/munin09:15:251
220885996251,8cyclictest200318-21kworker/u8:0+flush-179:008:50:001
220887996158,2cyclictest0-21swapper/210:37:592
220881996155,4cyclictest283699-21kworker/u8:0+flush-179:011:15:240
220881996154,5cyclictest200318-21kworker/u8:0+flush-179:007:20:230
220887996026,2cyclictest0-21swapper/207:17:592
220881996054,4cyclictest248116-21kworker/u8:2+events_unbound09:00:010
220881996050,8cyclictest293435-21/usr/sbin/munin10:55:270
220881995949,8cyclictest291851-21/usr/sbin/munin10:50:240
220881995853,4cyclictest322181-21kworker/u8:1+flush-179:012:29:560
220881995848,7cyclictest309881-21kworker/u8:0+flush-179:011:55:170
220885995749,6cyclictest267237-21latency_hist09:35:011
220885995749,6cyclictest267237-21latency_hist09:35:001
220881995752,4cyclictest303009-21kworker/u8:2+flush-179:011:50:180
220881995751,4cyclictest278662-21kworker/u8:1+flush-179:010:40:220
220881995750,5cyclictest219304-21kworker/u8:2+events_unbound07:10:250
220885995649,5cyclictest723-21Xorg10:37:591
220885995649,5cyclictest286814-21kworker/u8:2+flush-179:010:35:011
220885995646,8cyclictest272274-21apt-get09:50:121
220881995651,4cyclictest268881-21kworker/u8:3+flush-179:009:45:240
220885995435,16cyclictest271-21systemd-journal11:51:271
220881995448,5cyclictest224016-21kworker/u8:1+flush-179:007:30:230
220890995351,1cyclictest0-21swapper/312:04:593
220885995344,6cyclictest268806-21kworker/u8:0+flush-179:009:39:591
220881995346,5cyclictest306105-21kworker/u8:1+flush-179:011:35:260
220885995248,3cyclictest262429-21kworker/u8:2+flush-179:009:20:301
220885995216,34cyclictest723-21Xorg07:17:591
220885995145,4cyclictest268881-21kworker/u8:3+flush-179:009:45:261
220885995143,6cyclictest200318-21kworker/u8:0+events_unbound08:15:281
22088599510,2cyclictest723-21Xorg10:13:581
220881995146,4cyclictest296632-21kworker/u8:1+flush-179:011:25:230
220881995145,4cyclictest262429-21kworker/u8:2+flush-179:009:25:240
220881995142,7cyclictest200318-21kworker/u8:0+flush-179:007:20:010
220885995046,3cyclictest278662-21kworker/u8:1+flush-179:010:21:291
220885995046,3cyclictest278662-21kworker/u8:1+flush-179:010:21:291
220885995029,6cyclictest310443-21diskstats11:45:171
220881994944,4cyclictest283699-21kworker/u8:0+flush-179:011:10:290
220881994944,4cyclictest283699-21kworker/u8:0+flush-179:011:10:280
220881994941,6cyclictest256149-21kworker/u8:3+flush-179:009:10:260
4670791480,13ptp4l61690irq/136-eno1-TxRx-007:09:563
220885994844,3cyclictest322181-21kworker/u8:1+flush-179:012:25:231
220885994840,6cyclictest283699-21kworker/u8:0+flush-179:010:50:241
220881994842,4cyclictest306105-21kworker/u8:1+flush-179:011:40:220
220881994842,4cyclictest277030-21kworker/u8:3+events_unbound10:15:260
220881994841,5cyclictest278662-21kworker/u8:1+flush-179:010:25:180
220881994837,8cyclictest236738-21latency_hist08:00:000
220885994742,4cyclictest200318-21kworker/u8:0+flush-179:008:25:161
220885994739,5cyclictest309901-21latency_hist11:45:021
220885994734,9cyclictest221042-21apt-get07:10:141
220885994731,2cyclictest723-21Xorg07:51:581
220881994738,7cyclictest314934-21apt-get12:00:130
220885994640,4cyclictest239913-21latency_hist08:10:011
220885994639,5cyclictest301406-21cron11:19:591
220881994641,3cyclictest273642-21systemd-run09:55:010
220885994539,4cyclictest278662-21kworker/u8:1+events_unbound10:45:011
220881994540,3cyclictest238366-21systemd-run08:05:010
220881994538,5cyclictest268806-21kworker/u8:0+events_unbound09:55:270
220881994538,5cyclictest268806-21kworker/u8:0+events_unbound09:55:270
220881994537,6cyclictest283699-21kworker/u8:0+flush-179:010:35:180
220885994431,10cyclictest244709-21apt-get08:25:001
220885994428,5cyclictest236720-21turbostat08:04:551
22088599440,42cyclictest299830-21awk11:15:001
22088599440,42cyclictest299830-21awk11:14:591
220881994438,4cyclictest248116-21kworker/u8:2+events_unbound08:35:260
220881994437,5cyclictest322181-21kworker/u8:1+events_unbound12:20:280
220885994336,5cyclictest254732-21kworker/u8:0+flush-179:009:00:011
220881994337,4cyclictest241533-21kworker/u8:1+flush-179:008:20:190
220881994332,9cyclictest246797-21apt-get08:30:120
22089099420,1cyclictest0-21swapper/309:06:583
22088599420,40cyclictest290045-21turbostat10:49:561
220881994236,4cyclictest283699-21kworker/u8:0+flush-179:010:45:180
220890994134,6cyclictest0-21swapper/310:59:593
220887994138,2cyclictest0-21swapper/209:57:592
220887994138,2cyclictest0-21swapper/209:57:582
220885994135,4cyclictest304318-21kworker/u8:3+flush-179:011:30:001
220881994136,4cyclictest238305-21kworker/u8:3+flush-179:008:05:180
220881994136,3cyclictest277030-21kworker/u8:3+events_unbound10:30:200
22088199410,39cyclictest723-21Xorg07:44:590
22089099400,38cyclictest260876-21turbostat09:15:003
22088799400,1cyclictest0-21swapper/210:15:582
22088599400,38cyclictest723-21Xorg08:52:591
22088599400,38cyclictest319715-21turbostat12:19:551
22088599400,38cyclictest228749-21turbostat07:39:551
220881994033,5cyclictest303063-21latency_hist11:25:000
220881994032,6cyclictest316962-21kworker/u8:1+flush-179:012:10:180
220881994032,6cyclictest283699-21kworker/u8:0+flush-179:011:05:000
220881993933,4cyclictest262429-21kworker/u8:2+flush-179:009:35:260
220881993932,5cyclictest283699-21kworker/u8:0+flush-179:011:05:150
220881993932,5cyclictest256149-21kworker/u8:3+flush-179:009:05:250
220881993931,6cyclictest221321-21kworker/u8:3+flush-179:007:30:000
4670791380,6ptp4l431rcuc/309:50:013
220890993836,1cyclictest0-21swapper/311:42:583
220890993836,1cyclictest0-21swapper/311:21:593
22088799380,36cyclictest0-21swapper/210:31:582
22088799380,35cyclictest0-21swapper/208:59:542
22088599380,37cyclictest0-21swapper/107:32:591
220890993731,5cyclictest0-21swapper/307:29:553
22089099370,36cyclictest0-21swapper/311:07:593
22089099370,35cyclictest0-21swapper/308:42:003
220887993734,2cyclictest0-21swapper/210:49:002
220887993734,2cyclictest0-21swapper/210:04:542
22088799370,35cyclictest0-21swapper/211:04:002
220885993731,4cyclictest200318-21kworker/u8:0+events_unbound07:25:001
220885993730,5cyclictest271-21systemd-journal11:00:001
220885993729,5cyclictest235367-21/usr/sbin/munin07:55:201
22088599370,1cyclictest0-21swapper/109:01:581
220881993730,5cyclictest238305-21kworker/u8:3+events_unbound08:10:170
220881993730,5cyclictest238305-21kworker/u8:3+events_unbound08:10:160
220881993729,6cyclictest243098-21latency_hist08:20:010
220890993634,1cyclictest0-21swapper/309:19:553
220890993634,1cyclictest0-21swapper/309:02:593
220890993634,1cyclictest0-21swapper/308:39:563
220890993634,1cyclictest0-21swapper/308:14:593
220890993634,1cyclictest0-21swapper/308:14:583
22089099360,35cyclictest0-21swapper/308:48:593
22089099360,35cyclictest0-21swapper/308:04:593
22089099360,1cyclictest0-21swapper/312:12:593
22089099360,1cyclictest0-21swapper/309:41:593
22088799360,34cyclictest0-21swapper/209:14:582
22088799360,1cyclictest0-21swapper/212:10:582
220885993632,3cyclictest314709-21systemd-run12:00:011
220885993632,3cyclictest271-21systemd-journal11:34:281
220881993631,4cyclictest262429-21kworker/u8:2+events_unbound09:20:270
220881993629,5cyclictest256149-21kworker/u8:3+flush-179:009:15:260
220890993533,1cyclictest0-21swapper/309:55:003
220887993532,2cyclictest0-21swapper/212:28:592
220887993532,2cyclictest0-21swapper/211:29:552
220885993528,5cyclictest314723-21kworker/u8:3+flush-179:012:00:231
220885993526,7cyclictest0-21swapper/108:35:011
220890993432,1cyclictest0-21swapper/307:52:593
22089099340,27cyclictest0-21swapper/310:03:063
220887993429,3cyclictest0-21swapper/208:28:592
220885993431,2cyclictest723-21Xorg11:24:001
220885993424,7cyclictest249728-21latency_hist08:40:011
220881993429,3cyclictest280405-21latency_hist10:15:000
220890993331,1cyclictest0-21swapper/312:29:563
220890993331,1cyclictest0-21swapper/307:14:593
22089099330,27cyclictest0-21swapper/309:25:133
22089099330,26cyclictest0-21swapper/311:35:143
220887993330,2cyclictest0-21swapper/209:00:592
220885993330,2cyclictest723-21Xorg11:03:591
220885993328,3cyclictest723-21Xorg09:57:591
220885993328,3cyclictest723-21Xorg09:57:581
220890993230,1cyclictest0-21swapper/308:06:593
22089099320,26cyclictest0-21swapper/310:25:143
220887993229,2cyclictest0-21swapper/207:25:592
220885993228,3cyclictest256149-21kworker/u8:3+flush-179:009:14:551
220885993227,4cyclictest321303-21turbostat12:24:551
220885993227,4cyclictest248116-21kworker/u8:2+flush-179:008:40:171
220885993223,8cyclictest306460-21apt-get11:35:031
22088599321,29cyclictest0-21swapper/110:05:161
220890993128,2cyclictest0-21swapper/310:38:003
220890993124,6cyclictest0-21swapper/311:27:583
22088599311,29cyclictest723-21Xorg07:49:591
220890993028,1cyclictest0-21swapper/307:34:553
220890993027,2cyclictest0-21swapper/309:35:143
22089099300,28cyclictest0-21swapper/307:35:143
220887993027,2cyclictest0-21swapper/209:15:142
220887993027,2cyclictest0-21swapper/207:37:592
220887993026,2cyclictest0-21swapper/208:43:592
220887993026,2cyclictest0-21swapper/207:11:592
220885993025,4cyclictest220911-21latency_hist07:10:011
220890992927,1cyclictest0-21swapper/311:14:563
220890992927,1cyclictest0-21swapper/311:14:553
22089099290,27cyclictest0-21swapper/311:30:033
22089099290,27cyclictest0-21swapper/310:20:153
22089099290,27cyclictest0-21swapper/310:20:153
22089099290,27cyclictest0-21swapper/310:15:023
22089099290,27cyclictest0-21swapper/307:40:163
22089099290,1cyclictest0-21swapper/307:15:133
220887992926,2cyclictest0-21swapper/210:50:142
220887992926,2cyclictest0-21swapper/209:25:152
220887992926,2cyclictest0-21swapper/208:34:552
220887992926,2cyclictest0-21swapper/207:45:162
22088799290,27cyclictest0-21swapper/212:35:182
22088799290,27cyclictest0-21swapper/210:06:002
22088799290,27cyclictest0-21swapper/208:45:152
22088799290,27cyclictest0-21swapper/208:05:152
22088799290,21cyclictest0-21swapper/211:30:032
220885992925,2cyclictest239926-21perf08:14:561
220885992925,2cyclictest239926-21perf08:14:561
220881992922,5cyclictest220758-21latency_hist07:10:010
220890992826,1cyclictest0-21swapper/308:19:553
22089099280,4cyclictest327653-21turbostat12:40:003
22089099280,2cyclictest0-21swapper/310:10:163
22089099280,27cyclictest0-21swapper/311:15:163
22089099280,27cyclictest0-21swapper/310:05:173
22089099280,26cyclictest0-21swapper/309:30:143
22089099280,26cyclictest0-21swapper/309:30:143
22089099280,26cyclictest0-21swapper/308:50:133
22089099280,21cyclictest0-21swapper/312:05:173
220887992825,2cyclictest0-21swapper/210:40:142
220887992825,2cyclictest0-21swapper/209:21:592
220887992825,2cyclictest0-21swapper/208:50:142
22088799280,2cyclictest0-21swapper/211:35:042
22088799280,26cyclictest0-21swapper/208:37:582
220885992823,4cyclictest256149-21kworker/u8:3+events_unbound09:05:161
220881992824,2cyclictest723-21Xorg12:16:590
4670791270,6ptp4l61690irq/136-eno1-TxRx-012:25:003
4670791270,6ptp4l61690irq/136-eno1-TxRx-009:25:013
220890992725,1cyclictest0-21swapper/308:25:023
22089099270,1cyclictest0-21swapper/310:46:593
220887992724,2cyclictest0-21swapper/209:42:592
22088799270,25cyclictest0-21swapper/212:08:592
22088799270,25cyclictest0-21swapper/211:50:592
22088799270,25cyclictest0-21swapper/210:59:542
4670791260,6ptp4l61690irq/136-eno1-TxRx-007:59:553
4670791260,6ptp4l441ktimers/308:34:553
220890992624,1cyclictest0-21swapper/310:33:593
220890992623,2cyclictest0-21swapper/311:04:593
220890992623,2cyclictest0-21swapper/307:20:593
22089099260,25cyclictest0-21swapper/311:46:593
22089099260,24cyclictest0-21swapper/308:20:163
220887992623,2cyclictest0-21swapper/212:15:592
220887992622,2cyclictest0-21swapper/207:21:592
22088799260,2cyclictest0-21swapper/211:40:582
22088799260,2cyclictest0-21swapper/209:54:002
22088799260,24cyclictest0-21swapper/211:10:582
22088799260,24cyclictest0-21swapper/211:10:582
22088799260,24cyclictest0-21swapper/210:24:002
22088799260,24cyclictest0-21swapper/210:23:592
220885992619,5cyclictest270417-21sshd09:41:291
4670791250,6ptp4l61690irq/136-eno1-TxRx-011:55:003
4670791250,6ptp4l441ktimers/312:20:013
4670791250,5ptp4l441ktimers/307:49:563
220890992523,1cyclictest0-21swapper/308:59:593
22089099250,24cyclictest0-21swapper/310:44:563
220887992522,2cyclictest0-21swapper/211:45:592
220887992521,2cyclictest0-21swapper/211:59:002
22088799250,2cyclictest0-21swapper/207:41:592
22088799250,24cyclictest0-21swapper/211:25:002
4670791240,5ptp4l61690irq/136-eno1-TxRx-012:00:013
4670791240,5ptp4l441ktimers/312:34:553
4670791240,5ptp4l441ktimers/309:59:553
4670791240,5ptp4l441ktimers/309:59:553
220887992421,2cyclictest0-21swapper/209:47:582
220887992421,2cyclictest0-21swapper/208:23:592
220887992421,2cyclictest0-21swapper/208:02:592
4670791230,5ptp4l441ktimers/310:54:563
220887992320,2cyclictest0-21swapper/207:53:592
220887992319,2cyclictest0-21swapper/208:13:002
220887992319,2cyclictest0-21swapper/208:13:002
220887992218,2cyclictest0-21swapper/212:22:592
220887992117,2cyclictest0-21swapper/210:27:592
220887992017,2cyclictest0-21swapper/209:33:592
220887992017,2cyclictest0-21swapper/209:33:582
220887992017,2cyclictest0-21swapper/208:15:592
220887992016,2cyclictest0-21swapper/207:30:592
220887991917,1cyclictest0-21swapper/208:00:002
220887991815,1cyclictest0-21swapper/210:15:012
220887991513,1cyclictest0-21swapper/211:09:592
220887991310,2cyclictest0-21swapper/211:16:002
220887991210,1cyclictest0-21swapper/212:30:162
2208879996,2cyclictest0-21swapper/209:38:002
2208879974,2cyclictest0-21swapper/209:07:332
2208879963,2cyclictest0-21swapper/212:04:122
2208879930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional