You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 04:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Thu Jan 01, 2026 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
553604999987,9cyclictest613698-21kworker/u8:1+flush-179:022:20:011
553604998983,4cyclictest592285-21kworker/u8:3+flush-179:021:20:001
553604998878,7cyclictest624612-21latency_hist22:50:001
553604998682,3cyclictest555342-21kworker/u8:0+events_unbound20:15:241
553604998478,4cyclictest553474-21kworker/u8:2+flush-179:019:10:251
553604998378,3cyclictest649106-21kworker/u8:0+events_unbound00:30:251
553604998373,7cyclictest590666-21latency_hist21:05:011
553604998275,5cyclictest637643-21kworker/u8:2+flush-179:023:45:001
553604998274,6cyclictest615945-21kworker/u8:2+events_unbound23:00:001
553604998271,4cyclictest649106-21kworker/u8:0+events_unbound00:25:141
553603998274,6cyclictest556696-21latency_hist19:20:010
553604998076,3cyclictest637643-21kworker/u8:2+events_unbound00:20:251
553604997872,4cyclictest606598-21kworker/u8:1+events_unbound21:55:241
553604997872,4cyclictest606598-21kworker/u8:1+events_unbound21:55:241
553604997869,7cyclictest618246-21latency_hist22:30:001
553604997771,4cyclictest555342-21kworker/u8:0+flush-179:020:00:251
553604997769,5cyclictest615945-21kworker/u8:2+events_unbound23:00:241
553604997764,10cyclictest601797-21apt-get21:40:001
553604997461,12cyclictest647218-21kworker/u8:3+events_unbound00:00:021
553604997365,6cyclictest558302-21latency_hist19:25:001
553604997266,4cyclictest553474-21kworker/u8:2+events_unbound19:30:251
553604997265,5cyclictest598597-21kworker/u8:1+flush-179:021:30:011
553604997062,5cyclictest631839-21kworker/u8:3+events_unbound23:35:191
553604996962,5cyclictest613698-21kworker/u8:1+flush-179:022:35:001
553604996955,11cyclictest568103-21latency_hist19:55:021
553603996959,7cyclictest582946-21/usr/sbin/munin20:40:240
553604996861,5cyclictest615945-21kworker/u8:2+flush-179:022:45:011
553604996860,6cyclictest555342-21kworker/u8:0+flush-179:020:30:001
553604996858,7cyclictest652267-21apt-get00:20:001
553604996858,5cyclictest629431-21kworker/u8:1+events_unbound23:15:281
553604996761,4cyclictest601883-21kworker/u8:3+flush-179:022:10:261
553604996756,9cyclictest647179-40mandb00:00:011
553604996661,3cyclictest637643-21kworker/u8:2+flush-179:023:30:161
553604996661,3cyclictest629431-21kworker/u8:1+flush-179:023:15:011
553604996656,4cyclictest595684-21kworker/u8:0+flush-179:021:45:211
553603996657,7cyclictest578163-21/usr/sbin/munin20:25:240
553604996559,4cyclictest592285-21kworker/u8:3+events_unbound21:15:001
553604996558,6cyclictest564708-21kworker/u8:1+flush-179:019:55:211
553604996556,7cyclictest579489-21kworker/u8:1+flush-179:020:30:251
553603996555,7cyclictest649071-21latency_hist00:10:010
553603996555,7cyclictest649071-21latency_hist00:10:000
553604996449,6cyclictest555342-21kworker/u8:0+events_unbound20:55:011
553604996257,4cyclictest640835-21kworker/u8:1+flush-179:023:50:281
553604996252,7cyclictest574731-21latency_hist20:15:001
553603996255,4cyclictest576576-21kworker/u8:2+flush-179:020:20:260
553604996155,4cyclictest579489-21kworker/u8:1+flush-179:020:35:161
553604996155,4cyclictest579489-21kworker/u8:1+flush-179:020:35:161
553604996147,5cyclictest555342-21kworker/u8:0+events_unbound20:25:001
553603996154,5cyclictest598597-21kworker/u8:1+flush-179:021:40:210
553603996048,9cyclictest555342-21kworker/u8:0+events_unbound19:40:020
553604995954,4cyclictest615945-21kworker/u8:2+flush-179:022:20:251
553604995952,6cyclictest608288-21kworker/u8:0+events_unbound22:05:131
553604995952,5cyclictest592285-21kworker/u8:3+events_unbound21:20:221
553604995851,5cyclictest595684-21kworker/u8:0+events_unbound21:45:011
553603995847,8cyclictest647179-40mandb00:00:020
553604995749,6cyclictest615945-21kworker/u8:2+events_unbound22:40:011
553603995733,21cyclictest613220-21perf22:14:590
55360399571,3cyclictest645560-21munin-run23:54:590
553604995652,3cyclictest615945-21kworker/u8:2+flush-179:023:09:551
553604995652,3cyclictest555342-21kworker/u8:0+events_unbound19:45:231
553604995651,4cyclictest555342-21kworker/u8:0+flush-179:019:25:141
553604995647,6cyclictest629431-21kworker/u8:1+events_unbound23:20:171
553604995645,5cyclictest658301-21kworker/u8:1+events_unbound00:39:551
553603995648,6cyclictest657525-21/usr/sbin/munin00:35:180
553604995550,4cyclictest587706-21kworker/u8:2+events_unbound21:05:171
553604995550,4cyclictest555342-21kworker/u8:0+flush-179:019:15:251
553608995449,2cyclictest0-21swapper/219:22:592
553604995447,5cyclictest555342-21kworker/u8:0+flush-179:020:45:191
553603995446,6cyclictest601883-21kworker/u8:3+flush-179:022:00:010
553603995446,6cyclictest601883-21kworker/u8:3+flush-179:022:00:010
553603995444,8cyclictest636274-21apt-get23:25:110
553604995347,5cyclictest555342-21kworker/u8:0+events_unbound20:05:191
553604995347,4cyclictest579489-21kworker/u8:1+flush-179:020:55:211
553604995345,5cyclictest631839-21kworker/u8:3+events_unbound23:25:151
553603995347,4cyclictest553474-21kworker/u8:2+events_unbound19:45:010
553604995248,3cyclictest555342-21kworker/u8:0+flush-179:020:40:211
553604995247,4cyclictest601883-21kworker/u8:3+events_unbound22:00:241
553603995243,6cyclictest623063-21kworker/u8:3+flush-179:022:45:150
553603995144,4cyclictest637643-21kworker/u8:2+flush-179:023:49:590
553603994944,4cyclictest628062-21apt-get23:00:120
4577091490,13ptp4l61490irq/134-eno1-TxRx-019:10:003
553609994845,1cyclictest0-21swapper/319:23:003
553604994842,4cyclictest553474-21kworker/u8:2+flush-179:019:40:311
553603994842,5cyclictest631839-21kworker/u8:3+flush-179:023:35:210
553603994829,3cyclictest618252-21munin-run22:30:000
553604994742,4cyclictest640835-21kworker/u8:1+flush-179:023:45:161
553603994739,6cyclictest613698-21kworker/u8:1+events_unbound22:30:230
553603994720,5cyclictest0-21swapper/023:23:000
55360399471,43cyclictest685-21snmpd23:40:170
553603994642,3cyclictest714-21Xorg20:49:590
553603994440,3cyclictest610033-21idleruntime-cro22:05:000
553603994433,9cyclictest555342-21kworker/u8:0+flush-179:020:40:010
553603994433,9cyclictest555342-21kworker/u8:0+flush-179:020:40:010
553608994340,2cyclictest0-21swapper/220:02:592
55360899430,41cyclictest0-21swapper/220:20:592
553603994337,4cyclictest601883-21kworker/u8:3+events_unbound22:05:240
553603994336,5cyclictest555342-21kworker/u8:0+flush-179:019:25:170
553604994238,3cyclictest649106-21kworker/u8:0+events_unbound00:10:211
55360399428,30cyclictest605502-21apt-get21:50:110
553603994238,3cyclictest640835-21kworker/u8:1+flush-179:000:00:230
553603994235,5cyclictest555342-21kworker/u8:0+flush-179:020:50:240
553603994234,6cyclictest601883-21kworker/u8:3+flush-179:021:50:010
55360999410,40cyclictest0-21swapper/300:09:583
55360999410,40cyclictest0-21swapper/300:09:583
553608994138,2cyclictest0-21swapper/219:59:552
553603994137,3cyclictest714-21Xorg22:20:590
553603994132,7cyclictest566258-21latency_hist19:50:000
553603994130,9cyclictest573399-21apt-get20:10:140
553609994038,1cyclictest0-21swapper/320:20:593
553608994038,1cyclictest0-21swapper/200:24:582
553604994035,4cyclictest598597-21kworker/u8:1+events_unbound21:31:241
553604994035,4cyclictest598597-21kworker/u8:1+events_unbound21:31:241
553603994037,2cyclictest0-21swapper/023:19:560
553603994034,5cyclictest574741-21turbostat20:19:560
553603994032,6cyclictest553490-21latency_hist19:10:010
553603994031,7cyclictest591657-21kworker/u8:0+flush-179:021:05:250
553608993935,2cyclictest0-21swapper/220:14:582
553608993935,2cyclictest0-21swapper/219:13:592
553603993935,3cyclictest714-21Xorg21:33:590
553603993935,3cyclictest714-21Xorg21:33:590
553603993935,3cyclictest714-21Xorg20:07:590
553603993932,4cyclictest0-21swapper/020:01:590
553603993930,5cyclictest0-21swapper/020:30:590
553609993835,2cyclictest0-21swapper/320:02:593
553608993835,2cyclictest0-21swapper/221:40:002
553608993835,2cyclictest0-21swapper/200:08:002
553608993835,2cyclictest0-21swapper/200:07:592
553608993834,2cyclictest0-21swapper/223:54:542
553608993834,2cyclictest0-21swapper/222:56:582
553608993834,2cyclictest0-21swapper/200:13:592
55360899380,36cyclictest0-21swapper/223:27:592
55360899380,36cyclictest0-21swapper/200:38:592
55360899380,1cyclictest0-21swapper/220:27:592
553604993833,4cyclictest615945-21kworker/u8:2+events_unbound22:53:391
553604993833,4cyclictest595684-21kworker/u8:0+events_unbound21:50:131
553604993833,4cyclictest553474-21kworker/u8:2+flush-179:019:35:251
553603993831,5cyclictest564708-21kworker/u8:1+flush-179:019:55:180
553603993830,6cyclictest615945-21kworker/u8:2+flush-179:023:00:000
553609993734,2cyclictest0-21swapper/319:53:593
553608993734,2cyclictest0-21swapper/223:47:002
553608993734,2cyclictest0-21swapper/221:08:592
553608993734,2cyclictest0-21swapper/219:54:592
553608993734,2cyclictest0-21swapper/219:48:002
553608993734,2cyclictest0-21swapper/219:39:592
553608993733,2cyclictest0-21swapper/223:43:592
553608993733,2cyclictest0-21swapper/222:39:002
553608993733,2cyclictest0-21swapper/221:13:592
553608993733,2cyclictest0-21swapper/220:34:552
55360899370,35cyclictest0-21swapper/222:08:002
553603993734,2cyclictest714-21Xorg21:22:590
553603993733,2cyclictest592285-21kworker/u8:3+events_unbound21:15:160
553603993732,4cyclictest608288-21kworker/u8:0+flush-179:022:15:220
553603993732,3cyclictest590696-21systemd-run21:05:010
553603993731,4cyclictest649106-21kworker/u8:0+events_unbound00:15:240
553603993728,5cyclictest0-21swapper/022:39:000
55360999360,35cyclictest0-21swapper/319:58:593
55360899360,1cyclictest0-21swapper/223:01:592
55360899360,1cyclictest0-21swapper/200:02:002
553603993633,2cyclictest0-21swapper/022:53:590
553603993629,5cyclictest553474-21kworker/u8:2+events_unbound19:15:010
553609993533,1cyclictest0-21swapper/323:27:583
553609993533,1cyclictest0-21swapper/322:08:003
553609993533,1cyclictest0-21swapper/320:29:593
553609993533,1cyclictest0-21swapper/300:24:583
553609993532,2cyclictest0-21swapper/300:38:583
553609993529,5cyclictest576349-21turbostat20:20:013
553608993533,1cyclictest0-21swapper/220:20:002
553608993532,2cyclictest0-21swapper/221:49:582
553608993531,2cyclictest0-21swapper/223:34:542
553608993531,2cyclictest0-21swapper/222:13:592
553603993531,3cyclictest601815-21munin-run21:40:000
553603993529,4cyclictest588164-21hwlatdetect20:55:170
553603993528,5cyclictest637643-21kworker/u8:2+flush-179:023:35:000
553609993433,1cyclictest0-21swapper/322:39:003
553609993431,2cyclictest0-21swapper/320:41:593
553608993431,2cyclictest0-21swapper/223:11:592
553608993430,2cyclictest0-21swapper/221:54:002
553608993430,2cyclictest0-21swapper/220:40:002
553608993430,2cyclictest0-21swapper/220:39:592
55360899340,2cyclictest0-21swapper/223:37:592
553604993429,4cyclictest637643-21kworker/u8:2+events_unbound00:06:021
553604993429,4cyclictest637643-21kworker/u8:2+events_unbound00:06:021
553603993429,4cyclictest587706-21kworker/u8:2+flush-179:021:10:220
55360399341,2cyclictest714-21Xorg21:29:580
4577091340,6ptp4l441ktimers/320:35:003
553609993331,1cyclictest0-21swapper/322:46:593
553609993331,1cyclictest0-21swapper/322:27:593
553609993330,2cyclictest0-21swapper/323:46:593
553609993330,2cyclictest0-21swapper/321:30:593
553609993330,2cyclictest0-21swapper/321:30:593
553609993330,2cyclictest0-21swapper/321:08:593
553609993330,2cyclictest0-21swapper/319:47:593
553609993330,2cyclictest0-21swapper/300:15:003
553608993330,2cyclictest0-21swapper/220:59:552
55360899330,1cyclictest0-21swapper/222:29:592
55360399330,25cyclictest0-21swapper/023:05:590
553609993230,1cyclictest0-21swapper/321:40:003
553609993230,1cyclictest0-21swapper/300:04:003
553609993229,2cyclictest0-21swapper/322:56:593
553609993229,2cyclictest0-21swapper/320:14:583
553608993229,2cyclictest0-21swapper/223:23:002
553608993229,2cyclictest0-21swapper/223:17:582
553603993228,3cyclictest657287-21cron00:35:010
553603993225,5cyclictest555342-21kworker/u8:0+flush-179:019:20:230
553609993129,1cyclictest0-21swapper/323:54:543
553609993129,1cyclictest0-21swapper/321:13:593
553609993129,1cyclictest0-21swapper/319:39:593
553609993129,1cyclictest0-21swapper/319:13:593
553609993128,2cyclictest0-21swapper/323:02:593
553609993128,2cyclictest0-21swapper/321:49:593
55360899310,2cyclictest0-21swapper/222:54:552
55360899310,28cyclictest0-21swapper/220:44:592
553603993123,5cyclictest0-21swapper/019:52:590
55360999300,2cyclictest0-21swapper/320:55:143
553608993027,2cyclictest0-21swapper/221:40:022
553608993027,2cyclictest0-21swapper/221:00:142
553608993027,2cyclictest0-21swapper/220:50:132
55360899300,28cyclictest0-21swapper/222:15:132
55360899300,28cyclictest0-21swapper/221:55:142
55360899300,28cyclictest0-21swapper/221:55:142
553603993026,3cyclictest655701-21grep00:30:010
553609992927,1cyclictest0-21swapper/323:43:593
553609992927,1cyclictest0-21swapper/323:34:543
553609992927,1cyclictest0-21swapper/323:10:023
553609992927,1cyclictest0-21swapper/322:15:133
553609992927,1cyclictest0-21swapper/322:13:593
553609992927,1cyclictest0-21swapper/321:55:143
553609992927,1cyclictest0-21swapper/321:55:133
553609992926,2cyclictest0-21swapper/321:40:133
55360999290,28cyclictest0-21swapper/320:50:133
55360999290,28cyclictest0-21swapper/300:27:593
553608992926,2cyclictest0-21swapper/222:45:592
553608992926,2cyclictest0-21swapper/222:20:132
553608992926,2cyclictest0-21swapper/200:28:592
55360899290,27cyclictest0-21swapper/222:00:132
55360899290,27cyclictest0-21swapper/221:20:152
55360899290,27cyclictest0-21swapper/219:40:232
55360399291,26cyclictest714-21Xorg00:15:000
55360399291,26cyclictest560982-21/usr/sbin/munin19:30:220
4577091290,7ptp4l441ktimers/300:35:013
553609992826,1cyclictest0-21swapper/323:35:123
553609992826,1cyclictest0-21swapper/323:20:133
553609992825,2cyclictest0-21swapper/320:39:593
553609992825,2cyclictest0-21swapper/320:39:593
55360999280,27cyclictest0-21swapper/322:20:143
55360999280,27cyclictest0-21swapper/321:20:153
55360999280,27cyclictest0-21swapper/320:05:133
55360999280,27cyclictest0-21swapper/319:40:243
55360999280,27cyclictest0-21swapper/300:15:133
55360999280,26cyclictest0-21swapper/322:00:143
55360999280,26cyclictest0-21swapper/321:00:143
553608992825,2cyclictest0-21swapper/221:29:592
553608992825,2cyclictest0-21swapper/220:05:132
55360899280,2cyclictest0-21swapper/200:15:142
553603992825,2cyclictest0-21swapper/023:10:270
553603992824,3cyclictest653188-21fschecks_time00:20:160
55360399280,2cyclictest714-21Xorg22:40:590
4577091280,7ptp4l61490irq/134-eno1-TxRx-022:54:593
4577091280,6ptp4l441ktimers/319:35:003
553609992725,1cyclictest0-21swapper/323:17:593
553609992725,1cyclictest0-21swapper/321:54:003
4577091270,7ptp4l441ktimers/319:30:003
4577091270,6ptp4l441ktimers/321:20:013
4577091270,6ptp4l441ktimers/319:20:003
55360899260,2cyclictest0-21swapper/222:41:582
55360899260,24cyclictest0-21swapper/221:32:022
55360899260,24cyclictest0-21swapper/221:32:022
55360899260,24cyclictest0-21swapper/221:17:482
55360899260,24cyclictest0-21swapper/219:29:562
55360899260,24cyclictest0-21swapper/219:15:582
55360899260,24cyclictest0-21swapper/200:31:192
4577091260,6ptp4l61490irq/134-eno1-TxRx-023:10:013
4577091260,6ptp4l61490irq/134-eno1-TxRx-022:45:013
4577091260,4ptp4l441ktimers/322:34:593
553609992523,1cyclictest0-21swapper/321:27:473
55360999250,2cyclictest647160-21turbostat00:00:013
553608992523,2cyclictest0-21swapper/219:34:482
553608992522,2cyclictest0-21swapper/223:08:152
4577091250,6ptp4l441ktimers/320:50:003
553608992421,2cyclictest0-21swapper/220:45:592
553608992320,2cyclictest0-21swapper/222:32:592
553604991714,2cyclictest0-21swapper/119:10:011
553608991411,2cyclictest0-21swapper/223:57:462
5536089930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional