You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 11:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sun Dec 07, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4008699910290,9cyclictest501363-21latency_hist00:20:020
4008699910290,9cyclictest501363-21latency_hist00:20:010
400871999982,15cyclictest484848-21latency_hist23:30:011
400871999571,15cyclictest473185-21latency_hist22:55:001
400869999370,20cyclictest490126-21/usr/sbin/munin23:45:250
400871998676,7cyclictest408949-21kworker/u8:4+flush-179:020:00:001
400871998577,6cyclictest271-21systemd-journal23:00:011
400871998274,5cyclictest407126-21apt-get19:30:001
400871997872,4cyclictest455349-21kworker/u8:1+flush-179:022:20:261
400869997867,8cyclictest458821-21latency_hist22:10:010
400869997667,6cyclictest448995-21kworker/u8:0+flush-179:021:50:260
400871997364,7cyclictest498397-21/usr/sbin/munin00:10:241
400869997266,4cyclictest491711-21kworker/u8:1+flush-179:000:00:000
400871997140,16cyclictest460404-21latency_hist22:15:011
400869997160,8cyclictest463653-21latency_hist22:25:000
400869997060,7cyclictest456981-21latency_hist22:05:010
400869996959,7cyclictest489889-21latency_hist23:45:010
400871996856,9cyclictest271-21systemd-journal20:35:001
400869996858,7cyclictest493057-21latency_hist23:55:010
400871996761,4cyclictest418249-21kworker/u8:0+flush-179:021:00:261
400871996756,9cyclictest271-21systemd-journal22:19:591
400871996756,8cyclictest439464-21latency_hist21:10:011
400871996656,8cyclictest439722-21apt-get21:10:141
400869996654,8cyclictest462045-21true22:20:000
400869996630,26cyclictest486696-21latency_hist23:35:010
400869996454,8cyclictest442916-21apt-get21:20:110
400871996352,8cyclictest476640-21latency_hist23:05:021
400869996360,2cyclictest271-21systemd-journal19:40:010
400871996252,8cyclictest421935-21/usr/sbin/munin20:15:241
400869996253,6cyclictest400728-21latency_hist19:10:010
400871996050,7cyclictest445831-21latency_hist21:30:001
400869996049,8cyclictest405538-21latency_hist19:25:010
400869995950,7cyclictest502940-21latency_hist00:25:010
400871995852,5cyclictest448995-21kworker/u8:0+flush-179:022:00:001
400871995850,6cyclictest271-21systemd-journal00:06:451
400871995747,8cyclictest271-21systemd-journal20:55:431
40086999561,51cyclictest0-21swapper/019:25:140
400871995550,3cyclictest469475-21kworker/u8:0+flush-179:022:45:001
400871995544,9cyclictest481932-21apt-get23:20:131
400869995544,8cyclictest432844-21latency_hist20:50:010
40086999550,2cyclictest723-21Xorg21:33:590
400871995435,17cyclictest403941-21awk19:20:001
400869995445,7cyclictest271-21systemd-journal22:27:420
400869995443,9cyclictest441305-21apt-get21:15:120
40087199531,45cyclictest0-21swapper/123:30:131
400869995342,8cyclictest502-21rs:main0
400869995333,16cyclictest403944-21run-parts19:20:000
40086999530,2cyclictest723-21Xorg20:35:580
40086999521,49cyclictest408983-21idleruntime-cro19:35:000
400871995138,11cyclictest496566-21kworker/u8:2+flush-179:000:35:151
400869995142,7cyclictest465242-21kworker/u8:3+flush-179:023:20:000
400869995136,13cyclictest0-21swapper/019:59:560
400871995045,3cyclictest418567-21systemd-run20:05:011
400871995041,6cyclictest450612-21latency_hist21:45:011
400871995022,15cyclictest475-21dbus-daemon23:50:001
400869995032,17cyclictest449028-21idleruntime-cro21:40:000
400871994943,4cyclictest387873-21kworker/u8:3+flush-179:019:20:141
400871994933,14cyclictest447656-21/usr/sbin/munin21:35:171
4670791480,13ptp4l61690irq/136-eno1-TxRx-019:10:003
400871994842,5cyclictest448995-21kworker/u8:0+flush-179:021:55:001
400869994841,5cyclictest413782-21kworker/u8:2+flush-179:020:15:010
400869994828,16cyclictest436551-21apt-get21:00:150
40087199471,28cyclictest408972-21munin-run19:35:011
40087199470,38cyclictest410551-21cron19:39:591
400869994739,6cyclictest425131-21/usr/sbin/munin20:25:170
400871994629,15cyclictest271-21systemd-journal23:14:591
400871994628,15cyclictest0-21swapper/120:39:551
400871994535,8cyclictest418784-21apt-get20:05:131
400869994514,29cyclictest478215-21sshd23:09:430
40086999451,41cyclictest1-21systemd20:55:430
40087699440,43cyclictest437889-21turbostat21:05:003
400871994441,2cyclictest491711-21kworker/u8:1+flush-179:023:50:241
400871994434,7cyclictest421697-21latency_hist20:15:001
400871994433,7cyclictest491711-21kworker/u8:1+flush-179:000:00:161
400869994435,7cyclictest496577-21latency_hist00:05:010
400869994424,18cyclictest0-21swapper/022:40:140
40086999442,38cyclictest414003-21apt-get19:50:140
40087499435,2cyclictest0-21swapper/220:35:582
400871994339,3cyclictest411088-21kworker/u8:0+flush-179:019:40:261
400871994338,4cyclictest496566-21kworker/u8:2+events_unbound00:20:251
400871994338,4cyclictest486730-21kworker/u8:3+flush-179:023:35:271
400869994335,6cyclictest418249-21kworker/u8:0+flush-179:020:25:000
400869994323,18cyclictest444939-21hwlatdetect21:25:170
400874994238,3cyclictest0-21swapper/221:33:592
400871994222,16cyclictest426477-21latency_hist20:30:021
40087199421,39cyclictest506118-21cron00:35:011
40087199421,32cyclictest458823-21munin-run22:10:011
40087199420,38cyclictest433360-21apt-get20:50:151
40087199420,2cyclictest0-21swapper/121:30:161
400869994237,4cyclictest488227-21awk23:35:260
400869994233,7cyclictest471600-21latency_hist22:50:010
400869994223,16cyclictest0-21swapper/021:05:240
400869994222,18cyclictest401605-21latency19:10:180
40086999421,4cyclictest271-21systemd-journal20:09:590
400876994138,2cyclictest0-21swapper/323:08:593
40087699410,40cyclictest0-21swapper/320:48:593
40087699410,39cyclictest447434-21turbostat21:35:003
400874994139,1cyclictest0-21swapper/219:39:592
400871994138,2cyclictest0-21swapper/123:16:001
400869994122,17cyclictest0-21swapper/020:55:010
40086999411,3cyclictest723-21Xorg22:56:580
40086999410,2cyclictest0-21swapper/022:33:590
400871994030,8cyclictest467060-21/usr/sbin/munin22:35:221
40087199401,2cyclictest723-21Xorg22:33:591
400869994018,20cyclictest484865-21apt-get23:30:010
400869994018,20cyclictest0-21swapper/020:00:430
40086999401,35cyclictest3415387-21munin-node00:30:140
400876993934,4cyclictest407127-21turbostat19:30:003
40087699390,4cyclictest444260-21turbostat21:25:013
40087699390,38cyclictest0-21swapper/322:08:583
400871993934,4cyclictest408949-21kworker/u8:4+flush-179:019:45:251
400871993920,16cyclictest503375-21tr00:25:171
400869993936,2cyclictest507762-21systemd-run00:40:010
400876993833,3cyclictest499756-21turbostat00:15:003
40087699380,37cyclictest436278-21turbostat21:00:003
40087699380,36cyclictest0-21swapper/322:35:593
400874993835,2cyclictest0-21swapper/200:12:002
400874993834,2cyclictest0-21swapper/223:31:002
40087499380,36cyclictest0-21swapper/200:05:582
400871993832,4cyclictest423304-21turbostat20:24:551
40087199382,29cyclictest723-21Xorg21:45:591
40087199382,28cyclictest0-21swapper/123:05:591
40086999383,28cyclictest723-21Xorg23:02:580
400869993830,5cyclictest501388-21kworker/u8:0+flush-179:000:25:000
400876993731,5cyclictest445837-21turbostat21:30:003
400876993731,4cyclictest428073-21turbostat20:35:003
400874993734,2cyclictest0-21swapper/219:24:542
40087499370,2cyclictest0-21swapper/220:42:592
400871993733,3cyclictest494994-40gzip00:00:011
400871993733,2cyclictest723-21Xorg23:41:001
400871993723,12cyclictest441746-21cut21:15:171
40087199372,29cyclictest0-21swapper/119:52:591
400869993733,2cyclictest0-21swapper/019:47:590
400869993713,21cyclictest467174-21cat22:35:130
400876993633,2cyclictest0-21swapper/323:21:583
40087699360,35cyclictest0-21swapper/323:29:563
40087699360,35cyclictest0-21swapper/323:00:593
40087699360,35cyclictest0-21swapper/320:14:593
40087699360,35cyclictest0-21swapper/319:46:593
400874993633,2cyclictest0-21swapper/222:33:582
400874993633,2cyclictest0-21swapper/221:27:592
400874993633,2cyclictest0-21swapper/219:25:592
400874993633,2cyclictest0-21swapper/219:17:592
400874993632,2cyclictest0-21swapper/223:55:002
40087499360,1cyclictest0-21swapper/219:34:592
40087199361,30cyclictest0-21swapper/122:25:161
400869993630,4cyclictest418249-21kworker/u8:0+flush-179:020:15:250
40086999362,26cyclictest723-21Xorg19:43:590
400869993614,18cyclictest0-21swapper/020:40:170
40086999361,31cyclictest479806-21cron23:15:000
400876993530,4cyclictest452207-21turbostat21:50:003
40087699350,1cyclictest0-21swapper/321:07:593
400874993532,2cyclictest0-21swapper/222:46:592
40087499350,1cyclictest0-21swapper/221:16:592
40087199352,28cyclictest0-21swapper/122:04:591
40087199352,28cyclictest0-21swapper/120:48:591
40087199352,27cyclictest723-21Xorg19:11:591
400871993519,14cyclictest430263-21fschecks_count20:40:191
40087199351,3cyclictest0-21swapper/100:15:151
40087199351,3cyclictest0-21swapper/100:15:151
40086999352,28cyclictest723-21Xorg23:23:590
40086999352,28cyclictest0-21swapper/022:10:590
40086999351,27cyclictest723-21Xorg21:41:590
400876993432,1cyclictest0-21swapper/319:23:593
400876993431,2cyclictest0-21swapper/322:17:593
400876993428,5cyclictest412167-21turbostat19:44:593
400876993428,5cyclictest402357-21turbostat19:19:553
400874993431,2cyclictest0-21swapper/222:21:592
400874993431,2cyclictest0-21swapper/220:59:562
40086999343,27cyclictest723-21Xorg20:34:000
400869993431,2cyclictest0-21swapper/022:50:590
400869993425,5cyclictest0-21swapper/021:55:120
40086999342,28cyclictest0-21swapper/021:11:590
40086999341,28cyclictest0-21swapper/021:45:590
400876993332,1cyclictest0-21swapper/322:44:543
400876993331,1cyclictest0-21swapper/323:59:593
40087699330,4cyclictest441059-21turbostat21:15:003
400874993329,2cyclictest0-21swapper/221:06:592
400871993316,16cyclictest0-21swapper/121:20:241
40087199331,28cyclictest470673-21fschecks_time22:45:171
40086999333,28cyclictest497778-21mailstats00:05:270
400876993230,1cyclictest0-21swapper/300:22:003
400876993227,4cyclictest471611-21turbostat22:50:003
40087699320,2cyclictest416934-21turbostat20:00:013
400874993229,2cyclictest0-21swapper/220:05:592
400874993229,2cyclictest0-21swapper/219:41:592
400876993129,1cyclictest0-21swapper/300:19:563
400876993129,1cyclictest0-21swapper/300:19:553
400876993126,4cyclictest453788-21turbostat21:55:003
400874993128,2cyclictest0-21swapper/222:58:002
4670791300,11ptp4l61690irq/136-eno1-TxRx-019:39:563
40087699300,29cyclictest0-21swapper/323:43:003
400874993028,1cyclictest0-21swapper/221:54:592
400874993028,1cyclictest0-21swapper/221:40:002
400874993027,2cyclictest0-21swapper/223:16:002
400874993027,2cyclictest0-21swapper/219:58:592
400874993027,2cyclictest0-21swapper/200:05:002
400876992927,1cyclictest0-21swapper/323:38:583
400876992927,1cyclictest0-21swapper/300:29:583
400876992927,1cyclictest0-21swapper/300:05:153
40087699290,27cyclictest0-21swapper/323:15:173
400874992926,2cyclictest0-21swapper/221:55:122
400874992926,2cyclictest0-21swapper/221:00:152
400874992926,2cyclictest0-21swapper/200:30:142
400874992925,3cyclictest0-21swapper/220:48:592
400874992925,2cyclictest0-21swapper/220:17:592
40087499290,27cyclictest0-21swapper/223:20:142
40087499290,27cyclictest0-21swapper/222:38:592
40087499290,27cyclictest0-21swapper/222:25:162
40087499290,27cyclictest0-21swapper/220:50:032
40087499290,27cyclictest0-21swapper/219:50:142
40087499290,27cyclictest0-21swapper/200:15:142
40087499290,27cyclictest0-21swapper/200:15:142
4670791280,6ptp4l441ktimers/322:25:013
400876992826,1cyclictest0-21swapper/323:15:003
400876992825,2cyclictest0-21swapper/322:10:173
40087699280,27cyclictest465248-21turbostat22:30:003
40087699280,26cyclictest0-21swapper/321:35:593
40087699280,26cyclictest0-21swapper/321:15:143
40087699280,26cyclictest0-21swapper/319:30:023
400874992825,2cyclictest0-21swapper/200:35:132
400874992824,2cyclictest0-21swapper/220:33:592
400874992823,2cyclictest0-21swapper/222:50:592
40087499280,26cyclictest0-21swapper/223:15:002
4670791270,6ptp4l441ktimers/323:55:003
4670791270,6ptp4l441ktimers/300:05:013
4670791270,26ptp4l71-21kworker/3:1+events20:50:163
400876992725,1cyclictest0-21swapper/323:49:563
400876992725,1cyclictest0-21swapper/320:20:133
400876992725,1cyclictest0-21swapper/320:06:583
400876992724,2cyclictest0-21swapper/322:56:593
40087699270,26cyclictest0-21swapper/319:12:593
40087699270,1cyclictest0-21swapper/320:40:163
40087699270,1cyclictest0-21swapper/320:25:133
400874992725,1cyclictest0-21swapper/200:25:002
400874992724,2cyclictest0-21swapper/220:00:162
400874992724,2cyclictest0-21swapper/219:48:002
400874992724,2cyclictest0-21swapper/219:14:522
400874992724,2cyclictest0-21swapper/200:25:172
40087499270,1cyclictest0-21swapper/220:29:592
4670791260,6ptp4l61690irq/136-eno1-TxRx-000:35:003
4670791260,5ptp4l61690irq/136-eno1-TxRx-022:00:003
40087699260,1cyclictest0-21swapper/322:50:133
400874992623,2cyclictest0-21swapper/223:40:592
400874992622,2cyclictest0-21swapper/220:24:562
40087499260,24cyclictest0-21swapper/223:39:552
40087199261,5cyclictest400959-21latency_hist19:10:011
4670791250,6ptp4l441ktimers/322:34:563
4670791250,6ptp4l441ktimers/322:04:553
400876992523,1cyclictest0-21swapper/320:39:593
40087699250,24cyclictest0-21swapper/321:41:593
40087699250,23cyclictest0-21swapper/320:15:153
40087699250,1cyclictest0-21swapper/319:54:553
400874992522,2cyclictest0-21swapper/223:06:582
400874992522,2cyclictest0-21swapper/223:02:592
4670791240,4ptp4l61690irq/136-eno1-TxRx-000:39:563
400874992421,2cyclictest0-21swapper/223:48:582
400874992421,2cyclictest0-21swapper/223:25:592
400874992421,2cyclictest0-21swapper/222:40:152
400874992420,2cyclictest0-21swapper/222:04:552
4670791230,4ptp4l61690irq/136-eno1-TxRx-020:04:563
400874992321,1cyclictest0-21swapper/221:14:592
400874992320,2cyclictest0-21swapper/223:55:032
400874992320,2cyclictest0-21swapper/221:23:592
400874992218,2cyclictest0-21swapper/221:48:592
4670791200,3ptp4l61690irq/136-eno1-TxRx-023:34:563
400874991916,2cyclictest0-21swapper/222:05:592
400874991915,2cyclictest421711-21turbostat20:15:002
400874991815,2cyclictest0-21swapper/222:16:002
400874991815,2cyclictest0-21swapper/221:41:592
40087499129,2cyclictest0-21swapper/222:13:582
4008749996,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional