You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 03:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Fri Dec 19, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1636871998980,7cyclictest266-21systemd-journal00:04:001
1636870998878,7cyclictest1657727-21latency_hist20:15:000
1636871998672,10cyclictest1672593-21latency_hist21:00:021
1636871998672,10cyclictest1672593-21latency_hist21:00:011
1636870998480,3cyclictest1630942-21kworker/u8:0+flush-179:019:20:220
1636870998377,4cyclictest1636749-21latency_hist19:10:020
1636871997969,7cyclictest1706545-21latency_hist22:45:001
1636870997969,8cyclictest1690591-21/usr/sbin/munin21:55:230
1636870997967,6cyclictest1712318-21kworker/u8:2+events_unbound23:15:210
1636870997875,2cyclictest0-21swapper/023:13:590
1636870997769,6cyclictest1667470-21kworker/u8:0+flush-179:020:55:000
1636871997663,10cyclictest1722714-21latency_hist23:35:011
1636870997669,5cyclictest1712318-21kworker/u8:2+events_unbound00:00:010
1636870997262,7cyclictest1659307-21kworker/u8:2+events_unbound20:40:210
1636870997261,7cyclictest266-21systemd-journal21:05:010
1636870997167,3cyclictest1646306-21kworker/u8:1+flush-179:020:25:250
1636870997165,5cyclictest1721675-21kworker/u8:0+events_unbound23:45:200
1636870997065,4cyclictest1712318-21kworker/u8:2+flush-179:000:20:010
1636871996962,4cyclictest1659307-21kworker/u8:2+flush-179:020:25:131
1636870996964,4cyclictest1686962-21kworker/u8:3+flush-179:022:00:250
1636870996963,4cyclictest1683972-21kworker/u8:2+events_unbound22:15:020
1636870996863,4cyclictest1641590-21kworker/u8:2+flush-179:019:30:210
1636870996862,5cyclictest1646306-21kworker/u8:1+flush-179:020:45:260
1636870996861,6cyclictest1669433-21kworker/u8:3+flush-179:021:05:250
1636870996861,5cyclictest1669433-21kworker/u8:3+events_unbound21:25:140
1636871996759,6cyclictest1729339-21latency_hist23:55:011
1636871996758,6cyclictest1683972-21kworker/u8:2+flush-179:022:30:241
1636870996759,6cyclictest1734492-21latency_hist00:10:010
1636870996658,6cyclictest1646306-21kworker/u8:1+events_unbound20:30:200
1636871996559,3cyclictest0-21swapper/123:05:591
1636871996559,3cyclictest0-21swapper/123:05:591
1636871996558,5cyclictest1733147-21/usr/sbin/munin00:05:271
1636870996560,4cyclictest1701752-21kworker/u8:1+events_unbound22:35:220
1636870996559,5cyclictest1652442-21kworker/u8:0+events_unbound20:00:030
1636870996460,3cyclictest1706580-21kworker/u8:1+events_unbound22:50:250
1636870996454,8cyclictest1659334-21apt-get20:20:000
1636870996359,3cyclictest1737738-21kworker/u8:3+flush-179:000:25:240
1636870996358,4cyclictest1712318-21kworker/u8:2+events_unbound00:10:250
1636870996357,4cyclictest1693582-21kworker/u8:0+events_unbound22:10:140
1636870996357,4cyclictest1693582-21kworker/u8:0+events_unbound22:10:140
1636871996251,10cyclictest1652712-21apt-get20:00:011
1636870996257,4cyclictest1716579-21kworker/u8:1+events_unbound23:25:240
1636870996254,6cyclictest1701761-21latency_hist22:30:010
1636870996252,7cyclictest1650355-21kworker/u8:2+events_unbound19:55:190
1636871996151,8cyclictest1641538-21kworker/u8:1+flush-179:019:25:221
1636870996050,8cyclictest1654813-21apt-get20:05:120
1636877995956,2cyclictest0-21swapper/323:59:593
1636870995954,4cyclictest1646306-21kworker/u8:1+events_unbound19:50:170
1636870995953,4cyclictest1703349-21latency_hist22:35:010
1636876995856,1cyclictest0-21swapper/223:59:592
1636871995849,7cyclictest1684222-21/usr/sbin/munin21:35:251
1636870995853,4cyclictest1669433-21kworker/u8:3+events_unbound21:10:250
1636870995852,5cyclictest1646306-21kworker/u8:1+events_unbound19:40:230
1636870995851,6cyclictest1669433-21kworker/u8:3+events_unbound21:20:270
1636870995850,6cyclictest1744051-21latency_hist00:40:010
1636870995850,6cyclictest1669433-21kworker/u8:3+flush-179:021:30:260
1636871995748,6cyclictest1682936-21kworker/u8:0+flush-179:021:30:231
1636870995752,4cyclictest1675802-21kworker/u8:2+events_unbound21:15:180
1636870995651,4cyclictest1716579-21kworker/u8:1+events_unbound23:35:190
1636870995651,4cyclictest1716579-21kworker/u8:1+events_unbound23:35:180
1636870995647,3cyclictest1682936-21kworker/u8:0+events_unbound21:35:250
163687699550,1cyclictest0-21swapper/221:40:592
1636871995547,6cyclictest1649514-21latency_hist19:50:001
1636871995546,7cyclictest1729882-21diskstats23:55:161
1636870995550,4cyclictest1667470-21kworker/u8:0+events_unbound20:55:240
1636870995550,4cyclictest1667470-21kworker/u8:0+events_unbound20:55:240
1636870995449,4cyclictest1683972-21kworker/u8:2+flush-179:022:05:200
1636870995348,4cyclictest1659307-21kworker/u8:2+events_unbound20:20:220
1636870995345,6cyclictest1639967-21latency_hist19:20:000
1636870995247,4cyclictest1683972-21kworker/u8:2+flush-179:021:45:160
1636870995245,6cyclictest1716579-21kworker/u8:1+events_unbound23:20:240
1636870995245,6cyclictest1662775-21kworker/u8:3+flush-179:020:35:010
1636877995148,1cyclictest0-21swapper/319:53:593
1636876995148,2cyclictest0-21swapper/219:53:592
1636871995148,2cyclictest0-21swapper/122:28:591
1636871995144,5cyclictest1644735-21latency_hist19:35:001
1636871995043,4cyclictest1737645-21kworker/u8:1+flush-179:000:20:221
1636871995041,6cyclictest1664857-21/usr/sbin/munin20:35:191
1636870995043,6cyclictest1646320-21kworker/u8:3+flush-179:019:45:260
4577091490,13ptp4l441ktimers/319:10:013
1636876994944,3cyclictest0-21swapper/220:43:592
1636871994942,5cyclictest1721125-21latency_hist23:30:011
1636871994942,5cyclictest1710077-21kworker/u8:0+flush-179:023:10:241
1636877994845,1cyclictest0-21swapper/320:43:593
1636871994842,4cyclictest1700161-21kworker/u8:0+flush-179:022:25:001
1636871994840,6cyclictest1704781-21kworker/u8:0+flush-179:022:40:001
1636871994834,12cyclictest1686962-21kworker/u8:3+flush-179:022:05:251
1636870994844,3cyclictest1683972-21kworker/u8:2+flush-179:022:40:220
1636870994843,4cyclictest1724234-21kworker/u8:4+flush-179:023:50:260
1636871994741,4cyclictest1674418-21kworker/u8:1+flush-179:021:10:001
1636871994738,7cyclictest1659307-21kworker/u8:2+flush-179:020:20:251
1636870994739,4cyclictest1737645-21kworker/u8:1+events_unbound00:20:180
1636871994640,4cyclictest1737691-21latency_hist00:20:011
163687199461,44cyclictest1-21systemd21:05:001
1636870994641,4cyclictest1706580-21kworker/u8:1+flush-179:023:05:270
1636870994641,4cyclictest1706580-21kworker/u8:1+flush-179:023:05:270
1636877994542,2cyclictest0-21swapper/323:05:593
1636877994542,2cyclictest0-21swapper/323:05:593
1636876994542,2cyclictest0-21swapper/223:05:592
1636876994542,2cyclictest0-21swapper/223:05:582
1636871994434,9cyclictest1646601-21apt-get19:40:121
1636870994440,3cyclictest1712318-21kworker/u8:2+events_unbound00:30:140
1636870994437,5cyclictest1690352-21latency_hist21:55:000
1636870994436,6cyclictest1726166-21latency_hist23:45:010
163687699430,40cyclictest0-21swapper/220:38:592
1636870994337,5cyclictest1712318-21kworker/u8:2+events_unbound23:30:140
163687799420,41cyclictest0-21swapper/319:19:553
1636871994237,4cyclictest1653514-21kworker/u8:3+flush-179:020:10:241
163687199422,4cyclictest1669414-21turbostat20:54:551
163687199420,40cyclictest714-21Xorg21:44:591
163687799410,40cyclictest0-21swapper/320:38:583
163687699400,38cyclictest0-21swapper/222:27:002
1636871994036,3cyclictest1646354-21basename19:40:011
1636871994035,3cyclictest1740901-21systemd-run00:30:001
1636871994033,5cyclictest1716579-21kworker/u8:1+flush-179:023:35:221
1636871994033,5cyclictest1716579-21kworker/u8:1+flush-179:023:35:221
163687199401,3cyclictest714-21Xorg22:16:591
163687199401,2cyclictest714-21Xorg19:21:001
163687199400,32cyclictest1742467-21turbostat00:39:551
1636870994036,3cyclictest1712318-21kworker/u8:2+events_unbound00:00:210
1636870994036,3cyclictest1706580-21kworker/u8:1+flush-179:022:55:150
1636870994032,3cyclictest1712318-21kworker/u8:2+events_unbound23:00:220
163687799390,2cyclictest0-21swapper/319:40:593
163687699390,36cyclictest0-21swapper/222:54:552
163687699390,36cyclictest0-21swapper/222:44:592
163687699390,36cyclictest0-21swapper/220:49:552
1636871993935,3cyclictest1683972-21kworker/u8:2+flush-179:021:49:101
1636871993935,3cyclictest1654588-21systemd-run20:05:011
1636871993932,4cyclictest1710077-21kworker/u8:0+events_unbound23:00:241
163687199390,37cyclictest714-21Xorg22:14:001
163687199390,37cyclictest714-21Xorg22:13:591
163687199390,2cyclictest714-21Xorg20:43:001
1636870993936,2cyclictest714-21Xorg19:26:590
1636870993934,4cyclictest1685601-21kworker/u8:1+flush-179:021:40:290
163687099390,38cyclictest211irq_work/022:48:590
163687799380,36cyclictest0-21swapper/322:26:593
1636876993835,2cyclictest0-21swapper/219:31:592
163687699380,2cyclictest0-21swapper/221:24:552
1636871993834,2cyclictest714-21Xorg21:19:001
1636871993833,4cyclictest1724234-21kworker/u8:4+flush-179:023:40:231
1636871993831,5cyclictest1646306-21kworker/u8:1+flush-179:020:30:261
1636877993735,1cyclictest0-21swapper/321:53:593
1636877993735,1cyclictest0-21swapper/319:31:593
1636877993731,5cyclictest1719544-21turbostat23:25:003
163687799370,36cyclictest0-21swapper/322:44:593
1636876993734,2cyclictest0-21swapper/221:54:002
1636876993734,1cyclictest0-21swapper/200:09:592
1636876993733,2cyclictest0-21swapper/221:49:552
1636876993733,2cyclictest0-21swapper/200:38:592
163687699370,35cyclictest0-21swapper/219:29:592
163687699370,2cyclictest0-21swapper/219:40:582
1636871993734,2cyclictest0-21swapper/119:20:001
1636871993733,2cyclictest714-21Xorg22:01:591
1636871993731,5cyclictest1659402-21unin-run20:20:011
1636871993731,4cyclictest1711361-21systemd-run23:00:011
1636871993731,4cyclictest1688799-21kworker/u8:1+flush-179:022:00:011
1636870993733,3cyclictest1637015-21kworker/u8:3+flush-179:019:10:130
1636877993634,1cyclictest0-21swapper/300:09:593
1636877993633,2cyclictest0-21swapper/300:38:583
163687799360,35cyclictest0-21swapper/322:54:553
163687799360,35cyclictest0-21swapper/320:49:563
163687799360,35cyclictest0-21swapper/319:29:593
1636876993633,2cyclictest0-21swapper/222:56:592
1636876993633,2cyclictest0-21swapper/219:24:582
163687699360,1cyclictest0-21swapper/220:09:552
1636871993631,4cyclictest1706580-21kworker/u8:1+events_unbound22:45:251
1636871993628,5cyclictest0-21swapper/123:19:591
163687199362,30cyclictest1726425-21apt-get23:45:121
163687199360,1cyclictest0-21swapper/121:23:581
1636870993629,6cyclictest1643383-21kworker/u8:0+flush-179:019:35:210
1636877993533,1cyclictest0-21swapper/322:56:593
1636877993533,1cyclictest0-21swapper/321:49:553
1636877993533,1cyclictest0-21swapper/319:24:593
1636877993531,2cyclictest0-21swapper/320:05:003
1636876993532,2cyclictest0-21swapper/223:42:582
163687699350,33cyclictest0-21swapper/222:32:592
1636871993531,2cyclictest714-21Xorg19:51:591
1636871993523,9cyclictest1682145-21apt-get21:30:001
163687199351,5cyclictest1689022-21apt-get21:50:141
1636877993432,1cyclictest0-21swapper/323:42:593
1636876993431,2cyclictest0-21swapper/223:25:002
1636876993430,2cyclictest0-21swapper/223:36:592
1636876993430,2cyclictest0-21swapper/223:36:592
163687799330,32cyclictest0-21swapper/322:32:593
163687199330,5cyclictest1741123-21apt-get00:30:141
1636877993230,1cyclictest0-21swapper/320:24:583
1636876993230,1cyclictest0-21swapper/220:24:582
1636876993230,1cyclictest0-21swapper/220:04:592
1636876993229,2cyclictest0-21swapper/222:07:592
1636871993227,4cyclictest1706580-21kworker/u8:1+flush-179:022:50:221
1636877993129,1cyclictest0-21swapper/323:36:593
1636877993129,1cyclictest0-21swapper/323:36:593
1636877993129,1cyclictest0-21swapper/322:07:593
1636876993128,2cyclictest0-21swapper/220:52:002
1636876993127,3cyclictest0-21swapper/200:22:582
1636876993127,2cyclictest0-21swapper/222:24:552
1636871993127,3cyclictest1669248-21unixbench-2d20:45:251
163687199310,28cyclictest0-21swapper/120:05:131
1636870993125,5cyclictest1683972-21kworker/u8:2+events_unbound22:24:010
1636877993028,1cyclictest0-21swapper/322:24:553
1636877993028,1cyclictest0-21swapper/300:30:003
1636877993027,1cyclictest0-21swapper/300:22:593
163687799300,29cyclictest0-21swapper/321:25:153
163687799300,28cyclictest0-21swapper/320:50:163
163687799300,28cyclictest0-21swapper/300:10:143
163687799300,1cyclictest0-21swapper/323:45:133
1636876993027,2cyclictest0-21swapper/221:25:142
1636876993027,2cyclictest0-21swapper/200:30:002
1636876993026,2cyclictest0-21swapper/200:19:552
163687699300,2cyclictest0-21swapper/223:10:152
163687699300,28cyclictest0-21swapper/223:50:142
163687699300,28cyclictest0-21swapper/223:00:142
1636877992927,1cyclictest0-21swapper/323:50:143
1636877992926,2cyclictest0-21swapper/320:05:143
1636877992926,2cyclictest0-21swapper/300:19:553
163687799290,2cyclictest0-21swapper/321:15:133
163687799290,27cyclictest0-21swapper/323:10:153
163687799290,27cyclictest0-21swapper/323:00:143
163687799290,27cyclictest0-21swapper/322:16:593
163687799290,1cyclictest0-21swapper/321:24:563
1636876992926,2cyclictest0-21swapper/221:15:132
1636876992926,2cyclictest0-21swapper/220:25:132
1636876992926,2cyclictest0-21swapper/219:14:592
1636876992925,2cyclictest0-21swapper/223:15:592
1636876992925,2cyclictest0-21swapper/200:30:142
163687699290,2cyclictest0-21swapper/223:45:132
163687699290,27cyclictest0-21swapper/200:10:152
1636871992925,3cyclictest1718988-21ntp_kernel_pll_23:20:231
4577091280,7ptp4l61490irq/134-eno1-TxRx-021:14:563
4577091280,6ptp4l441ktimers/320:15:013
1636877992826,1cyclictest0-21swapper/300:30:133
163687799280,27cyclictest0-21swapper/300:00:033
163687799280,21cyclictest0-21swapper/320:25:143
1636876992825,2cyclictest0-21swapper/200:00:022
1636876992824,2cyclictest0-21swapper/222:16:592
163687699280,2cyclictest0-21swapper/221:05:592
163687699280,24cyclictest0-21swapper/219:18:002
1636871992824,3cyclictest714-21Xorg19:11:591
4577091270,6ptp4l61490irq/134-eno1-TxRx-019:14:563
4577091270,6ptp4l441ktimers/321:00:003
4577091270,6ptp4l441ktimers/321:00:003
4577091270,5ptp4l61490irq/134-eno1-TxRx-020:00:003
163687799270,26cyclictest0-21swapper/322:10:593
163687799270,26cyclictest0-21swapper/322:10:593
1636876992724,2cyclictest0-21swapper/219:55:592
1636871992721,4cyclictest1735169-21gltestperf00:10:191
1636871992721,4cyclictest0-21swapper/121:15:001
4577091260,6ptp4l61490irq/134-eno1-TxRx-023:35:013
4577091260,6ptp4l61490irq/134-eno1-TxRx-022:00:003
4577091260,6ptp4l61490irq/134-eno1-TxRx-021:39:563
4577091260,6ptp4l441ktimers/320:19:553
4577091260,5ptp4l441ktimers/322:04:553
4577091260,5ptp4l441ktimers/320:35:003
4577091260,5ptp4l441ktimers/319:40:003
1636877992624,1cyclictest0-21swapper/323:15:583
1636877992623,1cyclictest0-21swapper/321:40:593
163687799260,1cyclictest0-21swapper/321:01:003
1636876992624,1cyclictest0-21swapper/223:34:592
163687699260,24cyclictest0-21swapper/222:11:002
163687699260,24cyclictest0-21swapper/222:10:592
4577091250,5ptp4l61490irq/134-eno1-TxRx-023:30:003
4577091250,5ptp4l441ktimers/321:34:553
4577091250,4ptp4l441ktimers/321:09:573
1636877992523,1cyclictest0-21swapper/322:38:593
1636876992522,2cyclictest0-21swapper/222:38:592
1636876992522,2cyclictest0-21swapper/222:01:592
1636876992522,2cyclictest0-21swapper/220:55:592
1636876992522,2cyclictest0-21swapper/220:55:582
1636876992522,2cyclictest0-21swapper/220:34:002
1636876992521,1cyclictest0-21swapper/223:29:592
163687699250,2cyclictest0-21swapper/221:00:592
4577091240,5ptp4l61490irq/134-eno1-TxRx-022:49:553
4577091240,3ptp4l61490irq/134-eno1-TxRx-019:49:563
1636876992421,2cyclictest0-21swapper/220:18:002
1636876992320,2cyclictest0-21swapper/219:46:592
231999220,13rtkit-daemon23940-21sendmail-mta19:09:421
1636876992018,1cyclictest0-21swapper/221:15:002
1636876991714,2cyclictest0-21swapper/221:31:002
1636876991612,2cyclictest0-21swapper/221:59:552
1636876991311,1cyclictest0-21swapper/221:39:592
16368769996,2cyclictest0-21swapper/220:14:552
16368769970,6cyclictest0-21swapper/222:45:092
16368769952,2cyclictest0-21swapper/219:35:022
16368769930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional