You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 05:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Thu Mar 26, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5299039911081,27cyclictest576475-21kworker/u8:2+flush-179:021:35:011
5299019910084,12cyclictest266-21systemd-journal19:15:000
529903999184,5cyclictest536201-21kworker/u8:3+flush-179:019:40:001
529903999182,7cyclictest550485-21latency_hist20:15:011
529901998985,3cyclictest567980-21kworker/u8:1+events_unbound21:15:240
529903998879,6cyclictest532995-21latency_hist19:20:001
529903997969,7cyclictest597753-21latency_hist22:40:001
529901997873,4cyclictest628308-21kworker/u8:1+events_unbound00:15:260
52990399770,74cyclictest714-21Xorg00:34:591
529901997671,3cyclictest536201-21kworker/u8:3+events_unbound20:00:210
529901997667,7cyclictest565590-21/usr/sbin/munin21:00:220
529901997570,4cyclictest608355-21kworker/u8:3+events_unbound23:10:250
529901997369,3cyclictest633082-21kworker/u8:1+events_unbound00:35:200
529901997366,5cyclictest607308-21latency_hist23:10:010
529903997263,7cyclictest560572-21latency_hist20:45:001
529903997162,7cyclictest570112-21latency_hist21:15:001
529903996963,4cyclictest548943-21kworker/u8:0+flush-179:020:20:211
529901996965,3cyclictest612068-21kworker/u8:1+events_unbound23:39:550
529901996962,5cyclictest563776-21kworker/u8:0+events_unbound20:55:250
529901996962,5cyclictest563776-21kworker/u8:0+events_unbound20:55:240
529903996862,4cyclictest576475-21kworker/u8:2+events_unbound21:35:231
529903996661,4cyclictest620339-21kworker/u8:3+events_unbound00:04:031
529903996661,4cyclictest584756-21kworker/u8:2+flush-179:022:04:551
529901996662,3cyclictest584756-21kworker/u8:2+events_unbound22:05:000
529901996662,3cyclictest526813-21kworker/u8:0+events_unbound19:19:560
529903996560,3cyclictest603581-21kworker/u8:0+flush-179:023:00:241
529901996561,3cyclictest584756-21kworker/u8:2+events_unbound22:07:270
529901996560,4cyclictest595038-21kworker/u8:1+events_unbound23:01:150
529901996560,4cyclictest540981-21kworker/u8:1+events_unbound20:05:520
529901996556,4cyclictest548943-21kworker/u8:0+events_unbound20:21:290
529901996555,8cyclictest510-21rs:main0
529903996459,4cyclictest563776-21kworker/u8:0+flush-179:020:55:221
529903996459,4cyclictest563776-21kworker/u8:0+flush-179:020:55:221
529901996460,3cyclictest603581-21kworker/u8:0+events_unbound23:29:280
529901996459,4cyclictest607349-21kworker/u8:2+events_unbound23:16:180
529901996459,4cyclictest539327-21kworker/u8:0+events_unbound19:54:280
529901996453,9cyclictest533248-21apt-get19:20:110
529903996357,5cyclictest628351-21kworker/u8:3+events_unbound00:20:001
529903996357,5cyclictest603581-21kworker/u8:0+flush-179:023:15:211
529903996356,5cyclictest554178-21kworker/u8:3+events_unbound20:30:001
529901996359,3cyclictest533026-21kworker/u8:2+events_unbound19:38:270
529903996257,3cyclictest595836-21kworker/u8:3+flush-179:022:45:221
529903996253,6cyclictest592676-21latency_hist22:25:021
529901996258,3cyclictest578589-21kworker/u8:1+events_unbound21:46:070
529901996258,3cyclictest563776-21kworker/u8:0+events_unbound21:11:020
529901996258,3cyclictest540981-21kworker/u8:1+events_unbound20:45:280
529901996257,4cyclictest626720-21kworker/u8:2+events_unbound00:14:380
529901996257,4cyclictest595836-21kworker/u8:3+events_unbound22:38:280
529901996250,8cyclictest536165-21latency_hist19:30:010
529903996157,3cyclictest623451-21kworker/u8:1+flush-179:000:05:211
529903996157,3cyclictest526813-21kworker/u8:0+flush-179:019:24:561
529903996148,10cyclictest623519-40mandb00:00:011
529901996157,3cyclictest617177-21kworker/u8:2+events_unbound23:54:560
529901996157,3cyclictest595836-21kworker/u8:3+events_unbound22:48:290
529903996051,6cyclictest603581-21kworker/u8:0+flush-179:023:50:161
529903996049,8cyclictest571959-21/usr/sbin/munin21:20:231
529903996049,8cyclictest571959-21/usr/sbin/munin21:20:221
529901996055,4cyclictest540981-21kworker/u8:1+events_unbound19:56:280
529903995950,6cyclictest591068-21latency_hist22:20:001
52990399590,57cyclictest1-21systemd19:50:011
529901995955,3cyclictest617177-21kworker/u8:2+events_unbound23:49:280
529901995955,3cyclictest584756-21kworker/u8:2+events_unbound22:21:090
529901995955,3cyclictest581517-21kworker/u8:3+events_unbound22:14:290
529901995955,3cyclictest563776-21kworker/u8:0+events_unbound21:33:290
529901995954,4cyclictest603581-21kworker/u8:0+events_unbound23:41:280
529901995954,4cyclictest578589-21kworker/u8:1+events_unbound21:40:280
529903995852,5cyclictest595038-21kworker/u8:1+flush-179:023:10:001
529903995850,7cyclictest545963-21/usr/sbin/munin20:00:241
529901995854,3cyclictest589716-21kworker/u8:0+events_unbound22:26:280
529901995854,3cyclictest548943-21kworker/u8:0+events_unbound20:11:270
529901995854,3cyclictest536201-21kworker/u8:3+events_unbound19:43:290
529901995853,4cyclictest563776-21kworker/u8:0+events_unbound21:38:300
529901995853,4cyclictest540981-21kworker/u8:1+events_unbound20:52:280
529901995848,8cyclictest555530-21latency_hist20:30:010
529903995747,8cyclictest607564-21apt-get23:10:111
529901995753,3cyclictest603581-21kworker/u8:0+events_unbound23:22:290
529901995753,3cyclictest554178-21kworker/u8:3+events_unbound21:07:290
529903995547,6cyclictest603581-21kworker/u8:0+flush-179:023:00:001
529903995546,8cyclictest595038-21kworker/u8:1+events_unbound22:40:261
529903995544,8cyclictest567195-21apt-get21:05:101
529901995551,3cyclictest540981-21kworker/u8:1+events_unbound20:37:290
529901995551,3cyclictest533026-21kworker/u8:2+events_unbound19:30:020
529901995550,4cyclictest589716-21kworker/u8:0+events_unbound22:19:290
529903995449,3cyclictest554178-21kworker/u8:3+flush-179:020:50:151
529903995445,7cyclictest563776-21kworker/u8:0+flush-179:021:05:011
529903995442,10cyclictest630188-21apt-get00:20:121
529901995450,3cyclictest548943-21kworker/u8:0+events_unbound20:17:010
529903995346,6cyclictest612068-21kworker/u8:1+flush-179:023:35:001
529903995341,11cyclictest628351-21kworker/u8:3+events_unbound00:25:241
529903995244,6cyclictest557920-21diskstats20:35:151
529903995145,4cyclictest578589-21kworker/u8:1+flush-179:021:55:141
529903995143,6cyclictest530052-21apt-get19:10:121
529901995147,3cyclictest540981-21kworker/u8:1+events_unbound20:30:550
529901995146,4cyclictest595038-21kworker/u8:1+events_unbound22:50:270
529901995146,4cyclictest578589-21kworker/u8:1+events_unbound21:56:080
529901995143,6cyclictest266-21systemd-journal00:23:390
529903995045,3cyclictest533026-21kworker/u8:2+flush-179:019:25:201
529901995046,3cyclictest536201-21kworker/u8:3+events_unbound19:46:280
529901995042,6cyclictest633082-21kworker/u8:1+flush-179:000:30:010
529903994943,4cyclictest597817-21kworker/u8:2+flush-179:022:55:001
529903994844,3cyclictest603581-21kworker/u8:0+flush-179:023:20:151
529903994843,4cyclictest536201-21kworker/u8:3+events_unbound20:05:221
529903994842,5cyclictest589716-21kworker/u8:0+flush-179:022:25:201
529903994841,5cyclictest603581-21kworker/u8:0+flush-179:023:45:211
529903994841,5cyclictest589479-21latency_hist22:15:011
529903994841,5cyclictest556044-21apt-get20:30:121
529901994844,3cyclictest484-21dbus-daemon00:04:000
529901994843,4cyclictest633082-21kworker/u8:1+events_unbound00:32:470
529903994742,4cyclictest536201-21kworker/u8:3+flush-179:020:00:011
529903994741,4cyclictest567980-21kworker/u8:1+flush-179:021:25:241
529903994741,4cyclictest563776-21kworker/u8:0+flush-179:021:20:011
52990899465,39cyclictest441ktimers/300:05:003
529903994642,3cyclictest540981-21kworker/u8:1+flush-179:019:50:151
529903994637,6cyclictest603581-21kworker/u8:0+flush-179:023:25:211
529901994642,3cyclictest597817-21kworker/u8:2+events_unbound22:59:280
529901994642,3cyclictest595836-21kworker/u8:3+events_unbound22:42:280
529901994642,3cyclictest563776-21kworker/u8:0+events_unbound21:50:030
4577091460,12ptp4l441ktimers/319:09:563
529903994541,3cyclictest625097-21kworker/u8:0+flush-179:000:10:211
529903994540,4cyclictest578589-21kworker/u8:1+flush-179:021:40:131
529901994533,9cyclictest266-21systemd-journal22:31:270
529903994439,4cyclictest540981-21kworker/u8:1+flush-179:020:45:201
529903994438,4cyclictest612068-21kworker/u8:1+flush-179:023:40:001
529901994339,3cyclictest567980-21kworker/u8:1+events_unbound21:29:290
529903994236,5cyclictest578589-21kworker/u8:1+events_unbound21:45:211
529905994137,2cyclictest0-21swapper/220:03:592
529908994034,5cyclictest578332-21turbostat21:40:013
52990899400,38cyclictest537769-21turbostat19:35:003
52990899400,1cyclictest0-21swapper/323:48:593
529905994036,2cyclictest0-21swapper/200:38:592
529903994035,4cyclictest612068-21kworker/u8:1+flush-179:023:40:211
529901994034,5cyclictest625398-21apt-get00:05:110
529901994034,4cyclictest714-21Xorg20:44:000
529903993935,2cyclictest714-21Xorg00:37:001
529903993932,5cyclictest540981-21kworker/u8:1+events_unbound20:20:001
529901993935,3cyclictest612068-21kworker/u8:1+events_unbound23:34:550
529901993935,3cyclictest567980-21kworker/u8:1+events_unbound21:20:030
529901993935,3cyclictest567980-21kworker/u8:1+events_unbound21:20:030
529908993835,2cyclictest0-21swapper/320:04:003
52990899380,4cyclictest532988-21turbostat19:20:003
529905993834,2cyclictest0-21swapper/220:47:592
529903993833,4cyclictest581517-21kworker/u8:3+events_unbound22:05:251
529908993734,2cyclictest0-21swapper/322:18:593
529908993733,3cyclictest613688-21turbostat23:30:003
52990899370,36cyclictest0-21swapper/322:09:553
52990899370,35cyclictest0-21swapper/320:44:003
529905993734,2cyclictest0-21swapper/223:49:592
52990599370,35cyclictest0-21swapper/223:30:592
52990599370,35cyclictest0-21swapper/223:12:002
52990599370,35cyclictest0-21swapper/219:20:002
52990599370,2cyclictest0-21swapper/220:15:592
529903993734,2cyclictest0-21swapper/122:31:591
529903993733,3cyclictest537788-21systemd-run19:34:591
529903993729,7cyclictest578589-21kworker/u8:1+flush-179:021:50:201
529908993634,1cyclictest0-21swapper/323:54:593
529908993634,1cyclictest0-21swapper/300:39:003
529908993634,1cyclictest0-21swapper/300:25:593
52990899360,35cyclictest0-21swapper/319:38:593
529905993633,2cyclictest0-21swapper/220:54:592
529905993633,2cyclictest0-21swapper/200:24:592
529905993632,2cyclictest0-21swapper/222:06:592
529905993632,2cyclictest0-21swapper/219:10:582
52990599360,1cyclictest0-21swapper/220:44:002
529908993532,2cyclictest0-21swapper/320:54:593
529905993532,2cyclictest0-21swapper/221:30:592
529905993531,2cyclictest0-21swapper/220:29:562
52990899340,1cyclictest0-21swapper/322:01:583
529905993430,2cyclictest0-21swapper/221:03:582
529908993331,1cyclictest0-21swapper/320:47:583
529908993331,1cyclictest0-21swapper/300:25:003
529908993330,2cyclictest0-21swapper/321:30:593
529908993330,2cyclictest0-21swapper/319:10:593
529903993328,4cyclictest908065-21snmpd19:40:161
529908993129,1cyclictest0-21swapper/321:53:593
529905993128,2cyclictest351rcuc/220:56:002
529905993128,2cyclictest351rcuc/220:55:592
529905993127,2cyclictest0-21swapper/223:51:592
529905993127,2cyclictest0-21swapper/223:41:592
529905993127,2cyclictest0-21swapper/223:17:592
529905993127,2cyclictest0-21swapper/222:43:582
52990599310,29cyclictest0-21swapper/222:35:042
529908993028,1cyclictest0-21swapper/320:29:553
52990899300,29cyclictest0-21swapper/322:35:033
529905993026,2cyclictest0-21swapper/223:25:132
529905993026,2cyclictest0-21swapper/219:48:592
529905993026,2cyclictest0-21swapper/219:41:592
529905993026,2cyclictest0-21swapper/219:21:592
52990599300,2cyclictest0-21swapper/219:55:142
52990599300,23cyclictest0-21swapper/222:50:132
529908992927,1cyclictest0-21swapper/321:03:583
529908992927,1cyclictest0-21swapper/320:19:593
529908992926,2cyclictest0-21swapper/322:45:143
529908992925,2cyclictest0-21swapper/322:43:583
52990899290,27cyclictest0-21swapper/320:20:023
52990899290,27cyclictest0-21swapper/300:10:133
52990899290,21cyclictest0-21swapper/319:55:133
529905992926,2cyclictest0-21swapper/220:20:012
529905992925,2cyclictest0-21swapper/222:56:592
529905992925,2cyclictest0-21swapper/221:21:582
529905992925,2cyclictest0-21swapper/221:21:582
529905992925,2cyclictest0-21swapper/221:14:002
52990599290,27cyclictest0-21swapper/222:45:132
52990599290,27cyclictest0-21swapper/222:00:142
52990599290,27cyclictest0-21swapper/221:05:122
52990599290,27cyclictest0-21swapper/220:30:032
52990599290,27cyclictest0-21swapper/200:10:132
52990599290,26cyclictest0-21swapper/222:15:142
529908992826,1cyclictest0-21swapper/320:30:023
529908992825,2cyclictest0-21swapper/323:41:593
529908992825,2cyclictest0-21swapper/323:17:593
529908992825,2cyclictest0-21swapper/320:55:143
529908992825,2cyclictest0-21swapper/320:55:143
52990899280,27cyclictest0-21swapper/322:50:123
52990899280,27cyclictest0-21swapper/322:14:003
52990899280,26cyclictest0-21swapper/321:05:133
529905992825,2cyclictest0-21swapper/221:35:122
529905992824,2cyclictest0-21swapper/219:28:582
529905992824,2cyclictest0-21swapper/200:15:592
52990599280,2cyclictest0-21swapper/222:13:592
52990599280,2cyclictest0-21swapper/219:52:592
529908992725,1cyclictest0-21swapper/323:36:593
52990899270,26cyclictest0-21swapper/323:33:593
52990899270,26cyclictest0-21swapper/319:52:593
52990899270,1cyclictest0-21swapper/321:14:003
529905992724,2cyclictest0-21swapper/223:57:592
529905992724,2cyclictest0-21swapper/219:38:592
52990599270,25cyclictest0-21swapper/221:59:592
52990599270,25cyclictest0-21swapper/221:43:002
529908992623,2cyclictest0-21swapper/322:56:593
529908992623,2cyclictest0-21swapper/319:28:593
52990899260,24cyclictest0-21swapper/321:29:593
52990899260,1cyclictest0-21swapper/321:40:123
529905992623,2cyclictest0-21swapper/221:45:582
529905992622,2cyclictest0-21swapper/222:33:592
529905992622,2cyclictest0-21swapper/220:36:582
529908992523,1cyclictest0-21swapper/321:21:593
529908992523,1cyclictest0-21swapper/321:21:583
529908992523,1cyclictest0-21swapper/319:49:003
529908992523,1cyclictest0-21swapper/319:41:593
529908992523,1cyclictest0-21swapper/319:21:583
529908992520,3cyclictest0-21swapper/321:45:583
52990899250,23cyclictest0-21swapper/323:24:593
529905992522,2cyclictest0-21swapper/222:22:592
529905992521,2cyclictest0-21swapper/223:36:592
529905992521,2cyclictest0-21swapper/222:29:582
529905992520,3cyclictest0-21swapper/219:30:592
529903992518,5cyclictest529767-21latency_hist19:10:011
529908992422,1cyclictest0-21swapper/323:56:003
529908992422,1cyclictest0-21swapper/320:37:593
529908992422,1cyclictest0-21swapper/320:09:593
529908992421,2cyclictest0-21swapper/300:15:593
529905992420,2cyclictest0-21swapper/223:20:592
52990599240,1cyclictest0-21swapper/221:30:002
4577091240,5ptp4l441ktimers/320:14:593
4577091240,3ptp4l441ktimers/323:05:003
529908992320,2cyclictest0-21swapper/322:33:593
529905992319,2cyclictest0-21swapper/220:08:592
529905992319,2cyclictest0-21swapper/200:06:592
4577091230,3ptp4l61490irq/134-eno1-TxRx-023:10:003
4577091230,3ptp4l61490irq/134-eno1-TxRx-000:35:003
4577091230,3ptp4l61490irq/134-eno1-TxRx-000:09:563
4577091230,3ptp4l441ktimers/322:00:013
4577091230,3ptp4l441ktimers/321:19:553
529908992221,1cyclictest0-21swapper/322:21:593
529905992219,2cyclictest0-21swapper/200:28:592
4577091220,3ptp4l61490irq/134-eno1-TxRx-022:29:553
4577091220,3ptp4l441ktimers/323:14:553
529905992117,2cyclictest0-21swapper/221:16:002
529905991816,1cyclictest0-21swapper/221:54:592
529905991714,2cyclictest351rcuc/223:04:552
529905991714,2cyclictest0-21swapper/200:01:592
529905991613,2cyclictest0-21swapper/223:07:592
52990199163,10cyclictest529904-21cut19:10:010
52990599119,1cyclictest0-21swapper/220:15:002
52990599118,2cyclictest351rcuc/200:30:172
5299059941,2cyclictest351rcuc/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional