You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 14:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sat Nov 08, 2025 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27633219910058,29cyclictest2776200-21/usr/sbin/munin19:50:230
2763321999979,18cyclictest2806464-21apt-get21:25:030
276332199831,77cyclictest2796927-21true20:55:000
2763321998271,8cyclictest2768012-21latency_hist19:25:010
2763323998168,10cyclictest2824471-21latency_hist22:20:011
2763323998070,8cyclictest2866920-21/usr/sbin/munin00:30:251
2763321997948,30cyclictest0-21swapper/023:04:590
2763321997948,30cyclictest0-21swapper/023:04:580
2763321997267,4cyclictest2827955-21kworker/u8:3+events_unbound00:00:000
2763321997264,6cyclictest2785527-21apt-get20:20:010
2763323997161,8cyclictest2827955-21kworker/u8:3+flush-179:023:00:011
2763323996864,3cyclictest2839030-21kworker/u8:0+events_unbound23:39:551
2763323996863,4cyclictest2782306-21kworker/u8:1+events_unbound20:20:261
2763323996863,4cyclictest2782306-21kworker/u8:1+events_unbound20:20:261
2763323996455,7cyclictest2777782-21/usr/sbin/munin19:55:241
2763323996352,9cyclictest2840650-21apt-get23:10:001
2763323996258,3cyclictest2779390-21kworker/u8:1+events_unbound20:00:261
2763326996131,2cyclictest0-21swapper/223:04:582
2763326996131,2cyclictest0-21swapper/223:04:582
2763323996153,6cyclictest271-21systemd-journal22:10:251
2763323996153,6cyclictest271-21systemd-journal22:10:251
2763321996152,6cyclictest1-21systemd22:30:010
2763321996151,8cyclictest2863749-21/usr/sbin/munin00:20:240
2763323996055,4cyclictest2839030-21kworker/u8:0+events_unbound00:10:251
2763321995953,3cyclictest2774363-21kworker/u8:2+events_unbound20:10:230
2763321995950,6cyclictest2771184-21latency_hist19:35:010
2763321995935,22cyclictest2763203-21latency_hist19:10:010
2763323995854,3cyclictest2787411-21kworker/u8:3+events_unbound20:55:241
2763321995847,9cyclictest2808555-21apt-get21:30:120
2763323995753,3cyclictest2862135-21kworker/u8:2+events_unbound00:29:001
2763323995753,3cyclictest2772753-21kworker/u8:1+events_unbound19:52:001
2763323995753,2cyclictest723-21Xorg20:35:581
2763323995746,8cyclictest271-21systemd-journal19:35:001
2763321995746,9cyclictest2827955-21kworker/u8:3+flush-179:023:35:000
2763323995651,4cyclictest2787411-21kworker/u8:3+events_unbound20:43:271
2763323995649,3cyclictest2827955-21kworker/u8:3+events_unbound22:40:231
2763323995648,3cyclictest2839030-21kworker/u8:0+events_unbound23:10:171
2763323995642,12cyclictest271-21systemd-journal00:04:001
2763321995646,7cyclictest2803801-21diskstats21:15:160
276332199560,38cyclictest2782306-21kworker/u8:1+i91520:09:580
2763323995551,3cyclictest2827955-21kworker/u8:3+events_unbound23:42:481
2763323995551,3cyclictest2825789-21kworker/u8:1+events_unbound22:21:281
2763323995551,3cyclictest2793963-21kworker/u8:0+events_unbound21:07:291
2763323995450,3cyclictest2793963-21kworker/u8:0+events_unbound21:29:271
2763323995450,3cyclictest2793963-21kworker/u8:0+events_unbound20:52:131
2763323995442,9cyclictest2806454-21latency_hist21:25:011
2763321995444,7cyclictest2767498-21kworker/u8:0+events_unbound19:40:010
2763321995430,4cyclictest2832719-21systemd-run22:45:010
2763323995349,3cyclictest2839030-21kworker/u8:0+events_unbound23:51:261
2763323995349,3cyclictest2827955-21kworker/u8:3+events_unbound23:59:261
2763323995349,3cyclictest2827955-21kworker/u8:3+events_unbound23:45:271
2763323995349,3cyclictest2808346-21kworker/u8:3+events_unbound21:39:271
2763323995349,3cyclictest2800126-21kworker/u8:2+events_unbound21:11:291
2763323995349,3cyclictest2763183-21kworker/u8:1+events_unbound19:19:271
2763321995319,32cyclictest723-21Xorg23:29:590
2763323995248,3cyclictest2839030-21kworker/u8:0+events_unbound00:05:271
2763323995248,3cyclictest2793963-21kworker/u8:0+events_unbound20:48:281
2763323995248,3cyclictest2787411-21kworker/u8:3+events_unbound21:01:281
2763323995247,4cyclictest2868521-21kworker/u8:3+events_unbound00:39:271
2763323995247,4cyclictest2862135-21kworker/u8:2+events_unbound00:18:271
2763321995243,6cyclictest2790518-21kworker/u8:1+events_unbound20:35:270
2763323995147,3cyclictest2842502-21kworker/u8:2+events_unbound23:16:271
2763323995146,4cyclictest2793963-21kworker/u8:0+events_unbound21:56:271
2763321995142,6cyclictest2793963-21kworker/u8:0+flush-179:020:45:170
2763321995137,10cyclictest2775952-21kworker/u8:0+events_unbound20:25:000
2763321995137,10cyclictest2775952-21kworker/u8:0+events_unbound20:24:590
276332199512,32cyclictest2797151-21apt-get20:55:120
2763323995046,3cyclictest2839030-21kworker/u8:0+events_unbound23:23:271
2763323995046,3cyclictest2839030-21kworker/u8:0+events_unbound00:22:271
2763323995046,3cyclictest2821264-21kworker/u8:2+events_unbound22:27:361
2763323995046,3cyclictest2808284-21kworker/u8:1+events_unbound21:32:291
2763323995046,3cyclictest2763183-21kworker/u8:1+events_unbound19:12:271
2763321995040,7cyclictest2774363-21kworker/u8:2+events_unbound20:00:000
2763321995040,6cyclictest2839030-21kworker/u8:0+flush-179:023:10:020
2763323994945,3cyclictest2775952-21kworker/u8:0+events_unbound20:12:041
2763323994945,3cyclictest2768046-21kworker/u8:2+events_unbound19:26:561
2763323994944,4cyclictest2827955-21kworker/u8:3+events_unbound23:04:271
2763323994944,4cyclictest2827955-21kworker/u8:3+events_unbound23:04:261
2763323994944,4cyclictest2787343-21kworker/u8:2+events_unbound20:31:271
2763321994939,7cyclictest2863501-21latency_hist00:20:010
276332199490,5cyclictest2834285-21dump-pmu-power22:50:010
2763323994844,3cyclictest2827955-21kworker/u8:3+events_unbound22:50:091
2763323994844,3cyclictest2793963-21kworker/u8:0+events_unbound21:53:261
2763323994844,3cyclictest2760250-21kworker/u8:3+events_unbound19:22:281
4670791470,9ptp4l441ktimers/319:05:213
2763323994743,3cyclictest2821264-21kworker/u8:2+events_unbound22:30:291
2763323994743,3cyclictest2819677-21kworker/u8:1+events_unbound22:01:291
2763323994743,3cyclictest2793963-21kworker/u8:0+events_unbound21:47:281
2763323994743,3cyclictest2787343-21kworker/u8:2+events_unbound20:27:071
2763323994743,3cyclictest2775952-21kworker/u8:0+events_unbound20:16:281
276332899460,44cyclictest2839069-21turbostat23:05:003
276332899460,44cyclictest2839069-21turbostat23:04:593
2763323994642,3cyclictest2839030-21kworker/u8:0+events_unbound23:32:271
2763323994642,3cyclictest2827955-21kworker/u8:3+events_unbound22:48:261
2763323994642,3cyclictest2827955-21kworker/u8:3+events_unbound22:39:271
2763328994539,4cyclictest2803275-21turbostat21:15:013
2763321994537,6cyclictest2827955-21kworker/u8:3+events_unbound22:40:000
2763321994537,6cyclictest2767498-21kworker/u8:0+flush-179:019:25:160
2763321994537,5cyclictest2800091-21latency_hist21:05:000
2763323994440,3cyclictest2793963-21kworker/u8:0+events_unbound21:15:071
2763323994440,3cyclictest2774363-21kworker/u8:2+events_unbound19:44:271
2763323994436,3cyclictest2774363-21kworker/u8:2+events_unbound20:05:211
276332199442,37cyclictest2816761-21apt-get21:55:150
2763323994339,3cyclictest2808346-21kworker/u8:3+events_unbound21:42:231
2763321994340,2cyclictest0-21swapper/022:55:590
2763321994339,3cyclictest2803297-21systemd-run21:15:010
2763321994338,3cyclictest2816538-21systemd-run21:55:010
2763321994336,5cyclictest2839030-21kworker/u8:0+flush-179:023:45:000
276332199430,1cyclictest0-21swapper/000:35:590
2763328994240,1cyclictest0-21swapper/320:34:553
2763321994239,2cyclictest0-21swapper/021:22:580
2763321994236,4cyclictest2819677-21kworker/u8:1+flush-179:022:05:240
276332199420,40cyclictest723-21Xorg22:15:590
276332199420,2cyclictest723-21Xorg20:29:590
2763323994137,3cyclictest2772753-21kworker/u8:1+events_unbound19:39:261
276332399410,39cyclictest723-21Xorg19:50:001
2763321994136,3cyclictest2779390-21kworker/u8:1+events_unbound20:00:240
276332199410,39cyclictest723-21Xorg22:14:000
276332199410,39cyclictest723-21Xorg22:13:590
276332199410,39cyclictest2818097-21turbostat22:04:550
2763328994034,4cyclictest2785532-21turbostat20:20:013
276332899400,38cyclictest2788962-21turbostat20:30:003
2763321994022,2cyclictest2765072-21apt-get19:15:150
2763328993934,4cyclictest2796901-21turbostat20:55:003
276332899390,38cyclictest2795324-21turbostat20:50:013
276332899390,37cyclictest2808310-21turbostat21:30:003
2763323993935,3cyclictest2819677-21kworker/u8:1+events_unbound22:07:291
2763321993935,3cyclictest2853662-21systemd-run23:50:000
2763321993933,4cyclictest2793963-21kworker/u8:0+flush-179:021:05:220
2763328993833,4cyclictest2842491-21turbostat23:15:003
2763328993833,4cyclictest2809894-21turbostat21:35:013
2763328993831,6cyclictest2816514-21turbostat21:55:013
2763326993835,2cyclictest0-21swapper/200:17:592
2763326993834,2cyclictest0-21swapper/223:27:002
276332699380,35cyclictest0-21swapper/220:01:592
2763321993834,3cyclictest2855248-21systemd-run23:55:010
2763321993830,6cyclictest2767498-21kworker/u8:0+events_unbound19:40:270
276332199380,37cyclictest0-21swapper/021:46:000
276332199380,37cyclictest0-21swapper/021:36:590
2763328993735,1cyclictest0-21swapper/300:01:003
2763328993734,2cyclictest0-21swapper/320:05:003
276332899370,36cyclictest0-21swapper/321:19:563
2763326993734,2cyclictest0-21swapper/223:11:592
2763326993734,2cyclictest0-21swapper/222:03:002
2763326993733,2cyclictest0-21swapper/223:59:592
276332699370,35cyclictest0-21swapper/223:35:592
276332699370,35cyclictest0-21swapper/200:34:552
276332699370,2cyclictest0-21swapper/223:40:592
2763323993733,3cyclictest2839030-21kworker/u8:0+events_unbound23:25:281
2763321993734,2cyclictest723-21Xorg00:04:590
2763321993733,3cyclictest2764192-21needreboot19:10:210
2763321993729,6cyclictest2868267-21latency_hist00:35:010
276332899360,35cyclictest0-21swapper/320:23:593
276332899360,35cyclictest0-21swapper/320:23:583
276332899360,35cyclictest0-21swapper/319:33:593
276332899360,35cyclictest0-21swapper/300:27:593
276332899360,1cyclictest0-21swapper/319:57:593
276332899360,1cyclictest0-21swapper/319:29:593
2763326993633,2cyclictest0-21swapper/222:51:592
2763326993633,2cyclictest0-21swapper/221:52:582
276332699360,1cyclictest0-21swapper/220:34:002
2763328993533,1cyclictest0-21swapper/323:32:583
2763328993533,1cyclictest0-21swapper/322:15:003
2763328993533,1cyclictest0-21swapper/322:15:003
2763326993533,1cyclictest0-21swapper/200:39:592
2763326993532,2cyclictest0-21swapper/220:50:582
2763326993531,2cyclictest0-21swapper/220:19:002
276332699350,25cyclictest0-21swapper/220:27:592
2763321993532,2cyclictest723-21Xorg23:21:000
2763321993527,4cyclictest0-21swapper/000:05:130
2763328993432,1cyclictest0-21swapper/322:28:003
2763328993432,1cyclictest0-21swapper/322:22:583
2763328993432,1cyclictest0-21swapper/322:17:593
2763328993431,2cyclictest0-21swapper/300:31:593
2763328993429,4cyclictest2837458-21turbostat23:00:003
2763328993429,4cyclictest2809894-21turbostat21:39:563
2763326993432,1cyclictest0-21swapper/220:14:592
2763326993431,2cyclictest0-21swapper/221:32:592
2763326993430,2cyclictest0-21swapper/221:08:582
276332199341,31cyclictest2824993-21apt-get22:20:150
276332199340,30cyclictest2811943-21apt-key21:40:010
276332199340,30cyclictest0-21swapper/023:15:140
276332699330,27cyclictest0-21swapper/221:40:022
2763321993328,4cyclictest2839030-21kworker/u8:0+flush-179:000:10:170
2763328993230,1cyclictest0-21swapper/319:17:593
2763326993230,1cyclictest0-21swapper/200:29:592
2763326993229,2cyclictest0-21swapper/221:27:592
2763326993229,2cyclictest0-21swapper/221:16:002
2763326993225,6cyclictest0-21swapper/223:50:142
2763326993225,5cyclictest0-21swapper/220:55:122
2763326993225,5cyclictest0-21swapper/220:09:592
2763326993220,9cyclictest0-21swapper/222:50:002
276332699320,26cyclictest0-21swapper/200:05:132
2763321993226,4cyclictest690-21snmpd20:30:150
276332199322,3cyclictest2828166-21apt-get22:30:150
2763328993129,1cyclictest0-21swapper/323:25:003
2763328993129,1cyclictest0-21swapper/300:18:593
2763328993128,2cyclictest0-21swapper/320:57:593
2763328993126,4cyclictest2813341-21turbostat21:44:593
2763326993129,1cyclictest0-21swapper/222:59:592
2763326993128,2cyclictest0-21swapper/219:58:592
2763326993127,3cyclictest0-21swapper/222:30:162
2763321993127,2cyclictest0-21swapper/019:45:130
2763328993028,1cyclictest0-21swapper/323:53:593
2763328993028,1cyclictest0-21swapper/322:35:583
2763328993027,2cyclictest0-21swapper/321:09:583
2763328993025,4cyclictest2844073-21turbostat23:20:003
276332899301,27cyclictest2860314-21turbostat00:10:003
276332899300,29cyclictest0-21swapper/323:35:593
276332899300,29cyclictest0-21swapper/300:00:013
276332899300,28cyclictest2818097-21turbostat22:00:013
2763326993027,2cyclictest0-21swapper/221:55:162
276332699300,28cyclictest0-21swapper/219:15:142
276332199300,2cyclictest723-21Xorg20:44:580
4672291290,27phc2sys0-21swapper/322:00:133
2763328992927,1cyclictest0-21swapper/323:44:003
276332899290,27cyclictest0-21swapper/323:45:113
276332899290,27cyclictest0-21swapper/320:40:123
276332899290,27cyclictest0-21swapper/320:10:133
2763326992926,2cyclictest0-21swapper/223:30:592
2763326992926,2cyclictest0-21swapper/222:20:022
2763326992926,2cyclictest0-21swapper/220:20:132
2763326992926,2cyclictest0-21swapper/220:20:132
276332699290,28cyclictest0-21swapper/200:00:032
276332699290,27cyclictest0-21swapper/221:01:002
276332699290,1cyclictest0-21swapper/220:40:002
2763321992926,2cyclictest0-21swapper/023:09:590
2763321992923,5cyclictest2865102-21turbostat00:29:560
2763328992826,1cyclictest0-21swapper/322:45:183
2763328992826,1cyclictest0-21swapper/319:40:163
2763328992823,4cyclictest2792143-21turbostat20:40:003
276332899280,2cyclictest0-21swapper/323:25:143
276332899280,27cyclictest0-21swapper/321:00:123
276332899280,26cyclictest0-21swapper/322:40:143
276332899280,26cyclictest0-21swapper/319:35:123
2763326992825,2cyclictest0-21swapper/222:14:002
2763326992825,2cyclictest0-21swapper/222:13:592
2763326992824,2cyclictest0-21swapper/222:16:002
276332699280,26cyclictest0-21swapper/223:15:142
276332699280,26cyclictest0-21swapper/221:22:592
276332699280,26cyclictest0-21swapper/219:23:592
276332199280,27cyclictest0-21swapper/022:50:580
2763328992725,1cyclictest0-21swapper/321:45:113
2763328992725,1cyclictest0-21swapper/319:12:593
2763326992724,2cyclictest0-21swapper/221:40:022
2763326992724,2cyclictest0-21swapper/219:45:592
2763326992724,2cyclictest0-21swapper/200:21:592
276332699270,24cyclictest0-21swapper/223:25:002
276332699270,24cyclictest0-21swapper/221:49:542
276332699270,1cyclictest0-21swapper/220:40:592
2763321992718,7cyclictest2839030-21kworker/u8:0+flush-179:023:40:000
4670791260,6ptp4l441ktimers/320:09:543
4670791260,6ptp4l441ktimers/300:15:003
4670791260,5ptp4l441ktimers/319:55:013
2763328992623,2cyclictest0-21swapper/322:54:593
2763326992623,2cyclictest0-21swapper/219:10:262
276332699260,24cyclictest0-21swapper/219:44:592
276332699260,24cyclictest0-21swapper/219:30:592
4670791250,6ptp4l441ktimers/300:40:003
4670791250,5ptp4l441ktimers/322:10:003
2763328992523,1cyclictest0-21swapper/323:09:593
2763328992523,1cyclictest0-21swapper/319:45:583
2763328992523,1cyclictest0-21swapper/319:20:073
2763328992521,3cyclictest2829505-21turbostat22:35:003
276332899250,1cyclictest0-21swapper/300:22:003
2763326992523,1cyclictest0-21swapper/222:35:592
2763326992522,2cyclictest0-21swapper/222:26:592
2763326992522,2cyclictest0-21swapper/219:27:212
276332699250,2cyclictest0-21swapper/219:36:592
202199250,16rtkit-daemon685-21ntpd19:05:161
2763326992422,1cyclictest0-21swapper/221:11:582
2763326992321,1cyclictest0-21swapper/200:15:002
4670791220,4ptp4l441ktimers/321:25:003
2763326992220,1cyclictest0-21swapper/222:10:002
2763326992119,1cyclictest0-21swapper/222:40:592
2763326991916,2cyclictest0-21swapper/220:45:142
2763326991614,1cyclictest0-21swapper/223:10:002
27633269985,2cyclictest0-21swapper/223:49:562
27633269985,2cyclictest0-21swapper/219:52:592
27633269920,1cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional