You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-19 - 01:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Sat Oct 18, 2025 12:44:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12178549912286,24cyclictest1222531-21latency_hist07:25:000
121785499120109,8cyclictest1315262-21latency_hist12:10:010
121785799114103,8cyclictest1240681-21latency_hist08:20:021
121785799114103,8cyclictest1240681-21latency_hist08:20:011
12178579910795,9cyclictest1290579-21latency_hist10:55:011
12178549910697,6cyclictest1300410-21latency_hist11:25:010
12178549910277,22cyclictest1242274-21latency_hist08:25:000
1217854999987,9cyclictest1230599-21latency_hist07:50:010
1217857999787,7cyclictest1263500-21latency_hist09:30:021
1217854999482,9cyclictest1305437-21latency_hist11:40:010
1217854999383,7cyclictest1288993-21latency_hist10:50:000
1217854999383,7cyclictest1249543-21journal-offline08:45:180
1217857999084,4cyclictest1217130-21kworker/u8:2+flush-179:007:55:261
1217854998779,6cyclictest1238866-21kworker/u8:2+flush-179:008:40:160
1217854998776,8cyclictest1261910-21latency_hist09:25:020
1217854998680,4cyclictest1292705-21apt-get11:00:150
1217854998675,8cyclictest1235643-21latency_hist08:05:010
1217854998576,6cyclictest1274694-21kworker/u8:1+flush-179:010:15:240
1217854998576,6cyclictest1274694-21kworker/u8:1+flush-179:010:15:240
1217854998572,10cyclictest1229011-21latency_hist07:45:010
1217857998374,6cyclictest1255540-21latency_hist09:05:011
1217857998371,10cyclictest1270130-21apt-get09:50:121
1217857998361,12cyclictest1286073-21apt-get10:40:151
1217857998361,12cyclictest1286073-21apt-get10:40:141
1217857998270,9cyclictest601706-21systemd-journal12:05:011
1217857998063,14cyclictest1305420-21kworker/u8:2+flush-179:012:10:251
1217854998074,4cyclictest1252325-21kworker/u8:0+flush-179:009:05:540
1217854997967,8cyclictest1225709-21latency_hist07:35:010
1217857997853,14cyclictest601706-21systemd-journal11:24:421
1217854997870,5cyclictest1253960-21latency_hist09:00:010
1217854997868,8cyclictest1310455-21/usr/sbin/munin11:55:230
1217857997665,9cyclictest1321892-21apt-get12:30:121
1217857997656,17cyclictest601706-21systemd-journal09:15:431
1217854997665,8cyclictest1292437-21latency_hist11:00:010
1217857997461,11cyclictest1265630-21diskstats09:35:161
1217854997469,3cyclictest1238866-21kworker/u8:2+flush-179:008:30:310
1217857997263,6cyclictest1233782-21apt-get08:00:041
1217854997060,8cyclictest1321622-21kworker/u8:2+flush-179:012:30:150
1217857996961,5cyclictest1303827-21kworker/u8:0+flush-179:011:35:311
1217857996958,9cyclictest1307298-21apt-get11:45:111
1217857996958,8cyclictest1321640-21latency_hist12:30:011
1217854996961,7cyclictest1277817-21kworker/u8:0+events_unbound10:50:260
1217854996959,8cyclictest1237775-21diskstats08:10:160
1217857996862,4cyclictest1263536-21kworker/u8:4+flush-179:009:45:261
1217857996854,11cyclictest601706-21systemd-journal08:55:001
1217857996852,5cyclictest1317086-21/usr/sbin/munin12:15:181
121785799680,66cyclictest722-21Xorg10:24:581
1217854996857,9cyclictest601706-21systemd-journal09:45:000
121785499680,65cyclictest1-21systemd10:38:430
1217857996759,5cyclictest1305420-21kworker/u8:2+flush-179:011:50:261
1217857996759,5cyclictest1305420-21kworker/u8:2+flush-179:011:50:261
1217857996756,9cyclictest601706-21systemd-journal10:47:421
1217854996756,8cyclictest1318457-21latency_hist12:20:020
1217854996753,10cyclictest1248674-21kworker/u8:1+flush-179:008:50:440
1217857996658,6cyclictest1238866-21kworker/u8:2+events_unbound08:50:001
1217857996655,8cyclictest601706-21systemd-journal07:19:591
1217857996653,10cyclictest1224377-21apt-get07:30:121
1217857996649,14cyclictest1252325-21kworker/u8:0+flush-179:009:10:001
1217854996660,4cyclictest1262147-21kworker/u8:2+flush-179:011:05:260
1217854996660,4cyclictest1262147-21kworker/u8:2+flush-179:011:05:260
1217854996659,5cyclictest1262147-21kworker/u8:2+flush-179:011:10:180
1217854996653,9cyclictest1303827-21kworker/u8:0+flush-179:011:35:010
1217857996557,6cyclictest1277817-21kworker/u8:0+events_unbound10:15:011
1217857996553,9cyclictest1312082-21apt-get12:00:131
1217854996558,4cyclictest1295657-21kworker/u8:3+flush-179:011:25:200
1217854996555,7cyclictest601706-21systemd-journal11:48:420
1217857996451,10cyclictest1282876-21apt-get10:30:151
1217854996457,5cyclictest1271716-21apt-get09:55:130
1217854996453,7cyclictest1-21systemd07:37:410
1217863996360,2cyclictest0-21swapper/310:24:583
1217859996360,2cyclictest0-21swapper/209:45:002
121785999630,60cyclictest0-21swapper/208:35:002
1217854996353,8cyclictest1220928-21latency_hist07:20:010
1217857996256,5cyclictest1238866-21kworker/u8:2+flush-179:008:45:011
1217857996254,5cyclictest1237231-21latency_hist08:10:001
1217857996234,18cyclictest1225698-21kworker/u8:1+flush-179:008:15:011
1217854996252,8cyclictest1234113-21cat08:00:010
1217857996154,5cyclictest1277817-21kworker/u8:0+flush-179:010:25:141
4505291590,9ptp4l441ktimers/307:05:453
1217854995954,4cyclictest1305420-21kworker/u8:2+flush-179:011:55:010
1217854995954,4cyclictest1305420-21kworker/u8:2+flush-179:011:55:010
1217854995852,4cyclictest1279682-21/usr/sbin/munin10:20:260
1217854995749,6cyclictest1313416-21latency_hist12:05:000
1217854995548,5cyclictest1257364-21/usr/sbin/munin09:10:240
1217854995548,4cyclictest1262147-21kworker/u8:2+flush-179:010:05:260
1217854995547,6cyclictest1233754-21kworker/u8:0+events_unbound08:15:080
1217854995547,6cyclictest1233754-21kworker/u8:0+events_unbound08:15:080
1217863995452,1cyclictest0-21swapper/309:03:003
1217857995445,7cyclictest1277817-21kworker/u8:0+events_unbound11:25:021
1217854995447,5cyclictest1262147-21kworker/u8:2+flush-179:009:30:210
1217854995447,5cyclictest1262147-21kworker/u8:2+flush-179:009:30:000
1217854995446,6cyclictest1238866-21kworker/u8:2+flush-179:008:40:000
1217854995349,3cyclictest1262147-21kworker/u8:2+flush-179:010:40:330
1217854995349,3cyclictest1262147-21kworker/u8:2+flush-179:010:40:330
1217857995244,7cyclictest1263536-21kworker/u8:4+events_unbound10:00:001
1217857995243,7cyclictest601706-21systemd-journal10:38:421
1217857995243,7cyclictest1305712-21apt-get11:40:141
1217859995148,2cyclictest0-21swapper/210:04:592
1217857995147,3cyclictest1238866-21kworker/u8:2+flush-179:008:30:281
1217857995140,9cyclictest1273327-21apt-get10:00:141
1217854995145,5cyclictest1305420-21kworker/u8:2+flush-179:012:20:220
1217854995141,8cyclictest601706-21systemd-journal08:29:550
1217854995018,11cyclictest1307042-21latency_hist11:45:010
1217857994943,5cyclictest1217130-21kworker/u8:2+flush-179:007:40:001
1217854994942,4cyclictest1233754-21kworker/u8:0+flush-179:008:05:160
1217854994927,20cyclictest1323479-21apt-get12:35:130
1217857994841,5cyclictest1242310-21kworker/u8:3+flush-179:008:30:001
1217857994825,20cyclictest0-21swapper/110:05:151
1217854994836,9cyclictest601706-21systemd-journal07:50:080
1217857994740,5cyclictest0-21swapper/108:35:151
1217857994738,7cyclictest1240944-21apt-get08:20:141
1217857994737,7cyclictest1263536-21kworker/u8:4+flush-179:009:40:221
121785799470,44cyclictest1229269-21apt-get07:45:151
121785799460,32cyclictest1222570-21unin-run07:25:011
1217854994633,10cyclictest601706-21systemd-journal09:05:010
1217863994539,4cyclictest1302269-21turbostat11:30:013
1217857994536,7cyclictest1227683-21apt-get07:40:151
121785799450,2cyclictest722-21Xorg07:29:591
1217854994441,2cyclictest722-21Xorg09:51:000
1217854994437,5cyclictest1219347-21perf07:15:000
1217857994339,3cyclictest1265120-21systemd-run09:35:011
1217857994335,6cyclictest502-21rs:main1
1217857994333,7cyclictest1253921-21kworker/u8:3+flush-179:009:20:171
1217854994339,3cyclictest1320033-21kworker/u8:1+flush-179:012:25:270
1217857994237,4cyclictest476-21dbus-daemon11:10:011
1217857994237,4cyclictest476-21dbus-daemon11:10:011
1217857994224,17cyclictest1320051-21munin-run12:25:001
121785799420,40cyclictest1324832-21idleruntime-cro12:40:011
1217854994135,5cyclictest1274694-21kworker/u8:1+events_unbound10:10:230
1217854994133,6cyclictest601706-21systemd-journal10:28:420
1217854994121,2cyclictest0-21swapper/009:20:010
1217854994119,4cyclictest0-21swapper/009:35:260
1217863994038,1cyclictest0-21swapper/307:29:593
121786399400,38cyclictest1222538-21turbostat07:25:003
1217854994034,5cyclictest1277817-21kworker/u8:0+flush-179:010:30:240
121785499400,3cyclictest1224136-21dump-pmu-power07:30:010
1217863993936,2cyclictest0-21swapper/308:57:583
121785999390,36cyclictest0-21swapper/209:52:592
121785799390,37cyclictest722-21Xorg09:10:591
121785499390,37cyclictest722-21Xorg11:19:590
121785999380,35cyclictest0-21swapper/212:07:582
121785999380,30cyclictest0-21swapper/212:14:592
1217857993831,5cyclictest601706-21systemd-journal11:15:421
1217857993828,7cyclictest1311808-21latency_hist12:00:001
1217863993735,1cyclictest0-21swapper/309:38:003
1217863993734,2cyclictest0-21swapper/309:59:593
1217863993730,5cyclictest1324823-21turbostat12:40:003
121786399370,36cyclictest0-21swapper/312:19:593
1217859993734,2cyclictest0-21swapper/209:03:592
1217859993734,2cyclictest0-21swapper/207:16:592
121785999370,35cyclictest0-21swapper/208:13:592
1217854993732,4cyclictest1263536-21kworker/u8:4+flush-179:010:00:540
201699360,4rtkit-daemon816013-21ntpd07:11:171
1217863993633,2cyclictest0-21swapper/311:53:593
1217863993633,2cyclictest0-21swapper/311:53:593
1217863993631,4cyclictest1237245-21turbostat08:10:003
1217859993633,2cyclictest0-21swapper/211:08:592
1217859993633,2cyclictest0-21swapper/211:08:592
1217859993633,2cyclictest0-21swapper/210:44:002
1217859993633,2cyclictest0-21swapper/210:43:592
1217859993633,2cyclictest0-21swapper/208:05:592
1217859993633,2cyclictest0-21swapper/207:32:592
121785999360,1cyclictest0-21swapper/211:55:592
121785999360,1cyclictest0-21swapper/211:29:592
121785999360,1cyclictest0-21swapper/211:19:592
121785999360,1cyclictest0-21swapper/209:28:592
121785799360,21cyclictest1278828-21/usr/sbin/munin10:15:201
121785799360,21cyclictest1278828-21/usr/sbin/munin10:15:201
1217854993628,6cyclictest1313391-21kworker/u8:0+events_unbound12:10:260
1217863993533,1cyclictest0-21swapper/311:46:593
1217863993533,1cyclictest0-21swapper/308:51:593
1217863993532,2cyclictest0-21swapper/311:22:593
121785999350,1cyclictest0-21swapper/212:38:592
1217857993529,5cyclictest1295657-21kworker/u8:3+flush-179:011:30:281
1217863993432,1cyclictest0-21swapper/312:32:003
1217863993431,2cyclictest0-21swapper/310:54:553
121785799340,30cyclictest1231585-21netstat07:50:211
1217863993331,1cyclictest0-21swapper/309:10:583
1217863993331,1cyclictest0-21swapper/309:06:593
1217863993331,1cyclictest0-21swapper/308:17:593
1217863993331,1cyclictest0-21swapper/308:17:593
1217863993330,2cyclictest0-21swapper/308:50:003
1217859993330,2cyclictest0-21swapper/211:40:592
1217859993330,2cyclictest0-21swapper/209:35:592
1217859993330,2cyclictest0-21swapper/207:49:582
1217859993330,2cyclictest0-21swapper/207:37:592
1217854993324,7cyclictest1263536-21kworker/u8:4+events_unbound09:45:140
1217863993229,2cyclictest0-21swapper/308:22:593
1217859993229,2cyclictest0-21swapper/212:24:562
1217859993229,2cyclictest0-21swapper/209:56:592
121785799320,28cyclictest1295881-21dpkg11:10:161
1217863993128,2cyclictest0-21swapper/309:47:593
1217859993128,2cyclictest0-21swapper/207:22:592
1217857993128,2cyclictest0-21swapper/111:01:591
121786399300,28cyclictest0-21swapper/307:40:163
1217859993027,2cyclictest0-21swapper/210:06:592
1217859993027,2cyclictest0-21swapper/207:27:592
121785999300,29cyclictest0-21swapper/211:30:142
121785799301,26cyclictest722-21Xorg08:56:591
1217863992927,1cyclictest0-21swapper/310:18:593
1217863992927,1cyclictest0-21swapper/310:18:593
1217863992927,1cyclictest0-21swapper/309:52:003
1217863992927,1cyclictest0-21swapper/307:49:553
1217863992926,2cyclictest0-21swapper/311:10:163
1217863992926,2cyclictest0-21swapper/309:34:593
121786399290,1cyclictest0-21swapper/308:35:143
1217859992926,2cyclictest0-21swapper/209:49:552
1217859992926,2cyclictest0-21swapper/209:10:152
121785999290,28cyclictest0-21swapper/208:25:142
121785999290,27cyclictest0-21swapper/212:25:162
121785999290,27cyclictest0-21swapper/212:15:152
121785999290,27cyclictest0-21swapper/211:50:162
121785999290,27cyclictest0-21swapper/211:50:152
121785999290,27cyclictest0-21swapper/211:35:172
121785999290,27cyclictest0-21swapper/209:30:132
121785999290,27cyclictest0-21swapper/208:50:022
121785999290,21cyclictest0-21swapper/208:00:022
1217863992826,1cyclictest0-21swapper/310:36:003
1217863992825,2cyclictest1282630-21turbostat10:30:003
1217863992825,2cyclictest0-21swapper/311:01:593
1217863992825,2cyclictest0-21swapper/309:21:593
1217863992823,4cyclictest1248665-21turbostat08:45:003
121786399280,2cyclictest1295635-21turbostat11:10:013
121786399280,2cyclictest1295635-21turbostat11:10:013
121786399280,26cyclictest1303849-21turbostat11:35:003
121786399280,26cyclictest0-21swapper/310:05:143
121786399280,26cyclictest0-21swapper/310:00:143
121786399280,26cyclictest0-21swapper/307:30:143
1217859992826,1cyclictest0-21swapper/210:15:132
1217859992826,1cyclictest0-21swapper/210:15:132
1217859992826,1cyclictest0-21swapper/208:15:152
1217859992826,1cyclictest0-21swapper/208:15:142
1217859992825,2cyclictest0-21swapper/211:03:592
1217859992825,2cyclictest0-21swapper/210:46:592
1217859992825,2cyclictest0-21swapper/210:35:162
1217859992825,2cyclictest0-21swapper/210:31:592
1217859992825,2cyclictest0-21swapper/208:55:142
121785999280,26cyclictest0-21swapper/209:10:002
1217863992725,1cyclictest0-21swapper/312:01:583
1217863992725,1cyclictest0-21swapper/310:55:013
1217863992724,2cyclictest0-21swapper/312:06:593
1217863992724,2cyclictest0-21swapper/309:25:583
1217863992720,5cyclictest1220941-21turbostat07:20:003
121786399270,26cyclictest0-21swapper/308:00:043
121786399270,1cyclictest0-21swapper/312:25:173
121786399270,1cyclictest0-21swapper/308:29:593
1217859992724,2cyclictest0-21swapper/210:59:592
1217859992724,2cyclictest0-21swapper/210:53:592
1217859992724,2cyclictest0-21swapper/210:12:592
121785999270,2cyclictest0-21swapper/208:25:002
121785999270,25cyclictest0-21swapper/211:11:592
121785999270,24cyclictest0-21swapper/210:23:582
121786399260,25cyclictest0-21swapper/309:41:583
121786399260,25cyclictest0-21swapper/307:56:593
121786399260,24cyclictest0-21swapper/307:10:183
1217859992623,2cyclictest0-21swapper/212:34:552
121785999260,24cyclictest0-21swapper/207:10:182
4505291250,6ptp4l61790irq/134-eno1-TxRx-011:59:563
4505291250,6ptp4l441ktimers/308:14:553
1217863992523,1cyclictest0-21swapper/310:49:553
1217863992523,1cyclictest0-21swapper/310:40:163
1217863992523,1cyclictest0-21swapper/310:40:163
1217863992522,2cyclictest431rcuc/312:23:113
1217863992522,2cyclictest431rcuc/312:14:253
1217863992522,2cyclictest431rcuc/307:38:263
1217863992522,2cyclictest0-21swapper/310:32:593
121786399250,24cyclictest0-21swapper/310:13:593
1217859992523,1cyclictest0-21swapper/211:47:592
1217859992523,1cyclictest0-21swapper/207:50:152
1217859992522,2cyclictest0-21swapper/209:15:592
4505291240,4ptp4l61790irq/134-eno1-TxRx-011:19:553
121786399240,1cyclictest0-21swapper/311:35:593
121785799240,22cyclictest1217934-21sed07:10:021
4505291230,3ptp4l441ktimers/308:34:563
1217863992321,1cyclictest0-21swapper/307:51:593
4505291220,4ptp4l61790irq/134-eno1-TxRx-009:19:563
1217863992220,1cyclictest431rcuc/311:40:263
1217859992220,1cyclictest0-21swapper/210:28:002
1217859992219,2cyclictest0-21swapper/211:21:002
1217859992119,1cyclictest0-21swapper/207:44:542
1217854991915,3cyclictest0-21swapper/007:10:010
1217859991613,2cyclictest0-21swapper/208:45:012
1217859991512,2cyclictest0-21swapper/208:41:582
1217859991411,2cyclictest0-21swapper/208:38:002
1217859991310,2cyclictest0-21swapper/209:24:562
121785999107,2cyclictest0-21swapper/212:04:002
12178599974,2cyclictest0-21swapper/207:59:562
12178599941,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional