You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 14:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sat Nov 29, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3181506999988,7cyclictest3287898-21latency_hist00:40:011
3181505998978,9cyclictest1-21systemd00:20:010
3181506998774,10cyclictest3274855-40mandb00:00:011
3181505998172,7cyclictest3240645-21latency_hist22:15:020
3181506998075,3cyclictest3255453-21kworker/u8:0+flush-179:023:00:261
3181505998070,7cyclictest3186249-21latency_hist19:25:010
3181505997872,4cyclictest3255454-21kworker/u8:2+flush-179:023:00:000
3181506997672,3cyclictest3267929-21kworker/u8:0+flush-179:023:40:251
3181506997666,8cyclictest3183291-21/usr/sbin/munin19:15:231
3181506997666,8cyclictest3183291-21/usr/sbin/munin19:15:231
3181505997667,7cyclictest3278083-21latency_hist00:10:010
3181506997569,4cyclictest3205324-21kworker/u8:0+flush-179:020:35:261
3181506997563,9cyclictest3227921-21latency_hist21:35:021
3181505997060,7cyclictest3218384-21latency_hist21:05:010
3181505996859,6cyclictest3267929-21kworker/u8:0+flush-179:023:40:010
3181506996558,5cyclictest3197349-21kworker/u8:0+flush-179:020:00:231
3181506996558,5cyclictest3197349-21kworker/u8:0+flush-179:020:00:221
3181505996558,6cyclictest3179520-21kworker/u8:0+flush-179:019:20:010
3181505996558,6cyclictest3179520-21kworker/u8:0+flush-179:019:20:000
3181506996453,8cyclictest3237476-21latency_hist22:05:011
3181505996458,5cyclictest3276484-21kworker/u8:4+flush-179:000:25:000
3181505996357,4cyclictest3274816-21kworker/u8:0+events_unbound00:00:140
3181507996259,2cyclictest0-21swapper/221:46:002
3181507996259,2cyclictest0-21swapper/221:45:592
3181506996250,9cyclictest3265035-21latency_hist23:30:011
3181506996249,8cyclictest271-21systemd-journal00:20:011
318150599620,1cyclictest0-21swapper/021:55:590
3181505996155,4cyclictest3276484-21kworker/u8:4+flush-179:000:39:540
3181505996152,7cyclictest3260255-21latency_hist23:15:010
3181506996055,4cyclictest3189398-21kworker/u8:2+flush-179:019:50:261
3181505995953,5cyclictest3226368-21kworker/u8:0+flush-179:021:50:250
3181505995953,4cyclictest3261007-21kworker/u8:1+events_unbound23:30:240
3181505995850,6cyclictest3197349-21kworker/u8:0+flush-179:020:10:210
3181507995754,2cyclictest0-21swapper/200:16:002
3181505995750,5cyclictest3208526-21apt-get20:35:010
3181505995748,6cyclictest3284708-21latency_hist00:30:010
3181505995747,7cyclictest3248627-21latency_hist22:40:010
3181505995747,4cyclictest3213640-21kworker/u8:2+events_unbound21:05:230
3181505995744,11cyclictest3274855-40mandb00:00:010
3181506995650,5cyclictest3250442-21kworker/u8:1+events_unbound22:45:231
3181506995650,4cyclictest3215447-21apt-get20:55:131
3181505995652,3cyclictest3261007-21kworker/u8:1+flush-179:023:25:170
3181505995649,3cyclictest3276484-21kworker/u8:4+flush-179:000:10:170
3181505995644,11cyclictest3184118-21kworker/u8:1+flush-179:019:25:210
3181505995548,5cyclictest3255453-21kworker/u8:0+flush-179:023:00:210
3181505995443,8cyclictest3216781-21latency_hist21:00:000
3181505995439,4cyclictest3212258-21kworker/u8:1+flush-179:021:20:000
3181506995351,1cyclictest0-21swapper/119:46:581
3181506995345,6cyclictest3202386-21kworker/u8:1+flush-179:020:15:231
3181506995344,6cyclictest3187834-21latency_hist19:30:001
3181506995343,8cyclictest271-21systemd-journal20:10:371
3181506995342,9cyclictest3254147-21apt-get22:55:131
3181505995349,3cyclictest3242205-21kworker/u8:0+flush-179:022:40:160
3181506995248,3cyclictest3255453-21kworker/u8:0+events_unbound23:05:201
3181505995247,4cyclictest3213640-21kworker/u8:2+events_unbound21:20:230
318150699510,49cyclictest1-21systemd23:20:001
3181505995146,4cyclictest3271646-21kworker/u8:2+events_unbound00:30:250
3181506995046,3cyclictest3224762-21apt21:24:591
3181506995045,4cyclictest3213640-21kworker/u8:2+events_unbound21:10:231
3181506995044,4cyclictest3184919-21apt-get19:20:131
3181506995043,6cyclictest3189398-21kworker/u8:2+flush-179:019:40:011
3181505995045,4cyclictest3212258-21kworker/u8:1+flush-179:020:50:220
3181506994842,5cyclictest3197349-21kworker/u8:0+flush-179:020:10:011
3181506994841,5cyclictest3242205-21kworker/u8:0+events_unbound22:40:031
3181506994839,6cyclictest3247013-21latency_hist22:35:011
3181505994843,4cyclictest3205324-21kworker/u8:0+flush-179:020:25:000
3181505994839,7cyclictest3212260-21/usr/sbin/munin20:45:250
4670791470,13ptp4l441ktimers/319:10:013
3181506994741,4cyclictest3269798-21kworker/u8:1+flush-179:023:50:171
3181505994742,4cyclictest3179520-21kworker/u8:0+flush-179:019:35:230
3181505994741,4cyclictest3192650-21kworker/u8:1+flush-179:019:55:160
3181505994740,5cyclictest271-21systemd-journal21:33:360
318150899460,44cyclictest3194210-21turbostat19:50:003
3181506994640,4cyclictest3212258-21kworker/u8:1+flush-179:020:55:001
3181505994641,4cyclictest3189398-21kworker/u8:2+events_unbound20:05:180
3181505994637,6cyclictest3213640-21kworker/u8:2+flush-179:021:35:240
3181505994636,8cyclictest271-21systemd-journal20:34:590
3181506994542,2cyclictest3247547-21kworker/u8:2+events_unbound22:35:241
3181506994540,3cyclictest723-21Xorg23:31:591
318150699450,43cyclictest723-21Xorg21:49:001
318150699450,43cyclictest723-21Xorg21:48:591
3181505994542,2cyclictest3252085-21systemd-run22:50:010
3181505994538,4cyclictest3237713-21kworker/u8:1+flush-179:022:05:370
3181506994440,3cyclictest3242205-21kworker/u8:0+events_unbound22:25:191
3181506994438,5cyclictest3189398-21kworker/u8:2+flush-179:020:25:011
3181506994337,5cyclictest3274901-21kworker/u8:3+flush-179:000:00:231
3181506994335,6cyclictest3205324-21kworker/u8:0+flush-179:020:25:231
3181506994335,6cyclictest3205324-21kworker/u8:0+flush-179:020:25:221
3181505994338,4cyclictest3244076-21kworker/u8:1+events_unbound22:25:130
3181505994338,4cyclictest3213640-21kworker/u8:2+events_unbound21:10:130
3181505994336,6cyclictest271-21systemd-journal22:32:370
3181505994336,5cyclictest723-21Xorg22:20:580
318150899420,1cyclictest0-21swapper/323:34:003
3181506994237,4cyclictest3213640-21kworker/u8:2+flush-179:021:25:161
3181506994234,6cyclictest3276484-21kworker/u8:4+flush-179:000:10:151
3181506994232,8cyclictest3270328-21apt-get23:45:121
318150699420,40cyclictest723-21Xorg20:42:591
318150599420,41cyclictest0-21swapper/022:15:590
318150599420,40cyclictest3261859-21turbostat23:24:550
3181508994137,3cyclictest3231131-21turbostat21:45:013
318150899410,1cyclictest0-21swapper/322:31:593
3181506994136,4cyclictest723-21Xorg21:55:591
3181506994136,4cyclictest3276484-21kworker/u8:4+events_unbound00:05:251
3181506994136,3cyclictest1-21systemd23:15:001
3181506994134,6cyclictest3226368-21kworker/u8:0+flush-179:021:50:221
3181505994134,5cyclictest3189398-21kworker/u8:2+events_unbound19:35:000
318150599412,37cyclictest723-21Xorg23:48:000
3181508994033,5cyclictest3235894-21turbostat22:00:003
318150899400,4cyclictest3215201-21turbostat20:55:003
318150899400,38cyclictest3255496-21turbostat23:00:003
318150899400,38cyclictest3234287-21turbostat21:55:013
3181506994037,2cyclictest0-21swapper/122:17:591
3181506994033,6cyclictest3213640-21kworker/u8:2+events_unbound21:15:251
3181508993932,6cyclictest3191017-21turbostat19:40:013
3181507993936,2cyclictest0-21swapper/220:09:582
3181506993935,2cyclictest3284724-21idleruntime-cro00:30:001
318150699390,2cyclictest723-21Xorg22:21:591
3181505993933,4cyclictest3271896-21/usr/sbin/munin23:50:240
3181505993932,5cyclictest3267929-21kworker/u8:0+flush-179:023:40:150
3181505993931,7cyclictest3197349-21kworker/u8:0+flush-179:020:00:180
3181505993931,7cyclictest3197349-21kworker/u8:0+flush-179:020:00:170
318150599390,2cyclictest723-21Xorg23:15:590
3181507993835,2cyclictest0-21swapper/223:19:552
318150799380,36cyclictest0-21swapper/223:34:582
318150799380,36cyclictest0-21swapper/219:22:592
3181506993833,3cyclictest3261007-21kworker/u8:1+events_unbound23:35:241
3181506993829,7cyclictest3283127-21latency_hist00:25:021
3181506993829,6cyclictest3252285-21/usr/sbin/munin22:50:171
318150699380,37cyclictest0-21swapper/119:57:591
3181505993835,2cyclictest723-21Xorg21:29:590
318150599381,32cyclictest0-21swapper/022:00:130
3181508993731,4cyclictest3260263-21turbostat23:15:003
318150899370,36cyclictest0-21swapper/300:04:583
3181507993734,2cyclictest0-21swapper/220:04:562
3181507993734,2cyclictest0-21swapper/220:04:562
318150799370,35cyclictest0-21swapper/222:38:592
3181506993726,9cyclictest3237732-21apt-get22:05:121
3181505993732,4cyclictest3205324-21kworker/u8:0+events_unbound20:40:220
3181508993634,1cyclictest0-21swapper/320:04:003
3181508993634,1cyclictest0-21swapper/320:03:593
3181508993633,2cyclictest0-21swapper/323:44:593
318150899360,35cyclictest0-21swapper/323:58:593
318150899360,35cyclictest0-21swapper/319:59:563
318150899360,1cyclictest0-21swapper/321:45:593
318150899360,1cyclictest0-21swapper/321:45:593
318150899360,1cyclictest0-21swapper/319:30:583
3181507993633,2cyclictest0-21swapper/222:55:002
3181507993633,2cyclictest0-21swapper/221:54:542
3181507993632,2cyclictest0-21swapper/222:35:002
3181506993633,2cyclictest3179520-21kworker/u8:0+events_unbound19:30:141
3181506993633,2cyclictest0-21swapper/120:31:591
3181506993633,2cyclictest0-21swapper/119:12:581
3181506993630,5cyclictest3213640-21kworker/u8:2+events_unbound21:05:151
3181508993532,2cyclictest0-21swapper/321:21:593
318150899350,1cyclictest0-21swapper/323:09:553
318150899350,1cyclictest0-21swapper/321:09:593
3181507993532,2cyclictest0-21swapper/221:10:012
3181507993532,2cyclictest0-21swapper/200:36:592
318150799350,2cyclictest0-21swapper/222:01:592
318150799350,1cyclictest0-21swapper/223:05:592
318150799350,1cyclictest0-21swapper/222:29:552
3181506993533,1cyclictest0-21swapper/122:10:591
3181505993530,4cyclictest3223207-21kworker/u8:3+flush-179:021:45:000
3181505993529,4cyclictest3194888-21hwlatdetect19:50:180
3181508993432,1cyclictest0-21swapper/323:22:593
3181508993432,1cyclictest0-21swapper/322:35:583
3181508993428,5cyclictest3257094-21turbostat23:05:003
3181507993431,2cyclictest0-21swapper/222:48:592
3181507993431,2cyclictest0-21swapper/220:39:592
3181505993429,2cyclictest3202405-21apt-get20:15:150
3181505993428,4cyclictest0-21swapper/022:55:000
3181508993331,1cyclictest0-21swapper/300:21:583
3181508993328,4cyclictest3281535-21turbostat00:20:013
3181507993331,1cyclictest0-21swapper/219:20:002
3181507993331,1cyclictest0-21swapper/219:20:002
3181507993330,2cyclictest0-21swapper/200:05:582
318150599331,4cyclictest0-21swapper/019:45:140
3181508993226,4cyclictest3287923-21turbostat00:40:003
3181507993230,1cyclictest0-21swapper/223:49:592
3181507993229,2cyclictest0-21swapper/223:41:592
3181507993229,2cyclictest0-21swapper/221:14:002
3181507993229,2cyclictest0-21swapper/220:29:552
3181507993229,2cyclictest0-21swapper/220:29:552
3181507993229,2cyclictest0-21swapper/219:13:592
3181507993229,2cyclictest0-21swapper/200:31:002
3181507993229,2cyclictest0-21swapper/200:20:592
3181506993228,3cyclictest3286302-21cron00:34:591
3181506993226,5cyclictest3179520-21kworker/u8:0+events_unbound19:40:201
3181505993228,3cyclictest3192617-21idleruntime-cro19:45:000
3181505993227,3cyclictest0-21swapper/023:05:140
3181505993224,6cyclictest3223207-21kworker/u8:3+flush-179:021:45:210
3181505993224,6cyclictest3223207-21kworker/u8:3+flush-179:021:45:210
3181508993129,1cyclictest0-21swapper/300:13:593
3181508993128,2cyclictest0-21swapper/321:13:593
3181508993126,4cyclictest3245439-21turbostat22:30:003
3181507993129,1cyclictest0-21swapper/220:50:002
3181507993129,1cyclictest0-21swapper/219:55:012
3181507993128,2cyclictest0-21swapper/223:52:592
318150799310,2cyclictest0-21swapper/200:29:592
318150699311,2cyclictest0-21swapper/123:20:151
318150599310,29cyclictest3206944-21ls20:30:000
318150599310,29cyclictest3206944-21ls20:30:000
3181508993024,5cyclictest3271672-21turbostat23:50:003
318150899300,29cyclictest0-21swapper/322:10:153
318150899300,28cyclictest0-21swapper/323:50:143
3181507993027,2cyclictest0-21swapper/220:30:162
3181507993027,2cyclictest0-21swapper/219:55:582
318150799300,29cyclictest0-21swapper/223:25:142
3181508992926,2cyclictest0-21swapper/320:30:153
3181508992926,2cyclictest0-21swapper/300:09:593
318150899290,2cyclictest0-21swapper/323:25:143
318150899290,28cyclictest0-21swapper/320:25:143
318150899290,28cyclictest0-21swapper/320:25:133
318150899290,27cyclictest0-21swapper/320:35:013
3181507992926,2cyclictest0-21swapper/222:15:142
3181507992926,2cyclictest0-21swapper/220:41:582
3181507992926,2cyclictest0-21swapper/200:10:142
318150799290,28cyclictest0-21swapper/220:50:142
318150799290,27cyclictest0-21swapper/222:55:022
318150799290,27cyclictest0-21swapper/222:06:582
318150799290,1cyclictest0-21swapper/200:00:132
318150699290,3cyclictest723-21Xorg20:47:591
3181508992826,1cyclictest0-21swapper/322:41:593
3181508992826,1cyclictest0-21swapper/300:30:143
3181508992825,2cyclictest0-21swapper/323:38:593
3181508992825,2cyclictest0-21swapper/320:40:133
3181508992823,4cyclictest3183062-21turbostat19:15:013
318150899280,2cyclictest3284716-21turbostat00:30:003
318150899280,26cyclictest0-21swapper/319:27:593
318150899280,1cyclictest0-21swapper/320:45:153
3181507992826,1cyclictest0-21swapper/220:24:562
3181507992825,2cyclictest0-21swapper/223:20:592
3181507992825,2cyclictest0-21swapper/221:00:592
3181507992825,2cyclictest0-21swapper/219:36:592
3181507992825,2cyclictest0-21swapper/200:00:022
318150799280,2cyclictest0-21swapper/219:30:132
3181505992822,4cyclictest0-21swapper/019:15:010
4670791270,6ptp4l61690irq/136-eno1-TxRx-019:55:003
3181508992725,1cyclictest0-21swapper/319:21:003
318150899270,26cyclictest0-21swapper/323:15:123
318150899270,26cyclictest0-21swapper/320:12:593
318150899270,1cyclictest0-21swapper/322:15:133
3181506992722,4cyclictest3229356-21unixbench-2d21:35:261
3181506992721,4cyclictest3212258-21kworker/u8:1+flush-179:021:05:001
4670791260,7ptp4l61690irq/136-eno1-TxRx-022:05:003
4670791260,6ptp4l441ktimers/321:30:003
4670791260,5ptp4l441ktimers/321:20:003
3181508992624,1cyclictest0-21swapper/320:59:553
3181507992623,2cyclictest0-21swapper/222:44:552
3181507992623,2cyclictest0-21swapper/222:14:592
3181507992623,2cyclictest0-21swapper/220:11:582
318150799260,24cyclictest0-21swapper/222:23:592
318150799260,24cyclictest0-21swapper/219:28:412
4670791250,6ptp4l61690irq/136-eno1-TxRx-021:35:003
4670791250,5ptp4l61690irq/136-eno1-TxRx-019:45:003
3181508992523,1cyclictest0-21swapper/319:19:393
3181508992523,1cyclictest0-21swapper/319:19:393
318150899250,24cyclictest0-21swapper/322:50:593
318150899250,1cyclictest0-21swapper/320:05:123
3181507992522,1cyclictest0-21swapper/219:50:002
3181505992517,5cyclictest3181423-21latency_hist19:10:020
3181508992422,1cyclictest0-21swapper/322:20:583
3181508992422,1cyclictest0-21swapper/320:24:583
3181507992421,2cyclictest0-21swapper/221:19:592
4670791230,4ptp4l441ktimers/321:05:013
4670791230,3ptp4l441ktimers/322:50:003
3181508992320,2cyclictest0-21swapper/322:08:593
3181506992317,5cyclictest3226368-21kworker/u8:0+flush-179:021:40:161
4670791220,3ptp4l441ktimers/321:39:563
3181507992220,1cyclictest0-21swapper/221:26:002
202199220,13rtkit-daemon485-21gpm19:05:061
3181507992118,2cyclictest0-21swapper/221:23:582
3181507992017,2cyclictest0-21swapper/221:42:592
3181507991916,2cyclictest0-21swapper/220:17:002
3181508991816,1cyclictest0-21swapper/320:19:003
3181507991815,2cyclictest0-21swapper/221:33:592
3181507991814,2cyclictest3192620-21turbostat19:44:592
3181507991613,2cyclictest0-21swapper/223:38:592
3181507991613,2cyclictest0-21swapper/221:55:112
3181507991513,1cyclictest0-21swapper/220:57:592
318150799129,2cyclictest0-21swapper/223:13:592
31815079996,2cyclictest0-21swapper/223:05:002
31815079971,4cyclictest3227931-21turbostat21:39:552
31815079930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional