You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 14:54
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Fri Dec 05, 2025 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32377759910898,7cyclictest3269576-21kworker/u8:3+flush-179:021:00:000
3237779999993,4cyclictest3340943-21kworker/u8:0+events_unbound00:30:291
3237775999794,2cyclictest3311674-21kworker/u8:0+flush-179:023:05:000
3237775999786,7cyclictest3284380-21latency_hist21:35:010
3237775999588,4cyclictest3298932-21kworker/u8:4+flush-179:022:30:010
3237779999489,4cyclictest3306965-21kworker/u8:2+events_unbound23:00:001
3237775999486,6cyclictest3328159-21kworker/u8:2+flush-179:000:00:010
3237775999384,7cyclictest3286004-21latency_hist21:40:010
3237779999187,3cyclictest3317260-21kworker/u8:0+events_unbound23:15:251
3237779999079,9cyclictest3323480-21cp23:35:021
3237779998984,4cyclictest3331361-21kworker/u8:0+flush-179:000:10:251
3237779998782,4cyclictest3328159-21kworker/u8:2+events_unbound23:55:281
3237775998671,5cyclictest3302193-21kworker/u8:2+events_unbound22:35:000
3237779998578,5cyclictest3271171-21kworker/u8:0+events_unbound21:10:021
3237779998575,8cyclictest3287584-21apt-get21:45:021
3237775998373,4cyclictest3255212-21kworker/u8:2+flush-179:020:15:250
3237779998278,3cyclictest3236028-21kworker/u8:2+flush-179:019:25:241
3237779998277,4cyclictest2056724-21kworker/u8:4+flush-179:019:10:251
3237775998276,5cyclictest3320447-21kworker/u8:1+flush-179:023:29:590
3237775998269,4cyclictest3236028-21kworker/u8:2+events_unbound19:20:230
3237779998176,4cyclictest3262983-21kworker/u8:1+events_unbound20:30:311
3237779998171,7cyclictest3310098-21latency_hist22:55:011
3237775998072,6cyclictest3320447-21kworker/u8:1+flush-179:023:45:000
3237779997970,6cyclictest3298971-21latency_hist22:20:011
3237775997969,6cyclictest3255212-21kworker/u8:2+events_unbound20:20:240
3237779997873,4cyclictest3292080-21kworker/u8:1+events_unbound23:10:291
3237779997772,4cyclictest3292080-21kworker/u8:1+events_unbound21:55:271
3237775997768,7cyclictest3250637-21/usr/sbin/munin19:50:240
3237779997667,3cyclictest3269576-21kworker/u8:3+events_unbound21:00:291
3237779997664,9cyclictest3306930-21latency_hist22:45:001
3237779997570,4cyclictest3247187-21kworker/u8:0+events_unbound20:35:281
3237775997564,4cyclictest3269576-21kworker/u8:3+events_unbound21:10:240
3237779997470,3cyclictest3280982-21kworker/u8:1+events_unbound21:30:331
3237775997467,4cyclictest3269576-21kworker/u8:3+events_unbound21:20:250
3237779997368,4cyclictest3244698-21kworker/u8:3+events_unbound19:30:251
3237779997361,11cyclictest3247187-21kworker/u8:0+events_unbound20:40:211
3237779997267,4cyclictest3280982-21kworker/u8:1+flush-179:021:25:291
3237775997267,3cyclictest3292080-21kworker/u8:1+flush-179:022:20:230
3237775997261,4cyclictest3313337-21kworker/u8:3+events_unbound23:45:260
3237779997167,3cyclictest3255212-21kworker/u8:2+flush-179:020:20:001
3237779996959,3cyclictest3247187-21kworker/u8:0+events_unbound19:55:201
3237779996956,4cyclictest3292567-21kworker/u8:2+events_unbound22:10:231
3237775996962,4cyclictest3236028-21kworker/u8:2+events_unbound19:25:130
3237775996862,5cyclictest3292080-21kworker/u8:1+flush-179:022:55:260
3237775996861,4cyclictest3302193-21kworker/u8:2+flush-179:022:35:160
3237775996755,4cyclictest3313337-21kworker/u8:3+events_unbound00:25:000
3237775996754,6cyclictest3292080-21kworker/u8:1+flush-179:023:05:280
3237779996661,4cyclictest3302193-21kworker/u8:2+flush-179:022:35:251
3237775996659,5cyclictest3298932-21kworker/u8:4+flush-179:022:50:270
3237779996561,3cyclictest3324989-21kworker/u8:0+events_unbound23:40:241
3237779996560,4cyclictest3289452-21kworker/u8:0+flush-179:022:05:281
3237779996560,4cyclictest3289452-21kworker/u8:0+flush-179:022:05:281
3237779996555,4cyclictest3247187-21kworker/u8:0+events_unbound20:25:251
3237779996555,4cyclictest3247187-21kworker/u8:0+events_unbound20:25:241
3237775996561,2cyclictest723-21Xorg21:05:000
3237775996557,5cyclictest3248816-21latency_hist19:45:000
3237779996460,3cyclictest3242494-21kworker/u8:1+events_unbound19:35:141
3237775996457,4cyclictest3262983-21kworker/u8:1+events_unbound20:40:170
3237775996457,4cyclictest3242494-21kworker/u8:1+flush-179:019:35:170
3237775996451,5cyclictest3239490-21kworker/u8:0+events_unbound19:30:150
3237775996354,4cyclictest3331361-21kworker/u8:0+events_unbound00:15:000
3237779996257,4cyclictest3331361-21kworker/u8:0+events_unbound00:20:231
3237779996257,4cyclictest3328159-21kworker/u8:2+flush-179:023:50:261
3237779996257,4cyclictest3306965-21kworker/u8:2+flush-179:022:45:281
3237779996253,7cyclictest3277990-21/usr/sbin/munin21:15:251
3237779996253,3cyclictest3283030-21kworker/u8:2+events_unbound21:35:201
3237775996258,3cyclictest3284415-21kworker/u8:3+events_unbound21:55:190
3237775996257,4cyclictest3271171-21kworker/u8:0+flush-179:021:05:270
3237775996253,7cyclictest3323096-21apt-get23:35:000
3237779996157,3cyclictest3320447-21kworker/u8:1+events_unbound00:00:251
3237779996156,4cyclictest3289452-21kworker/u8:0+events_unbound22:00:261
3237779996152,3cyclictest3313337-21kworker/u8:3+events_unbound23:45:281
3237779996151,4cyclictest3292080-21kworker/u8:1+events_unbound22:31:291
3237775996157,3cyclictest3247187-21kworker/u8:0+flush-179:020:45:260
3237775996155,4cyclictest3317260-21kworker/u8:0+flush-179:023:20:200
3237775996155,4cyclictest3313337-21kworker/u8:3+flush-179:000:25:260
3237775996155,4cyclictest3280982-21kworker/u8:1+flush-179:021:45:250
3237775996154,5cyclictest3313337-21kworker/u8:3+flush-179:023:15:000
3237775996154,4cyclictest3297404-21kworker/u8:3+flush-179:022:15:140
3237775996152,7cyclictest3331361-21kworker/u8:0+flush-179:000:20:000
3237775996146,6cyclictest3247187-21kworker/u8:0+flush-179:020:05:220
3237779996056,3cyclictest3247264-21kworker/u8:2+events_unbound19:45:181
3237779996056,3cyclictest3242494-21kworker/u8:1+events_unbound19:40:251
3237775996056,3cyclictest3262983-21kworker/u8:1+flush-179:020:30:300
3237775996053,5cyclictest3340943-21kworker/u8:0+flush-179:000:30:310
3237779995955,3cyclictest3271171-21kworker/u8:0+events_unbound21:05:231
3237779995955,3cyclictest3262983-21kworker/u8:1+events_unbound20:45:241
3237779995954,3cyclictest3313337-21kworker/u8:3+flush-179:000:15:151
3237779995951,3cyclictest3292080-21kworker/u8:1+events_unbound22:20:191
3237775995951,5cyclictest3247187-21kworker/u8:0+events_unbound20:10:180
3237779995854,3cyclictest3247187-21kworker/u8:0+flush-179:020:05:161
3237779995853,4cyclictest3271171-21kworker/u8:0+flush-179:020:55:291
3237779995853,4cyclictest3253292-21kworker/u8:1+events_unbound20:00:221
3237775995853,3cyclictest3292567-21kworker/u8:2+flush-179:022:00:210
3237775995852,5cyclictest3283030-21kworker/u8:2+flush-179:021:40:210
3237775995851,5cyclictest3269576-21kworker/u8:3+events_unbound20:55:000
3237775995851,4cyclictest3269576-21kworker/u8:3+events_unbound21:15:220
3237775995848,7cyclictest3318330-21latency_hist23:20:020
3237779995752,4cyclictest3313337-21kworker/u8:3+events_unbound00:05:201
3237775995745,4cyclictest3320447-21kworker/u8:1+events_unbound23:55:010
3237779995651,4cyclictest3311674-21kworker/u8:0+flush-179:023:00:151
3237779995651,4cyclictest3269576-21kworker/u8:3+events_unbound20:50:221
3237775995652,3cyclictest3289452-21kworker/u8:0+events_unbound21:50:230
3237775995645,9cyclictest3252239-21apt-get19:55:140
3237779995446,3cyclictest3247264-21kworker/u8:2+events_unbound19:50:251
3237775995447,5cyclictest3280982-21kworker/u8:1+events_unbound21:25:170
3237775995445,7cyclictest3239491-21/usr/sbin/munin19:15:230
3237785995320,1cyclictest0-21swapper/319:40:583
3237779995348,4cyclictest3236028-21kworker/u8:2+flush-179:019:20:241
3237779995347,5cyclictest3313337-21kworker/u8:3+flush-179:000:25:261
3237775995347,5cyclictest3328159-21kworker/u8:2+flush-179:000:09:270
3237775995343,7cyclictest3250399-21latency_hist19:50:000
3237775995145,4cyclictest3253292-21kworker/u8:1+flush-179:020:00:250
3237779994945,3cyclictest3340946-21kworker/u8:2+flush-179:000:35:301
3237779994943,5cyclictest3298932-21kworker/u8:4+flush-179:022:29:561
3237775994944,4cyclictest3298932-21kworker/u8:4+flush-179:022:40:240
3237775994944,4cyclictest3292567-21kworker/u8:2+flush-179:022:10:250
3237779994844,3cyclictest3239490-21kworker/u8:0+events_unbound19:15:251
4670791470,13ptp4l61690irq/136-eno1-TxRx-019:10:003
3237779994736,9cyclictest3318586-21apt-get23:20:121
3237779994642,3cyclictest3280982-21kworker/u8:1+flush-179:021:40:251
3237775994641,4cyclictest979226-21gdbus00:04:000
3237775994541,3cyclictest3262983-21kworker/u8:1+flush-179:020:35:200
3237779994339,2cyclictest171rcu_preempt21:20:141
3237779994338,4cyclictest3289452-21kworker/u8:0+flush-179:021:50:151
3237775994339,3cyclictest3323081-21kworker/u8:2+flush-179:023:35:220
3237775994338,4cyclictest3340943-21kworker/u8:0+events_unbound00:35:190
3237779994237,4cyclictest3321757-21apt-get23:30:131
3237779994237,4cyclictest3255212-21kworker/u8:2+events_unbound20:20:281
3237775994236,4cyclictest3306965-21kworker/u8:2+flush-179:022:45:150
3237775994236,4cyclictest3289452-21kworker/u8:0+flush-179:022:05:240
3237775994236,4cyclictest3289452-21kworker/u8:0+flush-179:022:05:240
323778599410,39cyclictest3240850-21turbostat19:20:003
3237785994034,5cyclictest3311694-21turbostat23:00:003
3237785994034,5cyclictest3279362-21turbostat21:20:003
323778599400,38cyclictest3337746-21turbostat00:19:593
323778599400,38cyclictest3323095-21turbostat23:35:003
323778099390,36cyclictest0-21swapper/200:09:552
3237779993934,4cyclictest3247187-21kworker/u8:0+events_unbound20:13:271
3237775993935,3cyclictest3236028-21kworker/u8:2+events_unbound19:10:190
3237785993833,4cyclictest3328142-21turbostat23:49:593
3237785993832,5cyclictest3261600-21turbostat20:25:003
323778099380,2cyclictest0-21swapper/222:32:592
3237779993834,3cyclictest3292080-21kworker/u8:1+events_unbound23:05:031
3237779993833,4cyclictest3320447-21kworker/u8:1+events_unbound23:28:021
3237785993735,1cyclictest0-21swapper/320:57:593
323778599370,36cyclictest0-21swapper/322:51:003
3237780993735,1cyclictest0-21swapper/223:14:582
3237780993733,2cyclictest0-21swapper/220:21:592
323778099370,35cyclictest0-21swapper/222:15:002
323778099370,35cyclictest0-21swapper/220:04:002
3237785993634,1cyclictest0-21swapper/323:50:593
3237785993634,1cyclictest0-21swapper/323:20:593
3237785993634,1cyclictest0-21swapper/321:32:593
3237785993634,1cyclictest0-21swapper/320:04:553
3237785993633,2cyclictest0-21swapper/323:27:583
3237785993633,2cyclictest0-21swapper/323:11:593
3237780993633,2cyclictest0-21swapper/222:40:592
3237780993633,2cyclictest0-21swapper/222:24:552
3237780993633,2cyclictest0-21swapper/219:54:552
323778099360,1cyclictest0-21swapper/219:31:592
3237785993532,2cyclictest0-21swapper/322:36:593
3237785993529,5cyclictest3287583-21turbostat21:45:003
3237785993528,5cyclictest3267976-21turbostat20:45:003
3237780993532,2cyclictest0-21swapper/221:08:582
3237775993525,7cyclictest3263179-21latency_hist20:30:010
3237775993525,7cyclictest3263179-21latency_hist20:30:000
3237785993432,1cyclictest0-21swapper/322:18:593
3237785993432,1cyclictest0-21swapper/322:14:583
3237785993432,1cyclictest0-21swapper/300:31:003
3237785993429,4cyclictest3256824-21turbostat20:09:593
3237785993428,5cyclictest3239259-21turbostat19:14:593
3237785993427,5cyclictest3295791-21turbostat22:10:003
3237785993427,5cyclictest3295791-21turbostat22:10:003
3237785993331,1cyclictest0-21swapper/319:48:593
3237785993327,4cyclictest3339330-21turbostat00:25:003
3237780993331,1cyclictest0-21swapper/221:14:592
3237780993331,1cyclictest0-21swapper/220:44:592
3237780993330,2cyclictest0-21swapper/221:47:592
3237780993330,2cyclictest0-21swapper/220:09:542
3237785993226,4cyclictest3324957-21turbostat23:40:013
323778599320,30cyclictest0-21swapper/319:56:593
3237780993230,1cyclictest0-21swapper/221:34:592
3237780993230,1cyclictest0-21swapper/220:59:592
3237780993228,2cyclictest0-21swapper/219:37:592
3237785993129,1cyclictest0-21swapper/321:29:553
3237785993126,4cyclictest3326546-21turbostat23:45:003
3237785993126,4cyclictest3258415-21turbostat20:15:013
3237780993129,1cyclictest0-21swapper/220:20:002
3237785993026,3cyclictest3240850-21turbostat19:24:553
3237780993028,1cyclictest0-21swapper/221:24:592
3237780993026,2cyclictest0-21swapper/222:55:162
3237780993026,2cyclictest0-21swapper/220:50:172
323778099300,2cyclictest0-21swapper/223:45:142
323778099300,28cyclictest0-21swapper/200:10:142
3237785992927,1cyclictest0-21swapper/321:46:593
3237785992926,2cyclictest0-21swapper/300:35:583
323778599292,26cyclictest3277768-21turbostat21:15:003
3237780992926,2cyclictest0-21swapper/223:35:042
3237780992926,2cyclictest0-21swapper/222:03:592
3237780992926,2cyclictest0-21swapper/200:20:152
323778099290,27cyclictest0-21swapper/223:00:132
323778099290,27cyclictest0-21swapper/222:50:142
323778099290,27cyclictest0-21swapper/221:25:162
323778099290,27cyclictest0-21swapper/220:45:132
323778099290,27cyclictest0-21swapper/219:15:582
323778099290,1cyclictest0-21swapper/219:50:002
3237785992825,2cyclictest0-21swapper/321:36:593
323778599280,27cyclictest0-21swapper/300:13:003
323778599280,26cyclictest0-21swapper/322:30:173
323778599280,26cyclictest0-21swapper/321:00:193
323778599280,26cyclictest0-21swapper/320:15:133
323778599280,26cyclictest0-21swapper/300:25:183
3237780992826,1cyclictest0-21swapper/223:40:142
3237780992825,2cyclictest0-21swapper/223:50:122
3237780992825,2cyclictest0-21swapper/223:05:162
3237780992825,2cyclictest0-21swapper/222:07:592
3237780992825,2cyclictest0-21swapper/222:07:592
3237780992824,2cyclictest0-21swapper/221:54:552
323778099280,2cyclictest0-21swapper/200:00:162
323778099280,26cyclictest0-21swapper/220:30:202
4670791270,6ptp4l441ktimers/322:30:013
3237785992725,1cyclictest0-21swapper/323:17:003
3237785992725,1cyclictest0-21swapper/319:54:003
323778599270,26cyclictest0-21swapper/321:58:583
323778599270,26cyclictest0-21swapper/320:52:593
323778599270,26cyclictest0-21swapper/319:34:593
323778599270,1cyclictest0-21swapper/320:48:593
3237780992724,2cyclictest0-21swapper/223:32:592
323778099270,26cyclictest0-21swapper/200:19:592
323778099270,24cyclictest0-21swapper/200:34:562
4670791260,7ptp4l441ktimers/322:05:013
4670791260,6ptp4l441ktimers/321:55:003
4670791260,6ptp4l441ktimers/321:09:553
4670791260,6ptp4l441ktimers/320:35:003
4670791260,6ptp4l441ktimers/319:30:013
4670791260,6ptp4l441ktimers/300:05:003
4670791260,5ptp4l441ktimers/322:25:013
4670791260,5ptp4l441ktimers/300:10:013
3237785992624,1cyclictest0-21swapper/319:39:003
3237785992623,2cyclictest0-21swapper/322:48:593
3237780992623,2cyclictest0-21swapper/223:26:592
3237780992623,2cyclictest0-21swapper/220:37:592
3237780992623,2cyclictest0-21swapper/200:39:592
3237780992623,2cyclictest0-21swapper/200:29:592
3237780992623,2cyclictest0-21swapper/200:00:022
323778099260,24cyclictest0-21swapper/223:24:592
4670791250,6ptp4l441ktimers/322:45:013
4670791250,5ptp4l441ktimers/320:39:553
3237785992523,1cyclictest0-21swapper/320:28:003
3237785992523,1cyclictest0-21swapper/320:27:593
3237780992523,1cyclictest0-21swapper/222:00:002
4670791240,6ptp4l441ktimers/300:00:003
4670791240,5ptp4l61690irq/136-eno1-TxRx-021:25:003
3237780992422,1cyclictest0-21swapper/221:37:592
3237780992422,1cyclictest0-21swapper/221:19:592
3237780992422,1cyclictest0-21swapper/220:00:002
3237780992422,1cyclictest0-21swapper/219:30:012
3237780992421,2cyclictest0-21swapper/222:38:582
323778099240,1cyclictest0-21swapper/219:10:162
4670791230,5ptp4l61690irq/136-eno1-TxRx-023:09:593
4670791230,4ptp4l441ktimers/323:04:553
3237780992320,2cyclictest0-21swapper/220:11:582
3237780992219,2cyclictest0-21swapper/222:15:582
3237779992217,3cyclictest0-21swapper/119:10:021
3237780992119,1cyclictest0-21swapper/219:25:002
3237780991916,2cyclictest0-21swapper/219:44:552
3237780991712,3cyclictest3263197-21turbostat20:29:592
3237780991712,3cyclictest3263197-21turbostat20:29:592
3237780991613,2cyclictest0-21swapper/221:01:592
3237780991513,1cyclictest0-21swapper/221:45:002
3237780991310,2cyclictest0-21swapper/222:29:592
3237775991310,2cyclictest0-21swapper/019:10:010
323778099119,2cyclictest0-21swapper/223:19:002
32377809974,2cyclictest0-21swapper/222:45:032
32377809930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional