You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-20 - 20:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Fri Sep 20, 2024 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
348375299114108,4cyclictest3485272-21kworker/u8:3+events_unbound07:15:241
348375299111101,8cyclictest3511133-21kworker/u8:4+flush-179:009:00:011
34837529910290,9cyclictest3557858-21/usr/sbin/munin11:00:241
34837479910189,9cyclictest3556048-21kworker/u8:2+flush-179:011:00:000
3483752999992,6cyclictest3490124-21kworker/u8:0+events_unbound07:49:591
3483752999992,6cyclictest3490124-21kworker/u8:0+events_unbound07:49:591
3483747999784,10cyclictest3512774-21latency_hist08:40:000
3483747999688,6cyclictest3582567-21kworker/u8:2+flush-179:012:15:250
3483752999389,3cyclictest3523873-21kworker/u8:2+events_unbound09:15:001
3483752999383,8cyclictest3494949-21kworker/u8:3+events_unbound08:25:001
3483747999382,9cyclictest3515975-21kworker/u8:1+flush-179:009:30:010
3483747999380,10cyclictest3105085-21systemd-journal07:45:010
3483747999180,9cyclictest3523873-21kworker/u8:2+flush-179:010:15:010
3483752998982,5cyclictest3546502-21kworker/u8:1+events_unbound10:45:231
3483752998982,5cyclictest3546502-21kworker/u8:1+events_unbound10:45:231
3483747998972,15cyclictest3546502-21kworker/u8:1+flush-179:011:10:210
3483747998972,15cyclictest3546502-21kworker/u8:1+flush-179:011:10:210
3483752998882,4cyclictest3486946-21kworker/u8:2+events_unbound07:40:281
3483747998881,5cyclictest3556048-21kworker/u8:2+events_unbound11:25:000
3483747998876,10cyclictest3544654-21gpgv10:20:010
3483752998679,5cyclictest3546502-21kworker/u8:1+events_unbound10:50:141
3483747998677,7cyclictest3586803-21latency_hist12:30:020
3483752998579,4cyclictest3511133-21kworker/u8:4+events_unbound08:45:151
3483752998578,6cyclictest3546502-21kworker/u8:1+events_unbound11:30:231
3483752998375,6cyclictest3494949-21kworker/u8:3+flush-179:008:10:201
3483752998274,5cyclictest3515975-21kworker/u8:1+flush-179:009:05:151
3483752998272,8cyclictest3564255-21latency_hist11:20:011
3483752998073,5cyclictest3486946-21kworker/u8:2+flush-179:007:30:241
3483752998070,8cyclictest3577918-21kworker/u8:0+flush-179:012:35:191
3483752997970,7cyclictest3490124-21kworker/u8:0+flush-179:007:50:201
3483747997972,5cyclictest3494949-21kworker/u8:3+events_unbound08:00:010
3483747997972,4cyclictest3586780-21kworker/u8:1+events_unbound12:35:240
3483747997873,4cyclictest3527122-21kworker/u8:0+flush-179:009:25:000
3483752997768,7cyclictest3575620-21kworker/u8:1+flush-179:012:15:001
3483752997766,5cyclictest3485272-21kworker/u8:3+events_unbound07:20:141
3483752997671,4cyclictest3562619-21kworker/u8:0+flush-179:011:35:241
3483752997669,5cyclictest3511133-21kworker/u8:4+events_unbound08:55:011
3483752997571,3cyclictest3530258-21kworker/u8:3+flush-179:009:45:251
3483752997562,5cyclictest3523873-21kworker/u8:2+events_unbound09:25:191
3483747997569,4cyclictest3575620-21kworker/u8:1+events_unbound11:55:010
348374799750,73cyclictest1-21systemd10:55:000
3483747997466,6cyclictest3572517-21kworker/u8:2+flush-179:012:00:010
3483752997362,9cyclictest3493311-21gzip07:38:301
3483752997265,5cyclictest3539814-21kworker/u8:4+flush-179:010:10:221
3483752997261,8cyclictest3501338-21latency_hist08:05:011
3483752997064,4cyclictest3539814-21kworker/u8:4+events_unbound10:15:141
3483747997059,9cyclictest3491906-21/usr/sbin/munin07:35:220
3483752996963,4cyclictest3523873-21kworker/u8:2+flush-179:009:50:141
3483752996943,18cyclictest3485272-21kworker/u8:3+events_unbound07:14:551
3483747996963,5cyclictest3486946-21kworker/u8:2+flush-179:007:34:300
3483747996963,4cyclictest3515975-21kworker/u8:1+flush-179:009:04:290
3483747996962,5cyclictest3523873-21kworker/u8:2+flush-179:009:15:170
3483747996862,5cyclictest3562619-21kworker/u8:0+flush-179:011:44:550
3483747996862,4cyclictest3536991-21kworker/u8:0+flush-179:009:55:200
3483752996762,4cyclictest3556048-21kworker/u8:2+events_unbound11:20:221
3483752996762,4cyclictest3539814-21kworker/u8:4+flush-179:010:05:251
3483752996762,4cyclictest3512793-21kworker/u8:0+events_unbound08:40:251
3483752996761,4cyclictest3536991-21kworker/u8:0+events_unbound10:00:141
3483752996761,4cyclictest3523873-21kworker/u8:2+events_unbound10:25:241
3483747996759,6cyclictest3575620-21kworker/u8:1+flush-179:012:10:010
3483752996646,4cyclictest3523873-21kworker/u8:2+events_unbound09:15:161
3483747996662,3cyclictest3523873-21kworker/u8:2+flush-179:009:44:300
3483747996660,4cyclictest3530258-21kworker/u8:3+flush-179:009:35:150
348374799660,64cyclictest1-21systemd11:31:290
3483752996560,3cyclictest3577918-21kworker/u8:0+flush-179:012:30:211
3483747996561,3cyclictest191rcuc/010:05:000
3483747996553,6cyclictest3577918-21kworker/u8:0+events_unbound12:30:140
4505291640,19ptp4l61790irq/134-eno1-TxRx-007:06:053
3483752996458,4cyclictest3572517-21kworker/u8:2+flush-179:012:00:141
3483747996458,4cyclictest3494949-21kworker/u8:3+flush-179:008:10:000
3483752996358,4cyclictest3566337-21kworker/u8:3+events_unbound11:40:201
3483752996356,5cyclictest3508006-21kworker/u8:2+flush-179:008:25:231
3483752996354,7cyclictest3545887-21kworker/u8:0+events_unbound10:35:231
3483752996352,9cyclictest3584682-21kworker/u8:3+flush-179:012:25:211
3483752996351,10cyclictest3561309-21apt-get11:10:091
3483752996351,10cyclictest3561309-21apt-get11:10:091
3483747996357,5cyclictest3562619-21kworker/u8:0+flush-179:011:36:300
3483752996256,5cyclictest3546502-21kworker/u8:1+flush-179:010:40:251
3483752996253,6cyclictest3486946-21kworker/u8:2+flush-179:007:25:201
348374799621,58cyclictest3519167-21munin-run09:00:010
3483752996153,6cyclictest3523873-21kworker/u8:2+events_unbound09:40:191
3483752996150,4cyclictest3584682-21kworker/u8:3+events_unbound12:20:231
3483747996156,3cyclictest3494949-21kworker/u8:3+flush-179:008:10:130
3483747995948,9cyclictest3540091-21apt-get10:05:120
3483752995853,4cyclictest3546502-21kworker/u8:1+flush-179:010:55:311
3483752995852,4cyclictest3508006-21kworker/u8:2+flush-179:008:30:111
3483752995852,4cyclictest3508006-21kworker/u8:2+flush-179:008:30:101
3483752995849,6cyclictest3566337-21kworker/u8:3+flush-179:011:45:161
3483747995851,5cyclictest3546502-21kworker/u8:1+events_unbound10:40:010
3483752995750,4cyclictest0-21swapper/110:30:121
3483747995750,4cyclictest3582567-21kworker/u8:2+events_unbound12:20:120
3483752995650,4cyclictest3511133-21kworker/u8:4+events_unbound08:39:591
3483752995551,3cyclictest3559204-21kworker/u8:3+flush-179:011:05:191
3483752995550,4cyclictest3575620-21kworker/u8:1+flush-179:011:55:141
3483752995550,4cyclictest3536991-21kworker/u8:0+flush-179:009:55:261
3483752995549,5cyclictest3530258-21kworker/u8:3+flush-179:009:35:001
3483752995545,8cyclictest3506380-21latency_hist08:20:001
3483747995546,4cyclictest3546502-21kworker/u8:1+events_unbound10:40:130
3483747995545,8cyclictest3523873-21kworker/u8:2+flush-179:009:50:010
3483747995449,4cyclictest3515975-21kworker/u8:1+flush-179:009:34:550
3483747995449,4cyclictest3515975-21kworker/u8:1+flush-179:009:14:300
3483747995448,5cyclictest3511133-21kworker/u8:4+flush-179:008:35:010
3483747995448,5cyclictest3511133-21kworker/u8:4+flush-179:008:35:010
3483747995447,5cyclictest3546502-21kworker/u8:1+flush-179:011:25:220
3483747995347,5cyclictest3546502-21kworker/u8:1+flush-179:011:19:550
3483752995246,5cyclictest3515975-21kworker/u8:1+events_unbound09:20:241
3483747995246,4cyclictest3490124-21kworker/u8:0+events_unbound07:50:240
3483752995144,6cyclictest3539814-21kworker/u8:4+events_unbound10:20:151
3483752995144,5cyclictest3562619-21kworker/u8:0+events_unbound11:25:151
3483752995140,9cyclictest3498686-21diskstats07:55:161
3483752995140,8cyclictest3520716-21latency_hist09:05:011
3483747995145,5cyclictest3545887-21kworker/u8:0+flush-179:010:33:290
3483747995145,4cyclictest3546502-21kworker/u8:1+events_unbound11:05:010
3483747995145,4cyclictest3512793-21kworker/u8:0+events_unbound08:50:130
3483747995143,7cyclictest3559448-21/usr/sbin/munin11:05:210
348374799510,48cyclictest3496590-21munin-run07:50:000
348374799510,48cyclictest3496590-21munin-run07:49:590
3483747995047,2cyclictest3545887-21kworker/u8:0+flush-179:010:26:300
3483747995042,6cyclictest3554087-21expr10:45:230
3483747995042,6cyclictest3554087-21expr10:45:230
3483752994945,3cyclictest3575620-21kworker/u8:1+flush-179:011:51:301
3483747994944,4cyclictest3577918-21kworker/u8:0+flush-179:012:01:290
3483747994944,4cyclictest3490124-21kworker/u8:0+flush-179:008:04:310
3483752994845,2cyclictest0-21swapper/112:15:581
3483747994838,8cyclictest3523873-21kworker/u8:2+flush-179:009:50:210
3483755994745,1cyclictest0-21swapper/309:38:593
3483752994742,4cyclictest3494949-21kworker/u8:3+flush-179:008:05:161
3483747994640,5cyclictest3575620-21kworker/u8:1+flush-179:012:10:270
3483752994541,2cyclictest3580440-21cron12:10:001
3483747994534,4cyclictest3512793-21kworker/u8:0+flush-179:008:45:200
3483747994533,10cyclictest3487161-21apt-get07:20:110
3483747994439,4cyclictest3566337-21kworker/u8:3+flush-179:011:46:300
3483747994439,4cyclictest3508006-21kworker/u8:2+events_unbound08:25:160
3483747994439,4cyclictest3485272-21kworker/u8:3+flush-179:007:28:300
3483747994434,5cyclictest3515975-21kworker/u8:1+events_unbound09:05:160
348375399430,40cyclictest0-21swapper/208:30:592
348375399430,40cyclictest0-21swapper/208:30:592
3483752994334,7cyclictest3531857-21latency_hist09:40:021
348374799431,40cyclictest3508010-21munin-run08:25:000
348375599400,38cyclictest3511157-21turbostat08:35:003
348375599400,38cyclictest3511157-21turbostat08:34:593
3483755993934,4cyclictest3577259-21turbostat12:00:003
348375399390,36cyclictest0-21swapper/212:13:592
348375399390,36cyclictest0-21swapper/209:34:562
348375399390,36cyclictest0-21swapper/207:33:002
3483747993934,4cyclictest3523873-21kworker/u8:2+flush-179:010:20:300
3483753993835,2cyclictest0-21swapper/210:56:592
3483753993835,2cyclictest0-21swapper/210:28:592
3483753993835,2cyclictest0-21swapper/209:19:592
3483753993835,2cyclictest0-21swapper/208:53:582
348375399380,36cyclictest0-21swapper/209:47:582
348375399380,36cyclictest0-21swapper/209:11:002
348375399380,36cyclictest0-21swapper/207:35:592
348375399380,35cyclictest0-21swapper/210:20:002
3483747993832,5cyclictest3494949-21kworker/u8:3+events_unbound08:20:010
3483753993734,2cyclictest0-21swapper/212:39:002
3483753993734,2cyclictest0-21swapper/211:20:592
348375399370,35cyclictest0-21swapper/207:56:582
3483755993634,1cyclictest0-21swapper/308:10:583
348375599360,35cyclictest0-21swapper/309:30:003
348375599360,1cyclictest0-21swapper/312:26:593
3483753993633,2cyclictest0-21swapper/212:22:592
3483753993633,2cyclictest0-21swapper/209:36:592
3483753993633,1cyclictest0-21swapper/208:14:592
3483753993632,2cyclictest0-21swapper/208:50:002
348375399360,1cyclictest0-21swapper/211:30:582
348375399360,1cyclictest0-21swapper/210:39:592
348375399360,1cyclictest0-21swapper/210:35:002
3483747993633,2cyclictest3508006-21kworker/u8:2+flush-179:008:41:290
348375599350,1cyclictest0-21swapper/311:36:593
348375599350,1cyclictest0-21swapper/308:54:563
348375399350,1cyclictest0-21swapper/209:24:592
3483747993530,4cyclictest3485272-21kworker/u8:3+flush-179:007:15:200
3483755993432,1cyclictest0-21swapper/311:11:593
3483755993432,1cyclictest0-21swapper/311:11:593
3483755993431,2cyclictest0-21swapper/309:18:583
3483755993428,5cyclictest3574074-21turbostat11:50:003
3483755993428,4cyclictest3498145-21turbostat07:54:593
3483753993432,1cyclictest0-21swapper/212:35:002
3483753993431,2cyclictest0-21swapper/211:53:592
3483753993431,2cyclictest0-21swapper/211:45:002
3483753993431,2cyclictest0-21swapper/207:49:562
3483753993431,2cyclictest0-21swapper/207:49:562
3483755993331,1cyclictest0-21swapper/310:20:593
3483755993331,1cyclictest0-21swapper/307:25:593
3483753993330,2cyclictest0-21swapper/207:21:582
3483753993330,2cyclictest0-21swapper/207:14:592
3483755993230,1cyclictest0-21swapper/311:54:593
3483755993230,1cyclictest0-21swapper/311:05:593
3483755993227,3cyclictest3502943-21turbostat08:10:003
3483753993229,2cyclictest0-21swapper/212:15:592
3483753993229,2cyclictest0-21swapper/211:01:002
3483753993229,2cyclictest0-21swapper/209:40:592
3483753993229,2cyclictest0-21swapper/208:58:002
348375399320,30cyclictest0-21swapper/209:06:582
348375399320,2cyclictest0-21swapper/208:35:592
4505291310,5ptp4l61790irq/134-eno1-TxRx-009:35:003
3483755993129,1cyclictest0-21swapper/307:24:593
3483755993128,2cyclictest0-21swapper/312:38:003
3483755993126,4cyclictest3585229-21turbostat12:25:003
3483753993129,1cyclictest0-21swapper/210:54:592
3483753993128,2cyclictest0-21swapper/211:16:592
3483753993128,2cyclictest0-21swapper/210:41:592
3483753993128,2cyclictest0-21swapper/210:24:592
348375399310,29cyclictest0-21swapper/209:00:112
3483747993125,4cyclictest722-21Xorg07:11:590
3483755993028,1cyclictest0-21swapper/308:26:593
3483755993028,1cyclictest0-21swapper/308:22:593
3483755993027,2cyclictest0-21swapper/307:16:593
348375599300,28cyclictest0-21swapper/310:10:153
348375599300,28cyclictest0-21swapper/310:05:143
348375599300,28cyclictest0-21swapper/309:10:143
348375599300,27cyclictest0-21swapper/310:35:123
3483753993027,2cyclictest0-21swapper/211:10:022
3483753993027,2cyclictest0-21swapper/211:10:022
3483753993027,2cyclictest0-21swapper/208:05:012
348375399300,28cyclictest0-21swapper/209:57:592
348375399300,28cyclictest0-21swapper/207:50:132
4505291290,7ptp4l61790irq/134-eno1-TxRx-009:45:003
4505291290,7ptp4l61790irq/134-eno1-TxRx-008:40:003
3483755992927,1cyclictest0-21swapper/311:29:543
3483755992927,1cyclictest0-21swapper/311:24:553
3483755992927,1cyclictest0-21swapper/307:46:003
3483755992927,1cyclictest0-21swapper/307:46:003
3483755992926,2cyclictest0-21swapper/312:15:143
3483753992926,2cyclictest0-21swapper/208:15:012
3483753992926,1cyclictest0-21swapper/211:30:002
348375399290,27cyclictest0-21swapper/211:45:132
348375399290,27cyclictest0-21swapper/209:25:122
4505291280,6ptp4l61790irq/134-eno1-TxRx-010:50:003
4505291280,6ptp4l61790irq/134-eno1-TxRx-010:50:003
3483755992826,1cyclictest0-21swapper/307:45:003
3483755992825,2cyclictest0-21swapper/310:50:593
348375599280,2cyclictest0-21swapper/312:10:193
348375599280,28cyclictest0-21swapper/310:30:123
348375599280,26cyclictest0-21swapper/308:15:023
3483753992825,2cyclictest0-21swapper/211:09:592
3483753992825,2cyclictest0-21swapper/207:26:592
348375399280,26cyclictest0-21swapper/211:55:142
348375399280,25cyclictest0-21swapper/208:23:592
4505291270,6ptp4l441ktimers/309:50:023
348375599270,26cyclictest0-21swapper/310:42:003
348375599270,26cyclictest0-21swapper/307:11:593
3483753992723,2cyclictest0-21swapper/212:28:582
348375399270,24cyclictest0-21swapper/208:06:592
4505291260,6ptp4l61790irq/134-eno1-TxRx-009:55:003
4505291260,6ptp4l441ktimers/312:09:563
4505291260,6ptp4l441ktimers/312:05:003
4505291260,6ptp4l441ktimers/311:34:563
4505291260,5ptp4l61790irq/134-eno1-TxRx-009:09:563
4505291260,4ptp4l441ktimers/310:30:003
4505291260,4ptp4l441ktimers/310:05:003
4505291260,4ptp4l441ktimers/309:05:003
3483755992624,1cyclictest0-21swapper/310:15:123
3483755992624,1cyclictest0-21swapper/309:56:593
348375599260,25cyclictest0-21swapper/310:59:593
348375599260,25cyclictest0-21swapper/308:49:553
3483753992622,2cyclictest0-21swapper/209:53:002
4505291250,6ptp4l61790irq/134-eno1-TxRx-011:04:553
4505291250,6ptp4l441ktimers/307:35:003
4505291250,5ptp4l61790irq/134-eno1-TxRx-011:45:003
4505291250,4ptp4l61790irq/134-eno1-TxRx-007:40:003
4505291250,4ptp4l441ktimers/312:35:003
4505291250,4ptp4l441ktimers/309:25:003
3483755992523,1cyclictest0-21swapper/308:56:593
3483755992522,2cyclictest0-21swapper/308:00:003
3483755992522,1cyclictest0-21swapper/308:44:543
3483753992522,2cyclictest0-21swapper/210:14:002
3483753992522,2cyclictest0-21swapper/210:02:582
3483747992522,2cyclictest3483775-21latency_hist07:10:020
4505291240,4ptp4l61790irq/134-eno1-TxRx-011:20:003
348375599240,1cyclictest0-21swapper/308:01:003
3483753992422,1cyclictest0-21swapper/210:09:592
3483753992321,1cyclictest0-21swapper/208:40:142
3483753992320,2cyclictest0-21swapper/212:05:102
3483752992317,3cyclictest0-21swapper/107:10:021
3483753992219,2cyclictest0-21swapper/207:42:582
3483753992219,2cyclictest0-21swapper/207:19:002
3483753991815,2cyclictest0-21swapper/210:47:592
3483753991815,2cyclictest0-21swapper/210:47:592
348375399129,2cyclictest0-21swapper/208:25:142
348375399118,2cyclictest0-21swapper/212:02:592
348375399118,2cyclictest0-21swapper/211:38:592
34837539930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional