You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 17:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sun Nov 30, 2025 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31935879910091,7cyclictest3234568-21kworker/u8:1+flush-179:009:55:001
3193584999482,9cyclictest3198260-21latency_hist07:25:010
3193587998982,4cyclictest3277625-21kworker/u8:1+events_unbound11:30:251
3193584998879,7cyclictest3229555-21latency_hist09:00:010
3193584998681,4cyclictest3234568-21kworker/u8:1+flush-179:009:20:010
3193584998674,10cyclictest3277625-21kworker/u8:1+flush-179:012:40:000
3193587998476,6cyclictest3210494-21kworker/u8:0+flush-179:008:35:221
3193587998176,3cyclictest3223359-21kworker/u8:2+flush-179:008:45:251
3193587998175,4cyclictest3211573-21kworker/u8:3+flush-179:008:05:221
3193584998175,5cyclictest3193493-21kworker/u8:1+events_unbound08:00:240
3193584998175,4cyclictest3246030-21kworker/u8:0+flush-179:010:00:000
3193587998074,4cyclictest3193493-21kworker/u8:1+flush-179:008:00:001
3193587998073,5cyclictest3240789-21kworker/u8:0+flush-179:009:40:001
3193584998074,5cyclictest3277625-21kworker/u8:1+flush-179:012:10:250
3193587997975,3cyclictest3237615-21kworker/u8:3+flush-179:009:35:011
3193587997974,4cyclictest3277625-21kworker/u8:1+flush-179:012:35:251
3193587997972,5cyclictest3234568-21kworker/u8:1+events_unbound09:20:221
3193587997868,6cyclictest3193493-21kworker/u8:1+events_unbound07:35:011
3193584997872,4cyclictest3210494-21kworker/u8:0+flush-179:008:15:240
3193587997569,4cyclictest3193493-21kworker/u8:1+flush-179:007:30:001
3193587997468,4cyclictest3205206-21kworker/u8:0+flush-179:007:50:161
3193587997368,3cyclictest3280869-21kworker/u8:3+flush-179:011:50:141
3193587997367,5cyclictest3223359-21kworker/u8:2+flush-179:008:40:261
3193587997267,4cyclictest3234568-21kworker/u8:1+flush-179:009:40:231
3193587997267,4cyclictest3234568-21kworker/u8:1+flush-179:009:40:221
3193584997268,3cyclictest3289089-21kworker/u8:3+events_unbound12:25:010
3193584997265,6cyclictest3187442-21kworker/u8:0+events_unbound07:25:010
3193584997165,4cyclictest3289089-21kworker/u8:3+flush-179:012:15:160
3193584997162,6cyclictest3193493-21kworker/u8:1+flush-179:007:35:230
3193584997159,9cyclictest3214736-21kworker/u8:1+flush-179:008:40:160
3193584997065,4cyclictest3187442-21kworker/u8:0+events_unbound07:10:230
3193584997065,4cyclictest3187442-21kworker/u8:0+events_unbound07:10:230
3193587996960,7cyclictest3246030-21kworker/u8:0+flush-179:009:55:261
3193587996862,4cyclictest3284297-21kworker/u8:2+flush-179:012:05:001
3193587996862,4cyclictest3214736-21kworker/u8:1+events_unbound09:00:231
3193587996857,8cyclictest3256944-21latency_hist10:25:011
3193584996862,4cyclictest3261690-21kworker/u8:2+flush-179:010:45:230
3193584996861,6cyclictest3214736-21kworker/u8:1+flush-179:008:25:230
3193587996763,3cyclictest3269715-21kworker/u8:3+flush-179:011:10:271
3193587996761,4cyclictest3273299-21kworker/u8:0+events_unbound11:35:251
3193584996762,4cyclictest3249007-21kworker/u8:2+events_unbound10:15:230
3193584996759,7cyclictest3229808-21kworker/u8:2+flush-179:009:05:010
3193584996660,4cyclictest3234568-21kworker/u8:1+events_unbound10:05:170
3193587996561,3cyclictest3269715-21kworker/u8:3+flush-179:011:15:251
3193587996560,4cyclictest3277625-21kworker/u8:1+flush-179:012:15:001
3193587996559,4cyclictest3244131-21kworker/u8:2+flush-179:009:45:201
3193587996456,6cyclictest3214736-21kworker/u8:1+flush-179:008:55:211
3193587996456,5cyclictest3214736-21kworker/u8:1+flush-179:009:05:161
3193587996454,8cyclictest3263553-21/usr/sbin/munin10:45:251
3193584996460,3cyclictest3277625-21kworker/u8:1+flush-179:011:55:570
3193584996457,5cyclictest3289089-21kworker/u8:3+flush-179:012:25:010
3193587996357,4cyclictest3234568-21kworker/u8:1+flush-179:009:20:001
3193587996357,4cyclictest3210494-21kworker/u8:0+flush-179:008:10:151
3193587996353,7cyclictest271-21systemd-journal12:25:011
3193587996351,4cyclictest3273299-21kworker/u8:0+events_unbound11:25:241
3193587996345,17cyclictest3255376-21kworker/u8:0+flush-179:010:35:001
3193584996352,3cyclictest3234568-21kworker/u8:1+events_unbound10:10:180
3193587996258,3cyclictest3249007-21kworker/u8:2+flush-179:010:14:551
3193587996258,3cyclictest3214736-21kworker/u8:1+flush-179:008:25:151
3193587996255,5cyclictest3197219-21kworker/u8:2+flush-179:007:50:001
3193587996254,5cyclictest3268072-21latency_hist11:00:011
3193587996252,7cyclictest3195064-21latency_hist07:15:001
3193587996252,7cyclictest3195064-21latency_hist07:15:001
3193584996257,4cyclictest3234568-21kworker/u8:1+flush-179:009:35:160
3193584996255,6cyclictest3280869-21kworker/u8:3+flush-179:011:50:240
3193587996156,4cyclictest3214736-21kworker/u8:1+events_unbound08:30:251
3193584996156,4cyclictest3214736-21kworker/u8:1+flush-179:008:30:250
3193584996155,5cyclictest3277625-21kworker/u8:1+flush-179:011:40:000
3193584996154,5cyclictest3210494-21kworker/u8:0+flush-179:008:50:210
3193584996154,5cyclictest3210494-21kworker/u8:0+flush-179:008:50:210
3193589996057,2cyclictest0-21swapper/209:43:002
3193589996057,2cyclictest0-21swapper/209:42:592
3193589996056,2cyclictest0-21swapper/212:19:592
3193589996054,3cyclictest0-21swapper/210:11:002
3193587996055,4cyclictest3197219-21kworker/u8:2+events_unbound08:00:151
3193587996051,7cyclictest3229808-21kworker/u8:2+flush-179:009:10:221
3193584996055,4cyclictest3210494-21kworker/u8:0+flush-179:008:10:000
3193594995956,2cyclictest0-21swapper/309:42:593
3193594995956,2cyclictest0-21swapper/309:42:593
3193584995955,3cyclictest3284297-21kworker/u8:2+events_unbound12:00:250
3193584995953,4cyclictest3193493-21kworker/u8:1+flush-179:007:45:180
3193584995952,5cyclictest3261690-21kworker/u8:2+events_unbound11:00:140
3193587995851,4cyclictest3237615-21kworker/u8:3+flush-179:009:25:211
3193584995852,4cyclictest3210494-21kworker/u8:0+events_unbound08:10:220
3193584995849,6cyclictest3193493-21kworker/u8:1+flush-179:007:50:140
3193594995755,1cyclictest0-21swapper/312:19:593
3193587995752,4cyclictest3277625-21kworker/u8:1+flush-179:011:40:231
3193587995748,7cyclictest3266484-21latency_hist10:55:021
3193584995751,5cyclictest3234568-21kworker/u8:1+events_unbound10:55:210
3193584995749,6cyclictest3210494-21kworker/u8:0+flush-179:008:20:160
3193584995749,6cyclictest3210494-21kworker/u8:0+flush-179:008:20:150
3193584995747,8cyclictest3260375-21apt-get10:35:140
3193584995747,8cyclictest3260375-21apt-get10:35:130
3193594995653,1cyclictest0-21swapper/310:10:593
3193587995650,5cyclictest3289089-21kworker/u8:3+events_unbound12:20:251
3193587995647,7cyclictest271-21systemd-journal10:19:141
3193584995652,3cyclictest3261690-21kworker/u8:2+flush-179:010:50:230
3193584995645,9cyclictest3233001-21apt-get09:10:120
3193587995550,4cyclictest3269715-21kworker/u8:3+flush-179:011:24:551
3193587995549,4cyclictest3249007-21kworker/u8:2+flush-179:010:00:211
3193584995550,3cyclictest3197219-21kworker/u8:2+flush-179:007:55:260
3193584995549,4cyclictest3273299-21kworker/u8:0+events_unbound11:20:200
3193584995450,3cyclictest3277625-21kworker/u8:1+events_unbound11:30:150
3193584995450,3cyclictest3271503-21kworker/u8:2+flush-179:011:14:550
3193584995447,6cyclictest3273299-21kworker/u8:0+flush-179:011:40:150
3193584995347,5cyclictest3234568-21kworker/u8:1+flush-179:009:25:150
3193584995344,6cyclictest3237580-21latency_hist09:25:000
3193587995248,3cyclictest3269657-21kworker/u8:0+flush-179:011:09:551
3193587995247,4cyclictest3252196-21kworker/u8:3+events_unbound10:25:201
3193584995247,4cyclictest3244131-21kworker/u8:2+events_unbound09:45:250
3193584995247,4cyclictest3234568-21kworker/u8:1+flush-179:010:04:550
3193584995241,8cyclictest3276303-21apt-get11:25:120
3193587995145,5cyclictest3289089-21kworker/u8:3+flush-179:012:05:251
3193584995147,3cyclictest3214736-21kworker/u8:1+flush-179:009:05:190
3193584995141,8cyclictest271-21systemd-journal11:10:000
3193587995030,3cyclictest3277625-21kworker/u8:1+events_unbound11:50:011
3193584995045,4cyclictest3277625-21kworker/u8:1+events_unbound12:30:130
3193584995043,5cyclictest3277625-21kworker/u8:1+events_unbound11:45:200
3193584995040,7cyclictest3240755-21latency_hist09:35:010
3193587994941,6cyclictest3193493-21kworker/u8:1+events_unbound07:40:181
3193584994945,3cyclictest3234568-21kworker/u8:1+flush-179:010:30:280
3193584994944,3cyclictest3234568-21kworker/u8:1+flush-179:009:40:170
3193584994944,3cyclictest3234568-21kworker/u8:1+flush-179:009:40:170
3193584994938,9cyclictest3195323-21apt-get07:15:130
4670791480,14ptp4l61690irq/136-eno1-TxRx-007:10:013
3193584994842,4cyclictest3255376-21kworker/u8:0+flush-179:010:20:160
3193584994841,5cyclictest3263313-21latency_hist10:45:000
3193587994741,5cyclictest3261690-21kworker/u8:2+events_unbound10:40:191
3193584994641,4cyclictest3289089-21kworker/u8:3+events_unbound12:05:220
3193584994641,3cyclictest3225049-21apt-get08:45:130
319358999450,2cyclictest0-21swapper/209:11:592
3193587994539,5cyclictest3277625-21kworker/u8:1+flush-179:012:30:251
3193587994527,17cyclictest3214736-21kworker/u8:1+flush-179:008:20:001
3193587994439,4cyclictest3187442-21kworker/u8:0+events_unbound07:20:301
3193589994338,2cyclictest0-21swapper/210:54:552
319359499420,1cyclictest0-21swapper/309:12:003
3193587994227,6cyclictest3277625-21kworker/u8:1+events_unbound12:20:011
319359499410,39cyclictest3213139-21turbostat08:10:003
319359499410,39cyclictest3198273-21turbostat07:25:003
3193587994133,7cyclictest3187442-21kworker/u8:0+flush-179:007:20:001
3193584994136,4cyclictest3273299-21kworker/u8:0+events_unbound11:15:230
3193594994037,2cyclictest0-21swapper/310:54:553
319359499400,38cyclictest0-21swapper/312:34:553
319359499400,38cyclictest0-21swapper/311:07:593
319358999400,31cyclictest0-21swapper/211:07:592
3193594993936,2cyclictest0-21swapper/309:53:003
3193589993936,2cyclictest0-21swapper/209:53:002
3193589993935,2cyclictest0-21swapper/207:51:582
319358999390,36cyclictest0-21swapper/209:47:582
3193587993933,5cyclictest3261690-21kworker/u8:2+flush-179:011:00:161
3193584993933,5cyclictest3210494-21kworker/u8:0+flush-179:008:35:230
3193594993835,1cyclictest0-21swapper/310:37:003
3193594993835,1cyclictest0-21swapper/310:36:593
3193594993832,5cyclictest3205186-21turbostat07:45:003
3193594993832,4cyclictest3245547-21turbostat09:50:003
3193589993835,2cyclictest0-21swapper/210:36:592
3193589993835,2cyclictest0-21swapper/210:36:592
3193589993835,1cyclictest0-21swapper/211:25:002
319358999380,36cyclictest0-21swapper/212:34:552
319358999380,36cyclictest0-21swapper/207:25:592
319358999380,2cyclictest0-21swapper/211:13:592
3193584993830,6cyclictest3200622-21apt-get07:30:120
202199380,3rtkit-daemon2020-21rtkit-daemon11:57:471
3193594993733,3cyclictest0-21swapper/307:51:593
3193594993732,3cyclictest0-21swapper/308:15:593
3193594993729,6cyclictest3232754-21turbostat09:10:003
319359499370,36cyclictest0-21swapper/309:59:583
319359499370,1cyclictest0-21swapper/307:39:593
3193589993734,2cyclictest0-21swapper/212:37:592
3193589993733,2cyclictest0-21swapper/208:35:592
3193589993733,2cyclictest0-21swapper/208:15:592
3193589993714,1cyclictest0-21swapper/208:48:592
3193594993634,1cyclictest0-21swapper/311:21:593
3193594993633,2cyclictest0-21swapper/312:38:003
3193589993634,1cyclictest0-21swapper/207:25:002
3193589993632,2cyclictest0-21swapper/211:41:592
3193589993632,2cyclictest0-21swapper/209:39:562
319358999360,2cyclictest0-21swapper/209:59:592
319358999360,1cyclictest0-21swapper/211:40:002
319358999360,1cyclictest0-21swapper/208:09:552
3193584993631,4cyclictest3193493-21kworker/u8:1+flush-179:007:40:210
3193594993531,2cyclictest0-21swapper/311:17:003
3193589993531,2cyclictest0-21swapper/211:16:592
3193587993529,5cyclictest3234568-21kworker/u8:1+flush-179:010:35:281
3193587993529,5cyclictest3234568-21kworker/u8:1+flush-179:010:35:271
3193587993527,7cyclictest3216955-21fschecks_time08:20:171
3193587993527,7cyclictest3216955-21fschecks_time08:20:171
3193594993431,2cyclictest0-21swapper/311:41:593
3193594993431,2cyclictest0-21swapper/308:35:583
3193594993429,4cyclictest3217914-21turbostat08:25:013
3193594993429,4cyclictest3217914-21turbostat08:25:003
3193589993432,1cyclictest0-21swapper/207:45:002
3193589993430,2cyclictest0-21swapper/210:20:592
319358499340,31cyclictest3245544-21apt-get09:50:020
3193594993331,1cyclictest0-21swapper/309:39:563
3193594993326,5cyclictest3229584-21turbostat09:00:003
3193589993330,2cyclictest0-21swapper/208:25:012
3193589993330,2cyclictest0-21swapper/208:25:002
3193587993328,4cyclictest3197219-21kworker/u8:2+events_unbound07:35:131
319358799330,2cyclictest0-21swapper/110:05:151
3193594993229,2cyclictest0-21swapper/310:20:593
3193589993228,2cyclictest0-21swapper/212:14:552
3193594993128,2cyclictest0-21swapper/312:14:543
3193587993120,8cyclictest3227964-21latency_hist08:55:011
3193587993120,8cyclictest3227964-21latency_hist08:55:011
319359499300,28cyclictest0-21swapper/311:45:133
319359499300,28cyclictest0-21swapper/308:00:143
319359499300,27cyclictest0-21swapper/308:40:153
3193589993026,2cyclictest0-21swapper/207:16:592
319358999300,28cyclictest0-21swapper/207:32:592
319358999300,1cyclictest0-21swapper/207:39:592
3193594992926,2cyclictest0-21swapper/310:55:143
3193594992926,2cyclictest0-21swapper/307:28:583
319359499290,27cyclictest0-21swapper/309:00:143
319359499290,27cyclictest0-21swapper/307:15:153
3193589992927,1cyclictest0-21swapper/208:59:592
3193589992926,2cyclictest0-21swapper/210:55:132
3193589992926,2cyclictest0-21swapper/210:30:162
3193589992926,2cyclictest0-21swapper/208:30:142
3193589992925,2cyclictest0-21swapper/210:40:132
3193589992925,2cyclictest0-21swapper/209:29:542
3193589992925,2cyclictest0-21swapper/207:46:592
319358999290,2cyclictest0-21swapper/208:25:032
319358999290,27cyclictest0-21swapper/212:05:152
319358999290,27cyclictest0-21swapper/209:00:142
319358999290,27cyclictest0-21swapper/208:40:142
319358999290,27cyclictest0-21swapper/208:00:142
319358999290,21cyclictest0-21swapper/211:45:132
3193594992826,1cyclictest0-21swapper/308:45:033
3193594992825,2cyclictest0-21swapper/307:46:583
3193594992825,2cyclictest0-21swapper/307:10:133
3193594992825,2cyclictest0-21swapper/307:10:133
319359499280,27cyclictest0-21swapper/310:40:133
319359499280,27cyclictest0-21swapper/310:30:153
319359499280,27cyclictest0-21swapper/310:09:003
319359499280,27cyclictest0-21swapper/308:25:033
319359499280,26cyclictest0-21swapper/311:35:143
3193589992825,2cyclictest0-21swapper/209:08:002
3193589992824,1cyclictest0-21swapper/210:16:582
319358999280,27cyclictest0-21swapper/210:08:592
3193584992824,3cyclictest3234568-21kworker/u8:1+events_unbound10:25:020
4670791270,8ptp4l61690irq/136-eno1-TxRx-010:05:003
4670791270,6ptp4l61690irq/136-eno1-TxRx-008:15:013
4670791270,5ptp4l61690irq/136-eno1-TxRx-012:05:003
3193594992725,1cyclictest0-21swapper/312:05:153
3193594992725,1cyclictest0-21swapper/311:10:143
3193594992725,1cyclictest0-21swapper/309:16:593
3193594992725,1cyclictest0-21swapper/308:30:143
3193594992725,1cyclictest0-21swapper/307:30:143
3193594992724,2cyclictest0-21swapper/309:29:543
3193584992721,4cyclictest3193452-21latency_hist07:10:020
4670791260,6ptp4l441ktimers/309:35:013
4670791260,6ptp4l441ktimers/309:24:593
4670791260,5ptp4l61690irq/136-eno1-TxRx-012:00:003
4670791260,5ptp4l441ktimers/311:34:563
3193594992623,2cyclictest0-21swapper/310:16:583
319359499260,24cyclictest0-21swapper/310:27:593
319359499260,1cyclictest0-21swapper/312:29:593
3193589992623,2cyclictest0-21swapper/211:59:552
3193589992622,2cyclictest0-21swapper/207:59:002
319358999260,2cyclictest0-21swapper/210:27:582
3193587992614,11cyclictest3193452-21latency_hist07:10:011
4670791250,6ptp4l441ktimers/310:50:003
4670791250,5ptp4l61690irq/136-eno1-TxRx-011:55:013
4670791250,5ptp4l61690irq/136-eno1-TxRx-008:54:593
4670791250,5ptp4l61690irq/136-eno1-TxRx-008:54:593
4670791250,5ptp4l441ktimers/311:30:013
4670791250,5ptp4l441ktimers/307:59:563
4670791250,4ptp4l441ktimers/312:25:003
4670791250,4ptp4l441ktimers/311:05:003
3193589992522,2cyclictest0-21swapper/211:00:592
3193589992522,2cyclictest0-21swapper/209:17:582
3193589992520,2cyclictest0-21swapper/212:21:592
319358999240,1cyclictest0-21swapper/212:29:592
3193589992320,2cyclictest0-21swapper/207:10:192
3193589992320,2cyclictest0-21swapper/207:10:192
3193589992220,1cyclictest0-21swapper/210:50:002
3193589992017,2cyclictest0-21swapper/210:00:152
3193589991815,2cyclictest0-21swapper/212:00:592
3193589991815,2cyclictest0-21swapper/207:10:022
3193589991613,2cyclictest0-21swapper/209:24:002
3193589991512,2cyclictest0-21swapper/209:31:582
3193589991411,1cyclictest0-21swapper/211:30:002
319358999140,12cyclictest0-21swapper/211:54:562
319358999107,2cyclictest0-21swapper/208:54:002
319358999107,2cyclictest0-21swapper/208:54:002
319358999107,1cyclictest0-21swapper/208:14:592
31935899990,8cyclictest0-21swapper/211:31:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional