You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-07 - 02:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3.osadl.org (updated Sun Jul 06, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
253806996637,27cyclictest263926-21kworker/1:108:00:141
25380699640,2cyclictest0-21swapper/111:00:381
253806996360,1cyclictest313156-21fschecks_time10:05:151
253806996260,1cyclictest0-21swapper/107:45:271
253806996259,2cyclictest0-21swapper/112:01:171
253806996259,2cyclictest0-21swapper/110:26:541
25380699620,61cyclictest356804-21cron12:20:001
253806996142,17cyclictest327357-21kworker/1:211:45:181
253806996044,15cyclictest263926-21kworker/1:107:55:001
253806996043,16cyclictest327357-21kworker/1:211:21:281
253806996043,16cyclictest327357-21kworker/1:210:55:011
253806995943,15cyclictest350089-21kworker/1:012:25:171
253806995943,15cyclictest327357-21kworker/1:211:15:201
253806995943,15cyclictest327357-21kworker/1:210:50:301
253806995943,14cyclictest263926-21kworker/1:108:49:561
253806995843,14cyclictest263926-21kworker/1:108:11:271
253806995843,14cyclictest257446-21kworker/1:007:30:191
253806995843,14cyclictest251229-21kworker/1:207:19:481
253806995843,14cyclictest251229-21kworker/1:207:10:001
253806995842,15cyclictest3922255-1kworker/1:1H11:51:521
253806995842,15cyclictest3922255-1kworker/1:1H11:05:041
253806995842,15cyclictest327357-21kworker/1:211:41:171
253806995842,15cyclictest327357-21kworker/1:211:28:591
253806995842,15cyclictest327357-21kworker/1:211:13:281
253806995842,15cyclictest304144-21kworker/1:209:47:531
253806995842,15cyclictest296630-21kworker/1:109:35:061
253806995842,15cyclictest296630-21kworker/1:109:19:541
253806995842,15cyclictest287180-21kworker/1:009:10:351
253806995842,15cyclictest263926-21kworker/1:108:40:221
253806995842,15cyclictest257446-21kworker/1:007:39:521
253806995840,14cyclictest263926-21kworker/1:107:50:011
253806995839,17cyclictest3922255-1kworker/1:1H08:25:161
253806995742,14cyclictest350089-21kworker/1:012:05:151
253806995742,14cyclictest306278-21kworker/1:010:40:171
253806995742,14cyclictest304144-21kworker/1:209:40:231
253806995742,14cyclictest287180-21kworker/1:009:00:151
253806995742,14cyclictest263926-21kworker/1:108:35:001
253806995742,14cyclictest263926-21kworker/1:108:15:011
253806995741,15cyclictest350089-21kworker/1:012:35:151
253806995741,15cyclictest348438-21kworker/1:111:55:401
253806995741,14cyclictest3922255-1kworker/1:1H10:45:141
253806995741,14cyclictest3922255-1kworker/1:1H08:05:161
253806995740,16cyclictest287180-21kworker/1:009:05:161
253806995641,14cyclictest350089-21kworker/1:012:30:151
253806995641,14cyclictest287180-21kworker/1:008:55:171
253806995640,15cyclictest350089-21kworker/1:012:13:531
253806995640,15cyclictest263926-21kworker/1:108:24:581
253806995639,16cyclictest327357-21kworker/1:211:30:021
253806995639,16cyclictest306278-21kworker/1:009:50:161
253806995638,17cyclictest306278-21kworker/1:010:30:161
253806995638,16cyclictest306278-21kworker/1:010:20:161
253806995637,18cyclictest263926-21kworker/1:108:50:181
253806995540,14cyclictest306278-21kworker/1:010:10:061
253806995539,15cyclictest296630-21kworker/1:109:30:121
253806995539,15cyclictest263926-21kworker/1:107:40:181
253806995539,15cyclictest263926-21kworker/1:107:40:181
253806995439,14cyclictest257446-21kworker/1:007:25:011
253806995438,15cyclictest350089-21kworker/1:012:20:161
253806995438,15cyclictest306278-21kworker/1:010:15:151
253806995438,15cyclictest306278-21kworker/1:010:00:011
253806995438,15cyclictest306278-21kworker/1:009:55:141
253811995347,4cyclictest270487-21kworker/2:108:10:002
253806995338,14cyclictest296630-21kworker/1:109:25:161
253806995338,14cyclictest296630-21kworker/1:109:20:011
253806995338,14cyclictest263926-21kworker/1:108:30:241
253806995337,15cyclictest251229-21kworker/1:207:20:011
253811995245,4cyclictest824389-1kworker/2:0H12:27:272
253806995249,2cyclictest0-21swapper/110:35:291
253806995234,17cyclictest327357-21kworker/1:211:35:161
253805995249,2cyclictest3103946-1kworker/0:1H11:13:270
253805995249,2cyclictest3103946-1kworker/0:1H10:34:530
253805995248,2cyclictest297643-21kworker/0:009:26:310
253805995248,2cyclictest289039-21kworker/0:109:05:110
253805995246,3cyclictest3103946-1kworker/0:1H08:35:140
253814995147,2cyclictest289042-21kworker/3:109:46:283
253814995146,3cyclictest1007335-1kworker/3:2H09:42:533
253814995145,4cyclictest1007335-1kworker/3:2H11:21:273
253811995144,4cyclictest824389-1kworker/2:0H12:21:152
253811995144,4cyclictest824389-1kworker/2:0H08:15:122
253811995142,5cyclictest824389-1kworker/2:0H10:35:132
253805995149,1cyclictest332225-21kworker/0:211:30:570
253805995147,2cyclictest337080-21kworker/0:111:25:050
253805995147,2cyclictest271500-21kworker/0:008:42:280
253814995047,2cyclictest265592-21kworker/3:207:49:523
253814995046,2cyclictest350052-21kworker/3:012:04:553
253814995046,2cyclictest322240-21kworker/3:210:45:113
253814995046,2cyclictest319003-21kworker/3:110:28:283
253814995045,4cyclictest1007335-1kworker/3:2H12:09:553
253814995045,3cyclictest315744-21kworker/3:210:24:283
253814995045,3cyclictest268827-21kworker/3:107:58:533
253814995045,3cyclictest265592-21kworker/3:207:44:513
253814995045,3cyclictest265592-21kworker/3:207:44:503
253811995042,4cyclictest824389-1kworker/2:0H08:45:152
253814994946,2cyclictest289042-21kworker/3:109:21:533
253814994944,3cyclictest332222-21kworker/3:111:10:333
253814994944,3cyclictest1007335-1kworker/3:2H10:59:553
253805994946,2cyclictest330586-21kworker/0:010:58:530
253805994945,2cyclictest289039-21kworker/0:109:11:590
253805994944,3cyclictest3103946-1kworker/0:1H08:08:280
253814994844,3cyclictest289042-21kworker/3:110:03:283
253814994844,3cyclictest282302-21kworker/3:208:55:483
253814994844,2cyclictest278563-21kworker/3:008:23:573
253814994843,4cyclictest320621-21kworker/3:010:35:063
253814994843,3cyclictest1007335-1kworker/3:2H07:30:013
253814994842,3cyclictest1007335-1kworker/3:2H10:15:153
253814994842,2cyclictest278563-21kworker/3:008:40:113
253814994839,6cyclictest1007335-1kworker/3:2H10:05:223
253811994840,6cyclictest824389-1kworker/2:0H11:00:422
253811994838,5cyclictest824389-1kworker/2:0H11:25:182
253805994845,2cyclictest3103946-1kworker/0:1H12:15:270
253805994845,2cyclictest306021-21kworker/0:209:42:280
253805994844,2cyclictest3103946-1kworker/0:1H11:20:120
253805994843,3cyclictest3103946-1kworker/0:1H12:10:000
253814994744,2cyclictest278563-21kworker/3:008:26:203
253814994744,2cyclictest1007335-1kworker/3:2H09:11:063
253814994743,3cyclictest358435-21kworker/3:012:26:263
253814994743,3cyclictest289042-21kworker/3:109:55:063
253814994743,2cyclictest1007335-1kworker/3:2H12:14:533
253814994742,3cyclictest1007335-1kworker/3:2H07:20:233
253805994744,2cyclictest236793-21kworker/0:107:22:540
253805994743,3cyclictest236793-21kworker/0:107:19:280
253805994743,2cyclictest356295-21kworker/0:112:20:240
253805994743,2cyclictest312513-21kworker/0:110:40:060
253805994743,2cyclictest300077-21kworker/0:109:32:040
253805994743,2cyclictest271500-21kworker/0:008:32:280
253814994643,2cyclictest272068-21kworker/3:208:05:293
253814994642,3cyclictest272068-21kworker/3:208:04:043
253814994642,2cyclictest350052-21kworker/3:012:18:533
253811994638,4cyclictest824389-1kworker/2:0H07:44:512
253811994638,4cyclictest824389-1kworker/2:0H07:44:502
253811994636,5cyclictest824389-1kworker/2:0H07:25:162
253805994643,2cyclictest332225-21kworker/0:212:00:210
253805994643,2cyclictest332225-21kworker/0:211:08:290
253805994643,2cyclictest307641-21kworker/0:010:01:280
253805994643,2cyclictest271500-21kworker/0:008:12:260
253805994643,2cyclictest265584-21kworker/0:007:45:050
253805994643,2cyclictest260944-21kworker/0:007:34:540
253805994642,3cyclictest3103946-1kworker/0:1H10:24:550
253805994642,3cyclictest3103946-1kworker/0:1H09:15:290
253805994642,3cyclictest271500-21kworker/0:008:25:210
253805994642,2cyclictest343571-21kworker/0:011:45:240
253805994642,2cyclictest306021-21kworker/0:209:46:090
253814994542,2cyclictest338718-21kworker/3:011:26:103
253814994542,2cyclictest325747-21kworker/3:011:04:293
253814994542,2cyclictest282302-21kworker/3:208:49:273
253814994542,2cyclictest278563-21kworker/3:008:30:063
253814994542,2cyclictest265592-21kworker/3:207:52:593
253814994542,2cyclictest252164-21kworker/3:107:16:593
253814994542,2cyclictest252164-21kworker/3:107:11:433
253814994541,3cyclictest1007335-1kworker/3:2H12:20:293
253814994540,3cyclictest1007335-1kworker/3:2H12:37:543
253811994538,5cyclictest824389-1kworker/2:0H10:10:182
253805994542,2cyclictest3103946-1kworker/0:1H10:36:280
253805994542,2cyclictest297643-21kworker/0:009:21:020
253805994542,2cyclictest287170-21kworker/0:208:50:340
253805994542,2cyclictest262563-21kworker/0:107:42:580
253805994542,2cyclictest262563-21kworker/0:107:42:580
253805994542,2cyclictest259074-21kworker/0:207:25:520
253814994441,2cyclictest337081-21kworker/3:211:36:273
253814994441,2cyclictest282302-21kworker/3:208:53:283
253814994441,2cyclictest1007335-1kworker/3:2H09:39:273
253814994440,3cyclictest1007335-1kworker/3:2H08:19:553
253814994439,3cyclictest1007335-1kworker/3:2H10:10:063
253811994438,4cyclictest294204-21kworker/2:009:50:162
253811994437,4cyclictest824389-1kworker/2:0H09:00:152
253805994441,2cyclictest330586-21kworker/0:011:00:370
253805994441,2cyclictest236793-21kworker/0:107:10:380
253805994440,3cyclictest3103946-1kworker/0:1H11:43:270
253805994439,3cyclictest3103946-1kworker/0:1H12:15:000
253814994340,2cyclictest361656-21kworker/3:212:32:293
253814994340,2cyclictest345197-21kworker/3:111:51:583
253814994340,2cyclictest299288-21kworker/3:209:25:053
253814994340,2cyclictest289042-21kworker/3:109:18:283
253814994339,2cyclictest322240-21kworker/3:210:43:333
253814994338,3cyclictest289042-21kworker/3:109:05:053
253811994338,3cyclictest824389-1kworker/2:0H08:31:282
253811994338,3cyclictest824389-1kworker/2:0H08:10:182
253805994340,2cyclictest356295-21kworker/0:112:36:580
253805994340,2cyclictest312513-21kworker/0:110:17:360
253805994340,2cyclictest307641-21kworker/0:009:52:530
253805994339,2cyclictest356295-21kworker/0:112:26:200
253805994339,2cyclictest287170-21kworker/0:208:55:060
253805994338,3cyclictest3103946-1kworker/0:1H09:56:270
253805994338,3cyclictest3103946-1kworker/0:1H08:21:270
253814994239,2cyclictest337081-21kworker/3:211:43:583
253814994239,2cyclictest319003-21kworker/3:110:30:553
253814994239,2cyclictest289042-21kworker/3:109:31:033
253814994238,3cyclictest345197-21kworker/3:111:48:273
253811994238,2cyclictest294204-21kworker/2:009:17:072
253811994237,3cyclictest824389-1kworker/2:0H11:16:102
253811994237,3cyclictest338776-21kworker/2:211:36:162
253805994238,2cyclictest312513-21kworker/0:110:45:050
253805994238,2cyclictest312513-21kworker/0:110:14:270
253805994238,2cyclictest267202-21kworker/0:208:02:290
253814994137,2cyclictest332222-21kworker/3:111:08:543
253814994136,3cyclictest350052-21kworker/3:011:58:283
253811994137,2cyclictest824389-1kworker/2:0H08:27:282
253811994137,2cyclictest824389-1kworker/2:0H07:55:052
253811994136,3cyclictest824389-1kworker/2:0H09:25:002
253811994133,5cyclictest824389-1kworker/2:0H12:30:142
253805994138,2cyclictest259074-21kworker/0:207:38:030
253814994037,2cyclictest272068-21kworker/3:208:12:593
253814994036,3cyclictest289042-21kworker/3:109:04:283
253814994035,3cyclictest1007335-1kworker/3:2H11:35:013
253811994037,2cyclictest824389-1kworker/2:0H07:21:282
253811994035,3cyclictest824389-1kworker/2:0H09:00:002
253805994037,2cyclictest312513-21kworker/0:110:08:590
253805994037,2cyclictest300077-21kworker/0:109:38:290
253805993936,2cyclictest3103946-1kworker/0:1H10:25:470
253814993834,3cyclictest259080-21kworker/3:007:31:283
253811993835,2cyclictest824389-1kworker/2:0H12:07:292
253811993833,3cyclictest824389-1kworker/2:0H08:54:012
253811993833,3cyclictest824389-1kworker/2:0H08:20:122
253805993835,2cyclictest332225-21kworker/0:211:37:290
253805993834,2cyclictest3103946-1kworker/0:1H10:50:300
253814993731,3cyclictest1007335-1kworker/3:2H07:35:143
253811993734,2cyclictest824389-1kworker/2:0H09:27:272
253811993732,3cyclictest824389-1kworker/2:0H08:38:282
253811993732,3cyclictest824389-1kworker/2:0H07:39:272
253811993731,3cyclictest824389-1kworker/2:0H11:20:362
253811993729,3cyclictest294204-21kworker/2:009:10:132
253805993734,2cyclictest3103946-1kworker/0:1H12:30:140
253805993734,2cyclictest3103946-1kworker/0:1H08:48:530
253805993734,2cyclictest271500-21kworker/0:008:16:130
253814993631,3cyclictest1007335-1kworker/3:2H08:35:153
253811993630,4cyclictest824389-1kworker/2:0H12:15:002
253811993630,4cyclictest294204-21kworker/2:010:05:102
253811993529,4cyclictest824389-1kworker/2:0H11:41:282
253811993527,4cyclictest824389-1kworker/2:0H07:34:502
253805993532,2cyclictest3103946-1kworker/0:1H11:18:530
253814993431,2cyclictest1007335-1kworker/3:2H10:50:293
253811993431,2cyclictest824389-1kworker/2:0H10:23:282
253811993430,3cyclictest824389-1kworker/2:0H10:30:002
253811993429,3cyclictest824389-1kworker/2:0H11:52:332
253811993429,3cyclictest824389-1kworker/2:0H07:54:282
253805993431,2cyclictest3103946-1kworker/0:1H11:50:570
253811993329,3cyclictest338776-21kworker/2:212:00:062
253805993330,2cyclictest3103946-1kworker/0:1H07:55:500
253811993229,2cyclictest824389-1kworker/2:0H07:10:332
253811993228,2cyclictest824389-1kworker/2:0H09:05:162
253811993226,3cyclictest824389-1kworker/2:0H07:45:142
253811993224,4cyclictest384844-21systemd-journal07:20:012
25381199321,2cyclictest0-21swapper/208:00:182
25381199320,3cyclictest341684-21grep11:30:192
25380699323,26cyclictest251rcuc/107:10:011
253814993128,2cyclictest332222-21kworker/3:111:15:343
253811993124,4cyclictest363311-21sadc12:40:002
253811993122,7cyclictest325751-21cron10:45:002
253814992925,3cyclictest1007335-1kworker/3:2H09:53:543
25381199298,9cyclictest330035-21head10:55:182
253811992825,2cyclictest294204-21kworker/2:010:16:282
253811992824,2cyclictest327405-21kworker/2:011:10:572
253811992821,4cyclictest304171-21cron09:40:002
25381199281,17cyclictest327632-21apt-get10:50:132
253811992723,3cyclictest824389-1kworker/2:0H09:45:162
253811992722,3cyclictest824389-1kworker/2:0H10:45:222
253811992722,2cyclictest311172-21apt-get10:00:122
25381199271,18cyclictest333105-21irqstats11:05:172
25381199270,7cyclictest0-21swapper/209:40:142
253805992724,2cyclictest3103946-1kworker/0:1H09:02:280
25381199252,11cyclictest345478-21apt-get11:45:142
25381199250,22cyclictest301154-21/usr/sbin/munin09:30:122
25381199250,14cyclictest0-21swapper/208:40:022
25380599251,10cyclictest348711-21apt-get11:55:130
25381199246,4cyclictest321585-21cat10:30:182
253811992317,4cyclictest824389-1kworker/2:0H12:20:002
25381199231,10cyclictest310972-21latency_hist10:00:012
25380599231,14cyclictest267485-21apt-get07:50:130
253811992219,2cyclictest824389-1kworker/2:0H11:55:052
25381199173,5cyclictest0-21swapper/207:10:012
253814991411,2cyclictest40-21ksoftirqd/307:10:013
25380599114,4cyclictest253817-21latency_hist07:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional