You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Mon Jun 16, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168521130,1sleep1221ktimersoftd/110:11:521
20325211089,15sleep20-21swapper/207:09:302
20030211088,16sleep00-21swapper/007:05:390
20084210888,15sleep30-21swapper/307:06:233
20174210788,14sleep10-21swapper/107:07:321
312112840,1sleep331212-21fschecks_count07:35:143
45832800,1sleep10-21swapper/110:15:161
45132770,1sleep0101ktimersoftd/011:20:290
255192770,0sleep10-21swapper/110:35:401
205142720,1sleep320507-21ssh12:10:183
231452690,0sleep10-21swapper/111:40:191
305142660,1sleep20-21swapper/211:47:122
307732260,0sleep10-21swapper/111:47:351
148642240,0sleep30-21swapper/311:30:463
260862220,0sleep20-21swapper/210:04:272
2039799221,19cyclictest0-21swapper/310:30:153
2039799221,19cyclictest0-21swapper/310:30:143
20394992118,2cyclictest5335-21kworker/u8:012:18:040
20396992017,2cyclictest24834-21kworker/u8:109:50:042
20394992017,2cyclictest13700-21kworker/u8:209:28:050
20396991916,2cyclictest29902-21kworker/u8:312:29:042
20395991916,2cyclictest0-21swapper/110:30:141
20395991916,2cyclictest0-21swapper/110:30:141
2039599190,2cyclictest0-21swapper/108:15:121
20394991916,2cyclictest12707-21kworker/u8:111:48:040
20396991815,2cyclictest5335-21kworker/u8:012:05:052
20396991815,2cyclictest24383-21kworker/u8:008:50:052
20395991815,2cyclictest9777-21kworker/u8:010:08:051
125382180,1sleep20-21swapper/211:29:412
20396991714,2cyclictest6487-21kworker/u8:207:15:042
20396991714,2cyclictest4137-21kworker/u8:110:18:052
20396991714,2cyclictest24383-21kworker/u8:009:17:052
20396991714,2cyclictest21559-21kworker/u8:011:40:062
2039699171,3cyclictest31891-21munin-node2
20394991714,2cyclictest16158-21kworker/u8:010:35:040
20394991714,2cyclictest12707-21kworker/u8:111:50:050
2039799161,13cyclictest0-21swapper/307:25:143
20396991613,2cyclictest7980-21kworker/u8:011:01:042
20396991613,2cyclictest25304-21kworker/u8:007:33:042
20396991613,2cyclictest25006-21kworker/u8:112:20:052
2039699160,14cyclictest58192sleep209:43:322
2039699160,14cyclictest0-21swapper/212:30:412
20395991611,4cyclictest24834-21kworker/u8:109:54:041
20394991613,2cyclictest9777-21kworker/u8:010:07:040
20394991613,2cyclictest7980-21kworker/u8:010:55:040
2039499160,14cyclictest23924-21ntpq11:40:230
20397991510,3cyclictest32260-21ssh11:49:513
2039799150,3cyclictest0-21swapper/308:15:143
20396991512,2cyclictest9777-21kworker/u8:010:12:012
20396991512,2cyclictest4137-21kworker/u8:110:25:042
20396991512,2cyclictest30696-21kworker/u8:111:23:042
20396991512,2cyclictest0-21swapper/212:36:502
20396991511,2cyclictest18663-21kworker/u8:211:05:042
2039699150,13cyclictest0-21swapper/211:34:212
2039699150,13cyclictest0-21swapper/209:10:152
20394991512,2cyclictest9777-21kworker/u8:010:12:050
20394991512,2cyclictest24383-21kworker/u8:008:47:040
20394991512,2cyclictest24123-21kworker/u8:011:16:050
20394991512,2cyclictest12707-21kworker/u8:111:57:050
20394991512,2cyclictest0-21swapper/010:27:330
20394991512,2cyclictest0-21swapper/009:59:340
20394991512,2cyclictest0-21swapper/009:35:230
20394991512,2cyclictest0-21swapper/009:18:230
2039499150,13cyclictest0-21swapper/010:50:160
2039499150,13cyclictest0-21swapper/010:00:210
86572140,3sleep28668-21ssh10:51:512
26602141,3sleep22838-21perl07:45:152
2039799141,11cyclictest0-21swapper/307:50:143
20397991410,2cyclictest0-21swapper/309:45:123
2039799140,12cyclictest0-21swapper/311:44:523
20396991411,2cyclictest6487-21kworker/u8:207:38:042
20396991411,2cyclictest28080-21kworker/u8:210:56:012
20396991411,2cyclictest24383-21kworker/u8:009:08:052
20396991411,2cyclictest24383-21kworker/u8:008:48:042
20396991411,2cyclictest0-21swapper/211:55:012
20396991411,2cyclictest0-21swapper/211:15:122
20396991411,2cyclictest0-21swapper/209:00:222
2039699140,2cyclictest0-21swapper/212:15:212
2039699140,2cyclictest0-21swapper/211:35:012
2039699140,2cyclictest0-21swapper/211:14:312
2039699140,2cyclictest0-21swapper/210:47:512
2039699140,2cyclictest0-21swapper/210:09:022
2039699140,13cyclictest0-21swapper/210:31:422
2039699140,13cyclictest0-21swapper/210:31:412
2039699140,13cyclictest0-21swapper/209:57:112
2039699140,13cyclictest0-21swapper/209:29:512
2039699140,13cyclictest0-21swapper/209:21:112
2039699140,12cyclictest0-21swapper/212:02:412
2039599149,3cyclictest32316-21sed08:55:281
2039599142,6cyclictest0-21swapper/109:45:001
20395991410,2cyclictest0-21swapper/110:55:001
2039599140,12cyclictest30855-21sh09:35:271
2039599140,12cyclictest0-21swapper/111:36:311
20394991411,2cyclictest6487-21kworker/u8:207:27:040
20394991411,2cyclictest15209-21kworker/u8:212:11:040
20394991411,2cyclictest0-21swapper/011:06:170
20394991411,2cyclictest0-21swapper/010:43:360
20394991411,2cyclictest0-21swapper/009:52:400
20394991411,2cyclictest0-21swapper/009:05:210
2039499140,3cyclictest0-21swapper/010:30:350
2039499140,3cyclictest0-21swapper/010:30:350
2039499140,2cyclictest0-21swapper/012:30:580
2039499140,13cyclictest4355-21diskmemload11:28:030
2039499140,13cyclictest0-21swapper/010:46:350
2039499140,13cyclictest0-21swapper/009:21:110
2039499140,12cyclictest0-21swapper/011:36:370
2039799132,6cyclictest14490-21gpgv09:20:013
2039799132,6cyclictest0-21swapper/309:55:003
20397991310,2cyclictest0-21swapper/311:50:233
20397991310,2cyclictest0-21swapper/311:17:113
20397991310,2cyclictest0-21swapper/309:38:323
20397991310,2cyclictest0-21swapper/307:10:123
20397991310,2cyclictest0-21swapper/307:10:123
2039799130,2cyclictest0-21swapper/308:25:133
2039799130,11cyclictest0-21swapper/312:38:243
2039799130,11cyclictest0-21swapper/312:16:423
20396991310,2cyclictest8218-21kworker/u8:208:13:042
20396991310,2cyclictest6487-21kworker/u8:207:40:042
20396991310,2cyclictest3767-21kworker/u8:209:45:042
20396991310,2cyclictest19651-21kworker/u8:009:30:012
20396991310,2cyclictest14329-21kworker/u8:008:27:042
20396991310,2cyclictest0-21swapper/209:38:512
2039699130,2cyclictest0-21swapper/210:35:042
2039599132,6cyclictest0-21swapper/112:10:011
2039599132,5cyclictest0-21swapper/109:35:011
2039599130,11cyclictest0-21swapper/110:40:151
2039599130,11cyclictest0-21swapper/108:35:151
20394991310,2cyclictest30957-21kworker/u8:108:57:050
20394991310,2cyclictest0-21swapper/012:28:170
20394991310,2cyclictest0-21swapper/012:05:290
20394991310,2cyclictest0-21swapper/011:33:330
20394991310,2cyclictest0-21swapper/011:10:220
20394991310,2cyclictest0-21swapper/010:20:090
20394991310,2cyclictest0-21swapper/009:42:260
20394991310,2cyclictest0-21swapper/009:10:190
20394991310,2cyclictest0-21swapper/008:35:050
20394991310,2cyclictest0-21swapper/007:55:110
20394991310,2cyclictest0-21swapper/007:51:530
20394991310,2cyclictest0-21swapper/007:45:270
2039499130,2cyclictest12452-21ssh12:01:310
2039499130,2cyclictest0-21swapper/007:35:010
2039499130,1cyclictest0-21swapper/012:20:260
2039499130,12cyclictest0-21swapper/012:35:190
2039499130,12cyclictest0-21swapper/011:01:370
2039499130,12cyclictest0-21swapper/009:46:400
2039499130,12cyclictest0-21swapper/009:30:060
2039499130,11cyclictest0-21swapper/010:15:190
2039499130,11cyclictest0-21swapper/008:50:040
140392130,2sleep10-21swapper/110:57:341
2039799129,2cyclictest0-21swapper/310:57:213
2039799129,2cyclictest0-21swapper/310:12:423
2039799129,2cyclictest0-21swapper/310:02:023
2039799122,7cyclictest0-21swapper/310:38:583
2039799122,6cyclictest0-21swapper/312:35:003
2039799122,5cyclictest0-21swapper/310:30:003
2039799120,2cyclictest0-21swapper/312:03:213
2039799120,2cyclictest0-21swapper/310:41:123
2039799120,2cyclictest0-21swapper/309:31:423
2039799120,11cyclictest0-21swapper/312:09:513
2039799120,11cyclictest0-21swapper/310:51:323
2039799120,11cyclictest0-21swapper/310:18:123
2039799120,11cyclictest0-21swapper/310:08:023
2039799120,11cyclictest0-21swapper/309:20:523
2039799120,11cyclictest0-21swapper/309:12:523
2039799120,11cyclictest0-21swapper/307:40:133
2039799120,10cyclictest0-21swapper/311:37:513
2039699129,2cyclictest8218-21kworker/u8:208:21:042
2039699129,2cyclictest8218-21kworker/u8:208:16:022
2039699129,2cyclictest2444-21kworker/u8:107:53:052
2039699120,2cyclictest0-21swapper/211:53:512
2039699120,2cyclictest0-21swapper/210:41:222
2039699120,11cyclictest0-21swapper/212:10:122
2039599129,2cyclictest0-21swapper/111:06:091
2039599128,2cyclictest0-21swapper/109:20:061
2039599125,2cyclictest23-21ksoftirqd/111:02:011
2039599122,7cyclictest0-21swapper/111:20:581
2039599122,7cyclictest0-21swapper/108:43:591
2039599121,9cyclictest29447-21apt-get12:20:051
2039599121,9cyclictest10203-21munin-node1
2039599121,9cyclictest0-21swapper/108:00:261
2039599121,9cyclictest0-21swapper/107:30:141
2039599121,3cyclictest26653-21munin-node1
2039599121,3cyclictest22630-21munin-run08:34:591
2039599120,10cyclictest0-21swapper/112:31:111
2039499129,2cyclictest30957-21kworker/u8:109:04:040
2039499129,2cyclictest25304-21kworker/u8:007:31:040
2039499120,1cyclictest0-21swapper/007:15:280
2039499120,11cyclictest0-21swapper/008:20:200
2039499120,11cyclictest0-21swapper/008:05:270
2039499120,11cyclictest0-21swapper/007:10:240
2039499120,11cyclictest0-21swapper/007:10:240
80902110,2sleep14355-21diskmemload11:25:041
59992110,2sleep10-21swapper/109:43:471
2039799118,2cyclictest0-21swapper/312:23:403
2039799118,2cyclictest0-21swapper/311:05:253
2039799118,2cyclictest0-21swapper/311:01:123
2039799118,2cyclictest0-21swapper/309:40:433
2039799116,3cyclictest17245-21tr09:55:143
2039799116,3cyclictest10995-21tr08:05:143
2039799113,2cyclictest0-21swapper/310:24:153
2039799112,6cyclictest9432-21ssh11:25:283
2039799112,6cyclictest0-21swapper/307:24:583
2039799111,8cyclictest1028-21seq10:45:053
2039799111,7cyclictest0-21swapper/308:45:583
2039699118,2cyclictest8218-21kworker/u8:208:09:042
2039699118,2cyclictest24383-21kworker/u8:008:43:052
2039599118,2cyclictest0-21swapper/112:35:001
2039599118,2cyclictest0-21swapper/110:26:011
2039599118,2cyclictest0-21swapper/108:10:231
2039599118,2cyclictest0-21swapper/107:50:151
2039599117,2cyclictest18512-21users09:55:281
2039599116,3cyclictest6112-21ssh09:10:281
2039599116,3cyclictest19897-21cpu09:25:151
2039599113,7cyclictest0-21swapper/107:35:171
2039599113,6cyclictest0-21swapper/111:18:471
2039599112,6cyclictest0-21swapper/107:43:031
2039599111,8cyclictest526-21munin-node1
2039599111,8cyclictest0-21swapper/109:05:221
2039599111,7cyclictest0-21swapper/108:27:581
2039599111,3cyclictest24418-21munin-run07:20:011
2039599111,3cyclictest0-21swapper/112:25:161
2039599111,3cyclictest0-21swapper/111:55:221
2039599111,2cyclictest0-21swapper/108:50:051
2039599110,9cyclictest26915-21munin-node1
2039599110,9cyclictest13106-21apt-get11:30:141
2039599110,9cyclictest0-21swapper/112:15:191
2039599110,8cyclictest0-21swapper/107:45:071
2039599110,10cyclictest0-21swapper/112:13:551
2039599110,10cyclictest0-21swapper/110:46:051
2039599110,10cyclictest0-21swapper/110:20:311
2039499118,2cyclictest0-21swapper/007:40:210
2039499110,1cyclictest0-21swapper/008:30:300
2039499110,10cyclictest0-21swapper/008:17:000
110812110,2sleep20-21swapper/210:20:592
2039799107,2cyclictest0-21swapper/308:50:303
2039799107,2cyclictest0-21swapper/308:35:293
2039799105,3cyclictest4560-21ntpd07:55:153
2039799105,3cyclictest3161-21grep07:45:183
2039799105,3cyclictest19739-21aten2.4-expect09:25:143
2039799102,6cyclictest4355-21diskmemload12:30:003
2039799100,8cyclictest0-21swapper/311:20:153
2039799100,7cyclictest0-21swapper/308:04:593
2039799100,5cyclictest0-21swapper/309:05:003
2039799100,5cyclictest0-21swapper/308:35:003
2039699107,2cyclictest2444-21kworker/u8:107:56:042
2039599105,3cyclictest9853-21sh09:15:151
2039599105,3cyclictest32611-21sh11:50:051
2039599102,6cyclictest0-21swapper/111:13:451
2039599101,2cyclictest21917-21apt-get10:00:141
2039599100,8cyclictest16814-21munin-node1
2039599100,3cyclictest0-21swapper/107:20:141
2039499100,9cyclictest0-21swapper/008:05:000
2039499100,8cyclictest5010-21snmpd08:28:110
2039499100,3cyclictest25395-21ls07:20:180
117652100,2sleep10-21swapper/108:05:261
5187290,1sleep30-21swapper/311:55:123
203979995,2cyclictest4355-21diskmemload11:10:023
203979994,3cyclictest32754-21latency_hist09:00:003
203979994,3cyclictest25224-21df_abs08:40:153
203979994,3cyclictest17480-21interrupts08:20:183
203979994,3cyclictest0-21swapper/308:10:283
203979994,3cyclictest0-21swapper/307:30:143
203979991,6cyclictest0-21swapper/307:15:253
203969995,2cyclictest21608-21pzem_current07:10:182
203969995,2cyclictest21608-21pzem_current07:10:182
203969991,6cyclictest8853-21tail08:00:122
203959994,3cyclictest6826-21sh07:55:121
203959994,3cyclictest20377-21latency_hist07:10:001
203959994,3cyclictest20377-21latency_hist07:10:001
203949990,8cyclictest0-21swapper/008:10:360
203979983,3cyclictest2970-21cut09:05:183
203969984,3cyclictest4360-21lldpd07:20:162
203969982,5cyclictest278-21jbd2/sda1-808:32:312
203969980,7cyclictest0-21swapper/208:35:022
203949985,2cyclictest0-21swapper/008:40:060
203969972,3cyclictest266422chrt07:25:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional