You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 01:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 14, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17714211997,16sleep20-21swapper/207:09:472
17601211395,12sleep10-21swapper/107:08:211
17570211295,11sleep30-21swapper/307:07:593
17669211188,18sleep00-21swapper/007:09:130
311542740,1sleep30-21swapper/311:20:573
148102730,0sleep0101ktimersoftd/012:11:440
283522650,1sleep20-21swapper/210:45:122
64772640,1sleep20-21swapper/211:30:052
1776299230,11cyclictest0-21swapper/212:08:022
17761992217,4cyclictest21758-21kworker/u8:110:06:041
200132210,1sleep00-21swapper/012:17:260
1776399211,18cyclictest0-21swapper/308:00:143
1776399210,18cyclictest0-21swapper/310:10:133
1776199212,18cyclictest0-21swapper/109:32:041
17761992118,2cyclictest32370-21kworker/u8:109:12:051
17761992118,2cyclictest30070-21kworker/u8:009:16:041
17761992118,2cyclictest26259-21kworker/u8:109:36:061
17761992118,2cyclictest2615-21kworker/u8:212:29:051
17761992118,2cyclictest2615-21kworker/u8:212:18:051
17761992118,2cyclictest21758-21kworker/u8:110:10:061
17761992118,2cyclictest20945-21kworker/u8:111:50:021
17761992118,2cyclictest20112-21kworker/u8:207:17:041
17761992118,2cyclictest18379-21kworker/u8:009:41:041
1776199210,16cyclictest0-21swapper/111:33:041
1776399201,3cyclictest0-21swapper/311:40:123
17761992017,2cyclictest9436-21kworker/u8:107:26:011
17761992017,2cyclictest8239-21kworker/u8:209:51:051
17761992017,2cyclictest31355-21kworker/u8:110:52:051
17761992017,2cyclictest27641-21kworker/u8:011:45:041
17761992017,2cyclictest27641-21kworker/u8:011:19:051
17761992017,2cyclictest18379-21kworker/u8:010:26:051
17761992017,2cyclictest17230-21kworker/u8:310:33:041
17761992017,2cyclictest12758-21kworker/u8:111:03:051
17761992017,2cyclictest12131-21kworker/u8:108:57:051
17761992017,2cyclictest12131-21kworker/u8:108:19:041
1776199201,2cyclictest0-21swapper/108:41:041
47102190,0sleep00-21swapper/009:12:110
1776399190,2cyclictest0-21swapper/311:35:123
17761991916,2cyclictest8215-21kworker/u8:008:33:051
17761991916,2cyclictest28880-21kworker/u8:011:55:041
17761991916,2cyclictest27641-21kworker/u8:011:28:051
17761991916,2cyclictest2615-21kworker/u8:212:09:051
17761991916,2cyclictest18379-21kworker/u8:010:38:051
17761991916,2cyclictest18379-21kworker/u8:010:23:041
17761991916,2cyclictest18379-21kworker/u8:009:28:051
1776199190,18cyclictest0-21swapper/110:19:051
1776199190,17cyclictest0-21swapper/112:10:111
17760991915,2cyclictest0-21swapper/007:10:110
314172180,1sleep00-21swapper/007:40:240
1776399180,2cyclictest0-21swapper/310:25:143
17761991815,2cyclictest9436-21kworker/u8:107:32:051
17761991815,2cyclictest28270-21kworker/u8:207:41:051
17761991815,2cyclictest27641-21kworker/u8:011:24:041
17761991815,2cyclictest23620-21kworker/u8:112:36:041
17761991815,2cyclictest23620-21kworker/u8:112:23:041
17761991815,2cyclictest21910-21kworker/u8:209:03:051
17761991815,2cyclictest21910-21kworker/u8:208:46:051
17761991815,2cyclictest20112-21kworker/u8:207:23:051
17761991815,2cyclictest1850-21kworker/u8:108:05:051
103122180,0sleep00-21swapper/009:51:190
17761991714,2cyclictest5272-21kworker/u8:209:24:051
17761991714,2cyclictest28270-21kworker/u8:207:36:051
17761991714,2cyclictest2615-21kworker/u8:212:03:051
17761991714,2cyclictest1850-21kworker/u8:107:51:041
17761991714,2cyclictest12131-21kworker/u8:108:51:051
1776199170,2cyclictest0-21swapper/108:14:051
1776399161,13cyclictest0-21swapper/312:05:383
1776399161,13cyclictest0-21swapper/309:55:443
17761991613,2cyclictest8215-21kworker/u8:008:38:051
17761991613,2cyclictest22085-21kworker/u8:007:14:051
17761991613,2cyclictest1850-21kworker/u8:107:58:021
17761991613,2cyclictest18379-21kworker/u8:010:42:051
1776099161,12cyclictest1670-21diskmemload10:50:260
1776099160,14cyclictest0-21swapper/008:05:110
70892152,2sleep07096-21pzem08:00:250
17763991512,2cyclictest1670-21diskmemload12:18:263
1776399151,12cyclictest0-21swapper/309:49:003
17763991510,3cyclictest0-21swapper/311:00:003
17763991510,3cyclictest0-21swapper/308:35:273
1776399150,13cyclictest381ktimersoftd/310:44:303
1776399150,13cyclictest14049-21ssh10:30:103
1776399150,13cyclictest0-21swapper/309:10:263
1776299151,12cyclictest24330-21df07:25:132
1776199150,13cyclictest0-21swapper/111:35:481
1776099151,3cyclictest0-21swapper/010:37:580
1776099151,2cyclictest0-21swapper/008:55:120
1776099151,12cyclictest0-21swapper/012:05:060
17760991510,3cyclictest2036-21ssh11:59:270
1776399149,3cyclictest2374-21ssh12:33:183
1776399143,7cyclictest4099-21ssh10:20:003
17763991411,2cyclictest1670-21diskmemload09:23:063
1776399141,11cyclictest25622-21ssh12:23:453
1776399140,2cyclictest0-21swapper/311:05:113
1776399140,2cyclictest0-21swapper/310:49:513
1776399140,2cyclictest0-21swapper/309:44:113
1776399140,1cyclictest0-21swapper/311:50:113
1776399140,12cyclictest0-21swapper/312:00:193
1776399140,12cyclictest0-21swapper/310:05:103
17762991411,2cyclictest5129-21runrttasks11:55:202
17762991411,2cyclictest0-21swapper/211:25:212
17762991411,2cyclictest0-21swapper/211:02:512
17762991410,2cyclictest0-21swapper/209:15:242
1776299140,2cyclictest24484-21apt-get11:15:112
1776299140,2cyclictest1670-21diskmemload09:32:412
1776299140,12cyclictest0-21swapper/210:40:132
1776299140,12cyclictest0-21swapper/210:29:582
1776299140,12cyclictest0-21swapper/209:45:182
1776299140,12cyclictest0-21swapper/209:40:122
1776299140,11cyclictest0-21swapper/211:35:592
17761991411,2cyclictest28270-21kworker/u8:207:45:051
17761991410,3cyclictest28181-21kworker/1:112:30:261
1776199140,12cyclictest0-21swapper/111:42:301
1776199140,12cyclictest0-21swapper/110:57:461
1776199140,12cyclictest0-21swapper/110:04:061
1776199140,12cyclictest0-21swapper/109:45:491
1776099149,3cyclictest3686-21ssh11:26:170
1776099142,6cyclictest0-21swapper/011:08:020
1776099142,5cyclictest0-21swapper/011:45:580
17760991411,2cyclictest0-21swapper/012:36:210
17760991410,2cyclictest0-21swapper/012:31:390
1776099140,5cyclictest0-21swapper/012:23:020
1776099140,3cyclictest0-21swapper/012:02:560
1776099140,12cyclictest1670-21diskmemload10:55:390
1776099140,12cyclictest0-21swapper/011:12:580
232892130,2sleep00-21swapper/009:30:590
1776399138,3cyclictest5129-21runrttasks11:55:393
1776399138,3cyclictest18179-21ssh09:25:293
1776399138,3cyclictest18160-21ssh10:00:163
1776399132,6cyclictest0-21swapper/312:35:013
1776399132,6cyclictest0-21swapper/307:20:013
1776399131,7cyclictest0-21swapper/308:50:003
1776399131,2cyclictest381ktimersoftd/310:39:153
1776399131,11cyclictest37-21rcuc/309:35:083
17763991311,1cyclictest0-21swapper/312:12:013
1776399131,10cyclictest29914-21tr07:39:593
17763991310,2cyclictest0-21swapper/312:27:133
17763991310,2cyclictest0-21swapper/311:45:213
17763991310,2cyclictest0-21swapper/311:34:403
17763991310,2cyclictest0-21swapper/311:02:233
17763991310,2cyclictest0-21swapper/309:34:273
17763991310,2cyclictest0-21swapper/307:30:133
1776399130,2cyclictest381ktimersoftd/311:28:573
1776399130,2cyclictest24422-21ssh11:15:043
1776399130,12cyclictest0-21swapper/311:11:373
1776399130,12cyclictest0-21swapper/310:20:293
1776399130,12cyclictest0-21swapper/309:50:023
1776399130,12cyclictest0-21swapper/309:16:413
1776399130,12cyclictest0-21swapper/308:20:173
1776299133,7cyclictest0-21swapper/207:53:582
17762991310,2cyclictest18154-21ssh09:25:282
17762991310,2cyclictest0-21swapper/212:20:152
17762991310,2cyclictest0-21swapper/212:12:452
17762991310,2cyclictest0-21swapper/211:50:012
17762991310,2cyclictest0-21swapper/211:43:582
17762991310,2cyclictest0-21swapper/210:32:202
17762991310,2cyclictest0-21swapper/210:11:112
17762991310,2cyclictest0-21swapper/210:01:012
17762991310,2cyclictest0-21swapper/209:58:142
17762991310,2cyclictest0-21swapper/209:10:112
17762991310,2cyclictest0-21swapper/207:10:202
1776299130,3cyclictest6560-21ssh12:37:052
1776299130,2cyclictest23170-21ssh10:39:582
1776299130,2cyclictest0-21swapper/211:45:252
1776299130,1cyclictest0-21swapper/212:28:012
1776299130,12cyclictest0-21swapper/212:30:172
1776299130,12cyclictest0-21swapper/212:17:362
1776299130,11cyclictest0-21swapper/210:05:022
1776299130,11cyclictest0-21swapper/209:51:402
1776199131,11cyclictest32019-21ssh10:48:551
17761991310,2cyclictest32370-21kworker/u8:109:08:051
1776199130,2cyclictest23-21ksoftirqd/111:07:271
1776199130,2cyclictest0-21swapper/109:55:041
1776199130,2cyclictest0-21swapper/108:00:241
1776099139,2cyclictest0-21swapper/010:42:390
17760991310,2cyclictest0-21swapper/011:53:010
17760991310,2cyclictest0-21swapper/011:41:590
17760991310,2cyclictest0-21swapper/011:31:230
17760991310,2cyclictest0-21swapper/011:18:450
1776099130,3cyclictest0-21swapper/009:25:150
1776099130,12cyclictest0-21swapper/011:23:510
1776099130,12cyclictest0-21swapper/009:19:310
1776099130,11cyclictest1670-21diskmemload09:38:510
1776099130,11cyclictest0-21swapper/010:46:140
1776399129,2cyclictest381ktimersoftd/308:30:123
1776399129,2cyclictest0-21swapper/308:40:243
1776399122,6cyclictest0-21swapper/307:45:243
1776399120,11cyclictest0-21swapper/310:50:153
1776399120,11cyclictest0-21swapper/307:50:173
1776299129,2cyclictest0-21swapper/211:22:572
1776299129,2cyclictest0-21swapper/210:20:272
1776299129,2cyclictest0-21swapper/210:16:172
1776299129,2cyclictest0-21swapper/207:30:262
1776299122,7cyclictest0-21swapper/208:32:022
1776299122,7cyclictest0-21swapper/207:48:592
1776299122,6cyclictest0-21swapper/208:40:242
1776299120,2cyclictest0-21swapper/211:06:432
1776299120,2cyclictest0-21swapper/210:52:412
1776299120,2cyclictest0-21swapper/209:35:082
1776299120,2cyclictest0-21swapper/208:35:232
1776299120,2cyclictest0-21swapper/207:15:172
1776299120,11cyclictest0-21swapper/212:00:282
1776299120,11cyclictest0-21swapper/211:10:452
1776299120,11cyclictest0-21swapper/210:55:272
1776299120,11cyclictest0-21swapper/209:20:082
1776299120,11cyclictest0-21swapper/208:45:142
1776199120,11cyclictest0-21swapper/111:10:081
1776199120,11cyclictest0-21swapper/108:20:131
1776099129,2cyclictest0-21swapper/012:25:350
1776099129,2cyclictest0-21swapper/009:24:100
1776099127,3cyclictest11833-21cut11:35:140
1776099122,7cyclictest0-21swapper/008:47:590
1776099122,7cyclictest0-21swapper/008:28:020
1776099122,7cyclictest0-21swapper/007:19:580
1776099120,2cyclictest5210-21ssh09:46:010
1776099120,2cyclictest0-21swapper/010:15:410
1776099120,2cyclictest0-21swapper/010:05:110
1776099120,11cyclictest0-21swapper/011:00:300
1776099120,11cyclictest0-21swapper/010:33:240
1776099120,11cyclictest0-21swapper/010:13:110
1776099120,11cyclictest0-21swapper/009:57:500
327002110,2sleep00-21swapper/009:40:450
204862110,2sleep00-21swapper/010:02:220
1776399112,6cyclictest0-21swapper/309:00:233
1776399112,6cyclictest0-21swapper/308:09:593
1776399110,2cyclictest381ktimersoftd/307:25:113
1776299118,2cyclictest0-21swapper/209:04:272
1776299118,2cyclictest0-21swapper/208:10:132
1776299118,2cyclictest0-21swapper/208:05:152
1776299112,6cyclictest0-21swapper/207:43:022
1776299111,7cyclictest5911-21gdm-simple-gree08:29:582
1776299110,8cyclictest13988-21copy08:20:022
1776199112,7cyclictest0-21swapper/108:25:181
1776099118,2cyclictest0-21swapper/008:15:170
1776099118,2cyclictest0-21swapper/007:35:120
1776099118,2cyclictest0-21swapper/007:25:150
1776099116,3cyclictest9271-21sh10:25:120
1776099112,6cyclictest0-21swapper/010:25:000
1776099112,6cyclictest0-21swapper/008:35:000
1776099111,3cyclictest0-21swapper/007:45:160
302042100,0sleep30-21swapper/307:40:113
1776399107,2cyclictest0-21swapper/307:10:233
1776399104,2cyclictest37-21rcuc/308:12:023
1776399102,6cyclictest0-21swapper/308:55:253
1776399102,6cyclictest0-21swapper/308:25:273
1776399102,6cyclictest0-21swapper/307:20:243
1776399100,6cyclictest0-21swapper/309:05:233
1776299102,7cyclictest0-21swapper/207:59:022
1776299102,6cyclictest0-21swapper/208:50:232
1776299102,5cyclictest0-21swapper/208:17:012
1776299101,5cyclictest0-21swapper/208:55:002
1776299100,7cyclictest0-21swapper/208:03:582
1776299100,7cyclictest0-21swapper/207:36:582
1776099105,3cyclictest19750irq/25-ahci[00007:25:010
1776099102,6cyclictest0-21swapper/008:00:000
1776099102,6cyclictest0-21swapper/007:53:260
1776099100,8cyclictest0-21swapper/008:10:060
27592290,2sleep00-21swapper/007:31:480
177639994,4cyclictest4324-21nscd07:55:133
177639994,3cyclictest26287-21munin-node3
177639991,6cyclictest0-21swapper/308:15:253
177629990,8cyclictest0-21swapper/209:09:542
177629990,7cyclictest0-21swapper/207:24:592
177609994,3cyclictest433-21grep09:05:130
177609994,3cyclictest26539-21cpuspeed_turbos08:50:140
177609994,3cyclictest22580-21df_abs08:40:140
177609994,3cyclictest189022sleep008:30:150
177609994,3cyclictest14900-21kernelversion08:20:170
177609983,3cyclictest318002chrt09:01:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional