You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Mon Nov 17, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11319212298,19sleep00-21swapper/007:08:570
11337211895,18sleep30-21swapper/307:09:123
11228210897,6sleep20-21swapper/207:07:472
11083210796,6sleep10-21swapper/107:05:531
286372960,1sleep0101ktimersoftd/007:50:170
261002770,5sleep229-21rcuc/212:30:312
93212760,1sleep10-21swapper/111:05:161
11392750,0sleep10-21swapper/109:15:161
258582740,0sleep00-21swapper/011:56:180
21962680,0sleep10-21swapper/110:58:021
11441992415,7cyclictest1676-21ssh09:50:123
11441992312,3cyclictest0-21swapper/311:50:203
11441992211,2cyclictest0-21swapper/309:31:563
1144199210,13cyclictest0-21swapper/311:25:303
11441991814,2cyclictest21990-21ssh10:45:143
1144199180,3cyclictest27676-21latency_hist07:49:593
11440991815,2cyclictest25924-21kworker/u8:007:50:042
11440991714,2cyclictest5334-21kworker/u8:009:54:052
1143999171,13cyclictest0-21swapper/109:45:001
11441991613,2cyclictest0-21swapper/309:36:103
11440991613,2cyclictest5015-21kworker/u8:007:10:042
11440991613,2cyclictest32432-21kworker/u8:009:22:052
11440991613,2cyclictest26952-21kworker/u8:211:39:052
11440991613,2cyclictest20748-21kworker/u8:010:49:052
1144099161,13cyclictest7309-21ssh11:03:182
1143999161,3cyclictest0-21swapper/108:15:191
11439991611,3cyclictest26367-21sh10:50:001
1143999160,3cyclictest0-21swapper/109:31:261
11441991512,2cyclictest0-21swapper/311:10:313
1144199151,12cyclictest12774-21df10:35:143
11441991510,3cyclictest5274-21cut11:35:163
1144199150,2cyclictest0-21swapper/310:33:013
1144199150,13cyclictest29800-21popularity-cont07:54:273
1144099151,3cyclictest0-21swapper/210:01:092
1144099151,2cyclictest19804-21apt-get09:35:122
11440991512,2cyclictest4516-21kworker/u8:212:15:042
11440991512,2cyclictest19565-21kworker/u8:008:50:062
11440991512,2cyclictest0-21swapper/212:03:392
1144099150,3cyclictest6584-21ssh10:28:462
1144099150,13cyclictest0-21swapper/211:58:232
11439991512,2cyclictest0-21swapper/110:26:361
1143999151,12cyclictest0-21swapper/112:32:261
1143999151,12cyclictest0-21swapper/110:40:031
1143999151,12cyclictest0-21swapper/110:39:481
11439991510,3cyclictest12790-21sensors07:10:251
11439991510,3cyclictest0-21swapper/112:05:311
11439991510,3cyclictest0-21swapper/110:34:281
1143999150,13cyclictest27893-21diskmemload10:22:371
1143999150,13cyclictest0-21swapper/109:55:171
1143999150,13cyclictest0-21swapper/109:05:131
11435991512,2cyclictest26872-21kworker/u8:110:59:040
11435991512,2cyclictest0-21swapper/011:03:410
1143599150,13cyclictest0-21swapper/012:06:500
1144199149,3cyclictest14104-21fschecks_count07:15:153
1144199141,1cyclictest0-21swapper/310:11:103
11441991411,2cyclictest0-21swapper/311:58:203
11441991411,2cyclictest0-21swapper/311:41:003
11441991411,2cyclictest0-21swapper/311:08:203
11441991411,2cyclictest0-21swapper/309:11:203
1144199141,11cyclictest28037-21ssh10:16:433
11441991410,3cyclictest31363-21ssh09:46:113
1144199140,2cyclictest0-21swapper/309:28:303
1144199140,13cyclictest0-21swapper/312:08:303
1144199140,13cyclictest0-21swapper/308:55:003
1144199140,12cyclictest928-21ssh10:22:383
1144199140,12cyclictest0-21swapper/309:56:113
11440991411,2cyclictest0-21swapper/212:36:492
11440991411,2cyclictest0-21swapper/212:20:282
11440991410,3cyclictest10233-21kworker/u8:010:33:042
1144099140,3cyclictest24907-21df09:40:142
1144099140,2cyclictest0-21swapper/210:59:042
1144099140,13cyclictest0-21swapper/210:52:282
1144099140,12cyclictest27893-21diskmemload12:05:012
1143999149,3cyclictest0-21swapper/111:52:181
1143999149,3cyclictest0-21swapper/108:25:241
1143999142,6cyclictest0-21swapper/108:00:011
11439991411,2cyclictest0-21swapper/110:16:161
1143999140,2cyclictest0-21swapper/109:47:171
1143999140,13cyclictest0-21swapper/109:11:171
1143999140,13cyclictest0-21swapper/108:55:271
1143999140,12cyclictest18225-21apt-get11:15:141
1143999140,12cyclictest0-21swapper/112:36:461
1143999140,12cyclictest0-21swapper/112:24:161
1143999140,12cyclictest0-21swapper/110:10:591
11435991411,2cyclictest0-21swapper/011:10:100
1143599141,11cyclictest29047-21memory09:10:190
84172130,2sleep00-21swapper/009:55:490
41282130,2sleep10-21swapper/109:51:531
314192130,2sleep00-21swapper/012:36:310
272322130,2sleep00-21swapper/009:42:100
223872130,2sleep00-21swapper/011:19:390
215782130,2sleep00-21swapper/010:44:550
1144199139,2cyclictest19452-21apt-get12:25:063
11441991310,2cyclictest27893-21diskmemload11:49:013
11441991310,2cyclictest13058-21ssh12:17:193
11441991310,2cyclictest0-21swapper/310:42:103
11441991310,2cyclictest0-21swapper/310:28:003
11441991310,2cyclictest0-21swapper/310:01:003
11441991310,2cyclictest0-21swapper/309:15:253
11441991310,2cyclictest0-21swapper/309:00:163
11441991310,2cyclictest0-21swapper/308:35:163
1144199130,2cyclictest0-21swapper/312:11:113
1144199130,2cyclictest0-21swapper/310:54:343
1144199130,2cyclictest0-21swapper/310:05:453
1144199130,2cyclictest0-21swapper/307:40:243
1144199130,12cyclictest0-21swapper/309:42:453
1144199130,12cyclictest0-21swapper/309:22:113
1144099131,2cyclictest5129-21runrttasks09:25:322
1144099131,11cyclictest0-21swapper/211:11:232
11440991310,2cyclictest9207-21kworker/u8:011:40:042
11440991310,2cyclictest25924-21kworker/u8:007:57:052
11440991310,2cyclictest0-21swapper/212:27:002
11440991310,2cyclictest0-21swapper/211:50:212
11440991310,2cyclictest0-21swapper/211:30:142
11440991310,2cyclictest0-21swapper/211:21:162
11440991310,2cyclictest0-21swapper/211:05:072
11440991310,2cyclictest0-21swapper/210:05:212
11440991310,2cyclictest0-21swapper/209:55:272
11440991310,2cyclictest0-21swapper/209:12:402
11440991310,2cyclictest0-21swapper/208:05:202
1144099130,2cyclictest18216-21ssh11:15:132
1144099130,2cyclictest0-21swapper/210:15:092
1144099130,2cyclictest0-21swapper/210:11:072
1144099130,12cyclictest27893-21diskmemload10:42:282
1144099130,12cyclictest0-21swapper/211:45:512
1144099130,12cyclictest0-21swapper/210:21:092
1144099130,12cyclictest0-21swapper/209:49:192
1144099130,12cyclictest0-21swapper/209:15:282
1144099130,12cyclictest0-21swapper/208:35:172
1144099130,12cyclictest0-21swapper/208:15:182
1144099130,12cyclictest0-21swapper/207:45:282
1143999138,3cyclictest23768-21apt-get11:55:121
1143999138,3cyclictest0-21swapper/112:10:151
1143999132,6cyclictest0-21swapper/109:05:001
1143999131,9cyclictest0-21swapper/110:00:201
11439991310,2cyclictest0-21swapper/112:16:361
11439991310,2cyclictest0-21swapper/111:25:081
11439991310,2cyclictest0-21swapper/111:01:271
11439991310,2cyclictest0-21swapper/110:54:151
11439991310,2cyclictest0-21swapper/110:05:211
11439991310,2cyclictest0-21swapper/109:25:071
11439991310,2cyclictest0-21swapper/107:15:251
1143999130,2cyclictest27893-21diskmemload12:25:051
1143999130,2cyclictest0-21swapper/111:31:411
1143999130,2cyclictest0-21swapper/109:38:101
1143999130,2cyclictest0-21swapper/108:00:141
1143999130,12cyclictest0-21swapper/111:41:161
1143999130,12cyclictest0-21swapper/111:20:381
1143999130,12cyclictest0-21swapper/108:05:271
1143999130,11cyclictest27893-21diskmemload12:04:061
1143999130,11cyclictest27893-21diskmemload11:46:081
11435991310,2cyclictest27893-21diskmemload11:40:400
11435991310,2cyclictest0-21swapper/012:15:090
11435991310,2cyclictest0-21swapper/011:48:400
11435991310,2cyclictest0-21swapper/011:38:000
11435991310,2cyclictest0-21swapper/011:22:450
1143599130,2cyclictest7440-21ssh12:10:500
1143599130,2cyclictest32703-21ssh12:04:420
1143599130,2cyclictest0-21swapper/009:20:200
1143599130,12cyclictest32270-21ssh11:30:000
1143599130,11cyclictest7787-21latency_hist08:19:590
1143599130,11cyclictest27893-21diskmemload11:05:180
1143599130,11cyclictest0-21swapper/010:04:410
283452120,1sleep00-21swapper/010:17:140
1144199129,2cyclictest0-21swapper/312:00:293
1144199129,2cyclictest0-21swapper/310:55:403
1144199129,2cyclictest0-21swapper/309:05:203
1144199122,7cyclictest0-21swapper/308:41:023
1144199122,7cyclictest0-21swapper/308:26:583
1144199122,7cyclictest0-21swapper/308:13:583
1144199122,7cyclictest0-21swapper/307:30:583
1144199120,2cyclictest27893-21diskmemload12:30:353
1144199120,2cyclictest0-21swapper/312:35:263
1144199120,2cyclictest0-21swapper/312:20:573
1144199120,2cyclictest0-21swapper/311:30:033
1144199120,2cyclictest0-21swapper/311:20:183
1144199120,2cyclictest0-21swapper/311:15:193
1144199120,11cyclictest0-21swapper/311:00:053
1144199120,11cyclictest0-21swapper/308:00:133
1144199120,11cyclictest0-21swapper/307:55:153
1144199120,11cyclictest0-21swapper/307:35:193
1144199120,11cyclictest0-21swapper/307:10:263
1144099129,2cyclictest5015-21kworker/u8:007:27:042
1144099129,2cyclictest21946-21kworker/u8:107:43:052
1144099129,2cyclictest16124-21kworker/u8:209:08:042
1144099129,2cyclictest16124-21kworker/u8:208:43:042
1144099120,2cyclictest0-21swapper/212:10:582
1144099120,2cyclictest0-21swapper/208:25:192
1144099120,11cyclictest0-21swapper/211:25:082
1144099120,11cyclictest0-21swapper/210:36:232
1144099120,11cyclictest0-21swapper/209:32:182
1144099120,11cyclictest0-21swapper/208:55:242
1143999129,2cyclictest0-21swapper/111:35:221
1143999127,3cyclictest14394-21tr11:10:181
1143999121,9cyclictest18007-21munin-node1
1143999121,9cyclictest0-21swapper/108:50:051
1143999121,9cyclictest0-21swapper/108:30:151
1143999121,9cyclictest0-21swapper/108:10:151
1143999121,9cyclictest0-21swapper/107:50:261
1143999121,9cyclictest0-21swapper/107:20:141
1143999121,3cyclictest0-21swapper/107:45:171
1143999120,2cyclictest0-21swapper/109:20:101
1143999120,2cyclictest0-21swapper/108:40:261
1143999120,2cyclictest0-21swapper/107:39:591
1143599129,2cyclictest30622-21kworker/u8:110:25:040
1143599129,2cyclictest26043-21aten2.4-expect07:45:160
1143599129,2cyclictest0-21swapper/011:25:430
1143599129,2cyclictest0-21swapper/010:50:420
1143599129,2cyclictest0-21swapper/010:45:200
1143599129,2cyclictest0-21swapper/010:08:300
1143599129,2cyclictest0-21swapper/009:37:310
1143599129,2cyclictest0-21swapper/008:25:190
1143599128,2cyclictest0-21swapper/009:50:150
1143599120,2cyclictest27893-21diskmemload12:20:450
1143599120,2cyclictest0-21swapper/011:51:060
1143599120,11cyclictest0-21swapper/012:30:080
1143599120,11cyclictest0-21swapper/012:25:510
1144199118,2cyclictest0-21swapper/308:15:283
1144199113,6cyclictest0-21swapper/308:30:263
1144199112,6cyclictest0-21swapper/308:47:583
1144199112,6cyclictest0-21swapper/308:07:023
1144199112,6cyclictest0-21swapper/307:29:013
1144099119,1cyclictest0-21swapper/208:10:182
1144099118,2cyclictest7679-21kworker/u8:107:15:052
1144099118,2cyclictest16124-21kworker/u8:209:04:042
1144099118,2cyclictest0-21swapper/208:45:392
1144099118,2cyclictest0-21swapper/208:00:132
1143999116,3cyclictest14451-21cut08:35:161
1143999111,8cyclictest19798-21munin-node1
1143999110,2cyclictest0-21swapper/107:25:211
1143999110,1cyclictest0-21swapper/107:40:181
1143999110,10cyclictest0-21swapper/108:20:231
1143599118,2cyclictest24622-21kworker/u8:109:46:050
1143599118,2cyclictest22051-21kworker/u8:109:17:040
1143599118,2cyclictest19849-21copy08:50:000
1143599118,2cyclictest0-21swapper/010:11:410
1143599111,3cyclictest27356-1kworker/0:0H09:35:000
1143599110,9cyclictest0-21swapper/010:22:100
227142100,2sleep00-21swapper/008:55:170
1144199102,6cyclictest0-21swapper/308:55:253
1144199102,6cyclictest0-21swapper/307:20:243
1144099107,2cyclictest5015-21kworker/u8:007:35:042
1144099107,2cyclictest0-21swapper/208:30:282
1144099100,9cyclictest0-21swapper/207:32:402
1143599107,2cyclictest5015-21kworker/u8:007:16:010
1143599107,2cyclictest16470-21fschecks_count08:40:140
1143599104,2cyclictest0-21swapper/008:13:050
1143599100,2cyclictest27893-21diskmemload09:25:510
1143599100,2cyclictest0-21swapper/007:35:170
114419994,3cyclictest0-21swapper/308:20:183
114409994,3cyclictest8849-21open_inodes08:20:212
114409994,3cyclictest15976-21cpuspeed_turbos07:20:142
114359996,2cyclictest31755-21kworker/u8:207:58:050
114359996,2cyclictest27893-21diskmemload10:37:350
114359994,3cyclictest14708-21latency08:35:190
114359990,2cyclictest0-21swapper/010:31:280
114359990,2cyclictest0-21swapper/007:30:160
114359985,2cyclictest5129-21runrttasks09:01:010
114359985,2cyclictest19750irq/25-ahci[00008:50:000
114359985,2cyclictest1869-21kworker/u8:008:07:040
114359985,2cyclictest0-21swapper/008:00:160
114359984,3cyclictest5010-21snmpd07:20:150
114359983,3cyclictest9876-21cat08:25:010
114359983,3cyclictest25795-21latency_hist07:44:590
114359981,3cyclictest18354-21munin-node0
114359980,7cyclictest0-21swapper/007:10:160
114359980,2cyclictest0-21swapper/009:05:290
114359980,2cyclictest0-21swapper/008:31:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional