You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 11:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sun Dec 28, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9222211998,14sleep30-21swapper/319:05:303
93682116105,6sleep00-21swapper/019:07:250
9313211292,15sleep20-21swapper/219:06:422
943129881,12sleep10-21swapper/119:08:141
151092890,1sleep2301ktimersoftd/221:35:062
280812350,1sleep128080-21ssh00:19:351
960299320,30cyclictest0-21swapper/219:30:122
286022250,1sleep20-21swapper/222:27:172
187332230,0sleep10-21swapper/100:08:171
960299221,19cyclictest0-21swapper/200:00:122
960499211,18cyclictest0-21swapper/323:40:123
960499210,19cyclictest0-21swapper/322:00:153
9602992111,2cyclictest12-21rcu_preempt22:22:552
960299200,2cyclictest0-21swapper/223:25:142
960299200,13cyclictest0-21swapper/223:00:052
960199200,18cyclictest0-21swapper/123:35:111
960099200,2cyclictest0-21swapper/020:20:120
150642200,1sleep10-21swapper/100:04:241
960299190,2cyclictest0-21swapper/220:10:132
318732190,1sleep30-21swapper/323:45:263
9600991813,3cyclictest17734-21ssh23:30:120
960099180,2cyclictest0-21swapper/023:25:140
9601991714,2cyclictest23366-21kworker/u8:022:27:041
960499161,13cyclictest0-21swapper/321:55:003
960499160,14cyclictest0-21swapper/322:36:453
960299161,3cyclictest6675-21snmpd23:16:592
960199161,6cyclictest0-21swapper/122:50:001
960199161,13cyclictest0-21swapper/123:20:131
9600991611,3cyclictest1936-21turbostat22:35:000
48632160,0sleep30-21swapper/323:51:513
320342160,1sleep20-21swapper/221:16:062
320342160,1sleep20-21swapper/221:16:062
960499151,3cyclictest26983-21ssh22:25:183
9604991512,2cyclictest0-21swapper/300:22:053
960499151,12cyclictest14837-21apt-get21:35:073
9604991511,2cyclictest0-21swapper/321:25:173
960499150,3cyclictest30124-21munin-node3
960499150,3cyclictest30124-21munin-node3
960499150,13cyclictest17516-21ls23:30:023
960499150,13cyclictest11239-21vmstat20:30:293
960499150,13cyclictest0-21swapper/322:13:353
9602991512,2cyclictest0-21swapper/223:11:352
960299151,12cyclictest24134-21ssh23:36:302
960299150,13cyclictest25700-21diskmemload21:30:252
960299150,13cyclictest24775-21sed19:45:192
960299150,13cyclictest0-21swapper/222:04:552
960299150,13cyclictest0-21swapper/219:10:152
960199151,3cyclictest25700-21diskmemload00:37:181
9601991512,2cyclictest25700-21diskmemload22:20:541
9601991512,2cyclictest0-21swapper/119:45:141
960199151,12cyclictest28589-21df_abs21:50:171
9601991510,3cyclictest0-21swapper/120:00:201
960199150,13cyclictest30064-21taskset21:15:041
960199150,13cyclictest30064-21taskset21:15:041
960199150,13cyclictest25700-21diskmemload00:23:341
960199150,13cyclictest10724-21ssh22:44:541
960199150,13cyclictest0-21swapper/122:57:341
960199150,13cyclictest0-21swapper/100:25:121
316962150,1sleep20-21swapper/222:30:252
960499149,3cyclictest22685-21ssh22:58:083
960499142,6cyclictest0-21swapper/320:45:003
9604991410,2cyclictest0-21swapper/320:50:133
960499140,2cyclictest0-21swapper/323:08:353
960499140,2cyclictest0-21swapper/321:24:253
960499140,2cyclictest0-21swapper/320:00:153
960499140,12cyclictest25700-21diskmemload22:53:443
960499140,12cyclictest0-21swapper/323:56:293
960499140,12cyclictest0-21swapper/323:31:533
960499140,12cyclictest0-21swapper/323:21:373
960499140,12cyclictest0-21swapper/323:15:303
960499140,12cyclictest0-21swapper/322:46:423
960499140,12cyclictest0-21swapper/322:40:533
960499140,12cyclictest0-21swapper/322:09:503
960499140,12cyclictest0-21swapper/321:56:573
960499140,12cyclictest0-21swapper/321:40:383
960499140,12cyclictest0-21swapper/321:33:143
960499140,12cyclictest0-21swapper/321:13:543
960499140,12cyclictest0-21swapper/300:06:383
960299149,3cyclictest0-21swapper/223:41:172
9602991411,2cyclictest0-21swapper/223:46:162
9602991411,2cyclictest0-21swapper/223:32:062
9602991411,2cyclictest0-21swapper/223:20:352
9602991411,2cyclictest0-21swapper/222:47:252
9602991411,2cyclictest0-21swapper/222:39:252
9602991410,2cyclictest20129-21ssh22:17:332
960299140,2cyclictest25700-21diskmemload22:57:352
960299140,13cyclictest0-21swapper/200:33:362
960299140,12cyclictest25700-21diskmemload22:51:032
960299140,12cyclictest22446-21ssh21:43:032
960299140,12cyclictest0-21swapper/221:58:122
960299140,12cyclictest0-21swapper/221:13:192
960299140,12cyclictest0-21swapper/200:05:292
9601991411,2cyclictest0-21swapper/122:10:141
960199140,13cyclictest0-21swapper/121:10:341
960199140,12cyclictest25700-21diskmemload21:40:161
960199140,12cyclictest0-21swapper/123:41:491
960199140,12cyclictest0-21swapper/121:59:091
960099149,3cyclictest12347-21switchtime22:45:230
960099142,7cyclictest0-21swapper/022:30:000
960099142,6cyclictest0-21swapper/020:05:000
960099142,10cyclictest0-21swapper/021:14:580
202792140,2sleep10-21swapper/123:32:221
960499138,3cyclictest16558-21irqstats19:25:153
960499138,3cyclictest0-21swapper/322:15:203
9604991310,2cyclictest0-21swapper/322:31:103
9604991310,2cyclictest0-21swapper/321:45:293
9604991310,2cyclictest0-21swapper/320:05:203
9604991310,2cyclictest0-21swapper/300:10:163
960499130,2cyclictest25700-21diskmemload00:18:293
960499130,2cyclictest0-21swapper/319:55:143
960499130,2cyclictest0-21swapper/300:30:023
960499130,12cyclictest0-21swapper/323:35:093
960499130,12cyclictest0-21swapper/323:00:553
960499130,12cyclictest0-21swapper/300:25:213
960499130,11cyclictest0-21swapper/323:10:273
960299132,6cyclictest0-21swapper/219:40:012
9602991310,2cyclictest25700-21diskmemload00:21:222
9602991310,2cyclictest0-21swapper/222:13:452
9602991310,2cyclictest0-21swapper/222:05:062
9602991310,2cyclictest0-21swapper/221:46:112
9602991310,2cyclictest0-21swapper/221:27:012
9602991310,2cyclictest0-21swapper/221:05:182
9602991310,2cyclictest0-21swapper/200:27:182
960299130,2cyclictest25700-21diskmemload00:10:242
960299130,2cyclictest0-21swapper/223:05:142
960299130,2cyclictest0-21swapper/222:43:252
960299130,2cyclictest0-21swapper/221:50:272
960299130,2cyclictest0-21swapper/221:21:112
960299130,12cyclictest5685-21ssh23:53:222
960299130,12cyclictest0-21swapper/223:56:342
960299130,11cyclictest25700-21diskmemload00:15:312
960199133,8cyclictest0-21swapper/122:45:231
9601991310,2cyclictest25700-21diskmemload23:56:441
9601991310,2cyclictest0-21swapper/123:45:231
9601991310,2cyclictest0-21swapper/122:06:541
9601991310,2cyclictest0-21swapper/121:27:541
9601991310,2cyclictest0-21swapper/120:20:161
9601991310,2cyclictest0-21swapper/119:30:051
960199130,2cyclictest25700-21diskmemload22:01:211
960199130,2cyclictest0-21swapper/123:18:271
960199130,2cyclictest0-21swapper/123:09:341
960199130,2cyclictest0-21swapper/122:35:311
960199130,2cyclictest0-21swapper/121:45:061
960199130,2cyclictest0-21swapper/121:36:541
960199130,2cyclictest0-21swapper/120:45:261
960199130,2cyclictest0-21swapper/119:50:251
960199130,1cyclictest0-21swapper/123:52:041
960199130,12cyclictest0-21swapper/123:28:581
960199130,12cyclictest0-21swapper/123:10:161
960199130,11cyclictest25700-21diskmemload21:20:421
960199130,11cyclictest25700-21diskmemload00:30:231
960199130,11cyclictest14973-21ls21:35:011
960199130,11cyclictest0-21swapper/100:12:381
960099139,2cyclictest16763-21open_files20:45:190
960099138,3cyclictest19518-21chrt19:32:360
960099138,3cyclictest0-21swapper/023:00:120
960099132,8cyclictest0-21swapper/019:46:580
960099132,6cyclictest14883-21copy21:35:000
960099132,6cyclictest0-21swapper/022:20:000
960099132,6cyclictest0-21swapper/022:10:000
960099132,6cyclictest0-21swapper/021:45:010
960099132,6cyclictest0-21swapper/000:30:010
960099131,9cyclictest16917-21ssh00:05:260
960099131,6cyclictest0-21swapper/019:45:000
960099130,3cyclictest0-21swapper/021:44:500
960099130,11cyclictest25700-21diskmemload22:53:000
307462130,2sleep025700-21diskmemload23:07:050
236812130,2sleep00-21swapper/022:21:110
13632130,2sleep00-21swapper/021:19:270
13632130,2sleep00-21swapper/021:19:260
960499129,2cyclictest0-21swapper/322:21:433
960499129,2cyclictest0-21swapper/300:35:043
960499129,2cyclictest0-21swapper/300:00:053
960499122,5cyclictest0-21swapper/320:15:003
960499120,2cyclictest0-21swapper/320:55:193
960499120,2cyclictest0-21swapper/319:20:243
960499120,2cyclictest0-21swapper/319:10:163
960299129,2cyclictest0-21swapper/220:55:002
960299129,2cyclictest0-21swapper/220:20:202
960299122,7cyclictest0-21swapper/220:04:582
960299122,6cyclictest0-21swapper/220:45:012
960299121,6cyclictest0-21swapper/220:30:002
960299120,1cyclictest0-21swapper/200:35:422
960299120,11cyclictest0-21swapper/219:20:162
960299120,11cyclictest0-21swapper/219:15:212
960199129,2cyclictest0-21swapper/123:00:171
960199129,2cyclictest0-21swapper/122:30:201
960199121,9cyclictest27984-21munin-node1
960199120,2cyclictest0-21swapper/122:15:581
960199120,2cyclictest0-21swapper/121:00:151
960199120,2cyclictest0-21swapper/119:20:211
960199120,11cyclictest0-21swapper/120:25:151
960199120,11cyclictest0-21swapper/120:10:181
960099129,2cyclictest29701-21ssh23:43:450
960099129,2cyclictest0-21swapper/020:50:190
960099129,2cyclictest0-21swapper/000:10:350
960099122,7cyclictest0-21swapper/019:52:580
960099122,6cyclictest0-21swapper/023:55:000
960099122,6cyclictest0-21swapper/020:35:010
960099122,6cyclictest0-21swapper/000:22:190
960099120,11cyclictest12069-21ssh22:08:350
960099120,11cyclictest0-21swapper/022:55:120
960099120,11cyclictest0-21swapper/000:04:550
960099120,10cyclictest0-21swapper/023:12:090
960099120,10cyclictest0-21swapper/021:25:200
309832120,2sleep00-21swapper/021:52:450
960499118,2cyclictest0-21swapper/320:15:183
960499112,7cyclictest0-21swapper/320:20:583
960499110,2cyclictest0-21swapper/321:05:193
960299118,2cyclictest0-21swapper/220:50:192
960299110,10cyclictest0-21swapper/220:07:292
960199118,2cyclictest0-21swapper/120:05:191
960199112,6cyclictest0-21swapper/120:31:251
960199111,8cyclictest19878-21apt-get20:55:131
960199111,5cyclictest0-21swapper/120:40:001
960199111,3cyclictest13850-21munin-node1
960199110,9cyclictest23906-21munin-node1
960199110,9cyclictest19927-21munin-node1
960199110,9cyclictest0-21swapper/119:15:271
960099119,1cyclictest0-21swapper/022:36:050
960099118,2cyclictest25700-21diskmemload21:23:030
960099118,2cyclictest0-21swapper/023:57:110
960099118,2cyclictest0-21swapper/023:21:450
960099118,2cyclictest0-21swapper/021:58:150
960099113,5cyclictest0-21swapper/022:40:260
960099112,6cyclictest0-21swapper/020:27:580
960099110,9cyclictest0-21swapper/023:35:020
960099110,8cyclictest0-21swapper/000:27:580
960099110,2cyclictest0-21swapper/020:00:010
960099110,10cyclictest17856-21ssh21:37:350
65542110,2sleep00-21swapper/022:01:340
960499107,2cyclictest0-21swapper/319:40:183
960499105,3cyclictest13185-21grep19:15:273
960499102,6cyclictest0-21swapper/320:38:583
960499100,6cyclictest0-21swapper/319:54:553
960499100,2cyclictest0-21swapper/321:03:293
960299105,3cyclictest3857-21cpuspeed_turbos20:15:132
960299102,6cyclictest0-21swapper/220:30:262
960199105,3cyclictest22224-21tr19:40:151
960199101,7cyclictest0-21swapper/120:15:111
960199100,2cyclictest0-21swapper/119:10:291
960099105,3cyclictest6318-21Xorg20:39:580
960099105,3cyclictest4914-21tail20:15:250
960099102,7cyclictest0-21swapper/021:05:580
960099102,6cyclictest0-21swapper/021:00:240
960099102,6cyclictest0-21swapper/000:37:220
960099101,6cyclictest0-21swapper/000:15:100
960099101,5cyclictest0-21swapper/020:00:140
960099100,9cyclictest0-21swapper/023:15:550
960099100,7cyclictest0-21swapper/019:37:580
960099100,2cyclictest0-21swapper/019:20:140
8952100,6sleep00-21swapper/023:47:330
96049994,3cyclictest8747-21proc_pri20:25:213
96049994,3cyclictest24289-21df_abs19:45:163
96049994,3cyclictest0-21swapper/319:30:053
96029994,3cyclictest7458-21runrttasks19:29:192
96029994,3cyclictest26013-21aten2.4_r3power19:50:122
96029994,3cyclictest23642-21sshd21:04:292
96029994,3cyclictest22743-21wc19:40:192
96019995,2cyclictest17867-21munin-node1
96019991,6cyclictest16987-21chrt19:25:231
96009996,2cyclictest17250-21grep19:25:250
96009996,2cyclictest0-21swapper/019:15:140
96009994,3cyclictest7458-21runrttasks20:57:400
96009994,3cyclictest10255-21fschecks_count19:10:160
96009991,3cyclictest3366-21munin-run20:15:000
96049983,3cyclictest20141-21cpuspeed_turbos19:35:133
96049983,3cyclictest16020-21telnet20:45:143
96029983,3cyclictest28211-21cpuspeed_turbos19:55:142
96029983,3cyclictest15870-21basename20:45:132
96029983,3cyclictest12224-21chrt20:35:152
96009981,3cyclictest14244-21diskstats20:40:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional