You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 01:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sat Apr 11, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26640210897,6sleep20-21swapper/219:07:432
26640210897,6sleep20-21swapper/219:07:422
26678210579,21sleep00-21swapper/019:08:120
26678210579,21sleep00-21swapper/019:08:120
26043210384,14sleep10-21swapper/119:05:241
26043210384,14sleep10-21swapper/119:05:241
26728210184,12sleep30-21swapper/319:08:523
26728210184,12sleep30-21swapper/319:08:513
142812840,0sleep00-21swapper/023:26:400
92572810,0sleep20-21swapper/223:21:202
12822770,0sleep30-21swapper/323:14:163
121252690,0sleep00-21swapper/023:58:330
255992670,5sleep32684999cyclictest21:25:123
2684899320,2cyclictest0-21swapper/219:40:112
2684799280,26cyclictest0-21swapper/122:10:101
301342220,1sleep229-21rcuc/223:43:482
2684899221,19cyclictest0-21swapper/220:25:112
2684799221,19cyclictest0-21swapper/120:15:111
2684899211,2cyclictest0-21swapper/219:50:122
2684899210,19cyclictest10691-21diskmemload21:40:122
2684799210,2cyclictest0-21swapper/100:15:121
2684699210,19cyclictest0-21swapper/021:40:110
2684699210,19cyclictest0-21swapper/000:35:110
26849992017,2cyclictest0-21swapper/323:45:113
26848992017,2cyclictest0-21swapper/221:25:112
2684899200,2cyclictest0-21swapper/220:55:112
2684899200,15cyclictest0-21swapper/220:51:042
2684799202,16cyclictest23-21ksoftirqd/121:10:101
26846992017,2cyclictest0-21swapper/021:25:110
265082200,1sleep10-21swapper/123:05:591
26848991914,3cyclictest20143-21ssh23:00:112
2684899191,14cyclictest0-21swapper/221:36:042
2684899191,13cyclictest0-21swapper/221:12:012
2684899190,14cyclictest0-21swapper/219:37:042
2684899190,14cyclictest0-21swapper/219:21:042
2684699190,2cyclictest0-21swapper/019:40:120
26848991815,2cyclictest806-21kworker/u8:219:45:042
26848991714,2cyclictest17020-21kworker/u8:119:28:052
26848991714,2cyclictest17020-21kworker/u8:119:12:042
26848991713,2cyclictest10691-21diskmemload00:38:042
2684999161,13cyclictest0-21swapper/322:05:133
2684899161,3cyclictest0-21swapper/200:15:142
26848991613,2cyclictest9677-21kworker/u8:100:33:042
2684799161,3cyclictest10691-21diskmemload21:10:011
26847991612,2cyclictest207150irq/24-eth123:40:101
2684699160,14cyclictest15387-21fschecks_count20:00:140
26849991511,2cyclictest0-21swapper/323:51:003
26849991510,3cyclictest23633-21ssh00:10:153
26849991510,3cyclictest12507-21ssh22:51:073
26849991510,3cyclictest12507-21ssh22:51:063
2684999150,3cyclictest17761-21ssh22:56:503
2684899155,7cyclictest0-21swapper/223:48:012
2684899151,3cyclictest15790-21ssh21:15:082
2684899151,2cyclictest0-21swapper/222:32:052
26848991512,2cyclictest23162-21kworker/u8:020:22:042
26848991510,3cyclictest0-21swapper/221:47:552
2684899150,2cyclictest0-21swapper/220:46:052
2684899150,13cyclictest9677-21kworker/u8:100:24:242
2684899150,13cyclictest21201-21rm23:34:552
2684799151,13cyclictest23-21ksoftirqd/122:01:031
2684799151,12cyclictest9969-21cut19:45:501
2684799151,12cyclictest9071-21apt-get22:15:121
2684799151,12cyclictest32616-21cat22:05:161
2684799151,12cyclictest21-21rcuc/119:55:121
2684699157,5cyclictest0-21swapper/020:44:580
2684699151,3cyclictest19225-21ssh21:51:040
2684699151,12cyclictest27332-21expr19:10:120
26846991510,3cyclictest27558-21runrttasks20:30:150
2684699150,13cyclictest0-21swapper/022:35:140
141772150,2sleep00-21swapper/022:53:450
141772150,2sleep00-21swapper/022:53:450
2684999140,3cyclictest29463-21rm22:35:233
2684999140,3cyclictest14742-21munin-node3
2684999140,3cyclictest0-21swapper/300:32:513
2684999140,2cyclictest5174-21ssh00:25:133
2684999140,2cyclictest0-21swapper/300:20:173
2684999140,2cyclictest0-21swapper/300:01:053
2684999140,13cyclictest0-21swapper/321:50:143
2684999140,12cyclictest9031-21ssh22:15:083
2684999140,12cyclictest3649-21snmpd23:31:223
2684999140,12cyclictest22838-21ssh21:21:263
2684999140,12cyclictest0-21swapper/322:43:413
2684999140,12cyclictest0-21swapper/321:15:173
2684899149,3cyclictest16142-21python20:01:022
2684899141,3cyclictest0-21swapper/222:09:462
26848991411,2cyclictest769-21kworker/u8:022:46:042
26848991411,2cyclictest769-21kworker/u8:022:46:042
26848991411,2cyclictest10691-21diskmemload22:58:552
26848991411,2cyclictest0-21swapper/219:30:142
26848991411,2cyclictest0-21swapper/200:25:132
26848991410,2cyclictest0-21swapper/222:50:112
26848991410,2cyclictest0-21swapper/222:50:102
2684899140,2cyclictest0-21swapper/221:51:052
2684899140,13cyclictest0-21swapper/222:27:042
2684899140,12cyclictest11691-21ssh23:57:532
2684799142,6cyclictest0-21swapper/122:50:011
2684799142,6cyclictest0-21swapper/122:50:001
2684799141,11cyclictest0-21swapper/122:22:221
2684799140,3cyclictest27957-21ntp_states19:10:171
2684799140,2cyclictest0-21swapper/122:40:131
2684799140,12cyclictest221ktimersoftd/121:00:281
2684799140,12cyclictest0-21swapper/121:20:141
2684799140,12cyclictest0-21swapper/100:20:101
26846991411,2cyclictest0-21swapper/023:49:040
26846991411,2cyclictest0-21swapper/021:10:440
2684699140,2cyclictest18662-21ssh22:58:140
2684699140,2cyclictest0-21swapper/023:00:150
2684699140,2cyclictest0-21swapper/022:25:510
2684699140,2cyclictest0-21swapper/022:23:250
2684699140,2cyclictest0-21swapper/021:15:140
2684699140,2cyclictest0-21swapper/000:01:040
2684699140,12cyclictest0-21swapper/021:34:540
2684699140,12cyclictest0-21swapper/000:13:450
2684999139,2cyclictest16734-21ssh22:22:243
26849991310,2cyclictest10691-21diskmemload00:18:293
26849991310,2cyclictest0-21swapper/323:56:023
26849991310,2cyclictest0-21swapper/323:44:523
26849991310,2cyclictest0-21swapper/323:21:363
26849991310,2cyclictest0-21swapper/323:16:023
26849991310,2cyclictest0-21swapper/322:45:053
26849991310,2cyclictest0-21swapper/322:45:053
26849991310,2cyclictest0-21swapper/322:25:223
26849991310,2cyclictest0-21swapper/321:45:573
26849991310,2cyclictest0-21swapper/321:40:143
26849991310,2cyclictest0-21swapper/320:15:183
2684999130,2cyclictest0-21swapper/321:38:213
2684999130,2cyclictest0-21swapper/321:30:203
2684999130,12cyclictest0-21swapper/322:11:533
2684999130,11cyclictest10691-21diskmemload23:25:063
2684999130,11cyclictest10691-21diskmemload22:02:383
26848991311,1cyclictest0-21swapper/221:57:452
26848991310,2cyclictest21170-21kworker/u8:222:24:042
26848991310,2cyclictest10691-21diskmemload22:40:132
26848991310,2cyclictest0-21swapper/223:36:362
26848991310,2cyclictest0-21swapper/223:06:152
26848991310,2cyclictest0-21swapper/222:36:152
26848991310,2cyclictest0-21swapper/220:30:112
2684899130,2cyclictest0-21swapper/223:25:112
2684899130,2cyclictest0-21swapper/223:11:262
2684899130,2cyclictest0-21swapper/222:13:192
2684899130,2cyclictest0-21swapper/221:30:062
2684899130,2cyclictest0-21swapper/221:21:042
2684899130,2cyclictest0-21swapper/221:07:052
2684899130,2cyclictest0-21swapper/200:07:362
2684899130,2cyclictest0-21swapper/200:04:062
2684899130,11cyclictest0-21swapper/200:14:062
2684799138,3cyclictest4933-21cut23:50:291
2684799138,3cyclictest0-21swapper/120:30:161
2684799133,8cyclictest221ktimersoftd/120:50:591
2684799133,3cyclictest17127-21pzem_power21:15:391
2684799132,9cyclictest221ktimersoftd/120:36:021
2684799132,6cyclictest0-21swapper/119:45:011
2684799131,7cyclictest0-21swapper/100:35:001
2684799131,3cyclictest10691-21diskmemload22:51:151
2684799131,3cyclictest10691-21diskmemload22:51:151
2684799131,10cyclictest7281-21apt21:40:001
26847991310,2cyclictest23-21ksoftirqd/122:26:051
2684799130,11cyclictest10691-21diskmemload23:00:141
2684799130,11cyclictest0-21swapper/122:36:301
2684799130,11cyclictest0-21swapper/121:47:221
26846991310,2cyclictest0-21swapper/023:23:140
26846991310,2cyclictest0-21swapper/023:15:340
26846991310,2cyclictest0-21swapper/022:11:460
26846991310,2cyclictest0-21swapper/020:56:020
2684699130,3cyclictest7482-21sh00:26:370
2684699130,2cyclictest30926-21latency_hist19:20:010
2684699130,2cyclictest0-21swapper/021:48:110
2684699130,12cyclictest0-21swapper/023:11:440
2684699130,12cyclictest0-21swapper/022:42:140
2684699130,12cyclictest0-21swapper/022:05:030
2684699130,11cyclictest0-21swapper/023:36:220
2684699130,11cyclictest0-21swapper/022:03:360
2684699130,11cyclictest0-21swapper/000:30:040
213542130,1sleep011-21rcuc/000:07:550
14182130,0sleep00-21swapper/000:20:340
325172120,1sleep1221ktimersoftd/121:31:331
28002120,2sleep20-21swapper/223:15:152
2684999129,2cyclictest0-21swapper/322:32:093
2684999129,2cyclictest0-21swapper/320:56:003
2684999129,2cyclictest0-21swapper/320:40:163
2684999123,6cyclictest26532-21ssh23:06:003
2684999122,6cyclictest0-21swapper/319:40:003
2684999121,6cyclictest0-21swapper/319:35:003
26849991210,1cyclictest0-21swapper/323:00:053
2684999120,11cyclictest24186-21ssh23:37:053
2684999120,11cyclictest0-21swapper/321:55:223
2684999120,11cyclictest0-21swapper/321:11:333
2684999120,11cyclictest0-21swapper/320:30:153
2684899122,7cyclictest0-21swapper/220:11:012
2684899122,7cyclictest0-21swapper/219:19:582
2684899122,6cyclictest0-21swapper/220:40:002
2684899120,2cyclictest0-21swapper/223:50:332
2684899120,2cyclictest0-21swapper/222:16:262
2684899120,2cyclictest0-21swapper/222:01:042
2684799129,2cyclictest23-21ksoftirqd/123:14:131
2684799129,2cyclictest221ktimersoftd/123:30:131
2684799129,2cyclictest0-21swapper/122:55:111
2684799127,3cyclictest11930-21cut19:50:291
2684799127,3cyclictest0-21swapper/120:25:131
2684799127,3cyclictest0-21swapper/120:05:141
2684799122,6cyclictest0-21swapper/119:26:011
2684799121,9cyclictest29066-21munin-node1
2684799121,9cyclictest0-21swapper/123:56:021
2684799121,9cyclictest0-21swapper/120:10:171
2684799121,3cyclictest23145-21head00:10:121
2684799121,3cyclictest18220-21munin-node1
2684799121,3cyclictest0-21swapper/120:20:141
2684799121,10cyclictest21-21rcuc/121:43:361
26847991210,1cyclictest0-21swapper/123:15:181
2684799120,2cyclictest0-21swapper/123:20:111
2684799120,2cyclictest0-21swapper/100:36:001
2684799120,11cyclictest0-21swapper/121:28:131
2684799120,11cyclictest0-21swapper/120:45:391
2684799120,11cyclictest0-21swapper/100:00:171
2684799120,10cyclictest10691-21diskmemload23:35:031
2684799120,10cyclictest0-21swapper/122:31:031
2684699129,2cyclictest0-21swapper/022:45:410
2684699129,2cyclictest0-21swapper/022:45:400
2684699129,2cyclictest0-21swapper/021:57:510
2684699129,2cyclictest0-21swapper/019:56:020
2684699128,3cyclictest17375-21df_inode20:05:140
2684699122,7cyclictest4556-21metacity21:04:580
2684699122,7cyclictest0-21swapper/020:28:580
2684699120,2cyclictest31862-21ssh00:19:210
2684699120,2cyclictest0-21swapper/023:40:310
2684699120,2cyclictest0-21swapper/020:20:120
2684699120,11cyclictest21384-21ssh21:20:180
2684699120,11cyclictest0-21swapper/023:52:230
2684699120,11cyclictest0-21swapper/023:30:050
2684699120,11cyclictest0-21swapper/023:05:140
2684699120,11cyclictest0-21swapper/022:30:540
2684699120,11cyclictest0-21swapper/022:17:560
2684699120,11cyclictest0-21swapper/021:35:110
2684699120,11cyclictest0-21swapper/019:20:100
183792120,2sleep318636-21ssh00:05:153
2684999118,2cyclictest0-21swapper/321:05:143
2684999118,2cyclictest0-21swapper/320:00:513
2684999118,2cyclictest0-21swapper/319:52:403
2684999110,9cyclictest0-21swapper/320:14:593
2684899116,3cyclictest21847-21needreboot20:15:172
2684899112,6cyclictest18150-21smartctl20:06:012
2684799118,2cyclictest0-21swapper/120:55:181
2684799117,2cyclictest0-21swapper/121:55:151
2684799113,6cyclictest221ktimersoftd/123:45:051
2684799111,3cyclictest0-21swapper/120:40:131
2684799111,2cyclictest14978-21apt-get20:00:111
2684799110,9cyclictest0-21swapper/119:40:001
2684799110,9cyclictest0-21swapper/119:30:101
2684799110,3cyclictest0-21swapper/121:50:171
2684699111,7cyclictest0-21swapper/019:52:580
2684699110,2cyclictest0-21swapper/020:11:390
234262110,7sleep30-21swapper/320:20:143
2684999107,2cyclictest0-21swapper/319:20:103
2684999105,3cyclictest7826-21proc_pri21:00:183
2684999105,3cyclictest3581-21grep20:50:173
2684999101,5cyclictest0-21swapper/320:00:003
2684899108,1cyclictest0-21swapper/219:55:122
2684899100,2cyclictest0-21swapper/221:00:052
2684799105,4cyclictest21-21rcuc/123:25:081
2684799100,8cyclictest0-21swapper/100:25:031
2684699107,2cyclictest0-21swapper/019:35:130
268499996,2cyclictest0-21swapper/319:15:133
268499994,3cyclictest8140-21wc19:41:023
268499994,3cyclictest705-21aten2.4-expect19:25:133
268499994,3cyclictest27320-21expr19:10:123
268499994,3cyclictest17198-21cpuspeed_turbos20:05:123
268489996,2cyclictest27187-21kworker/u8:220:42:052
268479994,3cyclictest31395-21cpuspeed_turbos19:20:111
268469996,2cyclictest0-21swapper/021:07:420
268469996,2cyclictest0-21swapper/020:17:040
268469994,3cyclictest1010-21df19:25:130
268469992,5cyclictest0-21swapper/019:45:370
268469990,8cyclictest0-21swapper/020:53:390
268469990,8cyclictest0-21swapper/020:37:060
268469990,2cyclictest0-21swapper/020:45:150
268469990,2cyclictest0-21swapper/019:30:090
268499984,2cyclictest8968-21telnet19:45:123
268499983,3cyclictest29196-21aten2.4_r3power20:35:113
268499983,3cyclictest25624-21http_loadtime20:25:153
268499983,3cyclictest1204-21tr20:45:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional