You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 01:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Thu Aug 20, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12992211895,18sleep00-21swapper/019:05:490
11929211594,16sleep20-21swapper/219:05:072
13469211087,18sleep30-21swapper/319:06:483
13669210089,6sleep10-21swapper/119:09:231
244872790,0sleep20-21swapper/200:23:062
274602760,1sleep1221ktimersoftd/121:05:001
7542740,0sleep20-21swapper/221:13:302
251212720,0sleep20-21swapper/222:11:032
274422700,0sleep20-21swapper/221:40:162
255242490,1sleep025522-21ssh22:11:130
1374899253,20cyclictest0-21swapper/121:25:121
1374899252,9cyclictest32269-21ssh23:25:111
13750992313,2cyclictest0-21swapper/323:25:103
1375099230,21cyclictest0-21swapper/321:25:113
13748992218,2cyclictest25590-21kworker/u8:121:24:051
1374899221,18cyclictest29293-21diskmemload21:40:111
1374899220,2cyclictest0-21swapper/100:10:121
1374799220,2cyclictest0-21swapper/022:45:130
1374899202,16cyclictest207150irq/24-eth100:00:141
13748992015,3cyclictest207150irq/24-eth100:35:121
143212190,0sleep00-21swapper/021:26:310
207992170,1sleep120800-21munin-node1
1375099162,7cyclictest0-21swapper/321:03:013
1374999161,13cyclictest0-21swapper/200:26:232
1374899161,3cyclictest0-21swapper/123:56:401
1374899161,3cyclictest0-21swapper/123:00:431
1374899161,3cyclictest0-21swapper/120:20:151
13748991611,4cyclictest0-21swapper/123:54:291
1374899161,13cyclictest29628-21apt-get21:10:121
1374899161,13cyclictest18077-21sendmail-mta23:43:161
1374899161,13cyclictest1192-21sendmail-mta22:54:391
1374799160,14cyclictest5806-21ntp_states23:30:150
1374999151,3cyclictest5943-21ssh21:18:402
13749991510,3cyclictest29293-21diskmemload22:55:132
1374899152,7cyclictest0-21swapper/120:35:001
13748991512,2cyclictest0-21swapper/123:10:351
1374899151,12cyclictest17154-21sh23:09:521
13748991510,3cyclictest18974-21chrt21:31:091
13748991510,3cyclictest0-21swapper/122:00:131
1374899150,13cyclictest0-21swapper/122:47:231
1374799150,13cyclictest0-21swapper/023:41:350
1374799150,13cyclictest0-21swapper/000:00:150
312752140,1sleep2301ktimersoftd/222:17:332
312752140,1sleep2301ktimersoftd/222:17:322
273732140,2sleep10-21swapper/122:14:041
161242140,2sleep029293-21diskmemload22:01:480
1375099141,11cyclictest20025-21ssh00:18:463
1375099140,2cyclictest0-21swapper/321:16:013
1375099140,12cyclictest0-21swapper/321:30:133
1374999141,12cyclictest14341-21apt-get23:40:042
1374999141,11cyclictest7479-21sendmail-mta21:54:382
1374999141,11cyclictest0-21swapper/221:32:312
1374999140,2cyclictest0-21swapper/200:36:082
1374999140,12cyclictest31413-21ssh23:24:122
1374999140,12cyclictest22664-21ssh23:47:422
1374899142,7cyclictest1183-21copy20:00:001
1374899141,3cyclictest5600-21ssh21:50:531
13748991411,2cyclictest0-21swapper/121:55:191
13748991411,2cyclictest0-21swapper/100:32:201
13748991410,2cyclictest0-21swapper/123:20:111
1374899140,3cyclictest5502-21ssh22:57:021
1374899140,12cyclictest24534-21ssh21:37:171
1374899140,12cyclictest0-21swapper/123:37:431
1374899140,12cyclictest0-21swapper/119:15:111
13747991411,2cyclictest0-21swapper/023:55:550
13747991411,2cyclictest0-21swapper/021:54:450
13747991411,2cyclictest0-21swapper/000:37:160
13747991411,2cyclictest0-21swapper/000:11:160
1374799140,2cyclictest23404-21ssh22:42:250
1374799140,2cyclictest0-21swapper/022:36:150
1374799140,13cyclictest12122-21ssh22:30:460
1374799140,13cyclictest0-21swapper/023:04:250
1374799140,13cyclictest0-21swapper/022:54:050
1374799140,12cyclictest0-21swapper/023:28:340
1374799140,12cyclictest0-21swapper/021:41:480
57572131,3sleep15501-21apt-get22:25:051
52532130,1sleep00-21swapper/021:17:380
16902130,2sleep20-21swapper/221:47:032
13750991310,2cyclictest0-21swapper/300:38:213
13750991310,2cyclictest0-21swapper/300:30:563
13750991310,2cyclictest0-21swapper/300:26:283
13750991310,2cyclictest0-21swapper/300:24:593
1375099130,11cyclictest0-21swapper/321:10:133
1374999131,10cyclictest16640-21awk22:35:192
13749991310,2cyclictest0-21swapper/223:26:042
13749991310,2cyclictest0-21swapper/223:03:342
13749991310,2cyclictest0-21swapper/222:06:202
13749991310,2cyclictest0-21swapper/222:04:102
13749991310,2cyclictest0-21swapper/221:39:542
13749991310,2cyclictest0-21swapper/220:45:102
13749991310,2cyclictest0-21swapper/200:07:122
1374999130,2cyclictest20766-21ssh23:12:542
1374999130,2cyclictest19107-21ssh00:17:222
1374999130,2cyclictest0-21swapper/223:57:432
1374999130,2cyclictest0-21swapper/222:33:322
1374999130,12cyclictest29293-21diskmemload23:07:042
1374899132,6cyclictest0-21swapper/122:40:001
1374899131,3cyclictest0-21swapper/119:51:001
1374899131,10cyclictest0-21swapper/123:30:111
13748991310,2cyclictest1998-21kworker/u8:022:22:011
13748991310,2cyclictest0-21swapper/122:44:431
13748991310,2cyclictest0-21swapper/121:05:171
1374899130,2cyclictest0-21swapper/121:47:101
1374899130,2cyclictest0-21swapper/100:16:091
1374899130,1cyclictest0-21swapper/120:25:191
1374899130,12cyclictest0-21swapper/123:16:581
1374899130,12cyclictest0-21swapper/121:15:091
1374899130,12cyclictest0-21swapper/119:11:031
1374899130,12cyclictest0-21swapper/100:09:391
13747991310,2cyclictest12-21rcu_preempt21:10:130
13747991310,2cyclictest0-21swapper/021:47:090
13747991310,2cyclictest0-21swapper/021:01:020
13747991310,2cyclictest0-21swapper/000:30:140
13747991310,2cyclictest0-21swapper/000:15:120
13747991310,2cyclictest0-21swapper/000:08:570
1374799130,2cyclictest0-21swapper/023:52:550
1374799130,2cyclictest0-21swapper/023:22:260
1374799130,2cyclictest0-21swapper/021:56:370
1374799130,2cyclictest0-21swapper/020:15:170
1374799130,2cyclictest0-21swapper/019:40:140
1374799130,12cyclictest29293-21diskmemload22:25:000
1374799130,12cyclictest0-21swapper/023:17:060
1374799130,12cyclictest0-21swapper/023:10:390
1374799130,12cyclictest0-21swapper/022:59:260
1374799130,12cyclictest0-21swapper/022:15:460
1374799130,12cyclictest0-21swapper/022:15:460
1374799130,12cyclictest0-21swapper/021:33:350
1374799130,12cyclictest0-21swapper/020:55:140
1374799130,11cyclictest0-21swapper/023:05:470
1374799130,11cyclictest0-21swapper/000:22:160
100002130,2sleep20-21swapper/222:29:232
275202120,2sleep20-21swapper/223:52:402
1375099129,2cyclictest0-21swapper/323:52:503
1375099129,2cyclictest0-21swapper/323:35:413
1375099129,2cyclictest0-21swapper/323:01:203
1375099129,2cyclictest0-21swapper/322:56:003
1375099121,9cyclictest24582-21fschecks_time19:35:133
1375099120,2cyclictest0-21swapper/323:45:103
1375099120,2cyclictest0-21swapper/323:22:213
1375099120,2cyclictest0-21swapper/323:16:123
1375099120,2cyclictest0-21swapper/300:03:013
1375099120,11cyclictest0-21swapper/323:12:103
1375099120,11cyclictest0-21swapper/323:06:313
1374999129,2cyclictest0-21swapper/221:28:132
1374999129,2cyclictest0-21swapper/219:55:122
1374999128,2cyclictest0-21swapper/220:15:122
1374999121,9cyclictest0-21swapper/200:30:122
1374999120,2cyclictest0-21swapper/223:39:442
1374999120,2cyclictest0-21swapper/223:18:012
1374999120,2cyclictest0-21swapper/220:11:042
1374999120,1cyclictest0-21swapper/220:30:112
1374999120,11cyclictest0-21swapper/223:30:322
1374999120,11cyclictest0-21swapper/220:53:102
1374999120,11cyclictest0-21swapper/219:16:042
1374999120,10cyclictest0-21swapper/222:20:532
1374899129,2cyclictest0-21swapper/120:45:131
1374899129,2cyclictest0-21swapper/120:05:131
1374899128,2cyclictest0-21swapper/122:15:161
1374899128,2cyclictest0-21swapper/122:15:161
1374899128,2cyclictest0-21swapper/100:25:121
1374899121,9cyclictest0-21swapper/120:00:051
1374899121,3cyclictest21844-21df_inode20:50:121
1374899121,3cyclictest20035-21munin-node1
1374899121,3cyclictest18345-21grep19:20:141
1374899121,3cyclictest17440-21munin-node1
1374899121,3cyclictest16704-21sed20:36:011
1374899121,3cyclictest0-21swapper/120:15:171
1374899120,2cyclictest0-21swapper/122:06:521
1374899120,2cyclictest0-21swapper/119:45:291
1374899120,2cyclictest0-21swapper/100:20:111
1374899120,11cyclictest0-21swapper/122:30:051
1374799129,2cyclictest0-21swapper/022:05:250
1374799129,2cyclictest0-21swapper/021:37:460
1374799129,2cyclictest0-21swapper/020:05:160
1374799120,2cyclictest27224-21ssh00:25:260
1374799120,2cyclictest0-21swapper/020:30:120
1374799120,2cyclictest0-21swapper/020:10:160
1374799120,2cyclictest0-21swapper/019:55:180
1374799120,11cyclictest0-21swapper/023:46:000
1374799120,11cyclictest0-21swapper/023:35:260
1374799120,11cyclictest0-21swapper/022:23:090
1374799120,11cyclictest0-21swapper/021:20:220
1374799120,11cyclictest0-21swapper/020:50:170
99192110,1sleep3381ktimersoftd/300:08:163
233602110,7sleep323372-21ssh22:10:053
1375099118,2cyclictest0-21swapper/323:56:523
1375099118,2cyclictest0-21swapper/323:42:023
1375099118,2cyclictest0-21swapper/323:31:513
1375099118,2cyclictest0-21swapper/321:23:523
1375099118,2cyclictest0-21swapper/320:10:163
1375099118,2cyclictest0-21swapper/319:50:133
1375099118,2cyclictest0-21swapper/319:45:113
1375099116,3cyclictest0-21swapper/322:45:033
1375099116,3cyclictest0-21swapper/321:35:153
1375099112,7cyclictest0-21swapper/319:11:583
1375099112,6cyclictest0-21swapper/320:05:023
1375099111,5cyclictest0-21swapper/320:55:003
1375099110,9cyclictest0-21swapper/322:27:573
1375099110,9cyclictest0-21swapper/321:44:363
1375099110,2cyclictest0-21swapper/319:26:043
1375099110,10cyclictest0-21swapper/322:50:313
1375099110,10cyclictest0-21swapper/300:12:203
1374999118,2cyclictest0-21swapper/222:51:432
1374999118,2cyclictest0-21swapper/222:47:322
1374999118,2cyclictest0-21swapper/220:05:162
1374999118,2cyclictest0-21swapper/219:13:402
1374999116,3cyclictest27151-21taskset19:40:542
1374999110,3cyclictest11688-21sh21:24:592
1374999110,10cyclictest0-21swapper/220:24:032
1374899117,2cyclictest0-21swapper/119:30:141
1374899111,8cyclictest5346-21munin-node1
1374899111,3cyclictest0-21swapper/120:55:141
1374799118,2cyclictest0-21swapper/020:23:570
1374799118,2cyclictest0-21swapper/019:45:150
1374799118,2cyclictest0-21swapper/019:36:000
1374799118,2cyclictest0-21swapper/019:10:160
257452100,2sleep20-21swapper/219:38:512
24362100,1sleep00-21swapper/020:00:450
205622100,2sleep30-21swapper/322:06:073
177442100,2sleep00-21swapper/019:19:300
168112100,2sleep30-21swapper/322:02:503
1375099107,2cyclictest28138-21ethtool21:05:153
1375099106,2cyclictest0-21swapper/319:30:113
1375099105,3cyclictest20757-21grep22:40:113
1375099105,3cyclictest17203-21cron20:39:593
1375099105,3cyclictest13667-21tr20:30:123
1375099103,6cyclictest0-21swapper/322:31:023
1375099101,5cyclictest0-21swapper/320:20:003
1375099100,9cyclictest0-21swapper/321:57:223
1375099100,8cyclictest31119-21ssh22:17:183
1375099100,8cyclictest31119-21ssh22:17:173
1374999107,2cyclictest0-21swapper/219:45:002
1374999107,2cyclictest0-21swapper/200:01:072
1374999105,3cyclictest29293-21diskmemload21:10:002
1374999105,3cyclictest26365-21ntpq21:00:172
1374999105,3cyclictest25099-21ls22:45:012
1374999105,3cyclictest23878-21grep20:55:152
1374999105,3cyclictest20606-21tr19:25:162
1374999101,7cyclictest9284-21cut21:55:152
1374999101,7cyclictest23834-21sh19:35:012
1374999100,9cyclictest0-21swapper/220:43:212
1374899105,3cyclictest24950-21ntp_states19:35:171
1374799107,2cyclictest16145-21sed20:35:160
1374799107,2cyclictest0-21swapper/019:50:150
137509995,3cyclictest6090-21users21:51:043
137509995,2cyclictest20295-21needreboot20:45:153
137509994,3cyclictest2096-21ssh22:20:403
137509994,3cyclictest18146-21cut20:40:173
137509994,3cyclictest16269-21http_loadtime22:35:163
137509994,3cyclictest10134-21cat20:20:153
137509993,4cyclictest29293-21diskmemload21:45:073
137509992,6cyclictest0-21swapper/320:29:583
137509992,6cyclictest0-21swapper/320:01:063
137509992,5cyclictest0-21swapper/319:57:013
137509990,8cyclictest0-21swapper/319:23:543
137499996,2cyclictest0-21swapper/220:04:322
137499994,3cyclictest31329-21timerandwakeup19:51:012
137499994,3cyclictest18170-21aten2.4_r3power19:20:122
137499994,3cyclictest169142sleep220:36:202
137499994,3cyclictest12155-21latency20:25:172
137499991,4cyclictest11895-21munin-node2
137489994,3cyclictest26331-21tr19:40:111
137479996,2cyclictest0-21swapper/019:35:010
137479991,6cyclictest21491-21grep19:26:050
137479990,8cyclictest0-21swapper/021:07:590
137509984,2cyclictest23628-21cpu20:55:133
137509983,3cyclictest26921-21cut19:40:163
137509981,5cyclictest16090-21aten2.4-expect19:15:123
137479983,3cyclictest20176-21cat20:45:150
137479980,7cyclictest0-21swapper/020:43:180
137479980,7cyclictest0-21swapper/020:26:240
137479980,2cyclictest0-21swapper/019:24:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional