You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 07:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Thu Mar 19, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
735859920300,2026cyclictest4320902sleep323:19:563
735859919071,1902cyclictest1590182sleep320:14:573
73585993760,373cyclictest223385-21turbostat21:09:553
73585993330,330cyclictest533537-21turbostat00:29:563
19692821050,1sleep20-21swapper/220:40:222
7358599171,12cyclictest223352-21taskset21:04:573
7358599166,8cyclictest191217-21turbostat20:44:563
7358599166,7cyclictest1644852sleep320:15:223
73585991613,1cyclictest0-21swapper/320:23:053
7358599161,12cyclictest81229-21taskset19:15:023
7358599161,11cyclictest244138-21taskset21:16:283
7358599155,8cyclictest463544-21taskset23:40:013
7358599155,8cyclictest463544-21taskset23:40:013
7358599155,7cyclictest260899-21turbostat21:30:003
7358599155,7cyclictest216875-21taskset20:59:443
7358599155,7cyclictest191019-21taskset20:37:543
7358599155,7cyclictest139661-21taskset19:57:403
73585991513,0cyclictest0-21swapper/323:08:433
73585991513,0cyclictest0-21swapper/322:50:563
73585991513,0cyclictest0-21swapper/300:03:233
73585991512,1cyclictest0-21swapper/323:57:283
73585991512,1cyclictest0-21swapper/323:33:493
73585991512,1cyclictest0-21swapper/321:36:313
73585991512,1cyclictest0-21swapper/300:06:103
7358599151,11cyclictest95001-21taskset19:25:013
7358599151,11cyclictest416852-21taskset23:10:013
7358599151,11cyclictest360827-21taskset22:31:233
7358599151,11cyclictest331306-21taskset22:15:113
7358599151,11cyclictest317261-21taskset22:05:163
7358599151,11cyclictest120596-21taskset19:44:573
7358599151,10cyclictest432491-21taskset23:20:123
7358599151,10cyclictest133072-21taskset19:50:273
7358599150,12cyclictest120853-21taskset19:45:013
7358599150,12cyclictest120853-21taskset19:45:013
7358599150,11cyclictest527534-21taskset00:20:163
7358599145,7cyclictest146098-21taskset20:03:023
7358599145,6cyclictest2360312sleep321:10:313
7358599144,7cyclictest5181682sleep300:15:123
7358599141,9cyclictest4423112sleep323:25:193
73585991412,0cyclictest0-21swapper/321:22:213
7358599141,11cyclictest370509-21taskset22:40:143
7358599141,11cyclictest329450-21taskset22:10:343
7358599141,11cyclictest283297-21taskset21:42:183
73585991411,1cyclictest0-21swapper/322:25:253
7358599141,10cyclictest79649-21taskset19:11:213
7358599141,10cyclictest549846-21taskset00:35:163
7358599141,10cyclictest407286-21taskset23:00:313
7358599141,10cyclictest381975-21taskset22:45:193
7358599141,10cyclictest368846-21taskset22:37:093
7358599141,10cyclictest345154-21taskset22:21:013
7358599141,10cyclictest313266-21taskset22:00:193
7358599141,10cyclictest298385-21taskset21:50:413
7358599141,10cyclictest267230-21taskset21:30:433
7358599141,10cyclictest210129-21taskset20:50:453
7358599141,10cyclictest199414-21taskset20:45:163
7358599141,10cyclictest152293-21taskset20:05:383
7358599141,10cyclictest106199-21taskset19:30:183
7358599141,10cyclictest101096-21taskset19:26:413
7358599140,11cyclictest300116-21taskset21:55:023
4869292141,10sleep2486924-21apt-get23:55:132
3158152141,4sleep2315812-21apt-get22:05:122
7358599134,7cyclictest5103202sleep300:10:103
7358599131,9cyclictest547515-21taskset00:30:273
7358599131,9cyclictest484417-21taskset23:50:203
7358599131,9cyclictest393455-21chrt22:55:063
73585991310,1cyclictest0-21swapper/323:36:003
73585991310,1cyclictest0-21swapper/321:47:583
7358599130,11cyclictest477338-21taskset23:45:393
7358599130,11cyclictest477338-21taskset23:45:393
7358599130,10cyclictest178644-21taskset20:30:043
7358599130,10cyclictest178042-21taskset20:25:353
7358599130,10cyclictest113423-21taskset19:35:233
5324652131,4sleep1532469-21ssh00:22:031
3055062120,8sleep2305555-21ntpq21:55:222
2378672122,7sleep2237901-21apt-get21:15:112
2240562121,4sleep2224873-21irqrtprio21:05:172
1719482120,9sleep0172063-21latency_hist20:25:010
5288822110,7sleep0530836-21cut00:20:180
5109082111,2sleep00-21swapper/000:10:150
3534412111,7sleep1353447-21ssh22:27:331
3185532111,7sleep00-21swapper/022:05:180
890532100,6sleep189253-21cat19:20:141
3468442101,4sleep2346841-21apt-get22:25:142
3149902101,6sleep0314995-21sshd-session22:03:340
2374142100,4sleep0237420-21ssh21:14:370
2159132101,5sleep1215963-21ntpq20:55:221
1590222101,5sleep2159042-21apt20:15:012
1068002101,5sleep1106842-21ntpq19:30:211
67383291,3sleep20-21swapper/219:05:122
534293291,5sleep0535260-21sed00:25:150
437581291,5sleep2437721-21ntpq23:20:202
409170291,4sleep0409332-21diskstats23:05:140
390993291,4sleep0391034-21ntpq22:50:220
386309290,6sleep2387244-21kthreadcore22:50:162
385668291,4sleep1385703-21apt-get22:50:111
359861291,4sleep0359908-21ntpq22:30:210
349842291,2sleep0351832-21kthreadcore22:25:200
268206290,1sleep20-21swapper/221:33:362
235395291,5sleep2235445-21ntpq21:10:222
190355290,6sleep1190458-21grep20:35:251
138973290,5sleep1139006-21ntpq19:55:231
79934281,4sleep180288-21ntp_states19:15:011
554699281,3sleep1554752-21ntpq00:35:231
541642280,5sleep0541654-21grep00:30:130
525112280,4sleep2525061-21apt-get00:18:152
484369280,4sleep2484413-21ntpq23:50:202
476687280,5sleep1476735-21ntpq23:45:221
476687280,5sleep1476735-21ntpq23:45:221
458499280,5sleep2460455-21kthreadcore23:35:202
437777280,4sleep0437834-21ntp_states23:20:220
430012281,4sleep1430063-21ntpq23:15:221
424718281,4sleep0424873-21df_inode23:15:130
424670281,4sleep2424707-21apt-get23:15:112
416946281,3sleep1416943-21apt-get23:10:131
393484281,3sleep0393523-21apt-get22:55:120
382980280,4sleep0383126-21ntpq22:45:210
328819280,4sleep2328872-21ntpq22:10:222
315329280,6sleep1315335-21ssh22:04:391
305358281,3sleep0305450-21ntpq21:55:200
203996281,4sleep2204067-21munin-run20:50:012
196840280,6sleep1196925-21switchtime20:40:221
191465281,3sleep0191498-21apt-get20:40:090
164529280,5sleep0164579-21ntpq20:15:230
146554281,4sleep10-21swapper/120:05:131
134092281,4sleep2134637-21irqcore19:55:162
126196281,4sleep1126322-21sensors_temp19:45:241
126196281,4sleep1126322-21sensors_temp19:45:241
114913281,5sleep2115716-21irqrtprio19:40:162
101631281,3sleep0101653-21apt-get19:30:120
94127271,4sleep2635-21dbus-daemon19:20:232
67419270,4sleep30-21swapper/319:05:133
555728271,3sleep2229784-21diskmemload00:36:522
541812270,1sleep20-21swapper/200:30:152
526089271,3sleep20-21swapper/200:20:122
523503270,3sleep1523549-21ntpq00:15:221
507985271,3sleep1508106-21python300:05:231
494285271,3sleep1494291-21ssh23:59:501
471367272,3sleep2471402-21apt-get23:45:112
471367272,3sleep2471402-21apt-get23:45:112
463968270,5sleep1464104-21ps23:40:161
463968270,5sleep1464104-21ps23:40:161
444195271,3sleep0445345-21memory23:25:200
423524270,2sleep2423530-21ssh23:12:342
407846270,1sleep20-21swapper/223:02:112
400420270,1sleep20-21swapper/222:58:262
384150271,4sleep1384154-21ssh22:46:341
378009270,5sleep2378111-21/usr/sbin/munin22:45:152
362564270,4sleep0362769-21hddtemp_smartct22:35:140
354797270,5sleep1354877-21fschecks_time22:30:141
336706271,4sleep2336835-21ssh22:15:232
331259270,3sleep1331296-21apt-get22:15:101
305457271,3sleep1305497-21ntpq21:55:211
299773270,4sleep0299779-21ssh21:54:480
292323270,1sleep20-21swapper/221:50:092
283509270,1sleep20-21swapper/221:42:552
277782271,3sleep10-21swapper/121:40:171
275284271,4sleep10-21swapper/121:36:361
262673270,4sleep0263792-21grep21:30:170
259018270,4sleep1259284-21sed21:25:251
241817270,4sleep1237866-21/usr/sbin/munin21:15:181
218773272,3sleep00-21swapper/021:00:180
217580271,3sleep20-21swapper/221:00:162
216220270,5sleep2216477-21unixbench_multi20:55:272
183903270,4sleep2183945-21ntpq20:30:202
176557270,4sleep2177297-21missed_timers20:25:192
141587270,5sleep0143042-21cut20:00:190
108052270,4sleep0108202-21df_inode19:35:130
98003260,4sleep0100086-21kthreadcore19:25:190
90931261,2sleep00-21swapper/019:20:170
84554261,2sleep20-21swapper/219:15:182
82341260,2sleep082373-21apt-get19:15:120
69345261,2sleep00-21swapper/019:05:180
543245261,2sleep1545073-21kthreadcore00:30:181
541277260,1sleep20-21swapper/200:29:592
539128260,3sleep1539173-21ntpq00:25:201
519615260,4sleep0521278-21kthreadcore00:15:170
513685260,4sleep2515393-21expr00:10:172
511885260,4sleep1513683-21cut00:10:161
508107260,3sleep20-21swapper/200:05:242
499282261,1sleep00-21swapper/000:00:210
494727260,1sleep20-21swapper/200:00:032
485963260,3sleep0485969-21ssh23:53:150
485671260,3sleep1485677-21ssh23:52:201
476916261,2sleep00-21swapper/023:45:250
476916261,2sleep00-21swapper/023:45:250
470589260,1sleep00-21swapper/023:43:360
470589260,1sleep00-21swapper/023:43:360
468846260,3sleep2468905-21ssh23:40:212
468846260,3sleep2468905-21ssh23:40:212
462669260,1sleep10-21swapper/123:38:161
455793260,1sleep00-21swapper/023:35:100
447220260,3sleep2447224-21ssh23:28:342
416203260,1sleep10-21swapper/123:08:561
409121260,3sleep20-21swapper/223:05:122
408651260,1sleep1408657-21ssh23:04:351
401818260,3sleep0402809-21irqstats23:00:170
393970260,3sleep1394972-21irqstats22:55:171
370078260,1sleep20-21swapper/222:40:062
362349260,3sleep1362382-21apt-get22:35:111
346357260,3sleep1346363-21ssh22:24:411
346302260,1sleep00-21swapper/022:24:230
339052260,1sleep20-21swapper/222:20:082
330941260,1sleep10-21swapper/122:14:581
323936261,2sleep00-21swapper/022:10:160
315773260,3sleep1315811-21ssh22:05:111
297516260,3sleep1297660-21ntpq21:50:201
290524260,1sleep10-21swapper/121:45:381
284541260,3sleep2284547-21ssh21:45:112
276058260,1sleep2276064-21ssh21:38:542
268631260,3sleep1268637-21ssh21:34:481
258781260,4sleep2258820-21ntpq21:25:222
250824260,3sleep2250969-21ntpq21:20:212
245615261,3sleep0245620-21diskmemload21:20:030
244405260,1sleep00-21swapper/021:17:180
230139260,3sleep1230145-21ssh21:10:131
226065260,4sleep0228099-21kthreadcore21:05:190
223852260,3sleep1224055-21basename21:05:161
210396260,1sleep20-21swapper/220:54:142
209357260,3sleep1209497-21ntpq20:50:201
198064260,3sleep0198218-21/usr/sbin/munin20:45:140
190321260,3sleep0190349-21ntpq20:35:240
180109260,3sleep1181751-21ps20:30:151
177476260,3sleep1177523-21ntp_states20:25:221
168543261,2sleep20-21swapper/220:20:182
155715261,2sleep10-21swapper/120:10:191
154354260,4sleep0155714-21cut20:10:180
151747260,3sleep2151786-21ntp_states20:05:222
145366260,3sleep1145409-21ntp_states20:00:241
135594260,3sleep0137547-21grep19:55:180
129793261,2sleep20-21swapper/219:50:192
120865261,2sleep0120898-21apt-get19:45:110
120865261,2sleep0120898-21apt-get19:45:110
120489260,1sleep10-21swapper/119:43:361
116905261,2sleep00-21swapper/019:40:180
113344261,3sleep2113421-21ps19:35:232
95249250,2sleep20-21swapper/219:25:132
73947250,3sleep274067-21grep19:10:132
73790251,1sleep00-21swapper/019:10:110
507938251,2sleep00-21swapper/000:05:220
494828250,2sleep10-21swapper/100:00:141
449457251,2sleep10-21swapper/123:30:181
448501250,2sleep0449256-21ssh23:30:170
447966250,3sleep20-21swapper/223:30:032
440249250,2sleep10-21swapper/123:25:121
417227251,2sleep00-21swapper/023:10:160
375358250,3sleep0375414-21ntpq22:40:200
363355250,3sleep2364245-21kthreadcore22:35:162
359964250,2sleep20-21swapper/222:30:232
334391250,3sleep00-21swapper/022:15:170
308396251,2sleep20-21swapper/222:00:152
290543250,1sleep00-21swapper/021:45:450
283728251,2sleep00-21swapper/021:43:370
269308250,2sleep0269472-21ssh21:35:150
252528250,3sleep10-21swapper/121:23:081
210728250,2sleep00-21swapper/020:55:120
198219250,2sleep10-21swapper/120:45:151
185094250,2sleep2421rcuc/220:35:142
178686251,1sleep00-21swapper/020:30:100
167217251,2sleep10-21swapper/120:20:171
163657250,1sleep10-21swapper/120:15:201
159324251,1sleep20-21swapper/220:15:132
146553250,2sleep00-21swapper/020:05:120
140128251,1sleep20-21swapper/220:00:132
132386250,3sleep0132481-21ntpq19:50:210
128646250,3sleep1129791-21kthreadcore19:50:181
107601250,3sleep20-21swapper/219:33:072
101352250,2sleep1101483-21latency_hist19:30:011
75213240,2sleep176863-21ps19:10:161
67580240,1sleep10-21swapper/119:05:151
550811240,2sleep0551875-21cat00:35:180
492224240,2sleep0492271-21ntp_states23:55:230
438406240,2sleep1438415-21ssh23:20:351
371635240,2sleep1373436-21kthreadcore22:40:171
254233240,2sleep00-21swapper/021:25:160
172250240,1sleep00-21swapper/020:25:130
126062240,1sleep20-21swapper/219:45:212
126062240,1sleep20-21swapper/219:45:212
222472230,1sleep10-21swapper/121:00:231
204283230,1sleep00-21swapper/020:50:030
113301230,1sleep10-21swapper/119:35:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional