You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 06:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot3.osadl.org (updated Fri Nov 21, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
238462499130,12cyclictest2518728-21taskset00:35:013
238462499130,12cyclictest0-21swapper/323:15:123
238462499129,1cyclictest0-21swapper/319:10:133
238462499129,1cyclictest0-21swapper/300:15:153
2384624991211,0cyclictest0-21swapper/321:35:013
238462499121,0cyclictest0-21swapper/323:25:123
238462499120,1cyclictest0-21swapper/300:00:143
238462499120,11cyclictest69450irq/121-eno122:55:213
238462499120,11cyclictest0-21swapper/321:45:203
238462499120,11cyclictest0-21swapper/321:25:123
238462499120,11cyclictest0-21swapper/321:20:143
238462499120,11cyclictest0-21swapper/321:10:013
238462499120,11cyclictest0-21swapper/320:55:183
238462499120,11cyclictest0-21swapper/320:45:153
238462499120,11cyclictest0-21swapper/320:25:193
238462499120,11cyclictest0-21swapper/320:05:123
238462499120,11cyclictest0-21swapper/300:00:003
238462499110,10cyclictest69450irq/121-eno123:10:203
238462499110,10cyclictest69450irq/121-eno122:50:033
238462499110,10cyclictest69450irq/121-eno100:20:143
238462499110,10cyclictest69450irq/121-eno100:13:093
238462499110,10cyclictest0-21swapper/323:20:223
238462499110,10cyclictest0-21swapper/322:30:133
238462499110,10cyclictest0-21swapper/320:05:013
238462499110,10cyclictest0-21swapper/319:20:003
238462499110,10cyclictest0-21swapper/300:05:123
238462499110,0cyclictest0-21swapper/321:55:143
238462499110,0cyclictest0-21swapper/321:10:193
238462499110,0cyclictest0-21swapper/320:10:253
238462499110,0cyclictest0-21swapper/300:25:133
238462499109,0cyclictest0-21swapper/320:30:183
238462499109,0cyclictest0-21swapper/319:45:113
2384624991010,0cyclictest69450irq/121-eno123:30:123
2384624991010,0cyclictest69450irq/121-eno122:19:423
2384624991010,0cyclictest69450irq/121-eno121:15:133
2384624991010,0cyclictest69450irq/121-eno121:00:143
2384624991010,0cyclictest0-21swapper/323:45:123
2384624991010,0cyclictest0-21swapper/323:40:183
2384624991010,0cyclictest0-21swapper/321:51:123
2384624991010,0cyclictest0-21swapper/321:40:113
2384624991010,0cyclictest0-21swapper/320:40:193
2384624991010,0cyclictest0-21swapper/319:50:013
238462499100,9cyclictest69450irq/121-eno122:40:153
238462499100,9cyclictest69450irq/121-eno122:35:253
238462499100,9cyclictest69450irq/121-eno122:05:203
238462499100,9cyclictest69450irq/121-eno119:30:143
238462499100,9cyclictest69450irq/121-eno119:20:163
238462499100,9cyclictest69450irq/121-eno100:35:183
238462499100,9cyclictest0-21swapper/322:00:183
23846249999,0cyclictest69450irq/121-eno123:08:583
23846249999,0cyclictest69450irq/121-eno122:12:403
23846249999,0cyclictest0-21swapper/323:00:113
23846249999,0cyclictest0-21swapper/321:30:153
23846249988,0cyclictest69450irq/121-eno122:48:423
23846249988,0cyclictest0-21swapper/320:15:213
23846249986,0cyclictest0-21swapper/319:40:153
23846249980,7cyclictest0-21swapper/323:40:013
23846249977,0cyclictest69450irq/121-eno123:54:103
23846249977,0cyclictest69450irq/121-eno120:35:123
23846249976,0cyclictest0-21swapper/319:35:133
23846249970,6cyclictest69450irq/121-eno122:25:153
23846249970,6cyclictest69450irq/121-eno120:22:543
23846249970,6cyclictest69450irq/121-eno119:55:133
23846249966,0cyclictest0-21swapper/322:20:213
23846249954,0cyclictest0-21swapper/319:25:163
2384431253,1sleep10-21swapper/119:09:241
2382569251,2sleep30-21swapper/319:05:013
2392610243,1sleep02948619-21kworker/0:0+events19:25:520
23846249940,3cyclictest2424145-21chrt20:50:033
2384392242,1sleep20-21swapper/219:08:532
2383573241,1sleep00-21swapper/019:05:170
2493306230,1sleep22493323-21apt-get23:35:012
2489256230,1sleep12489547-21df23:25:161
2456932230,1sleep02456944-21apt-get22:10:020
2444723230,2sleep22444633-21/usr/sbin/munin21:40:142
2398703230,1sleep02398722-21apt-get19:45:000
2518702220,1sleep22518716-21munin-run00:35:012
2514103220,1sleep20-21swapper/200:21:162
2505882220,0sleep00-21swapper/000:02:370
2501420220,0sleep10-21swapper/123:50:251
2500712220,1sleep22500875-21ntpq23:50:182
2496372220,1sleep10-21swapper/123:40:171
2482818220,0sleep00-21swapper/023:10:060
2476290221,1sleep02476491-21latency_hist22:55:010
2476214220,0sleep20-21swapper/222:54:232
2472883220,1sleep00-21swapper/022:45:200
2466595220,1sleep02466749-21grep22:30:190
2465049220,1sleep00-21swapper/022:25:310
2445534220,1sleep02445705-21/usr/sbin/munin21:40:200
2444264220,1sleep22444265-21sshd21:38:542
2439259220,1sleep00-21swapper/021:25:190
2438366220,1sleep12438406-21aten_r1power_cu21:25:121
2432098220,1sleep22432129-21http21:10:122
2430980220,1sleep10-21swapper/121:05:191
2424640220,1sleep02424775-21memory20:50:170
2423286220,1sleep22423536-21users20:45:222
2411236220,0sleep00-21swapper/020:15:190
2410466220,1sleep22410482-21munin-run20:15:012
2385215220,1sleep22385299-21irqstats19:10:172
2519722210,0sleep10-21swapper/100:35:201
2518990210,0sleep00-21swapper/000:35:150
2518233210,0sleep00-21swapper/000:30:550
2517025210,0sleep10-21swapper/100:30:161
2515675210,0sleep10-21swapper/100:25:191
2514883210,0sleep00-21swapper/000:25:140
2513387210,0sleep10-21swapper/100:20:191
2512703210,0sleep00-21swapper/000:20:140
2511568210,0sleep10-21swapper/100:15:231
2511394210,0sleep00-21swapper/000:15:220
2510622210,0sleep20-21swapper/200:15:162
2508999210,0sleep20-21swapper/200:10:192
2508499210,0sleep12508529-21apt-get00:10:131
2508484210,0sleep00-21swapper/000:10:050
2507792210,0sleep00-21swapper/000:06:010
2507376210,0sleep20-21swapper/200:05:192
2506597210,0sleep10-21swapper/100:05:141
2505741210,0sleep20-21swapper/200:01:232
2504785210,0sleep10-21swapper/100:00:181
2502391210,0sleep00-21swapper/023:55:170
2502205210,0sleep20-21swapper/223:55:142
2501947210,0sleep10-21swapper/123:55:011
2501509210,0sleep00-21swapper/023:51:120
2498208210,0sleep20-21swapper/223:45:142
2497249210,0sleep00-21swapper/023:40:280
2495657210,0sleep20-21swapper/223:40:022
2495189210,0sleep00-21swapper/023:35:550
2495162210,0sleep10-21swapper/123:35:401
2493288210,0sleep20-21swapper/223:34:532
2491463210,0sleep10-21swapper/123:30:081
2491462210,0sleep00-21swapper/023:30:070
2490057210,0sleep20-21swapper/223:25:212
2489804210,0sleep00-21swapper/023:25:190
2488572210,0sleep00-21swapper/023:22:260
2488540210,0sleep20-21swapper/223:22:122
2487037210,0sleep10-21swapper/123:20:151
2486235210,0sleep10-21swapper/123:15:271
2485965210,0sleep20-21swapper/223:15:192
2485090210,1sleep00-21swapper/023:15:140
2484141210,0sleep20-21swapper/223:10:242
2482959210,0sleep10-21swapper/123:10:151
2481726210,0sleep10-21swapper/123:05:261
2481552210,0sleep00-21swapper/023:05:250
2480749210,1sleep20-21swapper/223:05:152
2478658210,0sleep00-21swapper/023:00:130
2477595210,0sleep10-21swapper/122:55:221
2476923210,0sleep00-21swapper/022:55:170
2476570210,0sleep20-21swapper/222:55:152
2474456210,0sleep10-21swapper/122:50:121
2473670210,0sleep10-21swapper/122:47:411
2472145210,0sleep20-21swapper/222:45:142
2471642210,0sleep00-21swapper/022:43:120
2471347210,0sleep20-21swapper/222:40:382
2471323210,1sleep12471325-21ssh22:40:251
2469517210,1sleep00-21swapper/022:38:040
2468324210,0sleep10-21swapper/122:35:161
2467653210,0sleep20-21swapper/222:35:002
2467169210,0sleep10-21swapper/122:30:481
2465816210,0sleep20-21swapper/222:30:132
2465541210,0sleep10-21swapper/122:29:521
2465039210,0sleep20-21swapper/222:25:252
2463238210,0sleep20-21swapper/222:22:572
2462088210,0sleep00-21swapper/022:20:180
2461650210,0sleep10-21swapper/122:20:151
2461269210,0sleep20-21swapper/222:19:082
2460265210,0sleep10-21swapper/122:15:191
2460146210,0sleep00-21swapper/022:15:180
2458996210,0sleep20-21swapper/222:12:322
2457436210,0sleep10-21swapper/122:10:071
2456071210,0sleep22456363-21grep22:05:192
2455919210,0sleep10-21swapper/122:05:181
2455169210,0sleep00-21swapper/022:05:130
2453030210,0sleep00-21swapper/022:00:130
2452734210,0sleep10-21swapper/122:00:021
2451326210,0sleep00-21swapper/021:55:170
2450929210,0sleep20-21swapper/221:55:152
2450644210,0sleep10-21swapper/121:55:021
2448950210,1sleep00-21swapper/021:50:170
2448809210,0sleep10-21swapper/121:50:141
2447002210,0sleep00-21swapper/021:45:160
2446751210,0sleep20-21swapper/221:45:142
2446723210,0sleep10-21swapper/121:45:131
2444632210,0sleep10-21swapper/121:40:131
2442764210,0sleep00-21swapper/021:35:160
2440365210,0sleep00-21swapper/021:30:140
2438407210,0sleep20-21swapper/221:25:132
2437600210,1sleep00-21swapper/021:22:070
2436928210,0sleep20-21swapper/221:20:212
2435794210,0sleep00-21swapper/021:19:190
2435222210,0sleep20-21swapper/221:15:192
2434490210,0sleep10-21swapper/121:15:151
2433447210,0sleep00-21swapper/021:10:500
2432360210,0sleep10-21swapper/121:10:151
2429675210,0sleep00-21swapper/021:05:020
2429608210,0sleep20-21swapper/221:04:122
2429389210,0sleep00-21swapper/021:01:250
2427998210,0sleep10-21swapper/121:00:061
2426609210,0sleep20-21swapper/220:55:232
2426544210,0sleep10-21swapper/120:55:221
2424776210,0sleep10-21swapper/120:50:181
2423897210,0sleep10-21swapper/120:49:541
2422351210,0sleep00-21swapper/020:45:160
2421131210,0sleep10-21swapper/120:40:211
2420394210,0sleep00-21swapper/020:40:160
2420322210,0sleep20-21swapper/220:40:062
2418990210,0sleep10-21swapper/120:35:171
2418840210,0sleep00-21swapper/020:35:160
2418207210,0sleep20-21swapper/220:35:012
2417906210,1sleep00-21swapper/020:31:560
2417350210,0sleep10-21swapper/120:30:251
2416933210,0sleep20-21swapper/220:30:222
2415572210,0sleep20-21swapper/220:25:212
2415429210,0sleep10-21swapper/120:25:201
2413514210,0sleep10-21swapper/120:20:201
2412382210,0sleep20-21swapper/220:20:012
2411091210,0sleep20-21swapper/220:15:172
2410455210,0sleep00-21swapper/020:14:550
2410204210,0sleep10-21swapper/120:11:421
2407321210,0sleep10-21swapper/120:05:151
2407073210,0sleep00-21swapper/020:05:140
2406887210,0sleep20-21swapper/220:05:122
2405771210,0sleep20-21swapper/220:00:202
2404975210,0sleep00-21swapper/020:00:140
2403903210,0sleep20-21swapper/219:55:192
2403041210,0sleep00-21swapper/019:55:130
2402161210,0sleep20-21swapper/219:50:192
2401986210,0sleep10-21swapper/119:50:181
2401880210,0sleep00-21swapper/019:50:170
2400876210,0sleep20-21swapper/219:49:592
2400062210,0sleep10-21swapper/119:45:181
2398513210,1sleep00-21swapper/019:42:370
2398371210,0sleep10-21swapper/119:40:501
2397093210,0sleep20-21swapper/219:40:112
2396766210,0sleep00-21swapper/019:40:000
2394157210,0sleep00-21swapper/019:30:200
2393916210,0sleep20-21swapper/219:30:182
2393186210,0sleep10-21swapper/119:30:121
2391270210,0sleep12391296-21apt-get19:25:131
2390721210,0sleep00-21swapper/019:21:110
2390217210,0sleep20-21swapper/219:20:172
2390111210,0sleep10-21swapper/119:20:161
2387708210,0sleep20-21swapper/219:15:172
2387599210,0sleep10-21swapper/119:15:161
2387392210,0sleep00-21swapper/019:15:150
2385600210,0sleep10-21swapper/119:10:201
2384823210,1sleep00-21swapper/019:10:150
2518940200,0sleep20-21swapper/200:35:042
2514764200,0sleep20-21swapper/200:25:042
2498006200,0sleep10-21swapper/123:45:051
2498005200,0sleep00-21swapper/023:45:040
2478636200,0sleep20-21swapper/223:00:032
2478635200,0sleep10-21swapper/123:00:021
2452990200,0sleep20-21swapper/222:00:032
2448791200,0sleep20-21swapper/221:50:032
2442589200,0sleep10-21swapper/121:35:051
2440347200,0sleep10-21swapper/121:30:041
2440109200,0sleep20-21swapper/221:30:012
2436120200,0sleep10-21swapper/121:20:041
2430172200,0sleep20-21swapper/221:05:042
2425825200,0sleep00-21swapper/020:55:010
2424144200,0sleep20-21swapper/220:50:032
2414553200,0sleep00-21swapper/020:25:030
2412638200,0sleep00-21swapper/020:20:030
2410702200,0sleep10-21swapper/120:15:041
2404961200,0sleep10-21swapper/120:00:031
2403026200,0sleep10-21swapper/119:55:021
2395087200,0sleep20-21swapper/219:35:022
2394855200,0sleep10-21swapper/119:35:011
2391256200,0sleep20-21swapper/219:25:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional