You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 18:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 05, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
218522831104,708sleep221998-21grep12:24:492
841727219,695sleep28386-21latency_hist11:44:182
841727219,695sleep28386-21latency_hist11:44:182
1072726698,648sleep210771-21ntpq08:24:412
30647249325,452sleep130711-21/usr/sbin/munin07:44:501
298532353161,163sleep20-21swapper/207:44:192
69582286104,57sleep20-21swapper/208:14:192
246182272112,58sleep20-21swapper/207:24:492
22205222548,75sleep222350-21unixbench_singl08:59:502
4252222460,65sleep20-21swapper/211:29:412
19960221655,149sleep219043-21/usr/sbin/munin10:34:512
15415221555,57sleep20-21swapper/212:04:462
21043220329,85sleep221089-21ntp_offset10:39:432
27653219736,124sleep227787-21unixbench_multi07:34:502
17871219211,164sleep017962-21dpkg08:49:180
12999218911,164sleep213072-21systemd-run11:59:162
12999218911,164sleep213072-21systemd-run11:59:162
20920217914,128sleep220961-21cut07:14:592
17733217231,125sleep217882-21cat12:14:192
1984217016,52sleep22015-21modprobe07:59:172
31902216815,137sleep331923-21tr07:49:373
17908216635,80sleep217952-21ntpq10:29:422
24345216312,56sleep20-21swapper/210:49:482
12252216385,20sleep20-21swapper/208:29:422
401921625,129sleep00-21swapper/008:04:400
13421216224,51sleep20-21swapper/208:34:302
12006215823,114sleep212145-21unixbench_multi10:10:212
557921577,16sleep20-21swapper/211:34:372
557921577,16sleep20-21swapper/211:34:372
4132215626,21sleep24201-21sort08:04:422
29630215513,126sleep0364-21snmpd07:42:070
13823215192,17sleep20-21swapper/211:59:452
1046521518,93sleep210481-21grep11:49:392
1046521518,93sleep210481-21grep11:49:392
1956721497,35sleep20-21swapper/212:19:322
18220214825,96sleep118232-21modprobe07:09:171
31888214713,53sleep20-21swapper/209:29:502
3948214618,101sleep34012-21fschecks_time08:04:393
27790214610,15sleep30-21swapper/307:34:513
2179214630,56sleep20-21swapper/209:39:442
26340214514,51sleep226353-21basename09:14:282
26340214514,51sleep226353-21basename09:14:282
8651214416,93sleep20-21swapper/211:44:312
8651214416,93sleep20-21swapper/211:44:312
32316214427,95sleep032464-21grep07:49:460
11815214411,120sleep011720-21/usr/sbin/munin11:54:340
11815214411,120sleep011720-21/usr/sbin/munin11:54:340
24964213910,88sleep225005-21ntpq09:09:432
9470213628,42sleep30-21swapper/310:04:283
18709213646,54sleep218773-21cat07:09:382
8782213246,47sleep20-21swapper/208:19:322
27586213219,88sleep327725-21unixbench_singl10:59:503
27586213219,88sleep327725-21unixbench_singl10:59:503
28130213126,82sleep128151-21basename11:04:161
28130213126,82sleep128151-21basename11:04:161
7103212914,100sleep27238-21grep09:54:502
9206212820,90sleep0225-21systemd-journal10:04:170
84721278,68sleep2344-21apt-get11:19:232
4722212422,82sleep34866-21unixbench_singl11:29:503
28499212410,90sleep20-21swapper/209:19:452
28499212410,90sleep20-21swapper/209:19:452
3471212116,52sleep20-21swapper/211:26:262
2771212118,88sleep20-21swapper/207:59:452
25407212027,78sleep225492-21perl10:54:372
7196211814,92sleep27242-21grep08:14:312
16423211820,46sleep30-21swapper/312:09:313
31423211726,18sleep20-21swapper/207:49:202
29213211527,67sleep129230-21munin-plugin-st09:24:161
29213211527,67sleep129230-21munin-plugin-st09:24:161
22665211516,55sleep20-21swapper/212:29:332
5711211426,48sleep20-21swapper/209:49:492
1796921139,44sleep20-21swapper/212:14:312
3956211115,80sleep13973-21apt11:29:271
26410211120,53sleep20-21swapper/207:30:502
10897211112,84sleep211094-21latency_hist10:09:202
17261211020,50sleep20-21swapper/210:29:182
454210937,57sleep2621-21cat11:19:192
28421210922,47sleep20-21swapper/211:04:332
2446821078,44sleep224483-21grep12:34:382
353821059,60sleep20-21swapper/209:44:402
3200421055,15sleep00-21swapper/009:30:210
24883210513,74sleep325009-21/usr/sbin/munin12:34:473
10705210512,81sleep00-21swapper/010:08:410
5473210262,22sleep05140-21apt-get08:09:200
607429915,66sleep16076-21cron09:54:161
1924529917,62sleep319273-21idleruntime-cro12:19:163
327229812,73sleep03341-21df09:44:340
147129813,71sleep11480-21systemd-run09:39:181
3160129612,70sleep031636-21fschecks_count11:14:370
2256329616,62sleep122608-21/usr/sbin/munin10:44:431
1886129513,54sleep319007-21latency_hist10:34:193
1456629514,64sleep114567-21sendmail-mta12:04:001
2172829412,67sleep121851-21cat12:24:471
1058929414,60sleep310301-21/usr/sbin/munin08:24:383
2405929219,19sleep30-21swapper/307:24:383
1280229213,66sleep012809-21basename10:14:260
335529118,59sleep33425-21fschecks_time09:44:373
2123629117,56sleep321321-21find10:39:483
1943529019,47sleep119472-21sed10:34:421
2637528914,30sleep10-21swapper/109:14:351
2637528914,30sleep10-21swapper/109:14:351
1318328729,25sleep30-21swapper/308:34:193
2890528613,56sleep328990-21mailstats11:04:473
2123128615,55sleep321190-21latency_hist08:59:193
1547228628,45sleep20-21swapper/208:39:472
2299228522,12sleep20-21swapper/209:04:222
326032847,42sleep20-21swapper/207:50:332
1704628426,43sleep017115-21grep07:04:360
89782836,42sleep20-21swapper/210:01:202
2747828312,56sleep227584-21ls10:59:492
2747828312,56sleep227584-21ls10:59:492
208432835,11sleep20-21swapper/208:55:182
137942835,43sleep20-21swapper/210:15:292
232752826,43sleep20-21swapper/210:47:022
1881828211,58sleep018896-21sed07:09:400
1866828218,43sleep118707-21fschecks_time07:09:371
1543628223,11sleep20-21swapper/210:21:142
117392825,15sleep30-21swapper/311:54:293
117392825,15sleep30-21swapper/311:54:293
39862818,15sleep00-21swapper/011:29:350
307192816,40sleep20-21swapper/209:28:452
1946528119,11sleep20-21swapper/208:52:312
1463728113,53sleep314749-21latency_hist12:04:183
1150328119,44sleep011481-21latency_hist11:54:180
1150328119,44sleep011481-21latency_hist11:54:180
234202806,40sleep20-21swapper/207:22:102
1811828045,16sleep20-21swapper/208:49:202
1750428012,55sleep017501-21apt-get10:29:280
1633328014,49sleep016364-21/usr/sbin/munin10:24:440
1618528025,32sleep30-21swapper/312:09:193
284152799,53sleep00-21swapper/009:19:430
284152799,53sleep00-21swapper/009:19:430
1441427913,31sleep10-21swapper/110:19:321
2993927819,11sleep20-21swapper/211:09:262
2141927827,25sleep10-21swapper/107:15:111
1796327819,25sleep10-21swapper/108:49:191
1019827827,31sleep310189-21apt-get11:49:323
1019827827,31sleep310189-21apt-get11:49:323
72412770,8sleep30-21swapper/309:54:503
12492775,36sleep20-21swapper/209:37:172
612527627,27sleep30-21swapper/309:54:183
580527617,39sleep05505-21/usr/sbin/munin11:34:430
580527617,39sleep05505-21/usr/sbin/munin11:34:430
2549327611,15sleep30-21swapper/310:54:383
2546527620,34sleep025497-21munin-run12:39:160
250072760,7sleep30-21swapper/309:09:433
1008627619,32sleep00-21swapper/008:24:190
98727515,41sleep01048-21/usr/sbin/munin09:34:490
82727512,44sleep3981-21unixbench_multi09:34:483
294612758,55sleep0364-21snmpd11:06:290
260327516,45sleep32647-21sort11:24:373
1763527529,21sleep30-21swapper/307:04:473
1365427516,44sleep313694-21uname08:34:393
1169327512,48sleep311730-21ntpq10:10:133
72692748,49sleep00-21swapper/008:14:330
1309227417,36sleep313109-21kernelversion10:14:373
2278827319,36sleep322877-21ls09:04:193
2208727320,21sleep10-21swapper/108:59:491
2050827319,25sleep00-21swapper/008:54:470
193272730,11sleep10-21swapper/112:19:181
1834427311,46sleep018417-21df08:49:340
511427213,46sleep11-21systemd08:09:171
2132527217,32sleep00-21swapper/010:39:490
2032527212,45sleep020368-21awk10:39:170
1378727217,22sleep10-21swapper/108:34:411
3058127122,26sleep00-21swapper/007:44:490
2926527114,28sleep30-21swapper/309:24:183
2926527114,28sleep30-21swapper/309:24:183
2550827121,29sleep10-21swapper/107:29:361
2019727118,26sleep00-21swapper/012:19:460
180227118,29sleep10-21swapper/109:39:351
1403927126,26sleep00-21swapper/008:34:490
914927017,28sleep30-21swapper/308:19:413
809827015,35sleep18156-21/usr/sbin/munin09:59:371
2087627016,27sleep10-21swapper/112:24:191
3188326914,37sleep131912-21/usr/sbin/munin11:14:421
153372698,13sleep00-21swapper/012:04:440
1030026918,34sleep010302-21/usr/sbin/munin08:24:270
879826821,26sleep00-21swapper/011:44:370
879826821,26sleep00-21swapper/011:44:370
840026817,31sleep18415-21latency_hist11:44:171
840026817,31sleep18415-21latency_hist11:44:171
2582826813,42sleep125829-21sshd09:14:071
2248526816,23sleep00-21swapper/007:19:380
2062326818,29sleep019824-21apt-get07:14:290
1994526823,27sleep320157-21gpgv07:14:203
149862680,7sleep00-21swapper/008:39:280
3244226725,24sleep30-21swapper/311:15:043
2818326719,21sleep00-21swapper/011:04:190
2818326719,21sleep00-21swapper/011:04:190
2723526719,26sleep10-21swapper/107:34:411
2509226710,15sleep10-21swapper/110:54:191
2293326718,33sleep00-21swapper/012:29:390
373226611,16sleep00-21swapper/011:29:180
2672626615,36sleep326764-21latency_hist07:34:193
170322667,16sleep20-21swapper/207:04:342
995726527,16sleep10-21swapper/111:49:181
995726527,16sleep10-21swapper/111:49:181
232092655,46sleep00-21swapper/010:46:110
1663926512,35sleep316765-21/usr/sbin/munin10:24:513
2471026418,20sleep10-21swapper/112:34:451
2458426410,37sleep00-21swapper/010:49:500
2302726418,28sleep123081-21cpuspeed09:04:341
1780426415,33sleep317869-21apt-config08:49:173
1499926418,27sleep314994-21apt-get08:39:323
2294126319,20sleep30-21swapper/307:19:493
1997826313,32sleep120061-21tune2fs08:54:361
561526216,30sleep35617-21/usr/sbin/munin08:09:283
302826220,21sleep00-21swapper/009:44:180
3009126215,21sleep10-21swapper/109:24:491
266026216,22sleep30-21swapper/309:40:093
973126113,22sleep10-21swapper/110:04:381
710426115,25sleep30-21swapper/311:39:313
710426115,25sleep30-21swapper/311:39:313
508326121,19sleep00-21swapper/009:49:390
206526113,31sleep02135-21cut07:59:190
1933526113,29sleep30-21swapper/310:34:403
1165826119,24sleep10-21swapper/108:29:201
731526017,20sleep30-21swapper/308:14:373
62526013,29sleep3789-21apt-config11:19:203
2414326017,23sleep10-21swapper/110:49:431
176952607,13sleep00-21swapper/012:14:170
1575126015,29sleep015901-21cat10:24:190
111126013,33sleep11157-21basename11:20:191
793725916,26sleep37932-21apt-get09:59:313
375625917,18sleep10-21swapper/111:29:191
3081125915,21sleep30-21swapper/309:29:193
295802596,11sleep10-21swapper/107:41:271
2850525912,25sleep10-21swapper/111:04:371
2449125915,17sleep10-21swapper/107:24:481
2442525911,32sleep324548-21unixbench_multi10:49:493
2154725911,31sleep021554-21diskstats08:59:360
210122596,12sleep10-21swapper/110:39:421
1899725917,20sleep10-21swapper/108:49:561
181792597,13sleep00-21swapper/012:14:370
1646925920,19sleep10-21swapper/110:24:491
865225817,25sleep38647-21apt-get11:44:323
865225817,25sleep38647-21apt-get11:44:323
598625813,24sleep10-21swapper/108:10:311
2854225814,25sleep328538-21apt-get07:39:313
2747925813,21sleep00-21swapper/007:34:480
2345125814,26sleep323463-21sh10:49:173
2326425813,28sleep323298-21/usr/sbin/munin09:04:403
230125817,25sleep32298-21apt-get07:59:303
227825814,28sleep12295-21/usr/sbin/munin07:59:281
1651125815,25sleep10-21swapper/112:09:371
1487325816,26sleep114870-21apt-get12:04:281
139825812,21sleep00-21swapper/011:20:260
645025715,24sleep10-21swapper/109:54:371
3105225717,23sleep10-21swapper/109:29:331
2996625716,25sleep329962-21apt-get11:09:313
299592579,24sleep10-21swapper/111:09:291
2518325714,28sleep025266-21latency_hist07:29:190
2334425717,16sleep10-21swapper/112:29:481
1758025711,22sleep10-21swapper/110:29:331
143252575,13sleep00-21swapper/012:00:520
844625614,27sleep1225-21systemd-journal08:19:071
553425614,26sleep15578-21/usr/sbin/munin11:34:361
553425614,26sleep15578-21/usr/sbin/munin11:34:361
368425615,24sleep13721-21ntp_states09:44:431
3009925619,16sleep30-21swapper/307:44:363
2302625616,24sleep023020-21apt-get09:04:330
2242025619,23sleep30-21swapper/312:29:183
607325515,24sleep36140-21swap11:34:513
607325515,24sleep36140-21swap11:34:513
1630825514,24sleep116409-21sed08:44:181
1462125517,21sleep30-21swapper/310:19:383
921125413,23sleep19235-21ntp_states08:19:431
73692547,37sleep00-21swapper/009:56:010
48525416,22sleep1614-21sort07:54:191
2522125414,17sleep00-21swapper/009:09:500
2306525423,17sleep30-21swapper/312:29:423
1980125413,25sleep319835-21irqstats12:19:373
1232925413,25sleep012336-21ntp_states08:29:440
1127625416,22sleep00-21swapper/010:09:300
362225315,17sleep00-21swapper/008:04:200
2363925314,16sleep10-21swapper/107:24:191
223652534,15sleep00-21swapper/012:29:110
1159625314,23sleep111652-21sed10:10:111
2637425213,23sleep026361-21apt-get09:14:340
2637425213,23sleep026361-21apt-get09:14:340
258532529,17sleep00-21swapper/007:29:430
2058025212,22sleep30-21swapper/307:14:283
1365825214,23sleep113727-21users10:14:481
1277825213,27sleep112780-21sshd11:56:251
1277825213,27sleep112780-21sshd11:56:251
1826425114,23sleep317825-21apt-get08:49:213
1233725117,15sleep10-21swapper/108:29:451
83602509,15sleep00-21swapper/011:44:160
83602509,15sleep00-21swapper/011:44:160
3063025011,24sleep00-21swapper/011:09:480
2586025015,21sleep025929-21switchtime10:54:470
2635424916,17sleep30-21swapper/309:14:293
2635424916,17sleep30-21swapper/309:14:293
118524911,14sleep10-21swapper/107:54:441
2977324811,21sleep029775-21irqstats09:24:400
92724610,22sleep3982-21if_eth007:54:383
3145524210,13sleep30-21swapper/309:29:433
1059724113,15sleep10-21swapper/111:49:421
1059724113,15sleep10-21swapper/111:49:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional