You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 11:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Jul 18, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168442353013,3505sleep311501-21kworker/3:200:27:003
168442353013,3505sleep311501-21kworker/3:200:27:003
465525686,551sleep13568-1kworker/1:1H23:48:541
14313247819,443sleep214375-21/usr/sbin/munin20:54:032
22959241931,112sleep10-21swapper/123:03:521
14004236613,340sleep114081-21awk22:34:001
29503236115,333sleep329541-21ntpq19:58:563
78223499,323sleep211610-1kworker/2:1H21:53:512
78223499,323sleep211610-1kworker/2:1H21:53:512
265392299115,124sleep10-21swapper/119:49:011
13520228927,247sleep113606-21latency_hist20:53:331
837228157,21sleep10-21swapper/123:38:001
4135226267,135sleep14325-21latency_hist23:48:341
98762250103,32sleep10-21swapper/120:39:041
18686224911,197sleep118798-21timerandwakeup22:49:021
27905224235,193sleep127944-21ntpq19:53:581
8320223537,97sleep10-21swapper/122:18:331
20223222718,102sleep120075-21latency_hist00:38:331
15605222314,41sleep115670-21latency20:58:541
21346222035,165sleep121358-21munin-run21:18:331
216622215182,18sleep10-21swapper/122:58:591
22023220814,171sleep122069-21fschecks_time21:19:541
22023220814,171sleep122069-21fschecks_time21:19:541
25602220214,60sleep10-21swapper/123:13:331
25602220214,60sleep10-21swapper/123:13:331
31028219870,51sleep10-21swapper/120:03:551
16290219269,22sleep10-21swapper/122:43:331
30236218913,126sleep130239-21cron23:28:311
7935216936,91sleep18003-21ls23:58:561
2957621626,51sleep10-21swapper/121:43:321
5779216187,55sleep10-21swapper/122:08:561
31132216013,78sleep131214-21latency_hist21:48:331
15175216034,78sleep115185-21modprobe00:23:301
15175216034,78sleep115185-21modprobe00:23:301
15587215917,126sleep215066-21/usr/sbin/munin19:14:092
17468215811,126sleep117554-21switchtime21:04:031
10386215510,34sleep110452-21fschecks_time22:23:501
8719215337,93sleep10-21swapper/122:18:411
3393215129,22sleep10-21swapper/120:19:011
6443215028,69sleep16477-21ps20:28:591
26890215020,46sleep10-21swapper/121:33:551
13297215025,18sleep10-21swapper/119:08:561
25126214846,40sleep10-21swapper/121:28:421
5642214620,113sleep25653-21kernelversion22:08:532
15505214518,111sleep215609-21latency_hist00:23:362
15505214518,111sleep215609-21latency_hist00:23:362
15465214237,45sleep10-21swapper/119:14:041
23915214012,26sleep10-21swapper/121:24:021
20040214010,93sleep120124-21cat19:28:541
19345214017,80sleep10-21swapper/121:09:081
12322213914,54sleep10-21swapper/122:28:591
26363213622,99sleep026411-21tr19:48:560
23354213630,51sleep123388-21ps19:38:591
23354213630,51sleep123388-21ps19:38:591
1970021367,47sleep10-21swapper/122:53:501
1479121366,12sleep10-21swapper/120:55:351
12988213621,96sleep313005-21fschecks_count19:08:493
27703213532,16sleep10-21swapper/123:18:541
22541213419,45sleep10-21swapper/119:38:021
2530213117,17sleep10-21swapper/120:18:001
17861212810,78sleep118012-21date22:48:331
16761212522,85sleep216801-21fschecks_count19:18:502
10834212511,76sleep110885-21cat20:43:521
291721208,52sleep13040-21timerwakeupswit21:59:031
2042211821,72sleep00-21swapper/021:58:330
2042211821,72sleep00-21swapper/021:58:330
16982211826,81sleep117011-21unin-run00:28:311
16982211826,81sleep117011-21unin-run00:28:311
20085211520,79sleep320110-21pluginstate22:54:013
16145211515,81sleep216176-21/usr/sbin/munin00:24:292
16145211515,81sleep216176-21/usr/sbin/munin00:24:292
32759211410,86sleep10-21swapper/120:09:001
12415211416,82sleep312503-21fschecks_time00:13:513
16618211333,60sleep216708-21/usr/sbin/munin22:43:502
14378211327,63sleep014546-21fschecks_count19:13:350
1281210718,64sleep31293-21fschecks_count23:38:513
6553210624,19sleep10-21swapper/123:53:591
443721059,35sleep10-21swapper/120:23:461
2885210534,46sleep02881-21apt-get20:18:440
22511210513,75sleep322548-21latency_hist23:03:333
9726210228,57sleep10-21swapper/100:04:001
21437210223,55sleep221533-21apt-key21:18:342
1822321029,34sleep10-21swapper/119:23:451
1681210249,41sleep1278-21dbus-daemon23:39:011
479821016,12sleep10-21swapper/122:04:441
4294210116,66sleep34334-21ps22:03:583
3918210119,59sleep23933-21cut22:03:482
9069210016,68sleep39162-21memory22:19:283
21999210018,59sleep022141-21grep19:34:020
21790210015,64sleep221290-21apt-get21:18:422
21790210015,64sleep221290-21apt-get21:18:422
77992999,73sleep07839-21latency20:33:540
315422995,13sleep10-21swapper/123:30:251
278422996,10sleep30-21swapper/321:37:003
2153529914,65sleep321733-21sed21:18:353
1450329814,67sleep014651-21unixbench_singl20:54:050
7732976,31sleep10-21swapper/121:53:501
7732976,31sleep10-21swapper/121:53:501
31922960,11sleep00-21swapper/021:59:050
321562949,19sleep30-21swapper/323:33:503
1533729413,65sleep215384-21grep20:58:472
1263329429,31sleep30-21swapper/319:08:333
249132939,39sleep112703-21cyclictest19:44:001
648029212,66sleep26556-21sendmail_mailtr20:29:002
2527929228,38sleep025337-21fschecks_time21:28:540
821429113,63sleep18376-21grep20:34:031
77422919,14sleep00-21swapper/023:58:500
583929118,37sleep30-21swapper/323:53:323
195229112,62sleep02093-21uptime23:39:040
2658929025,41sleep10-21swapper/123:14:011
2658929025,41sleep10-21swapper/123:14:011
101529014,60sleep01020-21modprobe20:13:310
93122886,13sleep20-21swapper/220:38:532
178262888,17sleep00-21swapper/021:04:230
766128623,39sleep37804-21sed22:14:033
297512868,63sleep129873-21sed19:59:021
1196828619,34sleep10-21swapper/100:12:441
775228520,44sleep27833-21grep23:58:522
241282846,15sleep30-21swapper/323:08:343
241282846,15sleep30-21swapper/323:08:343
1970728413,16sleep00-21swapper/021:13:310
2958628334,26sleep20-21swapper/221:43:332
2729828317,50sleep227345-21grep19:53:342
2014028316,18sleep30-21swapper/319:28:563
1856028313,20sleep10-21swapper/100:33:321
73152826,12sleep00-21swapper/023:58:140
2719528217,48sleep0364-21snmpd23:18:320
2719528217,48sleep0364-21snmpd23:18:320
2711528223,30sleep20-21swapper/223:18:302
2711528223,30sleep20-21swapper/223:18:302
2176328211,55sleep321774-21ntpq19:33:573
2910028131,27sleep30-21swapper/323:23:533
1239728122,19sleep10-21swapper/100:13:491
311262800,6sleep20-21swapper/220:03:592
2968228036,25sleep00-21swapper/019:59:010
2410528018,47sleep324251-21unixbench_singl21:24:043
1782927925,28sleep00-21swapper/022:48:320
700027813,49sleep07009-21df_inode22:13:480
480427824,29sleep30-21swapper/320:23:563
2646427822,24sleep30-21swapper/321:33:333
1985727811,53sleep219931-21fschecks_time19:28:512
1694327822,44sleep10-21swapper/119:18:531
122612788,16sleep20-21swapper/222:28:562
1138327819,38sleep011524-21head20:44:040
2569427714,45sleep225765-21sensors_temp21:29:042
3197127613,47sleep332080-21sed20:08:333
2670927613,48sleep026746-21grep21:33:500
1572627616,41sleep015846-21/usr/sbin/munin19:14:110
1278327612,46sleep20-21swapper/200:13:592
1192927619,39sleep212067-21sort19:04:012
3185627513,44sleep332000-21latency_hist23:33:343
300927417,41sleep03079-21cat23:43:520
2751127416,40sleep027530-21apt-get19:53:440
2724527419,36sleep327275-21idleruntime-cro19:53:313
1822827419,23sleep30-21swapper/321:08:333
317652736,12sleep10-21swapper/121:48:581
317652736,12sleep10-21swapper/121:48:581
2576027311,46sleep325902-21latency_hist19:48:343
2462527317,36sleep224654-21munin-run21:28:312
180427326,22sleep20-21swapper/220:14:022
1097927318,23sleep00-21swapper/000:08:520
799727217,25sleep20-21swapper/220:34:002
2791527212,25sleep30-21swapper/323:19:003
2419227221,31sleep20-21swapper/219:43:332
2419227221,31sleep20-21swapper/219:43:332
546527126,29sleep30-21swapper/322:08:503
3182727114,43sleep231855-21latency_hist23:33:332
2373927120,26sleep00-21swapper/021:23:570
67027021,29sleep332645-21apt-get21:53:443
67027021,29sleep332645-21apt-get21:53:443
49027015,38sleep3639-21unixbench_singl20:09:023
359827021,23sleep30-21swapper/322:03:333
1975827019,25sleep20-21swapper/221:13:332
1451927023,26sleep00-21swapper/000:19:010
1451927023,26sleep00-21swapper/000:19:010
2172826913,37sleep221799-21/usr/sbin/munin22:59:002
1938626923,23sleep00-21swapper/022:53:330
1844026920,33sleep218452-21/usr/sbin/munin22:48:552
1230726911,38sleep00-21swapper/022:28:580
2255026817,25sleep00-21swapper/023:03:340
1948226825,24sleep30-21swapper/319:28:313
1696126818,31sleep00-21swapper/000:28:300
1696126818,31sleep00-21swapper/000:28:300
1052826821,21sleep20-21swapper/200:08:342
886826721,27sleep30-21swapper/320:38:333
441226715,28sleep20-21swapper/220:23:432
297326713,29sleep30-21swapper/320:18:513
2874326721,22sleep00-21swapper/023:23:330
2005626716,30sleep320123-21gpgv00:38:313
1720226720,23sleep00-21swapper/019:19:000
1665426715,23sleep00-21swapper/021:03:340
966026621,20sleep30-21swapper/320:39:023
663926611,37sleep06681-21systemd-run22:13:310
238526630,19sleep20-21swapper/221:58:522
1535826611,41sleep015406-21ntpq22:38:560
1477726629,17sleep30-21swapper/319:13:383
1225426611,43sleep00-21swapper/019:04:230
1160926619,26sleep00-21swapper/022:28:340
5902659,44sleep00-21swapper/023:34:560
1906926516,27sleep30-21swapper/321:09:063
1211526519,29sleep312185-21cat22:28:533
1171826522,23sleep10-21swapper/119:03:551
3262926417,20sleep30-21swapper/321:53:323
3262926417,20sleep30-21swapper/321:53:323
223902645,46sleep20-21swapper/219:36:062
2563626315,23sleep20-21swapper/223:13:342
2563626315,23sleep20-21swapper/223:13:342
1274226312,26sleep30-21swapper/320:49:033
1247526322,19sleep00-21swapper/020:48:560
958526220,24sleep00-21swapper/000:03:550
3262626218,21sleep20-21swapper/223:34:012
3146726211,35sleep031559-21sort21:48:520
3146726211,35sleep031559-21sort21:48:520
25722625,12sleep20-21swapper/223:43:342
1997326217,27sleep019992-21apt-get21:13:430
175026221,23sleep20-21swapper/223:39:022
1670926219,27sleep316773-21/usr/sbin/munin22:43:513
1498026219,20sleep30-21swapper/322:38:463
347226120,25sleep20-21swapper/220:19:022
2964226113,26sleep20-21swapper/223:24:042
2275126118,25sleep322746-21apt-get23:03:463
1386826118,23sleep30-21swapper/322:33:583
1066826120,19sleep20-21swapper/222:23:552
72232605,44sleep00-21swapper/020:33:170
490826015,21sleep00-21swapper/023:49:010
314226017,22sleep30-21swapper/323:43:553
2828026022,18sleep00-21swapper/021:38:470
280562607,12sleep20-21swapper/219:54:032
2514626013,32sleep025249-21unixbench_multi19:44:030
191822608,14sleep00-21swapper/000:33:560
903925918,20sleep30-21swapper/300:03:343
88842596,10sleep30-21swapper/300:02:573
693025916,22sleep20-21swapper/222:13:462
2297825918,21sleep00-21swapper/019:38:500
2297825918,21sleep00-21swapper/019:38:500
218152596,26sleep00-21swapper/021:18:440
218152596,26sleep00-21swapper/021:18:440
1850825922,21sleep30-21swapper/300:33:303
3066225818,23sleep030656-21apt-get20:03:460
301472585,10sleep00-21swapper/023:27:210
1822425814,26sleep218216-21apt-get19:23:462
2825225715,26sleep328270-21apt21:38:423
2406225719,16sleep20-21swapper/223:08:332
2212725714,25sleep322178-21irqstats21:20:003
2212725714,25sleep322178-21irqstats21:20:003
1126425718,17sleep30-21swapper/320:44:033
2639425617,21sleep30-21swapper/323:13:593
2639425617,21sleep30-21swapper/323:13:593
23102567,39sleep00-21swapper/020:15:170
1840725615,19sleep00-21swapper/019:23:520
1272525619,23sleep00-21swapper/000:13:570
112232568,23sleep20-21swapper/200:08:582
848225516,22sleep28645-21apt-config22:18:342
453225514,25sleep24521-21apt-get23:48:472
3028225513,29sleep330284-21sh21:43:593
2780525519,20sleep30-21swapper/319:53:563
2321425512,18sleep00-21swapper/023:03:590
1977025514,24sleep019765-21apt-get19:28:450
1290325515,24sleep212899-21apt-get19:08:442
1227425516,24sleep212281-21df_inode20:48:502
509225416,20sleep00-21swapper/022:08:310
3194825415,22sleep20-21swapper/221:49:032
3194825415,22sleep20-21swapper/221:49:032
1697825418,17sleep20-21swapper/221:03:522
1036325418,21sleep00-21swapper/020:43:310
631325318,21sleep26347-21irqstats23:53:522
1474125311,17sleep20-21swapper/222:38:332
1390925314,23sleep313905-21apt-get00:18:443
1390925314,23sleep313905-21apt-get00:18:443
1196225314,17sleep20-21swapper/220:48:342
953925213,24sleep09685-21unixbench_singl22:19:370
744225215,20sleep30-21swapper/323:58:333
3120425213,17sleep00-21swapper/023:29:020
310372525,38sleep20-21swapper/221:47:582
2678125214,23sleep226790-21diskstats21:33:522
2575025214,21sleep20-21swapper/219:48:332
1962225215,22sleep019618-21apt-get22:53:450
1845325219,19sleep30-21swapper/322:48:563
1731425215,17sleep00-21swapper/000:28:470
1083325212,20sleep30-21swapper/322:24:013
97325114,22sleep2974-21sshd20:13:012
2430125114,15sleep00-21swapper/023:08:470
2430125114,15sleep00-21swapper/023:08:470
2346925116,20sleep323557-21switchtime19:39:013
2346925116,20sleep323557-21switchtime19:39:013
1664225113,22sleep316652-21munin-run21:03:333
1384325119,15sleep30-21swapper/320:53:523
71692504,35sleep30-21swapper/320:32:363
2987425011,19sleep20-21swapper/219:59:032
1970825011,20sleep20-21swapper/222:53:512
1667725013,22sleep316744-21df19:18:473
1340025013,21sleep013396-21apt-get22:33:430
1075225012,22sleep310759-21apt-get00:08:433
443624812,21sleep04432-21apt-get20:23:450
2817524812,19sleep20-21swapper/223:19:032
1543324713,19sleep215439-21ntp_states22:38:582
177032468,16sleep20-21swapper/200:28:572
3019224512,18sleep030224-21awk21:43:560
192522447,13sleep20-21swapper/200:33:582
13032439,17sleep30-21swapper/320:13:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional