You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 14:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed May 27, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19872615716,6129sleep131191-21kworker/1:022:47:571
1065728168,795sleep210714-21unin-run21:33:422
17203274025,662sleep217245-21ntpq23:34:092
17203274025,662sleep217245-21ntpq23:34:092
1464723908,367sleep214737-21grep21:44:002
1464723908,367sleep214737-21grep21:44:002
18811237434,301sleep218882-21switchtime20:14:172
880522938,250sleep28851-21grep23:08:582
25216226217,231sleep125237-21latency_hist22:18:431
568226113,233sleep0598-21awk00:24:050
16441224573,20sleep20-21swapper/220:08:432
16441224573,20sleep20-21swapper/220:08:432
32734222214,190sleep217237-1kworker/2:0H22:40:562
2097222243,125sleep22164-21latency_hist22:48:432
31394221810,144sleep230906-21/usr/sbin/munin20:54:102
31394221810,144sleep230906-21/usr/sbin/munin20:54:102
18115221840,26sleep20-21swapper/221:54:092
18115221840,26sleep20-21swapper/221:54:092
2231221118,41sleep20-21swapper/219:24:232
4017219619,146sleep20-21swapper/200:35:102
4017219619,146sleep20-21swapper/200:35:102
7127218311,135sleep27192-21ls19:39:052
9919218115,28sleep20-21swapper/219:48:412
1923021789,157sleep219271-21grep21:58:572
1923021789,157sleep219271-21grep21:58:572
26318217717,141sleep226336-21kernelversion20:39:162
26318217717,141sleep226336-21kernelversion20:39:162
19029217713,149sleep119054-21awk23:39:211
3220321757,157sleep00-21swapper/000:21:310
20907217316,141sleep321059-21sed23:44:123
262521710,30sleep02631-21unixbench_singl00:29:130
11060217132,15sleep20-21swapper/223:14:152
32509217051,56sleep232577-21sshd20:58:582
4198216921,62sleep20-21swapper/219:29:072
14334216610,134sleep214486-21latency_hist21:43:442
20794216519,89sleep20-21swapper/223:44:112
411421628,139sleep00-21swapper/019:29:050
23964216228,96sleep224102-21grep23:54:122
23964216228,96sleep224102-21grep23:54:122
4518215910,134sleep24674-21latency_hist21:13:442
29047215615,18sleep20-21swapper/222:29:092
23640215415,101sleep20-21swapper/222:13:432
32051215310,86sleep20-21swapper/219:14:142
1452821535,12sleep20-21swapper/220:02:462
567215243,37sleep2626-21idleruntime19:19:022
3069921528,54sleep20-21swapper/220:53:452
3069921528,54sleep20-21swapper/220:53:452
2435721528,135sleep3194-21jbd2/mmcblk2p2-23:56:193
2435721528,135sleep3194-21jbd2/mmcblk2p2-23:56:193
6303215119,90sleep20-21swapper/221:18:572
27984214816,116sleep328016-21tune2fs20:44:583
12268214827,101sleep312364-21fschecks_time23:19:003
36432147102,23sleep20-21swapper/222:53:442
30002214197,23sleep20-21swapper/200:14:102
30002214197,23sleep20-21swapper/200:14:102
1725921417,95sleep20-21swapper/220:09:162
1725921417,95sleep20-21swapper/220:09:162
6752213898,23sleep26857-21apt-config19:38:442
2799921385,11sleep20-21swapper/222:25:142
29428213794,25sleep229466-21fschecks_count19:09:012
1289621355,18sleep20-21swapper/219:57:092
8525213413,75sleep20-21swapper/221:24:132
8346213413,95sleep00-21swapper/021:24:110
3258213413,22sleep10-21swapper/121:09:021
6991213124,87sleep37140-21tail21:19:183
2773621310,7sleep30-21swapper/322:24:133
20404213118,96sleep320483-21sort20:19:223
20404213118,96sleep320483-21sort20:19:223
216621309,101sleep011-21ksoftirqd/019:23:520
18605212914,100sleep318099-21apt-get23:38:513
5543212816,96sleep15547-21diskstats22:59:001
1440521289,84sleep214545-21tail23:24:122
12610212822,87sleep312806-21cat21:38:443
2763121245,109sleep20-21swapper/200:08:012
2510321226,81sleep20-21swapper/222:18:072
2655021205,81sleep20-21swapper/222:21:262
12998211712,90sleep012466-21apt-get21:38:530
583621135,12sleep20-21swapper/221:16:472
5224211322,73sleep15313-21ls22:58:431
1750211326,35sleep20-21swapper/200:28:562
5548211112,81sleep25637-21fschecks_time22:59:012
4150210911,64sleep24198-21latency22:54:042
24670210910,15sleep30-21swapper/322:14:123
19650210911,84sleep0363-21snmpd23:40:290
11252210917,74sleep111378-21sort23:14:181
1709821088,55sleep00-21swapper/023:34:070
1709821088,55sleep00-21swapper/023:34:070
9611210710,14sleep00-21swapper/021:29:040
27963210715,76sleep228035-21cat19:04:462
1664421077,16sleep20-21swapper/221:49:132
1089321068,59sleep211046-21awk19:49:142
8853210511,81sleep38878-21df23:08:593
3011121057,15sleep20-21swapper/222:33:532
2318821050,6sleep30-21swapper/320:29:063
9239210411,20sleep09275-21ntp_states23:09:200
2363221049,52sleep20-21swapper/220:29:192
5803210312,70sleep15873-21/usr/sbin/munin19:34:111
22996210216,66sleep00-21swapper/020:29:000
10679210216,73sleep310723-21processes19:49:113
1252721018,60sleep212609-21gpgv21:38:432
22940210021,45sleep20-21swapper/223:52:592
1623921005,84sleep20-21swapper/223:29:572
1623921005,84sleep20-21swapper/223:29:572
1285221008,18sleep212978-21cat23:19:112
2950329921,41sleep10-21swapper/100:13:531
2950329921,41sleep10-21swapper/100:13:531
244602998,50sleep20-21swapper/220:33:552
2163729812,71sleep021664-21cut20:24:080
2163729812,71sleep021664-21cut20:24:080
396729717,56sleep34024-21grep22:54:013
279682978,49sleep20-21swapper/200:08:542
217682976,79sleep2363-21snmpd22:05:112
217682976,79sleep2363-21snmpd22:05:112
1065729717,66sleep110669-21ntp_states19:49:091
2508529629,29sleep20-21swapper/223:59:052
2233029631,29sleep20-21swapper/220:24:592
2233029631,29sleep20-21swapper/220:24:592
934229521,50sleep09497-21grep19:44:110
538329423,48sleep35399-21sort19:34:013
6422937,47sleep20-21swapper/200:24:072
2085429331,29sleep20-21swapper/220:20:512
2085429331,29sleep20-21swapper/220:20:512
1494729320,57sleep314983-21ntpq21:44:053
1494729320,57sleep314983-21ntpq21:44:053
3073229213,62sleep130860-21cut19:13:441
196522920,6sleep10-21swapper/121:59:081
196522920,6sleep10-21swapper/121:59:081
180729214,61sleep01997-21cat19:23:440
2946129119,48sleep129357-21/usr/sbin/munin20:49:001
2946129119,48sleep129357-21/usr/sbin/munin20:49:001
2182329030,32sleep30-21swapper/323:49:003
765128930,30sleep10-21swapper/121:23:441
2775328929,38sleep127750-21apt-get19:03:561
2696128815,50sleep026013-21/usr/sbin/munin20:39:300
2696128815,50sleep026013-21/usr/sbin/munin20:39:300
2984528725,42sleep329885-21ps20:49:103
2984528725,42sleep329885-21ps20:49:103
1055928619,47sleep310084-21apt-get23:13:513
311912858,16sleep20-21swapper/200:18:592
685228419,16sleep10-21swapper/123:03:441
3215028419,50sleep332158-21ntpq22:39:073
29302849,16sleep20-21swapper/221:08:432
223028413,55sleep12184-21apt-get19:24:221
196372845,67sleep20-21swapper/223:40:192
598428325,36sleep30-21swapper/321:18:413
278742835,46sleep20-21swapper/220:44:332
73192828,32sleep20-21swapper/223:04:052
3059528213,52sleep130657-21/usr/sbin/munin22:34:091
138628216,50sleep31536-21wc21:03:433
1342628224,27sleep10-21swapper/119:59:021
1225228216,52sleep1309-21rs:main1
49228020,30sleep10-21swapper/119:19:011
1683227814,52sleep016896-21idleruntime20:09:060
1683227814,52sleep016896-21idleruntime20:09:060
1534427828,34sleep315247-21/usr/sbin/munin23:28:583
1534427828,34sleep315247-21/usr/sbin/munin23:28:583
208632777,58sleep00-21swapper/020:20:570
208632777,58sleep00-21swapper/020:20:570
3218927622,26sleep10-21swapper/120:58:411
3218927622,26sleep10-21swapper/120:58:411
1673927622,30sleep00-21swapper/023:33:470
1673927622,30sleep00-21swapper/023:33:470
1061327618,31sleep00-21swapper/019:49:080
1006327641,15sleep10-21swapper/123:13:411
3127927522,29sleep10-21swapper/119:13:561
2928727517,17sleep30-21swapper/300:13:433
505927425,26sleep30-21swapper/319:33:453
33742738,56sleep00-21swapper/022:50:550
2462927318,29sleep10-21swapper/123:58:431
2462927318,29sleep10-21swapper/123:58:431
606427218,31sleep10-21swapper/121:18:431
3190127214,16sleep10-21swapper/122:39:011
2864327215,26sleep00-21swapper/022:28:590
2754527219,36sleep327682-21date20:43:453
2754527219,36sleep327682-21date20:43:453
2626427214,39sleep126013-21/usr/sbin/munin20:39:151
2626427214,39sleep126013-21/usr/sbin/munin20:39:151
1718727220,25sleep10-21swapper/123:34:081
1718727220,25sleep10-21swapper/123:34:081
1430727217,39sleep114332-21cat21:43:431
365827015,40sleep33732-21tr21:09:133
3145527018,28sleep30-21swapper/322:38:433
2906527026,22sleep00-21swapper/019:08:430
2797227017,39sleep127990-21cpu00:08:571
2618527015,39sleep026235-21latency_hist00:03:430
2505727024,26sleep10-21swapper/123:59:041
312426919,24sleep10-21swapper/100:33:431
2209326816,21sleep30-21swapper/322:08:433
2209326816,21sleep30-21swapper/322:08:433
1509626820,26sleep30-21swapper/320:03:523
1509626820,26sleep30-21swapper/320:03:523
1419226817,25sleep30-21swapper/323:24:093
1117226818,32sleep00-21swapper/023:14:170
549626714,23sleep00-21swapper/021:14:180
2462526715,27sleep10-21swapper/120:34:061
2212926719,20sleep00-21swapper/022:08:440
1741426716,24sleep00-21swapper/021:53:430
842726621,21sleep10-21swapper/121:24:121
2546726616,24sleep10-21swapper/122:18:591
200726618,27sleep32043-21ntpq21:04:083
1648026617,30sleep10-21swapper/121:49:071
155322667,11sleep00-21swapper/021:45:070
155322667,11sleep00-21swapper/021:45:070
876526513,35sleep18868-21perl19:44:001
2514326517,27sleep30-21swapper/323:59:063
152626519,19sleep00-21swapper/022:44:130
1424826517,21sleep00-21swapper/023:24:100
1860526417,22sleep30-21swapper/320:14:103
1381826412,28sleep30-21swapper/319:59:123
803426318,23sleep30-21swapper/321:24:023
398826314,33sleep14016-21/usr/sbin/munin00:35:091
398826314,33sleep14016-21/usr/sbin/munin00:35:091
323026317,22sleep10-21swapper/122:49:151
2144626317,30sleep021475-21munin-run23:48:410
204572637,48sleep00-21swapper/022:03:110
204572637,48sleep00-21swapper/022:03:110
1753426311,30sleep30-21swapper/323:34:553
1753426311,30sleep30-21swapper/323:34:553
1236626319,22sleep00-21swapper/023:19:010
80342625,16sleep00-21swapper/023:05:370
3226826221,24sleep30-21swapper/320:58:433
3226826221,24sleep30-21swapper/320:58:433
2902126214,31sleep129046-21/usr/sbin/munin22:29:081
2790226213,32sleep027969-21/usr/sbin/munin20:44:550
247026213,31sleep32615-21awk00:29:133
232722625,11sleep00-21swapper/022:09:370
1804426221,19sleep30-21swapper/321:54:063
1804426221,19sleep30-21swapper/321:54:063
155502625,9sleep10-21swapper/121:45:201
155502625,9sleep10-21swapper/121:45:201
1534626223,21sleep10-21swapper/120:04:141
1534626223,21sleep10-21swapper/120:04:141
1498826210,19sleep00-21swapper/020:03:450
1498826210,19sleep00-21swapper/020:03:450
122442626,43sleep10-21swapper/121:36:031
966926120,25sleep10-21swapper/121:29:051
3117526121,21sleep30-21swapper/319:13:503
2297326122,18sleep30-21swapper/322:09:123
1981526118,26sleep119809-21apt-get20:18:561
1981526118,26sleep119809-21apt-get20:18:561
66992608,42sleep00-21swapper/023:03:180
251826023,18sleep30-21swapper/319:24:443
2336626016,23sleep10-21swapper/122:10:511
218826019,18sleep00-21swapper/021:04:130
188112606,10sleep10-21swapper/121:57:231
188112606,10sleep10-21swapper/121:57:231
1183826015,21sleep30-21swapper/319:53:593
738625918,25sleep37419-21ntp_kernel_err23:04:063
425025918,24sleep00-21swapper/022:54:060
2973925917,22sleep329796-21irqstats00:14:033
2973925917,22sleep329796-21irqstats00:14:033
2955225915,26sleep029610-21/usr/sbin/munin19:09:030
2841125920,16sleep30-21swapper/319:04:553
2299625912,25sleep00-21swapper/023:53:410
2224225916,20sleep10-21swapper/123:49:101
2177125922,19sleep30-21swapper/320:24:113
2177125922,19sleep30-21swapper/320:24:113
1417325918,19sleep10-21swapper/123:24:071
1360425921,20sleep00-21swapper/019:59:050
3136325817,23sleep00-21swapper/019:14:000
296442586,17sleep30-21swapper/322:31:203
2700925816,26sleep027005-21apt-get22:23:540
427925713,26sleep14354-21sensors_temp19:29:101
2934225715,28sleep329409-21df19:08:583
1736625715,26sleep317412-21unin-run21:53:423
132025716,18sleep10-21swapper/121:03:411
1071725713,25sleep310714-21munin-run21:33:433
59925615,19sleep10-21swapper/100:24:061
3111125614,25sleep331106-21apt-get00:18:563
191682565,11sleep00-21swapper/020:14:350
1132525617,22sleep011411-21perl21:34:130
81092556,37sleep30-21swapper/319:40:193
758425514,24sleep17585-21ntp_states19:39:161
489425512,23sleep05044-21grep00:38:430
489425512,23sleep05044-21grep00:38:430
305492558,34sleep0363-21snmpd00:15:420
305492558,34sleep0363-21snmpd00:15:420
2773525513,16sleep00-21swapper/000:08:430
2446325514,25sleep024457-21apt-get20:33:570
2136325514,24sleep121357-21apt-get20:23:571
2136325514,24sleep121357-21apt-get20:23:571
1831025511,20sleep10-21swapper/120:13:591
171725512,27sleep31805-21apt-config19:23:433
1196125512,19sleep10-21swapper/119:54:011
443025413,19sleep00-21swapper/021:13:420
35512546,10sleep30-21swapper/300:34:063
35512546,10sleep30-21swapper/300:34:063
3114325410,16sleep00-21swapper/020:54:030
3114325410,16sleep00-21swapper/020:54:030
2772325414,23sleep327732-21munin-run00:08:423
158625413,22sleep11604-21apt-get21:03:541
1175425412,17sleep00-21swapper/019:53:560
79725313,17sleep00-21swapper/020:59:200
2546625312,25sleep025460-21apt-get22:18:580
2473825321,15sleep30-21swapper/320:34:083
761025217,15sleep00-21swapper/021:23:430
3265825212,24sleep332649-21apt-get00:23:563
3118125210,17sleep10-21swapper/100:18:581
241725213,23sleep32421-21diskstats22:49:003
1807825212,23sleep018107-21ntp_states21:54:070
1807825212,23sleep018107-21ntp_states21:54:070
1764525218,17sleep30-21swapper/320:09:583
1764525218,17sleep30-21swapper/320:09:583
1301225212,24sleep312994-21apt-get21:39:003
3167525114,22sleep031687-21apt-get22:38:520
2836025115,17sleep10-21swapper/122:28:441
2334725112,24sleep023358-21df_inode23:53:580
2334725112,24sleep023358-21df_inode23:53:580
1927225114,21sleep319303-21df21:58:583
1927225114,21sleep319303-21df21:58:583
1710025113,22sleep117137-21ntp_states20:09:111
1710025113,22sleep117137-21ntp_states20:09:111
712525012,23sleep07101-21apt-get19:39:020
582925016,17sleep30-21swapper/322:59:063
2914525014,18sleep10-21swapper/120:48:441
2587525011,22sleep325920-21ntp_states22:19:093
2394825010,16sleep00-21swapper/022:13:570
1527225017,15sleep10-21swapper/123:28:551
1527225017,15sleep10-21swapper/123:28:551
3008924913,22sleep030107-21apt-get22:33:510
2288924913,17sleep10-21swapper/120:28:531
2060324911,13sleep10-21swapper/123:44:061
19642496,31sleep30-21swapper/322:47:393
576124810,26sleep00-21swapper/019:34:100
2841924814,18sleep00-21swapper/000:09:080
2077924811,20sleep120774-21apt-get22:03:531
2077924811,20sleep120774-21apt-get22:03:531
2988924411,18sleep029965-21systemctl20:49:110
2988924411,18sleep029965-21systemctl20:49:110
910724311,16sleep30-21swapper/319:44:063
2078324210,19sleep020850-21df22:03:560
264242389,17sleep10-21swapper/100:03:561
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional