You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 21:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jun 26, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5031270849,640sleep35173-21grep07:45:273
18991267921,641sleep119024-21/usr/sbin/munin08:30:151
3827261756,433sleep33889-21fschecks_time11:10:093
24205257019,533sleep024220-21/usr/sbin/munin10:30:160
24205257019,533sleep024220-21/usr/sbin/munin10:30:160
12335241023,363sleep212352-21apt-get08:10:012
19237233130,121sleep30-21swapper/308:30:223
2834323219,295sleep028379-21ntpq09:00:150
32611226821,25sleep00-21swapper/009:15:050
9017224928,207sleep39031-21cpu08:00:053
9017224928,207sleep39031-21cpu08:00:053
4719224311,218sleep04764-21awk07:45:200
31645222045,113sleep30-21swapper/307:29:523
2640822209,153sleep326521-21swap07:14:533
20569222013,189sleep3225-21systemd-journal12:04:493
29677221627,176sleep329794-21users12:30:213
2596221611,152sleep32716-21apt-key07:39:543
6462220935,122sleep30-21swapper/309:35:063
6462220935,122sleep30-21swapper/309:35:063
3053220840,108sleep30-21swapper/309:24:533
13047220751,96sleep313173-21cat08:10:273
3371220413,131sleep33404-21fschecks_count09:25:093
3371220413,131sleep33404-21fschecks_count09:25:093
968422030,11sleep20-21swapper/211:29:492
1440219016,138sleep31477-21proc_pri07:35:173
17490218511,162sleep317603-21latency_hist11:54:513
25491218037,125sleep00-21swapper/008:50:210
633217221,93sleep30-21swapper/311:00:033
15760216746,101sleep30-21swapper/308:20:123
15760216746,101sleep30-21swapper/308:20:123
8417216410,54sleep30-21swapper/311:25:033
21921216343,24sleep30-21swapper/308:40:113
9258216110,94sleep39289-21latency_hist09:44:513
9742216018,116sleep09887-21sed08:00:220
9742216018,116sleep09887-21sed08:00:220
2386421599,133sleep00-21swapper/008:45:210
11094215945,14sleep30-21swapper/311:33:113
11094215945,14sleep30-21swapper/311:33:113
11831215511,28sleep311887-21apt-config08:09:503
20411215218,120sleep217009-21kworker/u8:010:19:522
2040521509,103sleep320419-21grep08:35:143
19034214510,63sleep318560-21apt-get10:15:003
12154214562,68sleep312273-21timerwakeupswit11:35:253
22115214350,80sleep322141-21munin-run12:09:503
22115214350,80sleep322141-21munin-run12:09:503
725221417,123sleep37297-21ntp_kernel_pll_11:20:143
27114213910,113sleep327233-21ls12:24:523
27114213910,113sleep327233-21ls12:24:523
10067213946,26sleep30-21swapper/309:45:203
2374221378,16sleep30-21swapper/310:30:023
2374221378,16sleep30-21swapper/310:30:023
13757213710,111sleep213872-21ls11:40:212
2793213011,103sleep12782-21apt-get07:40:041
30766212913,69sleep10-21swapper/107:26:021
28597212813,101sleep328610-21cron12:29:493
28597212813,101sleep328610-21cron12:29:493
15473212840,48sleep00-21swapper/008:20:010
15473212840,48sleep00-21swapper/008:20:010
6491212721,67sleep30-21swapper/307:50:203
15579212713,100sleep0364-21snmpd11:45:440
27083212614,91sleep127229-21/usr/sbin/munin08:55:231
418212550,60sleep3441-21sshd09:15:123
25318212511,98sleep325422-21cut12:19:513
25318212511,98sleep325422-21cut12:19:513
25014211942,53sleep30-21swapper/307:10:173
25014211942,53sleep30-21swapper/307:10:173
18443211920,11sleep30-21swapper/310:13:433
2799211821,79sleep22782-21apt-get07:40:092
5204211768,34sleep30-21swapper/309:30:143
30759211720,73sleep230804-21munin-run09:09:502
16958211616,11sleep30-21swapper/310:09:313
39121129,65sleep3435-21latency_hist10:59:513
326621127,96sleep00-21swapper/011:07:270
26555211212,83sleep026557-21sshd10:39:490
26555211212,83sleep026557-21sshd10:39:490
21325211019,77sleep021336-21open_files12:05:180
21325211019,77sleep021336-21open_files12:05:180
13935210921,64sleep113897-21latency_hist09:59:521
30824210818,72sleep030914-21ls09:09:520
2909121088,66sleep327637-21kworker/u8:109:03:163
1593521089,56sleep315956-21sort11:49:503
11468210810,62sleep311505-21ntp_states09:50:173
879221078,61sleep38929-21cut07:59:533
2190721067,62sleep321931-21munin-run10:24:493
2190721067,62sleep321931-21munin-run10:24:493
14409210515,73sleep114148-21/usr/sbin/munin10:00:121
6549210421,62sleep26580-21munin-plugin-st11:19:492
29185210445,18sleep30-21swapper/307:20:233
2801121049,62sleep3364-21snmpd10:43:153
2801121049,62sleep3364-21snmpd10:43:153
8180210312,78sleep08293-21cat11:24:520
5655210313,74sleep15734-21latency_hist07:49:531
24573210325,23sleep30-21swapper/308:49:523
14285210313,74sleep0364-21snmpd11:44:020
30489210220,69sleep30-21swapper/312:35:063
17719210110,51sleep317726-21sshd11:54:593
17719210110,51sleep317726-21sshd11:54:593
6841210012,74sleep06845-21sh09:35:160
6841210012,74sleep06845-21sh09:35:160
527021008,50sleep30-21swapper/311:15:013
22703210013,32sleep322776-21awk10:25:183
2571729911,71sleep225724-21users08:50:232
860029812,53sleep00-21swapper/009:40:200
84929810,74sleep2905-21/usr/sbin/munin11:00:102
2394629824,57sleep124075-21unixbench_multi08:45:221
1860129825,51sleep218684-21apt-config10:14:512
909929712,15sleep20-21swapper/208:00:082
909929712,15sleep20-21swapper/208:00:082
2528329613,60sleep125320-21ps08:50:181
2582629521,46sleep20-21swapper/212:20:272
2582629521,46sleep20-21swapper/212:20:272
1898729421,53sleep018994-21modprobe11:59:490
1898729421,53sleep018994-21modprobe11:59:490
1425929421,21sleep30-21swapper/310:00:103
125362944,78sleep00-21swapper/011:36:470
1207729315,65sleep212153-21cat11:35:242
459829235,36sleep20-21swapper/209:29:522
459829235,36sleep20-21swapper/209:29:522
1549129120,18sleep00-21swapper/010:04:510
301682909,64sleep230247-21store07:24:542
842028918,48sleep08392-21/usr/sbin/munin11:25:040
2577928919,52sleep125257-21apt-get12:20:011
2577928919,52sleep125257-21apt-get12:20:011
2062128920,46sleep10-21swapper/112:04:511
184932898,68sleep00-21swapper/010:14:200
687428828,34sleep20-21swapper/211:20:052
142428818,37sleep10-21swapper/109:19:501
1027028818,56sleep110372-21sed09:45:221
1535128719,45sleep215488-21tail11:45:212
3215128510,62sleep032166-21kernelversion07:30:130
2315028519,41sleep223292-21unixbench_singl12:10:212
141702856,65sleep30-21swapper/311:42:323
2980428424,36sleep20-21swapper/207:24:502
307662838,15sleep30-21swapper/310:50:223
1713528322,40sleep017223-21fschecks_time08:25:100
1713528322,40sleep017223-21fschecks_time08:25:100
967428217,46sleep1225-21systemd-journal11:29:491
491928221,41sleep24983-21cut09:30:092
3115028011,57sleep331204-21fschecks_count09:10:113
2370528034,31sleep30-21swapper/312:14:503
420527930,29sleep00-21swapper/011:10:180
2842427924,40sleep30-21swapper/310:45:053
263702798,55sleep30-21swapper/308:55:053
1638027926,34sleep00-21swapper/010:05:200
316092785,12sleep00-21swapper/012:38:420
2378427823,19sleep20-21swapper/207:05:162
172127815,43sleep01855-21grep07:35:220
2405227722,35sleep124145-21fschecks_time12:15:091
2405227722,35sleep124145-21fschecks_time12:15:091
2183527723,28sleep20-21swapper/208:40:102
73927627,37sleep30-21swapper/307:34:533
714527612,45sleep17238-21/usr/sbin/munin11:20:121
355327623,35sleep30-21swapper/311:09:523
2717827611,47sleep027235-21irqstats07:15:150
257172769,12sleep30-21swapper/310:35:153
257172769,12sleep30-21swapper/310:35:153
2542627616,33sleep025611-21gpgv12:19:520
2542627616,33sleep025611-21gpgv12:19:520
2052027616,34sleep220116-21/usr/sbin/munin08:35:172
182327613,47sleep21824-21diskstats09:20:072
1173627624,31sleep211894-21sed09:50:242
2523527520,39sleep21-21systemd12:19:502
2523527520,39sleep21-21systemd12:19:502
170452758,51sleep30-21swapper/308:25:053
170452758,51sleep30-21swapper/308:25:053
461927412,41sleep20-21swapper/207:45:182
3122627417,37sleep231228-21cron10:54:492
2645627412,45sleep026459-21/usr/sbin/munin08:55:100
1427527418,42sleep114288-21needreboot08:15:131
1260127420,38sleep30-21swapper/309:55:053
1233127415,41sleep111859-21apt-get08:10:001
311292730,7sleep20-21swapper/212:35:202
2705227321,23sleep20-21swapper/207:15:132
1826527316,25sleep10-21swapper/111:55:181
1826527316,25sleep10-21swapper/111:55:181
1271527316,43sleep012808-21fschecks_time09:55:100
1126627324,23sleep20-21swapper/211:34:512
1126627324,23sleep20-21swapper/211:34:512
577427211,46sleep05852-21basename11:15:190
3104527223,29sleep10-21swapper/112:35:201
2558727215,30sleep20-21swapper/207:10:322
2558727215,30sleep20-21swapper/207:10:322
2428727217,35sleep223717-21/usr/sbin/munin10:30:182
2428727217,35sleep223717-21/usr/sbin/munin10:30:182
1354127226,25sleep10-21swapper/111:40:151
43927120,28sleep00-21swapper/010:59:520
3005827113,18sleep30-21swapper/309:05:223
279202718,15sleep20-21swapper/209:00:002
248192717,13sleep30-21swapper/308:50:043
1612427118,35sleep216157-21/usr/sbin/munin10:05:142
2663627020,29sleep20-21swapper/210:39:512
2663627020,29sleep20-21swapper/210:39:512
589126921,27sleep00-21swapper/007:50:040
2917326912,40sleep229257-21memory12:30:122
201412699,13sleep00-21swapper/010:17:230
1636826910,44sleep216429-21cat11:50:102
138832695,11sleep30-21swapper/308:14:593
814826810,33sleep28292-21grep07:55:232
349926824,29sleep20-21swapper/211:09:512
314422686,51sleep00-21swapper/007:27:550
3001326818,31sleep130167-21apt-key07:24:531
2852226823,22sleep10-21swapper/109:00:201
2467226811,40sleep124708-21munin-run07:09:511
1727626817,28sleep10-21swapper/110:10:051
534226720,24sleep00-21swapper/009:30:190
2790826714,39sleep227978-21sed12:25:202
2790826714,39sleep227978-21sed12:25:202
2147926725,22sleep00-21swapper/008:39:510
1600126719,35sleep216053-21mailstats08:20:192
1600126719,35sleep216053-21mailstats08:20:192
123826724,24sleep10-21swapper/107:35:111
35426616,34sleep2417-21irqstats09:15:112
2946826620,20sleep10-21swapper/112:30:191
2851126618,24sleep00-21swapper/010:45:100
2350726615,24sleep00-21swapper/010:29:510
1505026616,29sleep00-21swapper/008:18:110
2304626521,22sleep00-21swapper/008:44:520
1367526516,24sleep00-21swapper/008:14:520
2725626419,27sleep10-21swapper/110:40:141
2725626419,27sleep10-21swapper/110:40:141
1461326426,19sleep10-21swapper/111:45:001
1428926424,22sleep20-21swapper/208:15:142
1231026414,28sleep0278-21dbus-daemon09:54:500
3158726310,38sleep231594-21df_inode10:55:052
308912636,13sleep00-21swapper/010:50:270
2522226322,21sleep10-21swapper/107:10:231
2522226322,21sleep10-21swapper/107:10:231
247432637,41sleep324889-21ls07:09:533
2090026314,23sleep10-21swapper/108:35:241
1113826315,29sleep111231-21grep09:50:111
2780726213,26sleep00-21swapper/012:25:170
2780726213,26sleep00-21swapper/012:25:170
2773426215,23sleep20-21swapper/208:59:522
1189126212,35sleep011991-21sed08:09:510
1106126218,27sleep211126-21awk08:05:212
998926117,22sleep20-21swapper/209:45:192
2687726126,20sleep00-21swapper/010:40:050
2687726126,20sleep00-21swapper/010:40:050
1033526119,23sleep10-21swapper/108:04:511
1033526119,23sleep10-21swapper/108:04:511
636026016,30sleep16401-21processes07:50:181
43872605,44sleep00-21swapper/009:27:480
43872605,44sleep00-21swapper/009:27:480
431726015,19sleep20-21swapper/211:10:202
32462604,13sleep10-21swapper/111:07:121
3240726014,23sleep00-21swapper/009:14:520
310382606,15sleep10-21swapper/110:52:221
2910026021,15sleep00-21swapper/012:30:100
2625626019,18sleep00-21swapper/007:14:500
2625626019,18sleep00-21swapper/007:14:500
2249326012,32sleep122505-21fschecks_count12:10:081
1572926014,24sleep115796-21df10:05:051
844325918,20sleep20-21swapper/209:40:182
554625914,27sleep15560-21irqstats11:15:121
2879425915,28sleep128825-21/usr/sbin/munin10:45:151
2821225917,20sleep20-21swapper/210:44:522
2821225917,20sleep20-21swapper/210:44:522
2655225917,23sleep10-21swapper/107:14:561
2539025912,33sleep125482-21fschecks_time10:35:091
2539025912,33sleep125482-21fschecks_time10:35:091
2368725921,17sleep00-21swapper/007:05:140
2105625916,24sleep121111-21if_eth012:05:111
2105625916,24sleep121111-21if_eth012:05:111
1929125920,19sleep10-21swapper/110:16:071
182112594,12sleep20-21swapper/208:28:392
182112594,12sleep20-21swapper/208:28:392
1417325913,28sleep214167-21apt-get10:00:052
1317725920,21sleep20-21swapper/209:55:202
125625915,25sleep21345-21memory07:35:122
109925923,22sleep00-21swapper/011:00:160
958925813,25sleep00-21swapper/009:45:090
62725818,23sleep1629-21apt-get11:00:011
3127625811,25sleep10-21swapper/109:10:131
2265725814,20sleep10-21swapper/110:25:161
2020425813,26sleep020218-21vmstat12:00:220
684625719,20sleep10-21swapper/109:35:171
684625719,20sleep10-21swapper/109:35:171
109672577,39sleep00-21swapper/011:31:300
109672577,39sleep00-21swapper/011:31:300
335625617,24sleep13363-21df_inode09:25:071
335625617,24sleep13363-21df_inode09:25:071
2195225619,21sleep10-21swapper/110:24:511
2195225619,21sleep10-21swapper/110:24:511
174132565,39sleep00-21swapper/011:54:280
170962558,12sleep10-21swapper/108:25:071
170962558,12sleep10-21swapper/108:25:071
1623425521,18sleep10-21swapper/108:20:221
1623425521,18sleep10-21swapper/108:20:221
65525414,22sleep10-21swapper/107:34:261
463925413,20sleep10-21swapper/111:10:231
2947525410,21sleep10-21swapper/109:05:001
2589925414,19sleep00-21swapper/012:20:290
2589925414,19sleep00-21swapper/012:20:290
1199225414,19sleep10-21swapper/108:09:521
80432538,16sleep10-21swapper/109:40:091
640225315,16sleep20-21swapper/207:50:192
637725315,24sleep26373-21apt-get09:35:012
637725315,24sleep26373-21apt-get09:35:012
3188325215,21sleep231881-21apt-get07:30:032
281025215,19sleep00-21swapper/007:40:150
2783125217,17sleep10-21swapper/112:25:181
2783125217,17sleep10-21swapper/112:25:181
2332625212,23sleep223327-21sshd10:28:122
2109225212,24sleep221103-21ntp_states10:20:162
2109225212,24sleep221103-21ntp_states10:20:162
1989225214,22sleep019902-21munin-run08:34:510
1859225212,22sleep218586-21apt-get08:30:042
1773025214,22sleep217722-21apt-get11:55:032
1773025214,22sleep217722-21apt-get11:55:032
173725213,23sleep01733-21apt-get09:20:010
1680425215,21sleep116878-21find11:50:331
295412516,33sleep00-21swapper/007:21:290
2371625113,18sleep00-21swapper/012:14:510
1994825110,18sleep20-21swapper/212:00:202
997125014,16sleep10-21swapper/111:30:061
997125014,16sleep10-21swapper/111:30:061
789825011,22sleep17907-21ntp_states07:55:181
744225013,19sleep00-21swapper/007:55:050
3232125011,16sleep20-21swapper/209:14:502
2217125011,22sleep122177-21ntp_states08:40:171
1285024912,15sleep10-21swapper/111:39:511
248662475,32sleep10-21swapper/110:32:591
248662475,32sleep10-21swapper/110:32:591
2145224610,18sleep021551-21users10:20:220
2145224610,18sleep021551-21users10:20:220
195024613,19sleep12007-21irqstats09:20:101
3152024511,16sleep10-21swapper/110:55:041
232412459,13sleep20-21swapper/208:45:022
209392448,14sleep20-21swapper/212:05:082
209392448,14sleep20-21swapper/212:05:082
2849524311,17sleep20-21swapper/210:45:082
48332418,17sleep10-21swapper/109:30:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional