You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jan 08, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15782297510,2948sleep115576-1kworker/1:2H09:30:071
69272163811,1612sleep025305-1kworker/0:0H11:41:020
846829868,966sleep26259-1kworker/2:2H09:53:472
25787267610,654sleep025305-1kworker/0:0H10:54:260
429524697,454sleep26259-1kworker/2:2H11:29:002
7516244859,281sleep37519-21cron08:04:513
7516244859,281sleep37519-21cron08:04:513
29428240844,305sleep329560-21unixbench_singl09:15:223
11149238127,338sleep311205-21fschecks_count10:05:113
19120237732,228sleep319265-21latency_hist08:44:533
19120237732,228sleep319265-21latency_hist08:44:533
15934236367,215sleep315960-21munin-plugin-st08:34:523
3557236212,337sleep13596-21ntpq11:26:211
3095223597,341sleep115576-1kworker/1:2H07:35:451
3095223597,341sleep115576-1kworker/1:2H07:35:451
22920235549,174sleep323067-21unixbench_multi07:10:233
21645234919,210sleep321780-21unixbench_singl08:50:273
27521232510,192sleep327496-21/usr/sbin/munin11:00:033
1934823067,262sleep319361-21cpu08:45:053
16143229615,265sleep216186-21/usr/sbin/munin10:20:162
197422283110,137sleep319807-21idleruntime12:20:103
1956226627,200sleep31997-21ntp_kernel_pll_07:45:163
10686225589,21sleep30-21swapper/311:50:183
25280223766,116sleep30-21swapper/310:50:013
13793222438,171sleep313907-21timerwakeupswit08:25:233
7692221534,119sleep30-21swapper/311:41:213
31949221541,125sleep331991-21true09:24:523
937822130,9sleep00-21swapper/009:59:540
937822130,9sleep00-21swapper/009:59:540
1756222087,165sleep317646-21cat10:25:143
11262219322,154sleep111280-21sh08:19:511
2802321828,16sleep10-21swapper/107:25:191
2802321828,16sleep10-21swapper/107:25:191
13746217810,132sleep313786-21ntp_kernel_pll_12:00:153
5203217710,113sleep35282-21switchtime11:30:193
17764216633,19sleep30-21swapper/312:14:533
29172215815,67sleep30-21swapper/311:05:113
26291215814,123sleep126323-21latency07:20:151
26291215814,123sleep126323-21latency07:20:151
26120215853,66sleep326189-21cut09:05:213
24640215811,110sleep324681-21ntpq12:35:163
22779215422,104sleep122843-21sort12:30:091
720215327,91sleep3721-21sshd09:25:443
20758215210,128sleep320795-21sed10:35:153
19308215136,45sleep30-21swapper/310:30:183
6247214514,108sleep26318-21/usr/sbin/munin09:45:062
11577214227,21sleep30-21swapper/308:20:063
11577214227,21sleep30-21swapper/308:20:063
23624214121,94sleep123726-21cat07:14:531
15139214021,102sleep015160-21awk08:30:200
8810213710,85sleep38846-21ntpq08:10:123
23941213784,41sleep324060-21perl07:14:553
167621348,113sleep31725-21grep09:30:213
26204213112,102sleep30-21swapper/307:20:123
26204213112,102sleep30-21swapper/307:20:123
24031213048,38sleep30-21swapper/309:00:043
999212929,80sleep11024-21idleruntime-cro11:19:511
4700212912,61sleep30-21swapper/309:40:073
4700212912,61sleep30-21swapper/309:40:073
8666212821,58sleep30-21swapper/309:56:183
12453212820,81sleep212600-21grep11:55:222
5132212716,19sleep20-21swapper/211:30:182
9990212527,26sleep30-21swapper/311:49:533
31956212526,26sleep00-21swapper/011:14:530
30900212419,13sleep30-21swapper/307:35:073
30900212419,13sleep30-21swapper/307:35:073
601121229,34sleep30-21swapper/307:59:543
49621217,73sleep3559-21processes07:40:203
3122212119,62sleep30-21swapper/309:35:063
3122212119,62sleep30-21swapper/309:35:063
28787212119,77sleep228803-21sort09:15:092
27727212126,13sleep30-21swapper/309:10:223
2455421215,11sleep30-21swapper/310:46:543
14516212126,53sleep314131-21/usr/sbin/munin10:15:123
947621209,78sleep39483-21basename09:59:583
398211910,67sleep30-21swapper/311:15:213
24033211813,90sleep024028-21apt-get09:00:050
28054211510,68sleep328120-21systemctl07:25:213
28054211510,68sleep328120-21systemctl07:25:213
1307421147,96sleep313108-21ps10:10:203
21649211313,79sleep221671-21idleruntime-cro10:39:502
16221211332,15sleep30-21swapper/308:35:043
3124211210,58sleep30-21swapper/307:50:053
24098211214,84sleep023660-21munin-node07:14:560
1539121128,93sleep330095-21snmpd12:05:203
158421118,38sleep30-21swapper/311:20:133
158421118,38sleep30-21swapper/311:20:133
3212211010,86sleep03249-21fschecks_count07:50:100
32602210916,77sleep232625-21netstat09:25:152
1827421099,59sleep30-21swapper/312:15:133
16602210915,78sleep216644-21sed08:35:152
20737210722,70sleep020813-21cut07:05:120
20737210722,70sleep020813-21cut07:05:120
1120721067,65sleep30-21swapper/308:19:093
6866210415,64sleep26964-21timerandwakeup08:00:262
6866210415,64sleep26964-21timerandwakeup08:00:262
1807321048,14sleep00-21swapper/008:40:140
1807321048,14sleep00-21swapper/008:40:140
2474321026,77sleep00-21swapper/012:35:210
11975210114,74sleep112008-21/usr/sbin/munin08:20:161
11975210114,74sleep112008-21/usr/sbin/munin08:20:161
9343210015,14sleep00-21swapper/008:10:210
24481210017,63sleep224421-21apt-get07:15:432
12301210018,65sleep012375-21/usr/sbin/munin11:55:190
161872988,78sleep316197-21ntp_states10:20:173
327329718,27sleep30-21swapper/311:26:153
1223929760,20sleep30-21swapper/311:55:183
277922957,11sleep20-21swapper/211:00:102
2150529511,57sleep30-21swapper/312:25:153
48829412,61sleep232450-21/usr/sbin/munin07:40:192
305932939,45sleep330552-21munin-run07:34:543
19452937,13sleep20-21swapper/207:45:152
319929214,63sleep23205-21df_inode07:50:082
1924429214,57sleep019289-21/usr/sbin/munin10:30:150
1910229120,52sleep219116-21latency_hist08:44:522
1910229120,52sleep219116-21latency_hist08:44:522
1113429116,60sleep111143-21df_inode10:05:091
3193129010,46sleep331954-21sed11:14:523
1926929019,54sleep019294-21munin-run12:19:500
815628929,45sleep38203-21systemctl08:09:523
50432885,12sleep20-21swapper/207:55:172
47592887,13sleep00-21swapper/007:55:110
2983828823,44sleep029827-21apt-get09:17:380
2809428710,60sleep128181-21sed11:00:171
2310928611,33sleep30-21swapper/312:30:153
75502857,63sleep37636-21gpgv09:49:523
2207128526,37sleep122163-21fschecks_time10:40:101
2009328516,53sleep020224-21unixbench_singl08:45:230
2378828413,57sleep023809-21latency_hist08:59:530
95722830,7sleep10-21swapper/110:00:031
1387028322,22sleep20-21swapper/210:14:512
3134828214,51sleep031416-21cat09:21:420
1934628213,50sleep10-21swapper/108:45:031
1085028224,32sleep20-21swapper/210:04:532
695528111,22sleep10-21swapper/109:45:221
257528123,38sleep12607-21munin-run11:24:511
257528123,38sleep12607-21munin-run11:24:511
225452818,50sleep30-21swapper/308:55:113
1952228120,38sleep219665-21grep10:30:212
147862817,12sleep20-21swapper/210:15:192
1332428019,44sleep113332-21/usr/sbin/munin12:00:011
2841327925,36sleep028425-21cron09:14:510
2578127920,43sleep125790-21wc07:19:541
2253727923,15sleep30-21swapper/310:40:203
2004727924,33sleep20-21swapper/212:20:172
1765827911,10sleep00-21swapper/012:14:090
1153627925,30sleep20-21swapper/211:54:532
271032788,59sleep330095-21snmpd10:58:293
1552727815,46sleep015670-21wc10:19:540
2559227714,18sleep00-21swapper/009:05:060
1194727720,31sleep00-21swapper/008:20:150
1194727720,31sleep00-21swapper/008:20:150
2887027621,31sleep20-21swapper/207:29:532
2887027621,31sleep20-21swapper/207:29:532
2196227618,41sleep121981-21pluginstate07:09:521
2196227618,41sleep121981-21pluginstate07:09:521
604227520,41sleep16045-21apt-key11:34:531
601027523,24sleep10-21swapper/109:44:531
601027523,24sleep10-21swapper/109:44:531
2727427523,27sleep20-21swapper/207:24:522
2727427523,27sleep20-21swapper/207:24:522
2119327517,26sleep20-21swapper/207:05:222
2119327517,26sleep20-21swapper/207:05:222
2400827429,26sleep10-21swapper/109:00:021
1554827419,38sleep115667-21sed12:05:221
950427315,39sleep09571-21/usr/sbin/munin10:00:030
927827322,27sleep00-21swapper/011:45:220
2196527312,30sleep20-21swapper/210:40:032
2857927213,49sleep00-21swapper/007:26:440
2857927213,49sleep00-21swapper/007:26:440
82692715,10sleep30-21swapper/311:43:193
2066027115,40sleep220766-21latency_hist08:49:532
1985627121,20sleep10-21swapper/112:20:121
155302716,14sleep10-21swapper/108:30:251
90242706,10sleep30-21swapper/309:57:443
90242706,10sleep30-21swapper/309:57:443
2732727017,31sleep00-21swapper/007:24:540
2732727017,31sleep00-21swapper/007:24:540
1655227024,28sleep10-21swapper/112:10:101
1616827021,33sleep28116-21systemd-journal12:09:512
3264126921,25sleep00-21swapper/011:15:180
285292695,54sleep00-21swapper/011:01:290
123762690,8sleep10-21swapper/111:55:201
935526813,18sleep10-21swapper/111:45:231
63126813,38sleep1705-21find07:40:221
3002926819,30sleep029088-21/usr/sbin/munin07:30:270
2755726812,24sleep20-21swapper/209:10:162
2389226828,21sleep10-21swapper/110:45:151
2126226823,27sleep321387-21sed07:05:233
2126226823,27sleep321387-21sed07:05:233
528926711,41sleep05400-21cat11:30:200
1763326718,26sleep10-21swapper/108:39:551
1763326718,26sleep10-21swapper/108:39:551
1729626725,24sleep10-21swapper/110:25:071
1649726711,40sleep016589-21load08:35:130
919226621,24sleep10-21swapper/109:59:521
919226621,24sleep10-21swapper/109:59:521
2878026612,39sleep128786-21diskstats09:15:081
2092426626,20sleep00-21swapper/010:35:200
1077726614,35sleep110850-21sort11:50:201
2965426517,27sleep10-21swapper/111:05:221
2530526516,23sleep20-21swapper/209:04:512
2286326516,33sleep222908-21/usr/sbin/munin08:55:182
844226422,23sleep20-21swapper/208:10:032
699826424,26sleep16911-21apt-get11:41:031
332426419,29sleep13363-21/usr/sbin/munin09:35:121
332426419,29sleep13363-21/usr/sbin/munin09:35:121
2649026422,22sleep126543-21users10:54:401
2372826419,27sleep223937-21df_inode07:14:542
1730926414,31sleep217356-21users12:10:242
824826321,19sleep10-21swapper/108:09:541
803026311,36sleep07524-21apt-get09:50:050
651526315,27sleep00-21swapper/009:45:130
58282635,11sleep00-21swapper/009:43:110
58282635,11sleep00-21swapper/009:43:110
515226312,36sleep25158-21ntp_states09:40:182
515226312,36sleep25158-21ntp_states09:40:182
438426322,22sleep10-21swapper/107:54:511
42392639,41sleep00-21swapper/011:28:180
321926310,36sleep03323-21grep09:35:110
321926310,36sleep03323-21grep09:35:110
277426322,31sleep00-21swapper/009:34:060
231682636,49sleep00-21swapper/010:44:000
206102638,10sleep00-21swapper/012:21:480
102626318,27sleep21077-21munin-run11:19:522
7942625,18sleep00-21swapper/009:26:380
446226215,30sleep20-21swapper/209:39:532
446226215,30sleep20-21swapper/209:39:532
3102826218,30sleep231070-21ntp_offset11:10:152
27922626,44sleep00-21swapper/007:49:170
2460926215,24sleep20-21swapper/212:35:152
2292826213,18sleep10-21swapper/108:55:211
1471026213,37sleep114785-21sed10:15:181
1443626213,37sleep214554-21latency_hist08:29:542
129626214,28sleep11498-21latency_hist09:29:541
625126116,29sleep06246-21apt-get08:00:080
625126116,29sleep06246-21apt-get08:00:080
2330626116,28sleep023412-21ls12:30:210
2223826119,23sleep10-21swapper/112:27:431
190262615,10sleep20-21swapper/212:16:502
1581926113,21sleep00-21swapper/012:05:250
136442615,17sleep10-21swapper/110:11:551
1207826120,22sleep20-21swapper/210:06:432
617626012,31sleep26346-21apt-config11:34:542
3116325911,32sleep230400-21/usr/sbin/munin09:21:352
2099625912,34sleep121094-21cat10:35:211
1838025915,16sleep10-21swapper/112:15:151
157192598,37sleep015738-21apt-get10:20:020
1352825917,22sleep10-21swapper/108:25:171
3050225819,20sleep20-21swapper/207:34:532
2478425816,26sleep224798-21apt-get10:49:522
2142825812,22sleep20-21swapper/212:25:142
999325714,27sleep09987-21apt-get08:15:060
2973625713,25sleep20-21swapper/211:05:232
157925715,25sleep21646-21df09:30:082
90392567,12sleep10-21swapper/108:10:181
300742566,39sleep10-21swapper/107:31:011
1775425622,17sleep00-21swapper/010:25:190
1143325623,19sleep00-21swapper/010:05:160
820525520,19sleep00-21swapper/008:09:530
3166825511,33sleep031775-21cat07:36:330
3166825511,33sleep031775-21cat07:36:330
2491525520,19sleep00-21swapper/010:49:540
2134625511,32sleep021109-21/usr/sbin/munin12:25:120
2014425516,24sleep120153-21munin-run10:34:531
1736725515,24sleep217373-21df_inode10:25:082
1654525516,23sleep116644-21users10:20:231
157825514,15sleep10-21swapper/107:45:061
1261125515,24sleep012607-21apt-get10:10:040
3201625418,19sleep10-21swapper/109:24:541
2432725413,21sleep10-21swapper/112:35:101
2355425416,23sleep223595-21users12:30:232
1466925415,17sleep20-21swapper/212:04:542
2715925315,22sleep127226-21df09:10:071
2546025316,22sleep225468-21df_inode10:51:052
1157625313,23sleep211572-21apt-get08:20:042
1157625313,23sleep211572-21apt-get08:20:042
668825215,22sleep16707-21ntp_states08:00:211
668825215,22sleep16707-21ntp_states08:00:211
24612525,36sleep00-21swapper/011:23:250
24612525,36sleep00-21swapper/011:23:250
2147825221,17sleep10-21swapper/108:50:251
2117225216,15sleep00-21swapper/008:50:150
1379025212,17sleep00-21swapper/012:00:160
1061625214,15sleep00-21swapper/011:50:150
925925111,27sleep28768-21systemd-journal09:59:532
925925111,27sleep28768-21systemd-journal09:59:532
3271325116,18sleep00-21swapper/007:40:130
2926625113,16sleep00-21swapper/011:05:120
234232515,35sleep00-21swapper/007:13:550
172225114,22sleep21767-21/usr/sbin/munin11:20:162
172225114,22sleep21767-21/usr/sbin/munin11:20:162
904725013,17sleep20-21swapper/211:45:162
3087025010,19sleep20-21swapper/207:35:022
3087025010,19sleep20-21swapper/207:35:022
1598925018,15sleep20-21swapper/208:34:552
803224913,20sleep18024-21apt-get09:50:061
1330824914,15sleep00-21swapper/008:25:110
306312488,16sleep10-21swapper/111:10:061
2728524813,18sleep20-21swapper/210:59:542
2152624712,15sleep20-21swapper/208:50:262
2563324612,17sleep10-21swapper/109:05:071
829224514,18sleep130095-21snmpd11:43:371
82332454,28sleep20-21swapper/211:42:502
2721124511,19sleep027242-21munin-run10:59:510
1632024514,14sleep10-21swapper/108:35:101
99852449,19sleep29987-21apt-get08:15:042
2580124414,16sleep20-21swapper/209:05:122
467524213,16sleep10-21swapper/107:55:081
272752396,15sleep10-21swapper/110:59:521
84422367,11sleep00-21swapper/011:44:540
79082317,14sleep20-21swapper/208:06:402
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional