You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 17:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Aug 13, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26145246887,4666sleep113879-21kworker/u8:111:55:331
259092119315,1157sleep013879-21kworker/u8:111:55:280
21755273019,695sleep321778-21users08:15:273
575026747,655sleep15791-21unin-run12:34:541
16360253611,511sleep216401-21ntpq08:00:202
16537247918,446sleep216517-21latency_hist09:44:542
16537247918,446sleep216517-21latency_hist09:44:542
535223999,355sleep15355-21systemctl09:07:381
189202394154,200sleep119053-21sort08:09:561
31755228997,167sleep10-21swapper/112:14:551
9761228311,250sleep027228-1kworker/0:1H07:40:110
9761228311,250sleep027228-1kworker/0:1H07:40:110
8694227571,188sleep18756-21modprobe09:19:541
655722689,198sleep16687-21grep09:10:251
2331226415,235sleep32373-21ntpq07:15:183
20960224942,191sleep121022-21fschecks_time08:15:131
1001722428,198sleep110049-21ps11:05:541
3329223635,178sleep13473-21latency_hist07:19:571
29589223640,172sleep129736-21unixbench_multi12:05:261
28993223416,195sleep029086-21fschecks_time12:05:130
14456222676,34sleep132223-21cyclictest11:20:151
2154221635,132sleep12265-21ls10:40:261
20406220418,55sleep220435-21cat11:39:552
107472202111,26sleep132223-21cyclictest09:25:151
5150219834,117sleep10-21swapper/110:50:231
38522195102,26sleep13970-21sed09:04:561
38522195102,26sleep13970-21sed09:04:561
1411221927,38sleep10-21swapper/107:54:531
2904217911,153sleep02422-21apt-get12:25:040
2265621798,159sleep122695-21wc11:45:191
2620721789,157sleep126299-21fschecks_time10:15:141
22987217813,150sleep123007-21apt-get10:05:041
1406821759,148sleep10-21swapper/109:35:201
1406821759,148sleep10-21swapper/109:35:201
30620216918,96sleep10-21swapper/110:29:561
3247521639,121sleep132516-21grep12:15:231
2912216210,73sleep10-21swapper/112:25:051
9330214718,66sleep29454-21latency_hist11:04:582
3508214554,17sleep10-21swapper/107:20:051
26866214510,66sleep10-21swapper/108:34:551
18805214315,18sleep00-21swapper/011:34:560
18805214315,18sleep00-21swapper/011:34:560
32227213814,103sleep232281-21/usr/sbin/munin12:15:162
21013213523,52sleep10-21swapper/111:40:181
3033213415,86sleep00-21swapper/010:45:090
35421329,109sleep1393-21ps10:35:211
15805212912,23sleep015814-21df_inode11:25:110
27888212851,26sleep10-21swapper/112:00:251
11064212810,20sleep10-21swapper/107:44:561
11064212810,20sleep10-21swapper/107:44:561
7509212719,50sleep10-21swapper/110:59:561
1918721279,40sleep119190-21diskstats11:35:381
20415212615,59sleep10-21swapper/109:55:231
20415212615,59sleep10-21swapper/109:55:231
30625212412,63sleep10-21swapper/108:45:171
17057212425,85sleep117196-21latency_hist11:29:561
29008211910,97sleep129119-21date10:24:551
4899211820,18sleep10-21swapper/107:24:571
12111211820,54sleep10-21swapper/109:30:071
32447211510,58sleep132223-21cyclictest07:10:071
3220721150,10sleep232200-21ls07:09:552
32691211311,87sleep032694-21/usr/sbin/munin08:50:250
15793211212,56sleep10-21swapper/109:40:251
15793211212,56sleep10-21swapper/109:40:251
16779211136,35sleep10-21swapper/109:45:061
12881211021,71sleep213010-21awk09:30:252
25409210858,33sleep10-21swapper/111:55:041
11124210734,33sleep10-21swapper/111:10:101
7398210612,77sleep07406-21acpi09:15:040
9795210511,77sleep09914-21users09:20:250
9795210511,77sleep09914-21users09:20:250
10028210516,73sleep310057-21kernelversion07:40:183
10028210516,73sleep310057-21kernelversion07:40:183
8713210449,36sleep10-21swapper/107:35:231
3029210410,51sleep10-21swapper/110:45:061
89021038,52sleep10-21swapper/108:55:061
17649210311,41sleep10-21swapper/111:30:171
17649210311,41sleep10-21swapper/111:30:171
1507521039,54sleep10-21swapper/107:55:261
14470210312,73sleep214553-21memory11:20:162
53821025,54sleep10-21swapper/108:54:051
458421024,89sleep00-21swapper/007:22:140
9678210114,69sleep39791-21timerwakeupswit09:20:243
9678210114,69sleep39791-21timerwakeupswit09:20:243
9194210115,51sleep19329-21cat11:04:571
31780210110,17sleep20-21swapper/208:50:072
31410210122,62sleep131494-21switchtime07:05:231
22546210054,31sleep122567-21fschecks_count08:20:131
304312986,69sleep00-21swapper/012:10:050
2771129810,44sleep10-21swapper/108:35:241
2379829816,65sleep023944-21latency_hist08:24:570
2493329717,63sleep324964-21/usr/sbin/munin10:10:193
1127329714,68sleep011413-21unixbench_singl09:25:260
250122968,76sleep125023-21ntpq10:10:211
1284529620,34sleep10-21swapper/111:15:151
1081529512,66sleep310916-21latency_hist11:09:553
192229425,37sleep20-21swapper/207:15:052
2868529210,40sleep10-21swapper/108:40:081
452329111,67sleep24559-21idleruntime09:06:062
3053729114,56sleep330432-21/usr/sbin/munin12:10:123
242462917,47sleep123982-21/usr/sbin/munin08:25:141
171992917,74sleep1367-21snmpd08:04:181
2145629010,16sleep10-21swapper/110:00:081
1964229020,16sleep10-21swapper/109:54:551
708528913,60sleep07091-21ntp_states07:30:210
3242228920,22sleep20-21swapper/207:10:042
2352628814,56sleep023591-21apt-key11:49:540
2345628817,49sleep223458-21cron11:49:522
193742877,39sleep10-21swapper/108:10:441
318028618,43sleep33318-21grep09:00:293
318028618,43sleep33318-21grep09:00:293
1996628618,51sleep219997-21tune2fs09:55:122
1996628618,51sleep219997-21tune2fs09:55:122
35728517,50sleep3401-21ntpq07:10:173
1851528521,39sleep00-21swapper/009:50:130
1328228529,23sleep30-21swapper/311:15:253
728628419,48sleep37306-21apt12:39:533
723128413,54sleep07373-21latency_hist09:14:560
1375028416,44sleep313763-21/usr/sbin/munin07:50:273
1151528419,50sleep311563-21if_eth007:45:143
1151528419,50sleep311563-21if_eth007:45:143
2866728322,39sleep228742-21latency_hist12:04:542
3134128228,34sleep00-21swapper/007:05:220
2681428215,50sleep326824-21modprobe08:34:533
23832825,11sleep10-21swapper/112:24:361
3259528118,44sleep332656-21switchtime12:15:253
2728928117,47sleep327347-21if_err_eth008:35:143
2245228113,53sleep222454-21apt-get08:20:062
1198028133,11sleep10-21swapper/107:45:251
1198028133,11sleep10-21swapper/107:45:251
943528019,38sleep39437-21cron07:39:533
302832805,11sleep10-21swapper/110:27:011
2648128020,29sleep20-21swapper/210:15:192
2419028010,58sleep024245-21if_err_eth008:25:130
2400728012,55sleep324035-21grep08:25:083
1656228017,44sleep316710-21wc09:44:553
1656228017,44sleep316710-21wc09:44:553
2544727915,45sleep325559-21latency_hist11:55:183
1959127920,36sleep319593-21cron09:54:533
3261727816,44sleep332638-21sed10:35:153
240727825,30sleep22432-21munin-run12:24:532
160102780,7sleep20-21swapper/211:25:162
225727725,26sleep30-21swapper/308:59:573
696127631,11sleep10-21swapper/112:35:431
2395627626,29sleep20-21swapper/211:50:042
1879327614,45sleep318834-21http08:09:543
424627515,45sleep34261-21latency_hist12:29:553
3089227519,39sleep330987-21fschecks_time10:30:143
2432427515,43sleep324409-21latency_hist10:09:553
2114327521,32sleep00-21swapper/008:15:160
1776527516,40sleep217837-21mailstats11:30:222
1776527516,40sleep217837-21mailstats11:30:222
107527519,30sleep00-21swapper/008:55:130
878527416,40sleep28883-21timerandwakeup07:35:242
621527429,22sleep20-21swapper/210:55:132
289512748,48sleep00-21swapper/008:40:150
2664327417,26sleep30-21swapper/310:15:243
1361227420,26sleep20-21swapper/207:50:262
94122737,55sleep19457-21ntpq09:20:181
94122737,55sleep19457-21ntpq09:20:181
650827319,34sleep26570-21wc12:35:232
2861727315,43sleep028618-21cron12:04:520
208762738,43sleep20-21swapper/208:15:102
788327229,11sleep10-21swapper/107:34:511
2118327210,19sleep30-21swapper/311:40:243
1214327110,47sleep312144-21sshd11:12:423
308027022,22sleep20-21swapper/209:00:282
308027022,22sleep20-21swapper/209:00:282
2735527016,22sleep30-21swapper/312:00:113
963326914,25sleep00-21swapper/011:05:450
640226912,33sleep30-21swapper/307:29:563
2531926920,23sleep20-21swapper/208:29:562
2478126915,25sleep00-21swapper/011:50:430
910226814,39sleep09193-21dpkg11:04:560
475126819,27sleep20-21swapper/212:30:142
2578726818,31sleep325263-21apt-get11:55:223
908926714,33sleep29096-21/usr/sbin/munin09:20:112
908926714,33sleep29096-21/usr/sbin/munin09:20:112
3215226711,40sleep332180-21/usr/sbin/munin08:50:163
3055526717,23sleep00-21swapper/010:29:550
292726712,23sleep20-21swapper/212:25:062
1499826715,26sleep00-21swapper/009:39:560
1499826715,26sleep00-21swapper/009:39:560
120982675,12sleep00-21swapper/011:12:070
3128226616,33sleep031354-21mailstats10:30:230
3031126612,38sleep230315-21diskstats08:45:102
214582667,16sleep20-21swapper/210:00:092
2102726615,35sleep221064-21/usr/sbin/munin11:40:192
1988626615,31sleep00-21swapper/008:11:360
1815026616,22sleep00-21swapper/008:05:220
1566126622,28sleep015686-21munin-run07:59:530
2578626518,29sleep225428-21apt-get11:55:212
2285926515,31sleep322901-21awk08:20:203
116326511,39sleep21253-21load08:55:152
190372649,41sleep318554-21apt-get11:35:033
1732226419,25sleep20-21swapper/209:45:232
1307526417,35sleep013133-21awk11:15:220
118222648,13sleep30-21swapper/309:29:533
2806426314,21sleep00-21swapper/010:20:180
2058526316,25sleep30-21swapper/308:14:543
1864226321,19sleep30-21swapper/311:34:553
1864226321,19sleep30-21swapper/311:34:553
161126315,30sleep31653-21wc12:20:223
1498826310,21sleep30-21swapper/311:20:253
456826219,19sleep30-21swapper/312:30:113
206226217,18sleep00-21swapper/010:40:250
1746426214,28sleep00-21swapper/009:45:250
1734126213,34sleep317482-21latency_hist08:04:573
1625526215,18sleep00-21swapper/008:00:180
1574026224,21sleep20-21swapper/207:59:552
582226115,23sleep00-21swapper/007:25:250
562726115,31sleep35638-21munin-run09:09:543
3177026117,17sleep20-21swapper/212:14:562
1530726118,26sleep315314-21vmstat07:55:283
877426016,22sleep30-21swapper/309:19:563
773826012,32sleep37779-21df11:00:103
589426019,21sleep20-21swapper/210:54:562
564026018,20sleep00-21swapper/009:09:550
516026014,28sleep00-21swapper/012:30:250
2917726021,18sleep30-21swapper/312:05:163
1374426019,21sleep20-21swapper/209:35:132
1374426019,21sleep20-21swapper/209:35:132
1366026011,36sleep313709-21grep09:35:103
1366026011,36sleep313709-21grep09:35:103
1176726013,23sleep211803-21wc07:45:222
1176726013,23sleep211803-21wc07:45:222
718425922,21sleep30-21swapper/309:14:553
360525914,31sleep33611-21df_inode07:20:113
2898325911,32sleep228989-21diskstats12:05:112
2796925913,29sleep328061-21load10:20:173
1675525913,34sleep31369-21munin-node09:45:033
155725916,25sleep31564-21df_inode10:40:113
638025821,20sleep20-21swapper/207:29:552
494425820,25sleep30-21swapper/310:50:173
3088025813,21sleep20-21swapper/210:30:132
303002586,40sleep30-21swapper/310:27:153
2702525813,20sleep00-21swapper/010:15:290
252425817,22sleep00-21swapper/009:00:140
252425817,22sleep00-21swapper/009:00:140
1987825816,26sleep019873-21apt-get09:55:060
1987825816,26sleep019873-21apt-get09:55:060
1572325812,24sleep00-21swapper/009:40:240
1572325812,24sleep00-21swapper/009:40:240
2764025715,21sleep00-21swapper/008:35:230
155725712,28sleep01592-21/usr/sbin/munin12:20:190
1130225716,25sleep011297-21apt-get07:45:070
1130225716,25sleep011297-21apt-get07:45:070
1113125711,30sleep211138-21/usr/sbin/munin11:10:112
729525614,25sleep00-21swapper/010:58:420
3200425618,22sleep00-21swapper/012:15:100
2624325614,19sleep00-21swapper/008:30:260
2480225613,18sleep00-21swapper/010:10:160
2338625615,24sleep023410-21netstat10:05:200
1230425618,19sleep30-21swapper/309:30:143
303125514,24sleep33026-21apt-get10:45:083
2466625512,30sleep224538-21/usr/sbin/munin10:10:142
2878525417,25sleep20-21swapper/210:22:422
2263625413,23sleep322647-21ntp_states11:45:173
2145325413,22sleep321450-21apt-get10:00:063
1905825414,18sleep20-21swapper/208:09:572
148725418,18sleep20-21swapper/210:40:102
752525312,25sleep27534-21vmstat07:30:282
659025313,23sleep36591-21ntp_states10:55:223
624625316,19sleep20-21swapper/209:10:182
3089425314,24sleep330984-21cat08:45:233
2965025317,16sleep20-21swapper/210:25:212
2948425314,22sleep229584-21users08:40:262
2190925316,19sleep00-21swapper/011:44:530
199152539,21sleep00-21swapper/011:35:530
194625311,24sleep01941-21apt-get07:15:070
1771025320,19sleep20-21swapper/208:05:122
1548725314,18sleep00-21swapper/011:24:550
1236425311,17sleep00-21swapper/009:30:150
121225312,26sleep21244-21munin-run10:39:542
648625216,15sleep00-21swapper/012:35:210
463225216,20sleep00-21swapper/010:50:100
2186925213,22sleep021911-21ntp_states10:00:190
1395125214,23sleep214020-21latency_hist11:19:562
1056925212,22sleep210661-21users07:40:292
1056925212,22sleep210661-21users07:40:292
1056125212,23sleep210554-21apt-get09:25:082
953625113,19sleep20-21swapper/211:05:342
817525112,23sleep08171-21apt-get07:35:060
2000025112,22sleep320087-21perl09:55:133
2000025112,22sleep320087-21perl09:55:133
1584525112,17sleep30-21swapper/311:25:143
3264225014,21sleep00-21swapper/010:35:160
1285225011,24sleep012847-21apt-get07:50:080
670524911,16sleep30-21swapper/307:30:123
3244624912,21sleep032441-21apt-get07:10:060
3118124913,17sleep30-21swapper/307:05:173
422424813,18sleep20-21swapper/207:20:262
2778924814,15sleep20-21swapper/208:35:252
2563524812,20sleep225642-21diskstats08:30:122
434324612,16sleep20-21swapper/210:49:562
2359124613,21sleep223666-21grep10:05:262
3022724511,17sleep00-21swapper/008:45:040
2875524512,14sleep30-21swapper/308:40:103
187872429,17sleep218793-21ntp_states09:50:192
78232386,12sleep20-21swapper/209:15:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional