You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 14:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Jul 25, 2026 00:44:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
320929496,928sleep13217-21ntpq22:03:341
1508425696,528sleep115126-21ntpq20:58:331
2256123176,251sleep122703-21latency_hist21:23:081
2256123176,251sleep122703-21latency_hist21:23:081
23078229515,264sleep123179-21fschecks_time23:08:251
8143229054,188sleep10-21swapper/100:03:241
23133228512,259sleep07203-21kworker/0:019:41:340
23133228512,259sleep07203-21kworker/0:019:41:340
13539228431,209sleep113517-21latency_hist19:13:081
13539228431,209sleep113517-21latency_hist19:13:081
7966227612,249sleep28036-21sensors_temp22:18:362
1987622587,238sleep00-21swapper/022:58:190
31001225459,133sleep10-21swapper/123:33:081
23237223245,138sleep122742-21/usr/sbin/munin21:23:371
23237223245,138sleep122742-21/usr/sbin/munin21:23:371
22748222528,181sleep1364-21snmpd23:08:091
1907221134,141sleep11882-21/usr/sbin/munin23:43:221
28727220464,84sleep10-21swapper/119:58:381
28727220464,84sleep10-21swapper/119:58:381
29860218616,51sleep10-21swapper/120:03:271
29860218616,51sleep10-21swapper/120:03:271
28814218611,159sleep228898-21find19:58:392
28814218611,159sleep228898-21find19:58:392
9690217512,118sleep19685-21apt-get00:08:211
9690217512,118sleep19685-21apt-get00:08:211
1870021709,51sleep118736-21ntpq22:53:311
31698216910,118sleep131740-21ntp_kernel_pll_20:08:311
127921629,140sleep11305-21fschecks_count21:58:251
25667216137,68sleep125741-21switchtime19:48:371
25667216137,68sleep125741-21switchtime19:48:371
1493321597,14sleep10-21swapper/100:23:391
1493321597,14sleep10-21swapper/100:23:391
7906215422,58sleep10-21swapper/122:18:351
7565215129,50sleep10-21swapper/120:33:391
18405214863,27sleep10-21swapper/121:09:151
26591214515,60sleep10-21swapper/121:33:381
5477214418,13sleep10-21swapper/123:53:351
3028321449,79sleep130313-21sed21:48:071
16683214013,53sleep10-21swapper/121:03:301
26564213925,93sleep326058-21apt-get23:18:163
20137213972,53sleep11-21systemd19:33:071
20137213972,53sleep11-21systemd19:33:071
10083213548,20sleep10-21swapper/120:43:231
26924212947,27sleep126960-21latency19:53:381
14401212921,88sleep214544-21unixbench_singl22:38:392
12957212913,69sleep10-21swapper/100:18:301
12957212913,69sleep10-21swapper/100:18:301
21704212812,99sleep020593-21kworker/0:121:18:350
21704212812,99sleep020593-21kworker/0:121:18:350
11636212813,58sleep10-21swapper/120:48:281
1797021238,79sleep118030-21find00:33:441
17722212012,30sleep017295-21/usr/sbin/munin19:23:370
250021187,51sleep12516-21kernelversion20:18:291
19608211527,43sleep30-21swapper/321:13:173
19608211527,43sleep30-21swapper/321:13:173
346121149,39sleep10-21swapper/123:48:221
346121149,39sleep10-21swapper/123:48:221
1036021108,90sleep210370-21needreboot20:43:292
24798210923,26sleep10-21swapper/123:13:381
26041210821,65sleep026046-21modprobe23:18:050
1577121089,29sleep10-21swapper/119:18:211
14212210819,65sleep014273-21/usr/sbin/munin22:38:370
24211210742,40sleep10-21swapper/119:43:411
24211210742,40sleep10-21swapper/119:43:411
21961210720,63sleep322097-21sed21:18:383
21961210720,63sleep322097-21sed21:18:383
12811210716,65sleep012952-21grep22:33:360
12811210716,65sleep012952-21grep22:33:360
21080210641,11sleep10-21swapper/123:02:161
19446210611,80sleep019577-21latency_hist21:13:100
11574210530,63sleep1364-21snmpd22:30:261
21412210414,18sleep20-21swapper/223:03:172
7846210224,57sleep37870-21munin-plugin-st00:03:063
31832210244,16sleep10-21swapper/121:53:081
31832210244,16sleep10-21swapper/121:53:081
773529934,11sleep10-21swapper/100:01:381
3060629918,62sleep230503-21/usr/sbin/munin21:48:242
3060629918,62sleep230503-21/usr/sbin/munin21:48:242
2557229921,63sleep225621-21unin-run21:33:062
2017929913,69sleep220156-21latency_hist19:33:082
2017929913,69sleep220156-21latency_hist19:33:082
69562986,14sleep30-21swapper/320:33:243
555529712,37sleep10-21swapper/122:13:061
2544529712,65sleep20-21swapper/223:14:082
1576829714,69sleep015770-21grep19:18:200
1363529712,68sleep313748-21timerandwakeup19:13:103
1363529712,68sleep313748-21timerandwakeup19:13:103
2103129616,61sleep321129-21timerandwakeup19:33:413
2103129616,61sleep321129-21timerandwakeup19:33:413
1966129519,21sleep00-21swapper/019:28:420
1966129519,21sleep00-21swapper/019:28:420
2339929421,12sleep10-21swapper/119:43:201
2339929421,12sleep10-21swapper/119:43:201
201402938,69sleep120235-21cut21:13:351
201402938,69sleep120235-21cut21:13:351
1654929311,45sleep23337-21kworker/2:121:03:272
1191429326,45sleep011917-21cron19:08:070
3247629220,33sleep10-21swapper/121:53:321
30262919,62sleep32214-21/usr/sbin/munin20:18:393
1207729119,12sleep10-21swapper/122:33:171
1207729119,12sleep10-21swapper/122:33:171
9752907,68sleep11101-21/usr/sbin/munin23:38:371
9752907,68sleep11101-21/usr/sbin/munin23:38:371
89752908,18sleep10-21swapper/122:23:201
2064229022,36sleep10-21swapper/119:33:311
2064229022,36sleep10-21swapper/119:33:311
112312908,66sleep10-21swapper/100:13:171
1061629018,55sleep210712-21fschecks_time22:28:252
987428917,56sleep30-21swapper/322:23:383
955028912,63sleep09619-21switchtime22:23:350
3062428918,52sleep00-21swapper/020:03:460
3062428918,52sleep00-21swapper/020:03:460
2090228924,49sleep020948-21systemctl21:18:060
2090228924,49sleep020948-21systemctl21:18:060
1649628919,34sleep10-21swapper/122:48:061
1348928919,11sleep10-21swapper/122:38:091
604928822,36sleep10-21swapper/122:13:271
116128820,17sleep10-21swapper/120:13:441
116128820,17sleep10-21swapper/120:13:441
1103128827,42sleep311058-21awk19:03:383
3123228710,16sleep10-21swapper/123:33:161
2658828720,31sleep10-21swapper/123:18:181
213442870,7sleep20-21swapper/223:03:082
1310428721,34sleep10-21swapper/120:53:211
187072868,17sleep10-21swapper/119:28:101
1645228613,50sleep216598-21unixbench_multi19:18:382
397228526,18sleep10-21swapper/120:23:291
319422856,11sleep30-21swapper/320:08:373
2783528528,19sleep127882-21ntpq21:38:301
719428422,29sleep20-21swapper/222:18:082
1990228313,53sleep219959-21latency21:13:282
1990228313,53sleep219959-21latency21:13:282
180602837,20sleep118091-21latency_hist22:53:071
166928318,45sleep01646-21latency_hist23:43:080
166928318,45sleep01646-21latency_hist23:43:080
97542829,14sleep10-21swapper/120:43:071
526728215,47sleep35336-21latency23:53:293
3204928216,20sleep30-21swapper/321:53:183
285502825,64sleep00-21swapper/021:41:160
200428230,31sleep20-21swapper/220:18:092
200428230,31sleep20-21swapper/220:18:092
154472825,65sleep10-21swapper/119:18:011
154472825,65sleep10-21swapper/119:18:011
658128114,35sleep20-21swapper/223:58:202
285902819,40sleep10-21swapper/123:23:311
688228013,52sleep05666-1kworker/0:0H20:33:210
2381028019,40sleep323351-21/usr/sbin/munin19:43:303
2381028019,40sleep323351-21/usr/sbin/munin19:43:303
1328028017,40sleep013081-21/usr/sbin/munin20:53:280
128792809,13sleep30-21swapper/300:18:243
128792809,13sleep30-21swapper/300:18:243
87827921,41sleep2915-21modprobe21:58:052
735127917,26sleep20-21swapper/220:33:362
3050227912,17sleep30-21swapper/321:48:173
3050227912,17sleep30-21swapper/321:48:173
2186527913,48sleep021889-21ntp_states23:03:320
1860727923,31sleep30-21swapper/321:09:213
560627825,39sleep35691-21latency_hist22:13:083
1873627819,22sleep30-21swapper/300:38:063
2345027718,43sleep023581-21grep21:23:400
2345027718,43sleep023581-21grep21:23:400
2187227722,29sleep00-21swapper/019:38:100
2187227722,29sleep00-21swapper/019:38:100
1785527713,49sleep017867-21netstat00:33:350
1395227714,17sleep30-21swapper/319:13:143
1395227714,17sleep30-21swapper/319:13:143
1193127727,30sleep111982-21munin-run19:08:081
868727620,36sleep38713-21grep22:23:053
2574427626,28sleep20-21swapper/219:48:382
2574427626,28sleep20-21swapper/219:48:382
1897527625,38sleep319095-21timerwakeupswit22:53:383
2868227513,42sleep328688-21/usr/sbin/munin19:58:363
2868227513,42sleep328688-21/usr/sbin/munin19:58:363
2647427520,26sleep00-21swapper/021:33:370
811327413,18sleep00-21swapper/022:18:380
46052740,7sleep00-21swapper/022:08:280
2431127414,30sleep30-21swapper/323:13:073
119527421,34sleep21259-21cpuspeed21:58:222
57942737,14sleep10-21swapper/120:28:341
2431927320,17sleep10-21swapper/121:28:251
2363127321,29sleep00-21swapper/019:43:270
2363127321,29sleep00-21swapper/019:43:270
637527220,28sleep20-21swapper/223:58:082
3190227218,35sleep332011-21wc23:33:353
3033427215,41sleep030462-21sed23:28:390
2212827215,40sleep322192-21/usr/sbin/munin19:38:383
2212827215,40sleep322192-21/usr/sbin/munin19:38:383
116602720,9sleep20-21swapper/220:48:282
1628627114,25sleep00-21swapper/021:03:210
2860927018,22sleep30-21swapper/323:23:333
250027014,33sleep31-21systemd22:03:073
2404527014,22sleep20-21swapper/219:43:372
2404527014,22sleep20-21swapper/219:43:372
169672706,12sleep20-21swapper/222:48:282
1604727016,23sleep00-21swapper/021:03:090
11292706,49sleep00-21swapper/020:13:430
11292706,49sleep00-21swapper/020:13:430
450726916,37sleep24564-21irqstats22:08:272
1660326919,25sleep30-21swapper/319:18:393
1534626927,24sleep30-21swapper/320:58:403
1529226919,26sleep00-21swapper/022:43:250
1396826919,27sleep00-21swapper/019:13:150
1396826919,27sleep00-21swapper/019:13:150
1206026916,35sleep312136-21/usr/sbin/munin20:48:383
1123326921,30sleep211228-21apt-get00:13:182
2548826823,24sleep00-21swapper/019:48:320
2548826823,24sleep00-21swapper/019:48:320
1629626825,23sleep20-21swapper/200:28:322
1629626825,23sleep20-21swapper/200:28:322
981726719,31sleep00-21swapper/000:08:320
981726719,31sleep00-21swapper/000:08:320
581126716,31sleep25309-21/usr/sbin/munin20:28:352
1319126723,24sleep30-21swapper/320:53:273
321426618,23sleep30-21swapper/323:48:083
2337826618,19sleep20-21swapper/223:08:302
1573626615,27sleep20-21swapper/200:28:162
1573626615,27sleep20-21swapper/200:28:162
47082657,12sleep30-21swapper/323:53:043
47082657,12sleep30-21swapper/323:53:043
34192658,44sleep20-21swapper/223:48:112
34192658,44sleep20-21swapper/223:48:112
2958726515,22sleep30-21swapper/321:43:373
232526517,23sleep20-21swapper/220:18:262
231026519,25sleep00-21swapper/020:18:240
1761126513,35sleep317718-21date21:08:083
169382655,28sleep10-21swapper/100:29:521
169382655,28sleep10-21swapper/100:29:521
1497026514,25sleep30-21swapper/322:43:083
1107226513,28sleep20-21swapper/200:13:102
1107226513,28sleep20-21swapper/200:13:102
95142648,41sleep20-21swapper/220:40:342
2562226418,17sleep30-21swapper/321:33:073
156812646,13sleep20-21swapper/222:43:352
882326320,22sleep30-21swapper/300:03:463
560226321,25sleep05671-21ls20:28:290
267572637,44sleep00-21swapper/023:19:220
2493526315,21sleep30-21swapper/323:13:563
192426314,24sleep30-21swapper/320:18:063
192426314,24sleep30-21swapper/320:18:063
1828126318,27sleep018300-21apt-get22:53:180
1684326314,32sleep016900-21fschecks_count22:48:260
648026220,22sleep30-21swapper/322:13:373
509626214,29sleep00-21swapper/020:28:080
2304926213,25sleep30-21swapper/321:23:303
2304926213,25sleep30-21swapper/321:23:303
148712626,12sleep00-21swapper/000:23:380
148712626,12sleep00-21swapper/000:23:380
1457326214,29sleep214618-21ntp_kernel_err19:13:462
1457326214,29sleep214618-21ntp_kernel_err19:13:462
3024926113,32sleep330332-21sed23:28:383
239442617,16sleep20-21swapper/221:27:052
239442617,16sleep20-21swapper/221:27:052
1778426112,34sleep217790-21ntp_states19:23:392
66432607,39sleep30-21swapper/320:33:083
387326016,19sleep30-21swapper/320:23:273
300952606,20sleep20-21swapper/221:46:152
2674126014,29sleep226777-21/usr/sbin/munin19:53:352
90722598,13sleep20-21swapper/222:23:252
814225917,26sleep08039-21apt-get00:03:230
3094925920,22sleep20-21swapper/223:33:052
2161925916,22sleep30-21swapper/323:03:273
157042596,44sleep20-21swapper/221:00:002
1145925917,23sleep00-21swapper/000:13:360
2821825814,26sleep028211-21apt-get19:58:210
2821825814,26sleep028211-21apt-get19:58:210
2585825814,27sleep225855-21apt-get21:33:192
255342585,11sleep00-21swapper/021:32:360
251125822,17sleep00-21swapper/022:03:080
161825817,25sleep21650-21munin-run23:43:062
161825817,25sleep21650-21munin-run23:43:062
1225325817,28sleep212254-21sshd19:08:332
1225325817,28sleep212254-21sshd19:08:332
808925720,20sleep20-21swapper/200:03:212
418325715,20sleep00-21swapper/023:48:370
418325715,20sleep00-21swapper/023:48:370
2811825720,23sleep228129-21sort23:23:162
2119225714,22sleep20-21swapper/221:18:172
2119225714,22sleep20-21swapper/221:18:172
1922325711,28sleep319240-21memory19:28:333
1922325711,28sleep319240-21memory19:28:333
1828925719,19sleep20-21swapper/221:09:122
112002575,13sleep30-21swapper/320:47:213
914325616,18sleep30-21swapper/320:38:373
2676925614,15sleep20-21swapper/223:19:292
2209525614,24sleep222126-21fschecks_count19:38:372
2209525614,24sleep222126-21fschecks_count19:38:372
2069225620,19sleep20-21swapper/219:33:322
2069225620,19sleep20-21swapper/219:33:322
169712566,38sleep30-21swapper/300:30:193
169712566,38sleep30-21swapper/300:30:193
1368725613,25sleep213740-21users00:18:432
1368725613,25sleep213740-21users00:18:432
1089925617,24sleep30-21swapper/322:28:303
312362558,11sleep00-21swapper/023:33:190
274925511,25sleep02743-21apt-get22:03:200
2711625515,20sleep30-21swapper/319:53:443
2036125515,26sleep30-21swapper/322:58:343
1675625514,24sleep316752-21apt-get22:48:213
581925413,14sleep00-21swapper/022:13:170
2719725415,16sleep30-21swapper/321:38:083
257425414,22sleep00-21swapper/023:43:370
1719225413,27sleep01-21systemd00:33:060
1719225413,27sleep01-21systemd00:33:060
971225314,24sleep09739-21munin-run20:43:060
3254525319,18sleep00-21swapper/021:53:350
3174125312,25sleep231773-21ntp_states20:08:322
3038825316,20sleep20-21swapper/220:03:442
3038825316,20sleep20-21swapper/220:03:442
2821425315,17sleep00-21swapper/023:23:220
2770225317,20sleep20-21swapper/221:38:272
2325525313,24sleep023349-21memory23:08:280
139672537,35sleep00-21swapper/000:21:320
139672537,35sleep00-21swapper/000:21:320
1349725312,28sleep013538-21/usr/sbin/munin19:13:070
1349725312,28sleep013538-21/usr/sbin/munin19:13:070
1269525317,18sleep30-21swapper/322:33:353
1269525317,18sleep30-21swapper/322:33:353
1198425317,20sleep20-21swapper/219:08:092
1059825311,16sleep00-21swapper/022:28:230
545925211,26sleep05476-21open_inodes23:53:340
279225214,24sleep22821-21df22:03:222
2545925216,14sleep30-21swapper/319:48:313
2545925216,14sleep30-21swapper/319:48:313
1888225215,21sleep20-21swapper/222:53:372
184152525,10sleep30-21swapper/319:25:583
1030525219,16sleep30-21swapper/300:08:433
1030525219,16sleep30-21swapper/300:08:433
277682518,32sleep00-21swapper/019:56:230
1450525114,21sleep314591-21perl00:23:283
1450525114,21sleep314591-21perl00:23:283
1207825112,24sleep212073-21apt-get22:33:182
1207825112,24sleep212073-21apt-get22:33:182
322772505,34sleep20-21swapper/223:35:192
3089625012,25sleep030503-21/usr/sbin/munin21:48:310
3089625012,25sleep030503-21/usr/sbin/munin21:48:310
1310525012,22sleep213099-21apt-get20:53:222
1042025014,17sleep00-21swapper/020:43:310
72924916,15sleep30-21swapper/323:38:313
72924916,15sleep30-21swapper/323:38:313
3046824913,15sleep30-21swapper/320:03:453
3046824913,15sleep30-21swapper/320:03:453
5222489,15sleep00-21swapper/020:13:110
401924814,18sleep20-21swapper/220:23:302
2751924811,22sleep327579-21fschecks_time21:38:243
1775024814,15sleep20-21swapper/200:33:332
2910424714,13sleep00-21swapper/021:43:260
666324610,22sleep06669-21diskstats23:58:260
304324610,15sleep20-21swapper/223:46:332
325624511,21sleep33332-21systemctl22:03:363
118092459,16sleep00-21swapper/020:48:310
1123424313,17sleep30-21swapper/300:13:193
152052428,19sleep315201-21apt-get22:43:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional