You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-28 - 02:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Sep 27, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34342315912,3135sleep220135-21kworker/2:009:15:292
266082201930,1966sleep226586-21/usr/sbin/munin07:07:132
266082201930,1966sleep226586-21/usr/sbin/munin07:07:132
453246356,347sleep3605-21grep12:28:403
24824245614,200sleep324844-21cron08:42:113
28913233725,202sleep328955-21fschecks_count07:12:313
19695227148,104sleep319752-21latency10:07:343
19695227148,104sleep319752-21latency10:07:343
12924226687,64sleep30-21swapper/308:03:073
12924226687,64sleep30-21swapper/308:03:073
24817224953,147sleep324963-21grep10:22:443
4356224747,60sleep30-21swapper/311:02:293
26170224316,210sleep026215-21ntp_kernel_pll_12:07:450
26170224316,210sleep026215-21ntp_kernel_pll_12:07:450
26525221917,122sleep326669-21head12:07:523
26525221917,122sleep326669-21head12:07:523
6777221315,182sleep16844-21switchtime07:42:411
27544221369,29sleep30-21swapper/307:07:433
3412220825,63sleep30-21swapper/310:57:443
20196220324,17sleep30-21swapper/308:27:143
29158220054,88sleep329309-21grep08:52:453
29158220054,88sleep329309-21grep08:52:453
3176219817,166sleep13221-21ntp_kernel_pll_10:57:381
4350219336,115sleep30-21swapper/309:17:403
27971219011,64sleep30-21swapper/312:12:403
3454218955,121sleep30-21swapper/307:32:383
9009217513,124sleep39030-21awk07:52:123
9009217513,124sleep39030-21awk07:52:123
27256217320,16sleep30-21swapper/310:32:293
27256217320,16sleep30-21swapper/310:32:293
3569217224,120sleep33572-21cron09:17:123
1007216728,71sleep31057-21sh07:27:133
1007216728,71sleep31057-21sh07:27:133
7546216518,129sleep00-21swapper/011:12:320
7546216518,129sleep00-21swapper/011:12:320
23042216248,99sleep323111-21mailstats10:17:393
17240215961,56sleep30-21swapper/311:42:113
12216215915,121sleep312369-21ls08:02:143
17530215123,17sleep30-21swapper/310:01:283
15478215055,78sleep015526-21systemd-run08:12:120
15478215055,78sleep015526-21systemd-run08:12:120
9137214432,98sleep39237-21timerandwakeup09:32:423
9137214432,98sleep39237-21timerandwakeup09:32:423
31616214321,65sleep30-21swapper/312:24:363
544214286,13sleep30-21swapper/310:47:593
544214286,13sleep30-21swapper/310:47:593
388214115,71sleep2542-21tail10:47:582
388214115,71sleep2542-21tail10:47:582
3140521399,70sleep331438-21date09:02:143
3140521399,70sleep331438-21date09:02:143
22647213943,83sleep322668-21apt-get11:57:223
4427213810,114sleep04435-21df_inode11:02:300
19179213810,21sleep30-21swapper/310:07:143
15766213712,18sleep30-21swapper/308:12:283
518213631,23sleep3612-21sed09:07:153
518213631,23sleep3612-21sed09:07:153
123221348,32sleep30-21swapper/312:32:273
2940021329,101sleep30-21swapper/310:37:423
20812213216,100sleep120846-21latency_hist10:12:161
20812213216,100sleep120846-21latency_hist10:12:161
27463213051,61sleep327529-21switchtime08:47:413
1619212910,82sleep30-21swapper/307:27:413
10578212935,62sleep310618-21uname09:37:423
2208721189,74sleep322193-21grep08:32:303
2208721189,74sleep322193-21grep08:32:303
4736211638,17sleep30-21swapper/307:37:313
10813211610,49sleep30-21swapper/311:20:303
10813211610,49sleep30-21swapper/311:20:303
31719211410,67sleep331736-21munin-run10:47:143
5873211310,90sleep35892-21apt-get11:07:223
18841211122,63sleep018888-21latency_hist11:47:130
8151211019,47sleep00-21swapper/007:47:370
8151211019,47sleep00-21swapper/007:47:370
6746211017,65sleep00-21swapper/009:27:140
29332211011,84sleep228760-21/usr/sbin/munin10:37:412
32557210913,81sleep032599-21ntp_states10:47:520
32557210913,81sleep032599-21ntp_states10:47:520
21751210832,61sleep321808-21sed08:32:143
1479721088,56sleep30-21swapper/308:07:433
1479721088,56sleep30-21swapper/308:07:433
8061210775,17sleep30-21swapper/311:12:433
8061210775,17sleep30-21swapper/311:12:433
29430210711,78sleep229508-21timerandwakeup07:12:422
16164210712,19sleep10-21swapper/108:12:381
1416721079,86sleep0367-21snmpd09:48:250
25260210624,62sleep324782-21apt-get07:02:223
25260210624,62sleep324782-21apt-get07:02:223
4117210517,73sleep04235-21date11:02:140
27186210525,48sleep227205-21apt-get10:32:242
27186210525,48sleep227205-21apt-get10:32:242
12448210418,32sleep30-21swapper/311:25:353
6983210333,54sleep30-21swapper/309:27:253
16591210320,64sleep00-21swapper/011:37:440
31732210119,61sleep131825-21grep09:02:321
31732210119,61sleep131825-21grep09:02:321
9795210015,67sleep09822-21proc_pri07:53:020
9795210015,67sleep09822-21proc_pri07:53:020
26617210028,17sleep326646-21munin-run08:47:123
24104210026,44sleep024100-21apt-get10:22:240
259092999,49sleep30-21swapper/310:27:323
2569929934,44sleep125736-21sed07:03:051
2569929934,44sleep125736-21sed07:03:051
617629730,41sleep16320-21unixbench_singl09:22:431
608529712,73sleep26118-21basename11:07:332
306432979,22sleep30-21swapper/307:17:243
2503229720,62sleep225146-21sed08:42:142
196282979,33sleep319694-21awk11:47:403
335529616,65sleep13421-21latency07:32:361
3079929617,63sleep130808-21awk10:42:401
1416529620,26sleep30-21swapper/311:32:143
1416529620,26sleep30-21swapper/311:32:143
924529514,65sleep29295-21if_eth011:17:342
924529514,65sleep29295-21if_eth011:17:342
1608729512,58sleep10-21swapper/109:57:141
2212429420,57sleep122164-21systemctl11:57:111
1188129412,68sleep011943-21cat11:22:420
1162829442,34sleep30-21swapper/309:42:233
101982948,13sleep20-21swapper/209:37:332
178182939,20sleep30-21swapper/308:17:413
3112529116,61sleep031168-21ntp_offset07:18:340
2160629127,50sleep321732-21timerwakeupswit11:53:543
1393829113,66sleep214077-21unixbench_singl09:47:432
2975629012,63sleep129758-21sshd10:38:281
2429929017,48sleep024441-21sed08:37:440
1320529010,30sleep30-21swapper/309:47:273
2847128926,41sleep128490-21modprobe10:37:111
2847128926,41sleep128490-21modprobe10:37:111
152172899,17sleep30-21swapper/309:52:403
599328816,53sleep16082-21tune2fs11:07:321
209742888,17sleep30-21swapper/310:12:263
1803128618,46sleep218164-21unixbench_multi08:17:442
2177828521,44sleep221838-21unixbench_multi10:12:582
895328411,51sleep10-21swapper/107:51:301
895328411,51sleep10-21swapper/107:51:301
1865928426,31sleep00-21swapper/008:22:140
103572845,11sleep30-21swapper/307:54:393
103572845,11sleep30-21swapper/307:54:393
2817928314,49sleep128299-21/usr/sbin/munin12:12:421
1764028318,47sleep017727-21grep10:02:140
106752838,14sleep00-21swapper/011:20:230
106752838,14sleep00-21swapper/011:20:230
566928223,14sleep30-21swapper/309:22:333
2513828212,43sleep30-21swapper/312:02:473
2386728215,52sleep223998-21latency_hist10:22:142
959728128,31sleep29662-21latency07:52:562
959728128,31sleep29662-21latency07:52:562
758928112,52sleep27672-21expr09:27:402
2602328117,45sleep126148-21ls08:42:541
3074627918,39sleep130888-21/usr/sbin/munin08:57:441
3074627918,39sleep130888-21/usr/sbin/munin08:57:441
248227952,14sleep30-21swapper/310:57:123
248227952,14sleep30-21swapper/310:57:123
2898227823,18sleep328978-21apt-get12:17:253
1177827811,49sleep211801-21/usr/sbin/munin11:22:392
294942778,54sleep00-21swapper/012:17:420
71242765,10sleep30-21swapper/307:42:523
596527622,26sleep20-21swapper/207:42:142
3257227616,39sleep232616-21ntpq07:22:362
3257227616,39sleep232616-21ntpq07:22:362
2861327617,25sleep00-21swapper/008:52:340
2861327617,25sleep00-21swapper/008:52:340
1929227622,28sleep20-21swapper/208:22:362
1140127615,47sleep111428-21fschecks_count11:22:301
879627516,43sleep28897-21sed11:17:142
879627516,43sleep28897-21sed11:17:142
380227521,27sleep20-21swapper/207:32:452
362927512,42sleep03692-21/usr/sbin/munin07:32:430
3120427521,21sleep00-21swapper/012:22:430
2685027512,51sleep1367-21snmpd07:07:201
2375727513,43sleep123758-21sshd10:21:291
2005127512,50sleep10-21swapper/108:25:541
302682746,53sleep10-21swapper/112:22:111
2216727311,48sleep21-21systemd11:57:132
1889127323,20sleep10-21swapper/111:47:141
185952738,9sleep30-21swapper/311:44:423
18222737,11sleep30-21swapper/309:09:453
155282739,10sleep30-21swapper/311:35:253
155282739,10sleep30-21swapper/311:35:253
906527219,23sleep20-21swapper/209:32:412
906527219,23sleep20-21swapper/209:32:412
2526227221,23sleep00-21swapper/007:02:230
2526227221,23sleep00-21swapper/007:02:230
205827220,26sleep10-21swapper/109:12:131
1286027214,46sleep00-21swapper/009:46:310
3216927113,36sleep132188-21apt-get07:22:221
3216927113,36sleep132188-21apt-get07:22:221
2470427116,27sleep10-21swapper/112:02:411
2347127118,26sleep00-21swapper/010:17:440
2044627118,32sleep220501-21sort08:27:292
1780327116,41sleep217892-21cat11:42:352
150527115,38sleep11022-21apt-get07:27:231
1645927024,32sleep20-21swapper/211:37:422
1283327011,40sleep112853-21fschecks_count08:03:051
1283327011,40sleep112853-21fschecks_count08:03:051
1213827023,30sleep112161-21munin-run08:02:121
113542709,50sleep0367-21snmpd09:42:030
686126918,20sleep20-21swapper/207:42:422
463226910,18sleep00-21swapper/007:37:240
3223126923,22sleep10-21swapper/110:47:451
3223126923,22sleep10-21swapper/110:47:451
2282826914,38sleep022903-21fschecks_time11:57:550
194926913,24sleep20-21swapper/212:32:472
1729226927,24sleep10-21swapper/111:42:131
1622926919,29sleep00-21swapper/008:12:410
3218626811,41sleep132211-21sshd12:27:291
3016526819,35sleep130253-21apt-config07:17:131
2753326811,19sleep00-21swapper/008:47:420
2471426811,21sleep20-21swapper/210:22:432
1898026812,38sleep018993-21fschecks_count08:22:300
561226719,29sleep15626-21idleruntime-cro11:07:111
3013826716,24sleep00-21swapper/010:42:150
2760526714,31sleep00-21swapper/010:32:380
2760526714,31sleep00-21swapper/010:32:380
2745526722,26sleep20-21swapper/207:07:422
2615026717,31sleep226269-21/usr/sbin/munin08:42:552
2469426714,38sleep024703-21ntp_states12:02:400
1661926720,22sleep10-21swapper/109:57:351
15462676,12sleep00-21swapper/007:27:340
1458926717,23sleep10-21swapper/108:07:371
1458926717,23sleep10-21swapper/108:07:371
2236026616,25sleep00-21swapper/011:57:150
1825126623,26sleep00-21swapper/010:02:380
1420226612,34sleep00-21swapper/011:32:150
1420226612,34sleep00-21swapper/011:32:150
106026617,22sleep00-21swapper/007:27:140
106026617,22sleep00-21swapper/007:27:140
1969626522,22sleep00-21swapper/011:47:410
1543126513,27sleep20-21swapper/209:52:432
1296126520,21sleep10-21swapper/111:27:311
1296126520,21sleep10-21swapper/111:27:311
1237326518,20sleep00-21swapper/008:02:150
1237326518,20sleep00-21swapper/008:02:150
112826517,30sleep20-21swapper/209:07:382
2541726417,27sleep10-21swapper/110:27:141
2270226417,29sleep222668-21apt-get11:57:412
2041926423,23sleep10-21swapper/108:27:241
746426312,15sleep00-21swapper/007:47:130
2632926320,18sleep00-21swapper/010:27:410
2606926315,35sleep226152-21sed12:07:432
2606926315,35sleep226152-21sed12:07:432
7872626,11sleep00-21swapper/012:30:100
2767226213,31sleep027765-21memory12:12:330
2611626215,26sleep126152-21wc10:27:381
524126119,24sleep10-21swapper/109:22:141
478126114,26sleep10-21swapper/107:37:331
3117926120,18sleep20-21swapper/207:18:362
2087726115,26sleep020899-21ntp_states08:27:390
829426019,16sleep00-21swapper/009:32:140
2135426015,29sleep021387-21/usr/sbin/munin10:12:440
1926426016,19sleep20-21swapper/211:47:312
1456726010,26sleep20-21swapper/211:32:332
1456726010,26sleep20-21swapper/211:32:332
1217126016,22sleep20-21swapper/208:02:132
10302606,11sleep00-21swapper/009:07:360
435525916,27sleep24349-21apt-get11:02:282
279525918,23sleep22821-21munin-run07:32:132
225712597,14sleep10-21swapper/108:32:411
225712597,14sleep10-21swapper/108:32:411
1541725911,35sleep1367-21snmpd11:33:571
1541725911,35sleep1367-21snmpd11:33:571
1308225915,28sleep213115-21/usr/sbin/munin08:03:102
1308225915,28sleep213115-21/usr/sbin/munin08:03:102
990925811,31sleep010053-21cat09:37:150
938325811,27sleep19434-21users09:32:441
938325811,27sleep19434-21users09:32:441
738925815,23sleep10-21swapper/109:27:351
3164125817,24sleep231638-21apt-get09:02:252
3164125817,24sleep231638-21apt-get09:02:252
899225714,27sleep09000-21awk09:32:390
519425712,30sleep05240-21latency_hist09:22:130
307082575,12sleep20-21swapper/210:42:372
2846425713,26sleep228458-21apt-get08:52:282
2846425713,26sleep228458-21apt-get08:52:282
2813625715,26sleep00-21swapper/007:12:120
1848725715,22sleep20-21swapper/210:02:442
1715625716,18sleep20-21swapper/209:57:442
1325625713,28sleep213293-21awk11:27:362
1325625713,28sleep213293-21awk11:27:362
783725617,20sleep20-21swapper/207:47:312
783725617,20sleep20-21swapper/207:47:312
2907525619,18sleep20-21swapper/212:17:322
1634325615,17sleep20-21swapper/208:12:432
580425514,15sleep20-21swapper/209:22:362
3201225520,18sleep10-21swapper/112:27:161
3060925514,23sleep00-21swapper/010:42:350
294232554,39sleep10-21swapper/108:53:241
294232554,39sleep10-21swapper/108:53:241
2423725514,24sleep224234-21apt-get12:02:262
1165325513,24sleep211647-21apt-get09:42:262
3098525410,29sleep030987-21modprobe08:58:110
3098525410,29sleep030987-21modprobe08:58:110
280025415,24sleep22794-21apt-get10:57:272
268852548,17sleep10-21swapper/108:47:221
2359525417,21sleep10-21swapper/111:58:091
1441125413,25sleep014405-21apt-get11:32:270
1441125413,25sleep014405-21apt-get11:32:270
129425415,24sleep01303-21df_inode10:52:280
129425415,24sleep01303-21df_inode10:52:280
1172625416,19sleep10-21swapper/109:42:291
932625314,22sleep19320-21apt-get07:52:471
932625314,22sleep19320-21apt-get07:52:471
3209325314,24sleep032099-21ntp_states09:02:390
3209325314,24sleep032099-21ntp_states09:02:390
278825316,15sleep10-21swapper/109:12:421
2746725310,17sleep20-21swapper/212:12:272
2585225310,30sleep225908-21if_eth010:27:312
256125315,16sleep10-21swapper/110:57:141
256125315,16sleep10-21swapper/110:57:141
2095825314,21sleep120970-21apt-get10:12:241
1963725314,24sleep219694-21irqstats10:07:332
1963725314,24sleep219694-21irqstats10:07:332
1940025313,24sleep019418-21apt10:07:230
1940025313,24sleep019418-21apt10:07:230
1841525314,20sleep10-21swapper/110:02:431
124225315,22sleep11304-21df12:32:291
572725214,15sleep00-21swapper/011:07:140
304725217,20sleep00-21swapper/009:12:450
2882825212,22sleep028895-21df07:12:280
2207225212,24sleep222086-21fschecks_count08:32:292
2207225212,24sleep222086-21fschecks_count08:32:292
1450925221,17sleep10-21swapper/109:52:141
2490225113,17sleep10-21swapper/108:42:131
1908125112,24sleep119130-21/usr/sbin/munin11:47:261
1607625114,22sleep016085-21munin-run09:57:130
103025112,16sleep20-21swapper/212:32:142
903025013,21sleep19028-21apt-get11:17:251
903025013,21sleep19028-21apt-get11:17:251
764425013,17sleep10-21swapper/111:12:331
764425013,17sleep10-21swapper/111:12:331
388825012,21sleep23881-21apt-get09:17:272
3195825015,21sleep132050-21cat07:22:141
2528225011,21sleep024837-21apt-get08:42:170
24102504,34sleep00-21swapper/012:36:080
2249425011,23sleep00-21swapper/008:32:400
2249425011,23sleep00-21swapper/008:32:400
1743024910,17sleep10-21swapper/108:17:301
122224911,22sleep21215-21apt-get10:52:262
122224911,22sleep21215-21apt-get10:52:262
1429024811,20sleep214298-21df_inode08:07:302
1429024811,20sleep214298-21df_inode08:07:302
111172489,15sleep00-21swapper/007:57:300
1019324811,22sleep110197-21diskstats09:37:321
2739524714,17sleep20-21swapper/208:47:402
262162479,16sleep10-21swapper/112:07:461
262162479,16sleep10-21swapper/112:07:461
3088924610,20sleep230941-21users08:57:452
3088924610,20sleep230941-21users08:57:452
306302417,15sleep20-21swapper/212:22:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional