You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 15:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Jun 19, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
230192833280,8220sleep122522-21apt-get07:02:291
261622573516,5703sleep225686-21kworker/2:108:52:372
1163828768,854sleep211611-21apt-get11:32:432
25700279725,737sleep325741-21unin-run08:52:183
6453250622,467sleep26476-21/usr/sbin/munin09:32:472
24887247219,436sleep224897-21/usr/sbin/munin12:12:362
18662234954,167sleep318664-21apt-get10:12:293
1110222917,273sleep3363-21snmpd08:06:023
7115228431,166sleep37157-21ntp_kernel_pll_11:17:423
2760122786,237sleep323758-21sshd12:21:453
9589227685,96sleep30-21swapper/311:27:193
2197227582,136sleep30-21swapper/307:37:393
11593226410,238sleep311611-21apt-get11:32:273
25546226130,151sleep325695-21grep12:12:493
11311224532,122sleep311509-21apt-key11:32:193
5250223337,149sleep35341-21fschecks_time11:12:373
5250223337,149sleep35341-21fschecks_time11:12:373
735422238,199sleep37503-21cut09:37:203
18326221814,190sleep118380-21sort11:52:321
3765221232,142sleep30-21swapper/311:07:363
23655221113,62sleep30-21swapper/308:42:493
18890220528,143sleep319018-21unixbench_multi08:27:553
20945220223,96sleep321092-21unixbench_multi10:17:503
4419219815,163sleep00-21swapper/009:27:280
23440219019,158sleep023486-21fschecks_count10:27:360
15101218910,93sleep315194-21latency_hist10:02:193
4270218621,152sleep24389-21sed11:07:472
24513218635,57sleep324536-21modprobe12:12:173
181321838,85sleep31854-21grep09:17:463
21910218130,111sleep321981-21cat08:37:483
27933217427,64sleep30-21swapper/312:22:303
21530216913,20sleep30-21swapper/312:02:363
6424216026,111sleep26432-21modprobe11:17:162
6424216026,111sleep26432-21modprobe11:17:162
25424216051,86sleep30-21swapper/307:07:513
23064215328,25sleep10-21swapper/110:27:171
13380215311,91sleep30-21swapper/308:12:373
3079221518,53sleep30-21swapper/312:32:193
17221214912,97sleep316626-21/usr/sbin/munin08:23:033
14461214859,50sleep314495-21munin-run11:42:173
1033214844,61sleep31053-21modprobe09:17:173
906121448,101sleep39176-21ls07:58:153
906121448,101sleep39176-21ls07:58:153
9853214040,80sleep324396-21cyclictest09:42:493
28780213832,90sleep328915-21timerwakeupswit07:17:483
30650213643,50sleep30-21swapper/309:07:283
4846213332,52sleep30-21swapper/307:47:203
4846213332,52sleep30-21swapper/307:47:203
15142213227,21sleep30-21swapper/308:17:413
14502213113,101sleep014583-21rm11:42:180
8887213015,101sleep08912-21proc_pri07:58:120
8887213015,101sleep08912-21proc_pri07:58:120
1428212918,13sleep30-21swapper/307:35:013
28874212827,49sleep30-21swapper/309:02:193
32512212728,49sleep30-21swapper/310:57:183
31642212747,46sleep331005-21/usr/sbin/munin12:32:483
24353212740,62sleep324396-21cyclictest07:07:193
30520212419,12sleep30-21swapper/310:48:063
30520212419,12sleep30-21swapper/310:48:063
21329212411,99sleep121397-21sh08:37:341
13940212435,66sleep20-21swapper/211:37:482
29420212221,79sleep029429-21modprobe07:22:180
16709212222,14sleep30-21swapper/310:07:203
15802212216,92sleep215826-21proc_pri10:02:472
7854212124,36sleep30-21swapper/309:37:403
7854212124,36sleep30-21swapper/309:37:403
17824212018,11sleep30-21swapper/310:08:093
2297211711,86sleep12330-21/usr/sbin/munin07:37:411
21090211729,52sleep321191-21sed08:37:203
2075211725,46sleep10-21swapper/111:02:461
25744211618,34sleep10-21swapper/108:52:201
13128211528,76sleep3363-21snmpd09:52:543
4151211218,74sleep20-21swapper/209:27:182
20529211117,81sleep220545-21/usr/sbin/munin10:17:412
490210930,43sleep3481-21apt-get10:57:313
27180210912,56sleep30-21swapper/307:12:483
16745210817,73sleep116262-21apt-get11:47:281
469321075,13sleep30-21swapper/307:46:443
1276021075,13sleep30-21swapper/308:09:173
1266321078,90sleep00-21swapper/011:34:260
2080621065,13sleep30-21swapper/311:57:513
2895221055,12sleep30-21swapper/310:42:533
2362321058,53sleep30-21swapper/310:27:393
8913210417,72sleep18995-21grep07:58:131
8913210417,72sleep18995-21grep07:58:131
5751210321,42sleep30-21swapper/309:32:193
23503210316,73sleep223512-21awk07:02:502
29516210210,76sleep329563-21ntpq09:02:443
11956210218,40sleep30-21swapper/309:52:183
4652210112,74sleep24699-21if_eth009:27:382
2650821018,53sleep326582-21mailstats08:52:463
12292210116,63sleep012184-21/usr/sbin/munin09:52:350
1078210114,65sleep2463-21/usr/sbin/munin10:58:062
25027210023,35sleep30-21swapper/310:32:363
1271210019,45sleep01416-21unixbench_singl10:58:080
76642988,78sleep00-21swapper/011:17:590
39042986,80sleep30-21swapper/309:24:083
1864329815,21sleep20-21swapper/210:12:282
1858129825,34sleep30-21swapper/311:52:393
1624829838,47sleep316311-21systemd-run11:47:173
1170429623,51sleep011222-21apt-get08:07:290
311342957,13sleep10-21swapper/110:52:321
2311629543,17sleep323115-21munin-run10:27:193
1848329429,50sleep118559-21latency08:27:451
2803829314,61sleep028103-21grep08:57:480
1304629313,64sleep213155-21latency_hist08:12:192
2354929218,49sleep223652-21timerwakeupswit08:42:482
243902916,12sleep00-21swapper/012:10:480
1800529111,67sleep218034-21sed08:27:222
1729028918,59sleep117335-21cat10:07:421
1695428931,44sleep30-21swapper/311:48:183
2615328723,17sleep10-21swapper/112:17:191
3251728625,21sleep312-21rcu_sched07:32:183
1631628621,41sleep016269-21latency_hist11:47:180
1725828515,49sleep017289-21/usr/sbin/munin10:07:410
2058628418,41sleep120735-21tail11:57:491
1057028414,47sleep010688-21/usr/sbin/munin08:02:480
702028311,57sleep17104-21memory11:17:401
450728310,50sleep00-21swapper/011:07:490
1856228319,47sleep218159-21/usr/sbin/munin08:27:462
1131128320,27sleep20-21swapper/209:47:492
632828115,50sleep16344-21grep07:52:191
632828115,50sleep16344-21grep07:52:191
2704528014,53sleep127083-21processes07:12:461
20512806,33sleep30-21swapper/311:02:443
300162799,50sleep00-21swapper/007:22:380
1733627910,15sleep20-21swapper/210:07:432
1424127913,53sleep114248-21sh09:57:451
1387127921,28sleep00-21swapper/009:57:360
524627811,42sleep20-21swapper/207:47:402
524627811,42sleep20-21swapper/207:47:402
1424927819,37sleep214280-21proc_pri09:57:462
2958627718,33sleep20-21swapper/210:47:322
2958627718,33sleep20-21swapper/210:47:322
570127622,33sleep15719-21modprobe09:32:171
3126427628,26sleep20-21swapper/212:32:392
302842767,11sleep00-21swapper/009:06:120
2997627618,18sleep30-21swapper/307:22:373
297927619,37sleep13017-21grep09:22:361
2847827622,39sleep228546-21/usr/sbin/munin12:22:452
2228927628,28sleep20-21swapper/210:22:462
796527416,14sleep10-21swapper/109:37:421
796527416,14sleep10-21swapper/109:37:421
3263027419,28sleep00-21swapper/010:57:200
2666127424,35sleep126792-21grep08:52:481
1495827417,24sleep10-21swapper/111:42:271
522527312,38sleep05130-21/usr/sbin/munin11:12:340
522527312,38sleep05130-21/usr/sbin/munin11:12:340
1044727313,44sleep110587-21latency_hist09:47:191
3181727220,33sleep10-21swapper/107:27:521
1062727220,34sleep010739-21/usr/sbin/munin11:27:480
1894727113,42sleep118960-21/usr/sbin/munin10:12:471
1626127116,30sleep116417-21latency_hist08:22:201
1125027114,37sleep211264-21systemd-run08:07:182
2681427016,27sleep20-21swapper/207:12:392
2666727012,42sleep026760-21fschecks_time07:12:370
263442705,30sleep30-21swapper/310:37:203
1983927020,34sleep119885-21fschecks_count08:32:381
1809227016,39sleep118114-21latency_hist08:27:291
78922698,48sleep37966-21cmp07:57:183
714726919,20sleep00-21swapper/007:52:470
607826917,26sleep10-21swapper/109:32:381
3259126917,21sleep00-21swapper/009:12:420
2979726918,28sleep129952-21grep09:02:501
2815426910,13sleep20-21swapper/210:42:362
2156926912,37sleep00-21swapper/010:22:190
1462426923,21sleep10-21swapper/108:17:191
650326821,31sleep16592-21ls11:17:191
650326821,31sleep16592-21ls11:17:191
2924926812,38sleep229324-21/usr/sbin/munin09:02:392
2830326813,41sleep128349-21/usr/sbin/munin12:22:401
279142686,14sleep00-21swapper/012:22:270
2665626818,32sleep026686-21kernelversion12:17:390
2342426812,25sleep20-21swapper/210:27:342
1497526817,30sleep214476-21apt-get11:42:282
783226721,26sleep27889-21idleruntime-cro07:57:172
2451926717,33sleep224581-21fschecks_time08:47:372
2173826712,23sleep20-21swapper/208:37:432
961326618,33sleep09758-21latency_hist11:27:200
3243826616,25sleep20-21swapper/210:57:172
2934926618,26sleep129437-21latency_hist10:47:191
2810526611,35sleep128168-21/usr/sbin/munin08:57:491
2781826621,28sleep227967-21sort10:42:202
2706426618,22sleep10-21swapper/112:17:481
2830126518,26sleep00-21swapper/010:42:380
2722526526,18sleep10-21swapper/110:37:511
114126516,30sleep11255-21/usr/sbin/munin07:32:531
879226422,22sleep20-21swapper/211:22:452
625326415,35sleep00-21swapper/009:32:410
306402646,48sleep10-21swapper/112:31:001
256642646,46sleep20-21swapper/207:09:322
2383826420,26sleep10-21swapper/112:07:591
2353526413,37sleep223628-21/usr/sbin/munin12:07:522
2311826414,31sleep00-21swapper/010:27:200
2027126417,22sleep20-21swapper/211:57:422
315122636,12sleep10-21swapper/112:32:461
2617326313,28sleep026196-21munin-run10:37:170
201326315,23sleep00-21swapper/007:37:360
1958726322,23sleep00-21swapper/010:12:580
1458426319,28sleep11-21systemd11:42:191
1105826315,22sleep00-21swapper/009:47:430
1042026320,18sleep00-21swapper/009:47:180
783226216,29sleep27850-21kernelversion09:37:392
783226216,29sleep27850-21kernelversion09:37:392
665726221,21sleep10-21swapper/107:52:351
356026214,29sleep03615-21fschecks_count07:42:370
2499626220,21sleep20-21swapper/210:32:352
1695126214,29sleep016626-21/usr/sbin/munin08:22:560
511226118,24sleep05187-21fschecks_time07:47:380
511226118,24sleep05187-21fschecks_time07:47:380
3271026116,21sleep10-21swapper/109:12:471
3198426122,24sleep231954-21latency_hist09:12:202
2314326120,22sleep10-21swapper/108:42:391
1616926119,25sleep016257-21munin-run08:22:190
1030826114,33sleep110322-21grep08:02:411
1023426120,25sleep110276-21awk11:27:411
85872607,18sleep00-21swapper/009:39:150
85872607,18sleep00-21swapper/009:39:150
1958526020,25sleep21-21systemd11:57:182
1675326017,25sleep216765-21apt-get11:47:292
164572606,10sleep00-21swapper/010:05:310
1179526017,23sleep10-21swapper/108:08:181
843325916,21sleep10-21swapper/111:22:361
748125920,22sleep20-21swapper/211:17:492
6202595,12sleep20-21swapper/207:32:412
2948025913,28sleep229474-21apt-get12:27:302
265625915,29sleep02672-21latency_hist09:22:190
2465425916,18sleep00-21swapper/008:47:390
142025921,21sleep10-21swapper/110:58:091
132252595,10sleep20-21swapper/209:54:112
1295425915,27sleep10-21swapper/111:37:181
899725818,18sleep20-21swapper/207:58:142
899725818,18sleep20-21swapper/207:58:142
2348925820,19sleep10-21swapper/110:27:371
1486925814,27sleep214865-21apt-get08:17:322
3134725719,24sleep20-21swapper/207:27:412
3111525714,25sleep031210-21perl12:32:370
267325716,21sleep10-21swapper/109:22:201
2276525717,18sleep10-21swapper/112:07:181
2198625717,18sleep00-21swapper/008:37:490
2012225715,19sleep20-21swapper/208:32:432
169025717,25sleep01630-21latency_hist07:37:200
908725615,18sleep20-21swapper/209:42:282
32025616,25sleep2389-21ls09:12:482
244425615,20sleep20-21swapper/211:02:552
208842565,39sleep00-21swapper/008:35:430
1893625620,21sleep00-21swapper/011:52:480
3151725512,25sleep031578-21users09:07:500
2512925515,26sleep025205-21systemctl07:07:480
2191025514,25sleep021916-21ntp_states12:02:450
2146125514,17sleep20-21swapper/212:02:352
192972555,38sleep00-21swapper/008:30:100
143525521,19sleep20-21swapper/209:17:372
1377125515,18sleep00-21swapper/008:12:460
1117625518,20sleep20-21swapper/211:32:182
3121025411,22sleep10-21swapper/109:07:461
2958425414,24sleep029578-21apt-get10:47:300
2958425414,24sleep029578-21apt-get10:47:300
420725318,20sleep24234-21sed07:42:512
2597925312,24sleep025973-21apt-get08:52:310
2520625313,21sleep10-21swapper/107:07:491
1703325313,16sleep00-21swapper/011:48:190
1328425317,21sleep213278-21apt-get08:12:322
35742526,34sleep10-21swapper/111:07:261
2487225212,23sleep024879-21df_inode12:12:340
215425214,22sleep02161-21diskstats11:02:490
192425213,23sleep21920-21apt-get07:37:302
38792516,33sleep00-21swapper/009:23:480
3125425114,21sleep00-21swapper/007:27:300
3026625114,20sleep130397-21unixbench_multi10:47:481
3026625114,20sleep130397-21unixbench_multi10:47:481
2991825113,17sleep20-21swapper/207:22:362
2764525111,26sleep227658-21fschecks_count08:57:382
2639025111,15sleep20-21swapper/212:17:332
2490925111,24sleep124904-21apt-get10:32:301
1534225111,24sleep115369-21grep10:02:331
3022225012,14sleep00-21swapper/012:27:480
2815925012,23sleep128156-21apt-get07:17:301
2649125017,18sleep00-21swapper/010:37:330
2299625010,23sleep022522-21apt-get07:02:240
2066325016,21sleep120238-21/usr/sbin/munin10:17:441
1066125012,22sleep110656-21apt-get09:47:311
46192495,10sleep10-21swapper/107:45:491
2983824913,15sleep10-21swapper/107:22:351
2871324914,21sleep20-21swapper/207:17:472
1288624911,22sleep10-21swapper/109:52:481
2308524713,18sleep00-21swapper/008:42:380
1990224612,17sleep00-21swapper/011:57:320
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional