You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 23:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Sep 22, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
263672600510,5981sleep326341-21kworker/3:209:29:073
10667249106,4892sleep29060-21kworker/2:112:07:062
2383329637,941sleep017607-1kworker/0:2H09:22:500
19934254014,510sleep119944-21/usr/sbin/munin10:52:481
22689250216,469sleep122700-21cron11:02:291
2401724087,353sleep2367-21snmpd11:04:402
6097234956,248sleep26239-21latency_hist11:52:332
15389232559,242sleep215399-21modprobe12:22:292
19279231515,284sleep019325-21/usr/sbin/munin09:07:560
3004722887,243sleep230052-21cpu09:42:462
2628122838,235sleep226355-21fschecks_time07:47:502
14669225518,219sleep114702-21install07:12:321
13109222940,98sleep20-21swapper/207:07:322
1117622037,147sleep211239-21cat12:07:512
30843219916,165sleep030895-21fschecks_count11:27:480
30843219916,165sleep030895-21fschecks_count11:27:480
3185219776,59sleep23315-21grep09:58:012
14778219610,173sleep014918-21latency_hist10:37:320
19945218851,53sleep20-21swapper/210:52:492
8068218111,19sleep10-21swapper/110:13:061
8068218111,19sleep10-21swapper/110:13:061
21430218119,144sleep321422-21apt-get10:57:463
16556218118,119sleep216572-21modprobe07:17:292
16556218118,119sleep216572-21modprobe07:17:292
2514621807,133sleep225295-21latency_hist09:27:322
2972217917,145sleep33013-21cut09:57:583
13019217418,102sleep20-21swapper/208:47:552
197221666,14sleep20-21swapper/208:12:582
14708216313,110sleep214829-21latency_hist07:12:332
2931321576,15sleep20-21swapper/207:57:432
20165215033,76sleep220200-21awk09:12:302
18525214911,86sleep20-21swapper/212:32:302
3115821456,127sleep231201-21ntpq11:27:542
3115821456,127sleep231201-21ntpq11:27:542
19563214411,62sleep219635-21mailstats12:33:272
485621437,124sleep24908-21cat08:22:512
15768214311,120sleep215857-21grep12:22:492
21699214248,75sleep20-21swapper/209:17:142
25006214117,106sleep025068-21wc11:07:570
3100214029,58sleep2367-21snmpd08:17:422
27341213821,103sleep327381-21awk11:17:293
27341213821,103sleep327381-21awk11:17:293
32458213717,99sleep332564-21wc09:48:133
8011213663,60sleep28066-21latency08:32:522
19026213231,78sleep319004-21apt-get12:32:513
13038213129,86sleep013076-21idleruntime-cro07:07:300
20639213010,106sleep020692-21if_err_eth009:12:520
18521213010,45sleep20-21swapper/207:22:502
18521213010,45sleep20-21swapper/207:22:502
1390212916,99sleep11398-21ntpq09:52:591
23794212321,30sleep323812-21apt09:22:413
17418212230,75sleep00-21swapper/007:18:000
17418212230,75sleep00-21swapper/007:18:000
15864212116,84sleep315956-21sed07:13:023
15864212116,84sleep315956-21sed07:13:023
1107721218,60sleep211096-21apt-get08:42:422
22408212029,73sleep222433-21awk09:17:552
17611211911,86sleep017677-21/usr/sbin/munin10:43:180
13893211962,39sleep20-21swapper/212:17:322
458211657,17sleep20-21swapper/211:33:352
458211657,17sleep20-21swapper/211:33:352
20378211634,28sleep20-21swapper/207:27:552
13825211619,82sleep013861-21wc07:07:580
611721138,66sleep26189-21mailstats10:08:022
611721138,66sleep26189-21mailstats10:08:022
2472621127,13sleep20-21swapper/207:42:492
3185211113,82sleep33192-21/usr/sbin/munin08:17:473
12417210913,82sleep312487-21latency_hist08:47:313
17158210811,55sleep20-21swapper/209:02:322
23097210614,79sleep123099-21grep07:37:431
11844210542,23sleep20-21swapper/210:27:532
12329210415,73sleep312413-21mailstats10:28:063
10788210417,68sleep310937-21latency_hist12:07:323
690210329,59sleep2785-21latency_hist09:52:322
386210313,76sleep1457-21tr11:33:341
386210313,76sleep1457-21tr11:33:341
25380210341,50sleep225383-21grep09:27:482
22702210341,46sleep2289-21dbus-daemon11:02:302
14716210231,18sleep214726-21munin-run10:37:302
8999210123,54sleep39020-21latency_hist08:37:303
669021019,18sleep20-21swapper/211:52:572
7690210012,24sleep00-21swapper/011:57:310
30462210013,70sleep030498-21ntp_states09:42:570
19947210010,78sleep019966-21apt-get07:27:410
18101210043,12sleep20-21swapper/207:22:062
18101210043,12sleep20-21swapper/207:22:062
10768210013,70sleep010830-21/usr/sbin/munin10:22:580
2554029912,69sleep025553-21/usr/sbin/munin07:43:040
1797029921,58sleep018120-21grep12:28:000
531829820,63sleep25342-21modprobe10:07:302
531829820,63sleep25342-21modprobe10:07:302
405229817,29sleep20-21swapper/211:45:062
225902986,11sleep20-21swapper/207:35:062
225902986,11sleep20-21swapper/207:35:062
2443829716,66sleep124506-21sed09:23:351
145842977,66sleep20-21swapper/208:52:552
1185829724,50sleep311757-21/usr/sbin/munin07:02:493
2513629629,55sleep225247-21ls11:07:592
1625229616,26sleep10-21swapper/108:57:591
834029535,12sleep20-21swapper/210:16:302
834029535,12sleep20-21swapper/210:16:302
700229522,53sleep17150-21unixbench_multi11:53:041
429029530,38sleep10-21swapper/111:47:311
280022949,66sleep20-21swapper/207:52:552
153329419,59sleep11640-21tune2fs11:37:501
153329419,59sleep11640-21tune2fs11:37:501
1048029432,11sleep20-21swapper/208:41:162
1048029432,11sleep20-21swapper/208:41:162
653229342,33sleep36589-21/usr/sbin/munin11:52:543
2782629314,65sleep327849-21fschecks_count07:52:523
2634829321,15sleep20-21swapper/211:12:522
2634829321,15sleep20-21swapper/211:12:522
1686829347,12sleep20-21swapper/209:00:062
1675529316,62sleep116803-21awk10:42:461
670229218,20sleep20-21swapper/208:28:002
473029116,60sleep34790-21if_eth011:47:533
32728913,61sleep1359-21/usr/sbin/munin08:07:561
3133428915,56sleep131366-21/usr/sbin/munin08:02:591
2929628933,38sleep20-21swapper/211:22:492
1965328929,39sleep20-21swapper/210:52:322
1062528928,32sleep00-21swapper/008:42:320
1062528928,32sleep00-21swapper/008:42:320
2209628815,58sleep322163-21switchtime07:33:013
2209628815,58sleep322163-21switchtime07:33:013
915428720,45sleep19184-21munin-run12:02:291
156762878,15sleep30-21swapper/312:22:423
93328633,13sleep20-21swapper/209:52:482
78932868,67sleep27928-21cpu11:57:462
1544028621,43sleep00-21swapper/012:22:310
1496428628,41sleep20-21swapper/210:37:422
100242869,59sleep00-21swapper/010:22:300
100242869,59sleep00-21swapper/010:22:300
269242858,17sleep20-21swapper/209:32:452
1786228541,10sleep20-21swapper/210:45:342
595928414,55sleep36052-21latency_hist08:27:323
1859028417,43sleep318736-21latency_hist12:32:313
15062848,18sleep20-21swapper/211:37:472
15062848,18sleep20-21swapper/211:37:472
1236628415,52sleep112370-21modprobe08:47:291
322372839,21sleep20-21swapper/208:07:332
2539828315,48sleep325538-21/usr/sbin/munin07:43:033
1375628216,47sleep113799-21ntp_kernel_err10:32:531
999428122,41sleep310022-21basename10:22:293
999428122,41sleep310022-21basename10:22:293
426928113,51sleep04036-21/usr/sbin/munin10:02:510
426928113,51sleep04036-21/usr/sbin/munin10:02:510
358728121,41sleep03660-21threads11:43:020
283342819,58sleep228455-21cat11:18:002
1448528111,52sleep014544-21latency08:52:530
1159528126,25sleep10-21swapper/110:27:321
1137728137,11sleep20-21swapper/210:25:232
3180628037,11sleep20-21swapper/208:03:212
288992806,16sleep20-21swapper/209:37:542
288992806,16sleep20-21swapper/209:37:542
2980927916,48sleep029922-21sed09:42:320
2980927916,48sleep029922-21sed09:42:320
1622327915,51sleep116266-21apt-get10:42:301
1178327922,37sleep111757-21/usr/sbin/munin07:02:471
92132787,50sleep20-21swapper/210:18:002
92132787,50sleep20-21swapper/210:18:002
2863627813,50sleep128706-21fw_packets09:37:491
2863627813,50sleep128706-21fw_packets09:37:491
236452788,56sleep223720-21expr07:38:012
441227612,49sleep04384-21latency_hist08:22:330
1248827624,28sleep10-21swapper/110:28:081
3021227517,43sleep330277-21basename09:42:523
1745527516,38sleep317582-21unixbench_multi10:43:173
1537727519,29sleep00-21swapper/007:12:510
1537727519,29sleep00-21swapper/007:12:510
1430227524,28sleep10-21swapper/108:52:501
1782927414,40sleep317891-21/usr/sbin/munin09:02:583
64727322,26sleep00-21swapper/008:08:030
295227321,25sleep10-21swapper/109:57:561
2524927315,43sleep325392-21unixbench_singl11:08:003
294032727,39sleep10-21swapper/107:57:501
2825727214,31sleep00-21swapper/009:37:320
2261327219,28sleep00-21swapper/009:18:010
1508527213,14sleep30-21swapper/310:37:473
607427113,44sleep16114-21ps10:08:011
607427113,44sleep16114-21ps10:08:011
2027527121,29sleep020309-21munin-run12:37:290
1748327017,28sleep10-21swapper/107:18:011
1748327017,28sleep10-21swapper/107:18:011
874026917,30sleep10-21swapper/110:17:461
874026917,30sleep10-21swapper/110:17:461
2635826917,38sleep326413-21idleruntime07:47:513
1973426916,35sleep319875-21sed07:27:323
1973426916,35sleep319875-21sed07:27:323
1314026916,24sleep00-21swapper/010:32:310
763126823,26sleep30-21swapper/310:13:003
763126823,26sleep30-21swapper/310:13:003
25552687,11sleep00-21swapper/008:14:330
1160526811,42sleep011637-21proc_pri08:43:240
2775626719,24sleep30-21swapper/311:17:493
2198326718,25sleep10-21swapper/107:32:591
2198326718,25sleep10-21swapper/107:32:591
2057826719,30sleep320686-21wc07:28:003
1072126713,37sleep117692-1kworker/1:1H08:42:331
1072126713,37sleep117692-1kworker/1:1H08:42:331
774526613,28sleep00-21swapper/008:32:460
3232526612,23sleep10-21swapper/109:48:111
301526617,23sleep00-21swapper/009:57:590
2668426519,21sleep10-21swapper/109:32:321
2329726520,29sleep11-21systemd09:22:301
18662659,15sleep30-21swapper/311:37:563
18662659,15sleep30-21swapper/311:37:563
3156526412,36sleep131657-21users11:28:011
3156526412,36sleep131657-21users11:28:011
209426415,21sleep00-21swapper/011:38:010
209426415,21sleep00-21swapper/011:38:010
2045726416,31sleep320506-21/usr/sbin/munin09:12:473
1900726413,33sleep119004-21apt-get12:32:411
1309326419,22sleep10-21swapper/112:12:591
114226416,19sleep30-21swapper/309:52:533
1057726413,34sleep31-21systemd08:42:303
1057726413,34sleep31-21systemd08:42:303
3239526320,27sleep332420-21/usr/sbin/munin11:33:123
3239526320,27sleep332420-21/usr/sbin/munin11:33:123
2750226314,24sleep00-21swapper/007:52:320
2278126318,28sleep022813-21munin-run07:37:300
2278126318,28sleep022813-21munin-run07:37:300
120026317,21sleep00-21swapper/009:52:540
119072638,15sleep00-21swapper/012:08:050
76952628,14sleep00-21swapper/010:13:010
76952628,14sleep00-21swapper/010:13:010
585626217,24sleep00-21swapper/011:52:310
473426218,18sleep00-21swapper/008:22:490
3256926214,33sleep032708-21sed09:48:140
315126217,25sleep13174-21df08:17:451
1253326219,26sleep012529-21apt-get12:12:410
3272526121,18sleep00-21swapper/011:33:290
3272526121,18sleep00-21swapper/011:33:290
1830926119,25sleep318302-21apt-get10:47:493
2430626012,33sleep324453-21latency_hist11:07:313
238402609,13sleep00-21swapper/007:38:030
1490326013,29sleep315031-21/usr/sbin/munin12:18:023
619925919,25sleep16266-21/usr/sbin/munin08:27:471
2897025923,17sleep10-21swapper/111:22:321
2595825919,17sleep10-21swapper/107:47:331
209325920,25sleep12181-21cat08:13:011
1867925916,24sleep018717-21ntp_kernel_err10:47:580
1812325913,23sleep10-21swapper/112:28:011
1623225914,31sleep016299-21switchtime12:22:590
1253625915,17sleep30-21swapper/312:12:453
317872587,41sleep00-21swapper/008:03:060
3130725816,24sleep11-21systemd09:47:301
2805725821,20sleep00-21swapper/011:17:540
1867325819,15sleep10-21swapper/109:07:321
1866025821,20sleep00-21swapper/009:07:310
1579225815,25sleep015786-21apt-get08:57:460
1567925815,26sleep115673-21apt-get12:22:441
941725714,27sleep19436-21apt-get12:02:411
646225716,24sleep30-21swapper/308:27:533
2958625711,30sleep029588-21irqstats07:57:530
2692525715,19sleep30-21swapper/309:32:463
2355725717,17sleep10-21swapper/111:03:011
2111025714,26sleep321133-21munin-run10:57:303
1966525718,17sleep30-21swapper/309:08:033
1565825716,26sleep10-21swapper/107:12:561
1565825716,26sleep10-21swapper/107:12:561
1154625718,18sleep30-21swapper/312:08:003
774125616,24sleep17736-21apt-get08:32:431
2699825615,24sleep127007-21diskstats09:32:481
2099125613,21sleep10-21swapper/109:13:011
1961125617,18sleep10-21swapper/110:52:311
1724325615,18sleep30-21swapper/307:17:553
1724325615,18sleep30-21swapper/307:17:553
1064525611,31sleep310655-21needreboot10:22:533
648025514,21sleep00-21swapper/008:27:540
2607525514,24sleep126070-21apt-get11:12:421
2607525514,24sleep126070-21apt-get11:12:421
2034025517,23sleep10-21swapper/107:27:541
160052554,10sleep00-21swapper/010:39:420
1468625520,18sleep10-21swapper/112:18:001
1284125513,17sleep30-21swapper/308:47:523
1175625514,24sleep111891-21uptime12:08:021
103572555,13sleep00-21swapper/012:03:050
628825412,18sleep00-21swapper/010:08:040
628825412,18sleep00-21swapper/010:08:040
463225414,24sleep14644-21apt-get08:22:421
406025412,25sleep14054-21apt-get10:02:441
406025412,25sleep14054-21apt-get10:02:441
2857625414,23sleep328581-21diskstats09:37:473
2857625414,23sleep328581-21diskstats09:37:473
2561525413,24sleep025649-21irqstats09:27:550
1890825412,24sleep118903-21apt-get09:07:441
1842525416,22sleep118419-21apt-get07:22:451
1842525416,22sleep118419-21apt-get07:22:451
1265925412,24sleep112653-21apt-get08:47:461
561025312,24sleep35605-21apt-get10:07:473
561025312,24sleep35605-21apt-get10:07:473
453625314,23sleep14530-21apt-get11:47:471
2920925313,24sleep029204-21apt-get11:22:430
2679925315,23sleep10-21swapper/107:48:011
2544725314,22sleep10-21swapper/111:08:061
2252625319,16sleep30-21swapper/309:18:003
423725215,22sleep34268-21munin-run11:47:293
415925210,15sleep30-21swapper/310:02:503
415925210,15sleep30-21swapper/310:02:503
2947925215,18sleep10-21swapper/111:22:521
2780725212,22sleep127818-21df_inode07:52:501
2026625212,24sleep020302-21ntp_states10:52:550
1472725216,16sleep30-21swapper/310:37:313
1344825217,16sleep30-21swapper/307:07:493
2926125112,21sleep329285-21df_inode11:22:463
261452518,17sleep00-21swapper/011:12:450
261452518,17sleep00-21swapper/011:12:450
1636525112,18sleep30-21swapper/308:58:013
151425114,21sleep31512-21apt-get08:12:423
792925012,20sleep37964-21df11:57:473
3120325012,23sleep331211-21ntp_states11:27:553
3120325012,23sleep331211-21ntp_states11:27:553
87412499,16sleep00-21swapper/011:58:040
1308224912,20sleep013114-21ntp_states08:47:570
2336924810,16sleep30-21swapper/307:37:543
2228624811,21sleep022332-21users10:58:040
1850224811,20sleep018508-21diskstats07:22:480
1850224811,20sleep018508-21diskstats07:22:480
151442486,31sleep30-21swapper/308:53:173
3087124711,19sleep30-21swapper/308:02:453
2981024718,16sleep30-21swapper/307:58:013
2820424613,20sleep028244-21ps07:53:010
2337824611,21sleep00-21swapper/011:02:560
54724311,19sleep3614-21switchtime08:08:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional