You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 03:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Wed Jul 01, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
67762536136,386sleep06812-21cat00:30:080
67762536136,386sleep06812-21cat00:30:080
27967247024,429sleep028095-21latency_hist23:54:350
2716024577,393sleep027204-21ntpq20:24:570
1561423890,191sleep04339-1kworker/0:0H19:49:350
8826235521,315sleep28820-21apt-get21:09:492
117362348139,185sleep011883-21latency_hist21:19:360
32392234517,312sleep124084-21kworker/u8:122:24:451
1843622979,229sleep018151-21/usr/sbin/munin21:39:550
7814228726,242sleep27792-21gpgv00:34:352
7814228726,242sleep27792-21gpgv00:34:352
31745225581,21sleep00-21swapper/020:39:510
31745225581,21sleep00-21swapper/020:39:510
23650224870,163sleep023658-21cpu20:14:480
168672244149,16sleep00-21swapper/021:34:540
24077224076,146sleep024124-21modprobe21:59:320
334922349,109sleep00-21swapper/019:09:580
334922349,109sleep00-21swapper/019:09:580
26494223428,190sleep026517-21munin-plugin-st20:24:330
3109220612,145sleep03175-21wc20:50:370
3109220612,145sleep03175-21wc20:50:370
19672220113,166sleep019815-21unixbench_singl20:00:100
14784220021,162sleep014853-21switchtime23:10:030
14784220021,162sleep014853-21switchtime23:10:030
1390221946,178sleep0364-21snmpd23:09:180
19571219328,107sleep019719-21grep23:25:040
1863221908,67sleep00-21swapper/019:59:330
16228218712,160sleep016266-21processes23:15:030
5149218412,144sleep00-21swapper/022:40:020
2988021828,161sleep029911-21tune2fs23:59:520
307821786,12sleep00-21swapper/022:34:490
2055521787,15sleep00-21swapper/020:04:500
2055521787,15sleep00-21swapper/020:04:500
21810217611,143sleep00-21swapper/021:50:040
2917217510,151sleep03026-21sed00:19:340
7747217313,145sleep37778-21grep22:49:503
921221709,126sleep09224-21awk00:35:340
18542216816,139sleep218569-21awk21:39:572
29132216617,18sleep00-21swapper/022:14:510
10546216612,102sleep010605-21modprobe22:59:330
20214216515,24sleep00-21swapper/021:45:030
857521638,138sleep00-21swapper/021:09:350
5334215825,101sleep00-21swapper/019:14:590
15730215821,18sleep00-21swapper/021:30:050
31667215618,25sleep20-21swapper/220:39:452
31667215618,25sleep20-21swapper/220:39:452
4973215115,22sleep00-21swapper/000:24:550
436621475,13sleep00-21swapper/000:23:540
4596214510,57sleep00-21swapper/020:55:010
2866921455,11sleep00-21swapper/023:55:000
20593214513,118sleep220600-21forks20:04:522
20593214513,118sleep220600-21forks20:04:522
8526214318,59sleep00-21swapper/019:25:010
2538021416,48sleep00-21swapper/022:00:540
253621405,12sleep00-21swapper/022:31:300
1473621385,49sleep00-21swapper/021:29:130
3103021369,68sleep031066-21ntpq22:19:590
2379821367,61sleep00-21swapper/023:39:540
2379821367,61sleep00-21swapper/023:39:540
3169821358,91sleep031739-21http00:09:330
889621348,71sleep00-21swapper/021:09:510
17611213411,86sleep00-21swapper/023:19:560
31375213111,99sleep31-21systemd20:39:343
134021318,15sleep00-21swapper/020:45:040
1204321318,90sleep012052-21forks21:19:520
2622321305,34sleep00-21swapper/023:47:410
1153221309,82sleep02756-21cyclictest19:34:580
10186212915,101sleep210253-21switchtime19:30:032
1007321298,87sleep010107-21ps19:30:010
3238421276,67sleep00-21swapper/022:24:400
2244621267,85sleep022447-21ntp_states23:35:010
290421240,10sleep20-21swapper/220:50:302
290421240,10sleep20-21swapper/220:50:302
1129921240,12sleep30-21swapper/322:59:593
4235212215,93sleep34271-21basename19:14:343
4235212215,93sleep34271-21basename19:14:343
3008421228,53sleep00-21swapper/020:34:430
1132021206,47sleep00-21swapper/023:00:010
21626211918,90sleep00-21swapper/023:33:400
7773211122,71sleep17796-21awk19:24:331
13920211020,73sleep313931-21modprobe23:09:333
22778210920,72sleep222897-21timerwakeupswit20:10:052
22778210920,72sleep222897-21timerwakeupswit20:10:052
9613210815,78sleep19692-21latency_hist00:39:351
3222221088,65sleep032211-21apt-get00:09:450
4330210719,69sleep34380-21sendmail22:39:333
28465210767,28sleep00-21swapper/022:10:470
32701210614,77sleep332713-21ntpq00:10:213
12692210622,63sleep112841-21grep21:20:051
23648210518,69sleep223736-21tune2fs23:39:522
23648210518,69sleep223736-21tune2fs23:39:522
6577210414,72sleep36583-21/usr/sbin/munin00:29:593
6577210414,72sleep36583-21/usr/sbin/munin00:29:593
5788210414,74sleep15795-21/usr/sbin/munin20:59:511
16104210359,25sleep02756-21cyclictest19:50:000
15837210318,70sleep215907-21df19:49:542
26367210211,20sleep226401-21munin-run23:49:332
16684210221,53sleep116753-21fschecks_time21:34:511
15760210230,39sleep20-21swapper/223:14:492
968210119,63sleep2761-21/usr/sbin/munin20:44:542
10909210020,61sleep10-21swapper/119:34:351
132429916,64sleep31467-21latency_hist22:29:353
479229812,69sleep34908-21timerwakeupswit20:55:043
2888329812,73sleep329008-21unixbench_multi23:55:033
909329615,57sleep38250-21/usr/sbin/munin00:35:333
1713229514,57sleep317164-21latency_hist19:54:353
264152947,26sleep00-21swapper/022:05:020
128072947,48sleep00-21swapper/019:39:510
127429319,52sleep21321-21cat22:29:342
754929257,16sleep00-21swapper/022:49:350
65922917,46sleep00-21swapper/019:19:520
1644029019,49sleep316573-21sed23:15:063
1289029015,56sleep312952-21/usr/sbin/munin19:39:543
59182886,45sleep00-21swapper/020:59:540
2880128811,59sleep228880-21find23:55:022
1785828826,43sleep317870-21modprobe21:39:333
125928819,41sleep21046-21apt-get00:14:362
651928613,59sleep26524-21cpu19:19:502
3182828628,41sleep231947-21cat00:09:352
356028513,56sleep13643-21mailstats22:35:021
3159928515,58sleep11-21systemd22:20:491
3008828517,49sleep229605-21apt-get20:34:452
1973128526,32sleep30-21swapper/321:44:503
85952848,15sleep00-21swapper/022:50:080
2971728317,46sleep129878-21cmp20:34:351
1733728320,39sleep217470-21head21:35:042
283992827,15sleep00-21swapper/020:29:500
139632829,60sleep314000-21ps21:25:003
3194828118,42sleep332036-21munin-run22:24:343
953628015,48sleep30-21swapper/321:10:063
470728035,29sleep24789-21switchtime20:55:032
1553828017,47sleep2225-21systemd-journal19:49:342
1241528024,36sleep30-21swapper/319:39:333
2579227920,26sleep30-21swapper/323:45:103
2302327918,29sleep00-21swapper/020:10:110
2302327918,29sleep00-21swapper/020:10:110
1083227926,26sleep20-21swapper/221:15:022
3031427827,33sleep230340-21cat22:19:332
201452787,16sleep10-21swapper/120:03:381
792827720,41sleep18058-21sed21:05:081
479027722,27sleep10-21swapper/100:24:521
218227720,35sleep30-21swapper/320:49:353
151527714,47sleep30-21swapper/322:29:473
897727622,37sleep29091-21timerwakeupswit00:35:322
709727627,26sleep27107-21vmstat22:45:092
1773827514,44sleep317781-21ntpq23:19:593
1904327421,37sleep319124-21grep19:59:573
1283627430,19sleep20-21swapper/223:05:362
782427314,18sleep30-21swapper/319:24:353
3222927316,39sleep132300-21/usr/sbin/munin20:40:051
3222927316,39sleep132300-21/usr/sbin/munin20:40:051
3040327318,32sleep10-21swapper/100:00:051
2964927320,34sleep329730-21cat22:15:023
854927224,28sleep10-21swapper/119:25:021
1555927217,40sleep215642-21mailstats21:30:032
2678427122,27sleep30-21swapper/320:24:483
1779627113,42sleep117802-21ntp_states23:20:011
1439427119,40sleep314456-21if_err_eth023:09:533
1439427119,40sleep314456-21if_err_eth023:09:533
1244327117,26sleep20-21swapper/221:20:022
1074127117,35sleep310349-21/usr/sbin/munin21:14:593
214427013,42sleep12240-21sed00:15:071
2019827026,23sleep20-21swapper/223:29:342
77626925,22sleep10-21swapper/100:11:151
2616026926,20sleep10-21swapper/122:04:541
2510726914,34sleep225252-21/usr/sbin/munin22:00:072
248202699,14sleep30-21swapper/323:44:333
248202699,14sleep30-21swapper/323:44:333
2324126920,23sleep30-21swapper/323:39:323
1386726918,27sleep20-21swapper/221:24:552
936726818,23sleep20-21swapper/219:29:352
852126819,34sleep38564-21munin-run21:09:343
473526816,21sleep20-21swapper/222:39:522
2958726814,38sleep329627-21http20:34:333
2671026815,21sleep30-21swapper/323:49:503
2415726817,35sleep224268-21date21:59:342
1300326819,20sleep10-21swapper/119:39:561
1013426810,14sleep10-21swapper/121:14:361
668526722,19sleep10-21swapper/122:45:041
590526724,21sleep30-21swapper/322:44:333
3240326719,20sleep30-21swapper/320:40:073
3240326719,20sleep30-21swapper/320:40:073
2733526715,22sleep10-21swapper/120:25:021
1935726713,39sleep119401-21processes23:25:011
608126616,33sleep36127-21munin-run00:29:353
530226610,12sleep30-21swapper/319:14:583
3104726617,21sleep10-21swapper/120:35:201
2532726614,38sleep125418-21/usr/sbin/munin20:19:541
2322226613,29sleep30-21swapper/321:54:593
22412660,7sleep20-21swapper/200:15:072
2186526620,21sleep10-21swapper/120:09:361
2186526620,21sleep10-21swapper/120:09:361
184892666,12sleep30-21swapper/319:57:423
1140526614,37sleep111428-21irqstats19:34:551
1062426624,20sleep20-21swapper/222:59:352
2998226518,32sleep230043-21if_eth023:59:542
2647226518,17sleep10-21swapper/123:49:361
24522659,12sleep20-21swapper/222:30:272
2347926520,20sleep10-21swapper/121:55:051
1784326517,29sleep117906-21ps19:55:021
162962657,11sleep10-21swapper/119:50:051
1027626519,23sleep30-21swapper/319:30:043
259026424,21sleep10-21swapper/120:50:021
259026424,21sleep10-21swapper/120:50:021
1949226423,21sleep10-21swapper/121:44:361
1489526418,26sleep10-21swapper/119:45:051
1218526420,23sleep10-21swapper/123:04:341
441526324,24sleep21-21systemd00:24:332
2562026311,32sleep125693-21/usr/sbin/munin23:45:081
2195826313,26sleep20-21swapper/221:50:062
2174726320,18sleep20-21swapper/223:34:352
2060326316,21sleep30-21swapper/320:04:533
2060326316,21sleep30-21swapper/320:04:533
176326323,18sleep00-21swapper/019:04:580
1758626316,19sleep20-21swapper/219:54:552
1421226316,23sleep10-21swapper/123:09:471
1421226316,23sleep10-21swapper/123:09:471
803626223,20sleep30-21swapper/319:24:433
437826219,24sleep24507-21df_inode19:14:372
389526216,24sleep30-21swapper/320:54:363
389526216,24sleep30-21swapper/320:54:363
270326219,26sleep12721-21munin-run19:09:351
2411826217,33sleep224165-21processes20:15:022
87226118,25sleep1967-21perl20:44:531
487326116,29sleep20-21swapper/200:24:532
41052615,17sleep20-21swapper/219:12:512
41052615,17sleep20-21swapper/219:12:512
278732617,12sleep20-21swapper/222:10:002
2499626123,22sleep30-21swapper/320:19:353
2332126116,24sleep10-21swapper/123:39:351
168926120,18sleep30-21swapper/300:14:573
146812619,23sleep30-21swapper/319:44:583
2812126017,22sleep228270-21sed20:29:362
1575926010,31sleep10-21swapper/123:14:481
1335926024,19sleep10-21swapper/123:05:481
2870925918,25sleep128799-21grep23:55:011
2830325918,25sleep228306-21/usr/sbin/munin20:29:442
2525325915,19sleep30-21swapper/322:00:083
2523425916,25sleep325230-21apt-get20:19:473
166225923,17sleep20-21swapper/219:04:562
1326525914,30sleep313375-21latency_hist21:24:353
681525818,21sleep30-21swapper/322:45:063
638225811,18sleep20-21swapper/221:00:042
2861325818,20sleep10-21swapper/122:12:421
2314025815,22sleep10-21swapper/120:11:421
2314025815,22sleep10-21swapper/120:11:421
2257725813,33sleep322558-21latency_hist21:54:353
928925717,23sleep39284-21apt-get22:54:473
903925719,17sleep20-21swapper/222:54:342
522425716,21sleep10-21swapper/122:40:031
348325710,17sleep10-21swapper/119:10:031
348325710,17sleep10-21swapper/119:10:031
2742525712,19sleep20-21swapper/223:50:052
1733825715,26sleep217335-21apt-get23:19:452
774525614,25sleep17740-21apt-get22:49:481
270742566,10sleep30-21swapper/322:07:473
2603325615,25sleep226037-21diskstats22:04:512
237125615,16sleep10-21swapper/122:30:061
2198425614,25sleep221980-21apt-get23:34:472
1093525614,16sleep20-21swapper/222:59:512
246192559,36sleep10-21swapper/123:41:571
246192559,36sleep10-21swapper/123:41:571
1457725514,21sleep20-21swapper/223:09:562
1457725514,21sleep20-21swapper/223:09:562
947325410,23sleep10-21swapper/122:54:531
776725413,27sleep11-21systemd00:34:341
776725413,27sleep11-21systemd00:34:341
650925412,24sleep36609-21users21:00:053
446425413,20sleep30-21swapper/300:24:343
315825414,24sleep13155-21apt-get00:19:481
3064525412,24sleep330698-21users00:00:073
3063525413,24sleep230628-21apt-get22:19:492
2951925418,19sleep10-21swapper/122:15:001
2449425412,30sleep124580-21fschecks_time21:59:541
2127725413,25sleep121271-21apt-get21:49:491
1505725414,25sleep115124-21/usr/sbin/munin21:29:501
315325312,24sleep23159-21diskstats22:34:512
216832536,11sleep30-21swapper/323:34:243
1408425314,20sleep10-21swapper/121:25:021
2654525217,19sleep20-21swapper/220:24:352
2364625212,23sleep123643-21apt-get20:14:451
493425113,22sleep14928-21apt-get19:14:481
2290025113,19sleep323005-21users20:10:063
2290025113,19sleep323005-21users20:10:063
2112225116,18sleep10-21swapper/123:30:061
1282025120,16sleep20-21swapper/219:39:532
1980725013,23sleep119813-21diskstats21:44:521
1161125019,17sleep20-21swapper/219:35:002
1981424912,22sleep219824-21/usr/sbin/munin21:44:532
1933724912,17sleep30-21swapper/323:24:593
1531624914,20sleep30-21swapper/321:29:563
139324911,19sleep30-21swapper/319:04:493
739324811,17sleep20-21swapper/221:04:562
1903824814,17sleep20-21swapper/219:59:562
1866124814,14sleep30-21swapper/321:40:023
453424410,20sleep10-21swapper/120:54:581
2875724412,16sleep30-21swapper/322:14:333
3174024110,15sleep10-21swapper/100:09:341
1064224113,14sleep10-21swapper/121:14:561
315542339,13sleep30-21swapper/300:07:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional