You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 23:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jun 14, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
643524956,447sleep3363-21snmpd11:18:403
8078239911,374sleep38091-21sort09:42:463
2108623560,14sleep10-21swapper/108:38:061
147362348135,194sleep314780-21awk10:03:053
84023356,289sleep3878-21awk07:37:353
10204227510,243sleep310242-21fschecks_count11:32:533
10204227510,243sleep310242-21fschecks_count11:32:533
10379224015,202sleep210399-21cut11:32:562
10379224015,202sleep210399-21cut11:32:562
30283222416,170sleep330434-21latency_hist07:27:383
2559822209,196sleep125681-21tune2fs10:37:561
8641221940,158sleep3225-21systemd-journal08:02:353
23110221931,171sleep023139-21latency_hist07:07:360
23110221931,171sleep023139-21latency_hist07:07:360
7295221614,187sleep07379-21cat11:22:570
7295221614,187sleep07379-21cat11:22:570
3219221617,177sleep30-21swapper/311:08:063
32299220911,69sleep332403-21timerandwakeup10:58:053
11750220924,164sleep111821-21grep09:53:051
28797220823,172sleep328772-21/usr/sbin/munin12:32:503
24207220487,37sleep324280-21switchtime10:33:053
14870219215,22sleep014875-21diskstats11:47:520
10322218910,166sleep310450-21cat09:48:073
744021887,166sleep3363-21snmpd11:23:003
744021887,166sleep3363-21snmpd11:23:003
30459218515,155sleep330474-21acpi07:27:463
24549217721,95sleep30-21swapper/312:18:003
24549217721,95sleep30-21swapper/312:18:003
1249521777,21sleep30-21swapper/308:13:053
4377217616,105sleep30-21swapper/311:13:033
26097217311,146sleep026142-21ntpq12:22:580
19429217010,18sleep30-21swapper/308:34:093
710216810,146sleep3755-21awk09:18:023
18289216534,94sleep318339-21ntp_kernel_pll_11:58:013
109672165130,21sleep311032-21awk08:08:043
7018216415,15sleep30-21swapper/309:38:013
2246921629,139sleep322588-21cat08:43:053
24676216112,112sleep324723-21find07:12:363
1567021618,139sleep315713-21latency08:22:583
28708215820,28sleep00-21swapper/007:22:380
16146215815,130sleep316159-21kernelversion10:07:573
15114215719,59sleep115162-21unin-run08:22:361
2701221569,23sleep30-21swapper/312:27:373
6737215418,76sleep26739-21cron11:22:352
3642215316,122sleep13671-21latency_hist11:12:361
2123721539,104sleep321391-21awk10:23:083
21620215113,86sleep30-21swapper/308:42:373
6438214917,112sleep00-21swapper/007:53:070
2909214516,111sleep30-21swapper/307:42:573
24522214519,110sleep224547-21netstat12:17:592
24522214519,110sleep224547-21netstat12:17:592
27600214016,86sleep30-21swapper/310:43:063
153621408,121sleep31577-21ntpq07:38:003
2875221316,87sleep30-21swapper/310:47:553
1478821316,21sleep30-21swapper/311:47:473
22591213020,87sleep00-21swapper/008:43:060
19379213014,22sleep30-21swapper/310:17:593
31760212910,17sleep30-21swapper/309:13:063
136821299,47sleep31442-21cat11:03:053
27243212710,78sleep30-21swapper/307:17:573
13243212790,18sleep30-21swapper/311:42:513
1501621266,12sleep30-21swapper/308:21:213
14084212610,83sleep314093-21munin-run10:02:363
968321248,81sleep3363-21snmpd11:30:393
968321248,81sleep3363-21snmpd11:30:393
21496212018,17sleep30-21swapper/312:08:033
26834211916,85sleep2309-21rs:main2
6351211711,75sleep30-21swapper/307:53:063
2654021178,77sleep30-21swapper/308:57:473
11943211614,82sleep311177-21/usr/sbin/munin09:53:073
22562211510,90sleep322603-21grep10:28:003
17398211584,19sleep317408-21find10:12:463
20188211322,30sleep320339-21unixbench_singl12:03:063
4159211120,73sleep10-21swapper/111:12:571
24066211074,22sleep324195-21ls08:48:063
25673210913,82sleep225676-21acpi12:22:432
30136210611,81sleep130243-21latency_hist12:37:361
18280210626,64sleep118399-21cat08:32:371
3065721047,62sleep30-21swapper/309:09:143
2351210410,60sleep32389-21processes09:23:033
31453210310,54sleep30-21swapper/310:57:363
1255210313,68sleep1769-21/usr/sbin/munin11:03:031
2857221027,22sleep30-21swapper/312:32:383
4980210118,42sleep20-21swapper/209:32:472
27966210120,64sleep228108-21awk12:28:042
18656210016,67sleep218798-21grep11:58:082
290572995,83sleep30-21swapper/309:03:453
249612998,57sleep324958-21apt-get08:52:463
105729921,22sleep20-21swapper/209:18:092
99802987,14sleep30-21swapper/308:04:533
129522989,56sleep312983-21munin-run11:42:343
3212129723,50sleep232160-21cut07:32:542
1587229714,70sleep115931-21processes08:23:041
1300229715,65sleep113043-21cut11:42:361
50582967,16sleep30-21swapper/309:32:523
251352966,14sleep30-21swapper/307:12:403
2471629614,69sleep124726-21pluginstate08:52:361
3009129531,32sleep00-21swapper/010:52:450
2664229510,70sleep126644-21sshd07:15:081
237702957,12sleep00-21swapper/007:08:090
931429415,66sleep29364-21ntp_kernel_pll_08:03:022
711129412,66sleep17196-21fschecks_time11:22:541
711129412,66sleep17196-21fschecks_time11:22:541
474629428,34sleep10-21swapper/109:32:381
172802936,59sleep30-21swapper/311:53:333
2710929225,49sleep00-21swapper/010:42:550
1721629148,21sleep30-21swapper/308:27:593
1353629124,41sleep013736-21gpgv08:17:380
1821229012,64sleep018279-21apt-key08:32:360
984928914,61sleep19909-21if_eth009:47:561
46802897,49sleep30-21swapper/309:32:243
2207628929,38sleep122095-21uname07:03:101
2207628929,38sleep122095-21uname07:03:101
712828817,53sleep17149-21sort07:57:361
258942887,44sleep30-21swapper/310:38:023
1954528613,53sleep018985-21/usr/sbin/munin10:18:040
3130528531,33sleep10-21swapper/109:12:551
2807228519,25sleep20-21swapper/209:02:472
2383828415,49sleep223840-21cron12:17:342
194822847,48sleep00-21swapper/012:02:510
1750228421,26sleep20-21swapper/210:12:532
65542839,62sleep00-21swapper/011:20:100
2428428321,46sleep024407-21timerwakeupswit10:33:060
1501628211,52sleep115111-21timerwakeupswit10:03:111
656228110,54sleep00-21swapper/009:37:500
52542817,40sleep30-21swapper/307:49:143
52542817,40sleep30-21swapper/307:49:143
238072818,40sleep30-21swapper/312:17:113
1610528118,43sleep116252-21latency_hist11:52:381
1596728114,53sleep016026-21fschecks_count10:07:540
1329928125,36sleep113324-21apt-get08:17:351
320032809,12sleep00-21swapper/007:32:400
191662797,57sleep00-21swapper/008:34:020
163372798,15sleep20-21swapper/211:52:512
1346927915,46sleep113510-21irqstats11:42:571
1168427919,45sleep011749-21awk09:53:040
2164227810,24sleep10-21swapper/112:08:051
2033627817,45sleep220293-21latency_hist10:22:382
2341827713,17sleep00-21swapper/008:47:510
1953827714,46sleep119547-21diskstats12:02:521
1058527712,52sleep010623-21processes11:33:030
1058527712,52sleep010623-21processes11:33:030
23012768,57sleep00-21swapper/007:41:080
2218527630,29sleep20-21swapper/210:27:512
2850827516,43sleep00-21swapper/012:32:350
1936427522,32sleep219428-21/usr/sbin/munin08:34:082
628627413,48sleep26350-21sendmail_mailst07:53:052
3235927415,44sleep232360-21sshd09:16:072
2927127413,42sleep029330-21/usr/sbin/munin12:33:030
2528527417,22sleep00-21swapper/010:37:380
2109827419,37sleep20-21swapper/212:07:532
1634227414,42sleep116374-21proc_pri10:08:031
2398927321,33sleep224064-21switchtime08:48:052
2076527313,37sleep120532-21/usr/sbin/munin10:22:571
350727215,40sleep03409-21/usr/sbin/munin09:27:530
320927217,25sleep20-21swapper/209:27:382
2694027118,24sleep00-21swapper/007:17:380
2549727115,27sleep00-21swapper/008:53:040
890927011,45sleep08914-21acpi08:02:480
2464327016,33sleep224657-21apt-get07:12:352
1264127014,17sleep20-21swapper/208:13:082
435226914,23sleep20-21swapper/211:13:022
1900926922,29sleep119004-21apt-get10:17:491
2890926821,24sleep00-21swapper/007:22:470
1759426824,30sleep117618-21munin-run11:57:351
1327926816,28sleep00-21swapper/009:58:050
1286526813,21sleep20-21swapper/209:57:552
117126821,24sleep20-21swapper/211:03:002
967126723,17sleep10-21swapper/108:03:101
525426714,35sleep05348-21memory09:32:570
2662226716,26sleep20-21swapper/210:41:082
2974426610,33sleep10-21swapper/109:07:551
2742026617,32sleep227454-21/usr/sbin/munin10:43:012
1183526615,20sleep10-21swapper/108:12:381
1138826613,24sleep00-21swapper/008:08:090
2966126515,24sleep20-21swapper/209:07:522
2102426511,33sleep00-21swapper/010:23:050
205642657,11sleep20-21swapper/212:05:312
49532646,11sleep10-21swapper/109:32:421
316626415,33sleep23200-21ps07:43:042
2546526418,20sleep00-21swapper/012:22:370
2546526418,20sleep00-21swapper/012:22:370
231526415,33sleep22334-21apt-get11:07:452
1847026420,25sleep00-21swapper/011:58:060
1304426420,19sleep20-21swapper/211:42:372
1206126423,23sleep00-21swapper/011:37:590
91382636,12sleep10-21swapper/111:28:031
91382636,12sleep10-21swapper/111:28:031
799426314,26sleep00-21swapper/007:58:080
286602636,12sleep20-21swapper/210:47:542
2250026311,35sleep022541-21ntpq10:27:570
1351226317,31sleep213594-21sed11:42:582
716326215,20sleep20-21swapper/209:38:042
1604726216,27sleep216079-21awk11:52:352
983126121,25sleep09841-21modprobe11:32:340
983126121,25sleep09841-21modprobe11:32:340
432526116,20sleep20-21swapper/207:47:522
432526116,20sleep20-21swapper/207:47:522
3121226121,18sleep00-21swapper/007:28:080
2322126114,31sleep023345-21ls12:13:040
2988026013,27sleep00-21swapper/010:52:360
2973426010,33sleep029743-21/usr/sbin/munin09:07:540
2317026018,18sleep20-21swapper/208:47:362
1840026018,20sleep20-21swapper/208:32:382
1671326015,25sleep20-21swapper/208:27:382
1548826013,20sleep00-21swapper/008:22:550
1427126016,25sleep214314-21ntpq08:18:132
425125920,19sleep00-21swapper/007:47:500
425125920,19sleep00-21swapper/007:47:500
2865425919,25sleep128546-21/usr/sbin/munin10:47:531
2785625915,23sleep10-21swapper/112:28:031
250425918,22sleep12650-21latency_hist07:42:381
2156625925,15sleep00-21swapper/012:08:040
522525813,29sleep05240-21munin-run11:17:360
44692586,12sleep20-21swapper/209:29:402
275302586,39sleep20-21swapper/208:58:592
2339725816,24sleep123464-21df07:07:581
1874125811,26sleep10-21swapper/108:32:541
988225722,20sleep20-21swapper/211:32:362
988225722,20sleep20-21swapper/211:32:362
965425716,25sleep29649-21apt-get09:47:492
92025721,13sleep10-21swapper/107:37:381
869325715,21sleep10-21swapper/108:02:371
308602576,12sleep10-21swapper/107:28:011
547825615,25sleep15472-21apt-get11:17:491
2974225616,21sleep10-21swapper/107:23:201
2699825614,17sleep10-21swapper/108:58:011
2391925621,17sleep10-21swapper/112:17:381
1638025612,16sleep20-21swapper/210:08:042
837325521,16sleep20-21swapper/209:42:572
2415525513,25sleep124148-21apt-get12:17:501
2415525513,25sleep124148-21apt-get12:17:501
2200525522,16sleep10-21swapper/108:42:551
2179625514,24sleep221790-21apt-get07:02:502
2179625514,24sleep221790-21apt-get07:02:502
2031025513,25sleep220307-21apt-get08:37:472
188325515,24sleep01879-21apt-get09:22:480
1753625516,23sleep017631-21perl10:12:550
164525513,21sleep10-21swapper/109:22:371
1382625517,17sleep00-21swapper/008:17:460
1084725514,20sleep20-21swapper/208:07:592
1002325515,25sleep00-21swapper/009:47:590
88625414,24sleep00-21swapper/009:18:070
831425413,17sleep10-21swapper/109:42:561
2834425412,18sleep00-21swapper/009:02:570
270325412,25sleep12697-21apt-get07:42:501
2526425417,23sleep225331-21memory08:52:582
2055625414,24sleep220552-21apt-get10:22:502
1660425414,18sleep00-21swapper/011:52:570
1434025416,22sleep014334-21apt-get10:02:530
115825417,16sleep20-21swapper/207:37:512
790325310,27sleep27926-21users11:23:072
790325310,27sleep27926-21users11:23:072
3255325315,20sleep132594-21users10:58:071
3185025313,19sleep00-21swapper/009:13:070
2809725313,23sleep128091-21apt-get09:02:501
2782225314,15sleep10-21swapper/110:43:091
278052535,38sleep00-21swapper/009:02:330
2496425314,23sleep124958-21apt-get08:52:481
234025313,23sleep02334-21apt-get11:07:470
2198225315,18sleep30-21swapper/307:03:073
2198225315,18sleep30-21swapper/307:03:073
1342525310,21sleep00-21swapper/011:42:560
761225213,22sleep27646-21irqstats07:57:582
3209925210,23sleep232144-21ntp_states10:58:002
319525218,21sleep10-21swapper/109:27:371
3179925213,18sleep20-21swapper/207:32:382
3024325215,18sleep20-21swapper/207:27:372
2577625211,19sleep10-21swapper/112:22:511
2254325215,21sleep122554-21ntp_states10:27:581
2492025116,18sleep10-21swapper/110:37:351
2841125021,16sleep10-21swapper/107:22:351
2096325012,21sleep021084-21timerwakeupswit08:38:060
79424914,17sleep00-21swapper/011:02:500
3169424912,23sleep031709-21grep10:57:490
3011424911,21sleep230110-21apt-get10:52:472
238792497,15sleep20-21swapper/210:32:562
1277924912,21sleep112772-21apt-get09:57:501
474024813,17sleep10-21swapper/107:48:031
474024813,17sleep10-21swapper/107:48:031
3016324817,15sleep10-21swapper/110:52:501
2906524820,14sleep20-21swapper/212:32:572
1182324814,17sleep20-21swapper/209:53:062
895424713,19sleep09006-21users09:43:070
663024713,15sleep10-21swapper/109:37:511
2846424711,20sleep228549-21apt-config07:22:362
1791324713,21sleep117906-21apt-get11:57:511
1062724713,15sleep10-21swapper/111:33:041
1062724713,15sleep10-21swapper/111:33:041
1792924616,17sleep10-21swapper/110:13:041
1446224611,13sleep20-21swapper/210:03:002
270224511,17sleep00-21swapper/007:42:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional