You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 12:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon May 25, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
262822490420,4873sleep014889-21kworker/0:121:27:050
188312464019,4608sleep214426-21kworker/2:122:45:042
188312464019,4608sleep214426-21kworker/2:122:45:042
243272295943,2853sleep013151-1kworker/0:2H23:04:000
1182251720,480sleep11200-21/usr/sbin/munin23:34:021
16920235418,320sleep116997-21/usr/sbin/munin22:39:181
12892234413,251sleep013043-21unixbench_multi20:44:200
10308233016,298sleep210321-21/usr/sbin/munin22:19:162
10308233016,298sleep210321-21/usr/sbin/munin22:19:162
2401823166,275sleep024064-21ntpq21:19:130
8832225655,141sleep08885-21latency_hist20:33:500
15886225410,143sleep00-21swapper/000:19:190
28991224845,135sleep029005-21acpi23:18:580
5436223634,189sleep00-21swapper/022:04:120
5436223634,189sleep00-21swapper/022:04:120
11375223315,152sleep011523-21unixbench_singl20:39:240
16719221825,179sleep016766-21ntpq22:39:130
16976220833,132sleep017001-21netstat19:14:120
22600220759,125sleep022683-21latency_hist19:33:500
19691220011,174sleep319778-21cat22:49:133
1671219313,163sleep21708-21proc_pri23:34:192
14330218314,150sleep214460-21grep00:14:212
917218263,82sleep0934-21idleruntime-cro23:33:480
2078521760,6sleep20-21swapper/200:34:232
2695821655,12sleep00-21swapper/023:11:270
14829216215,68sleep014860-21fschecks_count22:34:080
22705216117,20sleep00-21swapper/021:14:210
25069215693,40sleep00-21swapper/019:39:200
7970215030,96sleep37996-21munin-plugin-st22:13:483
7970215030,96sleep37996-21munin-plugin-st22:13:483
11696214914,18sleep00-21swapper/000:07:040
3104721488,104sleep031089-21ntpq19:59:140
8690214611,115sleep08734-21systemctl23:58:480
2643221467,124sleep026497-21awk19:44:160
14411214528,77sleep00-21swapper/019:08:510
1912121438,89sleep00-21swapper/021:04:230
21735214268,24sleep021738-21cron21:13:480
20758214047,53sleep020904-21sed22:53:500
23807213915,111sleep0307-21in:imuxsock23:03:480
17870213916,61sleep00-21swapper/022:44:050
17870213916,61sleep00-21swapper/022:44:050
2825221348,82sleep00-21swapper/021:34:030
2095021337,116sleep0363-21snmpd19:28:090
2878121329,108sleep028843-21sed23:18:500
14110213210,80sleep00-21swapper/020:49:130
11885213224,88sleep111933-21grep00:08:501
6341213115,60sleep00-21swapper/020:24:130
6341213115,60sleep00-21swapper/020:24:130
357521309,107sleep03585-21find21:58:590
357521309,107sleep03585-21find21:58:590
5161212222,76sleep35297-21grep20:19:203
15160212168,42sleep015232-21mailstats19:09:200
15160212168,42sleep015232-21mailstats19:09:200
1012121215,81sleep00-21swapper/000:01:430
1656211711,92sleep21687-21nfs_client20:09:132
11937211511,87sleep312006-21awk22:24:183
3109221136,46sleep031122-21munin-run21:43:480
694621095,93sleep00-21swapper/023:51:130
1647221095,70sleep00-21swapper/020:57:370
8723210816,73sleep38236-21/usr/sbin/munin22:14:163
8723210816,73sleep38236-21/usr/sbin/munin22:14:163
22221210816,71sleep122286-21modprobe22:58:481
17511210852,42sleep017547-21ntpq20:59:260
8777210715,78sleep28873-21cat23:58:502
6258210613,75sleep36292-21ntp_kernel_err23:49:143
3364210643,49sleep03310-21latency_hist21:58:510
2785421056,63sleep00-21swapper/021:32:220
8748210444,44sleep08824-21tr22:14:170
8748210444,44sleep08824-21tr22:14:170
5945210217,68sleep25946-21/usr/sbin/munin20:23:592
5945210217,68sleep25946-21/usr/sbin/munin20:23:592
13426210213,74sleep313435-21systemd-run00:13:493
12096210212,75sleep112100-21acpi00:08:581
27902210020,66sleep227943-21awk19:49:152
16080210018,66sleep316215-21latency_hist22:38:513
329129942,43sleep03378-21basename23:39:180
2714129925,47sleep227155-21apt23:13:472
2966929815,69sleep329767-21sed19:54:213
2951229813,71sleep129581-21sh21:38:491
329529749,37sleep03296-21ntp_states20:14:160
2019829723,17sleep00-21swapper/000:34:130
1658629726,45sleep316699-21latency_hist00:23:513
3240029625,13sleep00-21swapper/020:04:110
2977029623,52sleep00-21swapper/019:54:220
1813429616,66sleep218200-21cat00:28:502
173329638,38sleep00-21swapper/020:09:150
1470829636,42sleep00-21swapper/000:15:480
9462958,74sleep0984-21processes21:49:180
802229525,51sleep18106-21latency_hist22:13:501
802229525,51sleep18106-21latency_hist22:13:501
2884429522,57sleep128982-21modprobe23:18:511
246629526,52sleep32485-21modprobe23:38:483
2125929516,64sleep121298-21latency_hist00:38:511
156332959,70sleep215651-21vmstat22:34:222
3219429424,46sleep232196-21sshd20:04:052
2719129416,58sleep226681-21/usr/sbin/munin21:29:172
128672938,69sleep013015-21apt-key22:28:510
1273529321,54sleep212830-21timerandwakeup00:09:202
184722928,31sleep018581-21perl00:29:090
2388829125,51sleep00-21swapper/019:36:050
1917729023,31sleep10-21swapper/122:48:511
1917729023,31sleep10-21swapper/122:48:511
2738928921,24sleep10-21swapper/121:29:201
216942899,65sleep021845-21grep22:54:190
1750328918,45sleep117646-21grep00:24:171
286632885,48sleep00-21swapper/019:52:240
17842888,28sleep01853-21switchtime23:34:210
1148128815,50sleep111544-21cut22:24:071
3181128721,16sleep00-21swapper/021:44:130
2649828710,64sleep126624-21cat23:09:181
2733028619,50sleep327469-21cut19:48:513
2695528634,35sleep326978-21irqstats21:29:103
1892528615,51sleep218346-21/usr/sbin/munin00:29:192
1808128615,55sleep018084-21cron00:28:480
105028622,25sleep10-21swapper/120:08:521
328428514,52sleep23334-21tr21:58:492
1962928515,49sleep30-21swapper/300:33:483
1492628516,50sleep115015-21/usr/sbin/munin19:09:121
1492628516,50sleep115015-21/usr/sbin/munin19:09:121
108952859,60sleep210924-21irqstats20:39:132
2173828312,21sleep121743-21ntp_states19:29:151
1366428318,50sleep213727-21cut22:29:262
1297728332,14sleep00-21swapper/000:09:220
983328214,36sleep30-21swapper/320:34:223
1660428225,31sleep10-21swapper/119:14:011
671028110,20sleep20-21swapper/222:09:022
671028110,20sleep20-21swapper/222:09:022
965028028,22sleep10-21swapper/120:34:201
56028018,34sleep30-21swapper/321:49:093
3269228021,29sleep10-21swapper/121:48:501
2242428021,31sleep20-21swapper/221:14:152
1423228017,30sleep10-21swapper/120:49:181
417627918,45sleep34269-21sed21:59:183
417627918,45sleep34269-21sed21:59:183
2334327919,43sleep123417-21sendmail_mailtr19:34:201
215742790,6sleep30-21swapper/322:54:173
1699827923,31sleep217162-21grep22:39:192
1315227926,36sleep113159-21/usr/sbin/munin19:04:081
108127912,51sleep21158-21grep21:49:202
46122779,55sleep04675-21cat20:19:090
114762777,15sleep00-21swapper/022:24:060
67132769,16sleep00-21swapper/022:09:040
67132769,16sleep00-21swapper/022:09:040
646227614,45sleep16536-21mailstats20:24:191
646227614,45sleep16536-21mailstats20:24:191
445827613,46sleep24542-21grep23:44:082
3056427623,36sleep00-21swapper/023:24:030
2567427613,45sleep125700-21munin-run19:43:491
1774727621,33sleep317828-21cat20:59:333
2105727521,22sleep10-21swapper/121:09:211
91432737,14sleep00-21swapper/020:34:070
3184127322,34sleep231873-21/usr/sbin/munin21:44:152
3043927322,27sleep20-21swapper/221:39:182
2484827317,39sleep224855-21/usr/sbin/munin23:04:552
254527216,32sleep10-21swapper/120:13:491
1857127210,20sleep00-21swapper/019:19:130
1857127210,20sleep00-21swapper/019:19:130
1190627222,30sleep111934-21munin-run20:43:491
595427110,15sleep30-21swapper/320:24:003
595427110,15sleep30-21swapper/320:24:003
467627112,44sleep14699-21irqstats20:19:101
2492127119,23sleep10-21swapper/121:23:511
102552718,17sleep00-21swapper/022:19:140
102552718,17sleep00-21swapper/022:19:140
764227018,23sleep10-21swapper/123:54:081
751927019,23sleep37536-21uniq23:54:063
43342708,18sleep00-21swapper/023:44:020
3029227018,31sleep30-21swapper/321:39:153
2524527016,33sleep20-21swapper/221:24:082
2022927015,24sleep10-21swapper/119:24:181
1595827023,29sleep30-21swapper/319:13:513
1595827023,29sleep30-21swapper/319:13:513
1073727024,28sleep310623-21/usr/sbin/munin20:39:103
564926917,27sleep20-21swapper/222:04:182
564926917,27sleep20-21swapper/222:04:182
3117826920,24sleep30-21swapper/323:24:183
3063826918,31sleep230640-21sshd19:59:042
2479526925,25sleep20-21swapper/219:39:132
1437726932,21sleep30-21swapper/319:08:503
70026815,29sleep20-21swapper/223:31:042
482326821,21sleep30-21swapper/323:44:173
2649826819,35sleep126568-21sensors19:44:171
2604826822,27sleep30-21swapper/323:09:083
2034126812,40sleep320378-21ntp_states00:34:163
99026718,22sleep10-21swapper/121:49:191
2956426719,21sleep30-21swapper/323:19:183
2656926714,36sleep226645-21/usr/sbin/munin19:44:182
1599426718,30sleep116130-21unixbench_singl00:19:201
1306726727,18sleep20-21swapper/219:04:052
47826621,28sleep30-21swapper/320:04:223
284262668,13sleep30-21swapper/319:49:243
1983026611,34sleep319718-21/usr/sbin/munin19:24:083
1368026613,38sleep313750-21/usr/sbin/munin00:14:053
1246626619,26sleep30-21swapper/320:44:113
597526524,20sleep20-21swapper/223:49:092
2294826518,28sleep322993-21/usr/sbin/munin22:59:153
1939826521,26sleep30-21swapper/321:04:303
1855426515,37sleep318567-21grep19:19:123
1855426515,37sleep318567-21grep19:19:123
1838926514,33sleep2284-21dbus-daemon21:03:482
1588026517,22sleep10-21swapper/120:54:191
1036826519,24sleep20-21swapper/200:03:512
3145426413,33sleep131468-21/usr/sbin/munin23:24:211
2382626417,29sleep123872-21munin-run23:03:491
2350626417,30sleep323632-21sed19:34:223
2254626417,30sleep222577-21awk19:33:482
1425326419,23sleep10-21swapper/100:14:201
3090926317,25sleep10-21swapper/119:59:111
2825526310,40sleep228325-21sh21:34:052
2056726317,22sleep20-21swapper/221:09:102
1678326324,23sleep30-21swapper/322:39:163
491626222,19sleep20-21swapper/222:03:492
491626222,19sleep20-21swapper/222:03:492
433526217,19sleep10-21swapper/123:44:031
324332625,19sleep30-21swapper/321:46:103
3064126211,29sleep30-21swapper/319:59:053
2791726217,23sleep30-21swapper/323:14:173
202922626,12sleep20-21swapper/219:24:202
1608826219,22sleep30-21swapper/320:54:213
151072626,35sleep10-21swapper/122:34:131
1226726213,27sleep20-21swapper/220:44:062
85572616,18sleep10-21swapper/120:30:571
39826124,17sleep10-21swapper/123:29:211
3040026121,19sleep10-21swapper/119:58:511
223226118,23sleep30-21swapper/321:54:093
211126114,21sleep10-21swapper/121:54:071
991626024,18sleep30-21swapper/323:59:233
964026013,32sleep19763-21/usr/sbin/munin23:59:211
369826021,19sleep10-21swapper/121:59:081
369826021,19sleep10-21swapper/121:59:081
2818726016,25sleep228328-21/usr/sbin/munin23:14:202
238312609,11sleep20-21swapper/219:35:222
2239426017,27sleep122423-21/usr/sbin/munin21:14:141
1032226023,19sleep30-21swapper/322:19:173
1032226023,19sleep30-21swapper/322:19:173
928325914,26sleep10-21swapper/122:15:481
928325914,26sleep10-21swapper/122:15:481
409525919,23sleep14160-21date23:43:511
328225915,29sleep33293-21ntpq20:14:153
287125921,21sleep20-21swapper/223:39:072
2153625913,22sleep20-21swapper/222:54:172
1951225918,17sleep10-21swapper/121:04:321
1947125915,22sleep10-21swapper/122:49:071
1033025916,24sleep210355-21munin-run20:38:492
872325820,20sleep30-21swapper/320:33:053
2805925817,24sleep128185-21ls23:14:191
2370025814,26sleep20-21swapper/221:19:072
179425811,34sleep31871-21readlink20:09:193
1688025820,20sleep30-21swapper/319:14:113
1228425812,26sleep20-21swapper/222:24:212
358725714,25sleep33685-21users23:39:213
2572525722,18sleep30-21swapper/319:43:513
2268825712,17sleep20-21swapper/222:59:102
1968025724,16sleep10-21swapper/100:33:501
1950225716,18sleep20-21swapper/219:23:512
1950225716,18sleep20-21swapper/219:23:512
1501425721,23sleep315033-21modprobe20:53:483
1233325714,24sleep112407-21fw_packets20:44:091
3211225615,26sleep332108-21apt-get23:29:033
2445725613,25sleep324464-21df_inode23:04:443
2209125612,16sleep30-21swapper/319:29:213
2058025616,17sleep30-21swapper/321:09:113
1595925620,17sleep20-21swapper/220:54:202
1359925618,18sleep30-21swapper/322:29:223
1318525613,25sleep113174-21apt-get22:29:041
1126025615,19sleep20-21swapper/200:04:232
588625515,23sleep15878-21apt-get23:49:041
2990625515,25sleep129915-21diskstats21:39:051
2515725515,24sleep125151-21apt-get21:24:031
1660525514,25sleep216601-21apt-get19:14:022
3056125412,26sleep230557-21apt-get23:24:012
2947925414,22sleep229516-21ntp_states19:54:152
2881125415,23sleep128821-21munin-run19:53:501
2487025414,26sleep324906-21basename21:23:493
590025318,18sleep10-21swapper/122:04:211
590025318,18sleep10-21swapper/122:04:211
2939725320,18sleep20-21swapper/223:19:132
2562125311,26sleep325643-21/usr/sbin/munin21:24:173
229992536,36sleep30-21swapper/321:15:083
1754925314,17sleep10-21swapper/120:59:271
761225218,17sleep20-21swapper/220:29:082
442425214,22sleep24418-21apt-get20:19:032
1949725212,24sleep219503-21diskstats22:49:082
193912526,10sleep10-21swapper/119:23:011
193912526,10sleep10-21swapper/119:23:011
1722325215,18sleep30-21swapper/300:24:113
1783825112,24sleep317857-21apt-get22:44:003
1783825112,24sleep317857-21apt-get22:44:003
1657325113,23sleep216585-21munin-run00:23:492
1511525114,20sleep20-21swapper/219:09:182
1511525114,20sleep20-21swapper/219:09:182
834725018,17sleep20-21swapper/222:14:072
834725018,17sleep20-21swapper/222:14:072
2960325014,19sleep30-21swapper/321:38:513
283425010,15sleep10-21swapper/120:14:051
2447325020,16sleep10-21swapper/123:04:461
516124910,21sleep30-21swapper/322:04:033
516124910,21sleep30-21swapper/322:04:033
1183224811,21sleep311858-21munin-run00:08:483
1939124711,20sleep219397-21ntp_states21:04:292
1011724714,16sleep10-21swapper/122:19:111
1011724714,16sleep10-21swapper/122:19:111
283024611,20sleep22827-21apt-get20:14:022
1374424513,16sleep20-21swapper/220:49:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional