You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 07:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Sep 03, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34502354510,3505sleep231674-1kworker/2:2H21:43:512
11132350117,3473sleep132155-21kworker/1:121:35:221
23092171012,1684sleep11150-21kworker/1:219:57:221
2487528998,837sleep224916-21ntpq00:34:132
2487528998,837sleep224916-21ntpq00:34:132
3066128057,760sleep230697-21ntpq19:44:072
1244227536,735sleep212486-21ntpq20:29:032
2304626937,673sleep223092-21ntpq00:29:042
26037251599,398sleep226059-21kworker/u8:421:13:402
57152484105,351sleep25870-21unixbench_multi21:49:162
20297243713,410sleep120305-21ntpq20:54:051
20297243713,410sleep120305-21ntpq20:54:051
494524056,350sleep2367-21snmpd23:29:592
2502423367,295sleep2367-21snmpd19:28:122
261912320149,57sleep20-21swapper/222:54:112
172002306108,175sleep217225-21awk22:28:392
172002306108,175sleep217225-21awk22:28:392
2298422958,210sleep222389-21/usr/sbin/munin22:44:102
130762282103,164sleep213092-21cat23:58:412
3053022657,247sleep0367-21snmpd21:26:130
2411225514,200sleep22443-21munin-run19:58:402
15557224852,79sleep215602-21awk20:39:042
15557224852,79sleep215602-21awk20:39:042
21691223251,144sleep221752-21latency20:59:022
10196223043,172sleep210169-21/usr/sbin/munin23:48:562
10196223043,172sleep210169-21/usr/sbin/munin23:48:562
59192229106,20sleep20-21swapper/220:08:572
21936222918,191sleep221975-21run-parts19:18:402
13165222649,31sleep212778-21/usr/sbin/munin22:14:022
13165222649,31sleep212778-21/usr/sbin/munin22:14:022
18047222215,55sleep218132-21latency_hist20:48:412
21127221885,83sleep221129-21cpu00:23:552
19604220816,174sleep219655-21latency_hist20:53:422
14365220510,182sleep114372-21cat22:18:491
1069821969,168sleep20-21swapper/220:23:592
15154218343,38sleep20-21swapper/200:04:062
15154218343,38sleep20-21swapper/200:04:062
23874218213,154sleep323883-21/usr/sbin/munin21:04:123
30199218115,140sleep20-21swapper/223:09:002
20836217256,58sleep220856-21awk00:23:392
16192216715,135sleep116155-21latency_hist00:08:411
16192216715,135sleep116155-21latency_hist00:08:411
2994521658,145sleep230012-21systemctl21:24:092
2187216214,53sleep20-21swapper/223:23:412
957421606,15sleep20-21swapper/220:19:102
1940021518,128sleep219526-21cat19:09:112
31921445,12sleep20-21swapper/219:49:562
2748721429,121sleep227530-21ntp_offset22:59:032
5593213313,106sleep25637-21latency_hist20:08:412
19352213018,89sleep119503-21cat00:18:421
25552212912,100sleep325076-21apt-get19:28:463
14208212716,38sleep014291-21find20:34:100
14208212716,38sleep014291-21find20:34:100
6339212631,71sleep00-21swapper/021:53:420
399621247,34sleep24045-21unin-run20:03:402
3444212419,84sleep12961-21apt-get21:43:501
25723212421,86sleep025729-21modprobe00:38:390
25723212421,86sleep025729-21modprobe00:38:390
7466212248,22sleep20-21swapper/223:39:032
2916821228,55sleep229174-21ntp_states19:39:062
11017212013,95sleep211066-21grep22:08:412
6779211814,87sleep26783-21cron23:38:392
7470211518,81sleep37565-21perl20:13:593
26998211510,58sleep20-21swapper/221:14:122
1230021155,12sleep20-21swapper/222:10:512
3089211417,73sleep12701-21/usr/sbin/munin19:59:031
32219211218,20sleep30-21swapper/323:14:053
21669211223,50sleep20-21swapper/222:39:132
18046211222,71sleep00-21swapper/022:29:380
18046211222,71sleep00-21swapper/022:29:380
24472211117,74sleep10-21swapper/121:08:421
2910721106,13sleep00-21swapper/021:22:500
2463221108,49sleep224751-21sed22:49:102
1457921109,89sleep20-21swapper/220:34:532
1457921109,89sleep20-21swapper/220:34:532
31737210913,39sleep20-21swapper/223:13:522
2392121095,11sleep20-21swapper/221:04:352
8690210722,50sleep20-21swapper/223:43:552
17763210614,53sleep218518-21cyclictest22:29:202
17763210614,53sleep218518-21cyclictest22:29:202
873121059,51sleep20-21swapper/220:18:422
8363210522,46sleep20-21swapper/221:59:012
6621210522,49sleep20-21swapper/221:53:562
17202210514,72sleep00-21swapper/019:03:570
17202210514,72sleep00-21swapper/019:03:570
2847621048,44sleep228475-21munin-run19:38:422
25593210420,44sleep20-21swapper/219:28:572
2207521039,17sleep20-21swapper/219:18:442
17136210315,70sleep117185-21grep20:44:041
1451221027,43sleep214575-21fschecks_time22:18:592
828210116,59sleep0972-21unixbench_singl21:34:120
24263210113,66sleep123777-21/usr/sbin/munin19:24:081
2791321009,39sleep20-21swapper/221:18:562
1768210018,44sleep20-21swapper/221:38:582
729229915,66sleep17429-21unixbench_singl21:54:111
298022995,27sleep20-21swapper/223:07:222
167032998,79sleep216770-21latency00:09:022
136829921,58sleep31429-21sendmail-msp21:38:393
299732989,15sleep00-21swapper/019:43:400
177329824,51sleep31838-21fschecks_time21:38:593
1429429714,63sleep114422-21/usr/sbin/munin20:34:111
1429429714,63sleep114422-21/usr/sbin/munin20:34:111
1299529714,58sleep013013-40ogrotate23:58:390
184282967,35sleep20-21swapper/200:14:072
1306229619,63sleep113033-40mandb23:58:401
1591329527,37sleep115932-21apt-get22:23:501
1591329527,37sleep115932-21apt-get22:23:501
2193229312,21sleep322030-21users00:24:133
1975929214,66sleep119821-21if_err_eth000:18:581
2377629112,61sleep023784-21/usr/sbin/munin19:23:500
137929117,18sleep30-21swapper/323:19:103
223412906,12sleep10-21swapper/119:18:591
187242908,15sleep10-21swapper/119:08:541
3228628817,52sleep032352-21switchtime23:14:060
1561028817,52sleep315752-21head00:04:153
1561028817,52sleep315752-21head00:04:153
327228713,19sleep30-21swapper/319:59:093
3236128615,53sleep332295-21latency_hist21:33:433
255672868,44sleep00-21swapper/019:28:510
1929228653,17sleep20-21swapper/222:33:562
152542866,13sleep00-21swapper/000:04:080
152542866,13sleep00-21swapper/000:04:080
1309528518,51sleep313235-21cat23:58:423
2446328414,43sleep124457-21apt-get00:33:551
2446328414,43sleep124457-21apt-get00:33:551
1991928317,50sleep019925-21diskstats20:53:560
1991928317,50sleep019925-21diskstats20:53:560
544228114,50sleep35443-21sshd20:07:213
2839728121,38sleep028411-21modprobe19:38:400
1172528111,59sleep00-21swapper/020:28:000
2921828018,47sleep329311-21cat23:04:093
1965828014,50sleep319670-21/usr/sbin/munin00:18:563
429027918,31sleep00-21swapper/020:03:590
2527127915,49sleep125312-21latency_hist22:53:411
1587127922,40sleep116029-21unixbench_multi20:39:121
1587127922,40sleep116029-21unixbench_multi20:39:121
368327815,50sleep33690-21modprobe23:28:393
3100727810,17sleep20-21swapper/221:28:532
2922127818,22sleep30-21swapper/321:23:423
398527717,41sleep03445-21/usr/sbin/munin21:44:250
137772778,14sleep20-21swapper/223:59:052
98832765,58sleep00-21swapper/023:48:250
873927620,25sleep10-21swapper/123:43:581
445027617,43sleep04509-21/usr/sbin/munin23:29:080
254252767,15sleep10-21swapper/119:28:441
339627519,45sleep20-21swapper/223:24:582
216427522,35sleep02225-21wc21:39:090
3037927418,25sleep10-21swapper/119:44:021
2880127419,33sleep328866-21fschecks_time19:38:593
195442746,11sleep00-21swapper/020:53:310
1328527414,28sleep30-21swapper/322:14:073
1328527414,28sleep30-21swapper/322:14:073
1022927415,43sleep010274-21sh20:23:400
791327321,23sleep30-21swapper/321:58:413
2003527314,40sleep01-21systemd19:13:410
1839627321,32sleep318424-21munin-run19:08:403
1839627321,32sleep318424-21munin-run19:08:403
1463827316,42sleep114704-21latency_hist00:03:401
110327318,10sleep20-21swapper/221:35:152
3077327219,28sleep00-21swapper/023:09:140
3028327219,25sleep10-21swapper/123:09:031
2433927217,31sleep00-21swapper/021:08:410
2276327216,39sleep122914-21latency_hist21:03:421
2034627217,37sleep320436-21grep20:54:073
2034627217,37sleep320436-21grep20:54:073
1981027233,27sleep20-21swapper/220:53:462
1981027233,27sleep20-21swapper/220:53:462
1192327222,25sleep30-21swapper/322:09:153
2267627115,15sleep10-21swapper/100:28:551
1774227116,39sleep117851-21latency_hist00:13:411
1723227123,28sleep317270-21munin-run22:28:403
1723227123,28sleep317270-21munin-run22:28:403
942027011,14sleep10-21swapper/122:03:401
83927016,26sleep10-21swapper/119:53:421
320627017,26sleep00-21swapper/023:24:120
2597727013,19sleep30-21swapper/322:54:083
2592527017,23sleep30-21swapper/300:38:423
2592527017,23sleep30-21swapper/300:38:423
2283027021,36sleep022923-21cat19:19:100
1983527014,38sleep019903-21/usr/sbin/munin22:34:110
1966027022,27sleep30-21swapper/322:34:053
149742707,15sleep00-21swapper/022:19:090
610426913,27sleep10-21swapper/120:09:001
2766926917,22sleep30-21swapper/321:18:413
248852696,51sleep00-21swapper/022:49:200
2259826920,32sleep122389-21/usr/sbin/munin22:44:001
2095326915,37sleep120965-21/usr/sbin/munin22:39:001
1965726921,24sleep30-21swapper/320:53:433
1806826924,25sleep30-21swapper/300:14:003
123926918,26sleep30-21swapper/319:54:013
3236426814,18sleep00-21swapper/019:49:090
2150826826,19sleep30-21swapper/320:58:593
1494926820,26sleep30-21swapper/320:38:413
1494926820,26sleep30-21swapper/320:38:413
1337326819,33sleep013411-21latency_hist20:33:420
2720826718,26sleep10-21swapper/119:33:571
2265126717,23sleep30-21swapper/319:19:053
3150126623,20sleep30-21swapper/323:13:413
308526621,27sleep13271-21apt-config21:43:421
1659226620,26sleep00-21swapper/000:09:000
1653526620,21sleep30-21swapper/320:43:423
1653526620,21sleep30-21swapper/320:43:423
1215126624,20sleep10-21swapper/120:28:581
979526518,22sleep00-21swapper/022:03:590
718726512,39sleep07290-21grep21:54:100
2982326521,24sleep10-21swapper/121:24:041
235272656,16sleep00-21swapper/022:46:460
1651926520,21sleep30-21swapper/300:08:593
1182826516,35sleep111940-21latency_hist20:28:421
93752645,11sleep00-21swapper/022:03:060
678526416,34sleep36858-21unin-run23:38:403
2517926423,21sleep30-21swapper/321:09:093
1728026416,28sleep017413-21apt-config22:28:410
1728026416,28sleep017413-21apt-config22:28:410
3125526318,26sleep131278-21irqstats21:29:001
231622636,13sleep30-21swapper/300:29:093
1145226312,25sleep10-21swapper/122:09:051
80842625,18sleep00-21swapper/023:40:100
206652626,16sleep00-21swapper/000:21:270
2039826214,27sleep00-21swapper/019:13:450
475726119,22sleep10-21swapper/120:04:121
2759726119,27sleep327610-21/usr/sbin/munin19:34:073
2518126114,31sleep325205-21munin-run19:28:423
2188526115,30sleep021917-21/usr/sbin/munin20:59:090
1904926116,28sleep319172-21wc22:33:413
1904926116,28sleep319172-21wc22:33:413
1799626115,30sleep018018-21munin-run20:48:390
997826019,22sleep30-21swapper/322:04:023
760426020,23sleep37695-21mailstats23:39:083
550026022,18sleep30-21swapper/321:49:133
3023826017,22sleep00-21swapper/019:43:530
2405526017,20sleep30-21swapper/322:48:593
2359226019,19sleep10-21swapper/121:04:101
1937626016,23sleep10-21swapper/120:51:231
111992605,10sleep30-21swapper/323:50:263
111992605,10sleep30-21swapper/323:50:263
865025918,21sleep30-21swapper/321:59:103
66125911,34sleep3748-21rm19:53:403
373625921,20sleep10-21swapper/123:28:411
349325915,25sleep33464-21apt-get21:44:083
2064325920,19sleep10-21swapper/119:13:501
1771625912,25sleep10-21swapper/122:28:511
1771625912,25sleep10-21swapper/122:28:511
1417825915,19sleep10-21swapper/122:18:411
1417825915,19sleep10-21swapper/122:18:411
1136125918,20sleep00-21swapper/022:09:040
709525819,21sleep00-21swapper/020:13:390
1721425813,17sleep10-21swapper/119:03:581
1721425813,17sleep10-21swapper/119:03:581
1479025815,19sleep30-21swapper/322:19:043
56025718,24sleep1630-21systemd-run23:18:401
3149225724,17sleep00-21swapper/021:29:080
3001325714,21sleep30-21swapper/321:24:103
2917125720,22sleep129194-21munin-run21:23:401
2861925712,17sleep10-21swapper/123:03:501
2657625710,19sleep10-21swapper/121:14:031
2268525714,22sleep30-21swapper/322:44:023
202052577,11sleep10-21swapper/122:34:161
115525716,23sleep00-21swapper/019:53:580
373625611,33sleep00-21swapper/020:00:190
2121325610,26sleep10-21swapper/100:23:591
1019325613,26sleep010187-21apt-get23:48:540
1019325613,26sleep010187-21apt-get23:48:540
707225511,25sleep10-21swapper/123:38:501
620925515,21sleep30-21swapper/320:09:023
443525518,19sleep30-21swapper/323:29:073
2113125513,26sleep321144-21/usr/sbin/munin19:14:093
1382825516,16sleep00-21swapper/023:59:070
1027525514,19sleep10-21swapper/120:23:411
686025411,27sleep00-21swapper/023:38:410
1722125417,19sleep20-21swapper/219:03:592
1722125417,19sleep20-21swapper/219:03:592
1332425415,25sleep313357-21/usr/sbin/munin20:33:403
1280225413,24sleep012796-21apt-get22:13:520
1280225413,24sleep012796-21apt-get22:13:520
1177625413,23sleep311808-21munin-run20:28:403
922025318,21sleep39316-21sed23:44:093
88125312,24sleep1874-21apt-get23:18:561
1918725317,19sleep30-21swapper/319:09:083
1882825313,24sleep018895-21users00:14:130
738025213,24sleep07376-21apt-get20:13:520
530325214,19sleep10-21swapper/121:49:071
2931725213,20sleep00-21swapper/023:04:100
2840825219,15sleep00-21swapper/023:03:410
267222527,33sleep10-21swapper/122:58:011
896925110,23sleep39037-21df20:18:553
88025114,19sleep00-21swapper/023:18:550
397225112,22sleep13968-21apt-get23:28:531
2217625114,15sleep30-21swapper/322:43:423
1518725113,24sleep015181-21apt-get20:38:540
1518725113,24sleep015181-21apt-get20:38:540
664324912,26sleep1367-21snmpd23:36:521
2771324915,16sleep00-21swapper/021:18:420
2550824913,18sleep00-21swapper/022:53:520
208672499,20sleep00-21swapper/022:38:550
3034224814,17sleep30-21swapper/319:44:003
2920024812,15sleep10-21swapper/119:39:091
270572485,30sleep10-21swapper/122:58:461
1028024712,22sleep110353-21grep23:48:591
1028024712,22sleep110353-21grep23:48:591
2521624611,20sleep025238-21ntp_states21:09:100
2136524611,20sleep021102-21/usr/sbin/munin00:24:020
1673424611,19sleep316729-21apt-get20:43:543
872124415,15sleep10-21swapper/120:18:411
271652418,17sleep30-21swapper/322:58:563
98872388,12sleep10-21swapper/122:04:001
134682378,14sleep30-21swapper/323:58:583
231332368,14sleep00-21swapper/000:29:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional