You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 17:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Jan 10, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2909221049224,10435sleep327548-21kworker/3:210:02:393
2909221049224,10435sleep327548-21kworker/3:210:02:393
24312350615,3478sleep214395-21kworker/2:108:35:402
18037231575,3140sleep214395-21kworker/2:109:25:392
2209294712,925sleep115237-21kworker/u8:012:01:371
1171129075,891sleep37694-1kworker/3:1H07:25:243
2441126216,608sleep028896-1kworker/0:2H08:05:290
15302251330,24sleep20-21swapper/209:19:462
256312426111,241sleep325665-21munin-plugin-st08:09:443
1014323845,346sleep37694-1kworker/3:1H07:21:443
1014323845,346sleep37694-1kworker/3:1H07:21:443
248923575,340sleep37694-1kworker/3:1H10:21:293
2798823406,325sleep027708-1kworker/0:1H08:17:000
2143123267,306sleep321511-21grep11:20:123
31530231311,208sleep331545-21modprobe11:54:443
6083230322,263sleep26117-21/usr/sbin/munin12:15:072
518229913,199sleep3573-21fschecks_time10:15:033
31112226175,170sleep331153-21latency08:25:073
13717225072,93sleep30-21swapper/307:30:163
25876223423,200sleep330095-21snmpd09:51:333
25876223423,200sleep330095-21snmpd09:51:333
23104221411,130sleep323247-21latency_hist09:44:463
23104221411,130sleep323247-21latency_hist09:44:463
30033221218,181sleep229998-21latency_hist11:49:462
11983220311,172sleep012026-21awk10:50:090
11983220311,172sleep012026-21awk10:50:090
32601219929,18sleep30-21swapper/308:30:083
783621948,122sleep37883-21ntp_offset12:20:083
10542219214,161sleep110619-21cat07:22:071
10542219214,161sleep110619-21cat07:22:071
1254821869,165sleep312611-21cut12:35:193
5064218122,141sleep25076-21sh10:29:442
1390321816,92sleep30-21swapper/310:55:173
29053217510,128sleep328998-21latency_hist08:19:463
518521737,152sleep35268-21switchtime08:45:143
10291217312,149sleep310385-21latency_hist12:29:453
10291217312,149sleep310385-21latency_hist12:29:453
2987321677,103sleep330095-21snmpd11:48:223
1053221637,97sleep310583-21sort12:30:023
2028521607,72sleep30-21swapper/311:15:283
2028521607,72sleep30-21swapper/311:15:283
2876215419,118sleep02939-21sendmail_mailst10:21:550
1023621518,121sleep30-21swapper/309:00:413
1023621518,121sleep30-21swapper/309:00:413
819121495,12sleep20-21swapper/207:15:052
819121495,12sleep20-21swapper/207:15:052
2466421467,11sleep20-21swapper/208:06:572
26937214568,57sleep30-21swapper/311:39:463
22259214419,103sleep0471-21cron07:59:440
13386214011,11sleep00-21swapper/009:11:030
168921399,47sleep30-21swapper/312:01:063
14543213916,109sleep214600-21cat07:34:472
9416213842,25sleep30-21swapper/307:19:463
9416213842,25sleep30-21swapper/307:19:463
30884213641,79sleep331012-21dpkg10:09:463
24367212864,10sleep30-21swapper/309:47:053
11587212813,20sleep30-21swapper/310:50:003
11587212813,20sleep30-21swapper/310:50:003
10122212812,96sleep310155-21fschecks_count10:45:033
26128212716,25sleep30-21swapper/308:10:053
1674212626,16sleep30-21swapper/308:35:043
1532421247,77sleep315465-21latency_hist09:19:473
16053212217,83sleep30-21swapper/309:20:113
709421209,99sleep37135-21basename10:35:063
2760521197,12sleep30-21swapper/309:58:523
2760521197,12sleep30-21swapper/309:58:523
32215211811,85sleep30-21swapper/311:55:093
747921169,45sleep37605-21latency_hist08:54:473
6422211633,64sleep30-21swapper/312:15:163
20694211568,31sleep30-21swapper/307:54:343
3060521148,26sleep30-21swapper/308:24:473
5356211331,18sleep30-21swapper/310:30:013
21451211220,79sleep221473-21latency_hist09:39:452
21451211220,79sleep221473-21latency_hist09:39:452
18407211231,45sleep318489-21mailstats07:45:143
17617210930,66sleep31-21systemd07:44:453
3068210723,18sleep30-21swapper/308:40:013
16147210716,16sleep00-21swapper/009:20:120
22580210624,70sleep322570-21apt-get08:00:033
22580210624,70sleep322570-21apt-get08:00:033
22493210616,74sleep022625-21unixbench_singl09:40:150
22493210616,74sleep022625-21unixbench_singl09:40:150
1596621065,17sleep30-21swapper/311:04:373
12918210619,69sleep213067-21latency_hist10:54:472
12918210619,69sleep213067-21latency_hist10:54:472
9395210517,70sleep09503-21fschecks_time09:00:050
9395210517,70sleep09503-21fschecks_time09:00:050
3012210526,15sleep30-21swapper/312:05:063
18254210423,69sleep30-21swapper/309:28:273
8560210319,14sleep30-21swapper/310:40:033
2770121039,56sleep327150-21/usr/sbin/munin11:40:153
24227210325,66sleep324278-21cat11:30:053
538421027,84sleep30-21swapper/312:12:373
22314210214,67sleep222380-21/usr/sbin/munin09:40:132
22314210214,67sleep222380-21/usr/sbin/munin09:40:132
16439210015,67sleep016451-21fschecks_count07:40:020
356929914,55sleep20-21swapper/210:24:472
318602997,58sleep331903-21ntpq10:11:483
1478029821,15sleep30-21swapper/307:35:013
1944929717,68sleep019519-21fschecks_time11:15:040
1944929717,68sleep019519-21fschecks_time11:15:040
2392429616,66sleep024011-21sort09:45:160
2685729525,53sleep114386-21systemd-journal11:39:441
663329417,56sleep26671-21grep08:50:142
498929410,64sleep10-21swapper/107:05:111
2376529413,46sleep3286-21dbus-daemon11:29:453
217072949,16sleep30-21swapper/307:55:153
1942729313,59sleep019428-21/usr/sbin/munin07:49:550
1872329315,62sleep218732-21diskstats09:30:032
492829225,11sleep30-21swapper/310:27:593
2028029222,51sleep020324-21fschecks_count09:35:020
2028029222,51sleep020324-21fschecks_count09:35:020
1862029120,13sleep30-21swapper/309:29:563
96362907,11sleep20-21swapper/210:43:162
212092905,74sleep30-21swapper/309:36:393
212092905,74sleep30-21swapper/309:36:393
1542029012,62sleep215544-21timerwakeupswit07:35:162
710028923,30sleep30-21swapper/307:11:103
508128941,31sleep00-21swapper/007:05:140
3203928913,62sleep032131-21cat10:11:530
2767728920,32sleep10-21swapper/108:14:551
1911628918,45sleep219135-21latency_hist11:14:462
2462028816,59sleep224646-21latency_hist09:49:462
2775028712,62sleep027798-21latency_hist09:59:450
2775028712,62sleep027798-21latency_hist09:59:450
3084628620,29sleep10-21swapper/108:25:011
2617628620,46sleep226283-21latency_hist09:54:472
2617628620,46sleep226283-21latency_hist09:54:472
897828523,41sleep09002-21munin-plugin-st08:59:450
897828523,41sleep09002-21munin-plugin-st08:59:450
124628514,58sleep11283-21latency_hist08:34:461
35292848,20sleep00-21swapper/008:40:140
2066028418,50sleep120695-21ps09:35:111
2066028418,50sleep120695-21ps09:35:111
89528310,60sleep00-21swapper/010:15:120
238342837,32sleep00-21swapper/008:04:410
238342837,32sleep00-21swapper/008:04:410
1612028319,41sleep116248-21ls07:39:471
2780028214,51sleep127946-21sed09:59:471
1081528212,52sleep010937-21/usr/sbin/munin10:45:160
929728114,32sleep20-21swapper/208:59:592
929728114,32sleep20-21swapper/208:59:592
2738328121,29sleep20-21swapper/211:40:052
1626828117,41sleep216332-21cpuspeed11:05:012
2897728016,47sleep229052-21systemd-run08:19:452
1119228029,28sleep20-21swapper/207:24:462
1119228029,28sleep20-21swapper/207:24:462
2612527817,46sleep026154-21munin-run09:54:450
2612527817,46sleep026154-21munin-run09:54:450
1991427820,26sleep10-21swapper/107:50:121
919327718,43sleep19249-21/usr/sbin/munin12:25:071
919327718,43sleep19249-21/usr/sbin/munin12:25:071
2659827715,44sleep126728-21/usr/sbin/munin08:10:151
1773027719,26sleep10-21swapper/107:44:471
160102776,53sleep20-21swapper/209:20:102
2962927613,49sleep029676-21cut10:05:030
1606327618,30sleep00-21swapper/011:04:460
66327515,36sleep1398-21/usr/sbin/munin10:15:051
2070927518,29sleep10-21swapper/111:19:451
2070927518,29sleep10-21swapper/111:19:451
439727418,27sleep00-21swapper/010:25:140
857627325,29sleep10-21swapper/110:40:051
579727322,34sleep31-21systemd07:09:453
1439327324,31sleep214448-21awk10:59:442
323827212,36sleep12717-21/usr/sbin/munin12:05:121
2599927220,28sleep00-21swapper/008:10:020
2272827214,21sleep10-21swapper/108:00:101
2272827214,21sleep10-21swapper/108:00:101
1995727220,22sleep10-21swapper/109:34:471
1891527223,38sleep30-21swapper/311:12:413
1244927217,24sleep20-21swapper/212:35:142
1235027212,39sleep00-21swapper/012:35:120
119732720,8sleep00-21swapper/007:26:530
959327114,36sleep19357-21apt-get07:19:481
959327114,36sleep19357-21apt-get07:19:481
925027113,29sleep20-21swapper/212:25:082
925027113,29sleep20-21swapper/212:25:082
829027121,26sleep10-21swapper/108:55:131
829027121,26sleep10-21swapper/108:55:131
462227124,23sleep20-21swapper/207:05:042
1935827120,28sleep00-21swapper/009:30:180
1094727119,35sleep110984-21ntp_states12:30:121
910627016,36sleep09192-21fw_packets12:25:060
910627016,36sleep09192-21fw_packets12:25:060
84872705,27sleep30-21swapper/308:55:153
84872705,27sleep30-21swapper/308:55:153
593427014,41sleep06029-21fschecks_time12:15:040
2700227019,33sleep00-21swapper/011:39:470
188227016,25sleep00-21swapper/012:01:110
1583227018,33sleep115521-21/usr/sbin/munin09:20:051
98826912,35sleep2648-21apt-get11:59:482
867226915,23sleep00-21swapper/007:15:150
867226915,23sleep00-21swapper/007:15:150
699526912,13sleep10-21swapper/110:35:041
3152226915,35sleep2378-21sshd11:54:392
2950126911,27sleep20-21swapper/208:20:052
282726927,25sleep10-21swapper/108:39:471
2401226913,35sleep10-21swapper/109:45:171
190826913,16sleep00-21swapper/008:35:100
1746926917,27sleep10-21swapper/109:25:101
1694126915,36sleep117066-21sed11:05:161
1011326912,38sleep110199-21users09:00:181
1011326912,38sleep110199-21users09:00:181
2799526818,32sleep027989-21apt-get10:00:020
2799526818,32sleep027989-21apt-get10:00:020
2326126811,42sleep00-21swapper/011:25:140
172752685,30sleep30-21swapper/311:06:313
1199926824,13sleep30-21swapper/309:08:203
1167326818,33sleep111671-21apt-get07:24:571
115032688,11sleep00-21swapper/012:31:040
3227826723,23sleep20-21swapper/211:55:122
1758926626,16sleep00-21swapper/011:09:470
1142926616,22sleep20-21swapper/209:05:162
2937926520,29sleep029394-21cut08:20:030
2702826524,20sleep10-21swapper/109:55:141
1921726519,20sleep00-21swapper/007:49:470
134462656,27sleep30-21swapper/309:11:513
79926417,19sleep10-21swapper/111:59:471
2375426419,28sleep223764-21modprobe11:29:442
2131126418,25sleep20-21swapper/211:20:072
1241626423,21sleep10-21swapper/109:09:591
1217526413,39sleep012139-21latency_hist09:09:460
481126320,24sleep20-21swapper/208:45:052
3207826315,34sleep032097-21kernelversion11:55:060
2225626319,21sleep10-21swapper/111:24:461
187226319,23sleep20-21swapper/210:19:472
1350226311,19sleep00-21swapper/007:30:090
498026217,27sleep15007-21awk08:45:081
46522625,11sleep10-21swapper/112:10:091
431926218,30sleep24342-21idleruntime-cro08:44:452
260432625,44sleep10-21swapper/109:53:411
260432625,44sleep10-21swapper/109:53:411
2592326219,26sleep225990-21df08:10:002
1898926215,28sleep00-21swapper/011:13:390
114052628,24sleep20-21swapper/210:49:472
743326116,31sleep07470-21munin-node07:14:470
682926119,22sleep10-21swapper/107:10:151
433826110,37sleep24364-21df12:10:022
1485026118,21sleep00-21swapper/007:35:020
1080126114,19sleep20-21swapper/212:30:092
780226010,38sleep27811-21forks08:55:022
780226010,38sleep27811-21forks08:55:022
740726019,23sleep17418-21find12:19:541
389026019,24sleep10-21swapper/110:25:031
3071926015,20sleep10-21swapper/110:08:391
2927026014,25sleep129289-21apt-get08:19:561
2879326014,31sleep028799-21df_inode11:45:000
2808426020,24sleep10-21swapper/110:00:071
2808426020,24sleep10-21swapper/110:00:071
2559426016,26sleep00-21swapper/011:34:590
1320526012,18sleep00-21swapper/010:55:010
3132125916,20sleep10-21swapper/110:09:561
3093025917,20sleep00-21swapper/008:25:040
2898825914,20sleep10-21swapper/111:45:051
2614525913,26sleep226216-21ls11:35:132
252462598,18sleep00-21swapper/011:33:580
2404525916,23sleep20-21swapper/211:30:002
1497325919,20sleep10-21swapper/111:00:071
1479725918,18sleep20-21swapper/211:00:042
1216425916,26sleep112259-21/usr/sbin/munin10:50:141
1216425916,26sleep112259-21/usr/sbin/munin10:50:141
625725818,21sleep10-21swapper/108:50:051
3239025816,25sleep232387-21apt-get08:29:592
2396925820,22sleep20-21swapper/208:04:472
2396925820,22sleep20-21swapper/208:04:472
2302725818,16sleep10-21swapper/109:44:441
2302725818,16sleep10-21swapper/109:44:441
2233725812,25sleep20-21swapper/207:59:462
1368725814,18sleep10-21swapper/107:30:141
1268525819,26sleep112765-21timerandwakeup12:35:211
732725714,27sleep07364-21ntp_states10:35:110
667825711,29sleep06828-21sed10:34:470
2404425712,26sleep124038-21apt-get11:29:591
2099725717,25sleep120992-21apt-get07:54:571
2094525715,26sleep020940-21apt-get11:19:570
1209225720,20sleep20-21swapper/207:26:562
3183125615,24sleep20-21swapper/210:11:472
317325613,24sleep23192-21ntp_states12:05:092
3053825617,25sleep130552-21uname11:50:061
235842565,11sleep10-21swapper/111:27:331
583925515,24sleep15834-21apt-get12:14:571
3084225514,25sleep230838-21apt-get08:24:582
3005725517,22sleep230150-21mailstats10:05:132
1989825515,21sleep20-21swapper/211:15:142
1989825515,21sleep20-21swapper/211:15:142
1523125516,25sleep115301-21systemd-run09:19:451
461425414,24sleep04607-21apt-get08:44:590
2641425423,18sleep20-21swapper/209:54:592
2641425423,18sleep20-21swapper/209:54:592
1829825414,19sleep10-21swapper/111:10:131
1134825418,19sleep10-21swapper/110:49:461
307125317,16sleep10-21swapper/108:40:031
283725318,18sleep00-21swapper/012:05:030
1423325316,22sleep00-21swapper/009:15:070
1367025314,22sleep21-21systemd09:14:452
790125216,16sleep20-21swapper/212:20:112
697525213,22sleep27021-21users07:10:162
535425212,19sleep10-21swapper/110:29:591
429225213,23sleep04285-21apt-get12:10:000
2925625211,24sleep229288-21munin-run10:04:442
2925625211,24sleep229288-21munin-run10:04:442
2484625213,23sleep124885-21memory08:07:001
1084225210,25sleep010940-21users07:22:100
1084225210,25sleep010940-21users07:22:100
909725113,19sleep00-21swapper/010:40:160
743325112,22sleep07428-21apt-get12:19:570
616125114,22sleep06164-21apt-get08:49:550
532925113,17sleep20-21swapper/210:29:562
38025114,24sleep0391-21ntp_states08:30:130
2785525111,17sleep10-21swapper/111:40:171
1707625114,22sleep017095-21apt-get09:24:570
142172517,32sleep10-21swapper/110:57:291
1106425112,17sleep10-21swapper/109:05:071
1635624919,18sleep10-21swapper/107:39:591
3026824810,21sleep030264-21apt-get11:49:570
1907424815,15sleep20-21swapper/207:48:402
280924713,19sleep10-21swapper/110:21:521
765424614,16sleep20-21swapper/210:35:172
2071324613,16sleep20-21swapper/207:54:462
164142376,14sleep20-21swapper/207:40:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional