You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 07:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jan 09, 2026 00:44:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1510521046322,10420sleep312177-21kworker/u8:220:31:013
1510521046322,10420sleep312177-21kworker/u8:220:31:013
21198217576,1694sleep26259-1kworker/2:2H22:35:032
25872114410,1121sleep22588-21sshd19:46:412
25872114410,1121sleep22588-21sshd19:46:412
2245728949,876sleep321942-1kworker/3:0H22:39:493
539828499,832sleep027383-1kworker/0:2H21:40:130
306327787,761sleep115576-1kworker/1:2H21:32:021
6616275314,723sleep06661-21/usr/sbin/munin20:00:120
6616275314,723sleep06661-21/usr/sbin/munin20:00:120
1275526436,627sleep012197-1kworker/0:0H20:20:270
1275526436,627sleep012197-1kworker/0:0H20:20:270
1664525208,502sleep012197-1kworker/0:0H20:32:560
16901241947,190sleep216992-21fschecks_time00:05:082
2676923260,51sleep20-21swapper/221:05:122
124182309113,171sleep212546-21latency_hist20:19:532
124182309113,171sleep212546-21latency_hist20:19:532
27510224828,208sleep227535-21grep22:55:072
20758222666,55sleep20-21swapper/220:45:222
20758222666,55sleep20-21swapper/220:45:222
25156222539,170sleep224647-21/usr/sbin/munin19:15:192
26074220926,127sleep226133-21if_eth022:50:102
26074220926,127sleep226133-21if_eth022:50:102
32654220857,88sleep224233-21munin-node21:25:002
5812220412,153sleep25948-21latency_hist21:44:512
19871220118,166sleep119903-21/usr/sbin/munin22:30:181
24023218151,76sleep224071-21latency_hist19:14:512
24023218151,76sleep224071-21latency_hist19:14:512
11119217913,106sleep20-21swapper/222:00:142
19984217814,151sleep320080-21sed22:30:203
4209217715,137sleep34282-21/usr/sbin/munin21:35:483
17571217043,86sleep217582-21modprobe22:24:502
1560121687,152sleep115576-1kworker/1:2H22:16:331
29273216432,113sleep228978-21/usr/sbin/munin23:00:112
27606216411,139sleep227752-21latency_hist19:24:522
22809216311,125sleep20-21swapper/200:24:512
30894215515,116sleep20-21swapper/221:19:512
27408215130,66sleep20-21swapper/200:39:492
32444214914,77sleep232589-21latency_hist21:24:522
9436214820,84sleep20-21swapper/220:10:082
14030214726,63sleep20-21swapper/222:09:552
4839214627,74sleep20-21swapper/219:55:092
1169421467,10sleep30-21swapper/323:48:093
1169421467,10sleep30-21swapper/323:48:093
12883214421,104sleep312972-21sed22:05:203
29531214313,60sleep129587-21if_err_eth019:30:081
7223213967,48sleep20-21swapper/223:34:522
21174213917,44sleep122969-21systemd-journal00:19:491
21174213917,44sleep122969-21systemd-journal00:19:491
29138213431,66sleep229287-21sed19:29:532
32740213314,47sleep20-21swapper/219:40:112
15556213215,100sleep215691-21/usr/sbin/munin20:31:082
3110421307,52sleep20-21swapper/219:35:102
256721287,86sleep22573-21ntpq21:31:222
32158212425,75sleep132160-21cron19:39:501
24966212417,64sleep225062-21cut21:00:082
31896212349,27sleep230095-21snmpd23:09:522
31896212349,27sleep230095-21snmpd23:09:522
8070212114,94sleep18133-21cut21:50:201
30816212022,78sleep030832-21sh21:19:490
15293212053,50sleep215426-21df20:31:042
31300211718,77sleep131442-21unixbench_singl23:05:231
31300211718,77sleep131442-21unixbench_singl23:05:231
25199211647,45sleep20-21swapper/200:30:202
3210221159,69sleep232104-21apt-get23:10:012
21245211246,54sleep221278-21date00:19:502
21245211246,54sleep221278-21date00:19:502
26236211112,79sleep126294-21fschecks_time00:35:081
118721118,44sleep21206-21apt-get23:15:012
3736211020,73sleep33838-21timerandwakeup19:50:223
1336821107,65sleep213314-21apt-get20:25:512
1336821107,65sleep213314-21apt-get20:25:512
2802621087,13sleep00-21swapper/019:25:110
27042210818,25sleep10-21swapper/121:05:201
10331210833,64sleep210468-21latency_hist23:44:532
15623210754,39sleep215660-21ntpq00:00:162
15623210754,39sleep215660-21ntpq00:00:162
27564210511,77sleep127605-21latency_hist19:24:511
18693210537,44sleep20-21swapper/222:25:222
15948210514,71sleep116100-21unixbench_multi00:00:231
15948210514,71sleep116100-21unixbench_multi00:00:231
1944721046,13sleep00-21swapper/000:12:410
11828210428,62sleep211876-21unin-run23:49:502
11828210428,62sleep211876-21unin-run23:49:502
2018210317,70sleep01508-21apt-get21:30:030
1965210315,26sleep10-21swapper/119:45:151
1965210315,26sleep10-21swapper/119:45:151
2040421020,9sleep00-21swapper/000:15:190
2040421020,9sleep00-21swapper/000:15:190
19113210151,36sleep20-21swapper/220:40:172
19113210151,36sleep20-21swapper/220:40:172
11826210110,80sleep1471-21cron23:49:491
11826210110,80sleep1471-21cron23:49:491
12638210015,67sleep012651-21needreboot22:05:130
828829923,58sleep08450-21sed21:50:230
663129926,18sleep20-21swapper/221:45:202
2538129715,53sleep00-21swapper/021:00:180
325429616,64sleep33289-21proc_pri23:20:153
2401229629,11sleep20-21swapper/200:26:042
229442965,27sleep20-21swapper/220:53:502
914929548,32sleep20-21swapper/221:55:052
733929510,68sleep00-21swapper/021:49:500
411629532,45sleep20-21swapper/221:35:472
651529411,17sleep20-21swapper/220:00:102
651529411,17sleep20-21swapper/220:00:102
1880629418,14sleep20-21swapper/200:10:172
1518729419,30sleep10-21swapper/120:31:031
2850329339,39sleep20-21swapper/221:10:182
2464529354,20sleep20-21swapper/220:59:522
32929218,61sleep3474-21awk23:10:223
2841029216,55sleep128518-21grep19:25:201
2524529213,66sleep224233-21munin-node22:45:262
2524529213,66sleep224233-21munin-node22:45:262
1405429214,62sleep014057-21apt-get22:10:010
1337929229,11sleep20-21swapper/222:07:492
1238229227,36sleep30-21swapper/323:50:113
109882927,69sleep311038-21fschecks_count20:15:093
109882927,69sleep311038-21fschecks_count20:15:093
41942915,28sleep20-21swapper/219:52:272
1080929125,41sleep00-21swapper/022:00:080
441129019,12sleep20-21swapper/223:25:072
2399629026,11sleep20-21swapper/222:44:162
1127029015,60sleep011301-21/usr/sbin/munin20:15:140
1127029015,60sleep011301-21/usr/sbin/munin20:15:140
2192928916,17sleep30-21swapper/319:05:203
1462028915,56sleep113329-21apt-get20:26:391
1462028915,56sleep113329-21apt-get20:26:391
593228812,60sleep36001-21systemd-run19:59:513
2358628812,62sleep023709-21timerwakeupswit22:42:000
1940728717,12sleep20-21swapper/222:30:022
1366128719,13sleep20-21swapper/223:55:022
1341728710,17sleep20-21swapper/223:54:502
1198928735,31sleep10-21swapper/122:04:491
731128527,34sleep30-21swapper/321:49:493
40912859,63sleep24102-21munin-run23:24:502
3115728517,53sleep330874-21/usr/sbin/munin19:35:113
183372858,21sleep218340-21cron20:39:492
183372858,21sleep218340-21cron20:39:492
2594028419,45sleep025977-21systemctl19:19:500
2378628441,22sleep00-21swapper/000:25:210
1586028313,25sleep00-21swapper/000:00:220
1586028313,25sleep00-21swapper/000:00:220
1232628322,31sleep10-21swapper/120:19:521
1232628322,31sleep10-21swapper/120:19:521
81342819,12sleep20-21swapper/221:50:212
483228123,34sleep14856-21munin-run21:39:491
2620128114,50sleep130095-21snmpd21:04:521
91102809,10sleep00-21swapper/023:40:110
635328013,52sleep06367-21awk23:30:170
51822807,14sleep10-21swapper/121:39:531
262952809,18sleep30-21swapper/322:50:153
262952809,18sleep30-21swapper/322:50:153
2468528024,31sleep30-21swapper/300:30:093
2409027924,25sleep10-21swapper/122:44:521
2988827826,36sleep20-21swapper/219:30:182
2530727811,55sleep030095-21snmpd22:46:120
2530727811,55sleep030095-21snmpd22:46:120
2466227819,30sleep30-21swapper/320:59:533
2981427714,41sleep329520-21/usr/sbin/munin21:15:123
1881727721,28sleep30-21swapper/300:10:183
2377427617,45sleep123829-21processes20:55:191
2295027643,21sleep022958-21fschecks_time22:41:480
122732769,48sleep10-21swapper/122:05:021
90042755,17sleep20-21swapper/220:09:032
90042755,17sleep20-21swapper/220:09:032
736327522,34sleep115576-1kworker/1:2H21:49:511
340027519,28sleep30-21swapper/321:35:073
443327415,44sleep14515-21latency_hist19:54:521
264872740,8sleep00-21swapper/019:20:090
2430827420,34sleep324336-21munin-run00:29:493
2279727416,43sleep322808-21awk19:10:083
2279727416,43sleep322808-21awk19:10:083
758627315,42sleep07650-21/usr/sbin/munin23:35:100
2748427324,23sleep30-21swapper/300:39:503
2623027317,24sleep00-21swapper/000:35:070
2444627219,35sleep124531-21cat19:14:581
211427217,28sleep00-21swapper/023:15:240
2094827224,22sleep00-21swapper/020:45:240
2094827224,22sleep00-21swapper/020:45:240
135927213,38sleep31421-21/usr/sbin/munin23:15:103
2319827122,26sleep10-21swapper/119:10:181
2319827122,26sleep10-21swapper/119:10:181
1783727115,24sleep10-21swapper/122:25:011
1773127120,28sleep30-21swapper/320:35:583
1773127120,28sleep30-21swapper/320:35:583
1699527112,43sleep317053-21if_eth000:05:093
153327116,41sleep01584-21apt-key21:29:500
780627020,29sleep00-21swapper/020:05:050
780627020,29sleep00-21swapper/020:05:050
322427013,39sleep13246-21ntp_states23:20:131
2511027013,41sleep125198-21mailstats00:30:191
1465227028,22sleep30-21swapper/322:15:093
1434027010,45sleep014458-21sed23:55:210
410526921,26sleep00-21swapper/023:24:520
1876126917,38sleep10-21swapper/100:10:161
158526922,29sleep11718-21latency_hist21:29:511
2506826821,28sleep00-21swapper/000:30:180
2490826814,26sleep30-21swapper/319:15:123
2139426828,20sleep20-21swapper/219:05:112
210842689,45sleep121108-21who20:45:251
210842689,45sleep121108-21who20:45:251
1514426814,39sleep015177-21munin-run20:31:020
3037426724,19sleep30-21swapper/323:04:533
2802426716,21sleep10-21swapper/122:55:181
2210426718,25sleep30-21swapper/300:20:203
178626713,39sleep11870-21cat23:15:201
159402675,31sleep20-21swapper/222:19:042
124222670,8sleep00-21swapper/023:50:110
1204026717,34sleep312123-21wc22:04:513
1052226710,48sleep00-21swapper/020:13:410
1048726719,23sleep00-21swapper/021:59:520
670626622,24sleep30-21swapper/320:00:153
670626622,24sleep30-21swapper/320:00:153
506926619,23sleep00-21swapper/021:39:520
202826610,13sleep30-21swapper/319:45:173
202826610,13sleep30-21swapper/319:45:173
1922126615,23sleep00-21swapper/022:29:520
818626520,32sleep18220-21ntp_kernel_pll_20:05:141
818626520,32sleep18220-21ntp_kernel_pll_20:05:141
77102659,15sleep20-21swapper/223:35:122
3255726522,21sleep30-21swapper/319:40:083
2940726512,21sleep10-21swapper/123:00:141
2754626514,38sleep027548-21sshd21:07:540
2281026519,21sleep00-21swapper/019:10:090
2281026519,21sleep00-21swapper/019:10:090
1607726516,19sleep10-21swapper/122:19:521
1248426513,23sleep10-21swapper/123:50:131
101322658,37sleep30-21swapper/320:10:213
2793126411,40sleep028023-21mailstats22:55:170
2501026415,26sleep30-21swapper/322:45:223
2501026415,26sleep30-21swapper/322:45:223
2126526425,23sleep00-21swapper/019:05:090
1691526417,23sleep10-21swapper/120:35:361
3158626314,33sleep331592-21ntp_states21:20:163
2836626314,21sleep10-21swapper/121:10:131
2119326315,31sleep121190-21apt-get22:35:021
1630726315,30sleep30-21swapper/320:31:463
1922726219,19sleep00-21swapper/020:40:190
1922726219,19sleep00-21swapper/020:40:190
1412626219,26sleep10-21swapper/123:55:181
788826113,34sleep37901-21fschecks_count20:05:083
788826113,34sleep37901-21fschecks_count20:05:083
2440726114,32sleep124305-21/usr/sbin/munin22:45:081
2440726114,32sleep124305-21/usr/sbin/munin22:45:081
2197426113,33sleep122035-21/usr/sbin/munin00:20:181
1578826117,28sleep30-21swapper/300:00:213
1578826117,28sleep30-21swapper/300:00:213
797526010,35sleep18046-21sensors_temp23:35:191
493126020,20sleep30-21swapper/323:25:203
2796826018,25sleep30-21swapper/319:25:103
2769826019,18sleep30-21swapper/321:09:503
2625726013,21sleep30-21swapper/319:20:053
2534326011,26sleep30-21swapper/321:00:173
203502609,40sleep030095-21snmpd22:30:450
142462608,12sleep30-21swapper/323:55:203
965525922,23sleep39774-21timerwakeupswit23:40:223
537725919,20sleep10-21swapper/119:55:211
2218125916,21sleep00-21swapper/000:20:210
219322595,12sleep10-21swapper/120:50:101
176012596,12sleep10-21swapper/120:35:561
176012596,12sleep10-21swapper/120:35:561
979425713,27sleep19805-21ntp_states20:10:151
813125720,17sleep30-21swapper/323:35:213
2585425714,26sleep125873-21apt-get22:50:011
2585425714,26sleep125873-21apt-get22:50:011
2035025720,22sleep30-21swapper/320:45:113
2035025720,22sleep30-21swapper/320:45:113
2005325716,26sleep320114-21idleruntime00:15:113
2005325716,26sleep320114-21idleruntime00:15:113
1892425716,20sleep10-21swapper/120:40:121
1892425716,20sleep10-21swapper/120:40:121
674625615,22sleep16793-21users23:30:231
3115325614,17sleep10-21swapper/121:20:061
128725616,24sleep1286-21dbus-daemon19:44:501
2325425515,16sleep10-21swapper/100:25:101
760225412,25sleep37597-21apt-get21:50:053
2826725417,17sleep30-21swapper/321:10:113
2588025413,25sleep025873-21apt-get22:50:040
2588025413,25sleep025873-21apt-get22:50:040
2174825413,24sleep321743-21apt-get20:50:043
1781625418,17sleep00-21swapper/020:35:590
1781625418,17sleep00-21swapper/020:35:590
54512538,35sleep00-21swapper/023:27:550
535525312,26sleep34852-21apt-get21:40:003
326432536,12sleep10-21swapper/123:10:201
2995625311,23sleep330057-21/usr/sbin/munin19:30:193
1631625312,24sleep116309-21apt-get22:20:041
97425213,20sleep00-21swapper/023:14:510
624325213,23sleep16238-21apt-get20:00:011
624325213,23sleep16238-21apt-get20:00:011
471025216,17sleep10-21swapper/123:25:141
384225213,23sleep03100-21/usr/sbin/munin19:50:230
3056525215,16sleep00-21swapper/023:05:060
3056525215,16sleep00-21swapper/023:05:060
2967725212,16sleep00-21swapper/019:30:120
1599825216,17sleep30-21swapper/322:19:503
920325114,17sleep10-21swapper/123:40:121
1680625114,23sleep016801-21apt-get00:05:020
1656425111,15sleep00-21swapper/022:20:110
111662519,27sleep011209-21sed23:45:200
111662519,27sleep011209-21sed23:45:200
3113525016,15sleep00-21swapper/021:20:050
3098225012,24sleep030994-21fschecks_count19:35:080
3022225015,16sleep10-21swapper/121:15:221
2742725019,17sleep30-21swapper/322:55:043
2339325010,26sleep023418-21grep20:55:100
2183825012,21sleep021931-21perl20:50:090
616024918,18sleep30-21swapper/323:30:123
59624911,24sleep332655-21/usr/sbin/munin21:25:133
475024915,21sleep04754-21diskstats19:55:070
2931124916,17sleep00-21swapper/021:14:520
915024812,22sleep30-21swapper/321:55:063
3018424810,24sleep030185-21sshd23:03:410
1638724812,21sleep316394-21df_inode22:20:063
2351224715,18sleep30-21swapper/322:41:593
170782477,15sleep10-21swapper/100:05:111
308062469,15sleep30-21swapper/323:05:133
308062469,15sleep30-21swapper/323:05:133
1786024612,17sleep30-21swapper/322:25:033
215422456,14sleep10-21swapper/119:05:141
15682449,20sleep01570-21apt-get19:45:020
15682449,20sleep01570-21apt-get19:45:020
956924310,21sleep19602-21ntp_states21:55:161
89724212,14sleep00-21swapper/019:40:210
1543324114,13sleep30-21swapper/320:31:053
187222407,16sleep30-21swapper/320:40:063
187222407,16sleep30-21swapper/320:40:063
160662408,16sleep00-21swapper/022:19:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional