You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 12:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 12, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
128992400816,3963sleep315031-21kworker/u8:223:34:043
3603265491,423sleep03742-21grep21:19:370
2883826189,600sleep2195-21jbd2/mmcblk2p2-20:58:592
27436261015,580sleep327476-21ntpq00:19:193
215524337,12sleep00-21swapper/021:16:260
95952349200,127sleep00-21swapper/023:24:080
12220232818,231sleep00-21swapper/021:49:080
22783232170,224sleep022845-21mailstats00:04:220
19374230811,113sleep019477-21cut20:29:140
1799922758,252sleep318035-21ntpq23:49:183
24963224455,108sleep00-21swapper/000:11:280
840223919,207sleep3886-21wc21:13:543
840223919,207sleep3886-21wc21:13:543
10215223626,51sleep00-21swapper/019:59:170
24383222015,186sleep0225-21systemd-journal22:28:530
8900221610,190sleep29044-21cat21:38:572
6226220914,181sleep00-21swapper/023:13:550
3173922029,179sleep031777-21/usr/sbin/munin19:24:240
25178219010,169sleep025261-21tr22:29:220
25178219010,169sleep025261-21tr22:29:220
89021818,158sleep01040-21date21:13:550
28977217722,58sleep00-21swapper/000:24:210
23954217715,140sleep00-21swapper/020:43:550
13715217512,148sleep013840-21cat20:09:230
21017217420,45sleep00-21swapper/022:15:530
3238721719,109sleep00-21swapper/021:09:190
2731121665,31sleep00-21swapper/019:12:120
3184221659,146sleep0364-21snmpd22:49:480
3103121649,19sleep00-21swapper/019:23:550
11588216412,129sleep011650-21fschecks_time20:04:140
466721559,41sleep00-21swapper/023:08:540
3883215173,22sleep00-21swapper/019:39:180
26361214916,41sleep00-21swapper/020:49:250
28139214715,51sleep00-21swapper/019:14:100
1182821478,127sleep0364-21snmpd21:47:350
11349214745,18sleep00-21swapper/023:29:130
11349214745,18sleep00-21swapper/023:29:130
8447214612,45sleep00-21swapper/019:54:120
3050121467,127sleep030537-21ntpq00:29:170
18245214422,103sleep10-21swapper/123:49:241
1254214212,112sleep01324-21gpgv22:58:530
22161214015,31sleep30-21swapper/322:19:233
28863213910,49sleep028866-21apt-get20:59:050
14039213815,107sleep214108-21latency21:54:192
15430213510,107sleep00-21swapper/021:59:130
22018213412,110sleep022058-21systemd-run00:03:530
22018213412,110sleep022058-21systemd-run00:03:530
1773621338,48sleep017726-21apt-get20:24:130
1773621338,48sleep017726-21apt-get20:24:130
1628921339,46sleep016321-21irqstats23:44:150
1628921339,46sleep016321-21irqstats23:44:150
26966213223,89sleep327107-21unixbench_singl22:34:253
31099213111,105sleep231218-21timerwakeupswit21:04:292
31099213111,105sleep231218-21timerwakeupswit21:04:292
20716213113,104sleep320724-21acpi23:59:033
20716213113,104sleep320724-21acpi23:59:033
8408213011,98sleep00-21swapper/023:19:160
8408213011,98sleep00-21swapper/023:19:160
4888213027,77sleep25086-21grep19:43:562
15943213035,84sleep016047-21date20:18:560
111421300,13sleep10-21swapper/121:14:121
30639212912,95sleep00-21swapper/021:04:180
30639212912,95sleep00-21swapper/021:04:180
1760121277,38sleep00-21swapper/023:49:070
24274212611,96sleep00-21swapper/020:44:120
25441212531,14sleep00-21swapper/000:14:070
25441212531,14sleep00-21swapper/000:14:070
19756212211,96sleep019841-21cat22:13:550
27305211818,84sleep00-21swapper/000:19:160
27059211821,78sleep327188-21latency_hist20:53:563
23657211614,88sleep1296-21rs:main1
20508211675,18sleep00-21swapper/023:58:560
6994211526,75sleep00-21swapper/019:49:160
6994211526,75sleep00-21swapper/019:49:160
27819211516,77sleep227013-21/usr/sbin/munin00:19:262
7730211420,72sleep17783-21sendmail23:18:521
7730211420,72sleep17783-21sendmail23:18:521
13571211412,20sleep30-21swapper/321:53:563
248121139,84sleep00-21swapper/019:34:250
30026211210,17sleep10-21swapper/119:19:201
27523211111,81sleep327717-21date19:13:573
1377321109,84sleep00-21swapper/021:54:130
8033210817,76sleep38117-21grep21:34:213
2846210722,74sleep00-21swapper/023:00:550
7755210620,11sleep00-21swapper/019:49:440
27297210624,49sleep00-21swapper/020:54:090
16261210621,48sleep00-21swapper/020:19:120
16261210621,48sleep00-21swapper/020:19:120
10253210628,67sleep00-21swapper/021:42:140
29444210510,83sleep029448-21entropy22:44:150
7655210424,11sleep00-21swapper/023:17:540
7655210424,11sleep00-21swapper/023:17:540
754821049,39sleep00-21swapper/021:34:060
730221048,85sleep07311-21munin-run21:33:540
31682210418,48sleep00-21swapper/000:34:100
31682210418,48sleep00-21swapper/000:34:100
4056210315,73sleep14154-21cat19:39:231
15750210319,31sleep00-21swapper/023:43:420
15750210319,31sleep00-21swapper/023:43:420
29107210219,32sleep00-21swapper/022:43:550
2726321026,44sleep00-21swapper/022:36:000
24761210214,75sleep324834-21switchtime20:44:233
1331210218,10sleep00-21swapper/019:30:140
12918210221,50sleep00-21swapper/023:34:050
21778210120,10sleep00-21swapper/020:34:590
5487210015,11sleep00-21swapper/021:25:590
5487210015,11sleep00-21swapper/021:25:590
29935210015,68sleep329992-21irqstats19:19:183
28853210017,61sleep128874-21kernelversion00:24:171
22690210015,11sleep00-21swapper/022:21:590
22499210015,68sleep322101-21apt-get20:38:583
1810729915,12sleep00-21swapper/022:08:140
1810729915,12sleep00-21swapper/022:08:140
3175129814,68sleep131832-21date21:08:561
3175129814,68sleep131832-21date21:08:561
2940329627,45sleep329437-21modprobe19:18:543
2287029514,57sleep122613-21/usr/sbin/munin20:39:281
1912529512,69sleep119144-21apt-get23:54:051
1744229411,65sleep217490-21systemd-run20:23:542
1744229411,65sleep217490-21systemd-run20:23:542
1924729316,61sleep219333-21fschecks_time23:54:142
2591029139,34sleep225929-21munin-run19:08:552
2618329011,59sleep20-21swapper/219:09:072
1210528926,32sleep30-21swapper/320:04:253
2684528717,49sleep226964-21timerandwakeup22:34:242
2387028713,58sleep223889-21apt-get00:09:042
48242867,57sleep30-21swapper/321:24:163
48242867,57sleep30-21swapper/321:24:163
543328522,42sleep15488-21tune2fs19:44:441
543328522,42sleep15488-21tune2fs19:44:441
2868228519,50sleep328831-21date20:58:563
96928428,24sleep20-21swapper/219:29:272
3207428420,41sleep232130-21fschecks_count21:09:132
3207428420,41sleep232130-21fschecks_count21:09:132
2109528313,57sleep221179-21cat20:34:162
1095228324,27sleep30-21swapper/323:28:563
2807728117,51sleep128161-21cat22:39:161
3179628021,29sleep20-21swapper/222:49:262
763427920,40sleep17674-21sort21:34:111
242852790,6sleep10-21swapper/120:44:121
1561227914,51sleep315670-21sed21:59:173
3167127814,48sleep10-21swapper/119:24:211
256227823,30sleep10-21swapper/119:34:271
1984227818,32sleep20-21swapper/220:29:242
1813727816,32sleep30-21swapper/320:24:243
1813727816,32sleep30-21swapper/320:24:243
531027724,33sleep20-21swapper/219:44:372
531027724,33sleep20-21swapper/219:44:372
1665327718,39sleep116670-21latency_hist22:03:551
567827612,51sleep15679-21sshd21:28:261
567827612,51sleep15679-21sshd21:28:261
318627619,39sleep23330-21modprobe23:03:562
3073627618,36sleep230818-21grep00:29:232
199627615,29sleep20-21swapper/222:59:322
2895827511,50sleep128989-21fschecks_count20:59:151
2671827521,36sleep126743-21munin-run00:18:521
2671827521,36sleep126743-21munin-run00:18:521
1586427520,36sleep215913-21awk20:18:542
1339427516,28sleep20-21swapper/223:34:282
345327417,40sleep33455-21/usr/sbin/munin23:04:133
2591227418,31sleep20-21swapper/200:14:212
2591227418,31sleep20-21swapper/200:14:212
2303827418,39sleep323177-21awk00:04:253
1905627417,28sleep20-21swapper/220:28:552
1905627417,28sleep20-21swapper/220:28:552
1350727418,24sleep10-21swapper/120:09:201
2648827324,24sleep10-21swapper/120:49:261
2154127319,37sleep320800-21/usr/sbin/munin20:34:253
1841727322,28sleep10-21swapper/120:24:301
1841727322,28sleep10-21swapper/120:24:301
1281027325,24sleep10-21swapper/120:08:551
2416427223,29sleep10-21swapper/100:09:151
1991927221,29sleep30-21swapper/320:29:253
660827116,23sleep20-21swapper/221:29:242
214327114,42sleep3364-21snmpd19:34:163
3089127015,40sleep330930-21ps21:04:263
3089127015,40sleep330930-21ps21:04:263
2789327014,34sleep227982-21fschecks_time22:39:122
2475827017,15sleep20-21swapper/222:29:112
2475827017,15sleep20-21swapper/222:29:112
177727016,35sleep11273-21apt-get22:59:021
10832709,47sleep20-21swapper/221:14:102
1735626921,26sleep30-21swapper/323:48:543
1735626921,26sleep30-21swapper/323:48:543
166926924,28sleep30-21swapper/319:33:563
1462826911,39sleep10-21swapper/121:54:301
849126821,24sleep20-21swapper/219:54:142
3194926818,27sleep30-21swapper/319:24:273
2948526813,23sleep20-21swapper/219:18:572
2932826815,30sleep329472-21sed00:24:283
2770126813,15sleep10-21swapper/100:19:251
2045526817,38sleep120461-21open_files22:14:201
1524726816,38sleep215387-21unixbench_singl23:39:242
1524726816,38sleep215387-21unixbench_singl23:39:242
1038226820,21sleep20-21swapper/221:43:532
940726717,34sleep39332-21latency_hist23:23:553
3087626713,31sleep10-21swapper/122:49:051
2936126723,24sleep10-21swapper/122:44:121
2667826717,28sleep10-21swapper/119:09:231
2348026717,21sleep10-21swapper/122:24:181
2094426711,40sleep120986-21cat23:59:141
2094426711,40sleep120986-21cat23:59:141
576726616,34sleep25731-21latency_hist21:28:552
576726616,34sleep25731-21latency_hist21:28:552
2520626625,22sleep30-21swapper/319:04:243
2506826630,21sleep10-21swapper/119:04:221
1320426618,21sleep10-21swapper/123:34:231
970726523,22sleep30-21swapper/319:58:563
2985326514,34sleep329927-21grep22:44:263
3228526417,30sleep132359-21find00:34:271
3228526417,30sleep132359-21find00:34:271
3173326413,37sleep331757-21df00:34:133
3173326413,37sleep331757-21df00:34:133
3021826420,28sleep30-21swapper/300:29:123
2756926413,22sleep10-21swapper/122:38:551
2488126416,30sleep024966-21memory19:04:170
2106126412,36sleep121094-21irqstats20:34:151
4942637,12sleep30-21swapper/322:54:243
311212638,13sleep10-21swapper/100:30:271
2813526317,27sleep128132-21apt-get19:14:071
1253126323,16sleep20-21swapper/223:33:552
1253126323,16sleep20-21swapper/223:33:552
1064726310,29sleep10-21swapper/121:44:041
3225026220,19sleep30-21swapper/322:53:553
2798426221,18sleep30-21swapper/322:39:133
2796826219,25sleep128092-21sed20:54:261
2438626213,37sleep224440-21if_eth020:44:142
2133326219,20sleep30-21swapper/322:18:553
169726214,26sleep31794-21/usr/sbin/munin21:14:273
2972726022,22sleep20-21swapper/219:19:132
259742609,33sleep326053-21/usr/sbin/munin00:14:223
259742609,33sleep326053-21/usr/sbin/munin00:14:223
2320226012,33sleep323209-21diskstats22:24:123
1788726013,24sleep30-21swapper/322:05:263
1788726013,24sleep30-21swapper/322:05:263
1757926014,28sleep217707-21unixbench_multi22:04:242
1757926014,28sleep217707-21unixbench_multi22:04:242
161926017,27sleep11650-21munin-run19:33:541
808025913,26sleep21-21systemd19:53:542
66925913,23sleep10-21swapper/122:54:261
624125913,30sleep16274-21/usr/sbin/munin21:29:151
342925916,18sleep20-21swapper/221:19:352
319425924,21sleep30-21swapper/321:19:283
2004825912,28sleep320057-21/usr/sbin/munin23:54:273
1523425913,22sleep10-21swapper/120:14:251
127012597,37sleep10-21swapper/121:49:211
1090125923,16sleep20-21swapper/223:28:552
994325815,26sleep29940-21apt-get19:59:072
2389725820,22sleep30-21swapper/300:09:093
236925822,20sleep20-21swapper/219:34:232
1682025815,22sleep10-21swapper/120:19:251
1682025815,22sleep10-21swapper/120:19:251
1385625817,15sleep30-21swapper/321:54:163
1045225813,17sleep10-21swapper/121:43:561
812825715,16sleep30-21swapper/319:53:553
2599225711,29sleep30-21swapper/322:33:573
2599225711,29sleep30-21swapper/322:33:573
2467325713,27sleep124668-21apt-get22:29:061
2467325713,27sleep124668-21apt-get22:29:061
2126125718,27sleep2292-21systemd-logind22:18:542
1666725721,23sleep316740-21tr20:19:233
1666725721,23sleep316740-21tr20:19:233
1074525716,28sleep310751-21df_inode21:44:113
343725610,17sleep10-21swapper/123:04:111
2390225612,20sleep20-21swapper/220:43:532
1934825616,24sleep119354-21df_inode20:29:101
1690025612,27sleep116894-21apt-get22:04:071
1690025612,27sleep116894-21apt-get22:04:071
1468625616,18sleep30-21swapper/323:39:133
1468625616,18sleep30-21swapper/323:39:133
1459425614,26sleep314590-21apt-get20:14:063
145352560,8sleep20-21swapper/220:13:562
141225618,21sleep21568-21gpgv22:58:552
1229225614,21sleep20-21swapper/221:49:102
97612558,31sleep10-21swapper/123:24:131
1635725512,17sleep20-21swapper/223:44:172
1635725512,17sleep20-21swapper/223:44:172
1632225519,20sleep10-21swapper/123:44:161
1632225519,20sleep10-21swapper/123:44:161
912625412,26sleep39117-21apt-get21:39:103
865125417,19sleep10-21swapper/119:54:171
552925414,24sleep35581-21irqstats19:44:463
552925414,24sleep35581-21irqstats19:44:463
537425415,18sleep10-21swapper/123:09:201
45322548,10sleep30-21swapper/319:41:153
3171725417,17sleep20-21swapper/219:24:222
2939525413,25sleep229432-21df22:44:132
2398225413,25sleep224029-21users22:24:272
2166325411,16sleep20-21swapper/223:59:312
2166325411,16sleep20-21swapper/223:59:312
194022546,10sleep10-21swapper/122:10:001
1150025414,24sleep211496-21apt-get20:04:082
1114125412,27sleep311208-21df23:29:083
1114125412,27sleep311208-21df23:29:083
722725312,23sleep10-21swapper/119:49:211
669625312,18sleep30-21swapper/321:29:253
646525315,25sleep36506-21grep23:14:103
646525315,25sleep36506-21grep23:14:103
3198825313,24sleep131982-21apt-get21:09:081
3198825313,24sleep131982-21apt-get21:09:081
2809325312,24sleep228199-21users20:54:272
1806425315,17sleep20-21swapper/223:49:212
1534225314,22sleep215338-21apt-get21:59:072
1140625315,20sleep10-21swapper/123:29:141
1140625315,20sleep10-21swapper/123:29:141
1066925313,25sleep210666-21apt-get21:44:052
1003725313,24sleep110127-21perl19:59:141
288425213,23sleep12897-21apt-get21:19:051
1841825219,18sleep30-21swapper/322:09:013
491225113,22sleep34908-21apt-get23:09:053
823025016,17sleep10-21swapper/123:19:131
823025016,17sleep10-21swapper/123:19:131
813025013,18sleep30-21swapper/323:19:113
778425012,22sleep27808-21munin-run23:18:532
778425012,22sleep27808-21munin-run23:18:532
3168425015,18sleep20-21swapper/200:34:122
3168425015,18sleep20-21swapper/200:34:122
2989425015,15sleep20-21swapper/200:28:552
29792507,35sleep30-21swapper/323:02:353
2158025012,14sleep10-21swapper/122:19:091
855324911,16sleep20-21swapper/223:19:222
855324911,16sleep20-21swapper/223:19:222
2750624912,16sleep30-21swapper/320:54:163
268812495,36sleep20-21swapper/220:52:172
474124815,16sleep20-21swapper/223:08:562
108642487,31sleep30-21swapper/319:59:323
2275124714,18sleep20-21swapper/200:04:192
1844324710,20sleep218439-21apt-get22:09:052
1348224614,16sleep30-21swapper/320:09:183
2208424410,17sleep20-21swapper/222:19:222
145042438,17sleep10-21swapper/123:39:071
145042438,17sleep10-21swapper/123:39:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional