You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 03:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Sep 14, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9757298714,957sleep29770-21irqstats21:08:242
21794237718,339sleep021809-21sort20:03:200
21794237718,339sleep021809-21sort20:03:200
29580235210,285sleep129683-21fschecks_time20:28:211
498223487,304sleep1367-21snmpd19:10:111
6954234522,113sleep10-21swapper/120:58:301
5180233615,118sleep10-21swapper/120:53:251
19182225610,207sleep119263-21timerandwakeup19:53:321
907122490,48sleep10-21swapper/122:48:221
32366224856,58sleep10-21swapper/120:38:021
11511224549,173sleep111538-21modprobe22:58:001
1209022440,11sleep00-21swapper/022:58:220
2924722420,146sleep13734-21cyclictest22:08:281
7753224076,51sleep10-21swapper/122:43:301
2328422248,202sleep123365-21switchtime23:33:301
28411222378,127sleep13734-21cyclictest20:23:281
146622019,64sleep11613-21latency_hist20:43:031
24179219711,139sleep124231-21if_eth021:53:221
24179219711,139sleep124231-21if_eth021:53:221
12766218720,71sleep112740-21apt-get21:18:331
18433218517,113sleep118044-21/usr/sbin/munin23:18:271
9264217912,153sleep19300-21cpu19:23:151
3044217831,129sleep13734-21cyclictest00:13:201
9019217710,152sleep19164-21kworker/u8:000:33:021
9877216911,145sleep19920-21ntp_kernel_pll_21:08:271
13756216718,53sleep10-21swapper/123:03:251
3020021638,99sleep10-21swapper/122:13:171
26788216250,18sleep10-21swapper/123:44:001
2373421629,55sleep10-21swapper/121:53:021
13675215924,47sleep10-21swapper/121:18:571
13675215924,47sleep10-21swapper/121:18:571
2072321577,96sleep1367-21snmpd23:26:021
5325215632,113sleep15369-21unin-run22:38:011
821621539,132sleep18292-21switchtime00:28:301
16246215313,92sleep116369-21latency_hist23:13:021
2706021518,89sleep127184-21sed20:18:321
24867215149,58sleep10-21swapper/123:38:341
24867215149,58sleep10-21swapper/123:38:341
17597214417,111sleep017603-21cat21:33:180
17597214417,111sleep017603-21cat21:33:180
30480214028,98sleep130475-40mandb23:58:011
30480214028,98sleep130475-40mandb23:58:011
2800213713,110sleep22830-21nfs4_client22:28:292
19301213711,105sleep10-21swapper/121:38:121
19301213711,105sleep10-21swapper/121:38:121
23967213627,92sleep323997-21awk23:38:003
12469213610,100sleep10-21swapper/122:58:321
7269213418,47sleep10-21swapper/119:18:021
21347213433,82sleep10-21swapper/123:28:211
823821337,102sleep10-21swapper/119:18:551
823821337,102sleep10-21swapper/119:18:551
24569213314,103sleep224712-21latency_hist20:13:032
22180213228,39sleep10-21swapper/121:48:011
772121319,12sleep00-21swapper/019:18:170
772121319,12sleep00-21swapper/019:18:170
6576212918,96sleep36703-21unixbench_multi19:13:323
4551212925,43sleep10-21swapper/120:53:011
4551212925,43sleep10-21swapper/120:53:011
411212810,105sleep1480-21switchtime00:03:301
2076021288,105sleep120854-21timerandwakeup19:58:331
2076021288,105sleep120854-21timerandwakeup19:58:331
5705212619,83sleep25798-21fschecks_time22:38:222
27705212619,84sleep327797-21fschecks_time23:48:183
7416212325,40sleep10-21swapper/100:28:021
6363212212,96sleep06401-21processes19:13:290
12727212263,38sleep10-21swapper/119:33:251
30813212047,59sleep130835-21df23:58:211
29967212015,90sleep230111-21cat22:13:022
32603211913,84sleep232647-21/usr/sbin/munin20:38:162
22616211933,61sleep222404-21/usr/sbin/munin21:48:222
1146211912,52sleep10-21swapper/122:23:231
266621189,49sleep12701-21awk00:12:591
15032211845,39sleep1367-21snmpd19:40:561
9196211713,84sleep29237-21idleruntime-cro21:08:002
1652221178,59sleep10-21swapper/121:28:301
2604221168,14sleep20-21swapper/223:43:122
7128211418,81sleep27133-21sort22:43:112
951211113,85sleep0959-21df22:23:180
6917211010,55sleep10-21swapper/122:43:011
3113211013,80sleep13243-21grep22:28:361
31465210813,73sleep331495-21awk22:18:003
17276210828,30sleep00-21swapper/021:33:020
30740210720,67sleep0231-21systemd-journal20:33:000
27612210711,49sleep10-21swapper/122:03:321
27612210711,49sleep10-21swapper/122:03:321
2137210514,71sleep20-21swapper/222:28:002
13362210512,77sleep013370-21/usr/sbin/munin23:03:120
8294210313,72sleep28416-21timerwakeupswit00:28:312
2827021030,12sleep00-21swapper/020:23:220
9885210221,61sleep310028-21grep00:33:333
8963210217,62sleep09017-21latency_hist00:33:010
7655210212,74sleep37699-21/usr/sbin/munin00:28:163
31126210016,68sleep031134-21needreboot23:58:280
2336729917,70sleep223488-21timerwakeupswit23:33:312
46232988,60sleep10-21swapper/100:18:161
2613029825,54sleep10-21swapper/121:58:311
1853629817,66sleep218575-21grep23:18:322
1212129815,55sleep30-21swapper/319:33:023
2769329625,17sleep10-21swapper/123:48:161
1994629617,64sleep220053-21cat21:38:332
1994629617,64sleep220053-21cat21:38:332
164832967,41sleep13734-21cyclictest23:13:141
1703029512,55sleep20-21swapper/219:48:162
560829217,43sleep30-21swapper/322:38:153
1722629219,54sleep30-21swapper/321:33:013
1548129013,58sleep315456-21/usr/sbin/munin19:43:163
30628917,53sleep332285-21/usr/sbin/munin00:03:283
1991428915,48sleep319974-21latency_hist19:58:023
1471728927,33sleep20-21swapper/223:08:022
2085528817,27sleep20-21swapper/219:58:342
2085528817,27sleep20-21swapper/219:58:342
839028717,52sleep38508-21sed19:18:573
839028717,52sleep38508-21sed19:18:573
205642870,10sleep0374-21sshd19:58:270
205642870,10sleep0374-21sshd19:58:270
1081228717,55sleep310903-21latency_hist21:13:023
1441728622,48sleep10-21swapper/121:23:131
842128521,47sleep38481-21/usr/sbin/munin21:03:293
523528512,52sleep35387-21unixbench_multi00:18:313
1547928522,47sleep10-21swapper/119:43:141
150928524,30sleep20-21swapper/200:08:242
73632846,11sleep00-21swapper/022:43:210
1678128417,51sleep016799-21latency_hist19:48:020
2710528323,33sleep327100-21apt-get22:03:183
2710528323,33sleep327100-21apt-get22:03:183
252912836,58sleep10-21swapper/120:13:311
2456628350,21sleep124544-21/usr/sbin/munin20:13:021
1024328321,38sleep010395-21sed21:08:340
665228217,47sleep26746-21memory20:58:232
2437528216,46sleep20-21swapper/223:38:232
2437528216,46sleep20-21swapper/223:38:232
1930028219,44sleep018796-21apt-get21:38:110
1930028219,44sleep018796-21apt-get21:38:110
85522818,59sleep17913-21/usr/sbin/munin21:03:311
1852428118,10sleep10-21swapper/121:34:481
1852428118,10sleep10-21swapper/121:34:481
525028019,37sleep25326-21munin-node19:13:022
119892800,5sleep00-21swapper/021:13:380
2150827918,44sleep321628-21sort21:43:313
1192327918,10sleep10-21swapper/119:31:071
2607127714,46sleep226108-21modprobe20:18:002
252642777,20sleep00-21swapper/023:39:260
252642777,20sleep00-21swapper/023:39:260
194227717,46sleep10-21swapper/120:43:231
194227717,46sleep10-21swapper/120:43:231
848327611,51sleep08551-21awk21:03:300
394427624,35sleep03940-21apt-get19:08:160
1054327617,45sleep010574-21latency_hist00:38:020
38627516,11sleep10-21swapper/122:20:281
3139527518,25sleep30-21swapper/320:33:243
2722727514,41sleep027313-21/usr/sbin/munin22:03:230
2722727514,41sleep027313-21/usr/sbin/munin22:03:230
2678527523,33sleep30-21swapper/320:18:263
1396927516,50sleep00-21swapper/021:20:540
1396927516,50sleep00-21swapper/021:20:540
1190027521,33sleep311981-21cat21:13:383
2948127420,34sleep229478-21apt-get20:28:142
1814527416,10sleep10-21swapper/119:51:101
832127319,42sleep28389-21sed19:18:562
832127319,42sleep28389-21sed19:18:562
2015527316,30sleep30-21swapper/319:58:183
2015527316,30sleep30-21swapper/319:58:183
1235927324,27sleep012408-21cpuspeed19:33:160
1090827320,26sleep20-21swapper/219:28:212
529027218,35sleep0289-21dbus-daemon22:38:000
451427218,25sleep00-21swapper/020:52:590
451427218,25sleep00-21swapper/020:52:590
1509627210,25sleep30-21swapper/323:08:203
648327113,16sleep00-21swapper/000:23:240
597527112,44sleep25979-21diskstats19:13:192
373927124,23sleep30-21swapper/322:33:023
2924627121,19sleep30-21swapper/320:28:033
229162716,12sleep10-21swapper/120:07:211
229162716,12sleep10-21swapper/120:07:211
736127015,39sleep37391-21munin-plugin-st00:27:593
2308427017,23sleep00-21swapper/021:48:330
1447327017,24sleep30-21swapper/319:38:323
1219627017,31sleep211804-21/usr/sbin/munin22:58:252
39726922,22sleep20-21swapper/222:20:372
2522826915,36sleep225266-21awk21:58:002
2522826915,36sleep225266-21awk21:58:002
1941126917,22sleep00-21swapper/023:23:030
1910626918,29sleep00-21swapper/019:53:310
92612689,41sleep30-21swapper/319:23:133
878726817,22sleep00-21swapper/022:48:170
361626818,28sleep13644-21munin-run19:08:001
275426820,22sleep20-21swapper/219:03:262
1673826815,36sleep216777-21modprobe19:48:002
1284526817,22sleep20-21swapper/219:33:302
63826718,19sleep00-21swapper/022:23:010
2507526712,31sleep30-21swapper/320:13:253
2443526715,34sleep324497-21/usr/sbin/munin23:38:243
2443526715,34sleep324497-21/usr/sbin/munin23:38:243
1182926711,42sleep211899-21sensors21:13:372
10522675,51sleep00-21swapper/000:07:090
3103526628,21sleep30-21swapper/323:58:273
281326616,22sleep00-21swapper/019:03:280
1547126614,29sleep00-21swapper/023:08:290
83142655,51sleep32053-21kworker/3:022:46:433
67772656,13sleep00-21swapper/022:41:510
408126512,32sleep24187-21tune2fs22:33:212
325992658,36sleep30-21swapper/320:38:123
3043026520,29sleep030452-40ogrotate23:58:000
3043026520,29sleep030452-40ogrotate23:58:000
2686626515,24sleep00-21swapper/022:03:020
296126320,29sleep33055-21timerandwakeup19:03:313
2916726319,26sleep229234-21df23:53:132
2916726319,26sleep229234-21df23:53:132
2334126313,35sleep323353-21/usr/sbin/munin20:08:203
2296726320,25sleep323005-21modprobe20:08:003
2296726320,25sleep323005-21modprobe20:08:003
2213626320,20sleep00-21swapper/021:48:000
189926317,25sleep30-21swapper/300:08:333
1441926317,32sleep314438-21grep21:23:153
893626215,32sleep28970-21irqstats22:48:192
635626220,26sleep36375-21apt-get20:58:113
269226211,32sleep00-21swapper/022:28:270
2685126216,27sleep20-21swapper/220:18:292
2189226214,26sleep20-21swapper/220:03:222
2189226214,26sleep20-21swapper/220:03:222
1231626215,18sleep30-21swapper/322:58:303
37152615,11sleep30-21swapper/320:48:303
37152615,11sleep30-21swapper/320:48:303
3056026114,32sleep230569-21/usr/sbin/munin22:13:262
2739526110,36sleep027541-21date23:48:020
2737226122,20sleep30-21swapper/323:48:013
200592610,8sleep00-21swapper/023:23:260
2888026013,33sleep028908-21munin-run23:52:590
250922605,18sleep30-21swapper/321:56:153
250922605,18sleep30-21swapper/321:56:153
1751126017,27sleep317507-21apt-get21:33:133
1751126017,27sleep317507-21apt-get21:33:133
1675126020,21sleep00-21swapper/023:13:210
1078926019,22sleep00-21swapper/022:53:300
974925911,35sleep29817-21readlink19:23:282
541925918,19sleep30-21swapper/320:53:313
534425915,27sleep35463-21latency_hist19:13:033
2789025912,27sleep227979-21/usr/sbin/munin23:48:212
2247225918,24sleep30-21swapper/323:33:033
2221525920,20sleep20-21swapper/221:48:022
2116525912,21sleep00-21swapper/023:28:160
207125913,29sleep02117-21/usr/sbin/munin20:43:260
207125913,29sleep02117-21/usr/sbin/munin20:43:260
1877325919,25sleep218845-21gpgv21:38:002
1877325919,25sleep218845-21gpgv21:38:002
1833925918,25sleep018447-21sed19:53:020
1420725915,23sleep00-21swapper/019:38:250
1110325914,25sleep311156-21users22:53:343
38425816,25sleep0417-21irqstats20:38:220
3251425820,22sleep30-21swapper/322:18:323
3076125816,25sleep230757-21apt-get23:58:182
302825818,24sleep33034-21df_inode00:13:183
205012585,16sleep30-21swapper/321:41:533
205012585,16sleep30-21swapper/321:41:533
3043625715,22sleep00-21swapper/022:13:240
2646525714,26sleep026471-21diskstats20:18:190
2615125716,21sleep00-21swapper/020:18:020
159422576,39sleep20-21swapper/221:28:112
1468725715,18sleep20-21swapper/221:23:232
1408525712,20sleep30-21swapper/321:22:273
1408525712,20sleep30-21swapper/321:22:273
296392566,40sleep00-21swapper/022:09:330
2601025612,25sleep326042-21/usr/sbin/munin21:58:293
1926625610,22sleep20-21swapper/219:53:332
1775125619,22sleep217761-21modprobe23:18:002
620025517,21sleep20-21swapper/220:58:022
607925522,19sleep20-21swapper/200:23:102
2862425514,22sleep328642-21apt-get22:08:103
2092825511,25sleep30-21swapper/323:28:023
1502225517,18sleep20-21swapper/223:08:192
931425421,16sleep30-21swapper/322:48:293
586725422,16sleep20-21swapper/200:23:022
389125418,14sleep20-21swapper/220:48:332
389125418,14sleep20-21swapper/220:48:332
141325413,26sleep01408-21apt-get00:08:180
1291125417,18sleep30-21swapper/319:33:313
2793225314,24sleep227928-21apt-get20:23:132
2300125313,25sleep30-21swapper/321:48:323
2116225313,23sleep221158-21apt-get23:28:132
1971925315,22sleep219822-21perl23:23:212
2937425213,18sleep30-21swapper/323:53:183
2937425213,18sleep30-21swapper/323:53:183
2563225216,16sleep20-21swapper/221:58:202
215625214,23sleep22163-21ntp_states20:43:282
215625214,23sleep22163-21ntp_states20:43:282
192762525,14sleep30-21swapper/323:22:213
1739825212,27sleep317437-21ntpq19:48:253
1154725220,17sleep20-21swapper/222:58:012
990325115,17sleep00-21swapper/019:23:300
480525114,18sleep00-21swapper/020:53:160
454825113,23sleep04547-21apt-get00:18:110
2973225113,21sleep329753-21irqstats20:28:233
2335525112,17sleep00-21swapper/020:08:210
1441625114,23sleep014412-21apt-get21:23:120
1365425119,15sleep30-21swapper/323:03:233
920225015,18sleep20-21swapper/200:33:152
431225016,16sleep20-21swapper/200:18:012
2407525014,20sleep024178-21perl21:53:210
2407525014,20sleep024178-21perl21:53:210
2276425012,17sleep00-21swapper/023:33:160
1776225020,17sleep30-21swapper/323:18:013
16762506,15sleep30-21swapper/320:43:133
16762506,15sleep30-21swapper/320:43:133
1441425012,14sleep20-21swapper/219:38:312
321592496,32sleep20-21swapper/220:36:222
2841124916,18sleep20-21swapper/222:08:012
2841124916,18sleep20-21swapper/222:08:012
88024811,18sleep30-21swapper/322:23:173
686824813,19sleep00-21swapper/020:58:290
53712489,24sleep35475-21latency_hist22:38:033
303624615,15sleep00-21swapper/000:13:190
1418124612,15sleep20-21swapper/221:23:022
1418124612,15sleep20-21swapper/221:23:022
3102524512,16sleep00-21swapper/020:33:130
1088824512,17sleep00-21swapper/019:28:190
11172415,13sleep20-21swapper/200:07:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional