You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 17:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sat Aug 29, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122922642110,6398sleep032639-21kworker/0:109:39:370
122922642110,6398sleep032639-21kworker/0:109:39:370
20343235386,3519sleep214126-21kworker/2:008:22:022
293912220917,2174sleep015994-21kworker/u8:307:09:080
293912220917,2174sleep015994-21kworker/u8:307:09:080
18764279010,760sleep1201-21jbd2/mmcblk2p2-09:59:211
249552497115,330sleep225017-21fschecks_time10:19:162
28405247023,428sleep028492-21/usr/sbin/munin07:04:280
28405247023,428sleep028492-21/usr/sbin/munin07:04:280
19996244321,404sleep320132-21/usr/sbin/munin08:19:293
8376231224,267sleep38469-21fschecks_time07:44:163
8924229555,197sleep29033-21ls07:44:272
1366122950,7sleep10-21swapper/107:59:271
3273622510,43sleep229072-21cyclictest07:18:582
235482249107,20sleep20-21swapper/211:58:582
5397224444,184sleep21-21systemd09:18:562
108562244100,19sleep20-21swapper/211:18:562
5646223091,27sleep20-21swapper/207:34:262
21092221123,82sleep22238-21latency_hist09:08:582
21092221123,82sleep22238-21latency_hist09:08:582
7113222011,195sleep17157-21ntpq07:39:211
321552220109,69sleep20-21swapper/208:59:252
18915221832,167sleep218918-21cron08:18:552
8571221331,145sleep28646-21latency_hist09:28:572
3416220615,174sleep03344-21latency_hist07:28:590
21747220615,179sleep221724-21apt-get10:09:192
14482219916,122sleep214618-21grep09:44:402
3600219813,79sleep23622-21munin-plugin-st09:13:552
28442218615,111sleep228508-21grep10:29:262
18637218120,142sleep318722-21systemd11:43:573
18637218120,142sleep318722-21systemd11:43:573
16569218047,73sleep20-21swapper/209:53:572
2093321788,106sleep229072-21cyclictest11:49:172
27966217522,138sleep00-21swapper/010:29:120
2855121738,77sleep20-21swapper/208:49:122
3317217011,22sleep21-21systemd07:28:572
26078216737,108sleep226216-21grep12:04:272
1137821678,121sleep211438-21cpuspeed07:54:122
3685216418,24sleep20-21swapper/207:29:132
2658021598,134sleep20-21swapper/210:24:172
6810215846,58sleep20-21swapper/207:39:142
2212215828,109sleep20-21swapper/210:49:252
2212215828,109sleep20-21swapper/210:49:252
2106521586,92sleep2367-21snmpd10:07:052
7024215215,113sleep37156-21latency_hist09:23:573
942621507,128sleep29551-21cat09:29:262
3092521509,128sleep230937-21vmstat12:19:292
21450214725,97sleep20-21swapper/208:24:292
30992214433,77sleep231046-21timerandwakeup07:14:032
30992214433,77sleep231046-21timerandwakeup07:14:032
27034214410,111sleep00-21swapper/008:44:130
1858321429,31sleep218634-21systemd-run11:43:562
1858321429,31sleep218634-21systemd-run11:43:562
447521418,121sleep2367-21snmpd10:57:572
28258213916,105sleep228280-21awk08:48:552
15390213916,103sleep20-21swapper/211:33:022
4054213613,91sleep00-21swapper/010:54:280
18939213215,97sleep118597-21apt-get11:43:591
2376721317,14sleep20-21swapper/211:59:062
2376721317,14sleep20-21swapper/211:59:062
10191213127,17sleep20-21swapper/207:49:202
776021309,86sleep27908-21sed11:08:592
29486213027,70sleep20-21swapper/210:34:022
17009213063,54sleep217025-21modprobe11:38:552
3020021218,74sleep230307-21grep08:54:172
807212011,87sleep20-21swapper/210:44:282
50621200,8sleep10-21swapper/109:03:561
22692212012,63sleep022694-21sshd10:10:020
19823211659,15sleep20-21swapper/211:44:532
12824211612,88sleep012879-21idleruntime11:24:160
11686211615,84sleep011691-21apt-key09:38:570
24059211517,55sleep20-21swapper/208:34:182
1219921158,34sleep20-21swapper/209:39:272
1219921158,34sleep20-21swapper/209:39:272
23257211315,16sleep20-21swapper/210:14:082
16917211217,73sleep117025-21fschecks_time09:54:161
16917211217,73sleep117025-21fschecks_time09:54:161
2270821117,89sleep222749-21ntpq08:29:212
19505211117,78sleep119586-21sed08:19:191
1632521107,66sleep216399-21cat08:09:182
28266210916,76sleep328398-21unixbench_multi07:04:273
28266210916,76sleep328398-21unixbench_multi07:04:273
16782210918,75sleep316924-21sed08:09:283
13226210913,82sleep013257-21/usr/sbin/munin07:59:190
28851210843,42sleep128880-21munin-run07:08:571
28851210843,42sleep128880-21munin-run07:08:571
595210712,37sleep20-21swapper/207:19:162
43121078,65sleep2547-21date12:28:572
29388210520,68sleep328879-21apt-get07:09:073
29388210520,68sleep328879-21apt-get07:09:073
3156121047,63sleep231595-21ps10:39:232
1426121047,63sleep214268-21munin-run08:03:572
87221039,79sleep3884-21/usr/sbin/munin09:04:153
87221039,79sleep3884-21/usr/sbin/munin09:04:153
29611210222,60sleep129691-21tune2fs10:34:181
28823210112,70sleep328920-21/usr/sbin/munin08:49:213
28541210121,15sleep20-21swapper/212:14:162
15464210113,49sleep20-21swapper/208:05:022
2222210011,49sleep20-21swapper/212:34:102
1396521009,51sleep20-21swapper/211:28:582
1035921008,51sleep20-21swapper/209:34:102
1772629810,74sleep017733-21/usr/sbin/munin08:14:130
168182988,53sleep20-21swapper/209:54:092
168182988,53sleep20-21swapper/209:54:092
362329715,68sleep33715-21sed09:13:563
281512977,34sleep228172-21munin-run12:13:562
892929512,70sleep08930-21sshd11:10:020
2891329434,35sleep229010-21apt-config07:08:582
2891329434,35sleep229010-21apt-config07:08:582
2269329316,56sleep122706-21needreboot08:29:201
1240629224,46sleep20-21swapper/211:23:572
1240629224,46sleep20-21swapper/211:23:572
1062929120,54sleep310677-21latency09:34:193
67512895,29sleep20-21swapper/209:21:022
641428816,56sleep06478-21modprobe07:38:560
30228819,52sleep3396-21fschecks_time10:44:173
2415728813,45sleep10-21swapper/111:59:211
2415728813,45sleep10-21swapper/111:59:211
203428827,42sleep20-21swapper/212:33:582
195312885,26sleep20-21swapper/210:02:172
3264428620,41sleep332669-21munin-run07:18:563
3077128624,40sleep130921-21grep12:19:281
2381228510,55sleep123258-21/usr/sbin/munin10:14:271
454928422,44sleep34576-21munin-run10:58:553
3167828422,27sleep30-21swapper/312:24:183
3045228410,60sleep130454-21sshd07:11:031
3045228410,60sleep130454-21sshd07:11:031
2120528417,50sleep321232-21munin-run10:08:563
17482849,55sleep10-21swapper/110:49:121
17482849,55sleep10-21swapper/110:49:121
3070928317,52sleep130795-21ls08:54:281
162528327,36sleep3374-21sshd12:29:593
527228213,33sleep20-21swapper/210:59:222
2396528128,27sleep30-21swapper/310:14:293
1554528118,30sleep10-21swapper/109:49:191
114632815,56sleep10-21swapper/107:54:151
98028010,56sleep01017-21ps07:19:260
2922828016,34sleep30-21swapper/310:33:553
1401028023,28sleep10-21swapper/109:44:311
1807527916,37sleep018126-21grep09:58:550
1807527916,37sleep018126-21grep09:58:550
2648427833,11sleep20-21swapper/208:41:002
2641727828,23sleep10-21swapper/110:24:121
150452786,52sleep30-21swapper/308:04:273
3158227720,33sleep031578-21apt-get12:24:110
599427613,41sleep06088-21memory09:19:220
805527519,14sleep20-21swapper/211:09:152
797527515,41sleep37950-21/usr/sbin/munin11:09:123
145527522,33sleep10-21swapper/110:48:551
1174127525,21sleep10-21swapper/109:38:581
305427416,40sleep33195-21sed10:53:583
305427416,40sleep33195-21sed10:53:583
2665127415,32sleep00-21swapper/012:08:570
2170527416,39sleep121218-21apt-get10:09:061
54827320,28sleep30-21swapper/309:03:593
356027315,43sleep13670-21cat12:38:571
32912735,57sleep00-21swapper/012:36:260
301062735,15sleep30-21swapper/312:19:133
1741327317,36sleep117507-21fschecks_time11:39:151
1741327317,36sleep117507-21fschecks_time11:39:151
618427212,43sleep16216-21/usr/sbin/munin09:19:271
377927217,21sleep10-21swapper/110:54:251
2892227221,22sleep00-21swapper/008:49:220
2087927221,27sleep30-21swapper/308:24:173
1621727219,21sleep00-21swapper/008:09:160
818427117,26sleep10-21swapper/111:09:181
76332717,44sleep30-21swapper/309:24:223
719727116,27sleep00-21swapper/007:39:250
425227012,41sleep34374-21sed07:29:263
2819727012,17sleep00-21swapper/012:13:580
2708727022,23sleep30-21swapper/308:44:163
2452327015,23sleep00-21swapper/008:34:280
2126027023,19sleep00-21swapper/010:08:570
1301427018,33sleep30-21swapper/311:24:193
1242927019,28sleep30-21swapper/311:23:583
1242927019,28sleep30-21swapper/311:23:583
99032695,11sleep00-21swapper/009:31:460
955326926,24sleep30-21swapper/309:29:273
884626916,32sleep18922-21switchtime07:44:261
700726913,28sleep30-21swapper/307:39:193
3271826911,41sleep032724-21df_inode10:44:140
321872696,11sleep00-21swapper/010:41:440
1611626918,29sleep315727-21/usr/sbin/munin11:34:203
1271226923,19sleep30-21swapper/307:58:583
1815726814,38sleep318305-21latency_hist09:58:583
1815726814,38sleep318305-21latency_hist09:58:583
639626724,28sleep16407-21sort11:04:051
60226722,25sleep30-21swapper/307:19:173
3137326714,29sleep031304-21latency_hist12:23:590
2814526724,22sleep10-21swapper/112:13:551
2664126710,42sleep326684-21irqstats10:24:183
851926623,27sleep08534-21modprobe09:28:550
547726619,27sleep30-21swapper/310:59:273
2812926616,29sleep10-21swapper/110:29:171
2004626617,21sleep10-21swapper/110:04:181
485326517,28sleep10-21swapper/109:14:311
293622656,51sleep00-21swapper/012:14:340
493626418,22sleep00-21swapper/007:33:590
474326413,34sleep04850-21timerandwakeup09:14:300
3159826421,27sleep331672-21/usr/sbin/munin10:39:243
3057026414,23sleep30-21swapper/308:54:263
2121626417,28sleep121238-21sshd08:24:241
2117826423,21sleep00-21swapper/008:24:230
1929626417,26sleep30-21swapper/311:44:423
157426415,27sleep00-21swapper/010:48:580
157426415,27sleep00-21swapper/010:48:580
121562645,11sleep00-21swapper/011:21:130
121562645,11sleep00-21swapper/011:21:130
847126312,19sleep00-21swapper/007:44:170
336926324,19sleep30-21swapper/307:28:583
3228726318,23sleep00-21swapper/008:59:270
2260326318,25sleep00-21swapper/008:29:190
1670326313,37sleep0367-21snmpd11:34:580
1511726318,25sleep015203-21ls08:04:280
1316726311,35sleep313224-21irqstats07:59:183
613526217,20sleep00-21swapper/011:03:560
2379026220,20sleep00-21swapper/011:59:120
2379026220,20sleep00-21swapper/011:59:120
1762226212,36sleep117630-21/usr/sbin/munin08:14:061
1462126217,28sleep30-21swapper/309:44:413
204882615,11sleep00-21swapper/008:23:540
1793226118,20sleep00-21swapper/011:39:260
1793226118,20sleep00-21swapper/011:39:260
1507726114,25sleep00-21swapper/009:48:580
1420726117,25sleep00-21swapper/011:29:120
952426016,27sleep39519-21apt-get11:14:133
516226018,24sleep15229-21df07:34:131
242782608,17sleep00-21swapper/010:15:310
104826010,21sleep10-21swapper/112:29:211
3215825917,21sleep10-21swapper/112:24:281
2978325912,26sleep00-21swapper/010:34:210
2545825918,23sleep125525-21df08:39:131
2440025918,21sleep30-21swapper/311:59:273
2440025918,21sleep30-21swapper/311:59:273
18332596,42sleep00-21swapper/009:06:100
18332596,42sleep00-21swapper/009:06:100
1114925917,23sleep10-21swapper/107:53:591
78625816,26sleep1783-21apt-get09:04:091
78625816,26sleep1783-21apt-get09:04:091
409525817,22sleep10-21swapper/107:29:241
247922589,21sleep30-21swapper/308:34:313
1527325814,29sleep015266-21apt-get09:49:100
1227925816,25sleep112282-21diskstats09:39:301
1227925816,25sleep112282-21diskstats09:39:301
286625719,17sleep30-21swapper/312:34:273
2852125718,17sleep10-21swapper/112:14:151
279292575,17sleep30-21swapper/312:11:063
264392575,41sleep30-21swapper/308:40:253
2113125714,21sleep00-21swapper/011:49:230
195842575,40sleep00-21swapper/010:02:590
111032578,24sleep10-21swapper/111:19:081
111032578,24sleep10-21swapper/111:19:081
3136125614,23sleep131382-21apt-get07:14:141
296042565,11sleep30-21swapper/312:17:323
2855025615,25sleep128545-21apt-get08:49:111
275432566,41sleep00-21swapper/010:27:140
231352565,16sleep30-21swapper/311:54:393
2256725611,19sleep30-21swapper/308:29:183
1000025619,20sleep00-21swapper/011:14:260
769625514,24sleep17719-21ntp_states09:24:241
2486725512,25sleep024862-21apt-get10:19:100
231982555,39sleep00-21swapper/011:55:250
1965525514,28sleep319682-21awk10:03:553
16342556,38sleep10-21swapper/107:22:091
1265325513,26sleep112648-21apt-get11:24:081
1217825514,23sleep312164-21apt-get09:39:123
1217825514,23sleep312164-21apt-get09:39:123
993325414,25sleep19976-21ntp_states11:14:231
618225418,16sleep10-21swapper/111:03:571
380925416,22sleep10-21swapper/109:13:581
3176725415,24sleep031813-21ntpq07:14:330
3175725416,18sleep10-21swapper/110:39:261
2645325414,23sleep326454-21sshd12:07:023
2542625414,25sleep125427-21diskstats12:04:141
193225417,21sleep00-21swapper/012:33:560
903625313,23sleep19081-21irqstats09:29:161
413425314,23sleep34129-21apt-get09:14:123
2753225312,18sleep10-21swapper/108:44:261
1617925315,23sleep116205-21ntp_states11:34:221
1481625311,22sleep114880-21memory08:04:211
2119725212,22sleep121266-21sensors_temp11:49:241
1911425211,21sleep00-21swapper/011:44:140
683925112,22sleep06875-21ntp_states11:04:210
2173825114,22sleep321724-21apt-get10:09:123
161272519,16sleep10-21swapper/108:09:131
2521225014,18sleep10-21swapper/108:38:571
2075025013,15sleep30-21swapper/311:49:143
1892425015,19sleep30-21swapper/308:18:563
3129124916,18sleep00-21swapper/008:58:020
222152499,17sleep10-21swapper/111:54:131
214124910,23sleep12147-21diskstats07:24:151
990824812,22sleep09914-21diskstats07:49:140
2577424814,16sleep00-21swapper/008:39:210
1020124815,18sleep30-21swapper/307:49:213
146632479,15sleep30-21swapper/311:29:233
641924610,21sleep36416-21apt-get11:04:083
2973224617,15sleep30-21swapper/310:34:203
2597924514,16sleep10-21swapper/110:22:021
162752445,29sleep30-21swapper/309:50:393
147222449,15sleep10-21swapper/111:29:251
1195224410,21sleep012037-21cat07:54:270
37172438,17sleep03808-21gpgv09:13:570
51592429,18sleep30-21swapper/307:34:113
248652398,15sleep30-21swapper/310:19:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional