You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 12:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 27, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32408273845,600sleep011-21ksoftirqd/020:52:590
18227252515,93sleep00-21swapper/021:48:220
1780925045,486sleep00-21swapper/023:29:550
29302241217,372sleep129332-21awk20:42:591
32591238519,349sleep332593-21/usr/sbin/munin00:18:103
946423788,355sleep19475-21ntpq23:03:281
15166237342,319sleep015210-21ntpq21:38:250
2782023639,299sleep027965-21latency_hist20:38:010
30038234211,317sleep230049-21ntpq20:43:242
2743231414,229sleep02873-21unixbench_multi20:58:280
2754123059,280sleep00-21swapper/022:18:170
2726922790,68sleep031163-21cyclictest22:18:000
32011224913,222sleep132019-21ntpq19:08:321
32011224913,222sleep132019-21ntpq19:08:321
5284223111,205sleep25343-21ps00:33:272
27530222955,77sleep00-21swapper/000:03:010
28409221416,184sleep228420-21grep20:38:242
11280221420,172sleep211417-21unixbench_singl23:08:292
22518221110,186sleep122633-21sed20:18:491
4403220812,180sleep00-21swapper/022:48:170
386822056,11sleep00-21swapper/022:45:420
1186922058,94sleep00-21swapper/021:28:190
2277421989,124sleep00-21swapper/022:03:110
1726421935,48sleep00-21swapper/021:44:020
3024218917,111sleep00-21swapper/000:28:000
3253121886,119sleep0412-1kworker/0:0H22:33:270
217072187114,60sleep021284-21apt-get20:18:030
2375021856,164sleep023791-21ntpq20:23:240
2375021856,164sleep023791-21ntpq20:23:240
1586218074,26sleep00-21swapper/022:38:240
2508217815,149sleep32518-21ntp_states20:58:233
12398217869,24sleep00-21swapper/019:48:200
5863217327,124sleep26000-21grep21:08:312
20768217223,132sleep320910-21latency_hist21:58:013
8255217175,74sleep00-21swapper/019:34:270
29184216815,89sleep029196-21modprobe00:07:580
29184216815,89sleep029196-21modprobe00:07:580
357121636,141sleep00-21swapper/019:18:530
16776216012,91sleep016794-21latency_hist20:03:130
16776216012,91sleep016794-21latency_hist20:03:130
30534215964,78sleep00-21swapper/020:43:550
10683215765,21sleep00-21swapper/019:43:150
10683215765,21sleep00-21swapper/019:43:150
20474215315,124sleep05763-1kworker/0:1H23:38:310
3037821519,106sleep030391-21awk00:08:340
3989214469,56sleep00-21swapper/021:03:250
29287214164,53sleep00-21swapper/022:23:260
22275213952,71sleep022320-21apt-config22:02:590
468213643,71sleep00-21swapper/020:53:110
9831213449,22sleep010006-21cut21:23:020
18800213249,71sleep018839-21wc23:33:300
6191212945,16sleep00-21swapper/000:38:010
21449212811,100sleep00-21swapper/023:43:110
20164212851,21sleep020305-21uptime21:53:300
3166821279,104sleep031793-21timerwakeupswit00:13:310
31815212612,88sleep00-21swapper/019:08:270
31815212612,88sleep00-21swapper/019:08:270
19974212317,82sleep00-21swapper/020:13:100
1107221236,64sleep00-21swapper/021:24:200
6487212222,20sleep00-21swapper/021:13:010
30102212221,84sleep230227-21timerwakeupswit00:08:322
12956211817,19sleep20-21swapper/221:33:022
23100211724,73sleep123198-21fschecks_time23:48:211
9723211519,72sleep39823-21apt-config21:23:013
20477211312,83sleep220543-21sendmail_mailqu20:13:292
3110721095,87sleep00-21swapper/022:28:310
31059210936,51sleep2278-21dbus-daemon19:08:012
16027210813,81sleep316075-21/usr/sbin/munin21:43:003
4861210734,40sleep25003-21grep19:23:312
2567721066,11sleep00-21swapper/023:56:190
17398210513,76sleep217420-21/usr/sbin/munin20:03:482
922621048,22sleep00-21swapper/023:03:220
18851210410,81sleep018894-21ntpq20:08:260
16789210325,46sleep116783-21apt-get23:28:141
17571210215,72sleep117587-21idleruntime-cro21:47:581
12106210219,47sleep20-21swapper/223:13:132
14141210116,62sleep213751-21/usr/sbin/munin19:53:222
16551210010,78sleep016597-21latency_hist23:28:010
15318210011,72sleep115324-21diskstats23:23:181
3115729919,59sleep30-21swapper/319:08:023
168129813,63sleep00-21swapper/019:13:320
606829729,51sleep16092-21apt00:37:581
2276429713,69sleep122838-21latency_hist23:48:001
180929714,67sleep11864-21fschecks_time00:23:211
1425829713,69sleep314321-21/usr/sbin/munin19:53:273
2409129614,60sleep124219-21ls22:08:011
1833629617,61sleep318341-21cpu23:33:173
799329522,57sleep18039-21idleruntime-cro21:17:591
45732959,62sleep05763-1kworker/0:1H00:33:010
2972429525,32sleep00-21swapper/019:03:150
255029517,26sleep20-21swapper/222:43:022
1978529123,46sleep119829-21ntpq21:53:231
2144829015,56sleep121468-21/usr/sbin/munin21:58:281
1884029010,65sleep118919-21mailstats23:33:311
262302898,65sleep026306-21sed20:33:010
243002889,66sleep024322-21latency_hist23:53:000
231228819,46sleep32340-21modprobe19:17:593
2114628812,62sleep121147-21sshd20:16:261
722028614,54sleep37161-21latency_hist22:58:003
71982868,66sleep06703-21/usr/sbin/munin21:13:300
1488828623,11sleep00-21swapper/019:56:070
147728627,35sleep11602-21grep00:23:011
3098128534,25sleep10-21swapper/119:08:001
2355828528,31sleep30-21swapper/323:48:313
184028519,49sleep21983-21cat20:58:022
1414328514,51sleep214152-21/usr/sbin/munin23:18:262
876628430,29sleep20-21swapper/223:03:002
332828418,34sleep20-21swapper/221:02:582
12332845,68sleep00-21swapper/000:20:330
281222837,14sleep00-21swapper/020:38:180
137642836,14sleep00-21swapper/023:18:160
127062838,13sleep10-21swapper/121:30:261
38912820,7sleep20-21swapper/200:28:302
1642628219,47sleep316494-21tune2fs21:43:203
106502828,54sleep20-21swapper/219:43:092
106502828,54sleep20-21swapper/219:43:092
1007728218,30sleep10-21swapper/121:23:121
89782817,62sleep09003-21fschecks_count19:38:180
643028110,12sleep00-21swapper/022:53:290
2460728110,58sleep1364-21snmpd20:27:411
2460728110,58sleep1364-21snmpd20:27:411
110952818,58sleep00-21swapper/023:08:270
3141828017,46sleep331450-21latency20:48:223
3002728030,24sleep10-21swapper/100:08:311
142902808,13sleep00-21swapper/021:35:540
1328128038,24sleep10-21swapper/121:33:211
442827918,40sleep34524-21fschecks_time22:48:203
1231427929,29sleep30-21swapper/319:48:193
1198027939,25sleep1278-21dbus-daemon19:48:001
1198027939,25sleep1278-21dbus-daemon19:48:001
714027827,30sleep17167-21munin-run22:57:581
43892787,52sleep00-21swapper/019:23:210
123132786,11sleep00-21swapper/023:13:200
2759527718,24sleep20-21swapper/222:18:192
219327715,46sleep21693-21/usr/sbin/munin00:23:302
1524227714,46sleep315305-21/usr/sbin/munin23:23:163
662527625,19sleep20-21swapper/222:53:312
39627626,30sleep2426-21munin-node19:13:012
39627626,30sleep2426-21munin-node19:13:012
178902768,12sleep00-21swapper/020:04:260
131427617,41sleep31401-21awk20:53:513
3272927522,28sleep10-21swapper/122:33:321
243362755,12sleep00-21swapper/022:08:200
1495127511,48sleep314995-21systemd-run23:22:593
1385927518,40sleep113871-21/usr/sbin/munin19:53:171
1337427521,36sleep313397-21munin-run23:17:593
81352746,12sleep00-21swapper/022:58:300
2901527421,25sleep10-21swapper/122:23:191
67082735,11sleep00-21swapper/019:29:260
1629027314,27sleep20-21swapper/221:43:112
1500327322,33sleep115089-21grep23:23:011
889527214,28sleep10-21swapper/119:38:151
653227224,26sleep10-21swapper/121:13:021
2707927231,25sleep20-21swapper/220:33:312
2640327223,31sleep20-21swapper/223:58:232
2622027219,31sleep30-21swapper/323:58:203
55492719,18sleep30-21swapper/321:08:243
2935327119,21sleep30-21swapper/320:43:013
2500527118,28sleep20-21swapper/220:28:182
2312527115,24sleep30-21swapper/322:04:153
910127013,39sleep29153-21fschecks_count23:03:202
353227017,33sleep13604-21latency00:28:211
2375727012,29sleep20-21swapper/222:04:272
95412698,12sleep20-21swapper/221:22:362
2958426916,36sleep329656-21switchtime22:23:333
1021426921,30sleep210307-21apt-config19:43:012
947826813,43sleep39515-21ps23:03:303
3115926819,36sleep131236-21cut00:13:201
2957326821,27sleep30-21swapper/300:08:213
154712689,16sleep30-21swapper/319:58:203
154712689,16sleep30-21swapper/319:58:203
134302685,12sleep30-21swapper/321:33:233
3187526713,24sleep30-21swapper/322:33:023
1853926722,24sleep20-21swapper/220:08:202
64626615,34sleep1725-21munin-node19:13:041
3249526619,24sleep20-21swapper/220:53:012
2149926619,30sleep321704-21latency_hist20:18:023
1020926613,22sleep20-21swapper/221:23:372
430126512,26sleep30-21swapper/319:23:203
256632656,14sleep30-21swapper/320:28:323
249992650,6sleep10-21swapper/123:53:241
2710226412,37sleep227103-21sshd22:17:262
2271026419,29sleep322727-21modprobe23:47:583
163342647,47sleep20-21swapper/219:59:202
163342647,47sleep20-21swapper/219:59:202
1222826416,29sleep212222-21apt-get19:48:142
1116726410,45sleep30-21swapper/321:25:323
63926320,22sleep20-21swapper/200:18:292
609426321,26sleep36133-21ntpq19:28:243
39372636,47sleep20-21swapper/222:46:372
2614626313,21sleep10-21swapper/122:13:221
2245826319,27sleep222652-21apt-config22:03:012
58372625,10sleep30-21swapper/300:35:023
494726215,17sleep20-21swapper/222:48:312
333626215,28sleep13403-21switchtime22:43:291
292192626,12sleep20-21swapper/222:23:242
2074626212,34sleep220690-21latency_hist21:58:002
1825826215,28sleep10-21swapper/121:48:231
570026111,21sleep10-21swapper/119:28:101
400726112,12sleep30-21swapper/300:28:323
397526112,33sleep24125-21cat19:23:022
338226120,24sleep13497-21sed21:03:011
3241726121,21sleep20-21swapper/200:18:012
2805326115,28sleep20-21swapper/200:03:232
2805326115,28sleep20-21swapper/200:03:232
2349226110,32sleep3360-21ntpd20:23:193
2349226110,32sleep3360-21ntpd20:23:193
2221626116,28sleep222250-21/usr/sbin/munin20:18:422
2172426112,21sleep20-21swapper/223:43:212
1882126111,32sleep30-21swapper/320:08:253
158792617,44sleep20-21swapper/221:41:052
116242617,11sleep30-21swapper/323:10:283
108582610,8sleep30-21swapper/319:43:333
108582610,8sleep30-21swapper/319:43:333
2830426013,24sleep30-21swapper/300:03:293
2830426013,24sleep30-21swapper/300:03:293
2767626020,23sleep127879-21gpgv00:03:021
2767626020,23sleep127879-21gpgv00:03:021
1585426013,30sleep215951-21/usr/sbin/munin23:23:312
807325922,19sleep30-21swapper/321:18:013
772225916,23sleep20-21swapper/219:33:282
767625913,21sleep10-21swapper/119:33:271
45652598,23sleep10-21swapper/122:48:221
394825915,26sleep3364-21snmpd21:03:243
2667025916,20sleep10-21swapper/123:58:301
2225025917,20sleep30-21swapper/322:02:583
1860425915,24sleep20-21swapper/223:33:242
443025814,29sleep14524-21users21:03:341
3101625814,24sleep131023-21ntp_states22:28:281
1747925816,17sleep20-21swapper/223:28:312
150625823,18sleep30-21swapper/322:38:233
114732585,11sleep10-21swapper/123:08:321
99925714,20sleep20-21swapper/222:38:012
91532576,11sleep30-21swapper/319:38:213
879425718,18sleep10-21swapper/121:18:311
727525722,17sleep30-21swapper/319:33:173
44825716,15sleep30-21swapper/319:13:023
44825716,15sleep30-21swapper/319:13:023
3009725718,21sleep10-21swapper/120:43:271
28442576,11sleep30-21swapper/300:26:223
2700625714,27sleep127078-21/usr/sbin/munin20:33:301
2644425713,30sleep326465-21dpkg20:33:123
262725715,24sleep32623-21apt-get19:18:113
255582575,10sleep30-21swapper/323:54:493
2345725723,19sleep20-21swapper/220:23:182
2345725723,19sleep20-21swapper/220:23:182
2120925715,25sleep121218-21/usr/sbin/munin23:43:001
462725613,17sleep10-21swapper/119:23:261
3130425618,23sleep231450-21awk22:28:332
254725615,26sleep12528-21/usr/sbin/munin22:43:011
2454925614,25sleep224542-21apt-get23:53:142
2039725624,17sleep320466-21mailstats23:38:303
1844425613,26sleep217837-21/usr/sbin/munin21:48:282
1739125616,25sleep117397-21ntp_states20:03:471
717725513,18sleep20-21swapper/221:13:282
581225514,21sleep20-21swapper/219:28:192
1742125516,18sleep30-21swapper/320:03:493
1578525514,25sleep115804-21ntp_states19:58:261
1578525514,25sleep115804-21ntp_states19:58:261
1244025515,22sleep112496-21irqstats19:48:211
44725415,23sleep132423-21apt-get20:53:081
394125423,16sleep10-21swapper/119:23:011
179442549,12sleep30-21swapper/321:48:173
2987325314,18sleep30-21swapper/320:43:213
2814225314,17sleep10-21swapper/122:18:301
2796725317,22sleep30-21swapper/322:18:283
2485925312,25sleep324947-21grep22:08:483
2297525312,25sleep222994-21apt-get23:48:092
1607925312,17sleep10-21swapper/121:43:021
1324125313,24sleep213267-21df21:33:182
314525213,25sleep33154-21needreboot22:43:233
782525114,19sleep10-21swapper/122:58:231
3242825117,18sleep10-21swapper/120:53:001
1716625111,20sleep30-21swapper/323:28:243
2491825012,22sleep124912-21apt-get20:28:131
2007725014,19sleep30-21swapper/320:13:173
3159024916,16sleep30-21swapper/300:13:303
2564124913,18sleep10-21swapper/122:13:011
321122479,25sleep30-21swapper/322:33:143
2472224711,15sleep20-21swapper/222:08:422
143824712,16sleep20-21swapper/219:13:262
1218324711,21sleep112189-21diskstats23:13:161
272212459,16sleep30-21swapper/322:18:003
2121924513,15sleep20-21swapper/223:43:012
1180524516,16sleep30-21swapper/321:28:183
878424413,15sleep30-21swapper/323:03:013
1903224315,15sleep10-21swapper/120:08:311
370324211,13sleep20-21swapper/221:03:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional