You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 01:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Apr 06, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
155652189819,1863sleep314629-1kworker/3:1H11:17:223
7190213165,1300sleep218729-1kworker/2:1H12:32:192
23935254018,506sleep023969-21/usr/sbin/munin11:42:190
13351242520,384sleep013400-21unin-run11:11:490
2608123578,334sleep226157-21awk10:07:182
2608123578,334sleep226157-21awk10:07:182
157223207,303sleep213441-21snmpd08:50:162
5033229416,259sleep35142-21sed12:26:513
27859226375,90sleep20-21swapper/210:12:352
14883225466,172sleep215007-21timerwakeupswit07:47:212
28712223389,71sleep20-21swapper/210:17:032
5109222758,48sleep20-21swapper/207:18:212
18309222676,129sleep20-21swapper/209:42:182
13873222621,184sleep214015-21unixbench_singl09:27:202
13873222621,184sleep214015-21unixbench_singl09:27:202
9625222315,193sleep09679-21grep07:32:080
24466220873,124sleep224479-21awk08:17:222
24466220873,124sleep224479-21awk08:17:222
450422058,181sleep24669-21cat07:16:512
4119220338,87sleep24162-21ntpq12:22:142
3103622037,182sleep215920-21kworker/u8:008:41:442
27943219912,169sleep328085-21sed11:56:513
25554219543,85sleep224996-21/usr/sbin/munin11:47:172
29967219214,135sleep230031-21latency12:02:112
25619216626,53sleep225691-21sendmail_mailtr08:22:182
25619216626,53sleep225691-21sendmail_mailtr08:22:182
3947216559,88sleep24107-21gpgv10:41:512
17735216413,89sleep217782-21ntpq07:57:142
20437216263,53sleep20-21swapper/208:07:042
20602215812,132sleep020682-21sendmail_mailtr11:32:170
20602215812,132sleep020682-21sendmail_mailtr11:32:170
19537215416,123sleep119586-21basename09:47:101
5943214210,48sleep20-21swapper/212:27:192
24049214216,62sleep20-21swapper/211:42:212
4928214027,91sleep20-21swapper/210:42:242
717321378,117sleep27264-21cut09:07:172
412021377,118sleep24191-21cat08:57:192
16131213618,57sleep216176-21ntpq07:52:132
17559213413,104sleep317590-21latency_hist09:41:503
1468521348,34sleep20-21swapper/209:32:042
8673213351,20sleep28820-21grep07:27:212
8673213351,20sleep28820-21grep07:27:212
297621328,111sleep00-21swapper/008:53:250
217521327,30sleep20-21swapper/212:17:052
13142213157,59sleep013193-21/usr/sbin/munin07:42:180
577213019,14sleep20-21swapper/212:12:042
3133321309,87sleep20-21swapper/208:42:052
15157212929,60sleep215330-21cat11:16:522
1155321296,63sleep20-21swapper/209:21:592
23248212630,52sleep20-21swapper/208:16:492
23248212630,52sleep20-21swapper/208:16:492
1311212631,25sleep2876-21/usr/sbin/munin10:32:142
31997212530,84sleep028388-1kworker/0:1H07:02:040
16256212548,42sleep20-21swapper/209:37:012
1307621245,29sleep20-21swapper/211:08:162
3216212316,89sleep13369-21awk10:37:221
12659212328,22sleep2928-21cyclictest07:42:042
1290212119,86sleep11310-21/usr/sbin/munin10:32:131
20199212013,85sleep20-21swapper/208:06:512
22357211910,19sleep20-21swapper/211:37:522
5594211713,14sleep10-21swapper/109:02:191
2874621178,95sleep228842-21cat11:57:192
9249211618,30sleep19250-21cron07:31:491
9249211618,30sleep19250-21cron07:31:491
31893211611,83sleep20-21swapper/210:27:092
954721158,89sleep20-21swapper/209:16:502
11224211516,86sleep311374-21unixbench_multi11:02:203
2148211415,23sleep10-21swapper/112:17:001
21339211411,46sleep20-21swapper/211:36:272
21339211411,46sleep20-21swapper/211:36:272
1696211458,15sleep20-21swapper/207:07:182
19115211329,62sleep20-21swapper/211:27:182
6531211235,60sleep20-21swapper/207:22:082
3582211213,84sleep03652-21sendmail_mailtr07:12:210
31160211124,75sleep20-21swapper/212:06:512
29173211116,73sleep328687-21/usr/sbin/munin10:17:163
28846211128,66sleep20-21swapper/208:32:202
11167211034,43sleep211178-21munin-run09:21:512
3320210927,67sleep23392-21latency_hist08:56:502
17937210915,78sleep318018-21cat07:57:193
2790321088,84sleep227941-21latency_hist11:56:502
7972210715,72sleep08000-21latency_hist09:11:510
28396210714,78sleep028467-21sh10:16:490
9663210612,80sleep09777-21timerwakeupswit10:57:200
20614210620,66sleep020645-21idleruntime-cro09:51:490
10876210626,58sleep20-21swapper/207:36:512
21365210515,72sleep021385-21/usr/sbin/munin09:52:170
27092210423,17sleep20-21swapper/208:27:152
14149210414,74sleep00-21swapper/011:12:180
7414210313,22sleep20-21swapper/210:52:052
2449521038,85sleep224532-21ps10:02:172
2449521038,85sleep224532-21ps10:02:172
19958210314,69sleep320048-21timerandwakeup09:47:203
544210116,57sleep10-21swapper/112:11:591
31625210014,71sleep331669-21ntpq12:07:133
20656210019,17sleep20-21swapper/209:51:512
534929816,67sleep15353-21diskstats12:27:051
379129831,45sleep01-21systemd10:41:490
1372229812,68sleep113723-21/usr/sbin/munin11:12:071
2397029716,22sleep10-21swapper/111:42:201
650929631,28sleep26629-21timerwakeupswit10:47:202
650929631,28sleep26629-21timerwakeupswit10:47:202
233752967,76sleep00-21swapper/009:57:220
231102969,49sleep223240-21sed09:57:202
1573529623,14sleep20-21swapper/211:17:292
179622955,28sleep20-21swapper/211:22:502
30772949,20sleep10-21swapper/107:12:061
3101329325,35sleep10-21swapper/112:06:501
3026129317,36sleep20-21swapper/210:22:042
322062916,11sleep30-21swapper/307:02:113
237162909,67sleep123718-21sshd10:01:271
1000029017,56sleep110041-21grep07:32:171
2962828910,68sleep028388-1kworker/0:1H10:17:250
2627928922,45sleep126308-21grep11:51:481
1430828912,64sleep313441-21snmpd09:30:443
1430828912,64sleep313441-21snmpd09:30:443
117372895,27sleep20-21swapper/211:05:582
898628822,16sleep20-21swapper/210:57:012
1698728824,34sleep117006-21apt-get11:22:001
1107628715,55sleep111166-21apt-key09:21:491
1105628719,51sleep011070-21apt-get09:21:480
250882869,63sleep025100-21cat08:22:000
250882869,63sleep025100-21cat08:22:000
3178228510,58sleep00-21swapper/012:07:180
209782858,37sleep20-21swapper/209:52:072
3245828328,30sleep10-21swapper/107:02:171
2528928319,48sleep325357-21ls11:47:103
1551828317,18sleep00-21swapper/007:51:510
563628210,58sleep30-21swapper/312:27:113
1126428122,27sleep30-21swapper/309:21:523
3072228015,46sleep330742-21/usr/sbin/munin10:22:173
80427925,34sleep2856-21sh07:06:502
749827921,37sleep17514-21fschecks_count10:52:081
205427913,51sleep30-21swapper/308:52:073
1336827922,23sleep00-21swapper/009:27:100
1336827922,23sleep00-21swapper/009:27:100
615527828,23sleep30-21swapper/307:21:493
55102785,62sleep00-21swapper/010:45:570
2954527717,46sleep129665-21date08:36:511
394827622,31sleep10-21swapper/108:57:141
855227517,35sleep18333-21apt-get12:36:511
243927515,30sleep30-21swapper/307:11:523
1897627526,25sleep00-21swapper/008:02:090
1548127512,46sleep115453-21apt-get11:17:161
729027412,43sleep07343-21interrupts12:32:300
642327421,26sleep00-21swapper/009:06:500
3002227421,23sleep10-21swapper/110:21:511
1101127420,22sleep00-21swapper/011:02:160
607427320,29sleep00-21swapper/010:47:100
607427320,29sleep00-21swapper/010:47:100
306362739,11sleep00-21swapper/012:02:420
275762737,54sleep025003-21kworker/0:211:53:330
2665527321,24sleep10-21swapper/111:52:091
99227223,29sleep00-21swapper/010:32:080
3217527213,45sleep331782-21/usr/sbin/munin10:27:143
2144727213,37sleep121387-21latency_hist11:36:501
2144727213,37sleep121387-21latency_hist11:36:501
3178427118,38sleep131792-21ntp_states08:42:161
3052327118,34sleep330657-21grep08:37:213
2381427123,27sleep323837-21irqstats08:17:103
2381427123,27sleep323837-21irqstats08:17:103
229427113,14sleep30-21swapper/310:36:513
1680227117,39sleep016945-21latency_hist11:21:510
1396827128,22sleep10-21swapper/107:46:511
121082715,29sleep20-21swapper/207:38:282
796727019,35sleep37970-21true09:11:503
61722705,11sleep20-21swapper/209:04:182
54772709,46sleep05533-21interrupts12:27:080
3147927015,24sleep30-21swapper/308:42:103
2715127016,24sleep10-21swapper/108:27:181
161427018,28sleep10-21swapper/107:07:171
3185326919,31sleep10-21swapper/112:07:191
255822696,15sleep10-21swapper/110:07:051
255822696,15sleep10-21swapper/110:07:051
831326821,28sleep10-21swapper/109:12:081
562826815,28sleep10-21swapper/110:46:511
490926825,24sleep30-21swapper/309:01:523
3220826813,21sleep00-21swapper/010:27:150
2884326817,35sleep328989-21/usr/sbin/munin11:57:203
2857326821,20sleep30-21swapper/308:32:123
250282686,13sleep10-21swapper/108:21:531
250282686,13sleep10-21swapper/108:21:531
217626816,36sleep32246-21/usr/sbin/munin12:17:063
2151726820,26sleep30-21swapper/309:52:203
1961826829,19sleep10-21swapper/108:02:221
195626813,37sleep11975-21apt-get08:52:011
1675426815,20sleep00-21swapper/009:37:150
702726726,27sleep36559-21apt-get12:32:003
596326718,31sleep36072-21perl10:47:093
596326718,31sleep36072-21perl10:47:093
2584026715,30sleep325852-21/usr/sbin/munin10:07:113
2584026715,30sleep325852-21/usr/sbin/munin10:07:113
2510726718,21sleep00-21swapper/011:47:070
1633326712,39sleep016412-21cat07:52:190
681526613,21sleep00-21swapper/007:22:150
2534426614,36sleep025484-21sed10:06:510
2534426614,36sleep025484-21sed10:06:510
144652668,14sleep10-21swapper/107:47:111
1410626618,27sleep30-21swapper/311:12:173
498126518,30sleep05012-21munin-run12:26:480
3240626513,28sleep10-21swapper/110:27:201
3109426516,34sleep031166-21date08:41:500
1982226519,27sleep10-21swapper/111:31:491
91026413,35sleep1980-21mailstats08:47:171
712626414,34sleep37253-21unixbench_multi07:22:223
2014426424,22sleep30-21swapper/308:06:483
1859826418,29sleep1281-21dbus-daemon08:01:501
1476926415,19sleep10-21swapper/109:32:071
557626317,26sleep35605-21munin-run10:46:493
2355826320,25sleep30-21swapper/311:42:093
2195526312,26sleep10-21swapper/108:11:591
2195526312,26sleep10-21swapper/108:11:591
1837826315,23sleep30-21swapper/309:42:193
175726325,20sleep00-21swapper/008:51:520
1374626313,13sleep10-21swapper/109:27:191
1374626313,13sleep10-21swapper/109:27:191
2566626211,24sleep00-21swapper/010:07:080
2566626211,24sleep00-21swapper/010:07:080
2489226215,21sleep00-21swapper/008:21:520
2489226215,21sleep00-21swapper/008:21:520
2216326217,20sleep30-21swapper/308:12:103
2216326217,20sleep30-21swapper/308:12:103
21792625,10sleep00-21swapper/007:09:100
193726213,20sleep00-21swapper/012:16:510
177422620,7sleep10-21swapper/109:41:511
1340126211,25sleep10-21swapper/111:11:501
85262618,13sleep00-21swapper/007:27:190
85262618,13sleep00-21swapper/007:27:190
2666226116,21sleep00-21swapper/008:27:050
2318326113,28sleep30-21swapper/311:41:493
205692616,37sleep30-21swapper/311:32:163
205692616,37sleep30-21swapper/311:32:163
182592616,17sleep30-21swapper/311:26:393
94672605,11sleep00-21swapper/009:16:350
840726019,19sleep30-21swapper/307:27:143
840726019,19sleep30-21swapper/307:27:143
499226019,15sleep00-21swapper/007:17:380
2835226014,30sleep128410-21irqstats11:57:091
2741726013,31sleep127104-21/usr/sbin/munin10:12:261
1675226015,24sleep30-21swapper/307:53:283
1421626016,25sleep314279-21df07:47:053
1160626021,22sleep10-21swapper/107:37:181
3021525916,27sleep130238-21ntp_states08:37:151
2967325915,25sleep329692-21apt-get12:02:003
2841425916,23sleep00-21swapper/008:32:090
280225914,24sleep30-21swapper/310:37:113
1846025916,20sleep00-21swapper/009:42:200
956225817,17sleep30-21swapper/309:16:513
652925812,25sleep16530-21diskstats07:22:071
32222587,11sleep30-21swapper/307:12:123
2393525821,17sleep10-21swapper/108:17:121
2393525821,17sleep10-21swapper/108:17:121
2732825716,23sleep30-21swapper/310:12:243
2294125716,22sleep30-21swapper/309:57:173
2189225713,29sleep121388-21apt-get11:36:581
1746525715,26sleep017472-21ntp_states11:22:150
1701425719,24sleep117029-21modprobe07:56:491
1209225715,26sleep312182-21grep09:22:203
1156625716,25sleep311572-21ntp_states07:37:163
749025618,22sleep00-21swapper/010:52:070
1108925613,25sleep011108-21apt-get07:37:010
88025516,23sleep3886-21ntp_states08:47:153
2469725517,19sleep10-21swapper/110:02:201
2469725517,19sleep10-21swapper/110:02:201
2174425516,23sleep00-21swapper/008:11:500
898725412,24sleep38982-21apt-get10:57:023
800325416,20sleep10-21swapper/109:11:521
3033325413,24sleep030338-21df_inode10:22:060
3026025413,25sleep130254-21apt-get10:22:031
2179325411,26sleep121807-21/usr/sbin/munin09:52:231
2172625413,25sleep30-21swapper/308:11:493
899025314,23sleep19057-21df10:57:041
519525318,19sleep30-21swapper/309:02:083
2261925313,18sleep00-21swapper/011:37:580
2104925311,20sleep10-21swapper/108:07:191
1602125316,17sleep00-21swapper/009:36:510
1054125314,22sleep110538-21apt-get11:02:011
490025212,23sleep14885-21apt-get07:17:071
2813525214,23sleep028137-21apt-get11:57:000
1853725211,18sleep00-21swapper/011:27:000
1764225213,23sleep017700-21memory07:57:120
2511225114,23sleep325108-21apt-get08:22:033
2511225114,23sleep325108-21apt-get08:22:033
1883325113,20sleep318927-21memory11:27:113
1273525116,18sleep30-21swapper/311:07:203
1158825114,17sleep10-21swapper/109:22:061
979925012,18sleep30-21swapper/307:32:113
826225012,20sleep30-21swapper/310:52:223
586625011,23sleep15861-21apt-get10:47:031
586625011,23sleep15861-21apt-get10:47:031
3133125012,22sleep031326-21apt-get08:42:030
1490125012,21sleep314950-21irqstats09:32:093
482424911,14sleep00-21swapper/009:01:490
47972495,34sleep10-21swapper/112:24:271
2333024911,15sleep00-21swapper/008:16:510
2333024911,15sleep00-21swapper/008:16:510
224824810,15sleep00-21swapper/012:17:070
83024711,21sleep3856-21irqstats12:12:133
2663424712,21sleep30-21swapper/308:26:593
1826724713,14sleep10-21swapper/109:42:171
1336724713,16sleep30-21swapper/307:42:213
1143424710,19sleep011070-21apt-get09:21:530
2443024611,15sleep30-21swapper/310:02:143
2443024611,15sleep30-21swapper/310:02:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional