You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 17:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Apr 17, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30746274714,717sleep230757-21/usr/sbin/munin09:26:342
864927366,691sleep18695-21ntpq11:41:351
864927366,691sleep18695-21ntpq11:41:351
1199626576,635sleep112029-21ntpq11:51:441
2059426366,595sleep120630-21ntpq10:36:341
2059426366,595sleep120630-21ntpq10:36:341
29615262615,592sleep229734-21/usr/sbin/munin09:21:452
29615262615,592sleep229734-21/usr/sbin/munin09:21:452
18354246783,205sleep118382-21modprobe10:31:091
72452398107,204sleep17395-21unixbench_singl08:11:421
27714237837,293sleep127735-21awk11:01:091
2111723345,316sleep215044-1kworker/2:0H08:56:292
2111723345,316sleep215044-1kworker/2:0H08:56:292
3265123047,259sleep1324-21grep09:31:421
21069227779,179sleep125884-21systemd-journal07:16:101
21069227779,179sleep125884-21systemd-journal07:16:101
670225818,221sleep2709-21ntpq07:51:362
10277225252,110sleep125884-21systemd-journal10:02:311
2910922510,7sleep10-21swapper/109:21:351
2910922510,7sleep10-21swapper/109:21:351
9091223837,107sleep19235-21latency_hist10:01:131
26229221533,144sleep126373-21latency_hist10:56:121
28324221014,182sleep028363-21irqstats07:36:320
28324221014,182sleep028363-21irqstats07:36:320
28091220611,180sleep228083-21apt07:36:262
28091220611,180sleep228083-21apt07:36:262
285892204104,79sleep10-21swapper/109:21:121
25490220024,153sleep325630-21tail07:27:463
19168219984,90sleep10-21swapper/112:16:111
12159219812,119sleep112210-21latency_hist10:11:111
12159219812,119sleep112210-21latency_hist10:11:111
1542219325,146sleep00-21swapper/007:56:100
15242219331,122sleep115386-21tail08:36:411
27231218829,120sleep127352-21ls07:32:451
26068218826,150sleep126107-21ntpq09:11:351
7896218612,159sleep07922-21proc_pri09:56:480
22377218131,58sleep10-21swapper/110:41:391
649218075,22sleep10-21swapper/107:51:351
16835216915,138sleep316853-21wc10:26:103
16835216915,138sleep316853-21wc10:26:103
1449215837,29sleep10-21swapper/109:36:341
22671215647,45sleep10-21swapper/109:01:291
1868215616,122sleep21873-21entropy11:21:292
17591215535,65sleep10-21swapper/107:06:121
17591215535,65sleep10-21swapper/107:06:121
30044215144,85sleep10-21swapper/111:06:381
1757321508,102sleep117621-21latency_hist12:11:101
2962214211,109sleep00-21swapper/009:41:340
18456214110,32sleep117-21ksoftirqd/107:06:411
18456214110,32sleep117-21ksoftirqd/107:06:411
2523221386,12sleep20-21swapper/210:51:332
12582213810,14sleep00-21swapper/010:11:300
22010213726,100sleep113441-21snmpd12:23:341
14068213718,28sleep10-21swapper/110:16:281
14068213718,28sleep10-21swapper/110:16:281
12690213722,90sleep212712-21modprobe08:31:092
4440213626,99sleep113441-21snmpd11:29:151
2487721358,113sleep125002-21unixbench_singl09:06:421
5259213420,13sleep10-21swapper/109:49:221
3412213024,34sleep10-21swapper/108:01:241
24052212840,68sleep10-21swapper/112:31:251
504721278,29sleep10-21swapper/108:06:291
504721278,29sleep10-21swapper/108:06:291
20015212638,51sleep119380-21/usr/sbin/munin12:16:401
11934212642,69sleep112012-21mailstats08:26:401
20998212014,25sleep10-21swapper/108:56:201
20998212014,25sleep10-21swapper/108:56:201
20408212018,18sleep10-21swapper/108:55:231
23403211914,90sleep323435-21tune2fs10:46:293
765211714,84sleep10-21swapper/111:16:391
765211714,84sleep10-21swapper/111:16:391
21565211731,17sleep10-21swapper/107:16:301
21565211731,17sleep10-21swapper/107:16:301
4964211613,86sleep14990-21fschecks_count11:31:291
26306211628,66sleep326413-21cat09:11:413
7619211337,14sleep10-21swapper/109:56:411
787621127,73sleep113441-21snmpd11:40:091
31665211230,69sleep131731-21wc07:46:381
2022211281,16sleep10-21swapper/111:21:321
30350211115,78sleep030494-21unixbench_singl07:41:410
28705211114,75sleep10-21swapper/107:36:411
28705211114,75sleep10-21swapper/107:36:411
30375211033,48sleep330372-21apt-get09:26:233
24422211012,81sleep224453-21/usr/sbin/munin12:31:352
3093210921,69sleep03187-21latency_hist11:26:110
29350210912,80sleep329399-21unin-run07:41:113
29350210912,80sleep329399-21unin-run07:41:113
24770210927,69sleep124711-21apt-get07:27:001
16477210928,42sleep10-21swapper/108:41:341
8305210815,75sleep37797-21apt-get08:16:203
8305210815,75sleep37797-21apt-get08:16:203
15975210828,63sleep10-21swapper/112:06:101
15093210613,80sleep315162-21/usr/sbin/munin08:36:393
9237210514,74sleep28791-21apt-get10:01:142
23484210510,24sleep10-21swapper/107:21:431
25407210414,70sleep325545-21latency_hist12:36:123
5136210319,31sleep30-21swapper/308:06:313
5136210319,31sleep30-21swapper/308:06:313
18540210315,70sleep218693-21awk07:06:422
18540210315,70sleep218693-21awk07:06:422
12396210318,42sleep10-21swapper/110:11:231
11597210346,27sleep00-21swapper/011:51:350
10042210324,14sleep10-21swapper/111:46:311
4724210216,71sleep04854-21sort08:06:120
30463210222,38sleep10-21swapper/109:26:291
2590210221,15sleep10-21swapper/109:41:231
313210121,59sleep3406-21fschecks_time11:16:293
313210121,59sleep3406-21fschecks_time11:16:293
24500210112,75sleep024509-21uname12:31:370
9881210017,15sleep10-21swapper/108:21:231
9881210017,15sleep10-21swapper/108:21:231
267829914,69sleep32714-21fschecks_count09:41:293
2905529839,48sleep10-21swapper/111:03:311
2905529839,48sleep10-21swapper/111:03:311
1958829817,67sleep219680-21cpu07:11:182
1958829817,67sleep219680-21cpu07:11:182
2302629719,62sleep323046-21modprobe10:46:093
833029615,59sleep38374-21fschecks_count11:41:283
833029615,59sleep38374-21fschecks_count11:41:283
2704529512,65sleep227155-21latency_hist09:16:122
2338829518,15sleep10-21swapper/110:46:271
2970229420,13sleep10-21swapper/107:41:251
1999229320,53sleep320049-21if_eth007:11:313
1999229320,53sleep320049-21if_eth007:11:313
1635829311,68sleep316447-21fschecks_time12:06:283
3167929228,38sleep10-21swapper/111:11:391
1827029224,44sleep10-21swapper/108:46:391
155062929,66sleep10-21swapper/110:21:201
155062929,66sleep10-21swapper/110:21:201
141662925,56sleep10-21swapper/108:35:161
3210729121,43sleep232165-21sshd09:31:302
3149529112,62sleep031532-21ntp_kernel_pll_11:11:330
2648429114,60sleep226003-21apt-get07:31:212
2250429120,19sleep20-21swapper/212:26:262
3219829012,55sleep00-21swapper/009:31:320
1930129012,55sleep019339-21fschecks_count08:51:300
987828930,35sleep20-21swapper/211:46:282
259928920,42sleep32748-21unixbench_singl07:56:423
259928920,42sleep32748-21unixbench_singl07:56:423
381828816,57sleep33901-21cut11:26:383
2113528820,51sleep021170-21idleruntime12:21:310
1284128831,35sleep212868-21modprobe11:56:092
861528516,51sleep08219-21/usr/sbin/munin11:41:340
861528516,51sleep08219-21/usr/sbin/munin11:41:340
2273128515,54sleep222793-21fw_packets09:01:302
2095628514,53sleep320929-21/usr/sbin/munin12:21:263
84842846,41sleep18549-21fw_packets08:16:301
84842846,41sleep18549-21fw_packets08:16:301
1832628414,49sleep018396-21/usr/sbin/munin12:11:380
1016228415,46sleep310185-21/usr/sbin/munin11:46:333
1601328214,12sleep20-21swapper/210:21:372
1601328214,12sleep20-21swapper/210:21:372
2350028113,50sleep022560-21/usr/sbin/munin09:01:440
1990328127,17sleep119929-21munin-run10:36:101
1942828120,44sleep219485-21/usr/sbin/munin08:51:332
985728011,24sleep30-21swapper/308:21:213
985728011,24sleep30-21swapper/308:21:213
2853828021,39sleep328561-21modprobe09:21:103
227782808,54sleep10-21swapper/112:26:331
311022799,51sleep30-21swapper/311:11:243
2931227816,48sleep329347-21sed11:06:113
2931227816,48sleep329347-21sed11:06:113
2432127812,50sleep224328-21apt-key07:26:122
2419527813,48sleep024236-21http07:26:100
2115127812,52sleep021242-21latency_hist07:16:130
2115127812,52sleep021242-21latency_hist07:16:130
1764427826,26sleep20-21swapper/207:06:132
1764427826,26sleep20-21swapper/207:06:132
851527717,38sleep28553-21wc11:41:312
851527717,38sleep28553-21wc11:41:312
25912779,51sleep20-21swapper/209:41:242
21142776,33sleep10-21swapper/107:56:321
21142776,33sleep10-21swapper/107:56:321
1891127713,47sleep218982-21sort10:31:302
2696527610,42sleep00-21swapper/009:16:100
2298327614,45sleep323041-21sendmail_mailqu12:26:393
1880627515,42sleep018862-21idleruntime10:31:280
3118727410,50sleep031189-21http07:46:250
2099527412,47sleep320489-21apt-get08:56:183
2099527412,47sleep320489-21apt-get08:56:183
3086427320,26sleep20-21swapper/211:11:112
241362738,13sleep00-21swapper/010:46:430
2082927323,32sleep30-21swapper/310:36:403
2082927323,32sleep30-21swapper/310:36:403
1531927324,29sleep30-21swapper/312:01:393
705627221,25sleep30-21swapper/308:11:403
2715327214,42sleep027229-21switchtime07:32:440
527927116,28sleep30-21swapper/311:31:353
3180027120,26sleep30-21swapper/307:46:403
3018327115,36sleep230246-21/usr/sbin/munin07:41:392
2904227124,26sleep30-21swapper/309:21:343
2904227124,26sleep30-21swapper/309:21:343
2612327121,22sleep20-21swapper/207:31:132
1665727026,23sleep00-21swapper/012:06:330
1237227018,23sleep30-21swapper/310:11:213
1541326918,33sleep015563-21sed12:01:400
736626821,25sleep20-21swapper/211:36:472
2972226811,43sleep329723-21diskstats07:41:273
2964026823,23sleep30-21swapper/311:06:283
2755326816,25sleep20-21swapper/209:16:322
2432926812,14sleep20-21swapper/209:06:312
2147626826,20sleep00-21swapper/007:16:290
2147626826,20sleep00-21swapper/007:16:290
1424026822,30sleep214241-21cron08:36:102
718926720,32sleep27226-21idleruntime-cro09:56:092
3002026716,36sleep029530-21/usr/sbin/munin11:06:370
1334826714,38sleep313360-21netstat08:31:353
1112526716,28sleep20-21swapper/210:06:312
1112526716,28sleep20-21swapper/210:06:312
50126618,24sleep20-21swapper/211:16:322
50126618,24sleep20-21swapper/211:16:322
212442665,16sleep00-21swapper/010:38:070
212442665,16sleep00-21swapper/010:38:070
1834826617,24sleep20-21swapper/208:46:402
892926518,31sleep09090-21cat10:01:120
2909626518,23sleep00-21swapper/009:21:350
2909626518,23sleep00-21swapper/009:21:350
2620226516,36sleep026227-21latency_hist10:56:110
1940626516,26sleep20-21swapper/212:16:242
1762726515,27sleep20-21swapper/212:11:112
1752526514,35sleep013441-21snmpd07:06:110
1752526514,35sleep013441-21snmpd07:06:110
164552657,45sleep00-21swapper/010:21:480
164552657,45sleep00-21swapper/010:21:480
1650926417,30sleep216551-21/usr/sbin/munin08:41:352
141242645,27sleep10-21swapper/111:57:171
1300726419,31sleep20-21swapper/210:11:402
1151826418,32sleep311542-21fschecks_count08:26:303
678326313,34sleep06861-21sed08:11:330
620126319,27sleep26231-21munin-run11:36:102
388926313,32sleep23947-21sendmail_mailqu08:01:382
37942635,11sleep20-21swapper/211:26:372
3107526315,32sleep231092-21apt11:11:192
188642636,12sleep20-21swapper/212:13:172
185426317,24sleep00-21swapper/007:56:230
185426317,24sleep00-21swapper/007:56:230
1601526316,31sleep216026-21munin-run12:06:112
125642636,48sleep00-21swapper/008:29:340
976926215,27sleep39775-21ntp_states10:01:443
800826214,32sleep38145-21cat11:41:123
2358626212,34sleep223604-21/usr/sbin/munin10:46:322
1936426214,22sleep30-21swapper/310:31:403
178426211,37sleep31805-21cpu11:21:263
931726119,18sleep00-21swapper/010:01:330
230426120,29sleep0588-21cron09:41:100
2165726113,33sleep321742-21cat07:16:323
2165726113,33sleep321742-21cat07:16:323
213126117,20sleep20-21swapper/207:56:332
213126117,20sleep20-21swapper/207:56:332
1915826115,27sleep019300-21munin-setwarn07:11:120
1915826115,27sleep019300-21munin-setwarn07:11:120
1778026113,30sleep217927-21unixbench_multi10:26:422
1708626124,28sleep00-21swapper/008:42:340
1383726114,24sleep30-21swapper/311:56:393
112232617,12sleep00-21swapper/010:06:330
112232617,12sleep00-21swapper/010:06:330
883926014,26sleep30-21swapper/310:01:113
627526015,25sleep00-21swapper/008:11:120
627526015,25sleep00-21swapper/008:11:120
2671826013,27sleep00-21swapper/010:56:320
203492605,10sleep30-21swapper/308:54:363
1978526019,18sleep30-21swapper/312:16:343
1346826010,13sleep00-21swapper/008:31:400
1000126016,29sleep010041-21/usr/sbin/munin11:46:300
2708425916,23sleep30-21swapper/307:32:433
2257825914,20sleep00-21swapper/012:26:280
2146025920,21sleep20-21swapper/207:16:272
2146025920,21sleep20-21swapper/207:16:272
2071625915,18sleep00-21swapper/012:21:110
532025811,22sleep20-21swapper/208:06:342
532025811,22sleep20-21swapper/208:06:342
2800325817,26sleep00-21swapper/011:01:240
2800325817,26sleep00-21swapper/011:01:240
239942585,11sleep00-21swapper/009:06:120
2186425815,20sleep20-21swapper/210:41:282
1807225817,20sleep30-21swapper/308:46:333
1226625816,17sleep30-21swapper/311:51:503
113125817,25sleep31191-21/usr/sbin/munin09:36:283
668925716,25sleep26738-21sshd08:11:312
487825713,28sleep04873-21apt-get11:31:240
2598725714,25sleep026029-21gpgv07:31:110
964425617,16sleep20-21swapper/208:21:122
964425617,16sleep20-21swapper/208:21:122
3273525614,26sleep332731-21apt-get07:51:253
2148625614,22sleep20-21swapper/212:21:412
1407925623,19sleep20-21swapper/210:16:292
1407925623,19sleep20-21swapper/210:16:292
991325514,17sleep00-21swapper/008:21:260
991325514,17sleep00-21swapper/008:21:260
807725521,18sleep30-21swapper/309:56:513
45232556,10sleep30-21swapper/308:04:173
3173125518,24sleep00-21swapper/009:31:120
3159625514,24sleep231641-21ntp_states07:46:352
2414425517,22sleep324140-21apt-get09:06:233
1619025514,19sleep30-21swapper/308:41:283
1328025515,24sleep013364-21fw_packets11:56:280
104425512,26sleep21038-21apt-get09:36:232
80725413,18sleep30-21swapper/309:36:123
53332545,40sleep30-21swapper/309:50:213
376225413,23sleep03786-21ntp_states11:26:350
2728525419,20sleep00-21swapper/009:16:260
2279425415,17sleep30-21swapper/309:01:313
2824425313,25sleep228273-21irqstats11:01:302
2824425313,25sleep228273-21irqstats11:01:302
2293925319,18sleep30-21swapper/307:21:283
218302538,16sleep00-21swapper/008:56:430
218302538,16sleep00-21swapper/008:56:430
1784625313,24sleep317843-21apt-get07:06:223
1784625313,24sleep317843-21apt-get07:06:223
1732625311,16sleep00-21swapper/010:26:320
1673125320,16sleep30-21swapper/307:01:383
1673125320,16sleep30-21swapper/307:01:383
1417225314,20sleep30-21swapper/310:16:303
1417225314,20sleep30-21swapper/310:16:303
72032529,30sleep00-21swapper/011:36:450
460425216,18sleep20-21swapper/209:46:372
3268725213,23sleep032683-21apt-get11:16:220
3268725213,23sleep032683-21apt-get11:16:220
2644325212,24sleep226440-21apt-get10:56:212
1727025220,17sleep30-21swapper/310:26:313
1360525213,25sleep213614-21ntpq11:56:342
811325112,23sleep00-21swapper/008:16:130
811325112,23sleep00-21swapper/008:16:130
1455625112,22sleep014550-21apt-get08:36:240
1398125112,24sleep013976-21apt-get10:16:220
1398125112,24sleep013976-21apt-get10:16:220
1094825110,17sleep30-21swapper/310:06:283
1094825110,17sleep30-21swapper/310:06:283
988225012,22sleep29876-21apt-get08:21:242
988225012,22sleep29876-21apt-get08:21:242
2668225013,15sleep30-21swapper/310:56:313
2493424911,21sleep324943-21df_inode10:51:263
2202424911,21sleep022039-21irqstats10:41:300
878424814,20sleep28838-21apt-config10:01:102
2029424812,21sleep220303-21diskstats10:36:272
2029424812,21sleep220303-21diskstats10:36:272
1892424812,19sleep00-21swapper/008:51:100
544524610,20sleep05415-21apt-get09:51:110
145982458,19sleep20-21swapper/208:36:262
1908824412,14sleep30-21swapper/312:16:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional