You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 17:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sun Mar 15, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31110249177,4898sleep029531-21kworker/0:111:01:260
2504429425,920sleep225045-21ntp_states08:58:332
27245268917,654sleep127290-21/usr/sbin/munin10:48:471
25471267518,643sleep125473-21sshd12:27:261
9752255585,394sleep09904-21unixbench_singl08:08:390
17417243321,395sleep117428-21/usr/sbin/munin11:59:071
2078323956,352sleep020818-21ntpq10:28:300
28676234215,313sleep228677-21sshd09:09:272
28676234215,313sleep228677-21sshd09:09:272
11061234196,202sleep011114-21cpuspeed09:58:250
9242233528,268sleep09371-21sort09:53:080
191102314138,88sleep00-21swapper/010:23:290
22694230139,149sleep00-21swapper/012:18:160
14318228224,234sleep114337-21apt-get11:53:051
16632227128,105sleep00-21swapper/011:58:540
17072223128,154sleep017213-21latency_hist10:18:080
4201222614,199sleep24212-21ntpq11:18:332
1693222421,14sleep00-21swapper/007:43:380
12195222028,176sleep00-21swapper/011:43:350
30921221459,138sleep031009-21xargs07:33:370
25061221010,155sleep00-21swapper/010:43:200
13459220671,24sleep00-21swapper/008:23:090
25178220516,155sleep00-21swapper/007:15:390
6791219347,53sleep00-21swapper/009:43:300
3435218825,20sleep03509-21systemctl11:18:060
16156218832,116sleep015940-21latency_hist11:58:090
16156218832,116sleep015940-21latency_hist11:58:090
28385218249,117sleep028524-21tail09:08:370
28385218249,117sleep028524-21tail09:08:370
27937217949,114sleep027338-21/usr/sbin/munin12:33:350
1966521708,127sleep019713-21cat08:43:080
708216916,65sleep0718-21modprobe07:43:060
1237216933,107sleep00-21swapper/009:28:080
10064216950,74sleep010159-21fschecks_time11:38:260
2591321668,135sleep026017-21cut12:28:260
2591321668,135sleep026017-21cut12:28:260
14331216547,68sleep00-21swapper/008:23:370
21579215724,16sleep00-21swapper/012:13:320
3728215636,57sleep00-21swapper/009:33:370
17267215541,77sleep00-21swapper/008:33:350
9138215125,80sleep09285-21unixbench_singl11:33:370
9138215125,80sleep09285-21unixbench_singl11:33:370
12773215128,39sleep20-21swapper/211:48:052
29338214451,75sleep00-21swapper/009:13:250
19997214431,97sleep020098-21fschecks_time08:43:240
1372121438,97sleep013848-21timerwakeupswit11:48:360
2551321416,29sleep00-21swapper/012:27:580
155721417,94sleep00-21swapper/009:28:240
8105213918,105sleep18132-21idleruntime-cro11:33:061
21539213718,101sleep121620-21grep08:48:261
27466213520,58sleep00-21swapper/009:08:080
2129921345,12sleep10-21swapper/110:28:391
16079213425,92sleep30-21swapper/310:13:313
5754213315,97sleep15819-21fschecks_time07:58:261
2327921338,103sleep00-21swapper/010:38:070
2327921338,103sleep00-21swapper/010:38:070
26822212738,73sleep026823-21/usr/sbin/munin10:48:170
4260212616,89sleep14353-21mailstats11:18:361
3088621248,59sleep00-21swapper/009:18:220
21896212435,49sleep00-21swapper/007:08:080
21896212435,49sleep00-21swapper/007:08:080
18040212317,89sleep318055-21cron08:38:063
1563921238,76sleep015686-21ntpq08:28:300
7715212213,87sleep37862-21latency_hist09:48:103
5446212235,47sleep05588-21date07:58:080
9577212121,78sleep39684-21grep09:53:273
6276211910,46sleep00-21swapper/007:58:370
2864321190,11sleep228651-21modprobe12:38:052
29559211817,86sleep129560-21sendmail-mta10:56:511
16476211617,81sleep216460-21apt-get11:58:232
7431211322,72sleep36949-21apt-get08:03:163
621921125,30sleep00-21swapper/011:23:440
3249521126,19sleep00-21swapper/011:04:210
8056211133,37sleep00-21swapper/008:03:540
2814211115,77sleep12857-21ls07:48:281
10560211130,45sleep010553-21apt-get08:13:230
22831211010,16sleep00-21swapper/008:53:100
6147210918,28sleep20-21swapper/209:43:082
29171210911,63sleep029211-21ntp_kernel_pll_07:28:320
295021079,85sleep02996-21ntpq07:48:310
2083121075,10sleep00-21swapper/012:13:030
20299210621,69sleep120344-21awk08:43:291
496221055,69sleep00-21swapper/011:22:300
799721045,31sleep00-21swapper/011:31:430
620210422,42sleep00-21swapper/011:08:210
2606121048,48sleep00-21swapper/007:18:300
28150210317,69sleep328162-21ntpq09:08:313
28150210317,69sleep328162-21ntpq09:08:313
28765210126,14sleep00-21swapper/010:53:300
1822021015,85sleep00-21swapper/010:18:530
595721005,28sleep00-21swapper/009:41:200
25893210022,14sleep00-21swapper/009:03:070
21827210046,35sleep31-21systemd07:08:073
21827210046,35sleep31-21systemd07:08:073
216729818,58sleep12315-21tail09:28:371
2770029719,17sleep00-21swapper/007:23:350
2241229723,15sleep00-21swapper/007:08:410
2241229723,15sleep00-21swapper/007:08:410
2145329720,61sleep00-21swapper/008:48:210
1309929712,37sleep00-21swapper/010:03:340
2335829615,67sleep323474-21timerwakeupswit12:18:363
3159029522,56sleep131622-21munin-plugin-st07:38:061
1657629414,18sleep30-21swapper/311:58:403
2038529212,24sleep20-21swapper/210:28:202
2701729022,42sleep20-21swapper/207:23:082
1541628929,31sleep20-21swapper/210:13:052
14732897,13sleep10-21swapper/107:43:351
2397128831,37sleep00-21swapper/010:38:360
359628748,19sleep30-21swapper/307:50:283
2276028713,58sleep222769-21unin-run08:53:082
1500128723,44sleep315127-21latency_hist08:28:083
1463428716,50sleep114635-21/usr/sbin/munin11:53:171
1463428716,50sleep114635-21/usr/sbin/munin11:53:171
538628623,44sleep35476-21grep11:23:263
141582868,35sleep00-21swapper/010:08:190
2905728517,44sleep228759-21/usr/sbin/munin07:28:302
2341128510,59sleep223460-21/usr/sbin/munin08:53:322
2239928333,30sleep122434-21sendmail12:18:051
3187928216,52sleep331884-21sort07:38:163
1424428215,51sleep114280-21/usr/sbin/munin10:08:241
256628112,55sleep12607-21ntpq11:13:311
1486828114,18sleep30-21swapper/311:53:273
1486828114,18sleep30-21swapper/311:53:273
2739928039,26sleep227401-21cron09:08:062
1733328018,30sleep10-21swapper/108:33:361
2965027913,51sleep229687-21sed09:13:312
2723527912,37sleep10-21swapper/107:23:191
2601827818,41sleep126071-21if_eth012:28:271
2601827818,41sleep126071-21if_eth012:28:271
2496127820,24sleep20-21swapper/212:23:402
1469227810,55sleep30-21swapper/310:08:343
1105427828,26sleep10-21swapper/108:13:401
1026827820,41sleep310282-21apt08:13:063
3256527719,32sleep30-21swapper/309:23:263
3246627715,42sleep132469-21/usr/sbin/munin09:23:241
3022527721,40sleep1222-21systemd-journal09:14:271
2639127715,48sleep226460-21/usr/sbin/munin12:28:372
2639127715,48sleep226460-21/usr/sbin/munin12:28:372
2048427715,38sleep219878-21/usr/sbin/munin08:43:342
2485127621,34sleep124891-21latency08:58:281
2383127619,40sleep224002-21fschecks_time07:13:152
147732768,14sleep20-21swapper/211:53:262
147732768,14sleep20-21swapper/211:53:262
118527626,33sleep21214-21munin-run09:28:052
588627521,40sleep35936-21sshd07:58:283
444027516,17sleep30-21swapper/309:38:083
424027527,27sleep30-21swapper/307:53:273
2640327518,37sleep226518-21latency_hist10:48:072
2277927517,42sleep122888-21timerandwakeup07:08:501
2277927517,42sleep122888-21timerandwakeup07:08:501
1964827512,46sleep219740-21users10:23:402
1344027518,41sleep313456-21wc08:23:073
1070927510,17sleep00-21swapper/009:57:440
815527425,25sleep30-21swapper/311:33:083
3157727425,25sleep30-21swapper/309:18:373
1929527419,25sleep10-21swapper/110:23:341
1633427420,37sleep116449-21timerwakeupswit10:13:371
1363627421,23sleep30-21swapper/311:48:353
1392127321,28sleep10-21swapper/110:08:071
1073827323,31sleep110786-21modprobe09:58:061
2622727116,36sleep126297-21/usr/sbin/munin07:18:351
231727119,38sleep22378-21users09:28:382
1859827114,26sleep30-21swapper/310:23:083
181022716,29sleep00-21swapper/012:03:220
939127026,23sleep10-21swapper/108:08:311
3214627022,29sleep232272-21latency_hist09:23:092
286472700,7sleep20-21swapper/210:53:272
948126920,30sleep29475-21apt-get09:53:222
488326919,37sleep14912-21cpu09:38:551
403626923,22sleep30-21swapper/311:18:303
2532826926,26sleep325459-21sed08:58:383
1934526917,21sleep30-21swapper/310:23:363
1188326923,23sleep10-21swapper/111:43:281
758726820,28sleep27728-21sed11:28:382
2549326822,19sleep30-21swapper/307:18:083
1032126814,35sleep20-21swapper/208:13:092
2949526725,20sleep30-21swapper/307:28:393
2611426725,21sleep10-21swapper/109:03:161
2265526729,20sleep30-21swapper/307:08:483
2265526729,20sleep30-21swapper/307:08:483
1591726718,27sleep20-21swapper/208:28:362
1570726727,19sleep20-21swapper/210:13:212
1113826714,35sleep211207-21/usr/sbin/munin08:13:412
526426620,24sleep30-21swapper/309:39:053
3079826627,20sleep20-21swapper/207:33:342
2818526613,37sleep328328-21ls10:53:093
1726426617,28sleep117283-21apt-get10:18:171
1340026618,30sleep213432-21munin-run08:23:062
1050626618,25sleep20-21swapper/211:38:362
7112659,15sleep10-21swapper/111:08:261
2841126524,20sleep10-21swapper/112:35:261
2083326518,32sleep220867-21awk12:13:052
1811126519,28sleep118114-21/usr/sbin/munin12:03:231
769726423,18sleep20-21swapper/209:48:092
3173626416,25sleep10-21swapper/111:03:251
2327326422,20sleep20-21swapper/212:18:352
1408026415,27sleep20-21swapper/208:23:312
1269426410,41sleep212705-21awk10:03:242
790826318,28sleep17927-21apt-get09:48:161
2741726313,34sleep227482-21swap10:48:522
205252638,45sleep30-21swapper/312:09:073
2982726220,20sleep30-21swapper/309:13:363
141522626,12sleep10-21swapper/108:23:341
126112626,15sleep30-21swapper/310:03:213
614526123,20sleep20-21swapper/207:58:352
537726115,22sleep10-21swapper/111:23:241
37626120,18sleep30-21swapper/311:08:073
355126113,33sleep23697-21cat11:18:082
3124226118,24sleep20-21swapper/211:03:062
2721026119,25sleep227227-21apt07:23:162
2651126119,25sleep326550-21ntpq09:03:313
2511626117,25sleep325171-21users12:23:413
2380226111,32sleep223886-21memory10:38:302
2379726118,18sleep10-21swapper/107:13:141
2361826120,21sleep10-21swapper/112:18:381
2361326124,17sleep30-21swapper/308:53:373
2351826116,25sleep323585-21df10:38:233
2277026122,19sleep30-21swapper/308:53:093
1211726118,25sleep112136-21apt-get08:18:171
1057726122,18sleep30-21swapper/311:38:373
225626012,24sleep30-21swapper/307:48:063
2202426013,33sleep122027-21entropy10:33:241
2202426013,33sleep122027-21entropy10:33:241
99625919,24sleep2990-21apt-get07:43:202
313382595,11sleep30-21swapper/307:34:533
2921225914,30sleep129238-21ntp_states07:28:331
101612595,11sleep10-21swapper/111:38:271
972825813,16sleep20-21swapper/211:38:072
972825813,16sleep20-21swapper/211:38:072
324025816,22sleep23390-21grep07:48:372
2994825814,24sleep129967-21apt-get10:58:161
2788025812,28sleep328002-21sed07:23:383
245025817,21sleep30-21swapper/309:29:283
2225725816,25sleep222352-21perl07:08:392
2225725816,25sleep222352-21perl07:08:392
216625813,20sleep30-21swapper/311:13:213
591525717,17sleep20-21swapper/211:23:382
2533125713,26sleep325363-21/usr/sbin/munin10:43:283
224122576,12sleep30-21swapper/310:33:343
224122576,12sleep30-21swapper/310:33:343
221282575,11sleep20-21swapper/212:14:432
2112125713,25sleep321117-21apt-get12:13:193
1550425712,20sleep10-21swapper/108:28:271
121602576,10sleep30-21swapper/310:01:023
477625620,16sleep20-21swapper/207:53:382
40952566,16sleep20-21swapper/209:35:452
342925617,24sleep13474-21/usr/sbin/munin09:33:301
2477325616,20sleep124792-21munin-run10:43:051
2057825616,18sleep30-21swapper/308:43:353
1849125612,24sleep20-21swapper/212:03:322
1728625614,26sleep217283-21apt-get10:18:182
86092558,17sleep10-21swapper/108:06:421
3244725519,21sleep20-21swapper/207:38:352
2902625517,21sleep30-21swapper/310:53:383
2577525516,18sleep30-21swapper/307:18:253
2300425515,23sleep123001-21apt-get08:53:191
1957825514,24sleep119572-21apt-get12:08:201
1326825514,27sleep213037-21/usr/sbin/munin11:48:252
877325315,22sleep28879-21latency_hist08:08:082
2817125321,16sleep10-21swapper/109:08:341
2817125321,16sleep10-21swapper/109:08:341
2634125318,20sleep10-21swapper/110:48:061
2065225315,19sleep10-21swapper/107:03:251
2065225315,19sleep10-21swapper/107:03:251
1182725314,23sleep311899-21systemd-run08:18:073
3197525215,21sleep131978-21df07:38:221
2581625213,24sleep325883-21/usr/sbin/munin12:28:213
2581625213,24sleep325883-21/usr/sbin/munin12:28:213
2203025214,16sleep20-21swapper/210:33:252
2203025214,16sleep20-21swapper/210:33:252
2075325219,12sleep30-21swapper/310:28:293
196122526,34sleep30-21swapper/308:43:033
192772526,34sleep10-21swapper/108:38:451
1654625216,15sleep30-21swapper/308:33:093
1113625117,20sleep20-21swapper/209:58:272
976225014,15sleep30-21swapper/311:38:093
976225014,15sleep30-21swapper/311:38:093
846325012,22sleep18469-21df_inode11:33:221
846325012,22sleep18469-21df_inode11:33:221
3152625011,23sleep231522-21apt-get11:03:182
2071225015,15sleep20-21swapper/207:03:262
2071225015,15sleep20-21swapper/207:03:262
193025013,22sleep32008-21cat11:13:083
1915025014,19sleep219258-21users08:38:382
1803925014,22sleep318100-21df12:03:213
16912505,36sleep20-21swapper/211:10:412
808024912,21sleep27909-21/usr/sbin/munin09:48:252
1196324913,21sleep111964-21users09:58:421
815124813,21sleep38200-21irqstats09:48:263
638524812,18sleep10-21swapper/109:43:191
3037824819,14sleep30-21swapper/310:58:303
3083424712,19sleep130829-21apt-get09:18:191
2785724610,15sleep30-21swapper/312:33:343
1211624613,14sleep30-21swapper/311:43:343
172372446,14sleep20-21swapper/208:33:332
46982439,14sleep10-21swapper/107:53:371
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional