You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 13:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Aug 22, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
36402152310,1497sleep322152-1kworker/3:1H21:44:503
17978262822,515sleep318110-21/usr/sbin/munin19:04:543
2667724766,434sleep0367-21snmpd19:33:010
1322240152,329sleep01384-21fschecks_time23:19:420
1322240152,329sleep01384-21fschecks_time23:19:420
1518238082,201sleep01521-21cron21:39:220
3095232868,245sleep03118-21dpkg21:44:220
3095232868,245sleep03118-21dpkg21:44:220
1348923150,55sleep018658-21cyclictest20:34:220
26551230638,253sleep226557-21df_inode21:14:402
60322947,275sleep0367-21snmpd23:15:260
60322947,275sleep0367-21snmpd23:15:260
1681822816,260sleep00-21swapper/022:24:530
10525227511,217sleep010647-21timerwakeupswit22:04:540
17368226461,144sleep017430-21sendmail_mailqu20:44:490
17368226461,144sleep017430-21sendmail_mailqu20:44:490
14290225311,121sleep014406-21latency_hist22:19:240
14290225311,121sleep014406-21latency_hist22:19:240
188912241143,83sleep018923-21munin-run22:34:220
7279223748,128sleep00-21swapper/023:39:250
1700223532,142sleep01837-21sort19:54:580
26896221223,171sleep026931-21modprobe22:59:220
8822221014,106sleep09010-21apt-config20:19:250
11936220911,119sleep0367-21snmpd20:29:230
8778220819,170sleep38924-21latency_hist23:44:243
2096322069,183sleep322152-1kworker/3:1H22:39:303
842220529,138sleep0981-21latency_hist19:54:250
2114922046,163sleep00-21swapper/020:57:580
1188122039,181sleep011946-21sendmail_mailst22:09:510
3184522009,132sleep031970-21cat21:30:010
3184522009,132sleep031970-21cat21:30:010
7136219617,158sleep27214-21unin-run23:39:232
1416321967,176sleep014227-21mailstats23:59:520
30034219119,156sleep130148-21latency_hist19:44:251
16233217852,113sleep00-21swapper/000:06:060
16233217852,113sleep00-21swapper/000:06:060
22113217617,142sleep322140-21latency_hist19:19:233
2568217417,141sleep22711-21latency_hist23:24:242
2568217417,141sleep22711-21latency_hist23:24:242
28785216521,123sleep328923-21head21:19:543
15745215665,78sleep015781-21ntpq20:39:490
1463721520,82sleep114641-21unixbench_singl20:34:521
31745214445,73sleep00-21swapper/019:49:240
22194214417,110sleep122266-21date19:19:251
2923121438,85sleep00-21swapper/023:04:510
24430214217,110sleep224437-21ntpq19:24:472
21806214120,97sleep221278-21apt-get20:59:342
575521408,17sleep00-21swapper/020:09:360
1541021400,8sleep20-21swapper/200:04:422
1541021400,8sleep20-21swapper/200:04:422
18722213917,57sleep00-21swapper/020:49:460
10924213811,63sleep00-21swapper/023:49:460
3021213711,20sleep00-21swapper/019:59:460
4967213516,103sleep25071-21timerwakeupswit23:29:542
13865213410,58sleep013963-21fschecks_time20:34:400
648121338,89sleep06539-21latency_hist21:54:240
15840213311,108sleep015991-21latency_hist22:24:250
2387921326,111sleep024031-21date22:49:250
4813213032,86sleep04886-21tr23:29:510
21738213010,58sleep021819-21latency00:24:440
2301921286,49sleep00-21swapper/000:29:360
20472212748,26sleep00-21swapper/000:19:550
9603212544,66sleep00-21swapper/023:44:540
19761212521,85sleep219862-21sed20:54:252
19018212327,75sleep00-21swapper/019:09:430
31045212121,75sleep30-21swapper/319:45:403
23052211924,70sleep00-21swapper/019:19:520
2477121189,70sleep024912-21unixbench_multi22:49:530
278021177,36sleep00-21swapper/023:24:340
32051211610,13sleep30-21swapper/319:49:403
31758211610,89sleep031968-21apt-config23:14:250
23640211530,70sleep023711-21latency21:04:450
2562121145,11sleep00-21swapper/000:36:470
3164211324,25sleep20-21swapper/221:44:242
3164211324,25sleep20-21swapper/221:44:242
23589211312,17sleep30-21swapper/321:04:443
9533211212,84sleep19567-21ntp_kernel_pll_20:19:551
21819211226,25sleep00-21swapper/020:59:400
31239211133,55sleep131375-21grep19:45:421
2776621117,19sleep10-21swapper/121:19:241
2971021105,12sleep00-21swapper/019:41:530
21550210923,67sleep121592-21proc_pri22:41:021
1890521099,88sleep018978-21wc00:14:500
1890521099,88sleep018978-21wc00:14:500
770021089,88sleep0367-21snmpd21:55:500
30152210816,73sleep230338-21apt-config19:44:262
2105221077,57sleep00-21swapper/022:40:200
29762210614,17sleep20-21swapper/221:24:422
18570210626,62sleep0231-21systemd-journal19:09:220
13852210614,79sleep313863-21awk20:34:393
12768210616,75sleep112889-21timerwakeupswit23:54:521
12768210616,75sleep112889-21timerwakeupswit23:54:521
12310210621,26sleep30-21swapper/320:29:423
3283210513,72sleep33486-21cat21:44:263
3283210513,72sleep33486-21cat21:44:263
4670210412,65sleep00-21swapper/020:04:510
3059721045,10sleep00-21swapper/021:25:530
20230210424,53sleep220291-21munin-node19:14:252
4065210122,15sleep00-21swapper/021:45:030
18021210118,55sleep10-21swapper/122:29:471
10245210119,67sleep210283-21ntpq22:04:482
24239210013,71sleep124255-21/usr/sbin/munin19:24:421
12013210017,65sleep212143-21cat22:09:532
2774929913,28sleep027752-21cron21:19:230
2933929846,17sleep029372-21munin-run21:24:230
2048129811,25sleep020539-21unin-run22:39:230
2045929817,60sleep320481-21/usr/sbin/munin20:54:503
131932975,31sleep00-21swapper/023:57:180
131932975,31sleep00-21swapper/023:57:180
2748329612,17sleep20-21swapper/222:59:452
256312965,63sleep30-21swapper/322:54:333
1521029520,53sleep215342-21tail22:19:552
2686829414,59sleep37437-21kworker/u8:419:34:253
771129226,47sleep27726-21kernelversion23:39:432
281892925,23sleep00-21swapper/019:37:330
2500829212,64sleep125015-21df_inode21:09:421
2044129229,37sleep320462-21apt-get22:39:223
91929121,52sleep1934-21modprobe23:19:221
91929121,52sleep1934-21modprobe23:19:221
701329118,58sleep27041-21awk20:14:222
2631429118,57sleep126459-21sed22:54:511
231112910,11sleep30-21swapper/321:04:223
1686629120,53sleep00-21swapper/000:09:410
1404529015,57sleep313653-21/usr/sbin/munin23:59:473
643028923,44sleep2492-21cron21:54:222
1686628814,52sleep316867-21/usr/sbin/munin20:44:323
1686628814,52sleep316867-21/usr/sbin/munin20:44:323
1126228812,61sleep111331-21/usr/sbin/munin20:25:241
3157028732,37sleep131589-21apt-get23:14:221
239722877,16sleep00-21swapper/019:24:330
1957728727,32sleep30-21swapper/319:09:543
84452865,46sleep00-21swapper/020:17:440
274172868,41sleep027469-21cat22:59:430
457928511,57sleep14616-21ntp_kernel_err20:04:481
209652847,42sleep021051-21perl19:14:430
95012836,64sleep00-21swapper/022:03:370
654128316,48sleep16682-21modprobe21:54:251
312128317,51sleep13163-21apt-config21:44:231
312128317,51sleep13163-21apt-config21:44:231
3007928317,34sleep10-21swapper/123:09:251
2814728314,54sleep228151-21diskstats21:19:412
249362837,64sleep024929-21apt-get21:09:370
359228220,37sleep23110-21apt-get21:44:332
1355128116,48sleep213622-21latency_hist23:59:262
1355128116,48sleep213622-21latency_hist23:59:262
925528020,39sleep39262-21fschecks_count20:19:493
2708328015,51sleep327160-21grep21:14:533
2460728020,18sleep20-21swapper/222:49:512
2135028025,37sleep121432-21munin-run20:59:241
799327931,28sleep30-21swapper/320:14:563
2754527920,43sleep327609-21find19:34:493
2591227917,46sleep225979-21latency_hist00:39:242
2455827916,49sleep124606-21processes22:49:501
2339327912,49sleep223395-21sshd19:20:352
2183827925,25sleep20-21swapper/200:24:462
189952790,6sleep30-21swapper/320:49:523
1625127912,51sleep116309-21cat22:24:421
242772789,17sleep10-21swapper/100:34:221
1832327822,29sleep20-21swapper/200:14:362
1832327822,29sleep20-21swapper/200:14:362
1321427823,24sleep20-21swapper/222:14:462
1321427823,24sleep20-21swapper/222:14:462
372727719,43sleep13737-21forks21:44:561
2219427725,29sleep10-21swapper/100:24:531
2129627721,40sleep221346-21if_eth022:40:552
305227618,33sleep10-21swapper/123:24:441
587527519,39sleep00-21swapper/023:34:360
432927510,15sleep30-21swapper/323:29:383
2767327514,44sleep327703-21/usr/sbin/munin22:59:503
1158727514,46sleep311589-21sshd23:51:353
101882757,18sleep20-21swapper/223:48:272
857327417,35sleep18611-21systemctl20:19:221
1578627418,41sleep115800-21modprobe22:24:221
1148327418,26sleep30-21swapper/322:09:423
410427322,30sleep20-21swapper/223:29:252
2309427319,39sleep30-21swapper/322:44:513
1662027314,27sleep20-21swapper/222:24:512
1649027323,25sleep10-21swapper/100:09:221
1649027323,25sleep10-21swapper/100:09:221
855827219,36sleep30-21swapper/321:59:443
643127215,41sleep16560-21unixbench_multi20:09:531
2589327219,38sleep125916-21/usr/sbin/munin19:29:471
1503127217,37sleep215033-21cron20:39:222
1062127215,29sleep10-21swapper/123:49:391
3167127126,29sleep331746-21find23:14:243
2932027110,16sleep20-21swapper/219:39:532
150427124,31sleep31546-21latency23:19:453
150427124,31sleep31546-21latency23:19:453
2053827018,27sleep10-21swapper/100:19:561
1489727021,24sleep30-21swapper/322:19:473
131627021,26sleep30-21swapper/319:54:493
2528926923,25sleep30-21swapper/319:29:243
2520426918,34sleep225220-21/usr/sbin/munin21:09:472
1194726916,30sleep10-21swapper/122:09:521
627226825,22sleep30-21swapper/323:34:473
3092126711,14sleep20-21swapper/223:09:542
2223926717,31sleep122281-21/usr/sbin/munin21:00:051
2010926714,38sleep320152-21cat00:19:463
1960226714,34sleep3492-21cron00:19:223
1960226714,34sleep3492-21cron00:19:223
1917126728,22sleep10-21swapper/120:49:551
1858926725,25sleep118640-21munin-run19:09:231
1704826726,25sleep317114-21latency00:09:443
2602026630,18sleep20-21swapper/219:29:522
159826616,34sleep31716-21latency_hist21:39:253
790926513,30sleep27990-21cat20:14:552
51222655,12sleep30-21swapper/320:04:583
2747026524,23sleep10-21swapper/122:59:441
1788226526,19sleep20-21swapper/219:04:522
1462326521,27sleep114629-21cron00:00:221
1353526516,24sleep20-21swapper/220:34:262
2266526421,29sleep322699-21irqstats19:19:433
1430126418,26sleep20-21swapper/223:59:542
117126420,25sleep10-21swapper/119:54:471
1113326416,30sleep111158-21munin-run22:09:231
2380426317,24sleep20-21swapper/222:49:232
3117126220,26sleep131238-21df21:29:461
3117126220,26sleep131238-21df21:29:461
278326221,22sleep32853-21df23:24:373
235532629,12sleep20-21swapper/221:04:432
2026826216,20sleep20-21swapper/220:54:452
2004726221,26sleep220108-21cut00:19:452
160426212,23sleep10-21swapper/123:19:471
160426212,23sleep10-21swapper/123:19:471
1133226222,17sleep20-21swapper/220:25:252
869426113,30sleep28739-21/usr/sbin/munin21:59:472
2318226114,26sleep10-21swapper/119:19:531
2070126117,23sleep20-21swapper/222:39:252
2019726119,21sleep10-21swapper/119:14:241
1335426116,26sleep313377-21munin-run23:59:233
1335426116,26sleep313377-21munin-run23:59:233
99826020,24sleep30-21swapper/323:19:243
99826020,24sleep30-21swapper/323:19:243
3257926020,20sleep20-21swapper/223:15:032
3257926020,20sleep20-21swapper/223:15:032
2969626027,18sleep10-21swapper/121:24:411
179852605,17sleep20-21swapper/220:47:142
179852605,17sleep20-21swapper/220:47:142
1668926012,20sleep30-21swapper/322:24:523
1530526014,25sleep30-21swapper/300:04:403
1530526014,25sleep30-21swapper/300:04:403
2306625917,20sleep20-21swapper/200:29:382
13642596,18sleep20-21swapper/221:37:222
1294625916,26sleep312938-21apt-get22:14:383
1294625916,26sleep312938-21apt-get22:14:383
86812587,39sleep10-21swapper/123:43:581
85225816,27sleep1945-21/usr/sbin/munin21:35:031
291102589,12sleep30-21swapper/319:39:503
2456625818,17sleep10-21swapper/100:34:381
2146625816,25sleep321462-21apt-get00:24:353
1535025816,25sleep215343-21apt-get20:39:382
1229225810,34sleep112302-21df_inode20:29:401
1118825816,24sleep211314-21sed23:49:532
1067425814,27sleep310667-21apt-get20:24:373
1042525819,21sleep30-21swapper/322:04:533
3244225716,22sleep10-21swapper/119:49:501
273725718,25sleep32773-21expr19:59:413
2251825715,26sleep122537-21apt-get22:44:331
1859825714,24sleep10-21swapper/100:14:431
1859825714,24sleep10-21swapper/100:14:431
163142576,17sleep10-21swapper/120:40:381
956025613,18sleep10-21swapper/122:04:231
2600025612,28sleep226031-21nfs_client22:54:442
2337225615,26sleep123439-21/usr/sbin/munin21:04:371
2142025618,21sleep221506-21sed19:14:542
1677825617,22sleep216773-21apt-get00:09:352
1611825615,20sleep30-21swapper/320:39:563
1335225612,26sleep113397-21/usr/sbin/munin22:14:491
1335225612,26sleep113397-21/usr/sbin/munin22:14:491
563425514,25sleep35649-21/usr/sbin/munin21:49:513
558825514,23sleep15625-21ntp_states21:49:491
322732555,16sleep30-21swapper/321:32:303
322732555,16sleep30-21swapper/321:32:303
2539825515,23sleep30-21swapper/322:54:243
1971725515,19sleep10-21swapper/122:34:521
1770425513,16sleep20-21swapper/222:29:402
1696225514,24sleep116971-21df_inode20:44:381
1696225514,24sleep116971-21df_inode20:44:381
900425412,25sleep18996-21apt-get23:44:391
596225415,23sleep36012-21if_eth020:09:433
488725420,19sleep10-21swapper/123:29:531
2704425414,21sleep20-21swapper/219:34:322
956925315,23sleep29590-21ntp_states20:19:562
3198325314,23sleep232050-21df19:49:392
2509325313,21sleep325167-21sensors_temp00:34:523
125625312,18sleep20-21swapper/219:54:482
614825210,17sleep10-21swapper/123:34:451
552125213,20sleep20-21swapper/220:09:262
325625212,24sleep23351-21sed19:59:522
2979025215,22sleep329791-21sshd23:06:203
1894325212,18sleep30-21swapper/322:34:253
804625114,22sleep38063-21grep21:59:243
2982825114,22sleep329877-21irqstats21:24:433
243272518,18sleep30-21swapper/300:34:243
90732509,17sleep30-21swapper/323:44:413
3167125016,21sleep231746-21mailstats21:29:592
3167125016,21sleep231746-21mailstats21:29:592
2903825017,15sleep20-21swapper/223:04:452
2814025011,15sleep10-21swapper/121:19:401
1807525014,21sleep118085-21munin-run00:14:231
2502324814,18sleep30-21swapper/321:09:443
71142457,15sleep20-21swapper/221:54:472
265124511,17sleep10-21swapper/119:59:351
2839724313,16sleep10-21swapper/119:39:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional