You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 20:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 30, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12276260815,60sleep10-21swapper/108:55:081
27386239916,361sleep027387-21cron09:44:350
19377236316,262sleep119516-21unixbench_multi07:35:071
102323600,16sleep110273-21cyclictest08:19:571
12596230249,159sleep112676-21fschecks_time07:14:561
1067322907,49sleep10-21swapper/112:15:121
30178228348,193sleep129805-21/usr/sbin/munin08:09:571
28678225818,140sleep10-21swapper/111:29:371
7010223217,200sleep17013-21cron12:04:341
2739022138,193sleep127437-21unin-run09:44:361
4309220815,179sleep24351-21latency_hist10:14:382
1025422088,188sleep0364-21snmpd10:31:400
25179220417,21sleep00-21swapper/007:55:100
25179220417,21sleep00-21swapper/007:55:100
11771220020,138sleep111793-21idleruntime-cro07:14:351
11771220020,138sleep111793-21idleruntime-cro07:14:351
4724219511,144sleep14810-21switchtime11:55:051
24819219432,147sleep124949-21latency_hist07:54:381
7675219135,85sleep10-21swapper/110:24:511
7675219135,85sleep10-21swapper/110:24:511
180721888,169sleep0364-21snmpd08:22:030
17212218518,108sleep117302-21fschecks_time12:34:551
12942218315,154sleep213005-21cut07:15:052
32305218223,59sleep10-21swapper/108:15:141
29054217549,51sleep10-21swapper/108:05:081
29054217549,51sleep10-21swapper/108:05:081
11383217523,55sleep10-21swapper/110:35:041
1765721716,152sleep117717-21cat10:54:561
3084621709,17sleep10-21swapper/109:54:541
26410216911,140sleep126455-21awk07:59:351
26410216911,140sleep126455-21awk07:59:351
2342021686,150sleep1364-21snmpd11:11:361
14745216611,138sleep114885-21latency_hist09:04:371
22188216022,120sleep10-21swapper/107:45:011
25811215413,88sleep125839-21sh09:39:351
12868215423,113sleep312893-21sendmail08:59:343
6211215146,56sleep10-21swapper/110:19:541
1568221518,101sleep115793-21latency_hist10:49:381
3766214851,44sleep10-21swapper/110:10:091
20980214853,48sleep10-21swapper/109:24:371
25596214757,41sleep10-21swapper/111:19:521
25596214757,41sleep10-21swapper/111:19:521
3053621459,99sleep130612-21latency_hist09:54:381
1815921457,14sleep20-21swapper/209:14:522
18174214130,33sleep00-21swapper/009:14:540
22104214018,97sleep222130-21cut11:09:352
7014213810,114sleep21-21systemd12:04:352
630621387,12sleep20-21swapper/208:35:102
14790213646,45sleep10-21swapper/107:20:091
15605213312,104sleep215599-21apt-get07:24:502
26874213024,46sleep10-21swapper/107:59:551
26874213024,46sleep10-21swapper/107:59:551
2383212924,49sleep10-21swapper/111:49:361
7012212612,101sleep37038-21df08:39:523
32071212333,52sleep132141-21fschecks_time11:39:551
341212215,20sleep00-21swapper/010:00:020
1953212145,40sleep10-21swapper/110:05:041
16238212119,83sleep20-21swapper/209:09:352
14760212011,96sleep214867-21unixbench_singl12:25:072
2347211813,82sleep10-21swapper/108:24:511
16637211819,11sleep10-21swapper/107:26:431
3818211737,50sleep33838-21apt-get08:29:463
2745211715,72sleep30-21swapper/311:49:553
17773211610,50sleep117901-21timerwakeupswit07:30:061
13469211628,56sleep1364-21snmpd10:42:481
5890211522,76sleep06016-21latency_hist10:19:370
2828321156,99sleep1788-21sendmail-mta11:25:541
2269211511,93sleep0364-21snmpd11:48:460
15469211510,65sleep114963-21/usr/sbin/munin09:05:061
31313211417,37sleep231118-21apt-get08:14:382
10152211428,50sleep110244-21dpkg12:14:361
20926211322,71sleep30-21swapper/309:24:353
1657521117,92sleep1364-21snmpd12:31:241
15942211113,45sleep10-21swapper/110:50:031
8605211029,43sleep18695-21grep08:44:551
8605211029,43sleep18695-21grep08:44:551
723521109,77sleep10-21swapper/108:39:581
20953211023,39sleep10-21swapper/107:40:081
2081421097,51sleep10-21swapper/111:04:451
20274210720,62sleep020268-21apt-get07:39:510
19945210648,19sleep119961-21awk11:00:091
6225210513,76sleep06295-21/usr/sbin/munin08:35:080
3927210530,41sleep13933-21munin-run11:54:371
3084210519,62sleep03148-21sort10:09:550
13397210420,64sleep212910-21apt-get08:59:462
13710210242,48sleep1364-21snmpd12:24:211
13710210242,48sleep1364-21snmpd12:24:211
20837210120,62sleep320834-21apt-get11:04:473
16823210117,70sleep016862-21modprobe12:34:340
742429914,72sleep27443-21open_inodes08:40:032
2520829928,53sleep025220-21modprobe11:19:340
1250229924,42sleep20-21swapper/212:20:142
1250229924,42sleep20-21swapper/212:20:142
1136229825,40sleep111373-21munin-run08:54:361
1136229825,40sleep111373-21munin-run08:54:361
185502978,46sleep10-21swapper/109:15:031
2528829615,26sleep20-21swapper/211:19:362
2158329612,68sleep321720-21latency_hist07:44:383
3098129517,62sleep331059-21switchtime11:35:043
295932956,12sleep00-21swapper/009:50:000
159972959,50sleep20-21swapper/210:50:042
384329419,36sleep10-21swapper/108:29:481
2408329415,13sleep10-21swapper/109:32:161
580429325,23sleep10-21swapper/108:35:011
3256729318,52sleep032659-21/usr/sbin/munin11:40:060
1747629317,54sleep017526-21cpuspeed10:54:510
2414929215,36sleep10-21swapper/111:14:551
2777329122,51sleep327860-21fschecks_time09:44:543
1364529123,41sleep313686-21munin-run10:44:363
1245629114,60sleep311932-21apt-get12:19:473
1245629114,60sleep311932-21apt-get12:19:473
1191029128,43sleep311934-21idleruntime-cro12:19:353
2579429020,47sleep225831-21grep11:19:572
2579429020,47sleep225831-21grep11:19:572
239329021,55sleep02497-21fschecks_time08:24:540
2232629029,34sleep30-21swapper/307:45:073
3223128914,56sleep032304-21/usr/sbin/munin08:15:130
2312928820,46sleep223107-21latency_hist07:49:382
284282879,11sleep00-21swapper/011:27:460
2743728715,58sleep227523-21perl11:24:582
2060528718,47sleep320330-21apt-get11:04:393
2931128620,39sleep329381-21fschecks_time09:49:553
1531528622,42sleep315345-21awk07:24:353
2297428418,46sleep022978-21find09:29:450
2011328414,51sleep220124-21ntp_states09:20:012
139932847,41sleep114009-21awk12:24:461
2342728315,55sleep323450-21netstat09:30:413
1413328315,55sleep214140-21sh07:19:542
1182328326,22sleep30-21swapper/307:14:373
1182328326,22sleep30-21swapper/307:14:373
927028124,17sleep10-21swapper/110:29:511
780428124,19sleep17876-21mailstats12:05:031
2898628124,45sleep128985-21munin-run09:49:371
1545528119,42sleep315575-21/usr/sbin/munin10:49:363
1281328027,30sleep20-21swapper/210:40:022
1124928015,49sleep310750-21/usr/sbin/munin10:35:023
941527922,44sleep29491-21tr07:05:042
243827914,34sleep230160-1kworker/dying11:49:372
2402627921,42sleep324056-21grep11:14:533
94922780,5sleep30-21swapper/307:05:043
2168127816,29sleep30-21swapper/309:25:033
3244627726,27sleep20-21swapper/211:40:042
1770327716,47sleep017771-21grep07:30:050
2898727621,25sleep20-21swapper/209:49:382
2336627617,25sleep30-21swapper/307:49:513
766727520,23sleep00-21swapper/012:04:580
42427523,33sleep2450-21munin-run08:19:352
610227422,31sleep36211-21grep08:35:073
2720927416,44sleep227265-21grep08:00:042
2720927416,44sleep227265-21grep08:00:042
171727426,23sleep20-21swapper/210:04:562
454427310,18sleep10-21swapper/110:14:491
189082730,9sleep30-21swapper/307:34:563
110682736,12sleep20-21swapper/208:51:222
110682736,12sleep20-21swapper/208:51:222
313662726,18sleep00-21swapper/011:35:420
112202726,48sleep00-21swapper/012:16:040
3080427121,27sleep20-21swapper/211:34:592
1714827120,11sleep10-21swapper/109:10:071
1714827120,11sleep10-21swapper/109:10:071
1631027118,36sleep216449-21awk12:30:082
804827012,26sleep30-21swapper/310:25:013
804827012,26sleep30-21swapper/310:25:013
325302709,24sleep132590-21if_err_eth009:59:551
222852706,23sleep10-21swapper/109:27:071
1589527014,24sleep00-21swapper/010:49:460
848526918,31sleep00-21swapper/008:44:450
848526918,31sleep00-21swapper/008:44:450
54032696,20sleep00-21swapper/011:59:160
356426921,27sleep30-21swapper/310:10:063
3051926923,32sleep10-21swapper/111:34:541
2517026921,30sleep325141-21apt-get07:55:043
2517026921,30sleep325141-21apt-get07:55:043
2462226914,35sleep224641-21awk07:54:352
1659226915,27sleep30-21swapper/309:09:523
1659226915,27sleep30-21swapper/309:09:523
16222699,12sleep20-21swapper/211:45:052
1130926914,20sleep00-21swapper/010:35:030
866026813,29sleep00-21swapper/012:09:380
2744026813,39sleep327535-21sed09:44:383
2276426828,21sleep30-21swapper/311:10:003
1906526819,26sleep20-21swapper/207:35:002
1067926815,33sleep00-21swapper/008:50:060
1067926815,33sleep00-21swapper/008:50:060
939126723,23sleep10-21swapper/107:05:031
205526714,15sleep30-21swapper/308:24:373
2052426715,32sleep220330-21apt-get11:04:382
1400626710,18sleep00-21swapper/009:00:160
1190726713,28sleep00-21swapper/007:14:380
860726619,27sleep30-21swapper/312:09:373
320572666,15sleep30-21swapper/311:39:533
2697126612,37sleep227032-21/usr/sbin/munin09:40:092
2107926618,21sleep00-21swapper/011:05:130
153326613,28sleep30-21swapper/310:04:533
1381326618,20sleep00-21swapper/012:24:370
1381326618,20sleep00-21swapper/012:24:370
983626516,32sleep39975-21date08:49:373
983626516,32sleep39975-21date08:49:373
360826517,24sleep30-21swapper/308:29:383
301926518,21sleep00-21swapper/011:50:000
2728426517,27sleep30-21swapper/311:24:463
2159626519,30sleep021637-21/usr/sbin/munin09:25:000
1691426523,18sleep00-21swapper/007:29:370
1410926513,32sleep214121-21acpi10:44:472
1138926517,20sleep20-21swapper/212:16:062
2885926415,23sleep30-21swapper/308:05:063
2885926415,23sleep30-21swapper/308:05:063
2804726418,19sleep30-21swapper/308:04:383
2804726418,19sleep30-21swapper/308:04:383
2299726420,26sleep222491-21apt-get09:29:472
206012645,19sleep00-21swapper/009:20:240
1896626422,20sleep00-21swapper/007:34:580
1239026419,24sleep00-21swapper/007:14:470
121226422,20sleep30-21swapper/310:04:373
1206426422,22sleep00-21swapper/008:55:030
964826312,37sleep29671-21ntp_states10:30:012
813626315,29sleep28156-21/usr/sbin/munin10:25:042
813626315,29sleep28156-21/usr/sbin/munin10:25:042
659526317,20sleep30-21swapper/310:20:043
444826315,32sleep24539-21timerandwakeup08:30:052
429226319,23sleep30-21swapper/311:54:553
2923026314,23sleep00-21swapper/011:30:010
2032626315,35sleep020418-21cmp11:04:360
1174326315,27sleep00-21swapper/007:14:140
827326224,21sleep00-21swapper/008:44:370
706526212,23sleep00-21swapper/012:04:370
2886226218,27sleep328855-21apt-get11:29:513
2835426220,25sleep20-21swapper/208:04:532
2835426220,25sleep20-21swapper/208:04:532
2561126220,28sleep325656-21fschecks_count11:19:543
2561126220,28sleep325656-21fschecks_count11:19:543
1881126212,20sleep20-21swapper/210:59:372
145692626,12sleep00-21swapper/012:25:050
1370626214,33sleep013835-21apt-config10:44:370
987726119,23sleep30-21swapper/310:30:063
957226123,24sleep09702-21wc07:05:060
463026119,22sleep30-21swapper/310:14:553
1768826127,20sleep30-21swapper/312:35:063
1535226122,20sleep00-21swapper/009:05:010
105126113,31sleep31068-21/usr/sbin/munin11:44:463
270526016,27sleep02711-21modprobe10:09:350
234762606,14sleep00-21swapper/011:12:190
1532726017,27sleep315458-21latency_hist12:29:363
1406126015,14sleep00-21swapper/007:19:520
1392926015,30sleep314004-21grep09:00:153
884525915,27sleep28839-21apt-get12:09:482
3214025915,22sleep00-21swapper/009:59:370
2469625916,27sleep024818-21cmp07:54:370
18342595,10sleep20-21swapper/208:22:262
155372596,16sleep30-21swapper/312:29:443
2913025820,24sleep229189-21latency11:29:582
2515925818,22sleep20-21swapper/207:54:552
2515925818,22sleep20-21swapper/207:54:552
2074725816,23sleep20-21swapper/207:40:052
1808825813,26sleep318084-21apt-get09:14:493
1086025811,31sleep210949-21grep10:34:532
809625716,25sleep08128-21ntp_states10:25:020
809625716,25sleep08128-21ntp_states10:25:020
680525714,28sleep26931-21/usr/sbin/munin10:20:072
3203125713,25sleep332075-21/usr/sbin/munin08:15:083
183572577,36sleep20-21swapper/212:39:252
893125616,23sleep38997-21fschecks_time12:09:533
3135025619,16sleep20-21swapper/209:55:072
2954025618,24sleep029590-21systemd-run08:09:350
2954025618,24sleep029590-21systemd-run08:09:350
447025514,26sleep24537-21memory11:54:582
3259125515,23sleep232617-21irqstats09:59:562
223692555,11sleep20-21swapper/209:28:132
167172554,10sleep30-21swapper/307:27:463
1589225520,19sleep30-21swapper/310:49:453
869925412,26sleep28746-21/usr/sbin/munin08:44:562
869925412,26sleep28746-21/usr/sbin/munin08:44:562
3013925414,16sleep20-21swapper/209:50:102
1779925414,24sleep20-21swapper/209:14:362
1779925414,24sleep20-21swapper/209:14:362
1498625414,23sleep314981-21apt-get09:04:523
1271625414,24sleep312744-21irqstats07:14:583
580825315,23sleep25831-21munin-run10:19:352
3143525310,20sleep30-21swapper/309:55:083
2463325313,20sleep00-21swapper/009:34:540
1500625312,25sleep015098-21users10:45:540
1308825311,22sleep20-21swapper/208:59:382
666225218,18sleep00-21swapper/010:20:050
2815425214,18sleep00-21swapper/009:45:030
2670325217,16sleep00-21swapper/007:59:460
2670325217,16sleep00-21swapper/007:59:460
246752528,18sleep20-21swapper/209:34:562
1808525218,17sleep30-21swapper/310:55:063
1757025213,24sleep017576-21ntp_states12:35:030
1099425214,19sleep20-21swapper/207:10:042
1099425214,19sleep20-21swapper/207:10:042
729925112,23sleep27295-21apt-get12:04:472
3108925115,20sleep30-21swapper/308:14:353
284925113,14sleep20-21swapper/208:25:042
2552825112,16sleep00-21swapper/011:19:510
2552825112,16sleep00-21swapper/011:19:510
600125016,15sleep30-21swapper/311:59:563
513425013,20sleep00-21swapper/008:34:350
1727225012,23sleep217356-21perl07:29:552
2476224912,25sleep324815-21interrupts09:34:573
1125224812,25sleep3364-21snmpd08:53:453
1125224812,25sleep3364-21snmpd08:53:453
1752324716,14sleep30-21swapper/307:30:003
1654124716,18sleep00-21swapper/009:09:490
1654124716,18sleep00-21swapper/009:09:490
2310224614,17sleep20-21swapper/211:10:072
2982924511,17sleep00-21swapper/008:09:480
217962429,14sleep00-21swapper/007:44:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional