You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 19:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 16, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13352263626,595sleep1370-21snmpd11:46:081
13352263626,595sleep1370-21snmpd11:46:081
2510124917,449sleep125137-21ntpq10:40:571
12585241323,372sleep312601-21fschecks_count11:45:533
12585241323,372sleep312601-21fschecks_count11:45:533
7601241147,255sleep17749-21latency_hist11:30:331
17434238959,246sleep117571-21unixbench_singl10:16:041
30933233612,305sleep030926-21apt-get11:00:470
28515233447,197sleep128662-21unixbench_multi10:51:021
12680231745,238sleep112699-21latency_hist08:20:321
21924228413,124sleep10-21swapper/112:15:531
32309228216,56sleep10-21swapper/111:05:321
25190224798,67sleep10-21swapper/107:15:531
2492222928,85sleep12640-21latency_hist09:30:321
30179222458,151sleep130193-21apt09:15:301
24477222335,124sleep10-21swapper/108:55:591
28414221012,183sleep128460-21ntp_kernel_pll_07:25:551
13351220815,175sleep313483-21latency_hist10:05:323
9550220711,181sleep09605-21basename11:35:570
24078220728,146sleep124091-21awk10:36:201
24078220728,146sleep124091-21awk10:36:201
20556220713,22sleep10-21swapper/108:45:321
606921989,175sleep16213-21latency_hist11:25:321
2731921989,151sleep10-21swapper/109:05:521
24696219810,63sleep10-21swapper/107:15:331
24696219810,63sleep10-21swapper/107:15:331
17238219750,60sleep10-21swapper/112:00:541
8518219548,133sleep18559-21proc_pri08:06:061
26225219316,160sleep326347-21latency_hist07:20:323
4615219128,127sleep14661-21ntpq09:35:581
3093521918,168sleep131080-21latency_hist07:35:331
1560421898,88sleep10-21swapper/111:55:491
1560421898,88sleep10-21swapper/111:55:491
9133218116,26sleep39139-21latency_hist11:35:313
9619217240,21sleep10-21swapper/109:51:041
6231217041,86sleep10-21swapper/108:00:331
22696216919,90sleep10-21swapper/110:35:311
22696216919,90sleep10-21swapper/110:35:311
17278216412,136sleep317279-21sshd08:35:043
8271216341,52sleep10-21swapper/111:31:011
17373215732,101sleep217510-21sed06:51:072
1045821568,110sleep110558-21timerandwakeup08:11:111
20969215314,123sleep121124-21awk12:11:041
11291215234,60sleep10-21swapper/111:40:591
30689214936,67sleep130739-21latency_hist11:00:321
469221489,121sleep14833-21latency_hist07:55:321
469221489,121sleep14833-21latency_hist07:55:321
13540214726,46sleep10-21swapper/108:21:051
6919214618,110sleep26951-21proc_pri08:00:582
851821447,48sleep1370-21snmpd09:49:101
851821447,48sleep1370-21snmpd09:49:101
483521440,13sleep20-21swapper/207:55:322
483521440,13sleep20-21swapper/207:55:322
22506214228,38sleep10-21swapper/107:06:041
22506214228,38sleep10-21swapper/107:06:041
15227214223,94sleep10-21swapper/110:10:511
15227214223,94sleep10-21swapper/110:10:511
1842121409,44sleep10-21swapper/110:20:571
23776213713,72sleep123817-21fschecks_time12:21:141
14798213472,20sleep10-21swapper/111:51:051
14798213472,20sleep10-21swapper/111:51:051
6460213214,97sleep30-21swapper/309:41:053
16672213118,58sleep10-21swapper/108:31:071
21828213018,96sleep221898-21/usr/sbin/munin07:05:492
21828213018,96sleep221898-21/usr/sbin/munin07:05:492
692212910,37sleep10-21swapper/107:40:581
3983212814,24sleep30-21swapper/309:35:313
22804212819,17sleep20-21swapper/210:35:322
22804212819,17sleep20-21swapper/210:35:322
3730212748,41sleep13809-21sort11:16:031
19242211943,56sleep10-21swapper/112:05:591
26390211821,75sleep326520-21unixbench_singl09:01:063
23792211732,47sleep123841-21modprobe08:55:301
23127211718,79sleep222640-21apt-get10:35:412
23127211718,79sleep222640-21apt-get10:35:412
14676211714,82sleep014797-21/usr/sbin/munin11:51:040
14676211714,82sleep014797-21/usr/sbin/munin11:51:040
28135211524,65sleep028229-21fschecks_time07:25:500
352121110,12sleep00-21swapper/009:31:030
26461211111,82sleep10-21swapper/110:45:531
11735211114,80sleep011749-21/usr/sbin/munin08:15:550
12082210745,42sleep10-21swapper/110:00:501
9798210618,69sleep39315-21apt-get08:10:413
18953210512,77sleep219099-21latency_hist08:40:332
12614210516,74sleep212702-21timerandwakeup10:01:032
10898210510,80sleep010917-21apt-get11:40:450
7725210440,16sleep10-21swapper/108:05:301
6856210313,74sleep36892-21ntpq08:00:553
24999210211,74sleep325143-21latency_hist12:25:323
1891421027,80sleep118952-21latency_hist08:40:321
3206921015,12sleep10-21swapper/107:36:171
22972210115,71sleep022991-21ntp_states08:51:010
2154221018,76sleep10-21swapper/107:05:311
2154221018,76sleep10-21swapper/107:05:311
18420210111,72sleep018492-21systemd-run12:05:300
220282977,57sleep10-21swapper/108:47:431
66992968,77sleep1370-21snmpd09:41:111
31092967,53sleep13122-21munin-run07:50:311
309329620,37sleep10-21swapper/109:30:571
2654229412,20sleep30-21swapper/307:20:483
1799829313,67sleep318027-21latency_hist10:20:313
1146129320,35sleep10-21swapper/109:57:141
3260429221,36sleep10-21swapper/111:05:531
3260429221,36sleep10-21swapper/111:05:531
320192929,41sleep10-21swapper/109:20:471
2326329229,43sleep00-21swapper/012:20:320
198232928,17sleep10-21swapper/110:25:461
1789629022,50sleep218028-21unixbench_singl12:01:072
150392908,43sleep115109-21mailstats08:26:021
82128923,50sleep3851-21munin-plugin-st09:25:293
317622897,44sleep131776-21munin-run09:20:311
1864228911,36sleep10-21swapper/106:55:541
140282898,16sleep10-21swapper/110:05:591
673428816,58sleep36740-21/usr/sbin/munin11:26:023
240972889,18sleep20-21swapper/208:55:472
178128817,55sleep31903-21sed09:26:023
1731228822,46sleep017327-21idleruntime-cro08:35:300
935428726,31sleep30-21swapper/311:35:483
2985928717,54sleep329563-21/usr/sbin/munin07:30:543
2145928719,53sleep021499-21ps08:46:120
486228615,48sleep34985-21timerwakeupswit09:36:043
1886928511,60sleep018913-21systemd-run08:40:310
1877328513,52sleep218847-21/usr/sbin/munin06:55:592
1043428523,30sleep00-21swapper/009:55:440
2760128314,52sleep227209-21/usr/sbin/munin09:05:572
269282838,35sleep10-21swapper/107:20:581
2663728315,48sleep026258-21/usr/sbin/munin10:45:560
291628217,33sleep20-21swapper/211:15:312
291628217,33sleep20-21swapper/211:15:312
218992810,6sleep30-21swapper/307:05:493
218992810,6sleep30-21swapper/307:05:493
2658828035,34sleep012001-1kworker/0:0H09:01:270
2591728025,42sleep125975-21sed09:00:551
1067828013,49sleep210807-21latency_hist11:40:312
1920327915,32sleep00-21swapper/012:05:580
586527818,34sleep20-21swapper/209:40:512
241627819,16sleep10-21swapper/111:11:061
241627819,16sleep10-21swapper/111:11:061
1639927821,39sleep316425-21munin-plugin-st10:15:303
1639927821,39sleep316425-21munin-plugin-st10:15:303
696427719,36sleep07114-21sed11:26:070
3074127726,30sleep20-21swapper/211:00:332
2960527717,41sleep029754-21unixbench_multi09:11:040
1426627714,13sleep00-21swapper/008:25:320
2759127624,26sleep20-21swapper/210:50:312
1882227615,19sleep30-21swapper/312:05:493
1164927618,31sleep30-21swapper/308:15:543
2158227517,36sleep221667-21cat07:05:322
2158227517,36sleep221667-21cat07:05:322
2113327519,41sleep221239-21latency_hist10:30:312
1339427514,43sleep212908-21/usr/sbin/munin08:21:022
3192327421,22sleep00-21swapper/007:36:040
327672738,14sleep00-21swapper/007:40:490
1871627310,49sleep318727-21ntp_states06:55:563
2363227214,23sleep20-21swapper/207:10:542
2363227214,23sleep20-21swapper/207:10:542
189327219,29sleep00-21swapper/007:45:490
16112729,47sleep30-21swapper/311:10:473
16112729,47sleep30-21swapper/311:10:473
111627221,21sleep00-21swapper/009:25:430
1091127213,44sleep310917-21ntp_states09:55:593
511927121,26sleep20-21swapper/207:55:572
3268127124,23sleep20-21swapper/211:05:542
3268127124,23sleep20-21swapper/211:05:542
2470827112,41sleep024775-21df10:40:470
1925727117,39sleep219362-21fschecks_time08:40:532
166102716,11sleep00-21swapper/011:57:180
166102716,11sleep00-21swapper/011:57:180
960627018,35sleep10-21swapper/111:35:581
442127010,47sleep01-21systemd11:20:300
38992709,12sleep30-21swapper/307:51:023
38992709,12sleep30-21swapper/307:51:023
3068327018,31sleep30-21swapper/309:15:563
305672706,11sleep00-21swapper/010:59:340
228827017,27sleep00-21swapper/011:11:050
228827017,27sleep00-21swapper/011:11:050
123532705,11sleep20-21swapper/211:45:322
123532705,11sleep20-21swapper/211:45:322
936526915,38sleep29484-21dpkg08:10:312
703526919,30sleep07077-21awk09:45:300
2364926911,42sleep023711-21ps10:36:150
2364926911,42sleep023711-21ps10:36:150
236452698,41sleep30-21swapper/307:10:553
236452698,41sleep30-21swapper/307:10:553
231282696,13sleep30-21swapper/310:35:423
231282696,13sleep30-21swapper/310:35:423
1621926916,23sleep20-21swapper/211:56:062
1621926916,23sleep20-21swapper/211:56:062
1103026919,25sleep20-21swapper/209:56:022
510626817,31sleep35150-21/usr/sbin/munin11:20:573
2495626812,43sleep224997-21latency_hist12:25:312
192462685,51sleep00-21swapper/006:56:500
1498426823,30sleep00-21swapper/008:26:010
381026720,26sleep20-21swapper/211:16:042
2513226712,36sleep025189-21irqstats07:15:520
245612677,13sleep00-21swapper/008:56:020
1395826714,25sleep30-21swapper/310:05:573
804326621,26sleep08198-21grep09:46:060
804326621,26sleep08198-21grep09:46:060
780326615,34sleep27811-21/usr/sbin/munin11:30:462
46626613,38sleep3499-21irqstats07:40:523
273282665,49sleep30-21swapper/310:47:413
2546426619,26sleep20-21swapper/209:00:322
241622660,6sleep00-21swapper/007:11:030
241622660,6sleep00-21swapper/007:11:030
2262726623,20sleep30-21swapper/310:35:293
2262726623,20sleep30-21swapper/310:35:293
2079026620,21sleep00-21swapper/007:01:020
2079026620,21sleep00-21swapper/007:01:020
1965526618,22sleep30-21swapper/308:41:023
1460326624,24sleep30-21swapper/311:51:033
1460326624,24sleep30-21swapper/311:51:033
423726521,19sleep00-21swapper/009:35:480
2158426516,36sleep31-21systemd12:15:303
1800126524,26sleep318034-21sort06:55:323
2418426421,27sleep324194-21/usr/sbin/munin08:55:533
2310026412,28sleep20-21swapper/212:20:302
193872648,11sleep00-21swapper/010:23:590
312426317,18sleep30-21swapper/307:50:333
2830126318,26sleep20-21swapper/210:50:592
2126626312,20sleep20-21swapper/208:46:062
2039726318,22sleep00-21swapper/010:26:010
145922636,13sleep00-21swapper/010:07:110
3122326217,29sleep331247-21fschecks_count07:35:513
2536726226,23sleep325485-21cat10:41:033
2300026224,20sleep20-21swapper/208:51:032
113926220,25sleep21135-21apt-get09:25:452
990026115,27sleep00-21swapper/009:52:040
32426111,35sleep0398-21sort11:05:560
32426111,35sleep0398-21sort11:05:560
287792617,12sleep30-21swapper/310:51:293
1736326122,21sleep20-21swapper/208:35:322
3089426011,17sleep00-21swapper/007:35:310
1958226022,20sleep30-21swapper/310:25:323
1217526017,22sleep30-21swapper/311:45:293
2827725914,27sleep228311-21irqstats07:25:522
2772225918,26sleep027764-21awk07:25:300
248525916,26sleep22613-21/usr/sbin/munin07:46:042
2190725914,19sleep00-21swapper/007:05:510
2190725914,19sleep00-21swapper/007:05:510
1805425910,33sleep318071-21ntp_states08:36:013
128692599,13sleep00-21swapper/011:45:590
128692599,13sleep00-21swapper/011:45:590
948525818,22sleep30-21swapper/308:10:323
60125813,29sleep2646-21/usr/sbin/munin07:40:552
2062725815,25sleep30-21swapper/307:00:573
2062725815,25sleep30-21swapper/307:00:573
2043525816,21sleep20-21swapper/212:10:532
1309325816,27sleep313152-21idleruntime08:20:553
1030725812,21sleep20-21swapper/208:11:082
394025715,24sleep34058-21users11:16:053
193282575,43sleep20-21swapper/210:23:122
177792576,39sleep00-21swapper/010:18:180
123182575,10sleep20-21swapper/208:16:212
930125620,21sleep09310-21modprobe08:10:290
671725613,23sleep00-21swapper/008:00:520
3047625614,25sleep030470-21apt-get09:15:490
2274525619,24sleep322795-21if_eth008:50:563
870225516,22sleep20-21swapper/209:50:332
870225516,22sleep20-21swapper/209:50:332
5912555,10sleep20-21swapper/209:22:302
549325513,25sleep25605-21users11:21:042
497325519,22sleep00-21swapper/011:20:540
3139925515,16sleep20-21swapper/207:35:542
2667425511,19sleep20-21swapper/207:20:512
2599725514,23sleep226041-21munin-run10:45:302
1763625514,20sleep30-21swapper/312:01:043
1714225516,25sleep017140-21ntp_states06:50:570
1652925512,20sleep20-21swapper/210:15:332
1652925512,20sleep20-21swapper/210:15:332
1532925514,24sleep215358-21munin-run11:55:302
1532925514,24sleep215358-21munin-run11:55:302
1514225513,26sleep015137-21apt-get10:10:460
1514225513,26sleep015137-21apt-get10:10:460
144802559,34sleep314499-21apt08:25:433
100542557,40sleep20-21swapper/209:54:042
668825412,16sleep20-21swapper/211:26:012
534825415,23sleep35371-21ntp_states07:56:023
466525419,22sleep00-21swapper/007:55:310
466525419,22sleep00-21swapper/007:55:310
423425413,23sleep24230-21apt-get09:35:462
3101225415,24sleep331016-21diskstats11:00:503
2946725414,24sleep329472-21diskstats10:55:533
2913525414,26sleep229192-21irqstats09:10:542
2889825412,16sleep30-21swapper/309:10:433
2783525412,25sleep027829-21apt-get10:50:450
2144725413,22sleep00-21swapper/010:30:500
2144725413,22sleep00-21swapper/010:30:500
2128025413,23sleep321363-21memory08:46:073
2055825412,19sleep00-21swapper/012:10:550
1959525416,23sleep019614-21ntp_states08:40:590
1873625415,24sleep218732-21apt-get12:05:442
167742548,24sleep10-21swapper/106:50:501
1327325413,23sleep013298-21munin-run10:05:290
87582536,37sleep00-21swapper/011:31:200
537325310,17sleep00-21swapper/007:56:030
1632925318,19sleep30-21swapper/311:56:073
1632925318,19sleep30-21swapper/311:56:073
2867425216,15sleep30-21swapper/309:10:313
2560025219,17sleep20-21swapper/207:16:032
1490225213,18sleep20-21swapper/210:10:322
1114125214,23sleep211190-21irqstats11:40:562
898725113,22sleep38993-21diskstats09:50:503
3259125110,16sleep30-21swapper/311:05:513
3259125110,16sleep30-21swapper/311:05:513
276762519,17sleep00-21swapper/009:05:590
2225225114,21sleep322289-21ntp_states12:15:593
96972509,17sleep00-21swapper/008:10:330
744425017,16sleep30-21swapper/309:45:533
744425017,16sleep30-21swapper/309:45:533
2803825013,21sleep328033-21apt-get07:25:453
1673325012,22sleep216765-21df10:15:492
3201824912,19sleep00-21swapper/009:20:460
313924912,20sleep20-21swapper/209:30:582
1982024911,22sleep319817-21apt-get10:25:443
157102498,32sleep20-21swapper/208:29:502
2365724812,15sleep00-21swapper/012:21:090
3115124710,21sleep231199-21irqstats11:00:532
3050424614,19sleep229563-21/usr/sbin/munin07:31:052
3093724413,14sleep20-21swapper/209:16:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional