You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 17:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Apr 12, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9045217267,1668sleep08282-21kworker/0:112:21:460
240712150513,1480sleep22729-21kworker/2:009:46:462
1729269921,660sleep21770-21/usr/sbin/munin08:36:482
1867325058,487sleep013441-21snmpd09:29:000
7147230315,149sleep07240-21fschecks_time07:11:460
32245226911,208sleep032329-21mailstats11:51:580
32245226911,208sleep032329-21mailstats11:51:580
12622223999,62sleep012690-21sshd10:52:460
14754223812,75sleep014812-21sendmail_mailqu09:16:560
14600223221,13sleep00-21swapper/007:34:070
16423223024,183sleep316526-21latency_hist07:41:293
8675220232,68sleep08829-21latency_hist12:21:300
2382521989,173sleep023905-21switchtime11:26:540
2044621988,154sleep020480-21wc11:16:490
19693218914,160sleep019726-21grep07:51:270
2846421880,6chrt0-21swapper/111:41:481
22360218817,155sleep322487-21cat11:21:573
2841218218,56sleep02884-21ls12:01:470
7511218012,144sleep00-21swapper/008:56:290
7511218012,144sleep00-21swapper/008:56:290
718221808,27sleep00-21swapper/010:36:390
22677218012,66sleep022715-21ntp_kernel_pll_09:41:520
22677218012,66sleep022715-21ntp_kernel_pll_09:41:520
131221809,142sleep00-21swapper/011:56:550
21730217699,40sleep021786-21basename07:56:490
21730217699,40sleep021786-21basename07:56:490
2171217218,26sleep00-21swapper/008:36:580
1858321717,28sleep00-21swapper/011:11:400
1720121658,50sleep017245-21munin-run09:26:290
27271215812,132sleep027299-21fschecks_count09:56:450
1179821530,10sleep10-21swapper/110:51:291
1120721477,125sleep00-21swapper/009:06:420
32109214518,111sleep232166-21latency10:11:502
2596214511,56sleep02649-21fschecks_count10:21:450
31350214311,49sleep00-21swapper/008:26:570
31350214311,49sleep00-21swapper/008:26:570
2408621425,104sleep00-21swapper/009:46:560
16857214211,47sleep00-21swapper/007:41:380
1268721408,53sleep00-21swapper/009:11:430
11447213812,105sleep011478-21munin-run07:26:280
8906213716,97sleep30-21swapper/307:16:503
9326213523,88sleep19480-21grep07:17:001
4191213519,99sleep04350-21sed07:01:570
4191213519,99sleep04350-21sed07:01:570
3136321359,43sleep00-21swapper/011:51:290
2788321357,115sleep027900-21fschecks_count08:16:470
10954213427,18sleep00-21swapper/009:06:300
16858213321,95sleep116860-21/usr/sbin/munin07:41:391
503121329,40sleep00-21swapper/008:46:520
1224621309,47sleep012295-21if_eth007:27:370
1197221308,20sleep00-21swapper/012:31:390
31548212814,96sleep031571-21modprobe10:11:270
13909212720,20sleep30-21swapper/310:56:493
1151921269,105sleep213441-21snmpd10:51:112
3230021228,17sleep00-21swapper/010:11:560
21646212221,76sleep321727-21fschecks_time07:56:483
21646212221,76sleep321727-21fschecks_time07:56:483
1966521216,102sleep30-21swapper/307:51:063
15811212118,85sleep115957-21grep07:36:591
6226211627,77sleep013441-21snmpd07:09:480
4329211616,44sleep00-21swapper/010:26:500
30895211619,79sleep230978-21timerandwakeup10:06:572
26139211617,72sleep126150-21apt-key11:36:281
1240211627,69sleep11265-21modprobe08:36:271
1049211426,73sleep21068-21fschecks_count10:16:462
8402211219,76sleep38510-21latency_hist07:16:293
26076211215,36sleep026137-21modprobe11:36:270
9698211125,12sleep00-21swapper/007:18:530
2997121115,57sleep00-21swapper/010:06:060
26659211122,48sleep00-21swapper/011:36:560
26659211122,48sleep00-21swapper/011:36:560
1520621090,6sleep10-21swapper/111:01:311
8302210818,47sleep00-21swapper/008:57:180
18056210815,79sleep3222-21systemd-journal11:11:273
15099210822,69sleep00-21swapper/007:36:420
53221079,47sleep0613-21sed08:31:580
2934021079,17sleep00-21swapper/008:21:400
6400210624,58sleep36442-21latency_hist07:11:293
28446210522,66sleep00-21swapper/011:41:480
9417210423,57sleep19550-21unixbench_multi10:41:571
21725210412,55sleep00-21swapper/011:21:420
15331210322,43sleep00-21swapper/011:01:430
3786210232,29sleep03916-21grep08:41:590
1689921028,79sleep016987-21perl11:06:470
2466021018,16sleep00-21swapper/008:06:420
8500210014,73sleep38565-21latency_hist10:41:313
29971210014,71sleep230038-21df11:46:432
1246029932,30sleep10-21swapper/112:31:571
866429820,60sleep38671-21cat12:21:293
556129819,59sleep25073-21/usr/sbin/munin07:06:562
490129814,69sleep15017-21cat12:06:571
263072988,18sleep026392-21perl08:11:470
1891729815,59sleep118410-21/usr/sbin/munin07:46:561
51392976,13sleep00-21swapper/012:07:000
2020629736,12sleep00-21swapper/009:33:480
1044729729,42sleep110442-21apt-get12:26:401
1593429625,49sleep116075-21unixbench_multi11:01:561
88742955,15sleep30-21swapper/312:21:373
2379529520,60sleep223949-21unixbench_multi08:01:592
2166129436,11sleep00-21swapper/009:37:150
827029317,52sleep20-21swapper/207:15:482
2702629315,60sleep227054-21nfs4_client11:37:062
2702629315,60sleep227054-21nfs4_client11:37:062
2448029312,66sleep324497-21memory09:47:403
414429216,54sleep14236-21fschecks_time10:26:471
1922429214,62sleep319233-21df_inode09:31:453
269492919,60sleep00-21swapper/009:56:290
111512916,46sleep00-21swapper/010:47:010
542929045,28sleep00-21swapper/010:31:300
3243829014,60sleep232483-21grep08:31:432
1590528911,20sleep20-21swapper/209:21:472
682628836,11sleep00-21swapper/012:13:430
561828811,50sleep20-21swapper/210:31:442
370428732,28sleep30-21swapper/308:41:583
1430128742,24sleep00-21swapper/010:56:580
1352728711,14sleep30-21swapper/307:31:443
1122228720,49sleep211255-21grep09:06:452
122328615,53sleep31096-21/usr/sbin/munin11:56:543
98632856,18sleep00-21swapper/010:44:470
81522840,6sleep10-21swapper/112:16:581
3154728415,52sleep131239-21apt-get11:51:301
3154728415,52sleep131239-21apt-get11:51:301
2390628416,54sleep124033-21cat11:26:551
1777328423,44sleep317842-21basename09:27:013
2462528313,55sleep124704-21/usr/sbin/munin09:47:461
241512835,41sleep00-21swapper/008:03:390
473728216,53sleep34825-21fschecks_time08:46:473
2088428221,13sleep00-21swapper/007:52:000
712128113,15sleep10-21swapper/112:16:301
1718928113,52sleep117232-21awk11:06:531
121328118,26sleep20-21swapper/211:56:532
1020728124,25sleep20-21swapper/212:26:282
3110428010,55sleep231156-21/usr/sbin/munin08:26:512
3110428010,55sleep231156-21/usr/sbin/munin08:26:512
1276528017,16sleep20-21swapper/210:52:482
1261528012,46sleep112711-21sed07:27:461
1228228017,39sleep312043-21/usr/sbin/munin10:52:373
88772789,14sleep30-21swapper/308:57:293
1139527839,29sleep00-21swapper/012:27:230
477027728,29sleep24798-21munin-run07:06:272
477027728,29sleep24798-21munin-run07:06:272
19762776,38sleep00-21swapper/010:18:180
1053727720,38sleep310593-21fschecks_count12:26:463
758127620,39sleep17715-21sed08:56:301
758127620,39sleep17715-21sed08:56:301
3045527620,41sleep130491-21wc11:47:101
96627523,37sleep31019-21cpuspeed10:16:433
576727526,24sleep10-21swapper/107:06:591
350627516,44sleep20-21swapper/208:41:532
2818727513,43sleep227771-21/usr/sbin/munin08:16:542
1974327517,45sleep219831-21latency_hist07:51:302
1090027423,37sleep310951-21ls09:06:293
2190727314,41sleep121966-21/usr/sbin/munin11:21:471
2041327315,36sleep120472-21sendmail-msp09:36:271
338927217,38sleep33534-21awk12:01:583
1325627222,30sleep313294-21sendmail12:36:273
131927226,25sleep30-21swapper/308:36:293
506427113,43sleep15109-21ntp_offset08:46:531
2058527120,27sleep10-21swapper/107:51:571
1212427120,34sleep212155-21tune2fs07:27:352
2836927014,38sleep328438-21switchtime08:16:593
2755727018,21sleep30-21swapper/308:16:303
2519527015,23sleep30-21swapper/311:31:543
1628727023,24sleep30-21swapper/309:21:563
2873826920,29sleep128735-21apt-get10:01:391
1808626922,24sleep20-21swapper/209:27:082
1572126913,23sleep20-21swapper/211:01:532
1408526914,38sleep214142-21/usr/sbin/munin07:31:592
582526817,28sleep30-21swapper/310:31:503
2619126818,26sleep20-21swapper/211:36:292
1071926819,32sleep210760-21ls12:26:492
613826726,21sleep20-21swapper/212:11:512
2882426715,31sleep328717-21/usr/sbin/munin10:01:453
2366926718,32sleep223815-21sed09:46:302
2366926718,32sleep223815-21sed09:46:302
1855126714,32sleep218070-21apt-get11:11:342
1642526716,29sleep10-21swapper/109:21:581
1558326718,26sleep20-21swapper/209:21:302
1281026721,28sleep112662-21/usr/sbin/munin09:11:481
481726618,26sleep34839-21munin-run07:06:283
481726618,26sleep34839-21munin-run07:06:283
397326622,18sleep10-21swapper/107:01:541
397326622,18sleep10-21swapper/107:01:541
2991626617,26sleep10-21swapper/108:21:571
1736626622,23sleep20-21swapper/211:06:582
1407026619,22sleep20-21swapper/210:56:522
436426516,32sleep14409-21systemd-run08:46:281
400326520,20sleep20-21swapper/212:06:302
3220126524,17sleep10-21swapper/108:31:301
3220126524,17sleep10-21swapper/108:31:301
2600926516,30sleep325589-21/usr/sbin/munin09:51:523
2441226515,35sleep324422-21munin-run08:06:293
1997626522,19sleep10-21swapper/111:16:301
111726516,29sleep11114-21apt-get11:56:431
724426415,19sleep10-21swapper/107:11:471
3029026421,25sleep30-21swapper/310:06:423
27502646,11sleep10-21swapper/108:40:391
2684826418,30sleep326906-21if_eth011:37:033
2684826418,30sleep326906-21if_eth011:37:033
2282526419,28sleep2222-21systemd-journal08:01:272
2282526419,28sleep2222-21systemd-journal08:01:272
2034926415,21sleep30-21swapper/307:51:513
1576026412,34sleep315854-21mailstats11:01:543
930326329,19sleep10-21swapper/112:21:511
377026320,18sleep20-21swapper/210:26:272
3185426313,20sleep30-21swapper/311:51:493
3185426313,20sleep30-21swapper/311:51:493
227002636,16sleep10-21swapper/107:59:521
227002636,16sleep10-21swapper/107:59:521
1427426318,18sleep30-21swapper/309:16:433
1295326325,17sleep30-21swapper/309:11:503
431626213,32sleep24409-21fschecks_time12:06:462
2730026217,26sleep127394-21perl09:56:461
2674426214,28sleep10-21swapper/111:37:011
2674426214,28sleep10-21swapper/111:37:011
2346626224,22sleep30-21swapper/311:26:453
2336626222,19sleep20-21swapper/211:26:442
1218526222,26sleep10-21swapper/110:52:071
955326122,23sleep20-21swapper/209:01:402
666226120,24sleep26668-21ntp_states08:51:552
666226120,24sleep26668-21ntp_states08:51:552
2305826111,29sleep323177-21/usr/sbin/munin09:41:593
2305826111,29sleep323177-21/usr/sbin/munin09:41:593
2121826124,19sleep10-21swapper/109:36:561
2071126117,21sleep30-21swapper/311:16:563
1403626110,36sleep214126-21latency_hist09:16:302
620626018,20sleep30-21swapper/308:51:433
620626018,20sleep30-21swapper/308:51:433
55226021,23sleep2648-21apt-key11:56:282
55226021,23sleep2648-21apt-key11:56:282
447426018,24sleep34511-21ntp_states10:26:533
2944026012,24sleep20-21swapper/211:46:272
2849526020,20sleep30-21swapper/310:01:293
212902608,11sleep20-21swapper/211:19:382
1962926013,29sleep219688-21/usr/sbin/munin09:31:562
1734926019,24sleep217355-21ntp_states07:42:012
1390626024,19sleep10-21swapper/107:31:541
1113926019,22sleep30-21swapper/309:06:413
3115725921,19sleep30-21swapper/308:26:523
3115725921,19sleep30-21swapper/308:26:523
3056725916,30sleep330659-21sed11:47:123
2466225918,25sleep224729-21df08:06:442
2338425920,26sleep123457-21sed08:01:501
1928225910,19sleep30-21swapper/311:12:083
86822587,11sleep10-21swapper/108:57:271
561625818,24sleep15610-21apt-get10:31:431
3000525819,20sleep20-21swapper/208:21:582
2596925820,22sleep326022-21sort08:11:293
1789525812,16sleep10-21swapper/109:27:031
1249925810,34sleep312533-21ps07:27:443
911025715,16sleep20-21swapper/207:16:572
3249025718,21sleep30-21swapper/310:11:593
3183525714,25sleep131830-21apt-get10:11:401
2686925717,23sleep226894-21munin-run09:56:272
2561325715,26sleep125607-21apt-get09:51:421
2357925713,18sleep10-21swapper/109:46:291
2357925713,18sleep10-21swapper/109:46:291
2006725717,21sleep20-21swapper/207:51:462
1173225716,22sleep30-21swapper/307:26:313
727025617,23sleep27274-21diskstats10:36:452
2930925612,23sleep30-21swapper/311:44:463
2179925613,16sleep20-21swapper/211:21:442
155625614,25sleep31552-21apt-get08:36:413
532625514,17sleep20-21swapper/210:31:282
320725512,27sleep13279-21/usr/sbin/munin12:01:561
2678225514,18sleep20-21swapper/208:11:572
2026025520,20sleep10-21swapper/111:16:461
548225413,16sleep30-21swapper/312:11:273
2931725412,20sleep30-21swapper/308:21:393
2915825417,21sleep229195-21ntp_states10:01:522
2669525415,15sleep10-21swapper/108:11:561
2474825417,18sleep10-21swapper/108:06:471
2080725415,15sleep20-21swapper/209:36:452
1121525416,23sleep111221-21diskstats09:06:431
1071925412,26sleep110791-21/usr/sbin/munin07:21:571
1045725420,20sleep20-21swapper/207:21:502
754125318,16sleep20-21swapper/212:16:472
740125315,23sleep17458-21awk10:36:481
493125317,17sleep20-21swapper/208:46:502
277725310,20sleep30-21swapper/310:21:483
2073025312,24sleep320725-21apt-get09:36:423
154225314,16sleep10-21swapper/110:16:581
913625224,13sleep30-21swapper/310:41:513
623925217,17sleep30-21swapper/312:11:563
251225212,24sleep12582-21df10:21:421
1726225218,17sleep30-21swapper/307:41:583
1157925214,23sleep311584-21apt-key10:51:283
982825114,21sleep19851-21irqstats09:01:511
146952519,17sleep10-21swapper/109:16:531
771625016,16sleep20-21swapper/208:56:312
2845325016,20sleep2281-21dbus-daemon10:01:282
919924912,23sleep29239-21ps10:41:542
1509624912,22sleep215093-21apt-get07:36:402
1199524912,20sleep211992-21apt-get12:31:412
2899524814,20sleep10-21swapper/108:20:441
302882457,17sleep10-21swapper/110:06:401
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional