You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 15:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 17, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13361271013,681sleep37836-21kworker/3:109:44:033
6146266613,603sleep26293-21grep07:39:052
5378250812,339sleep25496-21latency_hist11:03:352
25819237820,340sleep325843-21/usr/sbin/munin10:24:043
2819235511,202sleep22828-21apt-key09:13:372
15181235594,210sleep215327-21unixbench_singl09:49:062
2396023387,320sleep2364-21snmpd12:00:112
3035323318,251sleep2364-21snmpd12:22:112
8493228029,60sleep20-21swapper/209:28:562
19980227795,170sleep220043-21latency08:23:592
19980227795,170sleep220043-21latency08:23:592
14494226557,70sleep214496-21cron08:08:332
14494226557,70sleep214496-21cron08:08:332
30200226179,75sleep230282-21fschecks_time07:13:542
18652222712,198sleep018696-21ntpq08:20:340
18652222712,198sleep018696-21ntpq08:20:340
14388222537,108sleep213591-21/usr/sbin/munin11:29:072
4461221518,179sleep24542-21sort07:34:052
30375221534,159sleep20-21swapper/208:58:352
30375221534,159sleep20-21swapper/208:58:352
18235221444,109sleep20-21swapper/208:19:472
18235221444,109sleep20-21swapper/208:19:472
15159220615,21sleep20-21swapper/211:33:472
28459220330,89sleep228493-21cpu10:33:482
235142203139,21sleep223666-21tail08:34:162
539220246,67sleep2525-21latency_hist07:23:352
10604220216,170sleep210658-21fschecks_count11:18:532
5280219411,163sleep25322-21fschecks_time09:19:142
330321837,163sleep211610-1kworker/2:1H09:13:492
3565218212,49sleep23581-21munin-run10:58:342
7987218018,52sleep20-21swapper/209:28:352
251172178104,26sleep225184-21latency_hist10:23:342
251172178104,26sleep225184-21latency_hist10:23:342
28126217713,73sleep20-21swapper/207:08:452
11443217711,150sleep211525-21latency_hist07:58:352
17334217212,17sleep20-21swapper/209:58:352
21748217018,54sleep20-21swapper/208:29:082
27584216928,116sleep327743-21grep10:29:043
27584216928,116sleep327743-21grep10:29:043
13354216837,16sleep20-21swapper/209:44:022
2805216026,47sleep20-21swapper/212:38:342
8696215912,16sleep10-21swapper/111:13:351
31123215914,128sleep331191-21sendmail_mailtr08:59:043
31123215914,128sleep331191-21sendmail_mailtr08:59:043
864421569,66sleep28678-21munin-run11:13:322
26568214568,20sleep20-21swapper/210:28:022
27399214318,41sleep20-21swapper/212:13:352
367421418,79sleep23828-21latency_hist07:33:362
367421418,79sleep23828-21latency_hist07:33:362
3075521398,15sleep20-21swapper/208:58:552
3075521398,15sleep20-21swapper/208:58:552
679021374,11sleep00-21swapper/011:04:460
679021374,11sleep00-21swapper/011:04:460
2333721365,11sleep20-21swapper/210:16:362
2333721365,11sleep20-21swapper/210:16:362
21089213530,59sleep221119-21awk11:53:332
21089213530,59sleep221119-21awk11:53:332
27588213333,78sleep327669-21apt-config07:08:343
27588213333,78sleep327669-21apt-config07:08:343
744221327,53sleep2360-21ntpd07:44:022
744221327,53sleep2360-21ntpd07:44:022
209921309,38sleep20-21swapper/207:28:362
2592821299,40sleep20-21swapper/208:43:452
27671212740,64sleep027762-21apt-config07:08:350
27671212740,64sleep027762-21apt-config07:08:350
2761021278,38sleep20-21swapper/212:13:482
20434212518,89sleep020483-21grep10:08:340
1920521259,38sleep20-21swapper/210:03:522
5838212328,84sleep25830-21apt-get11:03:472
5838212328,84sleep25830-21apt-get11:03:472
20440212218,88sleep020545-21timerandwakeup11:49:030
20440212218,88sleep020545-21timerandwakeup11:49:030
16977212141,25sleep20-21swapper/211:38:552
23432212011,94sleep123501-21timerandwakeup08:34:151
2417521185,102sleep00-21swapper/012:02:590
6446211720,76sleep00-21swapper/009:23:370
23570211710,91sleep323679-21latency_hist10:18:343
23570211710,91sleep323679-21latency_hist10:18:343
3275021166,98sleep2364-21snmpd10:47:242
9019211378,17sleep20-21swapper/207:49:032
12151211315,81sleep012224-21fschecks_time11:23:520
12151211315,81sleep012224-21fschecks_time11:23:520
22295211211,75sleep00-21swapper/010:13:510
22295211211,75sleep00-21swapper/010:13:510
17570211222,72sleep20-21swapper/209:58:472
11436211216,24sleep30-21swapper/309:38:543
571211012,80sleep0726-21awk12:29:040
21758211011,88sleep221803-21ntp_kernel_pll_11:53:572
21758211011,88sleep221803-21ntp_kernel_pll_11:53:572
15961211011,17sleep30-21swapper/311:34:043
2286921090,6sleep00-21swapper/011:58:350
2286921090,6sleep00-21swapper/011:58:350
11350210722,68sleep20-21swapper/209:38:492
782210625,35sleep20-21swapper/212:29:262
16704210618,70sleep016751-21awk08:14:000
1662210521,64sleep31729-21latency12:33:553
31586210415,20sleep30-21swapper/310:43:503
21898210418,11sleep20-21swapper/210:13:092
3091921039,16sleep20-21swapper/212:23:532
15859210318,10sleep20-21swapper/208:11:112
12997210318,11sleep20-21swapper/211:24:222
12997210318,11sleep20-21swapper/211:24:222
1039210320,32sleep20-21swapper/210:48:582
1017821037,43sleep210218-21ntpq09:34:002
10128210318,11sleep20-21swapper/211:17:402
10128210318,11sleep20-21swapper/211:17:402
564210120,26sleep30-21swapper/307:23:363
32139210117,62sleep131560-21/usr/sbin/munin10:44:041
31288210016,34sleep20-21swapper/210:43:282
19530210015,11sleep20-21swapper/211:48:242
75229811,67sleep0761-21diskstats10:48:520
2548729815,11sleep20-21swapper/212:04:552
198929813,68sleep01997-21ntpq09:09:110
1630929818,28sleep20-21swapper/208:13:452
92829719,10sleep20-21swapper/209:04:532
2475229719,29sleep20-21swapper/208:39:012
1010229711,30sleep010142-21memory09:33:580
2862029617,62sleep128637-21open_inodes07:09:051
240612967,17sleep10-21swapper/110:18:571
240612967,17sleep10-21swapper/110:18:571
2192329617,66sleep321990-21swap11:54:023
2192329617,66sleep321990-21swap11:54:023
2903829512,71sleep129046-21cpu08:53:501
2903829512,71sleep129046-21cpu08:53:501
855529420,41sleep30-21swapper/307:48:473
2810229416,63sleep128182-21grep12:14:031
1768629418,54sleep317772-21apt-config08:18:353
278352939,34sleep20-21swapper/210:29:162
278352939,34sleep20-21swapper/210:29:162
2879729218,60sleep328863-21cat08:53:353
2879729218,60sleep328863-21cat08:53:353
2736029223,33sleep10-21swapper/112:13:341
464329117,55sleep34731-21gpgv09:18:353
3139129125,44sleep131513-21latency_hist10:43:361
2459629126,43sleep324636-21fschecks_count12:03:513
1983029113,62sleep319857-21/usr/sbin/munin08:23:563
1983029113,62sleep319857-21/usr/sbin/munin08:23:563
1024129120,54sleep110281-21/usr/sbin/munin09:34:031
275692906,33sleep20-21swapper/208:49:022
275692906,33sleep20-21swapper/208:49:022
465428820,51sleep04736-21cat10:59:090
278328813,60sleep12805-21munin-run09:13:361
953128716,58sleep19570-21latency_hist09:33:351
1647428724,43sleep30-21swapper/309:54:043
1542928613,59sleep115431-21sshd09:50:021
34022846,13sleep00-21swapper/007:31:040
34022846,13sleep00-21swapper/007:31:040
2293028417,40sleep322931-21/usr/sbin/munin11:58:433
1608128420,41sleep116094-21fschecks_count09:53:541
1987528320,31sleep10-21swapper/110:04:071
1032128317,45sleep010466-21latency_hist11:18:350
1725528223,41sleep017291-21idleruntime-cro09:58:330
1278128218,48sleep112929-21awk11:24:051
1278128218,48sleep112929-21awk11:24:051
1004328228,28sleep30-21swapper/309:33:573
584728123,38sleep15830-21apt-get11:03:541
584728123,38sleep15830-21apt-get11:03:541
2845828114,52sleep10-21swapper/110:33:471
267442818,53sleep026887-21grep12:09:050
1609728112,18sleep00-21swapper/008:13:350
571828017,41sleep10-21swapper/109:19:261
2506628018,41sleep025068-21cron10:23:320
2506628018,41sleep025068-21cron10:23:320
3259327912,15sleep30-21swapper/309:04:213
1950927924,29sleep30-21swapper/308:23:363
1950927924,29sleep30-21swapper/308:23:363
831827829,24sleep30-21swapper/307:48:353
831827829,24sleep30-21swapper/307:48:353
2629327815,45sleep326302-21/usr/sbin/munin08:43:583
2487927816,23sleep00-21swapper/012:03:560
1804827812,51sleep118116-21latency_hist11:43:351
1757127821,25sleep30-21swapper/309:58:483
3134227714,46sleep031390-21wc10:43:350
1801127714,47sleep118022-21ntp_states09:58:581
1310827717,24sleep00-21swapper/009:43:550
126502775,26sleep20-21swapper/207:59:452
3266827620,24sleep10-21swapper/112:28:561
3257027616,43sleep332632-21cat12:28:543
713927512,52sleep07179-21ps09:24:010
2425927516,44sleep324305-21latency_hist08:38:373
1351027523,29sleep10-21swapper/108:03:591
1351027523,29sleep10-21swapper/108:03:591
639327412,52sleep00-21swapper/007:39:200
2887027423,30sleep00-21swapper/008:53:360
2887027423,30sleep00-21swapper/008:53:360
2054727416,39sleep120689-21/usr/sbin/munin11:49:041
2054727416,39sleep120689-21/usr/sbin/munin11:49:041
148027416,27sleep00-21swapper/012:33:520
106522740,13sleep30-21swapper/307:54:033
834327316,30sleep00-21swapper/007:48:360
834327316,30sleep00-21swapper/007:48:360
765827316,41sleep37750-21/usr/sbin/munin11:08:573
60727319,23sleep00-21swapper/009:04:350
2693827321,19sleep10-21swapper/107:04:041
2693827321,19sleep10-21swapper/107:04:041
2138227219,27sleep10-21swapper/111:53:481
2138227219,27sleep10-21swapper/111:53:481
197727222,24sleep00-21swapper/007:27:410
364927115,23sleep10-21swapper/107:33:351
364927115,23sleep10-21swapper/107:33:351
2591327125,32sleep10-21swapper/110:24:061
1813127110,26sleep10-21swapper/108:18:451
1813127110,26sleep10-21swapper/108:18:451
1268227123,25sleep00-21swapper/009:43:350
2977427016,37sleep329892-21users07:13:393
2349127015,41sleep123550-21basename10:18:321
2349127015,41sleep123550-21basename10:18:321
2334627015,16sleep10-21swapper/111:58:571
2255627013,43sleep122579-21latency_hist08:33:351
386826922,25sleep10-21swapper/109:15:011
3050526916,33sleep00-21swapper/012:23:340
27572696,46sleep30-21swapper/307:29:013
27572696,46sleep30-21swapper/307:29:013
32742688,47sleep00-21swapper/009:13:430
27822687,14sleep00-21swapper/010:54:080
1177626713,38sleep011836-21/usr/sbin/munin07:58:580
404526614,22sleep10-21swapper/110:58:501
3045826611,42sleep330502-21/usr/sbin/munin12:23:333
2849526613,36sleep328531-21df10:33:493
2633226626,21sleep20-21swapper/207:03:492
2633226626,21sleep20-21swapper/207:03:492
1078726612,28sleep10-21swapper/111:18:561
910526522,19sleep00-21swapper/007:49:050
595326514,33sleep35962-21/usr/sbin/munin11:04:213
595326514,33sleep35962-21/usr/sbin/munin11:04:213
3107526521,28sleep131082-21sshd08:59:021
3107526521,28sleep131082-21sshd08:59:021
2888026521,23sleep30-21swapper/312:18:333
2601526517,23sleep00-21swapper/008:43:510
2533126517,29sleep00-21swapper/010:23:440
1578326516,34sleep015929-21cat09:53:360
1037626517,32sleep00-21swapper/007:53:570
550026418,20sleep30-21swapper/311:03:363
524526417,23sleep10-21swapper/107:38:361
409526421,19sleep00-21swapper/007:33:550
2874626413,35sleep128781-21munin-run08:53:331
2874626413,35sleep128781-21munin-run08:53:331
1575726416,34sleep315781-21cut09:53:353
948426311,23sleep00-21swapper/009:33:340
676226319,30sleep36783-21fschecks_count09:23:523
2912626318,22sleep30-21swapper/308:53:563
2912626318,22sleep30-21swapper/308:53:563
1427126315,31sleep114387-21/usr/sbin/munin11:29:061
1300126322,19sleep10-21swapper/108:03:351
806426221,21sleep00-21swapper/011:09:060
629826217,29sleep36367-21/usr/sbin/munin07:39:063
454326212,34sleep34659-21/usr/sbin/munin07:34:063
2876926217,21sleep00-21swapper/007:09:080
2494126222,18sleep00-21swapper/008:39:070
1682826220,24sleep00-21swapper/011:38:530
87626120,18sleep30-21swapper/307:23:523
3206126122,16sleep00-21swapper/012:28:350
2641426116,30sleep126446-21ps08:44:131
223332616,10sleep30-21swapper/308:31:153
1959426122,22sleep10-21swapper/111:48:351
193626116,29sleep11972-21basename10:53:331
198726029,16sleep30-21swapper/310:53:353
1536726020,18sleep00-21swapper/011:33:530
143762606,17sleep30-21swapper/308:07:043
143762606,17sleep30-21swapper/308:07:043
883225922,21sleep30-21swapper/309:29:053
3212725918,20sleep00-21swapper/009:03:380
3212725918,20sleep00-21swapper/009:03:380
2928625917,29sleep129336-21fschecks_count12:18:511
2438025918,20sleep30-21swapper/308:38:503
3243025817,25sleep132404-21apt-get09:04:031
2989425816,27sleep029958-21munin-node07:13:400
2839525814,17sleep30-21swapper/307:08:593
2817225813,20sleep00-21swapper/010:33:330
2817225813,20sleep00-21swapper/010:33:330
2711125813,19sleep10-21swapper/110:28:541
2711125813,19sleep10-21swapper/110:28:541
1849725816,17sleep00-21swapper/011:43:540
124812588,14sleep10-21swapper/109:41:591
1051525816,25sleep310512-21apt-get11:18:463
1008025816,25sleep110099-21apt-get07:53:461
933425713,19sleep00-21swapper/011:13:590
864225715,26sleep10-21swapper/107:48:541
718225717,23sleep17273-21mailstats09:24:021
532425721,21sleep30-21swapper/309:19:153
3170125714,22sleep10-21swapper/107:18:501
1991025718,17sleep30-21swapper/311:48:503
1991025718,17sleep30-21swapper/311:48:503
1735425715,22sleep30-21swapper/311:39:053
1603525714,18sleep30-21swapper/308:13:263
3245625613,17sleep00-21swapper/010:44:070
3056025617,19sleep10-21swapper/107:14:011
1939725614,23sleep30-21swapper/310:03:573
3145725512,17sleep10-21swapper/112:24:041
2972325513,26sleep029848-21/usr/sbin/munin08:54:090
2972325513,26sleep029848-21/usr/sbin/munin08:54:090
2633025514,20sleep30-21swapper/312:08:563
2068125512,17sleep10-21swapper/110:08:511
1954125514,24sleep319575-21munin-run11:48:333
1401725511,29sleep314063-21ntpq11:29:003
1369625515,24sleep013700-21diskstats11:28:520
554525312,24sleep15549-21diskstats07:38:521
2853325314,15sleep00-21swapper/010:33:500
2198725319,17sleep00-21swapper/010:13:350
1884025314,23sleep018874-21munin-run10:03:340
1675225311,27sleep116789-21ntp_states08:14:011
1515825312,25sleep115152-21apt-get11:33:461
1323925315,20sleep00-21swapper/008:03:500
1323925315,20sleep00-21swapper/008:03:500
893225210,25sleep18926-21apt-get11:13:481
893225210,25sleep18926-21apt-get11:13:481
348625213,22sleep13564-21apt-config10:58:331
3043025213,25sleep330467-21sed10:38:573
271725212,22sleep12804-21gpgv12:38:331
2387825214,22sleep323884-21df_inode10:18:513
2387825214,22sleep323884-21df_inode10:18:513
2281725211,25sleep022798-21apt-get08:33:580
1475825211,22sleep114777-21apt-get08:08:441
1027625213,22sleep310316-21cut11:18:343
1027625213,22sleep310316-21cut11:18:343
2484025112,20sleep124848-21ntp_states08:39:041
2140625111,21sleep00-21swapper/008:28:570
1968325112,18sleep00-21swapper/008:23:450
1968325112,18sleep00-21swapper/008:23:450
116452515,35sleep00-21swapper/011:21:540
3082225013,22sleep330829-21df_inode12:23:503
289125015,21sleep32958-21latency_hist09:13:383
230525013,21sleep32311-21df_inode10:53:553
2067525011,23sleep320668-21apt-get10:08:493
139652505,35sleep10-21swapper/109:46:071
125225013,19sleep10-21swapper/107:24:031
49724916,14sleep00-21swapper/007:23:330
206602485,32sleep10-21swapper/108:24:391
206602485,32sleep10-21swapper/108:24:391
2770924714,17sleep00-21swapper/012:13:540
250162479,15sleep10-21swapper/112:04:011
1206424712,17sleep30-21swapper/311:23:473
1206424712,17sleep30-21swapper/311:23:473
119572456,16sleep30-21swapper/307:59:013
786324311,15sleep10-21swapper/107:44:091
786324311,15sleep10-21swapper/107:44:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional