You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 10:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat May 30, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21388245179,285sleep021456-21latency_hist20:43:330
1263924096,390sleep012680-21ntpq20:13:550
2434624067,386sleep024393-21ntpq22:33:560
2434624067,386sleep024393-21ntpq22:33:560
146423969,375sleep01502-21grep19:38:490
3203123400,9sleep10-21swapper/119:33:321
1606423187,231sleep016216-21sed23:49:010
1069122955,69sleep00-21swapper/021:49:010
1069122955,69sleep00-21swapper/021:49:010
18970226485,163sleep019041-21true22:18:310
1139222738,21sleep00-21swapper/019:38:320
10426221212,184sleep210427-21http23:33:462
1506220111,174sleep11586-21fw_packets19:38:501
24486218716,154sleep124437-21latency_hist00:18:321
24486218716,154sleep124437-21latency_hist00:18:321
22144218726,24sleep00-21swapper/022:28:310
22144218726,24sleep00-21swapper/022:28:310
11551218311,156sleep311611-21fschecks_time21:53:493
27153218012,31sleep027222-21fschecks_time22:43:490
27153218012,31sleep027222-21fschecks_time22:43:490
30055217219,131sleep130197-21unixbench_multi00:34:011
2667421710,7sleep20-21swapper/200:23:562
17967217016,137sleep218028-21latency22:13:532
11765217013,36sleep011878-21latency_hist23:38:320
8083216719,133sleep28131-21unin-run21:43:312
8083216719,133sleep28131-21unin-run21:43:312
2944216512,135sleep0363-21snmpd23:08:580
17358216215,132sleep117396-21processes23:53:581
5006216116,121sleep00-21swapper/021:33:320
18148215613,125sleep018164-21modprobe23:58:290
26629215017,115sleep026736-21timerandwakeup19:13:590
19773214810,124sleep019859-21mailstats22:18:590
8455214679,54sleep08464-21forks21:43:490
286162141101,20sleep00-21swapper/022:48:450
2112621399,119sleep0363-21snmpd00:06:240
2112621399,119sleep0363-21snmpd00:06:240
290962138113,15sleep0363-21snmpd21:08:060
10225213868,60sleep0363-21snmpd20:06:000
18262213610,21sleep00-21swapper/020:33:320
15608213629,25sleep00-21swapper/020:23:520
15608213629,25sleep00-21swapper/020:23:520
31613213515,28sleep20-21swapper/221:14:132
24028213512,110sleep224064-21fschecks_count22:33:502
24028213512,110sleep224064-21fschecks_count22:33:502
8394213310,28sleep08440-21latency_hist23:28:310
8394213310,28sleep08440-21latency_hist23:28:310
22266213326,82sleep10-21swapper/100:09:031
22266213326,82sleep10-21swapper/100:09:031
12447213312,20sleep00-21swapper/023:38:560
325532132105,15sleep032567-21munin-run21:18:310
14032213114,14sleep00-21swapper/020:18:510
30468212916,99sleep130438-21latency_hist19:28:331
22824212932,76sleep022905-21wc19:03:590
641821287,48sleep00-21swapper/021:37:230
31425212819,30sleep031428-21cron22:58:290
4565212711,98sleep04607-21fschecks_count19:48:490
13759212711,100sleep013779-21apt-get23:43:400
10712212719,86sleep310396-21/usr/sbin/munin23:34:033
3055321266,10sleep00-21swapper/021:11:450
24443212416,89sleep124466-21latency_hist20:53:301
23696212210,30sleep023826-21latency_hist22:33:320
17784212225,70sleep30-21swapper/322:13:503
30714212011,19sleep00-21swapper/000:38:320
739821159,91sleep07476-21switchtime21:39:010
739821159,91sleep07476-21switchtime21:39:010
3916211510,92sleep03923-21acpi23:13:380
2724921155,11sleep00-21swapper/000:24:520
25102211420,30sleep10-21swapper/120:53:551
13510211471,27sleep00-21swapper/021:58:580
32005211311,80sleep00-21swapper/019:33:320
26284211369,21sleep00-21swapper/020:58:420
2317421139,26sleep023190-21grep20:48:450
1218211314,82sleep21244-21netstat23:03:562
1887721128,27sleep018913-21ntpq23:58:580
14085211218,28sleep10-21swapper/120:18:521
11837211110,79sleep00-21swapper/021:53:540
876621105,11sleep00-21swapper/020:02:090
2895321106,11sleep00-21swapper/000:31:500
29239210947,43sleep00-21swapper/019:23:480
27545210923,69sleep127610-21sendmail_mailqu22:43:581
27545210923,69sleep127610-21sendmail_mailqu22:43:581
19839210916,73sleep020002-21latency_hist20:38:320
27911210619,67sleep127950-21fschecks_count00:28:491
1009821060,8sleep10-21swapper/121:48:491
1009821060,8sleep10-21swapper/121:48:491
2429721055,68sleep00-21swapper/000:17:050
2429721055,68sleep00-21swapper/000:17:050
10471210518,23sleep30-21swapper/320:08:333
3011210362,24sleep00-21swapper/019:43:490
27877210375,16sleep027925-21latency19:18:510
580521029,15sleep00-21swapper/023:18:510
29641210216,65sleep229722-21mailstats19:23:582
29403210228,45sleep329422-21apt-get21:08:413
25707210110,75sleep225709-21sshd20:54:522
2472221017,14sleep00-21swapper/000:18:430
2450821017,14sleep00-21swapper/019:09:350
2450821017,14sleep00-21swapper/019:09:350
201521015,11sleep00-21swapper/023:07:210
30426210017,28sleep20-21swapper/200:35:522
2158221009,75sleep00-21swapper/000:08:460
2158221009,75sleep00-21swapper/000:08:460
1918129826,50sleep119327-21sed20:34:011
318402976,11sleep00-21swapper/022:58:500
318402976,11sleep00-21swapper/022:58:500
2036129618,57sleep120446-21/usr/sbin/munin20:38:541
2924629512,68sleep329380-21timerwakeupswit22:49:003
118372955,12sleep00-21swapper/020:11:490
93129411,69sleep1932-21sshd19:36:521
267462945,11sleep00-21swapper/022:43:040
13262945,11sleep00-21swapper/021:19:520
103329418,58sleep3825-21/usr/sbin/munin23:03:533
172602935,10sleep00-21swapper/022:11:330
158362935,12sleep00-21swapper/022:08:100
60429215,59sleep2709-21latency_hist23:03:322
60429215,59sleep2709-21latency_hist23:03:322
2601429215,62sleep226093-21sendmail_mailtr22:38:582
24342926,11sleep00-21swapper/021:24:000
215932925,19sleep00-21swapper/020:43:370
1397029219,54sleep213993-21fschecks_count23:44:352
63582917,68sleep00-21swapper/019:53:540
193712917,49sleep20-21swapper/222:18:492
1208929115,22sleep10-21swapper/123:38:491
96632905,11sleep00-21swapper/023:30:330
305629013,57sleep13135-21fw_packets19:43:501
256602906,11sleep00-21swapper/020:54:140
2133429021,43sleep220809-21/usr/sbin/munin22:23:582
2133429021,43sleep220809-21/usr/sbin/munin22:23:582
1674029019,45sleep216898-21latency_hist20:28:342
1517129031,41sleep215329-21latency_hist20:23:332
1548528922,51sleep215533-21fschecks_count20:23:502
2133628719,51sleep221337-21cron20:43:312
2886628527,38sleep328881-21apt19:23:303
1394028513,56sleep314030-21fschecks_time20:18:503
3123328419,48sleep131207-21apt-get21:13:561
45242830,7sleep30-21swapper/323:13:573
1692428321,27sleep20-21swapper/220:28:422
1173428321,37sleep211820-21sshd21:53:522
2673928227,38sleep126899-21unixbench_singl19:14:001
2440428126,35sleep324442-21munin-run00:18:303
2440428126,35sleep324442-21munin-run00:18:303
1353928111,55sleep113622-21sendmail_mailtr21:58:591
105928121,42sleep21073-21modprobe19:38:302
490528011,49sleep10-21swapper/121:32:521
3081528017,44sleep230709-21latency_hist21:13:332
2769428018,27sleep10-21swapper/119:18:481
2753528026,34sleep30-21swapper/300:28:303
2959027923,20sleep20-21swapper/222:49:522
1665127916,50sleep316668-21cut23:53:323
44227819,43sleep3548-21fschecks_time21:18:503
3248727815,39sleep30-21swapper/321:18:303
236727816,48sleep32432-21processes21:23:593
2134827819,31sleep20-21swapper/200:08:322
2134827819,31sleep20-21swapper/200:08:322
1894227825,35sleep30-21swapper/323:59:013
887327721,41sleep28952-21systemctl21:43:592
780827630,26sleep20-21swapper/223:24:012
780827630,26sleep20-21swapper/223:24:012
2047927617,43sleep320525-21/usr/sbin/munin20:38:563
1299827616,16sleep30-21swapper/320:14:023
2598227516,16sleep10-21swapper/122:38:571
2296527523,26sleep10-21swapper/120:48:341
1010727520,40sleep110262-21date23:33:331
1986027416,37sleep119935-21switchtime22:19:001
1079227419,28sleep20-21swapper/220:08:492
444727311,45sleep24523-21/usr/sbin/munin23:13:572
2733527318,25sleep30-21swapper/322:43:523
2733527318,25sleep30-21swapper/322:43:523
2402027320,28sleep10-21swapper/122:33:491
2402027320,28sleep10-21swapper/122:33:491
1046127324,25sleep210470-21pluginstate20:08:322
666727216,39sleep26668-21sshd23:21:522
629527220,29sleep36001-21/usr/sbin/munin19:53:533
2758827217,39sleep127559-21latency_hist00:28:321
2481027214,45sleep224897-21fschecks_time00:18:492
1196827218,30sleep10-21swapper/120:13:321
112327212,42sleep11217-21memory23:03:551
684727116,37sleep16978-21latency_hist23:23:321
2449427123,25sleep30-21swapper/320:53:323
2356627120,31sleep123615-21munin-run19:08:311
1510127119,25sleep30-21swapper/323:48:323
311812705,21sleep10-21swapper/122:55:241
2729027012,37sleep327301-21modprobe19:18:303
250527016,23sleep10-21swapper/121:24:011
1979127013,33sleep20-21swapper/200:03:322
2668026911,13sleep10-21swapper/120:58:551
2545526921,25sleep325607-21memory19:13:343
2545526921,25sleep325607-21memory19:13:343
225752697,19sleep30-21swapper/300:10:063
225752697,19sleep30-21swapper/300:10:063
2011926920,27sleep120211-21tune2fs00:03:511
2011926920,27sleep120211-21tune2fs00:03:511
198326925,25sleep20-21swapper/221:23:502
1823426923,27sleep218217-40mandb23:58:312
389926823,25sleep20-21swapper/219:44:522
3123926815,23sleep30-21swapper/319:28:593
3078026815,27sleep20-21swapper/222:53:592
2766326815,36sleep327822-21sed21:03:333
2258326822,28sleep222622-21memory19:03:532
1666026817,32sleep316661-21cron20:28:313
1666026817,32sleep316661-21cron20:28:313
1207326816,19sleep30-21swapper/323:38:473
2807126721,25sleep20-21swapper/219:18:582
2609426713,39sleep326191-21sed22:38:593
2375926724,23sleep20-21swapper/220:48:592
32426621,20sleep30-21swapper/319:34:003
2826426613,37sleep228298-21/usr/sbin/munin21:03:562
2586026618,24sleep20-21swapper/219:13:372
2432426611,26sleep10-21swapper/119:09:321
2432426611,26sleep10-21swapper/119:09:321
227872669,42sleep222836-21ntp_states22:28:542
2180826618,23sleep30-21swapper/320:43:523
2005726615,23sleep30-21swapper/322:19:023
1746126622,24sleep20-21swapper/222:13:322
989326519,30sleep29910-21apt-get23:33:302
3087626517,32sleep331033-21unixbench_singl22:54:003
1000126513,37sleep310005-21diskstats21:48:483
1000126513,37sleep310005-21diskstats21:48:483
968326418,20sleep30-21swapper/321:48:323
952326414,32sleep19532-21/usr/sbin/munin20:03:551
3238426421,27sleep332515-21sed22:59:013
3238426421,27sleep332515-21sed22:59:013
2917226418,30sleep129191-21unin-run21:08:311
2795326413,33sleep228033-21fw_packets00:28:502
178226424,21sleep20-21swapper/219:38:552
1117226417,21sleep30-21swapper/320:08:583
1039426417,27sleep10-21swapper/123:33:411
618026321,22sleep10-21swapper/119:53:511
2910026321,23sleep10-21swapper/122:48:581
2163726312,33sleep121735-21users22:24:011
2163726312,33sleep121735-21users22:24:011
1587726316,21sleep20-21swapper/223:48:592
2836126215,25sleep30-21swapper/300:28:593
2237726214,21sleep30-21swapper/319:03:503
1904226210,33sleep10-21swapper/122:18:321
1827526224,19sleep30-21swapper/323:58:323
918726118,22sleep30-21swapper/323:28:593
747826112,22sleep10-21swapper/121:39:021
747826112,22sleep10-21swapper/121:39:021
2500026117,27sleep325070-21ls20:53:533
1900626118,21sleep30-21swapper/320:33:593
134842618,12sleep10-21swapper/120:17:551
837126022,20sleep10-21swapper/121:43:461
716626011,34sleep17170-21diskstats23:23:481
716626011,34sleep17170-21diskstats23:23:481
3000726017,26sleep130106-21/usr/sbin/munin21:09:001
2299026018,24sleep323101-21ls22:28:593
1460426014,27sleep10-21swapper/122:03:471
1283826017,21sleep20-21swapper/223:39:022
126932606,12sleep10-21swapper/121:58:021
50232595,13sleep20-21swapper/219:48:592
2441025919,23sleep323917-21/usr/sbin/munin22:33:593
2441025919,23sleep323917-21/usr/sbin/munin22:33:593
166125917,20sleep10-21swapper/121:23:321
1047425919,18sleep20-21swapper/221:48:592
1047425919,18sleep20-21swapper/221:48:592
495425820,24sleep24961-21modprobe21:33:302
316282586,13sleep20-21swapper/219:29:102
2977125812,32sleep329805-21ntp_kernel_pll_00:33:553
2842025821,19sleep30-21swapper/321:04:013
266525814,19sleep10-21swapper/123:08:511
2521325818,24sleep30-21swapper/300:18:583
248225817,17sleep20-21swapper/223:08:482
1670425816,21sleep10-21swapper/120:28:331
1670425816,21sleep10-21swapper/120:28:331
71142575,11sleep20-21swapper/219:56:182
64462576,11sleep30-21swapper/321:37:473
228592577,11sleep10-21swapper/122:28:571
892125613,26sleep39063-21latency_hist20:03:333
537025618,18sleep10-21swapper/119:49:031
1615025619,22sleep116145-21apt-get22:08:431
1435425612,21sleep20-21swapper/222:03:322
44102559,32sleep14444-21ps23:13:561
2922925521,15sleep30-21swapper/319:23:473
2747625514,23sleep227521-21ntp_states22:43:552
2747625514,23sleep227521-21ntp_states22:43:552
2091125516,16sleep30-21swapper/322:23:463
2091125516,16sleep30-21swapper/322:23:463
1436525520,17sleep30-21swapper/322:03:333
1358525510,19sleep10-21swapper/123:43:331
496225417,23sleep35004-21munin-run21:33:313
2518225415,23sleep125188-21ntp_states00:18:561
231012549,18sleep20-21swapper/200:13:402
231012549,18sleep20-21swapper/200:13:402
2171225415,16sleep20-21swapper/220:43:512
1500225415,19sleep30-21swapper/322:03:573
887125314,24sleep18901-21munin-run20:03:311
83722539,20sleep20-21swapper/219:59:002
689225314,24sleep36898-21df_inode21:38:483
689225314,24sleep36898-21df_inode21:38:483
2045125312,24sleep320486-21ntp_states00:03:573
2045125312,24sleep320486-21ntp_states00:03:573
1639625312,27sleep116397-21sshd23:50:521
1287425317,16sleep20-21swapper/220:14:012
578325217,19sleep30-21swapper/319:53:323
319425219,18sleep30-21swapper/319:43:523
2539625211,15sleep20-21swapper/219:13:332
2539625211,15sleep20-21swapper/219:13:332
1932825212,24sleep219401-21users20:34:022
1591325216,20sleep215984-21latency_hist22:08:322
607125112,15sleep30-21swapper/323:18:583
257825118,15sleep30-21swapper/323:08:493
1698925112,23sleep317022-21df20:28:473
1692225112,16sleep30-21swapper/322:09:023
164525114,20sleep30-21swapper/319:38:523
708025012,21sleep37078-21apt-get23:23:423
708025012,21sleep37078-21apt-get23:23:423
533025014,17sleep20-21swapper/221:33:502
2652025012,17sleep20-21swapper/220:58:522
2168225012,21sleep221792-21perl00:08:522
2168225012,21sleep221792-21perl00:08:522
1740724910,24sleep117413-21ntp_states20:28:571
568424712,19sleep15700-21ntp_states21:33:571
34224710,19sleep20-21swapper/221:18:452
3174224111,15sleep10-21swapper/122:58:431
3174224111,15sleep10-21swapper/122:58:431
1847723910,15sleep10-21swapper/123:58:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional