You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 01:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 31, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30264263626,6345sleep325885-21kworker/3:009:23:343
19414247257,4706sleep313403-21kworker/3:108:49:343
102752317520,3141sleep224366-21kworker/2:208:22:002
24489210689,1044sleep324533-21ntpq12:33:093
24489210689,1044sleep324533-21ntpq12:33:093
3076125546,535sleep330807-21ntpq07:43:083
7051242278,227sleep37130-21fschecks_time11:38:063
828323998,377sleep38371-21mailstats09:58:143
18155235767,102sleep30-21swapper/308:47:453
19185230329,207sleep319224-21latency_hist12:17:433
19185230329,207sleep319224-21latency_hist12:17:433
25105230212,273sleep125113-21ntpq09:08:101
25105230212,273sleep125113-21ntpq09:08:101
23944227624,199sleep323949-21fschecks_count10:48:043
15761225675,72sleep30-21swapper/308:38:123
24763223645,177sleep324852-21timerandwakeup07:23:143
24763223645,177sleep324852-21timerandwakeup07:23:143
15342223154,163sleep315430-21timerandwakeup12:03:143
23920222310,117sleep323927-21cat07:22:453
1264222177,172sleep312706-21ps08:28:153
16199221463,138sleep316235-21ntpq10:23:093
273252209129,36sleep30-21swapper/310:58:093
3122322008,155sleep31-21systemd11:11:343
6842219329,123sleep36975-21sort09:53:143
4410219111,47sleep30-21swapper/309:47:453
111621816,160sleep31157-21ntpq11:18:083
460321759,114sleep30-21swapper/311:28:133
19076217275,77sleep30-21swapper/307:07:593
11935216448,101sleep312073-21latency_hist08:27:463
22612216263,42sleep30-21swapper/312:28:013
14011216153,88sleep314060-21latency_hist10:17:443
5382215910,128sleep35448-21cpuspeed11:32:573
20170215648,15sleep30-21swapper/312:18:123
20170215648,15sleep30-21swapper/312:18:123
259721518,57sleep30-21swapper/307:57:583
1758215018,105sleep01886-21grep07:53:150
13722214875,15sleep30-21swapper/308:32:593
4728214712,43sleep30-21swapper/308:03:153
16899214715,118sleep117024-21timerandwakeup12:08:121
144372147108,17sleep30-21swapper/310:18:043
10386214611,60sleep30-21swapper/308:22:453
410214418,108sleep20-21swapper/211:17:422
861321420,9sleep20-21swapper/211:43:012
6262213813,108sleep16272-21df09:52:591
21502213731,21sleep30-21swapper/308:57:543
20483213614,63sleep30-21swapper/310:37:463
20483213614,63sleep30-21swapper/310:37:463
1168821369,113sleep311759-21cut11:53:003
10608213311,108sleep310680-21sort11:48:123
10939213110,61sleep30-21swapper/310:07:473
2734221298,106sleep327431-21grep07:33:033
24344212426,74sleep224363-21idleruntime-cro09:07:422
10410212113,79sleep10-21swapper/111:48:061
31703211840,64sleep331793-21cut11:13:033
30674211818,79sleep230827-21latency_hist09:27:452
18071211834,41sleep317324-21/usr/sbin/munin10:28:143
21210211733,71sleep3225-21systemd-journal08:57:423
21210211733,71sleep3225-21systemd-journal08:57:423
2047211523,78sleep02084-21latency_hist11:22:450
1448211514,83sleep21593-21/usr/sbin/munin11:18:152
7972211468,11sleep30-21swapper/308:13:253
3052721140,38sleep230530-21fschecks_time07:43:022
2554211424,35sleep30-21swapper/311:23:043
2554211424,35sleep30-21swapper/311:23:043
13762211423,23sleep30-21swapper/311:58:133
2646521139,21sleep30-21swapper/309:13:043
26106211316,82sleep226142-21ntpq07:28:092
26106211316,82sleep226142-21ntpq07:28:092
22368211329,71sleep322396-21latency_hist07:17:453
9599211116,77sleep29693-21memory10:03:062
29766211013,81sleep029808-21systemd-run11:07:430
17533211023,65sleep017324-21/usr/sbin/munin10:28:020
28828210921,41sleep00-21swapper/007:37:560
11169210918,21sleep30-21swapper/310:08:403
11169210918,21sleep30-21swapper/310:08:403
62221089,83sleep3756-21sed09:33:153
267921085,11sleep30-21swapper/309:41:063
23931210817,72sleep024078-21latency_hist07:22:460
19197210813,80sleep019283-21perl07:08:040
12428210823,59sleep00-21swapper/010:12:450
12428210823,59sleep00-21swapper/010:12:450
28831210740,49sleep30-21swapper/307:37:593
2276210518,23sleep12278-21apt-get11:22:541
2276210518,23sleep12278-21apt-get11:22:541
29399210413,74sleep229456-21switchtime07:38:152
18888210213,73sleep119032-21cat07:07:451
32135210131,18sleep332174-21modprobe09:32:423
32135210131,18sleep332174-21modprobe09:32:423
16556210011,68sleep20-21swapper/208:42:432
61632999,50sleep36205-21proc_pri08:08:123
2301529919,67sleep123092-21mailstats12:28:111
174502995,11sleep30-21swapper/312:11:333
323912989,75sleep332425-21ps07:48:133
49429710,72sleep0625-21latency_hist11:17:440
295272975,36sleep30-21swapper/311:04:393
440429611,69sleep14474-21/usr/sbin/munin11:28:111
230622969,44sleep30-21swapper/309:02:553
540929514,62sleep25413-21modprobe08:07:422
2786629511,64sleep10-21swapper/110:58:191
535329414,62sleep25355-21/usr/sbin/munin11:32:522
2282429411,19sleep30-21swapper/310:43:153
3136629319,54sleep031328-21latency_hist11:12:440
859429213,65sleep18612-21fschecks_count11:43:011
296429213,66sleep02979-21netstat07:58:070
2663229226,44sleep126659-21munin-run10:57:431
2205529114,61sleep122199-21latency_hist10:42:451
2080229122,49sleep120899-21fschecks_count07:12:501
280729014,60sleep12826-21modprobe09:42:421
2804529024,28sleep00-21swapper/009:18:060
2628029018,54sleep30-21swapper/307:28:143
2628029018,54sleep30-21swapper/307:28:143
2617128913,60sleep026190-21apt-get09:12:530
1334828919,44sleep213496-21grep10:13:152
435828819,50sleep14378-21modprobe09:47:421
3218728835,31sleep10-21swapper/109:32:441
3218728835,31sleep10-21swapper/109:32:441
2357228811,61sleep023649-21tr09:03:120
916128717,54sleep19193-21tune2fs08:18:031
757628615,56sleep27673-21timerandwakeup11:38:172
2523228512,24sleep00-21swapper/009:08:130
2523228512,24sleep00-21swapper/009:08:130
762528412,58sleep27626-21fschecks_count08:13:152
286128418,45sleep02985-21cat09:42:450
2851228420,42sleep228540-21munin-run07:37:422
2227828424,32sleep10-21swapper/110:43:011
2160028410,32sleep30-21swapper/310:38:353
2051728327,36sleep00-21swapper/010:37:470
2051728327,36sleep00-21swapper/010:37:470
10202838,33sleep30-21swapper/307:52:583
1711028219,33sleep20-21swapper/210:27:442
431828122,24sleep00-21swapper/011:28:100
1637728012,55sleep016446-21switchtime10:23:140
3232627914,43sleep031904-21/usr/sbin/munin07:48:100
2258327917,42sleep122602-21apt-get07:17:551
14362797,16sleep20-21swapper/207:53:092
267112787,14sleep30-21swapper/310:57:453
2454427810,46sleep10-21swapper/112:33:111
2454427810,46sleep10-21swapper/112:33:111
480027719,33sleep00-21swapper/008:03:160
1944727725,35sleep119486-21ntpq07:08:101
2515227624,40sleep325224-21mailstats09:08:123
2515227624,40sleep325224-21mailstats09:08:123
2179127611,46sleep220996-21/usr/sbin/munin07:13:152
1683127617,30sleep00-21swapper/012:08:110
131527617,28sleep10-21swapper/109:37:451
1108227623,35sleep110599-21apt-get10:07:531
1108227623,35sleep110599-21apt-get10:07:531
1885927521,38sleep118897-21proc_pri08:48:111
1810527418,41sleep218154-21/usr/sbin/munin08:47:442
1711727412,49sleep117140-21cut08:43:061
3095027317,28sleep00-21swapper/007:43:130
2202627324,29sleep00-21swapper/010:42:440
303542729,36sleep10-21swapper/107:42:541
2909027223,29sleep20-21swapper/209:22:442
2435427221,21sleep00-21swapper/012:33:060
2435427221,21sleep00-21swapper/012:33:060
2284827222,26sleep10-21swapper/109:02:451
2250927210,37sleep00-21swapper/012:27:540
215572729,47sleep30-21swapper/312:23:123
215572729,47sleep30-21swapper/312:23:123
1081827223,31sleep10-21swapper/108:23:041
391627125,21sleep20-21swapper/208:02:452
277812718,47sleep30-21swapper/309:17:573
2441927118,29sleep10-21swapper/109:07:451
2390627121,30sleep00-21swapper/010:48:010
1160427112,29sleep00-21swapper/011:52:570
814027018,27sleep10-21swapper/109:58:081
71902700,8sleep10-21swapper/111:38:071
506427037,16sleep30-21swapper/309:48:093
1795927022,23sleep10-21swapper/107:03:181
1645927019,21sleep10-21swapper/110:23:151
1418427014,43sleep014224-21ps08:33:120
815826916,26sleep20-21swapper/209:58:092
2786926912,22sleep10-21swapper/109:18:031
981526819,27sleep00-21swapper/011:47:440
2941726824,21sleep10-21swapper/109:23:031
1874726814,36sleep318776-21/usr/sbin/munin07:07:433
1874726814,36sleep318776-21/usr/sbin/munin07:07:433
1817326818,18sleep00-21swapper/007:03:210
1817326818,18sleep00-21swapper/007:03:210
136012685,11sleep10-21swapper/110:13:341
1172026826,23sleep20-21swapper/210:08:552
1172026826,23sleep20-21swapper/210:08:552
1021626814,38sleep210274-21idleruntime11:48:032
81082678,11sleep10-21swapper/108:13:271
123126721,21sleep20-21swapper/209:37:422
1099026711,41sleep011013-21/usr/sbin/munin08:23:070
883926616,36sleep08894-21sed08:17:460
3096126613,31sleep10-21swapper/111:08:171
2695326622,30sleep227004-21cpuspeed10:58:002
2549626618,20sleep20-21swapper/212:37:452
2549626618,20sleep20-21swapper/212:37:452
1705826615,17sleep00-21swapper/008:43:050
1348326622,20sleep10-21swapper/108:32:451
134212665,11sleep00-21swapper/008:32:350
981326516,34sleep09885-21mailstats10:03:120
695926521,30sleep26981-21/usr/sbin/munin08:12:432
514026514,32sleep25109-21latency_hist11:32:442
221692655,11sleep10-21swapper/112:26:501
221692655,11sleep10-21swapper/112:26:501
204952656,48sleep20-21swapper/212:20:122
204952656,48sleep20-21swapper/212:20:122
199752656,13sleep10-21swapper/112:18:101
199752656,13sleep10-21swapper/112:18:101
1780626516,24sleep20-21swapper/212:12:542
1780626516,24sleep20-21swapper/212:12:542
1139526515,25sleep00-21swapper/011:52:450
2634326419,24sleep10-21swapper/109:13:021
215302647,13sleep00-21swapper/008:57:590
1943026419,31sleep0369-21sshd12:17:560
1943026419,31sleep0369-21sshd12:17:560
1847926417,23sleep00-21swapper/008:48:020
1493126420,20sleep20-21swapper/210:18:162
1444326415,31sleep014447-21modprobe12:02:420
101126424,20sleep10-21swapper/111:18:061
628726313,31sleep16378-21/usr/sbin/munin08:08:141
267242636,18sleep00-21swapper/007:29:370
267242636,18sleep00-21swapper/007:29:370
2508226315,32sleep225112-21/usr/sbin/munin10:52:432
237326313,20sleep00-21swapper/011:23:010
237326313,20sleep00-21swapper/011:23:010
227622636,12sleep20-21swapper/210:43:142
2035226316,28sleep220393-21ntpq08:53:092
2035226316,28sleep220393-21ntpq08:53:092
1971226320,19sleep20-21swapper/208:52:442
1380126314,27sleep213806-21diskstats08:33:022
968526218,10sleep30-21swapper/311:46:423
587526212,33sleep05954-21/usr/sbin/munin11:33:110
3188926224,21sleep20-21swapper/209:29:352
3188926224,21sleep20-21swapper/209:29:352
2748726223,23sleep00-21swapper/010:58:140
2129026216,23sleep10-21swapper/108:57:441
2129026216,23sleep10-21swapper/108:57:441
212112626,12sleep20-21swapper/210:38:302
1907526214,25sleep20-21swapper/207:07:582
1425326217,26sleep114247-21apt-get10:17:581
139026216,28sleep11424-21ntp_kernel_err07:53:081
3131226116,29sleep231318-21modprobe11:12:422
2152726118,27sleep221523-21apt-get08:57:572
1665126115,27sleep20-21swapper/212:08:052
676926029,16sleep20-21swapper/209:53:132
595526015,28sleep16052-21sed11:33:121
3219026019,19sleep10-21swapper/107:48:071
3059026018,27sleep030635-21systemd-run09:27:430
2973126017,26sleep10-21swapper/107:38:181
2701726021,22sleep10-21swapper/107:32:451
2701726021,22sleep10-21swapper/107:32:451
2344226016,27sleep222501-21/usr/sbin/munin12:28:162
1603426018,18sleep10-21swapper/112:07:441
41232596,15sleep00-21swapper/009:44:430
3259925916,19sleep20-21swapper/207:48:162
2859125921,23sleep00-21swapper/007:37:440
1995425913,29sleep019947-21apt-get08:52:580
1995425913,29sleep019947-21apt-get08:52:580
1907725917,22sleep20-21swapper/210:33:052
1907725917,22sleep20-21swapper/210:33:052
1796125918,19sleep20-21swapper/210:28:132
1605025916,19sleep20-21swapper/212:07:452
1473725915,28sleep00-21swapper/012:02:580
915425818,23sleep09160-21diskstats08:18:020
2906325816,27sleep229142-21sed11:03:132
2850725818,26sleep128504-21apt-get11:02:561
1899825816,24sleep10-21swapper/110:33:041
1899825816,24sleep10-21swapper/110:33:041
1529725816,26sleep015292-21apt-get08:37:570
143392586,17sleep10-21swapper/112:01:221
1070225817,24sleep210706-21diskstats08:23:012
474625718,24sleep14849-21fschecks_time09:48:031
2416025716,24sleep124227-21df07:22:591
2416025716,24sleep124227-21df07:22:591
2165725715,25sleep021781-21sed12:23:130
2165725715,25sleep021781-21sed12:23:130
2071725714,26sleep020801-21munin-node07:12:490
697925618,17sleep00-21swapper/009:53:150
27132566,38sleep20-21swapper/209:41:342
2546925616,19sleep20-21swapper/210:53:032
1160025613,26sleep211594-21apt-get11:52:552
464925517,20sleep00-21swapper/009:47:580
249822554,11sleep10-21swapper/110:51:291
1731425512,24sleep216977-21apt-get07:02:462
1731425512,24sleep216977-21apt-get07:02:462
1286025513,21sleep10-21swapper/111:57:421
1266625514,25sleep012660-21apt-get10:12:570
425325411,18sleep10-21swapper/108:03:051
2860525414,17sleep10-21swapper/107:37:451
2264825412,23sleep222681-21df07:18:002
2228725411,20sleep20-21swapper/212:27:432
2228725411,20sleep20-21swapper/212:27:432
10892545,37sleep00-21swapper/009:35:540
3218325316,17sleep10-21swapper/111:13:131
2545625314,24sleep225547-21users09:08:152
2545625314,24sleep225547-21users09:08:152
696125212,25sleep06957-21apt-get11:37:590
577225213,22sleep05779-21df_inode08:08:000
2541725213,24sleep025454-21modprobe07:27:430
2541725213,24sleep025454-21modprobe07:27:430
1864925213,20sleep118658-21munin-run10:32:441
180742529,14sleep00-21swapper/012:13:040
180742529,14sleep00-21swapper/012:13:040
940425114,21sleep19411-21diskstats10:03:011
907625114,16sleep00-21swapper/010:02:430
2537325112,22sleep00-21swapper/010:52:570
1573425114,21sleep115753-21ntp_states08:38:101
1508825117,17sleep10-21swapper/108:37:451
1450225112,15sleep00-21swapper/010:18:050
3132925011,23sleep031335-21ntp_states09:28:110
3132925011,23sleep031335-21ntp_states09:28:110
2415525014,20sleep224152-21apt-get07:22:562
2415525014,20sleep224152-21apt-get07:22:562
1810925017,18sleep10-21swapper/112:13:051
1810925017,18sleep10-21swapper/112:13:051
1217625012,22sleep212246-21df08:28:012
1317824911,22sleep213174-21apt-get11:57:562
34802488,16sleep10-21swapper/107:58:161
2358424813,17sleep00-21swapper/010:47:440
309424611,17sleep20-21swapper/209:42:552
485124515,15sleep20-21swapper/209:48:042
34002445,30sleep20-21swapper/211:25:542
34002445,30sleep20-21swapper/211:25:542
2753424412,17sleep20-21swapper/209:17:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional