You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 01:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Wed May 06, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2557521790921,17810sleep217241-21kworker/u8:409:45:272
2557521790921,17810sleep217241-21kworker/u8:409:45:272
141112360812,3581sleep314099-21apt-get12:35:083
141112360812,3581sleep314099-21apt-get12:35:083
31024264335,485sleep231107-21head10:00:252
9246249116,457sleep39248-21sshd08:52:383
1177234513,316sleep11317-21cat11:54:561
900023179,100sleep20-21swapper/212:19:562
18145228577,192sleep218231-21ls09:20:232
5177228333,84sleep25266-21fschecks_time08:40:342
16773226895,58sleep20-21swapper/207:34:562
2984922448,225sleep3363-21snmpd09:59:093
2984922448,225sleep3363-21snmpd09:59:093
10484223737,181sleep210509-21idleruntime-cro12:24:532
23447223354,71sleep223450-21cron09:39:542
1936122197,157sleep219416-21cat09:25:162
11930221340,92sleep212050-21timerwakeupswit09:00:252
7391220917,151sleep27437-21latency_hist12:14:562
7391220917,151sleep27437-21latency_hist12:14:562
22455220916,179sleep122509-21sed09:35:161
13390220811,158sleep213480-21latency_hist10:49:562
14000220710,184sleep014079-21fschecks_time07:25:130
15033220534,127sleep215151-21timerwakeupswit09:10:262
20726219917,78sleep220747-21fschecks_count09:30:132
32303218913,121sleep20-21swapper/211:50:082
21878218448,119sleep221939-21sendmail_mailqu11:15:242
3344217648,56sleep20-21swapper/212:00:182
27634216927,93sleep227647-21cpu11:35:102
27634216927,93sleep227647-21cpu11:35:102
12609216620,29sleep10-21swapper/109:04:561
32262216350,31sleep20-21swapper/210:05:222
15450216394,48sleep20-21swapper/212:39:552
15450216394,48sleep20-21swapper/212:39:552
5839215710,125sleep25879-21latency_hist12:09:552
9295215116,120sleep39315-21open_inodes07:10:263
25025215111,121sleep20-21swapper/209:44:542
26729214997,14sleep20-21swapper/208:05:172
26729214997,14sleep20-21swapper/208:05:172
16421214586,17sleep20-21swapper/209:15:232
14980214216,26sleep20-21swapper/210:54:562
25522214013,109sleep025540-21/usr/sbin/munin09:45:050
25522214013,109sleep025540-21/usr/sbin/munin09:45:050
28426213544,77sleep228569-21latency_hist09:54:562
14457213513,16sleep20-21swapper/207:25:242
5370213445,72sleep25484-21latency_hist10:24:562
3803213327,91sleep23880-21sed08:35:252
3656213216,91sleep13794-21grep12:00:251
7322213026,38sleep10-21swapper/107:05:121
7322213026,38sleep10-21swapper/107:05:121
704721279,62sleep27253-21ls10:29:572
2034021279,77sleep220417-21basename11:10:232
32668212512,19sleep20-21swapper/208:25:112
3204521225,11sleep20-21swapper/208:21:362
235621229,97sleep20-21swapper/208:30:242
235621229,97sleep20-21swapper/208:30:242
18375212239,47sleep20-21swapper/211:05:102
23230212023,18sleep20-21swapper/207:55:072
19939212015,32sleep019951-21/usr/sbin/munin11:10:130
14394212018,29sleep10-21swapper/107:25:231
29292211920,77sleep329308-21fschecks_count11:40:123
29292211920,77sleep329308-21fschecks_count11:40:123
3183321185,12sleep20-21swapper/211:47:382
2298721187,86sleep20-21swapper/211:20:082
8225211726,14sleep20-21swapper/208:50:112
21864211717,82sleep121877-21/usr/sbin/munin11:15:231
17710211711,61sleep20-21swapper/207:35:252
10974211721,78sleep010994-21idleruntime-cro08:59:540
12846211630,31sleep20-21swapper/209:05:092
18233211424,74sleep318384-21unixbench_multi09:20:253
10985211418,17sleep20-21swapper/210:40:232
26058211323,59sleep126077-21apt-get11:30:041
26209211215,80sleep10-21swapper/109:45:591
26209211215,80sleep10-21swapper/109:45:591
2733211021,70sleep32750-21cut10:15:163
9288210942,13sleep20-21swapper/207:10:252
8954210939,14sleep20-21swapper/210:35:102
838721085,10sleep20-21swapper/210:31:302
758021085,47sleep20-21swapper/208:46:152
2728521085,10sleep20-21swapper/211:34:092
19856210715,76sleep119909-21ls11:10:101
4677210618,69sleep14875-21apt-key08:39:561
12539210518,70sleep112574-21grep10:45:221
18391210410,80sleep018418-21cut11:05:120
19357210212,45sleep218564-21/usr/sbin/munin07:40:262
13212210218,66sleep313241-21/usr/sbin/munin09:05:183
28651210114,72sleep028721-21df09:55:100
28651210114,72sleep028721-21df09:55:100
12756210117,28sleep20-21swapper/207:20:192
2434329918,66sleep324431-21cat09:40:233
118872996,82sleep20-21swapper/212:28:042
118872996,82sleep20-21swapper/212:28:042
1702329813,64sleep117116-21memory11:00:161
1259129817,69sleep312659-21ls12:30:153
794829726,54sleep28094-21unixbench_singl07:05:252
794829726,54sleep28094-21unixbench_singl07:05:252
568929620,59sleep05749-21fschecks_time10:25:100
2175529610,22sleep20-21swapper/207:50:112
206202967,49sleep20-21swapper/207:45:222
294522948,46sleep20-21swapper/208:15:062
265629437,40sleep20-21swapper/211:59:542
252562947,48sleep20-21swapper/208:00:202
252562947,48sleep20-21swapper/208:00:202
110532946,54sleep211052-21/usr/sbin/munin08:59:562
877029212,66sleep08849-21wc08:50:250
735329224,40sleep37217-21/usr/sbin/munin07:05:143
735329224,40sleep37217-21/usr/sbin/munin07:05:143
2478229210,22sleep30-21swapper/308:00:053
2478229210,22sleep30-21swapper/308:00:053
2941529116,59sleep129455-21/usr/sbin/munin11:40:141
2941529116,59sleep129455-21/usr/sbin/munin11:40:141
3235429017,22sleep30-21swapper/308:24:563
279092906,44sleep20-21swapper/208:10:052
3062028921,54sleep330662-21ntpq10:00:183
1595528917,55sleep116005-21/usr/sbin/munin09:15:101
3252028819,51sleep132661-21grep10:05:251
3201228823,41sleep332039-21modprobe11:49:533
300128812,52sleep33101-21latency_hist08:34:583
300128812,52sleep33101-21latency_hist08:34:583
1603928813,58sleep016051-21/usr/sbin/munin09:15:130
515528719,31sleep00-21swapper/012:05:250
123962877,47sleep20-21swapper/212:30:102
2928528617,14sleep20-21swapper/211:40:112
2928528617,14sleep20-21swapper/211:40:112
563328512,56sleep05700-21awk08:40:440
250092856,44sleep20-21swapper/211:25:212
922228411,60sleep09310-21cat10:35:160
1099228420,47sleep310506-21/usr/sbin/munin10:40:243
2399928310,62sleep00-21swapper/011:21:440
2116628317,52sleep321205-21cat11:14:563
1304728318,50sleep013177-21unixbench_singl12:30:240
1092428317,50sleep110994-21fw_packets07:15:171
786628224,43sleep17887-21munin-run08:49:541
155412828,16sleep20-21swapper/210:55:172
1416028114,16sleep00-21swapper/009:09:560
111862819,16sleep20-21swapper/207:15:222
943728014,51sleep39518-21cut10:35:233
998427912,31sleep30-21swapper/308:55:173
895227920,43sleep18953-21/usr/sbin/munin10:35:091
292232797,36sleep229309-21sed09:55:242
292232797,36sleep229309-21sed09:55:242
480127817,45sleep34814-21kernelversion12:05:163
450127824,37sleep34581-21true08:39:543
195222787,19sleep00-21swapper/007:40:440
620627714,24sleep30-21swapper/310:25:223
2784127612,47sleep027875-21/usr/sbin/munin11:35:160
2784127612,47sleep027875-21/usr/sbin/munin11:35:160
2579527629,24sleep30-21swapper/311:29:543
20652766,39sleep20-21swapper/210:13:252
1126627613,21sleep30-21swapper/309:00:053
841427512,26sleep00-21swapper/012:15:260
3131927513,44sleep031408-21load08:20:170
234342750,8sleep10-21swapper/111:20:181
1021827523,34sleep110225-21munin-node07:14:571
828827419,23sleep30-21swapper/312:15:253
1827027425,30sleep318298-21munin-run07:39:543
176727416,43sleep31807-21grep11:55:183
1408727412,48sleep314128-21ps10:50:213
2749727320,30sleep10-21swapper/109:50:191
171172737,40sleep20-21swapper/211:00:172
149627319,21sleep30-21swapper/310:10:243
39792728,55sleep00-21swapper/012:01:330
2587127222,33sleep126009-21latency_hist11:29:561
885027112,43sleep18973-21unixbench_multi08:50:261
337327130,30sleep20-21swapper/210:15:272
2665427116,37sleep326728-21cat11:30:233
2144927125,23sleep00-21swapper/007:49:560
2022327120,31sleep120283-21fschecks_time07:45:131
1418227124,27sleep30-21swapper/307:25:163
1212827121,25sleep00-21swapper/007:19:560
67827024,31sleep30-21swapper/310:09:553
2967827016,27sleep30-21swapper/308:15:153
1420027016,33sleep114278-21ls10:50:231
75532697,50sleep00-21swapper/008:45:530
744026917,22sleep30-21swapper/312:14:573
2424726919,31sleep324250-21cron11:24:543
151152698,49sleep00-21swapper/012:36:380
151152698,49sleep00-21swapper/012:36:380
527526814,25sleep10-21swapper/112:05:261
1761426818,28sleep10-21swapper/107:35:241
1104226819,24sleep00-21swapper/012:25:150
1104226819,24sleep00-21swapper/012:25:150
1075726825,22sleep30-21swapper/307:15:073
819426724,24sleep30-21swapper/310:31:233
2576726716,37sleep325823-21idleruntime09:45:483
2576726716,37sleep325823-21idleruntime09:45:483
2189926718,34sleep321913-21idleruntime-cro09:34:543
1967126722,23sleep10-21swapper/111:09:571
1730626720,18sleep30-21swapper/309:19:563
148426719,22sleep10-21swapper/111:55:121
324482668,12sleep30-21swapper/311:50:133
32402668,18sleep00-21swapper/008:35:110
2770526618,20sleep10-21swapper/108:09:561
2770526618,20sleep10-21swapper/108:09:561
2767026610,42sleep027704-21latency_hist08:09:550
2767026610,42sleep027704-21latency_hist08:09:550
21832666,12sleep00-21swapper/008:30:220
21832666,12sleep00-21swapper/008:30:220
145226615,39sleep31489-21sed08:29:573
1245126615,34sleep012458-21/usr/sbin/munin07:20:130
63426514,37sleep1720-21mailstats08:25:231
2809426516,25sleep30-21swapper/311:35:243
2809426516,25sleep30-21swapper/311:35:243
1311126525,24sleep10-21swapper/109:05:161
1085926513,27sleep10-21swapper/112:25:121
1085926513,27sleep10-21swapper/112:25:121
959326417,30sleep39627-21/usr/sbin/munin12:20:183
871526424,17sleep00-21swapper/010:34:560
60852647,13sleep00-21swapper/012:10:090
60852647,13sleep00-21swapper/012:10:090
2966926418,16sleep00-21swapper/011:40:210
2966926418,16sleep00-21swapper/011:40:210
2914226416,25sleep10-21swapper/109:55:231
2914226416,25sleep10-21swapper/109:55:231
2772726420,22sleep30-21swapper/309:50:253
251782647,12sleep00-21swapper/008:00:180
251782647,12sleep00-21swapper/008:00:180
1909526411,35sleep00-21swapper/009:25:100
883126311,37sleep00-21swapper/007:10:110
2945926313,23sleep00-21swapper/008:15:080
2385026314,24sleep00-21swapper/009:40:120
1543726320,23sleep30-21swapper/307:30:053
1362426326,22sleep113635-21modprobe07:24:541
1248626313,19sleep10-21swapper/112:30:131
115226318,27sleep00-21swapper/011:54:550
3230426221,26sleep132330-21awk08:24:541
2320626216,28sleep023226-21apt-get07:55:050
1886026216,19sleep10-21swapper/107:40:161
2762326114,31sleep327668-21systemd-run08:09:543
2762326114,31sleep327668-21systemd-run08:09:543
1917826119,25sleep30-21swapper/309:25:133
635326021,20sleep30-21swapper/312:10:163
2946026016,28sleep129454-21apt-get08:15:091
132642606,44sleep00-21swapper/010:48:570
1291126017,21sleep30-21swapper/307:20:243
1134025913,28sleep011364-21df09:00:100
560525810,20sleep10-21swapper/110:25:071
323952588,13sleep00-21swapper/010:05:240
3084225819,23sleep00-21swapper/011:45:140
273825813,28sleep02853-21sed11:59:560
2498625817,26sleep024992-21ntp_states11:25:190
234142587,42sleep30-21swapper/309:39:303
2299425818,17sleep30-21swapper/307:54:563
1700825816,25sleep017003-21apt-get07:35:070
802725715,23sleep10-21swapper/112:15:191
2352725719,24sleep023596-21sed09:39:560
1179025715,24sleep111818-21munin-run10:44:541
978325616,24sleep19789-21df_inode08:55:111
655125610,29sleep16598-21processes12:10:221
655125610,29sleep16598-21processes12:10:221
474725619,15sleep00-21swapper/010:20:260
3022325615,26sleep030218-21apt-get10:00:070
2710425611,32sleep027163-21cpuspeed09:50:090
255925617,16sleep00-21swapper/010:15:130
155402565,17sleep00-21swapper/009:13:190
1459525613,20sleep10-21swapper/112:35:381
1459525613,20sleep10-21swapper/112:35:381
975925515,20sleep00-21swapper/012:20:230
92425516,23sleep0918-21apt-get10:10:080
2616725512,17sleep00-21swapper/011:30:120
2413225520,17sleep10-21swapper/109:40:171
2182725510,19sleep10-21swapper/107:50:141
2168625515,25sleep021679-21apt-get07:50:090
163832556,37sleep00-21swapper/007:30:340
1562925513,19sleep10-21swapper/110:55:201
130322556,15sleep30-21swapper/310:46:013
3019825414,24sleep130216-21apt10:00:041
2612625416,17sleep10-21swapper/108:04:551
2612625416,17sleep10-21swapper/108:04:551
216832545,40sleep00-21swapper/009:32:080
2167525417,19sleep00-21swapper/011:15:180
145482546,38sleep00-21swapper/010:50:340
949425313,16sleep10-21swapper/112:20:161
731225313,23sleep07321-21df_inode07:05:110
731225313,23sleep07321-21df_inode07:05:110
412725315,23sleep14133-21df_inode10:20:111
3271225313,22sleep0333-21perl08:25:140
2835325312,24sleep328364-21ntp_states08:10:193
2818625313,26sleep128263-21cat11:35:261
2818625313,26sleep128263-21cat11:35:261
218325314,24sleep12209-21munin-run10:14:541
3075825214,22sleep130825-21df11:45:111
297325212,24sleep32970-21apt-get12:00:073
2800925210,17sleep10-21swapper/108:10:121
2788625214,19sleep00-21swapper/008:10:030
1683125216,21sleep016835-21diskstats11:00:110
1447625217,17sleep30-21swapper/309:10:113
705625111,18sleep30-21swapper/308:45:193
247525113,15sleep10-21swapper/110:15:101
233692517,16sleep10-21swapper/107:55:141
226425116,18sleep10-21swapper/108:30:231
226425116,18sleep10-21swapper/108:30:231
2068825112,24sleep320765-21cat07:45:233
1930725111,24sleep119360-21irqstats09:25:151
1858525112,23sleep318582-21apt-get07:40:063
502525013,22sleep34521-21apt-get08:40:023
466525012,22sleep34742-21timerandwakeup10:20:253
217282505,10sleep10-21swapper/109:32:421
1838425013,21sleep318389-21diskstats11:05:113
1753825013,20sleep017535-21apt-get09:20:050
1556525012,20sleep30-21swapper/310:55:183
1467725011,18sleep10-21swapper/109:10:171
1053025015,21sleep00-21swapper/010:40:080
508624912,18sleep10-21swapper/108:40:291
1571024913,15sleep00-21swapper/010:55:230
754424812,21sleep17553-21diskstats10:31:091
1714824814,20sleep317189-21ntpq11:00:183
1659024814,18sleep10-21swapper/107:33:141
242772474,31sleep30-21swapper/307:57:103
177522468,13sleep10-21swapper/109:20:141
2137624212,15sleep30-21swapper/311:15:043
111872419,16sleep30-21swapper/312:25:183
111872419,16sleep30-21swapper/312:25:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional