You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 13:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun May 03, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30445211888,1165sleep131198-1kworker/1:1H23:15:421
918290211,880sleep01489-1kworker/0:0H20:00:350
918290211,880sleep01489-1kworker/0:0H20:00:350
14352259417,560sleep214421-21/usr/sbin/munin00:05:462
1020825109,488sleep2191-21jbd2/mmcblk2p2-20:30:202
23625242020,382sleep123657-21modprobe22:55:151
1416235610,203sleep11482-21cpuspeed21:45:331
10337233816,301sleep110356-40ogrotate23:55:151
18566228896,178sleep10-21swapper/120:55:391
3668228578,179sleep13814-21unixbench_singl21:50:461
20446226069,64sleep10-21swapper/122:45:171
5359225884,90sleep15413-21latency_hist20:15:171
1557422578,215sleep115563-21apt-get00:10:291
2922223735,183sleep12430-21/usr/sbin/munin20:06:211
1748422297,208sleep117546-21cat19:10:371
4762221449,151sleep14764-21apt-get21:55:271
31260221116,139sleep131443-21apt-config19:55:191
24547220711,180sleep124635-21cat22:55:431
21109220710,58sleep121166-21/usr/sbin/munin21:05:161
15177220621,99sleep115297-21gpgv00:10:171
10340220512,99sleep110431-21fschecks_time20:30:351
25505218111,158sleep125549-21ntpq19:35:401
6417217313,147sleep06500-21fschecks_time22:00:340
22258217011,71sleep10-21swapper/100:30:451
1555216653,97sleep11640-21mailstats23:25:431
8008216528,89sleep18070-21sendmail_mailqu23:45:461
31045216514,136sleep231123-21find23:15:592
2323721637,144sleep123273-21awk19:30:151
19102216011,137sleep019110-21df22:40:320
22491215925,112sleep222624-21sed00:30:502
17741215874,25sleep10-21swapper/100:15:461
1557121559,131sleep023632-21kworker/u8:100:10:280
3256521539,33sleep10-21swapper/121:40:371
5282214615,45sleep00-21swapper/020:15:160
22440214515,27sleep30-21swapper/322:50:323
18371214210,115sleep018378-21sh20:55:340
10006213911,115sleep010123-21latency_hist20:30:180
10006213911,115sleep010123-21latency_hist20:30:180
27525213855,65sleep126631-21/usr/sbin/munin19:41:051
20790213742,28sleep10-21swapper/122:45:251
12584213771,51sleep112669-21mailstats20:35:561
12584213771,51sleep112669-21mailstats20:35:561
20443213658,39sleep10-21swapper/100:25:361
15018213415,27sleep10-21swapper/119:05:161
15018213415,27sleep10-21swapper/119:05:161
15167213019,51sleep10-21swapper/120:45:291
5502212947,46sleep15572-21sed23:40:171
32313212917,40sleep132431-21unixbench_multi19:56:121
308021296,108sleep03120-21ps23:30:430
1289221288,76sleep112980-21perl22:20:341
14078212633,80sleep114140-21uname20:40:451
29514212136,44sleep10-21swapper/119:50:181
11040212115,85sleep10-21swapper/122:15:171
23530212017,24sleep10-21swapper/119:30:311
16652211658,16sleep10-21swapper/119:10:171
16652211658,16sleep10-21swapper/119:10:171
6083211540,58sleep26055-21latency_hist22:00:162
5598211335,18sleep10-21swapper/120:15:291
11007211317,81sleep011039-21date22:15:160
26231211224,67sleep026376-21grep23:00:470
23167211256,44sleep122934-21/usr/sbin/munin21:10:371
23167211256,44sleep122934-21/usr/sbin/munin21:10:371
19711211121,67sleep219858-21grep22:40:462
28309211025,69sleep328313-21modprobe23:10:153
32524210810,80sleep032564-21/usr/sbin/munin21:40:360
1112921078,41sleep111130-21ntp_states23:55:431
20640210611,79sleep320681-21if_eth019:20:363
14351210512,55sleep114417-21cpuspeed22:25:311
30855210222,54sleep330920-21fschecks_time21:35:343
1431321028,13sleep20-21swapper/222:25:202
7136210110,49sleep17289-21gpgv23:45:171
7136210110,49sleep17289-21gpgv23:45:171
31562210125,52sleep231559-21apt-get19:55:282
30110210117,62sleep230271-21grep23:15:182
281252999,19sleep10-21swapper/121:25:431
281252999,19sleep10-21swapper/121:25:431
833029810,75sleep38332-21sshd20:24:193
719229812,64sleep00-21swapper/020:20:290
2617929710,72sleep026206-21fschecks_count21:20:330
2617929710,72sleep026206-21fschecks_count21:20:330
105829621,59sleep31098-21fschecks_time20:00:543
105829621,59sleep31098-21fschecks_time20:00:543
963729518,58sleep29153-21apt-get22:10:252
836229548,10sleep10-21swapper/122:05:441
2304429525,53sleep323060-21sort21:10:353
2304429525,53sleep323060-21sort21:10:353
294772948,25sleep10-21swapper/121:30:401
237429414,65sleep22415-21sed23:30:162
177562948,38sleep10-21swapper/122:35:361
1681129418,39sleep10-21swapper/120:50:341
1332029416,59sleep013334-21idleruntime-cro20:40:150
1332029416,59sleep013334-21idleruntime-cro20:40:150
82132930,7sleep30-21swapper/322:05:373
2904229319,20sleep00-21swapper/019:45:490
1627429312,67sleep016304-21awk22:30:390
878229212,43sleep115117-21cyclictest20:25:351
878229212,43sleep115117-21cyclictest20:25:351
1554429219,52sleep215052-21apt-get00:10:262
1353529214,63sleep113585-21wc00:05:161
454329123,25sleep30-21swapper/320:10:433
256852915,10sleep10-21swapper/121:18:441
256852915,10sleep10-21swapper/121:18:441
1730029112,63sleep017364-21/usr/sbin/munin19:10:320
757829015,11sleep10-21swapper/122:05:081
1783529019,24sleep20-21swapper/200:15:472
1133129036,13sleep10-21swapper/122:15:331
2860128934,15sleep10-21swapper/123:10:291
9032888,63sleep1907-21modprobe20:00:281
9032888,63sleep1907-21modprobe20:00:281
2681228822,34sleep20-21swapper/223:05:172
1570428816,56sleep315855-21cat22:30:173
114952887,14sleep00-21swapper/023:55:500
816928742,11sleep12133-21kworker/1:020:22:161
3153928710,58sleep00-21swapper/019:55:260
2827228735,14sleep10-21swapper/123:09:441
298728616,50sleep03052-21awk20:06:240
2162628636,12sleep10-21swapper/121:05:361
2000228630,13sleep10-21swapper/100:25:151
1805228613,59sleep018121-21sed22:35:430
3115328514,50sleep031259-21latency_hist19:55:180
2993928512,59sleep330000-21if_eth019:50:363
1204928514,57sleep311538-21apt-get20:35:263
1204928514,57sleep311538-21apt-get20:35:263
1134828213,34sleep30-21swapper/322:15:353
485328116,48sleep24914-21/usr/sbin/munin20:10:462
3112628113,51sleep331241-21sed23:16:003
2666728115,19sleep30-21swapper/321:20:443
2666728115,19sleep30-21swapper/321:20:443
2198328114,31sleep20-21swapper/219:25:322
2088628115,53sleep321012-21timerwakeupswit00:25:463
3174028035,12sleep10-21swapper/121:36:331
320432799,31sleep10-21swapper/123:20:301
278627915,49sleep22814-21kernelversion20:06:182
1358627911,52sleep013604-21apt20:40:270
2674227817,40sleep026631-21/usr/sbin/munin19:40:360
256862778,13sleep10-21swapper/123:00:361
1387927716,46sleep313909-21kernelversion20:40:393
1878727613,46sleep318795-21diskstats00:20:333
282627511,48sleep10-21swapper/123:30:361
2743427526,30sleep20-21swapper/223:05:412
245692757,14sleep00-21swapper/019:32:280
1092027516,33sleep00-21swapper/020:30:460
700027418,35sleep30-21swapper/323:45:153
700027418,35sleep30-21swapper/323:45:153
610927415,46sleep26132-21/usr/sbin/munin23:40:382
610927415,46sleep26132-21/usr/sbin/munin23:40:382
343327420,34sleep23495-21grep21:50:432
2799627414,17sleep20-21swapper/219:45:182
2689327411,46sleep10-21swapper/121:20:461
2689327411,46sleep10-21swapper/121:20:461
1737627411,29sleep00-21swapper/020:50:450
1168627424,24sleep30-21swapper/320:35:173
456427323,23sleep30-21swapper/323:35:383
2263127311,46sleep022646-21kernelversion22:50:370
2094027324,30sleep30-21swapper/322:46:003
925727217,16sleep20-21swapper/223:50:362
916327216,16sleep20-21swapper/220:25:442
916327216,16sleep20-21swapper/220:25:442
2901027216,39sleep329151-21date21:30:173
2547327221,29sleep00-21swapper/019:35:390
1795027212,28sleep30-21swapper/319:10:473
1503627214,41sleep315081-21gpgv00:10:153
913927123,27sleep00-21swapper/022:10:150
654027118,26sleep20-21swapper/222:00:372
58727116,32sleep3729-21grep21:40:473
241627116,24sleep30-21swapper/323:30:173
1871027110,26sleep00-21swapper/000:20:300
159032717,48sleep10-21swapper/122:30:271
1231527123,25sleep00-21swapper/000:00:350
310382708,16sleep20-21swapper/221:35:372
1925527018,29sleep30-21swapper/319:15:403
1455827017,20sleep00-21swapper/000:05:480
2585426918,28sleep00-21swapper/021:20:170
2585426918,28sleep00-21swapper/021:20:170
1507026919,29sleep215087-21munin-run19:05:172
1507026919,29sleep215087-21munin-run19:05:172
139226910,43sleep20-21swapper/220:01:012
139226910,43sleep20-21swapper/220:01:012
3223926812,27sleep20-21swapper/223:20:352
214262689,12sleep10-21swapper/119:21:511
314262678,47sleep00-21swapper/023:16:540
284762678,42sleep20-21swapper/219:45:392
2202026714,25sleep00-21swapper/000:30:400
1580726716,33sleep315820-21/usr/sbin/munin19:05:423
1580726716,33sleep315820-21/usr/sbin/munin19:05:423
1544026712,24sleep20-21swapper/220:45:382
1384626732,20sleep30-21swapper/319:00:343
1384626732,20sleep30-21swapper/319:00:343
505426616,27sleep30-21swapper/321:55:453
431226615,35sleep34421-21latency_hist21:55:173
3156326619,28sleep331029-21apt-get19:55:293
244312665,48sleep20-21swapper/219:30:492
2026026620,17sleep00-21swapper/000:25:310
193302667,13sleep10-21swapper/119:15:421
1152626619,22sleep20-21swapper/222:15:382
1130526617,20sleep00-21swapper/022:15:320
915326511,40sleep39160-21df_inode23:50:333
762926511,28sleep30-21swapper/323:45:363
3205826522,22sleep30-21swapper/323:20:323
250362656,13sleep20-21swapper/221:15:432
250362656,13sleep20-21swapper/221:15:432
2407726516,19sleep30-21swapper/322:55:333
2081226515,27sleep20-21swapper/200:25:452
1859426512,39sleep218595-21sshd22:38:202
1606726518,21sleep00-21swapper/020:45:480
153126523,25sleep00-21swapper/023:25:420
2829226422,23sleep30-21swapper/319:45:363
249222648,17sleep00-21swapper/022:56:310
2029026419,30sleep3281-21dbus-daemon22:45:153
180972647,19sleep00-21swapper/000:16:060
1494326418,31sleep0292-21in:imuxsock19:04:180
1494326418,31sleep0292-21in:imuxsock19:04:180
545826316,24sleep00-21swapper/021:55:540
3045226319,25sleep030475-21munin-run21:35:150
2818826318,28sleep20-21swapper/221:25:442
2818826318,28sleep20-21swapper/221:25:442
2791226311,14sleep30-21swapper/321:25:373
2791226311,14sleep30-21swapper/321:25:373
1495726312,34sleep015062-21/usr/sbin/munin22:25:460
46626223,15sleep20-21swapper/221:40:462
2865426217,21sleep00-21swapper/023:10:320
2658726214,29sleep226666-21/usr/sbin/munin21:20:432
2658726214,29sleep226666-21/usr/sbin/munin21:20:432
2486626211,22sleep30-21swapper/319:35:183
395626117,29sleep24058-21latency_hist23:35:172
3053026117,21sleep20-21swapper/221:35:172
296622616,11sleep20-21swapper/219:50:192
2772926121,22sleep00-21swapper/021:25:340
2772926121,22sleep00-21swapper/021:25:340
2402926118,21sleep20-21swapper/222:55:322
2289526112,19sleep00-21swapper/019:25:500
2003926121,18sleep20-21swapper/221:00:362
1961026118,25sleep30-21swapper/321:00:173
1897026116,25sleep20-21swapper/219:15:342
1103726113,27sleep20-21swapper/223:55:402
883926018,24sleep00-21swapper/023:50:170
83126012,26sleep0920-21latency_hist23:25:180
2986826016,27sleep329887-21apt-get23:15:153
2641525919,23sleep20-21swapper/219:40:182
225392596,14sleep00-21swapper/021:08:340
2219025915,21sleep30-21swapper/319:25:373
1249725920,22sleep30-21swapper/300:00:383
491125816,17sleep20-21swapper/223:35:452
32092587,12sleep30-21swapper/320:06:273
2538725816,21sleep30-21swapper/319:35:383
1279825815,27sleep012792-21apt-get22:20:290
689725716,23sleep06906-21munin-run20:20:170
2149725719,19sleep30-21swapper/321:05:343
209852578,40sleep00-21swapper/022:46:090
578125619,19sleep20-21swapper/220:15:352
265182566,13sleep30-21swapper/319:40:193
236225614,29sleep22363-21sshd21:46:192
2107125617,21sleep00-21swapper/021:04:460
169902567,37sleep20-21swapper/222:33:032
1612025617,16sleep30-21swapper/300:10:523
946625511,17sleep30-21swapper/322:10:183
928325517,20sleep20-21swapper/222:10:172
2558725512,26sleep325594-21/usr/sbin/munin23:00:343
2141025512,25sleep221404-21apt-get21:05:292
2119725515,17sleep30-21swapper/321:05:183
139972556,10sleep30-21swapper/322:24:283
1049825513,22sleep00-21swapper/023:55:180
909825413,22sleep30-21swapper/320:25:413
909825413,22sleep30-21swapper/320:25:413
641125413,25sleep36415-21diskstats22:00:333
419425413,26sleep04188-21apt-get23:35:270
3078325414,24sleep030842-21df21:35:310
295825414,23sleep02952-21apt-get21:50:290
2370125416,22sleep00-21swapper/021:10:480
2370125416,22sleep00-21swapper/021:10:480
1902825413,20sleep20-21swapper/220:55:482
1671425414,23sleep316710-21apt-get20:50:283
753225314,22sleep27521-21apt-get23:45:312
614525319,19sleep30-21swapper/320:15:443
264825312,28sleep313441-21snmpd21:50:003
2454025313,25sleep324536-21apt-get21:15:273
2454025313,25sleep324536-21apt-get21:15:273
1939325316,18sleep00-21swapper/019:15:450
1592325317,17sleep30-21swapper/320:45:473
1222525313,24sleep212221-21apt-get00:00:292
270325213,26sleep32731-21cut23:30:343
1703225210,16sleep20-21swapper/219:10:222
1534125212,24sleep015338-21apt-get19:05:270
1534125212,24sleep015338-21apt-get19:05:270
2236225110,16sleep20-21swapper/222:50:272
147192514,15sleep20-21swapper/220:43:202
1316725113,22sleep20-21swapper/222:20:392
759725011,23sleep27603-21ntp_states20:20:392
631625012,23sleep06400-21cat23:40:440
631625012,23sleep06400-21cat23:40:440
2181125011,23sleep30-21swapper/300:30:343
1919525015,14sleep20-21swapper/200:20:442
1561525012,22sleep215674-21sed19:05:372
1561525012,22sleep215674-21sed19:05:372
150125012,20sleep00-21swapper/021:45:370
2703024910,16sleep00-21swapper/023:05:270
1726824916,17sleep30-21swapper/300:15:363
1104724913,18sleep30-21swapper/323:55:413
2295824810,22sleep222954-21apt-get21:10:302
2295824810,22sleep222954-21apt-get21:10:302
208672488,15sleep00-21swapper/019:20:410
2082224810,22sleep220809-21apt-get22:45:312
1874124813,21sleep318772-21awk22:40:153
2042424712,18sleep20-21swapper/219:20:272
1206324610,21sleep212047-21apt-get20:35:332
1206324610,21sleep212047-21apt-get20:35:332
165602439,15sleep30-21swapper/322:30:463
179752427,14sleep20-21swapper/220:55:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional