You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 23:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jul 21, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28664231556,3138sleep325565-21kworker/3:208:59:363
723267919,643sleep0732-21/usr/sbin/munin10:58:370
1094124168,395sleep310942-21sshd09:47:363
1094124168,395sleep310942-21sshd09:47:363
20732235922,266sleep320766-21cut10:18:363
25627233934,282sleep325750-21sed12:18:203
32631229112,139sleep332653-21kernelversion09:13:453
23508227342,216sleep323597-21latency_hist10:28:213
14868226221,227sleep214940-21sort08:19:152
91242255102,52sleep30-21swapper/308:03:213
4296225592,83sleep34315-21sh09:28:183
420122189,57sleep34251-21if_eth011:08:413
420122189,57sleep34251-21if_eth011:08:413
3131822027,119sleep331447-21timerwakeupswit12:33:503
28138219612,139sleep328148-21modprobe10:43:183
1516221907,150sleep00-21swapper/011:43:420
260521898,167sleep32609-21grep11:03:533
260521898,167sleep32609-21grep11:03:533
18684218912,71sleep318652-21latency_hist08:33:213
4894218619,61sleep30-21swapper/307:48:403
32500218010,15sleep30-21swapper/307:33:433
20828217821,76sleep320831-21cron12:03:183
15775217811,152sleep215796-21users11:43:522
11000217817,35sleep111027-21munin-run09:48:181
11000217817,35sleep111027-21munin-run09:48:181
29040217513,149sleep028979-21latency_hist09:03:210
2387121756,157sleep0364-21snmpd12:12:320
29219217431,109sleep10-21swapper/109:03:301
29219217431,109sleep10-21swapper/109:03:301
114921719,60sleep30-21swapper/310:58:483
24027216318,129sleep024044-21kernelversion10:28:420
24027216318,129sleep024044-21kernelversion10:28:420
15919215919,127sleep316010-21fschecks_time08:23:403
10909215686,18sleep30-21swapper/308:08:343
9452215512,127sleep29525-21grep08:03:412
23980215018,105sleep30-21swapper/312:13:193
30434214620,108sleep230451-21grep07:28:212
1396721445,10sleep30-21swapper/309:56:403
1675214381,21sleep31694-21fschecks_count07:38:393
802521427,123sleep38038-21awk11:18:543
30453214029,94sleep30-21swapper/307:28:223
27338213947,54sleep327355-21grep07:18:213
27655213812,41sleep30-21swapper/307:18:373
1755621365,11sleep30-21swapper/311:51:323
26050213410,48sleep326036-21apt-get12:18:363
1939221329,48sleep319431-21/usr/sbin/munin08:33:493
1891321325,33sleep30-21swapper/311:54:023
14385213117,90sleep30-21swapper/308:18:513
903321296,33sleep30-21swapper/309:41:023
903321296,33sleep30-21swapper/309:41:023
19436212716,93sleep30-21swapper/310:13:433
1597121279,103sleep016020-21cpuspeed10:03:350
3247721257,107sleep00-21swapper/010:54:250
428321245,51sleep30-21swapper/307:46:543
29753212314,20sleep10-21swapper/110:48:211
25759212276,25sleep00-21swapper/012:18:210
24409212212,20sleep30-21swapper/308:48:383
14223212018,84sleep214372-21date09:58:212
14223212018,84sleep214372-21date09:58:212
561621196,96sleep30-21swapper/311:13:373
28088211926,80sleep328155-21systemctl12:23:493
2487921196,10sleep30-21swapper/310:31:363
2487921196,10sleep30-21swapper/310:31:363
7636211819,43sleep10-21swapper/109:38:211
3263221175,11sleep30-21swapper/310:56:263
2254521177,18sleep10-21swapper/107:03:381
1573211726,72sleep30-21swapper/309:18:383
26486211616,82sleep126510-21/usr/sbin/munin08:53:491
23729211639,58sleep223758-21munin-plugin-st07:08:192
31276211528,75sleep331275-21ntp_states09:08:573
2667221158,88sleep30-21swapper/308:53:553
19336211115,81sleep219483-21cat11:58:212
14126211022,67sleep30-21swapper/311:38:523
12910210921,12sleep30-21swapper/308:13:473
8672210824,14sleep30-21swapper/311:23:343
17934210817,72sleep118003-21grep08:28:551
20880210712,77sleep120946-21latency_hist12:03:201
12494210712,21sleep312639-21grep11:33:523
7437210615,61sleep30-21swapper/307:57:403
6613210613,78sleep26653-21uname09:33:542
29622210610,18sleep30-21swapper/312:28:493
25355210622,13sleep30-21swapper/307:13:203
21925210616,61sleep30-21swapper/308:42:253
25304210523,11sleep30-21swapper/310:33:333
17520210521,50sleep30-21swapper/310:08:353
9446210414,66sleep18649-21/usr/sbin/munin11:23:531
3037421048,59sleep330415-21ntpq10:48:433
3037421048,59sleep330415-21ntpq10:48:433
27528210414,77sleep227595-21latency_hist08:58:222
20413210416,69sleep00-21swapper/010:18:210
714621035,87sleep30-21swapper/309:35:083
593021035,85sleep00-21swapper/007:53:090
1486321038,17sleep10-21swapper/111:43:301
22495210212,75sleep222013-21apt-get08:43:292
4804210115,73sleep24851-21cat09:28:462
4804210115,73sleep24851-21cat09:28:462
4804210115,73sleep24851-21cat09:28:462
22822210120,13sleep30-21swapper/308:43:583
55912985,62sleep30-21swapper/309:30:043
55912985,62sleep30-21swapper/309:30:043
55912985,62sleep30-21swapper/309:30:043
2429129818,60sleep223800-21apt-get08:48:292
153032988,16sleep30-21swapper/311:43:453
146522988,13sleep30-21swapper/309:58:423
146522988,13sleep30-21swapper/309:58:423
2650429717,64sleep025976-21/usr/sbin/munin07:13:500
102192977,14sleep30-21swapper/311:28:353
752429626,33sleep20-21swapper/211:18:452
218429616,64sleep11929-21apt-get11:03:221
2665529519,20sleep20-21swapper/210:38:212
311012946,14sleep30-21swapper/307:28:463
311012946,14sleep30-21swapper/307:28:463
369629320,51sleep03710-21idleruntime-cro11:08:180
369629320,51sleep03710-21idleruntime-cro11:08:180
792129217,55sleep17428-21apt-get09:38:301
792129217,55sleep17428-21apt-get09:38:301
3199029229,36sleep10-21swapper/107:33:201
3199029229,36sleep10-21swapper/107:33:201
593828919,20sleep10-21swapper/109:33:211
593828919,20sleep10-21swapper/109:33:211
593828919,20sleep10-21swapper/109:33:211
2446328915,61sleep024511-21cat07:08:450
2315728915,60sleep123243-21mailstats12:08:491
1769628921,47sleep017725-21munin-run11:53:180
119028924,43sleep11050-21latency_hist07:38:211
1117628914,61sleep211192-21kernelversion08:08:412
3043728813,60sleep230471-21ps10:48:462
3043728813,60sleep230471-21ps10:48:462
283228822,36sleep20-21swapper/209:23:212
2548828820,47sleep125509-21apt-get12:18:181
2747528722,43sleep126857-21/usr/sbin/munin10:38:521
322128610,63sleep03277-21cat09:23:430
2158728636,34sleep321630-21processes12:03:473
2889628516,21sleep20-21swapper/207:23:202
2456528415,18sleep10-21swapper/108:48:451
40922839,50sleep20-21swapper/211:08:402
40922839,50sleep20-21swapper/211:08:402
3145028315,47sleep031533-21timerandwakeup09:09:020
2962728315,46sleep029218-21/usr/sbin/munin09:03:530
2962728315,46sleep029218-21/usr/sbin/munin09:03:530
1137628330,27sleep10-21swapper/109:48:391
286512826,60sleep30-21swapper/310:43:383
301932817,42sleep30-21swapper/309:04:213
301932817,42sleep30-21swapper/309:04:213
2884528127,32sleep00-21swapper/007:23:180
285962816,14sleep00-21swapper/012:24:550
24072817,24sleep20-21swapper/211:03:272
24072817,24sleep20-21swapper/211:03:272
228228038,23sleep00-21swapper/007:39:000
171272806,40sleep30-21swapper/310:07:053
1449928027,39sleep114557-21if_err_eth008:19:061
1129128011,32sleep20-21swapper/209:48:362
2414527912,52sleep224185-21sort07:08:392
124892797,42sleep30-21swapper/309:52:363
104582786,39sleep30-21swapper/308:06:163
464727724,27sleep10-21swapper/111:08:511
464727724,27sleep10-21swapper/111:08:511
2983627718,38sleep229973-21grep12:28:522
2190827721,38sleep121922-21modprobe10:23:181
1477727614,44sleep114786-21/usr/sbin/munin09:58:441
1477727614,44sleep114786-21/usr/sbin/munin09:58:441
917427415,43sleep29252-21mailstats11:23:502
3161027421,29sleep10-21swapper/110:53:371
2858927313,41sleep228650-21sort10:43:372
2324427319,37sleep223313-21switchtime12:08:502
1302127327,29sleep213047-21munin-run11:38:192
302762728,13sleep10-21swapper/110:48:411
302762728,13sleep10-21swapper/110:48:411
2923927222,25sleep10-21swapper/112:28:391
2422327224,24sleep20-21swapper/212:13:342
206012725,11sleep20-21swapper/212:00:232
1282427222,24sleep00-21swapper/008:13:440
1225327218,22sleep00-21swapper/011:33:490
712627116,26sleep10-21swapper/111:18:361
688027124,21sleep10-21swapper/111:18:201
2387227120,34sleep124004-21apt-config08:48:201
2380127122,25sleep00-21swapper/007:08:210
1152527118,26sleep00-21swapper/011:33:200
1098227115,29sleep10-21swapper/108:08:361
331327019,29sleep20-21swapper/209:23:452
3109227018,28sleep10-21swapper/112:33:441
186792707,54sleep20-21swapper/210:12:052
848526918,27sleep20-21swapper/209:39:402
848526918,27sleep20-21swapper/209:39:402
555726910,42sleep25615-21df11:13:362
444526914,25sleep00-21swapper/007:48:210
3223126910,45sleep00-21swapper/007:33:350
2204226922,26sleep10-21swapper/108:43:201
1776126920,26sleep00-21swapper/008:28:490
84482688,14sleep00-21swapper/009:39:380
84482688,14sleep00-21swapper/009:39:380
789326812,25sleep10-21swapper/107:58:391
433926823,21sleep10-21swapper/109:28:201
3199926823,26sleep20-21swapper/210:53:472
2809426812,42sleep228142-21ntp_offset08:58:472
2603826816,36sleep125509-21apt-get12:18:301
1717326826,19sleep00-21swapper/008:28:210
1389126817,31sleep214074-21latency_hist08:18:222
1157826822,21sleep10-21swapper/111:33:211
790426720,23sleep20-21swapper/207:58:402
2967426721,22sleep10-21swapper/107:23:521
2570826718,30sleep10-21swapper/110:33:431
2025826721,29sleep220280-21/usr/sbin/munin08:38:202
1798426714,29sleep10-21swapper/110:08:491
1708526718,34sleep217130-21systemd-run08:28:192
158826715,30sleep21584-21apt-get07:38:342
1266426716,32sleep112714-21/usr/sbin/munin08:13:411
3206726614,38sleep132068-21cron09:13:191
318802666,14sleep20-21swapper/207:32:372
318802666,14sleep20-21swapper/207:32:372
241226620,25sleep10-21swapper/111:03:301
241226620,25sleep10-21swapper/111:03:301
2142326517,30sleep221573-21sed08:38:532
2279026417,23sleep20-21swapper/210:23:522
467626315,21sleep00-21swapper/009:28:400
467626315,21sleep00-21swapper/009:28:400
467626315,21sleep00-21swapper/009:28:400
313072639,38sleep231374-21mailstats09:09:002
209542638,36sleep00-21swapper/008:38:420
149126316,31sleep00-21swapper/009:18:340
1469226314,24sleep00-21swapper/011:43:210
2376226220,24sleep323799-21munin-run07:08:203
293492617,13sleep00-21swapper/007:23:430
2422226113,16sleep20-21swapper/210:28:482
2422226113,16sleep20-21swapper/210:28:482
224682616,12sleep10-21swapper/110:23:421
207882616,11sleep00-21swapper/012:02:470
1843826118,20sleep00-21swapper/011:53:470
1103326120,21sleep00-21swapper/008:08:390
2768826016,19sleep10-21swapper/112:23:391
2375426017,26sleep123745-21apt-get10:28:351
2375426017,26sleep123745-21apt-get10:28:351
1904226018,26sleep219043-21/usr/sbin/munin10:13:302
1694626015,22sleep20-21swapper/211:48:482
143112605,15sleep20-21swapper/211:39:362
468425917,20sleep10-21swapper/107:48:341
2739325916,24sleep00-21swapper/010:38:510
2675525921,20sleep00-21swapper/008:53:560
2658025923,20sleep10-21swapper/107:13:511
2242725917,18sleep10-21swapper/112:08:211
210525917,19sleep10-21swapper/109:18:521
1604625918,23sleep216054-21diskstats10:03:372
1305425910,18sleep00-21swapper/009:53:420
724825813,32sleep0364-21snmpd09:36:270
679525812,33sleep16873-21switchtime07:53:511
672025815,18sleep00-21swapper/007:53:500
3264725815,26sleep232683-21ntp_states07:33:462
3251925815,27sleep132575-21/usr/sbin/munin09:13:431
2758625813,23sleep10-21swapper/107:18:351
2266925814,30sleep022732-21/usr/sbin/munin08:43:550
2041825815,27sleep10-21swapper/110:18:221
1727925818,18sleep10-21swapper/110:08:211
1341625817,20sleep10-21swapper/109:53:521
117125821,21sleep00-21swapper/009:18:180
687425725,17sleep20-21swapper/207:53:522
25432576,15sleep10-21swapper/107:39:261
2109125711,31sleep221102-21ntp_states10:18:432
1765525710,31sleep217744-21/usr/sbin/munin08:28:472
439325616,23sleep24422-21munin-run07:48:192
3077025610,29sleep030780-21forks07:28:390
3077025610,29sleep030780-21forks07:28:390
2888225612,26sleep028888-21ntp_states10:43:440
1892325616,24sleep018918-21apt-get08:33:340
3233625511,25sleep20-21swapper/209:13:322
2815925520,19sleep128223-21ps08:58:501
2734225514,26sleep227348-21ntp_states10:38:492
2726925514,23sleep227294-21munin-run07:18:202
104802557,36sleep00-21swapper/008:06:330
712425412,25sleep07115-21apt-get11:18:350
2997425413,25sleep029969-21apt-get10:48:320
2997425413,25sleep029969-21apt-get10:48:320
2577325414,23sleep00-21swapper/010:33:470
2520725410,31sleep0364-21snmpd12:14:460
215772549,22sleep20-21swapper/212:03:462
1931525414,18sleep10-21swapper/111:58:201
169152545,15sleep10-21swapper/108:26:081
944725315,22sleep19451-21diskstats08:03:401
3078125315,22sleep230672-21/usr/sbin/munin12:33:372
2986225313,26sleep229948-21timerandwakeup09:03:592
2986225313,26sleep229948-21timerandwakeup09:03:592
2835825312,19sleep20-21swapper/212:23:522
2606925313,24sleep226036-21apt-get12:18:472
19122539,28sleep21958-21apt-config11:03:192
974525216,19sleep20-21swapper/209:43:352
974525216,19sleep20-21swapper/209:43:352
864825210,18sleep00-21swapper/011:23:310
2942325216,24sleep029450-21kernelversion12:28:420
2432825210,27sleep124369-21ls07:08:421
2062525210,15sleep00-21swapper/010:18:290
2018125213,19sleep10-21swapper/108:38:191
156512525,35sleep00-21swapper/010:02:590
156512525,35sleep00-21swapper/010:02:590
1281525211,24sleep212834-21apt-get09:53:322
1254725212,24sleep012572-21munin-run09:53:180
3074325112,22sleep030770-21df12:33:350
2301125111,22sleep023103-21users10:23:540
2241725110,29sleep0364-21snmpd12:08:200
1108625115,21sleep011239-21sed09:48:220
1108625115,21sleep011239-21sed09:48:220
680225015,13sleep00-21swapper/011:18:190
172442506,17sleep10-21swapper/111:48:511
1378425011,23sleep113799-21ntp_states11:38:461
1043025010,25sleep210481-21cat11:28:422
1425424917,17sleep00-21swapper/008:18:320
524524811,21sleep05274-21munin-run11:13:180
524524811,21sleep05274-21munin-run11:13:180
47322489,17sleep20-21swapper/207:48:352
355324811,21sleep13590-21ntp_states07:43:461
2562724811,23sleep025650-21munin-run08:53:200
1219824814,19sleep112222-21ntp_states11:33:461
820624616,16sleep00-21swapper/007:58:460
1882524610,21sleep018797-21latency_hist10:13:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional