You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 05:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Apr 13, 2026 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
187882161111,1574sleep215044-1kworker/2:0H20:36:372
2274129407,885sleep122783-21ntpq19:06:521
5747265620,619sleep35779-21/usr/sbin/munin21:36:533
2908623716,354sleep113441-21snmpd21:07:161
20731235721,317sleep220793-21/usr/sbin/munin00:06:542
15734234443,291sleep113441-21snmpd22:09:421
26676226973,125sleep10-21swapper/100:26:471
46622617,154sleep1573-21fschecks_time21:21:451
624422529,189sleep16288-21awk23:22:551
491922428,222sleep00-21swapper/021:35:450
29017223232,72sleep129061-21ntpq22:51:501
14364222529,184sleep114341-21latency_hist22:06:271
26010222455,110sleep10-21swapper/120:57:441
26010222455,110sleep10-21swapper/120:57:441
3145221349,126sleep13189-21ntpq19:46:531
3145221349,126sleep13189-21ntpq19:46:531
10023221216,150sleep110129-21fschecks_time21:51:451
10023221216,150sleep110129-21fschecks_time21:51:451
18418220924,164sleep118577-21latency_hist20:36:281
18418220924,164sleep118577-21latency_hist20:36:281
1478220655,90sleep11604-21timerwakeupswit23:06:561
1571922019,137sleep115763-21ntpq23:51:501
1571922019,137sleep115763-21ntpq23:51:501
12225218564,17sleep10-21swapper/123:41:411
17801218315,66sleep10-21swapper/123:56:591
20113217445,114sleep120256-21date20:41:291
31863217029,126sleep131967-21date19:36:271
31863217029,126sleep131967-21date19:36:271
866921577,99sleep113441-21snmpd20:06:201
553321579,92sleep15663-21cat23:21:301
11980215715,125sleep112041-21grep21:56:541
3210215525,74sleep13211-21munin-run21:31:261
23818215217,110sleep323861-21ntp_kernel_pll_00:16:523
22102215214,121sleep322103-21cron22:31:253
22102215214,121sleep322103-21cron22:31:253
2070121518,20sleep00-21swapper/000:06:520
24053214466,58sleep10-21swapper/100:16:581
31952214169,54sleep10-21swapper/123:01:471
22159214110,76sleep10-21swapper/122:31:271
22159214110,76sleep10-21swapper/122:31:271
22013214149,46sleep122073-21basename00:11:451
31554214042,21sleep10-21swapper/121:16:491
20219213827,12sleep10-21swapper/122:22:101
28511213617,19sleep20-21swapper/221:06:492
28288213429,11sleep10-21swapper/122:50:531
7991212718,82sleep38134-21grep23:26:573
7991212718,82sleep38134-21grep23:26:573
29190212359,45sleep10-21swapper/119:26:461
29190212359,45sleep10-21swapper/119:26:461
3210121218,97sleep10-21swapper/119:36:401
3210121218,97sleep10-21swapper/119:36:401
25872212112,58sleep10-21swapper/122:41:491
864421199,68sleep113441-21snmpd23:31:271
864421199,68sleep113441-21snmpd23:31:271
15349211866,26sleep20-21swapper/222:06:572
15523211648,54sleep10-21swapper/120:26:471
623421159,72sleep113441-21snmpd21:37:281
837921149,88sleep08375-21apt-get21:46:400
837921149,88sleep08375-21apt-get21:46:400
14141211416,76sleep213749-21/usr/sbin/munin23:46:512
1066921129,65sleep110664-21apt-get23:36:401
1285211119,68sleep01373-21fschecks_time19:41:480
1285211119,68sleep01373-21fschecks_time19:41:480
21608211014,83sleep121617-21modprobe20:46:251
2763210835,55sleep10-21swapper/123:11:531
5544210622,64sleep05573-21idleruntime-cro19:56:250
27789210625,58sleep327807-21modprobe00:31:253
3069021049,21sleep10-21swapper/122:56:551
28369210411,79sleep228353-21latency_hist22:51:272
2663210417,70sleep32711-21latency23:11:513
24803210416,71sleep324945-21latency_hist00:21:283
13773210440,37sleep122096-21cyclictest23:46:421
13013210471,16sleep10-21swapper/120:17:391
24298210322,17sleep10-21swapper/119:11:401
17746210329,21sleep122096-21cyclictest22:16:411
24049210213,49sleep10-21swapper/122:36:441
20220210226,15sleep10-21swapper/100:06:361
1150321026,77sleep00-21swapper/021:56:410
14391210111,74sleep214539-21latency_hist22:06:282
2498821006,41sleep10-21swapper/100:21:421
16131210020,64sleep10-21swapper/122:11:361
3142129913,70sleep30-21swapper/321:16:473
2950929944,17sleep129682-21apt-config00:36:281
2443029945,42sleep113441-21snmpd20:52:471
162882994,10sleep20-21swapper/223:52:442
162882994,10sleep20-21swapper/223:52:442
2528129818,61sleep025329-21cat22:41:270
23942985,80sleep00-21swapper/023:11:440
2079429720,56sleep320866-21cat00:06:553
1097329745,40sleep110974-21ntp_states20:11:531
1194529621,47sleep012101-21latency_hist20:16:280
283532958,67sleep20-21swapper/200:31:492
767929431,42sleep07285-21/usr/sbin/munin23:26:500
767929431,42sleep07285-21/usr/sbin/munin23:26:500
47422946,77sleep10-21swapper/121:33:321
1415929442,41sleep114243-21cat20:21:501
84672935,12sleep20-21swapper/221:46:462
84672935,12sleep20-21swapper/221:46:462
845729321,14sleep10-21swapper/121:46:451
845729321,14sleep10-21swapper/121:46:451
2965929318,50sleep329800-21sed19:26:573
2965929318,50sleep329800-21sed19:26:573
224872939,18sleep10-21swapper/120:46:581
1117629329,43sleep010645-21/usr/sbin/munin23:37:200
2206529233,39sleep122087-21cpufreq19:06:271
285832915,53sleep10-21swapper/119:25:051
285832915,53sleep10-21swapper/119:25:051
2397829113,65sleep024004-21latency19:11:310
130232916,10sleep00-21swapper/022:01:300
2667529013,62sleep326687-21/usr/sbin/munin21:01:453
634428910,58sleep10-21swapper/119:56:551
2698628922,15sleep10-21swapper/121:01:511
235792895,50sleep10-21swapper/122:35:271
51742887,20sleep10-21swapper/119:51:581
94028725,40sleep21018-21tune2fs23:06:452
3256428716,55sleep232599-21grep19:36:532
3256428716,55sleep232599-21grep19:36:532
2944728721,38sleep00-21swapper/019:26:530
2944728721,38sleep00-21swapper/019:26:530
367828617,50sleep33194-21apt-get21:31:363
2585528613,58sleep025863-21/usr/sbin/munin19:16:360
2319928611,58sleep323312-21latency_hist00:16:273
525928513,59sleep25265-21acpi21:36:362
2142728520,39sleep321579-21grep19:01:583
2190128436,25sleep30-21swapper/320:46:443
1657628423,42sleep016644-40mandb23:56:260
1657628423,42sleep016644-40mandb23:56:260
156428433,34sleep10-21swapper/119:41:531
156428433,34sleep10-21swapper/119:41:531
809028319,43sleep28092-21cron21:46:252
545228322,38sleep05531-21latency_hist23:21:290
318028323,43sleep03208-21basename21:31:250
1811728316,51sleep318162-21ntp_kernel_pll_22:16:513
214628220,35sleep30-21swapper/323:11:263
420128114,53sleep24243-21cut21:32:122
718028023,31sleep20-21swapper/220:01:282
1900528013,53sleep122096-21cyclictest00:01:521
1397628017,47sleep214056-21tune2fs20:21:472
896527922,40sleep10-21swapper/120:06:401
1497827929,25sleep30-21swapper/322:06:503
803627830,11sleep10-21swapper/121:45:441
2793227721,26sleep20-21swapper/219:21:552
2793227721,26sleep20-21swapper/219:21:552
1750327721,32sleep30-21swapper/322:16:273
1737227712,18sleep00-21swapper/022:15:430
576327625,33sleep25747-21apt-get23:21:432
2754027616,15sleep30-21swapper/322:46:563
2371827519,37sleep223835-21memory19:11:292
2162327516,40sleep221658-21munin-run20:46:262
2000327510,50sleep020111-21unixbench_multi22:21:570
1601727524,30sleep00-21swapper/020:26:580
1599927523,34sleep316130-21grep23:51:573
1599927523,34sleep316130-21grep23:51:573
1070027517,28sleep30-21swapper/323:36:583
1041027517,29sleep20-21swapper/221:51:552
1041027517,29sleep20-21swapper/221:51:552
2856027420,32sleep028621-21grep00:31:550
1211127419,39sleep312218-21grep21:56:563
758627328,27sleep00-21swapper/020:01:460
302492739,51sleep10-21swapper/121:11:571
2072027318,18sleep30-21swapper/320:41:513
3184927216,33sleep00-21swapper/023:01:460
3044927215,30sleep30-21swapper/321:11:593
2512127220,28sleep30-21swapper/320:56:453
2512127220,28sleep30-21swapper/320:56:453
3028827119,20sleep20-21swapper/219:31:282
3028827119,20sleep20-21swapper/219:31:282
2415127124,30sleep224300-21sed00:16:592
1011527122,32sleep210136-21apt-get23:36:252
3149627017,34sleep031553-21irqstats21:16:480
2620927021,20sleep30-21swapper/322:42:033
1876227012,41sleep318275-21apt-get20:36:303
1868727014,31sleep318685-21apt-get00:01:383
147382706,10sleep00-21swapper/023:47:420
1063527018,29sleep20-21swapper/220:11:462
933526919,33sleep39366-21/usr/sbin/munin20:06:503
3028126920,24sleep30-21swapper/322:56:453
2667126913,39sleep226674-21diskstats21:01:442
2471426919,31sleep024725-21modprobe20:56:250
146426921,32sleep30-21swapper/319:41:513
146426921,32sleep30-21swapper/319:41:513
904326815,22sleep00-21swapper/023:31:470
2718826817,24sleep30-21swapper/319:21:273
1783726812,36sleep217929-21perl22:16:462
3096626717,32sleep230979-21modprobe21:16:262
2373426713,28sleep00-21swapper/022:36:270
1230026710,13sleep30-21swapper/323:41:433
1027326710,41sleep010334-21latency_hist20:11:280
92882666,44sleep10-21swapper/123:31:521
812626613,37sleep38141-21munin-run21:46:263
527426612,34sleep25360-21dpkg23:21:272
324922665,39sleep30-21swapper/319:36:503
324922665,39sleep30-21swapper/319:36:503
260102659,15sleep10-21swapper/119:16:451
2466826516,34sleep324704-21ntpq19:11:503
229426515,22sleep20-21swapper/221:26:512
1725226516,25sleep20-21swapper/220:31:492
1725226516,25sleep20-21swapper/220:31:492
1357926514,26sleep00-21swapper/023:46:280
2548226419,21sleep20-21swapper/220:56:522
2548226419,21sleep20-21swapper/220:56:522
2365626412,23sleep30-21swapper/322:36:263
1687726414,35sleep316879-21/usr/sbin/munin23:56:373
1054126419,27sleep010657-21cat21:51:570
1054126419,27sleep010657-21cat21:51:570
768926315,34sleep27767-21sed20:01:482
757226318,24sleep20-21swapper/223:26:482
757226318,24sleep20-21swapper/223:26:482
2934026317,24sleep329418-21systemd-run00:36:263
279826313,34sleep22846-21ntp_states23:11:542
27172636,12sleep20-21swapper/219:46:382
27172636,12sleep20-21swapper/219:46:382
884326212,27sleep30-21swapper/321:46:553
884326212,27sleep30-21swapper/321:46:553
1866626214,32sleep218182-21apt-get00:01:372
1368926216,31sleep313835-21cat20:21:283
2995626114,32sleep229895-21latency_hist22:56:272
274426116,23sleep30-21swapper/319:46:433
274426116,23sleep30-21swapper/319:46:433
814326014,31sleep08244-21sed21:46:270
457426012,22sleep30-21swapper/323:16:563
45326019,23sleep00-21swapper/021:21:440
338826015,24sleep33561-21gpgv21:31:283
3059826013,28sleep030610-21vmstat21:12:000
2724326011,22sleep00-21swapper/019:21:280
2724326011,22sleep00-21swapper/019:21:280
2721726017,27sleep027362-21sed00:26:580
139626022,19sleep00-21swapper/023:06:550
1395726017,21sleep30-21swapper/322:02:013
1233726016,27sleep012327-21apt-get20:16:410
317242599,34sleep331739-21acpi23:01:373
2898525919,21sleep00-21swapper/022:51:490
239852596,13sleep00-21swapper/000:16:570
2366425914,25sleep20-21swapper/220:51:472
2061725917,19sleep30-21swapper/322:26:283
1897325919,17sleep00-21swapper/000:01:510
1771425921,18sleep30-21swapper/320:31:583
1771425921,18sleep30-21swapper/320:31:583
1240825920,21sleep20-21swapper/223:41:462
1069425913,22sleep20-21swapper/223:36:532
583725812,26sleep05856-21apt-get19:56:380
455625814,31sleep34631-21cat19:51:483
317642585,12sleep00-21swapper/019:36:100
317642585,12sleep00-21swapper/019:36:100
3116825818,24sleep331291-21timerwakeupswit19:31:573
3116825818,24sleep331291-21timerwakeupswit19:31:573
2258625819,19sleep20-21swapper/220:46:592
2201125820,19sleep00-21swapper/019:06:260
2192925818,23sleep00-21swapper/000:11:440
169062587,13sleep00-21swapper/023:56:420
1580525813,32sleep215410-21/usr/sbin/munin20:26:522
2629625715,17sleep20-21swapper/200:26:282
2270825715,26sleep022740-21awk19:06:510
2233825710,24sleep30-21swapper/319:06:413
575125621,20sleep00-21swapper/023:21:370
36225616,24sleep2358-21apt-get21:21:382
3060825611,30sleep230620-21/usr/sbin/munin19:31:442
3060825611,30sleep230620-21/usr/sbin/munin19:31:442
298632566,38sleep30-21swapper/322:56:163
2832425617,19sleep30-21swapper/321:06:463
2631525621,18sleep00-21swapper/021:01:260
2631525621,18sleep00-21swapper/021:01:260
255152567,36sleep20-21swapper/222:41:382
2117425613,17sleep00-21swapper/022:26:500
2117425613,17sleep00-21swapper/022:26:500
1953025614,27sleep219562-21irqstats22:21:472
1905725611,29sleep019169-21cat22:21:290
166382566,12sleep20-21swapper/222:11:542
1593625615,25sleep30-21swapper/320:26:573
1185225618,24sleep311880-21munin-run20:16:273
1015625615,26sleep310221-21unin-run23:36:263
3267825514,18sleep00-21swapper/019:36:550
3267825514,18sleep00-21swapper/019:36:550
311425513,24sleep00-21swapper/019:46:520
311425513,24sleep00-21swapper/019:46:520
2075025515,23sleep220747-21apt-get19:01:402
652625419,18sleep30-21swapper/323:23:013
3019625412,25sleep030263-21df22:56:420
2769925415,25sleep30-21swapper/319:21:483
2769925415,25sleep30-21swapper/319:21:483
2395025413,25sleep323969-21apt-get22:36:383
1515425412,25sleep015222-21/usr/sbin/munin22:06:550
1430625413,17sleep00-21swapper/020:21:540
3125825313,25sleep231276-21apt21:16:382
2713925315,21sleep20-21swapper/219:21:262
2653225313,23sleep226528-21apt-get00:26:402
2542425312,24sleep025443-21ntp_states00:21:530
2248425317,17sleep20-21swapper/222:31:442
182382536,36sleep00-21swapper/020:36:230
182382536,36sleep00-21swapper/020:36:230
167725311,19sleep30-21swapper/321:26:283
2659125211,18sleep00-21swapper/021:01:420
2551925212,23sleep025513-21apt-get22:41:400
1150025211,23sleep211496-21apt-get21:56:392
992325112,22sleep39919-21apt-get21:51:393
992325112,22sleep39919-21apt-get21:51:393
2556425115,22sleep225595-21munin-run19:16:262
24402515,35sleep20-21swapper/219:46:182
24402515,35sleep20-21swapper/219:46:182
1911725112,22sleep019195-21memory20:37:080
1698225114,21sleep216878-21/usr/sbin/munin23:56:442
867625012,18sleep30-21swapper/320:06:263
528325013,22sleep05279-21apt-get21:36:380
1253925013,23sleep212599-21/usr/sbin/munin20:17:272
450624912,22sleep24555-21irqstats19:51:472
3151124914,15sleep20-21swapper/223:01:282
2443024913,20sleep224436-21ntp_states22:36:532
243752495,32sleep30-21swapper/320:52:043
94242489,17sleep20-21swapper/220:06:532
2814824811,20sleep028143-21apt-get21:06:390
2076724811,20sleep020804-21ntp_states20:41:520
1511924815,17sleep30-21swapper/323:51:283
2183924711,20sleep221835-21apt-get00:11:382
586124612,17sleep30-21swapper/319:56:413
1405924514,17sleep30-21swapper/320:21:483
218982438,17sleep00-21swapper/020:46:410
210852428,15sleep20-21swapper/222:26:482
210852428,15sleep20-21swapper/222:26:482
319952417,17sleep20-21swapper/223:01:482
270702398,14sleep00-21swapper/022:46:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional