You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 09:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 02, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292042187222,1809sleep33027-1kworker/3:1H19:34:473
229742141415,1384sleep122979-21ntp_states00:19:571
6135281330,759sleep36225-21fschecks_time20:04:523
6314270319,667sleep26319-21/usr/sbin/munin20:04:552
1629260523,563sleep21723-21/usr/sbin/munin19:49:532
19974257622,533sleep219766-21apt-get20:49:332
19974257622,533sleep219766-21apt-get20:49:332
189672511470,26sleep018969-21sshd00:07:560
16057251115,462sleep316062-21cron22:16:293
16057251115,462sleep316062-21cron22:16:293
1503324587,441sleep22877-1kworker/2:1H22:14:462
1503324587,441sleep22877-1kworker/2:1H22:14:462
25708236664,188sleep325746-21fschecks_count00:29:483
9561234316,301sleep29700-21latency_hist23:39:312
32534230910,192sleep332606-21latency19:44:533
10400227113,108sleep310403-21cron20:19:303
25385226113,236sleep325359-21latency_hist00:29:323
265112249103,22sleep30-21swapper/322:49:493
727922467,227sleep37354-21find23:29:593
15809224256,122sleep30-21swapper/323:59:313
26227223770,110sleep326270-21munin-run21:09:293
26227223770,110sleep326270-21munin-run21:09:293
23168223517,105sleep323332-21unixbench_multi19:15:493
23168223517,105sleep323332-21unixbench_multi19:15:493
184942221103,100sleep318557-21fschecks_time22:24:493
25651220854,140sleep325737-21fschecks_time19:24:503
25333220219,161sleep30-21swapper/322:44:563
7182220129,134sleep37284-21fschecks_time21:50:403
4479219245,33sleep30-21swapper/319:59:433
29269218119,117sleep329331-21unin-run22:59:303
3414217816,148sleep03504-21cut19:54:590
13240217017,38sleep30-21swapper/322:09:423
6482216423,125sleep16518-21modprobe21:49:291
3096216410,118sleep33159-21cat19:54:503
11405216235,111sleep30-21swapper/322:04:303
24026216111,111sleep324151-21sed21:00:003
16414215951,23sleep30-21swapper/300:00:043
29696215713,122sleep021803-21kworker/u8:021:19:320
29905215531,112sleep329959-21ls21:19:433
12897215510,99sleep312891-21apt-get23:49:423
26900215215,121sleep30-21swapper/321:09:543
18576215283,46sleep318664-21fschecks_time20:44:503
18576215283,46sleep318664-21fschecks_time20:44:503
23456213911,34sleep323511-21fschecks_time22:39:513
20982213812,111sleep221008-21sh00:14:452
1707221350,13sleep00-21swapper/022:20:370
29509213413,101sleep10-21swapper/119:34:571
713821328,105sleep00-21swapper/023:29:560
2033521318,22sleep30-21swapper/320:50:193
22375212616,92sleep121506-21apt-get19:14:421
22375212616,92sleep121506-21apt-get19:14:421
22278212412,75sleep322302-21cat00:19:303
1724212145,20sleep30-21swapper/319:49:543
28063212023,78sleep128127-21sort22:54:471
5731211812,91sleep35819-21grep21:44:593
27631211859,47sleep327650-21open_inodes00:34:573
9642211718,31sleep20-21swapper/220:15:002
2859721175,76sleep30-21swapper/319:34:233
1994621179,96sleep320018-21sort00:10:013
1994621179,96sleep320018-21sort00:10:013
26129211613,88sleep126205-21awk00:29:581
1164211627,74sleep31284-21cat21:30:013
15036211522,72sleep015019-21apt-get22:14:480
15036211522,72sleep015019-21apt-get22:14:480
19141211456,44sleep319278-21date00:09:333
2883421130,7sleep10-21swapper/119:34:321
13312211321,74sleep213321-21df_inode22:09:462
1134821139,65sleep30-21swapper/323:44:413
11057211210,58sleep30-21swapper/323:44:293
2976821099,82sleep029885-21/usr/sbin/munin19:35:050
20507210920,70sleep20-21swapper/219:10:032
16273210815,53sleep216259-21apt-get23:59:462
2185221078,64sleep321833-21apt-get20:54:553
2185221078,64sleep321833-21apt-get20:54:553
20772210715,76sleep120843-21sensors_temp19:10:101
2873921068,15sleep30-21swapper/321:14:593
289421057,37sleep30-21swapper/323:15:013
289421057,37sleep30-21swapper/323:15:013
1791221045,41sleep30-21swapper/320:40:463
13586210421,60sleep013736-21sed23:50:030
1071721048,53sleep30-21swapper/320:19:423
6984210315,67sleep10-21swapper/121:49:411
43821039,51sleep30-21swapper/323:09:443
3443210313,72sleep23596-21latency_hist21:39:332
13899210313,49sleep30-21swapper/320:29:473
17880210212,77sleep117907-21df00:04:501
8647210129,58sleep38672-21fschecks_count21:54:483
8647210129,58sleep38672-21fschecks_count21:54:483
31616210114,70sleep031631-21uname23:04:520
19651210139,40sleep219683-21munin-run19:09:302
16991210113,22sleep10-21swapper/122:20:271
16826210047,19sleep316333-21apt-get22:19:403
403729810,48sleep30-21swapper/323:19:583
3183829810,40sleep331906-21systemctl23:04:593
225922987,48sleep30-21swapper/300:19:463
1195529813,69sleep112031-21grep23:44:591
239952978,49sleep30-21swapper/319:19:413
124782978,46sleep30-21swapper/320:24:513
124782978,46sleep30-21swapper/320:24:513
579429612,69sleep05916-21timerwakeupswit23:24:590
314542967,49sleep30-21swapper/321:24:443
2018629617,60sleep019679-21apt-get19:09:400
1390329625,44sleep013944-21fschecks_count20:29:480
83052949,46sleep30-21swapper/323:34:483
2689229417,56sleep127016-21latency_hist19:29:321
153632937,46sleep30-21swapper/320:34:443
1246729311,69sleep212468-21sshd23:47:392
416229217,58sleep24197-21basename19:59:302
309982928,46sleep30-21swapper/319:39:563
1731029115,58sleep017338-21/usr/sbin/munin20:39:540
1072029120,53sleep110774-21ls20:19:441
200362908,45sleep30-21swapper/322:29:463
200362908,45sleep30-21swapper/322:29:463
1971329032,34sleep319809-21apt-config19:09:313
1545929012,60sleep114999-21/usr/sbin/munin22:15:381
1545929012,60sleep114999-21/usr/sbin/munin22:15:381
655128915,57sleep16631-21tr20:05:021
281828914,18sleep20-21swapper/223:15:002
281828914,18sleep20-21swapper/223:15:002
143682895,11sleep20-21swapper/223:54:332
2127728816,55sleep0225-21systemd-journal22:34:310
2127728816,55sleep0225-21systemd-journal22:34:310
2292228710,57sleep30-21swapper/322:39:313
264082867,12sleep00-21swapper/000:30:010
1662228526,42sleep016663-21systemctl20:39:300
2359628416,47sleep123654-21/usr/sbin/munin20:59:501
225428416,53sleep02267-21grep23:14:500
225428416,53sleep02267-21grep23:14:500
1846428422,31sleep10-21swapper/119:04:451
175528416,53sleep21818-21latency_hist23:14:312
2824628313,55sleep028289-21latency22:54:510
2977228219,49sleep329776-21acpi22:59:423
26852820,7sleep00-21swapper/021:35:020
2501828212,51sleep125091-21/usr/sbin/munin22:44:501
133042829,59sleep20-21swapper/223:49:572
1185228221,24sleep20-21swapper/222:04:492
82492818,13sleep30-21swapper/320:10:013
206322819,21sleep00-21swapper/020:50:480
109528111,57sleep21163-21swap21:30:002
266328015,48sleep12783-21latency_hist19:54:321
2260228023,36sleep222650-21grep19:15:362
2260228023,36sleep222650-21grep19:15:362
1474628019,39sleep014932-21apt-key22:14:320
51282797,14sleep30-21swapper/323:24:423
43832790,6sleep00-21swapper/023:20:020
262062790,7sleep20-21swapper/200:29:582
2743927815,20sleep10-21swapper/100:34:511
3197027722,36sleep031991-21modprobe19:44:290
2779027722,36sleep227820-21basename21:14:292
211652779,14sleep20-21swapper/220:50:592
1670227716,19sleep20-21swapper/220:39:322
2850227522,27sleep00-21swapper/000:39:300
1570327520,36sleep115743-21munin-run23:59:291
2207227422,21sleep00-21swapper/019:14:370
2207227422,21sleep00-21swapper/019:14:370
214242746,11sleep30-21swapper/319:13:473
1450027427,23sleep10-21swapper/122:14:291
46522737,11sleep30-21swapper/321:40:513
2524127311,20sleep10-21swapper/119:24:111
2160227316,29sleep10-21swapper/122:34:481
2104027316,18sleep10-21swapper/120:50:581
167392738,50sleep116750-21ntpq00:00:101
2707527213,43sleep027143-21/usr/sbin/munin21:09:590
753827115,39sleep27544-21ntp_states21:50:482
549227115,40sleep05510-21grep21:44:520
325827146,16sleep321686-21munin-node21:38:133
2778127117,36sleep2364-21snmpd19:30:422
2689327115,26sleep10-21swapper/122:49:591
267227125,27sleep10-21swapper/123:14:591
267227125,27sleep10-21swapper/123:14:591
266432716,11sleep00-21swapper/019:27:170
2459027120,29sleep00-21swapper/022:44:290
116582715,12sleep30-21swapper/322:04:343
559327020,24sleep20-21swapper/221:44:542
456727012,42sleep14572-21/usr/sbin/munin19:59:491
3191827020,21sleep20-21swapper/221:24:592
2980027022,27sleep00-21swapper/022:59:470
2421827014,23sleep00-21swapper/019:19:500
1979527013,42sleep019882-21dpkg20:49:310
1979527013,42sleep019882-21dpkg20:49:310
1321127010,12sleep00-21swapper/020:25:240
1321127010,12sleep00-21swapper/020:25:240
1293527018,30sleep10-21swapper/120:25:011
1293527018,30sleep10-21swapper/120:25:011
1084027017,37sleep0369-21sshd23:41:390
53626921,24sleep10-21swapper/123:09:501
2122426916,37sleep221254-21munin-run22:34:292
2122426916,37sleep221254-21munin-run22:34:292
1624726913,22sleep00-21swapper/023:59:400
52526814,37sleep0535-21/usr/sbin/munin23:09:490
498826815,22sleep10-21swapper/121:44:331
2497126819,26sleep00-21swapper/022:44:490
1961026813,43sleep00-21swapper/022:28:460
2714426728,22sleep10-21swapper/121:10:001
1748926713,24sleep20-21swapper/220:40:002
808926612,21sleep10-21swapper/120:09:591
3260826618,33sleep132642-21awk23:09:291
3244326615,32sleep132484-21/usr/sbin/munin19:44:511
235792666,18sleep00-21swapper/000:21:530
2316026620,29sleep20-21swapper/200:20:022
1591926617,28sleep015722-21apt-get23:59:320
1214026616,29sleep00-21swapper/022:04:550
1040626621,23sleep00-21swapper/021:59:500
960226515,37sleep19639-21ps20:14:591
367426516,30sleep03669-21apt-get21:39:430
2701826513,35sleep227214-21cat19:29:332
1981226519,23sleep00-21swapper/019:09:320
1460126511,23sleep10-21swapper/120:30:021
59626415,29sleep10-21swapper/121:29:471
218826419,21sleep10-21swapper/121:34:511
533726319,25sleep20-21swapper/223:24:492
3154226319,28sleep031598-21tune2fs21:24:490
1945326316,32sleep219459-21df_inode00:09:482
1945326316,32sleep219459-21df_inode00:09:482
1701326310,26sleep10-21swapper/120:39:471
2874326218,21sleep00-21swapper/019:34:320
211762628,43sleep1364-21snmpd22:33:521
211762628,43sleep1364-21snmpd22:33:521
2014726216,27sleep120288-21/usr/sbin/munin00:10:031
2014726216,27sleep120288-21/usr/sbin/munin00:10:031
1266126223,20sleep00-21swapper/023:49:310
1111026217,29sleep111212-21latency_hist23:44:311
1078026216,22sleep20-21swapper/222:00:002
2923326117,28sleep229268-21sendmail22:59:292
2393526122,19sleep20-21swapper/222:40:032
2212026120,20sleep00-21swapper/022:35:000
2169426119,19sleep20-21swapper/222:34:492
909626021,20sleep20-21swapper/221:55:002
909626021,20sleep20-21swapper/221:55:002
48092608,10sleep20-21swapper/221:42:522
3233426017,18sleep20-21swapper/219:44:482
309662606,11sleep10-21swapper/121:21:551
3071326015,25sleep10-21swapper/119:39:461
23932606,42sleep00-21swapper/019:51:400
2387526012,32sleep023896-21awk20:59:570
2184926017,25sleep021833-21apt-get20:54:520
2184926017,25sleep021833-21apt-get20:54:520
728725917,19sleep00-21swapper/021:50:410
720425912,31sleep27278-21mailstats23:29:582
624625911,33sleep16248-21sshd23:27:391
2997525919,26sleep229997-21grep22:59:572
2725525920,19sleep20-21swapper/200:34:482
2416225919,19sleep10-21swapper/100:24:491
2710625812,25sleep027230-21sed22:50:020
2531725817,23sleep20-21swapper/219:24:322
1666625818,26sleep116701-21munin-run20:39:311
831325712,20sleep00-21swapper/023:34:490
822925713,22sleep10-21swapper/123:34:461
298725723,18sleep10-21swapper/119:54:481
2693625712,18sleep20-21swapper/222:50:002
2159925719,19sleep00-21swapper/000:14:590
952225611,24sleep10-21swapper/121:55:391
952225611,24sleep10-21swapper/121:55:391
922325614,22sleep00-21swapper/020:14:500
1848225613,24sleep10-21swapper/122:24:471
1699825614,17sleep20-21swapper/222:20:282
1431525612,21sleep20-21swapper/220:29:592
128825615,19sleep00-21swapper/021:30:020
1148825618,16sleep10-21swapper/122:04:321
685425512,25sleep06946-21users20:05:050
580025512,20sleep00-21swapper/020:04:320
288002556,12sleep20-21swapper/221:15:022
2533325520,16sleep00-21swapper/000:29:290
2341125513,25sleep223407-21apt-get20:59:432
2002125515,23sleep020146-21sort00:10:020
2002125515,23sleep020146-21sort00:10:020
1290725514,23sleep112974-21df23:49:481
1236925514,18sleep10-21swapper/122:05:001
1071625513,25sleep210712-21apt-get20:19:412
1016125515,18sleep20-21swapper/223:39:512
398225418,17sleep10-21swapper/123:19:551
3226125415,20sleep00-21swapper/019:44:460
3202025412,18sleep20-21swapper/219:44:312
2102925415,24sleep121125-21grep00:14:481
3264425312,19sleep20-21swapper/223:09:302
2384225315,17sleep10-21swapper/122:40:021
2231325312,23sleep122319-21ntp_states20:55:101
2231325312,23sleep122319-21ntp_states20:55:101
1901425314,23sleep219112-21sed22:25:002
1237825315,17sleep20-21swapper/220:24:502
1237825315,17sleep20-21swapper/220:24:502
325882527,10sleep10-21swapper/121:27:531
2629325211,18sleep20-21swapper/221:09:322
2629325211,18sleep20-21swapper/221:09:322
2622325213,16sleep20-21swapper/222:49:322
1856725212,25sleep118570-21df_inode20:44:481
1856725212,25sleep118570-21df_inode20:44:481
837625112,21sleep08491-21users20:10:020
2832225112,23sleep128317-21apt-get21:14:441
2317225114,17sleep20-21swapper/220:59:312
2317225114,17sleep20-21swapper/220:59:312
1117325114,21sleep011179-21ntp_states20:19:550
856425014,19sleep00-21swapper/021:54:450
856425014,19sleep00-21swapper/021:54:450
2496725012,23sleep024962-21apt-get21:04:420
2496725012,23sleep024962-21apt-get21:04:420
758524910,22sleep27581-21apt-get20:09:432
297652498,19sleep10-21swapper/122:59:401
2407224912,21sleep224069-21apt-get00:24:422
484524713,18sleep20-21swapper/219:59:542
402024711,21sleep24036-21ntp_states23:19:562
299622479,15sleep00-21swapper/021:19:440
178292469,20sleep20-21swapper/200:04:472
308292454,11sleep20-21swapper/221:20:102
250492459,17sleep10-21swapper/121:04:471
250492459,17sleep10-21swapper/121:04:471
2962824411,17sleep20-21swapper/219:35:022
66732427,12sleep10-21swapper/123:29:411
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional