You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 14:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Tue Sep 22, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28699263279,6304sleep027479-21kworker/0:023:42:000
13895287212,844sleep013896-21diskstats19:27:520
1983624648,444sleep119877-21ntpq23:12:571
3411234629,273sleep13502-21sort22:22:341
3411234629,273sleep13502-21sort22:22:341
27002229145,188sleep127091-21dpkg23:37:321
27885227576,142sleep127923-21fschecks_count20:12:511
22180227516,237sleep022221-21fschecks_count21:37:500
9571226914,240sleep29615-21ntpq19:13:492
26501226810,244sleep126608-21latency_hist21:52:331
877622599,189sleep18871-21fschecks_time19:12:381
12860225668,130sleep113057-21apt-config22:52:341
5387223127,187sleep15532-21unixbench_multi20:43:041
5664222851,140sleep15698-21ps22:28:031
26931222517,56sleep10-21swapper/121:52:541
2852222110,197sleep32919-21unin-run20:37:333
1937220023,65sleep10-21swapper/123:58:081
12347219755,22sleep10-21swapper/119:22:521
4606219428,52sleep10-21swapper/100:07:491
9004219110,128sleep19032-21awk00:22:311
9466218411,155sleep09555-21load19:13:460
4272218133,126sleep14401-21timerwakeupswit22:23:021
4272218133,126sleep14401-21timerwakeupswit22:23:021
22462218159,58sleep10-21swapper/121:37:561
18481217743,76sleep10-21swapper/119:42:471
10552217556,64sleep110607-21true00:27:321
29846217432,124sleep129361-21/usr/sbin/munin20:18:001
11658217339,116sleep10-21swapper/100:28:041
1258621687,146sleep117692-1kworker/1:1H21:07:511
10905216810,15sleep00-21swapper/000:27:350
16380215542,76sleep116458-21readlink21:18:011
3142021539,130sleep131506-21mailstats20:23:011
1639321529,130sleep016417-21df23:02:510
27463215022,109sleep226945-21apt-get23:37:412
22335215010,125sleep122486-21cut23:22:341
32239214843,65sleep132327-21fschecks_time23:52:511
992214711,82sleep10-21swapper/122:13:051
330021477,127sleep13368-21systemctl00:03:001
330021477,127sleep13368-21systemctl00:03:001
9853214510,115sleep09872-21apt-get22:42:440
30592214225,57sleep10-21swapper/122:03:031
1389721429,121sleep113992-21perl19:27:531
26015214131,72sleep126158-21cat20:07:341
1803621419,77sleep118143-21sort21:23:011
17009214014,57sleep10-21swapper/119:37:501
17009214014,57sleep10-21swapper/119:37:501
2088421398,95sleep120914-21awk21:32:551
897521385,121sleep10-21swapper/120:56:161
6845213813,109sleep37002-21sed00:13:043
790213316,99sleep3826-21cat22:12:593
23238213311,60sleep10-21swapper/123:23:031
2810212936,49sleep00-21swapper/000:02:430
2810212936,49sleep00-21swapper/000:02:430
51821285,11sleep10-21swapper/120:28:021
32484212814,16sleep00-21swapper/020:27:480
20507212767,46sleep120564-21find19:48:001
1788621218,14sleep00-21swapper/023:07:430
8797212013,67sleep18859-21processes22:38:001
9966211930,74sleep19999-21cut22:42:531
31835211911,55sleep131859-21modprobe23:52:301
31835211911,55sleep131859-21modprobe23:52:301
2048721186,99sleep00-21swapper/019:47:590
6605211749,41sleep36633-21munin-run19:07:323
13856211711,91sleep213951-21timerandwakeup22:53:282
20265211613,24sleep220294-21irqstats19:47:532
26713211529,59sleep326732-21apt-get21:52:433
2158211528,75sleep12250-21cat20:33:021
8328211332,20sleep10-21swapper/119:12:341
2928621136,10sleep10-21swapper/121:58:431
10609211315,36sleep210661-21/usr/sbin/munin00:27:332
32342211129,71sleep1367-21snmpd22:08:121
2408621085,33sleep10-21swapper/123:27:391
2408621085,33sleep10-21swapper/123:27:391
12023210819,58sleep312000-21latency_hist19:22:343
3650210616,74sleep23656-21cpu22:22:472
3650210616,74sleep23656-21cpu22:22:472
21390210614,73sleep321413-21/usr/sbin/munin23:17:563
30407210516,71sleep330478-21swap22:03:013
11436210522,62sleep211582-21unixbench_singl19:18:052
1443121035,51sleep10-21swapper/122:57:021
15558210222,60sleep015571-21modprobe21:17:310
2978721008,79sleep229836-21/usr/sbin/munin20:17:572
14258210020,60sleep313776-21apt-get21:12:423
2912529928,56sleep129214-21perl23:42:531
2912529928,56sleep129214-21perl23:42:531
2796329940,48sleep128002-21ps23:38:131
1549329924,59sleep215632-21grep22:58:252
166229814,63sleep31237-21/usr/sbin/munin23:58:023
803029727,54sleep11-21systemd22:37:321
801129721,55sleep08029-21cron22:37:310
633329710,70sleep26340-21/usr/sbin/munin20:47:492
2031529715,35sleep120460-21latency_hist21:32:341
2031529715,35sleep120460-21latency_hist21:32:341
1102229733,18sleep10-21swapper/121:02:501
650529617,63sleep36626-21latency_hist22:32:333
1476729633,17sleep10-21swapper/121:13:121
1764929522,56sleep217628-21latency_hist23:07:332
45752946,13sleep00-21swapper/000:07:440
2359129447,33sleep10-21swapper/119:58:011
1104829419,55sleep310567-21apt-get00:27:423
2494529338,18sleep124953-21munin-run21:47:331
54192928,59sleep00-21swapper/019:02:490
2623329220,57sleep226306-21switchtime23:33:012
1105629214,29sleep20-21swapper/200:27:452
1188329119,59sleep211907-21proc_pri22:47:592
907029017,55sleep39071-21cron20:57:313
1638129011,13sleep00-21swapper/019:34:340
2749028915,56sleep327466-21apt-get23:37:553
2594428917,52sleep326085-21/usr/sbin/munin21:48:033
230292897,12sleep20-21swapper/223:23:002
15312895,74sleep00-21swapper/022:14:250
98772888,15sleep10-21swapper/120:58:021
51702889,60sleep25232-21/usr/sbin/munin20:43:012
2622628810,59sleep10-21swapper/120:07:421
1670928813,62sleep316727-21netstat23:02:583
2388928731,30sleep00-21swapper/023:27:340
662128612,51sleep06125-21/usr/sbin/munin00:13:010
1861828623,43sleep30-21swapper/323:08:033
1609928612,29sleep10-21swapper/119:33:061
1677228538,32sleep10-21swapper/123:03:001
667728312,58sleep16790-21latency_hist19:07:341
655528325,43sleep16601-21ntpq00:12:581
132132839,36sleep00-21swapper/022:52:490
825828216,50sleep38353-21grep00:18:053
3118128221,17sleep20-21swapper/222:07:322
252562820,9sleep20-21swapper/220:03:022
1742328220,17sleep20-21swapper/221:22:462
2671728121,45sleep226760-21processes20:08:002
2581728110,56sleep225942-21timerwakeupswit21:48:022
1661228123,37sleep316632-21latency_hist19:37:323
1132428114,19sleep10-21swapper/119:18:041
1070628132,28sleep00-21swapper/021:02:330
75302808,58sleep17551-21munin-run20:52:331
699328016,42sleep07086-21grep20:48:040
2877828014,49sleep328812-21/usr/sbin/munin21:58:013
1262828015,11sleep10-21swapper/100:32:491
343027910,14sleep10-21swapper/120:37:551
2217027910,53sleep10-21swapper/119:53:011
2217027910,53sleep10-21swapper/119:53:011
1323627921,33sleep313341-21timerandwakeup00:33:033
3064427823,28sleep00-21swapper/020:22:310
159782789,59sleep1367-21snmpd23:01:341
1913427718,10sleep10-21swapper/123:12:211
696327613,41sleep27019-21/usr/sbin/munin22:32:522
2915027622,24sleep00-21swapper/020:17:350
1428827615,43sleep214351-21sendmail_mailqu19:28:022
284127518,42sleep22849-21modprobe20:37:312
2386827524,25sleep30-21swapper/323:27:333
1713927513,47sleep017179-21/usr/sbin/munin19:37:530
1713927513,47sleep017179-21/usr/sbin/munin19:37:530
488627415,41sleep34890-21/usr/sbin/munin00:08:033
441727410,51sleep24457-21/usr/sbin/munin20:42:322
1361327417,29sleep30-21swapper/319:27:353
622927324,24sleep30-21swapper/320:47:423
2751927314,43sleep027641-21/usr/sbin/munin21:53:050
2389427328,27sleep30-21swapper/319:58:073
231552736,14sleep20-21swapper/219:57:502
1789927319,34sleep30-21swapper/321:22:593
1633627312,42sleep016379-21sshd21:18:000
45832725,55sleep00-21swapper/022:23:210
45832725,55sleep00-21swapper/022:23:210
3048127216,38sleep030591-21sort22:03:020
2871427213,35sleep028752-21ntpq21:57:570
2731427217,26sleep20-21swapper/221:53:032
118842726,17sleep00-21swapper/021:03:260
3103227116,40sleep031122-21fschecks_time20:22:520
2669727115,32sleep00-21swapper/020:07:580
1919927121,23sleep20-21swapper/223:12:342
1352027125,28sleep013537-21modprobe19:27:310
548227034,18sleep20-21swapper/219:02:512
2210427015,41sleep222107-21sshd23:20:172
2119327017,22sleep00-21swapper/021:33:020
20032706,13sleep30-21swapper/322:17:353
20032706,13sleep30-21swapper/322:17:353
1968827017,24sleep30-21swapper/321:28:083
1968827017,24sleep30-21swapper/321:28:083
1274827016,37sleep012859-21dpkg22:52:330
1610126922,22sleep30-21swapper/323:02:333
85722680,9sleep20-21swapper/200:18:072
2202526813,41sleep222033-21uname19:52:582
2202526813,41sleep222033-21uname19:52:582
1738526813,31sleep20-21swapper/219:37:592
1738526813,31sleep20-21swapper/219:37:592
159422684,53sleep00-21swapper/023:01:050
3192526720,28sleep20-21swapper/220:24:072
256652679,15sleep00-21swapper/023:32:470
1008726714,30sleep30-21swapper/322:42:553
972926610,15sleep00-21swapper/000:22:580
819626613,38sleep08208-21sh00:18:020
290292666,20sleep30-21swapper/323:42:473
290292666,20sleep30-21swapper/323:42:473
2608726617,25sleep00-21swapper/021:48:040
1971326613,37sleep319803-21memory23:12:553
83032659,44sleep38304-21/usr/sbin/munin22:37:423
728826517,21sleep00-21swapper/019:07:570
495426526,23sleep30-21swapper/322:27:323
495426526,23sleep30-21swapper/322:27:323
37282659,12sleep20-21swapper/200:03:062
37282659,12sleep20-21swapper/200:03:062
251672656,48sleep00-21swapper/023:29:520
251672656,48sleep00-21swapper/023:29:520
1916026519,31sleep019196-21/usr/sbin/munin21:27:570
1916026519,31sleep019196-21/usr/sbin/munin21:27:570
683126422,26sleep30-21swapper/322:32:493
501626416,30sleep25058-21awk00:08:062
958826317,25sleep09629-21systemd-run22:42:310
270326313,34sleep02781-21timerandwakeup22:18:000
270326313,34sleep02781-21timerandwakeup22:18:000
216122635,16sleep20-21swapper/221:35:222
173942635,12sleep30-21swapper/319:38:003
173942635,12sleep30-21swapper/319:38:003
896426218,28sleep39007-21ntp_states19:12:403
78132627,13sleep00-21swapper/020:52:480
497326222,23sleep00-21swapper/020:42:550
448126219,20sleep00-21swapper/020:42:340
3222526217,27sleep232323-21users22:08:052
2829526214,28sleep328374-21/usr/sbin/munin20:13:013
165226212,34sleep01661-21df_inode20:32:490
947426117,21sleep30-21swapper/320:57:523
290022615,12sleep20-21swapper/223:42:422
290022615,12sleep20-21swapper/223:42:422
224942617,39sleep20-21swapper/221:37:572
2185726114,21sleep30-21swapper/321:37:333
194982615,11sleep00-21swapper/019:44:270
1397526118,23sleep00-21swapper/021:12:350
1069326115,25sleep30-21swapper/300:27:343
735026017,26sleep27387-21sshd19:07:592
316826014,21sleep30-21swapper/300:02:543
316826014,21sleep30-21swapper/300:02:543
2457826017,25sleep20-21swapper/223:28:002
2457826017,25sleep20-21swapper/223:28:002
1445326021,21sleep20-21swapper/221:13:052
1397826018,20sleep30-21swapper/322:53:293
129812606,11sleep30-21swapper/321:08:013
720325921,18sleep30-21swapper/319:07:563
445825917,22sleep34474-21grep20:42:333
3212325919,23sleep032117-21apt-get23:52:460
3103425913,30sleep331041-21sshd23:48:003
3103425913,30sleep331041-21sshd23:48:003
1862625911,33sleep218690-21idleruntime19:42:522
1614725914,25sleep316240-21memory21:17:553
517625813,23sleep00-21swapper/022:27:450
3256025818,24sleep332576-21ntp_states23:52:573
290162585,14sleep20-21swapper/220:16:522
2542625818,16sleep30-21swapper/323:32:343
2542625818,16sleep30-21swapper/323:32:343
163625819,18sleep20-21swapper/223:58:012
3273925720,22sleep232752-21irqstats20:27:542
2948725715,25sleep029494-21ntp_states23:43:010
2948725715,25sleep029494-21ntp_states23:43:010
2133625712,18sleep30-21swapper/319:52:343
206492575,40sleep00-21swapper/023:16:480
915225617,16sleep20-21swapper/220:57:342
676725614,18sleep20-21swapper/200:13:032
3248225614,24sleep332477-21apt-get20:27:473
2363925614,25sleep323634-21apt-get21:42:463
2315325614,25sleep023149-21apt-get19:57:480
1316425611,21sleep20-21swapper/200:33:022
323242558,33sleep30-21swapper/322:08:063
3115825518,21sleep20-21swapper/223:48:032
3115825518,21sleep20-21swapper/223:48:032
294522556,38sleep20-21swapper/222:00:502
2646025514,24sleep326517-21awk20:07:533
1978725522,17sleep30-21swapper/319:47:333
195862556,15sleep20-21swapper/221:28:072
195862556,15sleep20-21swapper/221:28:072
148622554,12sleep30-21swapper/319:29:533
2495425415,25sleep225015-21latency_hist21:47:342
247062549,26sleep30-21swapper/320:02:483
2207625418,20sleep00-21swapper/019:53:000
2207625418,20sleep00-21swapper/019:53:000
1262025415,23sleep012618-21apt-get00:32:440
116902545,15sleep00-21swapper/019:18:550
966225316,16sleep20-21swapper/222:42:342
305822539,28sleep00-21swapper/023:47:490
305822539,28sleep00-21swapper/023:47:490
256692535,35sleep00-21swapper/020:03:460
2107225313,24sleep021097-21df23:17:490
134825313,23sleep01352-21diskstats23:57:550
1094625315,18sleep30-21swapper/321:02:483
101125318,14sleep20-21swapper/223:57:322
3228125216,21sleep332429-21latency_hist20:27:353
2217325212,22sleep322178-21diskstats21:37:493
185022525,38sleep00-21swapper/021:25:100
1720225117,16sleep20-21swapper/221:22:342
981825014,20sleep29842-21ntp_states20:57:592
2938425012,22sleep329380-21apt-get20:17:463
186625013,21sleep22002-21date22:17:342
1310625012,23sleep213164-21users19:23:052
1024725014,22sleep210277-21awk22:42:582
649524913,21sleep26504-21munin-run22:32:322
1146924913,22sleep311398-21/usr/sbin/munin22:47:473
491424811,21sleep34971-21irqstats20:42:543
2361524811,22sleep023634-21apt-get21:42:430
1969824815,17sleep20-21swapper/223:12:542
1380224811,23sleep213890-21gpgv21:12:322
1162024815,16sleep20-21swapper/221:03:042
2574824712,15sleep30-21swapper/323:32:503
210224712,20sleep22169-21df22:17:462
210224712,20sleep22169-21df22:17:462
2157224510,18sleep321639-21df19:52:463
2157224510,18sleep321639-21df19:52:463
3142124410,18sleep031416-21apt-get22:07:470
1559124412,16sleep20-21swapper/219:32:552
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional