You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 14:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Apr 10, 2026 00:43:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
96582102427,980sleep09660-21ntp_states23:27:150
12700241722,377sleep212767-21/usr/sbin/munin20:12:072
389223828,339sleep13901-21unin-run19:46:381
30341234822,307sleep030503-21apt-config22:51:380
9461230742,245sleep19525-21latency23:27:001
4836226153,190sleep14948-21unixbench_multi19:47:061
32680226032,212sleep1339-21wc21:16:381
6142225417,148sleep16289-21unixbench_singl21:33:061
2706922469,221sleep127217-21unixbench_singl00:22:111
2706922469,221sleep127217-21unixbench_singl00:22:111
3051224429,120sleep13113-21fschecks_time23:07:011
3051224429,120sleep13113-21fschecks_time23:07:011
24371221711,147sleep124497-21latency_hist20:51:391
21968221615,159sleep00-21swapper/020:42:050
15539221622,103sleep10-21swapper/123:47:031
15539221622,103sleep10-21swapper/123:47:031
21252221520,179sleep221305-21fschecks_count22:21:562
28299221220,177sleep328316-21idleruntime-cro19:21:363
28086221134,157sleep10-21swapper/100:27:031
1834022109,184sleep118371-21latency_hist23:56:401
1834022109,184sleep118371-21latency_hist23:56:401
7518219612,168sleep27596-21find21:37:062
3927219613,166sleep23956-21nfs4_client21:27:012
1414021899,47sleep10-21swapper/123:42:021
32051218860,56sleep10-21swapper/119:32:001
86321878,56sleep1901-21ntpq21:17:031
16721217710,114sleep116782-21sendmail_mailqu22:07:051
16721217710,114sleep116782-21sendmail_mailqu22:07:051
7114217430,85sleep17247-21gpgv19:56:391
7114217430,85sleep17247-21gpgv19:56:391
18385217211,148sleep118450-21ls20:31:521
90732167111,38sleep39156-21timerandwakeup21:42:083
16708216713,139sleep316717-21unin-run23:51:373
16708216713,139sleep316717-21unin-run23:51:373
17082216515,138sleep017130-21grep23:51:580
17082216515,138sleep017130-21grep23:51:580
11004216358,87sleep111021-21cut23:32:011
1857216248,102sleep11868-21ntpq23:02:051
1857216248,102sleep11868-21ntpq23:02:051
54021609,140sleep013441-21snmpd19:34:170
11262215629,72sleep10-21swapper/120:07:091
1854621538,14sleep10-21swapper/123:56:521
1854621538,14sleep10-21swapper/123:56:521
364521507,100sleep13679-21fschecks_count21:26:561
11334215011,120sleep111485-21latency_hist21:51:391
30695214912,119sleep330214-21apt-get22:51:453
30695214912,119sleep330214-21apt-get22:51:453
28667214811,120sleep128797-21grep21:02:091
25076214611,55sleep10-21swapper/100:17:061
25076214611,55sleep10-21swapper/100:17:061
21781214543,46sleep10-21swapper/122:22:081
26400213811,114sleep126419-21apt-get20:56:481
31346213728,96sleep131415-21ls22:52:321
31346213728,96sleep131415-21ls22:52:321
4504213413,88sleep10-21swapper/123:11:501
4504213413,88sleep10-21swapper/123:11:501
18424213314,95sleep123282-21cyclictest22:12:181
17710213117,92sleep217857-21cat22:11:392
17710213117,92sleep217857-21cat22:11:392
27374212819,52sleep10-21swapper/122:41:501
27374212819,52sleep10-21swapper/122:41:501
3245021265,109sleep00-21swapper/021:14:430
27639212617,87sleep227721-21find19:17:072
14960212513,24sleep30-21swapper/322:01:593
14960212513,24sleep30-21swapper/322:01:593
2303212240,66sleep11-21systemd19:41:371
8486212131,18sleep10-21swapper/121:41:541
8013211861,17sleep18049-21ntpq23:22:041
29634211811,20sleep10-21swapper/121:06:571
29634211811,20sleep10-21swapper/121:06:571
24884211714,26sleep10-21swapper/120:51:591
15063211718,57sleep10-21swapper/122:02:011
15063211718,57sleep10-21swapper/122:02:011
310211210,20sleep10-21swapper/122:57:121
310211210,20sleep10-21swapper/122:57:121
11996211119,73sleep0588-21cron23:36:360
6155210916,77sleep36215-21processes19:52:053
6155210916,77sleep36215-21processes19:52:053
1808221089,89sleep113441-21snmpd20:31:281
3051321067,90sleep113441-21snmpd00:34:141
25731210614,65sleep30-21swapper/319:12:003
12289210611,75sleep012300-21acpi23:36:480
31417210415,72sleep231546-21timerwakeupswit22:52:332
31417210415,72sleep231546-21timerwakeupswit22:52:332
12120210424,57sleep10-21swapper/121:52:111
28351210318,63sleep10-21swapper/119:21:381
13739210345,14sleep10-21swapper/120:16:501
7599210117,62sleep37739-21tail21:37:083
1771210123,51sleep01857-21latency_hist21:21:380
5015210022,56sleep00-21swapper/023:12:090
5015210022,56sleep00-21swapper/023:12:090
27157210023,19sleep10-21swapper/122:41:381
27157210023,19sleep10-21swapper/122:41:381
756129929,57sleep17493-21apt-get19:57:361
3014829927,21sleep10-21swapper/122:50:541
1973329914,61sleep019886-21cat20:36:400
2218029731,18sleep10-21swapper/100:07:091
2218029731,18sleep10-21swapper/100:07:091
1998729716,57sleep319595-21/usr/sbin/munin22:17:013
1538029627,47sleep015540-21grep22:02:080
1538029627,47sleep015540-21grep22:02:080
2159229425,48sleep321631-21fschecks_count20:41:563
215032947,70sleep10-21swapper/120:41:501
1434929417,60sleep214460-21timerandwakeup20:17:082
285892937,17sleep10-21swapper/119:21:511
227172939,71sleep122765-21awk22:26:521
284322925,73sleep00-21swapper/022:43:470
284322925,73sleep00-21swapper/022:43:470
2582729114,59sleep025850-21/usr/sbin/munin19:12:010
2314029117,52sleep223156-21sort20:46:552
2056629137,19sleep120600-21ps00:02:091
2056629137,19sleep120600-21ps00:02:091
1445029117,59sleep214426-21latency_hist22:01:372
1445029117,59sleep214426-21latency_hist22:01:372
1220029122,14sleep10-21swapper/120:11:541
765829020,49sleep07673-21fschecks_count19:57:430
2999329014,59sleep030062-21awk00:32:110
2388529033,37sleep10-21swapper/100:12:101
2388529033,37sleep10-21swapper/100:12:101
1908129014,58sleep2367-21sshd22:13:492
2995328910,23sleep10-21swapper/119:26:391
2995328910,23sleep10-21swapper/119:26:391
69022888,36sleep10-21swapper/123:17:151
2153428815,55sleep321568-21/usr/sbin/munin22:22:013
140602888,46sleep10-21swapper/121:57:161
140602888,46sleep10-21swapper/121:57:161
74252877,41sleep17517-21mailstats21:37:051
51428613,56sleep0642-21sort22:57:150
51428613,56sleep0642-21sort22:57:150
2019028610,40sleep10-21swapper/122:17:071
3071428525,37sleep030716-21apt-get22:51:460
3071428525,37sleep030716-21apt-get22:51:460
3022628518,51sleep230258-21tune2fs19:26:562
3022628518,51sleep230258-21tune2fs19:26:562
2774628517,52sleep327865-21latency_hist21:01:383
2280028517,48sleep00-21swapper/022:26:550
264532846,13sleep10-21swapper/119:13:151
2509828414,56sleep325181-21mailstats20:52:053
847028320,19sleep00-21swapper/021:41:530
687428311,59sleep00-21swapper/019:55:410
687428311,59sleep00-21swapper/019:55:410
624228320,44sleep26288-21fschecks_time23:17:002
242762839,17sleep10-21swapper/122:31:531
242762839,17sleep10-21swapper/122:31:531
2068328317,29sleep30-21swapper/300:02:113
2068328317,29sleep30-21swapper/300:02:113
1290228317,50sleep013003-21latency_hist21:56:390
103172837,42sleep110348-21munin-run20:06:371
881828222,27sleep30-21swapper/320:01:383
201028218,15sleep10-21swapper/121:21:511
914328122,44sleep29208-21uniq20:01:532
2586928119,41sleep225895-21grep20:56:362
1536928122,32sleep20-21swapper/220:21:562
2690128014,54sleep326966-21switchtime00:22:093
2690128014,54sleep326966-21switchtime00:22:093
130328013,48sleep21019-21/usr/sbin/munin19:36:582
114228014,47sleep31268-21/usr/sbin/munin21:17:093
1996027922,41sleep320009-21/usr/sbin/munin20:36:513
1217827921,37sleep212286-21wc21:52:122
1131527912,47sleep011331-21cat21:51:380
313022788,14sleep30-21swapper/321:11:483
235092789,51sleep10-21swapper/119:06:491
235092789,51sleep10-21swapper/119:06:491
1971927813,51sleep319731-21latency_hist20:36:393
331527725,31sleep30-21swapper/321:26:383
2553227721,28sleep00-21swapper/022:36:360
2553227721,28sleep00-21swapper/022:36:360
1750027718,43sleep317524-21awk22:11:363
1750027718,43sleep317524-21awk22:11:363
1453127717,39sleep214552-21awk23:42:152
597527620,27sleep30-21swapper/321:33:043
305627618,35sleep22570-21/usr/sbin/munin19:42:072
2835427619,40sleep228383-21/usr/sbin/munin21:02:022
2311527611,51sleep323159-21ntp_offset22:27:023
363827511,17sleep00-21swapper/021:26:550
169602758,50sleep20-21swapper/223:51:522
169602758,50sleep20-21swapper/223:51:522
2318227421,25sleep20-21swapper/222:27:052
577527321,34sleep25822-21fschecks_count19:51:562
577527321,34sleep25822-21fschecks_count19:51:562
434727311,30sleep30-21swapper/319:46:563
3098927316,34sleep331208-21apt-config21:11:393
2476827317,35sleep024789-21modprobe19:11:360
2476827317,35sleep024789-21modprobe19:11:360
2195827318,30sleep20-21swapper/220:42:032
1753027330,23sleep00-21swapper/022:11:370
1753027330,23sleep00-21swapper/022:11:370
146402730,6sleep00-21swapper/022:01:380
146402730,6sleep00-21swapper/022:01:380
313927212,24sleep00-21swapper/019:42:090
2940627216,42sleep329445-21wc22:47:053
2657327218,33sleep326708-21unixbench_singl22:37:073
2657327218,33sleep326708-21unixbench_singl22:37:073
2515927224,30sleep225232-21perl19:11:422
2087527221,28sleep20-21swapper/222:21:362
1354427215,38sleep10-21swapper/123:41:131
834327113,23sleep00-21swapper/023:22:110
335127112,38sleep03507-21sed21:26:390
3210127112,14sleep20-21swapper/219:32:012
163962717,12sleep00-21swapper/023:48:070
163962717,12sleep00-21swapper/023:48:070
1528927110,30sleep30-21swapper/320:21:533
1191527125,21sleep30-21swapper/320:11:393
2989727020,25sleep00-21swapper/019:26:380
2249227033,22sleep10-21swapper/119:02:051
2249227033,22sleep10-21swapper/119:02:051
201072709,30sleep00-21swapper/022:17:060
1386027021,30sleep013856-21apt-get23:41:490
1118227010,46sleep011260-21ls20:07:080
1107027015,29sleep20-21swapper/220:07:062
791326921,29sleep30-21swapper/323:22:023
3197426912,41sleep232067-21sort21:12:192
3152826921,29sleep031694-21dpkg19:31:390
3152826921,29sleep031694-21dpkg19:31:390
534426820,31sleep25333-21apt-get21:31:542
379726821,27sleep20-21swapper/223:07:152
379726821,27sleep20-21swapper/223:07:152
2596926821,28sleep00-21swapper/020:56:390
2495226826,24sleep30-21swapper/300:17:043
2495226826,24sleep30-21swapper/300:17:043
2526726711,41sleep20-21swapper/200:17:112
2526726711,41sleep20-21swapper/200:17:112
2406426712,42sleep224144-21sed19:07:062
2406426712,42sleep224144-21sed19:07:062
1854026716,35sleep218541-21sshd23:56:492
1854026716,35sleep218541-21sshd23:56:492
628926622,26sleep30-21swapper/323:17:013
3080226615,33sleep030808-21modprobe21:11:360
3080226615,33sleep030808-21modprobe21:11:360
2849126615,26sleep30-21swapper/321:02:073
2836426614,38sleep028396-21proc_pri00:27:100
2740226618,26sleep30-21swapper/322:41:563
2740226618,26sleep30-21swapper/322:41:563
25262666,20sleep00-21swapper/023:04:500
25262666,20sleep00-21swapper/023:04:500
2435326610,40sleep324358-21diskstats22:31:553
2435326610,40sleep324358-21diskstats22:31:553
1868626619,25sleep30-21swapper/323:57:143
1868626619,25sleep30-21swapper/323:57:143
153592668,41sleep10-21swapper/120:21:551
1512026617,32sleep315144-21modprobe23:46:363
1421826612,23sleep00-21swapper/020:17:060
973426514,34sleep39809-21sensors_temp23:27:183
577226520,28sleep25790-21idleruntime-cro23:16:352
577226520,28sleep25790-21idleruntime-cro23:16:352
2643126516,35sleep226459-21fschecks_count00:21:552
2643126516,35sleep226459-21fschecks_count00:21:552
224082655,12sleep00-21swapper/022:26:220
2240726520,21sleep00-21swapper/019:02:040
2240726520,21sleep00-21swapper/019:02:040
1565626515,20sleep00-21swapper/020:22:020
3143926418,32sleep331526-21gpgv19:31:383
3143926418,32sleep331526-21gpgv19:31:383
2748526418,17sleep20-21swapper/222:41:592
2748526418,17sleep20-21swapper/222:41:592
1720526424,22sleep20-21swapper/220:27:012
2896526323,19sleep30-21swapper/319:22:013
2233326317,32sleep322486-21awk00:07:113
2233326317,32sleep322486-21awk00:07:113
2118126320,10sleep10-21swapper/120:41:091
487726213,20sleep30-21swapper/323:12:043
487726213,20sleep30-21swapper/323:12:043
3199926217,28sleep30-21swapper/319:31:543
290242629,38sleep00-21swapper/021:03:490
2859026221,20sleep20-21swapper/219:21:522
2662026211,35sleep026707-21sed00:22:020
2662026211,35sleep026707-21sed00:22:020
2632526214,21sleep00-21swapper/022:37:040
2632526214,21sleep00-21swapper/022:37:040
2035826210,14sleep00-21swapper/020:37:000
2010526218,21sleep00-21swapper/000:01:550
2010526218,21sleep00-21swapper/000:01:550
159126213,34sleep31628-21/usr/sbin/munin23:01:593
159126213,34sleep31628-21/usr/sbin/munin23:01:593
1210926213,24sleep30-21swapper/323:36:393
120126219,19sleep30-21swapper/323:01:383
120126219,19sleep30-21swapper/323:01:383
660326117,27sleep06606-21cron21:36:360
2611826116,22sleep20-21swapper/222:36:582
2611826116,22sleep20-21swapper/222:36:582
2414526114,18sleep30-21swapper/319:07:073
2414526114,18sleep30-21swapper/319:07:073
1800426115,24sleep00-21swapper/022:11:490
1800026118,27sleep317518-21apt-get22:11:483
1723726122,19sleep30-21swapper/320:27:023
104726123,18sleep20-21swapper/221:17:082
323526022,21sleep00-21swapper/023:07:050
323526022,21sleep00-21swapper/023:07:050
1995726015,26sleep219951-21apt-get20:36:502
1994126022,17sleep00-21swapper/000:01:390
1994126022,17sleep00-21swapper/000:01:390
2691825921,20sleep20-21swapper/220:57:292
1446125916,25sleep314603-21awk20:17:093
1276825913,28sleep312894-21/usr/sbin/munin20:12:083
295825814,21sleep30-21swapper/323:06:513
295825814,21sleep30-21swapper/323:06:513
2480425815,25sleep024793-21apt-get00:16:570
2480425815,25sleep024793-21apt-get00:16:570
9592575,10sleep20-21swapper/222:59:362
9592575,10sleep20-21swapper/222:59:362
890025711,27sleep28944-21systemd-run23:26:372
78982577,39sleep00-21swapper/021:38:010
749125717,24sleep26976-21apt-get19:56:492
3019025712,27sleep20-21swapper/200:32:132
1742025717,21sleep30-21swapper/323:52:093
1742025717,21sleep30-21swapper/323:52:093
1081625716,26sleep210823-21diskstats23:31:542
98382568,14sleep20-21swapper/221:46:392
299302566,12sleep30-21swapper/300:32:103
2789625612,26sleep227966-21/usr/sbin/munin00:26:522
275225621,18sleep20-21swapper/221:22:082
230572569,29sleep323124-21df20:46:523
1313625617,23sleep313133-21apt-get21:56:503
1313625617,23sleep313133-21apt-get21:56:503
1231525614,21sleep30-21swapper/323:36:513
1036825614,24sleep310458-21sort20:06:393
2317225514,23sleep023192-21apt-get00:11:480
2317225514,23sleep023192-21apt-get00:11:480
131725512,23sleep31409-21memory19:36:593
413925413,25sleep04134-21apt-get19:46:490
2323525413,20sleep20-21swapper/219:06:372
2323525413,20sleep20-21swapper/219:06:372
1231425412,25sleep212309-21apt-get23:36:502
395725315,17sleep30-21swapper/321:27:023
390325316,20sleep34032-21sort19:46:403
3020025316,22sleep230218-21munin-run22:51:362
2994825317,18sleep30-21swapper/321:07:033
2955125315,20sleep20-21swapper/221:06:542
2955125315,20sleep20-21swapper/221:06:542
2817425312,24sleep328268-21memory00:27:053
2165125313,24sleep021718-21df00:06:520
2165125313,24sleep021718-21df00:06:520
2031225314,17sleep20-21swapper/200:02:022
2031225314,17sleep20-21swapper/200:02:022
655525214,23sleep06561-21ntp_states23:17:100
2750825214,17sleep00-21swapper/019:17:050
2371325221,16sleep30-21swapper/300:12:083
2371325221,16sleep30-21swapper/300:12:083
1837825215,20sleep20-21swapper/220:31:492
1153825216,16sleep30-21swapper/321:51:493
2540625118,17sleep20-21swapper/222:35:002
2540625118,17sleep20-21swapper/222:35:002
1127425113,23sleep311299-21munin-run21:51:373
104625110,15sleep00-21swapper/019:36:520
2305225013,21sleep023048-21apt-get20:46:490
2192425014,17sleep30-21swapper/319:01:503
2192425014,17sleep30-21swapper/319:01:503
186925017,17sleep20-21swapper/223:02:062
186925017,17sleep20-21swapper/223:02:062
926824914,16sleep00-21swapper/020:01:550
926624912,23sleep29272-21diskstats23:26:532
870424910,16sleep20-21swapper/221:41:592
2367024916,19sleep20-21swapper/200:12:072
2367024916,19sleep20-21swapper/200:12:072
2646424811,20sleep026419-21apt-get20:57:150
1600424818,15sleep20-21swapper/222:06:382
1600424818,15sleep20-21swapper/222:06:382
1215624813,14sleep00-21swapper/020:11:530
41162477,14sleep20-21swapper/219:46:472
2172824612,17sleep20-21swapper/200:06:542
2172824612,17sleep20-21swapper/200:06:542
1741024411,16sleep00-21swapper/020:27:070
1666724416,14sleep20-21swapper/223:51:362
1666724416,14sleep20-21swapper/223:51:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional