You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 06:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed May 13, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293382114510,1113sleep118714-1kworker/1:2H21:14:501
23746290914,875sleep323812-21memory00:19:553
31218284223,807sleep031262-21ntpq21:19:540
15836275517,720sleep315856-21/usr/sbin/munin23:54:593
1575327027,683sleep015797-21ntpq23:54:560
10114263425,597sleep010156-21ntpq21:54:560
3244126037,584sleep032477-21ntpq23:04:550
2741125987,556sleep027457-21ntpq19:25:070
27522246913,438sleep327563-21ntpq22:49:563
20050244810,392sleep020189-21latency_hist00:09:340
26465243111,399sleep016382-21kworker/u8:019:24:310
23657240719,372sleep123714-21/usr/sbin/munin00:19:531
21613237913,352sleep121622-21unin-run00:14:321
32332236924,309sleep032368-21ntpq19:40:180
11057236715,263sleep011200-21grep20:15:030
6739235658,207sleep06869-21unixbench_multi23:25:030
22448230911,80sleep00-21swapper/022:34:460
28589228244,96sleep00-21swapper/022:54:330
24971227965,186sleep025097-21latency_hist19:19:340
24971227965,186sleep025097-21latency_hist19:19:340
22539226158,72sleep022664-21latency_hist20:54:330
7899224539,186sleep07931-21munin-plugin-st21:49:310
18300224511,219sleep218311-21ntpq22:20:432
24756223313,205sleep024856-21latency_hist00:24:320
4994222641,146sleep05028-21ps23:20:140
16958221640,81sleep00-21swapper/022:16:080
16958221640,81sleep00-21swapper/022:16:080
24038221424,168sleep124078-21tr20:59:311
24038221424,168sleep124078-21tr20:59:311
25707221215,107sleep025762-21latency_hist21:04:350
25707221215,107sleep025762-21latency_hist21:04:350
449322107,190sleep04529-21ntpq19:54:560
8334220848,30sleep00-21swapper/021:49:510
23690220810,72sleep00-21swapper/020:55:060
23690220810,72sleep00-21swapper/020:55:060
18278220610,65sleep018375-21latency_hist00:04:330
18278220610,65sleep018375-21latency_hist00:04:330
19097219845,132sleep019117-21latency_hist22:24:320
187521938,144sleep02017-21head23:10:040
17705219319,151sleep217847-21grep00:00:072
17705219319,151sleep217847-21grep00:00:072
7869218830,118sleep07991-21sed20:05:020
12953218841,72sleep013019-21fschecks_time22:04:490
28725218242,51sleep00-21swapper/019:29:520
25178216631,81sleep00-21swapper/000:24:530
25178216631,81sleep00-21swapper/000:24:530
9038216142,71sleep08753-21/usr/sbin/munin20:09:540
447721565,14sleep00-21swapper/021:38:440
14714215521,117sleep214742-21awk20:29:312
2032621535,136sleep00-21swapper/022:26:020
1395121528,127sleep00-21swapper/023:49:510
6630215114,121sleep26628-21apt-get21:44:442
2411021448,99sleep024197-21tune2fs22:39:510
119821429,93sleep01233-21http21:29:310
119821429,93sleep01233-21http21:29:310
6140214031,47sleep00-21swapper/020:00:010
418621396,11sleep10-21swapper/121:35:061
30623213911,111sleep030767-21latency_hist21:19:340
11806213710,58sleep011887-21sendmail_mailtr22:00:010
22933213645,51sleep023000-21latency_hist19:14:330
28797213149,33sleep028829-21munin-run21:14:330
6661213017,88sleep16662-21cpu21:45:031
28403212815,94sleep228470-21cat22:54:312
27801212814,97sleep00-21swapper/000:34:300
27801212814,97sleep00-21swapper/000:34:300
1446421275,30sleep00-21swapper/020:26:180
3081121256,107sleep3363-21snmpd21:19:373
20973212440,71sleep020998-21fschecks_count22:29:500
8281211917,37sleep28395-21sed23:30:022
8281211917,37sleep28395-21sed23:30:022
22881211912,88sleep222882-21cron19:14:312
26906211617,33sleep20-21swapper/200:29:562
26906211617,33sleep20-21swapper/200:29:562
2816211521,75sleep32910-21memory19:49:543
10729211522,76sleep00-21swapper/023:39:450
25332211224,20sleep10-21swapper/122:44:331
600921115,11sleep00-21swapper/021:43:120
32507211126,65sleep10-21swapper/119:40:231
17032211111,51sleep00-21swapper/020:34:590
4503210912,80sleep23969-21apt-get23:19:442
1632210927,67sleep01211-21apt-get21:29:350
25737210816,70sleep325889-21sed00:25:043
25737210816,70sleep325889-21sed00:25:043
21863210850,17sleep00-21swapper/000:14:470
1605321076,18sleep00-21swapper/020:31:520
265721068,43sleep026574-1kworker/0:1H23:14:410
265721068,43sleep026574-1kworker/0:1H23:14:410
2547210526,38sleep00-21swapper/019:49:470
15457210518,68sleep115460-21/usr/sbin/munin23:54:491
2344210341,20sleep00-21swapper/019:49:350
14878210319,66sleep114916-21proc_pri22:09:591
29378210227,40sleep029415-21ps22:55:020
8399210114,69sleep18456-21irqstats21:49:521
30325210118,36sleep00-21swapper/019:34:530
30325210118,36sleep00-21swapper/019:34:530
16634210125,59sleep216645-21modprobe23:59:302
26963210024,58sleep326478-21apt-get19:24:423
2079621005,29sleep00-21swapper/020:47:470
1664529916,63sleep316788-21grep22:15:023
1664529916,63sleep316788-21grep22:15:023
1567529921,38sleep00-21swapper/022:14:310
2985229821,57sleep129965-21latency_hist19:34:331
266752965,57sleep00-21swapper/022:47:310
2589029622,49sleep126035-21unixbench_multi19:20:031
2219629611,69sleep222205-21systemd-run22:34:322
171829613,69sleep11211-21apt-get21:29:441
543529515,65sleep15532-21latency_hist19:59:341
2028229511,65sleep30-21swapper/300:09:453
3189429415,62sleep231365-21apt-get19:39:432
1513829314,66sleep215234-21cut23:54:332
3045929214,59sleep330504-21/usr/sbin/munin19:34:563
3045929214,59sleep330504-21/usr/sbin/munin19:34:563
610929123,44sleep16126-21munin-plugin-st21:44:311
460029112,62sleep10-21swapper/123:20:031
2629129121,43sleep326429-21unixbench_singl22:45:033
2176929114,60sleep321833-21/usr/sbin/munin19:09:513
84928920,47sleep3866-21modprobe23:09:313
3134828922,41sleep231365-21apt19:39:312
3134828922,41sleep231365-21apt19:39:312
3017328817,54sleep130064-21/usr/sbin/munin19:34:501
3017328817,54sleep130064-21/usr/sbin/munin19:34:501
123642887,68sleep012431-21fschecks_time23:44:500
575728711,60sleep25831-21fschecks_time19:59:512
2887728613,57sleep228397-21apt-get22:54:402
285232867,34sleep00-21swapper/000:34:580
2665628621,41sleep326854-21apt-config19:24:343
2455528612,57sleep224626-21sensors22:40:022
123922869,43sleep012455-21mailstats20:20:000
112462850,7sleep20-21swapper/221:59:332
3048728430,38sleep00-21swapper/022:59:460
264702846,41sleep012-21rcu_sched21:05:440
211742847,63sleep00-21swapper/019:07:550
341328322,41sleep23544-21unixbench_singl23:15:032
341328322,41sleep23544-21unixbench_singl23:15:032
2013528318,28sleep30-21swapper/319:04:513
2035828234,32sleep00-21swapper/000:09:500
1059728214,47sleep310689-21/usr/sbin/munin20:14:543
760528120,16sleep00-21swapper/023:29:430
760528120,16sleep00-21swapper/023:29:430
230022819,54sleep123143-21latency_hist19:14:341
240928012,51sleep32445-21unin-run23:14:323
2086128012,54sleep320878-21apt22:29:403
944427914,52sleep19519-21timerwakeupswit20:10:041
2600027812,15sleep10-21swapper/122:44:571
1631227817,44sleep316406-21latency_hist20:34:333
132627820,40sleep21448-21cmp21:29:332
132627820,40sleep21448-21cmp21:29:332
3186927722,32sleep30-21swapper/319:39:403
2342027719,39sleep223421-21/usr/sbin/munin00:19:422
1070827713,49sleep110753-21awk20:14:561
2507927611,46sleep225094-21fschecks_count00:24:512
2507927611,46sleep225094-21fschecks_count00:24:512
1654927620,32sleep30-21swapper/320:34:453
1322127612,15sleep10-21swapper/120:24:341
1288327613,43sleep312960-21/usr/sbin/munin23:45:013
2836527513,45sleep128457-21load00:34:551
2625327525,29sleep10-21swapper/100:29:301
2625327525,29sleep10-21swapper/100:29:301
2012327518,29sleep20-21swapper/219:04:502
3085627419,26sleep30-21swapper/322:59:583
1821227420,15sleep00-21swapper/020:39:510
2302227318,23sleep30-21swapper/322:35:023
1886827312,46sleep118883-21needreboot00:04:541
652627215,25sleep10-21swapper/123:25:001
1997827222,26sleep30-21swapper/300:09:333
2410527112,41sleep324108-21/usr/sbin/munin22:39:503
2170427121,34sleep321766-21wc20:50:003
992527012,42sleep310054-21awk23:35:043
992527012,42sleep310054-21awk23:35:043
768827017,35sleep17711-21pluginstate20:04:591
2618327015,23sleep20-21swapper/222:45:022
2131027019,33sleep221408-21grep20:49:512
191212700,9sleep10-21swapper/122:24:331
1508527016,31sleep315107-21awk23:54:303
3247826917,31sleep132565-21fschecks_time21:24:521
3247826917,31sleep132565-21fschecks_time21:24:521
3212826921,30sleep232167-21cut23:04:492
3100526920,20sleep10-21swapper/123:00:041
2847126919,37sleep328509-21ntp_kernel_pll_00:34:573
2838126918,34sleep128401-21idleruntime-cro22:54:301
239526925,23sleep20-21swapper/223:14:312
2004026925,26sleep30-21swapper/320:44:553
1994426912,15sleep10-21swapper/122:25:021
2315626813,28sleep30-21swapper/320:54:573
2315626813,28sleep30-21swapper/320:54:573
1266226819,33sleep312803-21uptime20:20:033
1008726819,28sleep20-21swapper/220:14:332
898426721,22sleep30-21swapper/320:09:533
812426714,30sleep20-21swapper/220:05:042
255512676,10sleep10-21swapper/121:03:391
255512676,10sleep10-21swapper/121:03:391
3261626622,25sleep10-21swapper/123:05:011
2774926618,27sleep227843-21/usr/sbin/munin21:09:582
2653326618,24sleep20-21swapper/219:24:332
1815726611,30sleep30-21swapper/322:20:403
1753626611,42sleep317624-21cat00:00:043
1753626611,42sleep317624-21cat00:00:043
1678426621,26sleep10-21swapper/120:34:511
1169026619,27sleep20-21swapper/220:19:342
492826519,23sleep30-21swapper/321:39:483
36342657,17sleep10-21swapper/123:15:351
36342657,17sleep10-21swapper/123:15:351
2941826519,27sleep129491-21awk22:55:031
2769026514,36sleep227830-21sed19:25:132
2357626515,35sleep323646-21df19:14:483
2357626515,35sleep323646-21df19:14:483
1626126518,30sleep116296-21basename20:34:311
3048926418,25sleep20-21swapper/222:59:492
2064326422,28sleep10-21swapper/119:05:011
134182648,17sleep10-21swapper/123:47:581
1281126416,22sleep20-21swapper/223:45:002
1030626418,12sleep00-21swapper/023:38:080
1030626418,12sleep00-21swapper/023:38:080
883026323,23sleep30-21swapper/321:50:023
612826318,31sleep26189-21find21:44:322
307726314,20sleep30-21swapper/323:14:563
307726314,20sleep30-21swapper/323:14:563
2923026318,30sleep229248-21acpi21:14:432
2354726315,21sleep20-21swapper/220:55:042
2354726315,21sleep20-21swapper/220:55:042
145226317,28sleep31629-21apt-config21:29:343
853926217,19sleep30-21swapper/320:09:333
594026221,26sleep10-21swapper/119:59:541
535626219,21sleep30-21swapper/319:59:313
2384026212,33sleep123550-21/usr/sbin/munin19:14:541
2384026212,33sleep123550-21/usr/sbin/munin19:14:541
2125526216,29sleep121278-21/usr/sbin/munin22:29:551
2058826222,19sleep20-21swapper/200:10:042
130452628,10sleep20-21swapper/220:22:572
52272616,43sleep20-21swapper/219:57:532
296942617,12sleep20-21swapper/219:33:092
216126111,37sleep10-21swapper/123:11:311
1636826115,26sleep10-21swapper/122:14:561
1636826115,26sleep10-21swapper/122:14:561
81872607,27sleep20-21swapper/221:49:452
2256926017,24sleep122655-21fw_packets22:34:511
1971026014,20sleep30-21swapper/322:24:563
183026014,29sleep11899-21/usr/sbin/munin19:45:041
2900925921,17sleep30-21swapper/322:54:533
2608625913,32sleep226149-21perl21:05:342
1812925914,16sleep10-21swapper/120:39:481
1485825912,30sleep30-21swapper/322:09:573
1403825918,25sleep114095-21awk23:49:521
129392595,11sleep20-21swapper/222:04:472
2928325820,20sleep30-21swapper/319:30:033
2183625819,22sleep10-21swapper/120:50:021
1717025820,16sleep20-21swapper/220:35:012
160052587,11sleep20-21swapper/220:31:132
117212588,37sleep10-21swapper/123:41:071
818625721,22sleep10-21swapper/123:30:011
818625721,22sleep10-21swapper/123:30:011
345625723,18sleep30-21swapper/321:34:513
201725718,17sleep20-21swapper/221:30:102
1741425717,17sleep20-21swapper/222:19:342
1741425717,17sleep20-21swapper/222:19:342
106922577,16sleep30-21swapper/321:55:403
913025614,22sleep20-21swapper/223:34:422
913025614,22sleep20-21swapper/223:34:422
3059925612,29sleep330621-21sed21:19:323
2696725612,17sleep10-21swapper/119:24:441
1418225618,21sleep30-21swapper/320:25:043
1131225620,22sleep311287-21/usr/sbin/munin21:59:483
658725516,21sleep20-21swapper/223:25:012
1985325521,19sleep20-21swapper/200:09:322
1944925519,19sleep20-21swapper/220:44:342
1368025513,15sleep10-21swapper/122:05:031
1265225513,17sleep20-21swapper/222:04:352
760825412,18sleep30-21swapper/323:29:463
760825412,18sleep30-21swapper/323:29:463
498625414,23sleep34992-21ntp_states23:20:133
35372548,35sleep20-21swapper/219:51:062
2275725413,27sleep222764-21users00:15:052
1942825413,18sleep20-21swapper/222:24:512
1126225413,20sleep30-21swapper/323:40:003
74625311,27sleep3839-21users21:25:063
74625311,27sleep3839-21users21:25:063
3210125312,14sleep10-21swapper/121:24:311
2773525314,22sleep127748-21irqstats21:09:571
1313525315,25sleep313197-21wc22:04:523
1109725310,18sleep10-21swapper/121:59:331
698625222,17sleep30-21swapper/320:04:323
673325213,22sleep30-21swapper/321:45:053
2818825213,22sleep228275-21perl00:34:522
2743725211,27sleep326518-21/usr/sbin/munin00:30:053
2743725211,27sleep326518-21/usr/sbin/munin00:30:053
2623825217,22sleep126320-21sed21:05:371
2186125212,23sleep321856-21apt-get00:14:463
2011925215,19sleep10-21swapper/120:44:571
1975825216,22sleep20-21swapper/220:44:502
1476725213,17sleep10-21swapper/120:29:341
1462425213,19sleep20-21swapper/222:09:522
1202925213,24sleep212039-21munin-run23:44:322
453825114,15sleep20-21swapper/221:39:312
3129425113,13sleep20-21swapper/221:19:562
958625011,23sleep19609-21ntp_states23:34:581
958625011,23sleep19609-21ntp_states23:34:581
614225016,20sleep36145-21entropy23:24:503
2754125012,23sleep327552-21df_inode21:09:513
2615025016,16sleep30-21swapper/321:05:353
2378325011,16sleep20-21swapper/222:39:332
1854225011,23sleep318579-21ntp_states20:39:583
2435124910,20sleep30-21swapper/320:59:453
2435124910,20sleep30-21swapper/320:59:453
1722724915,15sleep10-21swapper/123:59:541
1722724915,15sleep10-21swapper/123:59:541
1138824913,20sleep111394-21diskstats21:59:501
384624811,22sleep13855-21munin-run19:54:331
253492487,32sleep20-21swapper/221:01:062
253492487,32sleep20-21swapper/221:01:062
2489124710,21sleep224918-21munin-run19:19:322
2489124710,21sleep224918-21munin-run19:19:322
99624612,19sleep2992-21apt-get19:44:442
2699624610,18sleep10-21swapper/100:29:591
2699624610,18sleep10-21swapper/100:29:591
222302459,16sleep10-21swapper/100:14:561
99724311,15sleep30-21swapper/319:44:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional