You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 05:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Sep 23, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20100259621,556sleep320170-21/usr/sbin/munin19:22:583
9090256419,526sleep39108-21cron22:17:283
5382243333,358sleep15414-21awk23:47:271
6123231816,175sleep10-21swapper/120:22:581
19623228539,207sleep119710-21grep19:22:481
15504226972,144sleep10-21swapper/119:12:301
15604226463,160sleep115608-21acpi22:37:371
11779225128,163sleep111920-21latency_hist20:42:321
2067622426,201sleep120720-21ntpq22:52:551
9207223242,61sleep10-21swapper/120:32:511
21939223225,184sleep122029-21fschecks_time22:57:481
3318222712,192sleep13469-21head23:37:581
3318222712,192sleep13469-21head23:37:581
6202222230,179sleep16217-21kernelversion23:48:121
6202222230,179sleep16217-21kernelversion23:48:121
31736221627,61sleep10-21swapper/120:02:301
760221419,130sleep10-21swapper/123:32:291
834220549,106sleep10-21swapper/120:07:311
2532522027,161sleep124936-21/usr/sbin/munin23:07:581
13601220131,113sleep113654-21fschecks_count20:47:491
2350022009,152sleep123604-21cut23:02:491
2350022009,152sleep123604-21cut23:02:491
1672822009,130sleep116801-21switchtime19:12:591
8473219461,96sleep1231-21systemd-journal23:54:181
4204219010,66sleep14178-21/usr/sbin/munin20:17:451
12773218514,101sleep112478-21/usr/sbin/munin22:27:521
18225218116,81sleep10-21swapper/119:18:361
1946021809,33sleep10-21swapper/121:07:271
1946021809,33sleep10-21swapper/121:07:271
3016321756,133sleep129837-21apt-get23:22:311
3016321756,133sleep129837-21apt-get23:22:311
30611217338,18sleep10-21swapper/121:42:291
25990216740,90sleep126040-21latency_hist21:27:311
12169216734,97sleep112218-21if_eth020:42:521
31639216631,117sleep131774-21unixbench_singl21:43:011
13783216522,27sleep10-21swapper/100:12:471
29057216028,119sleep10-21swapper/123:18:031
19462216015,127sleep219601-21grep22:48:002
19462216015,127sleep219601-21grep22:48:002
2224215511,27sleep12373-21tail21:52:591
2224215511,27sleep12373-21tail21:52:591
3837215318,117sleep23858-21idleruntime-cro23:42:272
3837215318,117sleep23858-21idleruntime-cro23:42:272
19340215329,82sleep119461-21cat22:47:591
19340215329,82sleep119461-21cat22:47:591
2104321509,126sleep121070-21munin-run21:12:281
14823215015,120sleep014952-21awk22:32:590
1746214910,78sleep10-21swapper/120:07:591
20616214425,100sleep220764-21grep00:32:592
518921438,99sleep15231-21processes22:02:561
1121221407,74sleep10-21swapper/100:02:581
1121221407,74sleep10-21swapper/100:02:581
30495213920,95sleep330553-21cut23:23:303
30495213920,95sleep330553-21cut23:23:303
14243213913,75sleep10-21swapper/122:32:481
5416213218,100sleep25485-21apt-key23:47:282
31663213222,89sleep231790-21latency_hist23:27:292
31663213222,89sleep231790-21latency_hist23:27:292
28545213221,42sleep10-21swapper/121:33:001
17250213164,53sleep117331-21timerandwakeup20:57:591
9508213011,19sleep10-21swapper/123:57:571
9508213011,19sleep10-21swapper/123:57:571
21313212869,16sleep10-21swapper/121:12:451
21313212869,16sleep10-21swapper/121:12:451
11221212619,52sleep10-21swapper/122:22:511
20561212414,91sleep320643-21/usr/sbin/munin22:52:533
3324211817,34sleep10-21swapper/121:57:511
3324211817,34sleep10-21swapper/121:57:511
140621168,96sleep11452-21ntpq23:32:531
1564221158,40sleep10-21swapper/120:52:581
12609211433,57sleep312600-21apt-get19:02:443
6603211318,77sleep26726-21cat23:48:212
6603211318,77sleep26726-21cat23:48:212
2849211314,80sleep02938-21/usr/sbin/munin23:37:490
2849211314,80sleep02938-21/usr/sbin/munin23:37:490
1577621139,66sleep10-21swapper/100:17:561
16940211219,30sleep20-21swapper/222:42:302
31971211134,46sleep031967-21apt-get20:02:410
27438210819,27sleep10-21swapper/121:31:511
14854210821,68sleep214999-21latency_hist20:52:302
26825210710,75sleep326841-21sort23:12:483
32403210517,72sleep232482-21awk23:27:582
24442210415,75sleep324548-21latency_hist21:22:323
23991210418,51sleep10-21swapper/121:20:201
1683721036,35sleep10-21swapper/100:22:411
1683721036,35sleep10-21swapper/100:22:411
1263921029,48sleep112709-21mailstats00:07:591
1263921029,48sleep112709-21mailstats00:07:591
2884321018,32sleep10-21swapper/119:52:451
1820021017,60sleep1367-21snmpd22:44:231
2571821007,32sleep10-21swapper/119:42:391
2298429915,71sleep223009-21grep21:17:502
188012998,17sleep00-21swapper/022:47:450
188012998,17sleep00-21swapper/022:47:450
2642429816,68sleep226479-21if_eth021:27:482
211712989,70sleep221262-21/usr/sbin/munin19:27:502
1828729717,65sleep318298-21awk21:02:493
1828729717,65sleep318298-21awk21:02:493
605829515,66sleep06082-21grep23:48:070
605829515,66sleep06082-21grep23:48:070
41172957,75sleep00-21swapper/021:58:590
41172957,75sleep00-21swapper/021:58:590
3010229516,63sleep030145-21idleruntime-cro19:57:280
2846629510,67sleep228029-21apt-get23:17:322
1546229512,63sleep015043-21/usr/sbin/munin20:52:530
1393129545,34sleep10-21swapper/119:07:301
743529417,61sleep07454-21apt-get23:52:390
3039329319,24sleep30-21swapper/319:57:403
2789929332,41sleep10-21swapper/119:48:001
201072939,68sleep320144-21idleruntime00:32:483
1286329312,68sleep212873-21needreboot22:27:532
808929229,40sleep10-21swapper/122:12:491
1504329123,40sleep20-21swapper/200:17:292
628929014,57sleep26359-21sensors_temp20:23:032
583929013,60sleep35905-21fschecks_time20:22:523
1799028913,60sleep017968-21apt-get19:17:500
1474628915,53sleep314180-21/usr/sbin/munin19:08:043
2396128830,42sleep123901-21latency_hist19:37:301
1615428715,36sleep20-21swapper/219:12:442
1232728717,53sleep212560-21apt-key19:02:312
757328631,20sleep17561-21/usr/sbin/munin22:12:291
42542860,7sleep20-21swapper/220:17:452
1515828612,55sleep315043-21/usr/sbin/munin20:52:483
33852859,38sleep10-21swapper/120:13:001
241982859,33sleep10-21swapper/119:37:421
1997728517,54sleep320032-21interrupts21:07:543
1565428548,17sleep00-21swapper/000:17:510
828328220,29sleep10-21swapper/120:28:591
2729628219,43sleep227333-21cpu19:47:452
1906228213,32sleep10-21swapper/100:27:581
1574628211,56sleep315752-21ntp_states00:17:543
1506228225,33sleep314994-21latency_hist00:17:303
2091728119,28sleep10-21swapper/100:33:551
1861728115,47sleep218693-21switchtime19:18:452
102202818,31sleep10-21swapper/122:18:061
51812800,6sleep00-21swapper/022:02:540
2617328012,52sleep326197-21ntp_states19:42:533
1820528014,49sleep018180-21/usr/sbin/munin21:02:460
1820528014,49sleep018180-21/usr/sbin/munin21:02:460
1589028014,47sleep015904-21grep22:37:480
559127915,38sleep05397-21apt-get23:47:300
213512799,29sleep10-21swapper/119:27:531
1673127917,41sleep216771-21sort20:57:472
2558627819,38sleep025705-21timerwakeupswit23:08:050
169132780,5sleep20-21swapper/200:22:452
169132780,5sleep20-21swapper/200:22:452
1478127712,48sleep014821-21sh20:52:280
1279227725,21sleep30-21swapper/300:08:013
1279227725,21sleep30-21swapper/300:08:013
116227717,27sleep10-21swapper/121:51:131
2157327613,47sleep021585-21/usr/sbin/munin21:12:520
2157327613,47sleep021585-21/usr/sbin/munin21:12:520
1785827616,26sleep30-21swapper/322:43:003
2722627519,15sleep10-21swapper/123:13:141
261992749,16sleep00-21swapper/019:42:540
1754727419,37sleep017578-21/usr/sbin/munin22:42:530
127227423,32sleep21307-21latency23:32:502
2544427319,26sleep20-21swapper/223:08:032
2509727316,37sleep325135-21ntpq21:23:093
1097727316,37sleep311016-21/usr/sbin/munin00:02:523
1097727316,37sleep311016-21/usr/sbin/munin00:02:523
1068527325,26sleep00-21swapper/020:37:510
3131327220,24sleep00-21swapper/019:58:010
2782527213,45sleep227850-21/usr/sbin/munin21:32:452
2131527228,24sleep00-21swapper/000:37:300
1213027226,29sleep012149-21apt-get00:07:410
1213027226,29sleep012149-21apt-get00:07:410
1066327223,34sleep210694-21munin-run22:22:272
951927120,24sleep00-21swapper/020:32:580
2921527122,27sleep30-21swapper/323:18:053
288527118,26sleep00-21swapper/021:57:300
288527118,26sleep00-21swapper/021:57:300
2623027116,31sleep20-21swapper/219:42:562
1599827117,20sleep00-21swapper/019:12:380
99622706,13sleep00-21swapper/023:58:040
99622706,13sleep00-21swapper/023:58:040
410227013,26sleep30-21swapper/323:42:403
3157827016,40sleep231579-21sshd20:01:062
2943427022,25sleep20-21swapper/221:37:452
237927014,29sleep30-21swapper/320:12:303
1038527018,24sleep30-21swapper/300:02:313
1038527018,24sleep30-21swapper/300:02:313
1017627016,24sleep30-21swapper/320:37:293
2891126912,25sleep20-21swapper/219:52:462
226126919,25sleep30-21swapper/323:37:283
1685626915,34sleep016620-21/usr/sbin/munin20:57:490
1250426912,39sleep312574-21df22:27:453
74026817,36sleep3888-21unixbench_singl21:48:003
664726826,22sleep20-21swapper/222:07:582
3125026814,34sleep031311-21/usr/sbin/munin23:23:440
3125026814,34sleep031311-21/usr/sbin/munin23:23:440
3069926818,27sleep30-21swapper/321:42:313
281432685,14sleep30-21swapper/319:48:023
2466026818,32sleep324692-21munin-run23:07:273
2466026818,32sleep324692-21munin-run23:07:273
1831326818,32sleep318405-21load19:18:383
171122686,54sleep315501-21kworker/3:119:13:383
1507826818,26sleep20-21swapper/219:08:072
634926712,39sleep36358-21df_inode22:07:513
2466026711,27sleep30-21swapper/319:37:573
226232677,45sleep30-21swapper/319:32:443
919526627,22sleep20-21swapper/223:57:502
919526627,22sleep20-21swapper/223:57:502
396426615,37sleep33988-21latency_hist20:17:303
338426614,31sleep20-21swapper/221:57:522
338426614,31sleep20-21swapper/221:57:522
253482667,19sleep00-21swapper/019:41:050
2013626612,25sleep20-21swapper/221:07:572
1964526620,29sleep20-21swapper/200:32:272
147026616,31sleep01534-21ps23:32:560
1386926628,19sleep00-21swapper/019:07:290
1275326613,25sleep00-21swapper/020:43:040
2974826514,22sleep00-21swapper/021:37:510
1929826511,16sleep00-21swapper/000:28:010
1496526515,21sleep20-21swapper/222:33:022
105552657,13sleep00-21swapper/022:21:040
3242326418,30sleep232424-21munin-node21:47:392
2054226423,19sleep20-21swapper/222:52:522
1205126413,18sleep30-21swapper/320:42:503
881226316,26sleep20-21swapper/223:57:302
323392639,13sleep30-21swapper/320:02:533
2212126314,29sleep00-21swapper/022:57:510
2102026317,29sleep021040-21basename21:12:270
1845626319,28sleep318462-21df_inode00:27:433
738226220,24sleep30-21swapper/320:27:483
3090126217,28sleep030895-21apt-get21:42:440
2680126212,22sleep30-21swapper/321:27:573
2653626219,20sleep30-21swapper/323:12:323
1911526214,30sleep218180-21/usr/sbin/munin21:03:052
1911526214,30sleep218180-21/usr/sbin/munin21:03:052
890926118,26sleep28918-21/usr/sbin/munin20:32:442
790626119,24sleep20-21swapper/222:12:462
2480426119,23sleep20-21swapper/219:38:002
2203226119,29sleep222082-21basename22:57:492
1342426114,29sleep213458-21munin-run00:12:282
1342426114,29sleep213458-21munin-run00:12:282
1133226122,19sleep20-21swapper/200:02:592
1133226122,19sleep20-21swapper/200:02:592
88602607,40sleep00-21swapper/022:14:300
3261526014,24sleep20-21swapper/220:03:002
325826019,22sleep00-21swapper/020:12:590
2802126012,33sleep328044-21irqstats21:32:503
2761526014,26sleep227667-21users23:13:202
2270026013,20sleep20-21swapper/221:17:302
161252606,46sleep20-21swapper/220:54:532
153226010,12sleep20-21swapper/220:07:562
790725916,18sleep20-21swapper/220:27:592
1660125913,29sleep216705-21latency_hist00:22:292
1229825913,30sleep012444-21latency_hist22:27:300
512225811,35sleep25166-21ntpq22:02:532
2737225818,26sleep027470-21sed23:13:180
207682586,39sleep20-21swapper/219:27:132
1961025813,27sleep019622-21/usr/sbin/munin19:22:470
1711025815,27sleep317200-21/usr/sbin/munin00:22:513
1711025815,27sleep317200-21/usr/sbin/munin00:22:513
1326525812,32sleep213275-21unin-run20:47:302
89025720,21sleep00-21swapper/021:48:010
803725717,19sleep30-21swapper/323:52:583
62162576,30sleep00-21swapper/020:23:010
448625718,20sleep30-21swapper/322:02:303
448625718,20sleep30-21swapper/322:02:303
2420425718,20sleep00-21swapper/021:22:290
1225625716,25sleep312242-21latency_hist22:27:293
46962568,11sleep30-21swapper/322:02:383
319062568,13sleep00-21swapper/023:27:430
2929425615,21sleep30-21swapper/319:52:553
2842725614,22sleep00-21swapper/021:32:590
2340325615,24sleep223400-21apt-get23:02:412
2340325615,24sleep223400-21apt-get23:02:412
3108325518,22sleep231135-21cat21:42:502
30062558,12sleep20-21swapper/220:12:532
1684125519,17sleep00-21swapper/000:22:440
1684125519,17sleep00-21swapper/000:22:440
1432525513,23sleep314419-21memory22:32:503
1103825517,22sleep211120-21cut22:22:482
84772547,35sleep00-21swapper/020:31:290
654825417,18sleep00-21swapper/022:07:560
420325412,24sleep04197-21apt-get20:17:440
2689425415,17sleep00-21swapper/021:27:580
2323325414,21sleep00-21swapper/019:33:010
161425415,23sleep31620-21df_inode21:52:443
161425415,23sleep31620-21df_inode21:52:443
148032545,37sleep00-21swapper/000:15:120
1101825416,23sleep011053-21ntp_states00:02:530
1101825416,23sleep011053-21ntp_states00:02:530
587925312,22sleep30-21swapper/323:47:373
587925312,22sleep30-21swapper/323:47:373
2299425318,19sleep20-21swapper/219:32:552
2128925313,24sleep221307-21apt-get21:12:412
949325214,22sleep39497-21diskstats22:17:523
177772528,31sleep217855-21find22:42:592
1420425213,22sleep014198-21apt-get19:07:490
3235725114,22sleep332363-21ntp_states23:27:553
257642517,34sleep00-21swapper/021:25:110
2338025114,20sleep00-21swapper/023:02:390
2338025114,20sleep00-21swapper/023:02:390
2316525113,14sleep30-21swapper/323:02:303
191225110,20sleep20-21swapper/221:52:522
191225110,20sleep20-21swapper/221:52:522
1612225112,18sleep30-21swapper/322:37:553
386125010,23sleep30-21swapper/323:42:283
386125010,23sleep30-21swapper/323:42:283
2027925012,14sleep00-21swapper/022:52:460
107125012,23sleep01065-21apt-get20:07:420
1057925010,19sleep20-21swapper/220:37:492
286652498,14sleep00-21swapper/023:17:540
2604124912,17sleep20-21swapper/221:27:322
1677324913,20sleep30-21swapper/320:57:483
1391924913,19sleep30-21swapper/320:47:553
3113624810,26sleep330877-21/usr/sbin/munin21:42:513
2115424710,21sleep021163-21diskstats19:27:480
2782924511,20sleep027898-21sensors_temp19:47:590
196752457,27sleep30-21swapper/322:48:293
196752457,27sleep30-21swapper/322:48:293
1083024512,19sleep310867-21ntp_states20:37:543
211162448,14sleep30-21swapper/319:27:473
42012439,16sleep30-21swapper/320:17:433
130452438,13sleep00-21swapper/022:27:590
2151924014,14sleep30-21swapper/321:12:513
2151924014,14sleep30-21swapper/321:12:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional