You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 08:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Aug 20, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12858262017,586sleep112880-21/usr/sbin/munin20:09:581
2609323648,342sleep026164-21/usr/sbin/munin00:19:300
2609323648,342sleep026164-21/usr/sbin/munin00:19:300
51712353167,172sleep35181-21cpu21:29:463
283922707,250sleep32902-21latency_hist19:39:323
17799224542,28sleep30-21swapper/322:09:513
28867224115,142sleep328958-21fschecks_time22:44:513
28201223139,177sleep328351-21sed19:15:013
28201223139,177sleep328351-21sed19:15:013
10088222921,193sleep310137-21latency_hist23:29:313
11961222754,124sleep312048-21fschecks_time23:34:483
12403222115,191sleep112478-21sensors_temp23:34:581
2561322015,161sleep325624-21ntpq20:50:463
758219814,67sleep30-21swapper/322:59:323
758219814,67sleep30-21swapper/322:59:323
21363217764,25sleep30-21swapper/322:20:003
21363217764,25sleep30-21swapper/322:20:003
16152217413,148sleep216164-21fschecks_count22:04:492
6641217393,20sleep30-21swapper/319:49:583
6641217393,20sleep30-21swapper/319:49:583
6017216861,36sleep35622-21/usr/sbin/munin23:14:543
29462216777,74sleep329493-21fschecks_count19:20:393
2446821677,95sleep324600-21timerwakeupswit22:30:013
1735021679,92sleep317421-21latency20:24:543
23667216637,22sleep323274-21/usr/sbin/munin20:44:533
20517216256,24sleep30-21swapper/322:19:313
20517216256,24sleep30-21swapper/322:19:313
18577215862,80sleep30-21swapper/323:54:593
16069215551,39sleep316135-21sort22:04:463
32707214339,89sleep3403-21latency_hist21:14:323
2841921439,120sleep328489-21basename20:59:513
26009214223,102sleep00-21swapper/019:10:060
4388213942,84sleep34450-21latency_hist19:44:333
1794213813,108sleep11897-21uptime23:00:021
1794213813,108sleep11897-21uptime23:00:021
4718213740,82sleep34824-21grep19:44:493
23299213522,40sleep30-21swapper/300:09:523
10730213433,19sleep30-21swapper/321:45:003
11637213361,56sleep30-21swapper/323:34:323
939212937,19sleep30-21swapper/321:14:563
31687212817,95sleep031807-21sort00:35:000
31687212817,95sleep031807-21sort00:35:000
31604212532,56sleep331683-21sed00:34:593
31604212532,56sleep331683-21sed00:34:593
25176212422,75sleep225332-21latency_hist22:34:322
23063212429,16sleep30-21swapper/320:44:323
9537212213,93sleep09538-21sshd21:44:200
21728212237,38sleep00-21swapper/020:39:410
20257211934,35sleep30-21swapper/320:34:473
30183211819,19sleep30-21swapper/300:30:023
30183211819,19sleep30-21swapper/300:30:023
21240211838,20sleep321314-21latency_hist00:04:313
1044021185,37sleep30-21swapper/320:02:583
2479121169,86sleep324803-21apt-get00:14:403
2479121169,86sleep324803-21apt-get00:14:403
3115221157,71sleep331241-21ls21:09:323
5675211313,83sleep15752-21systemctl21:30:001
16256211323,74sleep316278-21cut23:49:303
12881211318,79sleep212942-21/usr/sbin/munin20:09:592
8322211136,18sleep30-21swapper/319:54:583
30873210935,18sleep30-21swapper/322:49:593
18845210917,74sleep218990-21uptime23:55:022
22978210622,55sleep223144-21unixbench_multi22:24:592
27959210514,71sleep028019-21/usr/sbin/munin19:14:580
27959210514,71sleep028019-21/usr/sbin/munin19:14:580
24561210210,78sleep024589-21basename20:49:300
23932210211,32sleep323951-21apt-get19:04:523
1646521025,13sleep30-21swapper/323:49:343
1646521025,13sleep30-21swapper/323:49:343
2891721015,12sleep30-21swapper/300:25:383
2891721015,12sleep30-21swapper/300:25:383
1571210140,15sleep30-21swapper/323:00:003
1571210140,15sleep30-21swapper/323:00:003
1261210129,16sleep30-21swapper/319:34:313
31410210014,71sleep131424-21needreboot00:34:531
31410210014,71sleep131424-21needreboot00:34:531
22010210016,68sleep022023-21sh22:24:290
22010210016,68sleep022023-21sh22:24:290
499029917,57sleep25134-21modprobe21:29:332
2873929918,60sleep128255-21apt-get22:44:401
2603729926,22sleep30-21swapper/322:35:023
1042229813,61sleep210508-21fschecks_time23:29:472
2382729729,36sleep00-21swapper/022:29:420
2562629629,14sleep30-21swapper/319:09:573
175729623,55sleep01791-21idleruntime-cro21:19:300
905029513,68sleep09144-21/usr/sbin/munin23:24:520
145922955,12sleep30-21swapper/323:43:003
95052945,11sleep30-21swapper/321:43:553
2959229417,63sleep029628-21latency_hist21:04:320
2773629417,59sleep10-21swapper/100:24:311
2773629417,59sleep10-21swapper/100:24:311
2683729418,56sleep226936-21latency_hist19:14:322
2154829415,58sleep021458-21latency_hist20:39:330
838929314,66sleep18465-21cut21:39:491
750129315,61sleep17449-21latency_hist19:54:321
750129315,61sleep17449-21latency_hist19:54:321
281322927,74sleep3367-21snmpd22:43:053
1437429230,15sleep30-21swapper/320:14:573
2256729015,60sleep122608-21latency22:24:501
199392909,29sleep10-21swapper/123:59:571
625728918,50sleep26290-21grep19:49:492
625728918,50sleep26290-21grep19:49:492
591928913,59sleep16008-21load23:14:521
1887628920,44sleep218905-21munin-run22:14:302
1460328910,30sleep30-21swapper/321:59:493
2483828714,53sleep124803-21apt-get00:15:031
2483828714,53sleep124803-21apt-get00:15:031
85628618,53sleep10-21swapper/121:14:541
412128615,32sleep20-21swapper/221:25:012
309532868,38sleep30-21swapper/319:24:503
271572867,41sleep327191-21ps20:54:583
691628521,42sleep26928-21modprobe23:19:292
513128529,29sleep00-21swapper/019:44:580
2771128516,21sleep00-21swapper/000:24:300
2771128516,21sleep00-21swapper/000:24:300
36322846,67sleep30-21swapper/323:07:113
276692847,16sleep30-21swapper/300:24:293
276692847,16sleep30-21swapper/300:24:293
183728414,55sleep21930-21latency_hist21:19:322
155222836,32sleep30-21swapper/320:19:413
976228214,50sleep19272-21/usr/sbin/munin20:00:011
3011628215,50sleep030054-21latency_hist22:49:320
2707628212,49sleep026956-21/usr/sbin/munin22:39:470
1988928223,23sleep20-21swapper/220:34:302
923928114,51sleep19271-21/usr/sbin/munin23:24:571
764328110,35sleep30-21swapper/323:19:553
121582817,33sleep30-21swapper/320:09:323
1131028120,45sleep111352-21wc20:04:581
2850028024,26sleep10-21swapper/122:44:321
2803228026,25sleep00-21swapper/020:59:310
231452807,14sleep30-21swapper/322:25:003
235282796,18sleep00-21swapper/022:29:050
2055027915,49sleep220638-21switchtime00:00:352
1738927911,52sleep117539-21latency_hist22:09:321
927227812,25sleep20-21swapper/223:24:582
308292786,58sleep00-21swapper/021:06:100
20742787,36sleep30-21swapper/321:19:453
46482775,15sleep00-21swapper/021:26:280
2930027710,17sleep00-21swapper/019:19:430
2459027730,27sleep10-21swapper/120:49:311
1550227712,50sleep015594-21mailstats23:44:580
139992775,62sleep30-21swapper/321:56:593
36212769,15sleep30-21swapper/321:24:453
3063527624,24sleep20-21swapper/219:24:332
2789227618,29sleep10-21swapper/119:14:551
2789227618,29sleep10-21swapper/119:14:551
41427511,39sleep00-21swapper/019:29:560
2952827515,31sleep10-21swapper/119:20:451
2537727517,43sleep225434-21irqstats20:50:412
233427518,38sleep22488-21sed23:04:312
233427518,38sleep22488-21sed23:04:312
209622757,56sleep00-21swapper/000:01:340
1935327526,31sleep1289-21dbus-daemon23:59:291
530527419,32sleep25443-21unixbench_multi19:45:002
2456427429,28sleep10-21swapper/119:05:111
3147427326,25sleep00-21swapper/021:09:490
2809227314,37sleep228199-21timerandwakeup19:15:002
2809227314,37sleep228199-21timerandwakeup19:15:002
1553327311,48sleep0367-21snmpd22:01:200
543427216,39sleep25583-21sed23:14:332
408427225,25sleep10-21swapper/121:25:001
3094027212,39sleep230831-21/usr/sbin/munin19:24:492
2638327229,25sleep20-21swapper/200:19:402
2638327229,25sleep20-21swapper/200:19:402
2429727215,33sleep10-21swapper/100:14:301
1953527216,35sleep019697-21apt-config23:59:320
251022716,10sleep00-21swapper/020:49:540
2417327116,45sleep00-21swapper/000:13:040
877527020,23sleep20-21swapper/221:39:582
3108827020,24sleep10-21swapper/121:09:291
2927027019,29sleep00-21swapper/000:29:310
2927027019,29sleep00-21swapper/000:29:310
2206127014,34sleep222037-21latency_hist22:24:312
2206127014,34sleep222037-21latency_hist22:24:312
1899427013,27sleep10-21swapper/120:29:531
1147327021,29sleep00-21swapper/021:49:490
72272696,29sleep30-21swapper/321:35:033
3040626919,25sleep10-21swapper/122:49:491
2681426918,34sleep126836-21munin-run19:14:311
2424126916,39sleep124266-21/usr/sbin/munin22:29:551
2158026919,21sleep10-21swapper/100:04:501
1722026917,30sleep20-21swapper/223:50:012
1722026917,30sleep20-21swapper/223:50:012
128172697,43sleep30-21swapper/320:09:563
797426815,31sleep18124-21grep23:20:011
50602685,28sleep30-21swapper/323:10:383
322852680,6sleep10-21swapper/119:29:321
174252680,7sleep00-21swapper/020:24:540
1419526830,22sleep014264-21systemd-run21:59:300
3268826715,38sleep232705-21latency_hist21:14:312
3235326715,39sleep232419-21cat22:54:592
3235326715,39sleep232419-21cat22:54:592
226342675,28sleep30-21swapper/300:08:103
2197626725,22sleep20-21swapper/200:04:592
201792678,14sleep10-21swapper/120:34:451
1871226712,14sleep00-21swapper/020:29:480
1684026719,25sleep10-21swapper/120:24:321
387926618,33sleep24030-21ls23:09:322
274152666,12sleep20-21swapper/222:39:532
2463226623,22sleep20-21swapper/220:49:322
3249526514,37sleep00-21swapper/022:55:010
3249526514,37sleep00-21swapper/022:55:010
2696526526,24sleep10-21swapper/120:54:521
2143826519,28sleep21-21systemd20:39:312
1870326510,41sleep30-21swapper/320:29:473
1321226524,23sleep20-21swapper/223:39:312
124662655,28sleep30-21swapper/321:52:103
119022657,14sleep00-21swapper/020:06:500
465326416,35sleep04729-21sed23:09:580
2678126412,37sleep226817-21fschecks_count20:54:492
1627526413,20sleep00-21swapper/022:04:510
141792646,10sleep20-21swapper/221:59:202
3106526313,28sleep00-21swapper/019:24:510
3085426312,34sleep031005-21sed00:34:310
2415326314,33sleep024214-21if_eth019:05:020
1892826324,20sleep00-21swapper/022:14:320
1451126314,29sleep114505-21apt-get21:59:431
418426211,36sleep14194-21/usr/sbin/munin23:09:471
1576626221,23sleep20-21swapper/222:04:202
1293826218,27sleep112931-21apt-get21:54:471
363226121,18sleep10-21swapper/119:39:591
338426114,32sleep23480-21latency_hist21:24:322
2326426116,19sleep20-21swapper/200:09:512
2232726113,23sleep20-21swapper/220:39:592
186726114,31sleep11897-21/usr/sbin/munin19:34:541
1476026113,31sleep214902-21cat23:44:322
897726018,27sleep29039-21sendmail-msp19:59:302
797726019,26sleep28009-21munin-run21:39:302
2563326020,23sleep10-21swapper/120:50:481
200426011,31sleep20-21swapper/219:34:592
1415526014,19sleep20-21swapper/220:14:522
1115326024,21sleep00-21swapper/021:49:330
1065826015,23sleep00-21swapper/020:04:320
805725916,18sleep00-21swapper/021:39:320
62992597,15sleep00-21swapper/021:32:500
294822596,10sleep20-21swapper/200:29:412
294822596,10sleep20-21swapper/200:29:412
2441725917,26sleep20-21swapper/219:05:082
1353725910,25sleep00-21swapper/023:39:500
1228325917,25sleep20-21swapper/221:50:042
24972586,41sleep00-21swapper/019:36:060
2215325825,17sleep00-21swapper/000:05:010
1916625817,23sleep019160-21apt-get22:14:440
1916625817,23sleep019160-21apt-get22:14:440
1409125815,20sleep10-21swapper/120:14:511
638225714,22sleep00-21swapper/019:49:510
638225714,22sleep00-21swapper/019:49:510
2950625718,23sleep029502-21apt-get00:29:430
2549425724,18sleep00-21swapper/022:34:500
2214625721,18sleep10-21swapper/120:39:541
1365525720,23sleep0231-21systemd-journal20:14:290
723625612,19sleep20-21swapper/223:19:452
2889225613,30sleep128893-21sshd00:25:201
2889225613,30sleep128893-21sshd00:25:201
1953425614,19sleep10-21swapper/122:14:531
1953425614,19sleep10-21swapper/122:14:531
166625616,20sleep00-21swapper/023:00:010
166625616,20sleep00-21swapper/023:00:010
1662225616,25sleep116629-21diskstats23:49:481
1662225616,25sleep116629-21diskstats23:49:481
1590725615,28sleep00-21swapper/020:19:550
983725520,18sleep20-21swapper/221:44:432
39425515,23sleep2400-21ntp_states19:29:542
370525514,20sleep20-21swapper/219:40:002
356325514,27sleep03631-21mailstats19:39:580
32432557,14sleep00-21swapper/021:23:210
3239825514,24sleep132395-21apt-get19:29:411
3195725517,17sleep10-21swapper/122:54:501
3195725517,17sleep10-21swapper/122:54:501
2824825514,25sleep228267-21apt-get20:59:412
2674725514,21sleep20-21swapper/222:39:332
263025512,15sleep10-21swapper/123:04:471
205125513,22sleep10-21swapper/121:19:431
1224425511,26sleep012283-21ntpq23:34:530
1014425514,18sleep00-21swapper/023:29:320
2877225414,24sleep228766-21apt-get22:44:452
1761725418,21sleep117611-21apt-get22:09:451
1744925416,22sleep00-21swapper/023:50:030
1744925416,22sleep00-21swapper/023:50:030
1730025415,22sleep217349-21irqstats20:24:532
1359725412,22sleep10-21swapper/123:39:511
659825313,23sleep06599-21sshd23:15:320
3153825317,19sleep20-21swapper/200:34:582
3153825317,19sleep20-21swapper/200:34:582
2795825312,24sleep227952-21apt-get00:24:442
2795825312,24sleep227952-21apt-get00:24:442
1059725317,18sleep10-21swapper/123:29:501
856625212,19sleep10-21swapper/119:55:001
2994925213,21sleep129962-21ntp_states00:29:561
2994925213,21sleep129962-21ntp_states00:29:561
2872725213,22sleep128733-21ntp_states20:59:571
1707925212,24sleep117074-21apt-get20:24:441
1598325210,17sleep20-21swapper/220:19:572
1567625216,16sleep20-21swapper/223:45:002
1010525211,19sleep10-21swapper/121:44:501
929625111,23sleep29292-21apt-get19:59:462
3040125114,23sleep030405-21diskstats22:49:480
2492325115,23sleep224784-21/usr/sbin/munin00:15:122
2492325115,23sleep224784-21/usr/sbin/munin00:15:122
217442515,10sleep10-21swapper/122:21:031
217442515,10sleep10-21swapper/122:21:031
255125013,20sleep00-21swapper/023:04:420
1499525015,20sleep114990-21apt-get23:44:431
1136425013,20sleep111383-21apt-get21:49:421
462224912,21sleep14617-21apt-get19:44:431
3141024911,23sleep231411-21sshd22:52:202
2982624912,20sleep229823-21apt-get21:04:422
2541024914,15sleep10-21swapper/122:34:471
82642486,31sleep00-21swapper/023:21:330
804424710,14sleep00-21swapper/019:54:510
3174424712,15sleep10-21swapper/121:09:541
1391924612,18sleep00-21swapper/020:14:420
109192457,14sleep20-21swapper/220:04:472
740224311,16sleep10-21swapper/121:35:051
192362439,19sleep219245-21df_inode22:14:462
192362439,19sleep219245-21df_inode22:14:462
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional