You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 19:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Dec 11, 2025 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13366291117,876sleep013477-21/usr/sbin/munin10:48:020
13366291117,876sleep013477-21/usr/sbin/munin10:48:020
3116278635,736sleep03126-21/usr/sbin/munin10:17:330
32236245510,397sleep232296-21find08:24:152
13113242541,294sleep227999-1kworker/2:2H09:03:202
9580241719,383sleep39588-21/usr/sbin/munin10:37:433
9580241719,383sleep39588-21/usr/sbin/munin10:37:433
26332233040,242sleep226527-21cat08:07:342
24101232310,300sleep224165-21ps11:23:022
24101232310,300sleep224165-21ps11:23:022
18429230915,280sleep018464-21unin-run07:42:320
1397823070,20sleep25613-21cyclictest07:27:352
6520230611,272sleep26677-21tail07:03:052
389722688,220sleep24042-21latency_hist08:37:352
23657226110,234sleep223677-21kernelversion07:57:532
8473225947,192sleep28626-21unixbench_multi07:08:052
12271222179,70sleep212214-21latency_hist07:22:352
31858221135,139sleep231910-21if_eth011:47:542
11787221130,160sleep00-21swapper/012:27:540
15604221011,134sleep215728-21latency_hist10:57:332
29136220710,143sleep228527-21/usr/sbin/munin11:38:042
14694220328,114sleep214814-21timerwakeupswit07:28:052
14694220328,114sleep214814-21timerwakeupswit07:28:052
14041219815,92sleep20-21swapper/210:52:332
14041219815,92sleep20-21swapper/210:52:332
21550219724,149sleep00-21swapper/009:32:340
20485219510,152sleep220524-21expr09:27:562
26874219431,126sleep226877-21fschecks_count08:08:072
8738219113,92sleep28876-21unixbench_multi10:33:062
8738219113,92sleep28876-21unixbench_multi10:33:062
1873721878,140sleep218677-21latency_hist11:07:352
5914218212,154sleep15974-21/usr/sbin/munin12:08:001
23476218022,143sleep323488-21fschecks_count07:57:503
10198218012,101sleep210286-21threads10:38:032
10198218012,101sleep210286-21threads10:38:032
3750217813,143sleep13789-21ntpq10:18:071
26806217125,106sleep226817-21needreboot09:47:592
26806217125,106sleep226817-21needreboot09:47:592
15811216848,58sleep20-21swapper/207:32:532
15811216848,58sleep20-21swapper/207:32:532
25286216566,63sleep225323-21sed09:42:582
25286216566,63sleep225323-21sed09:42:582
1576216528,98sleep21711-21latency_hist10:12:342
1576216528,98sleep21711-21latency_hist10:12:342
2142821629,141sleep221441-21awk11:13:042
24025215710,105sleep224164-21tail09:38:052
961215215,61sleep20-21swapper/208:27:542
22399215213,125sleep122475-21timerandwakeup09:33:041
8715215118,112sleep18746-21fschecks_time12:17:571
12951215130,107sleep213024-21unin-run12:32:322
16629215046,18sleep20-21swapper/209:17:342
7815214911,56sleep20-21swapper/210:32:342
5388214610,101sleep25415-21munin-run08:42:322
5388214610,101sleep25415-21munin-run08:42:322
1736421459,49sleep20-21swapper/207:37:542
3849214326,105sleep23896-21processes10:18:122
28993214348,69sleep20-21swapper/208:13:082
21876214211,68sleep221932-21fschecks_count09:32:522
28401213821,32sleep00-21swapper/008:12:540
3141221348,87sleep231540-21ls11:47:332
10286213412,107sleep010295-21vmstat07:13:160
18733213128,91sleep218735-21cpu09:23:402
10376212720,90sleep110462-21latency_hist08:57:331
20667212611,97sleep25666-21snmpd07:48:002
19174212420,84sleep00-21swapper/007:43:010
19174212420,84sleep00-21swapper/007:43:010
23393212331,37sleep20-21swapper/211:22:332
31668212020,80sleep131649-21apt-get08:22:521
19305212025,22sleep20-21swapper/207:43:032
19305212025,22sleep20-21swapper/207:43:032
9887211811,93sleep09892-21systemd-run12:22:330
30501211715,84sleep330537-21ntpq11:42:583
10614211711,19sleep30-21swapper/308:57:483
10186211664,16sleep25666-21snmpd08:55:452
21953211526,75sleep222018-21grep07:52:522
1242521158,96sleep212466-21unin-run10:47:322
1242521158,96sleep212466-21unin-run10:47:322
26127211315,79sleep026266-21tail11:28:210
24315211219,75sleep124454-21tail11:23:051
24315211219,75sleep124454-21tail11:23:051
10694211228,70sleep20-21swapper/207:17:342
591121108,38sleep20-21swapper/208:42:442
7957210911,37sleep27970-21awk12:13:222
7957210911,37sleep27970-21awk12:13:222
16962210818,72sleep016997-21grep09:17:520
16962210818,72sleep016997-21grep09:17:520
2995721079,38sleep229982-21cut08:17:522
8258210621,68sleep38321-21cut07:08:023
455210530,52sleep3480-21munin-run11:52:313
23611210512,72sleep123315-21/usr/sbin/munin09:37:561
1516210422,60sleep01654-21unixbench_singl11:53:050
9679210124,58sleep39806-21grep08:53:033
29336210121,67sleep30-21swapper/309:57:333
25496210113,74sleep025550-21switchtime09:43:040
25496210113,74sleep025550-21switchtime09:43:040
490621005,11sleep20-21swapper/212:04:262
724229917,58sleep30-21swapper/308:47:343
2021329921,17sleep20-21swapper/211:12:322
2916629814,48sleep2910-21sendmail-mta09:55:592
206132987,17sleep00-21swapper/011:12:500
2701829716,68sleep127033-21kernelversion08:08:101
189329721,13sleep20-21swapper/210:12:512
1826229610,47sleep218198-21munin-run09:22:342
1826229610,47sleep218198-21munin-run09:22:342
115692965,11sleep20-21swapper/208:58:362
1434829514,65sleep314455-21cat09:08:063
295782949,19sleep20-21swapper/209:57:482
2404129416,62sleep324084-21grep11:22:593
2404129416,62sleep324084-21grep11:22:593
18242945,11sleep20-21swapper/211:54:402
1714329416,61sleep117164-21latency_hist11:02:331
689529314,63sleep36932-21ntp_states10:27:583
550729226,42sleep25537-21munin-run07:02:322
3160229236,45sleep231642-21uname10:03:052
2717729215,64sleep327233-21cat11:32:523
1560229225,41sleep015596-21apt-get07:32:470
1560229225,41sleep015596-21apt-get07:32:470
1710129017,52sleep317129-21basename11:02:313
255242897,72sleep25666-21snmpd11:28:032
171662888,20sleep217311-21latency_hist11:02:342
486728717,48sleep15010-21unixbench_singl08:38:111
486728717,48sleep15010-21unixbench_singl08:38:111
3181528715,46sleep131946-21grep10:03:081
1220328712,58sleep00-21swapper/007:22:330
2985428615,51sleep329866-21apt-get08:17:453
2205628516,40sleep122075-21apt-get11:17:441
85282845,67sleep20-21swapper/208:49:472
805628410,23sleep10-21swapper/110:32:491
805628410,23sleep10-21swapper/110:32:491
528928411,56sleep35334-21/usr/sbin/munin10:22:573
252028413,56sleep12548-21sshd11:57:541
438028311,59sleep05666-21snmpd10:19:190
1499928331,16sleep215033-21munin-run09:12:332
824528211,45sleep10-21swapper/107:08:001
67882827,18sleep26793-21munin-run12:12:332
2589928210,60sleep00-21swapper/008:03:370
222262820,7sleep30-21swapper/307:52:563
1115628213,46sleep311224-21/usr/sbin/munin10:42:483
1115628213,46sleep311224-21/usr/sbin/munin10:42:483
2700028121,14sleep20-21swapper/211:32:472
471928029,23sleep00-21swapper/006:58:020
2562128021,28sleep30-21swapper/308:03:033
592427922,35sleep10-21swapper/107:02:511
1139427910,32sleep20-21swapper/212:27:322
975927820,43sleep39785-21netstat07:13:073
785627815,36sleep10-21swapper/108:48:001
2489227838,20sleep00-21swapper/011:27:320
2489227838,20sleep00-21swapper/011:27:320
1115127812,52sleep111221-21sensors_temp08:58:021
3211427718,34sleep10-21swapper/111:48:011
2895727720,25sleep10-21swapper/111:37:591
25732779,16sleep10-21swapper/108:32:461
2400727724,33sleep024111-21timerandwakeup07:58:030
60527623,33sleep10-21swapper/110:07:571
60527623,33sleep10-21swapper/110:07:571
185162760,7sleep10-21swapper/109:22:361
185162760,7sleep10-21swapper/109:22:361
992827511,25sleep210073-21date12:22:352
719127522,30sleep10-21swapper/108:47:321
2184227524,24sleep30-21swapper/311:17:343
931527424,35sleep39324-21modprobe10:37:313
931527424,35sleep39324-21modprobe10:37:313
708527417,40sleep07181-21timerandwakeup10:28:030
3112827424,32sleep131142-21apt-get08:22:321
1998927413,45sleep120131-21cat09:27:341
1040327415,39sleep110458-21sed12:22:541
637327322,25sleep00-21swapper/007:03:020
3173627313,44sleep031748-21/usr/sbin/munin11:47:520
1483527322,30sleep30-21swapper/310:53:023
2815627216,42sleep328265-21latency_hist08:12:363
255827219,13sleep20-21swapper/211:57:552
1140927216,39sleep311424-21/usr/sbin/munin07:18:163
83527122,25sleep00-21swapper/010:08:040
83527122,25sleep00-21swapper/010:08:040
74792717,13sleep00-21swapper/007:07:390
656527119,27sleep10-21swapper/110:27:521
278822710,8sleep10-21swapper/109:52:331
278822710,8sleep10-21swapper/109:52:331
2757627121,28sleep10-21swapper/111:33:031
2592027117,26sleep10-21swapper/111:28:181
279127012,26sleep30-21swapper/311:58:003
2692127016,39sleep326984-21sendmail_mailqu09:48:043
2692127016,39sleep326984-21sendmail_mailqu09:48:043
254927011,43sleep32550-21dpkg08:32:443
533626911,43sleep05372-21ntp_states10:22:580
2859126915,39sleep128611-21cut08:12:591
906326817,24sleep30-21swapper/312:18:083
558226811,41sleep20-21swapper/210:23:052
465226833,19sleep10-21swapper/106:57:591
4412685,10sleep30-21swapper/308:27:103
2553026818,32sleep325545-21fschecks_count11:28:083
218826818,31sleep30-21swapper/310:13:003
1747726812,29sleep30-21swapper/311:02:523
1189026820,26sleep10-21swapper/112:27:551
543626710,42sleep05437-21http12:07:470
2084226718,28sleep30-21swapper/309:28:053
199622676,12sleep30-21swapper/307:47:323
199622676,12sleep30-21swapper/307:47:323
1657926718,27sleep00-21swapper/009:17:320
1586826718,28sleep30-21swapper/307:32:543
1586826718,28sleep30-21swapper/307:32:543
1489926713,18sleep00-21swapper/010:53:030
1382026720,28sleep00-21swapper/009:07:540
1278426720,24sleep30-21swapper/310:47:493
1278426720,24sleep30-21swapper/310:47:493
591126616,34sleep35917-21df_inode07:02:493
2879126620,29sleep30-21swapper/308:13:053
1572426612,22sleep10-21swapper/107:32:521
1572426612,22sleep10-21swapper/107:32:521
1233426622,20sleep30-21swapper/312:28:053
2309826518,22sleep10-21swapper/107:57:321
2050326521,26sleep120522-21apt-get11:12:431
589726421,22sleep00-21swapper/008:42:420
3110126412,16sleep00-21swapper/010:02:430
3038126425,20sleep00-21swapper/008:18:020
3018526415,27sleep10-21swapper/109:58:041
2994826414,32sleep329985-21sed09:57:573
1378426415,25sleep00-21swapper/007:27:340
1144826415,19sleep00-21swapper/012:27:340
1098626416,31sleep011020-21/usr/sbin/munin08:57:570
833826314,25sleep10-21swapper/112:17:331
833826314,25sleep10-21swapper/112:17:331
543126313,33sleep35493-21cat08:42:333
543126313,33sleep35493-21cat08:42:333
2620126323,18sleep10-21swapper/109:47:331
2620126323,18sleep10-21swapper/109:47:331
2160426321,22sleep00-21swapper/007:52:340
1932426320,22sleep00-21swapper/011:07:580
1866926317,25sleep00-21swapper/009:23:010
1508026320,20sleep10-21swapper/109:12:361
1099826317,26sleep10-21swapper/107:17:501
887326217,25sleep00-21swapper/008:52:360
2070026214,36sleep120768-21systemctl09:28:031
149432627,12sleep10-21swapper/107:28:081
149432627,12sleep10-21swapper/107:28:081
1332926213,32sleep312451-21/usr/sbin/munin07:23:133
857926113,19sleep00-21swapper/010:33:040
857926113,19sleep00-21swapper/010:33:040
76226113,19sleep10-21swapper/108:27:491
50232617,11sleep30-21swapper/312:05:563
2544926117,26sleep125525-21awk08:03:011
2073326113,28sleep120792-21/usr/sbin/munin07:48:031
2001426122,18sleep10-21swapper/107:47:341
2001426122,18sleep10-21swapper/107:47:341
1599426118,20sleep10-21swapper/109:13:041
475126013,28sleep00-21swapper/008:38:100
475126013,28sleep00-21swapper/008:38:100
301926011,24sleep00-21swapper/011:58:060
258126015,28sleep02575-21apt-get08:32:490
1840926016,29sleep318426-21ls07:42:313
1271526018,25sleep112786-21sensors_temp09:03:031
973025911,31sleep09816-21fw_packets10:37:530
973025911,31sleep09816-21fw_packets10:37:530
514725916,17sleep10-21swapper/112:07:311
3140225917,25sleep331433-21/usr/sbin/munin10:02:583
306432596,38sleep00-21swapper/010:00:000
2809425923,20sleep00-21swapper/009:52:500
2219625915,18sleep00-21swapper/009:32:590
1777325917,27sleep117815-21ntp_kernel_pll_11:02:581
17332595,15sleep00-21swapper/008:28:490
1678125915,29sleep0225-21systemd-journal10:58:240
967825811,31sleep19684-21df_inode10:37:501
967825811,31sleep19684-21df_inode10:37:501
824825814,25sleep30-21swapper/312:16:543
824825814,25sleep30-21swapper/312:16:543
2371025812,29sleep323754-21ntpq09:37:583
1903925817,25sleep319140-21fschecks_time11:07:523
179052586,12sleep30-21swapper/309:19:373
179052586,12sleep30-21swapper/309:19:373
1309225820,20sleep10-21swapper/107:23:111
1307525817,20sleep10-21swapper/110:47:551
1307525817,20sleep10-21swapper/110:47:551
829825713,23sleep08326-21munin-run12:17:320
829825713,23sleep08326-21munin-run12:17:320
2855225714,26sleep328547-21apt-get11:37:493
232992575,13sleep30-21swapper/311:21:583
1992325713,31sleep319964-21systemd-run09:27:323
1942425714,26sleep119425-21ntp_states11:08:151
12902576,41sleep30-21swapper/310:09:533
12902576,41sleep30-21swapper/310:09:533
1286525716,20sleep30-21swapper/309:03:053
1171025716,26sleep111774-21wc10:43:031
1171025716,26sleep111774-21wc10:43:031
549525620,18sleep10-21swapper/110:23:041
336725615,26sleep33362-21apt-get10:17:483
3007525613,23sleep10-21swapper/111:42:441
2699825614,25sleep026993-21apt-get11:32:450
77625514,25sleep3771-21apt-get11:52:473
75925515,24sleep3754-21apt-get08:27:473
2318625512,24sleep00-21swapper/007:57:350
2235325517,16sleep00-21swapper/011:17:550
1536225510,29sleep315504-21cat07:32:343
1536225510,29sleep315504-21cat07:32:343
2976025413,28sleep029762-21sshd11:41:540
2749625411,27sleep027497-21sshd09:49:550
2749625411,27sleep027497-21sshd09:49:550
1813725419,18sleep00-21swapper/011:03:050
1115325415,23sleep011147-21apt-get10:42:450
1115325415,23sleep011147-21apt-get10:42:450
794025314,22sleep37948-21ntp_states08:48:023
3237225312,19sleep00-21swapper/008:24:170
213525317,19sleep10-21swapper/111:57:341
1790225315,21sleep118045-21sed07:38:051
106042537,13sleep30-21swapper/312:23:003
543425213,22sleep35430-21apt-get12:07:463
2532425214,23sleep325343-21ntp_states09:42:593
2532425214,23sleep325343-21ntp_states09:42:593
208625216,20sleep10-21swapper/110:12:581
1668625212,17sleep30-21swapper/309:17:353
3010025112,23sleep030095-21apt-get11:42:470
2411225115,18sleep10-21swapper/107:58:041
2333825115,19sleep00-21swapper/009:37:470
1426225114,22sleep114276-21apt-get10:52:441
1409825112,21sleep014107-21df_inode07:27:500
1409825112,21sleep014107-21df_inode07:27:500
1236525015,16sleep00-21swapper/009:02:540
2726324814,17sleep00-21swapper/008:08:170
2179224811,22sleep321859-21df09:32:493
1067524815,15sleep10-21swapper/107:17:331
757224613,16sleep00-21swapper/008:47:550
2157924513,17sleep30-21swapper/307:52:333
429624310,14sleep30-21swapper/306:57:493
41572407,14sleep00-21swapper/012:02:540
282882406,15sleep10-21swapper/109:52:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional