You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 17:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri May 08, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
136672733916,7310sleep24215-21kworker/2:110:33:012
151342119816,1166sleep215108-21/usr/sbin/munin08:54:492
759828726,852sleep023678-21kworker/u8:108:30:080
1478127987,778sleep314825-21ntpq10:35:403
1478127987,778sleep314825-21ntpq10:35:403
12815267019,634sleep112883-21/usr/sbin/munin08:45:201
3047426577,640sleep023967-1kworker/0:0H08:00:120
2896258813,561sleep02961-21unin-run09:59:480
9242255024,510sleep29275-21/usr/sbin/munin12:00:132
58682426117,184sleep30-21swapper/311:50:093
58682426117,184sleep30-21swapper/311:50:093
27123237718,311sleep327157-21fschecks_count11:15:053
4934232440,216sleep35083-21tail08:20:183
25218231117,103sleep30-21swapper/309:25:153
25218231117,103sleep30-21swapper/309:25:153
820229720,261sleep2850-21awk08:09:472
19407226899,131sleep319467-21fschecks_time07:25:063
19407226899,131sleep319467-21fschecks_time07:25:063
12063226314,150sleep312143-21fschecks_time12:10:033
12063226314,150sleep312143-21fschecks_time12:10:033
30446224113,208sleep230515-21sort11:25:092
30737222015,186sleep330830-21latency_hist09:44:493
4510221998,23sleep30-21swapper/310:04:483
4510221998,23sleep30-21swapper/310:04:483
10551221834,166sleep010595-21fschecks_time12:05:080
10551221834,166sleep010595-21fschecks_time12:05:080
2080221015,176sleep22113-21basename11:39:472
2080221015,176sleep22113-21basename11:39:472
20016220525,142sleep320135-21head10:50:183
8316220412,147sleep38349-21/usr/sbin/munin10:15:143
15742220386,28sleep30-21swapper/308:55:113
6070219917,80sleep36153-21latency_hist10:09:483
15803219340,115sleep30-21swapper/307:14:533
15803219340,115sleep30-21swapper/307:14:533
1777021899,167sleep317795-21cpu07:20:023
1777021899,167sleep317795-21cpu07:20:023
28401218811,134sleep328470-21switchtime09:35:163
1897217717,109sleep30-21swapper/309:55:113
1618021769,52sleep30-21swapper/310:40:173
25252217316,141sleep025370-21latency_hist11:09:490
1949521659,144sleep019555-21cat07:25:070
1949521659,144sleep019555-21cat07:25:070
20171216175,69sleep30-21swapper/312:35:113
20171216175,69sleep30-21swapper/312:35:113
2756021609,112sleep327624-21ps07:50:143
2756021609,112sleep327624-21ps07:50:143
5504215987,48sleep30-21swapper/311:49:493
14554215945,92sleep314699-21grep08:50:183
4527215810,132sleep04678-21cat10:04:490
4527215810,132sleep04678-21cat10:04:490
6118215121,108sleep00-21swapper/011:50:140
6118215121,108sleep00-21swapper/011:50:140
31433215140,90sleep313999-21cyclictest11:29:463
17679214646,65sleep30-21swapper/310:44:573
22965214392,20sleep30-21swapper/309:19:503
29105214210,115sleep329147-21uname07:55:153
27267214018,15sleep30-21swapper/309:31:003
1266921396,12sleep00-21swapper/010:30:080
323521388,84sleep30-21swapper/308:15:173
323521388,84sleep30-21swapper/308:15:173
8551213448,67sleep38612-21modprobe11:59:463
2954213211,20sleep30-21swapper/311:40:413
18225213242,16sleep30-21swapper/312:30:023
31722213118,92sleep30-21swapper/308:04:483
25815213037,51sleep30-21swapper/307:45:093
7054212838,15sleep30-21swapper/308:29:473
3222421276,11sleep30-21swapper/308:05:083
9539212511,18sleep30-21swapper/310:20:063
937221255,12sleep30-21swapper/312:00:193
16673212532,17sleep30-21swapper/312:25:013
31811212214,85sleep131897-21fschecks_time11:30:051
22410212110,23sleep30-21swapper/307:35:003
2647421175,11sleep30-21swapper/311:11:133
17791211323,68sleep217680-21/usr/sbin/munin10:45:042
7555211220,75sleep17575-21/usr/sbin/munin08:29:571
2658921127,96sleep00-21swapper/011:12:440
2632211223,69sleep02637-21find08:14:580
2632211223,69sleep02637-21find08:14:580
6307211112,82sleep30-21swapper/310:10:013
20342211043,54sleep320400-21sed09:10:093
1178621085,45sleep30-21swapper/308:42:233
1476021075,42sleep30-21swapper/312:19:123
13235210714,79sleep113243-21modprobe12:14:461
13235210714,79sleep113243-21modprobe12:14:461
29936210611,79sleep330075-21latency_hist11:24:503
26799210617,72sleep1309-21rs:main1
2090821060,26sleep120912-21unixbench_multi09:10:181
1532210630,45sleep30-21swapper/311:35:193
1532210630,45sleep30-21swapper/311:35:193
12734210635,46sleep112830-21fschecks_time07:05:061
1146121066,44sleep30-21swapper/312:06:533
1146121066,44sleep30-21swapper/312:06:533
5555210516,75sleep15597-21cut08:24:481
24348210542,30sleep3363-21snmpd09:23:173
32288210446,24sleep332414-21wc09:49:503
22124210418,71sleep322171-21systemd-run07:34:473
22124210418,71sleep322171-21systemd-run07:34:473
2629210327,46sleep22627-21apt-get11:39:592
22145210316,70sleep122269-21cat10:59:491
9125210221,56sleep09174-21awk10:19:470
6409210114,72sleep26506-21sed08:25:182
29500210115,61sleep229399-21/usr/sbin/munin09:40:042
9750210014,71sleep09861-21sed08:35:170
524210031,19sleep30-21swapper/311:34:473
15538210043,22sleep315591-21latency_hist07:14:493
3177429912,70sleep131916-21latency_hist08:04:501
239362997,19sleep323932-21apt-get11:05:003
223782998,16sleep30-21swapper/310:59:593
152042987,14sleep30-21swapper/312:20:053
3138629719,63sleep131431-21ntp_states09:45:121
239912977,14sleep30-21swapper/307:40:003
986229623,50sleep110005-21sed08:35:181
228772968,12sleep20-21swapper/207:35:152
808029414,61sleep37558-21/usr/sbin/munin08:30:363
2630329411,71sleep126328-21cpu09:30:011
1254029439,19sleep312538-21apt-get10:29:593
3005229322,49sleep130060-21apt-key07:59:491
260829319,61sleep12100-21apt-get11:39:581
1134629315,63sleep011475-21cat08:40:180
11112927,46sleep30-21swapper/308:10:003
172002916,46sleep30-21swapper/309:00:073
1420429024,31sleep00-21swapper/007:09:570
3252828910,45sleep30-21swapper/309:50:023
757628726,37sleep27577-21apt-get08:29:582
93342860,7sleep10-21swapper/112:00:161
187462867,46sleep30-21swapper/309:05:083
169628612,53sleep20-21swapper/209:55:062
1675728512,55sleep216769-21/usr/sbin/munin12:25:042
1628128522,47sleep116387-21tune2fs07:15:071
1628128522,47sleep116387-21tune2fs07:15:071
1380828511,55sleep00-21swapper/008:49:590
2638428414,52sleep026389-21diskstats09:30:040
1120028414,54sleep211267-21systemctl08:40:162
220282835,39sleep30-21swapper/310:59:183
3249728214,49sleep232518-21apt-get09:49:572
3172728213,55sleep231797-21df11:30:022
2634028212,51sleep226393-21users11:10:192
103472825,44sleep30-21swapper/308:39:123
1824628117,26sleep00-21swapper/007:20:150
1824628117,26sleep00-21swapper/007:20:150
67728018,39sleep132498-21/usr/sbin/munin09:50:161
491628012,23sleep00-21swapper/011:45:300
270342798,53sleep10-21swapper/111:14:591
217927928,32sleep0363-21snmpd11:39:490
217927928,32sleep0363-21snmpd11:39:490
142392790,6sleep10-21swapper/112:15:171
119502795,42sleep30-21swapper/310:25:513
119502795,42sleep30-21swapper/310:25:513
1948427817,45sleep0225-21systemd-journal12:34:470
1304127721,41sleep213086-21ntpq07:05:112
2676727612,16sleep20-21swapper/209:30:152
133152765,38sleep30-21swapper/308:47:073
667727514,45sleep06704-21netstat10:10:100
508627512,47sleep05155-21/usr/sbin/munin08:20:190
2923127514,20sleep20-21swapper/211:20:172
1235127514,44sleep212354-21/usr/sbin/munin08:45:082
2793427418,39sleep027937-21sshd09:35:050
1638327414,40sleep016409-21munin-run12:24:460
1590227420,22sleep00-21swapper/007:14:540
1590227420,22sleep00-21swapper/007:14:540
1431127416,35sleep113818-21apt-get10:34:581
1431127416,35sleep113818-21apt-get10:34:581
1837727313,45sleep218473-21timerandwakeup07:20:172
1837727313,45sleep218473-21timerandwakeup07:20:172
1070527315,43sleep01-21systemd10:24:470
2957327213,46sleep0363-21snmpd07:55:360
2053327213,42sleep020534-21ntp_states09:10:140
1776927213,49sleep00-21swapper/012:27:410
206312719,48sleep120737-21latency_hist07:29:491
206312719,48sleep120737-21latency_hist07:29:491
1604927110,30sleep20-21swapper/208:55:182
1264127124,21sleep00-21swapper/008:45:150
27302709,16sleep20-21swapper/208:15:042
27302709,16sleep20-21swapper/208:15:042
1474927015,41sleep214312-21/usr/sbin/munin10:35:392
1474927015,41sleep214312-21/usr/sbin/munin10:35:392
914326915,27sleep00-21swapper/012:00:110
794126917,29sleep20-21swapper/210:15:042
704226911,44sleep10-21swapper/111:54:471
704226911,44sleep10-21swapper/111:54:471
3017226918,29sleep030295-21grep09:40:180
2988526917,34sleep0225-21systemd-journal11:24:470
244826918,24sleep10-21swapper/108:14:511
122726912,24sleep10-21swapper/111:35:131
122726912,24sleep10-21swapper/111:35:131
2872526817,30sleep20-21swapper/207:55:052
2617326822,27sleep126339-21sed11:10:181
201626815,39sleep12054-21/usr/sbin/munin09:55:131
1576526821,29sleep20-21swapper/207:14:522
1576526821,29sleep20-21swapper/207:14:522
1576226814,36sleep015908-21latency_hist10:39:490
1576226814,36sleep015908-21latency_hist10:39:490
27812679,11sleep10-21swapper/108:15:071
27812679,11sleep10-21swapper/108:15:071
2709026619,29sleep227091-21sshd07:50:012
2709026619,29sleep227091-21sshd07:50:012
2389126618,29sleep00-21swapper/009:20:190
2375326614,27sleep00-21swapper/007:39:490
171462666,51sleep00-21swapper/010:41:150
1051326616,33sleep110525-21/usr/sbin/munin12:05:051
1051326616,33sleep110525-21/usr/sbin/munin12:05:051
985226521,22sleep10-21swapper/110:20:121
443626518,23sleep20-21swapper/211:45:122
1952726520,26sleep10-21swapper/112:34:481
1868426520,27sleep20-21swapper/209:05:072
1759526515,27sleep20-21swapper/209:00:192
1197126515,31sleep111985-21idleruntime-cro08:44:471
819726414,35sleep28200-21vmstat11:55:212
468526419,19sleep10-21swapper/111:45:191
2524926412,37sleep125260-21modprobe07:44:471
2248026415,33sleep222589-21perl11:00:062
1992626415,32sleep220002-21timerandwakeup07:25:182
1992626415,32sleep220002-21timerandwakeup07:25:182
531826317,28sleep05385-21switchtime10:05:170
4712638,16sleep00-21swapper/009:50:110
2606526322,18sleep20-21swapper/209:29:502
2606526322,18sleep20-21swapper/209:29:502
1987126311,36sleep020017-21latency_hist09:09:490
1896726314,31sleep018993-21munin-run10:49:460
1848126312,36sleep118500-21apt-get09:04:581
1587026317,20sleep00-21swapper/008:55:160
1463026314,22sleep20-21swapper/207:10:112
989826211,36sleep29904-21ntp_states10:20:132
512826212,19sleep20-21swapper/210:05:112
274972628,12sleep00-21swapper/007:50:110
274972628,12sleep00-21swapper/007:50:110
2414326220,23sleep20-21swapper/207:40:072
2051726217,21sleep00-21swapper/010:54:470
1694926211,12sleep10-21swapper/109:00:011
1533526215,28sleep20-21swapper/212:20:082
842626116,22sleep10-21swapper/110:15:161
651726113,23sleep10-21swapper/110:10:071
326382615,12sleep00-21swapper/011:30:200
288912618,10sleep10-21swapper/109:37:001
2502226120,23sleep10-21swapper/109:25:091
2502226120,23sleep10-21swapper/109:25:091
2114326117,22sleep10-21swapper/107:30:091
2114326117,22sleep10-21swapper/107:30:091
1934326118,28sleep219350-21forks10:50:052
1126326119,20sleep00-21swapper/010:25:080
1126326119,20sleep00-21swapper/010:25:080
109752616,13sleep20-21swapper/212:05:182
109752616,13sleep20-21swapper/212:05:182
2684526014,29sleep226877-21latency_hist07:49:482
2162326012,32sleep121691-21df09:15:031
1906926020,18sleep10-21swapper/107:24:481
1906926020,18sleep10-21swapper/107:24:481
1670226023,17sleep10-21swapper/108:59:491
1395426019,21sleep30-21swapper/307:09:483
1335226022,22sleep013502-21sed07:05:170
1108026012,32sleep111124-21grep10:25:051
1108026012,32sleep111124-21grep10:25:051
1039326014,27sleep210419-21munin-run08:39:472
559825912,26sleep25744-21latency_hist08:24:492
3112225918,25sleep00-21swapper/009:45:060
2123325919,19sleep20-21swapper/207:30:102
2123325919,19sleep20-21swapper/207:30:102
87325820,19sleep00-21swapper/008:09:490
601925818,19sleep10-21swapper/110:09:461
2942025816,24sleep129417-21apt-get09:39:591
2451525818,19sleep20-21swapper/209:24:492
217832588,12sleep00-21swapper/007:30:200
217832588,12sleep00-21swapper/007:30:200
2022925817,20sleep10-21swapper/112:35:131
2022925817,20sleep10-21swapper/112:35:131
1352625816,26sleep013518-21apt-get12:15:010
1112525816,26sleep211203-21fw_packets10:25:062
1112525816,26sleep211203-21fw_packets10:25:062
638725715,25sleep26393-21diskstats10:10:042
2338625711,21sleep10-21swapper/109:20:081
2056425711,26sleep10-21swapper/110:54:481
1711525715,26sleep117126-21ntp_states12:25:111
617125614,19sleep00-21swapper/008:25:120
398825613,19sleep00-21swapper/011:44:490
2394725610,32sleep124015-21df11:05:061
2004225615,19sleep00-21swapper/012:35:080
2004225615,19sleep00-21swapper/012:35:080
1608325610,18sleep20-21swapper/210:39:512
1389725611,26sleep213944-21ntpq12:15:112
2833525516,23sleep20-21swapper/209:35:152
414825421,19sleep20-21swapper/210:00:472
414825421,19sleep20-21swapper/210:00:472
2740725316,20sleep00-21swapper/011:15:100
2703525311,27sleep227031-21apt-get11:15:002
2399325313,24sleep023987-21apt-get07:40:010
1426725314,22sleep114304-21df07:10:021
766325215,20sleep10-21swapper/111:55:121
304482527,24sleep20-21swapper/207:59:582
2369925212,27sleep023676-21latency_hist11:04:480
2162025213,23sleep021612-21apt-get09:15:020
1770325214,22sleep117698-21apt-get10:44:591
1380925214,23sleep113803-21apt-get08:50:001
1246125217,18sleep00-21swapper/012:10:130
1246125217,18sleep00-21swapper/012:10:130
796425112,26sleep08085-21sed11:55:190
3148625116,21sleep131580-21sed11:29:481
2708925114,21sleep127084-21apt-get07:50:001
2708925114,21sleep127084-21apt-get07:50:001
2482525113,23sleep024831-21df_inode09:25:040
2482525113,23sleep024831-21df_inode09:25:040
1826825112,27sleep018364-21sed09:04:490
115525114,22sleep21188-21df08:10:032
718725017,18sleep17075-21latency_hist08:29:491
574925014,22sleep25819-21df11:50:042
574925014,22sleep25819-21df11:50:042
3247125014,21sleep132477-21ntp_states08:05:141
2238125012,22sleep122375-21apt-get11:00:011
366224910,18sleep10-21swapper/110:00:131
366224910,18sleep10-21swapper/110:00:131
3124724915,15sleep20-21swapper/209:45:082
2213024912,22sleep022144-21munin-run10:59:480
210472496,31sleep20-21swapper/209:11:092
202382499,17sleep20-21swapper/212:35:142
202382499,17sleep20-21swapper/212:35:142
1868424910,19sleep20-21swapper/212:30:132
1591024914,21sleep116082-21cmp10:39:501
11132499,14sleep00-21swapper/008:10:010
2869824814,18sleep10-21swapper/111:20:041
2554024813,19sleep00-21swapper/007:45:020
2287024812,20sleep122876-21ntp_states07:35:141
2123824710,21sleep221260-21ntp_states10:55:142
1194624712,21sleep20-21swapper/212:09:542
1194624712,21sleep20-21swapper/212:09:542
2212724613,17sleep20-21swapper/209:15:172
1293924613,15sleep10-21swapper/110:30:131
2434524512,17sleep20-21swapper/211:05:152
314652439,21sleep031485-21munin-run11:29:470
151262439,15sleep10-21swapper/112:20:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional