You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 21:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Aug 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
107812125319,1218sleep110782-21ntp_states08:25:281
1157623859,358sleep011616-21unin-run08:29:510
26462318108,188sleep22809-21unixbench_multi09:40:212
258282303137,148sleep225891-21sendmail-msp12:39:492
35322286141,34sleep23681-21grep11:25:212
22544223517,204sleep322598-21sort10:45:053
9023222812,203sleep29095-21mailstats08:20:202
9023222812,203sleep29095-21mailstats08:20:202
15822221013,114sleep20-21swapper/212:05:222
21564220919,47sleep20-21swapper/212:25:122
40112206170,23sleep04013-21sshd11:29:220
6584219218,131sleep20-21swapper/209:55:022
6584219218,131sleep20-21swapper/209:55:022
3276218916,146sleep13401-21latency_hist09:44:521
24937218336,19sleep20-21swapper/207:29:542
14746218210,148sleep20-21swapper/210:20:092
17424217711,26sleep20-21swapper/208:45:262
1870521769,144sleep20-21swapper/212:15:152
204602174143,15sleep20-21swapper/212:20:192
3168721707,150sleep231782-21perl07:50:082
2867721698,147sleep228755-21basename09:20:182
13489216513,78sleep313513-21fschecks_count08:35:103
111332165124,23sleep20-21swapper/210:09:522
27075216414,122sleep20-21swapper/207:35:152
122652163119,22sleep20-21swapper/208:30:172
27081215713,129sleep226581-21/usr/sbin/munin09:15:182
26371215311,14sleep20-21swapper/209:14:532
1715221536,131sleep217196-21ntpq12:10:142
20544215011,117sleep20-21swapper/210:38:422
32029214884,49sleep232171-21head09:30:232
4119214618,106sleep20-21swapper/209:45:212
2417621469,124sleep224268-21tune2fs10:50:092
9502214410,119sleep29557-21latency_hist10:04:512
9502214410,119sleep29557-21latency_hist10:04:512
20544214211,118sleep220562-21netstat08:55:562
20544214211,118sleep220562-21netstat08:55:562
14052213919,104sleep014086-21awk12:00:150
8718213718,99sleep10-21swapper/111:44:501
793213689,32sleep2840-21ntpq09:35:162
335921365,13sleep20-21swapper/208:02:282
30356213610,103sleep230473-21timerandwakeup09:25:232
15893213521,87sleep315986-21sshd12:05:233
22102213415,15sleep20-21swapper/209:00:162
22102213415,15sleep20-21swapper/209:00:162
1148213499,19sleep20-21swapper/207:55:172
30597213318,22sleep20-21swapper/207:45:202
5520213218,97sleep218059-21cyclictest09:50:182
259121327,14sleep20-21swapper/211:24:522
3186521307,29sleep231917-21latency_hist11:14:512
24513213019,93sleep324557-21awk09:09:493
13387212915,25sleep013399-21apt-get08:35:030
1027121299,23sleep20-21swapper/208:25:002
4339212819,13sleep20-21swapper/211:30:052
23563212721,87sleep323077-21apt-get07:25:023
14283212482,28sleep214362-21find12:00:212
850521235,11sleep20-21swapper/211:42:042
2225221228,101sleep222258-21ntp_states07:20:172
18810212011,20sleep20-21swapper/208:50:192
18810212011,20sleep20-21swapper/208:50:192
16303212011,18sleep216390-21fschecks_time10:25:102
1320021205,10sleep20-21swapper/211:57:432
693121198,16sleep20-21swapper/208:15:072
509321198,99sleep25121-21munin-run08:09:502
20582211916,85sleep10-21swapper/108:55:591
20582211916,85sleep10-21swapper/108:55:591
6930211830,58sleep16924-21apt-get08:15:061
29301211813,17sleep20-21swapper/211:05:272
2640021188,18sleep226420-21munin-run07:34:502
19531211724,77sleep019535-21modprobe07:14:500
1419221175,27sleep20-21swapper/210:17:512
1419221175,27sleep20-21swapper/210:17:512
858211513,85sleep1923-21awk09:35:191
634021159,14sleep20-21swapper/211:35:182
1543921159,91sleep20-21swapper/208:40:182
1452921155,28sleep20-21swapper/208:38:082
1776221147,13sleep20-21swapper/210:30:022
25910211279,13sleep20-21swapper/210:55:132
25910211279,13sleep20-21swapper/210:55:132
1997521116,13sleep20-21swapper/207:14:562
1997521116,13sleep20-21swapper/207:14:562
19593211116,79sleep319679-21latency_hist07:14:533
786211012,85sleep3813-21cut07:55:103
538821097,13sleep20-21swapper/208:10:072
31621086,11sleep20-21swapper/211:15:202
29804210779,16sleep229814-21munin-run07:44:512
2840721055,11sleep20-21swapper/211:03:532
2840721055,11sleep20-21swapper/211:03:532
1164021055,11sleep20-21swapper/211:52:342
4232210417,63sleep04265-21ntp_kernel_pll_08:05:160
2439821046,10sleep20-21swapper/209:08:232
2352921045,9sleep20-21swapper/210:46:212
1020421045,11sleep20-21swapper/211:49:022
6371210321,64sleep06342-21latency_hist09:54:520
2215421035,10sleep20-21swapper/210:43:352
1929921035,11sleep20-21swapper/207:11:532
2412821015,10sleep20-21swapper/212:32:512
1150021018,61sleep20-21swapper/210:10:002
1150021018,61sleep20-21swapper/210:10:002
714029827,42sleep17162-21apt11:39:491
1842729816,64sleep018548-21cat10:30:200
306329715,64sleep13152-21/usr/sbin/munin11:25:121
1544829519,21sleep30-21swapper/308:40:193
721929421,57sleep37229-21grep11:39:513
3121829413,66sleep031351-21cat11:10:210
3050729411,25sleep330513-21ntp_states07:45:173
501929325,44sleep15174-21unixbench_multi11:30:201
723229211,65sleep07379-21latency_hist11:39:520
2918929216,58sleep029251-21/usr/sbin/munin11:05:250
237329214,61sleep32481-21grep08:00:113
1597629111,62sleep016029-21latency_hist10:24:510
285929018,51sleep32865-21sshd11:25:063
1256129015,61sleep112568-21ntp_states11:55:151
2861228911,60sleep028793-21latency_hist11:04:520
2861228911,60sleep028793-21latency_hist11:04:520
163992890,13sleep116366-21apt-key08:44:521
2827728714,43sleep00-21swapper/009:20:080
2357028719,48sleep123579-21/usr/sbin/munin07:25:041
2902328617,49sleep128998-21apt-get11:05:131
2781528616,54sleep127894-21sort11:00:371
2781528616,54sleep127894-21sort11:00:371
794928514,51sleep08077-21latency_hist09:59:520
794928514,51sleep08077-21latency_hist09:59:520
1677328526,44sleep316827-21grep07:05:123
1534728419,45sleep315479-21grep10:20:223
1758328325,43sleep00-21swapper/007:05:260
1156528329,25sleep30-21swapper/308:29:503
705728215,21sleep10-21swapper/109:55:171
705728215,21sleep10-21swapper/109:55:171
2138028225,31sleep021406-21modprobe08:59:490
2138028225,31sleep021406-21modprobe08:59:490
131662828,53sleep10-21swapper/108:34:521
3255028120,30sleep00-21swapper/009:34:490
2555028118,25sleep00-21swapper/009:10:390
1673728125,37sleep116256-21apt-get08:45:011
2586828021,35sleep10-21swapper/107:30:251
2435928017,28sleep10-21swapper/110:50:121
150612786,13sleep10-21swapper/112:04:521
2859127725,30sleep00-21swapper/007:40:090
2068227716,45sleep120722-21/usr/sbin/munin07:15:191
2068227716,45sleep120722-21/usr/sbin/munin07:15:191
1419027723,28sleep10-21swapper/112:00:201
1705327615,25sleep00-21swapper/012:10:120
39132756,13sleep10-21swapper/108:05:091
3247227523,27sleep30-21swapper/311:15:133
2425827511,52sleep0367-21snmpd09:06:310
2300727519,42sleep123027-21latency_hist09:04:511
2300727519,42sleep123027-21latency_hist09:04:511
144532755,12sleep10-21swapper/108:37:101
503427422,29sleep10-21swapper/109:50:001
205512748,18sleep00-21swapper/010:38:440
1891527414,42sleep018984-21/usr/sbin/munin12:15:210
1861227425,33sleep10-21swapper/112:15:141
120522740,6sleep30-21swapper/311:54:523
40327318,39sleep3231-21systemd-journal07:54:503
2897727318,35sleep328978-21/usr/sbin/munin11:04:593
286227318,38sleep12954-21/usr/sbin/munin08:00:211
2149427321,25sleep10-21swapper/110:40:191
1983027318,31sleep00-21swapper/008:54:530
1983027318,31sleep00-21swapper/008:54:530
1618927319,27sleep10-21swapper/110:25:001
1494727316,35sleep31-21systemd12:04:503
3191927217,24sleep30-21swapper/311:14:523
251192725,50sleep00-21swapper/007:30:040
2319427218,26sleep10-21swapper/107:24:521
1968427215,26sleep00-21swapper/007:14:540
1968427215,26sleep00-21swapper/007:14:540
1230927217,29sleep30-21swapper/311:55:093
828227115,33sleep08388-21tune2fs10:00:090
828227115,33sleep08388-21tune2fs10:00:090
1799527125,28sleep218014-21munin-run07:09:512
2413626923,24sleep00-21swapper/007:25:480
1022426910,27sleep10-21swapper/110:05:181
482426818,35sleep14973-21cat09:49:521
283122687,45sleep028462-21sed07:39:530
2644026819,31sleep326420-21/usr/sbin/munin07:34:513
433626723,24sleep00-21swapper/011:30:030
3102226715,39sleep131059-21ps11:10:181
2714326718,28sleep30-21swapper/307:35:163
243226720,29sleep30-21swapper/309:40:173
2199726723,20sleep00-21swapper/012:25:220
1902826717,29sleep119056-21modprobe10:34:491
1708926724,28sleep117139-21ntp_states07:05:181
1687026721,23sleep10-21swapper/112:10:091
982226615,24sleep10-21swapper/108:24:511
982226615,24sleep10-21swapper/108:24:511
3020526622,24sleep30-21swapper/309:25:203
3014126624,20sleep00-21swapper/007:45:100
2887326615,20sleep10-21swapper/107:40:141
282812660,7sleep10-21swapper/109:20:081
2136326619,32sleep021389-21netstat10:40:130
1882726613,27sleep30-21swapper/312:15:203
173672667,51sleep00-21swapper/010:28:270
878526512,20sleep30-21swapper/308:20:133
878526512,20sleep30-21swapper/308:20:133
278126516,22sleep00-21swapper/008:00:200
2702026521,23sleep10-21swapper/107:35:141
2516326512,36sleep325167-21diskstats09:10:263
172402657,11sleep30-21swapper/308:45:233
142326514,25sleep30-21swapper/311:20:093
348926414,37sleep33508-21apt-get09:45:023
267122649,42sleep10-21swapper/110:57:261
267122649,42sleep10-21swapper/110:57:261
2669226420,21sleep10-21swapper/109:15:081
2546626415,25sleep10-21swapper/109:10:361
2483926419,27sleep124931-21/usr/sbin/munin12:35:121
2005126423,24sleep30-21swapper/307:14:573
2005126423,24sleep30-21swapper/307:14:573
1809326410,39sleep00-21swapper/008:49:520
564826317,25sleep00-21swapper/008:10:130
47226320,18sleep00-21swapper/009:35:100
2847726318,20sleep30-21swapper/307:40:003
2610126312,36sleep326115-21awk10:55:183
2610126312,36sleep326115-21awk10:55:183
2008926316,25sleep30-21swapper/308:55:003
2008926316,25sleep30-21swapper/308:55:003
1866426316,29sleep118700-21/usr/sbin/munin07:10:151
878826215,29sleep08936-21grep11:44:530
552726220,18sleep30-21swapper/309:50:193
3221126217,32sleep132326-21cat07:50:191
233932629,12sleep00-21swapper/012:30:140
186692625,12sleep10-21swapper/108:50:141
186692625,12sleep10-21swapper/108:50:141
1570226217,26sleep015740-21grep12:05:200
123226210,12sleep10-21swapper/107:55:201
1133026221,22sleep00-21swapper/011:50:220
1001226218,25sleep30-21swapper/310:05:123
2718526114,20sleep00-21swapper/011:00:120
2718526114,20sleep00-21swapper/011:00:120
2589226119,23sleep30-21swapper/312:39:503
236626116,22sleep00-21swapper/009:40:140
2269626120,20sleep00-21swapper/009:01:230
2269626120,20sleep00-21swapper/009:01:230
1908126114,28sleep30-21swapper/310:34:513
141322617,42sleep00-21swapper/010:17:040
141322617,42sleep00-21swapper/010:17:040
2039526011,27sleep10-21swapper/112:20:181
1977526018,19sleep30-21swapper/310:35:163
1538226010,37sleep015426-21ntpq08:40:160
1220926018,20sleep10-21swapper/110:10:231
1220926018,20sleep10-21swapper/110:10:231
1150426013,17sleep00-21swapper/010:10:020
1150426013,17sleep00-21swapper/010:10:020
849125917,27sleep08510-21apt-get08:20:020
849125917,27sleep08510-21apt-get08:20:020
3053025910,13sleep10-21swapper/107:45:191
2462325916,22sleep10-21swapper/109:09:511
1203625915,29sleep312112-21awk10:10:213
1203625915,29sleep312112-21awk10:10:213
740725814,25sleep37466-21/usr/sbin/munin08:15:203
3100325816,26sleep131014-21modprobe09:29:501
2459025815,21sleep00-21swapper/010:50:190
2306325814,25sleep323090-21munin-run07:24:503
1315625816,26sleep113163-21diskstats10:15:071
1315625816,26sleep113163-21diskstats10:15:071
773025717,19sleep30-21swapper/311:40:113
366225716,20sleep00-21swapper/009:45:110
3193225722,18sleep10-21swapper/109:30:221
2195225717,27sleep022019-21sort07:20:110
2174425718,19sleep10-21swapper/112:25:151
210702575,43sleep00-21swapper/012:23:200
2037425718,18sleep30-21swapper/312:20:163
1381125719,16sleep30-21swapper/310:15:223
1381125719,16sleep30-21swapper/310:15:223
904025613,26sleep19087-21users10:00:221
904025613,26sleep19087-21users10:00:221
669025616,25sleep36825-21latency_hist08:14:523
38425612,27sleep3379-21apt-get09:35:053
276862566,38sleep30-21swapper/309:17:243
1972225617,24sleep319824-21cat08:54:523
1972225617,24sleep319824-21cat08:54:523
1809725619,18sleep30-21swapper/312:14:513
1183025613,25sleep311849-21apt-get08:30:023
1104425625,18sleep111126-21mailstats11:50:191
909525515,21sleep00-21swapper/011:45:090
3216925513,17sleep00-21swapper/011:15:050
291042555,11sleep30-21swapper/309:20:233
2795525516,17sleep00-21swapper/009:19:520
2597825514,25sleep026112-21date12:39:510
2455225515,20sleep30-21swapper/310:50:183
2342025516,23sleep123464-21ntpq12:30:151
123825514,25sleep01233-21apt-get11:20:020
393225411,15sleep30-21swapper/308:05:113
2974225413,24sleep029738-21apt-get09:25:040
2261725412,26sleep122626-21diskstats10:45:071
2144825417,20sleep30-21swapper/312:25:003
44325312,24sleep01-21systemd07:54:510
2428125315,24sleep30-21swapper/312:34:503
1210125313,24sleep012096-21apt-get11:55:020
1185325315,20sleep00-21swapper/008:30:030
1100325315,24sleep011043-21apt-get10:09:490
2228225217,18sleep30-21swapper/309:00:213
2228225217,18sleep30-21swapper/309:00:213
535625112,24sleep15375-21apt-get08:10:021
2852125113,22sleep328610-21gpgv11:04:513
2852125113,22sleep328610-21gpgv11:04:513
226012519,26sleep022616-21df10:45:060
3168825016,20sleep30-21swapper/309:30:163
2067525012,23sleep120750-21systemd-run10:39:501
154925010,16sleep00-21swapper/007:55:230
938724913,20sleep19436-21ntpq11:45:141
1667824913,18sleep30-21swapper/310:25:193
717824813,20sleep07201-21irqstats08:15:130
697924813,20sleep30-21swapper/309:55:153
697924813,20sleep30-21swapper/309:55:153
2126824812,22sleep321361-21memory10:40:123
169524810,21sleep10-21swapper/111:20:151
1466624811,21sleep114663-21apt-get10:20:041
325352477,16sleep10-21swapper/111:15:151
433424610,20sleep34331-21apt-get11:30:023
90072448,18sleep30-21swapper/311:45:043
81842448,17sleep30-21swapper/310:00:043
81842448,17sleep30-21swapper/310:00:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional