You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 19:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Tue May 05, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
143982329910,3272sleep230894-1kworker/2:0H12:15:262
5885273712,709sleep3275-1kworker/3:3H10:00:113
5885273712,709sleep3275-1kworker/3:3H10:00:113
10422250111,427sleep010512-21fschecks_time10:20:150
16334246919,434sleep216378-21ntpq12:20:232
21379243720,399sleep221511-21/usr/sbin/munin12:35:292
25267237026,20sleep025281-21grep07:30:190
10822231120,29sleep20-21swapper/208:25:372
2140723109,18sleep00-21swapper/010:55:080
2140723109,18sleep00-21swapper/010:55:080
25104228928,192sleep025255-21unixbench_singl09:10:300
28800224931,204sleep028769-21latency_hist09:24:590
940224648,69sleep00-21swapper/009:39:570
19132223734,37sleep00-21swapper/012:30:140
2534223319,199sleep22542-21awk08:00:252
2904522318,209sleep029185-21unixbench_singl07:40:290
10516222615,197sleep110574-21if_eth010:20:161
9145221940,43sleep00-21swapper/010:15:190
6440220810,180sleep06456-21modprobe08:14:570
19412220037,54sleep00-21swapper/010:50:000
19412220037,54sleep00-21swapper/010:50:000
4496219611,172sleep04537-21ntpq09:55:240
4496219611,172sleep04537-21ntpq09:55:240
29338219658,67sleep029361-21fschecks_count11:20:160
29338219658,67sleep029361-21fschecks_count11:20:160
24940218610,52sleep00-21swapper/011:05:220
23633218216,15sleep00-21swapper/007:25:140
23633218216,15sleep00-21swapper/007:25:140
22170218155,75sleep00-21swapper/007:20:170
22170218155,75sleep00-21swapper/007:20:170
31374218012,152sleep331393-21apt-get07:50:093
21007217313,147sleep021016-21wc09:00:000
13146216926,91sleep013227-21cat12:10:270
17012216817,130sleep017046-21proc_pri10:40:260
26991216646,56sleep00-21swapper/007:35:200
26991216646,56sleep00-21swapper/007:35:200
15384216510,119sleep015469-21timerandwakeup08:40:290
15384216510,119sleep015469-21timerandwakeup08:40:290
23487216039,67sleep018338-21cyclictest11:00:250
398215420,116sleep2471-21sendmail_mailtr11:30:282
18222215422,112sleep318281-21systemd-run07:09:593
30883214512,120sleep030913-21fschecks_count11:25:170
2742721458,125sleep027446-21cat11:14:590
2742721458,125sleep027446-21cat11:14:590
6805214166,34sleep06958-21unixbench_multi11:50:290
6805214166,34sleep06958-21unixbench_multi11:50:290
8001213930,94sleep08074-21unin-run08:19:580
30674213837,54sleep018338-21cyclictest09:30:160
1230213728,56sleep00-21swapper/009:40:060
16234213626,57sleep00-21swapper/012:20:210
457521338,17sleep00-21swapper/011:45:120
3849213331,88sleep03910-21latency_hist09:54:590
1283121325,11sleep00-21swapper/008:33:590
19858212813,53sleep00-21swapper/007:15:000
19858212813,53sleep00-21swapper/007:15:000
15312212812,98sleep315381-21sensors_temp08:40:283
15312212812,98sleep315381-21sensors_temp08:40:283
20769212746,59sleep00-21swapper/007:15:210
20769212746,59sleep00-21swapper/007:15:210
917521250,12sleep20-21swapper/211:59:592
12331212330,81sleep012337-21ntpq10:25:240
9717212213,95sleep39776-21awk12:00:333
3258221226,12sleep00-21swapper/007:54:010
2951021218,75sleep029575-21ps09:25:280
24375212118,78sleep124516-21grep07:25:311
24375212118,78sleep124516-21grep07:25:311
17101212114,84sleep317221-21/usr/sbin/munin08:45:293
1465021205,11sleep00-21swapper/010:34:330
8967211910,57sleep09027-21dpkg11:59:580
494521189,69sleep05081-21date08:10:000
494521189,69sleep05081-21date08:10:000
27756211652,15sleep00-21swapper/011:15:150
9447211352,14sleep00-21swapper/012:00:020
27984211322,73sleep228030-21sendmail07:39:582
27984211322,73sleep228030-21sendmail07:39:582
2233211117,77sleep22274-21grep11:35:292
14347211110,63sleep014366-21apt-get12:15:080
26213210920,65sleep226311-21fschecks_time11:10:172
26213210920,65sleep226311-21fschecks_time11:10:172
10110210839,13sleep00-21swapper/008:25:190
9729210720,71sleep29933-21latency_hist08:25:012
30545210714,75sleep130557-21munin-run11:24:581
30545210714,75sleep130557-21munin-run11:24:581
11045210728,52sleep00-21swapper/012:05:140
14861210620,67sleep114950-21grep08:40:181
14861210620,67sleep114950-21grep08:40:181
13334210620,69sleep213454-21unixbench_singl12:10:292
23479210516,71sleep323480-21ntp_states11:00:243
1445210525,42sleep10-21swapper/111:35:111
18548210421,46sleep10-21swapper/107:10:131
18548210421,46sleep10-21swapper/107:10:131
29847210325,48sleep229844-21apt-get07:45:112
20826210212,73sleep320882-21/usr/sbin/munin12:35:183
17877210234,15sleep00-21swapper/012:25:210
1020210220,68sleep21110-21latency_hist09:39:592
21017210114,69sleep120946-21latency_hist09:00:011
791621009,64sleep30-21swapper/311:55:203
19988210023,52sleep120121-21fschecks_time07:15:011
19988210023,52sleep120121-21fschecks_time07:15:011
2279929935,17sleep00-21swapper/009:05:130
799529815,58sleep37997-21cron08:19:573
626529812,69sleep06303-21modprobe10:04:570
626529812,69sleep06303-21modprobe10:04:570
46929710,54sleep0363-21snmpd07:55:100
46929710,54sleep0363-21snmpd07:55:100
176022978,77sleep2225-21systemd-journal10:42:222
3117629425,40sleep30-21swapper/307:50:003
1194929418,55sleep311850-21/usr/sbin/munin10:25:153
432929314,22sleep30-21swapper/311:44:583
338329316,58sleep23392-21/usr/sbin/munin11:40:212
301729315,63sleep1225-21systemd-journal08:00:571
2444529317,63sleep324467-21fschecks_count09:10:163
1964929323,53sleep00-21swapper/010:50:120
878429241,39sleep08823-21processes08:20:260
679529228,55sleep06531-21/usr/sbin/munin10:05:210
679529228,55sleep06531-21/usr/sbin/munin10:05:210
33912918,31sleep03534-21date08:05:000
2039729121,50sleep220526-21grep10:50:302
1873229141,39sleep018338-21cyclictest08:50:350
2985029035,38sleep00-21swapper/007:45:130
1828428929,37sleep018338-21latencyplot07:10:000
131922899,19sleep00-21swapper/008:35:080
399728819,34sleep30-21swapper/308:05:233
399728819,34sleep30-21swapper/308:05:233
1761928810,62sleep017681-21cat08:49:580
379928731,32sleep30-21swapper/309:54:583
2821028716,53sleep228275-21/usr/sbin/munin11:15:262
161702875,71sleep00-21swapper/010:39:130
161702875,71sleep00-21swapper/010:39:130
934428618,52sleep19424-21wc10:15:241
781228619,49sleep37822-21modprobe10:09:573
781228619,49sleep37822-21modprobe10:09:573
2124328635,35sleep00-21swapper/009:00:120
849328525,16sleep00-21swapper/010:14:580
849328525,16sleep00-21swapper/010:14:580
437128525,16sleep00-21swapper/011:44:590
1059828411,56sleep310692-21/usr/sbin/munin10:20:183
2857228315,49sleep328274-21/usr/sbin/munin07:40:193
1267728323,17sleep10-21swapper/112:10:161
320128235,33sleep00-21swapper/009:49:590
1565428221,39sleep315792-21grep10:35:293
1565428221,39sleep315792-21grep10:35:293
2195528122,45sleep00-21swapper/012:39:590
152228133,32sleep00-21swapper/011:35:140
1456328122,26sleep30-21swapper/308:40:003
1141428116,48sleep111016-21/usr/sbin/munin12:05:241
1173428028,25sleep10-21swapper/108:30:171
58227920,38sleep1647-21fschecks_time07:55:201
58227920,38sleep1647-21fschecks_time07:55:201
259062797,15sleep00-21swapper/009:15:100
161882786,39sleep10-21swapper/110:39:271
161882786,39sleep10-21swapper/110:39:271
3234927732,30sleep00-21swapper/011:30:140
3143327717,45sleep231493-21/usr/sbin/munin09:30:312
2525527727,31sleep325389-21sed11:05:293
2255827617,44sleep322637-21latency_hist09:04:593
595427517,32sleep20-21swapper/208:10:312
902827412,46sleep18946-21latency_hist11:59:591
77472745,59sleep20-21swapper/210:09:072
77472745,59sleep20-21swapper/210:09:072
489127421,34sleep14917-21munin-run08:09:571
489127421,34sleep14917-21munin-run08:09:571
2411727418,27sleep20-21swapper/207:25:282
2411727418,27sleep20-21swapper/207:25:282
2320327413,40sleep322775-21/usr/sbin/munin09:05:243
2590427320,33sleep325901-21apt-get09:15:093
2140427313,45sleep220923-21apt-get10:55:062
2140427313,45sleep220923-21apt-get10:55:062
1849727323,26sleep20-21swapper/208:50:292
1596627314,42sleep116015-21/usr/sbin/munin12:20:131
1175827314,43sleep2363-21snmpd08:30:182
87127224,31sleep2905-21ntp_kernel_err07:55:252
87127224,31sleep2905-21ntp_kernel_err07:55:252
2758027219,26sleep30-21swapper/309:20:183
2338627220,29sleep10-21swapper/107:24:591
2338627220,29sleep10-21swapper/107:24:591
543027114,41sleep25480-21sshd11:45:302
2566627118,26sleep20-21swapper/209:15:002
2348227116,33sleep123623-21unixbench_multi09:05:301
1696227113,44sleep10-21swapper/107:05:131
962727011,30sleep10-21swapper/108:25:001
2328427020,34sleep222991-21/usr/sbin/munin11:00:192
2308627013,29sleep10-21swapper/111:00:141
2643626921,29sleep20-21swapper/207:34:572
220802697,45sleep20-21swapper/207:20:112
220802697,45sleep20-21swapper/207:20:112
1985926918,37sleep319916-21interrupts10:50:193
1202126924,30sleep30-21swapper/308:30:233
3070926811,37sleep10-21swapper/109:30:171
2774826817,19sleep30-21swapper/311:15:143
247626816,34sleep12551-21grep09:45:281
1810826816,32sleep118099-21apt-get08:50:111
1505326821,26sleep20-21swapper/210:35:162
1505326821,26sleep20-21swapper/210:35:162
503726714,20sleep10-21swapper/111:45:251
305872679,44sleep30-21swapper/309:30:113
2484026716,28sleep20-21swapper/211:05:202
682926618,24sleep10-21swapper/110:05:221
682926618,24sleep10-21swapper/110:05:221
631626622,25sleep20-21swapper/210:04:592
631626622,25sleep20-21swapper/210:04:592
3275026620,27sleep1324-21ntpq11:30:231
2893326615,34sleep128976-21awk11:19:571
2776326620,27sleep20-21swapper/209:20:212
2658926627,21sleep20-21swapper/209:15:282
255326614,27sleep20-21swapper/209:45:292
2012526618,30sleep220226-21munin-node07:15:022
2012526618,30sleep220226-21munin-node07:15:022
177326618,20sleep30-21swapper/307:59:583
177326618,20sleep30-21swapper/307:59:583
1473226619,20sleep20-21swapper/210:34:592
1370226613,39sleep113779-21readlink08:35:261
807626516,35sleep28178-21latency_hist08:20:002
679726520,25sleep10-21swapper/108:15:141
2132926518,32sleep121400-21cut09:00:171
2059126524,18sleep10-21swapper/112:35:071
558226412,36sleep35624-21ntpq08:10:243
3127726418,31sleep231339-21sendmail_mailqu11:25:272
2631326422,18sleep30-21swapper/311:10:183
2631326422,18sleep30-21swapper/311:10:183
2090226419,28sleep320946-21sshd07:15:243
2090226419,28sleep320946-21sshd07:15:243
1768226415,24sleep10-21swapper/108:49:591
877626317,22sleep30-21swapper/308:20:253
639126322,18sleep30-21swapper/311:50:203
639126322,18sleep30-21swapper/311:50:203
2679526314,31sleep126911-21sed11:10:281
2679526314,31sleep126911-21sed11:10:281
1883326316,25sleep20-21swapper/212:30:002
1748426319,31sleep30-21swapper/312:25:083
588426220,21sleep30-21swapper/311:49:593
3148726216,25sleep131634-21unixbench_multi11:25:301
226826212,35sleep32298-21awk09:45:213
2037926213,20sleep10-21swapper/112:34:591
1389126219,26sleep313979-21systemd-run12:14:583
1011126215,30sleep110092-21apt-get08:25:201
1000826218,25sleep210036-21munin-run10:19:572
406526114,21sleep20-21swapper/209:55:102
406526114,21sleep20-21swapper/209:55:102
2525726116,28sleep125266-21/usr/sbin/munin09:10:311
1863526123,19sleep30-21swapper/308:50:343
1224526115,32sleep212286-21ntpq10:25:222
633326016,27sleep26390-21/usr/sbin/munin11:50:182
633326016,27sleep26390-21/usr/sbin/munin11:50:182
363026017,27sleep23623-21apt-get08:05:142
363026017,27sleep23623-21apt-get08:05:142
2636126017,29sleep126368-21/usr/sbin/munin09:15:231
2145126023,20sleep10-21swapper/110:55:221
2145126023,20sleep10-21swapper/110:55:221
2035426021,18sleep20-21swapper/208:55:302
1789926018,26sleep117938-21awk12:25:221
1495226015,31sleep214985-21/usr/sbin/munin08:40:192
1495226015,31sleep214985-21/usr/sbin/munin08:40:192
1048226015,30sleep310521-21ntpq08:25:303
588025920,24sleep15375-21apt-get10:00:091
588025920,24sleep15375-21apt-get10:00:091
2140525913,24sleep30-21swapper/310:55:073
2140525913,24sleep30-21swapper/310:55:073
2032825917,26sleep320354-21munin-run12:34:573
1818625915,28sleep118281-21fschecks_time10:45:161
1818625915,28sleep118281-21fschecks_time10:45:161
1699025917,23sleep20-21swapper/208:45:282
1089625918,19sleep20-21swapper/210:20:262
945425816,25sleep28947-21apt-get12:00:082
482325818,18sleep30-21swapper/309:55:313
482325818,18sleep30-21swapper/309:55:313
292325816,25sleep32975-21users08:00:313
2681125815,29sleep126845-21fschecks_count07:35:171
2681125815,29sleep126845-21fschecks_count07:35:171
1829325813,31sleep218350-21cat10:45:172
1829325813,31sleep218350-21cat10:45:172
1456325819,24sleep114618-21/usr/sbin/munin12:15:331
1428525717,23sleep114332-21users10:30:341
1339425716,24sleep313413-21apt-get10:30:083
302325616,26sleep13017-21apt-get11:40:121
2850625611,20sleep10-21swapper/107:40:171
2590925614,24sleep126055-21latency_hist11:09:591
2400925616,19sleep20-21swapper/209:09:232
207522567,11sleep10-21swapper/110:52:551
1863425621,18sleep20-21swapper/207:10:182
1863425621,18sleep20-21swapper/207:10:182
171722568,11sleep30-21swapper/312:24:193
1700425617,23sleep317010-21ntp_states10:40:253
949525515,24sleep19475-21apt-get12:00:191
3271125514,16sleep20-21swapper/207:55:002
3259825513,25sleep10-21swapper/109:35:261
3079725513,25sleep330793-21apt-get11:25:123
3055825512,22sleep20-21swapper/211:24:592
3055825512,22sleep20-21swapper/211:24:592
304232555,17sleep30-21swapper/311:23:513
304232555,17sleep30-21swapper/311:23:513
2803125512,23sleep30-21swapper/307:39:593
2803125512,23sleep30-21swapper/307:39:593
1187525511,25sleep111850-21/usr/sbin/munin10:25:131
530725410,34sleep1365-21lldpd09:59:251
530725410,34sleep1365-21lldpd09:59:251
3238025412,25sleep10-21swapper/107:51:241
3213425413,24sleep332130-21apt-get09:35:123
3033825415,24sleep3284-21dbus-daemon09:29:593
2911725413,23sleep129123-21diskstats09:25:171
2386925415,24sleep323902-21irqstats07:25:213
2386925415,24sleep323902-21irqstats07:25:213
2802325316,20sleep1363-21snmpd09:20:281
1932425314,18sleep30-21swapper/307:10:313
1932425314,18sleep30-21swapper/307:10:313
180492537,33sleep20-21swapper/207:07:482
1378025316,17sleep20-21swapper/208:35:272
851525217,14sleep10-21swapper/110:14:591
851525217,14sleep10-21swapper/110:14:591
2829825211,25sleep228294-21apt-get07:40:102
1967225214,22sleep319741-21df08:55:143
1906725213,24sleep219061-21apt-get12:30:122
1125325214,17sleep20-21swapper/212:05:212
888425117,18sleep30-21swapper/310:15:143
515625117,18sleep10-21swapper/108:10:101
2171225120,16sleep20-21swapper/209:00:262
1808725111,24sleep318083-21apt-get10:45:103
1808725111,24sleep318083-21apt-get10:45:103
612525010,22sleep16192-21df11:50:131
612525010,22sleep16192-21df11:50:131
3021425015,18sleep10-21swapper/107:45:221
2543925014,22sleep325153-21/usr/sbin/munin07:30:223
1654425012,22sleep116541-21apt-get10:40:111
1540425021,16sleep30-21swapper/312:16:563
2216424914,20sleep322168-21diskstats07:20:163
2216424914,20sleep322168-21diskstats07:20:163
165724714,16sleep10-21swapper/109:44:591
1131224715,14sleep30-21swapper/312:05:223
323212466,15sleep30-21swapper/311:30:083
2649124513,16sleep10-21swapper/107:35:001
294942447,16sleep20-21swapper/209:25:262
273024412,17sleep32782-21munin-run11:39:583
168924311,14sleep20-21swapper/209:45:002
84522367,14sleep30-21swapper/310:14:573
84522367,14sleep30-21swapper/310:14:573
11112366,12sleep30-21swapper/309:40:003
819423510,14sleep20-21swapper/210:11:372
819423510,14sleep20-21swapper/210:11:372
35572317,14sleep10-21swapper/109:51:501
33312316,11sleep20-21swapper/209:50:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional