You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 04:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sat Jul 04, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
42132156215,1537sleep118479-1kworker/1:0H21:59:391
42132156215,1537sleep118479-1kworker/1:0H21:59:391
50492102026,959sleep25093-21ntpq23:44:462
12082267822,639sleep312089-21/usr/sbin/munin22:24:403
22240259214,553sleep222401-21unixbench_multi22:54:512
2475249019,454sleep02482-21/usr/sbin/munin21:54:410
8415236915,276sleep28544-21timerwakeupswit23:54:512
1750023277,309sleep2364-21snmpd19:15:542
1750023277,309sleep2364-21snmpd19:15:542
733022880,13sleep30-21swapper/322:09:373
3748225023,212sleep13874-21timerwakeupswit23:39:521
26714222765,94sleep20-21swapper/219:44:522
13801222314,194sleep113815-21uname22:29:431
27790221211,187sleep227841-21sort23:14:382
2216822046,183sleep222300-21latency_hist21:14:262
2216822046,183sleep222300-21latency_hist21:14:262
26847220310,113sleep226914-21latency_hist21:29:242
12280220129,106sleep212302-21apt-get20:44:232
13346219421,158sleep113394-21ntpq19:04:471
25067218614,89sleep20-21swapper/219:39:522
29942217495,43sleep20-21swapper/223:19:512
11506216817,135sleep311590-21grep20:39:513
28344216517,131sleep01-21systemd21:34:220
3158216319,129sleep13203-21cat20:14:421
4705215815,127sleep34725-21open_inodes22:00:133
4705215815,127sleep34725-21open_inodes22:00:133
11280215817,127sleep311292-21grep00:04:463
8860215412,123sleep00-21swapper/022:14:320
5861214979,59sleep25883-21sort20:24:242
1294421458,81sleep20-21swapper/220:44:512
18177214412,34sleep218180-21cron22:44:222
2755021439,116sleep027512-21latency_hist23:14:240
7868214112,117sleep27887-21apt20:29:332
28231214042,60sleep228310-21sort19:49:532
30866213232,80sleep230944-21fschecks_time19:59:422
14400213213,104sleep314449-21ntpq00:14:473
29292212891,19sleep20-21swapper/219:54:412
19433212814,25sleep20-21swapper/221:04:492
6375212753,59sleep26393-21grep20:24:442
6375212753,59sleep26393-21grep20:24:442
18128212511,99sleep218255-21timerwakeupswit20:59:532
3737212415,88sleep23811-21gpgv21:59:232
2143212327,33sleep20-21swapper/221:54:232
18844212325,55sleep20-21swapper/219:19:532
16545212310,91sleep216712-21gpgv22:39:252
16545212310,91sleep216712-21gpgv22:39:252
15871212310,98sleep215912-21service19:14:242
7559212111,92sleep07588-21irqstats22:09:430
23476212120,80sleep20-21swapper/219:34:512
10783212120,81sleep110912-21latency_hist20:39:251
20743212019,78sleep020887-21grep22:49:530
3162521199,56sleep231756-21unixbench_multi23:24:562
16323211517,18sleep20-21swapper/220:54:482
24790211420,69sleep324884-21fschecks_time23:04:423
22773211410,40sleep222809-21ntpq21:14:462
18466211412,85sleep218553-21latency_hist00:29:242
691211312,85sleep2856-21tail23:29:522
151221130,7sleep30-21swapper/323:34:243
7779211269,22sleep20-21swapper/222:09:492
21996211111,79sleep20-21swapper/219:29:512
27742210625,58sleep20-21swapper/221:29:522
5764210528,38sleep30-21swapper/322:04:333
3258121058,14sleep30-21swapper/320:04:433
1013521057,32sleep20-21swapper/222:19:222
3679210419,65sleep03109-21/usr/sbin/munin23:39:510
19366210419,61sleep319424-21systemd-run19:24:233
18731210418,64sleep30-21swapper/322:44:433
25242210222,52sleep025387-21tail19:39:540
18865210212,66sleep20-21swapper/222:44:462
28559210118,64sleep127633-21/usr/sbin/munin19:49:561
2737121005,17sleep20-21swapper/223:12:442
2565921007,16sleep20-21swapper/223:05:382
2129121005,12sleep20-21swapper/200:36:342
18699210027,60sleep2364-21snmpd00:29:322
312772995,11sleep20-21swapper/221:41:552
297522996,12sleep20-21swapper/221:37:352
1628929933,45sleep115896-21/usr/sbin/munin20:54:471
114032995,12sleep20-21swapper/222:20:272
39182985,17sleep20-21swapper/220:15:202
2775629821,55sleep027861-21fschecks_time19:49:430
252302986,11sleep20-21swapper/221:24:072
1159329813,71sleep011666-21wc20:39:520
669129711,70sleep36728-21ps23:49:523
101812969,20sleep20-21swapper/220:34:542
903329524,44sleep09070-21munin-run23:59:210
1410029517,63sleep114136-21fschecks_count00:14:411
1000029519,21sleep20-21swapper/223:59:522
211542948,73sleep221191-21idleruntime-cro19:29:222
2396529315,61sleep123984-21apt-get21:19:341
1455329213,60sleep20-21swapper/220:49:432
3233629116,51sleep132311-21/usr/sbin/munin20:04:371
573729022,12sleep20-21swapper/222:04:282
242262906,74sleep00-21swapper/019:39:050
2359829015,38sleep20-21swapper/222:59:482
2359829015,38sleep20-21swapper/222:59:482
1456529024,38sleep014559-21apt-get19:09:380
3233228912,57sleep20-21swapper/220:04:342
1365028927,42sleep313789-21sed19:04:533
422828818,15sleep20-21swapper/223:42:162
2289828813,60sleep322952-21mailstats21:14:513
1937828827,34sleep10-21swapper/100:29:511
148652886,14sleep20-21swapper/222:34:242
1176528820,47sleep011917-21latency_hist22:24:250
317512879,14sleep20-21swapper/221:44:342
2909628710,64sleep00-21swapper/021:34:510
11302879,58sleep00-21swapper/021:49:440
73742866,68sleep20-21swapper/223:52:582
2888028610,60sleep028881-21sshd23:17:060
14612867,14sleep20-21swapper/220:09:392
1207428618,13sleep20-21swapper/222:24:392
1185628624,30sleep20-21swapper/200:05:172
2489028516,51sleep00-21swapper/023:04:430
207042856,29sleep10-21swapper/119:25:181
588428426,30sleep30-21swapper/320:24:253
2440428417,49sleep324424-21modprobe23:04:213
2440428417,49sleep324424-21modprobe23:04:213
1688128414,54sleep316891-21/usr/sbin/munin19:14:463
1688128414,54sleep316891-21/usr/sbin/munin19:14:463
166332840,6sleep00-21swapper/020:54:530
1569128424,41sleep315783-21fschecks_time00:19:403
55582837,17sleep25707-21latency_hist22:04:242
55582837,17sleep25707-21latency_hist22:04:242
210352835,51sleep20-21swapper/222:51:202
737028227,38sleep07394-21idleruntime-cro20:29:220
737028227,38sleep07394-21idleruntime-cro20:29:220
3249528224,28sleep00-21swapper/021:44:520
2624028214,32sleep10-21swapper/123:09:361
229882828,22sleep10-21swapper/119:34:341
156862817,35sleep20-21swapper/200:19:392
289728022,30sleep10-21swapper/123:39:241
2625028021,40sleep326369-21grep21:24:573
1980228011,17sleep00-21swapper/022:49:240
1644328015,47sleep116541-21latency_hist22:39:241
1644328015,47sleep116541-21latency_hist22:39:241
1363428023,30sleep20-21swapper/200:13:132
1303128016,48sleep313079-21if_eth020:44:523
1786727924,27sleep10-21swapper/119:19:241
1786727924,27sleep10-21swapper/119:19:241
1466127922,36sleep10-21swapper/119:09:431
1379327924,32sleep00-21swapper/019:04:540
1364027923,32sleep30-21swapper/322:29:413
27592789,52sleep02916-21latency_hist20:14:250
2679527817,37sleep326955-21unixbench_multi19:44:533
2673727815,47sleep326816-21awk23:09:503
2037327815,49sleep120459-21gpgv21:09:241
801527719,44sleep08036-21fschecks_count20:30:030
2145227715,41sleep020831-21/usr/sbin/munin21:10:080
2145227715,41sleep020831-21/usr/sbin/munin21:10:080
673227615,20sleep00-21swapper/023:49:530
1635927618,40sleep316364-21modprobe22:39:223
1635927618,40sleep316364-21modprobe22:39:223
1209227615,39sleep011974-21/usr/sbin/munin22:24:410
3217927527,19sleep00-21swapper/023:29:220
2208027522,23sleep30-21swapper/319:29:523
2170327517,42sleep321767-21idleruntime22:54:403
2153527520,26sleep00-21swapper/019:29:400
210627521,23sleep10-21swapper/121:54:221
1218227512,47sleep112245-21if_eth022:24:421
777327414,42sleep10-21swapper/123:54:341
707827418,40sleep17195-21latency_hist22:09:241
2114927419,23sleep10-21swapper/121:10:011
2114927419,23sleep10-21swapper/121:10:011
177292747,13sleep00-21swapper/019:18:380
177292747,13sleep00-21swapper/019:18:380
83462735,13sleep10-21swapper/122:11:261
273527321,22sleep30-21swapper/320:14:243
195027316,25sleep00-21swapper/023:34:470
156127321,25sleep30-21swapper/323:34:373
132027321,32sleep10-21swapper/123:34:231
1310327327,27sleep10-21swapper/100:09:521
932527214,43sleep39361-21wc22:14:473
3002227224,27sleep30-21swapper/323:19:523
18312726,11sleep20-21swapper/223:34:452
384327117,33sleep34050-21latency_hist21:59:243
3046527120,33sleep330475-21modprobe19:59:223
2975527123,25sleep00-21swapper/019:54:510
2466127119,37sleep124766-21tune2fs19:39:431
1512627121,40sleep20-21swapper/200:17:162
1091827126,27sleep10-21swapper/122:19:501
3078227014,40sleep330852-21df19:59:393
1942727020,22sleep00-21swapper/019:24:240
1277927023,23sleep10-21swapper/120:44:341
1248827016,28sleep10-21swapper/120:44:261
2491126912,36sleep324912-21sshd21:20:043
2321826923,33sleep123274-21tune2fs22:59:391
2321826923,33sleep123274-21tune2fs22:59:391
1637226918,33sleep0364-21snmpd22:39:230
1637226918,33sleep0364-21snmpd22:39:230
3074226815,34sleep030183-21/usr/sbin/munin21:39:510
2574926817,30sleep00-21swapper/021:24:450
1959026820,26sleep30-21swapper/321:04:543
1952526825,24sleep10-21swapper/119:24:251
365626717,29sleep00-21swapper/020:14:530
317562677,14sleep10-21swapper/121:44:371
1724126712,25sleep00-21swapper/000:24:410
1301726718,24sleep00-21swapper/000:09:510
2929126617,19sleep30-21swapper/321:34:543
2208826613,23sleep00-21swapper/022:54:490
2085726621,25sleep10-21swapper/100:34:521
1786826610,39sleep017914-21/usr/sbin/munin20:59:460
1339526622,18sleep00-21swapper/020:45:010
2553326513,29sleep10-21swapper/121:24:341
1591326517,24sleep30-21swapper/319:14:253
529426416,20sleep00-21swapper/023:44:520
2031326413,34sleep020372-21gpgv21:09:230
135112648,12sleep00-21swapper/022:29:250
2426026314,33sleep024307-21cat21:19:450
2158426321,24sleep00-21swapper/000:39:230
1844326313,28sleep30-21swapper/321:00:223
140442635,15sleep20-21swapper/219:06:532
854626217,18sleep30-21swapper/323:54:523
39402625,12sleep30-21swapper/320:15:373
3057726211,22sleep30-21swapper/321:39:463
2058426218,23sleep00-21swapper/019:25:170
1445526217,20sleep00-21swapper/020:49:410
3139126121,15sleep10-21swapper/119:59:531
1652626119,21sleep30-21swapper/320:54:533
1584126121,19sleep10-21swapper/100:19:421
128582615,13sleep30-21swapper/300:09:463
920826019,24sleep10-21swapper/120:34:241
2611426012,25sleep00-21swapper/019:44:380
2182026018,19sleep10-21swapper/119:29:461
993025915,23sleep10-21swapper/123:59:511
992325916,28sleep39423-21/usr/sbin/munin20:34:513
675725915,27sleep30-21swapper/320:24:533
675725915,27sleep30-21swapper/320:24:533
2919125915,28sleep129335-21modprobe23:19:251
2840225920,15sleep10-21swapper/123:14:531
2754525918,21sleep30-21swapper/321:29:493
2632025915,25sleep10-21swapper/119:44:431
2587525917,22sleep30-21swapper/319:44:253
1504525920,17sleep00-21swapper/000:16:220
1461125914,29sleep314640-21irqstats20:49:443
610825816,26sleep16105-21apt-get20:24:351
610825816,26sleep16105-21apt-get20:24:351
442725816,18sleep00-21swapper/022:00:060
442725816,18sleep00-21swapper/022:00:060
3186425811,36sleep031865-21sshd20:02:020
2778625818,24sleep327782-21apt-get23:14:353
2708225816,25sleep127077-21apt-get21:29:351
1878325815,26sleep018788-21diskstats00:29:380
83262578,30sleep38364-21/usr/sbin/munin20:30:143
2049925718,22sleep30-21swapper/319:25:163
2018325720,18sleep10-21swapper/122:49:421
1818125716,26sleep318226-21munin-run22:44:233
1222725712,28sleep112305-21latency_hist00:09:241
3014725612,25sleep030269-21users23:19:530
2600225611,26sleep10-21swapper/123:09:241
2321325612,17sleep00-21swapper/022:59:380
2321325612,17sleep00-21swapper/022:59:380
84625514,24sleep3842-21apt-get21:49:353
4142555,11sleep30-21swapper/321:47:333
3020825515,22sleep130201-21apt-get21:39:361
2983225519,19sleep10-21swapper/119:54:521
2249125515,26sleep122567-21cut21:14:411
2132125517,17sleep30-21swapper/322:54:243
1920825514,26sleep319278-21/usr/sbin/munin00:29:493
1845025514,24sleep118458-21apt-get22:44:331
576625414,24sleep05761-21apt-get22:04:340
455925413,24sleep14554-21apt-get20:19:351
3221925414,25sleep132229-21munin-run23:29:231
2882425412,24sleep128882-21if_eth021:34:441
2432625415,19sleep30-21swapper/319:39:243
948125319,14sleep30-21swapper/323:59:413
465525311,17sleep10-21swapper/123:44:331
2831125317,16sleep30-21swapper/319:49:543
2136525317,19sleep30-21swapper/321:10:073
2136525317,19sleep30-21swapper/321:10:073
1511125320,18sleep10-21swapper/120:49:551
1071825320,15sleep30-21swapper/322:19:433
886125213,23sleep18855-21apt-get22:14:331
2240525212,24sleep022397-21apt-get21:14:360
1805025217,22sleep118127-21cat20:59:521
1769225216,18sleep30-21swapper/300:24:533
1688525213,22sleep116379-21apt-get22:39:321
127325215,16sleep00-21swapper/023:34:220
943125113,19sleep00-21swapper/023:59:380
666725114,23sleep16683-21ntp_states23:49:501
598825115,20sleep35960-21latency_hist23:49:233
3094825113,21sleep330995-21df23:24:403
2110025111,23sleep321153-21users00:34:543
1822825111,17sleep00-21swapper/022:44:240
1219025116,19sleep00-21swapper/000:09:230
442425018,16sleep30-21swapper/323:44:233
119225019,15sleep00-21swapper/020:09:240
2216024910,16sleep10-21swapper/122:54:501
14032498,16sleep10-21swapper/120:09:331
952524811,21sleep09531-21diskstats20:34:390
456324715,18sleep00-21swapper/020:19:380
365424713,19sleep13736-21systemd-run21:59:221
1560824514,14sleep00-21swapper/000:19:370
176202399,13sleep10-21swapper/100:24:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional