You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 15:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Aug 05, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
170492667714,6650sleep212091-21kworker/2:022:49:422
17636257814,547sleep317639-21/usr/sbin/munin22:52:463
27033252510,491sleep018905-1kworker/0:1H19:57:470
13496246718,437sleep220252-21kworker/u8:320:57:422
27439236314,335sleep127464-21awk19:58:241
296923419,262sleep13085-21sed20:22:581
30923232312,295sleep030935-21/usr/sbin/munin21:52:450
21941232012,293sleep221982-21/usr/sbin/munin23:07:262
16815223922,194sleep016836-21fschecks_count00:32:490
16307223249,165sleep116326-21modprobe21:07:261
2789222219,187sleep02822-21fschecks_count23:47:440
19839221785,43sleep119927-21fschecks_time21:17:471
18750221716,182sleep118720-21latency_hist19:32:291
2988922128,192sleep129858-21latency_hist20:07:281
2835122026,185sleep1364-21snmpd23:26:431
12084219711,169sleep0364-21snmpd00:17:430
23488219526,150sleep123511-21awk23:12:261
25961219429,139sleep00-21swapper/021:37:280
2245219220,152sleep22388-21unixbench_singl22:03:012
20963219017,27sleep10-21swapper/119:37:541
8661218747,34sleep18700-21cut20:42:441
8661218747,34sleep18700-21cut20:42:441
22837217716,146sleep322864-21grep21:27:423
765521689,102sleep17669-21kernelversion00:02:511
4648216811,143sleep24691-21ntpq23:52:512
17311216851,22sleep10-21swapper/122:52:281
322216313,103sleep10-21swapper/121:57:551
6518216010,24sleep16661-21grep23:57:591
9335215741,23sleep10-21swapper/100:07:511
4366215528,115sleep14385-21open_inodes20:27:551
20328215411,102sleep120475-21latency_hist19:37:291
4404215359,77sleep14540-21cat22:12:281
4404215359,77sleep14540-21cat22:12:281
24159215216,32sleep124271-21apt-config21:32:281
3111921469,124sleep131236-21wc23:33:001
848221440,11sleep30-21swapper/322:22:543
26902214291,34sleep126910-21systemd-run23:22:271
11774214232,89sleep111928-21unixbench_multi22:32:581
11510214210,121sleep2364-21snmpd00:14:182
31425214012,15sleep00-21swapper/020:12:280
31205213584,12sleep10-21swapper/121:52:501
2132213572,42sleep10-21swapper/122:03:001
11880213127,78sleep211940-21latency_hist19:12:282
874213019,16sleep10-21swapper/120:17:411
13269212910,81sleep113451-21sed20:57:291
13937212812,99sleep113952-21munin-run22:42:271
29897212614,97sleep230013-21cat21:47:552
140621259,26sleep10-21swapper/123:42:491
18037212386,24sleep118101-21mailstats22:52:561
5508212264,44sleep15576-40mandb23:57:261
3538212111,23sleep10-21swapper/123:47:581
18247212115,87sleep218263-21sort21:12:482
26707212064,40sleep10-21swapper/121:37:571
26707212064,40sleep10-21swapper/121:37:571
12056212013,56sleep312075-21apt-get00:17:383
2115211913,17sleep00-21swapper/020:22:290
14814211910,92sleep014968-21cat21:02:280
14425211715,85sleep313942-21apt-get22:42:373
23779211612,76sleep10-21swapper/123:12:421
23779211612,76sleep10-21swapper/123:12:421
9399211526,69sleep39443-21ntp_offset19:03:013
9399211526,69sleep39443-21ntp_offset19:03:013
6178211124,70sleep06324-21grep20:32:580
2149921118,17sleep00-21swapper/023:03:000
9266211023,74sleep09284-21kernelversion19:02:580
9266211023,74sleep09284-21kernelversion19:02:580
6062211062,13sleep10-21swapper/122:16:401
29102211016,70sleep229160-21fschecks_time23:28:012
25697211077,18sleep10-21swapper/119:52:551
3066210817,72sleep03134-21sshd22:07:430
3066210817,72sleep03134-21sshd22:07:430
21094210811,84sleep121134-21ntpq23:02:531
15885210713,75sleep10-21swapper/119:22:401
11958210714,64sleep00-21swapper/020:52:520
25598210610,45sleep30-21swapper/323:17:403
20444210619,66sleep020543-21latency_hist23:02:280
15506210612,80sleep215514-21ntp_states21:02:552
1108721069,78sleep10-21swapper/119:08:151
29262210517,72sleep329280-21apt21:47:363
29001210510,20sleep129040-21modprobe21:47:261
29001210510,20sleep129040-21modprobe21:47:261
15578210415,73sleep015627-21ntpq00:27:520
14192210362,22sleep10-21swapper/119:17:481
22202210224,11sleep10-21swapper/119:42:461
17731210258,12sleep10-21swapper/121:10:411
1517210214,71sleep00-21swapper/022:02:420
12126210210,80sleep112253-21uname19:12:311
12126210210,80sleep112253-21uname19:12:311
1012921029,16sleep10-21swapper/120:47:441
1012921029,16sleep10-21swapper/120:47:441
9068210113,72sleep39148-21sendmail_mailtr20:42:543
9068210113,72sleep39148-21sendmail_mailtr20:42:543
1168721018,73sleep10-21swapper/120:52:451
9630210010,73sleep10-21swapper/122:27:461
9630210010,73sleep10-21swapper/122:27:461
13601210022,12sleep10-21swapper/100:22:371
13601210022,12sleep10-21swapper/100:22:371
2490829915,26sleep30-21swapper/321:33:073
738329821,10sleep10-21swapper/122:18:251
2493029810,14sleep10-21swapper/119:52:271
167052986,20sleep10-21swapper/100:32:421
129529822,57sleep31445-21latency_hist22:02:293
897329718,12sleep10-21swapper/122:23:541
2892629716,64sleep328422-21apt-get23:27:373
83262968,75sleep18335-21munin-run20:42:271
83262968,75sleep18335-21munin-run20:42:271
3057929616,63sleep230667-21tune2fs23:32:492
206632969,71sleep020752-21fschecks_time19:37:490
1693729614,67sleep216987-21if_eth000:32:512
1609129618,62sleep316162-21fschecks_time22:47:463
225982949,47sleep122597-21/usr/sbin/munin21:27:281
152582947,17sleep10-21swapper/100:27:451
311032935,37sleep10-21swapper/120:08:451
2657029314,21sleep00-21swapper/021:37:520
2657029314,21sleep00-21swapper/021:37:520
1413129312,68sleep014208-21cat00:22:560
1413129312,68sleep014208-21cat00:22:560
191232927,16sleep1364-21snmpd21:13:391
176072929,14sleep00-21swapper/021:09:020
1590329217,57sleep315882-21apt-get19:22:503
1576329223,48sleep315911-21unixbench_singl21:03:003
390429127,40sleep03899-21apt-get20:27:410
148262918,12sleep10-21swapper/122:42:511
1003329116,60sleep210112-21awk22:27:552
1003329116,60sleep210112-21awk22:27:552
321402905,22sleep10-21swapper/123:37:441
1187629015,51sleep10-21swapper/100:17:281
96632898,11sleep00-21swapper/020:45:130
96632898,11sleep00-21swapper/020:45:130
42528911,16sleep30-21swapper/320:16:423
2905228923,16sleep30-21swapper/321:47:283
2905228923,16sleep30-21swapper/321:47:283
286652899,12sleep20-21swapper/220:02:452
2598128914,53sleep026111-21/usr/sbin/munin19:52:580
432228819,46sleep34337-21apt-get22:12:263
432228819,46sleep34337-21apt-get22:12:263
3023628820,57sleep10-21swapper/121:48:271
2574928811,10sleep10-21swapper/121:35:431
2237828816,55sleep022390-21/usr/sbin/munin19:42:490
124982889,63sleep10-21swapper/100:17:521
195222875,12sleep10-21swapper/122:57:531
914428617,35sleep10-21swapper/119:02:551
914428617,35sleep10-21swapper/119:02:551
300002865,12sleep10-21swapper/123:29:531
88928518,41sleep3948-21grep20:17:433
66532856,11sleep10-21swapper/120:36:421
2893828526,37sleep00-21swapper/023:27:420
326282836,12sleep10-21swapper/120:13:301
2853828315,44sleep228711-21cat23:27:282
2427528316,41sleep224483-21apt-config21:32:292
2024528223,41sleep220270-21modprobe19:37:262
1362928215,46sleep313682-21cpuspeed00:22:433
1362928215,46sleep313682-21cpuspeed00:22:433
946628124,29sleep20-21swapper/219:03:042
946628124,29sleep20-21swapper/219:03:042
678728118,43sleep26839-21ls20:37:282
2662928120,33sleep226722-21apt-config19:57:282
1921428118,47sleep319250-21fschecks_count22:57:473
1225528114,13sleep20-21swapper/219:12:322
1225528114,13sleep20-21swapper/219:12:322
773528020,44sleep37745-21unin-run22:22:273
56128025,26sleep00-21swapper/020:17:280
1535828021,38sleep315385-21apt-get19:22:263
1403428021,37sleep214072-21munin-run22:42:282
1143828013,52sleep011490-21ls20:52:270
1143828013,52sleep011490-21ls20:52:270
308627918,34sleep23221-21grep20:22:592
2519727927,23sleep30-21swapper/323:17:283
2519727927,23sleep30-21swapper/323:17:283
233362798,14sleep20-21swapper/219:46:592
2218627916,48sleep322089-21/usr/sbin/munin19:42:443
2187027912,51sleep321948-21sed19:42:283
1698227916,29sleep20-21swapper/221:07:512
2511627821,34sleep225137-21munin-run23:17:272
2511627821,34sleep225137-21munin-run23:17:272
874227723,26sleep30-21swapper/300:07:293
1919927716,45sleep019256-21/usr/sbin/munin19:32:480
2438527617,45sleep024464-21timerandwakeup23:12:570
2438527617,45sleep024464-21timerandwakeup23:12:570
735627514,13sleep20-21swapper/200:02:322
1591527514,47sleep015956-21users21:03:010
1109327510,14sleep20-21swapper/222:32:432
48127416,41sleep2483-21cron20:17:262
2581827414,45sleep025823-21/usr/sbin/munin23:17:540
1888227420,28sleep00-21swapper/022:57:270
1089927418,21sleep20-21swapper/220:48:012
1089927418,21sleep20-21swapper/220:48:012
1454527324,28sleep00-21swapper/022:42:460
141292730,9sleep30-21swapper/322:42:283
1275727321,29sleep00-21swapper/022:37:460
1084427315,41sleep310875-21sed22:32:283
1084427315,41sleep310875-21sed22:32:283
2064527213,42sleep220653-21/usr/sbin/munin21:18:002
2548327115,40sleep325572-21load19:52:493
88772700,7sleep00-21swapper/000:07:290
39062708,14sleep20-21swapper/220:27:432
291327018,29sleep20-21swapper/223:47:462
2262927025,28sleep30-21swapper/323:07:523
2254527018,35sleep322565-21latency_hist21:27:263
124922709,12sleep30-21swapper/319:12:373
124922709,12sleep30-21swapper/319:12:373
1209827018,29sleep00-21swapper/019:12:300
1209827018,29sleep00-21swapper/019:12:300
1539426922,24sleep00-21swapper/019:22:270
719426818,21sleep20-21swapper/222:18:022
547726821,26sleep30-21swapper/322:13:353
2259926811,41sleep222741-21latency_hist21:27:292
2763226727,19sleep30-21swapper/319:58:303
2136426717,28sleep30-21swapper/323:02:593
992526620,22sleep20-21swapper/200:08:572
740126623,26sleep30-21swapper/320:37:533
740126623,26sleep30-21swapper/320:37:533
70772667,48sleep30-21swapper/300:02:083
3104326614,23sleep00-21swapper/023:32:590
2419626624,21sleep30-21swapper/319:47:573
2039326613,36sleep220424-21awk23:02:262
1816326619,23sleep30-21swapper/321:12:453
1727326614,28sleep30-21swapper/300:33:003
1637526617,32sleep216376-21ntp_states19:23:102
1480026611,41sleep314809-21systemd-run21:02:273
3252426517,33sleep332561-21proc_pri23:37:553
3245726516,24sleep00-21swapper/023:37:520
2741926521,22sleep00-21swapper/023:22:460
2366226517,30sleep023656-21apt-get19:47:420
2223126517,30sleep022226-21apt-get23:07:410
2220726522,26sleep222226-21apt-get23:07:382
711926413,23sleep20-21swapper/220:37:482
711926413,23sleep20-21swapper/220:37:482
3132426424,23sleep231409-21grep21:52:562
277126415,26sleep30-21swapper/322:07:253
2320826415,14sleep00-21swapper/021:27:510
521226312,37sleep35354-21latency_hist20:32:293
253926314,34sleep32600-21cat20:22:483
107262637,11sleep00-21swapper/022:31:280
107262637,11sleep00-21swapper/022:31:280
2983626217,22sleep30-21swapper/320:07:263
2745526223,23sleep20-21swapper/221:42:262
2745526223,23sleep20-21swapper/221:42:262
1494126210,36sleep215007-21/usr/sbin/munin19:18:012
127412626,47sleep20-21swapper/220:54:522
903726118,22sleep00-21swapper/000:07:460
556226114,18sleep00-21swapper/022:13:360
213826121,22sleep00-21swapper/023:43:530
1576626119,27sleep20-21swapper/222:47:282
699526018,22sleep00-21swapper/022:18:000
1876426013,30sleep218920-21sort19:32:302
1218626019,20sleep30-21swapper/320:52:593
1183726015,19sleep00-21swapper/000:17:270
79612599,20sleep20-21swapper/222:22:382
111932596,12sleep30-21swapper/300:12:583
367825815,24sleep23800-21sed22:07:572
367825815,24sleep23800-21sed22:07:572
3012925816,23sleep30-21swapper/320:07:433
2155425816,22sleep00-21swapper/021:22:470
196025814,25sleep32058-21users23:42:593
1333825817,27sleep213375-21modprobe00:22:252
1243225817,20sleep30-21swapper/322:37:283
1096725823,22sleep00-21swapper/019:08:100
3097625720,23sleep30-21swapper/323:32:583
2736825716,24sleep327417-21irqstats23:22:453
270125717,25sleep32697-21apt-get23:47:393
1981925713,26sleep319826-21diskstats21:17:453
902425612,23sleep30-21swapper/300:07:453
710725614,18sleep00-21swapper/020:37:460
710725614,18sleep00-21swapper/020:37:460
603725613,27sleep20-21swapper/223:57:472
2771825622,19sleep00-21swapper/021:42:360
2771825622,19sleep00-21swapper/021:42:360
2756425612,27sleep20-21swapper/219:58:292
235425615,24sleep02349-21apt-get20:22:410
1821525615,25sleep018239-21df_inode21:12:460
1648125610,32sleep016546-21sendmail_mailqu22:47:550
1517225614,26sleep315168-21apt-get00:27:393
929925513,22sleep00-21swapper/022:27:280
691825510,33sleep0364-21snmpd00:00:030
424925514,23sleep34245-21apt-get23:52:403
1846525511,29sleep217528-21/usr/sbin/munin22:53:012
1744025513,23sleep30-21swapper/319:27:433
3012825412,25sleep230122-21apt-get20:07:412
2497025412,26sleep225047-21cut19:52:282
122625416,23sleep01258-21idleruntime-cro22:02:260
3220925313,23sleep332363-21sed21:57:293
256502537,35sleep20-21swapper/221:34:272
2038725311,19sleep00-21swapper/021:17:580
161325313,23sleep21619-21ntp_states23:42:542
140925320,18sleep20-21swapper/220:17:552
1390225312,23sleep213995-21memory00:22:502
1390225312,23sleep213995-21memory00:22:502
870325213,23sleep28784-21/usr/sbin/munin20:42:452
870325213,23sleep28784-21/usr/sbin/munin20:42:452
86725215,23sleep20-21swapper/223:42:262
2687825216,15sleep30-21swapper/321:37:593
2687825216,15sleep30-21swapper/321:37:593
1526225212,18sleep20-21swapper/200:27:462
1235025214,24sleep012376-21sendmail22:37:250
553425113,22sleep35538-21diskstats20:32:453
289672517,36sleep20-21swapper/221:46:582
289672517,36sleep20-21swapper/221:46:582
1087325113,18sleep30-21swapper/319:08:093
425125015,17sleep00-21swapper/023:52:410
3249125012,22sleep232500-21df_inode21:57:482
2607525012,22sleep226087-21ntpq23:18:122
1762425011,24sleep017686-21if_err_eth019:27:480
488624912,21sleep20-21swapper/222:13:062
2944824814,18sleep00-21swapper/021:47:450
2099824812,21sleep221020-21ntp_states19:37:552
1598824810,22sleep015997-21diskstats19:22:590
199422464,30sleep30-21swapper/319:33:323
1012524612,17sleep30-21swapper/320:47:423
1012524612,17sleep30-21swapper/320:47:423
1682324510,22sleep316587-21/usr/sbin/munin21:07:483
2861224111,14sleep00-21swapper/020:02:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional