You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 17:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Aug 04, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
187832758816,7556sleep015762-21kworker/0:009:51:280
29240261423,577sleep029233-21apt-get12:07:430
20517235825,314sleep120593-21/usr/sbin/munin11:38:001
29019231959,208sleep329166-21latency_hist12:07:303
12694230327,129sleep30-21swapper/309:32:303
16585229012,163sleep316642-21latency_hist08:02:293
16585229012,163sleep316642-21latency_hist08:02:293
3272022798,233sleep332603-21/usr/sbin/munin08:52:493
31744224784,58sleep30-21swapper/308:48:003
3088422418,42sleep30-21swapper/310:27:583
6315223121,191sleep06461-21unixbench_multi10:53:020
447221821,187sleep3364-21snmpd10:34:443
27952221555,143sleep328086-21grep10:18:013
3971221431,168sleep23972-21sshd12:28:282
21539221217,181sleep221571-21tune2fs11:42:462
17929220529,96sleep317867-21munin-run11:32:293
22896220010,177sleep022932-21ntpq10:02:510
207692190130,17sleep30-21swapper/308:12:593
16339219062,48sleep30-21swapper/309:42:523
9130218413,153sleep09136-21df_inode07:37:470
9424217434,81sleep30-21swapper/307:37:543
3235721726,43sleep332373-21awk12:17:393
30550217111,139sleep330696-21latency_hist12:12:303
19669217027,80sleep319791-21latency_hist11:37:303
10085216926,106sleep310226-21latency_hist11:07:303
11914216817,129sleep212039-21latency_hist07:47:302
24641216615,25sleep30-21swapper/311:52:483
2323721669,145sleep323370-21grep10:02:593
1499421629,21sleep30-21swapper/311:22:443
1499421629,21sleep30-21swapper/311:22:443
4737216134,89sleep34794-21ls12:32:493
5639215910,133sleep35670-21idleruntime-cro07:27:283
28372215810,88sleep30-21swapper/308:37:553
26808215818,120sleep326928-21timerwakeupswit11:58:013
3488215538,72sleep33529-21proc_pri12:27:573
19906215035,99sleep30-21swapper/311:37:423
15271214814,116sleep00-21swapper/007:57:420
15271214814,116sleep00-21swapper/007:57:420
4998214715,116sleep15082-21fschecks_time09:07:531
2327321466,13sleep30-21swapper/311:47:523
22234214423,42sleep122261-21munin-plugin-st10:02:281
7682213976,18sleep30-21swapper/309:17:283
25715213923,61sleep30-21swapper/308:28:023
25715213923,61sleep30-21swapper/308:28:023
1858521397,16sleep30-21swapper/308:07:383
1858521397,16sleep30-21swapper/308:07:383
12281213428,49sleep30-21swapper/307:47:483
2517213222,90sleep12590-21sshd07:17:291
13943213127,71sleep3364-21snmpd09:34:203
13943213127,71sleep3364-21snmpd09:34:203
32062212811,83sleep332065-21cron12:17:273
11862212851,42sleep311886-21cpu11:12:433
11862212851,42sleep311886-21cpu11:12:433
7481212723,85sleep27518-21cpu07:32:442
3150321268,80sleep331595-21date07:07:303
5448212550,25sleep30-21swapper/309:08:033
4078212544,39sleep30-21swapper/310:47:403
4078212544,39sleep30-21swapper/310:47:403
611212110,66sleep3623-21munin-node07:12:313
26310212118,84sleep026459-21unixbench_multi10:13:010
2591212113,90sleep22624-21cat07:17:302
31432212047,46sleep131457-21munin-plugin-st07:07:281
7287211945,32sleep30-21swapper/310:57:493
30775211910,71sleep330811-21awk08:47:283
30775211910,71sleep330811-21awk08:47:283
29034211724,75sleep129094-21latency10:22:551
20169211619,76sleep120236-21df08:12:451
13408211518,80sleep013457-21unin-run07:52:290
2346621140,13sleep10-21swapper/108:22:501
211821149,70sleep32154-21ntpq08:57:543
14700211216,80sleep214722-21basename11:22:272
7676211122,75sleep17718-21uname10:57:591
5392211026,24sleep10-21swapper/110:52:301
5392211026,24sleep10-21swapper/110:52:301
30304211013,79sleep030349-21/usr/sbin/munin10:27:430
8604210916,80sleep18681-21grep09:17:581
19122210912,80sleep019123-21sendmail-mta09:52:340
1025210936,47sleep30-21swapper/307:12:353
21663210812,79sleep021711-21if_eth011:42:480
2103210830,38sleep30-21swapper/312:23:003
10305210831,37sleep30-21swapper/311:07:383
6505210615,74sleep06514-21/usr/sbin/munin09:12:490
30318210616,30sleep230416-21users08:42:592
30318210616,30sleep230416-21users08:42:592
1073210613,74sleep01085-21/usr/sbin/munin10:37:480
26891210528,66sleep326913-21munin-run10:17:283
13422210510,19sleep10-21swapper/111:17:421
11605210517,71sleep111619-21munin-run11:12:291
4079210342,38sleep30-21swapper/309:03:033
26937210326,38sleep30-21swapper/308:32:583
871221027,12sleep30-21swapper/307:37:103
19276210214,73sleep119277-21sshd11:33:281
31192210112,76sleep231240-21cat12:12:542
1559321015,12sleep30-21swapper/309:40:173
2191629940,48sleep321428-21/usr/sbin/munin11:42:553
1822529815,47sleep331546-21cyclictest08:07:303
885629723,26sleep10-21swapper/107:37:321
605429722,34sleep30-21swapper/307:27:493
290229723,55sleep02936-21ntp_kernel_err10:42:530
154632977,13sleep00-21swapper/011:22:580
154632977,13sleep00-21swapper/011:22:580
287612968,50sleep328753-21apt-get10:22:453
2529629510,72sleep225299-21sshd10:12:022
2356929511,40sleep30-21swapper/308:22:533
1915829510,16sleep30-21swapper/309:52:453
959729317,35sleep30-21swapper/309:22:453
3247729324,48sleep032563-21tune2fs12:17:480
313629319,51sleep22508-21/usr/sbin/munin10:42:592
2465129325,15sleep30-21swapper/310:07:593
2191329312,66sleep021969-21idleruntime08:17:490
218362938,25sleep321908-21fschecks_time08:17:483
339429215,61sleep23423-21fschecks_count09:02:492
1886229128,49sleep30-21swapper/309:52:283
1699829117,56sleep217049-21idleruntime08:02:492
1402129114,38sleep30-21swapper/311:18:003
2378929021,49sleep123907-21users11:48:021
1450929016,48sleep10-21swapper/109:37:451
582928926,44sleep15847-21idleruntime-cro12:37:271
1837728919,48sleep118466-21ls08:07:321
718728824,41sleep27213-21awk07:32:282
3267928819,20sleep00-21swapper/008:52:460
328828716,49sleep13002-21/usr/sbin/munin12:27:511
142502878,32sleep30-21swapper/307:53:013
59922869,39sleep30-21swapper/310:52:533
3233228525,42sleep232357-21munin-run08:52:272
1958728519,43sleep019605-21apt11:37:270
1718228523,22sleep10-21swapper/111:27:591
414428419,34sleep20-21swapper/207:22:312
2729728319,45sleep227361-21sort10:17:472
2229128321,33sleep00-21swapper/010:02:310
123928315,49sleep11246-21latency_hist12:22:291
3225628216,30sleep20-21swapper/207:07:572
2309128216,21sleep00-21swapper/011:47:480
937128118,21sleep10-21swapper/111:03:001
2460928128,29sleep00-21swapper/008:27:300
1224828111,55sleep012283-21ntpq11:12:520
1224828111,55sleep012283-21ntpq11:12:520
837028027,41sleep38414-21ntpq09:17:523
306052809,59sleep00-21swapper/008:45:190
306052809,59sleep00-21swapper/008:45:190
2041228020,40sleep220475-21sendmail-msp09:57:282
1320828015,49sleep213353-21latency_hist11:17:302
1320828015,49sleep213353-21latency_hist11:17:302
183627929,29sleep20-21swapper/208:57:482
2493427819,35sleep20-21swapper/208:27:482
2493427819,35sleep20-21swapper/208:27:482
158332780,6sleep10-21swapper/107:57:581
158332780,6sleep10-21swapper/107:57:581
1113827815,17sleep00-21swapper/007:43:000
703527717,24sleep10-21swapper/109:13:021
209372777,32sleep30-21swapper/309:57:373
98527617,14sleep30-21swapper/310:37:433
2765227620,39sleep027668-21idleruntime-cro08:37:270
1418427525,30sleep20-21swapper/207:53:002
1283027521,37sleep212978-21grep07:47:592
78627422,26sleep20-21swapper/208:53:002
56052749,15sleep20-21swapper/210:52:402
2929927414,43sleep029370-21sensors10:23:020
2854327425,24sleep10-21swapper/110:22:311
2697127425,25sleep10-21swapper/110:17:301
2545927419,31sleep10-21swapper/108:28:001
2545927419,31sleep10-21swapper/108:28:001
3176227313,42sleep131812-21/usr/sbin/munin07:07:441
1397327319,29sleep00-21swapper/007:52:530
1087627323,25sleep10-21swapper/107:42:531
930427214,39sleep29306-21cron09:22:282
2760927214,16sleep00-21swapper/010:17:530
2898727112,14sleep20-21swapper/212:07:292
186002719,25sleep20-21swapper/208:07:412
186002719,25sleep20-21swapper/208:07:412
1558627113,40sleep214969-21/usr/sbin/munin11:23:002
1558627113,40sleep214969-21/usr/sbin/munin11:23:002
1156827114,32sleep011596-21munin-run11:12:280
3245927011,41sleep20-21swapper/212:17:462
3147527017,23sleep00-21swapper/012:13:000
3146727017,27sleep10-21swapper/108:47:541
2202827022,24sleep10-21swapper/111:42:571
202827017,29sleep20-21swapper/212:22:592
1965727022,23sleep10-21swapper/109:52:591
196527015,39sleep12027-21/usr/sbin/munin12:22:581
1918427026,25sleep10-21swapper/108:08:091
1918427026,25sleep10-21swapper/108:08:091
1801427017,33sleep017813-21apt-get11:32:300
32882698,45sleep30-21swapper/307:17:563
1896726913,24sleep20-21swapper/211:33:132
1006126914,30sleep210083-21ls11:07:292
2031526811,13sleep00-21swapper/011:37:550
1665526818,27sleep20-21swapper/209:42:592
146122689,25sleep20-21swapper/209:37:502
1241926819,28sleep10-21swapper/107:47:501
116572689,46sleep211733-21latency09:27:532
1039626818,22sleep20-21swapper/211:07:452
661426720,27sleep20-21swapper/209:12:512
1060026710,15sleep30-21swapper/307:42:433
760926615,32sleep27651-21/usr/sbin/munin10:57:562
724126611,20sleep00-21swapper/007:32:300
485026621,28sleep10-21swapper/107:22:581
456426619,29sleep04596-21awk09:07:280
314452665,37sleep10-21swapper/110:30:591
2581326618,30sleep225834-21sendmail11:57:272
229426613,14sleep20-21swapper/210:42:302
1356626618,32sleep213692-21sed09:33:012
1356626618,32sleep213692-21sed09:33:012
589926519,26sleep15918-21sed10:52:511
2960826511,40sleep129642-21ntp_kernel_err12:07:521
2876526514,36sleep228767-21cpu10:22:482
2797226517,29sleep027966-21apt-get08:37:440
26192657,22sleep32692-21grep10:42:473
2313326515,35sleep223235-21timerandwakeup10:02:572
1862226517,27sleep00-21swapper/008:07:470
1862226517,27sleep00-21swapper/008:07:470
177842655,11sleep00-21swapper/008:03:350
1135526524,21sleep10-21swapper/109:27:391
484426419,25sleep00-21swapper/007:22:570
3214226420,26sleep10-21swapper/112:17:291
306522647,37sleep20-21swapper/207:02:572
2712026413,34sleep127259-21unixbench_singl08:33:001
21322645,47sleep10-21swapper/110:41:041
1817026420,25sleep20-21swapper/209:47:592
71726313,18sleep10-21swapper/108:52:591
1988426317,28sleep219903-21apt-get11:37:412
2492226216,21sleep00-21swapper/011:52:530
951626121,20sleep20-21swapper/207:37:582
4232618,13sleep20-21swapper/210:34:272
2640326120,25sleep10-21swapper/111:57:511
24912615,15sleep00-21swapper/007:17:080
125862616,16sleep00-21swapper/009:31:460
466026018,19sleep20-21swapper/212:32:482
409226013,21sleep00-21swapper/007:22:290
2769126017,26sleep127684-21apt-get12:02:431
208632606,12sleep00-21swapper/008:13:000
176002606,16sleep20-21swapper/211:29:542
306625917,19sleep10-21swapper/110:42:581
2825625914,30sleep127947-21/usr/sbin/munin08:37:531
2745225916,19sleep00-21swapper/012:02:300
2687225920,17sleep20-21swapper/208:32:572
2326725911,36sleep2364-21snmpd08:22:352
2228525915,22sleep10-21swapper/108:17:581
189652590,8sleep10-21swapper/109:52:291
1376925913,30sleep113773-21df07:52:461
905525813,27sleep19049-21apt-get07:37:441
822125817,24sleep08270-21irqstats09:17:490
307225817,18sleep10-21swapper/109:02:321
2960225816,20sleep10-21swapper/108:42:451
2960225816,20sleep10-21swapper/108:42:451
2235625819,18sleep20-21swapper/208:17:592
1102025815,23sleep00-21swapper/009:27:300
550225717,22sleep00-21swapper/009:08:040
3079125722,17sleep00-21swapper/007:02:590
302525716,25sleep03021-21apt-get12:27:410
2589425714,27sleep126030-21latency_hist11:57:301
2584025713,25sleep225897-21irqstats10:12:512
1591125720,20sleep20-21swapper/207:58:002
1591125720,20sleep20-21swapper/207:58:002
1450825715,25sleep014502-21apt-get09:37:440
5422567,40sleep00-21swapper/010:35:580
215725611,20sleep00-21swapper/008:57:550
2106225616,23sleep20-21swapper/209:57:532
2069925614,17sleep20-21swapper/208:12:582
2843925513,28sleep228472-21ps08:37:582
1605425513,25sleep116051-21apt-get09:42:411
1550125517,19sleep10-21swapper/111:22:591
1550125517,19sleep10-21swapper/111:22:591
1418925517,24sleep114215-21munin-run09:37:281
1418925517,24sleep114215-21munin-run09:37:281
649625417,16sleep10-21swapper/107:27:591
38772546,36sleep10-21swapper/107:19:431
3213525421,20sleep132205-21cat10:32:521
3176125414,23sleep031755-21apt-get07:07:430
3080525421,18sleep20-21swapper/210:27:572
2977425414,22sleep20-21swapper/212:07:572
2474225418,20sleep00-21swapper/010:08:000
2378425417,21sleep10-21swapper/110:07:291
983425315,17sleep20-21swapper/209:22:522
884325312,23sleep08849-21diskstats11:02:470
463225318,18sleep00-21swapper/010:47:570
463225318,18sleep00-21swapper/010:47:570
265652539,23sleep20-21swapper/211:57:562
489325215,21sleep20-21swapper/207:22:592
338725214,19sleep10-21swapper/109:02:481
3187725213,24sleep031977-21users08:48:010
2813925212,23sleep00-21swapper/012:02:540
881725112,25sleep08855-21sed07:37:310
58062515,16sleep20-21swapper/209:08:222
1228525112,22sleep112314-21ntp_states11:12:531
1228525112,22sleep112314-21ntp_states11:12:531
788825015,16sleep00-21swapper/010:58:030
406425013,21sleep24075-21apt-get10:47:392
406425013,21sleep24075-21apt-get10:47:392
278924914,20sleep02859-21latency_hist12:27:290
1045324912,22sleep110500-21users09:23:031
498924811,21sleep05027-21ntp_states12:32:540
2537424811,21sleep125386-21munin-run10:12:291
1727124812,15sleep00-21swapper/009:46:560
2484624712,18sleep00-21swapper/008:27:420
2484624712,18sleep00-21swapper/008:27:420
1345024710,18sleep00-21swapper/011:17:450
1293324612,18sleep00-21swapper/009:32:430
1293324612,18sleep00-21swapper/009:32:430
2321424513,17sleep20-21swapper/211:47:512
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional