You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 05:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 20, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21100253933,5379sleep25086-21kworker/2:019:35:532
32310213917,1372sleep030872-21kworker/0:020:11:530
17494262538,570sleep117640-21gpgv21:08:261
17494262538,570sleep117640-21gpgv21:08:261
10490254913,522sleep210498-21ntpq19:03:512
4170246311,436sleep04209-21ntpq23:48:590
30560237311,294sleep130562-21apt-get21:48:361
1983123338,287sleep119867-21ntpq21:13:511
9858231710,293sleep29859-21sshd00:07:022
25771231513,289sleep322660-21kworker/3:023:16:513
377422929,269sleep13773-21apt-get23:48:351
247052291109,125sleep124721-21fschecks_count23:13:421
270022284106,163sleep127095-21latency_hist21:38:261
270022284106,163sleep127095-21latency_hist21:38:261
2566228270,32sleep10-21swapper/122:03:261
13474224360,23sleep10-21swapper/122:38:261
121502243102,103sleep112295-21unixbench_singl20:48:561
13520224272,26sleep10-21swapper/100:18:441
29783222434,172sleep129849-21idleruntime20:03:541
25425221523,173sleep125531-21latency_hist21:33:261
2988221232,30sleep10-21swapper/122:03:461
12347221112,181sleep312416-21/usr/sbin/munin00:13:533
10064221147,80sleep10-21swapper/100:08:261
23792221044,76sleep10-21swapper/119:43:471
10036220427,139sleep110097-21if_eth020:43:441
10036220427,139sleep110097-21if_eth020:43:441
27968220228,126sleep10-21swapper/123:23:361
11901220255,86sleep111963-21grep00:13:431
10329219656,25sleep10-21swapper/122:28:251
12181219115,15sleep10-21swapper/119:08:541
12181219115,15sleep10-21swapper/119:08:541
15249218813,159sleep315305-21fschecks_time19:18:453
10579217916,97sleep10-21swapper/122:28:411
7893217411,118sleep18031-21grep23:58:571
28907217345,91sleep128980-21fschecks_time21:43:441
1956821699,120sleep119650-21apt-config19:33:251
710621646,101sleep10-21swapper/120:33:541
2885521648,15sleep20-21swapper/221:43:402
19736216414,133sleep219676-21latency_hist22:58:252
19736216414,133sleep219676-21latency_hist22:58:252
6832216348,76sleep16903-40mandb23:58:251
415216318,124sleep2555-21grep21:54:102
981216111,104sleep10-21swapper/120:14:001
16941215950,73sleep10-21swapper/122:48:431
16941215950,73sleep10-21swapper/122:48:431
16790215929,93sleep116796-21cat19:23:441
22834215818,119sleep222856-21kernelversion21:23:462
1868421540,10sleep10-21swapper/121:08:581
1868421540,10sleep10-21swapper/121:08:581
2673215010,90sleep10-21swapper/123:43:551
21427215011,98sleep121561-21apt-key19:38:261
3800214912,123sleep33801-21cpu23:48:503
23178214828,106sleep123323-21latency_hist19:43:271
19296214810,123sleep319321-21/usr/sbin/munin00:38:243
11843214619,112sleep311869-21awk22:33:233
31875214311,93sleep10-21swapper/123:33:541
31875214311,93sleep10-21swapper/123:33:541
16617214215,109sleep10-21swapper/122:48:271
13660214221,34sleep30-21swapper/319:13:453
365321396,12sleep20-21swapper/220:23:462
16416213918,101sleep116427-21cron19:23:241
31825213210,102sleep00-21swapper/023:33:530
31825213210,102sleep00-21swapper/023:33:530
5059213056,59sleep15170-21tail22:08:551
14940213053,35sleep10-21swapper/100:23:361
19009212738,25sleep10-21swapper/122:53:561
19009212738,25sleep10-21swapper/122:53:561
2974421258,13sleep10-21swapper/123:28:281
2963121207,14sleep00-21swapper/020:03:450
23250212024,72sleep323389-21grep21:23:553
3894211913,65sleep13972-21awk20:23:531
19976211644,46sleep10-21swapper/122:58:411
876221157,74sleep10-21swapper/120:38:511
2647721158,72sleep126555-21sed23:18:471
31271211421,72sleep331362-21fschecks_time20:08:493
2864121149,61sleep10-21swapper/119:58:531
1741021127,94sleep113085-21kworker/u8:200:29:011
2307211123,50sleep10-21swapper/120:18:521
445210913,23sleep3512-21df23:38:403
445210913,23sleep3512-21df23:38:403
18291210917,75sleep20-21swapper/221:08:522
18291210917,75sleep20-21swapper/221:08:522
31424210821,71sleep331516-21sort23:33:433
31424210821,71sleep331516-21sort23:33:433
11619210828,36sleep00-21swapper/019:08:370
11619210828,36sleep00-21swapper/019:08:370
3117121077,87sleep131166-21apt-get20:08:391
2514210729,59sleep32538-21modprobe22:03:243
10666210510,79sleep210702-21fschecks_count22:28:462
650221047,64sleep15871-21/usr/sbin/munin22:13:561
650221047,64sleep15871-21/usr/sbin/munin22:13:561
32760210418,41sleep10-21swapper/121:54:091
22023210410,79sleep122099-21mailstats23:03:531
875321037,61sleep18788-21systemd-run22:23:241
21823210318,67sleep221318-21apt-get19:38:352
27326210210,52sleep10-21swapper/121:38:431
3385210114,70sleep33275-21latency_hist23:48:253
23155210117,25sleep00-21swapper/019:43:260
12931210110,48sleep10-21swapper/120:53:371
16010210018,30sleep00-21swapper/021:03:330
16010210018,30sleep00-21swapper/021:03:330
1999529914,71sleep220056-21/usr/sbin/munin21:13:562
798629820,61sleep28033-21sendmail20:38:242
2462229614,66sleep224683-21cpuspeed23:13:392
1477829623,31sleep10-21swapper/120:58:491
1477829623,31sleep10-21swapper/120:58:491
7092959,16sleep10-21swapper/123:38:461
7092959,16sleep10-21swapper/123:38:461
61072957,12sleep00-21swapper/022:13:460
61072957,12sleep00-21swapper/022:13:460
2658329519,59sleep226675-21perl19:53:462
2649429542,13sleep10-21swapper/119:53:401
2438029516,58sleep324096-21/usr/sbin/munin21:28:473
1803829523,33sleep10-21swapper/100:33:381
497529321,31sleep10-21swapper/120:28:411
497529321,31sleep10-21swapper/120:28:411
2129629325,45sleep321318-21apt-get19:38:243
1965729315,59sleep019682-21munin-run22:58:230
1965729315,59sleep019682-21munin-run22:58:230
788829216,59sleep37933-21ntp_states22:18:503
207842929,67sleep320809-21latency_hist21:18:263
1471329218,10sleep10-21swapper/119:16:461
2975029115,57sleep229270-21apt-get23:28:332
1953729015,62sleep019635-21cut21:13:460
1368429010,65sleep113002-21munin-node22:38:341
249222897,15sleep10-21swapper/119:48:361
2350828916,57sleep023584-21sort19:43:420
2314928923,45sleep10-21swapper/123:08:421
182128915,55sleep01823-21/usr/sbin/munin20:18:350
1801328914,55sleep218080-21latency_hist19:28:252
1564428912,64sleep015706-21sendmail_mailst19:18:550
1417728913,22sleep00-21swapper/022:38:530
2796728818,50sleep027480-21apt-get23:23:350
841728611,13sleep00-21swapper/022:19:150
402028613,58sleep0225-21systemd-journal22:07:530
1059128518,42sleep010740-21grep20:43:550
1059128518,42sleep010740-21grep20:43:550
225662849,14sleep10-21swapper/121:23:371
211312847,17sleep121233-21perl21:18:481
1816128436,29sleep30-21swapper/322:53:263
1816128436,29sleep30-21swapper/322:53:263
494328315,51sleep04973-21cpu20:28:400
494328315,51sleep04973-21cpu20:28:400
3272428310,57sleep00-21swapper/020:13:420
3021528313,55sleep030418-21latency_hist21:48:270
3010728313,53sleep330210-21date21:48:263
1776828313,41sleep00-21swapper/022:48:590
1776828313,41sleep00-21swapper/022:48:590
795728223,25sleep20-21swapper/222:18:532
103372826,14sleep10-21swapper/100:08:421
103372826,14sleep10-21swapper/100:08:421
723328114,52sleep37239-21diskstats23:58:433
316012810,6sleep20-21swapper/223:33:452
316012810,6sleep20-21swapper/223:33:452
1617928118,44sleep316193-21grep21:03:483
1617928118,44sleep316193-21grep21:03:483
1578028014,49sleep215901-21/usr/sbin/munin19:18:572
3098327915,46sleep331015-21awk21:49:193
3054227917,24sleep00-21swapper/021:48:350
291127915,46sleep01973-21/usr/sbin/munin23:43:580
2612727919,28sleep00-21swapper/021:33:530
2612727919,28sleep00-21swapper/021:33:530
510727820,29sleep20-21swapper/223:53:252
1567327816,40sleep315817-21grep00:23:553
962327717,33sleep29707-21awk22:23:542
560727710,42sleep20-21swapper/222:13:242
54142779,50sleep05060-21apt-get23:53:270
1471127723,26sleep30-21swapper/300:23:263
1241827716,45sleep012502-21timerandwakeup00:13:540
1049927723,30sleep310009-21/usr/sbin/munin19:03:523
587027617,28sleep20-21swapper/222:13:362
587027617,28sleep20-21swapper/222:13:362
1523127623,36sleep315385-21sed20:58:593
1523127623,36sleep315385-21sed20:58:593
1370727611,32sleep30-21swapper/322:38:363
127652760,5sleep20-21swapper/222:33:542
1218327615,25sleep20-21swapper/200:13:482
2155627524,26sleep30-21swapper/321:18:583
3039727411,48sleep030513-21sed23:28:560
488227320,28sleep30-21swapper/322:08:533
2210027321,25sleep20-21swapper/223:03:542
1764227321,25sleep20-21swapper/221:08:272
554527213,43sleep35622-21grep20:28:563
554527213,43sleep35622-21grep20:28:563
2946427224,20sleep30-21swapper/323:28:263
1694227215,22sleep00-21swapper/019:23:470
1236327227,23sleep30-21swapper/319:08:563
1236327227,23sleep30-21swapper/319:08:563
3088327116,28sleep30-21swapper/320:08:243
24222715,54sleep00-21swapper/022:02:120
1731527012,35sleep20-21swapper/222:48:532
1731527012,35sleep20-21swapper/222:48:532
949426914,15sleep00-21swapper/022:23:520
2293726918,32sleep022543-21/usr/sbin/munin21:23:480
209326916,38sleep22180-21fschecks_time23:43:442
1339626915,27sleep00-21swapper/000:18:390
1042026918,31sleep210471-21ls20:43:532
1042026918,31sleep210471-21ls20:43:532
691526817,29sleep20-21swapper/223:58:262
3257626816,36sleep032599-21ntp_states21:54:040
1740326811,27sleep00-21swapper/021:08:250
1740326811,27sleep00-21swapper/021:08:250
956126712,39sleep29602-21systemd-run20:43:242
3144626716,19sleep20-21swapper/220:08:522
2216226717,26sleep30-21swapper/323:03:553
1774926718,33sleep317768-21munin-run00:33:243
103012679,16sleep30-21swapper/320:43:503
103012679,16sleep30-21swapper/320:43:503
56302668,16sleep30-21swapper/323:54:233
258632667,49sleep00-21swapper/023:18:010
2486326619,26sleep20-21swapper/221:28:592
1494126618,28sleep214935-21apt-get00:23:372
1492026622,25sleep00-21swapper/019:18:260
79682659,44sleep00-21swapper/020:38:090
2785426518,22sleep00-21swapper/021:38:540
2667126515,35sleep326678-21/usr/sbin/munin23:18:533
2147626520,23sleep00-21swapper/021:18:550
126272658,14sleep00-21swapper/022:33:530
1250026521,27sleep312529-21/usr/sbin/munin22:33:483
722526413,35sleep27232-21df_inode23:58:422
239022647,12sleep20-21swapper/219:43:522
2189726410,16sleep00-21swapper/023:03:480
1871526415,25sleep20-21swapper/219:28:542
1271826415,23sleep30-21swapper/320:53:263
1157326414,25sleep00-21swapper/000:13:260
1157326414,25sleep00-21swapper/000:13:260
1043526426,21sleep00-21swapper/019:03:490
97352634,50sleep00-21swapper/000:05:260
2938626314,25sleep20-21swapper/220:03:272
2732726322,23sleep30-21swapper/319:53:593
2541026319,27sleep325443-21/usr/sbin/munin19:48:543
2135926319,21sleep00-21swapper/019:38:250
2852726220,27sleep328559-21/usr/sbin/munin19:58:513
2307626216,23sleep30-21swapper/323:08:403
61812614,45sleep20-21swapper/220:33:112
61812614,45sleep20-21swapper/220:33:112
288826113,24sleep30-21swapper/322:03:443
2649326116,27sleep026488-21apt-get19:53:390
1577326114,20sleep30-21swapper/322:43:533
1186126117,24sleep011867-21ntp_states20:48:510
82782606,42sleep00-21swapper/000:01:350
291782608,11sleep20-21swapper/223:27:242
2811026014,30sleep228143-21users21:38:562
2039826015,30sleep320427-21ntp_states22:58:513
1189226014,31sleep211974-21grep20:48:532
55925913,25sleep30-21swapper/321:54:113
170225912,32sleep21708-21ntp_states21:58:512
148652595,11sleep00-21swapper/000:23:270
1011125916,27sleep110117-21diskstats19:03:421
960325817,21sleep30-21swapper/320:43:253
875425813,28sleep08761-21/usr/sbin/munin20:38:500
723425816,26sleep37241-21/usr/sbin/munin22:18:263
723425816,26sleep37241-21/usr/sbin/munin22:18:263
205325819,23sleep20-21swapper/220:18:452
1389625811,16sleep20-21swapper/200:18:532
2629425714,19sleep20-21swapper/223:18:442
219742578,36sleep20-21swapper/221:19:302
2562225617,22sleep20-21swapper/219:48:572
2344425613,29sleep023490-21ntpq23:08:490
1877125615,22sleep30-21swapper/322:53:503
1877125615,22sleep30-21swapper/322:53:503
148062565,12sleep20-21swapper/220:58:502
148062565,12sleep20-21swapper/220:58:502
480225512,25sleep04808-21ntp_states22:08:500
44425515,24sleep2438-21apt-get23:38:392
44425515,24sleep2438-21apt-get23:38:392
242062557,29sleep00-21swapper/021:28:440
2307525513,24sleep223069-21apt-get23:08:392
2132425511,20sleep20-21swapper/223:03:262
181952555,12sleep20-21swapper/200:33:432
1170525513,17sleep20-21swapper/219:08:432
1170525513,17sleep20-21swapper/219:08:432
112625512,25sleep21172-21users20:14:012
902725415,24sleep39022-21apt-get22:23:383
669925413,25sleep36721-21dpkg20:33:363
192325414,24sleep31929-21diskstats20:18:423
1855925412,21sleep00-21swapper/021:08:580
1855925412,21sleep00-21swapper/021:08:580
1848525415,24sleep00-21swapper/000:33:490
176232545,10sleep00-21swapper/000:31:470
1694625416,22sleep216952-21ntp_states00:28:502
228852536,16sleep30-21swapper/319:40:303
1951225311,17sleep00-21swapper/019:33:240
1789125314,23sleep317884-21apt-get21:08:403
1789125314,23sleep317884-21apt-get21:08:403
1484025313,20sleep20-21swapper/219:18:242
1023125311,23sleep20-21swapper/200:08:352
1023125311,23sleep20-21swapper/200:08:352
45572526,34sleep00-21swapper/020:27:110
339325212,23sleep33388-21apt-get20:23:393
3199625213,23sleep231953-21apt-get21:53:502
3000925212,24sleep330050-21irqstats23:28:473
2985125215,23sleep229904-21irqstats20:03:552
291392525,36sleep00-21swapper/020:00:540
2626325218,18sleep00-21swapper/019:53:270
152742526,34sleep00-21swapper/022:43:300
1062425218,18sleep30-21swapper/300:08:483
1062425218,18sleep30-21swapper/300:08:483
2765425112,23sleep327691-21ntp_states21:38:493
2760925111,24sleep027800-21apt-config23:23:260
1429525117,16sleep30-21swapper/320:58:273
1428925111,27sleep214361-21switchtime22:38:562
1055225114,16sleep00-21swapper/022:28:360
2611825015,19sleep326124-21ntp_states21:33:523
2611825015,19sleep326124-21ntp_states21:33:523
2573925013,22sleep225748-21diskstats21:33:432
2573925013,22sleep225748-21diskstats21:33:432
1670325012,22sleep316700-21apt-get19:23:383
153072509,24sleep20-21swapper/222:43:402
32924911,23sleep3338-21diskstats20:13:453
2058924910,19sleep00-21swapper/022:58:560
137612498,19sleep20-21swapper/220:53:582
1295624911,21sleep012950-21apt-get20:53:400
2348124712,22sleep323504-21cut19:43:413
176124613,17sleep00-21swapper/023:43:250
176124613,17sleep00-21swapper/023:43:250
1910824511,20sleep319156-21users19:28:593
304224412,19sleep23075-21irqstats22:03:472
1609424210,18sleep20-21swapper/221:03:432
911624011,15sleep30-21swapper/300:03:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional