You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 22:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Aug 16, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3179326837,664sleep031834-21ntpq08:35:080
2508126598,613sleep025089-21ntpq11:40:080
20609263359,396sleep020736-21sort11:25:140
9214256115,531sleep39228-21/usr/sbin/munin09:05:153
24515241611,352sleep024557-21ntpq09:55:080
3112924049,356sleep01-21systemd08:34:430
9812239719,363sleep19848-21ntpq07:25:101
1757423548,11sleep00-21swapper/011:15:160
1757423548,11sleep00-21swapper/011:15:160
829229751,129sleep00-21swapper/012:05:160
829122878,267sleep08191-21/usr/sbin/munin12:29:590
829122878,267sleep08191-21/usr/sbin/munin12:29:590
20529228018,231sleep00-21swapper/009:42:080
13771227782,108sleep013805-21munin-run07:39:430
13771227782,108sleep013805-21munin-run07:39:430
17870226170,74sleep017881-21sort09:34:530
5901225455,77sleep05168-21/usr/sbin/munin08:55:150
1434224369,131sleep0518-21/usr/sbin/munin08:40:160
15623222333,177sleep01-21systemd07:44:430
15623222333,177sleep01-21systemd07:44:430
236121989,172sleep00-21swapper/010:29:450
25328219210,168sleep1201-21jbd2/mmcblk2p2-09:59:091
16584218949,26sleep00-21swapper/011:14:440
1033721805,140sleep00-21swapper/007:25:210
8374217813,149sleep28412-21ntpq07:21:312
4518217138,49sleep00-21swapper/012:16:100
32666217128,113sleep00-21swapper/010:20:160
920721677,148sleep09254-21ntpq10:50:080
7010216760,69sleep07058-21wc10:44:430
2985921679,145sleep1201-21jbd2/mmcblk2p2-10:12:131
1948215717,124sleep00-21swapper/012:09:520
1948215717,124sleep00-21swapper/012:09:520
26878215416,14sleep00-21swapper/011:45:130
2128421548,135sleep021302-21kernelversion09:45:060
4731215351,63sleep04857-21sed07:10:150
21807215050,63sleep021845-21modprobe08:04:430
12276214838,52sleep012415-21latency_hist07:34:460
2669214613,114sleep00-21swapper/010:30:010
846421457,127sleep0367-21snmpd10:49:070
8082214532,54sleep00-21swapper/009:04:440
25901214511,57sleep00-21swapper/010:00:050
21966214427,48sleep00-21swapper/011:30:100
786121429,121sleep07857-21apt-get07:19:560
13016214224,52sleep00-21swapper/009:19:580
3413214025,58sleep00-21swapper/008:49:450
1285214016,18sleep30-21swapper/308:40:153
10372214025,75sleep00-21swapper/012:35:120
10372214025,75sleep00-21swapper/012:35:120
26866213915,72sleep026951-21fschecks_time08:20:050
23877213930,70sleep023943-21latency_hist09:54:440
23877213930,70sleep023943-21latency_hist09:54:440
823021387,106sleep39496-1kworker/3:2H12:29:583
823021387,106sleep39496-1kworker/3:2H12:29:583
11066213857,63sleep00-21swapper/010:55:170
12780213572,18sleep00-21swapper/009:19:460
12780213572,18sleep00-21swapper/009:19:460
391721329,77sleep00-21swapper/008:50:060
391721329,77sleep00-21swapper/008:50:060
2227821289,54sleep00-21swapper/008:05:030
35021260,12sleep10-21swapper/110:20:161
30763212423,82sleep330785-21apt11:59:413
30763212423,82sleep330785-21apt11:59:413
6686212317,41sleep00-21swapper/007:15:150
23534212311,63sleep00-21swapper/011:35:110
23534212311,63sleep00-21swapper/011:35:110
7963212110,74sleep07978-21munin-run12:29:430
7963212110,74sleep07978-21munin-run12:29:430
27655212051,54sleep027701-21ntpq10:05:100
20627211921,79sleep120660-21fschecks_count08:00:031
3725211816,87sleep13794-21fschecks_time08:50:031
3725211816,87sleep13794-21fschecks_time08:50:031
28832211855,49sleep0289-21dbus-daemon08:25:140
6282211515,78sleep36356-21cat10:40:143
30841211419,68sleep330916-21mailstats10:15:143
25935211414,84sleep326028-21systemd-run11:44:433
11521211020,75sleep211609-21tune2fs09:15:042
11521211020,75sleep211609-21tune2fs09:15:042
5391210811,55sleep05427-21awk10:39:420
5205210815,15sleep30-21swapper/312:20:013
16848210812,17sleep10-21swapper/109:30:131
15247210845,49sleep015283-21ntpq09:25:440
2363721068,30sleep00-21swapper/008:09:560
3766210310,74sleep13802-21/usr/sbin/munin12:15:061
7393210218,60sleep17480-21apt-config07:19:451
5461210217,71sleep35507-21sed08:55:063
28230210214,73sleep128277-21cat11:50:061
28797210126,63sleep028845-21awk10:09:580
19927210135,11sleep00-21swapper/007:55:540
1662421018,75sleep00-21swapper/007:45:160
3503210016,65sleep23504-21/usr/sbin/munin12:14:552
26157210022,61sleep326181-21proc_pri10:00:123
10154210026,60sleep00-21swapper/009:10:050
2926929922,14sleep00-21swapper/011:54:460
906629818,64sleep29213-21awk09:05:142
316732976,77sleep00-21swapper/012:00:150
316732976,77sleep00-21swapper/012:00:150
1110229728,46sleep311153-21latency_hist12:39:433
1110229728,46sleep311153-21latency_hist12:39:433
558329665,18sleep05097-21/usr/sbin/munin12:20:100
2766929521,57sleep127692-21idleruntime-cro11:49:421
1119229547,24sleep30-21swapper/309:14:443
1539329430,40sleep215476-21tune2fs11:10:012
2110829325,41sleep121148-21cut09:45:031
2430729120,50sleep2492-21cron11:39:412
2430729120,50sleep2492-21cron11:39:412
2233129117,59sleep222435-21latency_hist09:49:452
2678228917,57sleep126852-21/usr/sbin/munin08:20:021
251702898,63sleep00-21swapper/008:14:560
1755628920,12sleep00-21swapper/009:34:040
297792887,37sleep00-21swapper/011:55:060
297792887,37sleep00-21swapper/011:55:060
1418328837,36sleep00-21swapper/011:05:110
130942885,72sleep00-21swapper/011:00:220
490028717,39sleep35047-21grep12:19:443
1459928724,38sleep314287-21apt-get09:24:463
953828618,11sleep00-21swapper/009:09:080
705928613,19sleep10-21swapper/110:44:441
1867028619,12sleep00-21swapper/007:54:370
1503828616,18sleep10-21swapper/111:09:441
780928518,47sleep37883-21switchtime10:45:153
3151028523,38sleep331599-21fschecks_time08:35:023
3123828526,26sleep20-21swapper/208:34:452
264982856,11sleep20-21swapper/211:45:022
1867828517,18sleep10-21swapper/111:20:061
1690828518,52sleep316952-21grep11:15:033
1690828518,52sleep316952-21grep11:15:033
73628418,47sleep2738-21cron10:24:422
725228414,22sleep10-21swapper/109:00:131
1872628426,31sleep30-21swapper/307:54:443
3147128315,12sleep00-21swapper/010:17:580
985028113,53sleep19925-21fschecks_time12:35:011
985028113,53sleep19925-21fschecks_time12:35:011
2793228115,47sleep228080-21unixbench_multi10:05:162
2497528121,30sleep10-21swapper/108:14:451
1301228123,36sleep113009-21apt-get09:19:551
418027919,41sleep33615-21/usr/sbin/munin08:50:133
418027919,41sleep33615-21/usr/sbin/munin08:50:133
1868227923,40sleep218725-21unin-run07:54:432
792627822,33sleep37954-21awk12:29:423
792627822,33sleep37954-21awk12:29:423
659727830,24sleep36681-21cat07:15:143
2502827815,49sleep20-21swapper/211:40:062
2300327810,48sleep30-21swapper/309:50:103
2300327810,48sleep30-21swapper/309:50:103
380527713,49sleep23861-21idleruntime08:50:042
380527713,49sleep23861-21idleruntime08:50:042
2646027726,21sleep20-21swapper/208:19:432
1006727721,37sleep210131-21systemd-run10:54:432
3106527618,38sleep20-21swapper/208:33:562
2304627614,32sleep10-21swapper/111:34:561
2304627614,32sleep10-21swapper/111:34:561
1848027621,14sleep00-21swapper/011:20:010
1607927614,47sleep116119-21/usr/sbin/munin07:45:051
787727511,51sleep17857-21apt-get07:20:051
1348827527,25sleep20-21swapper/211:04:442
927127420,28sleep30-21swapper/310:50:113
259027423,28sleep10-21swapper/107:05:031
966427315,42sleep19807-21latency_hist09:09:461
2878127317,25sleep20-21swapper/208:25:122
2150127320,34sleep121498-21apt-get11:29:551
2499027225,27sleep10-21swapper/111:40:051
206627223,32sleep22078-21fschecks_count12:10:022
206627223,32sleep22078-21fschecks_count12:10:022
149227224,29sleep31516-21munin-run12:09:433
1238627212,15sleep10-21swapper/111:00:071
618127112,46sleep2231-21systemd-journal12:22:102
1025327121,29sleep20-21swapper/209:10:072
3084627025,21sleep10-21swapper/111:59:431
3084627025,21sleep10-21swapper/111:59:431
2686927026,24sleep30-21swapper/311:45:123
1611327023,24sleep10-21swapper/109:29:451
734326923,23sleep20-21swapper/212:25:132
734326923,23sleep20-21swapper/212:25:132
339226919,36sleep33410-21ls08:49:443
2906426917,24sleep30-21swapper/310:10:053
265626915,29sleep30-21swapper/310:30:003
2257326915,38sleep122636-21/usr/sbin/munin09:50:001
2257326915,38sleep122636-21/usr/sbin/munin09:50:001
2152926918,35sleep321616-21basename09:45:143
1917126912,37sleep319221-21/usr/sbin/munin07:55:043
1591726915,26sleep10-21swapper/111:10:131
1013226921,26sleep20-21swapper/212:35:062
1013226921,26sleep20-21swapper/212:35:062
74026811,44sleep3786-21unin-run10:24:433
1180326814,34sleep30-21swapper/309:15:093
1180326814,34sleep30-21swapper/309:15:093
1006326815,36sleep110065-21cron10:54:421
2078626718,34sleep320934-21date09:44:453
1923126715,37sleep319267-21latency_hist09:39:443
1512426717,22sleep10-21swapper/107:41:461
1512426717,22sleep10-21swapper/107:41:461
94132666,18sleep10-21swapper/112:33:561
94132666,18sleep10-21swapper/112:33:561
738226621,24sleep30-21swapper/309:00:153
3273626615,36sleep1370-21/usr/sbin/munin12:05:041
2431226619,29sleep3307-21systemd-logind11:39:423
2431226619,29sleep3307-21systemd-logind11:39:423
1402926617,25sleep30-21swapper/307:39:463
110382666,11sleep30-21swapper/307:30:003
2276026521,23sleep20-21swapper/209:50:052
2276026521,23sleep20-21swapper/209:50:052
2053726522,22sleep30-21swapper/311:25:133
1709126518,28sleep217127-21latency11:15:062
1709126518,28sleep217127-21latency11:15:062
1642526525,18sleep20-21swapper/207:45:142
100126520,28sleep21020-21apt-get10:24:542
3042826412,30sleep10-21swapper/110:15:041
2878926412,39sleep328828-21ps08:25:133
2724926417,30sleep127243-21apt-get10:04:581
2121326415,30sleep121255-21systemd-run11:29:431
724426320,21sleep10-21swapper/112:25:111
724426320,21sleep10-21swapper/112:25:111
2654126316,25sleep10-21swapper/108:19:461
3202926221,23sleep20-21swapper/212:00:412
3202926221,23sleep20-21swapper/212:00:412
318362629,13sleep10-21swapper/108:35:091
3176526220,24sleep131891-21/usr/sbin/munin12:00:161
3176526220,24sleep131891-21/usr/sbin/munin12:00:161
3010326221,22sleep20-21swapper/210:14:452
2761826222,19sleep30-21swapper/310:05:093
2719626215,22sleep20-21swapper/208:20:112
2577826214,26sleep20-21swapper/210:00:032
1497026221,23sleep30-21swapper/309:25:393
950426122,19sleep30-21swapper/307:25:043
890726118,20sleep20-21swapper/212:30:132
890726118,20sleep20-21swapper/212:30:132
619526117,16sleep20-21swapper/210:40:132
381926119,25sleep23841-21ls07:09:442
2409026113,33sleep224092-21acpi09:54:542
2340626116,20sleep10-21swapper/108:09:441
226426120,28sleep32347-21sed12:10:153
226426120,28sleep32347-21sed12:10:153
1838426114,19sleep20-21swapper/211:19:552
1703326119,26sleep117090-21irqstats11:15:051
1703326119,26sleep117090-21irqstats11:15:051
1498426121,25sleep315015-21munin-run11:09:423
1170526119,23sleep111833-21sed07:30:151
2626926016,19sleep10-21swapper/110:00:141
1595626019,21sleep30-21swapper/307:45:033
1429726014,26sleep314316-21apt-get07:39:543
1429726014,26sleep314316-21apt-get07:39:543
1388626018,19sleep20-21swapper/207:39:452
1388626018,19sleep20-21swapper/207:39:452
136622605,12sleep30-21swapper/307:38:203
1246026012,20sleep30-21swapper/311:00:093
1204226012,33sleep312071-21latency_hist10:59:573
476525921,22sleep14882-21grep10:35:151
406925914,17sleep10-21swapper/107:09:551
300125926,19sleep20-21swapper/210:30:072
2804725915,22sleep20-21swapper/211:50:032
257025916,25sleep12567-21apt-get10:29:541
233025915,31sleep32357-21latency_hist10:29:443
218525917,19sleep20-21swapper/208:45:032
1760925914,22sleep20-21swapper/209:34:422
1699625915,19sleep30-21swapper/309:30:153
855225812,20sleep30-21swapper/307:21:363
519425813,28sleep15261-21df08:54:591
459425818,25sleep24616-21pluginstate10:35:122
3080925818,20sleep20-21swapper/210:15:132
274025814,27sleep10-21swapper/108:45:141
2544625816,26sleep30-21swapper/309:59:433
2185825823,17sleep20-21swapper/208:04:452
188392587,32sleep20-21swapper/209:35:392
1816925811,20sleep10-21swapper/109:35:061
145025817,22sleep20-21swapper/212:09:422
748125713,26sleep27654-21cut07:19:462
3024025712,21sleep10-21swapper/108:30:081
2193425710,33sleep221957-21ntp_states11:30:082
2158725719,24sleep321594-21/usr/sbin/munin11:30:013
1547825711,19sleep20-21swapper/209:25:502
409225613,25sleep34088-21apt-get07:09:573
3014025618,24sleep30-21swapper/308:30:063
210142566,11sleep20-21swapper/211:27:092
171725614,22sleep10-21swapper/112:09:451
171725614,22sleep10-21swapper/112:09:451
1117925616,20sleep20-21swapper/210:55:192
1947425515,18sleep10-21swapper/109:39:591
1382825517,22sleep113871-21munin-run07:39:441
1382825517,22sleep113871-21munin-run07:39:441
725725417,16sleep10-21swapper/110:45:011
2704825414,16sleep30-21swapper/308:20:083
250325415,23sleep32501-21apt-get07:04:573
1993825419,17sleep20-21swapper/209:40:122
1202025417,18sleep10-21swapper/110:59:551
1051825414,24sleep110629-21timerwakeupswit09:10:141
764725324,15sleep20-21swapper/210:45:102
3148425318,16sleep30-21swapper/312:00:103
3148425318,16sleep30-21swapper/312:00:103
1896525313,23sleep118959-21apt-get07:54:581
96325212,22sleep1997-21ntp_states08:40:091
540225218,16sleep20-21swapper/207:14:452
540225214,21sleep25460-21irqstats08:55:052
2859425211,19sleep10-21swapper/108:25:071
1565925211,28sleep315698-21tr11:10:063
146612526,35sleep20-21swapper/211:05:332
2876625114,21sleep328863-21users11:50:163
2584925117,19sleep325967-21wc08:15:163
1971625115,21sleep319781-21latency_hist11:24:443
1739325116,21sleep117409-21awk07:49:531
130932517,18sleep20-21swapper/209:20:002
511825012,22sleep15115-21apt-get12:19:541
202942508,30sleep220320-21latency_hist07:59:442
1438925014,23sleep20-21swapper/207:40:462
1438925014,23sleep20-21swapper/207:40:462
503024914,21sleep35043-21vmstat10:35:173
2477624914,18sleep20-21swapper/208:13:092
21352495,33sleep10-21swapper/110:27:481
1183424910,15sleep20-21swapper/207:30:162
1067624811,21sleep210707-21munin-run07:29:432
718624711,17sleep20-21swapper/209:00:092
250624715,15sleep00-21swapper/007:04:580
1189024712,19sleep111896-21ntp_states09:15:111
1189024712,19sleep111896-21ntp_states09:15:111
99824610,13sleep20-21swapper/208:40:102
220962469,19sleep30-21swapper/308:04:583
2490324512,20sleep31-21systemd08:14:433
220952437,18sleep20-21swapper/208:04:572
2053924012,14sleep30-21swapper/307:59:573
295502388,16sleep20-21swapper/211:55:002
295502388,16sleep20-21swapper/211:55:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional