You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 21:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 20, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
207922130127,12986sleep08775-21kworker/0:212:33:350
207922130127,12986sleep08775-21kworker/0:212:33:350
179432524915,5220sleep016392-21kworker/0:210:42:350
32643219808,1959sleep021218-21kworker/0:208:03:350
6759295630,908sleep36760-21ntp_states11:48:503
27933260183,502sleep327939-21diskstats11:13:423
2914425599,535sleep029155-21ntpq09:33:490
18432244534,386sleep118585-21grep12:23:561
5737244364,359sleep15861-21timerwakeupswit08:18:541
21411242041,325sleep121539-21latency_hist10:53:261
21411242041,325sleep121539-21latency_hist10:53:261
31907235317,318sleep031912-21/usr/sbin/munin09:43:420
14319234553,278sleep114370-21cpuspeed12:13:381
2782523447,326sleep1364-21snmpd11:13:321
25867231355,183sleep10-21swapper/109:23:451
8399229826,250sleep18427-21awk10:13:221
18683229621,257sleep318721-21/usr/sbin/munin10:43:473
379022567,212sleep13820-21sed09:58:241
1630222412,58sleep10-21swapper/111:33:381
119502220129,43sleep10-21swapper/108:38:481
23262221547,111sleep123270-21sshd07:33:331
10083221560,99sleep110120-21fschecks_count08:33:431
3001421885,12sleep00-21swapper/007:53:540
3001421885,12sleep00-21swapper/007:53:540
9955218773,94sleep19958-21cron10:18:221
13096217972,86sleep113110-21munin-run10:28:231
8540217719,90sleep114775-21cyclictest08:28:471
6060217610,110sleep16171-21date11:48:231
25359217258,34sleep125497-21latency_hist09:23:241
3027321689,108sleep130276-21grep09:38:381
927321637,97sleep19420-21latency_hist11:58:251
14971216326,100sleep114587-21apt-get07:08:261
17069216219,125sleep217122-21ls07:13:402
17069216219,125sleep217122-21ls07:13:402
16424216022,118sleep316451-21munin-plugin-st10:38:223
27786215141,81sleep127912-21sed09:28:531
27786215141,81sleep127912-21sed09:28:531
9629214954,47sleep10-21swapper/111:58:331
24625214916,24sleep20-21swapper/209:18:532
26310214612,120sleep026347-21grep11:08:390
1683721456,12sleep30-21swapper/310:38:433
32284214322,40sleep10-21swapper/109:43:511
8409214214,113sleep38488-21switchtime11:53:553
22165214037,18sleep10-21swapper/112:38:251
22165214037,18sleep10-21swapper/112:38:251
19446213916,85sleep119465-21fschecks_count09:03:421
29800213817,34sleep10-21swapper/109:36:241
7238213784,14sleep10-21swapper/108:23:541
1601213612,110sleep01606-21sort11:33:330
3027821320,13sleep20-21swapper/209:38:382
13765213130,89sleep113851-21fschecks_time10:29:081
21905213039,78sleep10-21swapper/110:53:501
1331213015,95sleep10-21swapper/109:48:511
19956212921,93sleep220051-21latency_hist07:23:242
13486212962,17sleep113511-21awk08:43:481
5861212761,17sleep15917-21latency10:03:451
5861212761,17sleep15917-21latency10:03:451
27398212718,33sleep10-21swapper/107:44:511
16504212724,77sleep10-21swapper/110:38:241
27680212512,98sleep227722-21munin-run07:48:232
5020212424,81sleep35034-21kernelversion11:43:453
11718212415,50sleep10-21swapper/112:03:541
23801212350,59sleep123809-21systemd-run09:18:241
20299212315,13sleep10-21swapper/112:30:421
7416212224,51sleep10-21swapper/111:51:341
12764212240,53sleep212761-21apt-get12:08:352
31582212018,20sleep30-21swapper/309:43:243
29515212062,19sleep129505-21apt-get07:53:391
29515212062,19sleep129505-21apt-get07:53:391
2579621205,12sleep10-21swapper/107:39:101
122321196,14sleep10-21swapper/111:31:531
4609211842,36sleep117166-1kworker/1:1H09:58:521
4609211842,36sleep117166-1kworker/1:1H09:58:521
1780221188,51sleep10-21swapper/108:58:381
14634211838,55sleep314735-21sort07:08:243
18343211715,50sleep1278-21dbus-daemon07:18:231
18343211715,50sleep1278-21dbus-daemon07:18:231
16225211740,25sleep114775-21cyclictest08:53:371
1517921149,64sleep115258-21awk08:48:531
14739211416,78sleep014969-21latency_hist07:08:250
27614211227,20sleep10-21swapper/111:13:241
22463211233,50sleep222459-21apt-get09:13:362
18075211213,60sleep118227-21latency_hist10:43:251
539211017,76sleep3571-21proc_pri11:28:523
2475621108,40sleep114775-21cyclictest11:03:351
2475621108,40sleep114775-21cyclictest11:03:351
23686211031,66sleep10-21swapper/110:58:511
23686211031,66sleep10-21swapper/110:58:511
24963210919,68sleep225054-21cut07:38:422
2187221098,15sleep00-21swapper/010:53:490
8414210820,67sleep37928-21apt-get08:28:323
1499210730,64sleep11517-21munin-run08:08:241
18378210618,68sleep10-21swapper/110:43:411
15336210519,64sleep215396-21if_eth007:08:592
31448210412,54sleep10-21swapper/111:23:491
1645821040,12sleep016449-21munin-run10:38:220
6655210317,40sleep30-21swapper/308:23:363
23361210213,16sleep30-21swapper/307:33:393
2050210231,20sleep12060-21df_inode08:08:401
2050210231,20sleep12060-21df_inode08:08:401
12742210020,19sleep10-21swapper/112:08:341
3229629815,29sleep30-21swapper/311:28:233
58642969,67sleep20-21swapper/208:18:552
3088029615,63sleep030982-21sed09:38:530
2092429627,52sleep10-21swapper/109:08:391
408129524,53sleep10-21swapper/108:13:531
2454529514,63sleep324694-21latency_hist11:03:253
2454529514,63sleep324694-21latency_hist11:03:253
2087229413,56sleep221006-21grep07:23:532
1360629411,54sleep213093-21/usr/sbin/munin08:43:532
2125029311,68sleep00-21swapper/007:25:450
141592938,12sleep30-21swapper/308:45:363
368529227,47sleep10-21swapper/111:38:511
1766929214,61sleep317185-21apt-get12:23:333
2922029023,56sleep129248-21munin-run07:53:231
2922029023,56sleep129248-21munin-run07:53:231
2691829015,61sleep326991-21sendmail_mailtr07:43:523
2296528918,54sleep323006-21date10:58:243
298802888,24sleep10-21swapper/111:18:491
92128715,52sleep3974-21cpuspeed09:48:413
566628720,52sleep05735-21grep08:18:530
2782928724,36sleep027846-21acpi11:13:350
1839128721,40sleep218497-21fschecks_time10:43:422
1647728714,52sleep016636-21cut07:13:250
1054228714,55sleep310573-21/usr/sbin/munin10:18:443
2952428519,46sleep229571-21fschecks_count11:18:422
191122857,42sleep119152-21ps07:18:511
663228419,43sleep26651-21apt-get08:23:352
2627828315,43sleep226432-21unixbench_multi09:23:542
1320528327,34sleep013268-21fschecks_time08:43:430
53752827,12sleep00-21swapper/011:43:550
39012829,54sleep03947-21/usr/sbin/munin08:13:480
2655528210,58sleep226578-21/usr/sbin/munin11:08:452
207792829,24sleep10-21swapper/107:23:521
707428115,48sleep07200-21/usr/sbin/munin11:48:550
230522817,41sleep123051-21/usr/sbin/munin07:33:241
1455528130,32sleep214631-21apt-config07:08:232
479028018,42sleep24822-21munin-run08:18:222
1641128015,11sleep10-21swapper/112:18:511
1641128015,11sleep10-21swapper/112:18:511
559327815,29sleep20-21swapper/210:03:382
559327815,29sleep20-21swapper/210:03:382
530127820,40sleep25315-21apt10:03:212
530127820,40sleep25315-21apt10:03:212
251112789,16sleep00-21swapper/007:38:440
2454427826,31sleep224557-21modprobe07:38:222
2090827817,30sleep20-21swapper/212:33:412
2090827817,30sleep20-21swapper/212:33:412
1956727824,22sleep30-21swapper/309:03:443
1593127824,28sleep20-21swapper/208:53:222
88782777,35sleep00-21swapper/010:13:410
322912775,13sleep10-21swapper/108:02:351
2537427712,52sleep324735-21/usr/sbin/munin11:03:533
2537427712,52sleep324735-21/usr/sbin/munin11:03:533
1603427719,42sleep00-21swapper/012:18:420
1603427719,42sleep00-21swapper/012:18:420
691927616,40sleep07069-21latency_hist10:08:240
691927616,40sleep07069-21latency_hist10:08:240
1952327618,38sleep019226-21/usr/sbin/munin12:28:470
1701727512,50sleep217110-21cut10:38:462
1307827523,30sleep00-21swapper/010:28:220
974927412,48sleep29793-21latency_hist08:33:242
211402749,46sleep20-21swapper/209:08:442
138922747,14sleep30-21swapper/310:29:103
1282627410,48sleep312846-21idleruntime-cro08:43:223
3166627311,16sleep30-21swapper/307:58:543
194827320,34sleep21464-21apt-get08:08:332
194827320,34sleep21464-21apt-get08:08:332
1513727314,46sleep315140-21ntpq08:48:513
3200527218,30sleep20-21swapper/209:43:442
284572729,50sleep028513-21latency_hist09:33:240
284572729,50sleep028513-21latency_hist09:33:240
226942726,52sleep20-21swapper/210:55:532
2133227212,44sleep021112-21apt-get10:53:250
2133227212,44sleep021112-21apt-get10:53:250
1826327222,33sleep218287-21/usr/sbin/munin08:58:512
158427219,33sleep21716-21latency_hist08:08:252
1413027214,41sleep214214-21mailstats10:29:172
109282725,11sleep00-21swapper/012:03:200
422027113,24sleep00-21swapper/009:58:430
422027113,24sleep00-21swapper/009:58:430
3084827125,23sleep20-21swapper/207:58:242
3084827125,23sleep20-21swapper/207:58:242
1356827122,22sleep10-21swapper/107:03:461
1319627116,37sleep313220-21ntp_states12:08:483
1012327124,23sleep20-21swapper/208:33:442
2058127018,33sleep220696-21unixbench_singl10:48:552
2058127018,33sleep220696-21unixbench_singl10:48:552
1714227011,45sleep017151-21diskstats07:13:420
1714227011,45sleep017151-21diskstats07:13:420
1222127014,30sleep30-21swapper/310:23:493
833426918,38sleep28408-21mailstats11:53:542
1450026914,42sleep214564-21if_err_eth012:13:432
39712687,10sleep30-21swapper/309:58:263
39712687,10sleep30-21swapper/309:58:263
362926813,39sleep23653-21ntp_states11:38:482
2117126812,25sleep30-21swapper/312:33:473
2117126812,25sleep30-21swapper/312:33:473
932326723,20sleep20-21swapper/210:13:512
64822677,41sleep00-21swapper/008:23:250
2299926720,31sleep323032-21munin-run07:33:223
1658726715,21sleep30-21swapper/312:18:533
1658726715,21sleep30-21swapper/312:18:533
124872676,19sleep20-21swapper/208:38:572
1087926724,22sleep00-21swapper/010:18:540
3074726620,24sleep00-21swapper/011:23:240
2754326616,24sleep30-21swapper/309:28:473
2754326616,24sleep30-21swapper/309:28:473
2473426613,38sleep024753-21apt-get11:03:340
2473426613,38sleep024753-21apt-get11:03:340
170526615,32sleep31712-21/usr/sbin/munin11:33:403
1027626619,31sleep10-21swapper/110:18:381
356826515,34sleep03594-21netstat11:38:460
138326516,36sleep01398-21open_inodes09:48:540
1516826415,30sleep215160-21apt-get10:33:372
846726315,23sleep00-21swapper/010:13:250
2927026314,35sleep329324-21latency_hist07:53:253
2927026314,35sleep329324-21latency_hist07:53:253
2356426313,31sleep00-21swapper/010:58:460
2356426313,31sleep00-21swapper/010:58:460
1429126320,21sleep00-21swapper/010:29:190
1253126318,18sleep00-21swapper/012:08:250
1130726319,24sleep30-21swapper/312:03:433
1026426320,23sleep010293-21kernelversion08:33:460
3262326213,31sleep232635-21/usr/sbin/munin11:28:432
1093026225,20sleep20-21swapper/212:03:222
2926026112,34sleep229269-21systemd-run07:53:242
2926026112,34sleep229269-21systemd-run07:53:242
261526112,25sleep30-21swapper/308:09:113
261526112,25sleep30-21swapper/308:09:113
2442126122,21sleep00-21swapper/009:18:470
2413826121,22sleep30-21swapper/309:18:423
2286626117,16sleep30-21swapper/309:13:503
2138026115,26sleep00-21swapper/009:08:500
2065726112,29sleep320695-21grep07:23:503
1730126116,28sleep217448-21latency_hist12:23:242
1730126116,28sleep217448-21latency_hist12:23:242
435926011,35sleep3364-21snmpd11:42:043
273326011,23sleep20-21swapper/209:53:452
248302606,45sleep30-21swapper/307:38:273
2238226022,22sleep20-21swapper/207:28:532
1982226015,23sleep20-21swapper/209:03:522
1980326010,13sleep30-21swapper/312:28:543
1622026018,25sleep316216-21apt-get08:53:353
145526019,20sleep20-21swapper/209:48:562
1134126015,23sleep30-21swapper/308:38:253
970725916,25sleep39649-21apt-get11:59:113
47425916,26sleep00-21swapper/011:28:490
371825926,18sleep00-21swapper/009:58:230
321762596,17sleep00-21swapper/008:01:040
2950525915,25sleep229596-21users09:33:552
1886625923,16sleep00-21swapper/010:43:520
713825819,20sleep30-21swapper/310:08:353
688225816,22sleep20-21swapper/211:48:532
3098825816,25sleep030980-21apt-get11:23:360
2100325817,20sleep30-21swapper/309:08:413
1287725811,35sleep2364-21snmpd10:25:462
1285325813,31sleep012864-21awk12:08:410
90282579,35sleep00-21swapper/011:57:230
559825715,27sleep35663-21awk08:18:523
2021825717,25sleep320262-21ntpq10:48:483
2021825717,25sleep320262-21ntpq10:48:483
1865025715,26sleep318643-21apt-get07:18:373
1770025715,26sleep217690-21apt-get12:23:362
154092576,40sleep00-21swapper/012:15:590
1514325712,25sleep315111-21apt-get07:08:523
1167625714,20sleep00-21swapper/008:38:430
350225621,19sleep30-21swapper/308:13:353
288992566,38sleep00-21swapper/007:49:140
288992566,38sleep00-21swapper/007:49:140
2691925617,17sleep30-21swapper/311:08:543
1961725614,24sleep019631-21irqstats09:03:450
1924925616,25sleep219244-21apt-get12:28:372
1863125614,23sleep018643-21apt-get07:18:340
1027025615,25sleep210266-21apt-get10:18:352
1003425615,18sleep30-21swapper/310:18:243
328725521,14sleep00-21swapper/008:13:240
328725521,14sleep00-21swapper/008:13:240
2685925514,23sleep026870-21ntp_states07:43:490
1633825515,17sleep20-21swapper/208:53:422
1630225514,24sleep016308-21diskstats08:53:400
294402549,15sleep30-21swapper/311:18:393
244962544,16sleep00-21swapper/007:37:430
2343925417,22sleep223473-21/usr/sbin/munin10:58:442
2343925417,22sleep223473-21/usr/sbin/munin10:58:442
171925416,22sleep31924-21sed08:08:263
2834125319,19sleep20-21swapper/211:13:532
2475925311,25sleep224753-21apt-get11:03:362
2475925311,25sleep224753-21apt-get11:03:362
1787925314,24sleep017886-21/usr/sbin/munin08:58:410
1767825314,13sleep00-21swapper/012:23:340
1439425316,23sleep314400-21diskstats12:13:403
3206425212,24sleep332093-21irqstats09:43:453
3103825210,29sleep231061-21sh11:23:382
305572525,16sleep30-21swapper/307:55:393
305572525,16sleep30-21swapper/307:55:393
2840625213,23sleep228438-21munin-run09:33:222
2840625213,23sleep228438-21munin-run09:33:222
2577525217,17sleep30-21swapper/309:23:433
2222825213,18sleep00-21swapper/007:28:510
1598325216,18sleep00-21swapper/008:53:240
810725111,24sleep28108-21sshd10:09:352
639625111,22sleep36493-21users10:03:553
639625111,22sleep36493-21users10:03:553
3240425118,17sleep20-21swapper/208:03:252
24332516,28sleep30-21swapper/309:53:343
1179825114,21sleep211881-21cat12:03:552
498825011,21sleep20-21swapper/211:43:442
2920125014,15sleep30-21swapper/309:33:523
1683525013,23sleep316844-21munin-node07:13:283
1683525013,23sleep316844-21munin-node07:13:283
2559424912,20sleep025589-21apt-get09:23:350
169824710,20sleep21704-21df_inode11:33:392
841624612,18sleep00-21swapper/008:28:330
1787124511,19sleep317877-21df_inode08:58:403
2304524310,15sleep00-21swapper/009:13:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional