You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 21:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 19, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27202140214,1374sleep028977-21kworker/0:211:04:550
3753243617,237sleep10-21swapper/107:43:501
3753243617,237sleep10-21swapper/107:43:501
198322367153,191sleep119903-21acpi10:18:361
198322367153,191sleep119903-21acpi10:18:361
10131235726,274sleep1364-21snmpd09:46:261
86772338153,163sleep18820-21unixbench_singl11:23:581
42412338142,168sleep10-21swapper/109:28:491
42412338142,168sleep10-21swapper/109:28:491
31112229512,265sleep031029-21latency_hist12:38:280
29458225573,22sleep10-21swapper/109:03:571
22633224411,216sleep022726-21grep08:43:460
14536223726,190sleep314537-21cpu08:18:413
8499222920,186sleep08500-21cron09:43:260
18494222713,200sleep218513-21fschecks_count10:13:572
18494222713,200sleep218513-21fschecks_count10:13:572
1407222097,26sleep10-21swapper/109:18:561
1407222097,26sleep10-21swapper/109:18:561
32715221424,168sleep232730-21fschecks_count10:58:432
5178221329,26sleep10-21swapper/107:48:431
5178221329,26sleep10-21swapper/107:48:431
10700221025,172sleep110734-21fschecks_time09:48:481
27122220430,155sleep127178-21processes12:23:551
2203220250,135sleep12251-21grep07:38:501
31816220096,25sleep10-21swapper/107:28:561
31816220096,25sleep10-21swapper/107:28:561
3052321977,175sleep130635-21/usr/sbin/munin07:24:001
2810221950,36sleep328106-21unixbench_singl08:58:583
25409219412,24sleep10-21swapper/108:53:281
21201218815,154sleep121206-21modprobe10:23:251
30041218619,143sleep10-21swapper/112:33:481
1719121846,143sleep117232-21ntpq10:08:551
26989217949,92sleep127139-21latency_hist08:58:281
1254221798,138sleep112586-21ntpq09:54:011
730217511,150sleep00-21swapper/007:34:000
730217511,150sleep00-21swapper/007:34:000
25091217410,104sleep125150-21wc10:33:591
9930217212,72sleep10-21swapper/108:03:451
2612621715,11sleep20-21swapper/208:53:572
11661217010,143sleep111754-21timerandwakeup11:33:561
11661217010,143sleep111754-21timerandwakeup11:33:561
1667121696,13sleep10-21swapper/111:49:181
1861621688,107sleep10-21swapper/111:58:271
2749821656,18sleep10-21swapper/108:58:491
148621597,115sleep11496-21munin-run11:03:271
9302215413,126sleep29446-21cat11:28:282
275921517,128sleep10-21swapper/109:23:521
21933214810,92sleep10-21swapper/112:08:361
14214214723,106sleep014235-21sh08:18:260
1423721468,66sleep10-21swapper/108:18:271
22838214418,111sleep022773-21latency_hist10:28:280
14794213910,91sleep114898-21wc11:43:571
2700521365,12sleep10-21swapper/107:13:541
2700521365,12sleep10-21swapper/107:13:541
9485213423,88sleep00-21swapper/011:28:380
78021338,90sleep1779-21munin-run09:18:271
2061321316,11sleep10-21swapper/108:37:371
1081921319,62sleep110903-21wc11:33:271
2351921307,11sleep10-21swapper/108:44:481
12720213016,34sleep112809-21fschecks_time11:38:441
9957212712,99sleep39967-21sh11:28:533
2898121275,12sleep10-21swapper/110:45:181
889212621,90sleep3922-21ntp_kernel_err07:34:033
889212621,90sleep3922-21ntp_kernel_err07:34:033
2141221256,13sleep10-21swapper/112:05:051
5844212321,86sleep25937-21ls07:49:042
5844212321,86sleep25937-21ls07:49:042
1743121218,97sleep117585-21mktemp08:28:281
1037211910,97sleep11119-21mailstats07:34:091
1037211910,97sleep11119-21mailstats07:34:091
267621186,77sleep10-21swapper/111:04:201
6976211610,66sleep10-21swapper/111:18:581
6976211610,66sleep10-21swapper/111:18:581
15385211617,78sleep315416-21fschecks_time10:03:483
15385211617,78sleep315416-21fschecks_time10:03:483
26658211310,85sleep10-21swapper/110:39:081
661621120,7sleep00-21swapper/007:53:290
661621120,7sleep00-21swapper/007:53:290
1496921128,67sleep11-21systemd10:03:261
1496921128,67sleep11-21systemd10:03:261
11975211218,79sleep311977-21sshd11:34:023
11975211218,79sleep311977-21sshd11:34:023
2465721119,68sleep124735-21latency_hist07:08:411
15733211133,38sleep00-21swapper/011:48:410
15681211110,57sleep10-21swapper/110:03:541
15681211110,57sleep10-21swapper/110:03:541
775021109,82sleep10-21swapper/109:39:031
3153121109,91sleep1360-21ntpd09:12:331
4025210921,62sleep34171-21sed11:08:583
373421087,63sleep13740-21ntp_states11:08:521
31145210864,32sleep131237-21fschecks_time10:53:461
2935621086,68sleep10-21swapper/107:20:451
2272521088,67sleep1364-21snmpd10:28:051
1140421089,57sleep10-21swapper/108:08:421
5019210712,78sleep25084-21latency_hist07:48:362
5019210712,78sleep25084-21latency_hist07:48:362
2383321056,56sleep10-21swapper/112:13:491
6922210432,60sleep16975-21munin-run09:38:261
6922210432,60sleep16975-21munin-run09:38:261
2614121047,32sleep10-21swapper/107:13:301
2614121047,32sleep10-21swapper/107:13:301
25151210313,65sleep20-21swapper/210:34:002
30207210113,72sleep230267-21processes12:33:532
2856321008,14sleep00-21swapper/012:28:510
26622210020,29sleep00-21swapper/012:23:380
20029210021,55sleep220062-21fschecks_count10:18:452
2599729911,73sleep0369-21sshd07:12:560
2599729911,73sleep0369-21sshd07:12:560
654329817,62sleep36613-21grep07:53:293
654329817,62sleep36613-21grep07:53:293
994329624,47sleep310006-21fschecks_time08:03:473
216192964,11sleep30-21swapper/310:23:513
1132429619,21sleep00-21swapper/011:33:470
1132429619,21sleep00-21swapper/011:33:470
67322956,17sleep10-21swapper/107:53:441
1912529559,22sleep119158-21latency11:58:471
483129361,16sleep10-21swapper/111:13:411
2951029321,35sleep10-21swapper/110:48:411
1932229218,60sleep319385-21processes11:58:533
2685829111,64sleep226914-21irqstats12:23:482
661929018,53sleep06656-21if_err_eth011:18:480
661929018,53sleep06656-21if_err_eth011:18:480
2078829015,59sleep320832-21ntpq12:03:503
262902895,72sleep10-21swapper/112:22:341
230422898,37sleep10-21swapper/110:28:421
230422898,37sleep10-21swapper/110:28:421
1783428917,33sleep10-21swapper/108:28:321
2326828815,22sleep00-21swapper/012:13:280
154712887,71sleep10-21swapper/108:19:161
87682878,18sleep18830-21processes07:58:571
285912878,15sleep10-21swapper/112:28:521
213922876,22sleep10-21swapper/108:38:541
617628623,37sleep00-21swapper/011:18:280
609628622,48sleep2225-21systemd-journal11:18:262
175862860,9sleep20-21swapper/208:28:282
2784128520,52sleep027914-21latency_hist10:43:350
2635728524,38sleep026378-21modprobe12:23:250
2365128514,53sleep223715-21/usr/sbin/munin12:13:462
1214728517,51sleep212299-21sed08:08:592
1853528418,37sleep318566-21sh11:58:253
1789928413,60sleep00-21swapper/008:29:000
3091928122,29sleep30-21swapper/309:08:563
2945328125,38sleep229481-21modprobe12:33:252
1635428116,44sleep316411-21sed08:23:503
272952795,57sleep00-21swapper/007:14:010
272952795,57sleep00-21swapper/007:14:010
2861327819,41sleep228688-21switchtime10:44:542
1654527812,50sleep016558-21open_files08:23:550
3141127721,34sleep231408-21apt-get07:28:412
3141127721,34sleep231408-21apt-get07:28:412
242932775,62sleep00-21swapper/010:33:210
242932775,62sleep00-21swapper/010:33:210
151327724,33sleep31660-21latency_hist11:03:293
994027617,33sleep20-21swapper/208:03:462
46792768,56sleep00-21swapper/007:45:470
46792768,56sleep00-21swapper/007:45:470
2831327617,29sleep20-21swapper/207:18:442
69827522,36sleep3715-21cron09:18:253
386827527,25sleep00-21swapper/009:28:280
2626127514,45sleep026344-21grep08:53:590
2434027422,37sleep224349-21unin-run10:33:272
2434027422,37sleep224349-21unin-run10:33:272
2386327427,25sleep30-21swapper/307:03:593
21872748,15sleep30-21swapper/311:03:533
2002927417,32sleep00-21swapper/008:33:570
1784027413,23sleep20-21swapper/208:28:382
104932749,48sleep00-21swapper/009:48:310
2959227312,46sleep029595-21entropy10:48:440
25572737,45sleep20-21swapper/209:23:412
25572737,45sleep20-21swapper/209:23:412
225327320,38sleep22292-21fw_packets07:38:512
3159927218,36sleep031604-21modprobe09:13:250
3138827219,36sleep031390-21/usr/sbin/munin07:28:390
3138827219,36sleep031390-21/usr/sbin/munin07:28:390
2725227217,34sleep327332-21timerandwakeup12:23:573
2655427219,34sleep226695-21fschecks_count07:13:392
2655427219,34sleep226695-21fschecks_count07:13:392
918127119,24sleep20-21swapper/209:44:092
1489927117,26sleep20-21swapper/211:43:582
1142327119,30sleep20-21swapper/211:33:492
1142327119,30sleep20-21swapper/211:33:492
808027024,32sleep28151-21latency_hist07:58:292
376727019,28sleep00-21swapper/011:08:550
32627027,23sleep30-21swapper/307:33:303
32627027,23sleep30-21swapper/307:33:303
3156127018,36sleep331639-21sendmail_mailtr10:53:563
3124027018,29sleep20-21swapper/210:53:472
2094627017,25sleep00-21swapper/012:03:550
2797026912,41sleep228099-21grep08:58:582
1844126915,24sleep30-21swapper/310:13:463
1844126915,24sleep30-21swapper/310:13:463
1111026915,24sleep30-21swapper/309:48:583
73952687,43sleep30-21swapper/307:54:063
2991826816,24sleep30-21swapper/310:48:513
2465526814,38sleep024650-21apt-get07:08:400
1886026815,23sleep00-21swapper/010:14:070
1886026815,23sleep00-21swapper/010:14:070
570326714,36sleep05742-21df09:33:450
570326714,36sleep05742-21df09:33:450
364126722,20sleep20-21swapper/211:08:492
268426717,25sleep00-21swapper/007:39:010
704726617,24sleep20-21swapper/211:18:592
704726617,24sleep20-21swapper/211:18:592
605026614,30sleep20-21swapper/209:33:592
605026614,30sleep20-21swapper/209:33:592
2890326612,14sleep20-21swapper/209:03:462
283862669,43sleep028394-21df07:18:460
241682666,15sleep00-21swapper/008:48:420
1957426626,19sleep20-21swapper/208:33:462
148526612,37sleep31491-21ntp_states09:18:583
148526612,37sleep31491-21ntp_states09:18:583
1429526618,25sleep20-21swapper/208:18:282
536026522,19sleep30-21swapper/311:13:553
3084226512,38sleep20-21swapper/209:08:552
3026026518,20sleep30-21swapper/307:23:533
247152655,12sleep30-21swapper/312:17:353
2409226513,38sleep224148-21cpuspeed08:48:402
2165126516,29sleep221715-21/usr/sbin/munin10:23:542
2125226522,27sleep321293-21latency_hist10:23:273
184442657,12sleep00-21swapper/011:57:130
1480726516,24sleep20-21swapper/208:18:482
1260626523,24sleep00-21swapper/009:54:040
89912647,41sleep30-21swapper/309:43:333
56926418,26sleep00-21swapper/010:58:490
2681526421,17sleep30-21swapper/307:13:483
2681526421,17sleep30-21swapper/307:13:483
2585526420,28sleep325860-21modprobe10:38:253
230826419,21sleep20-21swapper/211:03:562
1541926416,27sleep00-21swapper/010:03:490
1541926416,27sleep00-21swapper/010:03:490
872326321,20sleep30-21swapper/309:43:293
3186426316,27sleep00-21swapper/009:13:380
264226314,28sleep20-21swapper/209:23:452
2229726316,26sleep20-21swapper/208:43:272
2212126315,20sleep20-21swapper/212:08:462
1906626317,29sleep019124-21/usr/sbin/munin11:58:460
1758626313,24sleep00-21swapper/010:09:020
1513226316,20sleep20-21swapper/210:03:392
1513226316,20sleep20-21swapper/210:03:392
1290426313,37sleep012993-21cat11:38:470
112026321,23sleep20-21swapper/207:34:102
112026321,23sleep20-21swapper/207:34:102
860026221,27sleep28721-21apt-key09:43:282
77032627,12sleep00-21swapper/007:54:520
3117726219,18sleep00-21swapper/007:28:350
2640826217,23sleep30-21swapper/312:23:283
2526426225,21sleep30-21swapper/307:08:593
2526426225,21sleep30-21swapper/307:08:593
2164726217,23sleep30-21swapper/308:39:003
177682628,25sleep30-21swapper/308:28:303
1303526214,30sleep313081-21fschecks_count08:13:453
2743226118,27sleep327458-21munin-run10:43:263
1594026115,31sleep315990-21if_eth011:48:483
123962619,43sleep00-21swapper/008:10:020
3032726014,31sleep230359-21/usr/sbin/munin07:23:572
1439126016,23sleep30-21swapper/311:43:473
1268426010,30sleep20-21swapper/211:38:412
1230026024,20sleep30-21swapper/308:09:003
411225917,25sleep34109-21apt-get09:28:393
411225917,25sleep34109-21apt-get09:28:393
3036125913,26sleep230414-21users10:48:582
2586725918,27sleep025934-21munin-run10:38:260
2385225916,20sleep20-21swapper/208:48:282
2050625916,19sleep20-21swapper/212:03:452
1894125916,22sleep20-21swapper/211:58:442
153282598,29sleep20-21swapper/210:03:432
153282598,29sleep20-21swapper/210:03:432
1062425917,25sleep210648-21df_inode09:48:412
7812587,11sleep20-21swapper/209:18:282
288672589,23sleep30-21swapper/312:28:583
2486225817,17sleep00-21swapper/012:18:290
1772525811,31sleep217734-21ntpq11:53:522
137822589,33sleep013786-21diskstats09:58:440
137822589,33sleep013786-21diskstats09:58:440
114925813,19sleep00-21swapper/009:18:500
114925813,19sleep00-21swapper/009:18:500
830025714,26sleep38367-21df07:58:423
299522578,38sleep00-21swapper/009:07:200
2804325714,23sleep00-21swapper/010:44:400
2377425717,24sleep223862-21sed07:03:582
2262625717,25sleep322631-21diskstats08:43:453
1993225715,25sleep019927-21apt-get10:18:390
964625614,15sleep00-21swapper/009:44:270
823425616,18sleep00-21swapper/011:23:490
820925620,18sleep30-21swapper/311:23:483
588925612,25sleep35947-21irqstats09:33:563
588925612,25sleep35947-21irqstats09:33:563
3243425617,21sleep00-21swapper/010:58:290
651125516,26sleep36615-21fschecks_time11:18:473
651125516,26sleep36615-21fschecks_time11:18:473
449425518,20sleep00-21swapper/009:28:560
449425518,20sleep00-21swapper/009:28:560
388025514,22sleep33903-21irqstats07:43:523
388025514,22sleep33903-21irqstats07:43:523
259025516,16sleep30-21swapper/307:39:003
2553425512,23sleep225550-21ntp_states12:18:512
2345125521,17sleep30-21swapper/310:28:563
2345125521,17sleep30-21swapper/310:28:563
1549425511,24sleep00-21swapper/011:48:290
1259825514,25sleep312602-21ntp_states09:54:033
829925412,25sleep28292-21apt-get07:58:412
3107425416,16sleep30-21swapper/312:38:273
2495225414,23sleep325005-21users08:48:583
2203125414,24sleep322040-21diskstats12:08:433
1573525412,24sleep215802-21df11:48:432
850625312,25sleep08546-21if_eth007:58:470
2573025313,18sleep30-21swapper/308:53:463
220322536,38sleep00-21swapper/008:40:350
1456125310,17sleep00-21swapper/008:18:420
950825212,24sleep29504-21apt-get11:28:402
439525213,22sleep24489-21memory09:28:542
439525213,22sleep24489-21memory09:28:542
2762225213,18sleep20-21swapper/210:43:292
2730125215,18sleep00-21swapper/008:58:440
1422325216,18sleep30-21swapper/309:59:043
1422325216,18sleep30-21swapper/309:59:043
1338425213,24sleep213410-21munin-run09:58:262
987925112,17sleep00-21swapper/008:03:440
2831225113,22sleep328406-21memory10:44:473
386425012,23sleep33866-21systemd-run09:28:273
2511725012,22sleep225156-21memory07:08:542
2511725012,22sleep225156-21memory07:08:542
1963925014,21sleep319690-21irqstats08:33:473
130492508,18sleep30-21swapper/311:38:513
2877824913,21sleep328775-21apt-get09:03:393
2900024717,14sleep30-21swapper/307:19:053
1716224614,16sleep30-21swapper/310:08:533
1688424610,21sleep216889-21diskstats10:08:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional