You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-07 - 20:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun Jun 07, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165696121820,6sleep30-21swapper/308:15:393
164458321810,3sleep375-21kworker/3:1-events08:00:423
158954699641,61cyclictest1-21systemd11:42:140
1589548996157,3cyclictest1850086-21kworker/u8:2+flush-8:011:42:371
1589551995949,7cyclictest1704890-21systemd-journal11:42:392
1589551995532,20cyclictest1740722-21cp09:40:002
1589548995545,8cyclictest1676014-21kworker/u8:2+flush-8:008:55:461
1589548995346,5cyclictest1784997-21kworker/u8:1+events_unbound11:00:501
1589548995144,5cyclictest1758112-21kworker/u8:0+events_unbound11:45:461
1589548995143,6cyclictest1784997-21kworker/u8:1+flush-8:010:30:471
1589548995044,4cyclictest753075-21kworker/u8:0+flush-8:007:35:451
1589548994944,3cyclictest1758112-21kworker/u8:0+events_unbound10:45:501
1589548994939,8cyclictest1838993-21ntpq11:15:451
1589548994843,4cyclictest1676014-21kworker/u8:2+events_unbound09:35:421
1589548994842,4cyclictest753075-21kworker/u8:0+flush-8:007:20:501
1589548994842,3cyclictest1758112-21kworker/u8:0+events_unbound10:35:481
1589546994836,9cyclictest1704890-21systemd-journal12:20:400
1589551994728,16cyclictest1704890-21systemd-journal09:53:562
1589548994742,4cyclictest1715596-21kworker/u8:1+events_unbound09:15:451
1589548994742,3cyclictest1850086-21kworker/u8:2+flush-8:011:35:481
1589548994740,5cyclictest1850086-21kworker/u8:2+flush-8:012:35:461
1589548994641,4cyclictest1784997-21kworker/u8:1+flush-8:011:10:381
1589548994641,3cyclictest1758112-21kworker/u8:0+events_unbound10:15:391
1589548994641,3cyclictest1758112-21kworker/u8:0+events_unbound10:15:391
1589548994640,4cyclictest1527706-21kworker/u8:2+events_unbound07:10:471
1589548994639,5cyclictest1890193-21kworker/u8:3+events_unbound12:15:441
1589548994634,9cyclictest1848886-21ntpq11:25:441
1589546994638,6cyclictest1784997-21kworker/u8:1+flush-8:010:30:460
1589546994634,9cyclictest1799274-21ntpq10:35:440
1589548994540,4cyclictest1631055-21kworker/u8:3+events_unbound08:20:531
1589548994539,5cyclictest1758112-21kworker/u8:0+events_unbound10:40:511
1589548994539,4cyclictest1631055-21kworker/u8:3+events_unbound08:25:511
1589548994539,4cyclictest1631055-21kworker/u8:3+events_unbound08:25:511
1589548994538,5cyclictest753075-21kworker/u8:0+flush-8:007:55:471
1589548994538,5cyclictest1701177-21kworker/u8:3+flush-8:011:20:461
1589548994536,7cyclictest1785579-21/usr/sbin/munin10:25:471
1589548994536,7cyclictest1675114-21ntpq08:30:441
1589551994433,9cyclictest1704890-21systemd-journal12:00:372
1589548994440,3cyclictest1665766-21kworker/u8:0+events_unbound08:45:321
1589548994439,4cyclictest1631055-21kworker/u8:3+events_unbound08:10:451
1589548994438,4cyclictest1890193-21kworker/u8:3+flush-8:012:20:451
1589548994437,5cyclictest1758112-21kworker/u8:0+flush-8:010:20:451
1589548994436,6cyclictest1758112-21kworker/u8:0+flush-8:011:55:461
1589548994338,4cyclictest1890193-21kworker/u8:3+flush-8:012:10:461
1589548994338,4cyclictest1721427-21kworker/u8:0+events_unbound09:30:491
1589548994335,6cyclictest1701177-21kworker/u8:3+events_unbound09:25:431
1589546994333,8cyclictest1769606-21ntpq10:05:440
1589546994332,9cyclictest753075-21kworker/u8:0+flush-8:008:00:440
1589548994238,3cyclictest1758112-21kworker/u8:0+events_unbound10:05:011
1589548994235,5cyclictest1850086-21kworker/u8:2+flush-8:012:00:461
1589551994125,14cyclictest1816435-21kthreadcore10:55:392
1589548994136,3cyclictest1650271-21kworker/u8:2+flush-8:008:20:001
1589548994136,3cyclictest1589653-21kworker/u8:1+events_unbound08:40:501
1589548994131,8cyclictest1749816-21ntpq09:45:441
1589548994131,7cyclictest1610350-21ntpq07:25:441
1589546994135,4cyclictest1704890-21systemd-journal08:49:560
1589546994132,7cyclictest1891043-21latency_hist12:10:010
1589546994132,7cyclictest1891043-21latency_hist12:10:000
1589551994032,5cyclictest1666285-21cp08:25:002
1589548994035,3cyclictest1665766-21kworker/u8:0+flush-8:009:05:001
1589548994031,7cyclictest1650107-21ntpq08:05:461
1589546994030,7cyclictest1760558-21latency_hist10:00:010
1589551993931,6cyclictest1704890-21systemd-journal07:34:412
1589548993934,4cyclictest753075-21kworker/u8:0+flush-8:007:15:471
1589548993933,4cyclictest1784997-21kworker/u8:1+events_unbound11:05:421
1589548993833,4cyclictest753075-21kworker/u8:0+events_unbound07:30:481
1589548993833,4cyclictest1701177-21kworker/u8:3+events_unbound11:30:401
1589548993833,3cyclictest753075-21kworker/u8:0+events_unbound07:50:381
1589546993832,4cyclictest753075-21kworker/u8:0+flush-8:007:50:360
1589546993832,4cyclictest1631055-21kworker/u8:3+flush-8:008:20:380
1589551993730,5cyclictest1704890-21systemd-journal12:00:002
1589548993731,5cyclictest1665766-21kworker/u8:0+flush-8:009:10:471
1589548993729,6cyclictest1758112-21kworker/u8:0+flush-8:012:25:461
1589546993728,7cyclictest1704890-21systemd-journal11:20:000
1589551993628,5cyclictest1704890-21systemd-journal10:50:212
1589551993621,13cyclictest0-21swapper/209:15:012
1589548993629,5cyclictest1704890-21systemd-journal07:46:541
15892402364,10sleep10-21swapper/107:08:441
1589548993530,3cyclictest1758112-21kworker/u8:0+events_unbound10:50:401
1589548993529,4cyclictest1701177-21kworker/u8:3+events_unbound09:20:531
158955199344,24cyclictest1691347-21apt-get08:50:312
158955199342,24cyclictest1820487-21apt-get11:00:332
158955199342,12cyclictest1596643-21apt-get07:15:332
1589546993427,5cyclictest1715544-21apt-get09:11:380
1589551993313,16cyclictest1621163-21awk07:40:002
158955199331,12cyclictest1740605-21apt-get09:40:032
1589548993329,3cyclictest1665766-21kworker/u8:0+flush-8:009:05:391
1589546993329,2cyclictest1891677-21fschecks_time12:10:360
16803252320,3sleep3481ktimers/308:35:483
1589551993218,12cyclictest1646623-21cpuspeed_turbos08:05:332
1589548993228,3cyclictest1701177-21kworker/u8:3+flush-8:009:50:351
1589548993226,5cyclictest1784997-21kworker/u8:1+events_unbound10:55:221
1589548993226,5cyclictest1665766-21kworker/u8:0+flush-8:008:50:421
1589546993228,2cyclictest1750724-21systemd-run09:50:000
1589546993223,6cyclictest1704890-21systemd-journal07:55:010
1589548993126,4cyclictest1758112-21kworker/u8:0+events_unbound10:05:451
1589548993126,4cyclictest1665766-21kworker/u8:0+events_unbound08:35:551
1589548993125,4cyclictest1701177-21kworker/u8:3+flush-8:009:55:411
1589548993125,4cyclictest1589653-21kworker/u8:1+flush-8:007:40:411
1589546993126,3cyclictest1771006-21cpuspeed_turbos10:10:340
1589546993125,4cyclictest1850086-21kworker/u8:2+flush-8:011:45:320
1589546993124,5cyclictest1593504-21ntpq07:10:430
1589546993122,6cyclictest1611234-21latency_hist07:30:000
16531672300,2sleep30-21swapper/308:10:413
158955199308,19cyclictest1896538-21grep12:15:342
158955199302,8cyclictest1755912-21apt-get09:55:322
1589551993016,12cyclictest0-21swapper/210:25:502
1589551993012,15cyclictest0-21swapper/208:45:382
1589548993025,3cyclictest1850086-21kworker/u8:2+flush-8:011:50:411
1589546993022,6cyclictest1784997-21kworker/u8:1+flush-8:011:25:330
158955199293,19cyclictest1850083-21apt-get11:30:322
158955199292,8cyclictest1840222-21apt-get11:20:332
1589551992925,2cyclictest1891411-21aten2_r5power_p12:10:332
1589551992924,3cyclictest1631408-21aten2_r5power_p07:50:342
158955199291,8cyclictest1675707-21taskset08:31:282
1589551992916,11cyclictest1601912-21fschecks_count07:20:372
1589551992912,15cyclictest1708443-21tr09:05:392
158955199291,11cyclictest1767999-21ps10:05:402
158955199291,10cyclictest1636430-21apt-get07:55:032
1589548992925,3cyclictest1701177-21kworker/u8:3+events_unbound09:40:551
1589548992924,4cyclictest753075-21kworker/u8:0+flush-8:008:01:251
1589546992925,3cyclictest1820170-21cron11:00:010
1589546992919,7cyclictest1919835-21ntpq12:35:440
1589546992918,9cyclictest1740605-21apt-get09:40:010
158954699290,28cyclictest1735294-21sshd09:30:550
158955199289,15cyclictest1860049-21sed11:40:012
158955199288,17cyclictest1846723-21tr11:25:402
158955199287,7cyclictest1904658-21tr12:20:432
158955199284,6cyclictest1654866-21grep08:10:432
158955199282,7cyclictest1719908-21ntpq09:15:442
1589551992824,2cyclictest1726110-21aten2_r5power_e09:25:322
1589548992825,2cyclictest1701177-21kworker/u8:3+events_unbound10:13:381
1589546992825,2cyclictest1830045-21latency_hist11:09:590
1589546992824,3cyclictest1710893-21systemd-run09:10:010
1589546992816,9cyclictest1729782-21ntpq09:25:420
158955199272,5cyclictest1666367-21apt-get08:25:332
158955199272,5cyclictest1666367-21apt-get08:25:332
1589551992723,2cyclictest1911238-21aten2_r5power_p12:30:322
1589551992710,15cyclictest0-21swapper/212:35:452
158955199270,9cyclictest1705098-21/usr/sbin/munin09:00:472
158955199270,8cyclictest1700043-21awk08:55:452
158955199270,10cyclictest1809512-21/usr/sbin/munin10:45:482
158955199270,10cyclictest1790522-21/usr/sbin/munin10:30:312
1589548992722,4cyclictest1758112-21kworker/u8:0+events_unbound12:30:481
1589548992721,4cyclictest1865228-21kworker/u8:1+events_unbound12:05:421
1589548992721,4cyclictest1865228-21kworker/u8:1+events_unbound12:05:421
1589546992720,5cyclictest1780532-21latency_hist10:20:010
1589546992720,5cyclictest1780532-21latency_hist10:20:000
158955199267,5cyclictest1837550-21cut11:15:422
158955199264,6cyclictest1608389-21cut07:25:402
158955199262,9cyclictest1779506-21ntpq10:15:442
158955199262,9cyclictest1779506-21ntpq10:15:442
158955199262,4cyclictest1621456-21apt-get07:40:342
1589551992622,2cyclictest1780886-21cpuspeed_turbos10:20:332
158955199261,8cyclictest1871545-21/usr/sbin/munin11:50:322
158955199261,8cyclictest1680052-21ntpq08:35:442
158955199261,7cyclictest1801062-21/usr/sbin/munin10:40:372
1589551992611,12cyclictest0-21swapper/207:45:342
158955199260,9cyclictest1834051-21ntpq11:10:452
158955199260,8cyclictest1886399-21/usr/sbin/munin12:05:312
158955199260,8cyclictest1886399-21/usr/sbin/munin12:05:312
158955199260,8cyclictest1725917-21grep09:25:002
1589546992621,3cyclictest1871775-21cpuspeed_turbos11:50:350
1589546992619,5cyclictest1656243-21/usr/sbin/munin08:15:460
1589546992618,6cyclictest1784997-21kworker/u8:1+flush-8:010:45:450
15892042262,6sleep00-21swapper/007:08:150
1589551992522,2cyclictest1799764-21unixbench_singl10:35:492
1589551992521,2cyclictest1825638-21cpuspeed_turbos11:05:332
158955199251,9cyclictest1906652-21/usr/sbin/munin12:25:372
158955199251,7cyclictest1641529-21cut08:00:342
158955199250,9cyclictest1866747-21tr11:45:322
158955199250,9cyclictest1681230-21/usr/sbin/munin08:40:302
158955199250,8cyclictest1764656-21ntpq10:00:442
158955199250,8cyclictest1750174-21grep09:45:502
158955199250,8cyclictest1592690-21cut07:10:422
158955199250,7cyclictest1735179-21/usr/sbin/munin09:30:502
1589546992521,3cyclictest1758112-21kworker/u8:0+flush-8:012:30:450
1589546992520,4cyclictest1877234-21tr11:55:380
1589546992520,4cyclictest1705556-21unixbench_singl09:00:500
1589546992520,4cyclictest1589653-21kworker/u8:1+flush-8:007:40:490
158955199249,13cyclictest0-21swapper/208:15:422
158954699242,21cyclictest0-21swapper/007:35:490
1589546992420,3cyclictest1898481-21tr12:15:410
1589546992420,3cyclictest1811300-21tr10:50:390
1589546992419,4cyclictest1735644-21kworker/u8:1+flush-8:009:54:260
158954699240,23cyclictest1765576-21true10:05:000
158955199231,4cyclictest1771187-21/usr/sbin/munin10:10:362
158954699232,20cyclictest0-21swapper/011:35:500
1589546992318,4cyclictest1665766-21kworker/u8:0+flush-8:008:30:350
1589546992318,3cyclictest1680584-21unixbench_singl08:35:500
17044462220,2sleep3471rcuc/309:00:413
158954699222,19cyclictest0-21swapper/010:40:510
158954699222,19cyclictest0-21swapper/008:10:500
158954699222,14cyclictest583-21snmpd10:20:320
1589546992218,3cyclictest1865228-21kworker/u8:1+flush-8:012:00:470
1589546992218,3cyclictest1844459-21unixbench_multi11:20:510
1589546992218,3cyclictest1821412-21kthreadcore11:00:400
1589546992217,3cyclictest0-21swapper/010:25:490
1589546992216,5cyclictest1527706-21kworker/u8:2+flush-8:007:16:260
1589546992216,4cyclictest0-21swapper/009:15:380
158954699220,20cyclictest1670730-21sshd08:25:560
158954699220,20cyclictest1670730-21sshd08:25:550
17359842210,2sleep3471rcuc/309:35:323
158954699212,13cyclictest1721123-21taskset09:20:330
1589546992119,1cyclictest0-21swapper/008:05:520
1589546992117,3cyclictest1681835-21sort08:40:370
1589546992117,3cyclictest1605307-21tr07:20:440
1589546992116,3cyclictest1741874-21tr09:40:390
1589546992116,3cyclictest0-21swapper/011:30:380
1589546992115,5cyclictest753075-21kworker/u8:0+flush-8:007:30:460
1589546992115,4cyclictest1784997-21kworker/u8:1+flush-8:011:14:260
190514622010,6sleep30-21swapper/312:20:503
1589546992016,2cyclictest0-21swapper/008:50:410
1589546992015,3cyclictest0-21swapper/008:55:390
1589546992013,5cyclictest1626702-21grep07:45:360
17559112194,11sleep30-21swapper/309:55:313
15891872195,8sleep30-21swapper/307:08:023
15891492194,10sleep20-21swapper/207:07:332
158954699189,7cyclictest1909840-21sed12:25:420
15975262154,8sleep30-21swapper/307:15:403
18201652142,8sleep30-21swapper/311:00:003
17907802143,8sleep30-21swapper/310:30:353
19207482132,7sleep30-21swapper/312:40:013
18307882132,7sleep30-21swapper/311:10:403
18002682132,6sleep30-21swapper/310:40:013
17959642132,7sleep30-21swapper/310:35:373
17794252133,6sleep30-21swapper/310:15:423
17794252133,6sleep30-21swapper/310:15:423
17508982132,6sleep30-21swapper/309:50:023
16013612132,6sleep30-21swapper/307:20:023
19137052122,6sleep30-21swapper/312:30:413
19064572122,6sleep30-21swapper/312:25:363
18951492121,7sleep30-21swapper/312:10:473
18885752122,6sleep30-21swapper/312:05:423
18885752122,6sleep30-21swapper/312:05:423
18817002122,6sleep30-21swapper/312:00:363
18717962123,6sleep30-21swapper/311:50:373
18653232122,6sleep30-21swapper/311:42:063
18552952123,6sleep30-21swapper/311:35:363
18500552122,6sleep30-21swapper/311:30:293
18454022122,6sleep30-21swapper/311:25:373
18349622122,5sleep30-21swapper/311:15:003
18289222122,6sleep30-21swapper/311:05:423
18148782122,6sleep30-21swapper/310:50:543
17744982122,6sleep30-21swapper/310:10:443
17605922122,5sleep30-21swapper/310:00:013
17499232122,7sleep30-21swapper/309:45:493
17354372122,6sleep30-21swapper/309:32:493
17258452122,6sleep30-21swapper/309:25:013
17115352122,6sleep30-21swapper/309:10:373
17070822122,6sleep30-21swapper/309:05:393
16965352122,6sleep30-21swapper/308:55:373
16862082122,6sleep30-21swapper/308:45:333
16500652122,6sleep30-21swapper/308:05:443
16368692122,6sleep30-21swapper/307:55:373
16319282122,6sleep30-21swapper/307:50:393
16263952122,6sleep30-21swapper/307:45:343
16201542122,6sleep30-21swapper/307:35:423
15937832122,6sleep30-21swapper/307:10:513
18962722112,6sleep30-21swapper/312:15:243
18802872112,6sleep30-21swapper/311:55:463
18703022112,5sleep30-21swapper/311:45:423
18213832110,7sleep30-21swapper/311:00:413
18056432113,5sleep30-21swapper/310:45:323
17856082112,6sleep30-21swapper/310:25:323
17806212112,5sleep30-21swapper/310:20:163
17660922112,5sleep30-21swapper/310:05:353
17457772112,5sleep30-21swapper/309:44:593
17249512112,5sleep30-21swapper/309:20:493
17166782112,5sleep30-21swapper/309:15:393
16913192112,5sleep30-21swapper/308:50:303
16822642112,6sleep30-21swapper/308:40:403
16667582112,5sleep30-21swapper/308:25:393
16667582112,5sleep30-21swapper/308:25:383
16609542112,5sleep30-21swapper/308:20:013
16217042112,5sleep30-21swapper/307:40:373
16115182112,5sleep30-21swapper/307:30:333
16068332112,5sleep30-21swapper/307:25:363
18442142102,5sleep30-21swapper/311:20:513
16712852102,5sleep30-21swapper/308:30:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional