You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 07:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jan 01, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1456058994132,7cyclictest1617075-21kworker/u8:2+flush-259:022:30:010
1456069994034,4cyclictest252-21systemd-journal19:50:003
1456065994037,2cyclictest1643488-21kworker/u8:0+events_unbound22:15:222
1456069993934,3cyclictest1560208-21latency_hist20:50:013
1456065993732,3cyclictest1466345-21kworker/u8:0+flush-259:020:05:012
1456065993729,5cyclictest1497238-21latency_hist19:50:002
1456062993731,4cyclictest1764024-21latency_hist00:10:011
1456065993631,3cyclictest1705063-21kworker/u8:1+flush-259:023:50:252
1456069993429,3cyclictest1549514-21latency_hist20:40:013
1456058993430,3cyclictest1508006-21kworker/u8:2+flush-259:020:10:200
1456069993330,2cyclictest1705063-21kworker/u8:1+flush-259:023:50:193
1456069993330,2cyclictest1544357-21kworker/u8:3+flush-259:020:54:593
1456065993327,4cyclictest1762957-21latency_hist00:05:012
1456065993325,5cyclictest252-21systemd-journal23:40:012
1456058993330,2cyclictest1794848-21fwupd00:39:210
1456058993327,4cyclictest1673231-21ntpq22:35:190
1456058993326,5cyclictest1731371-21cp23:34:590
1456065993128,2cyclictest1663854-21kworker/u8:3+events_unbound22:35:012
1456065993126,4cyclictest1679504-21kworker/u8:0+flush-259:000:15:192
1456065993123,3cyclictest1508006-21kworker/u8:2+events_unbound20:30:222
1456058993125,4cyclictest1612128-21/usr/sbin/munin21:40:200
1456069993027,2cyclictest1705063-21kworker/u8:1+flush-259:023:45:223
1456069993027,2cyclictest1679504-21kworker/u8:0+flush-259:023:20:163
1456069993024,4cyclictest509-21rs:main3
1456065993026,3cyclictest1565390-21kworker/u8:4+flush-259:021:10:222
1456069992923,4cyclictest252-21systemd-journal00:13:233
1456065992923,4cyclictest1647494-21ntpq22:10:212
1456065992825,2cyclictest1544357-21kworker/u8:3+flush-259:020:50:182
1456062992825,2cyclictest1466345-21kworker/u8:0+flush-259:019:30:171
1456062992823,3cyclictest252-21systemd-journal22:07:321
1456058992720,5cyclictest1497352-21gpgv19:50:010
1456069992622,2cyclictest1663854-21kworker/u8:3+events_unbound22:35:173
1456065992622,3cyclictest1523487-21kworker/u8:1+flush-259:020:40:202
1456065992619,5cyclictest1674158-21latency_hist22:40:012
1456069992519,4cyclictest252-21systemd-journal19:20:003
1456065992522,2cyclictest1601448-21kworker/u8:3+events_unbound22:05:132
1456065992520,3cyclictest1565390-21kworker/u8:4+events_unbound21:40:122
1456065992519,4cyclictest509-21rs:main2
1456065992519,4cyclictest1476604-21latency_hist19:30:002
1456062992521,3cyclictest1653600-21kworker/u8:1+flush-259:022:25:261
1456065992421,2cyclictest1684699-21apt-get22:50:122
1456065992420,3cyclictest1544357-21kworker/u8:3+events_unbound20:45:212
1456062992420,3cyclictest1565390-21kworker/u8:4+flush-259:021:30:221
1456062992419,4cyclictest1460264-21ntpq19:10:211
1456058992417,5cyclictest1737210-21cat23:40:010
1456069992318,4cyclictest1466345-21kworker/u8:0+flush-259:019:30:243
1456065992320,2cyclictest1679504-21kworker/u8:0+events_unbound23:25:152
1456062992318,4cyclictest1764257-21/usr/sbin/munin00:10:181
145606299231,20cyclictest1-21systemd00:05:011
1456069992218,3cyclictest1679504-21kworker/u8:0+flush-259:023:30:183
1456069992218,3cyclictest1565390-21kworker/u8:4+flush-259:022:10:263
1456069992216,4cyclictest1741394-21ntpq23:40:223
1456065992219,2cyclictest1705063-21kworker/u8:1+flush-259:023:45:172
1456065992219,2cyclictest1705063-21kworker/u8:1+events_unbound23:15:182
1456065992215,5cyclictest1683566-21ntpq22:45:232
1456062992218,3cyclictest1565390-21kworker/u8:4+flush-259:021:00:231
1456062992217,4cyclictest1590346-21ntpq21:15:241
1456062992217,3cyclictest1700142-21/usr/sbin/munin23:05:201
1456062992217,3cyclictest1560635-21diskstats20:50:131
1456062992216,4cyclictest1632673-21cp22:00:001
145606299220,21cyclictest1-21systemd19:55:001
1456069992118,2cyclictest1674209-21kworker/u8:2+flush-259:022:45:183
1456065992118,2cyclictest1784620-21kworker/u8:3+flush-259:000:35:212
1456065992118,2cyclictest1679504-21kworker/u8:0+flush-259:000:20:122
1456065992118,2cyclictest1632746-21latency_hist22:00:002
1456065992118,2cyclictest1596539-21kworker/u8:1+events_unbound21:25:182
1456065992117,2cyclictest1565390-21kworker/u8:4+events_unbound21:20:172
1456065992116,3cyclictest1456005-21kworker/u8:1+events_unbound19:55:182
1456062992117,3cyclictest1742323-21kworker/u8:2+events_unbound23:55:191
1456062992117,3cyclictest1674209-21kworker/u8:2+flush-259:022:40:261
1456062992116,3cyclictest1679504-21kworker/u8:0+flush-259:000:20:221
1456058992116,3cyclictest1779886-21diskstats00:25:140
145605899210,19cyclictest1-21systemd00:20:000
1456069992016,3cyclictest1715352-21apt-get23:20:003
1456069992014,4cyclictest1731845-21/usr/sbin/munin23:35:133
1456065992017,2cyclictest1705063-21kworker/u8:1+flush-259:023:35:262
1456065992017,2cyclictest1508006-21kworker/u8:2+events_unbound20:10:182
1456065992016,3cyclictest1466345-21kworker/u8:0+events_unbound20:20:222
1456065992016,3cyclictest1456005-21kworker/u8:1+events_unbound19:15:212
1456062992016,3cyclictest252-21systemd-journal20:27:341
1456058992017,2cyclictest1555083-21munin-plugin-st20:45:010
1456058992015,4cyclictest1456005-21kworker/u8:1+flush-259:019:50:140
1456058992014,4cyclictest1565348-21latency_hist20:55:010
1456058992013,5cyclictest1778574-21ntpq00:20:210
1456069991916,2cyclictest1520872-21kthreadcore20:10:203
1456069991915,3cyclictest1679504-21kworker/u8:0+flush-259:022:55:013
1456069991915,3cyclictest1674209-21kworker/u8:2+flush-259:022:40:223
1456069991912,5cyclictest1763504-21cp00:05:003
1456065991916,2cyclictest1716445-21irqcore23:20:162
1456065991916,2cyclictest1705063-21kworker/u8:1+events_unbound00:05:042
1456065991916,2cyclictest1565390-21kworker/u8:4+events_unbound21:35:172
1456065991916,2cyclictest1565390-21kworker/u8:4+events_unbound21:35:162
1456065991915,3cyclictest1674209-21kworker/u8:2+events_unbound22:40:342
1456065991915,3cyclictest1456005-21kworker/u8:1+events_unbound19:34:332
1456065991912,5cyclictest252-21systemd-journal00:15:002
1456062991917,1cyclictest1689621-21kworker/u8:3+events_unbound22:55:211
1456062991916,2cyclictest1622893-21irqcore21:50:141
1456062991914,4cyclictest1492361-21apt-get19:45:121
1456062991913,4cyclictest1446018-21kworker/u8:2+events_unbound19:25:011
1456058991917,1cyclictest1596410-21kworker/0:1-events21:53:320
1456058991916,2cyclictest1528592-21cron20:20:000
1456058991915,3cyclictest1544357-21kworker/u8:3+flush-259:020:45:130
1456058991915,2cyclictest509-21rs:main0
1456058991913,4cyclictest1785001-21diskstats00:30:140
1456065991815,3cyclictest1565390-21kworker/u8:4+events_unbound21:00:242
1456065991815,2cyclictest1617075-21kworker/u8:2+events_unbound21:50:202
1456065991814,3cyclictest1617075-21kworker/u8:2+flush-259:022:25:222
1456065991814,3cyclictest1617075-21kworker/u8:2+events_unbound22:20:142
1456065991813,3cyclictest1580912-21kworker/u8:0+flush-259:021:15:202
1456065991813,3cyclictest1455968-21latency_hist19:10:012
1456062991815,2cyclictest1507978-21sh20:00:001
1456062991814,3cyclictest1705063-21kworker/u8:1+flush-259:023:50:181
1456062991813,4cyclictest1544357-21kworker/u8:3+flush-259:020:45:221
1456058991814,3cyclictest1565390-21kworker/u8:4+flush-259:021:20:190
1456058991812,4cyclictest1481791-21latency_hist19:35:010
1456058991811,5cyclictest1758194-21diskstats00:00:150
1456069991712,4cyclictest1544357-21kworker/u8:3+flush-259:021:13:383
1456065991714,2cyclictest1742323-21kworker/u8:2+flush-259:000:25:202
1456065991714,2cyclictest1523487-21kworker/u8:1+flush-259:020:15:262
1456065991713,3cyclictest1679504-21kworker/u8:0+flush-259:023:05:382
1456065991713,3cyclictest1679504-21kworker/u8:0+events_unbound00:33:432
1456062991714,2cyclictest1643488-21kworker/u8:0+flush-259:022:15:171
1456062991711,4cyclictest252-21systemd-journal23:43:411
1456062991710,5cyclictest1653799-21/usr/sbin/munin22:20:151
1456062991710,4cyclictest1669029-21latency_hist22:35:011
1456058991714,2cyclictest60350irq/132-enp0s29f1:rx-021:16:340
1456058991714,2cyclictest60350irq/132-enp0s29f1:rx-019:40:000
1456058991712,4cyclictest1565390-21kworker/u8:4+flush-259:021:00:170
1456058991711,4cyclictest252-21systemd-journal00:10:000
1456058991711,4cyclictest1466568-21apt-get19:20:120
1456069991613,2cyclictest1466345-21kworker/u8:0+flush-259:020:00:183
1456069991612,3cyclictest252-21systemd-journal22:35:013
1456065991613,2cyclictest1466345-21kworker/u8:0+events_unbound20:09:332
1456065991612,3cyclictest1580912-21kworker/u8:0+flush-259:021:34:592
1456065991612,3cyclictest1466345-21kworker/u8:0+events_unbound20:28:352
1456065991612,3cyclictest1446018-21kworker/u8:2+events_unbound19:20:232
1456065991610,4cyclictest1699897-21latency_hist23:05:002
1456065991610,4cyclictest1570556-21latency_hist21:00:012
145606299167,1cyclictest0-21swapper/122:50:011
1456062991613,2cyclictest1679504-21kworker/u8:0+flush-259:023:30:201
1456062991612,3cyclictest1705063-21kworker/u8:1+events_unbound23:15:241
1456062991612,3cyclictest1643488-21kworker/u8:0+flush-259:022:15:001
1456058991614,1cyclictest60350irq/132-enp0s29f1:rx-022:40:240
1456058991613,2cyclictest61150irq/133-enp0s29f1:tx-021:25:130
1456058991613,2cyclictest60350irq/132-enp0s29f1:rx-023:02:260
1456058991613,2cyclictest60350irq/132-enp0s29f1:rx-021:55:220
1456058991613,2cyclictest60350irq/132-enp0s29f1:rx-020:22:560
1456058991613,2cyclictest60350irq/132-enp0s29f1:rx-000:10:200
1456058991613,2cyclictest1491250-21ntp_kernel_pll_19:40:370
145605899160,15cyclictest1-21systemd21:13:330
145606999159,5cyclictest1601495-21latency_hist21:30:013
1456069991511,3cyclictest1456005-21kworker/u8:1+flush-259:019:55:173
1456069991510,4cyclictest1513431-21apt-get20:05:123
1456065991512,2cyclictest1689621-21kworker/u8:3+events_unbound23:10:022
1456065991512,2cyclictest1622214-21latency_hist21:50:002
1456065991512,2cyclictest1565390-21kworker/u8:4+events_unbound22:05:012
1456065991512,2cyclictest1466345-21kworker/u8:0+flush-259:019:35:282
145606299159,4cyclictest252-21systemd-journal00:35:411
145606299159,4cyclictest1736917-21base6423:40:001
1456062991511,3cyclictest1705063-21kworker/u8:1+flush-259:023:45:151
1456062991511,3cyclictest1663854-21kworker/u8:3+flush-259:022:35:151
1456062991511,3cyclictest1544357-21kworker/u8:3+flush-259:020:55:171
1456062991510,4cyclictest252-21systemd-journal20:44:331
1456062991510,3cyclictest1471728-21apt-get19:25:121
1456058991513,1cyclictest60350irq/132-enp0s29f1:rx-023:20:110
1456058991513,1cyclictest60350irq/132-enp0s29f1:rx-019:29:010
1456058991513,1cyclictest60350irq/132-enp0s29f1:rx-019:12:410
1456058991512,2cyclictest60350irq/132-enp0s29f1:rx-023:48:230
1456058991512,2cyclictest60350irq/132-enp0s29f1:rx-021:06:020
1456058991512,2cyclictest60350irq/132-enp0s29f1:rx-020:25:220
1456058991511,3cyclictest60350irq/132-enp0s29f1:rx-023:50:140
1456058991511,2cyclictest1742299-21turbostat23:45:000
1456058991510,4cyclictest1461398-21/usr/sbin/munin19:15:140
1456069991411,2cyclictest1508006-21kworker/u8:2+flush-259:020:30:163
1456069991410,3cyclictest1637762-21journal-offline22:01:253
1456065991411,2cyclictest1544357-21kworker/u8:3+events_unbound20:35:172
1456065991410,3cyclictest1565390-21kworker/u8:4+events_unbound21:09:202
1456065991410,3cyclictest1456005-21kworker/u8:1+flush-259:019:50:182
145606299149,3cyclictest509-21rs:main1
145606299148,4cyclictest1575981-21apt-get21:05:131
1456062991412,1cyclictest509-21rs:main1
1456062991410,2cyclictest1549915-21latency_hist20:40:011
1456058991412,2cyclictest60350irq/132-enp0s29f1:rx-020:34:310
1456058991412,1cyclictest60350irq/132-enp0s29f1:rx-023:10:410
1456058991412,1cyclictest60350irq/132-enp0s29f1:rx-022:55:140
1456058991412,1cyclictest60350irq/132-enp0s29f1:rx-019:55:250
1456058991411,2cyclictest60350irq/132-enp0s29f1:rx-021:37:320
1456058991411,2cyclictest60350irq/132-enp0s29f1:rx-021:37:310
1456058991411,2cyclictest60350irq/132-enp0s29f1:rx-020:00:140
1456058991411,2cyclictest1710895-21irqcore23:15:180
145606999139,2cyclictest1653600-21kworker/u8:1+events_unbound22:25:173
1456069991311,1cyclictest1575701-21cron21:05:003
1456069991311,1cyclictest0-21swapper/323:30:003
1456069991310,2cyclictest1570563-21sh20:59:593
1456069991310,2cyclictest0-21swapper/323:00:173
145606599139,3cyclictest1742323-21kworker/u8:2+events_unbound23:55:232
1456065991310,2cyclictest1689621-21kworker/u8:3+flush-259:022:55:132
1456062991310,2cyclictest1466345-21kworker/u8:0+flush-259:019:40:001
145605899139,3cyclictest1679504-21kworker/u8:0+flush-259:023:25:250
145605899139,2cyclictest60350irq/132-enp0s29f1:rx-021:30:150
1456058991312,1cyclictest60350irq/132-enp0s29f1:rx-023:57:410
1456058991311,1cyclictest60350irq/132-enp0s29f1:rx-023:09:200
1456058991310,2cyclictest60350irq/132-enp0s29f1:rx-022:20:020
1456058991310,2cyclictest509-21rs:main0
1456058991310,2cyclictest1617380-21cpuspeed_turbos21:45:140
145606999129,2cyclictest1784620-21kworker/u8:3+flush-259:000:35:203
145606999129,2cyclictest1742323-21kworker/u8:2+flush-259:023:55:153
145606999129,2cyclictest1565390-21kworker/u8:4+flush-259:021:49:593
145606999129,2cyclictest1466790-21fschecks_time19:20:153
145606999129,2cyclictest1466345-21kworker/u8:0+flush-259:020:20:253
145606999128,3cyclictest1565390-21kworker/u8:4+flush-259:021:05:153
145606999121,9cyclictest1596300-21awk21:24:593
145606999121,3cyclictest1789763-21turbostat.cron00:35:003
1456069991210,2cyclictest1523487-21kworker/u8:1+flush-259:020:40:103
1456069991210,1cyclictest0-21swapper/321:55:173
145606599129,2cyclictest1446018-21kworker/u8:2+events_unbound19:11:222
145606299129,2cyclictest0-21swapper/120:00:201
145606299129,1cyclictest0-21swapper/119:20:001
145606299128,3cyclictest1508006-21kworker/u8:2+flush-259:020:25:001
145606299127,3cyclictest1466345-21kworker/u8:0+flush-259:019:40:271
145606299127,1cyclictest0-21swapper/121:35:261
145606299127,1cyclictest0-21swapper/121:35:261
145606299126,4cyclictest252-21systemd-journal21:44:591
145606299121,9cyclictest1622220-21awk21:50:001
145605899129,2cyclictest1648657-21/usr/sbin/munin22:15:200
145606999119,1cyclictest0-21swapper/321:15:193
145606999118,2cyclictest1781996-21kthreadcore00:25:193
145606999118,2cyclictest1656121-21kthreadcore22:20:193
145606999118,2cyclictest0-21swapper/320:30:013
145606999117,3cyclictest252-21systemd-journal00:05:013
145606999110,10cyclictest1490974-21sshd19:40:333
145606299117,3cyclictest1679504-21kworker/u8:0+flush-259:023:10:391
145606299116,3cyclictest252-21systemd-journal20:15:331
145605899118,2cyclictest60350irq/132-enp0s29f1:rx-022:30:260
1456058991110,1cyclictest60350irq/132-enp0s29f1:rx-020:06:020
145606999109,0cyclictest2149817-21kworker/3:1+events_power_efficient21:30:253
145606999108,1cyclictest0-21swapper/319:55:013
145606999107,2cyclictest1611477-21unixbench_singl21:35:263
145606999107,2cyclictest1611477-21unixbench_singl21:35:253
145606999107,2cyclictest1456461-21fschecks_count19:10:153
145606999106,3cyclictest1689621-21kworker/u8:3+flush-259:023:05:013
145606999105,4cyclictest1-21systemd21:55:013
145606999100,8cyclictest1710203-21awk23:15:003
145606299108,1cyclictest0-21swapper/123:00:141
145606299104,4cyclictest1539352-21base6420:30:011
14560699997,1cyclictest0-21swapper/300:20:263
14560699991,2cyclictest1774315-21awk00:20:003
14560699984,3cyclictest1475119-21tr19:25:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional