You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 05:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Dec 15, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1521681994942,6cyclictest1562854-21fwupd19:49:210
1521691993835,2cyclictest1786983-21kworker/u8:1+events_unbound00:05:253
1521688993429,4cyclictest1584070-21latency_hist20:10:012
1521686993226,4cyclictest252-21systemd-journal22:10:011
1521681993225,5cyclictest252-21systemd-journal23:05:000
1521691993129,1cyclictest1516537-21kworker/u8:3+events_unbound20:00:203
1521686993128,2cyclictest1636650-21kworker/u8:1+flush-259:021:05:221
1521681993122,6cyclictest1573740-21latency_hist20:00:010
1521691993027,2cyclictest1756105-21kworker/u8:0+events_unbound23:20:163
1521686993024,4cyclictest1760311-21ntpq22:55:201
1521681993024,4cyclictest1739340-21ntpq22:35:220
1521691992927,2cyclictest1688178-21kworker/u8:1+events_unbound22:40:153
1521691992926,2cyclictest1833048-21kworker/u8:0+events_unbound00:10:193
1521691992926,2cyclictest1568797-21kworker/u8:1+flush-259:020:05:163
1521686992926,2cyclictest1745575-21kworker/u8:2+flush-259:023:10:211
1521686992926,2cyclictest1688178-21kworker/u8:1+events_unbound21:50:001
1521681992922,5cyclictest1744522-21ntpq22:40:220
1521691992825,2cyclictest1786983-21kworker/u8:1+events_unbound23:45:243
1521691992825,2cyclictest1584119-21kworker/u8:2+events_unbound20:35:263
1521691992825,2cyclictest1584119-21kworker/u8:2+events_unbound20:20:243
1521686992825,2cyclictest1537158-21kworker/u8:2+events_unbound19:35:221
1521686992824,3cyclictest1838531-21kworker/u8:2+events_unbound00:35:251
1521686992824,3cyclictest1786983-21kworker/u8:1+flush-259:023:45:211
1521686992824,3cyclictest1756105-21kworker/u8:0+flush-259:023:50:191
1521681992825,2cyclictest1515701-21kworker/u8:0+flush-259:019:25:000
1521691992725,2cyclictest1516537-21kworker/u8:3+events_unbound19:50:193
1521691992724,2cyclictest1838531-21kworker/u8:2+flush-259:000:35:233
1521691992724,2cyclictest1688178-21kworker/u8:1+events_unbound22:45:183
1521691992724,2cyclictest1610385-21kworker/u8:1+events_unbound20:40:013
1521686992724,2cyclictest1646604-21kworker/u8:4+events_unbound21:10:211
1521691992624,2cyclictest1786983-21kworker/u8:1+events_unbound00:25:133
1521691992623,2cyclictest1756105-21kworker/u8:0+events_unbound23:30:003
1521686992623,2cyclictest1662295-21kworker/u8:2+flush-259:021:35:491
1521681992619,5cyclictest1646980-21latency_hist21:10:000
1521691992523,2cyclictest1625500-21kworker/u8:0+flush-259:020:55:083
1521691992523,2cyclictest1604664-21kworker/u8:0+flush-259:020:30:223
1521691992522,2cyclictest1568797-21kworker/u8:1+events_unbound19:55:333
1521686992522,2cyclictest1584119-21kworker/u8:2+flush-259:021:02:081
1521686992522,2cyclictest1568797-21kworker/u8:1+flush-259:019:55:191
1521681992518,5cyclictest1693822-21cat21:55:020
1521691992422,2cyclictest1756105-21kworker/u8:0+events_unbound23:10:153
1521691992422,2cyclictest1699037-21kworker/u8:3+events_unbound22:55:223
1521691992422,2cyclictest1646604-21kworker/u8:4+events_unbound21:10:313
1521691992422,2cyclictest1631112-21kworker/u8:3+events_unbound21:30:123
1521691992422,2cyclictest1515701-21kworker/u8:0+events_unbound19:15:163
152168899246,6cyclictest1646056-21ntpq21:05:212
1521688992420,3cyclictest1516537-21kworker/u8:3+flush-259:019:20:192
1521688992418,4cyclictest1568777-21cp19:55:012
1521686992421,2cyclictest1838531-21kworker/u8:2+flush-259:000:17:211
1521686992421,2cyclictest1745575-21kworker/u8:2+events_unbound22:50:181
1521686992421,2cyclictest1688178-21kworker/u8:1+events_unbound22:25:201
1521686992421,2cyclictest1631112-21kworker/u8:3+flush-259:020:55:251
1521681992418,4cyclictest1792550-21diskstats23:30:150
1521691992322,1cyclictest1631112-21kworker/u8:3+events_unbound21:05:163
1521691992321,2cyclictest1657112-21kworker/u8:0+events_unbound21:50:223
1521691992321,2cyclictest1516537-21kworker/u8:3+events_unbound19:41:313
152168899230,10cyclictest1833302-21latency_hist00:10:012
1521686992320,2cyclictest1657112-21kworker/u8:0+flush-259:021:50:231
1521686992320,2cyclictest1531904-21kworker/u8:1+events_unbound19:25:211
1521686992320,2cyclictest1516537-21kworker/u8:3+flush-259:019:40:221
1521686992317,2cyclictest1756105-21kworker/u8:0+events_unbound23:25:211
1521681992318,3cyclictest1531904-21kworker/u8:1+flush-259:019:30:000
1521691992219,2cyclictest1688178-21kworker/u8:1+flush-259:022:29:313
1521691992219,2cyclictest1631112-21kworker/u8:3+flush-259:021:35:553
1521688992217,3cyclictest1662689-21/usr/sbin/munin21:25:152
1521686992219,2cyclictest1838531-21kworker/u8:2+events_unbound00:20:151
1521686992219,2cyclictest1833048-21kworker/u8:0+flush-259:000:10:221
1521686992219,2cyclictest1756105-21kworker/u8:0+flush-259:023:30:261
1521686992219,2cyclictest1699037-21kworker/u8:3+flush-259:023:35:191
1521686992217,3cyclictest252-21systemd-journal21:57:181
1521681992219,2cyclictest1719957-21apt-get22:20:140
1521681992214,6cyclictest1599608-21latency_hist20:25:000
152168199221,19cyclictest1-21systemd00:18:220
1521691992119,2cyclictest1756105-21kworker/u8:0+flush-259:023:44:313
1521691992118,2cyclictest1699037-21kworker/u8:3+events_unbound23:37:443
1521691992117,3cyclictest1516537-21kworker/u8:3+events_unbound19:21:523
1521691992114,5cyclictest252-21systemd-journal22:54:593
1521688992118,2cyclictest0-21swapper/222:45:002
1521688992117,3cyclictest1625891-21http20:50:002
1521686992118,2cyclictest1756105-21kworker/u8:0+events_unbound23:20:141
1521686992118,2cyclictest1557507-21kworker/u8:0+flush-259:019:45:201
1521686992118,2cyclictest1516537-21kworker/u8:3+flush-259:019:30:141
1521686992117,3cyclictest1568797-21kworker/u8:1+events_unbound20:10:091
1521686992117,2cyclictest1756105-21kworker/u8:0+flush-259:023:40:191
1521691992018,2cyclictest1657112-21kworker/u8:0+events_unbound22:00:163
1521691992018,2cyclictest1557507-21kworker/u8:0+events_unbound19:47:323
1521691992018,2cyclictest1537158-21kworker/u8:2+events_unbound19:35:243
1521691992017,2cyclictest1531904-21kworker/u8:1+events_unbound19:25:193
1521691992015,2cyclictest1709407-21kworker/u8:0+events_unbound22:35:243
152168899201,5cyclictest1739389-21ntpq22:35:232
1521688992014,4cyclictest1636622-21latency_hist21:00:002
1521686992018,2cyclictest1610385-21kworker/u8:1+events_unbound20:45:221
1521686992017,2cyclictest1786983-21kworker/u8:1+events_unbound00:05:221
1521686992017,2cyclictest1688178-21kworker/u8:1+flush-259:022:40:161
1521686992017,2cyclictest1631112-21kworker/u8:3+flush-259:021:15:221
1521686992016,3cyclictest1688178-21kworker/u8:1+events_unbound22:20:191
1521686992016,2cyclictest1515701-21kworker/u8:0+flush-259:019:15:121
1521686992015,4cyclictest1497795-21kworker/u8:2+flush-259:019:10:131
1521686992015,3cyclictest252-21systemd-journal20:15:311
1521691991917,2cyclictest1703917-21kworker/u8:4+events_unbound22:14:323
1521691991917,2cyclictest1557507-21kworker/u8:0+events_unbound20:10:183
1521691991914,4cyclictest1657534-21/usr/sbin/munin21:20:163
152168899196,7cyclictest1797287-21munin-run23:35:002
152168899194,5cyclictest1825545-21grep00:00:182
1521688991914,3cyclictest252-21systemd-journal21:45:012
1521686991916,2cyclictest1568797-21kworker/u8:1+events_unbound20:05:211
1521686991916,2cyclictest1547436-21kworker/u8:4+events_unbound19:52:331
1521686991915,3cyclictest1786983-21kworker/u8:1+flush-259:000:25:241
1521686991915,3cyclictest1746025-21/usr/sbin/munin22:45:211
1521686991915,2cyclictest0-21swapper/120:24:591
1521681991912,5cyclictest1729035-21ntpq22:25:180
1521691991816,2cyclictest1786983-21kworker/u8:1+flush-259:000:15:463
1521691991816,2cyclictest1699037-21kworker/u8:3+events_unbound23:07:323
1521691991816,2cyclictest1688178-21kworker/u8:1+events_unbound22:20:243
1521691991816,2cyclictest1584119-21kworker/u8:2+flush-259:020:25:273
1521691991815,2cyclictest1657112-21kworker/u8:0+events_unbound21:48:323
1521688991813,3cyclictest252-21systemd-journal21:16:312
1521688991813,3cyclictest1786966-21latency_hist23:25:012
1521688991811,5cyclictest1568523-21cp19:55:012
1521686991815,2cyclictest1812683-21kworker/u8:2+flush-259:023:55:231
1521686991815,2cyclictest1699037-21kworker/u8:3+flush-259:022:15:151
1521686991815,2cyclictest1699037-21kworker/u8:3+events_unbound23:15:231
1521686991814,3cyclictest1699037-21kworker/u8:3+events_unbound22:36:331
1521681991814,2cyclictest0-21swapper/020:29:590
1521681991812,4cyclictest1812662-21latency_hist23:50:010
1521681991811,5cyclictest1531085-21ntpq19:15:200
1521691991714,2cyclictest1786983-21kworker/u8:1+events_unbound23:52:273
1521691991714,2cyclictest1699037-21kworker/u8:3+events_unbound23:15:273
1521691991714,2cyclictest1688178-21kworker/u8:1+flush-259:022:19:323
1521691991714,2cyclictest1688178-21kworker/u8:1+events_unbound21:56:313
1521691991714,2cyclictest1516537-21kworker/u8:3+events_unbound19:33:313
1521691991713,3cyclictest1625500-21kworker/u8:0+flush-259:020:48:513
1521688991714,2cyclictest0-21swapper/220:30:002
1521688991713,3cyclictest500-21systemd-logind21:50:012
1521688991713,3cyclictest0-21swapper/200:40:002
152168899170,5cyclictest1526113-21sensors19:10:202
1521686991714,2cyclictest1631112-21kworker/u8:3+events_unbound21:25:151
1521681991712,3cyclictest1626560-21apt-get20:50:120
1521691991613,2cyclictest1786983-21kworker/u8:1+events_unbound00:21:453
1521691991613,2cyclictest1709407-21kworker/u8:0+events_unbound22:32:273
1521691991612,3cyclictest1662295-21kworker/u8:2+flush-259:021:25:023
152168899168,7cyclictest2971-21cron22:00:012
1521688991613,2cyclictest0-21swapper/221:55:002
1521688991611,4cyclictest1657112-21kworker/u8:0+flush-259:022:00:142
1521688991610,4cyclictest509-21rs:main2
152168899160,6cyclictest1729890-21sshd22:29:312
152168899160,5cyclictest1734211-21ntpq22:30:222
1521686991613,2cyclictest1709407-21kworker/u8:0+flush-259:022:30:161
1521686991613,2cyclictest1631112-21kworker/u8:3+flush-259:021:40:201
1521686991613,2cyclictest1584119-21kworker/u8:2+events_unbound20:29:321
1521681991614,1cyclictest60350irq/132-enp0s29f1:rx-021:30:230
1521681991614,1cyclictest60350irq/132-enp0s29f1:rx-021:25:250
1521681991613,2cyclictest60350irq/132-enp0s29f1:rx-023:15:190
1521681991613,2cyclictest60350irq/132-enp0s29f1:rx-021:36:190
1521681991613,2cyclictest60350irq/132-enp0s29f1:rx-000:03:310
1521681991613,2cyclictest0-21swapper/023:20:190
1521681991611,3cyclictest60350irq/132-enp0s29f1:rx-022:15:220
152168199161,13cyclictest252-21systemd-journal22:03:300
1521681991611,3cyclictest1537158-21kworker/u8:2+flush-259:019:40:010
152169199159,4cyclictest252-21systemd-journal19:11:313
1521691991513,2cyclictest1646604-21kworker/u8:4+flush-259:021:15:173
1521691991512,2cyclictest1786983-21kworker/u8:1+events_unbound00:34:453
1521691991512,2cyclictest1610385-21kworker/u8:1+flush-259:020:52:503
1521691991512,2cyclictest1584119-21kworker/u8:2+flush-259:021:02:333
1521691991512,2cyclictest1568797-21kworker/u8:1+events_unbound20:16:323
1521691991511,3cyclictest1657112-21kworker/u8:0+events_unbound21:41:323
152168899159,4cyclictest1750951-21latency_hist22:50:002
152168899151,5cyclictest1548316-21irqcore19:35:182
1521688991511,2cyclictest0-21swapper/223:50:212
1521688991511,2cyclictest0-21swapper/223:40:242
152168899150,4cyclictest1822072-21ntpq23:55:212
152168899150,4cyclictest1673111-21apt-get21:35:422
1521686991512,2cyclictest1699037-21kworker/u8:3+flush-259:023:05:161
1521686991512,2cyclictest1615727-21kworker/u8:3+events_unbound20:42:041
152168199158,5cyclictest1864195-21latency_hist00:40:010
1521681991513,2cyclictest60350irq/132-enp0s29f1:rx-019:53:590
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-022:30:040
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-022:05:360
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-021:20:160
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-020:15:160
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-019:10:200
1521681991513,1cyclictest60350irq/132-enp0s29f1:rx-000:06:550
1521681991511,3cyclictest60350irq/132-enp0s29f1:rx-023:25:240
1521681991511,3cyclictest1833048-21kworker/u8:0+flush-259:000:10:150
1521681991511,3cyclictest1646604-21kworker/u8:4+flush-259:021:10:250
1521681991510,3cyclictest1557718-21turbostat19:45:000
1521681991510,3cyclictest1542514-21apt-get19:30:120
1521681991510,3cyclictest1-21systemd23:55:410
152169199141,5cyclictest491ktimers/323:55:173
1521691991411,2cyclictest1699037-21kworker/u8:3+flush-259:022:05:273
1521691991410,3cyclictest1756105-21kworker/u8:0+flush-259:023:03:023
152168899149,3cyclictest1516537-21kworker/u8:3+flush-259:019:30:162
152168899148,4cyclictest252-21systemd-journal21:11:312
152168899148,4cyclictest1839016-21cut00:15:152
1521688991411,2cyclictest507-21in:imuxsock20:34:592
1521688991411,2cyclictest1710652-21kthreadcore22:10:202
152168899140,4cyclictest1792131-21/usr/sbin/munin23:29:592
152168899140,4cyclictest1671899-21collect221:30:212
1521686991411,2cyclictest1786983-21kworker/u8:1+events_unbound00:34:571
1521686991411,2cyclictest1578445-21unixbench_multi20:00:231
1521686991410,3cyclictest1610385-21kworker/u8:1+flush-259:020:50:311
152168199148,4cyclictest1859084-21latency_hist00:35:000
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-023:40:230
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-021:55:250
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-021:00:240
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-020:46:460
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-020:31:060
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-020:10:150
1521681991412,1cyclictest60350irq/132-enp0s29f1:rx-000:20:270
1521681991411,2cyclictest60350irq/132-enp0s29f1:rx-023:35:450
1521681991411,2cyclictest60350irq/132-enp0s29f1:rx-021:47:260
1521681991411,2cyclictest60350irq/132-enp0s29f1:rx-020:42:350
152168899139,3cyclictest1626484-21base6420:50:002
152168899139,3cyclictest1547436-21kworker/u8:4+flush-259:019:50:002
152168899139,2cyclictest1802407-21cron23:40:002
152168899137,4cyclictest252-21systemd-journal20:36:322
152168899136,5cyclictest1579018-21cat20:05:012
152168899131,9cyclictest1594411-21latency_hist20:20:002
1521688991310,2cyclictest0-21swapper/222:55:002
152168899130,4cyclictest1761465-21/usr/sbin/munin23:00:262
152168899130,4cyclictest1589285-21munin-run20:14:592
152168899130,3cyclictest1777091-21tune2fs23:15:162
152168699139,3cyclictest1662295-21kworker/u8:2+events_unbound21:32:321
152168699139,3cyclictest1516537-21kworker/u8:3+flush-259:019:20:201
1521686991310,2cyclictest1636650-21kworker/u8:1+events_unbound21:22:311
1521686991310,2cyclictest1604664-21kworker/u8:0+flush-259:020:34:371
1521681991312,1cyclictest60350irq/132-enp0s29f1:rx-023:51:120
1521681991311,2cyclictest60350irq/132-enp0s29f1:rx-023:09:410
1521681991311,1cyclictest60350irq/132-enp0s29f1:rx-022:55:130
1521681991311,1cyclictest60350irq/132-enp0s29f1:rx-021:40:120
1521681991310,2cyclictest60350irq/132-enp0s29f1:rx-021:17:130
1521681991310,2cyclictest60350irq/132-enp0s29f1:rx-020:35:290
1521681991310,2cyclictest60350irq/132-enp0s29f1:rx-020:05:190
1521681991310,2cyclictest60350irq/132-enp0s29f1:rx-000:25:130
1521681991310,2cyclictest0-21swapper/022:10:200
152169199128,1cyclictest191rcu_preempt00:00:223
152168899129,1cyclictest0-21swapper/200:20:142
152168899127,4cyclictest1-21systemd20:45:002
152168899126,4cyclictest1709367-21latency_hist22:10:012
152168899121,3cyclictest1775744-21ntpq23:10:212
152168899120,4cyclictest1837631-21ntpq00:10:222
152168699129,2cyclictest1761559-21cpuspeed_turbos23:00:151
152168699129,2cyclictest1703917-21kworker/u8:4+events_unbound22:10:191
152168699129,2cyclictest1699681-21sed22:00:171
152168699129,2cyclictest1604664-21kworker/u8:0+events_unbound20:37:321
152168199129,2cyclictest60350irq/132-enp0s29f1:rx-020:00:220
1521681991210,1cyclictest60350irq/132-enp0s29f1:rx-020:58:250
152168199120,9cyclictest1751213-21apt-get22:50:140
152168899118,2cyclictest1714838-21cpuspeed_turbos22:15:162
152168899118,1cyclictest0-21swapper/219:30:012
152168899111,3cyclictest1766891-21fw_conntrack23:05:152
152168899111,2cyclictest1854362-21df_inode00:30:152
152168899111,2cyclictest1760386-21ntpq22:55:232
152168899110,3cyclictest1553140-21hddtemp_smartct19:40:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional