You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 00:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Jul 02, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1391788995547,5cyclictest1496285-21latency_hist08:50:000
1391791995347,4cyclictest1396896-21latency_hist07:15:011
1391801994441,2cyclictest1673076-21kworker/u8:3+flush-259:012:35:013
1391801994337,4cyclictest1459602-21cp08:15:003
1391788994336,5cyclictest1444434-21diskstats08:00:150
1391796993833,3cyclictest1474936-21kworker/u8:3+flush-259:008:40:162
1391796993831,5cyclictest1443597-21https08:00:012
1391788993725,10cyclictest1423245-21apt-get07:40:120
1391791993632,3cyclictest1391732-21kworker/u8:0+flush-259:007:20:161
1391791993630,4cyclictest335557-21systemd-journal09:00:001
1391788993630,4cyclictest1703747-21latency_hist12:10:000
1391801993430,3cyclictest1553382-21kworker/u8:0+flush-259:010:20:213
1391801993330,2cyclictest1412732-21kworker/u8:2+flush-259:007:55:233
1391791993329,2cyclictest1402053-21kworker/u8:1+events_unbound07:55:011
1391791993227,4cyclictest335557-21systemd-journal07:30:341
1391791993225,5cyclictest1667848-21cp11:35:001
1391788993223,6cyclictest1465004-21latency_hist08:20:000
1391788993215,2cyclictest0-21swapper/012:30:010
1391801993126,3cyclictest1583355-21ntpq10:10:193
1391796993127,3cyclictest1412732-21kworker/u8:2+events_unbound07:40:182
1391796993026,3cyclictest1656935-21kworker/u8:0+events_unbound11:50:002
1391796993024,4cyclictest1407167-21apt-get07:25:002
1391791993027,2cyclictest1474936-21kworker/u8:3+flush-259:008:50:151
1391791993026,2cyclictest1474936-21kworker/u8:3+flush-259:008:55:251
1391788993024,4cyclictest1574388-21diskstats10:05:130
1391801992925,3cyclictest1735105-21latency_hist12:40:013
1391801992925,3cyclictest1412946-21/usr/sbin/munin07:30:123
1391796992926,2cyclictest1599836-21kworker/u8:0+flush-259:011:00:142
1391796992924,4cyclictest1698981-21diskstats12:05:132
1391788992925,3cyclictest1708874-21kworker/u8:1+flush-259:012:15:130
1391801992825,2cyclictest1517287-21kworker/u8:3+events_unbound10:30:003
1391801992822,4cyclictest335557-21systemd-journal12:00:003
1391796992825,2cyclictest1496331-21kworker/u8:2+flush-259:009:00:202
1391796992822,4cyclictest1630768-21gpgv11:00:002
1391791992824,3cyclictest1517579-21apt-get09:10:121
1391788992822,4cyclictest1615301-21kworker/u8:1+flush-259:010:55:170
1391801992725,2cyclictest1517287-21kworker/u8:3+events_unbound09:40:233
1391801992723,3cyclictest335557-21systemd-journal10:52:433
1391796992723,3cyclictest1402053-21kworker/u8:1+flush-259:008:10:142
1391788992722,3cyclictest1698579-21kworker/u8:0+flush-259:012:05:000
1391788992720,5cyclictest1667207-21latency_hist11:35:010
1391801992623,2cyclictest1641469-21kworker/u8:1+flush-259:011:30:003
1391801992623,2cyclictest1579116-21kworker/u8:2+flush-259:010:39:593
1391801992622,3cyclictest1412732-21kworker/u8:2+flush-259:007:40:163
1391796992620,4cyclictest1548201-21latency_hist09:40:012
1391788992619,5cyclictest509-21rs:main0
1391788992617,6cyclictest1693423-21latency_hist12:00:010
1391801992522,2cyclictest1402053-21kworker/u8:1+events_unbound07:45:143
1391796992520,3cyclictest1397126-21/usr/sbin/munin07:15:142
1391791992521,3cyclictest1698561-21latency_hist12:05:011
1391791992519,4cyclictest335557-21systemd-journal08:11:441
1391788992517,5cyclictest1636305-21latency_hist11:05:000
139178899251,22cyclictest1-21systemd12:24:430
1391801992421,2cyclictest1641469-21kworker/u8:1+flush-259:011:10:183
1391801992421,2cyclictest1465248-21kworker/u8:1+flush-259:008:45:203
1391796992420,3cyclictest1332800-21kworker/u8:2+events_unbound07:10:202
1391796992417,6cyclictest1465248-21kworker/u8:1+flush-259:008:45:152
1391791992421,2cyclictest1402053-21kworker/u8:1+flush-259:008:05:151
1391791992420,3cyclictest1641469-21kworker/u8:1+events_unbound11:15:201
1391791992419,4cyclictest1609288-21ntpq10:35:221
1391801992318,4cyclictest1444094-21cat08:00:013
1391801992312,9cyclictest1474936-21kworker/u8:3+flush-259:008:50:223
1391796992320,2cyclictest509-21rs:main2
1391791992319,3cyclictest1579116-21kworker/u8:2+flush-259:011:00:431
1391791992317,5cyclictest1579110-21latency_hist10:10:011
1391801992218,3cyclictest1719204-21latency_hist12:25:013
1391801992218,3cyclictest1698579-21kworker/u8:0+events_unbound12:10:013
1391801992218,3cyclictest1517287-21kworker/u8:3+flush-259:009:25:253
1391801992218,3cyclictest1402053-21kworker/u8:1+events_unbound07:25:253
1391801992217,3cyclictest1719442-21/usr/sbin/munin12:25:143
1391796992218,3cyclictest1527674-21kworker/u8:2+events_unbound09:45:202
1391796992218,3cyclictest1412732-21kworker/u8:2+flush-259:008:05:012
1391791992219,2cyclictest1553382-21kworker/u8:0+flush-259:010:00:141
1391791992219,2cyclictest1465248-21kworker/u8:1+flush-259:009:05:151
1391791992217,3cyclictest1579116-21kworker/u8:2+flush-259:011:05:171
1391788992213,6cyclictest335557-21systemd-journal12:15:010
1391801992118,2cyclictest1656935-21kworker/u8:0+events_unbound11:40:223
1391801992118,2cyclictest1615301-21kworker/u8:1+flush-259:010:55:163
1391801992118,2cyclictest1391732-21kworker/u8:0+flush-259:007:20:203
1391801992117,3cyclictest1465248-21kworker/u8:1+flush-259:009:06:003
1391801992117,3cyclictest1464130-21ntpq08:15:213
1391796992119,2cyclictest1512053-21kworker/u8:0+events_unbound09:05:212
1391796992117,3cyclictest1517287-21kworker/u8:3+events_unbound10:30:032
1391791992119,2cyclictest1656935-21kworker/u8:0+flush-259:011:45:231
1391791992117,3cyclictest1465277-21apt-get08:20:111
1391791992117,3cyclictest1465248-21kworker/u8:1+flush-259:008:30:161
1391791992117,3cyclictest1391732-21kworker/u8:0+events_unbound07:15:181
1391791992116,4cyclictest1598939-21ntpq10:25:231
1391791992116,4cyclictest1412714-21latency_hist07:30:011
1391788992117,3cyclictest1730160-21kworker/u8:2+flush-259:012:35:250
1391788992116,4cyclictest335557-21systemd-journal11:08:440
1391801992018,2cyclictest1537522-21kworker/u8:1+events_unbound09:30:223
1391801992017,2cyclictest1698579-21kworker/u8:0+flush-259:012:10:133
1391801992016,3cyclictest1641469-21kworker/u8:1+flush-259:011:15:233
1391801992016,3cyclictest1502036-21gpgv08:55:013
1391796992017,2cyclictest1474936-21kworker/u8:3+flush-259:008:35:152
1391796992017,2cyclictest1402053-21kworker/u8:1+flush-259:007:50:162
1391796992016,3cyclictest1568822-21sh10:00:002
1391796992016,3cyclictest1402053-21kworker/u8:1+events_unbound08:05:202
139179699200,11cyclictest1625610-21munin-run10:54:592
1391791992017,3cyclictest1527674-21kworker/u8:2+events_unbound09:30:131
1391791992017,2cyclictest1698579-21kworker/u8:0+events_unbound12:10:231
1391791992017,2cyclictest1579116-21kworker/u8:2+flush-259:010:30:231
1391791992016,3cyclictest1517287-21kworker/u8:3+flush-259:009:25:161
1391791992016,3cyclictest1422110-21ntpq07:35:201
1391788992014,4cyclictest1465248-21kworker/u8:1+flush-259:008:25:150
1391801991913,4cyclictest335557-21systemd-journal10:35:003
1391801991913,4cyclictest1667765-21cp11:35:013
1391796991916,2cyclictest1599836-21kworker/u8:0+events_unbound10:35:152
1391796991916,2cyclictest1553382-21kworker/u8:0+flush-259:010:20:152
1391796991916,2cyclictest1517287-21kworker/u8:3+flush-259:009:15:242
1391796991916,2cyclictest1412732-21kworker/u8:2+events_unbound07:45:162
1391796991916,2cyclictest1412732-21kworker/u8:2+events_unbound07:35:162
1391796991915,3cyclictest1599836-21kworker/u8:0+events_unbound10:45:182
1391791991915,3cyclictest1527674-21kworker/u8:2+events_unbound09:35:141
1391791991914,3cyclictest1491085-21cp08:45:001
1391788991912,5cyclictest1604116-21ntpq10:30:210
1391801991815,2cyclictest1553382-21kworker/u8:0+flush-259:009:50:183
1391801991815,2cyclictest1465248-21kworker/u8:1+flush-259:008:35:253
1391796991815,2cyclictest1474936-21kworker/u8:3+flush-259:008:55:232
1391796991814,3cyclictest1698579-21kworker/u8:0+events_unbound12:10:122
1391796991814,3cyclictest1537522-21kworker/u8:1+flush-259:009:30:232
1391796991814,3cyclictest1465248-21kworker/u8:1+events_unbound08:30:242
1391796991813,3cyclictest1501474-21latency_hist08:55:012
1391791991815,2cyclictest1673076-21kworker/u8:3+events_unbound12:25:241
1391791991815,2cyclictest1553382-21kworker/u8:0+flush-259:010:15:151
1391791991815,2cyclictest1465248-21kworker/u8:1+events_unbound08:45:231
1391791991814,3cyclictest1688301-21kworker/u8:1+flush-259:011:55:211
1391791991814,3cyclictest1465248-21kworker/u8:1+flush-259:008:25:171
1391791991814,3cyclictest1402053-21kworker/u8:1+flush-259:008:00:251
1391801991714,2cyclictest1698579-21kworker/u8:0+flush-259:012:15:163
1391801991714,2cyclictest1656935-21kworker/u8:0+events_unbound11:30:193
1391801991712,4cyclictest1454297-21latency_hist08:10:003
1391801991711,4cyclictest509-21rs:main3
1391796991714,2cyclictest1708874-21kworker/u8:1+flush-259:012:25:002
1391796991714,2cyclictest1402053-21kworker/u8:1+events_unbound07:30:192
1391796991710,5cyclictest335557-21systemd-journal10:29:422
1391791991714,2cyclictest1673076-21kworker/u8:3+events_unbound12:20:231
1391788991714,2cyclictest60350irq/132-enp0s29f1:rx-009:46:440
1391788991713,2cyclictest0-21swapper/010:45:190
1391788991712,4cyclictest1656935-21kworker/u8:0+flush-259:011:50:010
1391788991711,4cyclictest1475530-21/usr/sbin/munin08:30:230
1391788991711,4cyclictest1412732-21kworker/u8:2+flush-259:007:55:150
1391801991613,2cyclictest1527674-21kworker/u8:2+events_unbound09:20:143
1391801991613,2cyclictest1412732-21kworker/u8:2+events_unbound07:35:223
1391801991613,2cyclictest1332800-21kworker/u8:2+events_unbound07:10:223
1391801991612,3cyclictest1517287-21kworker/u8:3+events_unbound09:10:153
1391796991613,2cyclictest1651499-21kworker/u8:2+events_unbound11:35:142
1391796991613,2cyclictest0-21swapper/211:50:012
1391796991613,2cyclictest0-21swapper/211:25:002
1391796991613,2cyclictest0-21swapper/211:24:592
1391796991612,3cyclictest1708874-21kworker/u8:1+flush-259:012:30:212
1391796991612,3cyclictest1667249-21kworker/u8:5+flush-259:012:25:202
1391791991614,2cyclictest1673076-21kworker/u8:3+flush-259:012:05:151
1391791991613,2cyclictest509-21rs:main1
1391791991613,2cyclictest1527674-21kworker/u8:2+flush-259:009:45:211
1391791991613,2cyclictest1517287-21kworker/u8:3+flush-259:009:40:201
1391791991612,3cyclictest1657172-21apt-get11:25:101
1391791991612,2cyclictest1517287-21kworker/u8:3+flush-259:010:00:001
1391791991611,4cyclictest1594212-21kworker/u8:1+flush-259:010:21:001
1391791991611,3cyclictest1615704-21diskstats10:45:131
1391788991613,2cyclictest61150irq/133-enp0s29f1:tx-007:20:150
1391788991613,2cyclictest60350irq/132-enp0s29f1:rx-011:35:190
1391788991613,2cyclictest60350irq/132-enp0s29f1:rx-009:43:490
1391788991613,2cyclictest60350irq/132-enp0s29f1:rx-009:10:360
1391788991613,2cyclictest60350irq/132-enp0s29f1:rx-008:54:070
1391788991612,3cyclictest1656935-21kworker/u8:0+flush-259:011:25:010
1391788991612,3cyclictest1656935-21kworker/u8:0+flush-259:011:25:000
1391788991612,2cyclictest60350irq/132-enp0s29f1:rx-009:55:140
1391788991612,2cyclictest1470151-21turbostat08:25:000
1391788991612,2cyclictest1408270-21irqcore07:25:160
1391801991511,3cyclictest1641469-21kworker/u8:1+events_unbound11:20:153
1391801991511,3cyclictest1641469-21kworker/u8:1+events_unbound11:20:143
1391796991512,2cyclictest1730160-21kworker/u8:2+flush-259:012:35:292
1391796991512,1cyclictest0-21swapper/210:45:002
1391796991511,3cyclictest1651499-21kworker/u8:2+events_unbound11:25:192
1391796991511,3cyclictest1460070-21kworker/u8:3+events_unbound08:15:232
1391791991512,2cyclictest1708874-21kworker/u8:1+events_unbound12:15:131
1391791991512,2cyclictest1656935-21kworker/u8:0+events_unbound11:40:141
1391791991512,2cyclictest1623008-21kthreadcore10:50:181
1391791991512,2cyclictest1412732-21kworker/u8:2+flush-259:007:40:151
139178899159,4cyclictest1615280-21latency_hist10:45:000
1391788991514,1cyclictest60350irq/132-enp0s29f1:rx-009:05:010
1391788991513,2cyclictest60350irq/132-enp0s29f1:rx-011:28:120
1391788991513,2cyclictest60350irq/132-enp0s29f1:rx-010:04:330
1391788991513,1cyclictest60350irq/132-enp0s29f1:rx-009:50:220
1391788991513,1cyclictest60350irq/132-enp0s29f1:rx-009:36:240
1391788991513,1cyclictest60350irq/132-enp0s29f1:rx-009:15:160
1391788991513,1cyclictest60350irq/132-enp0s29f1:rx-007:45:230
1391788991513,1cyclictest60350irq/132-enp0s29f1:rx-007:35:380
1391788991513,1cyclictest171ktimers/011:13:410
1391788991512,2cyclictest60350irq/132-enp0s29f1:rx-010:35:220
1391788991512,2cyclictest60350irq/132-enp0s29f1:rx-010:28:220
1391788991512,2cyclictest60350irq/132-enp0s29f1:rx-009:00:150
1391788991512,2cyclictest60350irq/132-enp0s29f1:rx-008:35:170
1391788991512,2cyclictest0-21swapper/008:15:000
1391788991510,3cyclictest60350irq/132-enp0s29f1:rx-011:50:110
1391801991412,2cyclictest1553382-21kworker/u8:0+flush-259:010:15:243
1391801991410,3cyclictest1537522-21kworker/u8:1+flush-259:009:35:273
139179699149,3cyclictest335557-21systemd-journal11:30:452
1391796991411,2cyclictest1465248-21kworker/u8:1+events_unbound08:20:232
1391796991410,2cyclictest1651499-21kworker/u8:2+flush-259:011:20:002
1391791991411,2cyclictest1553382-21kworker/u8:0+events_unbound09:50:141
1391791991411,2cyclictest1412732-21kworker/u8:2+flush-259:007:45:201
1391791991411,2cyclictest1412732-21kworker/u8:2+events_unbound07:55:171
139178899149,4cyclictest1553382-21kworker/u8:0+flush-259:010:10:230
139178899148,4cyclictest509-21rs:main0
1391788991412,2cyclictest60350irq/132-enp0s29f1:rx-007:11:430
1391788991412,1cyclictest60350irq/132-enp0s29f1:rx-010:51:440
1391788991411,2cyclictest60350irq/132-enp0s29f1:rx-010:15:100
1391788991411,2cyclictest60350irq/132-enp0s29f1:rx-009:22:500
1391788991411,2cyclictest60350irq/132-enp0s29f1:rx-007:50:110
1391788991411,2cyclictest1489813-21ntp_kernel_err08:40:200
139180199139,3cyclictest1656935-21kworker/u8:0+events_unbound11:50:153
139180199139,3cyclictest1474936-21kworker/u8:3+events_unbound08:33:453
139179699137,4cyclictest1563667-21latency_hist09:55:012
1391796991310,2cyclictest1688950-21irqcore11:55:162
1391796991310,2cyclictest1574431-21fschecks_count10:05:142
1391796991310,2cyclictest1553382-21kworker/u8:0+events_unbound10:16:452
139179199139,3cyclictest1615301-21kworker/u8:1+events_unbound10:55:191
139179199139,3cyclictest1610401-21apt-get10:40:121
1391791991311,1cyclictest0-21swapper/111:35:011
1391788991312,1cyclictest60350irq/132-enp0s29f1:rx-007:15:280
1391788991310,2cyclictest60350irq/132-enp0s29f1:rx-011:40:110
1391788991310,2cyclictest60350irq/132-enp0s29f1:rx-009:25:060
1391788991310,2cyclictest60350irq/132-enp0s29f1:rx-008:55:210
1391788991310,2cyclictest60350irq/132-enp0s29f1:rx-007:30:230
139180199129,2cyclictest1678710-21irqcore11:45:153
139180199129,2cyclictest1599836-21kworker/u8:0+events_unbound11:00:263
139179699128,3cyclictest1517287-21kworker/u8:3+events_unbound09:10:212
139179199129,2cyclictest0-21swapper/110:15:011
139179199128,3cyclictest1527674-21kworker/u8:2+events_unbound09:20:111
139178899129,2cyclictest60350irq/132-enp0s29f1:rx-008:05:210
1391788991210,1cyclictest60350irq/132-enp0s29f1:rx-011:15:110
139180199119,1cyclictest1612139-21kthreadcore10:40:173
139180199119,1cyclictest0-21swapper/310:05:253
139180199116,4cyclictest509-21rs:main3
139180199116,4cyclictest1568825-21latency_hist10:00:013
139179699118,2cyclictest1693745-21cpuspeed_turbos12:00:132
139179699118,2cyclictest1584268-21idleruntime-cro10:15:002
139179199118,1cyclictest0-21swapper/109:15:161
139180199108,1cyclictest0-21swapper/309:45:173
139180199106,3cyclictest335557-21systemd-journal07:19:443
139180199104,4cyclictest1491153-21sed08:45:013
139179199107,2cyclictest1412732-21kworker/u8:2+events_unbound08:15:001
139179199107,1cyclictest0-21swapper/111:55:001
13917969997,1cyclictest0-21swapper/209:25:142
13917969984,3cyclictest1407830-21munin-node07:25:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional