You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 04:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Jul 15, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1461934993831,5cyclictest335557-21systemd-journal22:20:002
1461934993731,4cyclictest1772302-21latency_hist00:10:002
1461934993432,1cyclictest1627634-21kworker/u8:2+events_unbound21:55:242
1461934993330,2cyclictest1690023-21kworker/u8:0+flush-259:023:00:242
1461937993228,3cyclictest1595987-21kworker/u8:3+flush-259:022:30:013
1461937993226,4cyclictest1803669-21latency_hist00:40:013
1461934993225,5cyclictest335557-21systemd-journal19:20:002
1461934993225,5cyclictest335557-21systemd-journal19:20:002
1461934993128,2cyclictest1492903-21kworker/u8:2+events_unbound20:00:012
1461922993024,4cyclictest1684632-21latency_hist22:45:000
1461937992924,4cyclictest335557-21systemd-journal21:16:203
1461934992926,2cyclictest1751585-21kworker/u8:4+events_unbound00:15:172
1461934992923,2cyclictest1690023-21kworker/u8:0+events_unbound23:45:202
1461937992825,2cyclictest1710617-21kworker/u8:1+events_unbound00:00:283
1461934992825,2cyclictest1467273-21kworker/u8:4+events_unbound20:25:242
1461928992824,3cyclictest335557-21systemd-journal21:29:201
1461934992724,2cyclictest1710617-21kworker/u8:1+events_unbound00:20:182
1461934992723,3cyclictest1710617-21kworker/u8:1+flush-259:000:25:172
1461934992723,3cyclictest1710617-21kworker/u8:1+events_unbound00:30:202
1461934992720,5cyclictest335557-21systemd-journal22:30:012
1461934992720,2cyclictest1492903-21kworker/u8:2+events_unbound19:50:222
1461922992721,4cyclictest1549928-21latency_hist20:35:010
1461934992623,2cyclictest1690023-21kworker/u8:0+events_unbound23:14:592
1461934992623,2cyclictest1643016-21kworker/u8:4+events_unbound22:35:212
1461934992623,2cyclictest1643016-21kworker/u8:4+events_unbound22:05:242
1461934992623,2cyclictest1467273-21kworker/u8:4+flush-259:020:05:202
1461934992623,2cyclictest1422852-21kworker/u8:0+events_unbound19:20:242
1461934992622,3cyclictest1710617-21kworker/u8:1+flush-259:023:20:182
1461934992622,3cyclictest1555303-21kworker/u8:2+events_unbound21:15:122
1461934992621,3cyclictest509-21rs:main2
1461934992621,3cyclictest1710617-21kworker/u8:1+events_unbound23:25:262
1461934992621,3cyclictest1487956-21/usr/sbin/munin19:35:152
1461928992622,3cyclictest1663677-21gpgv22:25:011
1461934992522,2cyclictest1690023-21kworker/u8:0+events_unbound23:58:332
1461934992522,2cyclictest1690023-21kworker/u8:0+events_unbound22:50:142
1461934992522,2cyclictest1643016-21kworker/u8:4+events_unbound22:45:252
1461934992522,2cyclictest1595987-21kworker/u8:3+flush-259:023:05:202
1461934992522,2cyclictest1549976-21kworker/u8:1+flush-259:022:00:152
1461934992522,2cyclictest1467273-21kworker/u8:4+flush-259:019:45:162
1461934992518,5cyclictest1627588-21latency_hist21:50:002
1461928992521,3cyclictest1575903-21kworker/u8:3+flush-259:021:10:001
1461922992520,4cyclictest1560233-21latency_hist20:45:010
1461934992421,2cyclictest1643016-21kworker/u8:4+flush-259:022:30:252
1461934992421,2cyclictest1595987-21kworker/u8:3+flush-259:022:55:242
1461934992417,5cyclictest335557-21systemd-journal19:30:002
1461928992420,3cyclictest1736160-21kworker/u8:2+flush-259:023:39:591
1461928992420,3cyclictest1549976-21kworker/u8:1+flush-259:021:00:251
1461928992420,2cyclictest1617601-21apt-get21:40:131
1461922992421,2cyclictest1690023-21kworker/u8:0+flush-259:023:30:000
1461922992421,2cyclictest1428560-21kworker/u8:3+flush-259:019:10:210
1461922992418,4cyclictest1580809-21latency_hist21:05:010
1461937992320,2cyclictest1633016-21apt-get21:55:133
1461934992320,2cyclictest1751585-21kworker/u8:4+events_unbound00:00:152
1461934992320,2cyclictest1653376-21kworker/u8:0+events_unbound22:20:162
1461934992320,2cyclictest1555303-21kworker/u8:2+events_unbound20:50:152
1461922992320,2cyclictest1595987-21kworker/u8:3+flush-259:022:45:190
1461922992319,3cyclictest1503471-21apt-get19:50:120
1461937992219,2cyclictest1663648-21sh22:25:003
1461937992218,3cyclictest1472231-21latency_hist19:20:003
1461937992218,3cyclictest1472231-21latency_hist19:20:003
1461934992219,2cyclictest1772527-21kworker/u8:2+events_unbound00:36:402
1461934992219,2cyclictest1492903-21kworker/u8:2+events_unbound20:10:162
1461934992218,3cyclictest1570535-21kworker/u8:0+flush-259:021:05:202
1461934992218,3cyclictest1549976-21kworker/u8:1+flush-259:021:10:252
1461934992217,3cyclictest1422852-21kworker/u8:0+flush-259:020:30:192
1461937992118,2cyclictest1595987-21kworker/u8:3+flush-259:021:45:193
1461937992117,3cyclictest1467273-21kworker/u8:4+flush-259:020:10:003
1461934992118,2cyclictest1528946-21kworker/u8:1+flush-259:020:20:252
1461934992118,2cyclictest1467273-21kworker/u8:4+events_unbound19:40:232
1461934992117,3cyclictest1751585-21kworker/u8:4+events_unbound00:10:342
1461934992117,3cyclictest1726045-21kworker/u8:3+events_unbound23:38:332
1461934992117,3cyclictest1549976-21kworker/u8:1+events_unbound20:40:142
1461928992117,3cyclictest1643016-21kworker/u8:4+flush-259:022:35:001
1461928992115,4cyclictest1565371-21latency_hist20:50:001
1461922992115,5cyclictest1658923-21diskstats22:20:140
1461937992017,2cyclictest1606773-21store21:30:003
1461934992017,2cyclictest1595987-21kworker/u8:3+events_unbound22:10:232
1461934992017,2cyclictest1549976-21kworker/u8:1+events_unbound20:55:212
1461934992015,3cyclictest1549976-21kworker/u8:1+events_unbound21:39:222
1461934992015,3cyclictest1549976-21kworker/u8:1+events_unbound21:39:222
1461928992016,3cyclictest1477680-21apt-get19:25:111
146192899200,19cyclictest1761915-21dirname00:00:001
1461937991917,1cyclictest0-21swapper/323:59:593
1461937991914,4cyclictest1700379-21apt-get23:00:103
1461934991916,2cyclictest1595987-21kworker/u8:3+flush-259:022:40:162
1461928991916,2cyclictest1550198-21apt-get20:35:141
1461928991915,3cyclictest507-21in:imuxsock23:45:001
1461922991915,3cyclictest1637908-21kworker/u8:0+flush-259:022:05:000
1461922991915,3cyclictest1422852-21kworker/u8:0+flush-259:019:20:430
1461937991816,1cyclictest1564459-21ntp_kernel_err20:45:203
1461937991814,3cyclictest335557-21systemd-journal20:59:553
1461934991815,2cyclictest1595987-21kworker/u8:3+events_unbound21:25:232
1461934991815,2cyclictest1549976-21kworker/u8:1+events_unbound21:50:152
1461934991815,2cyclictest1467273-21kworker/u8:4+events_unbound20:35:252
1461934991814,3cyclictest1710617-21kworker/u8:1+events_unbound23:41:502
1461928991816,1cyclictest0-21swapper/120:10:001
1461928991815,2cyclictest1690023-21kworker/u8:0+flush-259:000:05:181
1461928991815,2cyclictest1539191-21cron20:25:011
1461928991814,3cyclictest1492903-21kworker/u8:2+events_unbound20:12:031
1461928991814,2cyclictest0-21swapper/119:40:001
1461928991813,4cyclictest1793599-21apt-get00:30:111
1461928991813,3cyclictest335557-21systemd-journal22:45:231
1461922991812,4cyclictest1498517-21diskstats19:45:140
1461937991711,4cyclictest1700118-21latency_hist23:00:013
1461934991713,3cyclictest1690023-21kworker/u8:0+events_unbound23:15:082
1461934991713,3cyclictest1482809-21kworker/u8:1+flush-259:019:30:162
1461934991713,3cyclictest1467273-21kworker/u8:4+flush-259:020:17:262
1461934991710,4cyclictest509-21rs:main2
1461928991713,3cyclictest1710617-21kworker/u8:1+flush-259:023:51:061
1461928991713,3cyclictest1607252-21apt-get21:30:121
1461928991713,3cyclictest1595987-21kworker/u8:3+flush-259:023:05:191
1461928991712,4cyclictest1679493-21latency_hist22:40:011
1461922991714,2cyclictest1595987-21kworker/u8:3+flush-259:023:01:080
1461922991714,2cyclictest1595987-21kworker/u8:3+flush-259:021:21:120
1461922991713,3cyclictest1565638-21apt-get20:50:120
1461922991713,2cyclictest60350irq/132-enp0s29f1:rx-000:30:010
1461922991712,3cyclictest0-21swapper/023:00:000
1461934991613,2cyclictest1726045-21kworker/u8:3+events_unbound23:31:462
1461934991613,2cyclictest1549976-21kworker/u8:1+flush-259:021:00:292
1461928991612,3cyclictest509-21rs:main1
1461928991612,3cyclictest1751585-21kworker/u8:4+events_unbound00:15:211
1461922991614,2cyclictest60350irq/132-enp0s29f1:rx-023:51:500
1461922991614,1cyclictest60350irq/132-enp0s29f1:rx-023:21:500
1461922991614,1cyclictest60350irq/132-enp0s29f1:rx-022:35:150
1461922991614,1cyclictest60350irq/132-enp0s29f1:rx-021:35:270
1461922991614,1cyclictest60350irq/132-enp0s29f1:rx-021:35:260
1461922991614,1cyclictest60350irq/132-enp0s29f1:rx-020:55:260
1461922991614,1cyclictest0-21swapper/023:15:000
1461922991612,3cyclictest1595987-21kworker/u8:3+flush-259:022:15:110
1461922991611,4cyclictest1736406-21apt-get23:35:120
1461922991611,3cyclictest335557-21systemd-journal19:32:190
146193799159,4cyclictest509-21rs:main3
1461937991512,1cyclictest0-21swapper/319:35:003
1461937991510,3cyclictest1746677-21/usr/sbin/munin23:45:143
146193799150,13cyclictest1-21systemd23:19:593
1461934991512,2cyclictest1549976-21kworker/u8:1+events_unbound20:47:492
1461928991513,1cyclictest0-21swapper/120:00:201
1461928991512,2cyclictest1725797-21cron23:25:001
1461928991512,2cyclictest1695218-21apt-get22:55:121
1461928991512,2cyclictest1642545-21unixbench_multi22:00:241
1461928991511,3cyclictest1595987-21kworker/u8:3+flush-259:021:51:071
1461922991514,1cyclictest60350irq/132-enp0s29f1:rx-023:31:570
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-023:55:210
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-023:15:250
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:190
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-022:10:260
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-021:43:420
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-020:45:250
1461922991513,1cyclictest60350irq/132-enp0s29f1:rx-019:40:250
146193799149,3cyclictest509-21rs:main3
1461937991412,1cyclictest0-21swapper/323:44:593
1461937991411,2cyclictest1467273-21kworker/u8:4+events_unbound20:00:203
1461937991410,3cyclictest1422852-21kworker/u8:0+flush-259:019:20:213
1461937991410,2cyclictest1731709-21sed23:30:183
1461934991410,3cyclictest1595987-21kworker/u8:3+events_unbound21:41:342
1461934991410,3cyclictest1428560-21kworker/u8:3+events_unbound19:14:172
1461928991410,3cyclictest1539204-21apt-get20:25:011
1461922991412,2cyclictest60350irq/132-enp0s29f1:rx-022:32:040
1461922991412,1cyclictest60350irq/132-enp0s29f1:rx-022:25:160
1461922991412,1cyclictest60350irq/132-enp0s29f1:rx-022:05:290
1461922991412,1cyclictest60350irq/132-enp0s29f1:rx-021:48:520
1461922991412,1cyclictest60350irq/132-enp0s29f1:rx-021:10:230
1461922991412,1cyclictest60350irq/132-enp0s29f1:rx-021:05:190
1461922991412,1cyclictest0-21swapper/020:25:010
1461922991411,2cyclictest60350irq/132-enp0s29f1:rx-020:35:140
1461922991411,2cyclictest60350irq/132-enp0s29f1:rx-020:27:270
1461922991411,2cyclictest60350irq/132-enp0s29f1:rx-000:15:130
1461922991411,2cyclictest1797539-21needreboot00:30:180
1461922991411,2cyclictest1783267-21irqcore00:20:170
1461922991411,2cyclictest1509024-21irqcore19:55:170
146193799139,3cyclictest1710617-21kworker/u8:1+flush-259:000:30:413
146193799139,2cyclictest1538247-21cut20:20:203
146193799138,3cyclictest335557-21systemd-journal22:40:013
1461937991311,1cyclictest0-21swapper/321:25:003
1461937991310,2cyclictest1751585-21kworker/u8:4+flush-259:023:51:373
1461937991310,2cyclictest1550456-21fschecks_time20:35:153
1461937991310,2cyclictest1467273-21kworker/u8:4+flush-259:019:35:423
146193499139,3cyclictest1467273-21kworker/u8:4+flush-259:020:04:242
146192899139,3cyclictest335557-21systemd-journal21:36:201
146192899139,3cyclictest335557-21systemd-journal21:36:201
146192899139,3cyclictest1710659-21apt-get23:10:121
146192899136,6cyclictest1591517-21grep21:15:141
146192899136,5cyclictest335557-21systemd-journal21:25:011
1461928991311,1cyclictest0-21swapper/100:25:001
1461928991310,2cyclictest1731018-21cron23:30:001
146192299138,4cyclictest1745569-21ntpq23:40:200
146192299137,4cyclictest1607413-21diskstats21:30:150
1461922991312,1cyclictest60350irq/132-enp0s29f1:rx-019:29:010
1461922991312,1cyclictest60350irq/132-enp0s29f1:rx-019:16:430
1461922991312,1cyclictest60350irq/132-enp0s29f1:rx-019:16:430
1461922991311,1cyclictest60350irq/132-enp0s29f1:rx-022:54:210
1461922991311,1cyclictest60350irq/132-enp0s29f1:rx-020:00:100
1461922991311,1cyclictest60350irq/132-enp0s29f1:rx-019:35:160
1461922991311,1cyclictest60350irq/132-enp0s29f1:rx-000:14:520
146193799129,2cyclictest1690124-21cpuspeed_turbos22:50:143
146193799129,2cyclictest1607657-21irqcore21:30:163
146193799129,2cyclictest0-21swapper/319:50:013
146193799129,1cyclictest0-21swapper/323:40:003
146193799128,3cyclictest1595987-21kworker/u8:3+flush-259:021:50:423
146193799128,3cyclictest1560231-21sh20:45:013
1461937991210,1cyclictest0-21swapper/321:04:593
146192899129,2cyclictest1798819-21apt-get00:35:131
146192899129,2cyclictest1772673-21cpuspeed_turbos00:10:131
146192899129,2cyclictest1549976-21kworker/u8:1+flush-259:020:55:201
146192899129,2cyclictest1502242-21needreboot19:45:201
146192899128,3cyclictest509-21rs:main1
146192899128,3cyclictest1586218-21apt-get21:10:111
146192899128,2cyclictest0-21swapper/123:05:011
146192899127,4cyclictest1643274-21apt-get22:05:151
1461922991211,1cyclictest60350irq/132-enp0s29f1:rx-021:15:570
1461922991210,1cyclictest60350irq/132-enp0s29f1:rx-021:25:160
1461922991210,1cyclictest60350irq/132-enp0s29f1:rx-020:13:210
1461922991210,1cyclictest60350irq/132-enp0s29f1:rx-000:05:160
146193799119,1cyclictest0-21swapper/322:40:223
146193799119,1cyclictest0-21swapper/322:10:193
146193799119,1cyclictest0-21swapper/319:25:253
146193799117,3cyclictest1555303-21kworker/u8:2+flush-259:020:50:273
146192899118,2cyclictest1792008-21ntpq00:25:221
146192899118,2cyclictest1751067-21unixbench_multi23:45:251
146192899118,2cyclictest1555455-21cpuspeed_turbos20:40:131
146192899118,2cyclictest0-21swapper/120:30:181
146192899117,3cyclictest335557-21systemd-journal22:11:241
146192899117,2cyclictest0-21swapper/122:30:001
146192899117,2cyclictest0-21swapper/119:35:001
146192899116,3cyclictest335557-21systemd-journal19:20:431
146192299118,2cyclictest60350irq/132-enp0s29f1:rx-023:45:220
146192299111,3cyclictest1803661-21awk00:40:000
146193799108,1cyclictest0-21swapper/322:00:153
146193799108,1cyclictest0-21swapper/320:20:003
146193799107,2cyclictest1774845-21kthreadcore00:10:183
146193799104,5cyclictest1669499-21apt-get22:30:113
146192899104,4cyclictest1715958-21diskstats23:15:141
146192299108,1cyclictest60350irq/132-enp0s29f1:rx-021:55:140
146192299107,2cyclictest1628452-21irqcore21:50:180
14619379995,3cyclictest1786196-21kthreadcore00:20:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional