You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Mar 27, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1648537994440,3cyclictest1811659-21kworker/u8:0+events_unbound23:00:141
1648534994034,4cyclictest1907207-21/usr/sbin/munin23:45:140
1648537993934,4cyclictest1922812-21latency_hist00:00:011
1648537993834,3cyclictest3776209-21kworker/u8:4+events_unbound20:55:141
1648537993633,2cyclictest1834152-21kworker/u8:2+events_unbound23:10:161
1648537993532,2cyclictest1773921-21kworker/u8:4+events_unbound21:20:151
1648537993432,2cyclictest1747514-21kworker/u8:1+events_unbound20:50:231
1648537993229,2cyclictest1876247-21kworker/u8:3+events_unbound23:20:141
1648537993229,2cyclictest1747514-21kworker/u8:1+events_unbound21:00:131
1648537993228,3cyclictest1891502-21kworker/u8:0+flush-259:023:50:161
1648537993128,2cyclictest1811659-21kworker/u8:0+events_unbound22:40:131
1648537993126,4cyclictest1732011-21latency_hist20:30:001
1648537993027,2cyclictest1922838-21kworker/u8:2+flush-259:000:05:171
1648537993024,5cyclictest1906971-21latency_hist23:45:011
1648534993025,4cyclictest1917273-21gpgv23:55:010
1648534993025,3cyclictest1881444-21apt-get23:20:110
1648537992926,2cyclictest2790750-21kworker/u8:1+events_unbound19:45:211
1648537992926,2cyclictest2790750-21kworker/u8:1+events_unbound19:35:151
1648537992926,2cyclictest1953760-21kworker/u8:0+events_unbound00:30:241
1648537992923,4cyclictest1779092-21/usr/sbin/munin21:15:131
1648542992825,2cyclictest1811659-21kworker/u8:0+flush-259:022:40:203
1648537992826,2cyclictest1953760-21kworker/u8:0+flush-259:000:35:151
1648537992825,2cyclictest3776209-21kworker/u8:4+events_unbound19:20:181
1648537992825,2cyclictest1695889-21kworker/u8:2+events_unbound20:10:211
1648537992824,3cyclictest1773921-21kworker/u8:4+events_unbound22:20:231
1648540992724,2cyclictest1695889-21kworker/u8:2+flush-259:020:05:162
1648540992722,4cyclictest1844814-21apt-get22:45:122
1648537992725,2cyclictest2790750-21kworker/u8:1+flush-259:019:10:251
1648537992724,2cyclictest1901316-21kworker/u8:1+events_unbound00:25:141
1648537992724,2cyclictest1811659-21kworker/u8:0+events_unbound23:05:231
1648537992720,5cyclictest1834606-21diskstats22:35:141
1648542992625,1cyclictest1737128-21kworker/u8:0+events_unbound20:35:143
1648542992625,1cyclictest1737128-21kworker/u8:0+events_unbound20:35:133
1648542992623,2cyclictest1695889-21kworker/u8:2+flush-259:020:20:153
1648540992623,2cyclictest1901316-21kworker/u8:1+flush-259:000:25:182
1648540992623,2cyclictest1773921-21kworker/u8:4+flush-259:021:15:192
1648537992624,2cyclictest1891502-21kworker/u8:0+events_unbound23:35:131
1648537992623,2cyclictest1901316-21kworker/u8:1+flush-259:000:15:161
1648537992623,2cyclictest1811659-21kworker/u8:0+events_unbound22:30:131
1648537992623,2cyclictest1783763-21kworker/u8:0+events_unbound21:35:231
1648537992623,2cyclictest1747514-21kworker/u8:1+events_unbound21:25:111
1648537992623,2cyclictest1737128-21kworker/u8:0+events_unbound20:40:211
1648537992623,2cyclictest1737128-21kworker/u8:0+events_unbound20:40:201
1648537992618,2cyclictest2790750-21kworker/u8:1+events_unbound19:55:191
164853499260,24cyclictest1-21systemd22:09:590
1648542992522,3cyclictest3776209-21kworker/u8:4+flush-259:019:15:243
1648540992522,2cyclictest1747514-21kworker/u8:1+events_unbound21:25:132
1648540992520,3cyclictest252-21systemd-journal22:16:152
1648537992523,2cyclictest3776209-21kworker/u8:4+flush-259:019:50:161
1648537992523,2cyclictest1773921-21kworker/u8:4+flush-259:022:25:261
1648537992521,3cyclictest1737128-21kworker/u8:0+flush-259:020:40:011
1648537992521,3cyclictest1737128-21kworker/u8:0+flush-259:020:40:001
1648537992520,3cyclictest1658835-21latency_hist19:20:011
1648534992522,2cyclictest1844573-21kworker/u8:4+events_unbound23:00:010
1648542992421,2cyclictest1891713-21kworker/u8:4+flush-259:023:35:203
1648542992421,2cyclictest1747514-21kworker/u8:1+events_unbound21:25:163
1648540992420,3cyclictest1901316-21kworker/u8:1+flush-259:023:45:102
1648540992419,3cyclictest1742597-21apt-get20:40:112
1648540992419,3cyclictest1742597-21apt-get20:40:102
1648540992419,2cyclictest1695889-21kworker/u8:2+events_unbound20:00:132
1648537992422,2cyclictest1773921-21kworker/u8:4+events_unbound22:00:141
1648537992421,2cyclictest1811659-21kworker/u8:0+events_unbound22:55:171
1648537992419,4cyclictest1938281-21latency_hist00:15:001
1648542992320,2cyclictest1773921-21kworker/u8:4+flush-259:021:40:223
1648542992319,3cyclictest2790750-21kworker/u8:1+flush-259:019:40:213
1648542992319,1cyclictest1820615-21kworker/u8:1+events_unbound22:35:173
1648542992317,5cyclictest1773686-21latency_hist21:10:003
1648542992316,5cyclictest1675376-21cat19:35:013
1648540992320,2cyclictest2790750-21kworker/u8:1+events_unbound19:45:192
1648540992319,3cyclictest1811659-21kworker/u8:0+flush-259:023:00:202
1648537992320,2cyclictest3776209-21kworker/u8:4+events_unbound20:45:241
1648537992320,2cyclictest2790750-21kworker/u8:1+events_unbound19:25:131
1648542992219,2cyclictest2790750-21kworker/u8:1+events_unbound19:10:113
1648540992218,3cyclictest3776209-21kworker/u8:4+flush-259:019:25:232
1648540992218,3cyclictest1876247-21kworker/u8:3+flush-259:023:25:202
1648537992219,2cyclictest1820615-21kworker/u8:1+events_unbound22:50:141
1648537992219,2cyclictest1783763-21kworker/u8:0+events_unbound21:40:221
1648537992219,2cyclictest1768764-21/usr/sbin/munin21:05:201
1648534992215,5cyclictest1726892-21latency_hist20:25:010
164853499220,20cyclictest1876036-21true23:15:000
1648542992118,3cyclictest1953760-21kworker/u8:0+events_unbound00:30:173
1648542992115,4cyclictest1943426-21latency_hist00:20:003
1648537992118,2cyclictest1773921-21kworker/u8:4+flush-259:021:30:191
1648537992116,4cyclictest1706515-21apt-get20:05:141
1648542992017,2cyclictest1695889-21kworker/u8:2+flush-259:020:00:243
1648542992016,3cyclictest1834152-21kworker/u8:2+events_unbound23:15:143
1648542992016,3cyclictest1811659-21kworker/u8:0+flush-259:023:00:213
1648540992016,3cyclictest1811659-21kworker/u8:0+events_unbound22:40:172
1648540992014,4cyclictest252-21systemd-journal19:57:112
1648537992017,2cyclictest2790750-21kworker/u8:1+events_unbound19:40:221
1648537992017,2cyclictest1773921-21kworker/u8:4+events_unbound21:10:121
1648537992017,2cyclictest1674782-21cut19:35:011
1648537992014,4cyclictest1811085-21latency_hist21:55:001
1648534992016,3cyclictest1922838-21kworker/u8:2+flush-259:000:10:160
1648534992016,2cyclictest1896930-21apt-get23:35:110
1648534992013,4cyclictest1948578-21latency_hist00:25:000
1648542991915,3cyclictest3776209-21kworker/u8:4+flush-259:020:45:163
1648540991916,2cyclictest1773921-21kworker/u8:4+flush-259:022:30:232
1648540991915,3cyclictest1876247-21kworker/u8:3+flush-259:023:15:152
1648537991916,2cyclictest1923504-21irqcore00:00:191
1648534991912,5cyclictest1767178-21ntpq21:00:200
1648542991815,2cyclictest1773921-21kworker/u8:4+events_unbound21:30:133
1648542991815,2cyclictest1716585-21kworker/u8:0+events_unbound20:25:273
1648542991814,3cyclictest2790750-21kworker/u8:1+flush-259:019:59:593
1648540991814,3cyclictest3776209-21kworker/u8:4+events_unbound19:15:142
1648540991814,3cyclictest2790750-21kworker/u8:1+events_unbound20:10:142
1648540991814,3cyclictest1783763-21kworker/u8:0+flush-259:021:35:002
1648540991814,3cyclictest1773921-21kworker/u8:4+flush-259:021:20:252
1648537991815,2cyclictest1816886-21kworker/u8:2+events_unbound22:13:171
1648537991813,2cyclictest1820615-21kworker/u8:1+events_unbound22:45:241
1648534991814,3cyclictest1876247-21kworker/u8:3+flush-259:023:29:590
1648542991714,2cyclictest1839618-21kworker/u8:3+flush-259:022:50:003
1648542991713,3cyclictest1891502-21kworker/u8:0+events_unbound00:00:133
1648542991713,3cyclictest1773921-21kworker/u8:4+events_unbound22:30:233
1648542991713,3cyclictest1610697-21kworker/u8:2+flush-259:019:20:053
1648542991710,5cyclictest1648454-21latency_hist19:10:013
1648540991714,2cyclictest1747514-21kworker/u8:1+events_unbound21:15:012
1648540991714,2cyclictest1663704-21kworker/u8:0+events_unbound19:20:222
1648540991713,3cyclictest2790750-21kworker/u8:1+flush-259:019:40:242
1648540991713,3cyclictest1891502-21kworker/u8:0+flush-259:000:04:062
1648540991713,3cyclictest1876247-21kworker/u8:3+flush-259:023:20:162
1648540991713,3cyclictest1834152-21kworker/u8:2+flush-259:022:54:592
1648540991711,4cyclictest1958909-21apt-get00:35:002
1648537991714,2cyclictest1901316-21kworker/u8:1+events_unbound23:45:161
1648537991714,2cyclictest1818764-21cron22:20:001
1648534991714,2cyclictest60350irq/132-enp0s29f1:rx-020:52:260
1648534991712,3cyclictest1758062-21apt-get20:55:110
1648542991613,2cyclictest1891502-21kworker/u8:0+flush-259:023:40:213
1648542991613,2cyclictest1876247-21kworker/u8:3+events_unbound23:27:163
1648542991613,2cyclictest1820615-21kworker/u8:1+events_unbound23:10:203
1648542991612,3cyclictest252-21systemd-journal20:15:003
1648542991612,3cyclictest1948847-21apt-get00:25:133
1648542991612,3cyclictest1891502-21kworker/u8:0+events_unbound23:30:233
1648542991612,3cyclictest1783763-21kworker/u8:0+flush-259:021:35:253
1648542991612,3cyclictest1747514-21kworker/u8:1+events_unbound21:11:173
1648542991611,4cyclictest2790750-21kworker/u8:1+events_unbound20:10:123
164854099164,7cyclictest1829041-21apt22:29:592
1648540991613,2cyclictest1917250-21apt-get23:55:012
1648540991613,2cyclictest1901316-21kworker/u8:1+flush-259:023:55:222
1648540991613,2cyclictest1732691-21irqcore20:30:172
1648540991612,3cyclictest1773921-21kworker/u8:4+flush-259:021:45:002
1648540991612,2cyclictest1873424-21sort23:10:172
1648537991613,2cyclictest1922838-21kworker/u8:2+events_unbound00:20:151
1648537991613,2cyclictest1773921-21kworker/u8:4+events_unbound21:47:301
164853499169,5cyclictest3776209-21kworker/u8:4+flush-259:019:55:000
1648534991614,1cyclictest60350irq/132-enp0s29f1:rx-023:30:240
1648534991613,2cyclictest60350irq/132-enp0s29f1:rx-000:15:210
1648534991612,2cyclictest0-21swapper/021:40:000
1648534991612,1cyclictest1906982-21turbostat23:44:590
1648534991611,4cyclictest1659518-21apt-get19:20:090
1648534991611,3cyclictest1922838-21kworker/u8:2+flush-259:000:05:530
1648542991512,2cyclictest1901316-21kworker/u8:1+flush-259:023:55:163
164854099154,5cyclictest1816838-21awk22:05:002
1648540991512,3cyclictest1811659-21kworker/u8:0+events_unbound23:05:202
1648540991512,2cyclictest1820615-21kworker/u8:1+flush-259:022:35:252
1648540991512,2cyclictest1748166-21irqcore20:45:172
1648540991511,3cyclictest1901316-21kworker/u8:1+flush-259:000:35:262
1648540991511,3cyclictest1648483-21kworker/u8:3+flush-259:019:35:162
1648537991511,3cyclictest1834152-21kworker/u8:2+events_unbound23:18:161
164853499159,4cyclictest1808906-21ntpq21:40:210
1648534991514,1cyclictest60350irq/132-enp0s29f1:rx-022:54:330
1648534991513,2cyclictest60350irq/132-enp0s29f1:rx-022:17:280
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-022:00:140
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:250
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-021:20:170
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-020:10:130
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-019:45:230
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-019:15:310
1648534991513,1cyclictest60350irq/132-enp0s29f1:rx-019:11:150
1648534991512,2cyclictest60350irq/132-enp0s29f1:rx-023:59:480
1648534991512,2cyclictest60350irq/132-enp0s29f1:rx-023:05:210
1648534991512,2cyclictest60350irq/132-enp0s29f1:rx-020:35:130
1648534991512,2cyclictest60350irq/132-enp0s29f1:rx-020:35:120
1648534991512,2cyclictest60350irq/132-enp0s29f1:rx-019:35:000
164854299149,2cyclictest1922838-21kworker/u8:2+events_unbound00:10:183
1648542991412,2cyclictest1901316-21kworker/u8:1+events_unbound00:35:243
1648542991411,2cyclictest3776209-21kworker/u8:4+flush-259:020:50:173
1648542991411,2cyclictest0-21swapper/321:05:003
1648540991411,2cyclictest1783763-21kworker/u8:0+flush-259:021:40:012
1648540991411,2cyclictest1747514-21kworker/u8:1+flush-259:020:50:102
1648540991411,2cyclictest1648483-21kworker/u8:3+events_unbound19:30:232
1648540991410,3cyclictest3776209-21kworker/u8:4+events_unbound20:55:162
1648537991411,2cyclictest1799534-21kworker/u8:2+events_unbound21:56:561
1648537991410,3cyclictest1891713-21kworker/u8:4+events_unbound23:31:351
1648534991413,1cyclictest60350irq/132-enp0s29f1:rx-022:11:450
1648534991412,2cyclictest60350irq/132-enp0s29f1:rx-019:59:370
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-022:40:170
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-021:55:370
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-021:51:350
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-020:45:110
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-020:40:550
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-020:40:540
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-020:06:110
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-019:44:010
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:260
1648534991412,1cyclictest60350irq/132-enp0s29f1:rx-000:25:160
1648534991411,2cyclictest60350irq/132-enp0s29f1:rx-022:30:140
1648534991411,2cyclictest60350irq/132-enp0s29f1:rx-022:20:240
1648534991411,1cyclictest1849738-21basename22:49:590
1648534991410,3cyclictest1768789-21apt-get21:05:110
164854299139,3cyclictest1834152-21kworker/u8:2+events_unbound23:06:173
164854299139,3cyclictest1674793-21gpgv19:35:013
1648542991311,2cyclictest1811659-21kworker/u8:0+events_unbound22:55:023
1648542991310,2cyclictest1928663-21sed00:05:183
1648542991310,2cyclictest1911665-21unixbench_singl23:45:243
1648542991310,2cyclictest1820615-21kworker/u8:1+flush-259:022:25:253
1648542991310,2cyclictest1811659-21kworker/u8:0+flush-259:022:52:353
1648542991310,2cyclictest1773921-21kworker/u8:4+events_unbound22:00:123
1648542991310,2cyclictest1773921-21kworker/u8:4+events_unbound21:15:103
1648542991310,2cyclictest0-21swapper/322:00:003
164854099139,3cyclictest1811659-21kworker/u8:0+flush-259:022:05:272
164854099138,4cyclictest1901316-21kworker/u8:1+flush-259:000:10:062
1648540991310,2cyclictest1891502-21kworker/u8:0+events_unbound23:40:242
1648540991310,2cyclictest1855186-21apt-get22:55:122
1648540991310,2cyclictest1769204-21tr21:05:162
1648540991310,2cyclictest1716585-21kworker/u8:0+flush-259:020:25:202
164853799139,3cyclictest1695889-21kworker/u8:2+events_unbound20:30:291
164853799135,1cyclictest191rcu_preempt20:15:181
1648537991311,1cyclictest1716585-21kworker/u8:0+events_unbound20:20:151
1648537991310,2cyclictest1695889-21kworker/u8:2+flush-259:020:02:501
164853499139,3cyclictest60350irq/132-enp0s29f1:rx-022:35:200
164853499139,2cyclictest1695889-21kworker/u8:2+flush-259:020:30:200
1648534991310,2cyclictest60350irq/132-enp0s29f1:rx-023:15:120
1648534991310,2cyclictest60350irq/132-enp0s29f1:rx-022:25:140
1648534991310,2cyclictest60350irq/132-enp0s29f1:rx-019:25:180
1648534991310,2cyclictest60350irq/132-enp0s29f1:rx-000:32:140
164854299129,2cyclictest1817712-21kworker/u8:1+events_unbound22:11:333
164854299129,2cyclictest1773921-21kworker/u8:4+events_unbound22:20:023
164854299128,3cyclictest1811659-21kworker/u8:0+events_unbound22:05:013
164854099129,2cyclictest1922838-21kworker/u8:2+events_unbound00:20:132
164854099129,2cyclictest1651339-21kthreadcore19:10:172
164854099128,3cyclictest3776209-21kworker/u8:4+flush-259:019:50:242
164854099128,2cyclictest0-21swapper/223:34:592
164853799129,2cyclictest1816886-21kworker/u8:2+flush-259:022:05:211
164853799121,1cyclictest1891466-21cron23:30:001
164853499129,2cyclictest60350irq/132-enp0s29f1:rx-021:15:260
1648534991211,1cyclictest60350irq/132-enp0s29f1:rx-021:49:150
1648534991210,1cyclictest60350irq/132-enp0s29f1:rx-021:25:140
1648534991210,1cyclictest60350irq/132-enp0s29f1:rx-021:10:130
1648534991210,1cyclictest60350irq/132-enp0s29f1:rx-020:00:230
164854299119,1cyclictest0-21swapper/319:50:133
164854099118,2cyclictest1773921-21kworker/u8:4+flush-259:022:00:002
164854099118,2cyclictest1762870-21kworker/u8:2+flush-259:021:00:242
164854299107,2cyclictest1695889-21kworker/u8:2+events_unbound20:30:153
164854099107,2cyclictest1928501-21fschecks_time00:05:162
16485429994,4cyclictest1773921-21kworker/u8:4+events_unbound21:54:043
16485409991,3cyclictest1810433-21apt21:50:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional