You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 16:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Mar 24, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4170084995047,3cyclictest235147-21kworker/u8:3+events_unbound12:30:150
4170084994744,2cyclictest95355-21kworker/u8:3+events_unbound10:00:140
4170092994641,3cyclictest85318-21/usr/sbin/munin08:55:152
4170084994238,3cyclictest75-21kswapd012:25:020
4170084994038,2cyclictest1146027-21kworker/u8:2+events_unbound07:20:260
4170084994037,2cyclictest4185474-21kworker/u8:0+events_unbound07:45:160
4170088993935,3cyclictest100495-21kworker/u8:4+flush-259:011:25:181
4170084993936,2cyclictest1146027-21kworker/u8:2+events_unbound07:55:140
4170084993835,2cyclictest48338-21kworker/u8:1+events_unbound08:20:240
4170084993834,3cyclictest100495-21kworker/u8:4+events_unbound11:25:200
4170084993834,2cyclictest245251-21kworker/u8:1+events_unbound11:35:250
4170092993735,1cyclictest250279-21fwupd11:34:262
4170092993735,1cyclictest250279-21fwupd11:34:252
4170084993734,2cyclictest69195-21kworker/u8:1+events_unbound08:45:260
4170084993633,2cyclictest4185474-21kworker/u8:0+events_unbound07:40:230
4170088993529,4cyclictest100476-21latency_hist09:10:011
4170084993531,2cyclictest100495-21kworker/u8:4+events_unbound10:45:250
4170088993430,3cyclictest147212-21apt-get09:55:121
4170084993431,2cyclictest79519-21kworker/u8:2+events_unbound09:55:190
4170084993330,2cyclictest100495-21kworker/u8:4+events_unbound10:10:220
4170084993329,3cyclictest183623-21kworker/u8:2+events_unbound10:35:230
4170094993229,2cyclictest1146027-21kworker/u8:2+events_unbound07:43:273
4170084993229,2cyclictest79519-21kworker/u8:2+events_unbound09:00:150
4170084993229,2cyclictest4185474-21kworker/u8:0+events_unbound08:50:220
4170084993229,2cyclictest286499-21kworker/u8:0+events_unbound12:10:240
4170094993128,2cyclictest75-21kswapd012:25:033
4170092993125,4cyclictest95755-21diskstats09:05:142
4170084993126,3cyclictest302011-21apt-get12:25:010
4170092993026,3cyclictest100495-21kworker/u8:4+events_unbound10:50:192
417008899306,17cyclictest259836-21ntpq11:40:191
4170084993027,3cyclictest79519-21kworker/u8:2+events_unbound09:25:270
4170084993027,2cyclictest1146027-21kworker/u8:2+events_unbound07:50:250
4170094992924,3cyclictest17831-21cat07:50:013
4170084992926,2cyclictest100495-21kworker/u8:4+events_unbound10:30:220
4170084992925,3cyclictest100495-21kworker/u8:4+flush-259:011:10:280
4170084992925,2cyclictest224571-21kworker/u8:1+events_unbound11:19:590
4170094992826,1cyclictest235147-21kworker/u8:3+events_unbound12:20:143
4170094992825,2cyclictest100495-21kworker/u8:4+events_unbound10:49:533
4170094992822,4cyclictest266240-21diskstats11:50:133
4170094992821,5cyclictest161505-21ntpq10:05:213
4170092992824,3cyclictest255531-21kworker/u8:0+events_unbound11:50:122
4170092992824,3cyclictest183623-21kworker/u8:2+events_unbound10:40:162
4170084992827,1cyclictest1146027-21kworker/u8:2+events_unbound08:10:280
4170084992826,2cyclictest173348-21kworker/u8:0+events_unbound10:20:240
4170084992825,2cyclictest100495-21kworker/u8:4+events_unbound10:05:200
4170094992724,2cyclictest4175189-21kworker/u8:3+events_unbound07:32:263
4170092992724,2cyclictest1146027-21kworker/u8:2+flush-259:008:15:152
4170092992723,3cyclictest69195-21kworker/u8:1+flush-259:009:20:002
4170092992723,3cyclictest48338-21kworker/u8:1+events_unbound08:30:132
4170092992722,3cyclictest1146027-21kworker/u8:2+events_unbound08:25:232
4170084992724,2cyclictest79519-21kworker/u8:2+events_unbound08:55:210
4170084992724,2cyclictest235147-21kworker/u8:3+events_unbound11:40:120
4170084992723,3cyclictest79519-21kworker/u8:2+events_unbound09:30:270
4170084992723,3cyclictest79519-21kworker/u8:2+events_unbound09:05:260
4170084992723,3cyclictest291951-21apt-get12:15:110
4170094992623,2cyclictest1146027-21kworker/u8:2+events_unbound07:57:253
4170094992620,5cyclictest218674-21ntpq11:00:223
4170092992623,2cyclictest100495-21kworker/u8:4+events_unbound11:10:172
4170092992622,3cyclictest1146027-21kworker/u8:2+events_unbound07:30:192
4170092992620,4cyclictest48860-21apt-get08:20:122
4170084992623,2cyclictest95355-21kworker/u8:3+events_unbound09:15:260
4170084992623,2cyclictest255531-21kworker/u8:0+events_unbound11:55:000
4170092992523,1cyclictest4175189-21kworker/u8:3+events_unbound07:15:182
4170092992519,4cyclictest275251-21ntpq11:55:192
4170084992522,2cyclictest313233-21kworker/u8:2+events_unbound12:35:260
4170084992522,2cyclictest183623-21kworker/u8:2+events_unbound10:55:200
4170084992521,2cyclictest48338-21kworker/u8:1+events_unbound08:25:210
4170092992420,3cyclictest235147-21kworker/u8:3+flush-259:011:40:112
4170092992420,3cyclictest235147-21kworker/u8:3+events_unbound12:30:132
4170092992417,5cyclictest171759-21ntpq10:15:202
4170084992421,2cyclictest79519-21kworker/u8:2+events_unbound09:35:280
4170084992421,2cyclictest245251-21kworker/u8:1+events_unbound12:05:150
4170094992321,2cyclictest183623-21kworker/u8:2+events_unbound10:34:463
4170092992320,2cyclictest95355-21kworker/u8:3+events_unbound09:45:202
4170092992320,2cyclictest235147-21kworker/u8:3+events_unbound12:25:032
4170092992320,2cyclictest235147-21kworker/u8:3+events_unbound12:15:152
4170092992320,2cyclictest100495-21kworker/u8:4+flush-259:011:25:152
4170092992319,3cyclictest1146027-21kworker/u8:2+flush-259:007:50:162
4170084992321,2cyclictest4185474-21kworker/u8:0+events_unbound08:34:240
4170084992321,2cyclictest235147-21kworker/u8:3+events_unbound11:55:230
4170084992321,2cyclictest1146027-21kworker/u8:2+events_unbound08:05:270
4170084992320,2cyclictest4185474-21kworker/u8:0+events_unbound07:38:250
4170084992319,3cyclictest183623-21kworker/u8:2+events_unbound11:09:590
4170084992319,3cyclictest100495-21kworker/u8:4+flush-259:009:54:410
4170092992219,2cyclictest79519-21kworker/u8:2+flush-259:009:35:252
4170092992218,3cyclictest4185474-21kworker/u8:0+events_unbound08:50:242
4170092992218,3cyclictest245251-21kworker/u8:1+events_unbound12:05:232
4170092992218,3cyclictest235147-21kworker/u8:3+flush-259:011:20:182
4170092992218,3cyclictest1146027-21kworker/u8:2+flush-259:007:20:242
4170092992218,3cyclictest100495-21kworker/u8:4+events_unbound11:05:182
4170092992217,4cyclictest252-21systemd-journal12:03:252
4170084992219,2cyclictest1146027-21kworker/u8:2+events_unbound07:30:140
4170084992219,2cyclictest100495-21kworker/u8:4+events_unbound10:54:260
4170084992217,2cyclictest4185474-21kworker/u8:0+events_unbound08:00:270
4170094992118,2cyclictest255531-21kworker/u8:0+events_unbound11:44:553
4170094992117,3cyclictest79760-21apt-get08:50:123
4170094992116,4cyclictest152087-21latency_hist10:00:013
417008899211,9cyclictest317516-21ntpq12:35:211
4170088992117,3cyclictest235147-21kworker/u8:3+flush-259:011:55:161
4170088992115,4cyclictest111087-21apt-get09:20:121
417008899210,12cyclictest58630-21taskset08:27:181
417008899210,11cyclictest69088-21seq08:39:191
4170084992117,2cyclictest507-21in:imuxsock08:20:000
4170084992114,5cyclictest172686-21latency_hist10:20:010
4170094992018,2cyclictest286499-21kworker/u8:0+events_unbound12:31:263
4170094992017,2cyclictest235147-21kworker/u8:3+events_unbound11:26:093
4170092992017,2cyclictest95355-21kworker/u8:3+flush-259:010:20:172
4170092992016,3cyclictest4185474-21kworker/u8:0+flush-259:007:40:202
4170092992016,3cyclictest4185474-21kworker/u8:0+events_unbound08:00:252
4170092992016,3cyclictest100495-21kworker/u8:4+events_unbound10:30:262
4170092992015,4cyclictest142189-21diskstats09:50:132
4170088992015,3cyclictest252-21systemd-journal08:30:251
4170084992017,2cyclictest235147-21kworker/u8:3+events_unbound11:33:270
4170084992017,2cyclictest235147-21kworker/u8:3+events_unbound11:33:260
4170084992017,2cyclictest131489-21kworker/u8:0+events_unbound09:40:270
4170084992016,3cyclictest178700-21diskstats10:25:130
417009499193,3cyclictest50-21ksoftirqd/312:00:003
4170094991917,2cyclictest100495-21kworker/u8:4+events_unbound09:24:393
417009299193,9cyclictest116043-21idleruntime-cro09:25:002
4170092991915,3cyclictest79519-21kworker/u8:2+events_unbound09:10:202
4170092991915,3cyclictest100495-21kworker/u8:4+flush-259:010:10:262
417008899199,8cyclictest252-21systemd-journal12:05:021
417008899191,9cyclictest292932-21irqstats12:15:171
417008899191,8cyclictest183405-21munin-run10:30:001
4170088991915,3cyclictest131489-21kworker/u8:0+flush-259:009:40:171
4170088991915,2cyclictest17757-21latency_hist07:50:011
4170088991914,4cyclictest4175387-21/usr/sbin/munin07:15:201
4170088991913,5cyclictest0-21swapper/112:30:011
417008899191,17cyclictest240088-21latency_hist11:25:011
417008899191,11cyclictest198620-21taskset10:42:061
417008899190,9cyclictest209041-21runrttasks10:53:391
417008899190,17cyclictest193734-21latency_hist10:40:001
417008899190,11cyclictest90113-21seq08:59:021
4170084991917,2cyclictest4185474-21kworker/u8:0+events_unbound08:40:260
4170084991917,2cyclictest100495-21kworker/u8:4+events_unbound11:24:560
417009499186,6cyclictest50-21ksoftirqd/308:25:003
4170094991816,2cyclictest79519-21kworker/u8:2+events_unbound10:14:283
4170094991815,2cyclictest100495-21kworker/u8:4+flush-259:010:59:563
4170094991813,3cyclictest4190641-21latency_hist07:30:013
4170092991816,1cyclictest95355-21kworker/u8:3+events_unbound09:55:212
4170088991815,1cyclictest116703-21irqcore09:25:171
417008899180,9cyclictest281229-21seq12:03:421
417008899180,8cyclictest224447-21chrt11:07:581
417008899180,11cyclictest156892-21seq10:01:091
417008899180,10cyclictest188438-21taskset10:33:341
417008899180,10cyclictest177973-21taskset10:21:281
4170084991816,2cyclictest1146027-21kworker/u8:2+events_unbound07:10:240
4170084991815,2cyclictest276400-21kworker/u8:4+events_unbound12:00:190
4170084991815,2cyclictest198900-21kworker/u8:1+events_unbound11:00:160
417009499178,1cyclictest0-21swapper/308:10:183
417009499173,8cyclictest178874-21cat10:25:153
4170094991716,1cyclictest48338-21kworker/u8:1+events_unbound08:26:013
4170094991715,2cyclictest95355-21kworker/u8:3+events_unbound09:39:223
4170094991715,2cyclictest100495-21kworker/u8:4+events_unbound10:42:583
4170094991714,2cyclictest0-21swapper/309:55:003
4170092991715,1cyclictest224571-21kworker/u8:1+events_unbound11:18:252
4170092991714,2cyclictest286499-21kworker/u8:0+events_unbound12:10:222
4170092991714,2cyclictest245251-21kworker/u8:1+events_unbound11:35:232
4170092991713,3cyclictest4185474-21kworker/u8:0+flush-259:007:45:142
4170092991713,3cyclictest4185474-21kworker/u8:0+events_unbound07:56:252
4170092991713,3cyclictest4175189-21kworker/u8:3+flush-259:007:25:162
4170092991713,3cyclictest313233-21kworker/u8:2+events_unbound12:35:232
417008899173,12cyclictest252-21systemd-journal07:29:591
4170088991713,3cyclictest297108-21apt-get12:20:121
4170088991713,3cyclictest235147-21kworker/u8:3+flush-259:011:35:181
417008899170,9cyclictest4185120-21chrt07:21:111
417008899170,8cyclictest234730-21taskset11:18:011
417008899170,1cyclictest0-21swapper/110:05:211
417008899170,10cyclictest74240-21taskset08:44:111
4170084991714,2cyclictest95355-21kworker/u8:3+events_unbound09:13:250
4170084991714,2cyclictest1146027-21kworker/u8:2+events_unbound07:17:080
4170094991615,1cyclictest235147-21kworker/u8:3+events_unbound12:18:283
4170094991614,2cyclictest69195-21kworker/u8:1+events_unbound09:06:133
4170094991614,1cyclictest278750-21tr12:00:183
417009499160,4cyclictest317609-21ntpq12:35:213
4170092991613,2cyclictest198900-21kworker/u8:1+flush-259:011:00:192
4170092991612,3cyclictest4185474-21kworker/u8:0+events_unbound08:39:272
4170092991611,4cyclictest1146027-21kworker/u8:2+events_unbound08:05:152
417008899161,8cyclictest39561-21irqstats08:10:181
4170088991613,2cyclictest503-21tpm2-abrmd08:55:001
4170088991612,2cyclictest503-21tpm2-abrmd12:35:001
4170088991611,3cyclictest4165109-21kworker/u8:1+flush-259:007:10:161
417008899161,13cyclictest33866-21cut08:05:161
417008899161,13cyclictest252-21systemd-journal12:15:011
417008899161,13cyclictest252-21systemd-journal08:00:011
417008899161,13cyclictest229795-21latency_hist11:14:591
417008899161,13cyclictest125919-21grep09:30:241
417008899160,14cyclictest172680-21idleruntime-cro10:20:001
417008899160,13cyclictest28405-21grep08:00:131
417008899160,12cyclictest105925-21apt-get09:15:131
417008499168,1cyclictest0-21swapper/009:45:170
4170084991613,2cyclictest4185474-21kworker/u8:0+events_unbound08:38:270
4170084991613,2cyclictest235147-21kworker/u8:3+events_unbound11:47:260
4170084991612,3cyclictest100495-21kworker/u8:4+flush-259:009:20:350
417009499156,2cyclictest0-21swapper/309:45:203
417009499152,8cyclictest50-21ksoftirqd/312:10:133
417009499152,4cyclictest4180572-21awk07:20:113
4170094991513,2cyclictest79519-21kworker/u8:2+events_unbound09:28:043
4170094991513,2cyclictest4185474-21kworker/u8:0+events_unbound08:34:073
4170094991513,2cyclictest4175189-21kworker/u8:3+events_unbound07:17:313
4170094991513,2cyclictest235147-21kworker/u8:3+events_unbound11:31:213
4170094991513,2cyclictest235147-21kworker/u8:3+events_unbound11:31:213
4170094991513,1cyclictest0-21swapper/310:35:253
4170094991513,1cyclictest0-21swapper/308:40:003
4170094991512,2cyclictest79519-21kworker/u8:2+events_unbound09:11:333
4170094991512,2cyclictest72624-21kthreadcore08:40:183
4170094991512,1cyclictest191rcu_preempt10:03:463
4170094991510,1cyclictest191rcu_preempt09:15:433
417009499150,8cyclictest240254-21latency_hist11:25:003
4170092991512,2cyclictest4185474-21kworker/u8:0+events_unbound08:49:282
4170092991511,3cyclictest235147-21kworker/u8:3+events_unbound12:20:182
4170092991511,3cyclictest100495-21kworker/u8:4+flush-259:010:45:202
417008899152,11cyclictest91143-21sed09:00:191
417008899151,12cyclictest7675-21ps07:40:161
417008899151,12cyclictest75017-21sed08:45:161
417008899151,12cyclictest49003-21cpuspeed_turbos08:20:131
417008899151,12cyclictest270135-21ntpq11:50:191
417008899151,12cyclictest209565-21grep10:55:141
417008899151,12cyclictest137297-21/usr/sbin/munin09:45:161
417008899151,11cyclictest43721-21apt-get08:15:151
417008899151,11cyclictest261044-21grep11:45:121
417008899151,11cyclictest126789-21fschecks_count09:35:141
417008899150,13cyclictest249370-21needreboot11:30:201
417008899150,13cyclictest249370-21needreboot11:30:191
417008899150,13cyclictest2255-21df07:35:121
417008899150,12cyclictest4190996-21cut07:30:131
417008899150,12cyclictest214712-21cpuspeed_turbos11:00:131
417008899150,11cyclictest23008-21sed07:55:011
4170084991511,3cyclictest4175189-21kworker/u8:3+events_unbound07:25:080
417009499146,1cyclictest0-21swapper/312:05:173
417009499144,1cyclictest191rcu_preempt09:40:233
417009499143,3cyclictest50-21ksoftirqd/311:15:173
417009499143,3cyclictest50-21ksoftirqd/310:25:003
417009499143,3cyclictest28673-21awk08:00:183
417009499142,4cyclictest50-21ksoftirqd/311:05:123
417009499142,3cyclictest50-21ksoftirqd/310:15:193
417009499142,2cyclictest0-21swapper/309:30:213
4170094991413,1cyclictest69195-21kworker/u8:1+events_unbound08:45:243
4170094991412,2cyclictest235147-21kworker/u8:3+events_unbound11:36:203
4170094991412,2cyclictest183623-21kworker/u8:2+events_unbound10:51:103
4170094991412,2cyclictest1146027-21kworker/u8:2+events_unbound08:08:333
4170094991411,2cyclictest79519-21kworker/u8:2+events_unbound09:01:123
4170094991411,2cyclictest79519-21kworker/u8:2+events_unbound08:57:543
4170094991411,2cyclictest0-21swapper/307:45:133
417009499140,4cyclictest225894-21/usr/sbin/munin11:10:193
417009299149,4cyclictest252-21systemd-journal08:45:002
4170092991411,2cyclictest0-21swapper/208:15:002
4170092991411,1cyclictest0-21swapper/209:30:002
4170092991410,3cyclictest183623-21kworker/u8:2+events_unbound10:35:242
4170092991410,3cyclictest173348-21kworker/u8:0+flush-259:010:25:182
417008899142,9cyclictest146089-21ntpq09:50:211
417008899141,11cyclictest199560-21ps10:45:181
417008899141,11cyclictest166658-21ntpq10:10:211
417008899141,11cyclictest100992-21fschecks_time09:10:161
417009499132,3cyclictest50-21ksoftirqd/311:45:173
417009499131,3cyclictest50-21ksoftirqd/308:15:193
4170094991311,2cyclictest1146027-21kworker/u8:2+events_unbound07:37:063
4170094991311,1cyclictest491ktimers/307:10:203
417009299138,4cyclictest235147-21kworker/u8:3+events_unbound11:50:012
4170092991311,1cyclictest4165109-21kworker/u8:1+events_unbound07:10:152
4170092991310,2cyclictest95355-21kworker/u8:3+flush-259:009:30:242
4170092991310,2cyclictest1146027-21kworker/u8:2+events_unbound07:35:212
417009299118,2cyclictest152762-21cut10:00:152
417009299117,3cyclictest79519-21kworker/u8:2+flush-259:009:01:362
417009299117,3cyclictest100495-21kworker/u8:4+events_unbound10:07:252
417009299108,1cyclictest0-21swapper/209:40:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional