You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 16:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Aug 19, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
239252240166,24sleep30-21swapper/307:08:313
237262236163,24sleep20-21swapper/207:05:552
239382234166,22sleep10-21swapper/107:08:401
237642202164,26sleep00-21swapper/007:06:260
24130999693,2cyclictest0-21swapper/210:08:472
24130999692,3cyclictest0-21swapper/210:27:462
24130999692,3cyclictest0-21swapper/210:27:462
24130997067,2cyclictest0-21swapper/210:13:042
60222600,2sleep30-21swapper/312:35:023
24130995754,2cyclictest0-21swapper/210:58:212
278962520,4sleep230-21rcuc/208:30:112
285642510,2sleep30-21swapper/312:10:213
26392510,1sleep00-21swapper/007:30:210
2413799394,6cyclictest11932-40dnf09:05:243
2411799389,13cyclictest10649-21idleruntime-cro09:05:010
24137993712,22cyclictest1279-21irqstats09:55:203
24117993712,12cyclictest0-21swapper/011:15:010
326292360,5sleep22413099cyclictest08:40:142
2413799361,9cyclictest0-21swapper/310:23:023
24137993612,21cyclictest32593-21grep08:40:153
2413799361,11cyclictest6591-21sshd10:06:433
2413799353,28cyclictest14289-21grep10:25:153
2413799353,28cyclictest14289-21grep10:25:153
2411799357,13cyclictest29470-21munin-run12:15:010
2413799343,2cyclictest87350irq/126-enp1s0-09:20:203
2413799338,3cyclictest29420-21/usr/sbin/munin07:20:213
2413799332,6cyclictest30765-21ntp_states12:15:243
2413799331,28cyclictest10525-21sshd09:03:423
2413799331,21cyclictest8798-21kworker/u8:010:36:453
24137993312,18cyclictest6122-21/usr/sbin/munin12:35:143
24137993312,18cyclictest25405-21sort09:35:253
24137993311,19cyclictest827-21runrttasks12:30:003
24137993311,19cyclictest827-21runrttasks07:25:043
24137993311,19cyclictest5504-21/usr/sbin/munin11:20:243
2411799335,13cyclictest8107-21perf12:40:010
2411799331,25cyclictest16897-21kworker/0:209:28:020
2413799322,5cyclictest488-40dnf11:06:223
24137993213,16cyclictest22745-21unixbench_multi08:15:273
2413799321,22cyclictest21800-21memory11:55:203
2411799326,13cyclictest5621-21munin-run07:40:000
2411799324,13cyclictest1209-21munin-run07:30:010
2413799319,3cyclictest827-21runrttasks11:29:113
2413799319,3cyclictest22463-21/usr/sbin/munin10:45:193
2413799317,3cyclictest27223-21/usr/sbin/munin07:15:133
2413799313,6cyclictest24700-21smart_sda08:20:243
2413799312,8cyclictest0-21swapper/312:06:473
2413799311,7cyclictest17224-21crond09:20:003
2413799311,5cyclictest20862-21cat11:55:023
2413799311,4cyclictest27936-21ntp_states10:55:233
24137993111,17cyclictest773-21NetworkManager09:28:263
24137993110,3cyclictest26874-21gltestperf09:40:153
2413799310,9cyclictest0-21swapper/311:00:163
2413799308,5cyclictest909-21sssd_nss10:30:183
2413799308,3cyclictest827-21runrttasks08:12:313
2413799307,4cyclictest22478-21grep09:30:163
2413799305,3cyclictest87650irq/129-enp1s0-11:35:123
2413799304,4cyclictest909-21sssd_nss10:10:193
2413799304,3cyclictest87650irq/129-enp1s0-08:55:153
2413799302,4cyclictest26708-21ntp_states08:25:243
2413799302,3cyclictest14417-21cpuspeed_turbos11:40:133
2413799302,3cyclictest14417-21cpuspeed_turbos11:40:133
2413799302,10cyclictest3016-21df10:00:143
24137993011,16cyclictest9354-21ls10:15:153
2413799298,4cyclictest2131-21/usr/sbin/munin08:45:133
2413799298,2cyclictest25196-21cat12:05:023
2413799297,4cyclictest5135-21latency08:50:183
2413799296,3cyclictest32546-21gltestperf12:20:163
2413799293,3cyclictest25887-21awk10:50:223
2413799291,14cyclictest0-21swapper/310:42:333
2413799291,10cyclictest0-21swapper/307:58:443
2413799291,10cyclictest0-21swapper/307:58:443
24137992910,2cyclictest1313-21expr11:10:123
2413799289,3cyclictest11416-21processes07:50:233
2413799288,3cyclictest28531-21idleruntime08:30:173
2413799288,3cyclictest1434-21acpi07:30:123
2413799287,3cyclictest31813-21grep09:50:203
2413799282,4cyclictest3527-21cat11:15:133
2413799281,13cyclictest0-21swapper/309:11:423
2413799281,13cyclictest0-21swapper/309:11:423
24137992811,14cyclictest6145-21expr07:40:133
24137992810,4cyclictest0-21swapper/309:46:023
2412499281,10cyclictest28786-21df09:45:151
2411799285,11cyclictest17243-21awk09:20:020
2411799281,13cyclictest12226-21idleruntime-cro07:55:000
2411799281,11cyclictest4591-21kworker/0:310:05:220
2411799281,10cyclictest9581-21kworker/0:211:38:310
48942270,4sleep02411799cyclictest08:50:170
2413799277,2cyclictest8041-21/usr/sbin/munin07:45:153
2413799276,3cyclictest15453-21kernelversion08:00:193
2413799272,5cyclictest31094-21ntp_states08:35:223
2413799272,3cyclictest87350irq/126-enp1s0-08:05:243
2413799272,3cyclictest3650-21/usr/sbin/munin07:35:183
2413799271,7cyclictest0-21swapper/311:45:573
2412499275,6cyclictest0-21swapper/111:51:271
2411799272,10cyclictest8835-21kworker/0:010:14:100
2411799272,10cyclictest26563-21kworker/0:310:58:030
2411799272,10cyclictest22762-21kworker/0:212:03:220
2411799271,9cyclictest24271-21kworker/0:210:51:430
2411799271,13cyclictest18817-21munin-run08:10:020
2413799265,3cyclictest25302-21missed_timers07:10:193
2413799261,12cyclictest0-21swapper/311:30:473
2411799262,9cyclictest5111-21kworker/0:111:19:430
2411799262,12cyclictest0-21swapper/012:35:010
2411799262,11cyclictest11078-21kworker/0:110:18:060
2411799262,10cyclictest3401-21kworker/0:012:26:550
2411799261,9cyclictest7335-21kworker/0:011:27:110
2411799261,17cyclictest15974-21kworker/0:111:45:540
2411799261,10cyclictest16897-21kworker/0:209:39:340
2413099251,5cyclictest0-21swapper/211:22:132
2412499251,15cyclictest366-21/usr/sbin/munin12:20:191
2411799251,9cyclictest15525-21kworker/0:011:04:470
2411799251,12cyclictest27610-21munin-run08:30:020
2411799251,10cyclictest440-21kworker/0:211:06:550
2413099249,4cyclictest0-21swapper/209:35:132
2413099241,7cyclictest24237-21awk12:00:252
2412499247,9cyclictest15642-21latency_hist10:30:011
2412499247,9cyclictest15642-21latency_hist10:30:011
2412499241,17cyclictest0-21swapper/111:42:101
2412499241,17cyclictest0-21swapper/111:42:101
2411799249,2cyclictest12123-21kworker/0:207:55:310
2411799249,2cyclictest12123-21kworker/0:207:55:310
2411799247,3cyclictest11078-21kworker/0:110:27:190
2411799247,3cyclictest11078-21kworker/0:110:27:180
2411799245,3cyclictest27566-21kworker/0:108:45:230
2411799242,9cyclictest15525-21kworker/0:010:37:270
2411799242,9cyclictest11078-21kworker/0:110:33:430
2411799242,8cyclictest6533-21kworker/0:107:18:470
2411799242,11cyclictest15525-21kworker/0:010:41:270
2411799241,8cyclictest24271-21kworker/0:210:48:470
2411799241,10cyclictest9581-21kworker/0:211:32:070
2411799240,10cyclictest5111-21kworker/0:111:22:390
2412499236,4cyclictest0-21swapper/109:36:121
2412499231,9cyclictest0-21swapper/110:20:391
2412499231,17cyclictest0-21swapper/110:53:051
2412499231,16cyclictest0-21swapper/112:10:231
2412499231,16cyclictest0-21swapper/111:10:211
2412499231,14cyclictest20332-21hddtemp_smartct09:25:181
2412499231,11cyclictest19516-21cat09:25:021
2411799239,3cyclictest1156-21kworker/0:312:21:230
2411799235,5cyclictest2265-21kworker/0:210:02:300
2411799232,9cyclictest20484-21kworker/0:011:54:180
2411799232,9cyclictest16897-21kworker/0:209:34:020
2411799232,10cyclictest24996-21kworker/0:012:08:380
2411799231,9cyclictest11078-21kworker/0:110:21:300
2411799231,8cyclictest8367-21kworker/0:309:05:590
2411799231,10cyclictest23773-21kworker/0:109:40:420
24117992310,3cyclictest20833-21kworker/0:008:17:430
2413099221,3cyclictest0-21swapper/209:08:462
2412499228,4cyclictest0-21swapper/110:13:091
2412499228,12cyclictest438-21kworker/u8:012:01:351
2412499225,3cyclictest0-21swapper/111:47:111
2412499224,12cyclictest87550irq/128-enp1s0-09:40:211
24124992216,3cyclictest16174-21memory09:15:201
24124992212,2cyclictest808-21snmpd07:17:031
2412499221,16cyclictest0-21swapper/112:33:121
2412499221,16cyclictest0-21swapper/111:04:201
2412499221,15cyclictest0-21swapper/110:30:351
2412499221,15cyclictest0-21swapper/110:18:591
2412499221,13cyclictest12913-21irqstats11:35:201
2412499221,11cyclictest0-21swapper/111:56:581
2411799225,2cyclictest15974-21kworker/0:111:56:460
2411799221,9cyclictest31902-21kworker/0:008:39:510
2411799221,9cyclictest15974-21kworker/0:111:44:020
2411799221,9cyclictest15974-21kworker/0:111:44:020
24130992114,4cyclictest28094-21grep12:10:162
2413099211,13cyclictest0-21swapper/211:55:162
2412499218,3cyclictest23823-21munin-run09:35:021
2412499211,3cyclictest0-21swapper/111:19:021
2412499211,3cyclictest0-21swapper/110:48:281
2412499211,18cyclictest0-21swapper/108:30:171
2412499211,15cyclictest0-21swapper/107:21:031
2412499211,14cyclictest0-21swapper/107:53:421
2412499211,14cyclictest0-21swapper/107:26:421
2412499211,11cyclictest0-21swapper/111:22:221
2412499211,11cyclictest0-21swapper/110:07:051
2412499211,10cyclictest0-21swapper/111:28:291
2412499210,16cyclictest0-21swapper/107:35:491
2412499210,15cyclictest0-21swapper/110:35:351
2411799219,2cyclictest15974-21kworker/0:112:16:060
2411799216,8cyclictest5277-21kworker/0:307:41:180
2411799211,9cyclictest3134-21kworker/0:007:48:140
2411799211,7cyclictest0-21swapper/009:24:120
2411799211,6cyclictest0-21swapper/009:57:250
24117992110,2cyclictest18576-21kworker/0:208:14:270
2412499209,3cyclictest7561-21awk08:55:241
2412499209,3cyclictest3492-21irqstats10:00:181
2412499203,9cyclictest0-21swapper/109:54:491
24124992016,3cyclictest0-21swapper/108:12:151
2412499201,5cyclictest0-21swapper/109:59:201
24124992013,2cyclictest0-21swapper/107:43:321
24124992012,3cyclictest0-21swapper/107:55:391
24124992012,3cyclictest0-21swapper/107:55:391
2412499201,18cyclictest0-21swapper/108:39:101
2412499201,17cyclictest0-21swapper/112:17:411
2412499201,17cyclictest0-21swapper/111:05:191
2412499201,14cyclictest0-21swapper/112:07:081
2412499201,14cyclictest0-21swapper/111:34:471
2412499201,14cyclictest0-21swapper/107:47:441
2412499201,12cyclictest0-21swapper/112:35:201
2412499201,11cyclictest9727-21ntp_states09:00:221
2412499201,11cyclictest1630-21ls07:30:111
2412499201,10cyclictest25427-21awk08:25:011
2411799207,8cyclictest29672-21kworker/0:308:34:110
2411799203,2cyclictest4611-21kworker/0:008:55:230
2411799202,10cyclictest27932-21kworker/0:009:45:060
130532201,2sleep10-21swapper/109:10:051
130532201,2sleep10-21swapper/109:10:051
2413099194,3cyclictest87350irq/126-enp1s0-12:35:212
2413099193,4cyclictest264172cat08:25:172
2413099193,4cyclictest0-21swapper/210:45:542
2413099192,4cyclictest0-21swapper/209:15:012
2413099192,4cyclictest0-21swapper/209:15:012
2413099191,8cyclictest12618-21cstates07:55:122
2413099191,8cyclictest12618-21cstates07:55:122
2413099191,6cyclictest813-21cat12:20:252
2413099191,6cyclictest16175-21memory09:15:192
2413099191,5cyclictest28999-21sshd07:17:432
2413099191,5cyclictest17039-21tune2fs11:45:142
24130991913,3cyclictest8350-21cut11:25:162
2413099191,16cyclictest0-21swapper/210:22:002
2413099191,15cyclictest4242-21grep10:00:272
2412499198,2cyclictest27735-21cat08:30:011
2412499196,3cyclictest25322-21head07:10:201
24124991916,2cyclictest0-21swapper/108:16:331
24124991915,3cyclictest0-21swapper/108:02:561
2412499191,17cyclictest0-21swapper/108:50:511
2412499191,17cyclictest0-21swapper/108:45:221
2412499191,16cyclictest0-21swapper/109:05:411
2412499191,14cyclictest0-21swapper/112:27:401
2412499191,12cyclictest25864-21kworker/u8:110:56:051
2412499190,14cyclictest0-21swapper/110:43:241
2411799195,8cyclictest6533-21kworker/0:107:11:430
2411799194,2cyclictest23046-21kworker/0:108:20:110
2411799191,7cyclictest0-21swapper/009:52:070
2411799191,10cyclictest32226-21/usr/sbin/munin08:40:230
2413099187,3cyclictest87350irq/126-enp1s0-11:35:282
2413099185,4cyclictest87650irq/129-enp1s0-10:40:112
2413099182,3cyclictest0-21swapper/211:13:272
2413099181,7cyclictest6459-21/usr/sbin/munin08:55:162
2413099181,7cyclictest1-21systemd08:00:202
24130991813,3cyclictest23215-21awk08:20:012
24130991812,3cyclictest1876-21cpuspeed_turbos12:25:122
24130991812,3cyclictest10015-21tr11:30:112
2413099181,10cyclictest0-21swapper/210:31:532
24124991815,2cyclictest0-21swapper/108:41:221
2412499181,3cyclictest0-21swapper/108:06:301
2411799181,9cyclictest14564-21tail09:10:270
2411799181,9cyclictest14564-21tail09:10:270
2411799181,10cyclictest3134-21kworker/0:008:01:430
2413099175,3cyclictest20182-21pmu-power08:10:212
2413099172,7cyclictest4260-21/usr/sbin/munin08:50:202
2413099172,6cyclictest909-21sssd_nss12:30:192
2413099172,14cyclictest0-21swapper/209:00:032
2413099171,7cyclictest29169-21cat09:45:192
2413099171,7cyclictest0-21swapper/208:05:202
2413099171,6cyclictest808-21snmpd08:37:182
2413099171,6cyclictest6995-21grep07:40:232
2413099171,6cyclictest26339-21wc09:40:122
2413099171,6cyclictest19061-21grep10:35:192
2413099171,6cyclictest0-21swapper/212:08:072
2413099171,6cyclictest0-21swapper/209:25:222
2413099171,5cyclictest10484-21processes10:15:252
2413099171,5cyclictest0-21swapper/211:50:132
2413099171,4cyclictest1067-21grep09:55:182
24130991711,4cyclictest30043-21/usr/sbin/munin12:15:162
24130991711,3cyclictest31272-21/usr/sbin/munin11:05:162
24130991711,3cyclictest21848-21/usr/sbin/munin09:30:172
24130991711,3cyclictest17457-21/usr/sbin/munin09:20:262
24130991711,3cyclictest15011-21cat11:40:192
24130991711,3cyclictest15011-21cat11:40:192
2413099169,4cyclictest3311-21/usr/sbin/munin11:15:242
2413099165,3cyclictest87350irq/126-enp1s0-07:50:132
2413099164,3cyclictest87350irq/126-enp1s0-07:45:212
2413099161,6cyclictest31632-21strings09:50:182
2413099161,5cyclictest25004-21hddtemp_smartct07:10:152
2413099161,5cyclictest24792-21cpuspeed_turbos10:50:112
2413099161,3cyclictest0-21swapper/211:03:072
24130991612,3cyclictest0-21swapper/208:22:202
24130991612,3cyclictest0-21swapper/207:26:092
24130991611,3cyclictest3876-21cat07:35:142
24130991610,3cyclictest2344-21expr08:45:132
24117991611,3cyclictest29420-21/usr/sbin/munin07:20:250
2413099159,3cyclictest1571-21grep07:30:122
2413099151,5cyclictest12791-21kworker/u8:107:20:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional