You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 10:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Fri Mar 27, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
60202221191,20sleep20-21swapper/219:09:392
40932203165,25sleep10-21swapper/119:05:111
59382202164,25sleep30-21swapper/319:08:353
40922202165,25sleep00-21swapper/019:05:100
2582921300,3sleep2615099cyclictest19:50:232
6154998986,2cyclictest0-21swapper/323:27:493
6154998885,2cyclictest0-21swapper/320:04:463
6154998077,2cyclictest0-21swapper/322:48:013
6154996562,2cyclictest0-21swapper/323:10:453
122962610,1sleep20-21swapper/200:35:242
6154995451,2cyclictest0-21swapper/319:13:323
6154995350,2cyclictest0-21swapper/300:25:393
260362480,1sleep00-21swapper/019:50:250
6154994136,2cyclictest19669-21ntp_kernel_pll_20:50:193
614399351,13cyclictest27701-40dnf23:59:501
6154993432,1cyclictest0-21swapper/320:22:033
326062340,2sleep10-21swapper/122:55:191
614399333,2cyclictest86550irq/126-enp1s0-21:35:581
614399331,8cyclictest0-21swapper/119:24:191
614399330,11cyclictest0-21swapper/100:33:151
614399321,11cyclictest0-21swapper/123:53:311
614399320,20cyclictest0-21swapper/119:12:191
615499316,19cyclictest26622-21kworker/3:123:46:553
614399313,2cyclictest0-21swapper/123:46:551
6154993012,7cyclictest26622-21kworker/3:119:45:473
614399301,12cyclictest0-21swapper/120:22:151
6154992926,2cyclictest0-21swapper/319:41:233
6154992912,7cyclictest26622-21kworker/3:122:50:233
6154992912,7cyclictest26622-21kworker/3:122:50:233
615499291,20cyclictest26622-21kworker/3:119:51:413
614399291,11cyclictest0-21swapper/120:11:281
6143992910,3cyclictest0-21swapper/120:09:221
615499281,20cyclictest26622-21kworker/3:120:46:563
614399281,17cyclictest0-21swapper/120:40:471
614399281,11cyclictest0-21swapper/119:44:591
6137992811,14cyclictest28794-21perf20:00:020
615499271,15cyclictest32175-21diskstats21:20:143
615499270,19cyclictest26622-21kworker/3:119:24:053
614399278,4cyclictest14552-21munin-run23:30:011
614399274,11cyclictest8055-21fschecks_count20:25:151
614399271,9cyclictest0-21swapper/119:36:071
614399271,24cyclictest0-21swapper/120:56:051
614399271,17cyclictest0-21swapper/123:11:361
614399271,12cyclictest0-21swapper/121:02:151
614399271,10cyclictest0-21swapper/120:31:471
614399270,9cyclictest0-21swapper/100:14:391
614399270,16cyclictest0-21swapper/119:15:391
614399270,16cyclictest0-21swapper/119:15:391
613799272,12cyclictest18408-21munin-run20:50:010
613799271,14cyclictest0-21swapper/021:05:010
615499260,19cyclictest26622-21kworker/3:120:08:493
615499260,19cyclictest26622-21kworker/3:119:27:333
614399261,9cyclictest0-21swapper/120:15:591
6143992614,2cyclictest27209-21diskmemload21:57:201
614399261,11cyclictest0-21swapper/123:04:091
613799269,14cyclictest3570-21basename23:05:010
613799261,12cyclictest5234-21idleruntime-cro20:20:010
615499252,8cyclictest26622-21kworker/3:123:31:473
615499251,16cyclictest0-21swapper/322:13:263
6154992510,7cyclictest26622-21kworker/3:100:20:553
614399251,6cyclictest10723-21df_inode23:20:131
6143992514,3cyclictest0-21swapper/123:30:301
614399251,2cyclictest0-21swapper/121:21:001
614399251,17cyclictest0-21swapper/100:07:111
614399251,16cyclictest0-21swapper/121:54:401
614399251,12cyclictest0-21swapper/122:01:171
614399251,11cyclictest0-21swapper/121:10:591
614399250,9cyclictest0-21swapper/120:53:271
614399250,10cyclictest0-21swapper/123:44:271
615499249,7cyclictest26622-21kworker/3:120:55:453
615499245,8cyclictest26622-21kworker/3:121:40:213
615499245,8cyclictest26622-21kworker/3:121:40:213
615499241,16cyclictest26622-21kworker/3:120:15:213
615499241,16cyclictest10625-21kworker/u8:200:39:373
615499241,15cyclictest3509-21kworker/u8:021:45:113
615499241,15cyclictest0-21swapper/321:50:123
615499240,9cyclictest26622-21kworker/3:122:43:133
615499240,19cyclictest26622-21kworker/3:120:37:293
615499240,16cyclictest16020-21kworker/u8:022:36:323
614399248,3cyclictest0-21swapper/121:07:011
614399241,3cyclictest29994-21/usr/sbin/munin21:15:141
614399241,20cyclictest27765-21fschecks_count22:45:151
614399240,8cyclictest0-21swapper/122:22:511
614399240,2cyclictest0-21swapper/119:31:521
614399240,13cyclictest0-21swapper/120:45:031
6137992410,3cyclictest0-21swapper/021:16:270
615499236,9cyclictest0-21swapper/300:14:173
615499235,2cyclictest26622-21kworker/3:123:00:133
615499235,2cyclictest26622-21kworker/3:100:02:233
615499234,2cyclictest26622-21kworker/3:100:17:153
615499232,4cyclictest0-21swapper/300:05:243
615499232,13cyclictest3509-21kworker/u8:021:55:393
615499231,14cyclictest0-21swapper/321:33:563
615499231,14cyclictest0-21swapper/321:17:133
615499231,10cyclictest26622-21kworker/3:119:37:053
615099231,6cyclictest0-21swapper/223:20:282
614399238,3cyclictest0-21swapper/119:47:331
614399237,3cyclictest0-21swapper/122:14:011
614399237,3cyclictest0-21swapper/120:35:451
614399233,2cyclictest0-21swapper/122:07:551
614399231,9cyclictest0-21swapper/100:04:081
614399231,3cyclictest30309-21switchtime20:00:231
614399231,12cyclictest0-21swapper/123:18:511
614399231,11cyclictest1723-21egrep21:25:111
614399230,13cyclictest0-21swapper/123:35:591
613799231,11cyclictest0-21swapper/022:04:470
615499228,3cyclictest26622-21kworker/3:122:09:433
615499227,8cyclictest26622-21kworker/3:122:15:133
615499227,2cyclictest26622-21kworker/3:121:39:053
615499227,2cyclictest26622-21kworker/3:121:05:213
615499226,9cyclictest26622-21kworker/3:122:03:373
615499226,6cyclictest0-21swapper/321:10:393
615499226,3cyclictest26622-21kworker/3:123:50:153
615499224,10cyclictest10891-21grep20:30:223
615499222,3cyclictest26622-21kworker/3:120:14:493
615499221,14cyclictest0-21swapper/323:41:533
615499221,14cyclictest0-21swapper/323:06:513
615099221,11cyclictest580-21systemd-journal21:25:012
614399227,3cyclictest0-21swapper/122:17:011
614399221,7cyclictest0-21swapper/119:56:231
614399221,20cyclictest0-21swapper/100:17:321
614399221,19cyclictest0-21swapper/100:35:161
614399221,15cyclictest0-21swapper/123:09:061
614399221,15cyclictest0-21swapper/121:45:041
614399220,9cyclictest0-21swapper/122:30:291
128702220,7sleep011-21rcuc/023:25:110
615499219,2cyclictest26622-21kworker/3:120:25:433
615499219,2cyclictest26622-21kworker/3:119:58:413
615499219,2cyclictest26622-21kworker/3:100:32:493
615499217,7cyclictest26622-21kworker/3:123:38:133
615499215,3cyclictest26622-21kworker/3:123:56:333
615499213,10cyclictest23749-21cat21:00:163
615499212,13cyclictest86550irq/126-enp1s0-23:19:513
615499211,8cyclictest26622-21kworker/3:119:15:173
615499211,8cyclictest26622-21kworker/3:119:15:163
615499211,6cyclictest0-21swapper/322:25:593
615499211,15cyclictest0-21swapper/321:25:373
615099211,7cyclictest7647-21/usr/sbin/munin20:25:242
6150992115,4cyclictest31347-21grep20:05:152
614399219,3cyclictest4937-21ntp_states21:30:211
614399211,9cyclictest0-21swapper/119:27:531
614399211,8cyclictest0-21swapper/122:35:491
614399211,2cyclictest86750irq/128-enp1s0-00:25:161
614399211,11cyclictest25462-21ls19:50:181
614399210,8cyclictest0-21swapper/122:51:091
614399210,8cyclictest0-21swapper/122:51:091
614399210,15cyclictest0-21swapper/121:42:321
614399210,15cyclictest0-21swapper/121:42:321
613799213,10cyclictest0-21swapper/021:13:290
613799211,6cyclictest0-21swapper/000:12:320
613799211,11cyclictest16969-21/usr/sbin/munin23:35:230
615499206,6cyclictest26622-21kworker/3:122:22:013
615499201,14cyclictest0-21swapper/323:24:113
615499201,14cyclictest0-21swapper/322:55:223
615499201,10cyclictest812-21snmpd22:30:163
615099206,4cyclictest8303-21sh19:15:012
615099204,5cyclictest29401-21sh21:15:022
614399201,10cyclictest19230-21smartctl22:25:181
613799204,9cyclictest277072chrt23:59:510
613799202,13cyclictest8303-21latency_hist19:15:010
613799202,10cyclictest5445-21/usr/sbin/munin20:20:170
613799201,6cyclictest11422-21/usr/sbin/munin19:20:180
6137992014,3cyclictest17081-21smart_sda19:30:240
613799201,11cyclictest21361-21/usr/sbin/munin23:45:230
613799201,10cyclictest7626-21grep21:35:260
613799201,10cyclictest0-21swapper/022:40:480
613799201,10cyclictest0-21swapper/020:02:490
6137992010,2cyclictest25771-1kworker/u9:000:35:400
240672201,2sleep00-21swapper/019:46:130
615499198,2cyclictest26622-21kworker/3:119:33:173
615499196,3cyclictest26622-21kworker/3:120:40:283
615099195,4cyclictest0-21swapper/223:59:502
615099192,7cyclictest0-21swapper/222:55:252
615099191,7cyclictest0-21swapper/222:38:582
615099191,7cyclictest0-21swapper/200:32:182
615099191,6cyclictest0-21swapper/200:03:522
615099191,5cyclictest21815-21ntp_states22:30:222
6150991913,4cyclictest25887-21cut21:05:152
615099191,15cyclictest0-21swapper/221:00:182
615099191,12cyclictest0-21swapper/222:54:502
615099191,12cyclictest0-21swapper/222:54:502
615099191,10cyclictest18558-21users23:35:252
6143991915,3cyclictest0-21swapper/100:20:401
614399191,11cyclictest24622-21kworker/u8:122:42:191
613799199,6cyclictest0-21swapper/021:24:560
613799197,3cyclictest9264-21cut21:40:190
613799197,3cyclictest9264-21cut21:40:190
613799195,3cyclictest53472missed_timers00:20:200
613799193,7cyclictest5979-21/usr/sbin/munin23:10:170
613799192,4cyclictest8603-21munin-run00:30:010
613799191,6cyclictest0-21swapper/022:12:110
613799191,5cyclictest0-21swapper/000:02:300
6137991913,3cyclictest14154-21grep20:40:010
615099185,4cyclictest24691-21grep23:50:222
615099184,6cyclictest0-21swapper/200:07:362
615099183,6cyclictest32-21ksoftirqd/223:11:372
615099182,13cyclictest7603-21unixbench_multi21:35:262
615099182,11cyclictest0-21swapper/222:48:002
615099181,8cyclictest4544-21hddtemp_smartct21:30:162
615099181,7cyclictest4761-21/usr/sbin/munin23:05:192
615099181,7cyclictest0-21swapper/221:47:082
615099181,6cyclictest0-21swapper/219:40:262
615099181,4cyclictest0-21swapper/220:12:072
615099181,3cyclictest0-21swapper/221:57:492
6150991813,3cyclictest13629-21users20:35:272
6150991812,3cyclictest18800-21expr22:25:142
615099181,11cyclictest0-21swapper/223:00:312
613799187,3cyclictest8101-21kworker/0:019:15:260
613799187,3cyclictest8101-21kworker/0:019:15:260
613799181,9cyclictest9848-21/usr/sbin/munin20:30:170
613799181,7cyclictest0-21swapper/023:51:070
613799181,7cyclictest0-21swapper/022:26:380
613799181,6cyclictest0-21swapper/022:06:560
613799181,4cyclictest0-21swapper/000:18:050
613799181,3cyclictest0-21swapper/000:10:010
312272181,2sleep00-21swapper/022:51:330
312272181,2sleep00-21swapper/022:51:330
615099179,2cyclictest86750irq/128-enp1s0-21:25:222
615099176,3cyclictest5750irq/9-acpi00:25:252
615099172,14cyclictest0-21swapper/219:28:182
615099171,7cyclictest9245-21egrep19:15:112
615099171,7cyclictest9245-21egrep19:15:112
615099171,7cyclictest29978-21diskstats21:15:132
615099171,7cyclictest16735-21runrttasks19:30:182
615099171,7cyclictest0-21swapper/223:25:192
615099171,6cyclictest0-21swapper/222:43:282
615099171,6cyclictest0-21swapper/221:43:372
615099171,6cyclictest0-21swapper/221:43:372
615099171,3cyclictest0-21swapper/223:40:272
6150991713,3cyclictest0-21swapper/222:05:222
6150991713,3cyclictest0-21swapper/220:31:542
6150991713,3cyclictest0-21swapper/219:37:022
6150991712,3cyclictest13556-21sendmail_mailtr22:00:222
615099171,15cyclictest0-21swapper/223:17:062
615099171,15cyclictest0-21swapper/222:17:562
615099171,14cyclictest0-21swapper/223:45:142
615099171,10cyclictest0-21swapper/222:20:262
615099171,10cyclictest0-21swapper/221:51:562
613799178,3cyclictest0-21swapper/000:30:250
613799177,3cyclictest86550irq/126-enp1s0-23:40:150
613799175,4cyclictest86750irq/128-enp1s0-20:51:160
613799175,3cyclictest86750irq/128-enp1s0-22:45:170
613799175,3cyclictest0-21swapper/022:21:490
613799172,14cyclictest0-21swapper/023:25:010
613799171,9cyclictest25618-21fschecks_count21:05:120
613799171,9cyclictest0-21swapper/022:30:200
613799171,9cyclictest0-21swapper/019:40:140
613799171,7cyclictest0-21swapper/022:39:140
613799171,6cyclictest0-21swapper/022:55:160
613799171,6cyclictest0-21swapper/021:32:530
613799171,14cyclictest0-21swapper/019:35:120
6137991711,3cyclictest833-21runrttasks20:55:270
6137991711,3cyclictest16381-21date20:45:020
613799171,12cyclictest8788-21grep23:15:160
613799170,10cyclictest0-21swapper/023:30:110
224322170,2sleep20-21swapper/220:55:292
181412170,3sleep20-21swapper/220:46:512
152512171,3sleep20-21swapper/222:11:502
615099165,3cyclictest12050irq/123-ahci[0020:25:012
615099161,6cyclictest4421-21/usr/sbin/munin00:20:132
615099161,6cyclictest0-21swapper/219:45:142
615099161,2cyclictest0-21swapper/200:13:182
615099161,14cyclictest0-21swapper/219:58:282
6150991611,3cyclictest18795-21expr20:50:112
6150991611,3cyclictest14238-21/usr/sbin/munin20:40:252
615099160,15cyclictest0-21swapper/220:02:012
613799167,2cyclictest86550irq/126-enp1s0-20:25:170
613799163,3cyclictest0-21swapper/022:15:540
613799161,5cyclictest0-21swapper/021:58:310
613799161,4cyclictest0-21swapper/021:50:430
6137991612,3cyclictest0-21swapper/021:46:060
6137991612,3cyclictest0-21swapper/021:29:060
6137991611,3cyclictest5211-21sed23:05:250
613799160,14cyclictest0-21swapper/020:10:520
22132161,2sleep227209-21diskmemload00:15:072
165852160,2sleep20-21swapper/223:33:092
615099151,5cyclictest0-21swapper/220:15:202
6150991510,3cyclictest11529-21grep19:20:102
613799155,3cyclictest86750irq/128-enp1s0-20:05:210
613799151,11cyclictest13617-21/usr/sbin/munin19:25:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional