You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 05:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Mon Mar 23, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
83952228159,56sleep20-21swapper/219:05:262
85802204166,25sleep30-21swapper/319:07:333
87072202167,23sleep00-21swapper/019:09:110
84662202165,24sleep10-21swapper/119:06:031
1463221600,2sleep30-21swapper/320:35:023
8874997569,3cyclictest27333-21df23:30:143
8874997470,3cyclictest0-21swapper/300:39:033
202992700,7sleep121-21rcuc/100:25:271
8874996461,2cyclictest0-21swapper/322:55:553
77622600,5sleep0885799cyclictest00:00:020
241442550,4sleep224142-21grep20:55:152
8874994542,2cyclictest0-21swapper/319:56:293
206592450,1sleep00-21swapper/022:00:190
887499331,24cyclictest26252-21kworker/3:121:59:543
887499321,19cyclictest26252-21kworker/3:119:32:153
886899321,12cyclictest0-21swapper/219:32:132
885799318,11cyclictest1986-21idleruntime-cro21:20:010
885799311,13cyclictest0-21swapper/021:49:290
887499302,18cyclictest26252-21kworker/3:119:45:153
887499302,12cyclictest26252-21kworker/3:123:25:553
885799301,13cyclictest0-21swapper/021:23:330
885799300,12cyclictest0-21swapper/022:35:120
201312300,5sleep018545-21/usr/sbin/munin19:30:260
887499292,18cyclictest26252-21kworker/3:121:06:513
887499292,11cyclictest26252-21kworker/3:120:59:113
887499292,10cyclictest26252-21kworker/3:121:42:073
887499291,19cyclictest26252-21kworker/3:121:14:033
887499291,19cyclictest26252-21kworker/3:121:14:033
887499291,17cyclictest26252-21kworker/3:122:53:193
887499291,11cyclictest26252-21kworker/3:122:24:433
887499291,11cyclictest26252-21kworker/3:122:24:433
886899291,12cyclictest0-21swapper/222:50:292
885799291,16cyclictest1900-21grep23:45:170
885799291,13cyclictest0-21swapper/019:35:210
885799291,12cyclictest0-21swapper/021:43:490
887499289,3cyclictest26252-21kworker/3:122:06:473
887499282,17cyclictest26252-21kworker/3:123:44:233
887499282,17cyclictest26252-21kworker/3:121:21:233
887499282,10cyclictest26252-21kworker/3:122:32:353
887499282,10cyclictest26252-21kworker/3:121:52:473
887499281,11cyclictest26252-21kworker/3:120:47:193
887499281,11cyclictest26252-21kworker/3:120:02:393
887499281,11cyclictest26252-21kworker/3:120:02:393
887499281,11cyclictest26252-21kworker/3:119:23:553
886899281,13cyclictest0-21swapper/220:46:532
886899281,12cyclictest0-21swapper/222:07:572
886899280,10cyclictest0-21swapper/221:06:292
887499272,11cyclictest26252-21kworker/3:120:13:383
887499272,10cyclictest26252-21kworker/3:100:04:393
887499271,18cyclictest26252-21kworker/3:121:38:073
887499271,11cyclictest26252-21kworker/3:100:14:313
887499270,11cyclictest26252-21kworker/3:121:02:273
886899271,9cyclictest0-21swapper/200:30:572
886899271,9cyclictest0-21swapper/200:27:412
886899271,12cyclictest0-21swapper/220:17:172
886899271,12cyclictest0-21swapper/219:57:132
886899271,11cyclictest25820-21/usr/sbin/munin19:45:162
886899271,11cyclictest0-21swapper/223:44:572
886899270,9cyclictest0-21swapper/223:08:452
886499270,20cyclictest0-21swapper/122:45:561
8857992711,3cyclictest0-21swapper/023:23:010
885799270,11cyclictest0-21swapper/021:14:170
885799270,11cyclictest0-21swapper/021:14:170
887499261,9cyclictest26252-21kworker/3:123:16:553
8874992613,3cyclictest26252-21kworker/3:100:30:033
887499261,10cyclictest26252-21kworker/3:123:49:393
8874992610,3cyclictest26252-21kworker/3:123:35:253
886899261,9cyclictest0-21swapper/220:10:532
886899261,12cyclictest0-21swapper/222:14:372
886899261,11cyclictest0-21swapper/220:36:132
886899261,10cyclictest0-21swapper/219:37:172
886899261,10cyclictest0-21swapper/219:27:052
886899260,12cyclictest0-21swapper/222:22:532
886899260,12cyclictest0-21swapper/222:22:532
886899260,10cyclictest0-21swapper/223:57:532
8864992619,4cyclictest12250-21/usr/sbin/munin00:10:111
886499260,18cyclictest0-21swapper/121:58:031
885799266,17cyclictest28347-21munin-run22:20:010
885799261,18cyclictest0-21swapper/021:58:370
885799261,13cyclictest25536-21awk21:00:000
885799260,13cyclictest0-21swapper/022:57:090
887499252,11cyclictest26252-21kworker/3:119:26:233
887499252,11cyclictest26252-21kworker/3:119:11:193
887499252,10cyclictest26252-21kworker/3:121:16:113
887499252,10cyclictest26252-21kworker/3:120:15:543
887499251,9cyclictest26252-21kworker/3:121:31:273
8874992511,3cyclictest26252-21kworker/3:119:41:513
887499251,11cyclictest26252-21kworker/3:122:45:233
887499251,11cyclictest26252-21kworker/3:122:02:193
8874992510,3cyclictest26252-21kworker/3:120:06:273
8874992510,3cyclictest26252-21kworker/3:100:08:033
886899251,9cyclictest0-21swapper/221:00:092
886899251,11cyclictest0-21swapper/221:47:412
886899251,11cyclictest0-21swapper/221:44:042
886899251,10cyclictest0-21swapper/220:09:132
886899250,9cyclictest0-21swapper/223:46:532
886899250,9cyclictest0-21swapper/200:35:132
886899250,11cyclictest0-21swapper/219:16:132
886899250,10cyclictest0-21swapper/223:53:052
886899250,10cyclictest0-21swapper/223:16:532
886899250,10cyclictest0-21swapper/220:33:372
886899250,10cyclictest0-21swapper/219:21:372
886499256,11cyclictest0-21swapper/123:37:311
885799257,3cyclictest0-21swapper/000:05:290
885799251,8cyclictest0-21swapper/019:45:250
885799251,12cyclictest8573-21awk21:35:010
885799251,12cyclictest13722-21grep00:10:250
885799251,12cyclictest0-21swapper/023:38:170
885799251,11cyclictest0-21swapper/021:25:410
885799251,10cyclictest0-21swapper/022:47:490
887499249,2cyclictest26252-21kworker/3:120:26:353
887499242,9cyclictest26252-21kworker/3:122:36:033
887499242,10cyclictest26252-21kworker/3:120:20:503
8874992412,3cyclictest26252-21kworker/3:121:26:233
8874992411,3cyclictest26252-21kworker/3:120:37:393
887499241,13cyclictest26252-21kworker/3:119:35:113
887499241,10cyclictest26252-21kworker/3:123:00:153
887499241,10cyclictest26252-21kworker/3:122:15:433
8874992410,3cyclictest26252-21kworker/3:120:54:473
8874992410,2cyclictest26252-21kworker/3:123:52:583
8874992410,2cyclictest26252-21kworker/3:120:40:353
886899241,9cyclictest0-21swapper/223:24:282
886899241,9cyclictest0-21swapper/222:37:212
886899241,11cyclictest0-21swapper/222:31:532
886899241,11cyclictest0-21swapper/221:58:332
886899241,11cyclictest0-21swapper/219:43:242
886899241,10cyclictest0-21swapper/223:28:002
886899240,9cyclictest0-21swapper/222:56:532
886899240,12cyclictest0-21swapper/221:54:492
886899240,10cyclictest0-21swapper/222:00:172
886899240,10cyclictest0-21swapper/200:22:492
886899240,10cyclictest0-21swapper/200:03:452
885799241,10cyclictest0-21swapper/023:51:050
885799241,10cyclictest0-21swapper/000:39:240
887499239,3cyclictest26252-21kworker/3:123:11:073
887499239,2cyclictest26252-21kworker/3:119:17:513
887499238,3cyclictest26252-21kworker/3:122:12:073
887499231,6cyclictest0-21swapper/321:48:283
8874992311,3cyclictest26252-21kworker/3:123:24:433
8874992311,3cyclictest26252-21kworker/3:123:05:233
887499231,10cyclictest26252-21kworker/3:100:17:433
8874992310,3cyclictest26252-21kworker/3:123:55:143
8874992310,3cyclictest26252-21kworker/3:100:29:273
886899237,3cyclictest19787-21users23:10:252
886899236,3cyclictest0-21swapper/221:12:532
886899236,3cyclictest0-21swapper/221:12:532
886899230,8cyclictest0-21swapper/220:20:292
886499231,3cyclictest13577-21munin-run23:00:011
8864992310,3cyclictest25524-21munin-run21:00:011
885799236,8cyclictest0-21swapper/020:03:150
885799236,8cyclictest0-21swapper/020:03:150
885799231,8cyclictest0-21swapper/000:23:050
885799231,5cyclictest0-21swapper/023:41:430
8857992313,3cyclictest0-21swapper/021:50:250
885799231,15cyclictest0-21swapper/022:31:040
885799231,13cyclictest0-21swapper/020:07:050
885799231,11cyclictest24915-21munin-run19:45:010
885799231,10cyclictest0-21swapper/023:00:370
887499228,3cyclictest26252-21kworker/3:122:26:433
8874992210,3cyclictest26252-21kworker/3:122:44:233
886899221,10cyclictest0-21swapper/222:17:132
886899220,10cyclictest0-21swapper/200:15:452
886499228,4cyclictest0-21swapper/121:50:261
886499222,15cyclictest0-21swapper/123:59:221
886499221,7cyclictest0-21swapper/123:15:221
886499221,6cyclictest0-21swapper/121:31:041
886499221,18cyclictest1336-21perf20:05:011
886499221,18cyclictest1336-21perf20:05:011
886499221,16cyclictest0-21swapper/123:54:441
886499220,4cyclictest0-21swapper/100:16:491
886499220,15cyclictest0-21swapper/123:27:311
885799228,3cyclictest0-21swapper/023:11:290
885799228,3cyclictest0-21swapper/022:29:000
885799224,9cyclictest0-21swapper/000:32:170
885799221,14cyclictest0-21swapper/019:57:290
885799221,11cyclictest0-21swapper/019:13:250
885799220,13cyclictest0-21swapper/020:45:240
886899212,6cyclictest0-21swapper/221:16:492
886899210,8cyclictest0-21swapper/221:33:252
886499216,3cyclictest147852tail19:20:161
886499212,9cyclictest30193-21kworker/u8:022:40:201
8864992117,3cyclictest0-21swapper/122:30:491
8864992117,3cyclictest0-21swapper/122:26:261
8864992116,3cyclictest9771-21awk22:50:141
8864992115,4cyclictest30055-21grep22:20:251
8864992115,4cyclictest30055-21grep22:20:251
886499211,2cyclictest6121-21taskset21:27:051
886499211,18cyclictest0-21swapper/121:35:191
886499211,18cyclictest0-21swapper/121:05:021
886499211,15cyclictest0-21swapper/123:47:511
886499211,15cyclictest0-21swapper/123:40:061
886499211,14cyclictest0-21swapper/100:22:261
886499211,11cyclictest23557-21missed_timers23:20:191
886499211,11cyclictest0-21swapper/122:14:341
886499211,11cyclictest0-21swapper/100:02:031
885799218,4cyclictest5966-21/usr/sbin/munin20:15:150
885799218,4cyclictest0-21swapper/022:54:130
885799218,3cyclictest0-21swapper/021:05:250
885799218,2cyclictest0-21swapper/021:35:210
885799217,3cyclictest0-21swapper/000:17:170
885799211,9cyclictest0-21swapper/019:25:130
885799211,11cyclictest8166-21/usr/sbin/munin20:20:260
885799210,9cyclictest0-21swapper/020:12:390
130192211,16sleep113095-21awk19:15:231
887499208,3cyclictest26252-21kworker/3:100:21:273
887499201,13cyclictest26252-21kworker/3:119:51:553
886899206,3cyclictest0-21swapper/220:53:532
886499202,16cyclictest0-21swapper/119:56:201
886499202,16cyclictest0-21swapper/119:53:181
8864992017,2cyclictest23-21ksoftirqd/100:40:011
8864992016,3cyclictest0-21swapper/122:16:411
8864992016,3cyclictest0-21swapper/120:10:421
886499201,5cyclictest21-21rcuc/120:15:161
886499201,4cyclictest0-21swapper/123:06:041
8864992013,2cyclictest0-21swapper/123:12:271
886499201,18cyclictest0-21swapper/120:24:271
886499201,17cyclictest0-21swapper/122:35:451
886499201,17cyclictest0-21swapper/120:32:041
886499201,17cyclictest0-21swapper/119:35:181
886499201,11cyclictest0-21swapper/122:05:141
886499201,10cyclictest12337-21latency_hist20:30:021
886499200,15cyclictest0-21swapper/123:30:571
885799209,2cyclictest0-21swapper/000:29:010
885799207,3cyclictest0-21swapper/020:27:570
885799204,3cyclictest86650irq/127-enp1s0-20:35:090
885799204,3cyclictest0-21swapper/019:17:210
885799202,10cyclictest16007-21/usr/sbin/munin23:05:150
885799201,12cyclictest0-21swapper/021:04:400
885799201,11cyclictest28575-21/usr/sbin/munin22:20:110
885799201,11cyclictest28575-21/usr/sbin/munin22:20:110
885799201,11cyclictest0-21swapper/023:26:130
885799201,11cyclictest0-21swapper/019:20:250
885799201,10cyclictest27337-21/usr/sbin/munin19:50:140
885799201,10cyclictest0-21swapper/022:43:530
8868991913,3cyclictest5652-21gltestperf22:40:172
886499198,3cyclictest0-21swapper/121:40:181
886499192,16cyclictest0-21swapper/120:05:211
886499191,9cyclictest833-21runrttasks00:30:211
8864991915,3cyclictest0-21swapper/121:02:491
8864991915,3cyclictest0-21swapper/119:46:191
8864991915,3cyclictest0-21swapper/119:41:471
886499191,17cyclictest0-21swapper/123:04:441
886499191,17cyclictest0-21swapper/121:48:141
886499191,17cyclictest0-21swapper/121:17:391
886499191,17cyclictest0-21swapper/119:31:301
886499191,16cyclictest0-21swapper/121:20:361
886499191,16cyclictest0-21swapper/119:10:421
886499191,10cyclictest17225-21latency19:25:191
886499190,10cyclictest0-21swapper/100:05:401
885799195,9cyclictest0-21swapper/023:19:090
885799191,11cyclictest0-21swapper/023:31:530
885799191,11cyclictest0-21swapper/022:05:290
143482190,2sleep00-21swapper/020:32:470
886899186,3cyclictest0-21swapper/221:21:492
886899185,3cyclictest0-21swapper/220:01:532
886899185,3cyclictest0-21swapper/220:01:532
886899184,3cyclictest0-21swapper/220:41:532
886899183,3cyclictest0-21swapper/223:36:052
886899181,9cyclictest9526-21if_enp1s021:35:152
886899181,7cyclictest0-21swapper/222:49:512
886899181,5cyclictest30776-21/usr/sbin/munin22:25:172
886899181,11cyclictest0-21swapper/223:34:312
886899181,10cyclictest0-21swapper/200:12:572
886899180,11cyclictest0-21swapper/221:27:572
8864991815,2cyclictest0-21swapper/120:35:231
8864991814,3cyclictest0-21swapper/120:45:291
886499181,16cyclictest0-21swapper/122:04:121
886499181,16cyclictest0-21swapper/121:11:241
886499181,16cyclictest0-21swapper/121:11:241
886499181,16cyclictest0-21swapper/120:40:531
886499180,16cyclictest0-21swapper/120:50:061
885799187,3cyclictest0-21swapper/022:12:370
885799184,3cyclictest86750irq/128-enp1s0-20:50:160
886899174,4cyclictest15308-21tail23:00:262
886899171,8cyclictest0-21swapper/220:26:092
886899171,6cyclictest27626-21expr19:50:122
885799171,3cyclictest0-21swapper/020:40:250
885799170,10cyclictest0-21swapper/000:03:010
886899161,14cyclictest0-21swapper/200:07:152
886899160,15cyclictest0-21swapper/219:10:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional