You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 01:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 06, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
389210264,34sleep00-21swapper/019:05:580
66128472,8sleep20-21swapper/219:09:042
35228370,8sleep30-21swapper/319:05:323
45428164,13sleep10-21swapper/119:06:431
82199280,0cyclictest0-21swapper/022:29:180
82199260,26cyclictest0-21swapper/021:37:030
821992523,1cyclictest0-21swapper/019:25:120
83199241,20cyclictest0-21swapper/323:11:083
82799240,21cyclictest0-21swapper/221:58:552
825992422,1cyclictest25-21ksoftirqd/123:04:351
82599240,1cyclictest0-21swapper/123:47:281
82199240,23cyclictest0-21swapper/023:05:530
82599230,1cyclictest0-21swapper/121:44:371
83199220,0cyclictest0-21swapper/321:38:283
82599220,0cyclictest0-21swapper/123:56:551
821992222,0cyclictest0-21swapper/023:12:580
827992120,1cyclictest0-21swapper/200:22:342
82199210,2cyclictest0-21swapper/000:35:310
82199210,20cyclictest0-21swapper/021:30:130
825992018,1cyclictest25-21ksoftirqd/122:22:391
82599200,19cyclictest25832-21diskmemload21:14:041
82199200,16cyclictest0-21swapper/022:55:560
82799195,3cyclictest0-21swapper/223:36:242
82599192,3cyclictest25832-21diskmemload00:11:381
82599192,1cyclictest25832-21diskmemload23:38:041
82599190,2cyclictest0-21swapper/120:50:141
82599190,1cyclictest0-21swapper/123:21:521
831991815,2cyclictest39-21ksoftirqd/322:36:113
82799181,1cyclictest7415-21ssh22:54:272
82799180,18cyclictest0-21swapper/223:15:032
82599185,12cyclictest241rcuc/100:00:121
82599180,2cyclictest0-21swapper/122:29:491
82599180,2cyclictest0-21swapper/122:04:231
82199180,1cyclictest0-21swapper/021:43:200
82199180,1cyclictest0-21swapper/000:02:330
82199180,16cyclictest0-21swapper/021:23:350
83199174,12cyclictest0-21swapper/322:22:563
83199171,1cyclictest141rcu_preempt00:22:583
83199171,1cyclictest0-21swapper/323:59:093
827991716,1cyclictest0-21swapper/223:14:522
825991715,1cyclictest25-21ksoftirqd/123:41:281
831991615,1cyclictest0-21swapper/322:30:333
83199160,16cyclictest0-21swapper/321:33:193
82799160,1cyclictest0-21swapper/222:29:072
82599162,2cyclictest141rcu_preempt21:53:161
82199164,8cyclictest24526-21grep22:45:140
82199160,0cyclictest141rcu_preempt23:54:470
82199160,0cyclictest0-21swapper/000:29:370
83199153,2cyclictest141rcu_preempt21:18:293
83199150,14cyclictest0-21swapper/322:00:033
827991515,0cyclictest0-21swapper/219:50:252
82799150,15cyclictest0-21swapper/220:26:462
82599158,1cyclictest141rcu_preempt00:25:221
82599150,1cyclictest0-21swapper/122:55:541
82599150,11cyclictest0-21swapper/120:25:141
82199154,6cyclictest5125-21sed00:10:010
82199154,6cyclictest5125-21sed00:10:000
821991513,2cyclictest13-21ksoftirqd/000:22:520
821991510,3cyclictest3727-21tr19:15:140
831991410,3cyclictest19320-21cut23:20:183
831991410,3cyclictest14246-21chrt23:36:113
831991410,3cyclictest1302-21tr19:10:133
831991410,3cyclictest12376-21tr21:40:143
831991410,2cyclictest25826-21sh21:10:003
83199140,14cyclictest0-21swapper/322:06:213
83199140,13cyclictest11484-21ls20:39:593
83199140,12cyclictest11807-21tr20:40:143
827991414,0cyclictest0-21swapper/221:23:592
827991414,0cyclictest0-21swapper/221:12:172
827991410,3cyclictest26405-21sed23:05:162
82799140,12cyclictest23997-21tr21:05:162
82799140,12cyclictest15406-21head22:20:162
82599148,2cyclictest55550irq/126-enp2s0-23:25:151
82599146,2cyclictest141rcu_preempt22:15:201
825991410,2cyclictest25-21ksoftirqd/121:55:131
82599140,2cyclictest20847-21kworker/1:122:50:371
82599140,14cyclictest0-21swapper/122:39:261
82199144,4cyclictest0-21swapper/023:28:200
82199141,9cyclictest14992-21cstates22:20:140
821991414,0cyclictest0-21swapper/021:19:440
821991410,3cyclictest4274-21head23:30:220
821991410,3cyclictest29231-21head20:05:260
82199140,2cyclictest0-21swapper/023:43:140
82199140,2cyclictest0-21swapper/021:49:220
82199140,1cyclictest0-21swapper/021:10:270
82199140,0cyclictest0-21swapper/021:59:440
83199139,3cyclictest3952-21sh23:30:193
83199136,4cyclictest28926-21tail23:45:173
83199136,3cyclictest25063-21sh23:05:013
83199133,5cyclictest225-21systemd-journal22:50:003
831991312,1cyclictest0-21swapper/322:55:133
831991310,2cyclictest5906-21ssh23:50:263
831991310,2cyclictest32162-21tail22:10:283
831991310,2cyclictest29629-21turbostat.cron20:09:593
831991310,2cyclictest26219-21ssh23:05:163
831991310,2cyclictest21350-21grep21:45:173
831991310,2cyclictest2086-21wc22:50:283
831991310,2cyclictest18750-21cat20:55:013
831991310,2cyclictest13927-21sed19:35:183
831991310,2cyclictest12689-21cron19:35:003
83199130,12cyclictest32001-21dump-pmu-power20:14:593
83199130,12cyclictest16121-21grep19:40:173
83199130,12cyclictest0-21swapper/300:08:373
83199130,12cyclictest0-21swapper/300:08:373
82799139,3cyclictest5629-21grep21:35:272
82799139,2cyclictest21313-21ssh21:45:162
827991310,2cyclictest410-21grep22:30:212
827991310,2cyclictest16087-21tr22:40:142
827991310,2cyclictest1423-21cat20:15:252
82799130,10cyclictest0-21swapper/221:40:132
82599138,4cyclictest13613-21cat19:35:161
82599135,4cyclictest225-21systemd-journal21:05:001
82599134,8cyclictest25-21ksoftirqd/121:25:131
82599133,6cyclictest0-21swapper/119:45:111
825991311,2cyclictest25-21ksoftirqd/123:52:561
825991310,2cyclictest5523-21grep21:35:261
825991310,2cyclictest26200-21grep22:45:251
825991310,2cyclictest13115-21ssh21:20:381
82599130,1cyclictest0-21swapper/100:05:481
82599130,1cyclictest0-21swapper/100:05:471
82199139,3cyclictest10822-21wc20:35:250
82199136,3cyclictest225-21systemd-journal23:59:590
82199133,5cyclictest15310-21lockfile-create22:40:000
821991310,2cyclictest20551-21grep20:55:250
821991310,2cyclictest18720-21grep19:45:150
821991310,2cyclictest17777-21wc20:50:230
821991310,2cyclictest15975-21grep22:40:150
821991310,2cyclictest12216-21sed19:30:260
821991310,2cyclictest12122-21grep23:35:160
82199130,2cyclictest0-21swapper/022:14:310
83199129,2cyclictest8953-21grep19:25:173
83199129,2cyclictest7591-21rm00:30:153
83199129,2cyclictest2178-21sort20:20:013
83199129,2cyclictest21143-21grep21:25:263
83199129,2cyclictest2021-21cron21:15:003
83199129,2cyclictest17782-21head22:40:273
83199129,2cyclictest15797-21ssh00:35:153
83199129,2cyclictest11473-21df21:20:163
83199128,3cyclictest20603-21tail20:55:253
83199128,2cyclictest23073-21apache_processe22:25:133
83199125,3cyclictest225-21systemd-journal00:05:003
83199123,3cyclictest7041-21ssh00:10:223
83199122,7cyclictest0-21swapper/321:50:133
831991211,1cyclictest0-21swapper/320:30:133
83199120,0cyclictest0-21swapper/321:57:573
82799129,2cyclictest9475-21wc19:25:212
82799129,2cyclictest31232-21cut00:25:152
82799129,2cyclictest2987-21ssh21:53:552
82799129,2cyclictest28871-21tune2fs23:45:162
82799129,2cyclictest25176-21ls22:45:172
82799129,2cyclictest22650-21head22:05:192
82799129,2cyclictest21064-21ls21:00:002
82799129,2cyclictest19651-21ssh23:20:212
82799129,2cyclictest18887-21cat23:00:262
82799129,2cyclictest17734-21sed20:50:232
82799129,2cyclictest15329-21cut00:35:142
82799129,2cyclictest13652-21wc00:15:012
82799129,2cyclictest13444-21grep19:35:152
82799128,3cyclictest28359-21grep21:30:162
82799126,3cyclictest4359-21grep23:50:152
82799122,9cyclictest0-21swapper/222:17:372
82799122,2cyclictest225-21systemd-journal22:40:012
82799121,11cyclictest25832-21diskmemload22:02:112
827991210,1cyclictest0-21swapper/223:56:002
82799120,12cyclictest0-21swapper/223:32:232
82799120,11cyclictest0-21swapper/200:32:552
82599129,2cyclictest9618-21tr20:35:151
82599129,2cyclictest4659-21ls19:15:221
82599129,2cyclictest31589-21cut22:30:141
82599129,2cyclictest27966-21grep21:30:141
82599129,2cyclictest24453-21grep00:20:271
82599129,2cyclictest23744-21sh22:45:011
82599129,2cyclictest1814-21head19:10:151
82599129,2cyclictest16792-21grep19:40:241
82599129,2cyclictest1065-21sed20:15:201
82599125,6cyclictest25-21ksoftirqd/122:14:081
825991210,1cyclictest22970-21cat22:05:221
82599120,2cyclictest0-21swapper/123:31:131
82599120,12cyclictest0-21swapper/123:16:101
82599120,11cyclictest31126-21grep20:10:221
82199129,2cyclictest8278-21tr00:30:200
82199129,2cyclictest6854-21cut22:15:170
82199129,2cyclictest6503-21tune2fs00:10:170
82199129,2cyclictest4398-21cron21:55:010
82199129,2cyclictest30600-21grep20:10:180
82199129,2cyclictest28559-21ssh23:45:140
82199129,2cyclictest15998-21tail00:15:260
82199129,2cyclictest13379-21grep20:40:260
82199128,3cyclictest13780-21sort19:35:170
82199128,2cyclictest815-21cyclictest23:21:250
821991210,1cyclictest2705-21grep19:10:250
82199120,2cyclictest0-21swapper/022:32:180
83199118,2cyclictest9797-21ssh22:16:413
83199118,2cyclictest713-21ssh00:25:383
83199118,2cyclictest3060-21sed20:20:193
83199118,2cyclictest27158-21chrt20:03:543
83199118,2cyclictest21856-21wc19:50:263
83199118,2cyclictest21470-21ssh23:40:263
83199118,2cyclictest15779-21tail20:45:253
83199118,2cyclictest15181-21wc00:15:183
83199117,3cyclictest10437-21grep23:15:133
83199110,8cyclictest0-21swapper/319:45:113
83199110,10cyclictest0-21swapper/323:25:183
82799118,2cyclictest32470-21sshd00:06:112
82799118,2cyclictest32470-21sshd00:06:112
82799118,2cyclictest28678-21rm00:04:452
82799118,2cyclictest27775-21tr20:05:152
82799118,2cyclictest25431-21tr20:00:142
82799118,2cyclictest24637-21chrt19:57:162
82799118,2cyclictest22802-21rm21:26:392
82799118,2cyclictest2262-21sh21:15:012
82799118,2cyclictest21725-21sed23:40:282
82799118,2cyclictest2158-21rm22:12:172
82799118,2cyclictest2068-21cat19:10:182
82799118,2cyclictest17381-21chrt19:41:182
82799118,2cyclictest14465-21expr20:45:142
82799117,3cyclictest28353-21wc23:25:222
82799115,4cyclictest17029-21sh00:16:102
82799110,10cyclictest7719-21head20:30:172
82599119,1cyclictest29617-21cron20:10:001
82599119,1cyclictest25-21ksoftirqd/121:17:001
82599119,1cyclictest21843-21grep21:45:201
82599118,2cyclictest8694-21grep20:30:261
82599118,2cyclictest6613-21ssh00:30:011
82599118,2cyclictest2691-21expr20:20:161
82599118,2cyclictest25960-21expr23:05:141
82599118,2cyclictest21169-21ssh00:38:301
82599118,2cyclictest12642-21cat20:40:191
82599118,2cyclictest11576-21cat19:30:201
82599113,4cyclictest141rcu_preempt00:15:531
82599111,10cyclictest3158-21sensors23:10:241
82599110,10cyclictest0-21swapper/121:05:131
82199118,2cyclictest5229-21sort20:25:160
82199118,2cyclictest26880-21ssh21:29:480
82199118,2cyclictest21589-21grep21:00:150
82199118,2cyclictest18348-21ssh23:00:190
82199118,2cyclictest1495-21tail22:50:210
82199118,2cyclictest12999-21ssh23:16:060
82199117,3cyclictest25957-21ssh22:07:070
82199117,2cyclictest9296-21wc20:35:010
83199108,1cyclictest6402-21sed19:20:153
83199108,1cyclictest6386-21chrt20:25:383
83199108,1cyclictest24501-21taskset19:55:313
83199107,2cyclictest23155-21chrt21:01:203
82799108,1cyclictest6234-21cat19:20:142
82799108,1cyclictest41182sleep220:22:002
82799108,1cyclictest31715-21taskset20:11:242
82799108,1cyclictest21455-21aten2_r7power_v21:00:152
82799108,1cyclictest19311-21grep19:45:202
82799108,1cyclictest12598-21cat20:40:192
82799108,1cyclictest112422sleep220:37:022
82799108,1cyclictest10966-21cat19:30:152
82799107,2cyclictest8429-21ssh22:55:032
82799107,2cyclictest5015-21ls19:15:262
82599108,1cyclictest5869-21cut19:20:131
82599108,1cyclictest25746-21cut20:00:161
82599108,1cyclictest23757-21cat19:55:191
82599108,1cyclictest21265-21cat19:50:181
82599107,2cyclictest9902-21sed19:25:251
82599107,2cyclictest19375-21expr20:55:141
82599106,3cyclictest14891-21sed20:45:161
82199108,1cyclictest8035-21sed19:25:000
82199108,1cyclictest3575-21grep20:20:240
82199108,1cyclictest27248-21cron20:05:000
82199108,1cyclictest24352-21grep21:05:170
82199108,1cyclictest16047-21cat19:40:160
82199107,2cyclictest32511-21tr20:15:130
82199107,2cyclictest23971-21tr19:55:220
82199107,2cyclictest22066-21head19:50:260
82199107,2cyclictest14957-21ssh22:00:260
82199107,2cyclictest14281-21cut20:45:130
8319997,1cyclictest4301-21cat19:15:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional