You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 00:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Wed Aug 26, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162432209104,76sleep20-21swapper/207:07:402
162572163108,30sleep10-21swapper/107:07:511
162362160104,28sleep00-21swapper/007:07:350
161872160110,28sleep30-21swapper/307:06:573
155192650,2sleep00-21swapper/011:42:450
105452620,1sleep30-21swapper/310:26:123
87892590,2sleep10-21swapper/108:05:161
60682580,3sleep1211rcuc/112:00:121
305422580,1sleep00-21swapper/011:05:210
269032580,2sleep00-21swapper/011:26:480
188042550,3sleep118801-21ntpq09:20:201
94902510,2sleep20-21swapper/209:13:552
277992510,1sleep00-21swapper/011:03:450
253522500,2sleep228-21ksoftirqd/210:13:582
227582500,8sleep228-21ksoftirqd/209:23:422
256452490,1sleep30-21swapper/312:38:583
321002480,1sleep00-21swapper/010:42:260
141452480,1sleep1211rcuc/110:53:311
77162470,2chrt0-21swapper/009:12:080
281962460,1sleep00-21swapper/009:27:250
281962460,1sleep00-21swapper/009:27:250
1642624530,8sleep20-21swapper/207:10:012
152072450,1sleep30-21swapper/312:06:213
174392360,2sleep10-21swapper/110:55:231
16698993127,3cyclictest211rcuc/110:39:581
16698993112,9cyclictest636-21runrttasks08:00:341
246952290,2chrt0-21swapper/009:49:240
16698992925,3cyclictest211rcuc/111:00:311
16698992925,3cyclictest211rcuc/110:03:501
16698992925,3cyclictest211rcuc/109:44:191
86852280,1sleep10-21swapper/110:49:451
1669899281,25cyclictest0-21swapper/107:39:471
1669899281,25cyclictest0-21swapper/107:22:131
1669799281,25cyclictest0-21swapper/008:12:160
1669799281,25cyclictest0-21swapper/008:07:250
1669799281,25cyclictest0-21swapper/007:53:270
1669799281,25cyclictest0-21swapper/007:38:220
16700992722,3cyclictest660-21lldpd07:41:303
16700992721,3cyclictest11742-21/usr/sbin/munin08:10:243
1670099271,24cyclictest0-21swapper/308:21:423
1670099271,24cyclictest0-21swapper/307:24:383
1669999271,24cyclictest0-21swapper/208:13:412
1669999271,24cyclictest0-21swapper/207:22:302
1669899278,7cyclictest22-21ksoftirqd/109:55:231
1669899274,7cyclictest22-21ksoftirqd/112:35:411
16698992723,3cyclictest211rcuc/111:43:531
16698992723,3cyclictest211rcuc/110:11:441
16698992723,3cyclictest211rcuc/109:11:491
1669899271,24cyclictest0-21swapper/109:03:351
1669899271,24cyclictest0-21swapper/108:10:251
1669899271,24cyclictest0-21swapper/107:41:301
1669899271,24cyclictest0-21swapper/107:31:301
1669899271,24cyclictest0-21swapper/107:26:091
16698992711,8cyclictest22-21ksoftirqd/110:16:041
16698992710,10cyclictest22-21ksoftirqd/112:34:561
1669799271,25cyclictest0-21swapper/007:31:360
1669799271,24cyclictest0-21swapper/008:57:100
1669799271,24cyclictest0-21swapper/008:48:010
1669799271,24cyclictest0-21swapper/008:33:200
1669799271,24cyclictest0-21swapper/008:27:090
1669799271,24cyclictest0-21swapper/008:21:410
1669799271,24cyclictest0-21swapper/008:16:260
1669799271,24cyclictest0-21swapper/007:46:360
1669799271,24cyclictest0-21swapper/007:25:480
1669799271,24cyclictest0-21swapper/007:21:280
1670099261,2cyclictest0-21swapper/307:53:273
1670099261,23cyclictest0-21swapper/308:56:483
1670099261,23cyclictest0-21swapper/308:45:483
1670099261,23cyclictest0-21swapper/308:26:103
1670099261,23cyclictest0-21swapper/307:45:523
1670099261,23cyclictest0-21swapper/307:38:203
1670099261,23cyclictest0-21swapper/307:31:013
1670099261,23cyclictest0-21swapper/307:26:073
1670099260,24cyclictest0-21swapper/308:36:303
1669999261,23cyclictest0-21swapper/208:17:162
1669999261,23cyclictest0-21swapper/208:06:092
1669999261,23cyclictest0-21swapper/207:40:322
1669999261,19cyclictest0-21swapper/207:28:092
1669899265,7cyclictest22-21ksoftirqd/112:21:301
1669899264,7cyclictest22-21ksoftirqd/109:16:021
1669899263,7cyclictest22-21ksoftirqd/112:25:001
1669899263,7cyclictest22-21ksoftirqd/112:11:321
16698992624,1cyclictest0-21swapper/108:56:271
16698992622,3cyclictest211rcuc/111:50:061
16698992622,3cyclictest211rcuc/111:28:201
16698992622,3cyclictest211rcuc/111:23:481
16698992622,3cyclictest211rcuc/109:26:501
16698992622,3cyclictest211rcuc/109:26:501
16698992622,3cyclictest211rcuc/107:48:331
16698992621,3cyclictest211rcuc/111:05:041
1669899261,7cyclictest11131-21sh10:26:461
1669899260,24cyclictest0-21swapper/108:40:521
1669899260,24cyclictest0-21swapper/108:26:331
1669899260,24cyclictest0-21swapper/108:15:391
16697992624,1cyclictest0-21swapper/007:41:080
1669799261,23cyclictest0-21swapper/008:41:250
1669799261,1cyclictest0-21swapper/008:53:410
89482250,2sleep20-21swapper/211:37:542
178402250,2chrt0-21swapper/211:45:002
1670099251,23cyclictest0-21swapper/308:06:383
1670099251,1cyclictest0-21swapper/308:33:213
1670099251,1cyclictest0-21swapper/308:16:363
1669999253,16cyclictest28374-21ssh10:40:102
1669999251,23cyclictest0-21swapper/208:52:052
1669999251,23cyclictest0-21swapper/207:36:232
1669999251,1cyclictest0-21swapper/208:46:052
1669999251,1cyclictest0-21swapper/207:32:142
1669999250,23cyclictest0-21swapper/208:26:412
1669899256,8cyclictest22-21ksoftirqd/112:05:141
1669899254,7cyclictest22-21ksoftirqd/112:16:321
1669899254,7cyclictest22-21ksoftirqd/111:59:411
1669899253,7cyclictest22-21ksoftirqd/111:18:151
1669899253,7cyclictest22-21ksoftirqd/109:49:051
16698992523,1cyclictest0-21swapper/108:46:081
16698992523,1cyclictest0-21swapper/108:21:501
1669899252,16cyclictest19335-21apt-get08:30:031
16698992521,3cyclictest211rcuc/110:41:361
16698992521,3cyclictest211rcuc/110:08:541
16698992521,3cyclictest211rcuc/108:50:031
16697992523,1cyclictest0-21swapper/009:03:460
16697992514,8cyclictest1630-21sed09:55:260
99582240,2sleep30-21swapper/311:38:553
308142240,1sleep20-21swapper/209:30:002
308142240,1sleep20-21swapper/209:30:002
16700992422,1cyclictest0-21swapper/308:52:533
16700992414,7cyclictest11375-21df_inode09:15:143
1669999243,14cyclictest1106-21ssh10:20:132
16699992422,1cyclictest0-21swapper/208:30:372
1669999242,16cyclictest25717-21ssh09:50:112
1669999241,1cyclictest0-21swapper/208:41:442
1669999241,10cyclictest223152grep10:35:162
1669999240,22cyclictest0-21swapper/208:57:082
1669899243,7cyclictest22-21ksoftirqd/111:46:591
1669899243,7cyclictest22-21ksoftirqd/109:50:491
1669899242,6cyclictest211rcuc/109:35:131
16698992420,3cyclictest211rcuc/111:10:341
16698992420,3cyclictest211rcuc/110:22:101
16698992419,3cyclictest211rcuc/108:36:431
16697992414,6cyclictest52950irq/141-enp3s0-12:35:120
216392230,2sleep30-21swapper/310:35:103
200802230,1sleep00-21swapper/007:15:230
16700992321,1cyclictest0-21swapper/308:41:043
16700992321,1cyclictest0-21swapper/308:04:553
1669999233,13cyclictest24907-21ssh11:50:132
1669999232,15cyclictest16058-21ssh10:55:132
16699992321,1cyclictest0-21swapper/207:45:482
1669999230,1cyclictest0-21swapper/208:21:462
1669899236,6cyclictest22-21ksoftirqd/110:30:461
1669899233,7cyclictest22-21ksoftirqd/111:32:141
1669899232,6cyclictest22-21ksoftirqd/109:30:541
16698992321,1cyclictest0-21swapper/109:09:561
1669899231,6cyclictest22-21ksoftirqd/111:35:121
16697992320,2cyclictest0-21swapper/008:04:550
5632220,19sleep3583-21ssh11:31:133
1670099227,3cyclictest141rcu_preempt11:05:203
16698992217,3cyclictest211rcuc/107:18:511
16697992213,5cyclictest52950irq/141-enp3s0-10:35:140
25572213,9sleep228-21ksoftirqd/211:33:132
1670099215,14cyclictest11918-21cat11:15:243
16700992118,2cyclictest0-21swapper/309:03:353
16700992113,6cyclictest5054-21swap11:10:233
16700992113,6cyclictest4947-21seq09:34:523
1670099211,18cyclictest6370-21perl12:00:143
1670099211,18cyclictest3837-21cron09:10:003
1670099211,18cyclictest22030-21seq12:11:273
16700992111,8cyclictest20438-21cat10:10:133
1669999217,4cyclictest66050lldpd10:09:472
16699992114,4cyclictest32525-21grep11:55:182
16699992111,7cyclictest278-21systemd-journal12:19:592
16698992118,2cyclictest211rcuc/107:12:211
16698992117,3cyclictest211rcuc/107:50:491
1669799215,3cyclictest3855-21diskmemload11:10:210
1669799212,14cyclictest19281-21apt-get12:10:110
1669799212,13cyclictest515-21irqbalance08:35:130
1669799211,5cyclictest5524-21apt-get09:35:130
16697992113,6cyclictest4435-21users10:45:290
16697992112,7cyclictest560-21dbus-daemon10:30:000
16697992112,7cyclictest11492-21tune2fs09:15:150
16697992112,6cyclictest278-21systemd-journal07:15:000
16697992111,6cyclictest12274-21ssh09:40:120
48412200,2chrt0-21swapper/210:45:532
1670099208,6cyclictest32176-21cut09:55:023
1670099207,10cyclictest2411-21sh09:56:063
1670099204,2cyclictest3855-21diskmemload10:45:223
1670099202,13cyclictest17782-21ssh09:20:133
1670099201,9cyclictest26715-21irqstats12:15:183
1670099201,6cyclictest13397-21awk10:05:113
1670099201,4cyclictest28357-21apt-get10:40:103
16700992014,4cyclictest4836-21sh10:22:263
16700992013,4cyclictest24701-21sh11:50:033
16700992012,6cyclictest7414-21tune2fs12:25:143
16700992012,6cyclictest18477-21seq07:13:153
16700992012,6cyclictest13216-21awk11:40:263
16700992011,7cyclictest9711-21ssh09:14:083
16700992011,7cyclictest23978-21cat11:25:133
1669999203,12cyclictest640-21snmpd09:00:122
1669999203,11cyclictest53050irq/142-enp3s0-11:05:132
16699992013,5cyclictest12181-21wc12:05:002
16699992013,4cyclictest8911-21open_inodes10:25:222
16699992012,6cyclictest24533-21cut11:50:012
16699992012,6cyclictest22637-21which08:35:242
16699992012,6cyclictest12284-21which09:15:222
16699992012,5cyclictest3855-21diskmemload09:42:142
1669899200,7cyclictest636-21runrttasks07:55:521
1669799204,3cyclictest3855-21diskmemload11:37:110
1669799202,4cyclictest6936-21apt-get12:25:110
1669799201,5cyclictest0-21swapper/010:05:320
16697992013,5cyclictest11949-21ls11:15:240
16697992013,4cyclictest3855-21diskmemload12:04:130
16697992012,6cyclictest5462-21seq07:58:210
16697992012,6cyclictest15681-21seq12:06:490
16697992012,6cyclictest13596-21seq10:52:570
1669799201,16cyclictest32084-21latency_hist09:55:010
16697992010,6cyclictest52950irq/141-enp3s0-09:30:110
240282190,1sleep00-21swapper/011:49:350
1670099198,3cyclictest52750irq/139-enp3s0-11:49:493
1670099197,4cyclictest52750irq/139-enp3s0-11:03:113
1670099192,3cyclictest3855-21diskmemload10:33:423
1670099191,5cyclictest0-21swapper/310:19:163
16700991914,3cyclictest27576-21sh09:26:503
16700991914,3cyclictest27576-21sh09:26:493
16700991913,4cyclictest4420-21sh11:59:003
16700991913,4cyclictest23458-21sh09:48:103
16700991913,4cyclictest19925-21sh12:34:593
16700991913,4cyclictest12166-21sh09:40:073
16700991911,6cyclictest19707-21expr07:15:193
16700991911,6cyclictest18273-21awk11:20:223
16700991911,5cyclictest3855-21diskmemload09:38:183
16700991910,6cyclictest4084-21cut07:55:163
1669999198,3cyclictest28-21ksoftirqd/209:55:182
1669999192,13cyclictest75-21kswapd011:10:112
16699991914,3cyclictest7382-21hwlatdetect10:00:162
16699991912,5cyclictest29654-21sh10:16:342
16699991912,4cyclictest21084-21apt-key10:35:002
16699991911,6cyclictest20783-21cat12:10:262
1669999191,15cyclictest21476-21ssh12:35:222
1669799194,12cyclictest15966-21processes10:30:220
1669799193,3cyclictest141rcu_preempt12:30:100
1669799192,3cyclictest3855-21diskmemload11:23:550
1669799191,5cyclictest0-21swapper/012:18:480
1669799191,5cyclictest0-21swapper/010:24:550
16697991911,6cyclictest19365-21cat09:20:260
16697991911,5cyclictest636-21runrttasks10:57:330
16697991910,6cyclictest32711-21apt-get12:20:130
1669799190,6cyclictest27854-21ntpq10:15:200
1670099181,4cyclictest141rcu_preempt10:50:353
16700991813,3cyclictest8498-21sh10:00:283
16700991813,3cyclictest2377-21sh12:20:483
16700991813,3cyclictest18785-21sh10:56:263
1669999189,6cyclictest53050irq/142-enp3s0-10:50:192
1669999182,10cyclictest75-21kswapd011:20:122
16699991813,3cyclictest27721-21sh11:03:402
1669999181,2cyclictest141rcu_preempt11:25:112
16699991812,4cyclictest22219-21sh09:46:572
16699991812,4cyclictest17006-21sh12:08:082
16699991811,5cyclictest2223-21expr09:05:132
16699991811,4cyclictest3855-21diskmemload11:19:382
16699991811,4cyclictest14255-21cut12:30:152
1669999181,12cyclictest7254-21users09:35:262
16699991810,6cyclictest7607-21ls12:25:162
16699991810,6cyclictest19127-21cat07:15:132
1669799184,12cyclictest31685-21cpuspeed_turbos11:55:120
16697991813,3cyclictest30473-21apt-get11:30:100
16697991812,4cyclictest22457-21sh10:11:030
16697991811,5cyclictest3145-21ls09:05:240
16697991810,6cyclictest6926-21cat10:00:140
16697991810,6cyclictest560-21dbus-daemon11:50:240
1669999174,3cyclictest3855-21diskmemload12:20:062
1669999172,10cyclictest1579-21apt-get07:50:132
1669999171,11cyclictest5617-21apt-get08:00:032
16699991710,5cyclictest4099-21fschecks_count07:55:152
16699991710,4cyclictest3855-21diskmemload09:30:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional