You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat Jun 27, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6060217097,24sleep00-21swapper/007:07:580
6082216999,23sleep20-21swapper/207:08:162
61682140104,25sleep10-21swapper/107:09:231
61252140103,26sleep30-21swapper/307:08:493
404921050,2sleep20-21swapper/210:05:232
344721000,2sleep20-21swapper/209:16:312
259542670,2sleep225955-21sh11:10:162
29002650,1sleep10-21swapper/109:15:581
190322590,2sleep20-21swapper/210:16:272
147202580,4sleep00-21swapper/011:50:110
279382570,3sleep30-21swapper/310:23:383
195172550,2sleep10-21swapper/111:53:351
16582540,2sleep10-21swapper/110:28:061
130642540,2sleep30-21swapper/309:24:273
128462540,1sleep10-21swapper/110:35:521
145762520,1sleep00-21swapper/011:01:170
306382510,1sleep20-21swapper/212:25:212
216322510,1sleep10-21swapper/110:42:591
6912500,2sleep00-21swapper/012:03:320
649499501,26cyclictest27136-21ssh11:35:141
6493995013,27cyclictest14147-21sh09:49:320
92542480,1sleep10-21swapper/109:45:201
649399471,32cyclictest20980-21ssh11:05:580
649399471,30cyclictest0-21swapper/009:32:540
178692470,1sleep30-21swapper/309:51:323
6494994611,25cyclictest7297-21sh09:44:191
649499460,31cyclictest932-21ssh09:39:401
649499460,31cyclictest19421-21ssh12:17:091
649499460,2cyclictest25697-21diskmemload11:46:131
649399464,22cyclictest28734-21sh11:12:010
649399461,31cyclictest14709-21rm10:37:460
267292460,1sleep20-21swapper/207:55:182
649499451,25cyclictest1987-21chrt08:14:271
649499450,31cyclictest15649-21ssh11:25:491
649499450,29cyclictest31534-21ssh09:13:541
649499450,29cyclictest31534-21ssh09:13:541
649499450,29cyclictest26885-21ssh09:58:531
649399453,25cyclictest26663-21rm11:58:480
649399451,25cyclictest30059-21ssh09:36:190
6493994511,24cyclictest22818-21sh11:31:170
649399450,31cyclictest16045-21ssh09:25:420
257892450,2sleep3141rcu_preempt09:33:463
179062450,1sleep30-21swapper/310:40:163
649499441,26cyclictest2514-21chrt11:16:171
649499440,30cyclictest24678-21ssh11:09:421
649399446,29cyclictest9675-21sh10:58:060
649399441,32cyclictest4010-21seq08:17:550
649399441,24cyclictest865-21sh10:03:350
649399440,34cyclictest16798-21seq08:46:040
649399440,30cyclictest26312-21ssh10:22:000
649399440,29cyclictest13526-21ssh10:12:390
311982440,1sleep00-21swapper/008:05:190
649499431,27cyclictest25913-21ssh12:21:561
649499431,26cyclictest27446-21taskset10:46:521
649499431,26cyclictest27446-21taskset10:46:521
649499431,26cyclictest25029-21ssh11:33:311
649499431,22cyclictest3665-21ntp_states12:05:221
649499430,26cyclictest19161-21chrt08:53:591
649499430,24cyclictest5659-21sed08:20:241
649399431,33cyclictest0-21swapper/012:13:450
649399431,31cyclictest5822-21sh11:19:380
649399431,27cyclictest4823-21sh09:17:550
649399431,26cyclictest20077-21ssh10:41:250
649399430,26cyclictest14894-21ssh12:38:030
649399430,25cyclictest9723-21ssh11:21:350
649499422,22cyclictest30027-21chrt08:03:071
649499421,29cyclictest25039-21ssh10:20:421
649499421,26cyclictest21378-21cron09:00:001
649399421,31cyclictest29004-21ssh10:48:260
649399421,31cyclictest29004-21ssh10:48:260
649399421,25cyclictest6744-21ssh12:07:530
649399421,23cyclictest12479-21ssh11:48:150
649399420,30cyclictest10809-21runrttasks07:20:070
270672420,2chrt0-21swapper/312:23:053
245142420,1sleep20-21swapper/211:33:002
649499411,28cyclictest18592-21ssh09:28:171
649499411,25cyclictest12381-21runrttasks07:20:451
649499411,24cyclictest15161-21sh11:01:501
649499410,29cyclictest10740-21ssh11:22:371
649499410,28cyclictest11666-21ssh12:11:051
649499410,24cyclictest14859-21seq08:43:391
649399412,24cyclictest0-21swapper/009:40:390
649399411,30cyclictest9201-21sh09:20:330
649399411,30cyclictest21222-21seq07:42:520
649399411,25cyclictest25161-21ssh12:21:090
649399411,23cyclictest1179-21ssh12:27:430
649399410,31cyclictest7963-21taskset12:32:490
649399410,31cyclictest15552-21ssh11:25:430
649399410,24cyclictest31068-21mailstats10:25:250
649399410,24cyclictest26224-21ssh09:10:060
649399410,24cyclictest26224-21ssh09:10:060
649499401,27cyclictest7460-21ssh10:32:111
649499401,26cyclictest8288-21seq07:13:301
649499401,25cyclictest7303-21ssh11:44:461
649499401,24cyclictest25675-21cron09:10:001
649499401,24cyclictest21632-21chrt09:00:051
649499401,23cyclictest25697-21diskmemload11:10:021
649499401,23cyclictest20407-21ssh10:17:501
649499401,23cyclictest10454-21chrt08:32:161
649499401,22cyclictest25697-21diskmemload12:33:351
649499401,22cyclictest10764-21cstates12:35:141
649499400,29cyclictest3566-21rm10:53:431
649499400,28cyclictest23949-21ssh09:31:541
649499400,28cyclictest2135-21ssh12:28:411
649399401,26cyclictest25535-21seq09:08:150
649399401,25cyclictest3393-21rm10:53:320
649399401,24cyclictest640-21snmpd08:52:150
649399401,24cyclictest21778-21ssh10:19:130
649399401,22cyclictest5122-21ssh10:06:070
649399400,30cyclictest20113-21sh12:17:500
649399400,26cyclictest6285-21seq08:23:450
649399400,26cyclictest10266-21seq07:16:230
649399400,25cyclictest25871-21sh09:57:510
649399400,22cyclictest17404-21ssh09:51:040
649499398,17cyclictest636-21runrttasks07:57:321
649499392,24cyclictest12763-21seq08:39:301
649499391,24cyclictest25697-21diskmemload10:05:051
649499391,22cyclictest25697-21diskmemload12:02:191
649499391,22cyclictest25697-21diskmemload10:00:351
649499391,22cyclictest25697-21diskmemload09:50:431
649499391,21cyclictest9264-21rm09:20:361
649499390,24cyclictest14697-21chrt07:28:041
649499390,23cyclictest8833-21chrt10:57:151
649399391,28cyclictest23521-21seq09:04:500
649399391,24cyclictest8260-21sh10:33:000
649399391,24cyclictest8143-21seq07:11:380
649399390,29cyclictest27929-21seq07:58:570
649399390,23cyclictest32214-21ssh11:39:000
649499381,21cyclictest21996-21ssh11:55:161
649499380,23cyclictest18872-21chrt07:36:421
649499380,23cyclictest16772-21chrt07:32:131
649399380,28cyclictest1124-21sh11:40:060
649499370,23cyclictest31991-21chrt08:05:521
649499370,23cyclictest10514-21chrt07:19:331
649499370,21cyclictest21141-21chrt07:41:481
649399371,23cyclictest8231-21taskset08:26:120
649399371,23cyclictest8231-21taskset08:26:120
649599364,29cyclictest10573-21turbostat07:20:002
649499361,22cyclictest636-21runrttasks07:47:011
649499361,22cyclictest3922-21chrt08:16:471
649499360,22cyclictest25625-21chrt07:51:471
649499360,22cyclictest12733-21ssh10:11:511
649399361,26cyclictest1731-21runrttasks08:11:090
649399360,27cyclictest14588-21seq07:26:380
649399360,22cyclictest636-21runrttasks07:33:050
6495993511,5cyclictest19439-21cat08:55:012
649399351,21cyclictest0-21swapper/008:56:030
649399350,22cyclictest25571-21seq07:51:060
649499341,21cyclictest636-21runrttasks08:49:171
649499340,21cyclictest8435-21chrt08:28:511
649499340,21cyclictest8435-21chrt08:28:511
649399340,17cyclictest22477-21ls07:45:180
6495993313,17cyclictest15209-21sh10:14:212
6495993312,4cyclictest9619-21sh09:20:582
649399339,22cyclictest30025-21seq08:03:060
649399330,20cyclictest0-21swapper/007:38:550
313332330,2chrt0-21swapper/209:13:422
313332330,2chrt0-21swapper/209:13:422
649599329,7cyclictest7189-21kernelversion07:10:202
649599322,12cyclictest27453-21apt-get09:35:122
6495993210,5cyclictest2458-21ls08:15:132
649399320,20cyclictest12716-21runrttasks08:38:530
649399320,20cyclictest0-21swapper/008:44:230
649599318,5cyclictest24296-21expr09:05:162
6495993113,15cyclictest30152-21sh12:00:322
6495993113,15cyclictest27291-21sh10:46:432
6495993113,15cyclictest27291-21sh10:46:432
6495993111,5cyclictest22831-21gzip11:55:232
649599308,4cyclictest11234-21tune2fs07:20:152
6495993013,3cyclictest23436-21sh10:44:492
6495993012,3cyclictest7930-21sh10:32:402
6495993012,3cyclictest16558-21sh09:26:132
6495993010,4cyclictest13793-21expr11:25:152
6495993010,4cyclictest0-21swapper/212:35:182
141332300,9sleep334-21ksoftirqd/312:13:343
649599297,4cyclictest3878-21basename12:30:132
649599297,3cyclictest0-21swapper/209:55:112
649599296,9cyclictest0-21swapper/212:15:362
649599291,13cyclictest32354-21ssh11:15:122
6495992911,3cyclictest11892-21sh11:47:382
6495992911,3cyclictest10469-21sh11:22:202
649599291,11cyclictest15235-21cut09:50:142
6495992910,5cyclictest12375-21grep08:35:272
649399295,21cyclictest10273-21grep08:30:270
204582290,9sleep228-21ksoftirqd/211:54:312
6495992811,3cyclictest5937-21sh11:43:242
6495992811,3cyclictest32317-21sh10:26:192
6495992811,3cyclictest22785-21sh12:20:122
649599281,11cyclictest19845-21cut11:05:232
6495992810,4cyclictest31819-21unixbench-2d08:05:252
6495992810,4cyclictest30346-21sh10:00:362
649599279,4cyclictest3717-21sh09:40:422
649599279,3cyclictest32000-21netstat10:50:212
6495992713,12cyclictest15165-21cpu07:30:132
6495992711,6cyclictest24013-21ntp_kernel_pll_10:20:182
6495992711,3cyclictest13648-21sh12:13:052
6495992710,4cyclictest11675-21latency_hist11:00:002
649599269,3cyclictest24558-21sh09:32:312
6495992610,3cyclictest12541-21ssh10:35:342
649699251,23cyclictest0-21swapper/307:11:493
649599252,10cyclictest13095-21apt-get08:40:142
649599251,9cyclictest2597-21apt-get12:05:152
649699242,16cyclictest6703-21ssh11:20:033
6496992413,7cyclictest25008-21seq11:33:303
649599242,7cyclictest21664-21apt-get09:00:112
70912230,5sleep37087-21fschecks_time08:25:163
70912230,5sleep37087-21fschecks_time08:25:163
649699232,7cyclictest32329-21apt-get11:15:113
649699232,15cyclictest25697-21diskmemload11:40:133
6496992315,5cyclictest14240-21sed07:25:243
649599221,10cyclictest23549-21apt-get07:50:012
649599220,6cyclictest11178-21ssh09:46:322
649699218,9cyclictest21524-1kworker/u9:010:25:143
649699211,9cyclictest11579-21turbostat.cron11:00:003
6496992113,6cyclictest20663-21python308:55:233
6496992112,7cyclictest3942-21smartctl12:05:253
6496992112,7cyclictest24287-21apt-get10:45:123
6496992112,7cyclictest24287-21apt-get10:45:123
6496992112,6cyclictest53050irq/142-enp3s0-09:10:143
6496992112,6cyclictest53050irq/142-enp3s0-09:10:143
6496992112,6cyclictest22234-21ls07:45:163
6495992113,6cyclictest16154-21ntp_states08:45:222
6495992112,6cyclictest1539-21unixbench-2d08:10:262
649699202,11cyclictest27465-21ssh09:35:123
6496992014,4cyclictest3531-21sh09:40:313
6496992014,4cyclictest30201-21sh12:00:353
6496992013,5cyclictest11008-21cstates07:20:133
6496992012,6cyclictest4442-21cut10:30:153
6496992012,5cyclictest26820-21grep11:10:253
6496992011,7cyclictest5051-21which12:30:243
6496992011,7cyclictest20516-21awk07:40:223
6496992010,7cyclictest10669-21apt-get10:35:123
649699200,9cyclictest257362sh09:57:443
6495992012,6cyclictest4919-21tune2fs08:20:162
6495992011,7cyclictest29401-21run-parts08:00:222
6495992011,6cyclictest21571-21date07:45:012
280252200,2sleep30-21swapper/311:35:223
649699197,7cyclictest53050irq/142-enp3s0-12:15:123
6496991914,3cyclictest23851-21cpu09:05:133
6496991914,3cyclictest17259-21sh11:51:173
6496991912,5cyclictest278-21systemd-journal08:55:003
6496991912,5cyclictest24884-21ntpq07:50:203
6496991912,4cyclictest18691-21/usr/sbin/munin11:05:263
6496991911,6cyclictest29216-21ntp_states08:00:203
6496991911,5cyclictest30422-21cat08:05:113
6496991910,7cyclictest560-21dbus-daemon12:30:003
6495991914,3cyclictest12186-21cpuspeed_turbos11:00:132
6495991910,6cyclictest17635-21grep07:35:162
172472190,1sleep30-21swapper/310:15:203
649699189,5cyclictest53050irq/142-enp3s0-09:25:133
649699182,11cyclictest21588-21apt-get11:55:133
649699181,8cyclictest30684-21ssh10:00:573
6496991813,3cyclictest9583-21acpi10:10:103
6496991813,3cyclictest13154-21sh12:36:173
6496991811,5cyclictest980-21needreboot08:10:203
6496991811,5cyclictest4209-21cat10:05:243
6496991811,5cyclictest18072-21sed07:35:203
6496991811,5cyclictest15382-21expr07:30:133
6496991810,6cyclictest24251-1kworker/u9:210:50:153
6496991810,5cyclictest11136-21cat08:35:153
649599182,11cyclictest19507-21apt-get08:55:122
6495991813,3cyclictest9699-21ntp_offset08:30:222
6495991813,3cyclictest28326-21switchtime11:35:242
6495991813,3cyclictest22006-21fschecks_count07:45:152
6495991811,5cyclictest13292-21expr07:25:142
649699179,6cyclictest4896-21perl08:20:153
649699178,6cyclictest2465-21cat08:15:123
6496991711,4cyclictest22298-21cat09:00:173
649699171,13cyclictest278-21systemd-journal08:35:003
649699171,11cyclictest15196-21grep11:25:273
6496991710,5cyclictest8780-21sed11:45:183
6496991710,5cyclictest34-21ksoftirqd/309:15:013
6496991710,5cyclictest27338-21cat07:55:243
6496991710,5cyclictest17166-21cut08:50:013
6496991710,4cyclictest9847-21switchtime07:15:233
6496991710,4cyclictest25697-21diskmemload11:00:123
6496991610,4cyclictest8597-21expr09:45:143
6495991610,4cyclictest7235-21idleruntime08:25:172
6495991610,4cyclictest7235-21idleruntime08:25:172
6496991511,2cyclictest13266-21cstates08:40:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional