You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 19:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Jun 01, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
124782178104,29sleep10-21swapper/107:08:581
123612177106,64sleep30-21swapper/307:07:283
124892173102,28sleep20-21swapper/207:09:082
106542162108,30sleep00-21swapper/007:05:040
229721510,2sleep00-21swapper/010:44:060
110762990,3sleep327567-21kworker/3:010:50:153
312882710,2sleep10-21swapper/109:31:501
209842690,2sleep00-21swapper/010:34:220
240012680,1sleep00-21swapper/012:08:490
245712670,1sleep30-21swapper/308:50:203
165162620,2sleep20-21swapper/212:03:212
40382600,1sleep10-21swapper/112:17:201
24582600,1sleep20-21swapper/211:30:152
108702600,2sleep10-21swapper/110:26:241
171072570,11sleep228-21ksoftirqd/208:35:132
26402540,5sleep10-21swapper/111:53:291
26402540,5sleep10-21swapper/111:53:281
147542520,3sleep334-21ksoftirqd/308:30:013
276292500,2sleep00-21swapper/011:01:310
240712500,2sleep10-21swapper/111:45:181
112902500,3sleep012-21ksoftirqd/011:13:300
105162500,2sleep10-21swapper/112:21:391
12832994823,18cyclictest0-21swapper/009:59:580
61172470,1sleep20-21swapper/210:23:332
61172470,1sleep20-21swapper/210:23:322
77642460,1sleep20-21swapper/209:15:192
239732450,2chrt0-21swapper/311:22:123
124002450,1sleep10-21swapper/109:18:581
287972440,1sleep20-21swapper/210:15:592
226982420,1sleep10-21swapper/110:58:321
156732420,6chrt0-21swapper/007:15:180
58902390,1sleep10-21swapper/110:23:191
58902390,1sleep10-21swapper/110:23:191
105742340,2chrt0-21swapper/110:03:371
1283599331,29cyclictest31998-21diskmemload10:03:583
287612310,1sleep30-21swapper/312:35:133
1283599304,24cyclictest53050irq/142-enp3s0-09:56:143
12834993020,2cyclictest141rcu_preempt10:05:442
1283499301,26cyclictest16822-21ssh09:45:082
245092290,2chrt0-21swapper/110:13:321
1283599298,11cyclictest34-21ksoftirqd/311:45:213
1283599297,10cyclictest34-21ksoftirqd/309:18:153
12835992910,4cyclictest34-21ksoftirqd/310:57:493
12832992923,4cyclictest0-21swapper/009:15:020
12832992918,9cyclictest28571-21ssh09:30:160
1283299291,26cyclictest30968-21ssh10:18:010
12835992811,4cyclictest34-21ksoftirqd/309:48:133
12835992810,9cyclictest34-21ksoftirqd/309:52:083
12834992822,4cyclictest28295-21ssh09:52:592
1283499281,25cyclictest10213-21ssh10:03:152
12833992823,3cyclictest30727-21ssh09:55:051
12833992822,4cyclictest31526-21sh10:18:331
12833992822,4cyclictest26835-21ssh09:51:351
1283299281,25cyclictest26255-21sh09:51:030
1283599276,10cyclictest34-21ksoftirqd/311:03:243
12835992723,2cyclictest31998-21diskmemload10:15:513
12835992710,9cyclictest34-21ksoftirqd/311:07:273
12835992710,9cyclictest34-21ksoftirqd/311:07:263
12835992710,3cyclictest141rcu_preempt10:28:093
1283399271,24cyclictest0-21swapper/109:45:081
1283399271,24cyclictest0-21swapper/109:00:131
1283299271,25cyclictest0-21swapper/010:02:400
1283299271,24cyclictest0-21swapper/010:05:440
184582260,2sleep20-21swapper/211:18:382
157272260,2sleep10-21swapper/108:30:191
1283599268,10cyclictest34-21ksoftirqd/310:40:503
1283599266,10cyclictest34-21ksoftirqd/311:34:093
1283599264,2cyclictest310102grep09:05:203
12835992612,5cyclictest34-21ksoftirqd/309:40:183
1283599261,23cyclictest0-21swapper/310:06:153
1283499261,24cyclictest0-21swapper/209:56:082
1283299261,23cyclictest0-21swapper/009:00:270
1283299260,24cyclictest0-21swapper/009:45:060
35592250,2sleep00-21swapper/009:12:120
265172250,2sleep00-21swapper/011:24:380
259462250,3sleep00-21swapper/012:32:480
143462250,13sleep012-21ksoftirqd/011:15:200
1283599258,5cyclictest34-21ksoftirqd/312:04:373
1283599258,5cyclictest34-21ksoftirqd/311:17:003
1283599258,5cyclictest34-21ksoftirqd/309:26:023
1283599257,10cyclictest34-21ksoftirqd/311:10:243
12835992512,4cyclictest34-21ksoftirqd/311:40:063
1283599251,23cyclictest0-21swapper/310:14:463
1283599251,23cyclictest0-21swapper/309:00:343
12835992511,8cyclictest34-21ksoftirqd/311:59:113
1283499251,2cyclictest0-21swapper/209:44:422
1283499251,22cyclictest0-21swapper/208:48:482
1283499250,23cyclictest0-21swapper/209:01:152
1283399251,1cyclictest0-21swapper/110:06:201
1283399250,1cyclictest0-21swapper/109:14:501
1283299250,24cyclictest0-21swapper/009:44:410
253712240,1sleep20-21swapper/211:45:552
140712240,1sleep20-21swapper/209:20:112
129072240,5sleep1221ksoftirqd/108:25:141
1283599249,2cyclictest141rcu_preempt11:28:063
1283599248,9cyclictest34-21ksoftirqd/311:50:523
1283599248,9cyclictest34-21ksoftirqd/311:50:513
1283599248,4cyclictest34-21ksoftirqd/309:13:083
1283599248,2cyclictest141rcu_preempt12:07:463
1283599247,4cyclictest34-21ksoftirqd/310:39:453
1283599246,8cyclictest34-21ksoftirqd/310:20:303
1283599246,8cyclictest34-21ksoftirqd/310:20:293
1283599245,8cyclictest34-21ksoftirqd/310:32:183
1283599242,15cyclictest8439-21taskset08:15:133
12835992418,4cyclictest34-21ksoftirqd/311:40:003
12835992410,4cyclictest34-21ksoftirqd/312:34:193
1283599239,4cyclictest34-21ksoftirqd/312:12:173
1283599238,4cyclictest34-21ksoftirqd/309:23:163
1283599237,4cyclictest34-21ksoftirqd/312:26:033
12835992313,6cyclictest53050irq/142-enp3s0-10:45:103
1283599231,10cyclictest34-21ksoftirqd/307:20:203
1283499231,2cyclictest141rcu_preempt08:05:012
1283399234,16cyclictest3270-21sh10:45:001
12833992321,1cyclictest0-21swapper/110:33:091
12833992321,1cyclictest0-21swapper/108:48:551
1283399231,11cyclictest4685-21apt-get11:55:121
1283299239,3cyclictest185682sh09:23:050
1283299238,3cyclictest53422sh12:18:360
12832992321,1cyclictest0-21swapper/009:00:000
1283299231,6cyclictest0-21swapper/011:34:540
1283299231,13cyclictest110732diskstats10:50:150
1283599229,4cyclictest34-21ksoftirqd/312:21:503
1283599229,3cyclictest141rcu_preempt09:33:303
12835992220,1cyclictest0-21swapper/309:00:003
12835992216,4cyclictest34-21ksoftirqd/312:20:013
12835992215,5cyclictest34-21ksoftirqd/307:40:003
12835992214,5cyclictest278-21systemd-journal07:35:003
1283599221,12cyclictest34-21ksoftirqd/309:39:573
1283599221,12cyclictest34-21ksoftirqd/309:39:563
12835992211,2cyclictest141rcu_preempt08:39:393
1283499225,8cyclictest0-21swapper/210:36:252
1283499221,8cyclictest157312head08:30:202
12834992213,7cyclictest4987-21ls08:05:222
12834992212,7cyclictest348-21forks09:10:142
12834992212,7cyclictest18702-21apt-get12:05:132
12833992220,1cyclictest0-21swapper/109:00:011
12833992218,2cyclictest0-21swapper/109:44:391
12832992213,6cyclictest14750-21cron07:15:000
1283599212,9cyclictest34-21ksoftirqd/307:10:253
12835992116,4cyclictest34-21ksoftirqd/308:05:003
12834992114,5cyclictest27131-21sh09:29:342
12834992113,6cyclictest14052-21ls08:25:242
12834992112,7cyclictest15621-21pluginstate12:25:222
12834992110,7cyclictest8090-21apt-get10:25:122
1283399212,13cyclictest14531-21ssh12:25:121
12833992114,5cyclictest11146-21ssh11:35:431
12833992113,6cyclictest31845-21seq09:08:221
12833992113,6cyclictest27105-21ls11:25:001
12833992112,7cyclictest9765-21cat08:15:261
1283299213,5cyclictest0-21swapper/012:10:050
12832992116,3cyclictest12-21ksoftirqd/010:59:590
12832992115,4cyclictest12-21ksoftirqd/011:55:010
12832992115,4cyclictest12-21ksoftirqd/011:55:000
12832992112,7cyclictest23133-21seq08:46:350
12832992112,7cyclictest18867-21seq07:22:360
12832992112,6cyclictest21885-21seq07:30:150
1283599206,2cyclictest141rcu_preempt07:50:153
12835992015,4cyclictest34-21ksoftirqd/308:00:013
12835992013,5cyclictest18950irq/143-ahci[0008:34:593
12835992012,6cyclictest7478-21cat08:10:233
12835992010,7cyclictest19643-21perl08:40:153
12834992015,3cyclictest271rcuc/207:45:232
12834992013,5cyclictest23116-21ls12:30:222
12834992013,5cyclictest12874-21ls11:15:012
12834992012,6cyclictest5104-21awk11:55:162
12834992012,6cyclictest4542-21date10:45:172
12834992012,6cyclictest32135-21ssh11:28:202
12834992012,6cyclictest31910-21seq09:32:252
12834992012,6cyclictest26439-21interrupts08:55:182
12834992012,6cyclictest23998-21seq10:13:032
12834992012,6cyclictest15725-21expr10:30:182
12834992012,6cyclictest10527-21ls08:20:002
12834992012,4cyclictest52950irq/141-enp3s0-10:56:232
12834992011,6cyclictest9603-21fw_packets11:35:132
12834992011,6cyclictest15711-21grep07:15:192
12833992012,6cyclictest23302-21seq09:25:551
12833992012,6cyclictest22924-21grep07:30:261
12833992012,6cyclictest16374-21expr11:40:141
12833992012,6cyclictest12067-21seq12:00:151
1283299204,3cyclictest31998-21diskmemload10:36:530
1283299202,12cyclictest19466-21rm10:10:100
12832992015,3cyclictest24728-21processes08:50:220
12832992014,4cyclictest21197-21cron07:30:010
12832992014,4cyclictest12-21ksoftirqd/012:05:000
12832992013,4cyclictest15648-21sh11:40:000
12832992012,6cyclictest32715-21perl07:55:150
12832992012,6cyclictest29771-21seq07:48:410
12832992012,6cyclictest23921-21cat11:45:170
12832992012,6cyclictest19224-21seq11:41:440
12832992012,5cyclictest8750-21processes12:20:220
3992190,3sleep122-21ksoftirqd/107:55:161
1283599199,2cyclictest141rcu_preempt08:45:133
12835991911,6cyclictest4294-21expr08:05:173
12835991911,5cyclictest16440-21grep07:15:253
12835991910,2cyclictest141rcu_preempt08:21:043
1283499198,7cyclictest52950irq/141-enp3s0-11:00:132
1283499196,11cyclictest7154-21ssh09:38:042
1283499196,11cyclictest7154-21ssh09:38:032
1283499194,3cyclictest141rcu_preempt08:55:002
1283499191,8cyclictest141rcu_preempt07:40:242
1283499191,4cyclictest0-21swapper/212:15:552
1283499191,3cyclictest141rcu_preempt11:20:132
12834991913,4cyclictest3846-21sh11:08:042
12834991913,4cyclictest3846-21sh11:08:042
12834991913,4cyclictest29007-21sh10:40:072
1283499191,2cyclictest141rcu_preempt08:40:162
1283499191,2cyclictest141rcu_preempt07:55:112
12834991912,5cyclictest13006-21expr07:10:132
12834991912,5cyclictest10782-21expr08:20:142
12834991911,6cyclictest7622-21cat08:10:252
12834991911,6cyclictest30479-21diskstats07:50:152
12834991911,6cyclictest18605-21seq11:41:082
12834991911,5cyclictest52950irq/141-enp3s0-12:35:112
12834991911,5cyclictest31998-21diskmemload10:51:532
12834991910,6cyclictest19061-21cron07:25:002
1283399192,12cyclictest27338-21sh11:25:111
1283399191,7cyclictest254132sort11:00:171
12833991914,3cyclictest14420-21open_inodes11:15:201
12833991913,4cyclictest23204-21timerwakeupswit12:30:241
12833991911,6cyclictest6258-21date08:10:001
12833991911,6cyclictest29762-21seq12:12:391
12833991910,7cyclictest15541-21sed07:15:161
1283299198,9cyclictest12-21ksoftirqd/009:40:000
1283299198,9cyclictest12-21ksoftirqd/009:40:000
1283299198,8cyclictest2350-21df_inode08:00:130
1283299192,4cyclictest14889-21diskstats12:25:160
12832991913,4cyclictest12464-21sh10:27:560
12832991912,5cyclictest20582-21grep08:40:250
12832991911,6cyclictest21327-21ls09:25:140
12832991911,6cyclictest14501-21seq08:28:330
12832991911,4cyclictest1252-21tr10:20:140
12832991911,4cyclictest1252-21tr10:20:130
12832991910,7cyclictest15627-21date08:30:170
12832991910,6cyclictest9205-21missed_timers08:15:190
11982190,2chrt22-21ksoftirqd/112:38:271
12835991810,2cyclictest141rcu_preempt07:46:073
1283599180,9cyclictest27385-21chrt07:40:333
1283499187,9cyclictest52950irq/141-enp3s0-12:20:092
1283499182,11cyclictest19320-21apt-get07:25:132
12834991813,3cyclictest31007-21ntp_offset09:05:212
12834991813,3cyclictest22958-21uptime07:30:252
12834991811,5cyclictest30991-21sed11:50:182
12834991811,5cyclictest30991-21sed11:50:172
12834991811,5cyclictest23852-21expr07:35:142
1283399189,6cyclictest2780-21memory08:00:171
1283399189,6cyclictest11178-21idleruntime08:20:161
1283399182,11cyclictest25859-21apt-get07:40:131
1283399182,11cyclictest10678-21apt-get10:50:111
12833991813,3cyclictest9762-21sh11:12:041
12833991813,3cyclictest17841-21kernelversion07:20:171
12833991812,4cyclictest5276-21sh11:31:461
12833991812,4cyclictest20859-21sh12:05:491
12833991812,4cyclictest20236-21sh07:25:201
12833991811,5cyclictest23849-21cat07:35:141
12833991811,5cyclictest23655-21basename08:50:101
12833991811,5cyclictest19428-21expr08:40:121
12833991811,5cyclictest15269-21run-parts09:20:221
12833991810,6cyclictest7219-21uname08:10:201
12833991810,6cyclictest3948-21expr10:45:131
12833991810,6cyclictest28311-21ls07:45:131
12833991810,6cyclictest1691-21seq11:06:011
12833991810,6cyclictest1691-21seq11:06:001
12833991810,6cyclictest13338-21fw_packets07:10:151
12833991810,5cyclictest30836-21latency07:50:191
1283299189,6cyclictest17151-21grep08:35:130
1283299186,3cyclictest141rcu_preempt08:10:150
1283299181,5cyclictest0-21swapper/011:05:410
1283299181,5cyclictest0-21swapper/011:05:400
1283299181,3cyclictest31998-21diskmemload12:37:350
12832991813,3cyclictest4138-21/usr/sbin/munin08:05:250
12832991810,6cyclictest4786-21cat11:55:140
12832991810,6cyclictest28039-21cat11:25:180
12832991810,6cyclictest25209-21seq07:36:420
1283299180,16cyclictest30314-21cstates07:50:130
12835991710,5cyclictest20107-21sed07:25:183
12834991711,4cyclictest29891-21sh12:12:452
1283399179,6cyclictest17366-21entropy08:35:161
12833991711,3cyclictest31998-21diskmemload09:36:141
12833991711,3cyclictest31998-21diskmemload09:36:131
12833991710,5cyclictest22390-21ls10:35:151
12832991712,3cyclictest31993-21sh09:10:000
12832991712,3cyclictest25833-21/usr/sbin/munin07:40:110
12832991710,5cyclictest4077-21diskstats10:45:140
12832991710,5cyclictest12480-21cron08:25:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional