You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Jul 27, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
681521840,2sleep20-21swapper/207:05:132
85332177142,25sleep10-21swapper/107:08:141
84382173151,15sleep30-21swapper/307:07:003
8532217297,24sleep00-21swapper/007:08:130
2968121510,1sleep20-21swapper/211:35:032
222272680,2sleep00-21swapper/009:29:210
109132680,7sleep0121ksoftirqd/011:45:130
26462670,3sleep0131rcuc/009:15:000
26462670,3sleep0131rcuc/009:15:000
206452600,1sleep30-21swapper/308:50:193
206692580,1sleep10-21swapper/111:51:481
200382570,2sleep30-21swapper/310:15:193
24992550,1sleep20-21swapper/209:14:542
24992550,1sleep20-21swapper/209:14:542
275752540,1sleep00-21swapper/010:45:130
29882530,3sleep20-21swapper/210:50:252
276192530,2sleep20-21swapper/212:20:382
101152510,2chrt0-21swapper/010:08:060
282192500,1chrt0-21swapper/211:57:272
306462460,1sleep10-21swapper/109:10:351
306462460,1sleep10-21swapper/109:10:351
153872450,1sleep10-21swapper/109:47:571
77462440,1sleep30-21swapper/312:05:553
277152440,2sleep10-21swapper/109:05:261
65822410,2chrt0-21swapper/311:17:323
18062410,2sleep30-21swapper/312:25:333
256572400,2chrt0-21swapper/312:20:113
251722400,2sleep20-21swapper/211:30:592
290642380,3sleep334-21ksoftirqd/311:34:523
184622380,1sleep30-21swapper/309:25:343
232122370,1sleep20-21swapper/207:40:242
894899362,13cyclictest24343-21apt-get11:55:043
8948993612,6cyclictest9183-21seq11:43:443
8948993513,5cyclictest9365-21ntp_kernel_pll_10:55:213
63572330,6sleep1141rcu_preempt10:53:291
196742320,3chrt0-21swapper/111:03:201
894899312,11cyclictest10003-21apt-get12:32:043
894699310,28cyclictest11568-21fschecks_time07:15:171
894899302,8cyclictest29193-21apt-get11:35:033
894899302,6cyclictest2370-21irqstats10:50:193
8946993011,5cyclictest22-21ksoftirqd/112:00:531
127052300,3chrt0-21swapper/008:30:260
8948992918,1cyclictest0-21swapper/307:16:503
8946992912,9cyclictest22-21ksoftirqd/110:55:211
252982290,12sleep012-21ksoftirqd/012:20:010
894599281,25cyclictest0-21swapper/007:16:160
132452280,1sleep00-21swapper/009:45:460
894799271,24cyclictest0-21swapper/207:15:172
248592270,2sleep012-21ksoftirqd/009:55:130
200792270,3sleep10-21swapper/112:15:241
894899265,6cyclictest0-21swapper/307:20:253
894699269,4cyclictest22-21ksoftirqd/109:42:031
894699269,4cyclictest22-21ksoftirqd/109:37:301
894699268,5cyclictest22-21ksoftirqd/110:07:221
894799252,16cyclictest5734-21ssh10:05:132
8946992519,4cyclictest22-21ksoftirqd/111:20:001
8946992510,2cyclictest141rcu_preempt10:17:161
288102250,11sleep122-21ksoftirqd/109:34:111
894799245,10cyclictest28-21ksoftirqd/211:05:012
894799241,11cyclictest36222gltestperf09:15:162
894699246,4cyclictest22-21ksoftirqd/110:10:591
894699246,4cyclictest22-21ksoftirqd/110:03:191
894699246,4cyclictest22-21ksoftirqd/109:28:031
894699241,8cyclictest22-21ksoftirqd/110:35:261
8946992418,4cyclictest22-21ksoftirqd/108:20:001
8946992416,6cyclictest21124-21ssh11:28:391
8946992416,6cyclictest21124-21ssh11:28:381
8945992413,6cyclictest3054-21ssh09:15:120
8945992410,9cyclictest52750irq/139-enp3s0-12:35:130
207102240,1sleep30-21swapper/307:35:243
148152241,4sleep214833-21sh09:23:422
8948992316,4cyclictest14155-21irqstats12:35:183
8948992315,6cyclictest19351-21ssh12:15:173
8947992316,5cyclictest28-21ksoftirqd/208:30:002
894799231,13cyclictest28-21ksoftirqd/210:25:442
894799231,11cyclictest25574-21ls09:55:192
894699236,9cyclictest141rcu_preempt11:31:141
894699235,4cyclictest22-21ksoftirqd/112:37:041
894699235,15cyclictest141rcu_preempt11:55:431
894699234,4cyclictest22-21ksoftirqd/110:32:061
894699234,10cyclictest22-21ksoftirqd/109:53:071
894699232,3cyclictest141rcu_preempt09:21:071
894699231,9cyclictest22-21ksoftirqd/112:14:551
894699231,9cyclictest22-21ksoftirqd/112:14:551
8946992314,6cyclictest560-21dbus-daemon11:50:001
164522230,1sleep00-21swapper/010:36:310
894899221,8cyclictest1481-21processes10:25:243
894899221,10cyclictest23454-21unixbench_singl11:05:263
8948992210,7cyclictest2618-21apt-get08:10:143
8947992216,5cyclictest28-21ksoftirqd/210:10:272
8947992214,5cyclictest5278-21cron10:05:002
894799221,13cyclictest28-21ksoftirqd/212:11:512
894799221,13cyclictest28-21ksoftirqd/212:11:512
8947992210,4cyclictest141rcu_preempt12:30:242
894699228,2cyclictest141rcu_preempt12:23:371
894699227,4cyclictest22-21ksoftirqd/111:09:121
894699226,3cyclictest141rcu_preempt12:05:061
894699225,2cyclictest141rcu_preempt10:25:441
894699223,3cyclictest141rcu_preempt12:30:241
8946992213,6cyclictest17987-21entropy08:45:181
8946992212,7cyclictest24445-21df_inode07:45:161
894699221,17cyclictest19776-21apt-get08:50:121
8946992211,2cyclictest141rcu_preempt10:49:541
894599229,8cyclictest30086-21ssh09:35:120
8945992213,7cyclictest923-21cut08:05:160
894599221,18cyclictest1048-21sed10:25:200
64152220,1sleep20-21swapper/211:40:572
253252220,2sleep00-21swapper/011:07:160
8948992116,3cyclictest13084-21sh08:35:003
8948992114,5cyclictest34-21ksoftirqd/310:45:013
8948992114,5cyclictest34-21ksoftirqd/310:45:003
8948992113,6cyclictest4387-21python309:15:233
8948992113,6cyclictest21125-21seq11:52:153
894899211,10cyclictest0-21swapper/309:30:153
894799211,9cyclictest283122diskmemload09:45:462
8947992114,5cyclictest28-21ksoftirqd/212:39:592
8947992112,7cyclictest19684-21cut08:50:012
8947992111,8cyclictest19413-21ssh11:03:052
894699212,4cyclictest22-21ksoftirqd/111:11:511
8946992119,1cyclictest0-21swapper/107:15:011
8946992115,4cyclictest22-21ksoftirqd/111:40:001
8946992115,4cyclictest22-21ksoftirqd/111:25:001
8946992114,5cyclictest22-21ksoftirqd/108:05:001
8946992113,6cyclictest29981-21seq07:56:271
8946992112,7cyclictest20376-21gdbus10:40:011
8946992112,7cyclictest20376-21gdbus10:40:011
894699211,18cyclictest640-21snmpd11:42:571
894599219,8cyclictest13074-21sed10:10:180
894599217,12cyclictest4795-21seq11:15:430
8945992112,6cyclictest5598-21seq12:29:230
292322210,1sleep20-21swapper/209:34:362
254942210,2chrt0-21swapper/109:55:181
894899209,6cyclictest52950irq/141-enp3s0-09:10:143
894899209,6cyclictest52950irq/141-enp3s0-09:10:133
894899207,4cyclictest0-21swapper/309:59:103
8948992015,3cyclictest27832-21unixbench_multi09:05:273
8948992012,6cyclictest5688-21cat10:05:113
8948992012,6cyclictest29369-21awk07:55:223
8948992012,6cyclictest2483-21cut08:10:003
8948992011,6cyclictest22603-21cat07:40:173
8948992011,6cyclictest14050-21cat08:35:183
8948992010,7cyclictest16211-21cat07:25:183
894899200,17cyclictest4865-21expr08:15:123
894799209,2cyclictest28-21ksoftirqd/211:00:002
894799209,2cyclictest28-21ksoftirqd/210:40:232
894799209,2cyclictest28-21ksoftirqd/210:40:222
8947992015,3cyclictest25887-21apt10:20:092
8947992014,4cyclictest28-21ksoftirqd/209:40:012
8947992012,6cyclictest10918-21date07:15:012
8947992012,6cyclictest1008-21ntpq12:25:222
8947992010,8cyclictest22836-21rm09:53:392
8947992010,8cyclictest12828-21ssh11:22:022
8947992010,7cyclictest52850irq/140-enp3s0-09:25:132
894799200,8cyclictest192391cpu10:15:132
894699201,9cyclictest326-21fschecks_count12:25:151
8946992014,4cyclictest22-21ksoftirqd/107:40:001
894699201,3cyclictest141rcu_preempt09:15:121
8946992013,4cyclictest1025-21sh08:05:161
8946992012,6cyclictest17070-21seq08:41:051
8946992011,6cyclictest19530-21cut07:35:001
8946992011,6cyclictest11521-21expr08:30:131
894599201,5cyclictest0-21swapper/011:31:410
8945992013,4cyclictest22763-21sh10:17:170
8945992012,6cyclictest18394-21ls07:30:200
8945992012,5cyclictest278-21systemd-journal07:30:000
8945992011,6cyclictest6273-21uname12:05:180
894599201,14cyclictest29844-21apt-get11:35:120
8945992010,6cyclictest52750irq/139-enp3s0-10:00:140
54252200,1sleep00-21swapper/009:40:210
292862200,2sleep30-21swapper/310:22:063
894899199,8cyclictest19945-21apt-get10:40:003
894899199,6cyclictest52950irq/141-enp3s0-10:30:123
894899192,11cyclictest15051-21ssh11:00:113
894899191,6cyclictest4821-21diskstats09:40:153
8948991912,5cyclictest9800-21sed08:25:183
8948991912,5cyclictest20752-21seq11:28:153
8948991912,5cyclictest20752-21seq11:28:153
8948991912,4cyclictest28312-21diskmemload10:14:413
8948991911,6cyclictest304-21cat12:00:243
8948991911,6cyclictest24750-21fw_packets09:00:183
8948991911,6cyclictest11996-21sed09:45:193
8948991911,5cyclictest28312-21diskmemload10:04:053
8948991911,5cyclictest18484-21grep08:45:223
8948991910,6cyclictest31621-21sed09:35:243
894799199,8cyclictest22385-21ssh11:29:552
894799199,8cyclictest22385-21ssh11:29:552
894799192,11cyclictest2621-21chrt08:10:142
894799191,8cyclictest0-21swapper/210:46:162
8947991914,3cyclictest7388-21sh12:05:342
8947991914,3cyclictest568-21sh12:00:292
8947991914,3cyclictest19024-21unixbench_singl07:30:272
8947991913,4cyclictest23847-21sh11:55:002
894799191,2cyclictest141rcu_preempt12:15:142
8947991912,5cyclictest27492-21ps09:05:242
8947991912,5cyclictest18950irq/143-ahci[0010:40:002
8947991911,6cyclictest6095-21grep09:40:262
8947991911,6cyclictest26826-21idleruntime07:50:152
8947991911,6cyclictest14047-21memory07:20:182
8947991911,5cyclictest28817-21cat11:10:152
894799191,11cyclictest10907-21sh10:32:422
8947991910,7cyclictest16214-21sed08:40:192
8947991910,7cyclictest14270-21open_files08:35:212
8947991910,7cyclictest11682-21sed11:45:182
8947991910,6cyclictest4108-21mailstats11:15:252
894699192,12cyclictest25888-21apt-get10:20:101
8946991911,6cyclictest25985-21seq09:02:491
8946991911,6cyclictest22300-21df07:40:141
8946991910,7cyclictest10566-21users08:25:261
8946991910,6cyclictest27929-21seq07:52:341
894599199,7cyclictest7045-21grep10:30:140
894599199,7cyclictest6913-21seq08:20:110
894599197,8cyclictest8986-21apt-get07:10:120
894599196,3cyclictest141rcu_preempt10:50:270
894599193,3cyclictest28312-21diskmemload12:20:050
8945991914,3cyclictest27924-21uptime09:05:270
8945991914,3cyclictest18669-21processes08:45:250
8945991913,4cyclictest17387-21idleruntime-cro08:45:000
8945991913,4cyclictest12268-21sh09:21:080
8945991912,5cyclictest29567-21seq11:10:210
8945991912,5cyclictest17056-21sed11:25:200
8945991912,5cyclictest17056-21sed11:25:190
8945991912,4cyclictest28312-21diskmemload10:41:160
8945991912,4cyclictest28312-21diskmemload10:41:160
8945991911,6cyclictest27184-21seq09:32:340
8945991911,6cyclictest21837-21cut07:40:000
8945991911,5cyclictest21306-21seq08:50:360
894599191,13cyclictest24554-21ssh11:55:170
894599191,12cyclictest22129-21taskset08:55:140
8945991910,7cyclictest24234-21expr07:45:150
8945991910,6cyclictest18787-21missed_timers09:50:170
166802190,2sleep20-21swapper/207:25:232
154022191,14sleep10-21swapper/107:25:121
894899189,6cyclictest34-21ksoftirqd/311:45:233
894899187,9cyclictest0-21swapper/309:50:153
8948991811,5cyclictest9434-21cut07:10:153
8948991811,5cyclictest52950irq/141-enp3s0-11:21:013
8948991811,5cyclictest25895-21seq07:48:533
8948991811,5cyclictest23315-21cat08:55:253
8948991811,5cyclictest15612-21grep08:40:143
894899181,12cyclictest9827-21apt-get09:20:123
8948991810,6cyclictest30797-21expr08:00:143
8948991810,5cyclictest18379-21cat07:30:193
894799189,6cyclictest30658-21expr08:00:132
8947991814,2cyclictest21985-21apt-config08:55:022
8947991814,2cyclictest13105-21idleruntime-cro08:35:002
8947991810,6cyclictest25202-21ntp_states09:00:232
8947991810,6cyclictest19840-21cat08:50:132
8946991811,5cyclictest22-21ksoftirqd/107:25:001
894599189,6cyclictest648-21ntpd08:15:190
894599189,6cyclictest5843-21grep11:40:260
894599181,7cyclictest27373-21awk07:50:210
8945991813,3cyclictest8972-21hddtemp_smartct10:55:180
8945991813,3cyclictest19589-21sh11:03:160
8945991813,3cyclictest12456-21cstates12:10:140
8945991813,3cyclictest12456-21cstates12:10:140
8945991812,4cyclictest28312-21diskmemload11:52:340
8945991812,3cyclictest28312-21diskmemload12:32:160
8945991811,5cyclictest30998-21cut08:00:150
8945991811,5cyclictest29111-21sed07:55:180
8945991811,4cyclictest9322-21sh11:20:020
8945991810,6cyclictest32110-21sed12:00:180
8945991810,6cyclictest26523-21which10:20:150
894599180,15cyclictest13521-21cat08:35:130
894899171,7cyclictest13202-21ssh12:10:213
894899171,7cyclictest13202-21ssh12:10:203
8948991712,3cyclictest32109-21sh11:12:213
8948991710,5cyclictest7187-21expr08:20:133
8948991710,5cyclictest27545-21expr10:45:133
8948991710,5cyclictest27045-21cat07:50:173
894799179,6cyclictest25517-21sed07:45:282
894799179,6cyclictest19853-21grep07:35:142
8947991711,4cyclictest28-21ksoftirqd/208:20:002
894799171,11cyclictest6916-21apt-get08:20:122
8947991710,5cyclictest29266-21ntp_kernel_err07:55:202
894699178,6cyclictest8898-21cat08:25:001
894699174,4cyclictest22-21ksoftirqd/108:55:021
8946991711,4cyclictest3709-21sendmail_mailst08:10:241
894699171,11cyclictest13342-21apt-get08:35:131
894599179,6cyclictest560-21dbus-daemon07:45:000
894599179,6cyclictest4323-21seq08:12:520
8945991712,3cyclictest10442-21unixbench-2d08:25:260
8945991710,4cyclictest13461-21cstates07:20:130
8947991611,3cyclictest2390-21latency_hist08:10:002
8945991611,3cyclictest24666-21fschecks_count09:00:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional