You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 09:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu May 07, 2026 00:45:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
299082192154,15sleep20-21swapper/219:07:062
299492166109,29sleep30-21swapper/319:07:383
300812162111,28sleep10-21swapper/119:09:211
299512159107,28sleep00-21swapper/019:07:390
691121370,2sleep10-21swapper/120:45:151
467421230,2sleep10-21swapper/123:48:171
319952690,1sleep30-21swapper/322:32:063
243912640,1sleep20-21swapper/222:26:152
279972630,3sleep20-21swapper/221:17:132
43132610,1sleep10-21swapper/100:11:161
169592590,2sleep0131rcuc/023:57:040
43622580,4sleep0131rcuc/000:35:190
306572570,2sleep20-21swapper/222:07:142
307622560,2sleep00-21swapper/021:43:350
144812540,1sleep10-21swapper/100:19:361
97122510,3sleep20-21swapper/219:35:212
225692500,4sleep022567-21irqstats22:25:170
287602490,4sleep20-21swapper/222:30:132
212362490,1sleep10-21swapper/122:48:371
158242490,2sleep30-21swapper/319:50:143
40452480,3sleep14044-21ssh00:35:161
258652480,3sleep3331rcuc/323:40:123
36742470,2sleep10-21swapper/119:20:231
206632470,2sleep122-21ksoftirqd/100:00:131
12242460,8sleep012-21ksoftirqd/022:57:350
47002440,3chrt4683-21ssh00:35:223
3040699341,29cyclictest6141-21ssh21:49:441
30408993217,10cyclictest5621-21df_inode23:25:153
312282310,4sleep0121ksoftirqd/020:25:180
25772310,2sleep20-21swapper/223:22:452
47882300,2sleep00-21swapper/021:48:210
30407993019,9cyclictest2463-21ssh21:22:202
30407992922,5cyclictest5590-21ssh21:49:092
89592280,2sleep012-21ksoftirqd/023:50:490
3040599281,25cyclictest0-21swapper/019:23:120
135682280,2sleep0131rcuc/022:42:390
3040899271,3cyclictest6082-21ssh21:49:403
30406992714,10cyclictest12516-21sed23:30:171
3040699271,2cyclictest17254-21diskmemload21:51:351
3040699271,24cyclictest0-21swapper/119:17:541
3040599271,24cyclictest0-21swapper/021:50:410
100752270,6sleep2141rcu_preempt20:50:242
3040799261,2cyclictest0-21swapper/219:22:482
3040699261,24cyclictest0-21swapper/121:44:511
3040599260,24cyclictest0-21swapper/019:17:560
30372260,1sleep30-21swapper/300:10:223
3040899256,9cyclictest34-21ksoftirqd/322:51:003
3040899251,1cyclictest0-21swapper/321:50:363
3040899251,1cyclictest0-21swapper/319:20:043
3040899251,1cyclictest0-21swapper/319:18:003
30407992521,2cyclictest14610-21strings21:55:212
3040799251,2cyclictest0-21swapper/219:17:082
30407992512,9cyclictest28-21ksoftirqd/223:35:112
3040799251,23cyclictest0-21swapper/221:50:402
305312240,2sleep00-21swapper/022:30:390
3040899243,15cyclictest27133-21ssh22:05:103
30408992422,1cyclictest0-21swapper/321:55:163
3040899241,1cyclictest0-21swapper/321:44:573
30407992421,2cyclictest0-21swapper/221:44:382
30407992411,9cyclictest22550-21uniq00:25:152
30406992420,2cyclictest0-21swapper/121:55:541
30405992415,7cyclictest13023-21irqstats21:30:150
30405992414,7cyclictest15685-21df21:05:150
314572230,2sleep20-21swapper/223:44:242
3040899239,12cyclictest25753-21mailstats21:15:243
3040899234,10cyclictest34-21ksoftirqd/322:43:193
3040799232,2cyclictest141rcu_preempt22:35:112
3040699232,5cyclictest640-21snmpd23:05:401
3040699232,15cyclictest15383-21ssh00:20:131
30405992321,1cyclictest0-21swapper/021:56:320
30405992310,11cyclictest12718-21ls23:30:180
174172230,2sleep10-21swapper/122:21:001
138962230,2sleep00-21swapper/019:45:190
3040899229,9cyclictest560-21dbus-daemon22:15:233
3040899226,8cyclictest141rcu_preempt23:17:193
3040899222,14cyclictest1437-21ssh22:10:113
3040899222,13cyclictest2907-21runrttasks22:35:123
3040899222,12cyclictest141rcu_preempt00:29:593
30408992213,2cyclictest238501ssh23:15:003
30408992213,2cyclictest238501ssh23:15:003
30408992212,6cyclictest52950irq/141-enp3s0-23:30:133
30408992212,6cyclictest15043-21timerandwakeup23:55:253
30408992211,2cyclictest141rcu_preempt21:27:073
30408992210,2cyclictest141rcu_preempt22:25:243
30407992214,6cyclictest278-21systemd-journal19:35:002
30407992213,7cyclictest28709-21perl20:20:142
30407992212,7cyclictest4476-21apt-get19:25:132
30407992212,7cyclictest31308-21ntp_kernel_pll_20:25:192
30407992212,7cyclictest13426-21tune2fs19:45:142
3040799221,18cyclictest16917-21tail22:45:162
30407992210,7cyclictest53050irq/142-enp3s0-23:00:122
30407992210,10cyclictest23205-21ssh23:14:252
30407992210,10cyclictest23205-21ssh23:14:252
3040699229,8cyclictest52850irq/140-enp3s0-00:05:121
3040699221,6cyclictest17254-21diskmemload22:09:081
30406992213,7cyclictest8327-21seq23:02:581
30405992212,7cyclictest18079-21cat21:10:150
30405992211,3cyclictest141rcu_preempt23:41:490
280982220,3chrt12-21ksoftirqd/022:53:470
257102220,1sleep10-21swapper/120:15:011
186992220,2sleep00-21swapper/023:35:110
140422220,2sleep20-21swapper/221:30:262
3040899218,3cyclictest141rcu_preempt21:34:013
3040899215,7cyclictest141rcu_preempt23:38:073
3040899213,5cyclictest34-21ksoftirqd/322:46:513
30408992113,6cyclictest24681-21ntp_states20:10:203
30408992110,3cyclictest141rcu_preempt20:35:113
3040799219,9cyclictest560-21dbus-daemon00:15:222
3040799219,9cyclictest30647-21kworker/u8:122:15:112
30407992113,6cyclictest16116-21ntpq22:20:192
30407992112,6cyclictest16627-21readlink00:20:222
30407992111,8cyclictest20199-21head21:35:212
3040699214,3cyclictest17254-21diskmemload22:38:221
3040699214,3cyclictest17254-21diskmemload21:20:041
3040699211,6cyclictest17254-21diskmemload21:12:331
3040699211,5cyclictest14656-21munin-run22:20:001
30406992112,7cyclictest2113-21cut20:35:001
30406992112,6cyclictest13620-21sed19:45:151
30406992111,7cyclictest23304-21grep22:25:251
30405992113,6cyclictest6697-21awk21:25:220
30405992113,6cyclictest6111-21ls23:25:200
30405992113,6cyclictest28620-21ls22:05:230
30405992113,6cyclictest2618-21diskstats20:35:150
30405992112,7cyclictest17188-21memory22:45:180
180402210,1sleep00-21swapper/019:55:170
3040899209,3cyclictest141rcu_preempt21:20:273
3040899207,11cyclictest278-21systemd-journal22:25:003
3040899206,3cyclictest141rcu_preempt00:09:593
30408992014,4cyclictest6392-21cron23:49:593
30408992014,4cyclictest34-21ksoftirqd/321:00:003
3040899201,3cyclictest141rcu_preempt23:50:543
30408992013,5cyclictest20383-21ntpq20:00:183
30408992013,5cyclictest20383-21ntpq20:00:183
3040899201,16cyclictest10814-21ssh23:05:113
30408992010,8cyclictest4007-21ssh23:00:083
30408992010,3cyclictest141rcu_preempt23:23:543
3040799209,9cyclictest8466-21ls20:50:002
3040799201,9cyclictest0-21swapper/223:51:552
3040799201,9cyclictest0-21swapper/200:05:132
3040799201,8cyclictest11608-21chrt22:40:402
3040799201,7cyclictest5941-21diskstats21:25:142
3040799201,7cyclictest2758-21ntp_states00:10:192
3040799201,7cyclictest19256-21ls21:10:242
30407992016,2cyclictest143052sleep223:31:042
30407992014,3cyclictest17254-21diskmemload23:25:142
3040799201,3cyclictest141rcu_preempt22:15:002
30407992013,4cyclictest23003-21unixbench-2d20:05:262
30407992012,6cyclictest560-21dbus-daemon21:00:012
30407992012,6cyclictest16287-21ntpq21:05:212
30407992012,6cyclictest15760-21sed19:50:152
3040799201,10cyclictest141rcu_preempt00:31:292
3040699204,3cyclictest17254-21diskmemload22:31:431
3040699204,3cyclictest17254-21diskmemload21:16:161
3040699202,7cyclictest10940-21python322:40:221
3040699201,6cyclictest5613-21df23:25:151
30406992012,6cyclictest29707-21awk20:20:241
30406992012,5cyclictest27142-21timerandwakeup23:40:241
30406992011,7cyclictest15299-21expr19:50:101
30406992011,4cyclictest52850irq/140-enp3s0-23:50:151
3040599209,3cyclictest141rcu_preempt23:04:110
3040599204,3cyclictest66050lldpd21:16:030
30405992013,5cyclictest27767-21seq20:19:540
30405992012,6cyclictest9575-21date19:35:180
30405992012,6cyclictest9500-21grep22:15:220
30405992012,6cyclictest25578-21seq20:14:150
30405992012,6cyclictest21301-21seq20:04:160
30405992012,6cyclictest21301-21seq20:04:160
30405992012,6cyclictest1694-21seq20:30:480
30405992011,6cyclictest15101-21cat00:20:010
30405992010,3cyclictest141rcu_preempt22:38:140
30405992010,3cyclictest141rcu_preempt00:22:030
281352200,2sleep20-21swapper/222:53:482
3040899196,3cyclictest141rcu_preempt21:38:093
3040899193,5cyclictest34-21ksoftirqd/321:10:013
3040899193,3cyclictest141rcu_preempt00:15:103
3040899192,12cyclictest19904-21apt-get00:00:033
30408991914,4cyclictest34-21ksoftirqd/321:10:003
30408991912,5cyclictest1385-21seq00:33:313
30408991911,6cyclictest2100-21ls20:35:003
30408991911,6cyclictest13223-21apt21:00:123
30408991911,5cyclictest17842-21cut19:55:163
3040899191,10cyclictest0-21swapper/322:55:093
30408991910,6cyclictest12244-21cat19:40:233
3040799191,9cyclictest0-21swapper/223:17:522
3040799191,8cyclictest0-21swapper/223:45:012
3040799191,6cyclictest2638-21cut20:35:152
30407991914,3cyclictest53050irq/142-enp3s0-00:36:162
30407991913,4cyclictest28-21ksoftirqd/219:15:002
30407991913,4cyclictest28-21ksoftirqd/219:15:002
30407991912,5cyclictest21648-21gzip22:00:232
30407991911,6cyclictest24502-21cat20:10:172
30407991911,6cyclictest19655-21cat20:00:132
30407991911,6cyclictest19655-21cat20:00:122
30407991910,6cyclictest31216-21grep22:55:252
3040699199,7cyclictest15214-21grep21:05:011
3040699198,7cyclictest52850irq/140-enp3s0-21:25:111
3040699193,3cyclictest17254-21diskmemload22:56:371
3040699193,2cyclictest141rcu_preempt22:51:371
3040699192,12cyclictest30421-21apt-get20:25:121
3040699191,8cyclictest23123-21netstat00:25:191
30406991913,4cyclictest30569-21grep00:30:251
30406991913,4cyclictest22-21ksoftirqd/123:20:011
30406991913,3cyclictest636-21runrttasks22:01:361
30406991912,5cyclictest1489-21seq23:21:391
30406991912,4cyclictest13944-21sh23:55:131
30406991911,6cyclictest26630-21sed20:15:181
30406991910,6cyclictest19516-21cat23:35:171
3040599199,7cyclictest1694-21apt-get00:10:120
3040599199,3cyclictest141rcu_preempt23:11:130
3040599199,3cyclictest141rcu_preempt23:11:120
3040599192,12cyclictest10915-21apt-get19:40:120
30405991913,4cyclictest31178-21sh21:20:090
30405991913,4cyclictest24338-21sh00:02:270
30405991912,5cyclictest6744-21cat20:45:120
30405991912,5cyclictest29613-21basename00:30:170
30405991912,5cyclictest12380-21seq20:55:320
30405991911,6cyclictest30411-21seq19:10:030
30405991911,6cyclictest30411-21seq19:10:030
30405991911,6cyclictest26850-21seq00:28:210
30405991911,6cyclictest11629-21ls23:05:180
30405991910,6cyclictest8109-21uptime19:30:250
3040899189,6cyclictest29718-21grep20:20:243
3040899183,2cyclictest141rcu_preempt00:20:033
30408991810,6cyclictest7245-21sed20:45:173
30408991810,6cyclictest7044-21tune2fs19:30:153
30408991810,6cyclictest5431-21grep20:40:203
30408991810,6cyclictest10119-21unixbench_multi20:50:263
3040799185,10cyclictest28-21ksoftirqd/223:05:132
3040799182,11cyclictest13598-21apt-get23:55:132
30407991811,5cyclictest5181-21sed20:40:172
30407991811,4cyclictest53050irq/142-enp3s0-00:00:132
30407991810,6cyclictest757-21if_err_enp3s020:30:172
30407991810,6cyclictest11475-21sh19:40:152
3040699188,7cyclictest278-21systemd-journal21:35:001
3040699182,11cyclictest17328-21apt-get19:55:121
3040699182,10cyclictest515-21irqbalance19:30:131
3040699182,10cyclictest17254-21diskmemload21:35:111
3040699181,7cyclictest4197-21perf20:40:001
30406991813,3cyclictest20288-21sh23:11:301
30406991813,3cyclictest20288-21sh23:11:291
30406991812,4cyclictest5548-21rt-features19:25:221
30406991810,6cyclictest9530-21sed19:35:181
30406991810,6cyclictest2099-21ls22:10:161
30406991810,6cyclictest20268-21cat20:00:181
30406991810,6cyclictest20268-21cat20:00:171
30406991810,5cyclictest5510-21awk20:40:201
3040599189,6cyclictest29519-21wc20:20:230
3040599181,5cyclictest31599-21/usr/sbin/munin23:20:170
30405991813,3cyclictest9955-21threads20:50:250
30405991813,3cyclictest7123-21sh22:14:240
30405991813,3cyclictest22564-21sh21:37:110
30405991813,3cyclictest17633-21sh22:21:130
30405991811,5cyclictest4721-21cat20:40:120
30405991811,5cyclictest27645-21expr00:05:120
3040599181,12cyclictest20417-21ssh22:00:130
30405991810,6cyclictest32684-21expr23:45:130
30405991810,6cyclictest24159-21seq23:15:090
3040899179,6cyclictest21744-21expr20:05:133
3040899171,12cyclictest13054-21apt-get19:45:123
30408991710,5cyclictest24337-21seq22:02:373
30407991711,4cyclictest12792-21apt-key21:00:012
30407991710,5cyclictest26602-21sed20:15:182
3040699179,6cyclictest11357-21grep20:55:151
30406991712,3cyclictest16458-21mailstats21:05:221
30406991710,4cyclictest12791-21cron19:44:591
30405991712,3cyclictest13887-21kernelversion21:00:190
30405991711,4cyclictest4738-21expr19:25:140
3040599171,12cyclictest21625-21apt-get20:05:130
3040899169,5cyclictest4942-21gltestperf19:25:163
3040899169,5cyclictest30654-21expr20:25:143
3040899169,5cyclictest26109-21cat20:15:153
3040899167,2cyclictest34-21ksoftirqd/319:35:173
30407991611,3cyclictest19219-21sh20:00:002
3040699168,6cyclictest10024-21ls20:50:241
30406991611,3cyclictest636-21runrttasks19:10:081
30406991611,3cyclictest636-21runrttasks19:10:081
30406991611,3cyclictest22229-21if_enp3s020:05:171
3040899158,5cyclictest31119-21cut19:10:173
3040899158,5cyclictest31119-21cut19:10:173
3040599153,7cyclictest15187-21apt-get19:50:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional