You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 04:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue May 12, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
233552209170,32sleep30-21swapper/319:07:063
23346216795,49sleep10-21swapper/119:07:001
23421214597,22sleep20-21swapper/219:07:582
23226214299,35sleep00-21swapper/019:05:270
170022860,3sleep10-21swapper/100:04:181
221042750,2sleep00-21swapper/000:31:550
221042750,2sleep00-21swapper/000:31:540
245722630,2sleep30-21swapper/300:10:023
176662630,1sleep30-21swapper/321:14:543
219532610,1sleep10-21swapper/122:05:241
68572600,2sleep20-21swapper/222:19:252
152212600,3sleep1211rcuc/122:25:181
251022560,2sleep10-21swapper/123:20:481
151972540,1sleep20-21swapper/221:12:242
53012520,2chrt0-21swapper/221:29:532
28102520,1sleep20-21swapper/221:51:002
34102510,2sleep30-21swapper/321:51:373
23854995012,21cyclictest17485-21sh00:28:583
106692490,1sleep10-21swapper/122:21:291
2385499436,4cyclictest11103-21diskmemload23:16:293
23854994116,22cyclictest16561-21ntp_states22:50:253
2385499399,18cyclictest636-21runrttasks19:38:433
2385499375,7cyclictest34-21ksoftirqd/321:25:243
2385499370,9cyclictest1512-21ssh23:28:063
2385499363,28cyclictest22341-21taskset20:20:123
2385499363,27cyclictest2191-21ssh22:40:133
23854993612,22cyclictest9061-21cut21:05:003
2385499360,6cyclictest29386-21ssh22:35:253
163582360,2sleep00-21swapper/000:27:500
2385499352,3cyclictest0-21swapper/321:35:123
2385499352,3cyclictest0-21swapper/321:35:123
2385499352,11cyclictest32652-21taskset19:30:043
23854993510,22cyclictest1350-21sed20:45:183
2385499350,6cyclictest27028-21ssh22:34:333
2385499349,22cyclictest560-21dbus-daemon22:20:003
2385499343,5cyclictest29067-21expr23:00:153
23854993410,21cyclictest26730-21cat21:45:173
2385499340,5cyclictest4135-21ssh23:30:133
2385499340,5cyclictest18316-21rm22:03:173
2385499339,21cyclictest25676-21sed21:20:183
2385499338,6cyclictest10002-21sed21:05:193
23854993312,18cyclictest31777-21sendmail00:40:003
2385499330,7cyclictest17753-21cut20:10:143
2385499329,20cyclictest5186-21cut19:40:153
2385499322,3cyclictest0-21swapper/300:30:133
2385499322,3cyclictest0-21swapper/300:30:133
2385499321,5cyclictest6677-21ntpq00:20:213
2385499321,28cyclictest348-21munin-run20:45:003
23854993212,18cyclictest21438-21ntpq22:05:193
23854993210,19cyclictest11103-21diskmemload23:44:403
23854993210,18cyclictest11103-21diskmemload23:50:113
21052320,1sleep00-21swapper/021:26:380
97392310,3sleep334-21ksoftirqd/321:32:393
2385499319,19cyclictest25398-21unixbench_multi23:45:263
2385499318,20cyclictest3862-21missed_timers23:05:203
2385499318,20cyclictest3862-21missed_timers23:05:193
2385499318,20cyclictest18967-21kernelversion21:15:183
2385499318,20cyclictest12434-21timerandwakeup19:55:253
2385499318,20cyclictest11103-21diskmemload00:03:113
2385499317,5cyclictest5384-21gltestperf20:55:163
2385499317,21cyclictest8108-21needreboot21:55:193
2385499315,23cyclictest0-21swapper/323:21:023
23854993110,18cyclictest11103-21diskmemload00:19:563
23854993110,18cyclictest11103-21diskmemload00:09:453
23854993110,17cyclictest20251-21sh21:40:233
123932310,2sleep30-21swapper/322:47:193
111352310,2chrt0-21swapper/021:34:050
105412310,8sleep334-21ksoftirqd/323:59:323
2385499306,5cyclictest27150-21grep19:15:223
2385499304,6cyclictest25291-21unixbench-2d19:10:243
2385499304,6cyclictest25291-21unixbench-2d19:10:243
2385499302,11cyclictest10015-21df_inode23:10:143
2385499301,11cyclictest14609-21ssh22:25:143
2385499301,10cyclictest7855-21apt-get22:20:113
2385499298,18cyclictest15657-21cstates20:05:133
2385499294,5cyclictest8289-21unixbench-2d19:45:253
2385499292,3cyclictest0-21swapper/320:50:133
2385499291,8cyclictest9507-21tune2fs19:50:163
2385499291,8cyclictest27043-21ssh22:10:133
2385499291,8cyclictest25931-21sed20:25:253
2385499291,8cyclictest25892-21ssh22:57:313
2385499291,8cyclictest13346-21apt-get20:00:123
2385499291,8cyclictest13346-21apt-get20:00:123
23854992910,17cyclictest28935-21cstates20:35:143
23854992910,17cyclictest11103-21diskmemload23:39:203
191612290,2sleep20-21swapper/221:40:132
2385499281,7cyclictest31738-21grep19:25:253
142442280,1sleep10-21swapper/121:11:261
115742280,2sleep10-21swapper/122:46:291
2385499271,9cyclictest20308-21diskstats20:15:163
2385499271,7cyclictest28543-21/usr/sbin/munin19:20:143
2385499271,6cyclictest26727-21cut20:30:133
2385399261,24cyclictest0-21swapper/219:12:152
2385399261,24cyclictest0-21swapper/219:12:152
2385299262,17cyclictest16287-21ssh23:15:111
23852992618,6cyclictest18137-21ssh22:51:241
2385299261,18cyclictest0-21swapper/119:12:201
2385299261,18cyclictest0-21swapper/119:12:201
2385199261,24cyclictest0-21swapper/019:12:120
2385199261,24cyclictest0-21swapper/019:12:120
139492260,3sleep00-21swapper/023:13:050
59882240,2chrt0-21swapper/021:54:160
31922240,1sleep10-21swapper/123:29:471
23853992413,8cyclictest15226-21cron20:05:002
23853992413,8cyclictest15226-21cron20:05:002
23852992418,4cyclictest22-21ksoftirqd/123:34:591
23852992414,7cyclictest560-21dbus-daemon21:20:001
23852992414,6cyclictest52950irq/141-enp3s0-22:00:131
23851992414,7cyclictest27393-21df22:10:160
2385199241,11cyclictest131002timerwakeupswit21:10:270
108812240,9sleep122-21ksoftirqd/121:57:291
2385399234,16cyclictest28-21ksoftirqd/222:00:202
23853992313,7cyclictest19808-21cat20:15:012
23852992316,4cyclictest5210-21idleruntime-cro00:20:001
23852992314,6cyclictest2944-21/usr/sbin/munin23:05:171
23852992314,6cyclictest2944-21/usr/sbin/munin23:05:171
2385299231,16cyclictest25859-21ssh21:45:031
23851992312,7cyclictest0-21swapper/023:20:100
190322230,9sleep012-21ksoftirqd/022:52:180
315182220,1sleep00-21swapper/023:01:290
296802220,2sleep10-21swapper/121:23:301
252502220,2sleep00-21swapper/022:56:520
2385399223,13cyclictest18934-21apt-get00:30:122
2385399223,13cyclictest18934-21apt-get00:30:122
23853992215,5cyclictest28-21ksoftirqd/221:40:002
23853992215,5cyclictest28-21ksoftirqd/221:40:002
23853992210,9cyclictest3044-21kworker/u8:022:20:102
23853992210,4cyclictest315192diskmemload23:01:292
23852992212,7cyclictest10925-21cron19:55:001
204042220,2sleep10-21swapper/120:15:171
193282220,4sleep0131rcuc/021:40:150
185342220,2sleep20-21swapper/223:40:252
112972220,2chrt0-21swapper/221:34:152
2385399217,3cyclictest28-21ksoftirqd/222:40:122
23853992116,3cyclictest4233-21ssh23:55:012
2385399211,5cyclictest18752-21ssh22:52:012
23853992113,6cyclictest28298-21diskstats22:35:152
23853992113,5cyclictest52850irq/140-enp3s0-22:55:122
2385399211,14cyclictest25861-21ssh21:45:032
2385299219,8cyclictest27024-21apt-get22:10:131
2385299213,10cyclictest23784-21apt-get23:45:121
2385299212,17cyclictest5023-21cpuspeed_turbos23:55:141
2385299212,13cyclictest32765-21ssh21:50:031
23852992113,4cyclictest11103-21diskmemload22:55:241
23852992112,6cyclictest10097-21ntp_kernel_err21:05:201
2385199211,8cyclictest152222gltestperf22:25:180
23851992113,6cyclictest3102-21ls22:40:190
23851992112,7cyclictest5610-21cat00:20:110
23851992112,7cyclictest28872-21seq00:12:520
23851992112,6cyclictest31205-21sed23:25:250
205842210,1sleep10-21swapper/100:06:111
2385399209,7cyclictest18122-21apt-get00:05:122
2385399209,3cyclictest212312sh21:16:442
2385399207,2cyclictest19688-21rm23:17:072
23853992012,6cyclictest485-21ls00:15:252
23853992012,6cyclictest4759-21find19:40:112
23853992012,6cyclictest21875-21idleruntime22:30:172
23853992012,6cyclictest20732-21memory20:15:202
23853992012,6cyclictest1534-21ntp_kernel_pll_20:45:202
23853992012,5cyclictest11103-21diskmemload23:10:322
23853992011,6cyclictest7044-21cpuspeed_turbos19:45:132
23853992011,6cyclictest11059-21sh00:00:002
23853992011,6cyclictest10494-21timerandwakeup21:05:252
2385399201,15cyclictest11103-21diskmemload23:05:032
2385399201,15cyclictest11103-21diskmemload23:05:032
2385399200,8cyclictest115752ssh22:46:292
2385299209,2cyclictest22-21ksoftirqd/122:31:511
23852992012,6cyclictest12878-21entropy21:35:151
23852992012,6cyclictest12878-21entropy21:35:151
2385199209,3cyclictest27732sh00:17:330
23851992012,6cyclictest29355-21sensors19:20:220
23851992012,6cyclictest2666-21ls22:15:230
23851992012,6cyclictest22711-21ls22:30:240
23851992012,6cyclictest10665-21unixbench_singl21:05:260
23851992011,7cyclictest27081-21perl20:30:150
23851992011,7cyclictest12312-21seq23:35:320
23851992011,6cyclictest14069-21cat20:00:170
23851992011,6cyclictest14069-21cat20:00:170
2385199201,16cyclictest0-21swapper/023:31:460
23851992010,7cyclictest4276-21sed20:50:260
2385199200,9cyclictest201132wc22:05:010
259072190,2sleep20-21swapper/222:09:192
2385399199,8cyclictest24656-21rm00:10:082
2385399199,7cyclictest4719-21cut19:40:012
2385399198,4cyclictest28-21ksoftirqd/221:20:262
2385399196,2cyclictest141rcu_preempt23:22:352
23853991913,4cyclictest317-21sh23:26:552
2385399191,2cyclictest141rcu_preempt00:00:142
23853991912,5cyclictest26294-21cat19:15:132
23853991912,4cyclictest14777-21sh00:26:122
23853991911,6cyclictest28265-21cat19:20:122
23853991910,7cyclictest4223-21cat23:30:152
23853991910,7cyclictest4042-21cat20:50:252
23853991910,7cyclictest31828-21awk19:25:252
2385299193,12cyclictest27903-21apt-get22:35:131
2385299192,11cyclictest30339-21apt-get19:25:131
2385299191,8cyclictest4657-21cron20:55:001
23852991914,3cyclictest2159-21idleruntime-cro19:35:001
23852991914,3cyclictest13812-21gltestperf20:00:151
23852991914,3cyclictest13812-21gltestperf20:00:151
23852991913,4cyclictest636-21runrttasks23:35:461
23852991912,5cyclictest30649-21ls20:40:001
23852991912,4cyclictest636-21runrttasks00:31:591
23852991912,4cyclictest636-21runrttasks00:31:591
23852991911,6cyclictest5440-21sed19:40:171
23852991911,6cyclictest3492-21seq21:28:031
23852991911,5cyclictest31762-21cron00:40:011
23852991911,5cyclictest11152-21timerwakeupswit23:10:251
23852991910,6cyclictest26498-21fw_packets19:15:151
2385199199,8cyclictest11103-21diskmemload23:58:070
23851991914,3cyclictest31581-21timerandwakeup19:25:240
23851991913,4cyclictest17012-21sh23:40:050
23851991913,4cyclictest13320-21sh00:00:320
23851991913,4cyclictest12-21ksoftirqd/023:20:000
23851991913,4cyclictest12-21ksoftirqd/000:10:000
23851991912,5cyclictest7767-21expr21:55:160
23851991912,5cyclictest21808-21seq20:19:330
23851991911,6cyclictest7827-21ntp_states19:45:200
23851991911,6cyclictest5262-21grep19:40:150
23851991911,6cyclictest30333-21seq23:50:090
23851991911,6cyclictest27932-21seq00:36:040
23851991911,6cyclictest23574-21threads20:20:240
23851991911,5cyclictest3552-21open_inodes19:35:210
23851991910,7cyclictest26283-21cat21:45:140
23851991910,6cyclictest31813-21ntp_kernel_pll_20:40:200
2385399189,6cyclictest7505-21cat21:55:132
2385399187,9cyclictest15578-21needreboot22:25:222
2385399182,11cyclictest9048-21apt-get19:50:132
23853991813,3cyclictest26978-21sh22:10:112
23853991813,3cyclictest11070-21hddtemp_smartct23:35:152
23853991812,5cyclictest28-21ksoftirqd/220:40:002
23853991812,4cyclictest28636-21sh23:48:382
23853991812,4cyclictest25420-21open_files20:25:212
23853991811,5cyclictest8231-21sensors21:00:222
23853991811,5cyclictest28575-21wc20:35:002
23853991810,6cyclictest5593-21seq00:20:112
23853991810,6cyclictest26495-21sed00:35:182
2385299189,6cyclictest8205-21awk21:00:221
2385299189,5cyclictest52950irq/141-enp3s0-00:20:131
2385299183,10cyclictest75-21kswapd023:50:111
2385299182,12cyclictest7448-21apt22:20:001
23852991812,4cyclictest9426-21sh21:32:201
23852991811,5cyclictest882-21expr20:45:131
23852991811,5cyclictest26234-21seq20:28:281
23852991811,4cyclictest636-21runrttasks23:43:231
2385299181,14cyclictest2657-21cat23:05:001
23852991810,6cyclictest26131-21cat00:10:231
2385199189,6cyclictest13451-21cut21:35:200
2385199189,6cyclictest13451-21cut21:35:200
2385199182,11cyclictest32686-21apt-get19:30:100
23851991813,3cyclictest26654-21if_enp3s019:15:160
23851991812,4cyclictest10007-21sh22:20:500
23851991811,5cyclictest9336-21cat19:50:140
23851991811,5cyclictest5848-21awk20:55:210
23851991811,5cyclictest4007-21ntp_kernel_pll_23:05:200
23851991811,5cyclictest4007-21ntp_kernel_pll_23:05:200
23851991811,5cyclictest278-21systemd-journal20:09:550
23851991811,5cyclictest25442-21unixbench_singl23:45:260
23851991811,5cyclictest25220-21ls21:20:140
23851991811,5cyclictest24667-21cat21:20:010
23851991811,5cyclictest11404-21expr19:55:140
23851991811,4cyclictest8978-21cpuspeed_turbos22:45:120
2385199181,12cyclictest27922-21chrt22:35:130
2385199180,16cyclictest7211-21grep21:00:130
2385399179,5cyclictest31802-21ntp_kernel_err20:40:202
23853991712,3cyclictest4903-21acpi20:55:112
2385399171,11cyclictest22338-21apt-get20:20:132
23853991710,5cyclictest16121-21cat20:05:172
23853991710,5cyclictest11923-21cat19:55:182
23853991710,5cyclictest1030-21expr19:30:182
2385299179,6cyclictest32490-21seq20:41:041
2385299179,6cyclictest30107-21basename19:25:001
2385299179,6cyclictest20785-21unixbench_multi21:40:271
2385299179,6cyclictest17717-21cut20:10:131
2385299178,6cyclictest26618-21awk20:30:111
2385299178,6cyclictest22449-21expr20:20:131
2385299174,3cyclictest11103-21diskmemload22:44:471
23852991712,3cyclictest2612-21apt19:35:111
2385299171,11cyclictest12297-21apt-get00:25:141
23852991710,5cyclictest7043-21expr19:45:131
2385199179,6cyclictest21627-21cut22:05:210
2385199171,7cyclictest30643-21cron20:40:000
23851991713,2cyclictest19128-21users20:10:260
2385199171,13cyclictest25024-21grep20:25:160
2385299169,5cyclictest5523-21cat20:55:171
2385299169,5cyclictest15704-21expr20:05:131
2385199163,8cyclictest630-21apt-get20:45:130
2385299158,5cyclictest11512-21df19:55:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional