You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 20:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jun 06, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3251899573,44cyclictest17586-21ssh23:35:122
3253199565,8cyclictest0-21swapper/322:50:133
3250999541,37cyclictest24994-21ssh21:50:131
3253199452,27cyclictest35966irq/58-eth021:55:123
3251899451,26cyclictest21827-21ssh21:30:132
3251899443,34cyclictest16150-21ssh21:45:142
3253199434,26cyclictest35966irq/58-eth022:00:123
3253199433,24cyclictest35966irq/58-eth023:05:123
3253199413,26cyclictest35966irq/58-eth000:05:133
3250999411,27cyclictest1597-21ssh21:55:131
3253199403,23cyclictest35966irq/58-eth000:00:103
32518994010,21cyclictest27-21ksoftirqd/222:20:122
3253199391,24cyclictest35966irq/58-eth000:35:133
32518993911,9cyclictest27-21ksoftirqd/222:50:132
3250999391,24cyclictest20720-21ssh23:55:121
3250999391,24cyclictest20720-21ssh23:55:121
3250999390,32cyclictest2972-21ssh23:45:121
32501993914,4cyclictest131rcu_preempt22:40:130
3253199384,18cyclictest35966irq/58-eth022:20:113
3253199381,24cyclictest35966irq/58-eth019:30:123
3251899381,19cyclictest6246-21ssh00:05:122
3253199364,21cyclictest35966irq/58-eth022:35:133
3253199351,10cyclictest0-21swapper/323:35:113
3251899358,16cyclictest25717-21ssh22:45:142
3251899352,7cyclictest131rcu_preempt23:20:312
3251899352,17cyclictest5295-21ssh23:26:172
3253199344,19cyclictest35966irq/58-eth022:45:123
3251899312,21cyclictest10517-21ssh22:00:132
32518993111,8cyclictest27-21ksoftirqd/200:20:122
3250999310,19cyclictest21-21ksoftirqd/120:55:141
3253199302,19cyclictest35966irq/58-eth021:20:153
3253199301,15cyclictest35966irq/58-eth021:15:123
3251899303,19cyclictest27-21ksoftirqd/221:10:132
3250999301,22cyclictest24132-21ssh00:15:141
3250199302,7cyclictest131rcu_preempt22:35:120
3253199299,5cyclictest0-21swapper/321:50:253
3253199299,17cyclictest0-21swapper/323:55:133
3253199299,17cyclictest0-21swapper/323:55:123
3251899292,7cyclictest0-21swapper/219:10:132
3251899292,21cyclictest0-21swapper/220:20:132
3251899292,21cyclictest0-21swapper/200:15:132
3251899292,20cyclictest0-21swapper/223:05:132
3251899292,11cyclictest12045-21users21:40:302
3251899292,11cyclictest12045-21users21:40:302
3253199286,10cyclictest35966irq/58-eth021:30:133
3251899282,19cyclictest0-21swapper/220:40:122
3251899282,17cyclictest27346irq/40-dwc2_hso22:30:152
3250199281,23cyclictest27430-21ssh21:15:130
3251899275,14cyclictest27346irq/40-dwc2_hso19:25:132
3251899275,14cyclictest27346irq/40-dwc2_hso19:25:132
3250999274,19cyclictest7202-21ssh21:40:121
3250999274,19cyclictest7202-21ssh21:40:121
3250999271,19cyclictest28763-21ssh23:21:161
3250199271,20cyclictest19534-21ssh22:05:140
3253199264,14cyclictest35966irq/58-eth000:25:123
3253199264,14cyclictest0-21swapper/300:30:123
3253199263,15cyclictest0-21swapper/319:20:133
3253199262,16cyclictest0-21swapper/323:26:173
3251899263,15cyclictest27346irq/40-dwc2_hso23:15:122
3251899261,16cyclictest1539-21ls20:15:272
3251899261,12cyclictest12024-21ls21:00:202
3250199263,20cyclictest7221-21ssh21:40:130
3250199263,20cyclictest7221-21ssh21:40:120
32501992618,5cyclictest20114-21sh23:00:110
3253199252,6cyclictest0-21swapper/323:21:163
3253199251,9cyclictest0-21swapper/323:15:263
3253199251,9cyclictest0-21swapper/320:55:213
3253199251,5cyclictest0-21swapper/300:10:213
3253199251,4cyclictest0-21swapper/321:45:313
3253199251,14cyclictest35966irq/58-eth023:30:123
3251899258,7cyclictest131rcu_preempt19:40:022
3251899252,19cyclictest27346irq/40-dwc2_hso00:35:122
32518992514,8cyclictest0-21swapper/220:30:162
32518992512,10cyclictest0-21swapper/220:10:162
32518992511,11cyclictest0-21swapper/219:20:302
3250999253,17cyclictest29688-21ssh00:00:121
3250999251,2cyclictest131rcu_preempt21:05:121
3253199249,6cyclictest0-21swapper/323:52:573
3253199248,6cyclictest0-21swapper/321:12:233
3253199246,6cyclictest0-21swapper/323:02:443
3253199246,6cyclictest0-21swapper/321:41:253
3253199246,6cyclictest0-21swapper/321:41:253
3253199243,8cyclictest0-21swapper/323:40:323
3253199242,4cyclictest0-21swapper/322:40:183
3251899243,6cyclictest0-21swapper/223:48:172
3251899242,8cyclictest0-21swapper/223:50:142
3251899242,19cyclictest0-21swapper/220:50:122
3251899242,15cyclictest8099-21apt-get22:35:132
3251899242,11cyclictest0-21swapper/219:15:142
3251899242,10cyclictest0-21swapper/219:40:122
32518992413,8cyclictest0-21swapper/220:05:222
3250199245,15cyclictest12931-21ssh21:25:130
32501992411,9cyclictest121ksoftirqd/020:55:130
3253199239,3cyclictest0-21swapper/300:24:283
3253199231,8cyclictest0-21swapper/300:15:163
3251899239,10cyclictest630-21Async09:36:312
3251899232,15cyclictest0-21swapper/222:05:032
32518992312,9cyclictest27554-21ssh23:57:552
32518992312,9cyclictest27554-21ssh23:57:542
32518992312,9cyclictest25110-21ssh00:32:352
32509992317,4cyclictest21-21ksoftirqd/123:10:111
32509992312,3cyclictest9622-21sshd21:21:301
3250999230,17cyclictest740-21ssh00:20:101
3250199232,7cyclictest131rcu_preempt00:04:310
3251899223,14cyclictest27346irq/40-dwc2_hso20:55:142
3251899223,13cyclictest27346irq/40-dwc2_hso20:35:132
3251899223,12cyclictest27346irq/40-dwc2_hso00:10:112
3251899222,5cyclictest0-21swapper/223:42:312
32518992213,7cyclictest0-21swapper/219:30:222
3251899221,12cyclictest8852-21fschecks_count23:30:152
32518992210,4cyclictest0-21swapper/219:50:132
3251899220,12cyclictest18154-21diskmemload21:15:292
3250999226,5cyclictest131rcu_preempt20:40:121
3250999221,16cyclictest31645-21ssh22:30:141
3250199225,11cyclictest131rcu_preempt22:25:120
3250199225,11cyclictest131rcu_preempt22:25:110
3250199222,18cyclictest13552-21/usr/sbin/munin23:30:310
3250199221,9cyclictest32289-21df23:25:140
3250199221,8cyclictest8696-21cat21:20:330
32501992214,5cyclictest0-21swapper/000:15:290
3250199221,19cyclictest19734-21ssh23:16:150
3250199221,16cyclictest27459-21apt-get00:35:120
3253199216,6cyclictest0-21swapper/322:56:143
3253199212,8cyclictest0-21swapper/323:14:313
3251899218,10cyclictest8232-21cut23:10:212
3251899213,15cyclictest27346irq/40-dwc2_hso21:50:132
3251899211,13cyclictest0-21swapper/219:45:132
3251899211,12cyclictest0-21swapper/221:55:132
3251899211,11cyclictest23363-21irqrtprio22:25:182
3251899211,11cyclictest23363-21irqrtprio22:25:182
3251899211,10cyclictest855-21kthreadcore21:35:212
3250999218,10cyclictest19525-21diskmemload22:05:141
3250999211,6cyclictest30340-21sed21:35:011
32509992112,6cyclictest26114-21cron23:40:021
32509992111,7cyclictest27247-21kthreadcore23:40:171
3250999211,17cyclictest20116-21ssh23:00:121
3250999211,16cyclictest16141-21ssh21:45:141
3250999211,10cyclictest18154-21diskmemload23:50:311
3250199215,5cyclictest25922-21irqrtprio20:10:180
3250199215,14cyclictest32157-21kthreadcore21:35:190
3250199211,5cyclictest0-21swapper/000:09:400
3250199211,15cyclictest13793-21find22:20:110
3250199211,13cyclictest18768-21uniq21:10:160
3253199209,8cyclictest0-21swapper/322:31:033
3253199203,5cyclictest0-21swapper/319:45:223
3253199202,10cyclictest35966irq/58-eth020:40:133
3251899205,6cyclictest27-21ksoftirqd/222:10:192
3251899203,9cyclictest31429-21wc20:50:022
3251899201,9cyclictest9634-21sh00:25:132
32518992014,4cyclictest0-21swapper/223:00:122
3251899201,16cyclictest7445-21find20:25:112
3251899201,15cyclictest12477-21apt-get21:25:012
3251899201,10cyclictest31039-21kthreadcore00:00:192
3251899201,10cyclictest14620-21grep19:55:272
3250999202,13cyclictest12236-21apt-get20:30:121
32509992012,6cyclictest22263-21sshd22:41:091
3250999201,11cyclictest4651-21kthreadcore22:50:201
3250999201,11cyclictest0-21swapper/122:55:141
32509992010,8cyclictest29993-21cron20:15:011
3250199201,9cyclictest28760-21tr22:45:220
3250199201,8cyclictest1509-21kthreadcore00:20:180
32501992014,3cyclictest0-21swapper/022:15:140
3250199201,3cyclictest0-21swapper/021:50:220
3250199201,16cyclictest0-21swapper/019:10:120
3250199201,15cyclictest18572-21ssh00:30:130
3250199201,10cyclictest24675-21kthreadcore23:20:200
32501992010,7cyclictest21292-21ps23:55:180
32501992010,7cyclictest21292-21ps23:55:180
3253199193,11cyclictest0-21swapper/323:47:303
3253199192,11cyclictest0-21swapper/321:25:053
3253199192,11cyclictest0-21swapper/319:40:123
3253199191,13cyclictest0-21swapper/320:45:153
3253199191,13cyclictest0-21swapper/320:15:143
3251899195,11cyclictest12688-21sh22:55:202
3251899191,9cyclictest13040-21df21:25:152
3251899191,8cyclictest5389-21sed22:15:182
3251899191,14cyclictest17290-21grep21:05:272
3251899191,10cyclictest16952-21ssh22:40:142
3250999192,14cyclictest8128-21ssh22:35:151
3250999191,9cyclictest18154-21diskmemload21:36:291
3250999191,6cyclictest17208-21cut20:35:161
3250999191,4cyclictest0-21swapper/119:40:201
3250999191,2cyclictest0-21swapper/122:45:161
3250999191,13cyclictest7090-21ssh23:28:301
3250999191,13cyclictest6261-21ssh00:05:131
3250999191,10cyclictest18154-21diskmemload22:27:221
3250999191,10cyclictest18154-21diskmemload22:27:221
3250999190,6cyclictest12478-21sed22:00:201
3250199198,9cyclictest26440-21cron20:45:010
3250199192,13cyclictest2182-21apt-get22:50:120
3250199191,9cyclictest25513-21kthreadcore19:35:190
3250199191,3cyclictest0-21swapper/023:44:010
3250199191,3cyclictest0-21swapper/022:11:480
32501991912,5cyclictest0-21swapper/019:15:260
32501991911,5cyclictest10395-21kthreadcore00:25:210
3250199191,13cyclictest73-21kswapd022:55:140
3253199182,10cyclictest0-21swapper/321:05:123
3253199182,10cyclictest0-21swapper/321:00:133
3253199182,10cyclictest0-21swapper/320:50:133
3253199182,10cyclictest0-21swapper/319:10:133
3253199181,13cyclictest0-21swapper/322:25:283
3253199181,13cyclictest0-21swapper/322:25:273
3253199181,12cyclictest0-21swapper/320:25:143
3250999189,7cyclictest614-21dbus-daemon20:00:331
3250999189,6cyclictest28460-21cut21:15:201
3250999189,3cyclictest6531-21irqcore19:50:181
3250999188,8cyclictest25148-21users20:05:291
3250999188,7cyclictest21-21ksoftirqd/123:15:131
3250999186,2cyclictest131rcu_preempt19:55:281
3250999181,9cyclictest20120-21kthreadcore19:30:191
3250999181,9cyclictest18154-21diskmemload00:38:361
3250999181,14cyclictest15177-21ssh00:10:121
3250999181,14cyclictest12948-21sh21:25:141
3250999181,13cyclictest31116-21tr22:10:211
3250999181,11cyclictest0-21swapper/100:27:201
3250999181,10cyclictest18154-21diskmemload23:32:421
3250999180,8cyclictest22912-21/usr/sbin/munin00:30:301
3250199189,6cyclictest29267-21wc19:40:150
3250199188,7cyclictest7541-21df_inode20:25:140
3250199188,7cyclictest20899-21cat21:45:320
3250199187,8cyclictest15231-21kthreadcore23:50:230
3250199186,9cyclictest10079-21unixbench_singl19:50:290
3250199186,3cyclictest131rcu_preempt21:55:150
3250199183,12cyclictest0-21swapper/019:55:140
3250199182,14cyclictest12048-21latency21:00:210
3250199182,12cyclictest2541466ssh21:30:250
3250199181,9cyclictest2309-21kthreadcore22:30:230
3250199181,9cyclictest0-21swapper/022:03:000
32501991812,4cyclictest0-21swapper/020:15:240
32501991811,4cyclictest3073-21df_inode23:45:130
3250199181,14cyclictest17025-21find20:35:110
3250199181,13cyclictest5410-21ssh23:10:110
3250199181,13cyclictest14662-21apt-get19:25:130
3250199181,13cyclictest14662-21apt-get19:25:120
3253199179,6cyclictest0-21swapper/322:11:243
3253199176,7cyclictest35966irq/58-eth022:05:153
3253199172,12cyclictest0-21swapper/321:35:113
3253199172,10cyclictest0-21swapper/322:15:143
3253199172,10cyclictest0-21swapper/320:20:133
3253199172,10cyclictest0-21swapper/320:00:153
3253199172,10cyclictest0-21swapper/319:55:143
3253199172,10cyclictest0-21swapper/319:25:143
3253199172,10cyclictest0-21swapper/319:25:143
3253199171,11cyclictest0-21swapper/320:10:023
3250999179,6cyclictest12126-21cut19:20:191
3250999172,11cyclictest17433-21ntp_192.168.11422:20:241
3250999171,6cyclictest0-21swapper/123:05:211
3250999171,13cyclictest8638-21ssh22:15:271
3250999171,13cyclictest2105-21tr20:50:211
3250999170,2cyclictest131rcu_preempt20:50:011
3250999170,12cyclictest18546-21ssh21:10:121
3250199179,6cyclictest12-21ksoftirqd/021:05:120
3250199179,5cyclictest19128-21/usr/sbin/munin00:10:270
3250199176,8cyclictest17581-21apt-get23:35:120
3250199174,10cyclictest16811-21grep20:35:020
3250199173,12cyclictest12-21ksoftirqd/020:05:130
3250199172,12cyclictest3187-21kthreadcore19:45:200
3250199172,12cyclictest28964-21ssh23:05:120
3250199171,8cyclictest14431-21apt19:25:010
3250199171,12cyclictest24001-21munin-run19:35:010
32501991710,5cyclictest2508-21ntpq20:50:250
3250999168,6cyclictest27694-21open_files19:35:251
3250999163,10cyclictest809-21kthreadcore20:15:211
32509991611,3cyclictest18044-21nfs_client19:25:241
32509991611,3cyclictest18044-21nfs_client19:25:231
3250999161,12cyclictest8917-21kthreadcore20:25:191
3250199164,9cyclictest2747-21fschecks_count20:20:150
3250199161,12cyclictest27167-21grep20:45:170
3250199161,11cyclictest16469-21kthreadcore20:00:200
3253199152,10cyclictest0-21swapper/319:35:293
3250999152,11cyclictest304-21fschecks_count19:10:151
3250999151,11cyclictest5023-21apt-get19:15:141
3250999151,11cyclictest2807-21kthreadcore19:45:201
32509991510,3cyclictest2724-21fschecks_count20:20:151
3253199149,3cyclictest0-21swapper/320:30:313
3253199141,10cyclictest0-21swapper/319:15:273
3251899141,10cyclictest32612-21latency_hist19:10:022
3250999146,5cyclictest9081-21irqcore21:00:161
3250999141,10cyclictest32532-21sort19:10:021
3253199131,10cyclictest0-21swapper/319:50:223
3250199131,9cyclictest32526-21cat19:10:020
3253199121,9cyclictest0-21swapper/320:35:193
3253199111,5cyclictest0-21swapper/320:10:143
3253199101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional