You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 17:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Thu May 21, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1244499521,38cyclictest639-21ssh11:35:151
12436995211,2cyclictest131rcu_preempt12:20:140
1244499481,32cyclictest27590-21ssh11:50:121
12444994810,4cyclictest131rcu_preempt09:00:111
12459994537,4cyclictest35966irq/58-eth011:45:133
1243699455,5cyclictest131rcu_preempt12:35:110
1243699454,36cyclictest2148-21ssh09:30:130
1245299401,32cyclictest20224-21ssh11:10:132
1243699401,32cyclictest27565-21ssh11:50:110
12459993833,3cyclictest35966irq/58-eth012:00:123
12452993830,6cyclictest30627-21diskmemload09:25:112
1245299370,25cyclictest14718-21ssh09:55:102
1244499372,26cyclictest400-21ssh10:05:141
1243699372,31cyclictest12947-21ssh10:30:130
12459993530,3cyclictest35966irq/58-eth009:50:133
1245299351,30cyclictest25594-21ssh10:55:112
1245299343,27cyclictest21989-21ssh10:35:122
1245299342,26cyclictest7589-21ssh09:15:142
1245299332,20cyclictest11216-21ssh11:05:122
12436993326,5cyclictest12451-21dpkg07:10:020
1244499324,21cyclictest13192-21ssh12:00:131
1245299314,20cyclictest27-21ksoftirqd/207:10:112
1245299314,13cyclictest27-21ksoftirqd/210:15:142
1245299313,25cyclictest9650-21ssh11:40:112
12452993112,4cyclictest7838-21ssh12:15:142
1245299311,24cyclictest2255-21ssh11:00:132
1243699312,23cyclictest9442-21ssh10:10:130
12436993113,10cyclictest0-21swapper/011:25:140
12436993113,10cyclictest0-21swapper/011:25:130
1245299309,15cyclictest27-21ksoftirqd/212:05:122
1245299303,19cyclictest27346irq/40-dwc2_hso08:00:112
1245299303,18cyclictest27346irq/40-dwc2_hso09:00:122
1245299302,20cyclictest30265-21ssh11:32:112
1245299301,20cyclictest27-21ksoftirqd/210:40:112
1244499307,15cyclictest131rcu_preempt09:20:111
12459992923,3cyclictest0-21swapper/311:25:133
12459992923,3cyclictest0-21swapper/311:25:123
12459992915,11cyclictest35966irq/58-eth012:15:143
1245299296,18cyclictest261rcuc/209:20:142
1245299294,20cyclictest25881-21ssh12:25:122
1245299293,17cyclictest27346irq/40-dwc2_hso08:50:132
1245299292,20cyclictest12081-21ssh11:22:102
1245299292,17cyclictest27523-21ssh10:00:282
12444992920,6cyclictest0-21swapper/107:10:011
1244499291,19cyclictest3980-21rm10:25:131
12436992913,2cyclictest12-21ksoftirqd/007:10:100
12459992816,9cyclictest321rcuc/310:45:123
1245299284,11cyclictest0-21swapper/212:00:122
1245299283,19cyclictest31042-21ssh09:10:152
1245299282,19cyclictest930-21snmpd08:15:162
1245299282,16cyclictest4202-21ssh11:55:142
12452992815,10cyclictest27-21ksoftirqd/209:35:122
1244499282,20cyclictest30669-21ssh10:55:471
1245999274,14cyclictest0-21swapper/311:00:123
1245999273,18cyclictest0-21swapper/312:05:113
1245299273,17cyclictest29377-21kworker/u8:010:10:102
1245299273,11cyclictest0-21swapper/210:30:102
1245299272,23cyclictest21651-21ssh10:50:462
1245299272,22cyclictest0-21swapper/208:45:152
1245299272,21cyclictest5756-21sh09:50:152
12452992716,8cyclictest0-21swapper/207:35:112
12452992711,12cyclictest32755-21missed_timers11:15:222
1245999261,17cyclictest0-21swapper/309:35:143
1245299262,10cyclictest14967-21ssh11:25:132
1245299262,10cyclictest14967-21ssh11:25:122
12452992612,11cyclictest25193-21ssh09:40:482
1244499262,19cyclictest31115-21ssh11:33:111
12444992616,7cyclictest29171-21ssh09:25:251
12444992610,11cyclictest21-21ksoftirqd/111:10:451
12459992511,9cyclictest35966irq/58-eth012:20:123
1245999251,13cyclictest35966irq/58-eth009:55:113
1245299254,18cyclictest0-21swapper/211:45:272
1245299254,13cyclictest27346irq/40-dwc2_hso08:35:132
1245299253,14cyclictest27346irq/40-dwc2_hso07:20:122
1245299252,20cyclictest0-21swapper/210:49:092
1245299251,20cyclictest436-21wc09:45:282
1244499251,18cyclictest73-21kswapd009:30:121
1243699251,19cyclictest0-21swapper/011:35:130
1245999243,18cyclictest0-21swapper/310:42:103
1245999242,17cyclictest35966irq/58-eth007:20:133
12459992412,8cyclictest35966irq/58-eth009:10:143
1245299248,11cyclictest27433-21sh10:20:152
1245299245,10cyclictest6596-21munin-run08:45:012
1245299244,10cyclictest9241-21idleruntime-cro11:40:022
1245299242,12cyclictest32468-21ssh11:50:302
1245299241,20cyclictest31773-21latency_hist07:30:022
1245299241,19cyclictest12105-21ssh12:35:182
1244499241,19cyclictest14975-21sh11:25:141
1244499241,19cyclictest14975-21sh11:25:141
1243699241,17cyclictest0-21swapper/010:05:130
12436992411,10cyclictest30158-21ssh12:25:300
1245999234,10cyclictest0-21swapper/310:20:163
1245999232,19cyclictest0-21swapper/310:36:113
1245999231,15cyclictest35966irq/58-eth010:00:133
1245299238,11cyclictest167-21systemd-network09:30:102
1245299236,4cyclictest11012-21ssh10:28:122
12452992313,8cyclictest0-21swapper/207:30:172
1245299231,12cyclictest589-21grep10:05:182
1245299231,11cyclictest4010-21kthreadcore12:30:212
1245299231,11cyclictest4010-21kthreadcore12:30:212
12452992311,10cyclictest31785-21irqrtprio12:10:192
12452992310,6cyclictest23495-21ssh12:22:382
1244499232,18cyclictest18618-21fschecks_time10:15:171
1244499231,6cyclictest5151-21kthreadcore08:40:211
1244499231,3cyclictest131rcu_preempt10:20:151
1243699239,2cyclictest12-21ksoftirqd/009:55:110
1243699232,3cyclictest131rcu_preempt08:00:120
1243699232,18cyclictest18270-21kthreadcore08:55:180
1243699231,7cyclictest10717-21sshd09:50:360
12436992314,5cyclictest0-21swapper/007:30:120
1245999222,14cyclictest0-21swapper/311:11:103
12459992213,6cyclictest35966irq/58-eth010:50:143
12459992211,9cyclictest0-21swapper/309:32:443
1245999221,17cyclictest35966irq/58-eth011:05:123
1245999221,14cyclictest0-21swapper/309:15:143
1245299223,16cyclictest20068-21switchtime08:55:262
1245299221,18cyclictest1907-21kthreadcore08:05:202
1245299221,17cyclictest22164-21kthreadcore08:25:212
1245299221,17cyclictest22164-21kthreadcore08:25:202
1245299221,17cyclictest15405-21grep08:20:152
1245299221,16cyclictest26438-21irqcore09:05:172
1245299221,11cyclictest17677-21switchtime07:45:282
1245299221,10cyclictest19037-21kthreadcore07:15:212
1244499223,16cyclictest10973-21grep08:15:191
1244499222,17cyclictest23868-21unixbench_multi08:25:291
1244499222,17cyclictest23868-21unixbench_multi08:25:281
1244499222,17cyclictest21041-21unixbench_singl12:20:301
1244499222,17cyclictest12830-21latency_hist12:00:011
1244499221,9cyclictest0-21swapper/112:27:111
1244499221,8cyclictest0-21swapper/110:34:041
1244499221,7cyclictest31503-21tr09:45:231
1244499221,3cyclictest131rcu_preempt10:50:241
12444992213,6cyclictest2611-21ssh12:30:161
12444992213,6cyclictest2611-21ssh12:30:161
1244499221,18cyclictest7474-21sed08:45:201
1244499221,17cyclictest5529-21ssh11:20:021
12444992210,9cyclictest1-21systemd08:30:211
12444992210,9cyclictest0-21swapper/108:50:261
12444992210,2cyclictest21-21ksoftirqd/110:44:181
1243699226,3cyclictest131rcu_preempt12:15:190
1243699223,3cyclictest131rcu_preempt10:40:120
1243699223,16cyclictest7614-21grep11:55:230
1243699222,17cyclictest7418-21grep10:25:220
1243699222,17cyclictest31207-21kthreadcore08:35:200
1243699222,17cyclictest11688-21grep09:15:300
12436992214,6cyclictest10811-21cron11:05:010
12436992213,6cyclictest29276-21ssh10:55:250
1243699220,7cyclictest5880-21ssh09:13:290
1245999219,7cyclictest0-21swapper/309:26:203
1245999215,10cyclictest35966irq/58-eth012:25:133
1245999213,14cyclictest0-21swapper/311:32:113
1245999213,11cyclictest0-21swapper/311:40:123
1245999212,12cyclictest35966irq/58-eth011:55:123
1245999211,16cyclictest0-21swapper/309:45:153
1245299217,11cyclictest27-21ksoftirqd/207:10:022
1245299213,15cyclictest5999-21grep08:10:182
1245299212,13cyclictest27346irq/40-dwc2_hso07:50:132
1245299212,11cyclictest13809-21cron07:45:012
1245299211,10cyclictest27404-21timerwakeupswit07:55:292
1244499218,7cyclictest0-21swapper/111:03:021
1244499217,3cyclictest131rcu_preempt08:20:241
1244499212,16cyclictest11952-21irqcore12:35:171
1244499211,7cyclictest131rcu_preempt09:55:131
12444992112,6cyclictest21984-21ssh10:35:131
12444992112,6cyclictest11075-21ssh09:35:131
12444992112,5cyclictest5709-21/usr/sbin/munin09:50:201
1244499211,17cyclictest22502-21kthreadcore09:40:201
1243699214,14cyclictest7319-21users12:30:320
1243699214,14cyclictest7319-21users12:30:310
1243699212,16cyclictest30452-21kthreadcore10:20:240
1243699212,16cyclictest2639-21kthreadcore08:05:190
1243699212,16cyclictest26232-21cut09:05:150
1243699212,16cyclictest22002-21kthreadcore08:25:210
1243699212,16cyclictest22002-21kthreadcore08:25:200
1243699212,16cyclictest10670-21unixbench_multi08:45:290
1243699211,6cyclictest22083-21/usr/sbin/munin10:15:270
1243699211,17cyclictest16647-21uniq09:20:160
1245999202,11cyclictest35966irq/58-eth010:10:113
12459992011,6cyclictest35966irq/58-eth012:30:133
12459992011,6cyclictest35966irq/58-eth012:30:123
12459992011,4cyclictest0-21swapper/307:50:133
12459992010,7cyclictest0-21swapper/312:10:203
1245299201,9cyclictest24969-21fschecks_count08:30:142
1244499206,4cyclictest21-21ksoftirqd/110:04:211
1244499203,3cyclictest131rcu_preempt10:45:111
1244499202,16cyclictest19416-21kernelversion11:45:211
1244499202,15cyclictest9847-21irqcore10:10:181
1244499202,15cyclictest22484-21switchtime07:50:261
1244499202,15cyclictest16480-21grep07:10:281
1244499202,14cyclictest32061-21ls07:30:141
1244499202,13cyclictest9144-21wc07:40:021
1244499201,17cyclictest27936-21kthreadcore09:05:211
1244499201,16cyclictest710-21cron09:00:021
1244499201,16cyclictest31536-21irqcore09:10:181
1244499201,16cyclictest21304-21grep07:15:291
12444992011,6cyclictest14417-21cat11:40:311
1244499201,16cyclictest10170-21ssh12:15:231
1244499201,15cyclictest11258-21rm09:15:281
1243699203,15cyclictest23411-21kthreadcore12:05:180
1243699203,14cyclictest24422-21kthreadcore07:55:190
1243699202,16cyclictest16744-21missed_timers12:00:240
1243699202,16cyclictest10974-21sed08:15:180
1243699202,15cyclictest32408-21grep12:10:210
1243699202,15cyclictest24252-21irqcore10:00:180
1243699202,15cyclictest22370-21acpi07:20:120
1243699202,15cyclictest20059-21kthreadcore07:15:220
1243699202,15cyclictest20039-21/usr/sbin/munin09:40:140
1243699202,12cyclictest16234-21ssh11:05:450
1243699201,16cyclictest17060-21kthreadcore08:20:200
1243699201,16cyclictest10560-21kthreadcore11:40:190
1245999195,10cyclictest0-21swapper/309:40:183
1245999193,10cyclictest0-21swapper/310:30:103
1245999192,10cyclictest0-21swapper/307:10:053
12459991914,3cyclictest0-21swapper/311:50:123
12459991913,4cyclictest0-21swapper/311:35:153
12459991911,6cyclictest35966irq/58-eth010:55:123
1245999191,11cyclictest0-21swapper/307:30:283
1244499192,14cyclictest9716-21irqcore07:40:171
1244499192,14cyclictest5783-21fschecks_count08:10:161
1244499192,14cyclictest31831-21kthreadcore08:35:201
1244499192,12cyclictest31238-21ssh12:10:151
12444991911,6cyclictest22042-21cron07:20:051
1244499191,15cyclictest24850-21tr07:55:211
1244499191,15cyclictest1243-21irqcore08:05:181
1243699192,15cyclictest8886-21kthreadcore08:10:220
1243699192,15cyclictest31507-21kthreadcore11:15:210
1243699192,15cyclictest2596-21cut08:40:170
1243699192,15cyclictest24133-21/usr/sbin/munin11:30:140
1243699192,14cyclictest22381-21ssh11:45:270
1243699191,9cyclictest29664-21unixbench_singl09:25:290
1243699191,9cyclictest0-21swapper/010:51:320
1243699191,9cyclictest0-21swapper/009:36:270
1243699191,3cyclictest0-21swapper/011:21:460
1243699191,2cyclictest131rcu_preempt07:35:100
12436991911,6cyclictest27019-21ssh10:35:430
1243699191,15cyclictest14935-21kthreadcore07:45:180
1245999182,9cyclictest0-21swapper/310:15:133
1245999182,11cyclictest0-21swapper/311:18:113
1245999182,10cyclictest0-21swapper/308:00:133
1245999182,10cyclictest0-21swapper/307:35:113
12459991812,4cyclictest0-21swapper/308:55:273
1245999181,13cyclictest0-21swapper/309:20:133
1245999181,12cyclictest0-21swapper/307:30:013
1245999181,11cyclictest0-21swapper/309:05:173
12459991810,6cyclictest0-21swapper/311:20:403
1244499183,12cyclictest28314-21kthreadcore07:25:201
1244499182,13cyclictest30849-21sort12:10:021
1244499181,5cyclictest9342-21memory11:20:231
1244499181,14cyclictest603-21cron08:05:011
1243699187,5cyclictest121ksoftirqd/007:40:120
1243699184,3cyclictest131rcu_preempt11:10:160
1243699184,12cyclictest21605-21ps09:00:170
1243699183,13cyclictest19163-21if_eth107:50:170
1243699183,12cyclictest12111-21irqcore08:50:180
1243699182,14cyclictest29316-21fschecks_count09:45:180
1243699182,11cyclictest0-21swapper/007:25:130
1243699181,4cyclictest0-21swapper/010:46:480
1245999179,6cyclictest0-21swapper/312:37:213
1245999179,6cyclictest0-21swapper/310:25:373
1245999172,10cyclictest0-21swapper/308:15:143
1245999172,10cyclictest0-21swapper/307:45:133
1245999172,10cyclictest0-21swapper/307:15:143
1244499172,10cyclictest11149-21ssh11:05:101
1244499171,13cyclictest16329-21kthreadcore07:45:211
1243699173,12cyclictest25356-21irqrtprio08:30:170
1245999168,6cyclictest0-21swapper/308:28:103
1245999168,6cyclictest0-21swapper/308:28:093
1245999161,9cyclictest0-21swapper/310:05:173
1245999152,10cyclictest0-21swapper/308:50:143
1245999151,7cyclictest0-21swapper/308:10:223
1245999148,4cyclictest33-21ksoftirqd/308:40:153
1245999141,10cyclictest0-21swapper/308:00:003
1245999138,3cyclictest0-21swapper/308:35:153
1245999138,3cyclictest0-21swapper/308:05:243
1245999131,10cyclictest0-21swapper/308:45:283
1245999122,7cyclictest0-21swapper/308:30:273
1245999121,8cyclictest0-21swapper/308:20:153
1245999121,8cyclictest0-21swapper/307:40:293
1245999121,7cyclictest0-21swapper/309:00:123
1245999111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional