You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 23:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jun 09, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2586799587,6cyclictest131rcu_preempt08:45:141
2587899551,36cyclictest1660-21ssh11:10:112
2586799521,9cyclictest131rcu_preempt11:10:111
2586699521,8cyclictest131rcu_preempt07:30:120
2586699521,8cyclictest131rcu_preempt07:30:120
2588799493,31cyclictest35966irq/58-eth009:15:133
2586699492,38cyclictest3330-21ssh10:35:140
2586799462,35cyclictest17421-21ssh11:55:161
2586699464,35cyclictest11933-21ssh12:10:140
25866994626,5cyclictest12-21ksoftirqd/011:20:130
25866994618,20cyclictest12-21ksoftirqd/010:50:140
25866994617,22cyclictest12-21ksoftirqd/010:45:110
2588799443,27cyclictest35966irq/58-eth011:30:133
2586799442,35cyclictest19370-21ssh09:50:131
25866994316,19cyclictest12-21ksoftirqd/011:30:120
2587899422,32cyclictest20111-21cat09:30:302
2586699418,21cyclictest12-21ksoftirqd/010:55:130
2586699418,21cyclictest12-21ksoftirqd/010:55:130
2586699413,26cyclictest23008-21ssh11:40:120
2586799391,13cyclictest30247-21switchtime12:00:301
2586799391,13cyclictest30247-21switchtime12:00:291
2586699394,23cyclictest31974-21ssh11:45:130
2586699392,8cyclictest131rcu_preempt11:05:130
2586699381,3cyclictest131rcu_preempt11:25:120
2587899372,28cyclictest27481-21cat10:10:292
2587899372,18cyclictest17358-21cat07:30:302
2587899372,18cyclictest17358-21cat07:30:292
2588799362,26cyclictest35966irq/58-eth011:25:133
2587899362,16cyclictest5062-21ssh10:00:122
2586799361,19cyclictest24312-21ssh12:35:151
2586699363,26cyclictest10550-21ssh11:15:130
25887993519,10cyclictest35966irq/58-eth010:45:143
2587899350,18cyclictest30551-21usage_EthcTask07:10:292
25866993514,14cyclictest12-21ksoftirqd/009:35:130
2588799342,15cyclictest35966irq/58-eth009:35:143
25887993417,9cyclictest0-21swapper/310:05:143
2587899343,23cyclictest0-21swapper/210:15:292
2586699346,19cyclictest12-21ksoftirqd/010:05:140
25878993318,9cyclictest0-21swapper/211:00:122
2586799331,26cyclictest21020-21ssh09:15:131
2586799331,12cyclictest17537-21ntp_states08:05:271
2586699332,28cyclictest17697-21ssh10:25:110
25866993317,13cyclictest12-21ksoftirqd/012:05:120
2588799323,18cyclictest35966irq/58-eth007:35:133
2586799323,7cyclictest0-21swapper/110:10:281
2586799322,9cyclictest21511-21grep10:45:161
2588799311,18cyclictest35966irq/58-eth011:45:133
25878993112,10cyclictest27-21ksoftirqd/211:20:142
25867993114,9cyclictest25865-21cyclictest08:50:121
2586799311,11cyclictest18013-21ssh10:41:291
2586699312,23cyclictest5315-21unixbench_multi09:40:280
25887993014,10cyclictest35966irq/58-eth012:05:103
2586799303,7cyclictest0-21swapper/107:30:301
2586799303,7cyclictest0-21swapper/107:30:291
2586799302,22cyclictest15917-21ssh12:10:301
2586699302,22cyclictest22047-21cat07:35:290
2586699302,22cyclictest18523-21ssh09:49:300
2587899292,21cyclictest14252-21ntp_states11:15:272
2586799292,19cyclictest9506-21sh10:00:301
2586699292,20cyclictest26354-21ssh12:00:150
2586699292,20cyclictest26354-21ssh12:00:140
2586699292,19cyclictest6409-21ssh09:25:020
2588799283,16cyclictest0-21swapper/310:15:283
2587899286,18cyclictest25865-21cyclictest11:40:112
2587899286,17cyclictest27346irq/40-dwc2_hso08:50:132
2587899282,15cyclictest27346irq/40-dwc2_hso11:55:162
25878992819,7cyclictest11927-21sh12:10:132
2586799282,19cyclictest31984-21/usr/sbin/munin10:15:111
2586799282,10cyclictest30561-21diskstats10:50:151
2586699289,8cyclictest12-21ksoftirqd/008:10:100
2586699281,19cyclictest15578-21ssh09:30:110
2588799273,16cyclictest0-21swapper/309:40:293
2588799272,18cyclictest35966irq/58-eth011:05:123
25887992712,9cyclictest35966irq/58-eth011:10:133
2587899274,14cyclictest27346irq/40-dwc2_hso08:00:122
2587899271,16cyclictest0-21swapper/210:05:162
2587899271,10cyclictest27573-21ssh09:54:282
2586699272,19cyclictest9507-21cat10:00:300
2586699271,20cyclictest32034-21sh10:15:120
2588799263,14cyclictest0-21swapper/307:30:303
2588799263,14cyclictest0-21swapper/307:30:293
2588799263,11cyclictest0-21swapper/310:13:293
2588799262,13cyclictest35966irq/58-eth009:50:123
2588799261,17cyclictest0-21swapper/310:00:303
2587899265,17cyclictest0-21swapper/210:50:152
2587899262,9cyclictest21475-21ssh10:45:152
25878992614,9cyclictest25521-21irqcore11:05:192
25878992613,10cyclictest0-21swapper/207:35:282
2586799262,18cyclictest31257-21cat10:30:291
2586799262,18cyclictest26383-21awk07:40:291
2586799261,13cyclictest14021-21sh11:35:121
2586699262,19cyclictest14039-21ssh11:35:120
2588799253,15cyclictest0-21swapper/312:00:153
2588799253,15cyclictest0-21swapper/312:00:153
2588799252,17cyclictest35966irq/58-eth008:00:113
25887992513,3cyclictest0-21swapper/309:30:113
25887992510,10cyclictest321rcuc/310:25:103
2587899256,9cyclictest21040-21sh09:15:132
2587899255,13cyclictest27346irq/40-dwc2_hso12:00:142
2587899255,13cyclictest27346irq/40-dwc2_hso12:00:132
2587899252,13cyclictest8752-21ssh10:20:142
25878992518,5cyclictest261rcuc/210:55:132
25878992518,5cyclictest261rcuc/210:55:122
25878992513,9cyclictest30660-21ssh12:37:212
2587899251,20cyclictest32194-21switchtime08:20:282
2587899251,18cyclictest28502-21sh11:25:142
2587899251,16cyclictest10299-21sh09:45:142
2586799252,19cyclictest15598-21ssh09:30:131
2586699253,17cyclictest20870-21ssh12:15:130
25866992521,2cyclictest0-21swapper/009:20:140
2588799243,13cyclictest0-21swapper/307:10:303
2587899245,14cyclictest27346irq/40-dwc2_hso09:00:122
2587899243,8cyclictest0-21swapper/208:25:112
2587899243,6cyclictest15371-21ssh12:30:112
25878992413,9cyclictest14298-21ssh12:29:082
2587899240,10cyclictest27243-21irqcore10:30:162
2586799247,12cyclictest21-21ksoftirqd/111:15:301
2586799241,19cyclictest5033-21ssh11:30:141
2586699242,7cyclictest131rcu_preempt10:10:090
2586699242,14cyclictest0-21swapper/007:25:130
2588799233,13cyclictest0-21swapper/309:25:023
2588799232,12cyclictest35966irq/58-eth012:10:143
25887992317,4cyclictest0-21swapper/312:25:013
2587899238,5cyclictest15578-21kthreadcore11:35:202
2587899236,15cyclictest27346irq/40-dwc2_hso09:35:132
2587899236,12cyclictest28428-21ssh09:55:132
2587899235,11cyclictest27346irq/40-dwc2_hso12:05:112
2587899235,11cyclictest27346irq/40-dwc2_hso08:55:122
2587899232,6cyclictest0-21swapper/210:29:292
2587899232,18cyclictest2746ksoftirqd/208:15:132
2587899231,7cyclictest8120-21munin-run11:50:002
25878992315,5cyclictest7654-21ssh11:30:212
2586799231,8cyclictest19517-21cut10:25:201
2586799231,17cyclictest30077-21ssh09:20:141
25866992313,6cyclictest0-21swapper/012:25:140
25887992214,5cyclictest0-21swapper/311:55:183
2587899228,5cyclictest3222-21latency_hist07:20:022
2587899227,7cyclictest27-21ksoftirqd/209:05:272
2587899223,14cyclictest27346irq/40-dwc2_hso08:40:132
2587899223,14cyclictest27346irq/40-dwc2_hso08:35:122
2587899222,6cyclictest0-21swapper/209:23:292
2587899221,18cyclictest24148-21kthreadcore12:15:232
25878992211,4cyclictest1215-21/usr/sbin/munin09:40:262
25878992210,5cyclictest15371-21ssh11:53:032
2587899220,14cyclictest4465-21find07:55:132
2587899220,14cyclictest4465-21find07:55:122
2586799228,3cyclictest131rcu_preempt09:45:251
2586799223,16cyclictest16008-21ssh11:00:131
2586799223,10cyclictest0-21swapper/107:15:121
2586799222,15cyclictest28610-21grep09:35:281
2586699222,14cyclictest2452-21cat07:15:290
2586699221,15cyclictest28446-21sh09:55:140
25866992210,10cyclictest391-21cron12:40:010
2588799213,13cyclictest0-21swapper/310:42:303
25887992114,4cyclictest0-21swapper/310:35:083
2588799211,14cyclictest35966irq/58-eth008:45:123
25887992110,8cyclictest0-21swapper/312:33:323
2587899217,6cyclictest5991-21cron12:25:002
2587899217,5cyclictest7852-21cron07:25:002
2587899213,15cyclictest0-21swapper/209:25:012
2587899212,15cyclictest3674-21if_eth110:35:202
2587899211,14cyclictest32147-21ls07:50:142
2587899211,12cyclictest24372-21kthreadcore07:40:222
2587899211,11cyclictest23356-21kthreadcore08:45:232
2587899211,11cyclictest15137-21ssh10:40:222
2586799211,4cyclictest0-21swapper/112:28:431
2586699217,8cyclictest0-21swapper/009:05:120
2586699212,14cyclictest24214-21users09:50:300
2586699211,17cyclictest8887-21grep10:20:170
2588799202,13cyclictest0-21swapper/311:40:123
25887992013,5cyclictest0-21swapper/312:20:153
2588799201,17cyclictest35966irq/58-eth009:20:133
25887992010,4cyclictest0-21swapper/310:55:143
25887992010,4cyclictest0-21swapper/310:55:133
2587899203,13cyclictest27346irq/40-dwc2_hso07:25:122
2587899201,9cyclictest17899-21grep08:05:282
2587899201,9cyclictest12820-21kthreadcore09:10:202
2587899201,15cyclictest6073-21if_eth108:30:182
2587899201,15cyclictest26030-21cat07:10:022
2587899201,15cyclictest26030-21cat07:10:022
2587899200,11cyclictest22770-21grep08:10:262
25867992013,4cyclictest28398-21/usr/sbin/munin09:55:161
2586799201,15cyclictest8088-21apt-get07:25:101
2586799201,15cyclictest28501-21ssh11:25:141
2586799201,15cyclictest20846-21ssh12:15:131
2586699206,4cyclictest18163-21kthreadcore11:55:230
2586699201,9cyclictest8593-21cpuspeed11:50:140
2586699201,12cyclictest1464-1kworker/0:0H08:35:110
2588799198,7cyclictest35966irq/58-eth010:20:153
2588799193,11cyclictest0-21swapper/311:15:293
2588799193,10cyclictest0-21swapper/309:45:053
25887991913,4cyclictest0-21swapper/311:50:053
25887991913,4cyclictest0-21swapper/311:00:263
25887991912,5cyclictest0-21swapper/311:22:083
25887991912,4cyclictest0-21swapper/312:15:253
25887991912,4cyclictest0-21swapper/311:35:253
25887991912,4cyclictest0-21swapper/310:50:213
2588799191,13cyclictest0-21swapper/310:30:113
2588799191,13cyclictest0-21swapper/308:10:193
2588799191,11cyclictest35966irq/58-eth009:05:123
25887991910,6cyclictest0-21swapper/309:10:173
2587899196,4cyclictest29131-21kthreadcore07:45:212
2586799196,10cyclictest4030-21cut09:40:221
2586799193,9cyclictest0-21swapper/110:05:301
2586799192,13cyclictest8225-21apt-get10:20:041
2586799192,11cyclictest23219-21ntp_states11:20:271
2586799191,15cyclictest25030-21ssh11:05:151
25867991910,6cyclictest0-21swapper/112:05:111
2586699193,9cyclictest0-21swapper/010:30:290
2586699192,2cyclictest131rcu_preempt07:50:120
2586699192,14cyclictest26248-21kthreadcore08:15:220
25866991911,6cyclictest7835-21sshd07:22:290
25866991911,5cyclictest14982-21kthreadcore10:40:220
2588799183,9cyclictest0-21swapper/307:40:283
2588799183,10cyclictest0-21swapper/312:35:163
2588799182,10cyclictest0-21swapper/307:55:153
2588799182,10cyclictest0-21swapper/307:55:153
2588799182,10cyclictest0-21swapper/307:50:143
2588799182,10cyclictest0-21swapper/307:25:123
25887991812,4cyclictest0-21swapper/309:55:443
2588799181,12cyclictest0-21swapper/309:00:143
2588799181,12cyclictest0-21swapper/308:15:243
2588799181,11cyclictest0-21swapper/308:55:163
2586799189,7cyclictest3857-21cat11:45:311
2586799189,6cyclictest7930-21cat10:55:201
2586799189,6cyclictest7930-21cat10:55:191
2586799188,7cyclictest6578-21apt-get09:25:131
2586799187,7cyclictest21-21ksoftirqd/107:50:141
2586799182,4cyclictest4564-21grep10:35:221
2586799181,5cyclictest29987-21diskstats12:20:171
2586799181,12cyclictest8506-21ssh11:50:121
2586799181,11cyclictest29790-21timerandwakeup07:10:271
2586799180,5cyclictest53646usb-storage08:10:051
2586699189,6cyclictest121ksoftirqd/007:40:130
2586699181,8cyclictest18939-21ntp_192.168.11412:30:220
2586699181,14cyclictest30750-21tr12:20:210
25866991811,4cyclictest0-21swapper/008:05:010
2586699181,13cyclictest10352-21apt-get08:35:020
2588799172,9cyclictest0-21swapper/308:40:143
2588799172,10cyclictest0-21swapper/308:35:123
2588799172,10cyclictest0-21swapper/308:25:133
2588799172,10cyclictest0-21swapper/307:45:133
2588799171,13cyclictest0-21swapper/308:20:163
2586799178,6cyclictest12068-21rm09:10:141
2586799174,10cyclictest19482-21unixbench_singl12:30:281
2586799173,9cyclictest0-21swapper/107:45:151
2586699179,6cyclictest8928-21cat07:55:310
2586699179,6cyclictest8928-21cat07:55:310
2586699179,6cyclictest23756-21open_files08:45:260
2586699177,8cyclictest20866-21cat11:00:310
2586699177,7cyclictest614-21dbus-daemon08:59:580
2586699177,6cyclictest29804-21wc08:55:020
2586699172,11cyclictest0-21swapper/008:05:150
2586699172,10cyclictest0-21swapper/007:45:150
2586699171,12cyclictest1906-21ssh11:10:160
25866991710,5cyclictest16390-21usage_Anybus09:10:300
2588799169,4cyclictest0-21swapper/308:30:293
2588799163,7cyclictest0-21swapper/307:15:283
2588799162,9cyclictest0-21swapper/307:20:143
2588799162,10cyclictest0-21swapper/308:05:153
2588799161,13cyclictest0-21swapper/308:50:233
2586799168,6cyclictest985-21wc08:55:271
2586799168,6cyclictest9216-21ntpq08:30:241
2586799168,6cyclictest3414-21grep07:20:161
2586799168,6cyclictest20232-21cut07:35:201
2586799165,8cyclictest7241-21tr07:55:221
2586799165,8cyclictest7241-21tr07:55:211
2586799164,9cyclictest8948-21tail09:05:201
2586799163,4cyclictest131rcu_preempt08:40:151
2586799162,9cyclictest0-21swapper/108:35:051
2586799162,11cyclictest1636-21tail08:25:181
2586799161,12cyclictest5288-21kthreadcore09:00:201
2586799161,12cyclictest27025-21latency08:15:231
25867991610,4cyclictest26621-21sh11:40:221
2586699163,9cyclictest0-21swapper/007:10:120
2586699163,10cyclictest2359-21cpuspeed09:00:130
2586699162,10cyclictest21139-21df_inode09:15:150
2586699162,10cyclictest0-21swapper/008:25:130
2586699162,10cyclictest0-21swapper/008:20:140
25866991610,4cyclictest18941-21ntp_states08:40:260
2586799152,9cyclictest0-21swapper/108:00:121
25867991510,3cyclictest29227-21cut08:20:171
2586799141,9cyclictest25952-21latency_hist07:10:011
2586799141,9cyclictest25952-21latency_hist07:10:001
2586699141,10cyclictest25875-21latency_hist07:10:010
2586699141,10cyclictest25875-21latency_hist07:10:000
2588799113,6cyclictest0-21swapper/307:10:023
2588799113,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional