You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 20:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jul 14, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6198996129,4cyclictest12-21ksoftirqd/009:50:120
621799551,33cyclictest930-21snmpd08:15:142
621799501,29cyclictest30468-21ssh12:30:122
619899506,8cyclictest131rcu_preempt08:25:140
619899491,35cyclictest25995-21ssh10:40:130
619899491,35cyclictest25995-21ssh10:40:130
619899472,35cyclictest7030-21ssh12:35:130
620699462,9cyclictest131rcu_preempt08:40:131
619899452,36cyclictest19640-21ssh09:25:160
620699442,36cyclictest31206-21ssh10:25:141
620699431,36cyclictest3435-21ssh12:15:131
619899422,24cyclictest844-21munin-node11:15:130
622699414,26cyclictest35966irq/58-eth010:45:123
622699413,21cyclictest35966irq/58-eth010:30:143
622699404,9cyclictest0-21swapper/309:20:133
622699401,23cyclictest35966irq/58-eth012:00:133
619899393,21cyclictest15653-21ssh11:10:110
6226993820,14cyclictest35966irq/58-eth012:25:143
6226993819,8cyclictest35966irq/58-eth010:15:133
621799382,29cyclictest20351-21sh11:47:182
622699374,18cyclictest35966irq/58-eth009:40:123
622699374,18cyclictest35966irq/58-eth009:40:113
620699376,24cyclictest21-21ksoftirqd/112:05:121
620699371,29cyclictest14522-21rm09:40:121
620699371,29cyclictest14522-21rm09:40:121
6198993722,8cyclictest12-21ksoftirqd/008:55:130
6198993714,16cyclictest12-21ksoftirqd/010:35:120
622699362,22cyclictest35966irq/58-eth010:25:133
622699362,17cyclictest35966irq/58-eth009:10:123
6217993612,19cyclictest27-21ksoftirqd/210:10:132
619899362,27cyclictest17827-21ssh12:05:130
622699353,21cyclictest35966irq/58-eth012:20:133
621799352,28cyclictest7651-21ssh11:59:162
620699353,22cyclictest28682-21ssh09:30:121
6206993518,10cyclictest0-21swapper/110:30:131
6206993510,19cyclictest21-21ksoftirqd/107:25:141
619899351,25cyclictest12449-21ssh12:20:140
622699334,19cyclictest0-21swapper/311:10:123
620699333,22cyclictest15667-21ssh11:10:141
6226993211,17cyclictest0-21swapper/312:30:133
621799322,15cyclictest2992-21ssh11:01:182
621799321,15cyclictest13112-21ssh10:15:152
620699323,20cyclictest7024-21ssh12:35:111
619899322,22cyclictest13371-21wc10:15:170
620699313,8cyclictest131rcu_preempt09:35:131
621799303,9cyclictest0-21swapper/210:20:172
6217993014,10cyclictest14785-21ssh09:56:182
620699302,22cyclictest29904-21ssh11:00:141
620699301,20cyclictest27365-21rm10:05:141
6206993010,12cyclictest21-21ksoftirqd/109:10:111
619899306,16cyclictest630-21Async09:36:310
619899302,22cyclictest29071-21/usr/sbin/munin09:30:180
619899302,22cyclictest18859-21sed10:00:180
622699295,11cyclictest35966irq/58-eth010:35:123
622699292,10cyclictest0-21swapper/311:15:133
621799292,20cyclictest0-21swapper/209:00:132
6217992914,7cyclictest27-21ksoftirqd/212:20:132
621799291,19cyclictest1460-21ssh09:15:122
620699296,9cyclictest131rcu_preempt10:00:131
619899292,19cyclictest22539-21awk10:20:170
622699283,17cyclictest35966irq/58-eth009:00:133
622699283,16cyclictest0-21swapper/310:00:183
622699283,16cyclictest0-21swapper/309:45:173
622699283,13cyclictest35966irq/58-eth010:50:143
622699283,11cyclictest0-21swapper/310:14:183
621799283,22cyclictest28711-21ssh09:30:142
621799283,17cyclictest27346irq/40-dwc2_hso08:00:112
621799281,19cyclictest7868-21ssh10:30:142
619899282,6cyclictest131rcu_preempt09:20:140
6198992813,6cyclictest5382-21ssh09:35:120
622699272,18cyclictest0-21swapper/309:56:173
622699272,17cyclictest35966irq/58-eth007:40:113
622699272,11cyclictest0-21swapper/311:59:173
622699272,11cyclictest0-21swapper/310:21:173
622699272,11cyclictest0-21swapper/309:53:513
621799273,21cyclictest2595-21ssh10:45:122
621799271,23cyclictest20781-21ssh10:55:142
621799271,23cyclictest19424-21ssh11:30:122
621799271,17cyclictest10542-21ssh09:20:122
620699272,19cyclictest4364-21ssh12:32:171
619899271,4cyclictest0-21swapper/012:30:130
622699262,18cyclictest0-21swapper/311:30:183
6226992613,10cyclictest35966irq/58-eth009:30:113
621799262,9cyclictest0-21swapper/207:30:132
621799262,20cyclictest0-21swapper/208:05:142
6217992615,8cyclictest6520-21/usr/sbin/munin11:05:172
621799261,18cyclictest19612-21find09:25:142
620699266,11cyclictest21-21ksoftirqd/108:45:121
620699264,14cyclictest0-21swapper/111:50:121
619899261,19cyclictest32122-21ssh11:55:140
622699253,15cyclictest0-21swapper/312:10:153
622699252,11cyclictest0-21swapper/311:01:173
622699252,10cyclictest0-21swapper/312:19:183
621799253,18cyclictest27346irq/40-dwc2_hso10:40:142
621799253,18cyclictest27346irq/40-dwc2_hso10:40:142
621799252,12cyclictest26817-21ssh12:10:142
620699251,19cyclictest24662-21ssh11:15:111
619899251,19cyclictest24495-21diskmemload11:40:150
619899251,19cyclictest24495-21diskmemload11:40:140
621799243,16cyclictest0-21swapper/209:53:502
621799242,17cyclictest0-21swapper/211:29:102
621799242,10cyclictest16880-21ssh10:35:122
621799240,15cyclictest10130-21timerwakeupswit07:10:272
620699243,18cyclictest24495-21diskmemload11:25:131
6206992415,5cyclictest0-21swapper/111:20:151
619899249,10cyclictest135-21systemd-journal09:58:510
619899242,6cyclictest131rcu_preempt11:29:100
619899240,6cyclictest131rcu_preempt11:45:100
622699233,13cyclictest0-21swapper/309:36:173
6226992310,6cyclictest0-21swapper/309:25:213
621799235,13cyclictest27346irq/40-dwc2_hso10:05:122
621799234,15cyclictest27346irq/40-dwc2_hso12:35:132
621799233,14cyclictest27346irq/40-dwc2_hso07:55:122
621799232,6cyclictest0-21swapper/212:19:182
621799232,6cyclictest0-21swapper/210:03:162
621799232,14cyclictest27346irq/40-dwc2_hso08:45:132
6217992312,9cyclictest24200-21ssh11:14:562
620699233,13cyclictest21-21ksoftirqd/109:15:131
6206992311,8cyclictest21-21ksoftirqd/111:35:141
619899239,10cyclictest12-21ksoftirqd/011:20:140
619899232,16cyclictest19389-21/usr/sbin/munin11:30:170
619899232,15cyclictest2384-21ssh12:14:160
622699223,13cyclictest0-21swapper/311:47:183
622699221,13cyclictest35966irq/58-eth011:05:123
622699221,12cyclictest35966irq/58-eth012:35:123
6226992210,5cyclictest0-21swapper/308:05:123
6226992210,10cyclictest0-21swapper/310:55:233
621799229,4cyclictest4590-21latency_hist11:40:022
621799229,10cyclictest15217-21load10:50:242
621799224,13cyclictest27346irq/40-dwc2_hso08:50:122
621799223,15cyclictest930-21snmpd09:40:142
621799223,15cyclictest930-21snmpd09:40:132
6217992211,9cyclictest6958-21ssh10:29:242
621799221,16cyclictest24884-21ssh09:10:112
620699228,4cyclictest131rcu_preempt08:50:161
620699222,4cyclictest131rcu_preempt08:30:151
620699222,4cyclictest131rcu_preempt08:30:151
620699222,4cyclictest131rcu_preempt07:45:151
620699222,16cyclictest21448-21ssh12:25:131
620699222,16cyclictest21-21ksoftirqd/111:30:141
6206992214,5cyclictest0-21swapper/110:45:111
6206992213,6cyclictest0-21swapper/109:24:161
619899222,16cyclictest17340-21apt-get07:55:110
6198992215,4cyclictest24443-21nfs_client10:55:270
622699211,11cyclictest35966irq/58-eth008:50:133
621799215,10cyclictest0-21swapper/207:15:122
621799214,5cyclictest0-21swapper/207:25:132
621799214,12cyclictest0-21swapper/207:45:142
621799214,11cyclictest0-21swapper/212:05:122
621799212,16cyclictest23605-21grep08:35:152
621799211,15cyclictest22411-21grep08:30:302
621799211,15cyclictest22411-21grep08:30:302
621799211,10cyclictest8522-21cut09:35:212
6217992110,8cyclictest28405-21processes11:15:282
620699218,10cyclictest31535-21cron08:10:011
620699215,9cyclictest0-21swapper/107:15:131
620699212,17cyclictest935-21irqstats09:50:171
620699212,17cyclictest24382-21kthreadcore09:45:181
620699211,4cyclictest131rcu_preempt10:12:481
620699211,17cyclictest3950-21ls08:15:021
620699211,15cyclictest9292-21sh09:55:151
620699211,15cyclictest20900-21rm10:55:171
619899211,8cyclictest28455-21sh11:35:150
619899211,18cyclictest31872-21ls11:55:020
622699203,12cyclictest0-21swapper/311:27:173
622699202,13cyclictest0-21swapper/312:05:133
622699202,10cyclictest35966irq/58-eth009:15:123
6226992011,6cyclictest0-21swapper/311:20:333
621799208,8cyclictest23532-21apt09:45:102
621799207,5cyclictest24382-21kthreadcore11:50:222
621799207,10cyclictest27-21ksoftirqd/211:20:162
621799205,12cyclictest27346irq/40-dwc2_hso07:40:132
621799202,9cyclictest14360-21kthreadcore07:50:222
621799202,15cyclictest0-21swapper/208:20:032
621799201,9cyclictest17809-21kthreadcore07:20:212
621799201,9cyclictest15663-21tr08:25:222
621799201,9cyclictest12140-21kthreadcore08:55:212
6217992014,4cyclictest0-21swapper/212:00:162
621799201,10cyclictest22287-21kthreadcore09:05:222
620699202,15cyclictest20015-21kthreadcore07:55:201
620699201,16cyclictest26814-21latency_hist08:05:011
620699201,16cyclictest25987-21apt-get10:40:121
620699201,16cyclictest25987-21apt-get10:40:121
6206992011,6cyclictest0-21swapper/110:15:241
620699201,14cyclictest10963-21latency_hist07:15:001
6206992010,7cyclictest3028-21sh12:15:021
620699200,3cyclictest131rcu_preempt10:23:111
619899209,8cyclictest31352-21grep10:25:150
619899207,4cyclictest131rcu_preempt10:50:010
619899205,3cyclictest131rcu_preempt10:30:260
619899204,11cyclictest0-21swapper/012:25:130
6198992010,7cyclictest0-21swapper/008:20:020
622699193,11cyclictest0-21swapper/310:06:163
6226991911,6cyclictest0-21swapper/311:53:573
622699191,13cyclictest0-21swapper/307:50:173
622699191,12cyclictest0-21swapper/307:30:273
621799196,10cyclictest27346irq/40-dwc2_hso11:40:132
621799196,10cyclictest27346irq/40-dwc2_hso11:40:132
621799193,7cyclictest21636-21fschecks_time12:25:172
621799191,9cyclictest31990-21hwlatdetect08:10:152
621799191,5cyclictest0-21swapper/207:35:132
620699193,9cyclictest0-21swapper/109:25:171
620699192,13cyclictest8626-21ntp_192.168.11411:40:261
620699192,13cyclictest8626-21ntp_192.168.11411:40:251
620699192,13cyclictest10389-21sendmail_mailtr11:05:281
619899198,8cyclictest6408-21apt-get07:10:120
619899197,4cyclictest131rcu_preempt08:25:010
619899192,9cyclictest0-21swapper/008:00:100
619899192,7cyclictest12-21ksoftirqd/008:40:120
619899192,13cyclictest7647-21ssh10:10:240
619899192,12cyclictest22155-21ssh09:43:490
619899192,12cyclictest22155-21ssh09:43:490
619899191,9cyclictest31167-21kthreadcore07:35:200
619899191,7cyclictest10143-21munin-run09:20:010
6198991911,5cyclictest0-21swapper/010:05:280
619899191,12cyclictest23747-21fschecks_time09:45:150
619899190,10cyclictest11667-21apt-get10:50:140
622699189,6cyclictest35966irq/58-eth010:40:143
622699189,6cyclictest35966irq/58-eth010:40:133
622699184,9cyclictest0-21swapper/311:35:153
622699182,10cyclictest0-21swapper/308:25:153
622699182,10cyclictest0-21swapper/307:45:153
622699182,10cyclictest0-21swapper/307:20:133
622699181,11cyclictest0-21swapper/308:55:233
621799184,11cyclictest27346irq/40-dwc2_hso08:40:122
621799181,8cyclictest6319-21latency_hist07:10:012
620699183,11cyclictest16240-21usage_EthcTask10:50:311
620699182,3cyclictest0-21swapper/112:25:021
6206991813,3cyclictest0-21swapper/107:50:331
620699181,15cyclictest27057-21kthreadcore07:30:191
6206991811,4cyclictest16000-21kthreadcore11:45:221
620699181,13cyclictest8790-21df_inode12:00:151
620699181,13cyclictest32285-21fschecks_count11:55:171
620699181,12cyclictest20436-21ls10:35:241
6206991810,5cyclictest20655-21latency_hist07:25:021
619899187,8cyclictest20968-21df07:25:160
619899182,2cyclictest131rcu_preempt11:05:220
619899182,10cyclictest1-21systemd12:19:180
6198991810,3cyclictest0-21swapper/009:00:130
622699179,6cyclictest0-21swapper/311:40:243
622699179,6cyclictest0-21swapper/311:40:243
622699172,10cyclictest0-21swapper/307:55:133
622699171,12cyclictest0-21swapper/307:15:163
622699171,10cyclictest0-21swapper/308:10:143
620699176,9cyclictest4116-21apt-get08:15:131
620699172,7cyclictest0-21swapper/107:35:131
620699172,11cyclictest19593-21awk09:05:011
620699171,8cyclictest11725-21kthreadcore08:55:211
6206991711,4cyclictest0-21swapper/109:05:251
620699171,13cyclictest13798-21apt-get08:25:161
620699171,13cyclictest11687-21kthreadcore08:20:201
619899178,6cyclictest4946-21kthreadcore07:40:190
619899173,11cyclictest29509-21sed07:30:270
619899172,12cyclictest19575-21apt-get09:05:030
619899172,11cyclictest0-21swapper/007:50:150
619899172,10cyclictest0-21swapper/008:05:140
619899171,8cyclictest5353-21/usr/sbin/munin08:50:170
619899171,7cyclictest26305-21kthreadcore08:35:200
619899171,12cyclictest3885-21apt08:15:010
619899171,10cyclictest1466-21ssh11:00:280
6198991710,5cyclictest28656-21tr09:10:270
622699167,6cyclictest0-21swapper/309:05:163
622699161,9cyclictest0-21swapper/308:15:283
622699161,12cyclictest0-21swapper/308:40:233
622699161,12cyclictest0-21swapper/308:35:273
620699162,11cyclictest4184-21kthreadcore07:40:191
620699161,13cyclictest23380-21apt-get08:35:101
619899162,9cyclictest0-21swapper/007:20:140
6198991611,3cyclictest11237-21cpu07:15:140
622699158,5cyclictest35966irq/58-eth008:20:173
622699151,7cyclictest0-21swapper/308:05:013
622699151,11cyclictest0-21swapper/308:30:203
622699151,11cyclictest0-21swapper/308:30:193
622699150,10cyclictest0-21swapper/307:35:153
619899151,11cyclictest8384-21sed07:45:200
619899151,11cyclictest18598-21apt-get08:30:140
619899151,11cyclictest18598-21apt-get08:30:130
619899150,8cyclictest627-21df_inode08:45:160
620699148,4cyclictest6212-21latency_hist07:10:011
619899141,9cyclictest6238-21latency_hist07:10:010
622699138,3cyclictest0-21swapper/308:45:203
622699132,6cyclictest0-21swapper/307:25:153
622699111,8cyclictest0-21swapper/307:10:023
622699111,7cyclictest0-21swapper/307:10:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional