You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 01:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed May 06, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5177996421,11cyclictest131rcu_preempt07:50:121
516999638,41cyclictest14655-21ssh11:10:120
5186995811,33cyclictest0-21swapper/211:10:122
517799573,47cyclictest18644-21ssh09:25:131
5186995530,12cyclictest4104-21apt-get11:40:132
518699512,28cyclictest369-21ssh11:20:112
5186995112,33cyclictest0-21swapper/209:00:122
516999512,35cyclictest16233-21ssh10:35:130
516999504,4cyclictest131rcu_preempt12:20:130
517799493,33cyclictest3090-21ssh10:10:121
517799492,34cyclictest9458-21ssh11:25:121
516999496,5cyclictest131rcu_preempt07:35:120
516999481,39cyclictest7154-21ssh10:30:120
5186994727,17cyclictest9544-21apt-get09:20:112
5177994733,8cyclictest0-21swapper/110:20:121
5186994632,9cyclictest12844-21apt-get08:25:122
5177994615,23cyclictest21-21ksoftirqd/107:40:131
516999462,37cyclictest25914-21ssh12:10:110
519699442,25cyclictest35966irq/58-eth010:10:123
517799443,32cyclictest20623-21ssh12:25:121
5169994420,17cyclictest12-21ksoftirqd/010:25:110
5196994319,16cyclictest321rcuc/307:50:133
5186994313,25cyclictest0-21swapper/212:35:132
516999433,30cyclictest23413-21diskmemload10:05:120
5169994326,5cyclictest12-21ksoftirqd/009:15:120
516999432,28cyclictest17441-21ssh10:00:130
5186994231,4cyclictest13184-21apt-get11:45:132
5177994232,6cyclictest0-21swapper/112:30:121
516999425,7cyclictest131rcu_preempt11:40:130
5186994128,3cyclictest19969-21apt-get10:55:132
5186994126,10cyclictest16880-21apt-get12:05:112
516999411,29cyclictest23719-21ssh11:15:130
5186994021,14cyclictest0-21swapper/209:35:132
517799401,31cyclictest4503-21ssh09:35:131
5186993922,13cyclictest27346irq/40-dwc2_hso08:45:112
517799392,29cyclictest22660-21ssh09:45:131
5186993814,18cyclictest23809-21ssh09:10:122
517799384,24cyclictest29037-21ssh11:00:121
5169993810,18cyclictest19981-21ssh10:55:120
519699373,22cyclictest35966irq/58-eth012:30:123
518699376,21cyclictest27-21ksoftirqd/210:10:122
5186993728,4cyclictest29636-21apt-get07:35:112
518699371,26cyclictest5660-21ssh11:05:142
517799371,29cyclictest18494-21ssh11:30:121
519699363,21cyclictest35966irq/58-eth011:00:113
5196993622,7cyclictest35966irq/58-eth009:35:123
518699362,26cyclictest0-21swapper/211:35:142
5186993620,11cyclictest0-21swapper/210:40:132
5169993616,9cyclictest12-21ksoftirqd/010:40:130
5186993526,7cyclictest29617-21apt-get12:30:132
5186993519,10cyclictest0-21swapper/209:45:122
516999354,6cyclictest131rcu_preempt10:15:130
518699345,12cyclictest22245-21ssh10:36:502
5186993424,6cyclictest2508-21apt-get12:15:132
5186993413,18cyclictest31638-21ssh09:30:272
5186993410,9cyclictest15381-21ps12:20:272
518699337,13cyclictest14685-21/usr/sbin/munin07:15:322
518699336,12cyclictest32068-21sed08:45:032
5186993319,11cyclictest29032-21ssh11:00:112
517799333,10cyclictest131rcu_preempt08:40:111
516999333,22cyclictest22247-21ssh11:50:130
518699323,12cyclictest7886-21ssh12:00:132
5186993211,17cyclictest21620-21unixbench_singl10:00:292
5177993216,10cyclictest5158-21cyclictest12:35:121
5169993216,10cyclictest5158-21cyclictest09:40:120
5196993122,6cyclictest35966irq/58-eth011:10:133
5196993117,11cyclictest35966irq/58-eth011:55:123
5196993115,10cyclictest35966irq/58-eth011:05:133
518699316,9cyclictest1418-21kthreadcore08:10:202
518699313,18cyclictest27346irq/40-dwc2_hso07:25:112
5186993116,11cyclictest2042-21apt07:40:122
5186993115,13cyclictest16432-21/usr/sbin/munin07:55:302
5186993115,12cyclictest0-21swapper/208:05:112
5186993115,12cyclictest0-21swapper/208:05:102
5186993110,13cyclictest1160-21cut11:55:202
5177993121,5cyclictest0-21swapper/112:10:141
516999312,25cyclictest9556-21ssh09:20:120
518699305,17cyclictest23413-21diskmemload11:33:402
518699305,12cyclictest0-21swapper/209:25:072
518699303,18cyclictest23413-21diskmemload09:52:472
518699302,21cyclictest0-21swapper/207:30:142
5186993014,13cyclictest7850-21ntp_192.168.11408:50:242
5186993011,14cyclictest11828-21apt-get10:15:042
517799303,25cyclictest27756-21ssh09:30:121
517799302,21cyclictest23818-21ssh09:10:141
5196992917,6cyclictest0-21swapper/311:25:123
5196992916,9cyclictest0-21swapper/310:30:123
5196992916,10cyclictest35966irq/58-eth010:25:123
518699296,13cyclictest10325-21ssh10:49:482
518699293,18cyclictest27346irq/40-dwc2_hso07:10:142
518699293,11cyclictest14535-21ntp_192.168.11410:50:232
518699292,21cyclictest0-21swapper/207:50:132
518699292,20cyclictest0-21swapper/212:25:122
518699292,11cyclictest0-21swapper/210:08:532
5186992914,11cyclictest0-21swapper/208:35:122
5186992913,12cyclictest6770-21ps08:15:262
5186992910,17cyclictest21175-21kthreadcore09:05:212
517799297,15cyclictest21-21ksoftirqd/109:00:131
516999294,18cyclictest12-21ksoftirqd/011:30:140
516999293,21cyclictest8119-21ssh09:35:240
518699289,7cyclictest27346irq/40-dwc2_hso09:18:382
518699286,18cyclictest32440-21ls11:20:022
518699282,21cyclictest0-21swapper/212:10:132
5186992814,11cyclictest13571-21ssh09:40:102
5186992813,13cyclictest28081-21ssh11:51:442
5186992812,13cyclictest16001-21latency_hist10:35:022
5186992812,13cyclictest10670-21unixbench_multi07:45:282
517799282,7cyclictest26490-21apt-get10:05:131
5169992823,3cyclictest0-21swapper/009:55:130
5196992711,8cyclictest0-21swapper/310:20:113
5186992716,9cyclictest21275-21apt08:00:112
5186992712,12cyclictest9442-21fschecks_time08:55:162
5186992712,12cyclictest9442-21fschecks_time08:55:152
5186992711,13cyclictest0-21swapper/209:55:112
517799278,10cyclictest0-21swapper/111:45:141
516999272,3cyclictest131rcu_preempt12:15:110
516999272,17cyclictest31264-21ssh11:55:140
519699263,15cyclictest35966irq/58-eth008:25:143
519699262,16cyclictest35966irq/58-eth009:40:123
518699266,6cyclictest23413-21diskmemload11:28:012
518699266,17cyclictest18923-21unixbench_singl07:20:292
518699262,15cyclictest27346irq/40-dwc2_hso10:20:142
519699252,16cyclictest321rcuc/308:10:113
5186992514,9cyclictest930-21snmpd08:32:252
5186992514,9cyclictest930-21snmpd08:32:242
5186992511,12cyclictest8730-21kthreadcore08:20:192
518699250,11cyclictest930-21snmpd10:26:572
517799251,19cyclictest21-21ksoftirqd/107:15:151
516999251,18cyclictest10905-21ssh10:50:120
519699243,14cyclictest35966irq/58-eth012:05:143
519699243,14cyclictest35966irq/58-eth008:50:123
519699242,17cyclictest0-21swapper/310:05:133
519699242,13cyclictest0-21swapper/311:40:123
5196992417,4cyclictest0-21swapper/312:35:243
519699241,19cyclictest0-21swapper/312:00:133
519699241,14cyclictest35966irq/58-eth009:15:123
5169992415,6cyclictest131rcu_preempt09:50:130
5169992411,10cyclictest9909-21cron07:15:020
5169992410,5cyclictest131rcu_preempt08:55:130
5169992410,5cyclictest131rcu_preempt08:55:120
519699235,14cyclictest35966irq/58-eth010:15:143
519699232,12cyclictest35966irq/58-eth009:30:123
5196992312,8cyclictest35966irq/58-eth011:50:123
5186992311,8cyclictest5250-21date07:10:012
517799232,15cyclictest23413-21diskmemload11:40:131
517799232,11cyclictest5644-21ssh11:05:111
5177992316,4cyclictest21-21ksoftirqd/112:00:241
5177992310,10cyclictest27656-21ssh11:35:161
516999231,15cyclictest1886-21ssh10:45:130
519699223,11cyclictest35966irq/58-eth007:55:113
519699222,13cyclictest0-21swapper/311:15:123
519699222,12cyclictest35966irq/58-eth009:20:133
5196992210,8cyclictest35966irq/58-eth009:55:143
517799223,14cyclictest630-21Async09:36:311
517799223,14cyclictest630-21Async09:36:311
5177992213,4cyclictest131rcu_preempt10:25:101
5177992210,3cyclictest131rcu_preempt11:57:411
516999222,15cyclictest13209-21sh11:45:130
5196992112,6cyclictest0-21swapper/310:40:203
519699211,13cyclictest0-21swapper/310:00:123
517799212,10cyclictest10182-21kworker/1:110:53:311
517799211,2cyclictest131rcu_preempt10:48:551
517799211,16cyclictest14799-21ssh11:10:161
517799210,18cyclictest17188-21ls12:05:161
516999219,3cyclictest131rcu_preempt08:15:000
516999211,18cyclictest16584-21timerwakeupswit08:25:280
516999211,17cyclictest6937-21timerandwakeup08:15:280
516999211,16cyclictest4093-21ssh11:20:250
516999211,15cyclictest867-21rm12:30:250
516999210,14cyclictest15237-21cut07:20:160
519699202,14cyclictest35966irq/58-eth010:45:133
519699202,13cyclictest321rcuc/308:05:113
519699202,13cyclictest321rcuc/308:05:103
5196992014,4cyclictest0-21swapper/310:50:063
5196992014,4cyclictest0-21swapper/310:35:263
5196992010,6cyclictest35966irq/58-eth012:15:123
517799202,10cyclictest2528-21ssh12:15:131
5177992013,5cyclictest21-21ksoftirqd/108:35:311
5177992012,6cyclictest20301-21ssh10:55:161
5177992011,6cyclictest22336-21ssh11:50:131
517799201,15cyclictest31726-21ssh09:50:111
517799200,16cyclictest20444-21ls12:25:011
517799200,14cyclictest9479-21ls08:55:171
517799200,14cyclictest9479-21ls08:55:161
516999209,8cyclictest3815-21sed11:40:010
516999201,8cyclictest12623-21awk08:25:010
516999201,14cyclictest23804-21apt-get09:10:130
519699192,12cyclictest35966irq/58-eth011:20:143
519699192,11cyclictest0-21swapper/307:20:043
5196991913,4cyclictest0-21swapper/311:31:413
5196991912,4cyclictest0-21swapper/311:45:133
519699191,13cyclictest0-21swapper/309:50:013
519699191,12cyclictest0-21swapper/308:45:123
519699191,12cyclictest0-21swapper/307:45:143
519699191,11cyclictest0-21swapper/308:20:153
5196991910,6cyclictest0-21swapper/309:52:513
517799197,10cyclictest0-21swapper/111:21:371
517799195,4cyclictest536-21usb-storage10:15:061
517799194,12cyclictest14143-21irqcore09:40:161
5177991911,5cyclictest28274-21usage_EthcTask11:15:301
517799190,2cyclictest131rcu_preempt09:19:461
516999197,3cyclictest131rcu_preempt12:05:170
516999197,3cyclictest131rcu_preempt09:30:230
516999193,13cyclictest10511-21ssh11:05:320
516999191,7cyclictest24449-21grep12:25:260
5169991912,4cyclictest0-21swapper/012:40:000
519699188,8cyclictest0-21swapper/309:25:133
519699182,10cyclictest0-21swapper/309:05:133
519699182,10cyclictest0-21swapper/309:00:143
519699182,10cyclictest0-21swapper/308:40:123
519699182,10cyclictest0-21swapper/307:35:043
519699181,8cyclictest0-21swapper/307:10:123
5196991812,4cyclictest0-21swapper/312:22:343
5196991812,4cyclictest0-21swapper/308:30:283
5196991812,4cyclictest0-21swapper/308:30:273
519699181,11cyclictest0-21swapper/307:30:043
5196991810,5cyclictest0-21swapper/312:25:253
5196991810,3cyclictest0-21swapper/311:36:083
517799187,9cyclictest18254-21sh09:25:011
517799185,3cyclictest131rcu_preempt09:55:411
517799184,11cyclictest25458-21df_inode10:40:161
517799181,7cyclictest18840-21/usr/sbin/munin09:05:131
5177991812,4cyclictest0-21swapper/108:50:281
517799181,13cyclictest21167-21ssh10:00:251
517799180,3cyclictest131rcu_preempt07:30:291
517799180,2cyclictest131rcu_preempt10:38:411
516999188,7cyclictest27369-21df_inode08:40:140
516999186,4cyclictest131rcu_preempt09:25:290
516999186,3cyclictest131rcu_preempt12:00:240
516999185,3cyclictest131rcu_preempt07:55:190
516999184,11cyclictest12-21ksoftirqd/008:00:140
516999183,9cyclictest0-21swapper/008:45:110
516999181,9cyclictest17447-21netstat09:00:220
516999181,3cyclictest0-21swapper/010:24:200
5169991812,4cyclictest0-21swapper/008:05:240
5169991812,4cyclictest0-21swapper/008:05:230
516999181,15cyclictest5721-21kthreadcore10:10:200
5169991811,4cyclictest25792-21kthreadcore09:45:210
516999181,14cyclictest14080-21ssh11:25:290
519699179,5cyclictest35966irq/58-eth009:10:133
519699178,6cyclictest35966irq/58-eth010:55:133
519699172,10cyclictest0-21swapper/308:00:143
519699171,9cyclictest35966irq/58-eth012:10:123
519699171,13cyclictest0-21swapper/308:35:163
517799179,6cyclictest17783-21cut08:30:161
517799179,6cyclictest17783-21cut08:30:151
517799172,12cyclictest25108-21grep08:00:291
517799172,11cyclictest5158-21cyclictest07:25:121
517799172,10cyclictest0-21swapper/108:20:141
517799171,7cyclictest9147-21kthreadcore10:30:211
517799171,13cyclictest8843-21grep07:10:261
517799171,13cyclictest3232-21apt-get08:15:121
517799170,8cyclictest3030-21latency_hist08:15:001
517799170,3cyclictest131rcu_preempt07:45:141
516999172,4cyclictest53646usb-storage07:30:110
516999172,3cyclictest11642-21apt-get07:50:120
516999172,12cyclictest7664-21kthreadcore08:50:220
516999172,12cyclictest6963-21cut07:45:150
516999171,13cyclictest17974-21grep08:30:180
516999171,13cyclictest17974-21grep08:30:170
516999171,12cyclictest29797-21kthreadcore11:00:200
519699162,9cyclictest0-21swapper/307:15:163
519699161,12cyclictest0-21swapper/307:25:283
517799169,5cyclictest0-21swapper/108:45:271
517799168,5cyclictest15238-21kthreadcore08:25:201
517799162,12cyclictest16503-21sort07:55:131
5177991610,4cyclictest0-21swapper/107:35:171
516999164,9cyclictest21636-21kthreadcore07:25:190
516999161,8cyclictest19532-21sed09:05:190
516999161,12cyclictest2633-21irqrtprio07:40:170
519699152,9cyclictest0-21swapper/308:55:143
519699152,9cyclictest0-21swapper/308:55:133
5196991510,3cyclictest0-21swapper/307:40:273
517799151,11cyclictest15712-21kernelversion07:20:181
516999154,9cyclictest11627-21kthreadcore07:15:210
516999151,11cyclictest5298-21latency_hist07:10:020
516999151,11cyclictest22575-21grep08:35:150
517799141,10cyclictest5158-21cyclictest07:10:021
519699121,8cyclictest0-21swapper/308:18:223
519699111,7cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional