You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 15:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jul 18, 2026 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2790999742,45cyclictest14353-21ssh11:15:152
27923996935,28cyclictest35966irq/58-eth010:30:123
2789999682,49cyclictest30997-21ssh09:55:151
2788799664,49cyclictest11622-21ssh12:25:140
2788799611,53cyclictest31234-21ssh12:00:170
2788799584,43cyclictest5225-21ssh11:10:130
27923995732,21cyclictest35966irq/58-eth011:25:133
27923995732,21cyclictest35966irq/58-eth011:25:123
2788799552,9cyclictest131rcu_preempt08:40:120
2788799528,27cyclictest14362-21ssh11:15:150
2788799527,7cyclictest131rcu_preempt08:55:140
2788799523,4cyclictest131rcu_preempt07:10:150
2790999511,31cyclictest19512-21ssh11:00:152
2792399491,33cyclictest35966irq/58-eth009:15:123
2788799492,36cyclictest23410-21ssh11:20:130
2788799491,39cyclictest3723-21ssh09:40:140
2789999481,35cyclictest20690-21ssh10:25:141
2788799483,33cyclictest13611-21diskmemload10:00:130
2792399475,14cyclictest35966irq/58-eth010:50:143
2788799450,32cyclictest17908-21ssh09:30:130
27923994430,7cyclictest35966irq/58-eth011:40:133
2789999443,30cyclictest12830-21ssh09:45:141
2788799442,27cyclictest14037-21ssh09:10:140
27909994327,6cyclictest27-21ksoftirqd/207:50:132
2788799431,35cyclictest29816-21ssh10:30:110
2792399424,27cyclictest35966irq/58-eth010:40:143
2792399422,23cyclictest35966irq/58-eth009:30:143
2792399412,23cyclictest35966irq/58-eth011:05:143
27899994128,9cyclictest29355-21apt-get07:45:141
27923994018,13cyclictest35966irq/58-eth010:00:133
27899994027,9cyclictest31865-21apt-get08:55:131
2788799403,28cyclictest25760-21ssh12:15:140
2788799403,28cyclictest25760-21ssh12:15:130
2788799401,33cyclictest10436-21ssh10:55:140
2788799401,33cyclictest10436-21ssh10:55:130
27899993731,3cyclictest7683-21apt-get08:30:151
2792399362,22cyclictest35966irq/58-eth008:55:153
2792399361,23cyclictest35966irq/58-eth012:00:163
2790999364,22cyclictest25775-21ssh12:15:152
2790999364,22cyclictest25775-21ssh12:15:142
2790999361,29cyclictest18235-21ssh11:35:122
27899993625,7cyclictest17401-21apt-get08:40:131
27887993614,16cyclictest0-21swapper/011:45:140
27887993614,16cyclictest0-21swapper/011:45:130
27923993519,10cyclictest35966irq/58-eth011:20:153
2792399351,21cyclictest35966irq/58-eth009:20:133
27899993524,6cyclictest20568-21apt-get12:30:131
2792399344,16cyclictest0-21swapper/311:35:133
2792399343,18cyclictest35966irq/58-eth011:15:153
2789999344,22cyclictest21882-21ssh09:50:121
2789999331,29cyclictest11457-21ssh10:20:121
2790999321,24cyclictest21898-21ssh09:50:132
27899993225,4cyclictest323-21apt-get07:15:111
2788799322,25cyclictest23100-21ssh09:15:140
2788799322,20cyclictest27332-21ssh11:40:130
2790999312,21cyclictest6523-21sh10:35:162
2789999314,4cyclictest131rcu_preempt07:50:121
27899993124,4cyclictest9154-21apt-get11:30:121
2788799311,23cyclictest16684-21ssh10:05:110
2792399302,14cyclictest35966irq/58-eth011:45:143
2792399302,14cyclictest35966irq/58-eth011:45:133
2790999303,19cyclictest27346irq/40-dwc2_hso08:40:122
2789999302,20cyclictest21-21ksoftirqd/108:50:161
27899993020,6cyclictest31207-21apt-get12:00:171
2788799302,19cyclictest1405-21ssh10:50:130
2792399292,15cyclictest35966irq/58-eth011:30:133
2790999299,11cyclictest27-21ksoftirqd/210:03:482
2789999293,18cyclictest6506-21ssh10:35:141
27899992921,6cyclictest14947-21apt-get07:30:141
27899992918,8cyclictest0-21swapper/109:20:141
2788799298,16cyclictest12-21ksoftirqd/010:45:140
2788799292,21cyclictest20597-21ssh12:30:150
2788799292,19cyclictest29717-21ssh12:35:140
2788799291,20cyclictest16787-21ssh12:10:160
2792399283,19cyclictest35966irq/58-eth012:25:133
27923992815,8cyclictest0-21swapper/310:10:153
2792399281,17cyclictest35966irq/58-eth012:15:143
2792399281,17cyclictest35966irq/58-eth012:15:133
2790999283,20cyclictest27346irq/40-dwc2_hso07:55:122
2790999282,21cyclictest27352-21ssh11:40:152
2790999282,11cyclictest0-21swapper/209:00:152
2790999281,22cyclictest9352-21cut11:30:162
2790999280,19cyclictest12681-21grep08:35:142
2789999288,15cyclictest0-21swapper/110:15:151
2789999285,6cyclictest21121-21rm11:54:161
2789999282,24cyclictest29827-21ssh10:30:131
2789999281,24cyclictest17920-21ssh09:30:131
2788799281,19cyclictest19513-21ssh11:00:140
2790999274,16cyclictest27346irq/40-dwc2_hso09:10:142
2790999273,17cyclictest0-21swapper/211:50:052
2790999272,9cyclictest27-21ksoftirqd/212:05:142
2790999272,19cyclictest0-21swapper/207:30:142
27909992710,9cyclictest0-21swapper/210:51:102
27909992710,13cyclictest261rcuc/211:55:132
27909992710,13cyclictest261rcuc/211:55:122
2789999275,7cyclictest9873-21ssh11:10:331
2789999274,18cyclictest7701-21sh12:05:131
27899992719,3cyclictest7616-21apt-get10:00:141
2790999269,6cyclictest27-21ksoftirqd/209:24:042
2790999268,3cyclictest131rcu_preempt08:55:292
2790999265,16cyclictest20595-21diskmemload12:30:142
2790999262,20cyclictest0-21swapper/207:25:132
2790999262,20cyclictest0-21swapper/207:25:122
2790999261,21cyclictest15355-21ls10:20:272
2790999261,16cyclictest23295-21cut09:15:162
2789999266,8cyclictest0-21swapper/110:51:111
2789999263,9cyclictest22726-21wc09:15:021
2789999263,15cyclictest3987-21ssh11:45:141
2789999263,15cyclictest3987-21ssh11:45:131
2789999263,13cyclictest18229-21ssh11:35:131
27899992620,2cyclictest4249-21apt-get09:00:151
27899992618,5cyclictest3708-21apt-get09:40:131
2788799266,16cyclictest13611-21diskmemload12:05:130
2788799263,15cyclictest13134-21ssh11:50:140
27923992510,11cyclictest35966irq/58-eth012:30:143
2790999259,7cyclictest0-21swapper/208:50:192
2790999258,4cyclictest27-21ksoftirqd/211:08:472
2790999252,15cyclictest27346irq/40-dwc2_hso08:10:162
2790999251,15cyclictest27346irq/40-dwc2_hso10:10:142
2790999251,10cyclictest15729-21cut10:40:162
27909992510,12cyclictest27-21ksoftirqd/210:45:142
2789999252,20cyclictest8871-21ssh09:25:131
2789999251,8cyclictest1981-21apt-get12:20:041
2788799251,15cyclictest20536-21ssh10:25:040
2792399249,6cyclictest0-21swapper/310:25:143
2792399243,17cyclictest33-21ksoftirqd/311:55:133
2792399243,17cyclictest33-21ksoftirqd/311:55:123
2790999248,10cyclictest27-21ksoftirqd/211:10:372
2790999247,11cyclictest27-21ksoftirqd/209:36:502
2790999247,11cyclictest27-21ksoftirqd/209:36:492
27909992414,8cyclictest0-21swapper/212:00:152
27909992410,8cyclictest27-21ksoftirqd/209:25:272
2789999242,7cyclictest0-21swapper/111:25:061
2789999242,7cyclictest0-21swapper/111:25:051
27899992418,3cyclictest19486-21apt-get11:00:141
27899992417,3cyclictest15564-21apt-get10:40:131
27899992416,6cyclictest14341-21apt-get11:15:151
27899992413,8cyclictest30464-21wc08:20:021
27899992411,9cyclictest24390-21ntp_states08:10:271
2789999241,17cyclictest26968-21ssh09:35:121
2789999241,17cyclictest26968-21ssh09:35:111
27899992410,11cyclictest0-21swapper/107:55:131
2788799243,12cyclictest0-21swapper/008:05:140
27887992410,11cyclictest8819-21cron09:05:010
2792399232,12cyclictest35966irq/58-eth012:05:123
2792399231,15cyclictest35966irq/58-eth010:35:143
2790999237,10cyclictest27-21ksoftirqd/209:56:332
2790999233,9cyclictest968-21ls10:30:232
2790999233,3cyclictest131rcu_preempt08:25:142
27909992312,9cyclictest0-21swapper/210:25:162
27909992311,5cyclictest10953-21ssh11:47:592
27909992311,5cyclictest10953-21ssh11:47:592
27909992311,4cyclictest27-21ksoftirqd/210:59:262
27909992311,4cyclictest27-21ksoftirqd/210:59:252
2789999237,12cyclictest23375-21/usr/sbin/munin11:20:271
2789999235,9cyclictest0-21swapper/109:19:131
2789999235,5cyclictest21-21ksoftirqd/110:09:071
2789999235,5cyclictest0-21swapper/111:07:531
27899992318,2cyclictest11612-21apt-get12:25:131
27899992315,6cyclictest10180-21fschecks_count07:25:151
27899992315,6cyclictest10180-21fschecks_count07:25:141
27899992313,6cyclictest0-21swapper/111:55:141
27899992313,6cyclictest0-21swapper/111:55:131
2789999230,17cyclictest16778-21ssh12:10:151
2788799236,7cyclictest12-21ksoftirqd/008:30:140
2788799234,11cyclictest0-21swapper/009:25:120
2788799231,8cyclictest6201-21ntp_states12:20:270
2792399224,8cyclictest0-21swapper/310:05:103
2792399224,12cyclictest35966irq/58-eth009:10:173
2792399221,17cyclictest35966irq/58-eth007:25:123
2792399221,17cyclictest35966irq/58-eth007:25:113
27923992210,9cyclictest0-21swapper/309:36:273
27923992210,9cyclictest0-21swapper/309:36:273
2790999228,4cyclictest27-21ksoftirqd/212:27:112
2790999228,4cyclictest27-21ksoftirqd/212:22:092
2790999228,4cyclictest27-21ksoftirqd/209:32:272
2790999226,3cyclictest27-21ksoftirqd/212:37:172
2790999226,10cyclictest131rcu_preempt07:10:232
27909992211,9cyclictest24819-21ssh12:14:212
2790999221,18cyclictest23623-21fschecks_time11:20:162
2790999221,16cyclictest11999-21kthreadcore08:00:202
2790999221,10cyclictest5300-21kthreadcore09:40:212
2789999229,9cyclictest718-21sed12:35:241
2789999229,9cyclictest19524-21/usr/sbin/munin08:05:251
2789999227,11cyclictest27176-21ssh11:40:031
2789999225,7cyclictest5781-21tr08:25:211
2789999221,7cyclictest30633-21sh10:10:331
27899992212,7cyclictest22311-21diskstats08:45:141
27899992212,7cyclictest0-21swapper/107:10:141
27899992210,9cyclictest24793-21fw_packets07:40:161
2788799222,4cyclictest7143-21sed10:35:210
2788799222,15cyclictest11477-21ssh10:20:130
2788799222,10cyclictest15583-21ssh10:40:130
27887992214,5cyclictest0-21swapper/009:21:210
2788799221,17cyclictest25746-21apt-get10:10:140
2788799221,16cyclictest26958-21ssh09:35:110
2788799221,16cyclictest26958-21ssh09:35:100
2788799221,14cyclictest18617-21ls07:30:280
2792399213,11cyclictest0-21swapper/311:10:133
27923992110,8cyclictest0-21swapper/311:51:053
27923992110,7cyclictest35966irq/58-eth009:40:153
2790999219,9cyclictest4176-21unixbench_multi11:25:302
2790999219,9cyclictest4176-21unixbench_multi11:25:292
2790999218,10cyclictest17599-21kthreadcore10:05:192
2790999214,3cyclictest131rcu_preempt08:20:142
2790999214,3cyclictest131rcu_preempt08:20:132
2790999213,9cyclictest13576-21sh09:45:212
2790999213,14cyclictest8345-21sed08:30:192
2790999212,16cyclictest0-21swapper/207:45:142
2789999219,9cyclictest8213-21kthreadcore07:20:221
2789999219,9cyclictest29458-21cat10:45:321
2789999219,9cyclictest14718-21ntp_192.168.11408:00:251
2788799218,8cyclictest10912-21cat11:30:200
2788799217,3cyclictest131rcu_preempt07:40:170
2788799214,9cyclictest27879-21cyclictest07:15:100
2788799211,16cyclictest31200-21cut09:55:170
2788799211,15cyclictest2385-21ssh10:15:150
2792399208,9cyclictest0-21swapper/309:49:123
2792399203,12cyclictest0-21swapper/309:50:143
27923992010,7cyclictest0-21swapper/310:55:293
27923992010,7cyclictest0-21swapper/310:55:283
2790999205,7cyclictest5483-21hwlatdetect07:20:152
2790999202,15cyclictest9227-21fschecks_time09:05:172
2790999201,9cyclictest22760-21kthreadcore07:35:222
2790999201,9cyclictest22760-21kthreadcore07:35:212
27909992014,4cyclictest0-21swapper/208:45:132
27909992012,5cyclictest2203-21ssh10:15:042
27909992011,7cyclictest0-21swapper/207:40:152
2790999201,10cyclictest16265-21fschecks_count08:05:162
2789999209,9cyclictest32390-21tail08:20:211
2789999209,9cyclictest32390-21tail08:20:201
2789999207,9cyclictest10095-21grep09:05:191
2789999206,11cyclictest25588-21ssh12:15:031
2789999206,11cyclictest25588-21ssh12:15:021
27899992013,4cyclictest14183-21open_inodes10:55:291
27899992013,4cyclictest14183-21open_inodes10:55:281
2789999200,16cyclictest19785-21ls07:35:141
2789999200,16cyclictest19785-21ls07:35:131
2788799208,7cyclictest48-21kcompactd007:45:130
2788799202,14cyclictest22015-21ssh09:50:140
27887992011,5cyclictest0-21swapper/008:45:240
2788799201,14cyclictest9055-21apt-get09:05:130
2792399199,7cyclictest0-21swapper/312:10:243
2792399192,11cyclictest35966irq/58-eth011:00:143
2792399192,11cyclictest0-21swapper/309:55:153
2792399191,13cyclictest0-21swapper/308:35:023
2792399191,10cyclictest0-21swapper/308:45:143
27923991910,6cyclictest0-21swapper/310:17:513
2790999192,14cyclictest937-21irqrtprio07:15:162
27909991910,7cyclictest0-21swapper/208:15:152
27899991910,7cyclictest16044-21ntp_states08:35:231
2788799196,10cyclictest2672-21idleruntime-cro08:25:000
2788799196,10cyclictest2672-21idleruntime-cro08:25:000
2788799191,6cyclictest22464-21if_eth011:55:170
2788799191,6cyclictest22464-21if_eth011:55:170
2788799191,14cyclictest28597-21apt-get11:05:140
27887991911,4cyclictest16664-21sendmail_mailtr09:45:300
2788799191,13cyclictest32526-21ssh11:25:130
2788799191,13cyclictest32526-21ssh11:25:120
2792399189,6cyclictest0-21swapper/310:20:523
2792399183,12cyclictest0-21swapper/312:35:063
2792399182,10cyclictest0-21swapper/309:05:143
2792399182,10cyclictest0-21swapper/307:50:133
2792399182,10cyclictest0-21swapper/307:40:153
2792399182,10cyclictest0-21swapper/307:30:033
2792399181,10cyclictest0-21swapper/307:45:143
27923991810,6cyclictest0-21swapper/312:22:593
27923991810,6cyclictest0-21swapper/310:45:333
2790999181,8cyclictest27931-21latency_hist07:10:022
2788799189,6cyclictest17144-21cron08:40:020
2788799188,7cyclictest12-21ksoftirqd/007:50:290
2788799182,14cyclictest25704-21/usr/sbin/munin08:15:230
2788799182,13cyclictest9053-21kthreadcore07:55:200
2788799182,13cyclictest12283-21kthreadcore08:00:200
2788799181,4cyclictest53646usb-storage07:20:120
27887991813,3cyclictest0-21swapper/008:25:140
2792399179,5cyclictest35966irq/58-eth009:25:193
2792399172,9cyclictest0-21swapper/307:15:113
2792399172,10cyclictest0-21swapper/308:20:133
2792399172,10cyclictest0-21swapper/308:20:133
2792399172,10cyclictest0-21swapper/308:10:163
2792399172,10cyclictest0-21swapper/308:00:143
2792399172,10cyclictest0-21swapper/307:55:133
2792399172,10cyclictest0-21swapper/307:20:053
2792399171,8cyclictest0-21swapper/308:15:133
2792399171,13cyclictest0-21swapper/308:40:023
2792399171,13cyclictest0-21swapper/308:05:173
2788799174,10cyclictest24157-21kthreadcore08:10:240
2788799171,7cyclictest19934-21kthreadcore11:35:200
2792399162,9cyclictest0-21swapper/308:40:133
2792399162,9cyclictest0-21swapper/307:35:133
2792399162,9cyclictest0-21swapper/307:35:133
2788799161,12cyclictest13456-21netstat07:25:230
2788799161,12cyclictest13456-21netstat07:25:220
2788799161,10cyclictest0-21swapper/008:50:150
2792399151,7cyclictest0-21swapper/309:00:133
2789999159,4cyclictest27767-21latency_hist07:10:011
2788799152,11cyclictest20018-21if_eth107:35:170
2788799152,11cyclictest20018-21if_eth107:35:160
2788799141,10cyclictest27904-21latency_hist07:10:010
2792399137,4cyclictest35966irq/58-eth007:10:213
2792399131,6cyclictest0-21swapper/308:25:293
2792399122,6cyclictest0-21swapper/308:50:153
279239991,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional