You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 18:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Aug 28, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2243999623,19cyclictest21-21ksoftirqd/109:10:131
2243999623,19cyclictest21-21ksoftirqd/109:10:121
2243399592,49cyclictest1536-21ssh10:00:130
22433995831,11cyclictest12-21ksoftirqd/011:40:120
2245999563,31cyclictest12048-21ssh12:30:132
22494995230,13cyclictest35966irq/58-eth010:00:133
2245999513,42cyclictest930-21snmpd12:20:132
2245999513,27cyclictest19209-21ssh11:40:122
2243999514,44cyclictest21094-21ssh09:35:091
2245999463,30cyclictest10564-21diskmemload10:05:132
2243999463,7cyclictest131rcu_preempt07:45:141
2245999441,8cyclictest15697-21ssh11:20:132
22459994416,21cyclictest0-21swapper/209:20:112
2243399443,34cyclictest22883-21ssh12:00:120
2243399442,33cyclictest8517-21sh12:10:120
2245999422,27cyclictest17765-21ssh10:45:142
2243399423,32cyclictest32173-21ssh10:35:120
2243399422,30cyclictest5083-21ssh10:20:120
2243399422,30cyclictest5083-21ssh10:20:120
2243999416,23cyclictest21-21ksoftirqd/111:05:141
22439994126,9cyclictest21-21ksoftirqd/110:24:061
22439994126,9cyclictest21-21ksoftirqd/110:24:051
2243999412,26cyclictest15917-21ssh09:50:121
2243999404,8cyclictest131rcu_preempt11:00:111
2243999396,23cyclictest21-21ksoftirqd/111:55:131
2243999393,24cyclictest21-21ksoftirqd/110:30:111
2243999392,29cyclictest24929-21ssh09:55:131
2243399392,28cyclictest3040-21sh09:25:130
2245999382,25cyclictest24644-21ssh11:25:112
2245999382,25cyclictest24644-21ssh11:25:102
2243999383,29cyclictest73-21kswapd012:15:131
2249499372,20cyclictest35966irq/58-eth010:45:113
2245999378,23cyclictest630-21Async09:36:312
2243399373,25cyclictest6687-21ssh11:15:140
2245999362,16cyclictest10205-21ssh11:35:132
2243999363,17cyclictest21-21ksoftirqd/108:15:121
2243999344,26cyclictest28172-21ssh11:45:121
2243999342,24cyclictest14135-21rm10:25:121
2249499332,20cyclictest35966irq/58-eth012:00:123
2249499332,20cyclictest35966irq/58-eth011:35:133
2249499331,22cyclictest35966irq/58-eth011:40:123
2243999334,21cyclictest21058-21ssh12:35:111
2243999331,19cyclictest21-21ksoftirqd/107:40:111
2245999322,13cyclictest7597-21ssh10:55:292
2245999322,13cyclictest7597-21ssh10:55:282
2243399322,22cyclictest26518-21ssh12:20:130
22494993117,7cyclictest0-21swapper/312:30:133
2249499302,15cyclictest35966irq/58-eth012:20:123
2245999302,22cyclictest0-21swapper/210:50:142
2243999302,22cyclictest12947-21ssh11:54:231
2243999302,20cyclictest17448-21ssh09:15:141
2243999302,18cyclictest21-21ksoftirqd/111:30:141
22439993018,6cyclictest536-21usb-storage09:40:111
2243999301,23cyclictest24668-21ssh11:25:131
2243999301,23cyclictest24668-21ssh11:25:121
2245999292,18cyclictest27346irq/40-dwc2_hso10:10:102
2245999292,17cyclictest27346irq/40-dwc2_hso08:20:122
22439992920,5cyclictest21-21ksoftirqd/112:25:131
2243399294,13cyclictest19493-21ssh10:10:100
2243399292,21cyclictest7619-21cat10:55:290
2243399292,21cyclictest7619-21cat10:55:280
2245999284,16cyclictest27346irq/40-dwc2_hso11:10:142
2245999284,16cyclictest27346irq/40-dwc2_hso09:45:132
2245999283,16cyclictest27346irq/40-dwc2_hso08:50:142
2243999282,19cyclictest11980-21ssh09:30:111
22439992811,12cyclictest21-21ksoftirqd/112:05:111
22494992712,7cyclictest35966irq/58-eth008:50:133
2245999273,19cyclictest0-21swapper/212:03:302
2245999272,18cyclictest3034-21ssh09:25:132
2243999275,4cyclictest131rcu_preempt10:35:121
2243399275,13cyclictest22413-21cyclictest08:45:130
2243399272,21cyclictest23136-21ssh10:30:100
22433992716,4cyclictest0-21swapper/007:35:120
2249499261,12cyclictest35966irq/58-eth007:50:123
2245999264,14cyclictest27346irq/40-dwc2_hso08:25:152
2245999262,17cyclictest27346irq/40-dwc2_hso07:20:122
22459992613,10cyclictest3180-21ssh12:25:152
2245999261,17cyclictest24098-21ls07:45:162
22433992617,4cyclictest0-21swapper/010:40:120
2249499253,16cyclictest0-21swapper/311:15:143
2249499253,15cyclictest0-21swapper/311:20:123
22494992511,5cyclictest0-21swapper/308:35:123
2245999253,17cyclictest27346irq/40-dwc2_hso08:30:112
2245999253,13cyclictest27346irq/40-dwc2_hso07:50:122
2243399252,18cyclictest3873-21rm11:49:230
2249499247,8cyclictest0-21swapper/309:25:133
22494992415,6cyclictest35966irq/58-eth009:30:123
2245999247,14cyclictest14143-21kthreadcore08:40:202
2245999242,6cyclictest0-21swapper/210:04:022
22459992412,9cyclictest17477-21ssh11:55:232
2243999247,3cyclictest131rcu_preempt11:10:131
2243999245,3cyclictest131rcu_preempt11:40:111
2243399246,14cyclictest28521-21ssh10:15:140
22433992415,5cyclictest10563-21ssh10:05:130
2249499232,16cyclictest0-21swapper/311:49:233
2249499230,15cyclictest35966irq/58-eth009:15:143
2245999239,11cyclictest22783-21cut11:05:212
2245999236,8cyclictest8047-21awk09:05:302
2245999235,16cyclictest27-21ksoftirqd/209:50:112
2245999233,14cyclictest27346irq/40-dwc2_hso12:10:122
2245999232,6cyclictest0-21swapper/212:19:292
2245999232,16cyclictest0-21swapper/211:46:282
2245999231,8cyclictest0-21swapper/212:06:102
22459992311,5cyclictest15255-21ssh09:30:202
2245999230,20cyclictest32269-21kthreadcore09:40:202
2243999234,9cyclictest0-21swapper/112:01:161
2243399232,8cyclictest24756-21df11:25:140
2243399232,8cyclictest24756-21df11:25:130
2243399232,18cyclictest21083-21apt-get09:35:090
2243399232,14cyclictest151rcuc/009:45:120
22433992314,6cyclictest2657-21cron09:25:010
2243399231,18cyclictest17928-21cut10:45:160
2243399231,15cyclictest30026-21cut08:25:170
2249499228,10cyclictest35966irq/58-eth009:40:123
2249499224,11cyclictest0-21swapper/311:25:103
2249499224,11cyclictest0-21swapper/311:25:103
2249499223,14cyclictest0-21swapper/310:15:163
2249499222,13cyclictest35966irq/58-eth007:10:133
22494992214,5cyclictest0-21swapper/310:30:123
22494992213,6cyclictest35966irq/58-eth009:55:133
2245999229,4cyclictest1524-21irqrtprio07:55:172
2245999225,13cyclictest27346irq/40-dwc2_hso08:10:122
2245999222,6cyclictest0-21swapper/211:17:292
2245999222,5cyclictest5095-21sh10:20:142
2245999222,5cyclictest5095-21sh10:20:132
22459992210,5cyclictest21239-21ssh10:28:122
2243999223,4cyclictest131rcu_preempt10:50:141
2243999223,14cyclictest21-21ksoftirqd/109:20:121
2243999222,15cyclictest9745-21ssh10:57:301
2243999222,15cyclictest9745-21ssh10:57:291
2243999221,18cyclictest16833-21cut08:45:171
22439992211,8cyclictest15220-21cron11:20:021
2243999221,18cyclictest11887-21find08:40:111
22433992211,7cyclictest121ksoftirqd/008:10:130
2249499212,14cyclictest0-21swapper/310:10:113
22494992115,3cyclictest35966irq/58-eth011:05:123
22494992111,5cyclictest35966irq/58-eth012:05:133
22494992110,8cyclictest0-21swapper/309:35:053
2245999215,12cyclictest24917-21apt-get09:55:122
2245999212,10cyclictest6404-21kworker/u8:110:40:132
2245999211,7cyclictest5246-21irqcore11:50:212
2245999211,6cyclictest913066ssh09:10:182
2245999211,6cyclictest913066ssh09:10:182
2245999211,4cyclictest1871266ssh11:02:302
22459992114,4cyclictest22554-21kthreadcore12:35:202
22459992112,7cyclictest0-21swapper/209:38:352
22459992112,6cyclictest31334-21sh10:15:252
2245999211,14cyclictest24546irq/40-fe98000007:10:112
2245999211,14cyclictest0-21swapper/207:15:142
2245999211,12cyclictest23164-21ssh10:30:132
2245999211,11cyclictest21170-21ntp_states09:15:282
2245999211,11cyclictest10624-21wc08:35:282
22459992110,9cyclictest1540-21ssh11:30:172
2243999218,10cyclictest6858-21cron08:35:021
22439992112,6cyclictest0-21swapper/110:10:081
2243999211,17cyclictest8104-21ls09:10:021
2243999211,13cyclictest27619-21cut07:15:151
22439992110,3cyclictest0-21swapper/107:55:161
2243399212,14cyclictest31911-21ssh12:05:120
2243399211,10cyclictest13842-21ntp_192.168.11411:35:240
2249499208,9cyclictest0-21swapper/310:54:383
2249499203,9cyclictest35966irq/58-eth007:20:123
2249499201,8cyclictest35966irq/58-eth007:45:123
22494992013,5cyclictest0-21swapper/312:25:163
22494992013,4cyclictest0-21swapper/312:35:253
22494992012,5cyclictest0-21swapper/309:50:213
22494992011,6cyclictest35966irq/58-eth010:35:123
2249499201,13cyclictest35966irq/58-eth008:55:133
2245999209,8cyclictest21856-21kthreadcore08:15:202
2245999207,6cyclictest9940-21sed07:30:172
2245999202,9cyclictest31241-21fschecks_count09:00:172
2245999202,10cyclictest28758-21kthreadcore08:55:202
2245999201,11cyclictest20541-21grep08:45:282
2243999201,5cyclictest15541-21munin-run09:50:021
22439992014,4cyclictest18801-21sh07:40:011
2243999201,3cyclictest536-21usb-storage11:20:041
22439992012,6cyclictest9709-21sshd10:04:291
22439992012,5cyclictest17192-21latency_hist12:15:011
22439992012,5cyclictest0-21swapper/107:25:161
22439992011,6cyclictest28654-21sh10:15:171
2243999201,15cyclictest28830-21timerwakeupswit08:20:271
2243999201,13cyclictest15069-21users11:35:321
2243399206,4cyclictest131rcu_preempt11:05:150
2243399203,12cyclictest0-21swapper/007:25:120
2243399203,11cyclictest13468-1kworker/u9:109:50:120
2243399201,8cyclictest17786-21ntp_192.168.11410:25:260
22433992013,5cyclictest4971-21ssh11:50:190
22433992012,6cyclictest19831-21ssh09:34:010
22433992011,6cyclictest1131-21cron10:00:010
2243399201,15cyclictest15705-21sh11:20:120
2243399201,13cyclictest20811-21latency_hist11:05:020
2249499199,8cyclictest0-21swapper/310:40:223
2249499199,7cyclictest35966irq/58-eth011:55:143
2249499199,7cyclictest35966irq/58-eth011:00:123
2249499193,13cyclictest0-21swapper/311:54:233
2249499193,11cyclictest0-21swapper/312:12:293
2249499193,11cyclictest0-21swapper/310:59:143
2249499193,11cyclictest0-21swapper/310:59:143
2249499193,11cyclictest0-21swapper/310:22:303
2249499193,11cyclictest0-21swapper/310:22:293
2249499193,11cyclictest0-21swapper/309:23:303
2249499192,11cyclictest0-21swapper/309:00:153
22494991912,4cyclictest0-21swapper/309:10:223
22494991912,4cyclictest0-21swapper/309:10:213
22494991910,6cyclictest35966irq/58-eth012:15:153
22494991910,6cyclictest0-21swapper/310:28:463
2245999192,9cyclictest19055-21cpu07:40:132
2245999191,9cyclictest9385-21grep08:00:272
2245999191,9cyclictest9385-21grep08:00:262
2245999191,9cyclictest7272-21tr07:25:222
2245999191,9cyclictest15997-21kthreadcore07:35:202
2245999191,15cyclictest12104-21kthreadcore08:05:202
2243999195,12cyclictest21-21ksoftirqd/108:35:281
2243999194,13cyclictest1029-21wc08:25:271
2243999194,12cyclictest26683-21fschecks_count12:20:151
2243999192,13cyclictest8742-21apt-get10:40:131
22439991913,3cyclictest21-21ksoftirqd/108:10:001
22439991911,5cyclictest11548-21kthreadcore10:05:201
2243999191,14cyclictest19983-21awk08:15:021
2243999191,12cyclictest167-21systemd-network10:45:131
22439991910,7cyclictest15029-21kthreadcore12:30:211
2243399198,9cyclictest3043-21apt-get12:25:130
2243399198,8cyclictest8825-21if_eth109:10:180
2243399198,8cyclictest8825-21if_eth109:10:170
2243399198,8cyclictest1274-21/usr/sbin/munin11:30:150
2243399191,8cyclictest18458-21cut12:15:210
2243399191,7cyclictest1682-21timerwakeupswit11:10:290
2243399191,15cyclictest26299-21cpu08:55:130
2243399191,11cyclictest17530-21ls09:15:150
2243399191,10cyclictest0-21swapper/012:33:540
2249499189,6cyclictest0-21swapper/311:30:583
2249499189,6cyclictest0-21swapper/310:08:113
2249499189,6cyclictest0-21swapper/307:40:213
2249499182,10cyclictest0-21swapper/308:30:133
2249499182,10cyclictest0-21swapper/308:25:153
2249499182,10cyclictest0-21swapper/307:35:133
2243999189,6cyclictest30110-21grep08:55:291
2243999189,6cyclictest22330-21latency_hist07:10:011
2243999188,7cyclictest21-21ksoftirqd/108:55:011
2243999186,9cyclictest32516-21grep07:50:271
2243999181,3cyclictest3019-21apt-get09:25:121
2243999181,15cyclictest7378-21kthreadcore08:00:201
2243999181,15cyclictest7378-21kthreadcore08:00:191
2243999180,9cyclictest131rcu_preempt07:35:011
2243999180,9cyclictest131rcu_preempt07:20:291
2243399189,6cyclictest30741-21irqcore09:40:180
2243399188,7cyclictest21597-21sed12:35:170
2243399182,11cyclictest0-21swapper/010:50:140
22433991812,3cyclictest12396-21irqrtprio08:40:180
22433991810,5cyclictest22472-21latency_hist12:00:010
2249499179,5cyclictest35966irq/58-eth011:10:253
2249499172,10cyclictest0-21swapper/307:25:133
22494991711,4cyclictest0-21swapper/309:45:053
2245999171,8cyclictest22532-21latency_hist07:10:022
2243999171,8cyclictest27200-21cron07:15:011
2243399179,6cyclictest6860-21cron08:35:020
2243399177,7cyclictest28867-21wc07:50:160
2243399174,11cyclictest2031-21cat09:00:240
2243399173,7cyclictest0-21swapper/007:20:120
2243399173,11cyclictest28080-21cut07:15:180
2243399172,10cyclictest0-21swapper/008:50:150
2243399171,13cyclictest11900-21cut07:30:200
2243399171,12cyclictest9901-21tr08:35:210
2243399171,12cyclictest6659-21tr08:00:190
2243399171,12cyclictest6659-21tr08:00:180
2249499168,6cyclictest0-21swapper/308:04:223
2249499168,6cyclictest0-21swapper/308:04:213
2249499162,10cyclictest35966irq/58-eth007:30:133
2249499162,10cyclictest0-21swapper/308:45:143
2249499161,12cyclictest0-21swapper/307:55:153
2243999161,12cyclictest31323-21cut09:00:161
2243399164,9cyclictest24060-21grep07:45:150
2243399164,10cyclictest1519-21irqcore07:55:170
2243399163,10cyclictest14190-21sendmail_mailtr08:05:270
2243399162,3cyclictest131rcu_preempt07:10:010
2243399162,11cyclictest21058-21cut07:40:200
2243399162,11cyclictest19972-21apt-get08:15:030
2243399162,10cyclictest0-21swapper/007:10:120
2243399161,6cyclictest27309-21kthreadcore08:20:200
2243399161,11cyclictest5904-21tr09:05:190
2249499151,11cyclictest0-21swapper/307:20:023
2249499138,3cyclictest0-21swapper/308:20:153
2249499132,7cyclictest0-21swapper/309:05:123
2249499131,10cyclictest0-21swapper/308:45:023
2249499122,8cyclictest0-21swapper/308:10:143
2249499121,9cyclictest0-21swapper/308:15:183
2249499121,9cyclictest0-21swapper/308:05:303
2249499101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional