You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 15:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Sep 12, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3493995120,3cyclictest131rcu_preempt11:20:120
351899496,23cyclictest35966irq/58-eth011:40:153
3518994913,23cyclictest35966irq/58-eth011:45:133
351899472,25cyclictest35966irq/58-eth012:05:133
351899463,28cyclictest35966irq/58-eth009:55:143
351899453,27cyclictest35966irq/58-eth010:20:153
351899442,27cyclictest35966irq/58-eth009:50:183
350199442,36cyclictest7315-21ssh12:20:121
3510994216,9cyclictest131rcu_preempt09:45:132
351899413,18cyclictest35966irq/58-eth010:55:133
351899412,24cyclictest35966irq/58-eth009:00:133
351899411,23cyclictest35966irq/58-eth008:15:133
350199411,4cyclictest131rcu_preempt10:00:151
3510994017,7cyclictest27-21ksoftirqd/210:40:482
351899395,22cyclictest35966irq/58-eth012:10:153
351899393,22cyclictest35966irq/58-eth011:30:133
350199391,33cyclictest2262-21ssh09:35:131
349399394,31cyclictest23725-21ssh11:35:130
350199389,4cyclictest131rcu_preempt10:05:151
350199389,4cyclictest131rcu_preempt10:05:141
351899371,21cyclictest35966irq/58-eth011:00:153
351099375,4cyclictest131rcu_preempt09:50:172
351099370,27cyclictest30777-21ssh12:15:112
351899362,22cyclictest35966irq/58-eth011:35:133
351899362,22cyclictest35966irq/58-eth007:55:123
350199361,28cyclictest18246-21ssh11:50:121
349399361,30cyclictest20303-21ssh09:45:130
350199342,17cyclictest0-21swapper/108:45:141
350199341,27cyclictest28996-21ssh11:20:131
350199341,25cyclictest12942-21ssh10:35:151
349399345,22cyclictest2016-21ssh12:35:150
351899339,16cyclictest35966irq/58-eth009:45:113
351899338,14cyclictest35966irq/58-eth010:50:123
351899338,14cyclictest35966irq/58-eth010:50:113
349399334,26cyclictest25686-21ssh09:30:160
3493993320,8cyclictest131rcu_preempt11:45:130
349399325,24cyclictest7499-21ssh10:50:110
349399325,24cyclictest7499-21ssh10:50:110
3493993214,3cyclictest131rcu_preempt08:20:110
350199312,23cyclictest21505-21ssh09:25:311
349399315,7cyclictest12-21ksoftirqd/009:05:130
3493993119,8cyclictest131rcu_preempt08:15:130
351899303,19cyclictest0-21swapper/312:15:133
351099302,23cyclictest27-21ksoftirqd/209:30:152
3510993015,13cyclictest11076-21ssh11:10:142
350199302,22cyclictest32603-21ssh12:33:241
3493993017,10cyclictest630-21Async09:36:310
349399301,18cyclictest12-21ksoftirqd/009:00:130
3518992917,9cyclictest0-21swapper/310:10:133
351899291,20cyclictest0-21swapper/309:30:163
351099293,22cyclictest27-21ksoftirqd/209:20:132
3510992913,8cyclictest261rcuc/208:35:152
349399292,22cyclictest9378-21ssh10:15:130
351899282,12cyclictest35966irq/58-eth008:30:143
351899281,18cyclictest35966irq/58-eth011:25:153
351099284,17cyclictest9287-21ssh11:45:142
351099281,14cyclictest0-21swapper/211:01:402
350199282,20cyclictest16411-21ssh12:25:141
350199282,19cyclictest2213-21ntpq10:45:251
349399281,21cyclictest151rcuc/007:40:140
351899273,18cyclictest0-21swapper/312:33:253
351899273,16cyclictest0-21swapper/310:40:483
351899272,18cyclictest35966irq/58-eth010:25:133
351899272,16cyclictest35966irq/58-eth011:10:123
3510992710,13cyclictest27-21ksoftirqd/210:15:122
350199272,18cyclictest12680-21ssh10:50:521
350199272,18cyclictest12680-21ssh10:50:511
349399271,21cyclictest5666-21ssh11:25:160
351899263,20cyclictest35966irq/58-eth009:40:143
351099263,7cyclictest23259-21unixbench_singl08:00:292
3510992616,4cyclictest27-21ksoftirqd/207:45:012
351099261,20cyclictest0-21swapper/210:49:262
351099261,17cyclictest0-21swapper/211:22:012
351099261,16cyclictest16187-21ls08:30:172
351099261,15cyclictest2111766ssh10:20:232
351099261,14cyclictest131rcu_preempt10:57:482
350199261,8cyclictest0-21swapper/108:20:121
349399262,21cyclictest27065-21sh10:40:480
349399262,20cyclictest31143-21ssh10:08:260
349399262,20cyclictest31143-21ssh10:08:250
349399261,19cyclictest29336-21sh09:50:170
351899253,12cyclictest35966irq/58-eth011:15:153
351099255,6cyclictest1019-21irqrtprio10:10:172
351099252,14cyclictest5667-21ssh11:25:152
3510992514,5cyclictest27-21ksoftirqd/212:25:192
351099251,16cyclictest0-21swapper/211:31:092
351099251,16cyclictest0-21swapper/209:18:072
351099251,14cyclictest0-21swapper/209:12:212
3510992510,2cyclictest27-21ksoftirqd/212:05:262
349399253,18cyclictest5818-21ssh09:55:150
349399252,18cyclictest27493-21apt-get10:25:140
349399252,16cyclictest12-21ksoftirqd/007:30:140
349399252,10cyclictest12-21ksoftirqd/007:35:150
3493992512,2cyclictest131rcu_preempt10:55:130
351099248,13cyclictest28044-21grep10:05:312
351099248,13cyclictest28044-21grep10:05:302
351099246,13cyclictest27346irq/40-dwc2_hso07:25:132
3510992414,5cyclictest27-21ksoftirqd/211:37:382
3510992414,4cyclictest27-21ksoftirqd/212:11:592
3510992414,4cyclictest27-21ksoftirqd/211:55:582
351099241,17cyclictest0-21swapper/212:35:152
351099241,14cyclictest0-21swapper/209:58:382
350199247,3cyclictest211ksoftirqd/111:05:121
350199245,13cyclictest844-21munin-node07:10:181
350199243,3cyclictest131rcu_preempt10:55:141
350199241,15cyclictest11282-21ssh09:40:151
349399243,16cyclictest20043-21ssh11:15:150
351099237,10cyclictest131rcu_preempt10:32:592
351099233,2cyclictest131rcu_preempt09:36:362
351099232,17cyclictest27346irq/40-dwc2_hso08:50:132
351099232,15cyclictest0-21swapper/209:43:002
3510992319,2cyclictest27-21ksoftirqd/212:32:242
351099231,5cyclictest0-21swapper/212:22:472
351099231,5cyclictest0-21swapper/207:50:122
351099231,16cyclictest0-21swapper/209:27:392
3510992311,5cyclictest0-21swapper/210:52:062
3510992311,5cyclictest0-21swapper/210:52:052
351099231,15cyclictest0-21swapper/210:38:342
3510992310,4cyclictest23438-21cron10:05:012
3510992310,10cyclictest0-21swapper/208:20:242
3501992314,5cyclictest3891-21/usr/sbin/munin12:00:291
350199231,14cyclictest21-21ksoftirqd/111:45:131
349399232,16cyclictest151rcuc/011:10:130
351899228,7cyclictest0-21swapper/310:15:453
351899224,14cyclictest35966irq/58-eth009:05:123
3518992211,7cyclictest35966irq/58-eth010:00:163
351099227,7cyclictest5372-21cut11:05:222
351099225,15cyclictest27346irq/40-dwc2_hso07:35:152
351099225,14cyclictest3922-21ssh12:00:142
351099225,12cyclictest27346irq/40-dwc2_hso08:40:132
351099224,11cyclictest27346irq/40-dwc2_hso09:00:142
351099221,5cyclictest27581-21ntp_192.168.11408:05:252
3510992211,9cyclictest23905-21ssh11:51:252
351099221,15cyclictest0-21swapper/211:19:102
351099221,13cyclictest0-21swapper/207:15:272
351099221,10cyclictest3273466ssh11:40:142
3510992210,9cyclictest0-21swapper/209:05:142
351099220,12cyclictest15248-21kthreadcore07:20:212
350199224,14cyclictest201rcuc/109:15:131
3501992212,5cyclictest392-21sh10:10:141
349399223,12cyclictest0-21swapper/012:00:160
349399222,15cyclictest4214-21ps10:10:250
3493992214,5cyclictest12-21ksoftirqd/012:20:010
3493992214,3cyclictest12-21ksoftirqd/010:00:200
3493992211,3cyclictest131rcu_preempt10:30:300
3493992210,6cyclictest0-21swapper/008:55:260
351899216,9cyclictest35966irq/58-eth012:20:133
351899213,12cyclictest0-21swapper/310:48:253
3518992111,6cyclictest0-21swapper/310:35:173
351899211,14cyclictest35966irq/58-eth012:25:133
351099213,15cyclictest27346irq/40-dwc2_hso07:10:162
351099211,4cyclictest0-21swapper/210:27:332
3510992111,2cyclictest0-21swapper/208:10:142
3510992111,2cyclictest0-21swapper/208:10:132
351099211,11cyclictest131rcu_preempt07:30:132
3510992110,9cyclictest12112-21sshd08:58:262
351099210,2cyclictest131rcu_preempt07:45:242
350199218,10cyclictest20151-21df_inode11:15:161
350199214,6cyclictest25707-21fschecks_count08:40:161
350199211,13cyclictest5805-21cut11:25:171
350199210,15cyclictest1998-21ls09:35:021
349399219,3cyclictest12-21ksoftirqd/009:20:150
349399217,9cyclictest30400-21cron12:15:020
349399217,9cyclictest12-21ksoftirqd/008:10:140
349399217,9cyclictest12-21ksoftirqd/008:10:130
3493992114,4cyclictest12-21ksoftirqd/009:15:210
349399211,14cyclictest7126-21ssh09:35:330
349399211,14cyclictest18238-21apt-get11:50:120
349399211,14cyclictest12993-21ls07:50:220
349399210,18cyclictest24249-21ls08:05:150
351899209,8cyclictest0-21swapper/309:13:303
351099201,13cyclictest0-21swapper/208:26:072
351099200,11cyclictest131rcu_preempt08:15:182
350199202,13cyclictest27349-21ssh10:25:031
350199201,9cyclictest10230-21tr09:20:231
350199201,7cyclictest10593-21munin-run12:40:011
350199201,3cyclictest131rcu_preempt11:35:111
350199201,13cyclictest20276-21apt-get09:45:111
350199201,12cyclictest23091-21cut07:30:181
350199201,10cyclictest6270-21kthreadcore07:45:191
3501992010,5cyclictest21-21ksoftirqd/111:30:141
349399209,3cyclictest12-21ksoftirqd/011:05:210
349399208,3cyclictest131rcu_preempt11:55:210
349399207,7cyclictest12-21ksoftirqd/010:39:410
3493992014,3cyclictest12-21ksoftirqd/011:43:570
3493992013,5cyclictest0-21swapper/007:50:010
3493992013,3cyclictest12-21ksoftirqd/012:30:190
349399201,16cyclictest18424-21apt-get10:20:130
351899198,8cyclictest0-21swapper/310:32:473
351899196,10cyclictest0-21swapper/308:45:143
351899193,11cyclictest0-21swapper/311:51:253
351899192,10cyclictest0-21swapper/308:40:133
351899192,10cyclictest0-21swapper/308:10:053
351899192,10cyclictest0-21swapper/308:10:043
351899191,12cyclictest0-21swapper/308:35:293
3518991910,7cyclictest0-21swapper/311:20:303
3518991910,6cyclictest0-21swapper/309:30:003
351099191,9cyclictest30917-21cut08:45:202
351099190,13cyclictest14523-21ls07:55:142
350199198,3cyclictest131rcu_preempt12:15:261
350199195,3cyclictest131rcu_preempt09:55:191
350199194,11cyclictest0-21swapper/110:20:141
350199193,13cyclictest9353-21apt-get10:15:111
350199191,9cyclictest32272-21latency_hist07:40:011
350199191,6cyclictest13218-21latency_hist07:20:011
3501991911,5cyclictest7003-21sh10:30:251
350199191,10cyclictest21773-21kworker/1:211:01:171
350199190,17cyclictest22539-21cut09:10:181
349399199,2cyclictest12-21ksoftirqd/009:11:060
349399196,4cyclictest12-21ksoftirqd/008:05:010
349399192,4cyclictest131rcu_preempt08:45:010
349399192,13cyclictest21575-21ntp_states07:25:250
3493991913,4cyclictest0-21swapper/012:25:260
3493991913,4cyclictest0-21swapper/011:30:240
3493991913,3cyclictest12-21ksoftirqd/010:45:180
3493991912,4cyclictest12-21ksoftirqd/009:40:490
3493991911,6cyclictest1-21systemd08:35:270
3493991911,5cyclictest19854-21ssh09:25:220
349399191,14cyclictest0-21swapper/007:55:130
3493991910,6cyclictest26684-21cat11:00:210
351899189,6cyclictest0-21swapper/309:20:173
351899183,10cyclictest0-21swapper/311:08:263
351899182,12cyclictest0-21swapper/307:10:153
351899182,11cyclictest35966irq/58-eth010:05:163
351899182,11cyclictest35966irq/58-eth010:05:163
351899182,10cyclictest0-21swapper/308:20:133
351899182,10cyclictest0-21swapper/307:50:143
351899182,10cyclictest0-21swapper/307:30:153
351899182,10cyclictest0-21swapper/307:15:153
351899181,12cyclictest0-21swapper/309:15:253
351899181,12cyclictest0-21swapper/307:25:013
351899181,10cyclictest0-21swapper/308:50:223
3518991810,6cyclictest0-21swapper/312:00:083
3518991810,6cyclictest0-21swapper/309:35:323
351099181,8cyclictest3573-21grep07:10:022
350199188,8cyclictest1527-21df_inode08:15:151
350199185,3cyclictest131rcu_preempt08:00:231
350199184,12cyclictest0-21swapper/112:05:191
350199183,2cyclictest131rcu_preempt07:40:141
350199181,9cyclictest15389-21kthreadcore09:00:231
350199181,9cyclictest0-21swapper/111:14:091
350199181,5cyclictest0-21swapper/111:57:251
350199181,4cyclictest21731-21diskmemload11:40:201
350199181,10cyclictest21797-21kworker/1:110:40:201
3501991810,6cyclictest7325-21sshd08:53:261
350199180,12cyclictest17997-21ls07:25:021
349399184,10cyclictest7297-21acpi12:20:120
3493991812,3cyclictest12-21ksoftirqd/008:25:180
3493991811,4cyclictest12-21ksoftirqd/008:50:170
3493991811,4cyclictest12-21ksoftirqd/007:25:010
3493991810,6cyclictest1964-21sshd08:45:310
351899179,6cyclictest0-21swapper/312:35:393
351899179,5cyclictest35966irq/58-eth011:55:123
351899172,9cyclictest0-21swapper/307:35:163
351899172,12cyclictest0-21swapper/308:00:133
351899172,10cyclictest0-21swapper/307:45:133
350199179,6cyclictest14227-21sshd07:53:261
350199179,5cyclictest0-21swapper/109:05:101
350199173,5cyclictest28160-21rm12:12:191
350199173,2cyclictest131rcu_preempt08:30:291
350199172,10cyclictest0-21swapper/108:25:151
350199171,8cyclictest8206-21uname08:55:191
350199171,7cyclictest464-21ntp_192.168.11409:50:301
350199171,7cyclictest21744-21tr08:35:211
350199171,4cyclictest53646usb-storage08:10:111
350199171,4cyclictest53646usb-storage08:10:101
3501991711,4cyclictest0-21swapper/107:25:171
3493991712,3cyclictest19429-21ntp_states08:30:260
3493991711,3cyclictest12-21ksoftirqd/007:15:170
351899161,7cyclictest0-21swapper/307:45:013
350199162,4cyclictest131rcu_preempt08:05:171
349399163,10cyclictest6974-21wc07:10:250
350199151,11cyclictest14471-21cpu07:55:121
350199150,6cyclictest3538-21latency_hist07:10:011
349399141,9cyclictest3492-21wc07:10:020
351899131,6cyclictest0-21swapper/308:55:243
351899131,10cyclictest0-21swapper/308:25:213
351899123,6cyclictest0-21swapper/307:30:033
351899121,6cyclictest0-21swapper/308:05:223
351899111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional