You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 18:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Tue Apr 14, 2026 12:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22403996547,13cyclictest35966irq/58-eth011:30:133
22403996547,13cyclictest35966irq/58-eth011:30:133
2239599642,40cyclictest24014-21ssh12:15:132
2237999563,46cyclictest17025-21ssh09:50:120
2238199543,40cyclictest14750-21ssh12:10:131
2238199492,38cyclictest17671-21ssh09:15:141
2238199483,34cyclictest9863-21sshd12:25:131
2237999483,37cyclictest17656-21ssh09:15:120
2237999472,40cyclictest20970-21ssh10:45:140
2237999467,26cyclictest12-21ksoftirqd/011:55:130
2240399449,23cyclictest35966irq/58-eth012:10:123
2238199431,36cyclictest24717-21ssh11:40:151
2237999433,7cyclictest131rcu_preempt11:25:150
2237999433,7cyclictest131rcu_preempt09:30:120
2237999433,7cyclictest131rcu_preempt09:30:120
2237999431,33cyclictest0-21swapper/008:05:130
22395994225,10cyclictest7804-21ssh09:45:132
2237999422,4cyclictest131rcu_preempt09:35:150
2237999422,4cyclictest131rcu_preempt09:35:150
2237999411,34cyclictest19039-21ssh12:30:140
22395993915,12cyclictest0-21swapper/211:30:132
22395993915,12cyclictest0-21swapper/211:30:122
2239599391,27cyclictest8095-21diskmemload11:50:132
2238199391,27cyclictest7421-21ssh10:20:121
2238199383,29cyclictest29015-21ssh12:00:131
2239599372,18cyclictest3827-21ssh12:03:242
22381993729,4cyclictest0-21swapper/108:50:141
2240399362,24cyclictest35966irq/58-eth007:50:123
2240399362,24cyclictest35966irq/58-eth007:50:123
2238199362,22cyclictest15475-21ssh11:35:151
2240399351,19cyclictest35966irq/58-eth007:30:133
2240399344,18cyclictest35966irq/58-eth011:50:133
2240399342,23cyclictest35966irq/58-eth010:05:123
2239599342,22cyclictest8499-21ssh09:10:132
2240399332,21cyclictest35966irq/58-eth007:20:133
2240399332,21cyclictest35966irq/58-eth007:20:133
2239599332,24cyclictest9881-21ssh12:25:142
22379993314,8cyclictest12-21ksoftirqd/010:10:130
22403993215,9cyclictest35966irq/58-eth009:35:153
22403993215,9cyclictest35966irq/58-eth009:35:143
2239599322,15cyclictest1605-21awk11:10:012
2237999322,22cyclictest19958-21ssh11:20:010
2237999322,19cyclictest1465-21ssh11:45:140
22395993110,15cyclictest0-21swapper/210:34:462
2239599304,19cyclictest0-21swapper/208:25:102
2239599302,21cyclictest10519-21ntp_192.168.11410:55:242
22395993013,9cyclictest27-21ksoftirqd/210:28:032
2239599301,21cyclictest11738-21ssh10:40:152
2239599301,21cyclictest11738-21ssh10:40:152
2238199302,21cyclictest10625-21ssh11:50:121
2240399292,17cyclictest35966irq/58-eth012:00:123
2239599292,22cyclictest5087-21ssh10:53:242
22395992912,4cyclictest27-21ksoftirqd/209:39:072
22395992912,4cyclictest27-21ksoftirqd/209:39:062
2238199292,19cyclictest0-21swapper/109:10:131
2237999298,8cyclictest0-21swapper/012:20:130
2237999293,16cyclictest6913-21ssh10:55:130
2240399282,14cyclictest35966irq/58-eth008:20:123
22403992814,10cyclictest35966irq/58-eth009:20:263
2239599284,16cyclictest27346irq/40-dwc2_hso07:50:132
2239599284,16cyclictest27346irq/40-dwc2_hso07:50:132
2239599283,19cyclictest0-21swapper/211:21:252
2239599282,21cyclictest10798-21ssh11:15:022
2239599282,18cyclictest27346irq/40-dwc2_hso11:40:162
2239599281,19cyclictest0-21swapper/207:40:142
2239599281,16cyclictest27346irq/40-dwc2_hso08:55:152
2239599281,16cyclictest27346irq/40-dwc2_hso08:55:152
22395992811,5cyclictest0-21swapper/211:49:192
2237999283,20cyclictest0-21swapper/011:43:240
22379992814,5cyclictest131rcu_preempt09:40:150
2240399272,13cyclictest0-21swapper/309:10:133
2239599278,8cyclictest27-21ksoftirqd/210:37:342
2239599278,13cyclictest27-21ksoftirqd/212:23:222
2239599277,10cyclictest0-21swapper/209:56:232
2238199276,9cyclictest21-21ksoftirqd/108:15:021
22381992710,8cyclictest0-21swapper/111:28:311
2240399262,18cyclictest0-21swapper/311:10:243
2240399262,14cyclictest35966irq/58-eth012:25:133
2239599264,9cyclictest3577-21ssh09:25:152
2239599264,17cyclictest27346irq/40-dwc2_hso08:45:122
2239599263,17cyclictest27346irq/40-dwc2_hso07:55:122
2238199269,4cyclictest21-21ksoftirqd/110:53:511
2238199269,11cyclictest0-21swapper/109:29:471
2238199268,8cyclictest21-21ksoftirqd/110:17:461
2238199262,18cyclictest8499-21ssh11:12:241
22381992613,7cyclictest0-21swapper/109:38:071
22381992613,7cyclictest0-21swapper/109:38:071
22381992613,7cyclictest0-21swapper/109:33:511
22381992613,7cyclictest0-21swapper/109:33:501
2238199261,17cyclictest26279-21ssh09:55:131
2237999268,3cyclictest131rcu_preempt09:20:290
22379992615,8cyclictest12-21ksoftirqd/012:05:130
22379992612,11cyclictest10601-21cat11:30:280
22379992612,11cyclictest10601-21cat11:30:280
2240399258,11cyclictest0-21swapper/310:53:233
2240399252,16cyclictest35966irq/58-eth012:15:133
2239599258,4cyclictest27-21ksoftirqd/211:56:212
2239599254,13cyclictest27346irq/40-dwc2_hso10:05:132
2239599252,18cyclictest16809-21ssh11:16:242
2239599252,17cyclictest0-21swapper/208:00:142
2239599252,17cyclictest0-21swapper/208:00:132
22395992512,6cyclictest614-21dbus-daemon08:54:352
2239599251,20cyclictest16882-21kthreadcore12:10:192
22395992511,4cyclictest27-21ksoftirqd/210:04:082
2238199259,8cyclictest21-21ksoftirqd/109:43:451
2238199257,5cyclictest21-21ksoftirqd/111:09:421
2238199256,10cyclictest21-21ksoftirqd/109:50:451
2238199253,17cyclictest24019-21ssh12:15:131
22381992511,4cyclictest0-21swapper/111:16:241
22381992510,10cyclictest21-21ksoftirqd/107:50:141
22381992510,10cyclictest21-21ksoftirqd/107:50:131
2237999257,3cyclictest131rcu_preempt09:25:120
2237999254,4cyclictest9850-21apt-get12:25:120
2237999253,17cyclictest25324-21ssh11:05:130
2237999251,19cyclictest2488-21ssh10:35:140
2239599249,8cyclictest27-21ksoftirqd/209:23:152
2239599248,8cyclictest27-21ksoftirqd/209:15:362
2239599248,7cyclictest27-21ksoftirqd/211:28:162
2239599248,12cyclictest16377-21fschecks_time11:00:162
2239599242,17cyclictest2090-21ssh12:37:232
2239599241,9cyclictest27-21ksoftirqd/210:45:492
22395992414,4cyclictest27-21ksoftirqd/209:30:202
22395992414,4cyclictest27-21ksoftirqd/209:30:192
22395992412,10cyclictest26782-21ssh10:10:592
22395992412,10cyclictest23851-21ssh12:30:322
22395992411,4cyclictest27-21ksoftirqd/211:36:152
2238199249,4cyclictest21-21ksoftirqd/110:55:411
2238199249,4cyclictest21-21ksoftirqd/110:38:041
2238199249,11cyclictest21-21ksoftirqd/110:41:251
2238199249,11cyclictest21-21ksoftirqd/110:41:251
2238199248,4cyclictest21-21ksoftirqd/111:46:561
2238199242,17cyclictest8095-21diskmemload12:30:231
2238199241,19cyclictest8407-21kthreadcore08:00:221
2238199241,19cyclictest8407-21kthreadcore08:00:221
22381992410,5cyclictest0-21swapper/112:06:371
2237999244,12cyclictest22378-21cyclictest10:30:150
2237999242,17cyclictest1-21systemd11:35:030
2237999242,15cyclictest23909-21memory11:20:250
2237999241,4cyclictest30657-21ssh10:15:140
22379992412,9cyclictest17246-21cron09:15:020
22403992315,6cyclictest0-21swapper/311:18:063
2240399231,13cyclictest35966irq/58-eth010:25:133
2239599239,3cyclictest131rcu_preempt09:05:252
2239599237,13cyclictest4334-21idleruntime-cro07:25:012
2239599237,13cyclictest4334-21idleruntime-cro07:25:012
2239599236,7cyclictest131rcu_preempt10:19:232
2239599234,11cyclictest10379-21kthreadcore07:30:192
2239599232,11cyclictest11154-21irqrtprio08:05:182
2239599231,13cyclictest7434-21sh10:20:122
22395992310,11cyclictest7145-21ssh09:44:442
2238199239,4cyclictest21-21ksoftirqd/112:00:001
2238199239,4cyclictest21-21ksoftirqd/108:40:291
2238199238,4cyclictest21-21ksoftirqd/109:22:001
2238199237,7cyclictest21-21ksoftirqd/110:25:041
2238199237,5cyclictest0-21swapper/109:46:361
2238199236,3cyclictest131rcu_preempt11:30:201
2238199236,3cyclictest131rcu_preempt11:30:191
2238199235,9cyclictest21-21ksoftirqd/110:35:011
22381992316,3cyclictest0-21swapper/107:40:141
22381992312,3cyclictest0-21swapper/110:03:171
2238199231,19cyclictest31859-21sendmail_mailqu12:35:291
2238199231,17cyclictest16873-21ls08:45:131
22381992311,3cyclictest0-21swapper/110:07:031
2237999235,11cyclictest630-21Async09:36:310
22379992311,9cyclictest16524-21grep11:00:170
2237999231,17cyclictest1141-21cut07:55:160
2240399223,14cyclictest35966irq/58-eth010:40:143
2240399223,14cyclictest35966irq/58-eth010:40:133
2240399223,13cyclictest0-21swapper/311:10:013
2239599225,9cyclictest4546-21sort07:25:122
2239599224,8cyclictest27-21ksoftirqd/209:51:212
2239599224,10cyclictest21418-21kthreadcore08:15:212
2239599224,10cyclictest21418-21kthreadcore08:15:212
2239599222,6cyclictest0-21swapper/209:00:132
22395992212,8cyclictest0-21swapper/212:05:432
2238199227,3cyclictest131rcu_preempt08:20:131
2238199225,3cyclictest131rcu_preempt11:24:521
2238199222,12cyclictest24564-21ntp_192.168.11410:45:241
22381992212,7cyclictest31960-21cron07:20:001
22381992210,4cyclictest21-21ksoftirqd/109:00:241
2237999222,14cyclictest14233-21ssh11:50:230
22379992214,5cyclictest0-21swapper/007:10:260
2237999221,16cyclictest32003-21ls12:35:290
2237999221,15cyclictest2027-21rm11:10:140
2237999220,18cyclictest32126-21ls07:20:020
2240399219,6cyclictest36066irq/59-eth009:45:143
2240399217,10cyclictest35966irq/58-eth011:40:163
2240399214,12cyclictest0-21swapper/311:35:153
22403992114,5cyclictest0-21swapper/311:21:103
2239599216,2cyclictest131rcu_preempt07:45:222
2239599216,2cyclictest131rcu_preempt07:45:222
2239599215,13cyclictest12351-21grep08:40:172
2239599213,15cyclictest15196-21kthreadcore07:35:212
2239599212,9cyclictest27443-21cpu07:15:122
2239599212,9cyclictest2410-21cpu08:30:162
2239599211,16cyclictest25948-21missed_timers07:10:222
2239599211,16cyclictest25349-21hwlatdetect08:20:182
2239599211,11cyclictest8207-21kthreadcore08:35:202
2239599211,10cyclictest18996-21cat08:10:282
2238199217,4cyclictest21-21ksoftirqd/111:02:471
2238199217,4cyclictest21-21ksoftirqd/108:35:161
2238199213,4cyclictest21-21ksoftirqd/110:10:431
2238199213,15cyclictest30740-21usage_EthcTask08:55:321
2238199213,15cyclictest30740-21usage_EthcTask08:55:311
2238199211,17cyclictest4631-21kthreadcore09:05:201
2238199211,17cyclictest1004-21fschecks_time12:20:171
2238199211,16cyclictest25823-21kthreadcore07:45:201
2238199211,16cyclictest25823-21kthreadcore07:45:191
2237999211,7cyclictest24778-21kthreadcore12:15:210
2237999211,16cyclictest26355-21apt-get08:55:140
2237999211,16cyclictest26355-21apt-get08:55:130
2237999211,16cyclictest23913-21cut07:45:160
2237999211,16cyclictest23913-21cut07:45:150
2237999211,14cyclictest16692-21cpuspeed10:25:150
2240399209,6cyclictest0-21swapper/310:00:173
2240399202,11cyclictest0-21swapper/308:50:153
2240399202,10cyclictest0-21swapper/308:40:133
22403992011,7cyclictest0-21swapper/310:10:173
22403992010,6cyclictest35966irq/58-eth010:45:143
2238199207,4cyclictest21-21ksoftirqd/107:30:261
2238199203,9cyclictest21-21ksoftirqd/107:15:011
2238199203,4cyclictest21-21ksoftirqd/108:15:261
2238199203,4cyclictest21-21ksoftirqd/108:15:261
2238199202,15cyclictest14505-21irqcore07:35:191
2238199202,11cyclictest0-21swapper/108:25:111
2238199201,16cyclictest7133-21latency_hist08:35:021
2238199201,16cyclictest3140-21kthreadcore07:55:211
2237999205,7cyclictest131rcu_preempt10:50:130
2237999203,9cyclictest11049-21sed08:35:300
22379992012,4cyclictest0-21swapper/008:10:140
2237999201,14cyclictest21292-21ssh12:12:230
2237999201,14cyclictest12017-21apt-get08:40:110
2237999201,13cyclictest1153-21switchtime08:25:260
2240399192,11cyclictest0-21swapper/308:00:143
2240399192,11cyclictest0-21swapper/308:00:133
22403991913,4cyclictest0-21swapper/310:33:313
22403991912,5cyclictest0-21swapper/310:20:263
22403991911,5cyclictest35966irq/58-eth009:55:143
22403991911,5cyclictest0-21swapper/311:25:143
2240399191,12cyclictest35966irq/58-eth009:15:143
2240399191,12cyclictest0-21swapper/312:35:163
2238199192,14cyclictest10815-21fschecks_time08:05:161
2238199192,10cyclictest844-21munin-node07:20:141
2238199192,10cyclictest844-21munin-node07:20:131
2238199191,15cyclictest5063-21irqrtprio07:25:191
2237999199,8cyclictest16200-21cron10:25:020
2237999196,10cyclictest26545-21grep09:55:170
2237999192,15cyclictest29673-21sh08:25:000
2237999191,15cyclictest20252-21ntp_states08:45:260
2237999191,14cyclictest7822-21sh09:45:150
2237999191,14cyclictest19115-21tune2fs07:40:160
2237999191,13cyclictest15983-21ssh10:05:260
2237999191,10cyclictest15361-21ntp_192.168.11410:40:250
2237999191,10cyclictest15361-21ntp_192.168.11410:40:240
2240399182,10cyclictest0-21swapper/309:05:143
2240399182,10cyclictest0-21swapper/308:45:143
22403991812,4cyclictest0-21swapper/312:30:383
22403991812,4cyclictest0-21swapper/312:20:193
22403991812,4cyclictest0-21swapper/310:55:153
22403991812,4cyclictest0-21swapper/310:15:253
22403991812,4cyclictest0-21swapper/309:50:173
22403991812,4cyclictest0-21swapper/309:40:263
22403991811,5cyclictest0-21swapper/312:07:043
22403991811,5cyclictest0-21swapper/311:00:123
2240399181,13cyclictest0-21swapper/311:55:143
2240399181,13cyclictest0-21swapper/309:25:303
22403991810,5cyclictest35966irq/58-eth011:45:143
22403991810,3cyclictest0-21swapper/310:36:223
2237999189,6cyclictest10041-21kthreadcore07:30:190
2237999186,3cyclictest131rcu_preempt07:50:210
2237999186,3cyclictest131rcu_preempt07:50:200
2237999183,9cyclictest0-21swapper/012:00:070
2237999182,10cyclictest0-21swapper/007:25:130
2237999181,9cyclictest3857-21irqcore09:05:180
2237999181,14cyclictest24422-21cut08:50:210
2240399172,10cyclictest0-21swapper/308:55:153
2240399172,10cyclictest0-21swapper/308:55:143
2240399172,10cyclictest0-21swapper/307:55:143
22403991711,4cyclictest0-21swapper/309:30:133
22403991711,4cyclictest0-21swapper/309:30:133
22403991711,4cyclictest0-21swapper/308:25:143
22403991710,4cyclictest0-21swapper/307:45:173
22403991710,4cyclictest0-21swapper/307:45:173
22403991710,4cyclictest0-21swapper/307:40:253
2239599173,11cyclictest0-21swapper/207:10:022
2237999177,7cyclictest3409-21cat07:20:280
2237999177,7cyclictest3409-21cat07:20:270
2240399169,4cyclictest0-21swapper/307:35:163
22403991611,3cyclictest0-21swapper/308:35:203
22403991611,3cyclictest0-21swapper/308:30:283
22403991611,3cyclictest0-21swapper/307:10:023
22403991610,4cyclictest0-21swapper/307:25:143
2237999162,11cyclictest6396-21irqrtprio08:00:180
2237999162,11cyclictest6396-21irqrtprio08:00:180
2237999161,12cyclictest4195-21kthreadcore10:00:200
2237999161,12cyclictest2407-21apt-get08:30:140
2237999161,12cyclictest23266-21tr08:15:230
2237999161,12cyclictest23266-21tr08:15:230
2240399159,4cyclictest0-21swapper/307:10:193
2240399151,8cyclictest0-21swapper/308:10:173
2240399151,11cyclictest0-21swapper/308:15:233
2240399151,11cyclictest0-21swapper/308:15:223
22403991510,3cyclictest0-21swapper/309:00:243
2238199151,10cyclictest22427-21latency_hist07:10:021
2237999151,10cyclictest31594-21irqcore09:00:170
2240399149,3cyclictest0-21swapper/308:08:233
2240399148,4cyclictest0-21swapper/307:15:163
2237999141,10cyclictest22557-21latency_hist07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional