You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 18:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Apr 30, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20078997540,25cyclictest35966irq/58-eth009:50:133
20078997540,25cyclictest35966irq/58-eth009:50:123
2007099542,29cyclictest4770-21ssh11:35:142
2004799503,36cyclictest24368-21ssh11:10:130
2007099491,39cyclictest9579-21ssh09:50:112
2007099491,39cyclictest9579-21ssh09:50:112
20078994729,12cyclictest35966irq/58-eth012:30:123
2007099473,33cyclictest16936-21ssh10:30:132
2005899473,4cyclictest131rcu_preempt11:30:121
2007899454,24cyclictest35966irq/58-eth010:55:143
2007099437,29cyclictest131rcu_preempt11:30:122
2007899412,25cyclictest35966irq/58-eth010:30:123
2005899412,29cyclictest31290-21ssh10:20:101
2005899411,33cyclictest13820-21ssh11:40:111
20058994014,9cyclictest21-21ksoftirqd/112:05:121
2007899392,22cyclictest35966irq/58-eth012:35:133
2007899382,20cyclictest35966irq/58-eth010:25:133
2007099362,28cyclictest3662-21ssh10:04:462
2007899352,20cyclictest35966irq/58-eth010:50:123
2007099353,9cyclictest0-21swapper/212:13:462
2007099343,7cyclictest131rcu_preempt12:30:102
2007099335,19cyclictest27346irq/40-dwc2_hso08:55:112
2007099335,19cyclictest27346irq/40-dwc2_hso08:55:102
2004799332,6cyclictest12-21ksoftirqd/009:25:140
2007099324,21cyclictest27346irq/40-dwc2_hso08:10:122
2005899322,19cyclictest30251-21ssh12:25:111
2004799325,20cyclictest20749-21ssh10:50:120
2007899314,15cyclictest35966irq/58-eth009:45:113
2007899304,19cyclictest35966irq/58-eth010:05:273
20078993015,9cyclictest35966irq/58-eth012:05:123
20078993014,8cyclictest35966irq/58-eth007:35:113
2007099302,11cyclictest6864-21ssh11:16:462
20070993017,8cyclictest261rcuc/211:05:462
2005899302,19cyclictest6408-21ssh11:00:131
2007899295,13cyclictest0-21swapper/309:35:123
2007899293,15cyclictest35966irq/58-eth011:00:133
2007899292,19cyclictest35966irq/58-eth008:15:133
2007099294,17cyclictest27346irq/40-dwc2_hso12:15:102
2007099292,19cyclictest27346irq/40-dwc2_hso07:45:112
2007099292,11cyclictest18432-21ssh10:10:462
2007099290,24cyclictest0-21swapper/210:25:132
2005899292,22cyclictest5895-21ssh09:30:111
20058992916,5cyclictest20997-21kthreadcore09:55:211
2005899291,19cyclictest10005-21ssh11:20:121
2005899291,16cyclictest21-21ksoftirqd/110:40:121
2004799291,7cyclictest12-21ksoftirqd/011:15:130
2007899281,17cyclictest35966irq/58-eth007:25:143
2007099284,16cyclictest27346irq/40-dwc2_hso07:35:132
2007099283,19cyclictest1379-21diskmemload11:27:462
2007099283,19cyclictest1379-21diskmemload11:27:462
2007099283,18cyclictest27346irq/40-dwc2_hso11:50:142
2007099283,17cyclictest27346irq/40-dwc2_hso10:45:132
2007099282,13cyclictest17528-21ssh12:00:132
2007099281,21cyclictest4477-21ssh10:05:282
2007099281,12cyclictest21195-21rm12:20:102
2004799282,20cyclictest24546-21ssh09:56:460
2007899272,16cyclictest0-21swapper/310:10:403
20078992718,6cyclictest35966irq/58-eth012:10:143
2007099273,19cyclictest0-21swapper/211:43:462
2007099273,19cyclictest0-21swapper/210:22:472
2007099273,17cyclictest27346irq/40-dwc2_hso09:20:182
2007099273,17cyclictest27346irq/40-dwc2_hso07:20:122
2007099272,20cyclictest29839-21ssh10:18:462
2007099272,20cyclictest29839-21ssh10:18:452
2007099272,18cyclictest27346irq/40-dwc2_hso08:40:122
2007099272,15cyclictest27346irq/40-dwc2_hso09:35:122
2007099271,21cyclictest930-21snmpd12:05:132
2007099271,18cyclictest27346irq/40-dwc2_hso08:35:122
2007099271,11cyclictest1379-21diskmemload09:25:152
2007099270,2cyclictest131rcu_preempt11:00:132
2005899272,19cyclictest1135-21ssh10:01:461
20058992711,10cyclictest13421-21sh10:10:161
2004799279,3cyclictest0-21swapper/011:02:200
2004799278,7cyclictest131rcu_preempt11:08:470
2004799272,19cyclictest20581-21ssh12:19:470
20047992712,3cyclictest0-21swapper/009:45:090
20047992711,3cyclictest0-21swapper/011:33:450
2007899268,12cyclictest0-21swapper/309:40:463
2007899263,15cyclictest0-21swapper/309:56:453
2007899263,14cyclictest0-21swapper/311:30:123
2007899262,17cyclictest0-21swapper/311:27:473
2007899262,17cyclictest0-21swapper/311:27:463
2007899261,15cyclictest35966irq/58-eth011:40:133
2007099262,18cyclictest27346irq/40-dwc2_hso08:05:122
20070992611,11cyclictest1-21systemd10:40:472
2005899268,12cyclictest131rcu_preempt10:30:081
2005899266,6cyclictest24415-21kthreadcore11:45:211
2005899261,19cyclictest6134-21ssh11:15:531
20058992610,5cyclictest131rcu_preempt09:45:121
2004799262,3cyclictest131rcu_preempt11:45:120
2004799262,18cyclictest22465-21sh09:38:450
2007099252,8cyclictest31525-21ssh09:43:462
2007099252,18cyclictest8986-21ssh09:49:472
2007099251,10cyclictest0-21swapper/208:30:142
2005899254,12cyclictest73-21kswapd011:35:141
2005899252,19cyclictest24375-21sh11:10:141
2004799259,3cyclictest12-21ksoftirqd/009:40:130
2004799257,3cyclictest131rcu_preempt09:15:130
2004799255,14cyclictest19116-21ssh11:25:130
2004799255,14cyclictest19116-21ssh11:25:120
20047992513,3cyclictest12-21ksoftirqd/012:12:050
20047992511,11cyclictest12-21ksoftirqd/008:10:110
2007899248,10cyclictest35966irq/58-eth011:15:543
2007099244,16cyclictest10884-21ssh09:15:122
2007099242,17cyclictest0-21swapper/209:59:462
2007099241,15cyclictest27346irq/40-dwc2_hso11:10:112
2005899248,12cyclictest21-21ksoftirqd/110:06:461
2005899246,11cyclictest21-21ksoftirqd/108:05:141
2005899241,6cyclictest10192-21ssh10:25:211
2005899241,17cyclictest15912-21ssh12:35:131
20058992410,10cyclictest21-21ksoftirqd/110:39:461
2004799248,3cyclictest131rcu_preempt08:45:130
2004799248,10cyclictest12-21ksoftirqd/008:15:110
2004799242,18cyclictest13418-21ssh10:10:150
20047992410,3cyclictest12-21ksoftirqd/009:05:230
2007899231,15cyclictest35966irq/58-eth011:35:143
2007099231,4cyclictest0-21swapper/207:40:122
20070992312,8cyclictest0-21swapper/209:00:182
2007099231,12cyclictest10410-21ssh11:20:152
2005899236,9cyclictest21472-21kthreadcore08:55:181
2005899236,9cyclictest21472-21kthreadcore08:55:171
2005899231,8cyclictest0-21swapper/112:30:441
2005899231,7cyclictest131rcu_preempt11:06:051
2005899231,7cyclictest0-21swapper/110:59:521
2005899231,5cyclictest29856-21cron12:25:021
2005899231,3cyclictest131rcu_preempt12:15:171
20058992313,4cyclictest8573-21tail08:35:281
2005899231,17cyclictest10723-21ssh12:13:451
20058992311,6cyclictest131rcu_preempt08:10:161
2004799239,3cyclictest0-21swapper/011:41:350
2004799237,4cyclictest12-21ksoftirqd/010:33:500
20047992310,3cyclictest12-21ksoftirqd/010:38:030
20047992310,3cyclictest12-21ksoftirqd/010:02:480
2007899223,13cyclictest0-21swapper/311:11:463
2007899223,12cyclictest0-21swapper/310:40:133
2007899223,10cyclictest0-21swapper/312:00:143
2007099228,11cyclictest2313-21sed09:10:172
2007099222,16cyclictest0-21swapper/210:38:452
2007099222,15cyclictest0-21swapper/212:26:472
2007099222,11cyclictest27658-21kthreadcore07:15:192
2007099221,18cyclictest27490-21find08:25:112
2005899229,6cyclictest0-21swapper/110:17:181
2005899229,6cyclictest0-21swapper/110:17:171
2005899229,6cyclictest0-21swapper/109:19:041
2005899223,13cyclictest0-21swapper/109:42:081
20058992213,3cyclictest0-21swapper/107:45:261
2005899221,15cyclictest31971-21ssh11:50:131
2005899221,15cyclictest0-21swapper/109:12:141
20058992211,2cyclictest131rcu_preempt12:01:161
20058992210,2cyclictest131rcu_preempt11:25:461
20058992210,2cyclictest131rcu_preempt11:25:451
2004799229,2cyclictest12-21ksoftirqd/009:00:220
2004799228,5cyclictest131rcu_preempt07:10:090
2004799224,3cyclictest131rcu_preempt11:35:200
2004799223,14cyclictest167-21systemd-network08:25:130
2004799222,4cyclictest0-21swapper/008:35:130
2004799222,15cyclictest3399-21kthreadcore10:40:180
20047992216,3cyclictest30258-21sh12:25:140
20047992213,5cyclictest12-21ksoftirqd/012:35:200
20047992213,4cyclictest0-21swapper/012:31:190
20047992212,3cyclictest12-21ksoftirqd/012:21:400
20047992212,3cyclictest12-21ksoftirqd/011:24:010
2004799221,16cyclictest29786-21ls07:20:030
20047992211,3cyclictest12-21ksoftirqd/010:27:060
20047992210,3cyclictest131rcu_preempt10:20:470
20047992210,3cyclictest12-21ksoftirqd/007:55:110
2007899219,9cyclictest0-21swapper/312:15:243
2007899211,12cyclictest0-21swapper/310:00:123
2007099217,11cyclictest22804-21sshd08:20:172
2007099216,12cyclictest27-21ksoftirqd/212:35:122
2007099213,13cyclictest27346irq/40-dwc2_hso07:30:132
2007099213,13cyclictest27346irq/40-dwc2_hso07:30:122
2007099213,13cyclictest27346irq/40-dwc2_hso07:25:132
2007099212,9cyclictest0-21swapper/207:50:132
2007099212,6cyclictest0-21swapper/207:55:112
2007099212,15cyclictest0-21swapper/209:33:462
2007099211,11cyclictest22446-21cron08:20:012
2005899219,6cyclictest20859-21kworker/1:211:56:331
2005899219,3cyclictest21-21ksoftirqd/108:05:001
2005899212,14cyclictest0-21swapper/110:47:521
2005899211,4cyclictest0-21swapper/108:33:471
2005899211,3cyclictest0-21swapper/107:20:161
2005899211,14cyclictest0-21swapper/108:40:291
2004799219,3cyclictest12-21ksoftirqd/008:00:210
2004799217,5cyclictest12-21ksoftirqd/009:20:250
2004799214,3cyclictest131rcu_preempt11:55:200
2004799214,3cyclictest131rcu_preempt08:55:180
2004799214,3cyclictest131rcu_preempt08:55:170
2004799212,7cyclictest31677-21cut07:20:190
2004799212,5cyclictest15309-21ntpq10:45:260
20047992115,4cyclictest0-21swapper/010:05:290
20047992113,3cyclictest12-21ksoftirqd/012:05:100
20047992112,2cyclictest12-21ksoftirqd/008:20:220
20047992110,3cyclictest12-21ksoftirqd/010:19:130
20047992110,3cyclictest12-21ksoftirqd/010:19:120
20047992110,3cyclictest12-21ksoftirqd/009:51:180
20047992110,3cyclictest12-21ksoftirqd/009:51:170
2007899209,7cyclictest35966irq/58-eth011:45:133
2007899203,12cyclictest0-21swapper/311:08:473
2007899203,12cyclictest0-21swapper/310:15:133
2007899203,12cyclictest0-21swapper/310:15:123
20078992013,4cyclictest33-21ksoftirqd/312:20:143
2007099208,9cyclictest29778-21irqrtprio09:05:182
2007099203,9cyclictest23122-21kthreadcore07:10:182
2007099203,14cyclictest6958-21ntpq08:00:262
2007099201,16cyclictest0-21swapper/211:45:122
2007099200,13cyclictest24401-21grep10:50:242
2005899208,3cyclictest28942-21usage_EthcTask09:00:311
2005899201,8cyclictest20810-21ssh09:20:231
2005899201,7cyclictest9568-21apt-get09:50:121
2005899201,7cyclictest9568-21apt-get09:50:111
2005899201,5cyclictest23431-21kthreadcore08:20:191
2005899201,4cyclictest22881-21rm09:39:141
2005899201,12cyclictest0-21swapper/110:50:451
2005899200,6cyclictest6801-21munin-run07:30:021
2004799209,9cyclictest17516-21apt-get12:00:130
2004799209,3cyclictest12-21ksoftirqd/010:55:270
2004799209,3cyclictest12-21ksoftirqd/007:50:290
2004799208,3cyclictest12-21ksoftirqd/007:40:200
2004799208,3cyclictest12-21ksoftirqd/007:25:250
2004799207,7cyclictest12-21ksoftirqd/009:32:290
2004799205,3cyclictest1379-21diskmemload11:53:020
20047992013,4cyclictest12-21ksoftirqd/008:05:290
20047992013,4cyclictest12-21ksoftirqd/007:45:250
2004799201,13cyclictest1762-21ssh09:10:110
20047992010,8cyclictest0-21swapper/007:35:260
2007899199,8cyclictest0-21swapper/309:25:563
2007899199,8cyclictest0-21swapper/309:23:203
2007899198,9cyclictest0-21swapper/311:54:453
2007899193,11cyclictest0-21swapper/311:22:463
2007899193,11cyclictest0-21swapper/310:35:133
20078991911,5cyclictest0-21swapper/310:20:103
2007899191,11cyclictest0-21swapper/308:25:273
2007099192,10cyclictest9931-21kthreadcore11:55:192
2007099191,9cyclictest19331-21latency_hist08:55:012
2007099191,4cyclictest844-21munin-node10:55:072
2007099191,14cyclictest14147-21apt-get08:45:042
2005899198,2cyclictest131rcu_preempt08:15:211
2005899195,4cyclictest0-21swapper/109:05:281
2005899191,9cyclictest0-21swapper/107:55:131
2005899191,7cyclictest0-21swapper/108:45:261
2005899191,15cyclictest11805-21find07:35:101
2005899191,10cyclictest0-21swapper/109:25:171
2005899190,16cyclictest21181-21latency_hist07:45:011
20047991913,3cyclictest12-21ksoftirqd/008:40:170
20047991913,3cyclictest12-21ksoftirqd/008:30:210
2007899189,6cyclictest0-21swapper/309:30:293
2007899189,6cyclictest0-21swapper/309:16:103
2007899183,8cyclictest0-21swapper/307:15:133
2007899183,10cyclictest0-21swapper/312:28:463
2007899182,10cyclictest0-21swapper/308:40:133
2007899182,10cyclictest0-21swapper/308:00:153
2007899182,10cyclictest0-21swapper/307:55:123
2007899182,10cyclictest0-21swapper/307:40:143
2007899182,10cyclictest0-21swapper/307:30:143
2007899182,10cyclictest0-21swapper/307:30:133
2007899181,9cyclictest0-21swapper/308:10:023
2007899181,13cyclictest0-21swapper/307:45:143
2007899181,12cyclictest0-21swapper/308:10:143
20078991810,6cyclictest0-21swapper/310:46:523
20078991810,6cyclictest0-21swapper/309:11:423
2007099186,9cyclictest19948-21latency_hist07:10:012
2005899185,4cyclictest23189-21kthreadcore07:10:181
2005899185,3cyclictest131rcu_preempt08:25:151
2005899182,2cyclictest131rcu_preempt07:50:201
20058991813,3cyclictest21-21ksoftirqd/107:30:161
20058991813,3cyclictest21-21ksoftirqd/107:30:151
20047991810,5cyclictest19280-21sshd08:54:470
2007899179,6cyclictest0-21swapper/311:55:043
2007899172,9cyclictest0-21swapper/308:30:133
2007899172,10cyclictest0-21swapper/309:00:133
2007899172,10cyclictest0-21swapper/308:35:143
2007899171,14cyclictest0-21swapper/307:50:123
2005899170,5cyclictest26111-21tr07:15:171
20047991712,3cyclictest12-21ksoftirqd/007:30:290
20047991712,3cyclictest12-21ksoftirqd/007:30:280
2007899169,4cyclictest0-21swapper/308:20:163
2007899161,12cyclictest0-21swapper/309:05:283
2007899161,12cyclictest0-21swapper/307:25:013
2005899161,4cyclictest0-21swapper/108:51:341
20078991510,3cyclictest0-21swapper/308:45:263
2007899141,6cyclictest0-21swapper/309:00:023
2007899141,6cyclictest0-21swapper/309:00:013
2005899141,10cyclictest20083-21latency_hist07:10:011
2004799141,10cyclictest20162-21latency_hist07:10:020
2007899111,8cyclictest0-21swapper/307:10:153
2007899111,7cyclictest0-21swapper/308:55:023
200789991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional