You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 00:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jun 12, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
783299474,29cyclictest35966irq/58-eth011:05:103
783299454,27cyclictest35966irq/58-eth010:45:123
783299433,26cyclictest35966irq/58-eth009:15:153
783299432,23cyclictest35966irq/58-eth007:25:143
782499426,28cyclictest898-21ssh11:20:132
7816994117,19cyclictest21-21ksoftirqd/111:35:121
7816994117,19cyclictest21-21ksoftirqd/111:35:121
780199414,2cyclictest131rcu_preempt08:30:130
7801994135,3cyclictest25794-21diskmemload09:45:140
782499401,31cyclictest29126-21ssh12:30:112
783299392,18cyclictest35966irq/58-eth010:10:133
783299391,23cyclictest35966irq/58-eth009:30:123
780199383,32cyclictest11288-21ssh12:20:130
783299373,19cyclictest35966irq/58-eth009:55:123
783299372,22cyclictest35966irq/58-eth012:25:113
781699370,30cyclictest0-21swapper/109:15:151
780199361,31cyclictest20211-21ssh12:25:130
780199360,3cyclictest131rcu_preempt12:35:250
783299342,21cyclictest35966irq/58-eth008:30:143
783299341,18cyclictest35966irq/58-eth010:40:113
782499343,17cyclictest26176-21ssh09:10:122
780199342,29cyclictest22423-21ssh11:50:130
783299332,20cyclictest35966irq/58-eth012:35:133
7816993324,2cyclictest536-21usb-storage10:10:121
783299322,16cyclictest35966irq/58-eth011:00:143
780199321,4cyclictest7979-21apt-get07:10:120
780199321,4cyclictest7979-21apt-get07:10:110
780199321,3cyclictest131rcu_preempt09:40:130
783299313,22cyclictest35966irq/58-eth011:20:133
783299311,19cyclictest35966irq/58-eth010:20:123
7824993117,4cyclictest25794-21diskmemload09:40:142
7824993111,12cyclictest13171-21kworker/u8:010:50:132
781699312,22cyclictest31903-21ssh10:41:261
781699311,19cyclictest18777-21ssh11:30:121
780199314,23cyclictest17299-21ssh10:35:120
780199311,26cyclictest26187-21ssh09:10:120
7801993110,3cyclictest131rcu_preempt11:10:140
783299301,15cyclictest35966irq/58-eth010:50:113
782499308,13cyclictest27-21ksoftirqd/211:45:272
782499303,19cyclictest0-21swapper/212:25:132
782499302,24cyclictest24351-21ssh11:15:112
781699308,16cyclictest131rcu_preempt12:20:141
781699302,22cyclictest24424-21awk10:55:261
781699302,22cyclictest1045-21/usr/sbin/munin11:00:261
780199308,4cyclictest131rcu_preempt09:55:130
780199307,4cyclictest131rcu_preempt12:30:140
780199302,7cyclictest12-21ksoftirqd/008:50:130
783299293,18cyclictest35966irq/58-eth009:10:123
783299293,17cyclictest35966irq/58-eth009:35:113
7824992910,13cyclictest0-21swapper/210:09:422
781699291,22cyclictest28096-21ssh10:05:131
780199297,17cyclictest13401-21ssh11:45:120
780199292,25cyclictest8013-21ssh12:00:140
780199292,25cyclictest8013-21ssh12:00:130
780199292,20cyclictest6404-21ssh09:35:110
782499285,3cyclictest131rcu_preempt11:00:242
782499284,16cyclictest27346irq/40-dwc2_hso10:40:122
782499284,11cyclictest27-21ksoftirqd/210:03:532
782499282,19cyclictest27346irq/40-dwc2_hso10:55:132
7824992810,5cyclictest0-21swapper/212:21:152
7824992810,12cyclictest0-21swapper/212:35:322
780199284,4cyclictest12-21ksoftirqd/010:20:100
780199281,23cyclictest20768-21ssh10:55:150
780199281,21cyclictest0-21swapper/011:20:130
7801992810,3cyclictest131rcu_preempt09:24:070
783299274,16cyclictest35966irq/58-eth008:20:133
783299274,12cyclictest35966irq/58-eth011:25:113
782499278,3cyclictest131rcu_preempt12:00:042
782499278,3cyclictest131rcu_preempt12:00:032
782499272,9cyclictest27-21ksoftirqd/208:45:122
7824992712,12cyclictest27-21ksoftirqd/211:05:162
7824992710,4cyclictest27-21ksoftirqd/207:30:172
781699276,14cyclictest131rcu_preempt07:35:141
781699276,14cyclictest131rcu_preempt07:35:131
7816992714,4cyclictest131rcu_preempt10:45:201
7816992713,11cyclictest3530-21kworker/1:211:05:111
7816992712,4cyclictest21-21ksoftirqd/112:05:231
780199277,3cyclictest131rcu_preempt11:25:330
780199276,2cyclictest131rcu_preempt11:15:100
7801992712,4cyclictest0-21swapper/010:12:110
7801992711,7cyclictest12-21ksoftirqd/012:16:500
782499264,7cyclictest131rcu_preempt12:11:202
782499264,14cyclictest27346irq/40-dwc2_hso07:15:132
782499262,11cyclictest0-21swapper/208:10:162
7824992616,8cyclictest27-21ksoftirqd/208:29:182
782499261,21cyclictest10160-21cut09:55:162
7824992611,7cyclictest27-21ksoftirqd/208:20:262
780199267,3cyclictest131rcu_preempt11:35:320
780199267,3cyclictest131rcu_preempt11:35:320
780199266,9cyclictest131rcu_preempt07:45:190
780199262,5cyclictest12-21ksoftirqd/008:05:140
780199261,7cyclictest12-21ksoftirqd/009:15:130
7801992614,8cyclictest0-21swapper/010:40:120
780199261,20cyclictest29883-21ssh09:30:140
783299259,8cyclictest0-21swapper/308:00:123
783299253,13cyclictest0-21swapper/309:25:053
782499259,13cyclictest27-21ksoftirqd/210:25:322
782499258,4cyclictest27-21ksoftirqd/210:38:282
782499258,3cyclictest0-21swapper/209:28:412
782499257,7cyclictest131rcu_preempt09:20:482
782499254,13cyclictest0-21swapper/209:45:142
782499253,16cyclictest31330-21ssh11:55:132
782499252,8cyclictest27-21ksoftirqd/211:36:002
782499252,8cyclictest27-21ksoftirqd/211:35:592
782499252,16cyclictest27346irq/40-dwc2_hso09:00:142
7824992513,5cyclictest12179-21ssh11:44:062
7824992513,5cyclictest12179-21ssh11:44:062
7816992516,5cyclictest4297-21/usr/sbin/munin11:40:281
7816992516,5cyclictest4297-21/usr/sbin/munin11:40:281
780199259,4cyclictest12-21ksoftirqd/012:15:020
780199259,2cyclictest12-21ksoftirqd/009:26:290
780199257,3cyclictest131rcu_preempt10:50:270
7801992512,7cyclictest0-21swapper/010:27:300
7801992510,7cyclictest12-21ksoftirqd/011:34:570
783299244,14cyclictest0-21swapper/309:20:143
783299243,14cyclictest35966irq/58-eth010:30:113
783299243,14cyclictest35966irq/58-eth010:30:113
782499249,6cyclictest131rcu_preempt11:32:282
782499249,12cyclictest2916-21df_inode10:45:142
782499248,3cyclictest131rcu_preempt10:22:562
782499247,3cyclictest131rcu_preempt09:30:222
782499246,7cyclictest131rcu_preempt08:35:132
782499244,7cyclictest131rcu_preempt10:31:212
782499244,7cyclictest131rcu_preempt10:31:202
782499244,17cyclictest28239-21munin-plugin-st08:05:012
782499244,13cyclictest27346irq/40-dwc2_hso07:55:112
782499242,11cyclictest0-21swapper/209:35:112
782499241,17cyclictest0-21swapper/211:10:122
782499241,14cyclictest27-21ksoftirqd/208:05:242
781699248,3cyclictest211ksoftirqd/108:30:121
781699248,11cyclictest21-21ksoftirqd/111:10:131
781699241,5cyclictest13402-21apt-get11:45:121
7816992412,4cyclictest0-21swapper/110:01:551
7816992410,11cyclictest21-21ksoftirqd/108:05:141
780199246,3cyclictest131rcu_preempt11:55:140
780199246,3cyclictest131rcu_preempt08:10:160
780199245,3cyclictest131rcu_preempt10:00:300
780199245,3cyclictest131rcu_preempt08:55:110
780199245,3cyclictest131rcu_preempt07:25:210
780199245,2cyclictest131rcu_preempt11:10:020
780199243,3cyclictest131rcu_preempt11:40:290
780199243,3cyclictest131rcu_preempt11:40:280
780199242,6cyclictest12-21ksoftirqd/011:00:190
7801992412,8cyclictest0-21swapper/007:55:120
7801992410,4cyclictest0-21swapper/010:17:500
783299233,14cyclictest35966irq/58-eth010:35:123
783299233,13cyclictest0-21swapper/311:10:133
783299232,12cyclictest0-21swapper/307:20:133
782499237,4cyclictest131rcu_preempt08:55:232
782499236,6cyclictest131rcu_preempt09:15:062
782499235,7cyclictest131rcu_preempt12:05:252
782499235,7cyclictest131rcu_preempt10:10:212
782499235,4cyclictest131rcu_preempt09:50:432
782499233,5cyclictest27-21ksoftirqd/210:17:202
782499232,6cyclictest0-21swapper/211:50:152
7824992311,5cyclictest16030-21ssh11:27:072
782499231,14cyclictest27-21ksoftirqd/212:16:382
782499231,13cyclictest0-21swapper/208:54:262
781699236,3cyclictest131rcu_preempt07:25:301
781699232,3cyclictest131rcu_preempt09:12:371
781699232,2cyclictest131rcu_preempt09:35:231
7816992313,8cyclictest131rcu_preempt11:50:131
7816992312,7cyclictest0-21swapper/107:50:021
780199239,6cyclictest131rcu_preempt09:53:030
780199237,3cyclictest131rcu_preempt08:25:240
780199236,4cyclictest131rcu_preempt12:05:190
780199236,3cyclictest131rcu_preempt09:00:200
780199235,8cyclictest12-21ksoftirqd/008:50:000
780199235,3cyclictest131rcu_preempt10:05:210
780199235,3cyclictest131rcu_preempt08:15:220
780199234,3cyclictest131rcu_preempt10:45:220
780199234,3cyclictest131rcu_preempt10:30:260
780199234,3cyclictest131rcu_preempt10:30:250
780199234,3cyclictest131rcu_preempt07:35:280
780199234,3cyclictest131rcu_preempt07:35:280
780199234,13cyclictest844-21munin-node08:20:130
783299226,10cyclictest35966irq/58-eth012:05:123
783299222,15cyclictest0-21swapper/310:00:133
7832992212,7cyclictest35966irq/58-eth010:15:133
783299221,13cyclictest35966irq/58-eth011:45:133
781699225,4cyclictest131rcu_preempt12:35:201
781699224,4cyclictest131rcu_preempt10:28:131
781699224,3cyclictest0-21swapper/109:42:011
781699222,3cyclictest131rcu_preempt09:20:131
781699222,2cyclictest131rcu_preempt10:15:131
781699221,2cyclictest131rcu_preempt10:20:381
7816992212,7cyclictest8801-21sshd12:17:261
781699221,16cyclictest20192-21ssh12:25:121
780199229,4cyclictest12-21ksoftirqd/008:35:160
780199228,4cyclictest12-21ksoftirqd/009:05:260
780199227,3cyclictest131rcu_preempt07:55:000
780199225,3cyclictest131rcu_preempt08:00:130
780199224,3cyclictest131rcu_preempt07:20:250
780199222,17cyclictest6294-21kthreadcore07:40:220
780199221,17cyclictest29759-21cut08:40:170
783299218,8cyclictest0-21swapper/312:30:123
782499217,11cyclictest32421-21irqcore07:35:202
782499217,11cyclictest32421-21irqcore07:35:202
782499214,2cyclictest131rcu_preempt09:05:202
782499213,12cyclictest0-21swapper/207:45:142
782499211,17cyclictest19753-21kthreadcore07:20:222
782499211,11cyclictest7031-21kthreadcore08:15:202
781699219,3cyclictest131rcu_preempt09:45:181
781699218,10cyclictest21-21ksoftirqd/109:30:201
781699216,4cyclictest812-21grep11:55:191
781699212,14cyclictest27208-21apt-get07:30:151
781699211,6cyclictest0-21swapper/109:50:161
781699211,15cyclictest26060-21uniq12:10:141
781699211,14cyclictest20027-21ls09:00:281
780199214,4cyclictest12-21ksoftirqd/007:30:290
783299209,7cyclictest0-21swapper/309:45:143
783299207,8cyclictest0-21swapper/311:15:123
783299202,11cyclictest0-21swapper/307:15:133
7832992012,5cyclictest35966irq/58-eth011:59:403
7832992011,6cyclictest35966irq/58-eth009:40:123
783299201,14cyclictest35966irq/58-eth011:30:113
7832992010,6cyclictest35966irq/58-eth011:40:143
7832992010,6cyclictest35966irq/58-eth011:40:143
7832992010,6cyclictest35966irq/58-eth009:50:163
782499207,4cyclictest21739-21kthreadcore08:30:202
782499203,8cyclictest10922-21kthreadcore07:10:212
782499203,8cyclictest10922-21kthreadcore07:10:212
782499203,14cyclictest4695-21if_eth107:40:202
782499201,9cyclictest30685-21cut08:40:202
782499201,3cyclictest0-21swapper/207:50:162
781699209,3cyclictest0-21swapper/108:50:241
781699202,4cyclictest131rcu_preempt11:29:451
781699201,3cyclictest131rcu_preempt08:40:121
781699201,2cyclictest131rcu_preempt12:32:561
781699201,14cyclictest18008-21irqstats10:35:191
781699200,9cyclictest9949-21/usr/sbin/munin09:55:231
780199204,3cyclictest131rcu_preempt07:15:240
7832991912,5cyclictest0-21swapper/310:55:213
782499191,9cyclictest22469-21ls07:25:182
782499191,8cyclictest7850-21latency_hist07:10:022
781699197,3cyclictest131rcu_preempt10:30:131
781699197,3cyclictest131rcu_preempt10:30:121
781699197,3cyclictest131rcu_preempt08:35:181
781699191,2cyclictest131rcu_preempt09:25:041
781699191,2cyclictest131rcu_preempt08:55:231
781699191,15cyclictest17666-21ls07:50:291
781699191,15cyclictest15187-21latency_hist08:25:021
781699191,14cyclictest12783-21apt-get07:15:131
781699190,3cyclictest0-21swapper/111:15:161
783299186,7cyclictest35966irq/58-eth012:00:143
783299186,7cyclictest35966irq/58-eth012:00:133
783299185,10cyclictest0-21swapper/312:15:293
783299182,9cyclictest0-21swapper/307:50:043
783299182,10cyclictest0-21swapper/308:40:153
783299182,10cyclictest0-21swapper/308:10:163
783299182,10cyclictest0-21swapper/307:55:123
783299182,10cyclictest0-21swapper/307:45:143
781699189,6cyclictest21-21ksoftirqd/111:20:131
781699189,2cyclictest131rcu_preempt07:10:121
781699189,2cyclictest131rcu_preempt07:10:111
781699186,3cyclictest131rcu_preempt12:00:141
781699186,3cyclictest131rcu_preempt12:00:141
781699183,2cyclictest131rcu_preempt08:00:131
781699182,4cyclictest21569-21irqcore09:05:181
781699182,2cyclictest131rcu_preempt10:50:241
781699181,9cyclictest5978-21ps08:15:171
781699181,2cyclictest131rcu_preempt08:45:321
781699181,2cyclictest131rcu_preempt07:55:231
783299179,6cyclictest0-21swapper/312:20:103
783299179,6cyclictest0-21swapper/311:35:443
783299179,6cyclictest0-21swapper/311:35:433
783299179,6cyclictest0-21swapper/310:25:023
783299178,6cyclictest0-21swapper/310:05:263
783299172,10cyclictest0-21swapper/312:10:123
783299171,13cyclictest0-21swapper/311:50:173
781699172,12cyclictest18534-21kthreadcore07:20:191
781699171,2cyclictest131rcu_preempt07:40:221
7816991712,3cyclictest0-21swapper/108:25:131
781699171,13cyclictest3099-21tr08:10:231
783299162,9cyclictest0-21swapper/309:00:153
783299161,9cyclictest0-21swapper/307:35:243
783299161,9cyclictest0-21swapper/307:35:233
783299161,10cyclictest0-21swapper/309:05:143
783299161,10cyclictest0-21swapper/308:15:173
783299152,9cyclictest0-21swapper/307:30:163
783299151,9cyclictest0-21swapper/308:05:243
781699151,5cyclictest0-21swapper/107:10:021
780199153,3cyclictest131rcu_preempt07:10:020
783299149,3cyclictest0-21swapper/308:45:253
783299144,6cyclictest0-21swapper/307:10:133
783299144,6cyclictest0-21swapper/307:10:123
783299141,11cyclictest0-21swapper/307:40:133
783299121,9cyclictest0-21swapper/308:35:213
783299112,5cyclictest0-21swapper/308:50:143
783299112,5cyclictest0-21swapper/308:25:133
783299111,7cyclictest0-21swapper/308:55:153
783299111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional