You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 15:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Mar 29, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28123995722,23cyclictest28366irq/58-eth012:30:123
28100995631,18cyclictest0-21swapper/110:00:111
2809299543,17cyclictest12-21ksoftirqd/009:10:130
28092995213,9cyclictest131rcu_preempt11:20:130
2810099450,37cyclictest30937-21ssh09:55:121
2812399433,25cyclictest28366irq/58-eth010:30:133
2811399411,31cyclictest0-21swapper/209:45:122
2810099412,32cyclictest29453-21ssh10:30:131
28113994020,9cyclictest27-21ksoftirqd/210:00:122
28113994015,20cyclictest131rcu_preempt09:10:132
28092993917,4cyclictest131rcu_preempt12:15:130
2812399382,23cyclictest28366irq/58-eth012:00:113
2811399385,8cyclictest131rcu_preempt07:25:112
2810099382,19cyclictest32372-21ssh09:20:121
2810099372,29cyclictest18690-21ssh11:00:131
2809299373,17cyclictest6077-21ssh10:35:140
2811399368,23cyclictest261rcuc/211:55:142
2809299368,9cyclictest12-21ksoftirqd/007:20:120
2812399341,22cyclictest28366irq/58-eth007:30:133
2812399341,20cyclictest28366irq/58-eth012:10:133
28113993417,11cyclictest18925-21ssh12:30:122
2811399341,19cyclictest4423-21ssh11:10:142
2812399333,14cyclictest28366irq/58-eth011:50:133
2811399331,15cyclictest9737-21ssh10:55:142
2811399322,25cyclictest7617-21ssh11:29:562
28113993217,13cyclictest0-21swapper/209:30:122
2809299321,19cyclictest11301-21ssh10:20:130
2809299321,19cyclictest11301-21ssh10:20:120
2812399311,20cyclictest28366irq/58-eth008:30:133
2811399315,18cyclictest22346irq/40-dwc2_hso07:20:122
2811399313,22cyclictest0-21swapper/211:49:362
2809299316,22cyclictest15328-21ssh12:10:130
2811399303,20cyclictest22346irq/40-dwc2_hso08:35:142
2810099302,20cyclictest29601-21ssh12:00:021
2809299302,21cyclictest29550-21ssh12:00:020
28092993014,11cyclictest8965-21diskmemload09:25:130
2809299301,22cyclictest12-21ksoftirqd/011:25:130
28092993012,14cyclictest11676-21ssh11:50:120
2812399293,14cyclictest28366irq/58-eth009:20:123
2811399295,16cyclictest22346irq/40-dwc2_hso11:35:142
2811399293,15cyclictest22346irq/40-dwc2_hso08:25:132
2811399292,11cyclictest6128-21ssh12:05:012
2811399291,9cyclictest131rcu_preempt10:05:132
2811399290,14cyclictest30946-21ssh09:55:122
2809299295,20cyclictest12-21ksoftirqd/009:40:130
28092992912,3cyclictest131rcu_preempt10:25:130
2811399286,14cyclictest22346irq/40-dwc2_hso07:15:112
2811399285,15cyclictest22346irq/40-dwc2_hso12:35:122
2811399285,15cyclictest22346irq/40-dwc2_hso12:35:122
2810099282,19cyclictest29554-21latency_hist12:00:021
2810099282,19cyclictest27856-21ssh12:35:141
2810099282,19cyclictest27856-21ssh12:35:141
2810099282,19cyclictest15161-21sh12:10:031
2810099281,19cyclictest8966-21rm09:25:141
28100992810,12cyclictest21-21ksoftirqd/107:30:131
2812399271,17cyclictest28366irq/58-eth012:05:123
2811399275,14cyclictest22346irq/40-dwc2_hso09:50:132
2811399275,12cyclictest22346irq/40-dwc2_hso10:45:122
2811399273,18cyclictest32388-21ssh09:20:142
2811399272,19cyclictest0-21swapper/211:54:362
2811399272,19cyclictest0-21swapper/208:00:132
2811399272,17cyclictest0-21swapper/210:15:052
2810099273,19cyclictest26124-21ssh11:40:151
28100992717,5cyclictest0-21swapper/111:25:151
2810099271,19cyclictest11262-21ssh11:50:001
2809299274,14cyclictest12-21ksoftirqd/009:55:110
2809299272,18cyclictest8117-21ssh11:30:140
2809299272,18cyclictest1093-21sh11:43:350
2811399265,13cyclictest22346irq/40-dwc2_hso09:15:122
2811399264,14cyclictest22346irq/40-dwc2_hso08:10:122
2811399263,17cyclictest0-21swapper/212:02:372
2811399262,16cyclictest22346irq/40-dwc2_hso08:05:142
2811399262,10cyclictest0-21swapper/208:15:132
2811399261,17cyclictest6806-21ls07:55:142
28113992611,7cyclictest531-21usb-storage07:50:112
2811399261,15cyclictest13420-21ls12:25:242
2810099264,8cyclictest131rcu_preempt11:39:371
2810099261,17cyclictest13817-21diskmemload09:10:141
2809299263,19cyclictest12854-21ssh09:45:120
28092992616,4cyclictest131rcu_preempt11:15:120
28092992616,4cyclictest131rcu_preempt11:15:120
2812399253,10cyclictest0-21swapper/311:42:373
2811399254,16cyclictest22346irq/40-dwc2_hso07:45:152
2811399253,18cyclictest22346irq/40-dwc2_hso09:35:152
2811399253,15cyclictest22346irq/40-dwc2_hso07:10:132
2811399253,15cyclictest22346irq/40-dwc2_hso07:10:132
2811399252,8cyclictest31017-21ssh11:40:372
2811399250,18cyclictest18271-21ls07:30:232
2811399250,17cyclictest29415-21find10:30:112
2810099258,8cyclictest131rcu_preempt09:45:281
2810099251,20cyclictest9743-21ssh10:55:141
2810099251,17cyclictest18036-21ssh09:30:141
28100992511,6cyclictest0-21swapper/107:10:181
28100992511,6cyclictest0-21swapper/107:10:171
28092992514,3cyclictest12-21ksoftirqd/011:45:310
28092992514,3cyclictest12-21ksoftirqd/008:40:290
28092992514,3cyclictest12-21ksoftirqd/007:55:290
28092992513,4cyclictest12-21ksoftirqd/008:05:230
2811399243,6cyclictest0-21swapper/207:40:112
2811399242,4cyclictest15103-21ssh10:40:132
2810099247,14cyclictest21-21ksoftirqd/108:30:131
2810099241,17cyclictest27108-21ssh09:35:161
2809299248,3cyclictest131rcu_preempt07:50:110
28092992413,3cyclictest12-21ksoftirqd/008:30:230
2809299241,17cyclictest3087-21ls08:25:020
2812399233,11cyclictest0-21swapper/309:55:123
2811399235,8cyclictest88166lldpd11:01:452
2811399234,11cyclictest22346irq/40-dwc2_hso08:40:132
2811399232,6cyclictest0-21swapper/212:21:372
2811399232,5cyclictest0-21swapper/208:55:122
2811399232,5cyclictest0-21swapper/208:55:112
2811399232,16cyclictest0-21swapper/211:34:362
2811399231,13cyclictest26229-21irqrtprio10:10:192
2811399230,20cyclictest31384-21ls11:05:252
2810099232,2cyclictest131rcu_preempt09:17:441
2810099231,17cyclictest13579-21ssh12:25:261
2809299235,15cyclictest12-21ksoftirqd/007:40:110
28092992314,6cyclictest18291-21cron11:00:020
28092992314,5cyclictest12-21ksoftirqd/012:20:180
28092992312,3cyclictest12-21ksoftirqd/011:37:420
28092992312,3cyclictest12-21ksoftirqd/009:35:170
2809299231,17cyclictest29461-21sh10:30:130
28092992311,3cyclictest12-21ksoftirqd/010:48:510
28092992310,4cyclictest12-21ksoftirqd/009:19:470
2812399223,13cyclictest0-21swapper/311:50:003
28123992215,5cyclictest28366irq/58-eth010:20:133
28123992215,5cyclictest28366irq/58-eth010:20:123
2811399222,16cyclictest0-21swapper/210:39:482
2811399221,9cyclictest24257-21ssh12:15:142
2811399221,16cyclictest15059-21ssh10:20:262
2811399221,16cyclictest15059-21ssh10:20:262
2811399221,15cyclictest31879-21latency_hist08:55:022
2811399221,11cyclictest4944-21cut09:40:202
2810099226,5cyclictest171-21systemd-network11:33:031
2810099223,3cyclictest131rcu_preempt12:17:021
2810099222,15cyclictest19940-21ssh11:54:361
2810099221,8cyclictest2097-21kthreadcore12:20:181
2810099221,17cyclictest6731-21apt-get07:55:111
28100992211,4cyclictest0-21swapper/107:50:051
2810099221,11cyclictest7128-21kthreadcore12:05:191
2809299229,3cyclictest12-21ksoftirqd/012:30:040
2809299227,3cyclictest131rcu_preempt07:15:120
2809299222,17cyclictest9959-21ssh12:25:150
2809299222,12cyclictest671-21ssh10:50:140
28092992217,2cyclictest4414-21ssh11:10:140
28092992214,5cyclictest0-21swapper/009:30:270
28092992211,7cyclictest10912-21basename10:20:000
28092992211,3cyclictest12-21ksoftirqd/011:08:010
28092992211,2cyclictest12-21ksoftirqd/009:20:300
2812399215,10cyclictest28366irq/58-eth009:40:143
28123992110,8cyclictest0-21swapper/310:04:083
2811399215,12cyclictest687-21in:imuxsock11:24:562
2811399214,12cyclictest22346irq/40-dwc2_hso09:05:142
2811399211,16cyclictest3897-21kthreadcore10:50:222
2811399211,10cyclictest10957-21kthreadcore09:25:222
2810099219,9cyclictest21930-21apt-get09:50:131
2810099216,4cyclictest13817-21diskmemload10:40:011
2810099212,9cyclictest131rcu_preempt10:52:401
2810099212,14cyclictest20414-21ssh10:25:121
2810099211,9cyclictest5495-21kthreadcore11:10:201
2810099211,5cyclictest28785-21ssh10:10:231
2810099211,5cyclictest0-21swapper/107:35:141
2810099211,4cyclictest0-21swapper/107:25:111
28100992112,6cyclictest25194-21cron10:10:011
28100992111,7cyclictest27171-21cat11:20:311
2809299219,3cyclictest12-21ksoftirqd/012:36:400
2809299219,3cyclictest12-21ksoftirqd/012:36:390
2809299214,2cyclictest131rcu_preempt08:00:140
28092992113,4cyclictest12-21ksoftirqd/012:00:190
28092992113,4cyclictest12-21ksoftirqd/008:20:020
2809299211,15cyclictest18812-21ssh11:00:150
28092992111,2cyclictest12-21ksoftirqd/010:05:140
28092992110,8cyclictest21955-21rm09:50:140
2812399202,12cyclictest0-21swapper/310:25:133
2811399208,10cyclictest20607-21fschecks_time10:25:162
2811399202,10cyclictest27074-21latency_hist08:50:012
2811399201,11cyclictest16595-21kthreadcore11:15:222
2811399201,11cyclictest16595-21kthreadcore11:15:222
2811399201,11cyclictest15513-21fschecks_count12:10:152
2811399201,10cyclictest4946-21irqrtprio09:00:172
2811399200,9cyclictest744-21kthreadcore08:20:202
2810099206,5cyclictest2982-21sh08:25:011
2810099203,3cyclictest131rcu_preempt10:15:201
2810099202,4cyclictest131rcu_preempt08:25:161
2810099201,8cyclictest25759-21apt08:15:011
2810099201,7cyclictest4212-21cron09:00:011
2810099201,7cyclictest4212-21cron09:00:011
2810099201,2cyclictest131rcu_preempt12:31:111
2810099201,13cyclictest20889-21latency08:40:211
2810099201,13cyclictest13434-21cpuspeed11:15:131
2810099201,13cyclictest13434-21cpuspeed11:15:131
28100992010,8cyclictest21-21ksoftirqd/109:00:121
28100992010,2cyclictest21-21ksoftirqd/109:41:431
2810099200,8cyclictest11248-21/usr/sbin/munin10:20:161
2810099200,8cyclictest11248-21/usr/sbin/munin10:20:161
2809299209,3cyclictest12-21ksoftirqd/012:06:570
2809299209,3cyclictest12-21ksoftirqd/010:44:410
2809299209,3cyclictest12-21ksoftirqd/008:55:210
2809299209,3cyclictest12-21ksoftirqd/008:55:210
2809299209,2cyclictest12-21ksoftirqd/010:10:130
2809299209,2cyclictest12-21ksoftirqd/007:30:160
2809299207,4cyclictest12-21ksoftirqd/008:45:190
28092992010,2cyclictest12-21ksoftirqd/010:00:040
2812399197,9cyclictest0-21swapper/309:13:473
2812399193,11cyclictest0-21swapper/311:59:373
2812399193,11cyclictest0-21swapper/310:44:493
2812399192,13cyclictest33-21ksoftirqd/311:25:153
28123991912,5cyclictest28366irq/58-eth009:30:133
28123991911,6cyclictest0-21swapper/312:35:313
28123991911,6cyclictest0-21swapper/312:35:303
2811399191,15cyclictest7984-21apt-get08:30:142
2811399191,10cyclictest20201-21irqcore07:35:202
2810099193,3cyclictest0-21swapper/109:09:361
2810099191,9cyclictest29992-21irqrtprio07:45:201
2810099191,9cyclictest131rcu_preempt10:45:201
28100991913,4cyclictest0-21swapper/108:45:171
2810099191,15cyclictest10020-21latency_hist07:25:021
28100991911,4cyclictest0-21swapper/110:40:151
2810099190,13cyclictest131rcu_preempt11:09:441
2809299199,2cyclictest12-21ksoftirqd/008:15:010
2809299198,3cyclictest12-21ksoftirqd/009:05:120
28092991911,6cyclictest20023-21fw_packets07:35:180
28092991911,4cyclictest12-21ksoftirqd/008:25:220
2809299191,14cyclictest10383-21fschecks_time07:25:140
2812399189,6cyclictest0-21swapper/311:10:203
2812399188,8cyclictest0-21swapper/310:55:583
2812399188,8cyclictest0-21swapper/310:08:183
2812399183,11cyclictest0-21swapper/311:24:353
2812399183,10cyclictest0-21swapper/311:37:363
2812399183,10cyclictest0-21swapper/310:38:363
2812399182,11cyclictest0-21swapper/312:22:383
2812399182,11cyclictest0-21swapper/311:30:353
2812399182,11cyclictest0-21swapper/311:02:373
2812399182,10cyclictest0-21swapper/311:05:053
2812399182,10cyclictest0-21swapper/308:10:053
2812399182,10cyclictest0-21swapper/307:35:163
2812399181,9cyclictest28366irq/58-eth009:25:133
28123991810,6cyclictest0-21swapper/312:26:453
28123991810,6cyclictest0-21swapper/312:15:203
28123991810,6cyclictest0-21swapper/310:50:023
28123991810,6cyclictest0-21swapper/309:50:403
28123991810,5cyclictest0-21swapper/309:45:203
2811399184,11cyclictest27-21ksoftirqd/207:10:022
2810099182,5cyclictest16785-21irqrtprio08:05:191
2810099181,5cyclictest31790-21apt08:55:011
2809299185,10cyclictest13174-21irqcore08:35:190
2809299181,9cyclictest28461-21sed07:10:150
2809299181,9cyclictest28461-21sed07:10:140
2809299181,2cyclictest131rcu_preempt09:00:140
2812399172,10cyclictest0-21swapper/311:15:133
2812399172,10cyclictest0-21swapper/311:15:123
2812399172,10cyclictest0-21swapper/309:00:133
2812399172,10cyclictest0-21swapper/308:00:143
28123991712,3cyclictest28366irq/58-eth010:15:033
2812399171,11cyclictest0-21swapper/308:40:223
28123991710,4cyclictest0-21swapper/309:17:123
2810099178,7cyclictest26193-21perl08:15:161
2810099177,7cyclictest554-21apt-get07:15:121
28100991712,3cyclictest0-21swapper/108:35:311
2809299176,2cyclictest12-21ksoftirqd/008:50:280
2809299171,4cyclictest48-21kcompactd007:45:200
2812399169,4cyclictest0-21swapper/310:10:333
2812399168,6cyclictest0-21swapper/307:45:343
2812399161,12cyclictest0-21swapper/309:35:233
2812399161,12cyclictest0-21swapper/307:10:153
2812399161,12cyclictest0-21swapper/307:10:153
28123991610,4cyclictest0-21swapper/310:48:213
2810099168,6cyclictest11703-21fschecks_time08:00:171
2810099161,12cyclictest24814-21fschecks_time07:40:141
28123991510,3cyclictest0-21swapper/309:05:143
2809299158,4cyclictest12-21ksoftirqd/007:10:020
2812399149,3cyclictest0-21swapper/307:25:063
2812399147,5cyclictest0-21swapper/308:26:403
2812399142,6cyclictest0-21swapper/308:35:163
2812399141,10cyclictest0-21swapper/307:55:213
2810099141,9cyclictest28182-21cat07:10:011
2812399132,6cyclictest0-21swapper/308:45:133
2812399131,8cyclictest0-21swapper/307:40:133
2812399127,3cyclictest28366irq/58-eth008:20:283
2812399127,3cyclictest0-21swapper/308:05:293
2812399122,7cyclictest0-21swapper/308:55:013
2812399122,6cyclictest0-21swapper/308:55:143
2812399122,6cyclictest0-21swapper/308:55:133
2812399121,9cyclictest0-21swapper/307:50:163
2812399121,7cyclictest0-21swapper/307:15:213
2812399111,7cyclictest0-21swapper/308:15:143
2812399111,7cyclictest0-21swapper/307:20:173
2812399101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional