You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 18:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 08, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9306996321,27cyclictest35966irq/58-eth009:10:143
926199624,8cyclictest131rcu_preempt10:30:140
927699592,49cyclictest24511-21ssh10:20:141
926199595,40cyclictest131rcu_preempt10:45:140
9306995850,4cyclictest35966irq/58-eth012:15:153
929299495,17cyclictest27-21ksoftirqd/210:30:142
929299490,28cyclictest6477-21ssh10:10:122
929299477,19cyclictest27-21ksoftirqd/210:25:142
929299471,35cyclictest27-21ksoftirqd/208:30:122
929299465,32cyclictest4442-21ssh12:15:162
926199467,7cyclictest131rcu_preempt12:15:160
926199466,6cyclictest131rcu_preempt08:30:120
9276994516,21cyclictest616-21systemd-logind08:10:151
926199452,31cyclictest29929-21ssh10:05:150
929299434,17cyclictest27-21ksoftirqd/209:10:142
927699431,35cyclictest31380-21ssh12:30:131
926199431,39cyclictest19209-21ssh10:35:160
9276994225,12cyclictest0-21swapper/110:15:161
927699413,24cyclictest4751-21ssh10:45:161
927699411,6cyclictest17226-21ssh09:40:141
9261994115,12cyclictest131rcu_preempt07:40:140
926199411,32cyclictest11777-21ssh09:55:130
9261994014,11cyclictest131rcu_preempt11:45:130
930699385,22cyclictest35966irq/58-eth010:40:133
930699384,27cyclictest35966irq/58-eth012:05:133
927699385,29cyclictest9846-21ssh12:00:151
927699371,9cyclictest10159-21awk10:10:261
927699371,29cyclictest11665-21sh11:25:141
9261993710,20cyclictest26249-21ssh11:15:150
9292993621,12cyclictest27-21ksoftirqd/212:30:132
927699361,9cyclictest1531-21grep08:05:301
927699361,10cyclictest27251-21tr09:45:191
927699361,10cyclictest24950-21ssh09:25:191
929299355,15cyclictest261rcuc/209:30:142
929299352,15cyclictest9164-21apt-get07:10:042
929299352,15cyclictest9164-21apt-get07:10:042
927699357,4cyclictest5052-21switchtime10:25:311
927699352,22cyclictest21-21ksoftirqd/107:25:131
927699351,9cyclictest32041-21grep12:10:311
927699351,7cyclictest27745-21apt10:40:011
927699351,11cyclictest26222-21apt-get11:15:151
927699351,10cyclictest8203-21ssh09:35:131
927699351,10cyclictest31941-21wc10:40:271
927699350,10cyclictest31681-21irqcore08:40:171
926199351,27cyclictest20823-21ssh10:00:120
930699343,18cyclictest35966irq/58-eth010:45:143
930699342,20cyclictest35966irq/58-eth010:20:133
9292993418,11cyclictest27-21ksoftirqd/208:10:142
927699342,9cyclictest28634-21ntp_kernel_err08:00:251
927699342,11cyclictest27545-21diskmemload12:15:151
927699342,10cyclictest25185-21kthreadcore10:55:241
927699341,9cyclictest17928-21tr09:00:011
927699341,9cyclictest10347-21cut10:30:171
927699341,8cyclictest5940-21ssh11:20:241
927699341,8cyclictest13979-21ps12:20:201
927699341,8cyclictest12277-21cat08:50:281
927699341,7cyclictest12112-21latency_hist08:20:021
9276993417,12cyclictest27774-1kworker/u9:209:15:121
9276993411,20cyclictest27545-21diskmemload11:35:281
927699341,10cyclictest9436-21latency_hist07:10:011
927699341,10cyclictest24813-21cut12:25:231
927699341,10cyclictest19007-21ntp_states11:45:261
926199343,21cyclictest1148-21sh10:25:150
930699333,19cyclictest35966irq/58-eth011:25:133
9292993313,15cyclictest27-21ksoftirqd/208:20:142
927699331,9cyclictest6084-21kthreadcore09:50:231
927699331,9cyclictest26415-21cat08:35:011
927699331,9cyclictest22687-21kthreadcore11:30:231
927699331,9cyclictest22343-21latency_hist10:55:011
927699331,9cyclictest12139-21if_eth109:55:171
927699331,10cyclictest2515-21kthreadcore11:55:231
927699331,10cyclictest21948-21kthreadcore12:05:231
927699331,10cyclictest10852-21kthreadcore07:10:201
927699331,10cyclictest10852-21kthreadcore07:10:201
927699330,9cyclictest11950-21ssh11:05:281
927699330,9cyclictest11950-21ssh11:05:281
927699330,8cyclictest20772-21irqcore07:55:171
926199332,7cyclictest131rcu_preempt09:34:390
930699323,18cyclictest321rcuc/311:50:143
929299324,18cyclictest27346irq/40-dwc2_hso12:20:162
929299321,17cyclictest11676-21ssh11:25:142
927699325,6cyclictest19980-21cat07:50:291
927699321,9cyclictest21203-21cat11:10:301
927699321,9cyclictest15370-21latency_hist07:50:011
927699321,8cyclictest15643-21ntpq08:20:251
927699321,8cyclictest13711-21cpuspeed09:20:161
927699321,8cyclictest13711-21cpuspeed09:20:151
927699321,7cyclictest7725-21cut11:40:201
927699321,10cyclictest796-21kthreadcore11:00:211
927699320,8cyclictest29522-21awk10:05:011
926199323,22cyclictest28155-21ssh10:40:130
930699313,14cyclictest35966irq/58-eth011:55:143
930699312,21cyclictest35966irq/58-eth011:45:123
930699312,16cyclictest35966irq/58-eth009:05:133
9306993112,12cyclictest35966irq/58-eth011:30:143
929299316,17cyclictest27-21ksoftirqd/210:50:132
929299311,19cyclictest18830-21ssh12:05:142
927699311,9cyclictest8493-21latency_hist08:50:021
927699311,8cyclictest6015-21df07:40:161
927699311,8cyclictest22951-21/usr/sbin/munin09:05:271
927699311,8cyclictest20737-21sort08:25:291
927699311,7cyclictest18106-21grep07:15:301
926199312,21cyclictest13293-21rm12:20:070
9261993115,5cyclictest9252-21cyclictest08:20:130
9261993111,11cyclictest15546-21diskmemload10:15:160
929299303,20cyclictest261rcuc/210:40:132
927699301,9cyclictest28595-21sed09:10:201
927699301,9cyclictest26889-21cut08:35:181
927699301,9cyclictest24409-21diskstats11:50:171
927699301,8cyclictest1601-21tr09:30:221
927699301,10cyclictest6089-21cron10:10:011
926199308,15cyclictest12-21ksoftirqd/007:50:120
930699293,14cyclictest0-21swapper/310:05:153
9306992921,5cyclictest35966irq/58-eth009:40:153
929299295,15cyclictest27346irq/40-dwc2_hso08:35:142
929299293,21cyclictest0-21swapper/209:27:392
929299292,21cyclictest0-21swapper/211:20:152
9292992913,12cyclictest17211-21apt-get11:10:142
927699291,9cyclictest21873-21grep09:00:291
927699291,9cyclictest1313-21sort07:35:171
927699291,11cyclictest8695-21uname12:35:211
927699290,7cyclictest19061-21/usr/sbin/munin07:20:271
926199292,21cyclictest1237-21sh09:48:390
926199291,21cyclictest17222-21sh11:10:130
930699283,18cyclictest0-21swapper/309:38:393
930699282,14cyclictest35966irq/58-eth012:20:143
929299284,18cyclictest27-21ksoftirqd/211:50:142
929299283,21cyclictest0-21swapper/209:38:392
929299281,18cyclictest808-21ssh11:55:142
927699281,9cyclictest28774-21fschecks_count07:30:171
9261992818,6cyclictest121ksoftirqd/009:05:130
930699273,13cyclictest35966irq/58-eth009:15:123
929299272,19cyclictest0-21swapper/208:15:152
929299272,17cyclictest27346irq/40-dwc2_hso10:20:132
929299272,16cyclictest27346irq/40-dwc2_hso09:20:132
929299272,16cyclictest27346irq/40-dwc2_hso09:20:132
929299272,11cyclictest0-21swapper/211:34:392
930699263,15cyclictest0-21swapper/312:10:263
9306992613,8cyclictest35966irq/58-eth009:45:133
9306992611,5cyclictest35966irq/58-eth012:30:133
929299265,13cyclictest27-21ksoftirqd/211:45:152
929299263,17cyclictest27346irq/40-dwc2_hso11:40:122
929299263,16cyclictest27-21ksoftirqd/209:45:122
929299262,16cyclictest15052-1kworker/u9:210:15:162
929299262,13cyclictest27346irq/40-dwc2_hso09:05:142
9292992613,10cyclictest0-21swapper/207:50:012
929299261,18cyclictest0-21swapper/207:55:142
926199262,17cyclictest31695-21ssh11:00:150
929299252,16cyclictest27346irq/40-dwc2_hso09:55:132
929299252,16cyclictest0-21swapper/207:35:042
929299252,15cyclictest27346irq/40-dwc2_hso11:15:142
929299251,21cyclictest6064-21grep07:40:172
926199252,7cyclictest131rcu_preempt09:25:390
930699242,16cyclictest0-21swapper/310:10:123
929299246,4cyclictest0-21swapper/209:40:142
929299242,16cyclictest27346irq/40-dwc2_hso11:00:142
929299241,18cyclictest16510-21ssh11:09:392
929299241,18cyclictest16510-21ssh11:09:392
926199242,19cyclictest13710-21ssh10:50:140
926199241,17cyclictest9847-21ssh12:00:150
929299232,16cyclictest0-21swapper/212:02:482
929299232,16cyclictest0-21swapper/210:49:392
929299231,6cyclictest0-21swapper/209:00:132
926199238,4cyclictest12-21ksoftirqd/007:50:010
926199234,14cyclictest22427-21ssh12:25:140
926199232,15cyclictest25448-21ssh09:44:390
929299226,7cyclictest27-21ksoftirqd/210:05:012
929299226,5cyclictest25237-21apt-get08:00:142
929299223,11cyclictest22844-21cut12:25:192
929299222,15cyclictest0-21swapper/210:39:392
929299222,15cyclictest0-21swapper/209:54:392
929299221,9cyclictest20207-21wc07:55:012
9292992215,5cyclictest25165-21ssh10:55:232
929299221,13cyclictest30481-21kernelversion11:35:202
929299221,11cyclictest30278-21tune2fs08:05:172
926199229,9cyclictest9252-21cyclictest11:54:390
9261992216,3cyclictest8214-21ssh09:35:130
926199221,3cyclictest131rcu_preempt11:25:140
930699214,9cyclictest35966irq/58-eth010:50:123
9306992116,3cyclictest0-21swapper/307:30:143
929299217,8cyclictest27-21ksoftirqd/208:25:302
929299217,7cyclictest12313-21getconf12:35:322
929299212,11cyclictest28829-21kthreadcore12:10:212
9292992114,5cyclictest0-21swapper/207:25:172
929299211,17cyclictest0-21swapper/207:30:242
929299211,12cyclictest12354-21unixbench_multi08:50:282
929299211,10cyclictest31109-21cut10:05:212
9292992110,8cyclictest4539-21find09:15:112
9261992111,7cyclictest9521-21cut11:05:220
9261992111,7cyclictest9521-21cut11:05:210
9261992111,5cyclictest12-21ksoftirqd/008:25:130
926199210,14cyclictest15114-21ls11:45:020
930699201,15cyclictest35966irq/58-eth010:35:163
9306992011,5cyclictest35966irq/58-eth010:00:133
9306992010,8cyclictest0-21swapper/310:18:373
929299205,13cyclictest27-21ksoftirqd/208:40:192
929299205,12cyclictest19752-21grep07:20:212
929299202,9cyclictest16842-21kthreadcore07:15:222
929299201,15cyclictest8504-21latency_hist08:50:022
929299201,15cyclictest14233-21kthreadcore08:55:202
926199206,10cyclictest21047-21ssh09:23:390
926199206,10cyclictest21047-21ssh09:23:390
926199201,6cyclictest23649-21munin-run07:25:010
926199201,16cyclictest1311-21grep07:35:170
926199201,15cyclictest28352-21tr09:10:190
926199201,14cyclictest828-21ssh11:55:150
9261992011,4cyclictest32140-21kthreadcore12:30:200
926199200,15cyclictest27905-21ssh10:20:230
930699193,11cyclictest0-21swapper/311:40:133
930699192,13cyclictest0-21swapper/311:00:153
930699192,11cyclictest0-21swapper/310:25:043
9306991915,2cyclictest0-21swapper/307:15:153
9306991913,4cyclictest0-21swapper/311:35:143
930699191,12cyclictest0-21swapper/308:10:153
9306991910,6cyclictest0-21swapper/309:20:323
9306991910,6cyclictest0-21swapper/309:20:323
929299191,9cyclictest9293-21rm07:10:012
926199198,8cyclictest18832-21apt-get12:05:130
926199196,10cyclictest24840-21grep11:30:310
926199191,8cyclictest10602-21sed10:10:300
9261991911,6cyclictest870-21date07:35:010
926199191,16cyclictest7567-21irqcore08:15:190
926199191,11cyclictest27851-21ssh12:10:130
9261991910,6cyclictest8272-21ntpq09:15:260
926199190,4cyclictest9252-21cyclictest11:35:260
930699189,6cyclictest0-21swapper/312:25:553
930699189,6cyclictest0-21swapper/311:21:173
930699189,6cyclictest0-21swapper/309:27:173
930699188,5cyclictest0-21swapper/310:30:143
930699182,10cyclictest0-21swapper/307:35:043
9306991812,4cyclictest0-21swapper/312:35:353
9306991811,5cyclictest0-21swapper/312:00:133
9306991811,5cyclictest0-21swapper/311:10:293
930699181,13cyclictest0-21swapper/307:30:023
926199183,10cyclictest0-21swapper/008:00:140
926199181,8cyclictest15573-21kthreadcore07:15:210
926199180,16cyclictest26264-21ls10:55:260
930699179,6cyclictest35966irq/58-eth011:05:163
930699179,6cyclictest35966irq/58-eth011:05:163
930699179,6cyclictest0-21swapper/309:51:143
930699178,6cyclictest35966irq/58-eth011:15:153
930699177,3cyclictest0-21swapper/310:59:253
930699172,9cyclictest0-21swapper/308:30:143
930699172,9cyclictest0-21swapper/307:10:043
930699172,9cyclictest0-21swapper/307:10:043
930699171,12cyclictest35966irq/58-eth009:55:133
930699171,12cyclictest0-21swapper/308:45:233
930699171,10cyclictest0-21swapper/309:35:023
930699171,10cyclictest0-21swapper/307:50:153
926199174,3cyclictest0-21swapper/011:20:230
926199172,12cyclictest6308-21sh09:50:240
9261991711,4cyclictest0-21swapper/008:50:170
9261991711,4cyclictest0-21swapper/007:25:250
926199171,12cyclictest2953-21irqcore08:10:190
9261991710,4cyclictest12745-21cat07:10:220
9261991710,4cyclictest12745-21cat07:10:220
9261991710,4cyclictest11679-21sh12:35:280
930699161,9cyclictest0-21swapper/308:50:223
9306991611,2cyclictest0-21swapper/308:25:143
930699161,12cyclictest0-21swapper/307:40:153
926199168,5cyclictest6944-21cut08:45:230
926199162,11cyclictest28773-21kthreadcore08:35:210
926199162,11cyclictest19075-21kthreadcore09:00:200
926199161,12cyclictest20531-21fschecks_count07:55:150
926199161,11cyclictest32437-21kthreadcore08:40:190
930699152,9cyclictest0-21swapper/309:00:143
926199159,4cyclictest0-21swapper/008:05:200
926199152,11cyclictest13321-21find08:55:130
930699141,9cyclictest35966irq/58-eth007:45:133
930699141,10cyclictest0-21swapper/308:40:143
930699141,10cyclictest0-21swapper/307:55:223
926199141,10cyclictest9273-21latency_hist07:10:020
930699132,8cyclictest0-21swapper/308:25:023
930699131,7cyclictest0-21swapper/308:05:163
930699131,10cyclictest0-21swapper/307:20:153
930699127,3cyclictest0-21swapper/308:55:183
930699123,7cyclictest0-21swapper/308:00:293
930699122,7cyclictest0-21swapper/308:35:143
930699112,5cyclictest0-21swapper/308:15:153
930699111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional