You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 18:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Apr 17, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1882399583,44cyclictest12995-21ssh10:45:121
1883199561,31cyclictest31720-21ssh09:25:122
1880799544,45cyclictest6372-21ssh10:05:130
1880799543,41cyclictest29857-21ssh10:00:120
1880799514,45cyclictest2066-21ssh11:15:110
1880799494,40cyclictest25481-21ssh11:10:130
1882399472,33cyclictest9769-21ssh10:25:121
1882399463,29cyclictest16474-21ssh11:05:121
1880799462,35cyclictest22821-21ssh09:20:130
1880799443,5cyclictest131rcu_preempt07:30:130
1880799433,26cyclictest3622-21ssh12:10:110
1884299421,27cyclictest35966irq/58-eth010:45:103
1880799421,14cyclictest26277-21ssh09:40:120
1883199392,31cyclictest2779-21ssh09:45:122
18823993926,8cyclictest536-21usb-storage07:45:111
1880799383,33cyclictest21992-21ssh10:50:140
1880799362,25cyclictest30952-21ssh10:55:110
1880799361,11cyclictest29093-21tr08:25:210
1880799361,11cyclictest29093-21tr08:25:200
18831993518,12cyclictest18947-21apt-get07:10:122
1882399352,25cyclictest0-21swapper/109:10:121
1880799343,28cyclictest11126-21ssh11:20:130
1880799343,21cyclictest12-21ksoftirqd/007:25:150
1880799342,25cyclictest0-21swapper/009:00:130
18807993417,11cyclictest4412-21diskmemload12:25:150
18807993411,4cyclictest24103-21unixbench_multi07:45:250
1884299331,19cyclictest35966irq/58-eth010:25:123
18831993318,10cyclictest27104-21apt-get12:05:122
1880799332,25cyclictest17247-21ssh09:35:140
18807993310,4cyclictest10505-21kthreadcore10:25:190
18842993223,6cyclictest0-21swapper/311:10:133
18831993223,4cyclictest20059-21apt-get11:25:122
18831993220,9cyclictest12655-21apt-get12:15:132
18831993220,9cyclictest12655-21apt-get12:15:122
1880799323,23cyclictest27130-21ssh12:05:130
1880799321,12cyclictest16300-21ssh12:35:150
18842993112,13cyclictest0-21swapper/309:55:163
1883199313,12cyclictest131rcu_preempt11:10:132
18831993119,8cyclictest27-21ksoftirqd/210:33:232
18831993119,8cyclictest27-21ksoftirqd/210:33:222
1883199311,15cyclictest4812-21ssh09:10:142
18831993111,15cyclictest0-21swapper/211:45:112
1880799311,11cyclictest18165-21ssh12:00:110
1880799311,11cyclictest11583-21sh09:50:020
1883199303,14cyclictest0-21swapper/209:42:122
18831993018,9cyclictest27590-21apt-get10:35:102
18831993017,4cyclictest9177-21apt-get11:55:132
1880799308,4cyclictest4920-21kthreadcore10:40:200
1880799302,24cyclictest27623-21ssh10:35:120
1880799301,10cyclictest20648-21grep11:05:280
18842992921,5cyclictest35966irq/58-eth011:15:123
18842992913,10cyclictest35966irq/58-eth009:45:143
1883199292,21cyclictest20790-21ssh09:35:212
1883199291,10cyclictest0-21swapper/207:30:122
1883199290,20cyclictest12053-21ssh10:06:322
18823992912,12cyclictest2788-21diskmemload09:45:141
1880799294,16cyclictest15264-21ssh10:10:110
1880799293,7cyclictest0-21swapper/009:34:220
1880799292,9cyclictest7568-21sh11:00:130
1880799292,20cyclictest12667-21ssh12:15:120
1880799292,20cyclictest12667-21ssh12:15:120
1880799291,11cyclictest23602-21apt-get11:45:130
18842992816,9cyclictest35966irq/58-eth010:00:113
18842992814,8cyclictest35966irq/58-eth012:10:123
1883199287,16cyclictest0-21swapper/210:40:112
18831992818,7cyclictest18150-21apt-get12:00:122
18831992816,2cyclictest20888-21apt-get09:55:162
1882399283,17cyclictest24253-21ssh10:15:141
1882399282,21cyclictest15350-21ssh11:04:091
1882399282,19cyclictest20793-21sed09:35:221
1880799283,19cyclictest7304-21ssh12:30:150
1880799283,18cyclictest7428-21kworker/u8:210:20:130
1880799283,14cyclictest9208-21ssh11:55:130
18807992815,8cyclictest0-21swapper/008:30:140
1880799281,23cyclictest29053-21ssh11:30:130
1884299279,11cyclictest35966irq/58-eth009:00:133
1884299273,16cyclictest0-21swapper/311:04:093
1883199277,9cyclictest0-21swapper/210:25:112
1883199276,18cyclictest0-21swapper/208:25:122
1883199276,18cyclictest0-21swapper/208:25:122
1883199272,4cyclictest29035-21apt-get11:30:142
18831992720,4cyclictest29899-21apt-get07:55:152
18831992720,2cyclictest3411-21apt-get08:35:132
1882399272,19cyclictest7306-21sh12:30:141
1880799271,8cyclictest8700-21switchtime09:10:280
1880799271,3cyclictest131rcu_preempt09:55:150
1880799271,11cyclictest32675-21df_inode11:50:150
1884299263,11cyclictest0-21swapper/310:44:223
1884299262,17cyclictest0-21swapper/310:59:063
18842992620,2cyclictest0-21swapper/310:30:133
18842992620,2cyclictest0-21swapper/310:30:123
18842992615,8cyclictest0-21swapper/311:05:103
18842992610,9cyclictest0-21swapper/310:35:093
1883199266,16cyclictest261rcuc/210:54:222
1883199261,16cyclictest27346irq/40-dwc2_hso11:05:122
1883199260,6cyclictest27-21ksoftirqd/211:42:182
1882399261,19cyclictest23627-21ssh11:45:131
18823992611,8cyclictest21-21ksoftirqd/107:30:011
18823992610,8cyclictest21-21ksoftirqd/108:45:011
1880799262,17cyclictest11885-21ssh09:50:130
18807992613,9cyclictest0-21swapper/008:05:120
18842992511,11cyclictest35966irq/58-eth012:20:263
1883199256,12cyclictest27-21ksoftirqd/210:23:032
1883199253,15cyclictest0-21swapper/209:20:082
1883199253,13cyclictest27-21ksoftirqd/209:30:032
1883199252,15cyclictest27-21ksoftirqd/211:35:262
1883199252,11cyclictest16148-21ssh12:35:042
18831992514,8cyclictest4412-21diskmemload11:00:132
1882399258,9cyclictest21-21ksoftirqd/109:16:301
1882399252,18cyclictest30019-21ssh10:54:231
18823992511,11cyclictest23812-21ssh09:55:251
18823992510,8cyclictest21-21ksoftirqd/109:34:471
1880799254,5cyclictest91-21pr/ttyS011:30:020
1880799253,16cyclictest0-21swapper/009:05:120
1884299249,9cyclictest35966irq/58-eth012:05:133
1884299242,14cyclictest35966irq/58-eth009:10:123
18842992418,2cyclictest0-21swapper/307:55:133
18842992417,4cyclictest0-21swapper/309:05:113
1884299241,17cyclictest35966irq/58-eth011:30:113
18842992410,6cyclictest0-21swapper/309:25:223
1883199248,3cyclictest0-21swapper/212:24:292
1883199243,13cyclictest27-21ksoftirqd/211:18:042
1883199243,11cyclictest11134-21sh11:20:142
1883199241,9cyclictest27-21ksoftirqd/210:13:242
1883199241,15cyclictest7174-21cut08:05:142
18831992410,11cyclictest24234-21ssh10:15:122
18831992410,11cyclictest0-21swapper/207:25:262
1883199240,9cyclictest12939-21ssh10:45:092
1882399248,9cyclictest21-21ksoftirqd/111:55:021
1882399248,8cyclictest21-21ksoftirqd/109:27:271
1882399247,9cyclictest21-21ksoftirqd/111:36:071
1882399247,9cyclictest21-21ksoftirqd/110:56:081
1882399247,7cyclictest21-21ksoftirqd/110:13:201
1882399246,8cyclictest21-21ksoftirqd/110:41:471
1882399245,3cyclictest131rcu_preempt12:37:191
18823992410,11cyclictest27335-21ssh09:20:301
1880799241,19cyclictest2168-21ls08:00:010
1880799241,18cyclictest753-21sendmail-mta07:10:120
1884299233,13cyclictest0-21swapper/310:54:233
18842992315,3cyclictest0-21swapper/311:50:113
18842992311,9cyclictest0-21swapper/309:35:123
1884299231,10cyclictest35966irq/58-eth009:40:113
18842992310,9cyclictest0-21swapper/312:25:323
1883199237,4cyclictest0-21swapper/212:25:132
1883199234,9cyclictest27-21ksoftirqd/210:57:152
1883199234,15cyclictest31017-21find08:30:122
1883199232,9cyclictest7036-21sh11:52:592
1883199232,12cyclictest27-21ksoftirqd/209:18:052
18831992314,7cyclictest25092-21apt-get07:50:142
18831992314,6cyclictest27216-21cron09:00:012
18831992312,8cyclictest1014-21nfs_client10:00:222
1883199231,19cyclictest16618-21find08:15:112
1883199231,17cyclictest10572-21find07:35:112
1882399236,8cyclictest21-21ksoftirqd/112:13:531
1882399236,7cyclictest21-21ksoftirqd/110:01:451
1882399232,5cyclictest21-21ksoftirqd/110:39:011
18823992317,4cyclictest21-21ksoftirqd/107:20:111
1884299223,11cyclictest35966irq/58-eth008:25:123
1884299223,11cyclictest35966irq/58-eth008:25:113
18842992216,2cyclictest0-21swapper/309:50:113
18842992214,3cyclictest0-21swapper/308:15:123
18842992213,6cyclictest0-21swapper/308:00:123
18842992211,8cyclictest0-21swapper/311:55:113
1883199229,9cyclictest27346irq/40-dwc2_hso08:45:132
1883199228,11cyclictest28641-21df_inode07:20:132
1883199224,6cyclictest27-21ksoftirqd/209:53:292
1883199223,15cyclictest0-21swapper/207:40:112
18831992215,4cyclictest7290-21apt-get12:30:132
18831992211,4cyclictest0-21swapper/208:10:172
1883199221,11cyclictest3782-21kthreadcore08:00:202
1882399229,7cyclictest21-21ksoftirqd/107:30:211
1882399229,6cyclictest131rcu_preempt11:43:241
1882399228,6cyclictest131rcu_preempt12:05:011
1882399227,8cyclictest21-21ksoftirqd/111:15:481
1882399227,7cyclictest131rcu_preempt11:20:081
1882399226,8cyclictest21-21ksoftirqd/110:21:541
1882399226,5cyclictest21-21ksoftirqd/111:31:381
1882399225,6cyclictest21-21ksoftirqd/111:25:401
1882399223,10cyclictest21-21ksoftirqd/109:40:331
1882399221,18cyclictest7111-21kthreadcore10:05:191
1880799223,13cyclictest844-21munin-node12:20:130
1880799222,16cyclictest7428-21kworker/u8:210:30:140
1880799222,16cyclictest7428-21kworker/u8:210:30:140
18842992116,2cyclictest0-21swapper/311:20:133
18842992116,2cyclictest0-21swapper/310:10:103
18842992114,3cyclictest0-21swapper/308:05:133
18842992111,7cyclictest35966irq/58-eth011:35:133
1883199212,7cyclictest0-21swapper/207:45:112
1883199212,15cyclictest0-21swapper/208:50:102
1883199212,15cyclictest0-21swapper/208:50:092
1883199212,13cyclictest0-21swapper/208:40:132
1883199211,9cyclictest31121-21wc09:00:262
1883199211,16cyclictest24764-21missed_timers08:20:232
1883199211,11cyclictest3640-21sh12:10:132
1883199211,10cyclictest23852-21grep07:15:162
1882399216,7cyclictest21-21ksoftirqd/108:20:171
1882399216,4cyclictest21-21ksoftirqd/112:18:161
1882399216,4cyclictest21-21ksoftirqd/112:18:151
1882399216,3cyclictest131rcu_preempt09:00:281
1882399215,8cyclictest21-21ksoftirqd/110:31:591
1882399215,8cyclictest21-21ksoftirqd/110:31:591
1882399212,15cyclictest32575-21diskmemload11:50:141
1882399212,10cyclictest21-21ksoftirqd/111:12:481
18823992110,4cyclictest21-21ksoftirqd/108:50:271
18823992110,4cyclictest21-21ksoftirqd/108:50:261
1880799211,17cyclictest8538-21sed08:40:170
1880799211,16cyclictest2486-21ls08:00:170
1884299208,9cyclictest0-21swapper/309:16:273
1884299203,12cyclictest0-21swapper/309:33:523
1884299203,11cyclictest0-21swapper/312:35:133
1884299203,10cyclictest35966irq/58-eth009:20:123
1884299202,14cyclictest35966irq/58-eth008:20:113
18842992016,2cyclictest0-21swapper/312:15:123
18842992016,2cyclictest0-21swapper/312:15:123
18842992015,3cyclictest35966irq/58-eth011:45:123
18842992015,2cyclictest0-21swapper/308:55:123
18842992014,3cyclictest0-21swapper/307:40:123
18842992013,4cyclictest0-21swapper/308:45:133
1883199201,11cyclictest1210-21kthreadcore09:05:202
1882399208,4cyclictest21-21ksoftirqd/112:23:301
1882399208,3cyclictest21-21ksoftirqd/108:35:161
1882399207,7cyclictest21-21ksoftirqd/108:55:181
1882399207,4cyclictest21-21ksoftirqd/108:15:171
1882399206,4cyclictest21-21ksoftirqd/112:05:361
1882399202,7cyclictest21-21ksoftirqd/107:50:221
1882399201,15cyclictest1944-21ssh12:25:261
18823992011,3cyclictest0-21swapper/108:25:151
18823992011,3cyclictest0-21swapper/108:25:151
1880799202,11cyclictest0-21swapper/007:40:140
18807992013,4cyclictest12-21ksoftirqd/009:25:300
1884299199,8cyclictest0-21swapper/312:01:383
1884299198,8cyclictest0-21swapper/311:41:223
1884299198,8cyclictest0-21swapper/310:09:393
1884299195,8cyclictest35966irq/58-eth010:15:143
18842991913,3cyclictest0-21swapper/307:30:133
18842991911,5cyclictest35966irq/58-eth008:30:133
18842991911,5cyclictest0-21swapper/312:30:243
1884299191,13cyclictest0-21swapper/307:20:213
1882399196,2cyclictest131rcu_preempt07:10:011
1882399195,3cyclictest131rcu_preempt09:05:251
1882399194,3cyclictest21-21ksoftirqd/108:45:231
1882399192,13cyclictest31026-21apt-get08:30:131
18823991912,4cyclictest0-21swapper/109:50:041
1880799198,8cyclictest28298-21cron07:20:010
1880799196,3cyclictest131rcu_preempt10:15:260
1880799191,9cyclictest16934-21kthreadcore09:15:210
18807991911,6cyclictest12016-21sshd11:37:220
1880799191,11cyclictest20361-21grep08:15:280
1884299188,7cyclictest0-21swapper/310:21:313
1884299187,8cyclictest0-21swapper/311:25:163
1884299181,13cyclictest0-21swapper/307:50:173
1884299181,12cyclictest0-21swapper/307:15:163
18842991810,6cyclictest0-21swapper/308:40:123
1882399187,4cyclictest21-21ksoftirqd/107:35:201
1882399181,8cyclictest0-21swapper/108:05:231
18823991812,4cyclictest0-21swapper/107:55:191
18807991814,2cyclictest0-21swapper/010:45:190
18807991810,5cyclictest11081-21grep07:35:170
1884299172,9cyclictest0-21swapper/307:45:053
1884299172,9cyclictest0-21swapper/307:35:123
18842991710,3cyclictest0-21swapper/307:10:123
1882399173,11cyclictest24039-21irqcore07:15:171
1882399172,4cyclictest21-21ksoftirqd/107:40:271
1880799179,5cyclictest29156-21irqrtprio07:20:170
1880799179,5cyclictest16097-21tr11:40:200
1880799172,9cyclictest0-21swapper/008:20:120
1880799172,10cyclictest0-21swapper/008:10:140
18807991711,4cyclictest16569-21processes08:45:250
1880799171,13cyclictest27413-21latency_hist09:00:010
1884299169,4cyclictest35966irq/58-eth008:35:123
1884299168,6cyclictest0-21swapper/308:10:303
1884299164,9cyclictest0-21swapper/308:50:103
1884299164,9cyclictest0-21swapper/308:50:103
1884299162,9cyclictest0-21swapper/307:25:153
1883199161,8cyclictest18794-21cyclictest07:10:022
1882399161,2cyclictest131rcu_preempt07:10:141
18823991611,3cyclictest0-21swapper/108:10:171
18823991611,3cyclictest0-21swapper/108:00:151
1880799162,11cyclictest3540-21fschecks_count08:35:160
1880799162,11cyclictest25741-21kernelversion07:50:190
18807991611,3cyclictest17989-21fschecks_time08:50:130
18807991611,3cyclictest17989-21fschecks_time08:50:130
1880799161,12cyclictest18862-21latency_hist07:10:020
1884299115,3cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional