You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 21:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 20, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1329299614,49cyclictest12497-21ssh12:15:140
1330399576,40cyclictest17751-21ssh09:20:121
1330399576,40cyclictest17751-21ssh09:20:111
1329299541,43cyclictest26968-21ssh09:25:130
1329299541,43cyclictest26968-21ssh09:25:130
1329299533,39cyclictest6829-21ssh10:25:150
1329299512,40cyclictest18704-21ssh11:25:130
13311995033,10cyclictest31408-21diskmemload11:40:122
1329299495,36cyclictest31205-21ssh09:45:150
1330399482,38cyclictest23962-21sh11:10:141
13311994715,21cyclictest27-21ksoftirqd/211:00:122
1329299471,34cyclictest22084-21ssh09:40:130
1330399461,40cyclictest0-21swapper/111:40:131
1330399451,5cyclictest0-21swapper/107:25:121
1330399441,38cyclictest0-21swapper/110:25:141
1329299446,26cyclictest516-21ssh11:15:150
1332399433,22cyclictest35966irq/58-eth010:15:113
1330399433,35cyclictest0-21swapper/110:35:141
1330399433,35cyclictest0-21swapper/110:35:141
1330399431,3cyclictest0-21swapper/107:45:141
1332399422,26cyclictest35966irq/58-eth011:00:133
1330399423,32cyclictest8434-21sh11:55:151
1332399414,19cyclictest35966irq/58-eth009:35:133
1330399411,4cyclictest23212-21apt-get07:20:131
13292994122,7cyclictest12-21ksoftirqd/008:35:140
1330399401,3cyclictest0-21swapper/108:10:141
1330399401,33cyclictest0-21swapper/110:30:131
1330399400,6cyclictest19551-21apt-get07:50:131
1329299402,31cyclictest15944-21ssh10:30:120
1329299402,31cyclictest10874-21ssh10:45:130
1330399390,3cyclictest0-21swapper/108:05:131
1330399384,27cyclictest14866-21ssh11:05:121
1329299387,28cyclictest1779-21ssh10:40:140
1329299384,20cyclictest12963-21ssh09:35:140
1332399371,20cyclictest35966irq/58-eth008:20:133
1331199371,25cyclictest19961-21ssh10:50:142
1330399372,27cyclictest5828-21ssh11:00:131
1330399371,5cyclictest0-21swapper/109:05:131
1332399362,18cyclictest35966irq/58-eth009:30:113
1330399363,29cyclictest0-21swapper/112:35:121
1330399361,31cyclictest30711-21rm12:25:131
1330399361,30cyclictest0-21swapper/107:55:131
1330399361,1cyclictest0-21swapper/107:35:111
1330399361,11cyclictest21713-21ntp_192.168.11407:15:251
1330399361,10cyclictest2994-21forks08:40:161
1330399360,1cyclictest0-21swapper/111:20:131
1329299363,20cyclictest26557-1kworker/u9:209:30:120
1330399352,29cyclictest21857-21kthreadcore08:25:201
1330399351,30cyclictest524-21sh11:15:161
1330399351,2cyclictest26184-21apt-get10:00:141
1330399351,2cyclictest0-21swapper/108:00:131
1330399350,3cyclictest22084-21apt-get09:00:131
1330399350,3cyclictest0-21swapper/108:15:141
1330399350,2cyclictest10883-21ssh10:45:141
1330399350,10cyclictest12737-21perl12:15:171
1332399344,19cyclictest35966irq/58-eth009:45:143
1332399342,24cyclictest35966irq/58-eth009:55:123
13323993420,8cyclictest35966irq/58-eth010:50:123
1330399344,27cyclictest1770-21ssh10:40:141
1330399342,29cyclictest32077-21idleruntime11:50:171
1330399341,8cyclictest31145-21irqstats08:35:191
1330399341,30cyclictest4299-21latency_hist11:35:021
1330399340,2cyclictest0-21swapper/112:05:131
1330399340,1cyclictest0-21swapper/109:50:121
13323993312,15cyclictest35966irq/58-eth010:00:143
1331199334,11cyclictest27-21ksoftirqd/209:05:122
13311993321,6cyclictest21568-21ssh12:20:112
1330399335,25cyclictest18163-21cut08:55:191
1330399331,8cyclictest654-21irqcore07:30:181
1330399331,8cyclictest654-21irqcore07:30:181
1330399331,8cyclictest13374-21ntp_192.168.11407:40:241
1330399331,7cyclictest15355-21kthreadcore07:10:201
1330399331,5cyclictest7793-21diskstats08:45:171
1330399331,30cyclictest16059-21grep10:10:291
1330399331,10cyclictest24806-21ssh09:40:231
1330399331,10cyclictest1752-21ssh10:55:491
1330399330,8cyclictest29021-21/usr/sbin/munin08:30:231
1330399330,5cyclictest12936-21apt-get09:35:131
13292993314,10cyclictest13287-21cyclictest11:20:140
13292993313,12cyclictest29842-21diskmemload10:00:250
1330399321,9cyclictest10245-21cut09:15:221
1330399321,8cyclictest27647-21sed09:25:191
1330399321,8cyclictest27647-21sed09:25:181
1330399321,8cyclictest24697-21kthreadcore11:45:221
1330399321,7cyclictest4482-21awk11:35:121
1330399321,6cyclictest24039-21kthreadcore10:15:201
1330399321,6cyclictest1855-21grep10:20:301
1330399321,27cyclictest20832-21kthreadcore12:00:231
1330399321,10cyclictest25511-21ssh09:59:421
1330399321,10cyclictest18820-21entropy11:25:151
1330399320,8cyclictest16209-21diskstats08:20:161
1329299322,25cyclictest4485-21ssh11:35:120
1332399316,18cyclictest35966irq/58-eth011:30:153
1332399314,13cyclictest35966irq/58-eth012:20:123
13323993115,12cyclictest35966irq/58-eth009:50:253
1331199315,16cyclictest31808-21ssh11:50:142
13311993111,12cyclictest0-21swapper/208:45:142
1330399311,9cyclictest7051-21ssh12:10:271
1330399311,7cyclictest12652-21/usr/sbin/munin08:50:151
1330399311,6cyclictest32681-21kthreadcore09:10:181
1330399311,6cyclictest1869-21kthreadcore09:45:231
1330399311,2cyclictest2858-21apt-get10:05:131
1330399311,27cyclictest22120-21cut12:20:171
1330399311,26cyclictest3754-21apt-get09:30:131
1330399310,9cyclictest13372-21date07:10:011
1332399308,18cyclictest33-21ksoftirqd/311:20:133
1332399302,17cyclictest35966irq/58-eth010:10:133
1332399302,16cyclictest35966irq/58-eth010:45:133
13311993020,7cyclictest0-21swapper/210:45:142
1330399300,9cyclictest11117-21ssh12:30:281
1329299305,3cyclictest131rcu_preempt10:35:130
1329299305,3cyclictest131rcu_preempt10:35:120
1329299301,23cyclictest17776-21ssh09:20:140
1329299301,23cyclictest17776-21ssh09:20:140
1331199292,16cyclictest3419-21ssh12:10:142
1331199291,19cyclictest26189-21ssh10:00:122
1331199291,11cyclictest22099-21ssh09:40:152
1330399291,8cyclictest20064-21df_inode10:50:151
1329299293,19cyclictest12-21ksoftirqd/007:20:110
13292992916,9cyclictest25194-21cron12:40:020
1332399283,12cyclictest35966irq/58-eth011:50:133
1332399283,11cyclictest35966irq/58-eth012:00:113
1332399282,23cyclictest35966irq/58-eth012:25:123
1331199287,11cyclictest27-21ksoftirqd/212:39:452
1331199286,8cyclictest27-21ksoftirqd/211:23:162
1331199285,13cyclictest27346irq/40-dwc2_hso11:10:142
1331199285,13cyclictest27346irq/40-dwc2_hso08:40:132
1331199282,18cyclictest31408-21diskmemload12:30:142
13311992818,4cyclictest27346irq/40-dwc2_hso10:15:122
13323992714,6cyclictest0-21swapper/311:05:143
1332399270,15cyclictest35966irq/58-eth010:25:153
1331199276,5cyclictest27-21ksoftirqd/211:08:072
1331199274,13cyclictest27346irq/40-dwc2_hso07:15:142
13311992711,8cyclictest27-21ksoftirqd/212:25:152
1331199271,16cyclictest27346irq/40-dwc2_hso10:05:142
1329299273,18cyclictest21585-21rm12:20:130
1329299272,16cyclictest12-21ksoftirqd/009:00:140
1332399264,18cyclictest35966irq/58-eth009:10:133
1332399261,15cyclictest35966irq/58-eth010:05:123
1332399261,15cyclictest35966irq/58-eth008:05:133
1331199265,13cyclictest27-21ksoftirqd/211:35:532
1331199264,12cyclictest27346irq/40-dwc2_hso09:55:132
1331199263,4cyclictest271ksoftirqd/208:35:212
1331199263,14cyclictest27-21ksoftirqd/210:10:242
1331199256,12cyclictest27-21ksoftirqd/210:22:452
1331199255,12cyclictest27346irq/40-dwc2_hso11:55:142
1331199254,3cyclictest27-21ksoftirqd/209:35:142
1331199253,2cyclictest131rcu_preempt08:00:252
1331199253,14cyclictest27346irq/40-dwc2_hso12:00:122
1331199252,17cyclictest27346irq/40-dwc2_hso09:00:132
1331199250,16cyclictest27346irq/40-dwc2_hso09:30:122
1329299251,5cyclictest11949-21ssh10:10:120
1332399243,13cyclictest0-21swapper/311:35:143
13323992414,5cyclictest0-21swapper/310:55:153
13323992411,8cyclictest35966irq/58-eth007:15:153
1331199247,7cyclictest27-21ksoftirqd/211:32:202
1331199247,4cyclictest27-21ksoftirqd/212:09:462
1331199246,12cyclictest27-21ksoftirqd/211:15:142
1331199246,12cyclictest27-21ksoftirqd/210:34:122
1331199245,12cyclictest27-21ksoftirqd/210:35:402
1331199245,12cyclictest27-21ksoftirqd/210:35:402
1331199244,11cyclictest27-21ksoftirqd/208:20:142
1331199242,16cyclictest0-21swapper/208:15:152
1331199242,15cyclictest31190-21ssh09:45:142
13311992414,3cyclictest0-21swapper/207:45:132
13311992413,4cyclictest261rcuc/209:29:352
13311992413,4cyclictest261rcuc/209:29:342
13311992410,10cyclictest11429-1kworker/u9:012:15:132
1329299244,17cyclictest22709-21ssh11:45:140
1329299243,17cyclictest2885-21ssh10:05:140
13292992415,6cyclictest0-21swapper/008:00:020
1332399239,11cyclictest0-21swapper/310:45:013
1332399235,13cyclictest0-21swapper/310:30:133
1332399235,12cyclictest36066irq/59-eth011:45:143
1332399233,13cyclictest0-21swapper/312:30:153
1331199236,8cyclictest27-21ksoftirqd/207:40:252
1331199236,3cyclictest131rcu_preempt11:47:202
1331199235,8cyclictest27-21ksoftirqd/209:16:402
1331199235,12cyclictest27-21ksoftirqd/210:26:072
1331199234,12cyclictest27-21ksoftirqd/209:24:412
1331199234,12cyclictest27-21ksoftirqd/209:24:402
1331199233,9cyclictest27-21ksoftirqd/210:55:522
1331199232,15cyclictest0-21swapper/207:30:142
1331199232,15cyclictest0-21swapper/207:30:132
1331199230,6cyclictest8119-21cut09:50:162
1329299232,18cyclictest151rcuc/012:10:150
1329299232,15cyclictest7931-21ssh09:50:130
1329299231,16cyclictest30146-21sh10:20:140
1329299231,15cyclictest25778-21ls08:30:140
1331199229,3cyclictest27-21ksoftirqd/210:43:492
13311992213,3cyclictest261rcuc/211:25:102
1331199221,17cyclictest32012-21fschecks_time09:10:162
1331199221,11cyclictest16994-21sendmail_mailqu07:10:272
1331199220,14cyclictest27346irq/40-dwc2_hso07:55:122
1329299223,16cyclictest6214-21missed_timers08:40:230
1329299221,18cyclictest26948-21ls09:05:150
1329299221,15cyclictest26616-21ssh12:05:150
1332399215,10cyclictest0-21swapper/311:15:153
1331199213,12cyclictest27346irq/40-dwc2_hso07:50:132
13311992112,3cyclictest0-21swapper/208:55:012
1331199211,16cyclictest25926-21sed08:30:162
1329299213,12cyclictest7361-21apt-get12:30:140
1329299212,15cyclictest3333-21ssh09:10:290
13292992112,6cyclictest448-21cut07:30:160
13292992112,6cyclictest448-21cut07:30:150
13292992111,6cyclictest0-21swapper/007:25:110
1329299211,15cyclictest27634-21ssh11:10:280
1329299211,14cyclictest16098-21find08:20:120
1331199204,8cyclictest25938-21kthreadcore07:20:212
1331199204,2cyclictest131rcu_preempt08:50:252
1331199202,9cyclictest8166-21kthreadcore08:10:222
1331199202,2cyclictest131rcu_preempt08:25:262
1331199201,9cyclictest3963-21kthreadcore08:05:212
1331199201,4cyclictest27-21ksoftirqd/207:35:192
1331199201,16cyclictest31830-21head07:25:272
1329299202,14cyclictest29085-21sh10:55:150
13292992013,4cyclictest12-21ksoftirqd/010:50:200
13292992012,6cyclictest18013-21cron07:15:000
1332399197,9cyclictest0-21swapper/311:25:143
1332399192,11cyclictest0-21swapper/307:25:123
1332399192,10cyclictest0-21swapper/307:50:133
13323991913,4cyclictest33-21ksoftirqd/309:25:293
13323991913,4cyclictest33-21ksoftirqd/309:25:293
13323991911,6cyclictest0-21swapper/312:13:123
13323991911,5cyclictest35966irq/58-eth009:40:133
1332399191,12cyclictest0-21swapper/309:20:143
1332399191,12cyclictest0-21swapper/309:20:143
1329299191,9cyclictest14172-21irqrtprio11:40:180
1329299191,8cyclictest19689-21kthreadcore07:15:210
13292991912,5cyclictest2985-21ssh11:50:250
13292991912,4cyclictest12-21ksoftirqd/007:55:000
13292991911,6cyclictest22685-21sshd11:09:000
1329299191,14cyclictest12295-21sh09:15:290
1329299191,13cyclictest1824-21ssh12:25:240
1332399189,6cyclictest0-21swapper/311:41:363
1332399182,12cyclictest33-21ksoftirqd/307:20:113
1332399182,10cyclictest0-21swapper/309:05:153
1332399182,10cyclictest0-21swapper/309:00:143
1332399182,10cyclictest0-21swapper/308:40:143
1332399182,10cyclictest0-21swapper/308:10:043
1332399182,10cyclictest0-21swapper/307:55:143
1332399182,10cyclictest0-21swapper/307:40:143
13323991810,5cyclictest35966irq/58-eth012:35:133
13323991810,5cyclictest35966irq/58-eth009:15:063
13323991810,3cyclictest0-21swapper/312:16:573
1329299189,7cyclictest17609-21df_inode12:00:150
1329299189,6cyclictest17471-21kworker/u8:010:15:100
1329299181,9cyclictest4695-21kthreadcore08:05:220
1329299181,9cyclictest31976-21unixbench_multi11:30:320
1329299181,3cyclictest131rcu_preempt11:00:110
13292991812,4cyclictest0-21swapper/008:50:170
13292991811,5cyclictest16206-21sshd11:59:000
1329299181,14cyclictest17249-21apt-get08:55:120
1329299181,10cyclictest6727-21ls08:10:170
1332399179,6cyclictest0-21swapper/312:06:063
1332399179,6cyclictest0-21swapper/311:14:193
1332399179,5cyclictest35966irq/58-eth010:20:123
1332399172,9cyclictest0-21swapper/308:15:153
1332399172,10cyclictest0-21swapper/310:35:143
1332399172,10cyclictest0-21swapper/310:35:133
1332399172,10cyclictest0-21swapper/308:55:143
1332399172,10cyclictest0-21swapper/307:45:143
13323991711,3cyclictest0-21swapper/311:55:303
1331199171,8cyclictest13358-21latency_hist07:10:022
1329299179,5cyclictest121ksoftirqd/008:45:140
1329299172,10cyclictest0-21swapper/008:00:030
1329299171,8cyclictest20903-21/usr/sbin/munin08:25:170
1329299171,6cyclictest16094-21kthreadcore07:45:200
1329299171,4cyclictest0-21swapper/007:36:340
1329299171,13cyclictest10471-21irqrtprio07:40:180
1332399162,10cyclictest0-21swapper/307:30:143
1332399162,10cyclictest0-21swapper/307:30:133
1332399161,12cyclictest0-21swapper/308:45:243
1332399161,12cyclictest0-21swapper/307:10:223
1329299163,11cyclictest17098-21apt-get09:55:120
1329299151,11cyclictest11475-21tune2fs08:15:170
1329299151,10cyclictest13388-21latency_hist07:10:020
1332399149,3cyclictest0-21swapper/308:27:363
1332399141,11cyclictest0-21swapper/308:55:013
1332399141,11cyclictest0-21swapper/308:35:123
1332399141,10cyclictest0-21swapper/307:35:183
1332399138,3cyclictest0-21swapper/308:33:003
1332399131,8cyclictest0-21swapper/308:00:033
1332399101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional