You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 14:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Sep 01, 2026 12:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1472599662,55cyclictest27035-21ssh11:30:151
14741995630,15cyclictest35966irq/58-eth012:30:133
1472599524,5cyclictest131rcu_preempt12:25:131
1472599521,42cyclictest27778-21ssh09:25:151
14733995024,19cyclictest0-21swapper/209:55:122
14741994827,17cyclictest35966irq/58-eth011:30:143
1472599483,8cyclictest131rcu_preempt07:30:131
14725994818,21cyclictest21-21ksoftirqd/110:25:141
1471299461,37cyclictest5145-21ssh11:00:130
1472599452,35cyclictest21624-21rm11:45:131
14741994321,15cyclictest35966irq/58-eth010:00:133
14733994333,6cyclictest22446-21ssh09:40:122
1472599435,29cyclictest2153-21ssh12:10:131
1472599435,29cyclictest2153-21ssh12:10:131
1472599412,4cyclictest131rcu_preempt10:30:121
1473399401,16cyclictest8951-21ssh11:20:122
1473399401,16cyclictest8951-21ssh11:20:122
1472599404,29cyclictest29726-21ssh10:20:111
14733993912,18cyclictest27-21ksoftirqd/210:45:122
1474199383,23cyclictest35966irq/58-eth011:35:153
1474199382,22cyclictest35966irq/58-eth012:35:143
14733993724,9cyclictest605-21sh09:10:132
14741993621,8cyclictest35966irq/58-eth009:25:153
1473399363,21cyclictest27346irq/40-dwc2_hso07:20:142
14733993610,6cyclictest5140-21sh11:00:122
1472599362,26cyclictest18863-21ssh09:20:151
1472599361,28cyclictest5863-21sshd12:30:131
14741993527,5cyclictest35966irq/58-eth010:55:133
1471299354,24cyclictest611-21ssh09:10:130
1474199343,23cyclictest35966irq/58-eth012:00:143
14733993413,15cyclictest27-21ksoftirqd/210:15:142
1472599342,25cyclictest14863-21ssh12:35:141
1471299342,23cyclictest28481-21ssh10:55:140
1473399338,14cyclictest0-21swapper/209:15:152
1473399338,14cyclictest0-21swapper/209:15:142
1473399335,15cyclictest25505-21ssh12:05:132
1472599332,24cyclictest10277-21ssh10:45:131
1471299336,21cyclictest26215-21ssh10:00:130
1471299331,24cyclictest9691-21ssh09:15:140
1471299331,24cyclictest9691-21ssh09:15:140
1471299331,21cyclictest12-21ksoftirqd/011:50:130
1471299331,21cyclictest12-21ksoftirqd/011:50:130
1474199321,18cyclictest35966irq/58-eth007:55:113
1473399321,20cyclictest32659-21diskmemload12:30:152
1471299322,22cyclictest24538-21usage_Anybus12:20:280
14712993221,6cyclictest0-21swapper/009:20:130
1474199314,17cyclictest35966irq/58-eth009:15:123
1474199314,17cyclictest35966irq/58-eth009:15:123
14741993117,11cyclictest35966irq/58-eth009:40:123
1473399310,7cyclictest16094-21munin-run12:00:012
1471299312,23cyclictest31541-21ssh09:45:160
14712993112,12cyclictest5856-21apt-get12:30:140
1474199303,18cyclictest35966irq/58-eth010:10:133
1473399304,14cyclictest27346irq/40-dwc2_hso09:00:112
1473399302,21cyclictest0-21swapper/208:05:132
1473399301,23cyclictest24553-21ssh10:35:162
1472599302,20cyclictest22450-21ssh09:40:121
1472599302,19cyclictest0-21swapper/110:05:141
1472599301,21cyclictest16519-21ssh12:00:141
1474199293,15cyclictest35966irq/58-eth011:50:123
1474199293,15cyclictest35966irq/58-eth011:50:123
1474199291,17cyclictest35966irq/58-eth011:55:123
1473399294,14cyclictest27346irq/40-dwc2_hso08:40:152
1473399291,17cyclictest29193-21munin-node12:25:122
1473399291,11cyclictest19408-21ssh10:30:272
14741992813,10cyclictest35966irq/58-eth011:00:143
14741992810,12cyclictest0-21swapper/310:23:283
1473399283,19cyclictest14702-21cyclictest11:09:592
1473399283,17cyclictest27346irq/40-dwc2_hso09:05:142
1473399282,20cyclictest0-21swapper/211:25:132
1473399281,18cyclictest7714-21cut08:10:162
1473399281,16cyclictest27346irq/40-dwc2_hso10:10:132
1472599281,21cyclictest7220-21ssh11:18:281
1471299282,19cyclictest8567-21ssh10:43:280
1471299282,19cyclictest8567-21ssh10:43:280
14741992717,6cyclictest35966irq/58-eth011:45:153
1474199271,15cyclictest35966irq/58-eth010:30:133
14741992710,8cyclictest35966irq/58-eth011:15:133
14741992710,11cyclictest0-21swapper/312:25:123
1473399274,14cyclictest27346irq/40-dwc2_hso09:45:142
1473399272,19cyclictest0-21swapper/207:50:142
1473399271,19cyclictest30754-21ssh11:50:142
1473399271,19cyclictest30754-21ssh11:50:142
1472599274,12cyclictest3613-21ssh11:35:141
1472599272,19cyclictest26683-21ssh12:22:281
14741992620,2cyclictest0-21swapper/310:40:133
14741992620,2cyclictest0-21swapper/310:40:133
1473399263,16cyclictest27346irq/40-dwc2_hso10:00:132
1473399261,5cyclictest27-21ksoftirqd/207:45:162
1473399261,15cyclictest4365-21ssh09:30:132
1472599262,20cyclictest23290-21ssh11:10:151
14725992612,9cyclictest8257-21df09:50:151
1471299265,17cyclictest6401-21apt-get10:25:130
1471299262,19cyclictest29231-21apt-get07:25:120
1471299262,18cyclictest31858-21ssh10:38:280
14712992620,3cyclictest32298-21ssh11:15:130
1473399259,3cyclictest131rcu_preempt12:03:482
1473399259,10cyclictest131rcu_preempt12:13:122
1473399259,10cyclictest131rcu_preempt12:13:122
1473399256,11cyclictest27-21ksoftirqd/210:21:182
1473399253,13cyclictest27-21ksoftirqd/211:37:592
1473399252,8cyclictest20159-21apt-get12:20:112
1473399252,16cyclictest0-21swapper/212:15:102
14733992511,11cyclictest13553-21kthreadcore11:40:212
1472599255,12cyclictest19331-21ssh10:50:121
1472599251,13cyclictest1201-21ssh10:40:131
1472599251,13cyclictest1201-21ssh10:40:131
1471299253,15cyclictest0-21swapper/009:35:140
1471299252,18cyclictest16267-21ssh11:23:280
1471299252,18cyclictest16267-21ssh11:23:280
1471299251,6cyclictest0-21swapper/009:00:120
14712992515,7cyclictest16412-21sshd12:00:080
1474199243,12cyclictest35966irq/58-eth007:45:133
1473399249,4cyclictest27-21ksoftirqd/210:59:022
1473399248,4cyclictest27-21ksoftirqd/211:15:442
1473399248,4cyclictest27-21ksoftirqd/210:51:482
1473399246,12cyclictest27-21ksoftirqd/210:41:062
1473399246,12cyclictest27-21ksoftirqd/210:41:062
1473399245,12cyclictest27-21ksoftirqd/209:39:102
1473399244,4cyclictest27-21ksoftirqd/208:55:242
1473399243,8cyclictest131rcu_preempt09:25:232
1473399242,15cyclictest0-21swapper/211:45:052
1473399242,11cyclictest131rcu_preempt09:50:222
1473399241,7cyclictest32659-21diskmemload09:22:072
14733992413,8cyclictest0-21swapper/208:30:142
14733992413,8cyclictest0-21swapper/208:15:172
14733992413,8cyclictest0-21swapper/208:15:172
14733992412,9cyclictest0-21swapper/208:25:152
1473399241,18cyclictest3817-21latency_hist08:40:012
1471299244,5cyclictest12-21ksoftirqd/012:15:580
1471299243,4cyclictest12-21ksoftirqd/011:55:210
14712992417,4cyclictest1581-21apt-get07:30:140
14712992410,7cyclictest630-21Async09:36:310
1471299240,7cyclictest22421-21apt-get09:40:120
1474199231,14cyclictest0-21swapper/309:10:123
1473399236,8cyclictest131rcu_preempt08:45:152
1473399232,6cyclictest0-21swapper/212:35:282
14733992313,8cyclictest0-21swapper/210:25:122
14733992313,8cyclictest0-21swapper/207:30:152
1473399231,12cyclictest0-21swapper/207:42:202
1472599232,15cyclictest28256-21ntp_states10:35:281
1472599232,15cyclictest11107-21ssh12:15:101
1472599231,17cyclictest15965-21find07:45:121
1471299235,7cyclictest131rcu_preempt11:41:490
1471299235,4cyclictest12-21ksoftirqd/011:11:220
1471299235,2cyclictest131rcu_preempt10:30:540
1471299234,5cyclictest12-21ksoftirqd/009:50:330
1471299234,4cyclictest12-21ksoftirqd/008:20:180
1471299233,6cyclictest12-21ksoftirqd/011:35:140
1471299233,6cyclictest12-21ksoftirqd/010:12:180
1471299233,5cyclictest12-21ksoftirqd/007:50:010
1471299232,6cyclictest12-21ksoftirqd/008:35:160
1471299232,5cyclictest12-21ksoftirqd/010:50:130
1471299231,7cyclictest12-21ksoftirqd/012:05:300
14712992317,4cyclictest151rcuc/010:20:540
14712992317,3cyclictest17953-21apt-get11:25:130
1471299231,6cyclictest12-21ksoftirqd/009:55:210
1471299231,6cyclictest12-21ksoftirqd/009:25:420
14712992311,10cyclictest151rcuc/011:50:010
14712992310,7cyclictest0-21swapper/012:13:570
14712992310,7cyclictest0-21swapper/012:13:570
1474199229,7cyclictest35966irq/58-eth010:25:143
1474199226,11cyclictest33-21ksoftirqd/309:55:123
1474199223,13cyclictest0-21swapper/310:38:283
1473399227,10cyclictest131rcu_preempt11:12:142
1473399227,10cyclictest131rcu_preempt07:35:282
1473399227,10cyclictest131rcu_preempt07:35:282
1473399225,13cyclictest131rcu_preempt10:05:142
1473399221,8cyclictest0-21swapper/207:55:172
1473399221,18cyclictest29378-21fschecks_count07:25:152
1473399221,15cyclictest27-21ksoftirqd/211:33:032
1472599222,14cyclictest17564-21munin-run11:25:011
1472599222,14cyclictest17564-21munin-run11:25:011
1472599221,5cyclictest0-21swapper/108:20:191
1472599221,15cyclictest32514-21ls10:55:301
14725992210,8cyclictest0-21swapper/109:10:221
1471299227,4cyclictest12-21ksoftirqd/010:15:410
1471299223,5cyclictest12-21ksoftirqd/012:25:520
1471299223,5cyclictest12-21ksoftirqd/008:50:250
1471299223,5cyclictest12-21ksoftirqd/007:55:130
1471299223,4cyclictest12-21ksoftirqd/011:05:150
1471299222,8cyclictest131rcu_preempt09:30:280
1471299222,4cyclictest12-21ksoftirqd/012:35:310
14712992213,4cyclictest10250-21apt-get10:45:120
1471299221,17cyclictest22039-21apt-get08:25:130
1471299220,19cyclictest20916-21ls07:50:140
1471299220,10cyclictest78461ls08:40:280
1474199212,10cyclictest35966irq/58-eth009:35:133
1474199211,2cyclictest0-21swapper/307:35:133
1474199211,2cyclictest0-21swapper/307:35:133
14741992110,8cyclictest35966irq/58-eth010:45:133
1473399211,5cyclictest27-21ksoftirqd/207:10:182
1473399211,5cyclictest27-21ksoftirqd/207:10:182
1473399211,15cyclictest30441-21apt-get08:00:142
1473399211,10cyclictest13738-21sort08:50:142
1472599219,3cyclictest21-21ksoftirqd/107:40:111
1472599215,10cyclictest844-21munin-node08:55:141
1472599212,10cyclictest14196-21ssh11:05:151
1472599211,8cyclictest32246-21irqstats09:45:191
1471299212,4cyclictest12-21ksoftirqd/010:05:310
1471299211,6cyclictest12-21ksoftirqd/007:20:170
1471299211,17cyclictest30432-21find08:00:120
1474199208,6cyclictest36066irq/59-eth009:20:153
1474199201,12cyclictest35966irq/58-eth010:05:143
1473399201,16cyclictest24347-21latency_hist07:20:012
1472599208,8cyclictest26586-21ps10:00:181
1472599206,8cyclictest0-21swapper/107:35:141
1472599206,8cyclictest0-21swapper/107:35:141
1472599204,9cyclictest0-21swapper/107:25:131
1472599202,13cyclictest20685-21memory09:55:231
14725992012,4cyclictest0-21swapper/111:25:121
1472599201,16cyclictest11718-21sh10:10:131
1472599201,10cyclictest13791-21irqcore09:35:181
1471299205,5cyclictest12-21ksoftirqd/009:05:340
1471299203,4cyclictest12-21ksoftirqd/007:10:200
1471299203,4cyclictest12-21ksoftirqd/007:10:200
1471299201,9cyclictest0-21swapper/008:10:150
1471299201,5cyclictest12-21ksoftirqd/007:15:250
1471299201,4cyclictest151rcuc/011:32:280
1471299201,4cyclictest12-21ksoftirqd/008:45:270
1471299201,15cyclictest2799-21apt-get08:05:110
1474199199,6cyclictest0-21swapper/312:05:163
1474199199,6cyclictest0-21swapper/309:46:133
1474199193,9cyclictest0-21swapper/312:20:283
1474199193,11cyclictest0-21swapper/311:25:023
1474199193,11cyclictest0-21swapper/311:23:283
1474199193,11cyclictest0-21swapper/311:23:283
1474199193,11cyclictest0-21swapper/310:52:283
1474199193,10cyclictest0-21swapper/312:15:103
1473399191,15cyclictest17453-21hwlatdetect08:20:162
1472599198,8cyclictest16803-21sshd07:10:201
1472599198,8cyclictest16803-21sshd07:10:201
1472599192,14cyclictest12657-21ssh11:40:141
1472599191,5cyclictest32347-21kthreadcore08:00:221
1472599191,14cyclictest24544-21awk10:15:281
1472599191,14cyclictest23301-21apt-get09:00:101
14725991910,6cyclictest11680-21kthreadcore09:15:211
14725991910,6cyclictest11680-21kthreadcore09:15:211
14725991910,6cyclictest0-21swapper/108:25:251
1471299191,6cyclictest12-21ksoftirqd/008:55:200
1471299191,5cyclictest12-21ksoftirqd/008:15:280
1471299191,5cyclictest12-21ksoftirqd/008:15:280
1471299191,15cyclictest13297-21kthreadcore07:40:200
1474199189,6cyclictest35966irq/58-eth012:10:153
1474199189,6cyclictest35966irq/58-eth012:10:143
1474199182,10cyclictest0-21swapper/309:00:123
1474199182,10cyclictest0-21swapper/308:50:143
1474199182,10cyclictest0-21swapper/308:35:163
1474199182,10cyclictest0-21swapper/308:10:133
1474199182,10cyclictest0-21swapper/307:30:143
1474199182,10cyclictest0-21swapper/307:20:153
1474199181,13cyclictest0-21swapper/311:45:023
1474199181,12cyclictest0-21swapper/311:05:143
1474199181,12cyclictest0-21swapper/309:50:243
14741991810,6cyclictest0-21swapper/311:10:463
14741991810,6cyclictest0-21swapper/310:15:163
14741991810,6cyclictest0-21swapper/309:31:023
1473399180,14cyclictest14769-21latency_hist07:10:012
1472599189,6cyclictest12188-21cron08:15:011
1472599188,8cyclictest19735-21df_inode07:15:151
1472599187,8cyclictest28075-21apt-get09:05:141
1472599182,13cyclictest4654-21irqrtprio08:40:181
1472599182,10cyclictest30863-21ssh11:50:151
1472599182,10cyclictest30863-21ssh11:50:151
1472599181,6cyclictest29234-21awk12:05:261
14725991812,4cyclictest0-21swapper/108:45:161
1472599181,13cyclictest7448-21ssh11:55:121
1472599181,12cyclictest31983-21tr08:35:191
14725991810,5cyclictest7518-21kthreadcore09:30:221
1471299180,4cyclictest12-21ksoftirqd/007:35:280
1471299180,4cyclictest12-21ksoftirqd/007:35:280
1474199172,9cyclictest0-21swapper/307:50:043
1474199172,10cyclictest0-21swapper/308:30:143
1474199172,10cyclictest0-21swapper/307:40:133
1474199171,12cyclictest0-21swapper/307:25:133
1472599172,13cyclictest30196-21awk08:00:021
1472599172,11cyclictest26824-21apt-get08:30:131
1472599171,13cyclictest5888-21ssh11:00:211
1474199162,9cyclictest0-21swapper/308:25:143
1474199162,10cyclictest0-21swapper/308:20:143
1474199161,10cyclictest0-21swapper/308:55:143
1472599162,11cyclictest20957-21fschecks_count07:50:161
1472599162,10cyclictest0-21swapper/108:05:131
1472599161,2cyclictest0-21swapper/107:20:191
1472599161,12cyclictest15552-21cut08:15:241
1472599161,12cyclictest15552-21cut08:15:241
1472599161,12cyclictest14147-21sed08:50:181
1474199151,8cyclictest0-21swapper/308:15:303
1474199151,8cyclictest0-21swapper/308:15:293
1474199151,8cyclictest0-21swapper/307:10:273
1474199151,8cyclictest0-21swapper/307:10:273
1474199151,7cyclictest0-21swapper/308:00:123
1474199151,11cyclictest0-21swapper/308:40:203
1472599152,10cyclictest14799-21cat07:10:021
1471299152,10cyclictest14767-21latency_hist07:10:010
1474199149,3cyclictest33-21ksoftirqd/308:05:243
1474199131,7cyclictest0-21swapper/309:05:163
1474199122,5cyclictest0-21swapper/307:15:143
1474199121,8cyclictest0-21swapper/308:45:113
147419991,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional