You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 19:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Sep 24, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1319099794,7cyclictest131rcu_preempt10:25:120
1319099762,60cyclictest15857-21ssh09:55:130
1319099692,31cyclictest6224-21ssh11:20:110
1319099645,45cyclictest22854-21ssh10:35:120
1319099641,9cyclictest131rcu_preempt09:40:120
13218995845,10cyclictest35966irq/58-eth012:20:103
13218995441,7cyclictest35966irq/58-eth011:30:183
1319899524,39cyclictest3074-21ssh09:30:131
1319099492,33cyclictest11387-21ssh12:35:110
1321899482,33cyclictest35966irq/58-eth008:00:133
1319099471,38cyclictest6765-21ssh09:50:110
1319099441,4cyclictest131rcu_preempt12:25:120
1319099441,4cyclictest131rcu_preempt12:25:110
13198994310,3cyclictest131rcu_preempt09:25:131
1321899425,22cyclictest35966irq/58-eth012:00:123
1319099422,33cyclictest17506-21ssh09:20:120
1319099422,33cyclictest17506-21ssh09:20:120
1321099412,30cyclictest17263-21kworker/u8:011:25:132
1319099411,8cyclictest131rcu_preempt07:25:120
1319099411,33cyclictest2855-21ssh11:00:150
1319099411,23cyclictest16854-21ssh12:20:100
13218994026,9cyclictest35966irq/58-eth009:10:113
1319899408,5cyclictest131rcu_preempt11:35:131
13218993918,12cyclictest0-21swapper/311:35:133
13218993819,11cyclictest35966irq/58-eth011:25:133
1321099382,18cyclictest17019-21ssh09:35:362
1319099378,20cyclictest8259-21ssh10:45:120
13190993716,9cyclictest12-21ksoftirqd/011:30:160
1321899362,20cyclictest35966irq/58-eth011:45:113
13210993614,14cyclictest261rcuc/212:00:132
1321899354,21cyclictest35966irq/58-eth011:20:113
13210993523,3cyclictest0-21swapper/209:55:122
1319899353,25cyclictest19427-21ssh10:15:151
1319099352,25cyclictest30236-21ls09:45:150
1319099352,25cyclictest30236-21ls09:45:140
1319099352,22cyclictest12-21ksoftirqd/010:40:120
13190993520,6cyclictest0-21swapper/010:30:160
1321899346,21cyclictest35966irq/58-eth010:25:133
1321099348,14cyclictest0-21swapper/211:55:112
1319899341,25cyclictest9725-21rm11:40:121
1319899341,19cyclictest20904-21ssh11:10:141
13218993324,5cyclictest33-21ksoftirqd/309:20:133
13218993324,5cyclictest33-21ksoftirqd/309:20:123
1321899331,18cyclictest35966irq/58-eth009:30:133
13218993215,13cyclictest0-21swapper/311:40:123
13218993213,10cyclictest0-21swapper/308:50:123
13218993213,10cyclictest0-21swapper/308:50:113
1321099323,21cyclictest27-21ksoftirqd/210:15:412
1319099324,18cyclictest12-21ksoftirqd/010:05:120
1319099323,25cyclictest24862-21ssh10:00:140
1321899312,20cyclictest35966irq/58-eth009:55:133
1321899311,17cyclictest35966irq/58-eth011:15:103
1321899311,17cyclictest35966irq/58-eth011:15:093
1321099313,18cyclictest27346irq/40-dwc2_hso08:55:132
13210993111,10cyclictest27-21ksoftirqd/211:15:112
13210993111,10cyclictest27-21ksoftirqd/211:15:102
1319899311,4cyclictest12075-21ssh09:35:111
1319899311,27cyclictest30156-21ssh09:45:131
1319899311,27cyclictest30156-21ssh09:45:121
1319099312,22cyclictest27762-21ssh11:50:150
1321899305,19cyclictest35966irq/58-eth010:40:113
1321099306,12cyclictest27346irq/40-dwc2_hso08:25:132
1319899304,17cyclictest29203-21sshd08:00:151
1319099302,22cyclictest2035-21diskmemload09:29:150
1319099301,22cyclictest2464-21ssh12:30:150
13218992923,3cyclictest0-21swapper/309:50:103
1321899291,20cyclictest35966irq/58-eth011:05:133
1321099293,17cyclictest27346irq/40-dwc2_hso07:55:132
1321099292,17cyclictest27346irq/40-dwc2_hso08:10:122
1321099291,22cyclictest0-21swapper/207:30:132
1321099291,22cyclictest0-21swapper/207:30:122
1319899292,20cyclictest29793-21ssh11:15:101
1319899292,20cyclictest29793-21ssh11:15:101
1319899291,11cyclictest21-21ksoftirqd/108:10:131
1321899284,13cyclictest35966irq/58-eth007:50:133
13210992814,9cyclictest0-21swapper/210:30:162
13198992812,8cyclictest23057-21ssh10:35:151
1321899273,16cyclictest0-21swapper/310:15:423
13218992721,4cyclictest0-21swapper/311:50:143
13218992719,5cyclictest0-21swapper/310:45:113
13218992717,7cyclictest0-21swapper/310:05:123
13218992717,6cyclictest0-21swapper/311:00:153
13218992713,7cyclictest0-21swapper/309:40:133
1321099272,18cyclictest930-21snmpd08:30:142
1321099271,8cyclictest870166ssh09:15:152
13198992720,3cyclictest21-21ksoftirqd/107:50:121
1319099275,13cyclictest0-21swapper/007:45:140
1321899264,12cyclictest0-21swapper/309:15:123
1321899262,20cyclictest35966irq/58-eth009:45:143
1321899262,20cyclictest35966irq/58-eth009:45:133
13218992620,3cyclictest0-21swapper/310:35:123
13218992613,7cyclictest0-21swapper/311:10:163
1319899263,3cyclictest131rcu_preempt09:05:141
1319899262,20cyclictest17522-21ssh09:20:141
1319899262,20cyclictest17522-21ssh09:20:131
1319899261,18cyclictest844-21munin-node07:30:141
1319899261,18cyclictest844-21munin-node07:30:131
1319099262,18cyclictest12188-21df_inode09:35:150
13190992615,7cyclictest0-21swapper/009:30:120
1319099261,19cyclictest746-21ssh11:35:130
13218992518,3cyclictest0-21swapper/309:25:133
1321899251,13cyclictest35966irq/58-eth012:25:113
1321899251,13cyclictest35966irq/58-eth012:25:113
1321099259,3cyclictest27-21ksoftirqd/211:05:152
1321099252,13cyclictest3079-21ssh09:30:132
1321099251,7cyclictest2343066ssh12:22:342
13210992514,8cyclictest0-21swapper/207:25:252
13210992510,6cyclictest0-21swapper/212:12:562
1319899257,7cyclictest156-21systemd-udevd12:34:481
1319099253,14cyclictest28422-21ssh10:20:130
1321899243,14cyclictest0-21swapper/309:35:353
1321899242,14cyclictest35966irq/58-eth007:55:133
13218992414,7cyclictest0-21swapper/311:55:113
1321099247,11cyclictest0-21swapper/209:20:132
1321099247,11cyclictest0-21swapper/209:20:122
1321099243,16cyclictest0-21swapper/209:52:152
1321099242,6cyclictest0-21swapper/209:42:132
1321099242,17cyclictest930-21snmpd07:50:122
1321099242,16cyclictest0-21swapper/210:46:142
1321099242,15cyclictest27346irq/40-dwc2_hso08:05:112
1321099241,7cyclictest1712466ssh10:12:452
1321099241,6cyclictest641866ssh11:36:302
1321099241,6cyclictest2756866ssh12:05:592
13210992413,9cyclictest0-21swapper/211:42:072
13210992412,5cyclictest29426-21ssh11:15:002
1321099241,19cyclictest27087-21ls09:05:162
1321099240,6cyclictest2440866rm11:46:282
1321099240,14cyclictest32216-21uniq09:10:142
13198992414,7cyclictest25268-21head09:40:291
1319099241,17cyclictest8591-21ssh09:15:130
13218992319,2cyclictest0-21swapper/312:10:173
13218992318,2cyclictest0-21swapper/310:20:123
13218992318,2cyclictest0-21swapper/308:30:133
13218992317,2cyclictest0-21swapper/308:45:123
13218992316,2cyclictest0-21swapper/308:20:133
13218992315,6cyclictest0-21swapper/310:10:263
13218992314,6cyclictest0-21swapper/310:50:233
13218992314,6cyclictest0-21swapper/310:30:213
13218992311,8cyclictest0-21swapper/308:40:133
13218992310,10cyclictest0-21swapper/310:55:213
1321099238,12cyclictest31332-21kthreadcore08:35:202
1321099233,15cyclictest0-21swapper/211:03:142
1321099232,6cyclictest0-21swapper/210:55:132
1321099231,7cyclictest3147466ssh11:33:252
13210992316,4cyclictest30091-21ssh09:25:242
13210992313,4cyclictest0-21swapper/212:28:182
13210992313,4cyclictest0-21swapper/212:28:172
13210992312,9cyclictest29163-21ssh10:37:172
13210992312,8cyclictest0-21swapper/210:21:452
13210992312,4cyclictest0-21swapper/207:10:232
1321099231,19cyclictest11565-21ssh12:35:152
1319899239,4cyclictest21-21ksoftirqd/111:27:541
1319899239,3cyclictest21-21ksoftirqd/111:05:141
1319899238,3cyclictest21-21ksoftirqd/112:28:571
1319899238,3cyclictest21-21ksoftirqd/112:28:561
1319899238,3cyclictest21-21ksoftirqd/111:47:501
1319899237,8cyclictest21-21ksoftirqd/112:00:121
13198992310,3cyclictest21-21ksoftirqd/110:41:061
1321899223,13cyclictest0-21swapper/312:15:113
13218992218,2cyclictest0-21swapper/312:35:113
13218992217,3cyclictest0-21swapper/312:30:163
13218992217,3cyclictest0-21swapper/308:35:123
13218992216,2cyclictest0-21swapper/308:05:103
13218992215,5cyclictest0-21swapper/307:15:243
13218992215,5cyclictest0-21swapper/307:15:243
13218992213,6cyclictest0-21swapper/312:05:353
13218992212,6cyclictest0-21swapper/307:40:283
13218992211,8cyclictest0-21swapper/308:15:113
1321899221,14cyclictest35966irq/58-eth008:25:113
1321099228,2cyclictest131rcu_preempt11:54:382
1321099222,9cyclictest9350-21kthreadcore11:20:202
13210992213,7cyclictest0-21swapper/210:07:192
13210992212,8cyclictest27-21ksoftirqd/210:40:242
13210992212,8cyclictest0-21swapper/210:54:302
13210992212,8cyclictest0-21swapper/210:25:352
1319899229,3cyclictest21-21ksoftirqd/110:00:241
1319899228,2cyclictest21-21ksoftirqd/110:45:551
1319899227,9cyclictest0-21swapper/111:50:331
1319899227,7cyclictest0-21swapper/112:06:121
1319899227,3cyclictest21-21ksoftirqd/109:54:161
1319899221,6cyclictest21-21ksoftirqd/112:10:181
1319899221,16cyclictest17356-21ssh10:50:191
13198992211,3cyclictest21-21ksoftirqd/110:34:551
13198992210,3cyclictest21-21ksoftirqd/112:35:191
13198992210,3cyclictest21-21ksoftirqd/110:06:521
1319099229,9cyclictest0-21swapper/010:10:170
1319099222,14cyclictest13428-21cut12:00:160
1319099222,14cyclictest11346-21ssh11:58:140
1319099221,18cyclictest24518-21idleruntime07:55:160
1319099221,18cyclictest22314-21cut09:00:180
1319099221,17cyclictest19620-21uniq07:50:140
1319099221,14cyclictest7832-21ssh12:15:110
13218992115,4cyclictest0-21swapper/307:10:153
13218992115,3cyclictest0-21swapper/310:00:123
13218992112,6cyclictest0-21swapper/309:05:273
13218992112,6cyclictest0-21swapper/307:35:233
13218992112,6cyclictest0-21swapper/307:25:203
1321899211,12cyclictest0-21swapper/309:00:143
1321099219,9cyclictest6394-21munin-plugin-st09:50:012
1321099219,9cyclictest6394-21munin-plugin-st09:50:002
1321099213,4cyclictest27-21ksoftirqd/212:15:142
1321099211,9cyclictest18284-21kthreadcore08:20:202
1321099211,4cyclictest14612-21acpi07:45:122
13210992112,7cyclictest0-21swapper/207:20:152
1321099211,17cyclictest12577-21tune2fs08:50:152
1321099211,17cyclictest12577-21tune2fs08:50:142
1321099211,15cyclictest2597-21diskstats12:30:182
1319899219,3cyclictest21-21ksoftirqd/112:16:251
1319899219,3cyclictest21-21ksoftirqd/110:28:571
1319899219,2cyclictest21-21ksoftirqd/109:19:391
1319899218,5cyclictest131rcu_preempt10:10:131
1319899218,4cyclictest21-21ksoftirqd/107:25:291
1319899217,3cyclictest21-21ksoftirqd/111:23:221
13198992115,4cyclictest0-21swapper/111:00:191
13198992111,8cyclictest0-21swapper/111:30:221
13198992110,8cyclictest17155-21if_err_eth012:20:151
13198992110,3cyclictest21-21ksoftirqd/110:20:071
1319099211,8cyclictest8412-21latency_hist09:15:010
1319099211,15cyclictest21046-21cut11:10:190
1319099211,15cyclictest15572-21ssh11:05:290
13190992110,3cyclictest131rcu_preempt11:45:010
1319099210,17cyclictest29850-21ls11:15:120
1319099210,17cyclictest29850-21ls11:15:110
13218992016,2cyclictest0-21swapper/307:45:113
13218992016,2cyclictest0-21swapper/307:30:113
13218992016,2cyclictest0-21swapper/307:30:113
1321099202,9cyclictest8695-21kthreadcore08:45:182
1321099202,9cyclictest5910-21kthreadcore07:35:202
1321099201,9cyclictest25052-21kthreadcore09:00:232
1321099201,9cyclictest24920-21sort10:00:152
1321099201,9cyclictest14920-21wc08:15:272
13210992014,4cyclictest0-21swapper/207:15:262
13210992014,4cyclictest0-21swapper/207:15:252
1321099201,10cyclictest11638-21kthreadcore07:40:202
1321099200,15cyclictest1525-21ls08:05:012
1319899208,4cyclictest21-21ksoftirqd/110:55:221
1319899208,3cyclictest21-21ksoftirqd/109:10:061
1319899206,8cyclictest0-21swapper/111:59:521
1319899203,11cyclictest0-21swapper/108:25:131
1319899201,5cyclictest13448-21fschecks_count07:10:171
1319899201,2cyclictest91-21pr/ttyS008:25:011
13198992011,6cyclictest0-21swapper/108:50:141
13198992011,6cyclictest0-21swapper/108:50:131
1319899201,13cyclictest17402-21ls08:55:141
1319899200,15cyclictest1768-21cut08:05:131
1319099209,8cyclictest24439-21sshd10:15:430
1319099203,14cyclictest14415-21kthreadcore08:50:190
1319099203,14cyclictest14415-21kthreadcore08:50:190
13190992013,4cyclictest20816-21latency10:50:280
1319099200,17cyclictest28918-21ls08:30:230
13218991914,3cyclictest0-21swapper/308:10:133
13218991910,6cyclictest0-21swapper/308:55:173
13218991910,6cyclictest0-21swapper/307:20:123
1321099191,9cyclictest2845-21/usr/sbin/munin08:40:122
1319899199,2cyclictest21-21ksoftirqd/107:35:201
1319899198,2cyclictest21-21ksoftirqd/107:40:251
1319899198,2cyclictest21-21ksoftirqd/107:20:161
1319899193,4cyclictest131rcu_preempt07:15:121
1319899193,4cyclictest131rcu_preempt07:15:111
13198991914,3cyclictest0-21swapper/109:55:141
13198991913,3cyclictest21-21ksoftirqd/107:10:011
13198991910,7cyclictest131rcu_preempt08:40:251
1319899190,7cyclictest2770-21cat08:40:011
1319099197,8cyclictest12-21ksoftirqd/008:20:140
1319099191,10cyclictest8538-21find07:35:270
13190991910,5cyclictest24265-21kthreadcore12:05:200
1319099190,12cyclictest14413-21ls07:45:010
1321099181,8cyclictest13259-21latency_hist07:10:022
1319899189,7cyclictest21-21ksoftirqd/108:45:151
1319899187,3cyclictest21-21ksoftirqd/108:30:131
1319899181,4cyclictest131rcu_preempt08:00:011
1319899180,3cyclictest131rcu_preempt08:20:011
1319099186,3cyclictest131rcu_preempt10:55:220
1319099183,9cyclictest0-21swapper/007:10:140
1319099181,9cyclictest6643-21wc08:10:160
1319099181,9cyclictest21059-21kthreadcore11:45:190
1319099181,14cyclictest32611-21apt-get07:30:120
1319099181,14cyclictest32611-21apt-get07:30:110
1319099181,14cyclictest25409-21kthreadcore07:20:220
1319099181,14cyclictest11601-21irqcore08:15:160
1319099181,13cyclictest31399-21ssh12:10:190
1319099181,11cyclictest20063-21ssh11:25:310
13198991712,3cyclictest26714-21idleruntime-cro09:05:011
1319099176,7cyclictest3068-21sshd08:40:140
1319099175,9cyclictest18178-21kthreadcore08:55:190
1319099174,10cyclictest21524-21netstat07:15:220
1319099174,10cyclictest21524-21netstat07:15:220
13190991712,3cyclictest31586-21vmstat09:05:330
13190991710,5cyclictest0-21swapper/008:00:230
1319899161,11cyclictest15032-21cut07:45:171
1319099167,6cyclictest31677-21kthreadcore08:35:190
1319099162,12cyclictest22349-21kthreadcore08:25:190
1319099161,12cyclictest7669-21apt-get08:45:120
1319099161,12cyclictest1613-21apt-get08:05:110
13218991410,2cyclictest0-21swapper/307:10:013
1319099136,4cyclictest12-21ksoftirqd/007:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional